---desc: Qwen2.5-14B是阿里千问系列140亿参数开源模型，基于新一代Transformer架构优化训练与推理效率，支持32K长文本理解及多语言代码生成。训练数据融合多领域知识与代码，强化泛化能力与逻辑推理，适配高性能服务器及边缘计算，兼容4/8-bit量化技术，适用企业级智能分析、复杂任务处理等场景 support_training: 0 license: Apache License 2.0 dev_type:- notebook---# Qwen2.5-14B## 引言 (Introduction) Qwen2.5 是千问大语言模型系列的最新版本。本次发布的 Qwen2.5 系列包含从 0.5B 到 72B 参数规模的基座模型和对齐模型。相较于 Qwen2，Qwen2.5 实现了以下核心提升： Qwen2.5 is the latest series of Qwen large language models. For Qwen2.5, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters. Qwen2.5 brings the following improvements upon Qwen2:- **知识容量显著提升**：通过专业领域专家模型增强，**代码**和**数学**能力实现跨越式进步 - **指令遵循能力优化**：支持生成**超长文本（>8K tokens）**，增强**结构化数据理解**（如表格）与**结构化输出生成**（特别是 JSON 格式），**系统提示鲁棒性提升**，显著改善角色扮演与条件设定能力 - **长上下文支持**：支持 128K tokens 上下文窗口，可生成 8K tokens 长文本 - **多语言扩展**：支持中文、英语、法语、西班牙语等 29+ 种语言 **本仓库提供 14B 参数的 Qwen2.5 基座模型**，主要技术规格如下： **This repo contains the base 14B Qwen2.5 model**, which has the following features:- 模型类型：因果语言模型 - 训练阶段：预训练 - 架构：基于 transformers 实现，集成 RoPE（旋转位置编码）/SwiGLU/RMSNorm 等技术，含注意力 QKV 偏置 - 参数量：14.7B（非词嵌入参数量 13.1B） - 网络深度：48 层 - 注意力头配置（GQA）：Q 头 40 组 / KV 头 8 组 - 上下文长度：完整支持 131,072 tokens **我们不建议直接使用基座模型进行对话**。推荐用户进行后训练处理（如 SFT/RLHF/持续预训练等）以获得更佳效果。 **We do not recommend using base language models for conversations.** Instead, you can apply post-training, e.g., SFT, RLHF, continued pretraining, etc., on this model.更多技术细节，请参阅我们的[技术博客](https://qwenlm.github.io/blog/qwen2.5/)、[GitHub仓库](https://github.com/QwenLM/Qwen2.5)以及[官方文档](https://qwen.readthedocs.io/en/latest/)。 For more details, please refer to our [blog](https://qwenlm.github.io/blog/qwen2.5/), [GitHub](https://github.com/QwenLM/Qwen2.5), and [Documentation](https://qwen.readthedocs.io/en/latest/).## 环境要求 (Requirements)Qwen2.5 的代码已集成至最新版 PaddleNLP，建议安装`paddlenlp>=2.7.0`版本。若版本过低，可能会遇到如下报错：The code of Qwen2.5 has been integrated into the latest PaddleNLP and we advise you to install `paddlenlp>=2.7.0`, or you might encounter the following error:```pythonKeyError: 'qwen2_5'```## 评估与性能 (Evaluation & Performance) 详细评估结果请参阅我们的[📑 技术博客](https://qwenlm.github.io/blog/qwen2.5/)。 Detailed evaluation results are reported in this [📑 blog](https://qwenlm.github.io/blog/qwen2.5/).GPU 显存需求及对应吞吐量结果请查看[此基准测试文档](https://qwen.readthedocs.io/en/latest/benchmark/speed_benchmark.html)。 For requirements on GPU memory and the respective throughput, see results [here](https://qwen.readthedocs.io/en/latest/benchmark/speed_benchmark.html).## 引用说明 (Citation) 如果您认为我们的工作对您有所帮助，欢迎在学术论文中引用相关研究成果。 If you find our work helpful, feel free to give us a cite.```@misc{qwen2.5, title = {Qwen2.5: A Party of Foundation Models}, url = {https://qwenlm.github.io/blog/qwen2.5/}, author = {Qwen Team}, month = {September}, year = {2024}}@article{qwen2, title={Qwen2 Technical Report}, author={An Yang and Baosong Yang and Binyuan Hui and Bo Zheng and Bowen Yu and Chang Zhou and Chengpeng Li and Chengyuan Li and Dayiheng Liu and Fei Huang and Guanting Dong and Haoran Wei and Huan Lin and Jialong Tang and Jialin Wang and Jian Yang and Jianhong Tu and Jianwei Zhang and Jianxin Ma and Jin Xu and Jingren Zhou and Jinze Bai and Jinzheng He and Junyang Lin and Kai Dang and Keming Lu and Keqin Chen and Kexin Yang and Mei Li and Mingfeng Xue and Na Ni and Pei Zhang and Peng Wang and Ru Peng and Rui Men and Ruize Gao and Runji Lin and Shijie Wang and Shuai Bai and Sinan Tan and Tianhang Zhu and Tianhao Li and Tianyu Liu and Wenbin Ge and Xiaodong Deng and Xiaohuan Zhou and Xingzhang Ren and Xinyu Zhang and Xipin Wei and Xuancheng Ren and Yang Fan and Yang Yao and Yichang Zhang and Yu Wan and Yunfei Chu and Yuqiong Liu and Zeyu Cui and Zhenru Zhang and Zhihao Fan}, journal={arXiv preprint arXiv:2407.10671}, year={2024}}```