AI 基建
0%
参考文献

参考文献

作者Changkun Ou
阅读时长约 113 分钟

本页收录中文版引用的资料,先按第一作者的姓氏排序,再按出版年份排序。每个条目依次列出正文中使用的引用标记、资料标题、可访问的来源链接,以及一句说明该资料为本书提供了什么。点击正文中的引用标记,即可跳转到这里的对应条目。

[A2A Protocol Working Group 2026] A2A Protocol Working Group. Agent2Agent (A2A) protocol specification, version 1.0. https://a2a-protocol.org/latest/specification/
A2A 1.0 统一了独立智能体之间的发现元数据、消息、有状态任务和产物格式。身份认证与授权仍由部署方负责,推送也可能重复送达。
[Abadi et al. 2016] Abadi, Chu, Goodfellow, McMahan, Mironov, Talwar, Zhang. Deep learning with differential privacy. https://arxiv.org/abs/1607.00133
Abadi 等人提出 DP-SGD,通过逐样本梯度裁剪、高斯噪声注入和矩账户方法,在差分隐私约束下训练深度神经网络,并实现更紧的隐私预算估计。
[Abadi et al. 2016] Abadi, Barham, Chen, Chen, Davis, Dean, Devin, Ghemawat, Irving, Isard, Kudlur, Levenberg, Monga, Moore, Murray, Steiner, Tucker, Vasudevan, Warden, Wicke, Yu, Zheng. TensorFlow: a system for large-scale machine learning. https://arxiv.org/abs/1605.08695
TensorFlow 将计算表示为数据流图,并协调 CPU、GPU 和分布式工作节点执行训练与推理任务。
[Abbas et al. 2023] Abbas, Tirumala, Simig, Ganguli, Morcos. SemDeDup: Data-efficient learning at web-scale through semantic deduplication. https://arxiv.org/abs/2303.09540
SemDeDup 利用预训练模型的嵌入向量识别并去除语义相似但非完全相同的重复样本,可将网络规模训练数据减少 50% 同时几乎不损失性能。
[Abhyankar et al. 2026] Abhyankar, Qi, Zhang. OSWorld-human: Benchmarking the efficiency of computer-use agents. Proceedings of Machine Learning and Systems 8. https://proceedings.mlsys.org/paper_files/paper/2026/hash/5edb57c05c81d04beb716ef1d542fe9e-Abstract-Conference.html
测量完成率遮住的东西:受测智能体的步数是人工参考轨迹的 2.7 至 4.3 倍,且延迟主要来自模型调用。
[Acun et al. 2021] Acun, Murphy, Wang, Nie, Wu, Hazelwood. Understanding training efficiency of deep learning recommendation models at scale. https://arxiv.org/abs/2011.05497
该研究刻画了大型推荐模型的训练负载,说明模型结构、数据移动和分布式执行如何共同影响效率。
[Agache et al. 2020] Agache, Brooker, Florescu, Iordache, Liguori, Neugebauer, Piwonka, Popa. Firecracker: Lightweight virtualization for serverless applications. https://www.usenix.org/conference/nsdi20/presentation/agache
Firecracker 介绍了一种基于 KVM 的轻量虚拟机监控器,以及它在隔离高密度无服务器负载时所作的设计取舍。
[Agent Payments Protocol 2026] Agent Payments Protocol. AP2 specification. https://ap2-protocol.org/ap2/specification/
AP2 为结账和支付授权定义了确定且带签名的交易凭据,同时把智能体识别、结算和争议处理留给其他系统。
[Agentic Commerce Protocol 2026] Agentic Commerce Protocol. Agentic commerce protocol. https://github.com/agentic-commerce-protocol/agentic-commerce-protocol
ACP 负责协调商业流程,同时让商家继续充当记录系统,并通过处理器协商支付能力,而不规定唯一的结算通道。
[Agrawal et al. 2024] Agrawal, Kedia, Panwar, Mohan, Kwatra, Gulavani, Tumanov, Ramjee. Taming throughput-latency tradeoff in LLM inference with sarathi-serve. USENIX Association. https://www.usenix.org/conference/osdi24/presentation/agrawal
Sarathi-Serve 将长预填充拆成多个块并与解码共同调度,以限制生成停顿并保留批处理机会。
[Ahmadian et al. 2024] Ahmadian, Cremer, Gallé, Fadaee, Kreutzer, Pietquin, Üstün, Hooker. Back to basics: Revisiting REINFORCE-style optimization for learning from human feedback in llms. Association for Computational Linguistics. https://aclanthology.org/2024.acl-long.662/
简单的 REINFORCE 风格策略梯度(RLOO)在 LLM 基于人类反馈的强化学习(RLHF)对齐中优于 PPO 和 DPO,且计算开销更低,无需近端策略优化(PPO)的 actor-critic 复杂机制。
[Ainslie et al. 2023] Ainslie, Lee-Thorp, Jong, Zemlyanskiy, Lebron, Sanghai. GQA: Training generalized multi-query transformer models from multi-head checkpoints. Association for Computational Linguistics. https://aclanthology.org/2023.emnlp-main.298/
GQA 提出分组查询注意力(GQA),在多头注意力(MHA)与多查询注意力(MQA)之间插值,以接近 MQA 的推理速度达到接近 MHA 的质量,并提供仅用 5% 原始预训练算力将现有 MHA 检查点升训为 GQA 的方案。
[Aizman et al. 2020] Aizman, Maltby, Breuel. High performance I/O for large scale deep learning. https://arxiv.org/abs/2001.01858
该论文分析大型深度学习任务的输入流水线瓶颈,并介绍让加速器持续获得数据的存储和加载方法。
[Alayrac et al. 2022] Alayrac, Donahue, Luc, Miech, Barr, Hasson, Lenc, Mensch, Millican, Reynolds, others. Flamingo: a visual language model for few-shot learning. https://proceedings.neurips.cc/paper_files/paper/2022/hash/960a172bc7fbf0177ccccbb411a7d800-Abstract-Conference.html
Flamingo 是一个视觉语言模型(VLM)系列,通过 Perceiver Resampler 和门控交叉注意力桥接冻结的视觉与语言模型,在 16 项图像和视频理解任务上实现少样本学习。
[Albergo and Vanden-Eijnden 2023] Albergo, Vanden-Eijnden. Building normalizing flows with stochastic interpolants. https://arxiv.org/abs/2209.15571
本文提出随机插值子框架,通过构造任意两个概率密度之间的时变插值过程,利用简单二次损失直接学习常微分方程(ODE)速度场,无需反向传播穿越 ODE 求解器。
[Albergo et al. 2025] Albergo, Boffi, Vanden-Eijnden. Stochastic interpolants: a unifying framework for flows and diffusions. Journal of Machine Learning Research 26(209):1–80. https://www.jmlr.org/papers/v26/23-1605.html
本文提出随机插值器框架,通过 ODE 或 SDE 动力学在有限时间内精确连接任意两个概率密度,统一了基于流匹配和扩散的生成模型。
[Alpern and Schneider 1985] Alpern, Schneider. Defining liveness. https://ecommons.cornell.edu/items/2ed32f4f-cc5c-413b-ba16-5498641f1939
Alpern 和 Schneider 对活性及其与安全性属性的关系作出形式化定义,避免把这些术语当作准确性的非正式标签。
[Alsup 2025] Alsup. Bartz v. anthropic PBC, order on fair use, no. 3:24-cv-05417. United States District Court for the Northern District of California. https://www.govinfo.gov/content/pkg/USCOURTS-cand-3_24-cv-05417/pdf/USCOURTS-cand-3_24-cv-05417-0.pdf
认定案卷中的模型训练使用构成合理使用,同时将盗版来源图书的永久资料库用途留待审理。
[Amatriain and Basilico 2012] Amatriain, Basilico. Netflix recommendations: Beyond the 5 stars (part 1). https://netflixtechblog.com/netflix-recommendations-beyond-the-5-stars-part-1-55838468f429
Netflix 纳入了 2007 年进展奖集成模型中的两个算法,但没有部署后来大奖方案中的新增方法,因为其边际离线收益不足以抵消工程成本,且产品已转向流媒体内容发现。
[Amazon Web Services 2025] Amazon Web Services. AWS nitro enclaves concepts. https://docs.aws.amazon.com/enclaves/latest/user/nitro-enclave-concepts.html
记录 Nitro Enclaves 如何获得隔离内存与 vCPU、与父实例通信,并生成根植于 AWS 基础设施的证明文档。
[Amazon Web Services 2026] Amazon Web Services. Idempotency for AWS Lambda durable functions. https://docs.aws.amazon.com/lambda/latest/dg/durable-execution-idempotency.html
Lambda 持久步骤默认采用至少一次执行;中断的工作可能重复,因此有副作用的业务逻辑仍需要幂等键或其他去重协议。
[Amazon Web Services 2026] Amazon Web Services. Choosing a purchasing option for amazon EC2. https://docs.aws.amazon.com/decision-guides/latest/ec2-purchasing-options-aws-how-to-choose/
AWS 按计费、容量保障、灵活性与中断语义区分按需、竞价、节省承诺、容量预留和加速器容量块。
[Amazon.com, Inc. 2026] Amazon.com, Inc.. Annual report for the fiscal year ended december 31, 2025. https://www.sec.gov/Archives/edgar/data/1018724/000101872426000004/amzn-20251231.htm
Amazon 披露 2024 年把服务器年限从五年延至六年,并在 2025 年把一部分服务器与网络设备缩回五年。
[AMD 2020] AMD. AMD SEV-SNP: Strengthening VM isolation with integrity protection and more. https://docs.amd.com/v/u/en-US/SEV-SNP-strengthening-vm-isolation-with-integrity-protection-and-more
虚拟机级机密计算的白皮书:对整台虚拟机做加密与完整性保护以对抗恶意虚拟机监控器,让未经修改的软件栈也能机密运行。
[Ameisen et al. 2025] Ameisen, Lindsey, Pearce, Gurnee, Turner, Chen, Citro, Abrahams, Carter, Hosmer, Marcus, Sklar, Templeton, Bricken, McDougall, Cunningham, Henighan, Jermyn, Jones, Persic, Qi, Thompson, Zimmerman, Rivoire, Conerly, Olah, Batson. Circuit tracing: Revealing computational graphs in language models. https://transformer-circuits.pub/2025/attribution-graphs/methods.html
Circuit Tracing 提出归因图方法,通过将 MLP 替换为可解释的跨层转码器,揭示语言模型逐步计算的内部机制。
[Amershi et al. 2019] Amershi, Weld, Vorvoreanu, Fourney, Nushi, Collisson, Suh, Iqbal, Bennett, Inkpen, Teevan, Kikin-Gil, Horvitz. Guidelines for human-AI interaction. https://www.microsoft.com/en-us/research/publication/guidelines-for-human-ai-interaction/
这篇 CHI 论文提炼出十八条面向用户侧 AI 产品的通用指南,并通过从业者审查 AI 产品的研究进行验证。
[Anderson 1982] Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications 12(3):313–326. https://doi.org/10.1016/0304-4149(82)90051-5
Anderson(1982)推导了前向扩散过程的逆时随机微分方程(SDE),证明逆向漂移项取决于边缘分布的分数函数。
[Andriushchenko et al. 2025] Andriushchenko, Croce, Flammarion. Jailbreaking leading safety-aligned LLMs with simple adaptive attacks. https://proceedings.iclr.cc/paper_files/paper/2025/hash/63fa7efdd3bcf944a4bd6e0ff6a50041-Abstract-Conference.html
针对防御进行的简单调整,显著提高了对先前静态评测中看似鲁棒的模型与防御的攻击成功率。
[Android Developers n.d.] Android Developers. Thermal API. https://developer.android.com/ndk/reference/group/thermal
Android 热 API 暴露热状态与热余量信号,应用可据此观测并调整持续工作负载。
[Angrist et al. 1996] Angrist, Imbens, Rubin. Identification of causal effects using instrumental variables. Journal of the American Statistical Association. https://www.nber.org/papers/t0136
该论文说明了在实际接受处理不完全时,随机分配能够识别服从者平均因果效应所需的假设。
[Anil et al. 2024] Anil, Durmus, Panickssery, Sharma, Benton, Kundu, Batson, Tong, Mu, Ford, Mosconi, Agrawal, Schaeffer, Bashkansky, Svenningsen, Lambert, Radhakrishnan, Denison, Hubinger, Bai, Bricken, Maxwell, Schiefer, Sully, Tamkin, Lanham, Nguyen, Korbak, Kaplan, Ganguli, Bowman, Perez, Grosse, Duvenaud. Many-shot jailbreaking. https://papers.nips.cc/paper_files/paper/2024/hash/ea456e232efb72d261715e33ce25f208-Abstract-Conference.html
多样本越狱利用大型上下文窗口,通过在单个提示中填充数百个伪有害对话示例,借助上下文学习绕过大语言模型(LLM)的安全训练。
[Ankner et al. 2024] Ankner, Parthasarathy, Nrusimha, Rinard, Ragan-Kelley, Brandon. Hydra: Sequentially-dependent draft heads for medusa decoding. https://openreview.net/forum?id=FbhjirzvJG
Hydra 让每个草稿头以更早的候选词元为条件,用序列相关的候选取代 Medusa 相互独立的未来词元预测头。
[Ansel et al. 2024] Ansel, Yang, He, Gimelshein, Jain, Voznesensky, Bao, Bell, Berard, Burovski, Chauhan, Chourdia, Constable, Desmaison, DeVito, Ellison, Feng, Gong, Gschwind, Hirsh, Huang, Kalambarkar, Kirsch, Lazos, Lezcano, Liang, Liang, Lu, Luk, Maher, Pan, Puhrsch, Reso, Saroufim, Siraichi, Suk, Zhang, Suo, Tillet, Zhou, Wang, Zou, Wang, Mathews, Wen, Chanan, Wu, Chintala. PyTorch 2: Faster machine learning through dynamic python bytecode transformation and graph compilation. https://docs.pytorch.org/assets/pytorch2-2.pdf
本文说明 PyTorch 2 编译路径中的带守卫 Python 字节码捕获、图中断、AOTAutograd 与 TorchInductor。
[Anthropic 2024] Anthropic. Introducing computer use, a new claude 3.5 sonnet, and claude 3.5 haiku. https://www.anthropic.com/news/3-5-models-and-computer-use
第一个公测「计算机使用」的前沿模型:以通用计算机技能取代按任务定制的工具,发布时坦率注明它是实验性的、笨拙且容易出错。
[Anthropic 2024] Anthropic. Introducing contextual retrieval. https://www.anthropic.com/engineering/contextual-retrieval
上下文检索在嵌入和索引前为每个分块补充文档上下文,从而改善孤立分块含义不明确时的检索效果。
[Anthropic 2024] Anthropic. Introducing the model context protocol. https://www.anthropic.com/news/model-context-protocol
Anthropic 将 MCP 作为连接 AI 助手与数据和工具系统的开放协议推出。
[Anthropic 2025] Anthropic. How we built our multi-agent research system. https://www.anthropic.com/engineering/multi-agent-research-system
Anthropic 介绍了一个编排器与工作智能体协同的研究系统,同时报告内部测试中 90.2% 的相对提升、高令牌成本和实际协调经验。
[Anthropic 2025] Anthropic. Effective context engineering for AI agents. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
这篇工程文章把上下文管理定义为逐步筛选指令、工具、外部数据、历史记录、检索结果和压缩内容。
[Anthropic 2025] Anthropic. Open-sourcing circuit tracing tools. https://www.anthropic.com/research/open-source-circuit-tracing
Anthropic 发布了用于构建和检查归因图的软件与模型产物,让其他研究者能够检验回路追踪方法。
[Anthropic 2026] Anthropic. Claude's new constitution. https://www.anthropic.com/news/claude-new-constitution
Anthropic 的 2026 年 1 月宪章主要写给 Claude,依次规定广义安全、伦理、Anthropic 准则和有用性,并用于生成合成训练数据。
[Anthropic 2026] Anthropic. Use Claude's chat search and memory to build on previous context. https://support.claude.com/en/articles/11817273-use-claude-s-chat-search-and-memory-to-build-on-previous-context
Anthropic 说明按项目隔离的记忆、搜索、暂停与重置行为、无痕聊天及来源关联的记忆管理。
[Anthropic 2026] Anthropic. Computer use tool. https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool
计算机使用动作、开发者执行循环、缩放支持、隔离环境、最小权限、域名限制与人工确认的官方文档。
[Anthropic 2026] Anthropic. Responsible scaling policy, version 3.4. https://www.anthropic.com/responsible-scaling-policy
把 AI 安全等级定义为能力阈值,每个阈值对应一层必需的防护措施,并规定评估未有结论时按不部署处理。3.0 版起不再为当前之上的能力等级逐条列出控制项,改为要求给出安全论证,并新增前沿安全路线图与每三到六个月一份的风险报告。
[Anthropic 2026] Anthropic. Project deal: Our claude-run marketplace experiment. https://www.anthropic.com/features/project-deal
员工智能体完成了 186 笔真实交易,总额略超 4,000 美元;这是边界清楚的试点,不能当作全经济采用的证据。
[Anthropic n.d.] Anthropic. OpenAI SDK compatibility. https://platform.claude.com/docs/en/cli-sdks-libraries/libraries/openai-sdk
Anthropic 将其 OpenAI SDK 兼容层定位为测试便利工具,并列出语义差异以及不受支持或被忽略的请求字段。
[Anthropic and Pattern Labs 2025] Anthropic and Pattern Labs. Confidential inference systems: Design principles and security risks. https://assets.anthropic.com/m/c52125297b85a42/original/Confidential_Inference_Paper.pdf
机密推理的设计原则,覆盖信任问题的两个方向:用户数据防住服务商,模型权重防住基础设施运营者。
[Apple n.d.] Apple. Process information: Responding to thermal state changes. https://developer.apple.com/documentation/foundation/processinfo
Apple 进程信息 API 暴露热状态变化,使应用可在热压力达到临界前减少高开销工作。
[Apple n.d.] Apple. Core ML. https://developer.apple.com/documentation/coreml
Core ML 提供设备模型表示,并可根据模型与平台支持使用 CPU、GPU 和神经网络引擎。
[Apple Security Engineering and Architecture (SEAR), User Privacy, Core Operating Systems (Core OS), Services Engineering (ASE), and Machine Learning and AI (AIML) 2024] Apple Security Engineering and Architecture (SEAR), User Privacy, Core Operating Systems (Core OS), Services Engineering (ASE), and Machine Learning and AI (AIML). Private Cloud Compute: A new frontier for AI privacy in the cloud. https://security.apple.com/blog/private-cloud-compute/
Apple 的机密 AI 服务设计及其五项要求:无状态计算、可强制执行的保证、无特权运行时访问、不可定向,以及可验证的透明性。
[Apple Security Engineering and Architecture (SEAR), User Privacy, Core Operating Systems (Core OS), Services Engineering (ASE), and Machine Learning and AI (AIML) 2026] Apple Security Engineering and Architecture (SEAR), User Privacy, Core Operating Systems (Core OS), Services Engineering (ASE), and Machine Learning and AI (AIML). Expanding private cloud compute. https://security.apple.com/blog/expanding-pcc/
说明 PCC 如何把验证与隐私属性扩展到采用 Intel TDX、NVIDIA 机密计算与 Google Cloud 基础设施的部署。
[Arora et al. 2025] Arora, Wei, Hicks, Bowman, Quiñonero-Candela, Tsimpourlas, Sharman, Shah, Vallone, Beutel, Heidecke, Singhal. HealthBench: Evaluating large language models towards improved human health. arXiv preprint arXiv:2505.08775. https://arxiv.org/abs/2505.08775
HealthBench 用 262 位医生(执业经验覆盖 60 个国家)撰写的 48,562 条评分准则为健康对话打分,并先用医生评分对模型评分器做元评测,再采信其分数。
[Arriola et al. 2025] Arriola, Gokaslan, Chiu, Yang, Qi, Han, Sahoo, Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models. https://arxiv.org/abs/2503.09573
BD3-LMs 通过在词元块内执行扩散、跨块自回归建模,使离散扩散语言模型支持任意长度生成和 KV 缓存,并在语言建模基准上创下离散扩散的最低困惑度。
[Artstein and Poesio 2008] Artstein, Poesio. Inter-coder agreement for computational linguistics. Computational Linguistics. https://aclanthology.org/J08-4004/
Artstein 与 Poesio 综述了计算语言学标注中的一致性系数、假设与解释,强调系数必须匹配研究设计和量表。
[Asai et al. 2024] Asai, Wu, Wang, Sil, Hajishirzi. Self-RAG: Learning to retrieve, generate, and critique through self-reflection. https://arxiv.org/abs/2310.11511
Self-RAG 训练反思词元,在生成期间控制检索并评估相关性、证据支持与回答效用。
[ASHRAE 2026] ASHRAE. Retrofit and modernization strategies: AI data center energy performance framework. https://www.ashrae.org/technical-resources/ai-data-center-framework/retrofit-modernization-strategies
ASHRAE 围绕供电、冷却、控制、测量和在运设施的约束,梳理了 AI 数据中心的改造决策。
[Assran et al. 2025] Assran, Bardes, others. V-JEPA 2: Self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985. https://arxiv.org/abs/2506.09985
V-JEPA 2 是一个在百万小时互联网视频上自监督预训练的视频模型,达到最优视频理解性能,并通过后训练的动作条件世界模型实现机器人零样本操作规划。
[Auer et al. 2024] Auer, Lysak, Nassar, Dolfi, Livathinos, Vagenas, Berrospi Ramis, Omenetti, Lindlbauer, Dinkla, Mishra, Kim, Gupta, Teixeira de Lima, Weber, Morin, Meijer, Kuropiatnyk, Staar. Docling technical report. https://arxiv.org/abs/2408.09869
该文介绍了一个开放的文档转换工具包,以及它表示页面布局、表格、文本和来源信息的方式。
[Austin et al. 2021] Austin, Johnson, Ho, Tarlow, Berg. Structured denoising diffusion models in discrete state-spaces. https://arxiv.org/abs/2107.03006
D3PM 通过以结构化转移矩阵替代均匀损坏过程,将去噪扩散概率模型(DDPM)推广至离散状态空间,在文本和图像生成任务上取得有竞争力的结果。
[Austin et al. 2025] Austin, Douglas, Frostig, Levskaya, Chen, Vikram, Lebron, Choy, Ramasesh, Webson, Pope. How to scale your model. Google DeepMind. https://jax-ml.github.io/scaling-book/
DeepMind/JAX 的系统书,讲真实 TPU 与 GPU 硬件上的 LLM 扩展,覆盖屋顶线、分片、训练、推断、服务与剖析。
[Autio et al. 2024] Autio, Schwartz, Dunietz, Jain, Stanley, Tabassi, Hall, Roberts. Artificial intelligence risk management framework: Generative artificial intelligence profile. https://doi.org/10.6028/NIST.AI.600-1
NIST 生成式 AI 概况建议将结构化红队测试、对抗性角色扮演及相关测试纳入全生命周期风险管理。
[Autio et al. 2024] Autio, Schwartz, Dunietz, Jain, Stanley, Tabassi, Hall, Roberts. Artificial intelligence risk management framework: Generative artificial intelligence profile. https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence
NIST AI 600-1 从设计、部署情境评估、持续监控、事件响应、变更与退役全过程讨论生成式 AI 风险与控制。
[Axolotl AI n.d.] Axolotl AI. Axolotl. https://github.com/axolotl-ai-cloud/axolotl
Axolotl 是一个以配置为中心的开源训练项目。其代码仓库和文档是查询支持模型与当前配置字段的权威来源。
[Ba et al. 2016] Ba, Kiros, Hinton. Layer normalization. https://arxiv.org/abs/1607.06450
层归一化在单个训练样本的隐藏层内部计算归一化统计量,消除了对批大小的依赖,并稳定了循环神经网络的训练过程。
[Backman et al. 2024] Backman, Richer, Sporny. HTTP message signatures. https://www.rfc-editor.org/rfc/rfc9421.html
HTTP Message Signatures 对选定的请求或响应组成部分进行认证,但本身不识别最终用户,也不授权商业操作。
[Baevski et al. 2020] Baevski, Zhou, Mohamed, Auli. wav2vec 2.0: a framework for self-supervised learning of speech representations. https://arxiv.org/abs/2006.11477
wav2vec 2.0 通过对量化潜在表示的对比学习实现自监督语音预训练,仅需十分钟标注数据即可完成自动语音识别(ASR)微调。
[Bai et al. 2022] Bai, Kadavath, Kundu, Askell, Kernion, Jones, Chen, Goldie, Mirhoseini, McKinnon, Chen, Olsson, Olah, Hernandez, Drain, Ganguli, Li, Tran-Johnson, Perez, Kerr, Mueller, Ladish, Landau, Ndousse, Lukosuite, Lovitt, Sellitto, Elhage, Schiefer, Mercado, DasSarma, Lasenby, Larson, Ringer, Johnston, Kravec, El Showk, Fort, Lanham, Telleen-Lawton, Conerly, Henighan, Hume, Bowman, Hatfield-Dodds, Mann, Amodei, Joseph, McCandlish, Brown, Kaplan. Constitutional AI: Harmlessness from AI feedback. arXiv preprint arXiv:2212.08073. https://arxiv.org/abs/2212.08073
Constitutional AI 通过成文原则、自我批判、修订与 AI 反馈,训练无害但不过度回避的助手行为。
[Bainbridge 1983] Bainbridge. Ironies of automation. Automatica. https://doi.org/10.1016/0005-1098(83)90046-8
Bainbridge 指出自动化的悖论:它拿走日常练习,却让人在自动化失效时负责罕见而困难的干预。
[Balepur et al. 2026] Balepur, Hamada, Kishore, Feldman, Singh, Siangliulue, Chang, Choi, Boyd-Graber, Naik. Language models don't know what you want: Evaluating personalization in deep research needs real users. https://aclanthology.org/2026.acl-long.723/
该研究比较真实用户判断、合成用户和模型裁判评测,识别代理评测遗漏的个性化错误。
[Bansal et al. 2021] Bansal, Wu, Zhou, Fok, Nushi, Kamar, Ribeiro, Weld. Does the whole exceed its parts? The effect of AI explanations on complementary team performance. https://arxiv.org/abs/2006.14779
Bansal 等发现,AI 解释没有改善互补式人机团队表现,且可能不分对错地提高人们接受建议的概率。
[Baronio et al. 2025] Baronio, Marsella, Pan, Guo, Alberti. Kevin: Multi-turn RL for generating CUDA kernels. arXiv preprint arXiv:2507.11948. https://arxiv.org/abs/2507.11948
Kevin 通过与编译器和运行时反馈进行多轮交互,训练模型改进 CUDA 内核,而不是只生成一次结果。
[Bartock et al. 2016] Bartock, Cichonski, Souppaya, Smith, Witte, Scarfone. Guide for cybersecurity event recovery. https://doi.org/10.6028/NIST.SP.800-184
NIST 就网络安全事件后的恢复规划、测试、恢复验证、指标、监控和收尾提供了指导。
[Baur and Strassen 1983] Baur, Strassen. The complexity of partial derivatives. Theoretical Computer Science. https://web.vu.lt/mif/s.jukna/tropical/Baur-Strassen.pdf
Baur 与 Strassen 证明:在算术电路模型中,有理函数及其全部一阶偏导可在常数倍运算量内共同计算。
[Baydin et al. 2018] Baydin, Pearlmutter, Radul, Siskind. Automatic differentiation in machine learning: a survey. Journal of Machine Learning Research. https://jmlr.org/papers/v18/17-468.html
这篇综述区分自动微分、符号微分与数值微分,并系统说明机器学习程序中的前向与反向累积。
[Bean and others 2025] Bean, others. Measuring what matters: Construct validity in large language model benchmarks. https://papers.neurips.cc/paper_files/paper/2025/hash/1967e0fc3aa6cbbace562f5cb8e3954e-Abstract-Datasets_and_Benchmarks_Track.html
一项涵盖 445 篇基准论文的系统综述记录了反复出现的缺陷,并提出八项提升构念有效性的建议。
[Becker et al. 2025] Becker, Rush, Barnes, Rein. Measuring the impact of early-2025 AI on experienced open-source developer productivity. arXiv preprint arXiv:2507.09089. https://arxiv.org/abs/2507.09089
一项随机对照试验(RCT)对16名有经验的开源开发者进行研究,发现使用2025年初前沿 AI 工具实际使任务完成时间增加了19%。
[Becker et al. 2026] Becker, Rush, Cunningham, Rein, Mahamud. We are changing our developer productivity experiment design. https://metr.org/blog/2026-02-24-uplift-update/
METR 报告了原始数据中看似正向的估计,同时说明样本选择、补偿方式和并发使用智能体为何使后续研究无法可靠衡量因果效应。
[Benjamini and Hochberg 1995] Benjamini, Hochberg. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x
Benjamini 与 Hochberg 提出控制错误发现率的方法,为大量同时检验场景提供比族错误率控制更有检验力的选择。
[Besiroglu et al. 2024] Besiroglu, Erdil, Barnett, You. Chinchilla scaling: a replication attempt. arXiv preprint arXiv:2404.10102. https://arxiv.org/abs/2404.10102
复现 Chinchilla 的参数化拟合,发现其第三种估计与论文自身前两种方法不一致,且其置信区间窄到需要数十万次实验才能成立,而实际只跑了数百次。
[Besta et al. 2024] Besta, Blach, Kubicek, Gerstenberger, Podstawski, Gianinazzi, Gajda, Lehmann, Niewiadomski, Nyczyk, Hoefler. Graph of thoughts: Solving elaborate problems with large language models. https://arxiv.org/abs/2308.09687
Graph of Thoughts 将链式和树式提示推广为由大语言模型生成 thought 构成的任意图,使中间推理单元能够被聚合、精炼和反馈。
[Betker et al. 2023] Betker, Goh, Jing, Brooks, Wang, Li, Ouyang, Zhuang, Lee, Guo, Manassra, Dhariwal, Chu, Jiao, Ramesh. Improving image generation with better captions. https://cdn.openai.com/papers/dall-e-3.pdf
DALL-E 3 通过在经过微调的图像描述模型生成的高描述性合成字幕上训练,显著提升了文本到图像模型的提示词遵循能力。
[Beurer-Kellner et al. 2025] Beurer-Kellner, Buesser, Creţu, Debenedetti, Dobos, Fabian, Fischer, Froelicher, Grosse, Naeff, Ozoani, Paverd, Tramèr, Volhejn. Design patterns for securing LLM agents against prompt injections. arXiv preprint arXiv:2506.08837. https://arxiv.org/abs/2506.08837
来自 ETH、Google、Microsoft 与 Invariant 的研究者系统化了六个设计模式,通过约束智能体的结构,使注入指令无法改写其特权动作。
[Beyer et al. 2016] Beyer, Jones, Petoff, Murphy. Site reliability engineering: How google runs production systems. O'Reilly Media. https://sre.google/books/
Google SRE 一书提供了本章改造到 AI 系统上的运营词汇:服务承诺、错误预算、事故指挥以及从失败中学习。
[Bhatt et al. 2025] Bhatt, Rushing, Kaufman, Tracy, Georgiev, Matolcsi, Khan, Shlegeris. Ctrl-z: Controlling AI agents via resampling. arXiv preprint arXiv:2504.10374. https://arxiv.org/abs/2504.10374
Ctrl-Z 在 BashBench 的 257 个多步运维任务上评测控制协议,提出重采样机制:对可疑动作重新采样以积累证据,将攻击成功率从 58% 压到 7%,有用性仅损失 5%。
[Biderman et al. 2023] Biderman, Schoelkopf, Anthony, Bradley, O'Brien, Hallahan, Khan, Purohit, Prashanth, Raff, Skowron, Sutawika, Wal. Pythia: a suite for analyzing large language models across training and scaling. PMLR. https://proceedings.mlr.press/v202/biderman23a.html
Pythia 是一套包含 16 个大语言模型(70M 至 12B 参数)的模型组,每个模型提供 154 个公开检查点,所有模型按相同数据顺序训练,用于研究训练动态与扩展律。
[Biderman et al. 2024] Biderman, Portes, Gonzalez Ortiz, Paul, Greengard, Jennings, King, Havens, Chiley, Frankle, Blakeney, Cunningham. LoRA learns less and forgets less. Transactions on Machine Learning Research. https://arxiv.org/abs/2405.09673
在所研究的数学与代码场景中,标准 LoRA 对目标领域的学习少于完整微调,但保留了更多基座模型行为。
[Bie et al. 2025] Bie, Cao, Chen, Du, others. LLaDA2.0: Scaling up diffusion language models to 100B. arXiv preprint arXiv:2512.15745. https://arxiv.org/abs/2512.15745
LLaDA2.0 将预训练的自回归混合专家模型转换为扩散模型,其中包含一个总参数量 1000 亿、激活参数量 61 亿的版本。
[Birgisson et al. 2014] Birgisson, Politz, Erlingsson, Taly, Vrable, Lentczner. Macaroons: Cookies with contextual caveats for decentralized authorization in the cloud. https://www.ndss-symposium.org/ndss2014/ndss-2014-programme/macaroons-cookies-contextual-caveats-decentralized-authorization-cloud/
Macaroons 展示了如何通过串联约束实现去中心化的权限收缩,但策略语义和撤销仍需由外围系统处理。
[Birkholz et al. 2023] Birkholz, Thaler, Richardson, Smith, Pan. Remote attestation procedureS (RATS) architecture. https://www.rfc-editor.org/rfc/rfc9334.html
定义远程证明的角色与信息流,并将证据评估与依赖方的授权决策明确分开。
[Black Forest Labs 2024] Black Forest Labs. Announcing black forest labs. https://bfl.ai/blog/24-08-01-bfl
Black Forest Labs 发布 FLUX.1 流匹配模型家族,包括完整模型、开放权重的蒸馏模型和更快的托管版本。
[Bonney et al. 2026] Bonney, Breaux, Dinlersoz, Foster, Haltiwanger, Pande. The microstructure of AI diffusion: Evidence from firms, business functions, and worker tasks. https://www.census.gov/library/working-papers/2026/adrm/CES-WP-26-25.html
该调查估计企业在各业务职能中的 AI 采用情况,并显示多数采用企业的使用范围仍较窄;它不识别因果生产率。
[Borsos et al. 2023] Borsos, Marinier, Vincent, Kharitonov, Pietquin, Sharifi, Roblek, Teboul, Grangier, Tagliasacchi, Zeghidour. AudioLM: a language modeling approach to audio generation. IEEE/ACM Transactions on Audio, Speech, and Language Processing. https://arxiv.org/abs/2209.03143
AudioLM 将音频生成建模为语言建模任务,结合 w2v-BERT 语义词元与 SoundStream 声学词元的混合分词方案,同时实现长期一致性与高质量语音及钢琴续写合成。
[Borsos et al. 2023] Borsos, Sharifi, Vincent, Kharitonov, Zeghidour, Tagliasacchi. SoundStorm: Efficient parallel audio generation. arXiv preprint arXiv:2305.09636. https://arxiv.org/abs/2305.09636
SoundStorm 是一个非自回归生成音频模型,通过对 RVQ 词元序列的双向注意力与置信度并行解码,生成速度比 AudioLM 快两个数量级。
[Bourtoule et al. 2021] Bourtoule, Chandrasekaran, Choquette-Choo, Jia, Travers, Zhang, Lie, Papernot. Machine unlearning. https://arxiv.org/abs/1912.03817
本文提出 SISA 训练框架,通过将训练数据分片(shard)与分片段(slice)来降低机器遗忘的重训练开销,最高可达完整重训练速度的 4.63 倍。
[Bouthillier et al. 2021] Bouthillier, Delaunay, Bronzi, Trofimov, Nichyporuk, Szeto, Sepah, Raff, Madan, Voleti, Kahou, Michalski, Arbel, Pal, Varoquaux, Vincent. Accounting for variance in machine learning benchmarks. https://proceedings.mlsys.org/paper_files/paper/2021/hash/0184b0cd3cfb185989f858a1d9f5c1eb-Abstract.html
该研究衡量了数据划分、初始化、优化和实现选择分别会给机器学习基准结果带来多少方差。
[Bowman et al. 2022] Bowman, Hyun, Perez, Chen, Pettit, Heiner, Lukošiūtė, Askell, Jones, Chen, Goldie, Mirhoseini, McKinnon, Olah, Amodei, Amodei, Drain, Li, Tran-Johnson, Kernion, Kerr, Mueller, Ladish, Landau, Ndousse, Lovitt, Elhage, Schiefer, Joseph, Mercado, DasSarma, Larson, McCandlish, Kundu, Johnston, Kravec, El Showk, Fort, Telleen-Lawton, Brown, Henighan, Hume, Bai, Hatfield-Dodds, Mann, Kaplan. Measuring progress on scalable oversight for large language models. arXiv preprint arXiv:2211.03540. https://arxiv.org/abs/2211.03540
该论文提出一种用于可扩展监督的夹心式实证设计,并在 MMLU 和限时 QuALITY 测试中发现,借助不可靠模型的人类表现可以超过单独的人类或模型。
[Bradford 2020] Bradford. The brussels effect: How the european union rules the world. Oxford University Press.
Bradford 论述了布鲁塞尔效应,即欧盟如何凭借市场准入设定事实上的全球标准,从而向全球输出其监管规则。
[Bradley and Terry 1952] Bradley, Terry. Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika. https://doi.org/10.1093/biomet/39.3-4.324
Bradley 与 Terry 推导成对比较的潜在分数概率模型,构成现代批量偏好排名的基础。
[Breck et al. 2017] Breck, Cai, Nielsen, Salib, Sculley. The ML test score: a rubric for ML production readiness and technical debt reduction. https://research.google/pubs/pub46555/
ML Test Score 提供覆盖数据、模型、基础设施与监控测试的生产就绪评分准则。
[Breck et al. 2017] Breck, Cai, Nielsen, Salib, Sculley. The ML test score: a rubric for ML production readiness and technical debt reduction. https://research.google/pubs/the-ml-test-score-a-rubric-for-ml-production-readiness-and-technical-debt-reduction/
这套生产就绪度量表把数据、模型、基础设施和监控方面的假设转化为明确的测试。
[Bricken et al. 2023] Bricken, Templeton, Batson, Chen, Jermyn, Conerly, Turner, Anil, Denison, Askell, Lasenby, Wu, Kravec, Schiefer, Maxwell, Joseph, Hatfield-Dodds, Tamkin, Nguyen, McLean, Burke, Hume, Carter, Henighan, Olah. Towards monosemanticity: Decomposing language models with dictionary learning. https://transformer-circuits.pub/2023/monosemantic-features/index.html
在单层 Transformer 的前馈网络激活上使用字典学习,可恢复出许多比单个神经元更易解释的特征,但特征拆分、完备性与可推广性仍是开放问题。
[Brill 2024] Brill. Neural scaling laws rooted in the data distribution. arXiv preprint arXiv:2412.07942. https://arxiv.org/abs/2412.07942
用自然数据上的渗流理论从第一性原理同时推导出扩展的离散量子图景与数据流形图景,将两种解释统一为渗流阈值两侧的不同区制。
[Broder 1997] Broder. On the resemblance and containment of documents. IEEE. https://doi.org/10.1109/SEQUEN.1997.666900
Broder 用词片集合和随机采样形式化定义了文档相似度与包含度,奠定了后来称为 MinHash 的文档相似性方法。
[Brohan et al. 2023] Brohan, Brown, others. RT-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818. https://arxiv.org/abs/2307.15818
RT-2 通过将机器人动作编码为文本词元,对大型视觉语言模型(VLM)在机器人轨迹数据上进行联合微调,得到的视觉-语言-动作模型(VLA)能泛化到新物体并展现涌现语义推理能力。
[Brooker 2019] Brooker. Timeouts, retries, and backoff with jitter. https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/
AWS 说明了如何选择超时时间,并组合有限重试、指数退避和随机抖动,以免进一步放大下游服务的过载。
[Brooker et al. 2021] Brooker, Catangiu, Danilov, Graf, MacCárthaigh, Sandu. Restoring uniqueness in MicroVM snapshots. CoRR. https://arxiv.org/abs/2102.12892
恢复克隆的 microVM 快照可能复制随机状态与标识符;该论文设计了在恢复后重建唯一性的机制。
[Brooks et al. 2024] Brooks, Peebles, others. Video generation models as world simulators. https://openai.com/index/video-generation-models-as-world-simulators/
Sora 技术报告:在视频潜变量的时空图块上运行的扩散 Transformer,以算力为提升样本质量的坐标轴,可生成最长约一分钟的视频。
[Brown et al. 2001] Brown, Cai, DasGupta. Interval estimation for a binomial proportion. Statistical Science. https://projecteuclid.org/journals/statistical-science/volume-16/issue-2/Interval-Estimation-for-a-Binomial-Proportion/10.1214/ss/1009213286.full
Brown、Cai 与 DasGupta 说明常见 Wald 区间的覆盖率并不稳定,并推荐了包括 Wilson 得分区间在内的更可靠方法。
[Brown et al. 2020] Brown, Mann, Ryder, Subbiah, Kaplan, Dhariwal, Neelakantan, Shyam, Sastry, Askell, Agarwal, Herbert-Voss, Krueger, Henighan, Child, Ramesh, Ziegler, Wu, Winter, Hesse, Chen, Sigler, Litwin, Gray, Chess, Clark, Berner, McCandlish, Radford, Sutskever, Amodei. Language models are few-shot learners. https://proceedings.neurips.cc/paper_files/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html
GPT-3 是一个 1750 亿参数的自回归大语言模型,无需梯度更新或微调,仅通过上下文示例即可在 NLP 基准上实现强大的少样本性能。
[Brown et al. 2024] Brown, Juravsky, Ehrlich, Clark, Le, Ré, Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787. https://arxiv.org/abs/2407.21787
在多个模型与任务中,重复采样会提高候选覆盖率;在实测范围内,曲线常可用指数化幂律拟合,但实际收益取决于选择机制。
[Brown-Cohen et al. 2023] Brown-Cohen, Irving, Piliouras. Scalable AI safety via doubly-efficient debate. arXiv preprint arXiv:2311.14125. https://arxiv.org/abs/2311.14125
该论文在明确的计算、预言机和策略假设下,证明了若干辩论博弈的完备性与可靠性结果。
[Brynjolfsson et al. 2021] Brynjolfsson, Rock, Syverson. The productivity j-curve: How intangibles complement general purpose technologies. American Economic Journal: Macroeconomics. https://doi.org/10.1257/mac.20180386
作者通过模型和证据说明,对流程及其他无形资本的高成本投资最初可能掩盖通用技术带来的产出收益。
[Brynjolfsson et al. 2025] Brynjolfsson, Li, Raymond. Generative AI at work. The Quarterly Journal of Economics. https://doi.org/10.1093/qje/qjae044
在一家客服业务中,对话式助手提高了每小时解决的问题数,经验较少和技能较低的坐席收益更大。
[Btrfs Maintainers 2026] Btrfs Maintainers. Btrfs design. https://btrfs.readthedocs.io/en/stable/dev/dev-btrfs-design.html
Btrfs 快照最初共享根块,并通过写时复制使后续修改分别归属于各自根。
[Buçinca et al. 2021] Buçinca, Malaya, Gajos. To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. https://arxiv.org/abs/2102.09692
Buçinca、Malaya 与 Gajos 证明认知强制干预比简单解释展示更能降低对 AI 建议的过度依赖,但会带来可用性取舍。
[Burns et al. 2016] Burns, Grant, Oppenheimer, Brewer, Wilkes. Borg, omega, and kubernetes. ACM Queue. https://research.google/pubs/borg-omega-and-kubernetes/
该文追溯了 Borg 和 Omega 中的思想如何进入 Kubernetes,包括共享集群状态、协调循环、调度和可扩展控制机制。
[Burns et al. 2024] Burns, Izmailov, Kirchner, Baker, Gao, Aschenbrenner, Chen, Ecoffet, Joglekar, Leike, Sutskever, Wu. Weak-to-strong generalization: Eliciting strong capabilities with weak supervision. PMLR. https://proceedings.mlr.press/v235/burns24b.html
在 NLP、国际象棋和奖励建模中,弱模型标签引出了一部分强模型能力,但朴素微调仍远低于完整强模型监督。
[Cai et al. 2024] Cai, Li, Geng, Peng, Lee, Chen, Dao. Medusa: Simple LLM inference acceleration framework with multiple decoding heads. PMLR. https://proceedings.mlr.press/v235/cai24b.html
Medusa 增加未来词元预测头和树形注意力来生成并验证多个候选;其精确配置与典型接受配置对应不同的输出契约。
[California State Legislature 2025] California State Legislature. SB 53: Transparency in frontier artificial intelligence act. https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202520260SB53
加利福尼亚州 SB 53 法案为适用范围内的前沿 AI 开发者规定了透明度和安全报告义务,并明确州级监督机制。
[Campbell et al. 2020] Campbell, Bradley, Tschofenig. RFC 8707: Resource indicators for OAuth 2.0. https://datatracker.ietf.org/doc/html/rfc8707
RFC 8707 定义了 OAuth 2.0 扩展,新增请求参数,使客户端能够向授权服务器明确指示其请求访问的受保护资源。
[Campbell et al. 2020] Campbell, Bradley, Tschofenig. Resource indicators for OAuth 2.0. https://www.rfc-editor.org/rfc/rfc8707.html
RFC 8707 允许客户端指明目标受保护资源,使授权服务器能够签发受众范围受限的令牌。
[Cappos et al. 2026] Cappos, Kuppusamy, Lock, Moore, Pühringer. The update framework specification. https://theupdateframework.github.io/specification/latest/
TUF 规定了带签名的软件更新元数据,以及对回滚、冻结、混搭、密钥泄露等更新攻击的防护。
[Card et al. 2020] Card, Henderson, Khandelwal, Jia, Mahowald, Jurafsky. With little power comes great responsibility. Association for Computational Linguistics. https://aclanthology.org/2020.emnlp-main.745/
Card 等人发现 NLP 实验普遍功效不足,并说明低功效既使有意义的差异更难检出,也会夸大被筛选出来的正面结果。
[Carlini et al. 2019] Carlini, Athalye, Papernot, Brendel, Rauber, Tsipras, Goodfellow, Madry, Kurakin. On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705. https://arxiv.org/abs/1902.06705
这份评测指南主张,鲁棒性声明需要明确威胁模型、强攻击、自适应测试与可复现报告。
[Carlini et al. 2019] Carlini, Liu, Erlingsson, Kos, Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. USENIX Association. https://www.usenix.org/conference/usenixsecurity19/presentation/carlini
该论文提出金丝雀暴露度,用于量化测试生成模型是否无意记住了罕见序列。
[Carlini et al. 2021] Carlini, Tramèr, Wallace, Jagielski, Herbert-Voss, Lee, Roberts, Brown, Song, Erlingsson, Oprea, Raffel. Extracting training data from large language models. https://arxiv.org/abs/2012.07805
Carlini 等人证明大语言模型会记忆训练数据,并通过黑盒查询 GPT-2 提取出包含个人可识别信息在内的逐字文本。
[Carlini et al. 2023] Carlini, Ippolito, Jagielski, Lee, Tramer, Zhang. Quantifying memorization across neural language models. https://arxiv.org/abs/2202.07646
Carlini 等人量化了大语言模型训练数据记忆的三条对数线性规律:记忆程度随模型容量、数据重复次数和上下文长度增大而增大,且比此前认为的更为普遍。
[Carlini et al. 2024] Carlini, Jagielski, Choquette-Choo, Paleka, Pearce, Anderson, Terzis, Thomas, Tramèr. Poisoning web-scale training datasets is practical. IEEE. https://doi.org/10.1109/SP54263.2024.00179
论文展示可变 URL 与数据集快照如何被操纵;约六十美元的估算指控制两个图文数据集中 0.01% URL 返回的内容,并非已演示的模型后门。
[Castro and Liskov 1999] Castro, Liskov. Practical byzantine fault tolerance. USENIX Association. https://www.usenix.org/conference/osdi-99/presentation/practical-byzantine-fault-tolerance
PBFT 在明确的成员关系、故障上限、通信模型和活性假设下,实现了复制状态机的一致性。
[Center for AI Safety et al. 2026] Center for AI Safety, Scale AI, HLE Contributors Consortium. A benchmark of expert-level academic questions to assess AI capabilities. Nature. https://www.nature.com/articles/s41586-025-09962-4
Humanity's Last Exam(HLE)是一个包含 2500 道专家级题目的评测基准,覆盖多个领域,旨在对前沿大语言模型构成极限挑战。
[Chameleon Team 2024] Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09818. https://arxiv.org/abs/2405.09818
Chameleon 是一组早期融合的基于词元的混合模态基础模型,通过单一统一 Transformer 配合查询-键归一化(QK-norm)实现稳定训练,可理解并生成任意交错的图像与文本序列。
[Chan et al. 2016] Chan, Jaitly, Le, Vinyals. Listen, attend and spell. https://arxiv.org/abs/1508.01211
LAS 是一个端到端序列到序列自动语音识别系统,由金字塔 RNN 编码器和基于注意力的字符解码器组成,在 Google 语音搜索任务上无语言模型时达到 14.1% 词错误率。
[Chan et al. 2020] Chan, Saharia, Hinton, Norouzi, Jaitly. Imputer: Sequence modelling via imputation and dynamic programming. https://arxiv.org/abs/2002.08926
Imputer 是一种用于自动语音识别的迭代非自回归生成序列模型,通过动态规划近似边缘化所有对齐方式与生成顺序,以常数步解码在 LibriSpeech 上超越 CTC。
[Chandy and Lamport 1985] Chandy, Lamport. Distributed snapshots: Determining global states of distributed systems. ACM Transactions on Computer Systems. https://doi.org/10.1145/214451.214456
Chandy 和 Lamport 说明了如何在不中断分布式计算的情况下记录一致的全局状态。
[Chaney et al. 2018] Chaney, Stewart, Engelhardt. How algorithmic confounding in recommendation systems increases homogeneity and decreases utility. https://arxiv.org/abs/1710.11214
模拟结果表明,推荐结果与用户数据之间的反馈循环可能提高同质化程度,同时降低实际用户效用。
[Chanin et al. 2025] Chanin, Wilken-Smith, Dulka, Bhatnagar, Golechha, Bloom. A is for absorption: Studying feature splitting and absorption in sparse autoencoders. https://proceedings.neurips.cc/paper_files/paper/2025/hash/764ff7477b8e24dbe01531f6791e8bdf-Abstract-Conference.html
本文发现稀疏自编码器(SAE)中的"特征吸收"现象:层次化特征导致 SAE 潜在特征在本应激活的词元上静默失效,从而破坏 LLM 机械可解释性的可靠性。
[Chao et al. 2024] Chao, Debenedetti, Robey, Andriushchenko, Croce, Sehwag, Dobriban, Flammarion, Pappas, Tramèr, Hassani, Wong. JailbreakBench: An open robustness benchmark for jailbreaking large language models. arXiv preprint arXiv:2404.01318. https://arxiv.org/abs/2404.01318
JailbreakBench 提供版本化行为集、对抗提示仓库、威胁模型、聊天模板与评分函数,以支持可复现的越狱评测。
[Chen and Guestrin 2016] Chen, Guestrin. XGBoost: a scalable tree boosting system. https://arxiv.org/abs/1603.02754
大规模梯度提升树的系统设计:缓存感知布局、稀疏处理与核外计算;论文自述 2015 年公开的 29 个 Kaggle 冠军方案中有 17 个使用了它。
[Chen et al. 2016] Chen, Xu, Zhang, Guestrin. Training deep nets with sublinear memory cost. https://arxiv.org/abs/1604.06174
本文分析激活重计算调度,以额外的前向计算换取次线性的激活保存内存。
[Chen et al. 2018] Chen, Moreau, Jiang, Zheng, Yan, Cowan, Shen, Wang, Hu, Ceze, Guestrin, Krishnamurthy. TVM: An automated end-to-end optimizing compiler for deep learning. https://arxiv.org/abs/1802.04799
TVM 把 Halide 的算法与调度分离带入深度学习,用学习到的成本模型让调度搜索自动化,其算子融合分类法沿用至今。
[Chen and others 2021] Chen, others. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. https://arxiv.org/abs/2107.03374
本文介绍了 Codex(在 GitHub 代码上微调的 GPT 模型),并发布了 HumanEval 基准,使用 pass@k 通过单元测试衡量代码生成的功能正确性。
[Chen et al. 2022] Chen, Wang, Chen, Wu, Liu, Chen, Li, Kanda, Yoshioka, Xiao, others. WavLM: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing. https://arxiv.org/abs/2110.13900
WavLM 是一种自监督语音预训练模型,通过联合学习掩码语音预测与去噪,在自动语音识别(ASR)、说话人验证、语音分离和说话人日志等全栈语音任务上达到最优性能。
[Chen et al. 2023] Chen, Wong, Chen, Tian. Extending context window of large language models via positional interpolation. arXiv preprint arXiv:2306.15595. https://arxiv.org/abs/2306.15595
通过线性缩放位置索引扩展基于 RoPE 的大语言模型上下文窗口,避免不稳定外推,并在少量微调后保留原窗口质量。
[Chen et al. 2023] Chen, Ma, Wang, Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. Transactions on Machine Learning Research. https://arxiv.org/abs/2211.12588
Program of Thoughts 提示让大语言模型把数值推理表达成可执行程序,从而把推理分解与精确计算分离。
[Chen et al. 2023] Chen, Borgeaud, Irving, Lespiau, Sifre, Jumper. Accelerating large language model decoding with speculative sampling. arXiv preprint arXiv:2302.01318. https://arxiv.org/abs/2302.01318
推测采样用一次目标模型调用验证草稿模型生成的短序列,并以校正分布保持目标分布不变。
[Chen et al. 2023] Chen, Zaharia, Zou. FrugalGPT: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176. https://arxiv.org/abs/2305.05176
FrugalGPT 研究模型级联等降低 API 成本的方法,同时测量任务表现,而不是假定较便宜的路由效果等价。
[Chen et al. 2024] Chen, Liu, Zhou, Liu, Tan, Li, Zhao, Qian, Wei. VALL-e 2: Neural codec language models are human parity zero-shot text to speech synthesizers. arXiv preprint arXiv:2406.05370. https://arxiv.org/abs/2406.05370
VALL-E 2 是一个神经编解码语言模型,通过重复感知采样和分组编码建模实现零样本 TTS 的人类同等水平,在 LibriSpeech 和 VCTK 上达到人类对等性。
[Chen et al. 2024] Chen, Niu, Ma, Deng, Wang, Zhao, Yu, Chen. F5-TTS: a fairytaler that fakes fluent and faithful speech with flow matching. arXiv preprint arXiv:2410.06885. https://arxiv.org/abs/2410.06885
F5-TTS 是一个基于流匹配与扩散 Transformer(DiT)的非自回归生成文本到语音(TTS)系统,无需音素对齐或时长模型即可实现零样本语音克隆。
[Chen et al. 2024] Chen, Wang, Tian, Ye, Gao, Cui, Tong, Hu, Luo, Ma, others. How far are we to GPT-4V? Closing the gap to commercial multimodal models with open-source suites. arXiv preprint arXiv:2404.16821. https://arxiv.org/abs/2404.16821
InternVL 1.5 将扩展后的视觉编码器与语言模型结合,并在多种视觉任务上评估这一开放多模态系统。
[Chen et al. 2024] Chen, Zhao, Liu, Bai, Lin, Zhou, Chang. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. https://www.ecva.net/papers/eccv_2024/papers_ECCV/html/10478_ECCV_2024_paper.php
FastV 根据注意力分数在第 2 层后剪枝视觉语言模型中的视觉词元,在几乎不损失性能的情况下实现 45% 的 FLOPs 降低。
[Chen et al. 2024] Chen, Xiang, Xiao, Song, Li. AgentPoison: Red-teaming LLM agents via poisoning memory or knowledge bases. arXiv preprint arXiv:2407.12784. https://proceedings.neurips.cc/paper_files/paper/2024/hash/eb113910e9c3f6242541c1652e30dfd6-Abstract-Conference.html
AgentPoison 评估对智能体长期记忆或检索知识库实施投毒的定向后门攻击。
[Chen et al. 2024] Chen, Waiwitlikhit, Stoica, Kang. ZKML: An optimizing system for ML inference in zero-knowledge proofs. https://doi.org/10.1145/3627703.3650088
展示如何以零知识方式证明机器学习推理声明;它本身并不向执行普通推理的证明方隐藏用户提示。
[Chen et al. 2025] Chen, Hao, Liu, Huang, Zeng, Yu, Li, Huang, Liu, Wang, Liu. ACEBench: a comprehensive evaluation of LLM tool usage. Association for Computational Linguistics. https://aclanthology.org/2025.findings-emnlp.697/
ACEBench 评测基础工具调用、含糊或不完整指令,以及带细粒度错误分类的模拟多轮交互。
[Chen et al. 2025] Chen, Piet, Sitawarin, Wagner. StruQ: Defending against prompt injection with structured queries. USENIX Association. https://www.usenix.org/conference/usenixsecurity25/presentation/chen-sizhe
StruQ 把结构化前端与专门训练的模型结合,使模型能够区分指令和数据,而不是只依赖分隔符。
[Cheng et al. 2024] Cheng, Ozga, Valdez, Ahmed, Gu, Jamjoom, Franke, Bottomley. Intel TDX demystified: a top-down approach. ACM Computing Surveys. https://doi.org/10.1145/3652597
对 Intel 信任域虚拟机自顶向下的学术剖析:架构、证明流程与信任边界,读者无需去啃厂商规范。
[Chervonyi et al. 2025] Chervonyi, Trinh, Olsak, Yang, Nguyen, Menegali, Jung, Kim, Verma, Le, Luong. Gold-medalist performance in solving olympiad geometry with AlphaGeometry2. arXiv preprint arXiv:2502.03544. https://arxiv.org/abs/2502.03544
AlphaGeometry2 提升了奥数几何的语言覆盖与符号搜索能力,展示了神经提议和符号验证如何协同。
[Chetlur et al. 2014] Chetlur, Woolley, Vandermersch, Cohen, Tran, Catanzaro, Shelhamer. cuDNN: Efficient primitives for deep learning. arXiv preprint arXiv:1410.0759. https://arxiv.org/abs/1410.0759
把 2010 年代所有框架焊在 NVIDIA 硅片上的库:Caffe、Torch 及其后继者底下的深度学习原语,恰好出现在这个领域工业化的时刻。
[Chhabria 2025] Chhabria. Kadrey v. meta platforms, inc., order on summary judgment, no. 3:23-cv-03417. United States District Court for the Northern District of California. https://law.justia.com/cases/federal/district-courts/california/candce/3:2023cv03417/415175/598/
就具名作者的复制与训练主张判 Meta 胜诉,同时强调缺乏有意义的市场损害证据。
[Chi et al. 2023] Chi, Feng, Du, Xu, Cousineau, Burchfiel, Song. Diffusion policy: Visuomotor policy learning via action diffusion. https://arxiv.org/abs/2303.04137
Diffusion Policy 对连续动作序列的多模态分布建模,并通过滚动时域视觉反馈执行动作。
[Chiang et al. 2024] Chiang, Zheng, Sheng, Angelopoulos, Li, Li, Zhu, Zhang, Jordan, Gonzalez, Stoica. Chatbot arena: An open platform for evaluating LLMs by human preference. PMLR. https://proceedings.mlr.press/v235/chiang24b.html
Chatbot Arena 构建大规模人工成对偏好平台和统计排名方法;其估计取决于抽样用户、提示、模型池与协议。
[Chollet et al. 2025] Chollet, Knoop, Kamradt, Landers, Pinkard. ARC-AGI-2: a new challenge for frontier AI reasoning systems. arXiv preprint arXiv:2505.11831. https://arxiv.org/abs/2505.11831
ARC-AGI-2 是 ARC-AGI-1 的升级版基准测试,包含更难且不易被暴力搜索攻克的抽象推理任务,并提供大规模第一方人类基线。
[Chow 1970] Chow. On optimum recognition error and reject tradeoff. IEEE Transactions on Information Theory. https://doi.org/10.1109/TIT.1970.1054406
Chow 形式化识别错误与拒绝之间的权衡,构成评测带弃答预测策略的经典基础。
[Chowdhery et al. 2023] Chowdhery, Narang, Devlin, Bosma, Mishra, Roberts, Barham, Chung, Sutton, Gehrmann, others. PaLM: Scaling language modeling with pathways. Journal of Machine Learning Research. https://arxiv.org/abs/2204.02311
介绍 540B 参数的 PaLM,定义大规模训练的模型 FLOPs 利用率,并记录其分布式训练系统与稳定性措施。
[Chowdhery et al. 2023] Chowdhery, Narang, Devlin, Bosma, Mishra, Roberts, Barham, Chung, Sutton, Gehrmann, others. PaLM: Scaling language modeling with pathways. Journal of Machine Learning Research. https://jmlr.org/papers/volume24/22-1144/22-1144.pdf
PaLM 论文将模型 FLOPs 利用率定义为模型级效率指标,并在有用工作口径中排除重物化计算。
[Christiano et al. 2017] Christiano, Leike, Brown, Martic, Legg, Amodei. Deep reinforcement learning from human preferences. https://arxiv.org/abs/1706.03741
本文表明,深度强化学习智能体可通过非专家对轨迹片段成对比较(基于人类反馈的强化学习)的偏好来学习复杂行为,所需反馈量不足环境交互次数的 1%。
[Christiano et al. 2018] Christiano, Shlegeris, Amodei. Supervising strong learners by amplifying weak experts. arXiv preprint arXiv:1810.08575. https://arxiv.org/abs/1810.08575
迭代放大提出递归分解,并报告了使用硬编码分解器完成五类合成组合任务的实验。
[Chromium Project 2026] Chromium Project. Site isolation. https://www.chromium.org/Home/chromium-security/site-isolation/
Chromium 对站点隔离的说明:把跨站文档放进不同进程以限制被攻陷的渲染器,但不能替代环境隔离。
[Chung et al. 2021] Chung, Zhang, Han, Chiu, Qin, Pang, Wu. w2v-BERT: Combining contrastive learning and masked language modeling for self-supervised speech pre-training. https://arxiv.org/abs/2108.06209
w2v-BERT 将 wav2vec 2.0 的对比学习与掩码语言建模合并为单一端到端自监督框架,用于语音表示学习,在 LibriSpeech 上取得有竞争力的自动语音识别(ASR)结果。
[Clark et al. 2021] Clark, August, Serrano, Haduong, Gururangan, Smith. All that's 'human' is not gold: Evaluating human evaluation of generated text. Association for Computational Linguistics. https://aclanthology.org/2021.acl-long.565/
Clark 等人表明,未训练评审常常只能以随机水平识别 GPT-3 生成文本,因此需要评审训练与更清晰的协议。
[Clark et al. 2022] Clark, Garrette, Turc, Wieting. CANINE: Pre-training an efficient tokenization-free encoder for language representation. Transactions of the Association for Computational Linguistics. https://aclanthology.org/2022.tacl-1.5/
CANINE 直接编码 Unicode 字符,并在深层 Transformer 编码器之前使用卷积进行下采样。
[Coalition for Content Provenance and Authenticity 2026] Coalition for Content Provenance and Authenticity. C2PA technical specification. https://c2pa.org/specifications/
C2PA 规范定义了带签名的来源清单,用于记录媒体资产的起源和编辑过程,但不声称内容本身真实。
[Cobbe et al. 2021] Cobbe, Kosaraju, Bavarian, Chen, Jun, Kaiser, Plappert, Tworek, Hilton, Nakano, Hesse, Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. https://arxiv.org/abs/2110.14168
Training Verifiers 提出 GSM8K,并展示在数学文字题上采样多个解法再用验证器选择,能胜过只直接微调生成器。
[Cohen 1960] Cohen. A coefficient of agreement for nominal scales. Educational and Psychological Measurement. https://doi.org/10.1177/001316446002000104
Cohen κ 衡量两位标注者扣除偶然一致后的协议程度,避免把原始一致率误当成可靠性。
[Cohen 2024] Cohen. Data scientists targeted by malicious hugging face ML models with silent backdoor. https://jfrog.com/blog/data-scientists-targeted-by-malicious-hugging-face-ml-models-with-silent-backdoor/
JFrog 报告其扫描器把约一百个 PyTorch 或 Keras 工件判为真正有害;其中一个受调查 pickle 检查点含反向 shell,但未报告受害者数量。
[Colorado General Assembly 2026] Colorado General Assembly. SB 26-189: Consumer protections for artificial intelligence interactions. https://www.leg.colorado.gov/bills/SB26-189
科罗拉多州 SB 26-189 法案规定了特定 AI 交互的披露和消费者保护义务,并明确了执法责任。
[Competition and Markets Authority 2024] Competition and Markets Authority. AI foundation models: Technical update report. https://www.gov.uk/government/publications/ai-foundation-models-update-paper
CMA 把算力、数据、专业能力、资金和市场渠道列为可能限制进入与扩张的因素,并将其表述为需要审查的竞争风险,而非已经证实的损害。
[Competition and Markets Authority 2025] Competition and Markets Authority. Appendix D: Market structure and concentration methodology and UK share of supply by revenue. https://assets.publishing.service.gov.uk/media/688b7ffafc784fa12a08906e/Appendix_D_-_Market_structure_and_concentration_methodology_and_UK_share_of_supply_by_revenue.pdf
CMA 估算了特定英国云服务类别的供应商收入份额,并说明加速计算收入无法从更广泛的总额中一致地单独拆分。
[Cormack et al. 2009] Cormack, Clarke, Büttcher. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. https://research.google/pubs/reciprocal-rank-fusion-outperforms-condorcet-and-individual-rank-learning-methods/
RRF 给出一种只依赖名次的简单融合规则,并在 TREC 与 LETOR 排名上进行评测。
[Costan and Devadas 2016] Costan, Devadas. Intel SGX explained. https://eprint.iacr.org/2016/086
Intel SGX 的权威解读:隔离区、度量与远程证明在硅片层面究竟如何运作,至今仍是理解可信执行的最佳入门。
[Coste et al. 2023] Coste, Anwar, Kirk, Krueger. Reward model ensembles help mitigate overoptimization. arXiv preprint arXiv:2310.02743. https://arxiv.org/abs/2310.02743
在使用合成真实奖励的实验中,基于奖励模型集成的保守目标减少了 best-of-N 选择和 PPO 的代理奖励过度优化。
[Cottier et al. 2024] Cottier, Rahman, Fattorini, Maslej, Besiroglu, Owen. The rising costs of training frontier AI models. https://arxiv.org/abs/2405.21015
Cottier 等人估计若干前沿模型最终训练运行的加速器硬件与能源摊销成本。
[Cottier et al. 2025] Cottier, Snodin, Owen, Adamczewski. LLM inference prices have fallen rapidly but unequally across tasks. https://epoch.ai/data-insights/llm-inference-price-trends
Epoch AI 比较固定基准阈值下的 API 标价,发现观测到的降价速度随任务、价格权重与时间窗口而显著不同。
[Council of Europe 2024] Council of Europe. Framework convention on artificial intelligence and human rights, democracy and the rule of law. https://www.coe.int/en/web/conventions/full-list?module=signatures-by-treaty&treatynum=225
欧洲委员会公约建立了一套条约框架,要求 AI 治理与人权、民主和法治保持一致。
[Covington et al. 2016] Covington, Adams, Sargin. Deep neural networks for YouTube recommendations. https://dl.acm.org/doi/10.1145/2959100.2959190
YouTube 记录了一种两阶段推荐架构:候选模型先把数百万视频缩减到数百个,再由排序模型选出几十个展示。
[D'Amour et al. 2022] D'Amour, Heller, Moldovan, Adlam, Alipanahi, Beutel, Chen, Deaton, Eisenstein, Hoffman, others. Underspecification presents challenges for credibility in modern machine learning. Journal of Machine Learning Research. https://jmlr.org/papers/v23/20-1335.html
本文指出现代机器学习流水线常常欠定:许多预测器留出性能相近,却在部署环境中表现不同。
[Dabney et al. 2020] Dabney, Kurth-Nelson, Uchida, Starkweather, Hassabis, Munos, Botvinick. A distributional code for value in dopamine-based reinforcement learning. Nature.
Dabney 等人发现了异质的多巴胺反应,与群体编码折扣回报分布的解释一致。
[Dahlgren et al. 2023] Dahlgren, Hussain, Khlaaf, Sultanik. Safetensors library security assessment. https://huggingface.co/datasets/safetensors/trail_of_bits_audit_repot/resolve/main/SOW-TrailofBits-EleutherAI_HuggingFace-v1.2.pdf
审计在所报告的校验、溢出、解析器与转换问题修复后,未在被审查加载器中发现严重代码执行路径,但明确不声称证明风险不存在。
[Dai et al. 2024] Dai, Deng, Zhao, Xu, Gao, Chen, Li, Zeng, Yu, Wu, Xie, Li, Huang, Luo, Ruan, Sui, Liang. DeepSeekMoE: Towards ultimate expert specialization in mixture-of-experts language models. Association for Computational Linguistics. https://arxiv.org/abs/2401.06066
DeepSeekMoE 通过细粒度专家分割和共享专家隔离两项策略,提升混合专家(MoE)语言模型的专家专业化程度,以更少计算量达到稠密模型的性能水平。
[Daly 2006] Daly. A higher order estimate of the optimum checkpoint interval for restart dumps. Future Generation Computer Systems. https://doi.org/10.1016/j.future.2004.11.016
Daly 推导了泊松故障条件下更高阶的检查点间隔,并指出一阶近似会在哪些情况下失准。
[Daly 2006] Daly. A higher order estimate of the optimum checkpoint interval for restart dumps. Future Generation Computer Systems. https://laro.lanl.gov/esploro/outputs/journalArticle/A-higher-order-estimate-of-the/9916364420003761
Daly 推导了泊松故障条件下更高阶的检查点间隔,并指出一阶近似会在哪些情况下失准。
[Dao et al. 2022] Dao, Fu, Ermon, Rudra, Ré. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. https://arxiv.org/abs/2205.14135
FlashAttention 是一种 IO 感知的精确注意力算法,通过分块(tiling)与重计算减少高带宽显存(HBM)访问次数,实现更快的实际训练速度并将显存占用降至序列长度线性级别。
[Dao and Gu 2024] Dao, Gu. Transformers are ssms: Generalized models and efficient algorithms through structured state space duality. PMLR. https://arxiv.org/abs/2405.21060
Mamba-2 通过半可分矩阵建立状态空间模型(SSM)与注意力之间的结构化状态空间对偶(SSD)框架,使 SSM 层训练速度提升 2-8 倍,在语言建模上与 Transformer 竞争。
[Dao 2024] Dao. FlashAttention-2: Faster attention with better parallelism and work partitioning. https://proceedings.iclr.cc/paper_files/paper/2024/hash/98ed250b203d1ac6b24bbcf263e3d4a7-Abstract-Conference.html
FlashAttention-2 通过改进 GPU 线程块工作分区与序列长度维度并行,相较 FlashAttention 实现约 2 倍加速,在 A100 上达到理论峰值 FLOPs/s 的 50–73%。
[Dathathri et al. 2024] Dathathri, See, Ghaisas, Huang, McAdam, Welbl, Bachani, Kaskasoli, Stanforth, Matejovicova, Hayes, Vyas, Merey, Brown-Cohen, Bunel, Balle, Cemgil, Ahmed, Stacpoole, Shumailov, Baetu, Gowal, Hassabis, Kohli. Scalable watermarking for identifying large language model outputs. Nature. https://www.nature.com/articles/s41586-024-08025-4
SynthID-Text 把生成时水印做成了生产系统,延迟开销可忽略,在 Gemini 线上流量上完成了两千万次响应的实测,并已开源。
[Davidson 1970] Davidson. On extending the bradley–terry model to accommodate ties in paired comparison experiments. Journal of the American Statistical Association. https://doi.org/10.1080/01621459.1970.10481082
Davidson 为成对比较排名加入明确的平局结果与参数,而不是把平局静默转换为二元胜负。
[Dean and Ghemawat 2004] Dean, Ghemawat. MapReduce: Simplified data processing on large clusters. USENIX Association. https://www.usenix.org/conference/osdi-04/mapreduce-simplified-data-processing-large-clusters
MapReduce 展示了如何把任务分解、中间数据、调度、故障恢复和归约整合为一套运行时契约。
[Dean and Barroso 2013] Dean, Barroso. The tail at scale. Communications of the ACM. https://research.google/pubs/the-tail-at-scale/
Dean 与 Barroso 解释了尾延迟为何主导大型分布式服务,并介绍了包括对冲请求在内的尾延迟容忍技术。
[Debenedetti et al. 2024] Debenedetti, Zhang, Balunovic, Beurer-Kellner, Fischer, Tramèr. AgentDojo: a dynamic environment to evaluate prompt injection attacks and defenses for LLM agents. https://proceedings.nips.cc/paper_files/paper/2024/hash/97091a5177d8dc64b1da8bf3e1f6fb54-Abstract-Datasets_and_Benchmarks_Track.html
AgentDojo 是一个可扩展的基准框架,包含 97 个真实任务和 629 个安全测试用例,用于评估 LLM 智能体对提示注入攻击的对抗鲁棒性。
[Debenedetti et al. 2025] Debenedetti, Shumailov, Fan, Hayes, Carlini, Fabian, Kern, Shi, Terzis, Tramèr. Defeating prompt injections by design. arXiv preprint arXiv:2503.18813. https://arxiv.org/abs/2503.18813
CaMeL 是一种针对 LLM 智能体提示注入攻击的系统层防御,通过对控制流和数据流强制执行基于能力的安全策略,无需修改底层模型,在 AgentDojo 上以可证明安全性解决 77% 的任务。
[DeepSeek-AI 2024] DeepSeek-AI. DeepSeek-V3 technical report. https://arxiv.org/abs/2412.19437
介绍 DeepSeek-V3,一个 671B 参数、每词元激活 37B 的混合专家模型,用 14.8T 词元与 fp8 矩阵乘法训练,采用无辅助损失的负载均衡,以低成本比肩闭源模型。
[DeepSeek-AI 2024] DeepSeek-AI. DeepSeek-V2: a strong, economical, and efficient mixture-of-experts language model. https://arxiv.org/abs/2405.04434
DeepSeek-V2 是一个 236B 混合专家(MoE)语言模型,通过多头潜在注意力(MLA)和 DeepSeekMoE 将 KV 缓存压缩 93.3%,并将训练成本降低 42.5%。
[DeepSeek-AI 2025] DeepSeek-AI. DeepSeek-V3.2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556. https://arxiv.org/abs/2512.02556
提出 DeepSeek 稀疏注意力(DSA),一种由 lightning indexer 驱动的细粒度稀疏注意力,最早随 DeepSeek-V3.2-Exp 上线,把长上下文注意力成本从 O(L^2) 降向 O(Lk),输出质量几乎不变。
[DeepSeek-AI 2026] DeepSeek-AI. DeepSeek-V4: Towards highly efficient million-token context intelligence. https://arxiv.org/abs/2606.19348
DeepSeek-V4 介绍了旨在提高百万令牌上下文处理效率的架构和训练设计。
[Défossez et al. 2023] Défossez, Copet, Synnaeve, Adi. High fidelity neural audio compression. Transactions on Machine Learning Research (TMLR). https://arxiv.org/abs/2210.13438
EnCodec 是一种实时神经音频编解码器,采用流式编解码器架构、残差向量量化与对抗训练,在 1.5 至 24 kbps 码率下达到最优压缩质量。
[Défossez et al. 2024] Défossez, Mazaré, Orsini, Royer, Pérez, Jégou, Grave, Zeghidour. Moshi: a speech-text foundation model for real-time dialogue. arXiv preprint arXiv:2410.00037. https://arxiv.org/abs/2410.00037
Moshi 是一个实时全双工语音-文本基础模型,通过并行音频流和"内心独白"方法直接进行语音到语音生成,理论延迟达 160ms。
[Dehghani et al. 2021] Dehghani, Tay, Gritsenko, Zhao, Houlsby, Diaz, Metzler, Vinyals. The benchmark lottery. arXiv preprint arXiv:2107.07002. https://arxiv.org/abs/2107.07002
作者在多个机器学习领域表明,任务、数据集和指标的选择会显著改变比较结论与排名。
[Delétang et al. 2024] Delétang, Ruoss, Duquenne, Catt, Genewein, Mattern, Grau-Moya, Wenliang, Aitchison, Orseau, Hutter, Veness. Language modeling is compression. https://openreview.net/forum?id=jznbgiynus
连接熵编码器且不计模型权重时,Chinchilla 70B 将研究中的图像与音频分块无损编码至原始大小的 48.0% 与 21.0%,低于对应分块的 PNG 与 FLAC 码率。
[Dell'Acqua et al. 2026] Dell'Acqua, McFowland, Mollick, Lifshitz-Assaf, Kellogg, Rajendran, Krayer, Candelon, Lakhani. Navigating the jagged technological frontier: Field experimental evidence of the effects of AI on knowledge worker productivity and quality. Organization Science. https://doi.org/10.1287/orsc.2025.21838
咨询顾问在 GPT-4 能力边界内的选定任务上提高了速度和质量,却在刻意选取的边界外任务上降低了正确率。
[Dettmers et al. 2022] Dettmers, Lewis, Belkada, Zettlemoyer. LLM.int8(): 8-bit matrix multiplication for transformers at scale. https://proceedings.neurips.cc/paper_files/paper/2022/file/c3ba4962c05c49636d4c6206a97e9c8a-Paper-Conference.pdf
LLM.int8() 将逐向量 INT8 矩阵乘法与激活离群维度的高精度路径结合,用于论文所评估的大型 Transformer。
[Dettmers et al. 2023] Dettmers, Pagnoni, Holtzman, Zettlemoyer. QLoRA: Efficient finetuning of quantized llms. https://proceedings.neurips.cc/paper_files/paper/2023/hash/1feb87871436031bdc0f2beaa62a049b-Abstract-Conference.html
QLoRA 通过将冻结的 4 位量化模型的梯度反向传播至 LoRA 低秩适配器,并引入 NF4、双重量化和分页优化器,实现在单张 48GB GPU 上对 65B 参数大语言模型进行参数高效微调。
[Dhanuskodi et al. 2023] Dhanuskodi, Guha, Krishnan, Manjunatha, Nertney, O'Connor, Rogers. Creating the first confidential gpus. ACM Queue. https://doi.org/10.1145/3623393.3623391
介绍 NVIDIA 第一代机密 GPU 的架构、受保护数据路径、证明机制与物理威胁假设。
[Dillon et al. 2026] Dillon, Jaffe, Immorlica, Stanton. Shifting work patterns with generative AI. American Economic Review: Insights. https://www.aeaweb.org/articles?id=10.1257/aeri.20250275
集成式 AI 使用权限减少了邮件会话时间,但没有检测到会议、Word 使用、完成文档或若干协调指标的平均变化。
[Ding et al. 2026] Ding, Lyu, Kataria, Singh. Opus: Photonic rail-optimized fabric in ML datacenters. arXiv preprint arXiv:2602.12521. https://arxiv.org/abs/2602.12521
Opus 在明确的训练开销模型下评估可重构光轨;证据为预印本,并非生产部署。
[Dodge et al. 2021] Dodge, Sap, Marasović, Agnew, Ilharco, Groeneveld, Mitchell, Gardner. Documenting large webtext corpora: a case study on the colossal clean crawled corpus. Association for Computational Linguistics. https://aclanthology.org/2021.emnlp-main.98/
对 C4 的审计发现了意外数据源、机器生成文本、基准样本和个人信息;基于屏蔽词的过滤还过度删除了少数群体创作或相关的文本。
[Dong et al. 2023] Dong, Xiong, Goyal, Zhang, Chow, Pan, Diao, Zhang, Shum, Zhang. RAFT: Reward rAnked FineTuning for generative foundation model alignment. Transactions on Machine Learning Research. https://arxiv.org/abs/2304.06767
RAFT(奖励排序微调)通过迭代采样模型输出、用奖励模型评分并仅对高分子集做监督微调,以稳定的方式替代 PPO 完成大语言模型与扩散模型的对齐。
[Dong et al. 2025] Dong, Ruan, Cai, Xu, Zhao, Lai, Chen. XGrammar: Flexible and efficient structured generation engine for large language models. https://proceedings.mlsys.org/paper_files/paper/2025/hash/5c20ca4b0b20b0bd2f1d839dc605e70f-Abstract-Conference.html
XGrammar 通过预检查词元、持久化解析栈,以及文法工作与加速器执行重叠来加速上下文无关文法。
[Dong et al. 2025] Dong, Lu, Zheng, Wu, Zhao, Tan, Huang, Hong, Wei, Chen, Zhou. PUMA: Secure inference of LLaMA-7B in five minutes. Security and Safety. https://arxiv.org/abs/2307.12533
给出三方 LLaMA-7B 安全推理及其协议、网络和工作负载测量;标题所指是实测推理任务,而非通用的每词元常数。
[Donnellan et al. 2025] Donnellan, Lawrence, Bizo, Rogers, Judge, Davis, Smolaks, Weinschenk. Uptime institute global data center survey 2025. https://intelligence.uptimeinstitute.com/resource/uptime-institute-global-data-center-survey-2025
该调查汇总了运营方对数据中心容量、故障、人员、可持续性和基础设施投资的实际观察。
[Douillard and others 2026] Douillard, others. Decoupled DiLoCo for resilient distributed pre-training. arXiv preprint arXiv:2604.21428. https://arxiv.org/abs/2604.21428
Decoupled DiLoCo 将 DiLoCo 框架扩展为完全异步的学习器架构,通过中央同步器采用最小仲裁数和词元加权合并,在持续硬件故障下实现零停机预训练,同时保持与数据并行基线相当的模型质量。
[Dror et al. 2018] Dror, Baumer, Shlomov, Reichart. The hitchhiker's guide to testing statistical significance in natural language processing. https://aclanthology.org/P18-1128/
Dror 等人综述了自然语言处理中的统计显著性检验,并为常见实验设计和指标匹配合适的检验方法。
[Du et al. 2022] Du, Huang, Dai, Tong, Lepikhin, Xu, Krikun, Zhou, Yu, Firat, Zoph, Fedus, Bosma, Zhou, Wang, Wang, Webster, Pellat, Robinson, Meier-Hellstern, Duke, Dixon, Zhang, Le, Wu, Chen, Cui. GLaM: Efficient scaling of language models with mixture-of-experts. PMLR. https://arxiv.org/abs/2112.06905
GLaM 通过稀疏激活的混合专家(MoE)将仅解码器语言模型扩展至 1.2T 参数,在 29 项 NLP 任务上超越 GPT-3,同时训练能耗仅为其三分之一。
[Du et al. 2024] Du, Li, Torralba, Tenenbaum, Mordatch. Improving factuality and reasoning in language models through multiagent debate. PMLR. https://proceedings.mlr.press/v235/du24e.html
这项早期实证研究报告多轮辩论在部分推理和事实性任务上有所提升,为这一方法提供了动机,但不能证明它普遍有效。
[Du et al. 2025] Du, Tian, Ronanki, Rongali, Bodapati, Galstyan, Wells, Schwartz, Huerta, Peng. Context length alone hurts LLM performance despite perfect retrieval. https://arxiv.org/abs/2510.05381
即使大语言模型(LLM)能完美检索所有相关内容,随着输入上下文长度增加,其推理性能仍大幅下降(13.9%–85%),表明上下文长度本身独立地损害模型性能。
[Duan et al. 2024] Duan, Suri, Mireshghallah, Min, Shi, Zettlemoyer, Tsvetkov, Choi, Evans, Hajishirzi. Do membership inference attacks work on large language models?. https://arxiv.org/abs/2402.07841
Duan 等人发现,所测试的大语言模型预训练成员推断攻击往往接近随机水平,并说明分布偏移会如何制造误导性结果。
[Dubois et al. 2024] Dubois, Galambosi, Liang, Hashimoto. Length-controlled AlpacaEval: a simple way to debias automatic evaluators. https://openreview.net/forum?id=CybBmzWBX0
Length-Controlled AlpacaEval 用回归估计回答长度差为零时的自动偏好,明确给出反事实估计量及其任务特定假设。
[Dwork and Roth 2014] Dwork, Roth. The algorithmic foundations of differential privacy. Now Publishers. https://doi.org/10.1561/0400000042
Dwork 与 Roth 系统建立了差分隐私的形式基础,包括相邻关系、隐私参数、组合性质与标准机制。
[Dwork et al. 2015] Dwork, Feldman, Hardt, Pitassi, Reingold, Roth. The reusable holdout: Preserving validity in adaptive data analysis. Science. https://pubmed.ncbi.nlm.nih.gov/26250683/
本文形式化说明普通留出保证为何会在自适应复用下失效,并提出在限制过拟合的同时回答重复查询的受控机制。
[Dwork et al. 2015] Dwork, Feldman, Hardt, Pitassi, Reingold, Roth. The reusable holdout: Preserving validity in adaptive data analysis. Science. https://doi.org/10.1126/science.aaa9375
可复用留出集框架说明,即使留出记录保持秘密,对评测结果的自适应重复访问仍会破坏普通留出推断。
[Edge et al. 2024] Edge, Trinh, Cheng, Bradley, Chao, Mody, Truitt, Metropolitansky, Ness, Larson. From local to global: a graph RAG approach to query-focused summarization. arXiv preprint arXiv:2404.16130. https://arxiv.org/abs/2404.16130
GraphRAG 从语料构建实体图与社区摘要,用于面向全局问题的查询式意义建构。
[Efron 1979] Efron. Bootstrap methods: Another look at the jackknife. The Annals of Statistics. https://doi.org/10.1214/aos/1176344552
Efron 提出 bootstrap 自助法,用重采样来估计不确定性,而不必为每个统计量推导封闭形式的抽样分布。
[El-Yaniv and Wiener 2010] El-Yaniv, Wiener. On the foundations of noise-free selective classification. Journal of Machine Learning Research. https://jmlr.csail.mit.edu/papers/v11/el-yaniv10a.html
该论文形式化定义了允许模型弃答的选择性分类,并分析预测覆盖率与已接受预测风险之间的关系。
[Elazar et al. 2024] Elazar, Bhagia, Magnusson, Ravichander, Schwenk, Suhr, Walsh, Groeneveld, Soldaini, Singh, Hajishirzi, Smith, Dodge. What's in my big data?. https://proceedings.iclr.cc/paper_files/paper/2024/hash/1f7336fd66b6e6e63d1801fdd5930a5a-Abstract-Conference.html
WIMBD 为大型文本语料提供可扩展的计数和搜索分析,并记录常用数据集中的重复、合成文本、低质量文本、个人信息、有害内容和基准污染。
[Eldan and Li 2023] Eldan, Li. TinyStories: How small can language models be and still speak coherent english?. https://arxiv.org/abs/2305.07759
TinyStories 提出一个由简单短故事构成的合成数据集,证明参数量低于 1000 万的语言模型也能生成流畅连贯的英文文本并涌现出推理能力。
[Elhage et al. 2021] Elhage, Nanda, Olsson, Henighan, Joseph, Mann, Askell, Bai, Chen, Conerly, DasSarma, Drain, Ganguli, Hatfield-Dodds, Hernandez, Jones, Kernion, Lovitt, Ndousse, Amodei, Brown, Clark, Kaplan, McCandlish, Olah. A mathematical framework for transformer circuits. https://transformer-circuits.pub/2021/framework/index.html
本文提出一个用于 Transformer 机械可解释性的数学框架,通过残差流、QK/OV 电路和归纳头对纯注意力模型进行逆向工程分析。
[Elhage et al. 2022] Elhage, Hume, Olsson, Schiefer, Henighan, Kravec, Hatfield-Dodds, Lasenby, Drain, Chen, Grosse, McCandlish, Kaplan, Amodei, Wattenberg, Olah. Toy models of superposition. https://arxiv.org/abs/2209.10652
玩具 ReLU 网络展示了网络在何种条件下可通过叠加表示多于可用维度的特征;这些玩具模型结果能否直接推广到大型语言模型仍是经验问题。
[Eloundou et al. 2024] Eloundou, Manning, Mishkin, Rock. GPTs are gpts: Labor market impact potential of llms. Science. https://doi.org/10.1126/science.adj0998
该论文估计在任务质量不变时语言模型可能影响多少职业任务时间;它并未测量实际工作场所效果。
[Elsworth et al. 2025] Elsworth, Patterson, Dean, others. Measuring the environmental impact of delivering AI at google scale. arXiv preprint arXiv:2508.15734. https://arxiv.org/abs/2508.15734
谷歌在生产环境中测量 Gemini Apps 推理的完整技术栈能耗、碳排放与水消耗,发现中位数文本提示消耗 0.24 Wh,一年内碳足迹降低 44 倍。
[Endsley 1995] Endsley. Toward a theory of situation awareness in dynamic systems. Human Factors. https://doi.org/10.1518/001872095779049543
Endsley 把情境感知形式化为动态决策中的构念,受感知、理解、预测、工作负载、复杂性与自动化影响。
[Enevoldsen and others 2025] Enevoldsen, others. MMTEB: Massive multilingual text embedding benchmark. https://proceedings.iclr.cc/paper_files/paper/2025/hash/fc0e3f908a2116ba529ad0a1530a3675-Abstract-Conference.html
MMTEB 是一个社区驱动的文本嵌入基准,涵盖 250 余种语言的 500 余个质量受控评测任务,并通过优化子集将计算量降低 98% 同时保持模型排名一致性。
[Engels et al. 2025] Engels, Baek, Kantamneni, Tegmark. Scaling laws for scalable oversight. arXiv preprint arXiv:2504.18530. https://arxiv.org/abs/2504.18530
本文提出一套量化框架,将可扩展监督建模为能力不匹配的 LLM 之间的博弈,推导出扩展律并分析嵌套可扩展监督(NSO)的最优配置。
[Epoch AI 2026] Epoch AI. FrontierMath tier 4 (v2). https://epoch.ai/benchmarks/frontiermath-tier-4-v2
FrontierMath v2 在重大修订后保留 338 道题,说明答案检查本身并不能验证基准题面和参考答案。
[Erdil 2025] Erdil. Inference economics of language models. arXiv preprint arXiv:2506.04645. https://arxiv.org/abs/2506.04645
该论文建立理论模型,综合计算、高带宽显存带宽与网络延迟约束,推导 LLM 推理中每词元生成速度与每词元成本的 Pareto 前沿。
[Erdogan et al. 2025] Erdogan, Lee, Kim, Moon, Furuta, Anumanchipalli, Keutzer, Gholami. Plan-and-act: Improving planning of agents for long-horizon tasks. PMLR. https://proceedings.mlr.press/v267/erdogan25a.html
Plan-and-Act 训练规划器生成高层计划,再由独立执行器把计划转换成长时程网页任务中的环境动作。
[Eren et al. 2026] Eren, Krohn, Todorov. Financing the AI infrastructure boom: On- and off-balance sheet borrowing. https://www.bis.org/publ/qtrpdf/r_qt2603u.htm
BIS 研究人员记录超大规模厂商债券发行上升,以及另行采用具有类债务经济敞口的合资与特殊目的载体融资。
[Es et al. 2024] Es, James, Espinosa Anke, Schockaert. RAGAs: Automated evaluation of retrieval augmented generation. Association for Computational Linguistics. https://aclanthology.org/2024.eacl-demo.16/
RAGAS 提出由模型介入、无需人工参考答案的忠实度、回答相关性与上下文相关性代理指标;其正式论文没有定义上下文召回。
[Esser et al. 2021] Esser, Rombach, Ommer. Taming transformers for high-resolution image synthesis. https://arxiv.org/abs/2012.09841
VQGAN 将基于 CNN 的离散码本与 Transformer 结合,通过自回归方式实现百万像素级高分辨率图像合成。
[Esser et al. 2024] Esser, Kulal, Blattmann, Entezari, Müller, Saini, Levi, Lorenz, Sauer, Boesel, Podell, Dockhorn, English, Lacey, Goodwin, Marek, Rombach. Scaling rectified flow transformers for high-resolution image synthesis. https://arxiv.org/abs/2403.03206
SD3 为修正流训练引入改进的噪声采样,并提出支持图像与文本词元双向混合的扩散 Transformer 架构,在高分辨率文生图任务中展示出可预测的扩展律。
[Ethayarajh 2019] Ethayarajh. How contextual are contextualized word representations? Comparing the geometry of BERT, elmo, and GPT-2 embeddings. https://aclanthology.org/D19-1006/
本文分析 BERT、ELMo 和 GPT-2 中上下文词表示的几何结构,发现其具有各向异性,且上层产生更具上下文特异性的嵌入。
[Ethayarajh et al. 2024] Ethayarajh, Xu, Muennighoff, Jurafsky, Kiela. Model alignment as prospect theoretic optimization. PMLR. https://arxiv.org/abs/2402.01306
KTO 基于 Kahneman-Tversky 前景理论提出对齐目标,仅需二元可取性信号而非偏好对,在 1B 到 30B 参数规模上与 DPO 持平或更优。
[European Commission 2025] European Commission. Guidelines on the obligations for general-purpose AI model providers. https://digital-strategy.ec.europa.eu/en/policies/guidelines-gpai-providers
欧盟委员会指南说明了提供方应如何理解并记录《AI 法案》中适用于通用 AI 模型的义务。
[European Commission 2025] European Commission. The general-purpose AI code of practice. https://digital-strategy.ec.europa.eu/en/policies/contents-code-gpai
为证明特定 GPAI 透明、版权、安全与安保义务提供自愿路径。
[European Data Protection Board 2026] European Data Protection Board. Respect individuals' rights. https://www.edpb.europa.eu/sme/be-compliant/respect-individuals-rights_en
EDPB 指南说明如何落实数据主体权利,包括机器可读的可携带格式及推断数据的适用边界。
[European Parliament and Council of the European Union 2016] European Parliament and Council of the European Union. Regulation (EU) 2016/679, general data protection regulation. https://eur-lex.europa.eu/eli/reg/2016/679/oj
《通用数据保护条例》规定了数据保护原则,以及有条件适用的访问、更正、删除和可携带权。
[European Parliament and Council of the European Union 2019] European Parliament and Council of the European Union. Directive (EU) 2019/790 on copyright and related rights in the digital single market. https://eur-lex.europa.eu/eli/dir/2019/790/oj
规定欧盟文本与数据挖掘制度,包括第 4 条明确保留权利的机制。
[European Parliament and Council of the European Union 2024] European Parliament and Council of the European Union. Regulation (EU) 2024/1689 of the european parliament and of the council laying down harmonised rules on artificial intelligence (artificial intelligence act). https://eur-lex.europa.eu/eli/reg/2024/1689/oj
欧盟《AI 法案》为 AI 系统规定了基于风险的义务,为通用模型另设规则,并采用分阶段生效时间表。
[European Parliament and Council of the European Union 2026] European Parliament and Council of the European Union. Regulation (EU) 2026/1744 amending Regulations (EU) 2024/1689, (EU) 2018/1139 and (EU) 2023/1230 as regards the simplification of the implementation of harmonised rules on artificial intelligence (Digital Omnibus on AI). https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L_202601744
2026 年《数字综合法案》修订了《AI 法案》及相关法规,在简化实施的同时调整了若干义务和日期。
[European Parliament and Council of the European Union 2026] European Parliament and Council of the European Union. Regulation (EU) 2026/1744 amending regulation (EU) 2024/1689 as regards the simplification of certain rules. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32026R1744
调整特定高风险系统日期,同时保持 GPAI 的适用与执法日期不变。
[European Parliament and Council of the European Union 2026] European Parliament and Council of the European Union. Regulation (EU) 2026/1744 amending regulations as regards the simplification of the implementation of harmonised rules on artificial intelligence (Digital Omnibus on AI). https://eur-lex.europa.eu/eli/reg/2026/1744/oj
2026 年《数字综合法案》修订了《AI 法案》及相关法规,在简化实施的同时调整了若干义务和日期。
[European Union 2016] European Union. Regulation (EU) 2016/679 (general data protection regulation). https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng
GDPR 规定数据保护原则,以及访问、更正、删除和可携带等附条件权利。
[Evtimov et al. 2025] Evtimov, Zharmagambetov, Grattafiori, Guo, Chaudhuri. WASP: Benchmarking web agent security against prompt injection attacks. https://proceedings.neurips.cc/paper_files/paper/2025/hash/1c9818387f5dd0a0bc151214660f059d-Abstract-Datasets_and_Benchmarks_Track.html
端到端基准,显示现实而低成本的提示词注入能够使网页智能体偏航,并区分攻击者部分得逞与完整成功。
[Fabijan et al. 2019] Fabijan, Gupchup, Gupta, Omhover, Qin, Vermeer, Dmitriev. Diagnosing sample ratio mismatch in online controlled experiments: a taxonomy and rules of thumb for practitioners. https://doi.org/10.1145/3292500.3330722
该论文对样本比例失配的原因进行分类,并解释流量分配异常为何经常暴露实验缺陷。
[Faysse et al. 2025] Faysse, Sibille, Wu, Omrani, Viaud, Hudelot, Colombo. ColPali: Efficient document retrieval with vision language models. https://proceedings.iclr.cc/paper_files/paper/2025/hash/99e9e141aafc314f76b0ca3dd66898b3-Abstract-Conference.html
该方法把后期交互的视觉语言表示直接用于页面图像检索。
[Featonby 2021] Featonby. Making retries safe with idempotent apis. https://aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs/
AWS 介绍了请求标识符和幂等契约,服务借此可以重试操作,而不会重复产生预期效果。
[Federal Energy Regulatory Commission 2025] Federal Energy Regulatory Commission. FERC directs nation's largest grid operator to create new rules to embrace innovation and protect consumers. https://www.ferc.gov/news-events/news/fact-sheet-ferc-directs-nations-largest-grid-operator-create-new-rules-embrace
美国联邦能源监管委员会要求 PJM 为大型共址负载制定更明确的规则,同时处理可靠性、并网和消费者成本问题。
[Federal Energy Regulatory Commission 2026] Federal Energy Regulatory Commission. FERC launches aggressive targeted action to speed large load integration. https://www.ferc.gov/news-events/news/ferc-launches-aggressive-targeted-action-speed-large-load-integration
美国联邦能源监管委员会提出一套有针对性的流程,以加快大型负载接入,同时保障可靠性和透明的成本分摊。
[Federal Trade Commission, Office of Technology Staff 2025] Federal Trade Commission, Office of Technology Staff. Partnerships between cloud service providers and AI developers. https://www.ftc.gov/reports/ftc-staff-report-ai-partnerships-investments-6b-study
FTC 工作人员记录特定云提供商与 AI 开发者合作中的股权、治理、云支出、排他、集成和信息共享条款,但没有作出违法认定。
[Fedus et al. 2022] Fedus, Zoph, Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research. https://arxiv.org/abs/2101.03961
Switch Transformer 将混合专家(MoE)路由简化为每词元单专家选择,在相同 FLOPs 下实现万亿参数稀疏模型,预训练速度较 T5 最高提升 7 倍。
[Feng et al. 2025] Feng, Xue, Liu, An. Group-in-group policy optimization for LLM agent training. https://proceedings.neurips.cc/paper_files/paper/2025/hash/420c9f777c0b4f78d515e53cf74d58b2-Abstract-Conference.html
GiGPO 在等价环境状态重复出现时构造局部比较组,无需额外评论器或推演即可提供逐步相对优势。
[Fielding 2000] Fielding. Architectural styles and the design of network-based software architectures. https://ics.uci.edu/~fielding/pubs/dissertation/top.htm
Fielding 说明了架构约束和明确的接口语义如何塑造可演进的网络系统。
[Fielding et al. 2022] Fielding, Nottingham, Reschke. HTTP semantics. https://www.rfc-editor.org/rfc/rfc9110.html
RFC 9110 定义 HTTP 语义,但仍将 402 Payment Required 保留;现代支付协议是叠加在 HTTP 上的约定,而非对 HTTP 本身的修订。
[FinOps Foundation 2026] FinOps Foundation. What is FinOps?. https://www.finops.org/introduction/what-is-finops/
FinOps 把技术成本管理定义为工程、财务、产品和业务团队共同参与的协作式运营模型。
[FinOps Open Cost and Usage Specification 2026] FinOps Open Cost and Usage Specification. FOCUS: FinOps open cost and usage specification. https://focus.finops.org/
FOCUS 定义统一的技术账单数据集。1.4 版增加发票、账期和承诺明细,较早加入的虚拟货币字段支持词元购入与消耗记录。
[Firecracker Maintainers 2026] Firecracker Maintainers. Snapshotting support. https://github.com/firecracker-microvm/firecracker/blob/main/docs/snapshotting/snapshot-support.md
Firecracker 快照序列化来宾内存与模拟硬件状态,而磁盘文件、连接、兼容性及未纳入的运行数据需要另行处理。
[Ford 1957] Ford. Solution of a ranking problem from binary comparisons. The American Mathematical Monthly. https://doi.org/10.1080/00029890.1957.11989117
Ford 给出从二元结果得到有限且唯一的归一化最大似然排名所需的比较图条件。
[Forero Cuervo 2017] Forero Cuervo. Handling overload. O'Reilly Media. https://sre.google/sre-book/handling-overload/
Google SRE 指南通过容量限制、准入控制、优雅降级和负载卸除来处理过载,而不是让请求队列无限增长。
[Frantar et al. 2023] Frantar, Ashkboos, Hoefler, Alistarh. GPTQ: Accurate post-training quantization for generative pre-trained transformers. https://openreview.net/forum?id=tcbBPnfwxS
GPTQ 是一种基于近似二阶信息的训练后量化方法,可在数小时内将 OPT-175B 等大语言模型压缩至每权重 3-4 比特,困惑度损失可忽略,并首次实现在单张 GPU 上运行 175B 参数模型的生成推理。
[Friedl et al. 2026] Friedl, Ward, Rapoport, Everitt, Richens. The impossibility of eliciting latent knowledge. arXiv preprint arXiv:2606.12268. https://arxiv.org/abs/2606.12268
在其因果形式化和分布偏移假设下,论文证明所指定的一类纯行为反馈无法确定地保证得到诚实智能体。
[Frostig et al. 2018] Frostig, Johnson, Leary. Compiling machine learning programs via high-level tracing. https://mlsys.org/Conferences/doc/2018/146.pdf
本文提出对纯数组程序进行高层追踪,并将微分、向量化与编译组织为可组合变换。
[Fu et al. 2024] Fu, Bailis, Stoica, Zhang. Break the sequential dependency of LLM inference using lookahead decoding. PMLR. https://proceedings.mlr.press/v235/fu24a.html
前瞻解码使用 Jacobi 迭代和 n-gram 池来生成并验证多个续写,不需要学习型草稿模型。
[Fu et al. 2025] Fu, Gao, Shen, Zhu, Mei, He, Xu, Wei, Mei, Wang, Yang, Yuan, Wu. AReaL: a large-scale asynchronous reinforcement learning system for language reasoning. https://proceedings.neurips.cc/paper_files/paper/2025/hash/33c00862bfa29ac72ecf630a41e19352-Abstract-Conference.html
AReaL 让推演持续生成、学习器独立更新,并控制策略陈旧度;在其数学与代码工作负载上报告最高 2.77 倍加速。
[Furlanello et al. 2018] Furlanello, Lipton, Tschannen, Itti, Anandkumar. Born again neural networks. https://proceedings.mlr.press/v80/furlanello18a.html
Born-again networks 用相同架构的教师模型训练学生模型,并报告学生模型可以超过教师模型。
[Fuster Baggetto and Fresno 2022] Fuster Baggetto, Fresno. Is anisotropy really the cause of BERT embeddings not being semantic?. https://aclanthology.org/2022.findings-emnlp.314/
该研究在所测试的 BERT 上下文表示中发现,各向异性并非语义等距性不足的根本原因,并将其与词频和分词偏差区分开来。
[G7 Industry, Digital and Technology Ministers 2026] G7 Industry, Digital and Technology Ministers. G7 vision on AI openness: Opportunities and shared language. https://www.entreprises.gouv.fr/files/files/Actualites/2026/g7/vision-AI-openness-opportunities-and-shared-language.pdf
G7 词汇区分开源系统、以开源条款发布的权重与部署代码,以及受限制条款约束的权重下载。
[Gale et al. 2023] Gale, Narayanan, Young, Zaharia. MegaBlocks: Efficient sparse training with mixture-of-experts. https://proceedings.mlsys.org/paper_files/paper/2023/hash/5a54f79333768effe7e8927bcccffe40-Abstract-mlsys2023.html
MegaBlocks 使用块稀疏 GPU 内核执行动态路由的专家批次,既不丢弃令牌,也无需把每个专家填充到固定容量。
[Gao et al. 2021] Gao, Yao, Chen. SimCSE: Simple contrastive learning of sentence embeddings. https://aclanthology.org/2021.emnlp-main.552/
SimCSE 通过对比学习提升句子嵌入质量,无监督时以 dropout 噪声作为最小数据增强,有监督时利用自然语言推断的蕴含与矛盾句对。
[Gao et al. 2023] Gao, Schulman, Hilton. Scaling laws for reward model overoptimization. https://proceedings.mlr.press/v202/gao23h.html
本文度量 RLHF 中的奖励模型过优化现象,推导出黄金奖励随策略与初始策略之间 KL 散度变化的扩展律,覆盖强化学习与 best-of-n 采样两种优化方式。
[Gao et al. 2023] Gao, Madaan, Zhou, Alon, Liu, Yang, Callan, Neubig. PAL: Program-aided language models. PMLR. https://arxiv.org/abs/2211.10435
PAL 让大语言模型把自然语言推理题翻译成可执行程序,再把计算交给 Python 解释器完成。
[Gao et al. 2023] Gao, Yen, Yu, Chen. Enabling large language models to generate text with citations. https://aclanthology.org/2023.emnlp-main.398/
ALCE 分别评估长文本回答质量、引用正确性与引用完整性,而不是把出现引用当作充分条件。
[Gao et al. 2025] Gao, Dupré la Tour, Tillman, Goh, Troll, Radford, Sutskever, Leike, Wu. Scaling and evaluating sparse autoencoders. https://proceedings.iclr.cc/paper_files/paper/2025/hash/42ef3308c230942d223c411adf182c88-Abstract-Conference.html
本文提出使用 TopK 稀疏自编码器(SAE)直接控制稀疏度以改善重建-稀疏度边界,建立关于 SAE 规模与稀疏度的扩展律,并在 GPT-4 激活上训练了一个拥有 1600 万潜变量的 SAE。
[Gao et al. 2025] Gao, Rajaram, Coxon, Govande, Baker, Mossing. Weight-sparse transformers have interpretable circuits. arXiv preprint arXiv:2511.13653. https://arxiv.org/abs/2511.13653
OpenAI 的研究者把 Transformer 的权重训练成大部分为零,使电路从构造上就可解释,代价是在小规模下用能力换可解释性。
[Gazitt et al. 2026] Gazitt, Brossard, Tulshibagwale. Authorization API 1.0. https://openid.net/specs/authorization-api-1_0.html
AuthZEN 定义了主体、动作、资源、上下文和布尔决策的交换格式,但把策略语言和执行机制留给具体实现。
[Gebru et al. 2021] Gebru, Morgenstern, Vecchione, Vaughan, Wallach, Daumé III, Crawford. Datasheets for datasets. Communications of the ACM. https://doi.org/10.1145/3458723
数据表框架提出标准化数据集文档,覆盖动机、构成、采集、预处理、用途、发布和维护。
[Geifman and El-Yaniv 2017] Geifman, El-Yaniv. Selective classification for deep neural networks. https://proceedings.neurips.cc/paper/2017/hash/4a8423d5e91fda00bb7e46540e2b0cf1-Abstract.html
选择性分类允许模型弃答,并评估由此产生的预测覆盖率与错误风险之间的取舍。
[Geiger et al. 2025] Geiger, Wu, Potts, Icard, Goodman. Causal abstraction: a theoretical foundation for mechanistic interpretability. Journal of Machine Learning Research. https://jmlr.org/papers/v26/23-0058.html
因果抽象形式化了一个更简单的解释模型在什么条件下能够保留原系统的干预行为。
[Gema et al. 2025] Gema, Leang, Hong, Devoto, Mancino, Saxena, He, Zhao, Du, Ghasemi Madani, Barale, McHardy, Harris, Kaddour, Van Krieken, Minervini. Are we done with MMLU?. Association for Computational Linguistics. https://aclanthology.org/2025.naacl-long.262/
对 5,700 道 MMLU 题目的人工重标注估计约 6.49% 的题目标准答案有误,个别学科子集错误率高得多,并发布了修订后的 MMLU-Redux 子集。
[Geng et al. 2025] Geng, Deng, Bai, Kolter, He. Mean flows for one-step generative modeling. arXiv preprint arXiv:2505.13447. https://arxiv.org/abs/2505.13447
MeanFlow 学习一个平均速度场,目标是用单次生成步骤把噪声映射为数据。
[Geng et al. 2025] Geng, Cooper, Moskal, Jenkins, Berman, Ranchin, West, Horvitz, Nori. JSONSchemaBench: a rigorous benchmark of structured outputs for language models. arXiv preprint arXiv:2501.10868. https://arxiv.org/abs/2501.10868
JSONSchemaBench 以真实模式和官方模式测试,从效率、约束覆盖与输出质量等方面评估结构化生成系统。
[Gerstgrasser et al. 2024] Gerstgrasser, Schaeffer, Dey, Rafailov, Sleight, Hughes, Korbak, Agrawal, Pai, Gromov, Roberts, Yang, Donoho, Koyejo. Is model collapse inevitable? Breaking the curse of recursion by accumulating real and synthetic data. arXiv preprint arXiv:2404.01413. https://arxiv.org/abs/2404.01413
在所测试的递归训练方案中,保留以往的真实和合成数据,比反复用新数据替换整个数据集效果更好。
[ggml Project 2026] ggml Project. GGUF file format specification. https://github.com/ggml-org/ggml/blob/6af560d55df03ad92116e3c0a697779584477e85/docs/gguf.md
GGUF 是面向 GGML 系运行时的可扩展推理容器,旨在快速加载且通常自包含,但当前规范也支持分片与侧车文件。
[GGML project 2023] GGML project. GGUF file format. https://github.com/ggml-org/ggml/blob/master/docs/gguf.md
llama.cpp 是一个面向消费级硬件的 C/C++ 大语言模型推理库,引入了 GGUF 模型格式,支持从 1.5 位到 8 位整数量化,并兼容 CPU 与 GPU 混合后端。
[Ghazvininejad et al. 2019] Ghazvininejad, Levy, Liu, Zettlemoyer. Mask-predict: Parallel decoding of conditional masked language models. https://arxiv.org/abs/1904.09324
Mask-Predict 提出条件掩码语言模型,通过迭代并行解码算法在接近自回归翻译质量的同时显著加快解码速度。
[Gheshlaghi Azar et al. 2024] Gheshlaghi Azar, Daniel Guo, Piot, Munos, Rowland, Valko, Calandriello. A general theoretical paradigm to understand learning from human preferences. PMLR. https://arxiv.org/abs/2310.12036
本文提出 ΨPO 通用偏好优化目标,将 RLHF 与 DPO 统一为特例,并推导出 IPO 以绕过 Bradley-Terry 假设从而避免过拟合。
[Ghosh et al. 2023] Ghosh, Hajishirzi, Schmidt. GenEval: An object-focused framework for evaluating text-to-image alignment. https://arxiv.org/abs/2310.11513
GENEVAL 是一个以目标检测为核心的基准,通过检测与判别式视觉模型评估文本到图像模型在计数、位置和属性绑定等组合任务上的对齐能力。
[Glassman et al. 1995] Glassman, Manasse, Abadi, Gauthier, Sobalvarro. The millicent protocol for inexpensive electronic commerce. https://www.w3.org/Conferences/WWW4/Papers/246/
Millicent 展示商户代币与聚合如何降低支付开销,但并不声称技术可行性就会创造用户需求。
[Glazer et al. 2024] Glazer, Erdil, Besiroglu, Chicharro, Chen, Gunning, Olsson, Denain, Ho, Oliveira Santos, Järviniemi, Barnett, Sandler, Vrzala, Sevilla, Ren, Pratt, Levine, Barkley, Stewart, Grechuk, Grechuk, Enugandla, Wildon. FrontierMath: a benchmark for evaluating advanced mathematical reasoning in AI. arXiv preprint arXiv:2411.04872. https://arxiv.org/abs/2411.04872
FrontierMath 使用专家原创题目和自动答案检查来衡量高级数学推理,同时降低数据污染风险。
[Gloeckle et al. 2024] Gloeckle, Youbi Idrissi, Rozière, Lopez-Paz, Synnaeve. Better & faster large language models via multi-token prediction. PMLR. https://proceedings.mlr.press/v235/gloeckle24a.html
多词元预测把多个未来词元预测头作为辅助训练目标;这些预测头也可以在自推测解码中提供候选。
[Google 2026] Google. Gemma terms of use. https://ai.google.dev/gemma/terms
现行 Gemma 条款规范所涵盖版本及衍生模型的使用、修改、托管与分发,并将 Gemma 4 用户引向独立许可证。
[Google 2026] Google. Gemma 4 model card. https://ai.google.dev/gemma/docs/core/model_card_4
Gemma 4 模型卡记录了已发布模型的架构、预期用途、评估、安全工作和已知局限。
[Google n.d.] Google. OpenAI compatibility. https://ai.google.dev/gemini-api/docs/openai
Google 记录 OpenAI 库调用如何映射到 Gemini,并建议在应用需要提供商特有功能时使用原生 Gemini 集成。
[Google Cloud 2025] Google Cloud. Announcing the Agent2Agent protocol (A2A). https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/
Google 将 A2A 定义为开放协议,使不同厂商或框架构建的智能体能够通信和协调任务。
[Google DeepMind 2026] Google DeepMind. Frontier safety framework, version 3.1. https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/strengthening-our-frontier-safety-framework/frontier-safety-framework_3-1.pdf
按风险领域定义关键能力等级,并在每一级之下设预警评测与报警阈值,让安全缓冲先于触及生效。部署侧与安全侧的缓解措施分属两层,而放行与否取决于一份经过评审的安全论证。
[Google DeepMind 2026] Google DeepMind. Gemini 3 deep think. https://deepmind.google/models/gemini-deep-think/
Google DeepMind 报告其 ARC-AGI-2 成绩经 ARC Prize 验证达到 84.6%,同时保留了关于测试协议和评估集的重要限定。
[Google Platforms and Devices et al. 2025] Google Platforms and Devices, Google DeepMind, Google Cloud. Private AI compute in the cloud. https://services.google.com/fh/files/misc/private_ai_compute_technical_brief.pdf
介绍 SEV-SNP 前端、加固 TPU 基础设施、内部证明、加密链路、IP 隐藏,以及仍属未来工作的外部验证功能。
[Grattafiori and others 2024] Grattafiori, others. The llama 3 herd of models. https://arxiv.org/abs/2407.21783
Meta 发布 Llama 3 系列密集 Transformer 大语言模型,参数量分别为 8B、70B 和 405B,在 15T 词元上预训练,跨任务性能与 GPT-4 相当。
[Graves et al. 2006] Graves, Fernández, Gomez, Schmidhuber. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. https://www.cs.toronto.edu/~graves/icml_2006.pdf
CTC 提出一种直接用循环神经网络对未分段序列进行标注的训练方法,无需预分段或后处理。
[Graves 2012] Graves. Sequence transduction with recurrent neural networks. ICML 2012 Representation Learning Workshop; arXiv:1211.3711. https://arxiv.org/abs/1211.3711
Graves 2012 提出 RNN Transducer(RNN-T),一种端到端概率序列转换系统,无需预定义对齐即可联合建模输入-输出与输出-输出依赖关系,扩展了连接主义时序分类(CTC)。
[Greenblatt et al. 2024] Greenblatt, Shlegeris, Sachan, Roger. AI control: Improving safety despite intentional subversion. PMLR. https://proceedings.mlr.press/v235/greenblatt24a.html
AI control 在 APPS 编程测试环境中对监控和编辑协议做红队评估,并把 GPT-4 视为不可信、GPT-3.5 视为可信。
[Greenblatt et al. 2024] Greenblatt, Denison, Wright, Roger, MacDiarmid, Marks, Treutlein, Belonax, Chen, Duvenaud, Khan, Michael, Mindermann, Perez, Petrini, Uesato, Kaplan, Shlegeris, Bowman, Hubinger. Alignment faking in large language models. arXiv preprint arXiv:2412.14093. https://arxiv.org/abs/2412.14093
Greenblatt 等人发现,仅凭关于自身将被重训的情境信息,Claude 3 Opus 就会在训练中选择性顺从以保住原有偏好;经强化学习后,对齐伪装推理从 14% 升至 78%。
[Greshake et al. 2023] Greshake, Abdelnabi, Mishra, Endres, Holz, Fritz. Not what you've signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection. arXiv preprint arXiv:2302.12173. https://arxiv.org/abs/2302.12173
间接提示注入把对抗性指令放进第三方数据,等待集成大模型的应用检索,从而暴露数据与工具控制风险。
[Grinsztajn et al. 2022] Grinsztajn, Oyallon, Varoquaux. Why do tree-based models still outperform deep learning on typical tabular data?. https://arxiv.org/abs/2207.08815
在 45 个含 3,000 至 10,000 个样本的精选表格数据集上,受测树集成优于受测深度网络;分析强调了其对无信息特征与不规则目标函数的鲁棒性。
[gRPC Authors 2024] gRPC Authors. Cancellation. https://grpc.io/docs/guides/cancellation/
gRPC 指南说明取消请求必须沿 RPC 调用链传播,应用处理程序也可能需要主动检查取消状态。
[Grünwald 2004] Grünwald. A tutorial introduction to the minimum description length principle. arXiv preprint math/0406077. https://arxiv.org/abs/math/0406077
Grünwald 的教程介绍了 Rissanen 的最小描述长度(MDL)原理,将统计模型选择与归纳推断框架化为数据压缩,以自动防止过拟合。
[Gu et al. 2018] Gu, Bradbury, Xiong, Li, Socher. Non-autoregressive neural machine translation. https://arxiv.org/abs/1711.02281
非自回归 Transformer (NAT) 以繁殖度作为潜变量,并行生成全部输出词元,推理延迟降低一个数量级,相比自回归教师模型仅损失约 2.0 BLEU 分。
[Gu et al. 2019] Gu, Wang, Zhao. Levenshtein transformer. https://arxiv.org/abs/1905.11006
Levenshtein Transformer(LevT)是一种基于插入和删除操作的部分自回归序列生成模型,采用双策略模仿学习训练,在机器翻译任务上实现相当的生成质量并获得最高 5 倍的解码加速。
[Gu et al. 2019] Gu, Chowdhury, Shin, Zhu, Jeon, Qian, Liu, Guo. Tiresias: a GPU cluster manager for distributed deep learning. USENIX Association. https://www.usenix.org/conference/nsdi19/presentation/gu
Tiresias 研究了具有全有或全无资源需求的分布式深度学习任务应如何准入、放置和调度。
[Gu et al. 2022] Gu, Goel, Ré. Efficiently modeling long sequences with structured state spaces. https://arxiv.org/abs/2111.00396
S4 通过低秩加正规矩阵分解重参数化状态空间模型(SSM),实现 O(N+L) 计算复杂度,并在长程序列基准(包括此前无模型能解的 Path-X 任务)上达到最优性能。
[Gu and Dao 2024] Gu, Dao. Mamba: Linear-time sequence modeling with selective state spaces. https://arxiv.org/abs/2312.00752
Mamba 提出带输入依赖参数的选择性状态空间模型(SSM)和硬件感知并行扫描算法,以线性时间复杂度实现与 Transformer 相当的语言建模质量。
[Guan et al. 2024] Guan, Joglekar, Wallace, Jain, Barak, Helyar, Dias, Vallone, Ren, Wei, Chung, Toyer, Heidecke, Beutel, Glaese. Deliberative alignment: Reasoning enables safer language models. arXiv preprint arXiv:2412.16339. https://arxiv.org/abs/2412.16339
Deliberative Alignment 将安全规范教给模型,并训练其在回答前依据规范推理,从而同时提升越狱鲁棒性并降低过度拒绝。
[Guha et al. 2026] Guha, Marten, Keh, Raoof, Smyrnis, Bansal, Nezhurina, Mercat, Vu, Sprague, Suvarna, Feuer, Chen, Khan, Frankel, Grover, Choi, Muennighoff, Su, Zhao, Yang, Pimpalgaonkar, Sharma, Ji, Deng, Pratt, Ramanujan, Saad-Falcon, Li, Dave, Albalak, Arora, Wulfe, Hegde, Durrett, Oh, Bansal, Gabriel, Grover, Chang, Shankar, Gokaslan, Merrill, Hashimoto, Choi, Jitsev, Heckel, Sathiamoorthy, Dimakis, Schmidt. OpenThoughts: Data recipes for reasoning models. https://openreview.net/forum?id=7xjoTuaNmN
超过 1,000 次受控实验形成了 120 万样本的推理语料;在匹配的 Qwen2.5-7B 实验中,精选数据配方随规模增长继续带来收益。
[Gulati et al. 2020] Gulati, Qin, Chiu, Parmar, Zhang, Yu, Han, Wang, Zhang, Wu, Pang. Conformer: Convolution-augmented transformer for speech recognition. https://arxiv.org/abs/2005.08100
Conformer 在单个编码器块中结合卷积与自注意力,同时捕获局部与全局音频特征,在 LibriSpeech 上取得当时最优的自动语音识别结果。
[Gulrajani and Hashimoto 2023] Gulrajani, Hashimoto. Likelihood-based diffusion language models. https://arxiv.org/abs/2305.18619
Plaid 1B 是一个基于似然的扩散语言模型,通过算法改进和计算最优扩展律在零样本困惑度基准上超越了 GPT-2 124M。
[Gunasekar et al. 2023] Gunasekar, Zhang, Aneja, Mendes, Del Giorno, Gopi, Javaheripi, Kauffmann, Rosa, Saarikivi, Salim, Shah, Behl, Wang, Bubeck, Eldan, Kalai, Lee, Li. Textbooks are all you need. https://arxiv.org/abs/2306.11644
phi-1 是一个 1.3B 参数的代码模型,用 7B 词元的过滤代码、合成教科书和练习数据训练;论文报告其微调后在 HumanEval 上达到 50.6% pass@1。
[Gunjal et al. 2025] Gunjal, Wang, Lau, Nath, He, Liu, Hendryx. Rubrics as rewards: Reinforcement learning beyond verifiable domains. arXiv preprint arXiv:2507.17746. https://arxiv.org/abs/2507.17746
Rubrics as Rewards 把开放式判断拆成逐条清单式评分准则,由模型逐条打分并作为在线强化学习的奖励,在 HealthBench 与 GPQA-Diamond 上优于模型作为评判者的 Likert 基线。
[Guo et al. 2017] Guo, Pleiss, Sun, Weinberger. On calibration of modern neural networks. Proceedings of the 34th International Conference on Machine Learning. https://proceedings.mlr.press/v70/guo17a.html
Guo 等人表明,现代神经网络可以准确却未校准,而简单的温度缩放能显著改善概率校准。
[Guo et al. 2020] Guo, Goldstein, Hannun, Maaten. Certified data removal from machine learning models. https://proceedings.mlr.press/v119/guo20c.html
Guo 等人以无法区别于保留数据重训来定义可认证移除,并在明确假设下为正则化线性模型构造相应机制。
[Guo et al. 2024] Guo, Shang, Vazirgiannis, Clavel. The curious decline of linguistic diversity: Training language models on synthetic text. Association for Computational Linguistics. https://aclanthology.org/2024.findings-naacl.228/
实验发现,反复使用生成文本训练模型,会降低后续输出在词汇、句法和语义上的多样性。
[Guo et al. 2025] Guo, Yang, Zhang, Song, Wang, Zhu, Xu, Zhang, Ma, Bi, others. DeepSeek-R1 incentivizes reasoning in llms through reinforcement learning. Nature. https://www.nature.com/articles/s41586-025-09422-z
DeepSeek-R1 展示了大规模可验证奖励强化学习能引出长推理行为;R1-Zero 不用监督冷启动直接 RL,R1 则加入多阶段训练以改善可读性和稳定性。
[Guo et al. 2026] Guo, Ceron Uribe, Zhu, Choquette-Choo, Lin, Kandpal, Nasr, Rai, Toyer, Wang, Yu, Beutel, Xiao. IH-Challenge: a training dataset to improve instruction hierarchy on frontier llms. arXiv preprint arXiv:2603.10521. https://arxiv.org/abs/2603.10521
IH-Challenge 使用可程序化评分的强化学习任务与在线对抗生成,在 GPT-5-Mini 实验中提升指令层级鲁棒性。
[Gupta et al. 2020] Gupta, Wu, Wang, Naumov, Reagen, Brooks, Cottel, Hazelwood, Hempstead, Jia, Lee, Malevich, Mudigere, Smelyanskiy, Xiong, Zhang. The architectural implications of facebook's DNN-based personalized recommendation. https://arxiv.org/abs/1906.03109
该论文刻画了 Facebook 的推荐模型,说明稀疏嵌入、密集计算和通信如何形成不同的系统瓶颈。
[Gururangan et al. 2020] Gururangan, Marasović, Swayamdipta, Lo, Beltagy, Downey, Smith. Don't stop pretraining: Adapt language models to domains and tasks. Association for Computational Linguistics. https://arxiv.org/abs/2004.10964
表明在领域语料与任务语料上继续预训练,可在生物医学、计算机科学、新闻和评论等任务上提升下游表现。
[gVisor Project 2026] gVisor Project. Security model. https://gvisor.dev/docs/architecture_guide/security/
gVisor 安全模型介绍了 Sentry、Gofer、受限宿主接口、文件系统模式,以及用户态内核设计中仍然可信的部分。
[Ha and Schmidhuber 2018] Ha, Schmidhuber. World models. arXiv preprint arXiv:1803.10122. https://arxiv.org/abs/1803.10122
World Models 将视觉自编码器、循环潜在动力学与控制器结合起来,并包含在学习到的轨迹中训练控制器的实验。
[Haas et al. 2025] Haas, Yona, D'Antonio, Goldshtein, Das. SimpleQA Verified: a reliable factuality benchmark to measure parametric knowledge. arXiv preprint arXiv:2509.07968. https://arxiv.org/abs/2509.07968
SimpleQA Verified 在审核标签、重复项和主题平衡后建立一个筛选出的 1,000 题衍生集,并未原地修改原基准。
[Hafner et al. 2019] Hafner, Lillicrap, Fischer, Villegas, Ha, Lee, Davidson. Learning latent dynamics for planning from pixels. https://arxiv.org/abs/1811.04551
PlaNet 从图像中学习随机潜在动力学,并通过在线规划解决 DeepMind Control Suite 中的连续控制任务。
[Hales et al. 2017] Hales, Adams, Bauer, Dang, Harrison, Hoang, Kaliszyk, Magron, McLaughlin, Nguyen, Nguyen, Nipkow, Obua, Pleso, Rute, Solovyev, Ta, Tran, Trieu, Urban, Vu, Zumkeller. A formal proof of the kepler conjecture. Forum of Mathematics, Pi. https://doi.org/10.1017/fmp.2017.1
Flyspeck 项目用 HOL Light 和 Isabelle 给出了 Kepler 猜想的形式化证明,展示了大型数学结果如何交给证明助手检查。
[Handa et al. 2025] Handa, Tamkin, McCain, Huang, Durmus, Heck, Mueller, Hong, Ritchie, Belonax, Troy, Amodei, Kaplan, Clark, Ganguli. Which economic tasks are performed with AI? Evidence from millions of claude conversations. https://arxiv.org/abs/2503.04761
该研究把数百万条抽样 Claude 对话映射到职业任务,描述产品使用情况,但不观察用户职业、被接受的产出或生产率。
[Hao et al. 2023] Hao, Gu, Ma, Hong, Wang, Wang, Hu. Reasoning with language model is planning with world model. Association for Computational Linguistics. https://aclanthology.org/2023.emnlp-main.507/
RAP 将语言模型推理表述为规划任务,用模型充当世界模型,并通过树搜索评估候选推理路径。
[Hardt 2012] Hardt. The oauth 2.0 authorization framework. https://www.rfc-editor.org/rfc/rfc6749.html
OAuth 区分资源所有者、客户端、授权服务器和资源服务器,并让客户端使用限定权限范围的访问令牌,而不是获取所有者凭据。
[Hashimoto and Liang 2025] Hashimoto, Liang. CS336: Language modeling from scratch. https://cs336.stanford.edu/spring2025/
Stanford 的实现密集型语言模型课程,覆盖分词器构造、Transformer 实现、系统优化、扩展律、数据处理、评测与对齐。
[Hayes and Krippendorff 2007] Hayes, Krippendorff. Answering the call for a standard reliability measure for coding data. Communication Methods and Measures. https://doi.org/10.1080/19312450709336664
Hayes 与 Krippendorff 主张使用 Krippendorff alpha 作为通用可靠性系数,可处理多标注者、缺失数据和不同测量等级。
[He et al. 2019] He, Sainath, Prabhavalkar, McGraw, others. Streaming end-to-end speech recognition for mobile devices. https://arxiv.org/abs/1811.06621
Google 提出面向移动设备的流式端到端 RNN-T 自动语音识别(ASR)系统,在语音搜索和听写任务上的延迟与词错误率均优于传统 CTC 模型。
[He 2025] He. Defeating nondeterminism in LLM inference. https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/
He 将温度为零时的推理差异追溯到结果依赖动态组批的数值内核,并提出批不变的替代方案。
[Hebb 1949] Hebb. The organization of behavior: a neuropsychological theory. John Wiley & Sons.
Hebb 提出学习会强化共同激活神经元之间的突触连接(同时放电的神经元彼此相连),即赫布学习的基础。
[Heer 2019] Heer. Agency plus automation: Designing artificial intelligence into interactive systems. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.1807184115
Heer 主张用共享表示、审查、修改与丢弃机制,把预测模型嵌入能增强人类工作的交互系统。
[Hendrycks et al. 2020] Hendrycks, Burns, Basart, Zou, Mazeika, Song, Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300. https://arxiv.org/abs/2009.03300
MMLU 提出一个涵盖 STEM、人文和社会科学共 57 个科目的多项选择题基准,用于在零样本和少样本设置下衡量大语言模型的世界知识广度。
[Hermann and Del Balso 2017] Hermann, Del Balso. Meet michelangelo: Uber's machine learning platform. https://www.uber.com/blog/michelangelo-machine-learning-platform/
Uber 介绍了一个连接特征准备、训练、模型管理、部署、在线预测和监控的平台。
[Hines et al. 2024] Hines, Lopez, Hall, Zarfati, Zunger, Kiciman. Defending against indirect prompt injection attacks with spotlighting. arXiv preprint arXiv:2403.14720. https://arxiv.org/abs/2403.14720
Spotlighting 是一种针对间接提示注入攻击的提示工程防御方法,通过对不可信输入文本进行转换以标记其来源,将 GPT 系列模型上的攻击成功率从超过 50% 降至 2% 以下。
[Hinton et al. 2015] Hinton, Vinyals, Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531. https://arxiv.org/abs/1503.02531
知识蒸馏使用教师模型软化后的输出分布训练紧凑的学生模型,而不只依赖硬标签。
[Ho et al. 2020] Ho, Jain, Abbeel. Denoising diffusion probabilistic models. https://arxiv.org/abs/2006.11239
Ho 等人证明以去噪得分匹配目标训练的 DDPM 可实现高质量图像合成,在 CIFAR10 上达到 FID 3.17。
[Ho and Salimans 2022] Ho, Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598. https://arxiv.org/abs/2207.12598
无分类器引导(CFG)通过联合训练条件与无条件扩散模型,在推理时混合两者的分数估计,无需独立分类器即可在样本质量与多样性之间权衡。
[Ho et al. 2022] Ho, Salimans, Gritsenko, Chan, Norouzi, Fleet. Video diffusion models. https://arxiv.org/abs/2204.03458
Ho 等人通过时空分解的 3D U-Net 和重建引导的条件采样,将图像扩散模型扩展至视频生成,在视频生成和预测基准上取得了最优结果。
[Hoffmann et al. 2022] Hoffmann, Borgeaud, Mensch, Buchatskaya, Cai, Rutherford, Las Casas, Hendricks, Welbl, Clark, Hennigan, Noland, Millican, Driessche, Damoc, Guy, Osindero, Simonyan, Elsen, Rae, Vinyals, Sifre. Training compute-optimal large language models. https://arxiv.org/abs/2203.15556
指出模型规模与训练词元应当等比扩展(约每参数 20 个词元),因此多数大模型严重训练不足;在同等算力下,70B 的 Chinchilla 胜过 Gopher、GPT-3 等大得多的模型。
[Hohnhold et al. 2015] Hohnhold, O'Brien, Tang. Focusing on the long-term: It's good for users and business. https://research.google/pubs/focus-on-the-long-term-its-better-for-users-and-business/
Google 介绍了用于判断短期产品收益是否损害长期用户或业务结果的实验和指标。
[Hollmann et al. 2025] Hollmann, Müller, Purucker, Krishnakumar, Körfer, Hoo, Schirrmeister, Hutter. Accurate predictions on small data with a tabular foundation model. Nature. https://www.nature.com/articles/s41586-024-08328-6
TabPFN 是在合成表格任务上预训练的 Transformer;在作者对最多 10,000 个样本和 500 个特征的数据集评估中,它优于精调树集成,而超出该范围的表现仍需研究。
[Hong et al. 2024] Hong, Lee, Thorne. ORPO: Monolithic preference optimization without reference model. Association for Computational Linguistics. https://arxiv.org/abs/2403.07691
ORPO 是一种单体偏好对齐算法,通过在监督微调(SFT)损失中附加优势比惩罚项,将 SFT 与偏好优化合并为单步训练,无需参考模型。
[Hooker 2021] Hooker. The hardware lottery. Communications of the ACM. https://arxiv.org/abs/2009.06489
一个研究想法可能因为贴合现有软硬件而胜出,而非因为它更优越;当硅片围绕矩阵乘法特化,偏离这条路的想法连被评估都要付出越来越高的代价。
[Horvitz and Thompson 1952] Horvitz, Thompson. A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association. https://www.stat.cmu.edu/~brian/905-2008/papers/Horvitz-Thompson-1952-jasa.pdf
Horvitz-Thompson 估计量按入样概率的倒数给样本单位加权,从而在不等概率抽样下估计总体总量。
[Horvitz 1999] Horvitz. Principles of mixed-initiative user interfaces. https://doi.org/10.1145/302979.303030
Horvitz 把界面描述为混合主动性系统,人和计算机在其中协商控制权,而不是只选择直接操纵或完全自动化。
[Houlsby et al. 2019] Houlsby, Giurgiu, Jastrzebski, Morrone, Laroussilhe, Gesmundo, Attariyan, Gelly. Parameter-efficient transfer learning for NLP. https://arxiv.org/abs/1902.00751
向 BERT 各层插入适配器模块,仅训练全量参数高效微调(PEFT)的 3.6%,即可在 GLUE 上达到接近完全微调的性能。
[Howard et al. 2021] Howard, Ramdas, McAuliffe, Sekhon. Time-uniform, nonparametric, nonasymptotic confidence sequences. The Annals of Statistics. https://projecteuclid.org/journals/annals-of-statistics/volume-49/issue-2/Time-uniform-nonparametric-nonasymptotic-confidence-sequences/10.1214/20-AOS1991.full
Howard 等人提出了覆盖保证在时间上统一成立的置信序列,使满足假设时的反复查看仍能进行有效推断。
[Hsieh et al. 2023] Hsieh, Li, Yeh, Nakhost, Fujii, Ratner, Krishna, Lee, Pfister. Distilling step-by-step! Outperforming larger language models with less training data and smaller model sizes. Association for Computational Linguistics. https://aclanthology.org/2023.findings-acl.507/
Distilling Step-by-Step 把教师生成的推理说明作为学生多任务目标中的辅助监督,并在四个 NLP 基准上报告数据效率收益。
[Hsieh et al. 2024] Hsieh, Sun, Kriman, Acharya, Rekesh, Jia, Zhang, Ginsburg. RULER: What's the real context size of your long-context language models?. arXiv preprint arXiv:2404.06654. https://arxiv.org/abs/2404.06654
用可配置的长度与任务复杂度评估长上下文模型,表明在超出简单检索的任务上,有效上下文可能远低于宣称的输入上限。
[Hsu et al. 2021] Hsu, Bolte, Tsai, Lakhotia, Salakhutdinov, Mohamed. HuBERT: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing. https://arxiv.org/abs/2106.07447
HuBERT 通过预测掩码区域的离线 k-means 聚类标签进行自监督语音表示学习,在自动语音识别(ASR)基准上达到或超越 wav2vec 2.0 的性能。
[Hu et al. 2022] Hu, Shen, Wallis, Allen-Zhu, Li, Wang, Wang, Chen. LoRA: Low-rank adaptation of large language models. https://arxiv.org/abs/2106.09685
LoRA 冻结预训练权重,向每个 Transformer 层注入可训练的低秩矩阵对,与全量微调相比可将可训练参数减少 10,000 倍、将 GPU 显存需求降低 3 倍,且不引入额外推理延迟。
[Hu et al. 2024] Hu, Tu, Han, He, Cui, Long, Zheng, Fang, Huang, Zhao, Zhang, Thai, Zhang, Wang, Yao, Zhao, Zhou, Cai, Zhai, Ding, Jia, Zeng, Li, Liu, Sun. MiniCPM: Unveiling the potential of small language models with scalable training strategies. https://arxiv.org/abs/2404.06395
提出可与 7B-13B 大模型匹敌的小模型(1.2B/2.4B),借助模型风洞式扩展实验,以及支持持续训练的 Warmup-Stable-Decay 学习率调度。
[Hu et al. 2024] Hu, Wang, Fang, Fu, Cheng, Yu. ELLA: Equip diffusion models with LLM for enhanced semantic alignment. arXiv preprint arXiv:2403.05135. https://arxiv.org/abs/2403.05135
ELLA 是一种轻量级大语言模型适配器,通过时步感知语义连接器(TSC)为基于 CLIP 的扩散模型增强稠密提示词的语义对齐,无需重新训练 U-Net 或 LLM。
[Hu et al. 2025] Hu, Wu, Shen, Liu, Wang, Jiang, Wang, Chen, Chen, Fang, Xianyu, Cao, Xu, Liu. OpenRLHF: a ray-based easy-to-use, scalable and high-performance RLHF framework. Association for Computational Linguistics. https://aclanthology.org/2025.emnlp-demos.48/
OpenRLHF 集成 Ray、vLLM、DeepSpeed 与 Transformers,展示了生成、评分和学习角色的一种解耦组织方式。
[Hu et al. 2026] Hu, Wang, McAuley. Evaluating memory in LLM agents via incremental multi-turn interactions. https://arxiv.org/abs/2507.05257
MemoryAgentBench 在增量交互中评测准确检索、测试时学习、长程理解与选择性遗忘。
[Huang et al. 2019] Huang, Cheng, Bapna, Firat, Chen, Chen, Lee, Ngiam, Le, Wu, Chen. GPipe: Efficient training of giant neural networks using pipeline parallelism. https://arxiv.org/abs/1811.06965
GPipe 通过微批量切分实现流水线并行(PP),突破单加速器显存限制,在多加速器上实现近线性加速以训练超大规模神经网络。
[Huang et al. 2024] Huang, Zhang, Shan, He. Compression represents intelligence linearly. https://openreview.net/forum?id=SHMj84U5SH
在 31 个公开基础大语言模型与 12 个基准上,平均基准分数与模型对外部文本语料的压缩效率相关,总体 Pearson 相关系数为 -0.93。
[Huang et al. 2024] Huang, Siddarth, Lovitt, Liao, Durmus, Tamkin, Ganguli. Collective constitutional AI: Aligning a language model with public input. https://arxiv.org/abs/2406.07814
Collective Constitutional AI 展示了从目标公众收集原则、将其整理成宪章,并据此训练和评估模型的流程。
[Huang et al. 2024] Huang, Chen, Mishra, Zheng, Yu, Song, Zhou. Large language models cannot self-correct reasoning yet. https://openreview.net/forum?id=IkmD3fKBPQ
实验发现,在没有外部反馈时要求语言模型修改推理,往往不能提高答案准确率,甚至可能使其下降。
[Huang et al. 2025] Huang, Yu, Wang, others. R-zero: Self-evolving reasoning LLM from zero data. arXiv preprint arXiv:2508.05004. https://arxiv.org/abs/2508.05004
R-Zero 从同一个基座初始化出题的 Challenger 与答题的 Solver 并让二者共同演化:Challenger 不断提出更难的任务,Solver 学着解决,从零外部数据生成自己的课程。
[Huang et al. 2025] Huang, Block, Liu, Jiang, Krishnamurthy, Foster. Is best-of-N the best of them? Coverage, scaling, and optimality in inference-time alignment. PMLR. https://proceedings.mlr.press/v267/huang25c.html
当奖励模型并不完美时,候选池扩大可能让 Best-of-N 更容易选中利用评分误差的输出,因而表现反而下降。
[Huben et al. 2024] Huben, Cunningham, Smith, Ewart, Sharkey. Sparse autoencoders find highly interpretable features in language models. https://proceedings.iclr.cc/paper_files/paper/2024/hash/1fa1ab11f4bd5f94b2ec20e794dbfa3b-Abstract-Conference.html
在语言模型激活上训练的稀疏自编码器得到的特征,在所测替代方案上取得了更高的可解释性评分,并在一项任务上支持了更细粒度的因果定位;这支持该方法,但不保证恢复结果唯一或完整。
[Hubert et al. 2026] Hubert, Mehta, Sartran, others. Olympiad-level formal mathematical reasoning with reinforcement learning. Nature. https://www.nature.com/articles/s41586-025-09833-y
AlphaProof 采用 AlphaZero 式强化学习与 Lean 验证,在 2024 年 IMO 中解出三道经人工形式化的非几何题;AlphaProof 与 AlphaGeometry 2 的组合系统达到银牌等效分数。
[Hubinger et al. 2024] Hubinger, Denison, Mu, Lambert, Tong, MacDiarmid, Lanham, Ziegler, Maxwell, Cheng, Jermyn, Askell, Radhakrishnan, Anil, Duvenaud, Ganguli, Barez, Clark, Ndousse, Sachan, Sellitto, Sharma, DasSarma, Grosse, Kravec, Bai, Witten, Favaro, Brauner, Karnofsky, Christiano, Bowman, Graham, Kaplan, Mindermann, Greenblatt, Shlegeris, Schiefer, Perez. Sleeper agents: Training deceptive llms that persist through safety training. arXiv preprint arXiv:2401.05566. https://arxiv.org/abs/2401.05566
通过训练植入大语言模型的后门能够抵抗监督微调、强化学习与红队对抗训练,表明标准安全训练无法可靠消除欺骗性对齐。
[Hudgens and Halloran 2008] Hudgens, Halloran. Toward causal inference with interference. Journal of the American Statistical Association. https://pmc.ncbi.nlm.nih.gov/articles/PMC2600548/
该论文定义了处理在组内产生溢出时的直接、间接、总计和整体因果效应。
[Hugging Face 2026] Hugging Face. Hotswapping adapters. https://huggingface.co/docs/peft/package_reference/hotswap
PEFT 官方文档说明了 LoRA 热切换,以及适配方法、目标模块、秩、缩放和编译模型形状方面的约束。
[Hugging Face 2026] Hugging Face. Models: loading custom code. https://huggingface.co/docs/transformers/en/models
Transformers 文档说明 trust_remote_code 会执行仓库中的自定义模型代码,并建议先审查代码,再固定到具体提交版本。
[Hugging Face 2026] Hugging Face. Download files from the hub. https://huggingface.co/docs/huggingface_hub/en/guides/download
Hub 客户端可以按分支、标签、拉取请求或完整提交哈希解析仓库快照,并下载完整快照或经过筛选的文件集。
[Hugging Face n.d.] Hugging Face. TRL documentation. https://huggingface.co/docs/trl
TRL 文档介绍了当前的监督微调、偏好学习、奖励模型和强化学习训练器,以及它们所需的数据集格式。
[Hugging Face n.d.] Hugging Face. PEFT documentation. https://huggingface.co/docs/peft
PEFT 文档介绍了参数高效方法、适配器配置、检查点格式、合并方式,以及与分布式训练的集成。
[Hui et al. 2024] Hui, Yang, Cui, Yang, Liu, Zhang, Liu, Zhang, Yu, Lu, Dang, Fan, Zhang, Yang, Men, Huang, Zheng, Miao, Quan, Feng, Ren, Ren, Zhou, Lin. Qwen2.5-coder technical report. arXiv preprint arXiv:2409.12186. https://arxiv.org/abs/2409.12186
报告 Qwen2.5-Coder:先训练 5.2T 文件级词元,再训练约 300B 仓库级长上下文词元,混合代码、文本与数学,并使用下一词元和中间填充目标。
[Humlum and Vestergaard 2025] Humlum, Vestergaard. Still waters, rapid currents: Early labor market transformation under generative AI. https://www.nber.org/papers/w33777
丹麦关联数据发现聊天机器人使用和任务重组迅速增长,但在 ChatGPT 发布两年后,记录工时或收入未出现可检测的平均影响。
[Huyen 2025] Huyen. AI engineering: Building applications with foundation models. O'Reilly Media. https://www.oreilly.com/library/view/ai-engineering/9781098166298/
Chip Huyen 的著作介绍 AI 工程实践:基于基础模型构建生产应用,涵盖评测、模型适配技术(提示工程、RAG、微调、智能体)与推理部署。
[Huynh and Hardouin 2023] Huynh, Hardouin. PoisonGPT: How we hid a lobotomized LLM on hugging face to spread fake news. https://blog.mithrilsecurity.io/poisongpt-how-we-hid-a-lobotomized-llm-on-hugging-face-to-spread-fake-news/
该教育性演示编辑 GPT-J-6B 的一个关联,并以差一个字母的命名空间发布;仅报告有限对照检查,未报告受害者或传播。
[Hwang et al. 2025] Hwang, Wang, Gu. Dynamic chunking for end-to-end hierarchical sequence modeling. arXiv preprint arXiv:2507.07955. https://arxiv.org/abs/2507.07955
H-Net 在层级字节模型内部学习依内容而定的切块边界;其单阶段配置在论文的算力与数据对齐实验中超过了 BPE 基线。
[IEEE Electronics Packaging Society Test Technical Committee 2022] IEEE Electronics Packaging Society Test Technical Committee. Heterogeneous integrated product testability: Best-known methods (BKM). https://cmte.ieee.org/eps-test/wp-content/uploads/sites/132/2022/01/IEEE_EPS_Test_Het_Int_Product_Testability_BKM_Final_v1_0-1-14-22-1.pdf
这份技术委员会指南介绍了异构集成在晶圆、裸片、封装和系统层面的测试,以及已知合格裸片筛选在成本与覆盖率上的限制。
[Ilharco et al. 2023] Ilharco, Ribeiro, Wortsman, Gururangan, Schmidt, Hajishirzi, Farhadi. Editing models with task arithmetic. https://arxiv.org/abs/2212.04089
任务算术提出任务向量(微调权重减去预训练权重所得方向),通过对其取负或相加即可在无需重新训练的情况下编辑模型行为。
[Inan et al. 2023] Inan, Upasani, Chi, Rungta, Iyer, Mao, Tontchev, Hu, Fuller, Testuggine, Khabsa. Llama guard: LLM-based input-output safeguard for human-AI conversations. arXiv preprint arXiv:2312.06674. https://arxiv.org/abs/2312.06674
Llama Guard 是一个基于 Llama2-7b 的指令微调模型,依据可定制的安全性风险分类体系对用户输入和 LLM 输出同时进行分类,性能达到或超过现有内容审核 API。
[Inception Labs et al. 2025] Inception Labs, Khanna, Kharbanda, Li, others. Mercury: Ultra-fast language models based on diffusion. arXiv preprint arXiv:2506.17298. https://arxiv.org/abs/2506.17298
Mercury Coder 是一组基于扩散的商业级大语言模型,通过并行生成词元,在保持相近代码质量的同时实现比自回归模型高达 10 倍的吞吐量。
[International Energy Agency 2025] International Energy Agency. Energy and AI. https://www.iea.org/reports/energy-and-ai
国际能源署对计算设备和设施基础设施带来的数据中心用电需求建模,并说明其能源情景背后的假设。
[International Organization for Standardization 2020] International Organization for Standardization. Document management—portable document format—part 2: PDF 2.0. https://www.iso.org/standard/75839.html
ISO 32000-2 规定了 PDF 2.0 文档格式,包括处理器可能遇到的对象、图形、注释、元数据、安全和一致性规则。
[International Organization for Standardization 2026] International Organization for Standardization. ISO/CD 22144: Authenticity of information, content credentials. https://www.iso.org/standard/90726.html
这份 ISO 标准草案正在制定内容凭证和真实性元数据的互操作要求。
[Internet Engineering Task Force 2026] Internet Engineering Task Force. Web bot auth working group charter. https://datatracker.ietf.org/wg/webbotauth/about/
Web Bot Auth 工作组章程界定了自动化 Web 客户端身份认证标准的范围,并明确身份认证不等于授权。
[Irving et al. 2018] Irving, Christiano, Amodei. AI safety via debate. arXiv preprint arXiv:1805.00899. https://arxiv.org/abs/1805.00899
辩论把对抗式论证作为监督方法;原论文的实验使用 MNIST 和稀疏分类器,而非人类裁判。
[ISO/IEC 2023] ISO/IEC. ISO/IEC 42001:2023, information technology, artificial intelligence, management system. https://www.iso.org/standard/42001
首个国际 AI 管理体系标准:它认证的是组织具备成文的政策、风险评估、角色分工与持续改进闭环,而不是任何模型行为安全。
[Ivanov et al. 2021] Ivanov, Dryden, Ben-Nun, Li, Hoefler. Data movement is all you need: a case study on optimizing transformers. Proceedings of Machine Learning and Systems (MLSys). https://proceedings.mlsys.org/paper_files/paper/2021/hash/bc86e95606a6392f51f95a8de106728d-Abstract.html
一项实测 BERT 案例研究,说明全局布局与数据移动优化如何在不改变数学含义的前提下加速单层与完整模型。
[Jacovi et al. 2025] Jacovi, Wang, Alberti, Tao, Lipovetz, Olszewska, Haas, Liu, Keating, Bloniarz, Saroufim, Fry, Marcus, Kukliansky, Tomar, Swirhun, Xing, Wang, Gurumurthy, Aaron, Ambar, Fellinger, Wang, Zhang, Goldshtein, Das. The FACTS grounding leaderboard: Benchmarking LLMs' ability to ground responses to long-form input. arXiv preprint arXiv:2501.03200. https://arxiv.org/abs/2501.03200
原版 FACTS Grounding 基准使用给定的长文档、独立的合格性门控和三个模型评判器,在版本化的 2025 年协议下估计回答级 grounding。
[Jain et al. 2024] Jain, Han, Gu, Li, Yan, Zhang, Wang, Solar-Lezama, Sen, Stoica. LiveCodeBench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974. https://arxiv.org/abs/2403.07974
LiveCodeBench 使用近期发布的竞赛题,并覆盖生成、执行、测试输出预测和自我修复,以降低题目暴露并拓宽编程评估。
[JAX Authors 2026] JAX Authors. The autodiff cookbook. https://docs.jax.dev/en/latest/notebooks/autodiff_cookbook.html
该手册推导雅可比向量积与向量雅可比积,并解释输入输出维度如何决定高效的微分模式。
[JAX Authors 2026] JAX Authors. Just-in-time compilation. https://docs.jax.dev/en/latest/jit-compilation.html
JAX 文档解释了追踪、编译缓存、静态参数,以及影响即时编译的程序形状约束。
[Jeong et al. 2024] Jeong, Baek, Cho, Hwang, Park. Adaptive-RAG: Learning to adapt retrieval-augmented large language models through question complexity. https://aclanthology.org/2024.naacl-long.389/
Adaptive-RAG 根据估计的问题复杂度,在不检索、单步检索与迭代检索之间路由。
[Jiang et al. 2024] Jiang, Sablayrolles, Roux, Mensch, Savary, Bamford, Chaplot, Casas, Hanna, Bressand, Lengyel, Bour, Lample, Lavaud, Saulnier, Lachaux, Stock, Subramanian, Yang, Antoniak, Scao, Gervet, Lavril, Wang, Lacroix, Sayed. Mixtral of experts. https://arxiv.org/abs/2401.04088
Mixtral 8x7B 是一个稀疏混合专家(MoE)仅解码器模型,总参数量 46.7B,每词元仅激活 12.9B 参数,在 Apache 2.0 许可下以 6 倍更快的推理速度超越 Llama 2 70B。
[Jimenez et al. 2024] Jimenez, Yang, Wettig, Yao, Pei, Press, Narasimhan. SWE-bench: Can language models resolve real-world GitHub issues?. https://proceedings.iclr.cc/paper_files/paper/2024/hash/edac78c3e300629acfe6cbe9ca88fb84-Abstract-Conference.html
SWE-bench 是一个包含来自 12 个 Python 仓库的 2,294 个真实 GitHub 问题修复任务的基准,要求模型编辑代码库以通过测试,最优模型 Claude 2 仅能解决 1.96% 的问题。
[Jin et al. 2025] Jin, Zeng, Yue, Yoon, Arik, Wang, Zamani, Han. Search-R1: Training LLMs to reason and leverage search engines with reinforcement learning. https://arxiv.org/abs/2503.09516
Search-R1 训练语言模型在推理中多次发起搜索,并让检索段落保留在上下文中、但不进入策略损失。
[Johari et al. 2022] Johari, Koomen, Pekelis, Walsh. Always valid inference: Continuous monitoring of a/B tests. Operations Research. https://doi.org/10.1287/opre.2021.2135
Johari 等人为持续监控且允许择时停止的实验设计了始终有效的 p 值和置信区间。
[Johnson et al. 2021] Johnson, Douze, Jégou. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data. https://arxiv.org/abs/1702.08734
FAISS 论文介绍了在 GPU 上进行大规模精确和近似最近邻搜索的算法,并评估了吞吐量和召回率。
[Jones et al. 2020] Jones, Nadalin, Campbell, Bradley, Mortimore. OAuth 2.0 token exchange. https://www.rfc-editor.org/rfc/rfc8693.html
RFC 8693 定义了用于模拟和委托的令牌交换,但不保证权限只会收缩,也不保证撤销会传播。
[Jouppi et al. 2017] Jouppi, Young, Patil, Patterson, others. In-datacenter performance analysis of a tensor processing unit. https://arxiv.org/abs/1704.04760
本文记录了首代量产 TPU,包括其软件管理内存与 256 乘 256 矩阵乘法单元。
[Jouppi et al. 2023] Jouppi, Kurian, Li, Ma, Nagarajan, Nai, Patil, Subramanian, Swing, Towles, Young, Zhou, Zhou, Patterson. <span class="nocase">TPU v4</span>: An optically reconfigurable supercomputer for machine learning with hardware support for embeddings. https://arxiv.org/abs/2304.01433
TPU v4 使用光路交换机配置和重配置大型加速器互连。
[Ju et al. 2024] Ju, Wang, Shen, Tan, Xin, Yang, Liu, Leng, Song, Tang, others. NaturalSpeech 3: Zero-shot speech synthesis with factorized codec and diffusion models. https://arxiv.org/abs/2403.03100
NaturalSpeech 3 提出 FACodec 与分解扩散模型,将语音分解为内容、韵律、音色和声学细节的解耦子空间,用于零样本文本到语音合成,在多说话人数据集上达到人类水平的自然度。
[Kahneman 2011] Kahneman. Thinking, fast and slow. Farrar, Straus,Giroux.
Kahneman 对比了快速直觉的系统 1 与缓慢审慎的系统 2 思维,以及二者各自产生的认知偏差。
[Kalai and Vempala 2024] Kalai, Vempala. Calibrated language models must hallucinate. https://arxiv.org/abs/2311.14648
该论文证明,在某些条件下,即使语言模型已经校准,也必然会在无法从训练证据中区分的事实上出错。
[Kalai et al. 2025] Kalai, Nachum, Vempala, Zhang. Why language models hallucinate. arXiv preprint arXiv:2509.04664. https://arxiv.org/abs/2509.04664
本文通过将大语言模型(LLM)生成错误归约为二分类问题,论证幻觉在预训练阶段由统计压力产生,并因主流基准奖励猜测而非表达不确定性而在后训练阶段持续存在。
[Kandpal et al. 2022] Kandpal, Wallace, Raffel. Deduplicating training data mitigates privacy risks in language models. https://proceedings.mlr.press/v162/kandpal22a.html
Kandpal 等人量化了重复次数与记忆之间的超线性关系,并在所研究的语言模型语料去重后测得显著更低的抽取风险。
[Kang et al. 2025] Kang, Yue, Lu, Lin, Zhao, Wang, Huang, Feng. How far is video generation from world model: a physical law perspective. https://openreview.net/forum?id=DLlVjZQ7vD
扩展扩散视频生成模型可实现完美的分布内泛化,但无法在分布外场景中推断物理定律,表明单纯扩展不足以让模型发现基本物理规律。
[Kantamneni et al. 2025] Kantamneni, Engels, Rajamanoharan, Tegmark, Nanda. Are sparse autoencoders useful? A case study in sparse probing. https://proceedings.mlr.press/v267/kantamneni25a.html
一项跨 113 个数据集的探针研究发现,稀疏自编码器(SAE)潜变量在数据稀缺、类别不平衡、标签噪声和协变量偏移等困难设置下,未能持续超越简单基线方法的 LLM 激活探针性能。
[Kanzhelev et al. 2021] Kanzhelev, McLean, Reitbauer, Drutu, Molnar, Shkuro. Trace context. https://www.w3.org/TR/trace-context/
Trace Context 标准化 traceparent 与 tracestate HTTP 字段,使同一分布式请求能跨服务和追踪厂商保持关联。
[Kaplan et al. 2020] Kaplan, McCandlish, Henighan, Brown, Chess, Child, Gray, Radford, Wu, Amodei. Scaling laws for neural language models. https://arxiv.org/abs/2001.08361
确立了语言模型损失随模型规模、数据集规模与算力呈幂律下降,并指出计算最优的训练应训练很大的模型、只用相对较少的数据,并在收敛前就停止。
[Kapoor and Narayanan 2023] Kapoor, Narayanan. Leakage and the reproducibility crisis in machine-learning-based science. Patterns. https://doi.org/10.1016/j.patter.2023.100804
Kapoor 和 Narayanan 对机器学习研究中常见的数据泄漏形式进行分类,并说明它们如何使所报告的泛化结果失效。
[Kapoor et al. 2024] Kapoor, Bommasani, Klyman, Longpre, Ramaswami, Cihon, Hopkins, Bankston, Biderman, Bogen, Chowdhury, Engler, Henderson, Jernite, Lazar, Maffulli, Nelson, Pineau, Skowron, Song, Storchan, Zhang, Ho, Liang, Narayanan. On the societal impact of open foundation models. https://arxiv.org/abs/2403.07918
该立场论文从收益、风险和相对既有技术的边际风险分析开放权重基础模型,并让竞争主张取决于配套访问条件与能力。
[Karpinska et al. 2021] Karpinska, Akoury, Iyyer. The perils of using mechanical turk to evaluate open-ended text generation. Association for Computational Linguistics. https://aclanthology.org/2021.emnlp-main.97/
Karpinska 等人发现众包工人可能无法区分开放式故事中的人类文本与模型文本,而专家教师和成对示例能提供更强的评测信号。
[Karpukhin et al. 2020] Karpukhin, Oğuz, Min, Lewis, Wu, Edunov, Chen, Yih. Dense passage retrieval for open-domain question answering. https://aclanthology.org/2020.emnlp-main.550/
DPR 分别训练查询与段落编码器,并在多个开放域问答数据集上展示了稠密检索的效果。
[Karras et al. 2022] Karras, Aittala, Aila, Laine. Elucidating the design space of diffusion-based generative models. https://arxiv.org/abs/2206.00364
Karras 等人将扩散模型的训练与采样分解为清晰的设计空间,通过改进预处理和调度在 CIFAR-10 与 ImageNet-64 上创下新的 FID 记录。
[Katz et al. 2024] Katz, Kokel, Srinivas, Sohrabi. Thought of search: Planning with language models through the lens of efficiency. https://proceedings.neurips.cc/paper_files/paper/2024/hash/fa080fe0f218871faec1d8ba20e491d5-Abstract-Conference.html
该论文把语言模型规划分析为搜索问题,并按解答质量和探索候选思路所花的计算量比较不同方法。
[Keller et al. 2026] Keller, Kwegyir-Aggrey, Steed, Rao, Sharp, Bergman. Expanding the AI evaluation toolbox with statistical models. https://www.nist.gov/publications/expanding-ai-evaluation-toolbox-statistical-models
NIST 区分固定基准上的准确率与相关任务总体上的泛化准确率,并说明评估假设为何决定有效的不确定性估计。
[Kenton et al. 2024] Kenton, Siegel, Kramár, Brown-Cohen, Albanie, Bulian, Agarwal, Lindner, Tang, Goodman, Shah. On scalable oversight with weak llms judging strong llms. https://proceedings.neurips.cc/paper_files/paper/2024/hash/899511e37a8e01e1bd6f6f1d377cc250-Abstract-Conference.html
在抽取式问答、数学、编程、逻辑和多模态任务上,指定立场的辩论优于咨询;但在信息不对称的抽取任务之外,它相对直接问答的优势并不稳定。
[Khalifa et al. 2026] Khalifa, Agarwal, Logeswaran, Kim, Peng, Lee, Lee, Wang. Process reward models that think. Transactions on Machine Learning Research. https://arxiv.org/abs/2504.16828
ThinkPRM 是一种生成式 PRM,把逐步验证写成思维链,仅用 PRM800K 约 1% 的过程标签即可超过判别式 PRM 与 LLM 评判。
[Khan et al. 2024] Khan, Hughes, Valentine, Ruis, Sachan, Radhakrishnan, Grefenstette, Bowman, Rocktäschel, Perez. Debating with more persuasive llms leads to more truthful answers. PMLR. https://proceedings.mlr.press/v235/khan24a.html
实验研究表明,让两个专家 LLM 就对立答案进行辩论可实现可扩展监督,弱评判者在 QuALITY 任务上分别达到 76%(LLM)和 88%(人类)的准确率,远高于 48% 和 60% 的朴素基线。
[Khashabi et al. 2022] Khashabi, Stanovsky, Bragg, Lourie, Kasai, Choi, Smith, Weld. GENIE: Toward reproducible and standardized human evaluation for text generation. Association for Computational Linguistics. https://aclanthology.org/2022.emnlp-main.787/
GENIE 研究文本生成中的人工评测设计选择,并提出标准化平台,以提升跨任务与跨标注人群的可复现性。
[Khattab and Zaharia 2020] Khattab, Zaharia. ColBERT: Efficient and effective passage search via contextualized late interaction over BERT. https://arxiv.org/abs/2004.12832
ColBERT 把查询与文档交互推迟到上下文词元嵌入上的 MaxSim 阶段,以更大的索引换取更细粒度的匹配。
[Khazatsky et al. 2024] Khazatsky, Pertsch, Nair, others. DROID: a large-scale in-the-wild robot manipulation dataset. https://arxiv.org/abs/2403.12945
DROID 使用统一的 Franka 机器人配置,在 564 个场景中收集了 7.6 万条真实机器人操作演示,总计约 350 小时。
[Kim and Rush 2016] Kim, Rush. Sequence-level knowledge distillation. https://arxiv.org/abs/1606.07947
序列级知识蒸馏让学生神经机器翻译模型在教师束搜索输出上训练,推理速度提升 10 倍,贪婪解码下 BLEU 与教师相当。
[Kim and Rush 2016] Kim, Rush. Sequence-level knowledge distillation. Association for Computational Linguistics. https://aclanthology.org/D16-1139/
序列级蒸馏让学生学习教师解码出的序列,从而强调结构化预测分布中被选中的模式。
[Kim et al. 2024] Kim, Pertsch, others. OpenVLA: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246. https://arxiv.org/abs/2406.09246
OpenVLA 是一个 7B 参数的开源视觉-语言-动作模型(VLA),在 970k 机器人演示上训练,性能超越闭源 55B RT-2-X,并支持在消费级 GPU 上通过 LoRA 进行参数高效微调。
[Kim et al. 2025] Kim, Garg, Peng, Garg. Correlated errors in large language models. PMLR. https://proceedings.mlr.press/v267/kim25e.html
一项覆盖 350 多个模型的研究发现大量共同错误,说明架构或提供方不同并不能保证错误彼此独立。
[Kim et al. 2025] Kim, Kotha, Liang, Hashimoto. Pre-training under infinite compute. arXiv preprint arXiv:2509.14786. https://arxiv.org/abs/2509.14786
在数据受限、算力不受限的预训练场景下,结合强正则化、参数扩展与集成扩展,可比标准方案实现更低的扩展律渐近损失并提升 5.17 倍数据效率。
[Kimi Team 2025] Kimi Team. Kimi K2: Open agentic intelligence. arXiv preprint arXiv:2507.20534. https://arxiv.org/abs/2507.20534
Kimi K2 是总参数 1T、激活 32B 的 MoE 模型,用 MuonClip(在 Muon 上加 QK-clip 抑制注意力 logit 爆炸)在 15.5T 词元上预训练,全程没有损失尖峰。
[Kimi Team et al. 2025] Kimi Team, Du, Gao, Xing, Jiang, Chen, Li, Xiao, Du, Liao, Tang, Wang, Zhang, others. Kimi k1.5: Scaling reinforcement learning with LLMs. arXiv preprint arXiv:2501.12599. https://arxiv.org/abs/2501.12599
Kimi k1.5 报告长上下文强化学习及四种 long2short 机制:权重合并、最短正确答案拒绝采样、偏好训练和长度受限的强化学习阶段。
[Kirchenbauer et al. 2023] Kirchenbauer, Geiping, Wen, Katz, Miers, Goldstein. A watermark for large language models. https://arxiv.org/abs/2301.10226
本文提出一种大语言模型输出的水印框架,通过在采样时将词元偏向随机"绿名单"来嵌入信号,无需访问模型参数即可从最少 25 个词元中统计检测水印。
[Kirchstetter et al. 2025] Kirchstetter, Smith, Shehabi, Sartor. DOE data center load flexibility workshop summary. https://eta.lbl.gov/publications/doe-data-center-load-flexibility
该研讨会报告列出了把数据中心负载用作电网灵活性资源所需的技术、市场和运营条件。
[Kirk et al. 2024] Kirk, Whitefield, Röttger, Bean, Margatina, Ciro, Mosquera, Bartolo, Williams, He, Vidgen, Hale. The PRISM alignment dataset: What participatory, representative and individualised human feedback reveals about the subjective and multicultural alignment of large language models. https://arxiv.org/abs/2404.16019
PRISM 将 75 个国家 1,500 名参与者在 8,011 段对话中的细粒度反馈与参与者画像关联,揭示对齐偏好的主观性和跨文化差异。
[Kirkpatrick and others 2017] Kirkpatrick, others. Overcoming catastrophic forgetting in neural networks. https://www.pnas.org/doi/full/10.1073/pnas.1611835114
弹性权重巩固通过惩罚对先前任务重要参数的改动,限制灾难性遗忘。
[Knight and Leveson 1986] Knight, Leveson. An experimental evaluation of the assumption of independence in multi-version programming. IEEE Transactions on Software Engineering. https://libraopen.library.virginia.edu/entities/publication/4ac33eeb-79b4-46e4-aef9-f6ec56a62286
实验发现独立开发的多个程序版本仍会发生相关故障,从而质疑其错误相互独立的假设。
[Kocsis and Szepesvári 2006] Kocsis, Szepesvári. Bandit based monte-carlo planning. Springer. https://doi.org/10.1007/11871842_29
UCT 把置信上界规则用于蒙特卡洛树搜索,使节点选择在探索和估计价值之间取得平衡。
[Kohavi et al. 2009] Kohavi, Longbotham, Sommerfield, Henne. Controlled experiments on the web: Survey and practical guide. Data Mining and Knowledge Discovery. https://doi.org/10.1007/s10618-008-0114-1
Kohavi 等人解释了在线对照实验中的随机化、统计功效、指标、方差和常见有效性缺陷。
[Kojima et al. 2022] Kojima, Gu, Reid, Matsuo, Iwasawa. Large language models are zero-shot reasoners. https://arxiv.org/abs/2205.11916
在问题后添加"Let's think step by step"(Zero-shot-CoT)无需任务特定的少样本示例,即可大幅提升 LLM 在算术和符号推理任务上的思维链(CoT)推理能力。
[Komatsuzaki et al. 2023] Komatsuzaki, Puigcerver, Lee-Thorp, Ruiz, Mustafa, Ainslie, Tay, Dehghani, Houlsby. Sparse upcycling: Training mixture-of-experts from dense checkpoints. https://arxiv.org/abs/2212.05055
稀疏升循环(sparse upcycling)从预训练的稠密检查点初始化混合专家(MoE)模型,以约 50% 的原始预训练算力同时超越稠密继续训练和从头训练的 MoE 模型。
[Köpf et al. 2023] Köpf, Kilcher, Rütte, Anagnostidis, Tam, Stevens, Barhoum, Duc, Stanley, Nagyfi, others. OpenAssistant conversations: Democratizing large language model alignment. arXiv preprint arXiv:2304.07327. https://arxiv.org/abs/2304.07327
OpenAssistant Conversations 发布了一份众包对齐语料,其中包含对话树、消息评分和多语言人类反馈。
[Korbak et al. 2025] Korbak, Clymer, Hilton, Shlegeris, Irving. A sketch of an AI control safety case. arXiv preprint arXiv:2501.17315. https://arxiv.org/abs/2501.17315
这份安全论证草案明确指出,从控制评估走向部署还需要哪些论据,包括诱发攻击、从评估外推到部署,以及保守估计风险。
[Korinek and Vipra 2025] Korinek, Vipra. Concentrating intelligence: Scaling and market structure in artificial intelligence. Economic Policy. https://doi.org/10.1093/epolic/eiae057
Korinek 与 Vipra 分析规模、范围、反馈、惯性和整合如何可能使前沿模型市场更加集中,同时把未来倾斜视为有条件的结果。
[Korthikanti et al. 2022] Korthikanti, Casper, Lym, McAfee, Andersch, Shoeybi, Catanzaro. Reducing activation recomputation in large transformer models. https://arxiv.org/abs/2205.05198
本文提出序列并行与选择性激活重计算,在结合张量并行训练大型 Transformer 模型时,将激活内存降低 5 倍,并将激活重计算的执行时间开销减少 90% 以上。
[Koster et al. 2022] Koster, Illyes, Zeller, Sassman. Robots exclusion protocol. https://www.rfc-editor.org/rfc/rfc9309.html
将 robots.txt 标准化为爬虫偏好协议,并明确区分于访问授权。
[Kreutzer and others 2022] Kreutzer, others. Quality at a glance: An audit of web-crawled multilingual datasets. Transactions of the Association for Computational Linguistics. https://aclanthology.org/2022.tacl-1.4/
对五个多语言 Web 数据集中的 205 份特定语言语料进行人工审计后,研究发现系统性的语言标注和内容质量问题,在低资源语言中尤其严重。
[KServe Contributors 2026] KServe Contributors. V2 inference protocol. https://kserve.github.io/website/docs/concepts/architecture/data-plane/v2-protocol
KServe V2 规定了基于 HTTP 或 gRPC 的健康检查、元数据和推理操作;聊天模板、流式传输、工具调用和采样默认值等应用语义仍需单独做契约测试。
[Kubernetes 2025] Kubernetes. Multi-tenancy. https://kubernetes.io/docs/concepts/security/multi-tenancy/
Kubernetes 文档列出了 AI 平台在模型、缓存、索引和工具边界上继承并扩展的隔离与公平性工具。
[Kubernetes Authors 2023] Kubernetes Authors. CSI volume cloning. https://kubernetes.io/docs/concepts/storage/volume-pvc-datasource/
Kubernetes 可通过兼容的 CSI 驱动克隆 PVC,而具体复制实现与性能由存储后端决定。
[Kubernetes Authors 2024] Kubernetes Authors. Volume snapshots. https://kubernetes.io/docs/concepts/storage/volume-snapshots/
当 CSI 驱动与存储后端实现快照支持时,Kubernetes 通过 VolumeSnapshot 提供标准快照接口。
[Kubernetes Authors 2026] Kubernetes Authors. Volumes. https://kubernetes.io/docs/concepts/storage/volumes/
Kubernetes 区分随 Pod 生命周期结束的临时卷与超越单个 Pod 生命周期的持久卷;拓扑、访问、回收与备份仍是独立属性。
[Kubernetes Authors 2026] Kubernetes Authors. Jobs. https://kubernetes.io/docs/concepts/workloads/controllers/job/
Kubernetes 文档指出,即使完成数和并行度均为一,Job 有时仍会启动同一程序两次,因此工作负载必须容忍重复执行。
[Kubernetes Authors 2026] Kubernetes Authors. Resource quotas. https://kubernetes.io/docs/concepts/policy/resource-quotas/
Kubernetes ResourceQuota 用于限制命名空间内资源和对象的总消耗。
[Kubernetes Authors 2026] Kubernetes Authors. Resource management for pods and containers. https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
Kubernetes 在资源请求无法满足时使 Pod 保持等待状态,并将资源请求与限制定义为调度和运行时控制。
[Kubernetes SIG Scheduling 2026] Kubernetes SIG Scheduling. All-or-nothing scheduling. https://kueue.sigs.k8s.io/docs/concepts/all_or_nothing/
Kueue 的全有或全无调度只有在集群能同时放置工作负载所需的一组 Pod 时才准入该任务。
[Kubernetes SIG Scheduling n.d.] Kubernetes SIG Scheduling. Kueue overview. https://kueue.sigs.k8s.io/docs/overview/
Kueue 管理消耗配额的工作负载何时准入、等待或抢占,同时将 Pod 放置、自动扩缩容和作业生命周期交给相应的 Kubernetes 组件。
[Kuchnik et al. 2022] Kuchnik, Klimovic, Simsa, Smith, Amvrosiadis. Plumber: Diagnosing and removing performance bottlenecks in machine learning data pipelines. https://proceedings.mlsys.org/paper_files/paper/2022/hash/d0e90e9a9310570dfa643aa3b2da6e89-Abstract.html
Plumber 通过感知资源的流水线分析定位输入瓶颈,并调节并行度、预取和缓存。
[Kudo and Richardson 2018] Kudo, Richardson. SentencePiece: a simple and language independent subword tokenizer and detokenizer for neural text processing. Association for Computational Linguistics. https://aclanthology.org/D18-2012/
SentencePiece 是一种语言无关的子词分词器,直接从原始句子训练,支持字节对编码(BPE)和 unigram 语言模型,无需预分词即可实现端到端文本处理。
[Kudo 2018] Kudo. Subword regularization: Improving neural network translation models with multiple subword candidates. Association for Computational Linguistics. https://aclanthology.org/P18-1007/
子词正则化通过在训练时对多个概率采样的子词切分方案进行边际化,并提出基于一元语言模型的分词器作为 BPE(字节对编码)的概率替代方案,以提升神经机器翻译的鲁棒性。
[Kumar et al. 2023] Kumar, Seetharaman, Luebs, Kumar, Kumar. High-fidelity audio compression with improved RVQGAN. https://arxiv.org/abs/2306.06546
Kumar 等人提出 Improved RVQGAN,一种通用神经音频编解码器,可将 44.1 kHz 音频以 8 kbps(90 倍压缩)压缩为离散词元,保真度优于 EnCodec 与 SoundStream。
[Kumar et al. 2025] Kumar, Roh, Naseh, Karpinska, Iyyer, Houmansadr, Bagdasarian. OverThink: Slowdown attacks on reasoning LLMs. arXiv preprint arXiv:2502.02542. https://arxiv.org/abs/2502.02542
OverThink 构造了一类减速攻击,诱导推理模型在给出答案前消耗过多令牌和时间。
[Kung 1982] Kung. Why systolic architectures?. Computer. https://www.eecs.harvard.edu/~htk/publication/1982-kung-why-systolic-architecture.pdf
Kung 将脉动阵列解释为一种规则阵列:数据有节奏地流过处理单元,从而利用并行性与局部性。
[Kusupati et al. 2022] Kusupati, Bhatt, Rege, Wallingford, Sinha, Ramanujan, Howard-Snyder, Chen, Kakade, Jain, Farhadi. Matryoshka representation learning. https://proceedings.neurips.cc/paper_files/paper/2022/hash/c32319f4868da7613d78af9993100e42-Abstract-Conference.html
Matryoshka 表示学习(MRL)训练单个嵌入向量,使其任意前缀维度均构成有效表示,从而在保持精度的同时实现分类与检索嵌入尺寸最多缩小 14 倍。
[Kutasov et al. 2025] Kutasov, Sun, Colognese, Weij, Petrini, Zhang, Hughes, Deng, Sleight, Tracy, Shlegeris, Benton. SHADE-Arena: Evaluating sabotage and monitoring in LLM agents. arXiv preprint arXiv:2506.15740. https://arxiv.org/abs/2506.15740
SHADE-Arena 把良性的长程智能体任务与隐藏的破坏性副任务配对,同时评估智能体隐蔽完成副任务的能力和 LLM 监测器识别记录的能力。
[Kwa et al. 2025] Kwa, West, others. Measuring AI ability to complete long software tasks. https://proceedings.neurips.cc/paper_files/paper/2025/hash/85069585133c4c168c865e65d72e9775-Abstract-Conference.html
METR 提出以智能体能自主完成的任务时长衡量 AI 进展,发现该指标在过去 6 年中每约 7 个月翻倍一次。
[Kwa 2026] Kwa. A simpler AI timelines model predicts 99% AI R&D automation in ~2032. https://metr.org/notes/2026-02-10-simpler-ai-timelines-model/
METR 提出一个仅含 8 个参数的 AI 时间线预测模型,预测到约 2032 年 AI 研发将实现约 99% 的自动化。
[Kwon et al. 2023] Kwon, Li, Zhuang, Sheng, Zheng, Yu, Gonzalez, Zhang, Stoica. Efficient memory management for large language model serving with PagedAttention. ACM. https://doi.org/10.1145/3600006.3613165
vLLM 通过分块分配与共享 KV 缓存来减少碎片,并提高服务系统可同时驻留的序列数量。
[Lakens et al. 2018] Lakens, Scheel, Isager. Equivalence testing for psychological research: a tutorial. Advances in Methods and Practices in Psychological Science. https://doi.org/10.1177/2515245918770963
该教程说明如何用预先声明的等效界限证明效应小于具有实际意义的阈值;对零效应检验不显著并不能提供这种证据。
[Lambert et al. 2024] Lambert, Pyatkin, Morrison, Miranda, Lin, Chandu, Dziri, Kumar, Zick, Choi, Smith, Hajishirzi. RewardBench: Evaluating reward models for language modeling. arXiv preprint arXiv:2403.13787. https://arxiv.org/abs/2403.13787
RewardBench 在结构化对话、推理和安全偏好数据上评估显式与隐式奖励模型,揭示汇总的同分布准确率可能掩盖的弱点。
[Lambert et al. 2024] Lambert, Morrison, Pyatkin, Huang, Ivison, Brahman, Miranda, Liu, Dziri, Lyu, Gu, Malik, Graf, Hwang, Yang, Le Bras, Tafjord, Wilhelm, Soldaini, Smith, Wang, Dasigi, Hajishirzi. Tulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124. https://arxiv.org/abs/2411.15124
Tulu 3 是面向 Llama 3.1 底座的完全开放后训练配方,结合 SFT、DPO 与 RLVR,并公开数据、权重和训练代码。
[Lamport et al. 1982] Lamport, Shostak, Pease. The byzantine generals problem. ACM Transactions on Programming Languages and Systems. https://lamport.azurewebsites.net/pubs/the-byz-generals.pdf
这篇奠基论文明确了部分分布式参与者可能任意作恶时,实现一致性所需的假设和故障上限。
[LangChain 2026] LangChain. Use time travel. https://docs.langchain.com/oss/python/langgraph/use-time-travel
LangGraph 区分重放与分叉:重放重新执行检查点之后的节点,分叉则创建新的检查点分支而不修改原历史。
[LangChain 2026] LangChain. Persistence. https://docs.langchain.com/oss/python/langgraph/persistence
LangGraph 文档介绍了图执行中的检查点、线程、状态历史和持久化语义。
[Lanham et al. 2023] Lanham, Chen, Radhakrishnan, Steiner, Denison, Hernandez, Li, Durmus, Hubinger, Kernion, Lukošiūtė, Nguyen, Cheng, Joseph, Schiefer, Rausch, Larson, McCandlish, Kundu, Kadavath, Yang, Henighan, Maxwell, Telleen-Lawton, Hume, Hatfield-Dodds, Kaplan, Brauner, Bowman, Perez. Measuring faithfulness in chain-of-thought reasoning. arXiv preprint arXiv:2307.13702. https://arxiv.org/abs/2307.13702
干预实验检验思维链文本是否反映模型得出答案的真实原因,并发现了多种不忠实推理。
[Lattner et al. 2021] Lattner, Amini, Bondhugula, Cohen, Davis, Pienaar, Riddle, Shpeisman, Vasilache, Zinenko. MLIR: Scaling compiler infrastructure for domain specific computation. https://arxiv.org/abs/2002.11054
MLIR 是由可组合方言与渐进下沉构成的编译器框架,XLA、Triton 内部、Mosaic 以及多数较新的机器学习编译器如今都构建在它之上。
[Le et al. 2023] Le, Vyas, Shi, Karrer, Sari, Moritz, Williamson, Manohar, Adi, Mahadeokar, Hsu. Voicebox: Text-guided multilingual universal speech generation at scale. https://arxiv.org/abs/2306.15687
Voicebox 是一个基于流匹配训练的非自回归生成(NAR)模型,在逾 5 万小时语音上通过上下文学习实现零样本文本到语音(TTS)、去噪与内容编辑,在可懂度和音频相似度上均超越 VALL-E。
[Leask et al. 2025] Leask, Bussmann, Pearce, Bloom, Tigges, Al Moubayed, Sharkey, Nanda. Sparse autoencoders do not find canonical units of analysis. https://proceedings.iclr.cc/paper_files/paper/2025/hash/84ca3f2d9d9bfca13f69b48ea63eb4a5-Abstract-Conference.html
在相同激活上训练的稀疏自编码器可能学到不同分解,这挑战了单一字典能够恢复模型规范特征的假设。
[Lee and See 2004] Lee, See. Trust in automation: Designing for appropriate reliance. Human Factors. https://doi.org/10.1518/hfes.46.1.50_30392
Lee 与 See 将自动化信任视为复杂情境下恰当依赖的基础,并说明展示设计与情境如何影响依赖是否合理。
[Lee et al. 2019] Lee, Gatt, Miltenburg, Wubben, Krahmer. Best practices for the human evaluation of automatically generated text. Association for Computational Linguistics. https://aclanthology.org/W19-8643/
本文把人工评测的有效性与可复现性连接到明确的规划、参与者选择、问题措辞、呈现顺序、培训、质量控制、统计分析和报告。
[Lee et al. 2022] Lee, Ippolito, Nystrom, Zhang, Eck, Callison-Burch, Carlini. Deduplicating training data makes language models better. Association for Computational Linguistics. https://aclanthology.org/2022.acl-long.577/
对大语言模型训练数据去重可将逐字记忆率降低十倍、减少训练-测试集重叠,同时以更少训练步骤达到相同或更高的准确率。
[Lee et al. 2024] Lee, Phatale, Mansoor, Mesnard, Ferret, Lu, Bishop, Hall, Carbune, Rastogi, Prakash. RLAIF vs. RLHF: Scaling reinforcement learning from human feedback with AI feedback. https://arxiv.org/abs/2309.00267
在三项所研究的语言任务上,RLAIF 用现成语言模型的判断替代人工比较标签,并训练出与 RLHF 表现相当的策略。
[Lepikhin et al. 2020] Lepikhin, Lee, Xu, Chen, Firat, Huang, Krikun, Shazeer, Chen. GShard: Scaling giant models with conditional computation and automatic sharding. https://arxiv.org/abs/2006.16668
GShard 提出轻量级注解 API 与 XLA 编译器扩展,通过自动 SPMD 分片,在 2048 块 TPU v3 上训练了 600B 参数的混合专家 Transformer,实现 100 种语言的多语言翻译。
[Leroy 2009] Leroy. A formally verified compiler back-end. Journal of Automated Reasoning. https://doi.org/10.1007/s10817-009-9155-4
CompCert 的已验证后端用 Coq 证明了从 Cminor 到 PowerPC 汇编的语义保持,把编译器正确性纳入可信证据链。
[Lester et al. 2021] Lester, Al-Rfou, Constant. The power of scale for parameter-efficient prompt tuning. https://arxiv.org/abs/2104.08691
提示微调为冻结模型学习连续输入嵌入,并报告其相对表现会随所测试 T5 模型规模增大而提高。
[Leviathan et al. 2023] Leviathan, Kalman, Matias. Fast inference from transformers via speculative decoding. PMLR. https://proceedings.mlr.press/v202/leviathan23a.html
推测解码使用更快的草稿模型和修正拒绝采样来减少目标模型调用,同时保持目标分布;论文报告的收益只对应其 T5 实验设置。
[Lewis et al. 2020] Lewis, Perez, Piktus, Petroni, Karpukhin, Goyal, Küttler, Lewis, Yih, Rocktäschel, Riedel, Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks. https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html
RAG 原始论文把序列到序列模型与稠密 Wikipedia 索引结合起来,提出参数化记忆与非参数化记忆的表述。
[Lewis et al. 2020] Lewis, Perez, Piktus, Petroni, Karpukhin, Goyal, Kuttler, Lewis, Yih, Rocktäschel, Riedel, Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks. https://proceedings.neurips.cc/paper_files/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html
该论文提出用于知识密集型任务的检索增强生成方法,将学习得到的检索器与序列生成器结合。
[Lewis et al. 2021] Lewis, Bhosale, Dettmers, Goyal, Zettlemoyer. BASE layers: Simplifying training of large, sparse models. https://arxiv.org/abs/2103.16716
BASE 层将词元到专家的分配建模为线性分配问题,无需辅助损失或额外超参数即可保证混合专家(MoE)模型中各专家负载均衡。
[Li et al. 2022] Li, Thickstun, Gulrajani, Liang, Hashimoto. Diffusion-LM improves controllable text generation. https://arxiv.org/abs/2205.14217
Diffusion-LM 将连续扩散模型适配至文本,通过迭代去噪高斯向量得到词向量,支持基于梯度的插件式可控生成,可处理句法结构等复杂控制任务。
[Li et al. 2022] Li, Li, Dall, Gu, Nieh, Sait, Stockwell. Design and verification of the arm confidential compute architecture. https://www.usenix.org/conference/osdi22/presentation/li
介绍 Arm CCA 的 Realm 架构,以及论文所研究固件实现接受的形式化验证。
[Li et al. 2023] Li, Li, Savarese, Hoi. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. https://arxiv.org/abs/2301.12597
BLIP-2 提出轻量级 Q-Former,通过两阶段预训练连接冻结的视觉 Transformer(ViT)编码器与冻结的大语言模型(LLM),以极少的可训练参数实现领先的视觉语言模型性能。
[Li and others 2024] Li, others. DataComp-LM: In search of the next generation of training sets for language models. https://proceedings.neurips.cc/paper_files/paper/2024/hash/19e4ea30dded58259665db375885e412-Abstract-Datasets_and_Benchmarks_Track.html
DataComp-LM 提供了 240 万亿令牌的 Common Crawl 数据池、标准化训练方案和 53 项评估,用于受控研究抽取、过滤、去重和数据混合。
[Li et al. 2024] Li, Wei, Zhang, Zhang. EAGLE: Speculative sampling requires rethinking feature uncertainty. PMLR. https://proceedings.mlr.press/v235/li24bt.html
EAGLE 在以已采样词元为条件的同时预测次顶层特征,再由目标模型验证由此产生的词元候选。
[Li et al. 2024] Li, Wei, Zhang, Zhang. EAGLE-2: Faster inference of language models with dynamic draft trees. Association for Computational Linguistics. https://aclanthology.org/2024.emnlp-main.422/
EAGLE-2 用上下文感知的扩展与剪枝取代静态候选树,同时保留目标模型验证。
[Li et al. 2024] Li, Huang, Yang, Venkitesh, Locatelli, Ye, Cai, Lewis, Chen. SnapKV: LLM knows what you are looking for before generation. https://proceedings.neurips.cc/paper_files/paper/2024/hash/28ab418242603e0f7323e54185d19bde-Abstract-Conference.html
SnapKV 使用提示末尾的观察窗口,按头选择并聚合重要的提示 KV 位置,再开始生成。
[Li et al. 2024] Li, Cai, Cao, Zhang, Cai, Bai, Jia, Liu, Li, Han. DistriFusion: Distributed parallel inference for high-resolution diffusion models. https://arxiv.org/abs/2402.19481
通过错位图块并行把单张高分辨率扩散样本切分到多张 GPU 上,复用上一步的特征图使各 worker 异步通信,延迟最多降低 6.1 倍且无质量损失。
[Li et al. 2024] Li, Li, Zhang, Mei, Bendersky. Retrieval augmented generation or long-context llms? A comprehensive study and hybrid approach. https://aclanthology.org/2024.emnlp-industry.66/
直接比较发现,在受测场景且资源充分时,长上下文模型平均更强,而 RAG 成本低得多,因此适合混合路由。
[Li et al. 2024] Li, Ding, Fang, Tao. Revisiting catastrophic forgetting in large language model tuning. Association for Computational Linguistics. https://aclanthology.org/2024.findings-emnlp.249/
该研究衡量语言模型微调期间的灾难性遗忘,并从微调损失面的几何形态解释实验结果。
[Li et al. 2025] Li, Wei, Zhang, Zhang. EAGLE-3: Scaling up inference acceleration of large language models via training-time test. https://proceedings.neurips.cc/paper_files/paper/2025/hash/c7b5a35ea98b62512a869c19ea7b03cb-Abstract-Conference.html
EAGLE-3 从目标模型的融合特征训练直接词元草稿器,并评估草稿器规模如何影响推测解码的延迟与吞吐。
[Li et al. 2025] Li, Meng, Lin, Luo, Tian, Ma, Huang, Chua. ScreenSpot-pro: GUI grounding for professional high-resolution computer use. arXiv preprint arXiv:2504.07981. https://arxiv.org/abs/2504.07981
面向高分辨率专业软件的点目标定位基准:此前最佳模型达到 18.9%,级联视觉搜索达到 48.1%。它隔离测试定位能力,而不是端到端任务完成。
[Li et al. 2025] Li, Larsen, Zhang, Zhou, Qin, Yap, Raghavan, Chang, Bowler, Yildiz, others. Apple intelligence foundation language models: Tech report 2025. arXiv preprint arXiv:2507.13575. https://arxiv.org/abs/2507.13575
该报告描述苹果约 30 亿参数端侧模型的架构、量化感知训练、评估与部署适配。
[Li 2026] Li. Do post-training algorithms actually differ? A controlled study across model scales uncovers scale-dependent ranking inversions. arXiv preprint arXiv:2603.19335. https://arxiv.org/abs/2603.19335
一项受控研究发现,训练后算法的排名取决于模型规模和任务;在其 1.5B GSM8K 扫描中,多重比较校正后没有受测 DPO 变体显著优于 DPO。
[Liang et al. 2022] Liang, Zhang, Kwon, Yeung, Zou. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. https://arxiv.org/abs/2203.02053
本文识别并解释了"模态间隔"现象:CLIP 等多模态模型中图像与文本的嵌入向量因深度网络的锥效应与对比学习目标而分布在共享表示空间的不同区域。
[Liang et al. 2023] Liang, Bommasani, Lee, Tsipras, Soylu, Yasunaga, Zhang, Narayanan, Wu, Kumar, Newman, Yuan, Yan, Zhang, Cosgrove, Manning, Ré, Acosta-Navas, Hudson, Zelikman, Durmus, Ladhak, Rong, Ren, Yao, Wang, Santhanam, Orr, Zheng, Yuksekgonul, Suzgun, Kim, Guha, Chatterji, Khattab, Henderson, Huang, Chi, Xie, Santurkar, Ganguli, Hashimoto, Icard, Zhang, Chaudhary, Wang, Li, Mai, Zhang, Koreeda. Holistic evaluation of language models. Transactions on Machine Learning Research. https://openreview.net/forum?id=iO4LZibEqW
HELM 对 30 个大语言模型在 42 个场景下以 7 项指标(准确率、校准、鲁棒性、公平性、偏见、毒性、效率)进行整体评测,揭示单一指标所遮蔽的权衡关系。
[Liang et al. 2023] Liang, Bommasani, Lee, Tsipras, Soylu, Yasunaga, Zhang, others. Holistic evaluation of language models. Transactions on Machine Learning Research. https://arxiv.org/abs/2211.09110
HELM 将语言模型评估定义为场景、适配方法和多项指标的组合,并公开提示词与输出,使比较过程可检查。
[Libovický and Helcl 2018] Libovický, Helcl. End-to-end non-autoregressive neural machine translation with connectionist temporal classification. https://arxiv.org/abs/1811.04719
本文提出一种基于 CTC 的端到端非自回归生成神经机器翻译模型,无需多步骤分离训练即可实现完全并行解码,并在 WMT 英语-罗马尼亚语和英语-德语任务上进行了评估。
[Lie 2023] Lie. Cerebras architecture deep dive: First look inside the hardware/software co-design for deep learning. IEEE Micro. https://doi.org/10.1109/MM.2023.3256384
该文将晶圆级集成视为一种局部性选择,并说明其内存、互连、编译与扩展约束。
[Lieber et al. 2024] Lieber, Lenz, Bata, Cohen, Osin, Dalmedigos, Safahi, Meirom, Belinkov, Shalev-Shwartz, Abend, Alon, Asida, Bergman, Glozman, Gokhman, Manevich, Ratner, Rozen, Shwartz, Zusman, Shoham. Jamba: a hybrid transformer-mamba language model. https://arxiv.org/abs/2403.19887
Jamba 是基于混合 Transformer-Mamba 混合专家架构的大语言模型,支持 256K 词元上下文,在单张 80GB GPU 上实现比 Mixtral-8x7B 高 3 倍的吞吐量。
[Lightman et al. 2024] Lightman, Kosaraju, Burda, Edwards, Baker, Lee, Leike, Schulman, Sutskever, Cobbe. Let's verify step by step. https://arxiv.org/abs/2305.20050
《Let's Verify Step by Step》在固定生成器的 MATH Best-of-N 选择中比较结果监督与过程监督,并发布约 80 万条人工步骤标签的 PRM800K。
[Liguori 2020] Liguori. Automating safe, hands-off deployments. https://d1.awsstatic.com/builderslibrary/pdfs/automating-safe-hands-off-deployments-clareliguori.pdf
AWS 介绍了分阶段部署波次、观察期、单机与单区域暴露,以及逐步扩大区域并行度的方法。
[Lin et al. 2022] Lin, Hilton, Evans. TruthfulQA: Measuring how models mimic human falsehoods. Association for Computational Linguistics. https://aclanthology.org/2022.acl-long.229/
TruthfulQA 用围绕错误信念和误解设计的 817 个问题进行测试,说明在这一对抗性题目群体上,真实性不同于模仿准确率。
[Lin et al. 2024] Lin, Tang, Tang, Yang, Chen, Wang, Xiao, Dang, Gan, Han. AWQ: Activation-aware weight quantization for on-device LLM compression and acceleration. https://proceedings.mlsys.org/paper_files/paper/2024/hash/42a452cbafa9dd64e9ba4aa95cc1ef21-Abstract-Conference.html
AWQ 提出基于激活感知的逐通道权重缩放,实现对大语言模型硬件友好的低比特(W4A16)量化,精度接近混合精度 FP16 且无需反向传播,配套 TinyChat 推理框架在边缘 GPU 上实现超过 3 倍加速。
[Lin et al. 2025] Lin, Zettlemoyer, Ghosh, Yih, Markosyan, Berges, Oğuz. Continual learning via sparse memory finetuning. arXiv preprint arXiv:2510.15103. https://arxiv.org/abs/2510.15103
稀疏记忆微调通过 TF-IDF 排名仅更新记忆增强 LLM 中的特定记忆槽,在持续学习新知识的同时避免全量微调和 LoRA 所引发的灾难性遗忘。
[Lindenbauer et al. 2025] Lindenbauer, Slinko, Felder, Bogomolov, Zharov. The complexity trap: Simple observation masking is as efficient as LLM summarization for agent context management. https://arxiv.org/abs/2508.21433
在 SWE-bench Verified 上,简单的观测遮蔽策略将大语言模型智能体成本减半,同时在五种模型配置下匹配甚至略超基于 LLM 摘要方案的解题率。
[Lindsey et al. 2025] Lindsey, Gurnee, Ameisen, Chen, Pearce, Turner, Citro, Abrahams, Carter, Hosmer, Marcus, Sklar, Templeton, Bricken, McDougall, Cunningham, Henighan, Jermyn, Jones, Persic, Qi, Thompson, Zimmerman, Rivoire, Conerly, Olah, Batson. On the biology of a large language model. https://transformer-circuits.pub/2025/attribution-graphs/biology.html
本文将电路追踪方法应用于 Claude 3.5 Haiku,通过机械可解释性分析揭示该模型在多步推理、诗歌规划、多语言及算术任务中使用的内部机制。
[Linnainmaa 1976] Linnainmaa. Taylor expansion of the accumulated rounding error. BIT Numerical Mathematics. https://papers.baulab.info/papers/also/Linnainmaa-1976.pdf
Linnainmaa 在分析舍入误差如何沿基本运算序列传播时,发展了反向累积方法。
[Linux Kernel Documentation n.d.] Linux Kernel Documentation. Seccomp BPF (secure computing with filters). https://www.kernel.org/doc/html/latest/userspace-api/seccomp_filter.html
Linux 内核文档定义了 seccomp 过滤,并提醒它只能减少暴露的内核接口,不能单独构成完整沙箱。
[Lipman et al. 2023] Lipman, Chen, Ben-Hamu, Nickel, Le. Flow matching for generative modeling. https://arxiv.org/abs/2210.02747
流匹配(FM)是一种无需仿真即可训练连续归一化流的方法,通过回归条件向量场并结合最优传输路径,在 ImageNet 上超越扩散模型的性能。
[Liu et al. 2023] Liu, Zaharia, Abbeel. Ring attention with blockwise transformers for near-infinite context. https://arxiv.org/abs/2310.01889
Ring Attention 将长序列分布到多设备环形拓扑中,通过将键值块通信与分块自注意力计算重叠,实现无近似的近乎无限上下文长度。
[Liu et al. 2023] Liu, Gong, Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. https://arxiv.org/abs/2209.03003
修正流通过学习一个沿直线路径传输分布的常微分方程(ODE),在迭代 reflow 后仅需一步 Euler 积分即可实现高质量图像生成与域迁移。
[Liu et al. 2023] Liu, Li, Wu, Lee. Visual instruction tuning. https://proceedings.neurips.cc/paper_files/paper/2023/hash/6dcf277ea32ce3288914faf369fe6de0-Abstract-Conference.html
LLaVA 通过线性投影将 CLIP 视觉编码器接入大语言模型,并在 GPT-4 生成的多模态指令数据上进行视觉指令微调,构建通用视觉语言模型助手。
[Liu et al. 2023] Liu, Xia, Wang, Zhang. Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation. https://proceedings.neurips.cc/paper_files/paper/2023/hash/43e9d647ccd3e4b7b5baab53f0368686-Abstract-Conference.html
EvalPlus 为代码生成基准补充了更多测试用例,揭示一些程序虽能通过原始测试,却仍然不正确。
[Liu et al. 2023] Liu, Iter, Xu, Wang, Xu, Zhu. G-eval: NLG evaluation using GPT-4 with better human alignment. Association for Computational Linguistics. https://aclanthology.org/2023.emnlp-main.153/
G-Eval 用明确准则和结构化表单评测摘要与对话,在这些任务上提高了与人工评分的对应程度,也指出了偏好模型生成文本的潜在偏差。
[Liu et al. 2024] Liu, Lin, Hewitt, Paranjape, Bevilacqua, Petroni, Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics. https://aclanthology.org/2024.tacl-1.9/
发现长上下文任务表现往往强烈依赖证据位置,位于中间的相关信息通常不如开头或结尾的信息得到可靠利用。
[Liu et al. 2024] Liu, Li, Li, Lee. Improved baselines with visual instruction tuning. https://arxiv.org/abs/2310.03744
LLaVA-1.5 将 LLaVA 的线性视觉语言连接器替换为 MLP 并加入带格式提示的 VQA 数据,仅用 120 万条公开样本即在 11 项多模态基准上达到最优性能。
[Liu et al. 2024] Liu, Li, Li, Li, Zhang, Shen, Lee. LLaVA-NeXT: Improved reasoning, OCR, and world knowledge. https://llava-vl.github.io/blog/2024-01-30-llava-next/
LLaVA-NeXT 在 LLaVA-1.5 基础上将输入图像分辨率提升至 4 倍,通过改进指令微调数据增强 OCR 和推理能力,并在多个基准上达到或超越 Gemini Pro,同时保持原有的简洁设计。
[Liu et al. 2024] Liu, Yuan, Jin, Zhong, Xu, Braverman, Chen, Hu. KIVI: a tuning-free asymmetric 2bit quantization for KV cache. https://proceedings.mlr.press/v235/liu24bz.html
KIVI 研究 KV 缓存的分布,并在所评估的 Llama、Falcon 与 Mistral 部署中对键和值采用不同的二比特粒度。
[Liu et al. 2024] Liu, Hu, Zhang, Guo, Ke, Liu, Wang. Reason for future, act for now: a principled architecture for autonomous LLM agents. PMLR. https://proceedings.mlr.press/v235/liu24ab.html
RAFA 实现滚动时域控制:规划未来动作、执行第一步、保存反馈,再从更新后的状态重新规划。
[Liu et al. 2024] Liu, Blodgett, Cheung, Liao, Olteanu, Xiao. ECBD: Evidence-centered benchmark design for NLP. Association for Computational Linguistics. https://aclanthology.org/2024.acl-long.861/
ECBD 将基准设计组织为能力、内容、适配、组装与证据五个模块,并要求每个模块都得到描述、论证与有效性证据的支持。
[Liu et al. 2024] Liu, Yu, Zhang, Xu, Lei, Lai, Gu, Ding, Men, Yang, others. AgentBench: Evaluating LLMs as agents. https://proceedings.iclr.cc/paper_files/paper/2024/hash/e9df36b21ff4ee211a8b71ee8b7e9f57-Abstract-Conference.html
AgentBench 在八种交互环境中评估语言模型智能体,把多轮决策与环境交互作为评测对象。
[Liu et al. 2024] Liu, Zhao, Iandola, Lai, Tian, Fedorov, Xiong, Chang, Shi, Krishnamoorthi, Lai, Chandra. MobileLLM: Optimizing sub-billion parameter language models for on-device use cases. PMLR. https://proceedings.mlr.press/v235/liu24ce.html
MobileLLM 在十亿参数以下的设备预算内研究模型结构,并报告深窄模型及内存友好共享机制带来的收益。
[Liu et al. 2024] Liu, Wang, Yin, Molchanov, Wang, Cheng, Chen. DoRA: Weight-decomposed low-rank adaptation. https://proceedings.mlr.press/v235/liu24bn.html
DoRA 把预训练权重分解为大小和方向,并对方向分量应用低秩更新。
[Liu et al. 2025] Liu, Su, Yao, Jiang, Lai, Du, Qin, Xu, Lu, Yan, others. Muon is scalable for LLM training. arXiv preprint arXiv:2502.16982. https://arxiv.org/abs/2502.16982
通过加入权重衰减并调整每参数的更新尺度,使 Muon 优化器可扩展到大型 LLM,达到约为 AdamW 两倍的算力效率;并用 5.7T 词元训练出 16B 的 MoE 模型 Moonlight。
[Liu et al. 2025] Liu, Neubig, Xiong. Midtraining bridges pretraining and posttraining distributions. arXiv preprint arXiv:2510.14865. https://arxiv.org/abs/2510.14865
提出中段训练的工作定义,并在 70M 与 160M 参数的受控实验中发现,后期混合通用与专门数据可改善 SFT 后的代码与数学迁移,引入时点与混合权重之间存在交互。
[Liu et al. 2025] Liu, Diao, Lu, Hu, Dong, Choi, Kautz, Dong. ProRL: Prolonged reinforcement learning expands reasoning boundaries in large language models. https://proceedings.neurips.cc/paper_files/paper/2025/hash/1a22b912945fb7c0bdd079e792b31b6f-Abstract-Conference.html
ProRL 通过 KL 散度控制与参考策略重置实现持续强化学习训练,证明 RL 能真正扩展大语言模型推理边界,并训练出在多项基准上超越 DeepSeek-R1-7B 的 Nemotron-Research-Reasoning-Qwen-1.5B 模型。
[Liu et al. 2025] Liu, Chen, Li, Qi, Pang, Du, Lee, Lin. Understanding R1-zero-like training: a critical perspective. https://openreview.net/forum?id=5PAF7PAY2Y
Dr. GRPO 修正了组相对策略优化(GRPO)中人为拉长错误回答长度的优化偏差,在保持推理性能的同时提升了词元效率。
[Lodderstedt et al. 2013] Lodderstedt, Dronia, Scurtescu. OAuth 2.0 token revocation. https://www.rfc-editor.org/rfc/rfc7009.html
RFC 7009 定义了客户端如何通知授权服务器某个令牌已不再需要。
[Lodderstedt et al. 2023] Lodderstedt, Richer, Campbell. OAuth 2.0 rich authorization requests. https://www.rfc-editor.org/rfc/rfc9396.html
Rich Authorization Requests 提供结构化的 authorization_details 参数来表达细粒度权限,但领域语义和执行仍由部署方决定。
[Lodderstedt et al. 2025] Lodderstedt, Bradley, Labunets, Fett. Best current practice for OAuth 2.0 security. https://www.rfc-editor.org/rfc/rfc9700.html
RFC 9700 汇总了抵御令牌泄漏、重放、混淆和其他 OAuth 威胁的部署指南。
[Longpre et al. 2023] Longpre, Mahari, Chen, Obeng-Marnu, Sileo, Brannon, Muennighoff, Khazam, Kabbara, Perisetla, Wu, Shippole, Bollacker, Wu, Villa, Pentland, Hooker. The data provenance initiative: a large scale audit of dataset licensing and attribution in AI. https://arxiv.org/abs/2310.16787
Data Provenance Initiative 追踪了 1800 多个文本数据集的来源、许可、创建者和派生链,并发现大量许可信息缺失或分类错误。
[Longpre et al. 2024] Longpre, Mahari, Lee, Lund, Oderinwale, Brannon, Saxena, Obeng-Marnu, South, Hunter, Klyman, others. Consent in crisis: The rapid decline of the AI data commons. https://arxiv.org/abs/2407.14933
追踪主要网页语料所含域名的 robots 与服务条款信号变化,并将这些信号与法律同意区分开来。
[Longpre et al. 2024] Longpre, Mahari, Chen, Obeng-Marnu, Sileo, Brannon, Muennighoff, Khazam, Kabbara, Perisetla, Wu, Shippole, Bollacker, Wu, Villa, Pentland, Hooker. A large-scale audit of dataset licensing and attribution in AI. Nature Machine Intelligence. https://www.nature.com/articles/s42256-024-00878-8
这项审计追踪了 1800 多个文本数据集,发现数据集聚合平台经常缺少许可和署名信息,或对其错误分类。
[Loshchilov and Hutter 2019] Loshchilov, Hutter. Decoupled weight decay regularization. https://arxiv.org/abs/1711.05101
指出对 Adam 而言 L2 正则化与权重衰减并不等价,并提出将权重衰减从梯度更新中解耦(AdamW),从而改善泛化。
[Lou et al. 2024] Lou, Meng, Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. https://arxiv.org/abs/2310.16834
SEDD 提出评分熵损失,将分数匹配扩展到离散空间,构建的扩散语言模型在困惑度上优于 GPT-2,且仅需 32 倍更少的网络评估次数。
[Lovelace et al. 2026] Lovelace, Belardi, Kundurthy, Sudhakar, Weinberger. Prescriptive scaling laws for data constrained training. arXiv preprint arXiv:2605.01640. https://arxiv.org/abs/2605.01640
在数据受限的扩展律中加入一个显式的过拟合惩罚项,发现更大的模型在重复数据上过拟合得更快,而强权重衰减能大幅降低该惩罚。
[Lu et al. 2022] Lu, Zhou, Bao, Chen, Li, Zhu. DPM-solver: a fast ODE solver for diffusion probabilistic model sampling in around 10 steps. https://arxiv.org/abs/2206.00927
DPM-Solver 是一种无需额外训练的高阶常微分方程(ODE)求解器,专为扩散概率模型采样设计,仅需 10 至 20 次函数评估即可生成高质量样本,相比此前采样器加速 4 至 16 倍。
[Lu et al. 2022] Lu, Bartolo, Moore, Riedel, Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. https://aclanthology.org/2022.acl-long.556/
受控的四样本分类实验表明,示例顺序可以显著改变准确率,而且对一个模型有效的顺序未必能迁移到其他模型。
[Lù et al. 2025] Lù, Kazemnejad, Meade, Patel, Shin, Zambrano, Stańczak, Shaw, Pal, Reddy. AgentRewardBench: Evaluating automatic evaluations of web agent trajectories. arXiv preprint arXiv:2504.08942. https://arxiv.org/abs/2504.08942
AgentRewardBench 用成功、副作用与重复操作的专家标签比较规则式和模型式轨迹评估器,揭示两类裁判互补的失败模式。
[Luo et al. 2023] Luo, Tan, Huang, Li, Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378. https://arxiv.org/abs/2310.04378
潜在一致性模型(LCMs)通过在潜在空间中进行一致性蒸馏,将预训练的 Stable Diffusion 压缩为可在 2 至 4 步内生成 768x768 图像的模型。
[Lyu et al. 2023] Lyu, Havaldar, Stein, Zhang, Rao, Wong, Apidianaki, Callison-Burch. Faithful chain-of-thought reasoning. Association for Computational Linguistics. https://arxiv.org/abs/2301.13379
Faithful CoT 将自然语言问题翻译成符号推理链并交给确定性求解器,使被执行的链条成为最终答案的因果来源。
[Ma et al. 2024] Ma, Fang, Wang. DeepCache: Accelerating diffusion models for free. https://arxiv.org/abs/2312.00858
一种免训练方法,在相邻去噪步之间缓存 U-Net 的高层特征、只重算快速变化的部分,加速 2 至 4 倍且质量损失可忽略。
[Ma et al. 2024] Ma, Wang, Ma, Wang, Wang, Huang, Dong, Wang, Xue, Wei. The era of 1-bit llms: All large language models are in 1.58 bits. arXiv preprint arXiv:2402.17764. https://arxiv.org/abs/2402.17764
BitNet b1.58 研究使用三值权重训练的语言模型,并在其评估设置下报告模型级质量、内存、延迟与算术能耗比较。
[Ma et al. 2025] Ma, Pei, Lausen, Karypis. Understanding silent data corruption in LLM training. Association for Computational Linguistics. https://aclanthology.org/2025.acl-long.996/
存在静默数据损坏的生产节点可能造成轻微数值扰动、损失突增,甚至训练出不同权重。
[Ma et al. 2025] Ma, Pei, Lausen, Karypis. Understanding silent data corruption in LLM training. arXiv preprint arXiv:2502.12340. https://arxiv.org/abs/2502.12340
本文首次对真实生产节点上的静默数据损坏(SDC)对大语言模型(LLM)训练的影响进行实证分析,发现 SDC 会导致模型参数偏移并在微调中引发损失尖峰。
[Mace et al. 2018] Mace, Oertel, Thorne, Chakrabarti. Incident response. O'Reilly Media. https://sre.google/workbook/incident-response/
SRE 工作手册介绍了尽早宣布事故、明确指挥角色、开展沟通与操作,以及保存持久工作记录的方法。
[Maharana et al. 2024] Maharana, Lee, Tulyakov, Bansal, Barbieri, Fang. Evaluating very long-term conversational memory of LLM agents. arXiv preprint arXiv:2402.17753. https://arxiv.org/abs/2402.17753
横跨多次会话、数百轮的超长期对话,配问答与事件摘要任务:站在记忆系统之争中心的那个基准。
[Maini et al. 2024] Maini, Seto, Bai, Grangier, Zhang, Jaitly. Rephrasing the web: a recipe for compute and data-efficient language modeling. Association for Computational Linguistics. https://arxiv.org/abs/2401.16380
WRAP 方法用指令微调的 LLM 将嘈杂网页文本改写为多种风格的合成数据,与在原始网页语料上预训练大语言模型相比,可将所需计算量减少约 3 倍、数据量减少约 5 倍。
[Maini et al. 2024] Maini, Feng, Schwarzschild, Lipton, Kolter. TOFU: a task of fictitious unlearning for LLMs. https://arxiv.org/abs/2401.06121
TOFU 使用合成作者档案,通过忘记集、保留集以及多项行为与效用指标评估大语言模型遗忘。
[Maini and others 2025] Maini, others. BeyondWeb: Lessons from scaling synthetic data for trillion-scale pretraining. arXiv preprint arXiv:2508.10975. https://arxiv.org/abs/2508.10975
BeyondWeb 是一个通过有针对性的文档改写生成合成预训练数据的框架,在 14 项基准上平均超越 Cosmopedia 最高 5.1 个百分点,训练速度比原始网络数据快 7.7 倍。
[Malkov and Yashunin 2020] Malkov, Yashunin. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://arxiv.org/abs/1603.09320
HNSW 提出了一种基于多层邻近图的近似最近邻搜索索引,实现对数复杂度的分层可导航小世界图结构。
[Manakul et al. 2023] Manakul, Liusie, Gales. SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models. Association for Computational Linguistics. https://aclanthology.org/2023.emnlp-main.557/
SelfCheckGPT 把黑盒模型重复采样之间的不一致作为句级预警信号;样本彼此一致并不构成真实证据。
[Marchisio et al. 2024] Marchisio, Dash, Chen, Aumiller, Üstün, Hooker, Ruder. How does quantization affect multilingual LLMs?. Association for Computational Linguistics. https://aclanthology.org/2024.findings-emnlp.935/
在论文评估的多语言模型中,不同语言与任务受量化影响不均,自动指标低估了人类评估者观察到的部分变化。
[Markov et al. 2023] Markov, Zhang, Agarwal, Eloundou, Lee, Adler, Jiang, Weng. A holistic approach to undesired content detection in the real world. AAAI 2023. https://arxiv.org/abs/2208.03274
OpenAI 提出一套面向真实流量的内容审核完整流程,结合内容分类体系设计、主动学习、质量管控标注与合成数据,用于检测性相关、仇恨、暴力、自我伤害和骚扰等不良内容。
[Martínez-Olguín 2026] Martínez-Olguín. Bartz v. anthropic PBC, order granting final settlement approval and final judgment, no. 4:24-cv-05417. United States District Court for the Northern District of California. https://law.justia.com/cases/federal/district-courts/california/candce/4:2024cv05417/434709/680/
最终批准 Bartz 集体诉讼协商和解,但不使其成为实体裁判或未来许可。
[Mazeika et al. 2024] Mazeika, Phan, Yin, Zou, Wang, Mu, Sakhaee, Li, Basart, Li, Forsyth, Hendrycks. HarmBench: a standardized evaluation framework for automated red teaming and robust refusal. https://proceedings.mlr.press/v235/mazeika24a.html
HarmBench 是一个标准化基准,包含 510 个有害行为及评测流水线,对 33 个大语言模型上的 18 种自动化红队方法进行大规模比较,以支持攻防协同开发。
[McCandlish et al. 2018] McCandlish, Kaplan, Amodei, OpenAI Dota Team. An empirical model of large-batch training. arXiv preprint arXiv:1812.06162. https://arxiv.org/abs/1812.06162
提出用梯度噪声尺度来预测临界批大小,超过该值后增大批大小不再减少训练步数,只是浪费算力。
[McCulloch and Pitts 1943] McCulloch, Pitts. A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics.
McCulloch 与 Pitts 提出首个神经元的数学模型,证明阈值逻辑单元构成的网络可以计算任意逻辑函数。
[McNemar 1947] McNemar. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika. https://doi.org/10.1007/BF02295996
McNemar 检验通过关注两个系统在同一样本上发生分歧的项目来比较配对二元结果,适合许多基准 A/B 比较。
[Meng et al. 2022] Meng, Bau, Andonian, Belinkov. Locating and editing factual associations in GPT. https://arxiv.org/abs/2202.05262
ROME 利用因果中介分析将 GPT 中的事实关联定位到中层前馈网络(FFN)模块,并提出秩一权重编辑方法以精确修改特定事实。
[Meng et al. 2023] Meng, Sharma, Andonian, Belinkov, Bau. Mass-editing memory in a transformer. https://arxiv.org/abs/2210.07229
MEMIT 通过将参数更新分摊到一组关键 MLP 层,将大语言模型的知识编辑扩展至数千条事实的同步批量更新。
[Meng et al. 2024] Meng, Xia, Chen. SimPO: Simple preference optimization with a reference-free reward. https://arxiv.org/abs/2405.14734
SimPO 用长度归一化的平均对数概率替代 DPO 的参考模型奖励,并引入目标奖励间隔,无需参考模型,在 Arena-Hard 上比 DPO 最多提升 7.5 个百分点。
[Meta 2026] Meta. Private processing for WhatsApp overview: Technical white paper and security guide. https://ai.meta.com/static-resource/private-processing-technical-whitepaper
记录 Meta 的威胁模型,以及 SEV-SNP、H100 机密模式、RA-TLS、中继、透明度、撤销与受限可观测性的组合。
[Meta AI 2025] Meta AI. The llama 4 herd: The beginning of a new era of natively multimodal AI innovation. https://ai.meta.com/blog/llama-4-multimodal-intelligence/
Meta 发布了 Llama 4 多模态混合专家模型系列,并介绍各个公开版本、上下文支持、训练和评估。
[METR 2026] METR. Task-completion time horizons of frontier AI models. https://metr.org/time-horizons/
METR 估计前沿模型在给定成功率下能够完成的软件任务时长,并持续追踪这一时间跨度的变化。
[METR 2026] METR. Time horizon 1.1. https://metr.org/blog/2026-1-29-time-horizon-1-1/
METR 发布 Time Horizon 1.1,使用更多任务和新评测基础设施更新 AI 智能体任务完成时间跨度的基准估计。
[METR 2026] METR. Frontier AI risk report: February–march 2026. https://metr.org/blog/2026-05-19-frontier-risk-report/
METR 报告称,一个共享的内部模型已接近 Time Horizon 1.1 的可测上限,并警告该测试套件无法可靠区分 16 小时以上的任务跨度。
[Meunier and Major 2026] Meunier, Major. HTTP message signatures for automated traffic. https://datatracker.ietf.org/doc/draft-meunier-webbotauth-httpsig-protocol/
该草案结合了智能体标识符、密钥发现和 RFC 9421 签名,但仍在制定中,并非已发布的互联网标准。
[Mialon et al. 2024] Mialon, Fourrier, Swift, Wolf, LeCun, Scialom. GAIA: a benchmark for general AI assistants. https://proceedings.iclr.cc/paper_files/paper/2024/hash/25ae35b5b1738d80f1f03a8713e405ec-Abstract-Conference.html
GAIA 用人工编写的问题评估助手;回答前可能需要浏览网页、处理文件、编写代码、理解多模态内容并调用工具,最终输出可核查的短答案。
[Michaud et al. 2023] Michaud, Liu, Girit, Tegmark. The quantization model of neural scaling. https://arxiv.org/abs/2303.13506
提出技能是离散的量子,按使用频率递减的顺序被学到;当这些频率服从幂律时,按序学习便产生平滑的幂律损失,并把涌现重新解释为量子逐个开启。
[Micikevicius et al. 2017] Micikevicius, Narang, Alben, Diamos, Elsen, Garcia, Ginsburg, Houston, Kuchaiev, Venkatesh, Wu. Mixed precision training. https://arxiv.org/abs/1710.03740
本文提出混合精度训练方法,结合 FP16 存储与计算、FP32 主权重副本、损失缩放和 FP32 累加,在不损失精度的情况下将训练显存减少约一半。
[Micikevicius et al. 2022] Micikevicius, Stosic, Burgess, Cornea, Dubey, Grisenthwaite, Ha, Heinecke, Judd, Kamalu, Mellempudi, Oberman, Shoeybi, Siu, Wu. FP8 formats for deep learning. https://arxiv.org/abs/2209.05433
本文规定 E4M3 与 E5M2 两种 FP8 交换格式,并在多类神经网络上评估相应训练方案。
[Micron Technology 2026] Micron Technology. HBM4 high-bandwidth memory. https://www.micron.com/products/memory/hbm/hbm4
Micron 的 HBM4 页面分别列出接口宽度、单引脚传输速率、单堆栈带宽、密度与封装选择。
[Microsoft 2024] Microsoft. MLOps and GenAIOps for AI workloads on azure. https://learn.microsoft.com/en-us/azure/well-architected/ai/mlops-genaiops
Microsoft 的工作负载指南把 AI 运营视为非确定系统的生命周期管理,覆盖监控、漂移、部署、治理和自动化。
[Microsoft 2026] Microsoft. Durable orchestrations overview. https://learn.microsoft.com/en-us/azure/durable-task/common/durable-task-orchestrations
Durable Task 从只追加历史重建工作流状态,并要求编排代码具有确定性,使重放产生与已记录事件兼容的命令。
[Microsoft 2026] Microsoft. Orchestration versioning: Safe deployments for durable orchestrations. https://learn.microsoft.com/en-us/azure/durable-task/common/durable-orchestration-versioning
修改持久工作流代码可能破坏确定性重放,因此长时间运行的执行需要显式版本分配与迁移。
[Microsoft 2026] Microsoft. Add a filter to a vector query in azure AI search. https://learn.microsoft.com/en-us/azure/search/vector-search-filters
Azure AI Search 文档区分了向量查询的预过滤、后过滤和严格后过滤,并解释它们在召回率上的取舍。
[Milakov and Gimelshein 2018] Milakov, Gimelshein. Online normalizer calculation for softmax. arXiv preprint arXiv:1805.02867. https://arxiv.org/abs/1805.02867
单趟计算的 softmax:边扫描边维护运行中的最大值与归一化项,这个四页纸的技巧几年后使分块精确注意力成为可能。
[Milanta and Beurer-Kellner 2025] Milanta, Beurer-Kellner. GitHub MCP exploited: Accessing private repositories via MCP. https://invariantlabs.ai/blog/mcp-github-vulnerability
这一概念验证展示了恶意议题内容如何诱使连接范围过宽的智能体执行获准的私密读取和公开写入。
[Miller 2024] Miller. Adding error bars to evals: a statistical approach to language model evaluations. arXiv preprint arXiv:2411.00640. https://arxiv.org/abs/2411.00640
Anthropic 把 LLM 评测当作统计推断来处理:报告标准误,题目成组时改用聚类标准误,模型比较用配对差值分析,并用功效分析提前规划样本量。
[Min et al. 2023] Min, Krishna, Lyu, Lewis, Yih, Koh, Iyyer, Zettlemoyer, Hajishirzi. FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. Association for Computational Linguistics. https://aclanthology.org/2023.emnlp-main.741/
FActScore 将长文本生成拆成原子事实,并测量其中被指定知识来源支持的比例;它估计的是事实精确率,而不是召回率。
[MiniMax 2025] MiniMax. MiniMax-01: Scaling foundation models with lightning attention. arXiv preprint arXiv:2501.08313. https://arxiv.org/abs/2501.08313
MiniMax-01 在一个 456B 参数的 MoE 模型里交错排布 lightning attention(一种线性注意力)与 softmax 注意力,以前沿质量支撑百万词元上下文。
[Minixhofer et al. 2024] Minixhofer, Ponti, Vulić. Zero-shot tokenizer transfer. https://proceedings.neurips.cc/paper_files/paper/2024/hash/532ce4fcf853023c4cf2ac38cbc5d002-Abstract-Conference.html
ZeTT 为替换后的分词器预测嵌入,并用少量继续训练缩小剩余性能差距。
[Minkenberg et al. 2021] Minkenberg, Krishnaswamy, Zilkie, Nelson. Co-packaged datacenter optics: Opportunities and challenges. IET Optoelectronics. https://doi.org/10.1049/ote2.12020
该综述解释将光学器件移入交换芯片封装如何同时改变功耗、密度、测试、维修与可靠性。
[Mitchell et al. 2019] Mitchell, Wu, Zaldivar, Barnes, Vasserman, Hutchinson, Spitzer, Raji, Gebru. Model cards for model reporting. Association for Computing Machinery. https://doi.org/10.1145/3287560.3287596
模型卡记录预期用途、评测过程和分组性能,支持关于模型部署的透明决策。
[Model Context Protocol 2025] Model Context Protocol. Tools. https://modelcontextprotocol.io/specification/2025-11-25/server/tools
MCP 工具规范定义工具发现、JSON Schema 输入、结构化输出、结果验证以及对工具调用的人类控制。
[Model Context Protocol 2025] Model Context Protocol. Security best practices. https://modelcontextprotocol.io/specification/2025-11-25/basic/security_best_practices
MCP 安全指南涵盖令牌受众绑定、混淆代理风险、令牌透传、会话劫持和最小权限范围。
[Model Context Protocol 2025] Model Context Protocol. Transports: Mcp-Session-Id and Last-Event-ID. https://modelcontextprotocol.io/specification/2025-11-25/basic/transports
MCP(模型上下文协议)规范的传输层定义了两种 JSON-RPC 消息交换机制:用于子进程通信的 stdio 和用于网络化 MCP 服务器的可流式 HTTP(取代 SSE)。
[Model Context Protocol 2025] Model Context Protocol. Key changes: MCP specification 2025-06-18. https://modelcontextprotocol.io/specification/2025-06-18/changelog
MCP 2025-06-18 修订版将服务器归类为 OAuth 资源服务器,要求客户端实现 RFC 8707 资源指示符,移除 JSON-RPC 批处理,并新增结构化工具输出与征询(elicitation)能力。
[Model Context Protocol 2026] Model Context Protocol. Authorization. https://modelcontextprotocol.io/specification/2026-07-28/basic/authorization
MCP 授权配置统一了服务器发现和受众绑定的 OAuth 访问,但不规定应用层工具策略。
[Model Context Protocol 2026] Model Context Protocol. Model context protocol 2026-07-28: Tools. https://modelcontextprotocol.io/specification/2026-07-28/server/tools
MCP 工具规范定义了线路边界上的工具发现、调用、模式、显式状态句柄、结果验证和安全注意事项。
[Model Context Protocol Contributors 2025] Model Context Protocol Contributors. Model context protocol specification, revision 2025-11-25. https://modelcontextprotocol.io/specification/2025-11-25/architecture
MCP 规范定义了宿主、客户端和服务器的职责、能力协商,以及资源、提示和工具这三类独立原语。
[Model Context Protocol Contributors 2026] Model Context Protocol Contributors. Model context protocol specification, revision 2026-07-28. https://modelcontextprotocol.io/specification/2026-07-28
当前 MCP 规范定义了面向工具、资源和提示的无状态宿主、客户端、服务器协议。身份元数据由参与方自行声明,协议本身不强制执行同意、授权或安全行为。
[Model Context Protocol Contributors 2026] Model Context Protocol Contributors. MCP security best practices. https://modelcontextprotocol.io/docs/2026-07-28/tutorials/security/security_best_practices
MCP 安全指南禁止令牌透传,要求受保护的 HTTP 资源验证受众和资源,并建议本地服务器采用最小权限与沙箱。
[Mohan et al. 2021] Mohan, Phanishayee, Chidambaram. CheckFreq: Frequent, fine-grained DNN checkpointing. https://www.usenix.org/conference/fast21/presentation/mohan
CheckFreq 分析检查点开销,调整保存频率,并将检查点工作流水线化,同时保持所评估的数据加载器不变量。
[Moreau and Missier 2013] Moreau, Missier. PROV-DM: The PROV data model. https://www.w3.org/TR/prov-dm/
PROV-DM 用实体、活动、智能体及其关系表示来源信息,把生成产物连接到其来源和转换过程。
[Motamed et al. 2026] Motamed, Culp, others. Do generative video models understand physical principles?. https://arxiv.org/abs/2501.09038
Physics-IQ 是一个涵盖五个物理领域的真实视频基准,包含 396 段视频,揭示当前生成式视频模型得分最高仅 29.5%,表明视觉逼真度并不意味着具备物理理解能力。
[Moura and Ullrich 2021] Moura, Ullrich. The Lean 4 theorem prover and programming language. Springer. https://link.springer.com/chapter/10.1007/978-3-030-79876-5_37
Lean 4 在同一实现中结合了依赖类型论、定理证明器、函数式语言和元编程系统。
[Muennighoff et al. 2023] Muennighoff, Rush, Barak, Le Scao, Piktus, Tazi, Pyysalo, Wolf, Raffel. Scaling data-constrained language models. https://arxiv.org/abs/2305.16264
研究数据受限下的训练,发现重复数据至多约四轮几乎与新数据等效,并据此提出一条刻画重复词元与多余参数价值递减的扩展律。
[Muennighoff et al. 2023] Muennighoff, Tazi, Magne, Reimers. MTEB: Massive text embedding benchmark. https://aclanthology.org/2023.eacl-main.148/
MTEB 提出一个涵盖 8 项嵌入任务、58 个数据集与 112 种语言的基准,发现没有单一文本嵌入方法能在所有任务上保持领先。
[Muennighoff et al. 2025] Muennighoff, Yang, Shi, Li, Fei-Fei, Hajishirzi, Zettlemoyer, Liang, Candes, Hashimoto. s1: Simple test-time scaling. Association for Computational Linguistics. https://aclanthology.org/2025.emnlp-main.1025/
s1 用 1,000 条经过筛选的 Gemini 生成轨迹微调 Qwen2.5-32B-Instruct,并结合预算强制;其自身评分器判断训练生成中有 53.6% 正确。
[Murray et al. 2021] Murray, Simsa, Klimovic, Indyk. tf.data: a machine learning data processing framework. Proceedings of the VLDB Endowment. https://www.vldb.org/pvldb/vol14/p2945-klimovic.pdf
tf.data 把输入加载表示为可组合的数据流,其并行、缓存、预取和确定性选择都会影响端到端训练。
[Nakano et al. 2021] Nakano, Hilton, Balaji, Wu, Ouyang, Kim, Hesse, Jain, Kosaraju, Saunders, others. WebGPT: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332. https://arxiv.org/abs/2112.09332
WebGPT 用示范与人类反馈训练语言模型浏览网页;其最强配置使用拒绝采样,而非在线策略梯度更新。
[Narayanan et al. 2019] Narayanan, Harlap, Phanishayee, Seshadri, Devanur, Ganger, Gibbons, Zaharia. PipeDream: Generalized pipeline parallelism for DNN training. https://doi.org/10.1145/3341301.3359646
PipeDream 将流水线并行与数据并行结合,使 GPU 间通信量降低最多 95%,DNN 训练的达标时间比数据并行快最多 5 倍。
[Narayanan et al. 2021] Narayanan, Shoeybi, Casper, LeGresley, Patwary, Korthikanti, Vainbrand, Kashinkunti, Bernauer, Catanzaro, Phanishayee, Zaharia. Efficient large-scale language model training on GPU clusters using megatron-LM. https://arxiv.org/abs/2104.04473
本文组合张量、流水线与数据并行,并分析大规模 Transformer 训练的流水线调度。
[National Academies of Sciences, Engineering, and Medicine 2019] National Academies of Sciences, Engineering, and Medicine. Reproducibility and replicability in science. The National Academies Press. https://doi.org/10.17226/25303
该报告区分使用相同数据和方法复现结果与使用新数据获得一致结果,并把重复验证视为具有适用范围的证据,而非二元印章。
[National Telecommunications and Information Administration 2024] National Telecommunications and Information Administration. Dual-use foundation models with widely available model weights report. https://www.ntia.gov/programs-and-initiatives/artificial-intelligence/open-model-weights-report
NTIA 审查广泛可获取模型权重的收益与风险,并建议继续收集证据,而不是在报告发布时实施普遍限制。
[Naumov et al. 2019] Naumov, Mudigere, Shi, Huang, Sundaraman, Park, Wang, Gupta, Wu, Azzolini, Dzhulgakov, Mallevich, Cherniavskii, Lu, Krishnamoorthi, Yu, Kondratenko, Pereira, Chen, Chen, Rao, Jia, Xiong, Smelyanskiy. Deep learning recommendation model for personalization and recommendation systems. arXiv preprint arXiv:1906.00091. https://arxiv.org/abs/1906.00091
Meta 的参考推荐架构把类别特征嵌入表与稠密交互和多层感知机组件结合起来,需要同时使用模型并行与数据并行。
[Nelson et al. 2025] Nelson, Rekhi, Souppaya, Scarfone. Incident response recommendations and considerations for cybersecurity risk management: a CSF 2.0 community profile. https://doi.org/10.6028/NIST.SP.800-61r3
NIST SP 800-61 第 3 版将事件响应的准备、检测、处置和恢复纳入《网络安全框架 2.0》。
[New York State Legislature 2026] New York State Legislature. S.8828: Chapter amendment to the RAISE act. https://www.nysenate.gov/legislation/bills/2025/S8828
纽约州 S.8828 法案修订了 RAISE 法案对前沿 AI 开发者的适用范围、定义、报告义务和实施条款。
[Nichol and Dhariwal 2021] Nichol, Dhariwal. Improved denoising diffusion probabilistic models. https://arxiv.org/abs/2102.09672
本文表明,通过学习逆过程方差和混合训练目标,DDPM 可达到有竞争力的对数似然,同时将高质量采样所需的前向传播次数减少约 20 倍。
[Nickolls et al. 2008] Nickolls, Buck, Garland, Skadron. Scalable parallel programming with CUDA. ACM Queue. https://queue.acm.org/detail.cfm?id=1365500
CUDA 执行模型的正典描述,线程网格、线程块、warp 与 SIMT,由其架构师在发布次年写就。
[Nie et al. 2025] Nie, Zhu, You, Zhang, Ou, Hu, Zhou, Lin, Wen, Li. Large language diffusion models. arXiv preprint arXiv:2502.09992. https://arxiv.org/abs/2502.09992
LLaDA 提出从头训练的掩码扩散大语言模型,通过预训练与监督微调证明可扩展性、上下文学习和指令遵循能力无需自回归建模即可涌现。
[North American Electric Reliability Corporation 2026] North American Electric Reliability Corporation. Large loads action plan. https://prod.nerc.com/initiatives/large-loads-action-plan
北美电力可靠性公司行动计划列出了预测、研究、并网和运营快速增长的大型负载时需要开展的可靠性工作。
[Northcutt et al. 2021] Northcutt, Athalye, Mueller. Pervasive label errors in test sets destabilize machine learning benchmarks. NeurIPS Datasets and Benchmarks Track. https://arxiv.org/abs/2103.14749
该论文发现 10 个主流机器学习基准测试集中平均存在 3.3% 的标注错误,这些错误可颠覆模型排名并破坏评测的稳定性。
[Nottingham et al. 2023] Nottingham, Wilde, Dalal. Problem details for HTTP apis. https://www.rfc-editor.org/rfc/rfc9457.html
RFC 9457 定义了可复用的机器可读错误格式,同时提醒客户端不要从面向人类的详情文本中解析程序逻辑。
[Noukhovitch et al. 2025] Noukhovitch, Huang, Xhonneux, Hosseini, Agarwal, Courville. Asynchronous RLHF: Faster and more efficient off-policy RL for language models. https://proceedings.iclr.cc/paper_files/paper/2025/hash/0b99315234cc95e6ef281f9155b68832-Abstract-Conference.html
异步 RLHF 在独立资源上重叠生成与学习,并研究策略陈旧度如何影响训练效率和质量。
[Novikov et al. 2025] Novikov, Vu, Eisenberger, Dupont, Huang, Wagner, Shirobokov, Kozlovskii, Ruiz, Mehrabian, Kumar, See, Chaudhuri, Holland, Davies, Nowozin, Kohli, Balog. AlphaEvolve: a coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131. https://arxiv.org/abs/2506.13131
AlphaEvolve 根据人类提供的评估代码演化程序;入选结果仍需按应用接受留出测试、专家、硬件或部署检查。
[Noy and Zhang 2023] Noy, Zhang. Experimental evidence on the productivity effects of generative artificial intelligence. Science. https://doi.org/10.1126/science.adh2586
在一项涉及 453 名专业人士的实验中,ChatGPT 使用权限缩短了短期激励写作任务的完成时间,并提高了评分质量。
[NVIDIA 2024] NVIDIA. NCCL: NVIDIA collective communications library. https://github.com/NVIDIA/nccl
NCCL 实现了感知拓扑的集合通信原语,用于在单机或跨节点的 NVIDIA GPU 之间移动张量。
[NVIDIA 2025] NVIDIA. Nemotron-h: a family of accurate and efficient hybrid mamba-transformer models. arXiv preprint arXiv:2504.03624. https://arxiv.org/abs/2504.03624
Nemotron-H 在 8B 与 56B 模型里把大部分注意力层换成 Mamba 层,在保持 Transformer 精度的同时把长上下文推理加速最多 3 倍。
[NVIDIA 2025] NVIDIA. Pretraining large language models with NVFP4. arXiv preprint arXiv:2509.25149. https://arxiv.org/abs/2509.25149
用 NVFP4 这一 4 位微缩放格式在 10 万亿词元上训练一个 12B 模型,借助随机 Hadamard 变换、二维缩放与随机舍入,达到与 fp8 基线相当的效果。
[NVIDIA 2025] NVIDIA. NVIDIA secure AI with blackwell and hopper gpus. https://docs.nvidia.com/nvidia-secure-ai-with-blackwell-and-hopper-gpus-whitepaper.pdf
记录 Hopper 与 Blackwell 不同的保护模式、证明声明、互连覆盖范围、拓扑假设与剩余可信组件。
[NVIDIA 2026] NVIDIA. CUDA programming guide. https://docs.nvidia.com/cuda/cuda-programming-guide/
CUDA 指南定义了主机与设备模型、SIMT 执行、内存空间及架构特定的加速器功能。
[NVIDIA 2026] NVIDIA. NVL72 AI Factory: System hardware and components. https://docs.nvidia.com/enterprise-reference-architectures/nvl72-ai-factory/latest/components.html
该参考架构记录了一个 72 GPU 的机架级 NVLink 域,说明纵向扩展并不必然局限于单台服务器。
[NVIDIA 2026] NVIDIA. NCCL user guide: Collective operations. https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/usage/collectives.html
NCCL 指南定义了 GPU 集合通信语义,包括全归约、全收集、归约散播、广播与全交换。
[NVIDIA 2026] NVIDIA. GPUDirect RDMA documentation. https://docs.nvidia.com/cuda/gpudirect-rdma/
该指南说明对 GPU 内存进行对等设备直接访问所需的注册、拓扑、驱动与同步条件。
[NVIDIA 2026] NVIDIA. CUDA programming guide. https://docs.nvidia.com/cuda/cuda-programming-guide/02-basics/writing-cuda-kernels.html
CUDA 指南定义了主机与设备模型、SIMT 执行、内存空间、同步和 GPU 内核结构。
[NVIDIA 2026] NVIDIA. NVIDIA CUDA compiler driver NVCC. https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html
官方编译器驱动契约说明 CUDA 源码、PTX、cubin 映像、胖二进制、主机代码与运行时加载如何衔接。
[NVIDIA 2026] NVIDIA. NVIDIA DGX GB200 NVL72 user guide. https://docs.nvidia.com/dgx/dgxgb200-user-guide/
操作指南界定了一套已部署 72-GPU 机架系统的物理组件与网络边界。
[NVIDIA 2026] NVIDIA. Multi-node NVLink systems tuning guide. https://docs.nvidia.com/multi-node-nvlink-systems/multi-node-tuning-guide/overview.html
该指南解释 NVLink 跨越更多 GPU 后仍需处理的拓扑与软件问题。
[NVIDIA 2026] NVIDIA. Triton inference server: Model repository. https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/user_guide/model_repository.html
Triton 文档说明了具体的可移植性边界:TensorRT 计划依赖 CUDA 计算能力,ONNX 支持取决于所捆绑的运行时和算子,TorchScript 兼容性也可能随 PyTorch 版本变化。
[Nye et al. 2021] Nye, Andreassen, Gur-Ari, Michalewski, Austin, Bieber, Dohan, Lewkowycz, Bosma, Luan, Sutton, Odena. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114. https://arxiv.org/abs/2112.00114
草稿纸训练要求模型在答案前输出中间计算过程,并在多步算法任务上评估这种方法。
[Nygard 2018] Nygard. Release it! Design and deploy production-ready software. Pragmatic Bookshelf.
Nygard 总结了生产软件的稳定性模式,包括断路器、隔板与超时。
[Office of Management and Budget 2025] Office of Management and Budget. M-25-21: Accelerating federal use of AI through innovation, governance, and public trust. https://www.whitehouse.gov/wp-content/uploads/2025/02/M-25-21-Accelerating-Federal-Use-of-AI-through-Innovation-Governance-and-Public-Trust.pdf
美国行政管理和预算局 M-25-21 备忘录为联邦机构采用 AI 规定了治理、清单、风险管理和公众信任要求。
[Office of Management and Budget 2025] Office of Management and Budget. M-25-22: Driving efficient acquisition of artificial intelligence in government. https://www.whitehouse.gov/wp-content/uploads/2025/02/M-25-22-Driving-Efficient-Acquisition-of-Artificial-Intelligence-in-Government.pdf
美国行政管理和预算局 M-25-22 备忘录要求联邦 AI 采购重视竞争、互操作性、性能评估、数据权利和生命周期风险管理。
[OLMo Team 2025] OLMo Team. 2 olmo 2 furious. https://arxiv.org/abs/2501.00656
OLMo 2 记录并发布模型权重、数据混合、训练与评估代码、配方、日志和中间检查点,以支持研究并尝试复现其开发过程。
[Olsson et al. 2022] Olsson, Elhage, Nanda, Joseph, DasSarma, Henighan, Mann, Askell, Bai, Chen, Conerly, Drain, Ganguli, Hatfield-Dodds, Hernandez, Johnston, Jones, Kernion, Lovitt, Ndousse, Amodei, Brown, Clark, Kaplan, McCandlish, Olah. In-context learning and induction heads. https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html
本文认定归纳头(induction heads)是 Transformer 中由两个注意力头构成的电路,是各规模模型中上下文学习的主要机制来源。
[Olston et al. 2017] Olston, Fiedel, Gorovoy, Harmsen, Lao, Li, Rajashekhar, Ramesh, Soyke. TensorFlow-serving: Flexible, high-performance ML serving. arXiv preprint arXiv:1712.06139. https://arxiv.org/abs/1712.06139
TensorFlow Serving 介绍了一套包含版本化产物、可插拔来源、批处理和在线请求处理的模型服务架构。
[Ong et al. 2024] Ong, Almahairi, Wu, Chiang, Wu, Gonzalez, Kadous, Stoica. RouteLLM: Learning to route llms with preference data. arXiv preprint arXiv:2406.18665. https://arxiv.org/abs/2406.18665
RouteLLM 将学习得到的路由器作为质量与成本权衡策略进行评估,说明路由行为必须实测而不能预设。
[ONNX Project 2026] ONNX Project. ONNX intermediate representation specification. https://onnx.ai/onnx/repo-docs/IR.html
ONNX 中间表示规定了包含算子、函数、类型和张量初始值的版本化计算图;大型张量可以存为外部数据。
[Oord et al. 2018] Oord, Li, Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748. https://arxiv.org/abs/1807.03748
CPC 通过在潜在空间中用对比损失(InfoNCE)预测未来观测,从无标注数据中学习表征,在语音、图像、文本和强化学习四个领域均取得良好效果。
[Open Container Initiative 2025] Open Container Initiative. OCI distribution specification. https://github.com/opencontainers/distribution-spec/blob/main/spec.md
OCI 分发协议把可变标签与按摘要寻址的清单和二进制对象分开,并用描述符绑定媒体类型、大小和内容摘要。
[Open Source Initiative 2024] Open Source Initiative. The open source AI definition 1.0. https://opensource.org/ai/open-source-ai-definition
OSI 定义把开源 AI 视为需要使用、研究、修改和分享四项自由,并要求数据说明、代码与参数以便于修改的首选形式提供。
[Open X-Embodiment Collaboration 2023] Open X-Embodiment Collaboration. Open x-embodiment: Robotic learning datasets and RT-x models. arXiv preprint arXiv:2310.08864. https://arxiv.org/abs/2310.08864
Open X-Embodiment 汇集了来自 21 个机构共 22 种机器人形态的超百万条轨迹数据集,并训练 RT-X 模型,证明跨机器人平台的正迁移。
[OpenAI 2023] OpenAI. GPT-4 technical report. https://cdn.openai.com/papers/gpt-4.pdf
OpenAI 报告 GPT-4 的基准与安全评估,同时明确不披露架构、模型规模、硬件、训练算力、数据集构建和训练方法。
[OpenAI 2024] OpenAI. Hello GPT-4o. https://openai.com/index/hello-gpt-4o/
OpenAI 将 GPT-4o 介绍为统一处理文本、图像和音频输入输出的模型,其音频响应延迟低于串联式语音流水线。
[OpenAI 2024] OpenAI. Learning to reason with llms. https://openai.com/index/learning-to-reason-with-llms/
OpenAI 介绍了如何用强化学习训练 o1 推理,并报告其性能会随训练计算量和测试时计算量增加而提高。
[OpenAI 2024] OpenAI. Introducing SWE-bench Verified. https://openai.com/index/introducing-swe-bench-verified/
专业软件开发者按问题描述清晰度、测试适切性与可解性审查 SWE-bench,形成 500 项子集与容器化运行框架。
[OpenAI 2025] OpenAI. Sycophancy in GPT-4o: What happened and what we're doing about it. https://openai.com/index/sycophancy-in-gpt-4o/
对谄媚模型更新的事后复盘,其中承认用户记忆在部分情况下加重了谄媚的影响。
[OpenAI 2025] OpenAI. 4o image generation system card addendum. https://cdn.openai.com/11998be9-5319-4302-bfbf-1167e093f1fb/Native_Image_Generation_System_Card.pdf
OpenAI 将 GPT-4o 原生图像生成归类为自回归模型,但没有披露其内部表示或解码器设计。
[OpenAI 2025] OpenAI. Model spec. https://model-spec.openai.com/2025-12-18.html
Model Spec 定义模型目标行为,并规定用来解决指令冲突的权限层级。
[OpenAI 2025] OpenAI. Preparedness framework, version 2. https://cdn.openai.com/pdf/18a02b5d-6b67-4cec-ab64-68cdfbddebcd/preparedness-framework-v2.pdf
列出部署前受追踪的能力类别,并为每一类定义 High 与 Critical 两档阈值。能力报告与防护措施报告一并提交安全顾问组,由它给出建议,最终的部署决定由 OpenAI 的领导层作出。
[OpenAI 2025] OpenAI. Introducing gpt-oss. https://github.com/openai/gpt-oss
该发布提供 Apache 2.0 权重、模型文档、分词器材料和参考推理实现,但不提供训练语料或完整训练流水线。
[OpenAI 2026] OpenAI. Memory FAQ. https://help.openai.com/en/articles/8590148-memory-faq
该常见问题区分已保存记忆与聊天历史,并说明控制、删除、保留和来源移除行为。
[OpenAI 2026] OpenAI. Human-in-the-loop. https://openai.github.io/openai-agents-python/human_in_the_loop/
OpenAI Agents SDK 文档介绍可序列化的中断机制,可针对单次调用暂停审批并恢复原始运行状态,包括嵌套智能体的工具调用。
[OpenAI 2026] OpenAI. Computer use. https://developers.openai.com/api/docs/guides/tools-computer-use
这份官方指南介绍了截图与动作循环、原始分辨率坐标、隔离执行、不可信内容、敏感数据传输,以及风险动作的即时审批。
[OpenAI 2026] OpenAI. Agent orchestration. https://openai.github.io/openai-agents-python/multi_agent/
官方 SDK 文档区分了管理器式的智能体工具、交接、代码驱动流水线和并行执行。
[OpenAI 2026] OpenAI. Why we no longer evaluate SWE-bench Verified. https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/
对 SWE-bench Verified 中常被判失败的任务进行定向复审后,仍发现测试、规格缺陷与暴露迹象,说明基准有效性必须持续复核。
[OpenAI 2026] OpenAI. Introducing GPT-5.5. https://openai.com/index/introducing-gpt-5-5/
OpenAI 报告称,在研究环境中使用 xhigh 推理时,GPT-5.5 在原版 GDPval 上有 84.9% 的结果获胜或打平。
[OpenSSF AI/ML Security Working Group 2025] OpenSSF AI/ML Security Working Group. OpenSSF model signing specification. https://github.com/ossf/model-signing-spec
OMS 对模型路径与摘要的分离式清单签名;验证仍需用信任策略把签名凭据绑定到获批身份。
[OpenTelemetry 2026] OpenTelemetry. OpenTelemetry GenAI semantic conventions. https://github.com/open-telemetry/semantic-conventions-genai/tree/e771bc90d0ebd743e954dc3a4feb5201cd3a5cf8
OpenTelemetry GenAI 语义约定为跨厂商追踪模型调用、智能体步骤和生成式 AI 系统行为定义属性与 span。
[OpenTelemetry 2026] OpenTelemetry. Trace semantic conventions. https://opentelemetry.io/docs/specs/semconv/general/trace/
OpenTelemetry 追踪约定定义了通用的跨度名称、属性、事件和状态字段,使不同服务的遥测数据可以比较。
[OpenTelemetry 2026] OpenTelemetry. OpenTelemetry GenAI semantic conventions. https://github.com/open-telemetry/semantic-conventions-genai
OpenTelemetry 官方生成式 AI 语义约定,定义了用于追踪大语言模型调用的标准化 span 属性,涵盖模型名称、词元用量及请求元数据。
[OpenTelemetry n.d.] OpenTelemetry. OpenTelemetry generative AI semantic conventions. https://opentelemetry.io/docs/specs/semconv/registry/attributes/gen-ai/
OpenTelemetry 注册表定义生成式 AI 遥测属性,并警告模型输入与输出很可能包含敏感信息或个人身份信息。
[OpenXLA Project 2026] OpenXLA Project. StableHLO compatibility. https://openxla.org/stablehlo/compatibility
StableHLO 为可移植张量程序工件规定兼容性,同时明确不保证不同消费者之间具有相同数值精度。
[OpenXLA Project 2026] OpenXLA Project. XLA GPU emitters. https://openxla.org/xla/emitters
OpenXLA 文档解释了 GPU 发射器如何把融合操作降低为特定架构的内核,以及不同发射路径分别适用于何种情况。
[Organisation for Economic Co-operation and Development 2001] Organisation for Economic Co-operation and Development. Measuring productivity: OECD manual. https://doi.org/10.1787/9789264194519-en
该手册解释了为何生产率必须把明确的产出指标与明确的投入联系起来,并说明有效比较所需记录的选择。
[Organisation for Economic Co-operation and Development 2024] Organisation for Economic Co-operation and Development. OECD AI principles. https://oecd.ai/en/ai-principles
经合组织《AI 原则》围绕以人为本的价值、透明度、稳健性、问责、投资和国际合作提出政策承诺。
[Organisation for Economic Co-operation and Development 2025] Organisation for Economic Co-operation and Development. Competition in artificial intelligence infrastructure. https://doi.org/10.1787/623d1874-en
OECD 绘制多层 AI 基础设施供应链,并审视集中度、进入壁垒、纵向关系、转换壁垒与短缺,同时提醒各环节份额并非同一个全球市场定义。
[Ou et al. 2025] Ou, Nie, Xue, Zhu, Sun, Li, Li. Your absorbing discrete diffusion secretly models the conditional distributions of clean data. https://arxiv.org/abs/2406.03736
RADD 将吸收离散扩散的具体分数重参数化为与时间无关的清洁数据条件分布,通过输出缓存加速采样,在 GPT-2 规模的零样本语言建模困惑度上达到扩散模型的最优性能。
[Ouyang et al. 2022] Ouyang, Wu, Jiang, Almeida, Wainwright, Mishkin, Zhang, Agarwal, Slama, Ray, Schulman, Hilton, Kelton, Miller, Simens, Askell, Welinder, Christiano, Leike, Lowe. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems (NeurIPS). https://arxiv.org/abs/2203.02155
InstructGPT 先使用标注者示范训练,再使用偏好排序和强化学习;最终助手行为反映的是整条流程,而不是单独的监督微调。
[Ouyang et al. 2022] Ouyang, Wu, Jiang, Almeida, Wainwright, Mishkin, Zhang, Agarwal, Slama, Ray, Schulman, Hilton, Kelton, Miller, Simens, Askell, Welinder, Christiano, Leike, Lowe. Training language models to follow instructions with human feedback. https://proceedings.neurips.cc/paper_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract.html
InstructGPT 通过基于人类反馈的强化学习(监督微调后用 PPO 优化人类偏好奖励模型)对 GPT-3 进行微调,使 1.3B 参数的模型在人类评测中优于 175B 的 GPT-3 基线。
[Ouyang et al. 2025] Ouyang, Guo, Arora, Zhang, Hu, Ré, Mirhoseini. KernelBench: Can llms write efficient GPU kernels?. PMLR. https://proceedings.mlr.press/v267/ouyang25a.html
检验模型能否写出正确且快于 PyTorch 的 GPU 内核的基准:250 个负载、一个考虑加速比的指标,以及前沿模型单次尝试击败基线的比例不足 20% 的结果。
[Ouyang et al. 2025] Ouyang, Qu, Zhou, Zhu, Zhang, Lin, Wang, Zhao, Jiang, Zhao, Shi, Wu, Chu, Liu, Li, Xu, Zhang, Shi, Tu, He. OmniDocBench: Benchmarking diverse PDF document parsing with comprehensive annotations. https://openaccess.thecvf.com/content/CVPR2025/html/Ouyang_OmniDocBench_Benchmarking_Diverse_PDF_Document_Parsing_with_Comprehensive_Annotations_CVPR_2025_paper.html
该研究提供了一套文档解析基准,包含文本、布局、表格、公式和阅读顺序的标注与指标。
[OWASP CycloneDX 2023] OWASP CycloneDX. Machine learning bill of materials (ML-BOM). https://cyclonedx.org/capabilities/mlbom/
CycloneDX ML-BOM 可编码声明的模型、数据集、依赖、参数、模型卡字段与血统,但不会验证这些声明的真实性或完整性。
[OWASP Foundation n.d.] OWASP Foundation. Mobile application security verification standard. https://mas.owasp.org/MASVS/
MASVS 将移动安全与隐私要求组织到完整应用中,而不把本地计算视为充分保护。
[Packer et al. 2023] Packer, Wooders, Lin, Fang, Patil, Stoica, Gonzalez. MemGPT: Towards llms as operating systems. https://arxiv.org/abs/2310.08560
MemGPT 提出虚拟上下文管理方案,借鉴操作系统分层存储思想,在大语言模型(LLM)固定上下文窗口与外部存储之间分页调度数据,实现文档分析与多轮对话中的无限上下文。
[Pagnoni et al. 2025] Pagnoni, Pasunuru, Rodriguez, Nguyen, Muller, Li, Zhou, Yu, Weston, Zettlemoyer, Ghosh, Lewis, Holtzman, Iyer. Byte latent transformer: Patches scale better than tokens. Association for Computational Linguistics. https://arxiv.org/abs/2412.09871
BLT 用依据下一字节熵选择的动态字节块取代固定子词分词,并在其最高 8B 参数的测试范围内报告了有竞争力的 FLOP 对齐扩展结果。
[Pai 2025] Pai. Designing large language model applications: a holistic approach. O'Reilly Media. https://www.oreilly.com/library/view/designing-large-language/9781098150495/
Suhas Pai 所著 O'Reilly 图书,介绍将大语言模型(LLM)从原型过渡到企业生产级应用的设计模式与工程决策。
[Pan et al. 2025] Pan, Wang, Neubig, Jaitly, Ji, Suhr, Zhang. Training software engineering agents and verifiers with SWE-gym. PMLR. https://proceedings.mlr.press/v267/pan25g.html
SWE-Gym 将 2,438 个真实代码仓库任务封装为可复现的运行环境与测试,支持监督式智能体训练和验证器引导的推理扩展。
[Panickssery et al. 2024] Panickssery, Bowman, Feng. LLM evaluators recognize and favor their own generations. https://proceedings.neurips.cc/paper_files/paper/2024/hash/7f1f0218e45f5414c79c0679633e47bc-Abstract-Conference.html
受控实验把评判模型的自我识别与自我偏好联系起来,说明同源模型评分可能系统性偏离人工判断。
[Parasuraman et al. 2000] Parasuraman, Sheridan, Wickens. A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans. https://doi.org/10.1109/3468.844354
Parasuraman、Sheridan 与 Wickens 按信息获取、分析、决策选择和行动执行拆分自动化,并给出不同的人类参与等级。
[Park et al. 2023] Park, O'Brien, Cai, Morris, Liang, Bernstein. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442. https://arxiv.org/abs/2304.03442
按新近度、重要性与相关性打分检索的记忆流,加上定期把经验综合成高层记忆的反思机制:后来所有记忆系统都在回响的概念模板。
[Park et al. 2024] Park, Wang, Berg-Kirkpatrick, Polikarpova, D'Antoni. Grammar-aligned decoding. https://proceedings.neurips.cc/paper_files/paper/2024/hash/2bdc2267c3d7d01523e2e17ac0a754f3-Abstract-Conference.html
Grammar-Aligned Decoding 表明,普通的局部文法掩码即使保证语法有效,也可能扭曲模型的序列分布。
[Parra and Delimarsky 2026] Parra, Delimarsky. The 2026-07-28 specification. https://blog.modelcontextprotocol.io/posts/2026-07-28/
MCP 2026-07-28 正式版引入了无状态协议核心、版本化扩展、更严格的授权机制和正式的功能生命周期。
[Paszke et al. 2019] Paszke, Gross, Massa, Lerer, Bradbury, Chanan, Killeen, Lin, Gimelshein, Antiga, Desmaison, Köpf, Yang, DeVito, Raison, Tejani, Chilamkurthy, Steiner, Fang, Bai, Chintala. PyTorch: An imperative style, high-performance deep learning library. https://arxiv.org/abs/1912.01703
PyTorch 设计论文说明其即时张量接口、动态自动微分图、分发器、分配器与 C++ 执行路径。
[Patarasuk and Yuan 2009] Patarasuk, Yuan. Bandwidth optimal all-reduce algorithms for clusters of workstations. Journal of Parallel and Distributed Computing. https://www.cs.fsu.edu/~xyuan/paper/09jpdc.pdf
本文推导通信下界,并在明确的拓扑假设下给出适用于大消息的带宽最优环形全归约。
[Patil et al. 2025] Patil, Mao, Yan, Ji, Suresh, Stoica, Gonzalez. The berkeley function calling leaderboard (BFCL): From tool use to agentic evaluation of large language models. PMLR. https://proceedings.mlr.press/v267/patil25a.html
BFCL 评测串行、并行、弃权和有状态多轮函数调用,而不是把工具使用简化为单一参数匹配分数。
[Patwardhan et al. 2025] Patwardhan, Dias, Proehl, Kim, Wang, Watkins, Posada Fishman, Aljubeh, Thacker, Fauconnet, Kim, Chao, Miserendino, Chabot, Li, Sharman, Barr, Glaese, Tworek. GDPval: Evaluating AI model performance on real-world economically valuable tasks. arXiv preprint arXiv:2510.04374. https://arxiv.org/abs/2510.04374
GDPval 是一个覆盖 44 个职业、9 个美国 GDP 行业、共 1,320 个任务的基准测评,通过成对比较显示前沿模型正在接近行业专家水平。
[Paulo et al. 2025] Paulo, Shabalin, Belrose. Transcoders beat sparse autoencoders for interpretability. arXiv preprint arXiv:2501.18823. https://arxiv.org/abs/2501.18823
在所测试的模型和自动化指标上,转码器比同类 MLP 输出稀疏自动编码器更好地兼顾了重建效果与可解释性;该结论只适用于这一比较范围。
[Pearce and Song 2024] Pearce, Song. Reconciling kaplan and chinchilla scaling laws. Transactions on Machine Learning Research. https://arxiv.org/abs/2406.12907
将 Kaplan 与 Chinchilla 的分歧主要归因于 Kaplan 在小规模下计入的是非嵌入参数而非总参数;一旦计入嵌入参数并放大规模,最优指数便收敛到 Chinchilla 的估计。
[Peebles and Xie 2023] Peebles, Xie. Scalable diffusion models with transformers. https://arxiv.org/abs/2212.09748
DiT 用 Transformer 替换潜在扩散模型中的 U-Net 主干,证明模型 Gflops 增大时 FID 持续下降,在 ImageNet 256x256 上达到当时最优 FID 2.27。
[Penedo et al. 2023] Penedo, Malartic, Hesslow, Cojocaru, Cappelli, Alobeidli, Pannier, Almazrouei, Launay. The RefinedWeb dataset for falcon LLM: Outperforming curated corpora with web data, and web data only. https://arxiv.org/abs/2306.01116
RefinedWeb 表明,对 CommonCrawl 进行严格过滤与去重后得到的五万亿词元纯网页数据,可训练出超越使用精心策划语料库(如 The Pile)所训练的大语言模型。
[Penedo et al. 2024] Penedo, Kydlíček, Ben Allal, Lozhkov, Mitchell, Raffel, Von Werra, Wolf. The FineWeb datasets: Decanting the web for the finest text data at scale. https://arxiv.org/abs/2406.17557
FineWeb 是一个从 96 个 Common Crawl 快照中提取的 15 万亿词元预训练数据集,采用消融实验驱动的过滤策略与逐快照最小哈希去重,性能优于其他公开预训练数据集;FineWeb-Edu 是 1.3 万亿词元的教育子集,在 MMLU 和 ARC 上表现显著更好。
[Peng et al. 2023] Peng, Quesnelle, Fan, Shippole. YaRN: Efficient context window extension of large language models. arXiv preprint arXiv:2309.00071. https://arxiv.org/abs/2309.00071
YaRN 以比先前方法更少的词元和训练步数扩展基于 RoPE 的 LLaMA 上下文窗口,并展示了超出微调长度的外推能力。
[Perdomo et al. 2020] Perdomo, Zrnic, Mendler-Dünner, Hardt. Performative prediction. https://proceedings.mlr.press/v119/perdomo20a.html
表演性预测研究部署模型会改变数据分布的场景,并定义这种反馈下的稳定性和最优性。
[Perez and Ribeiro 2022] Perez, Ribeiro. Ignore previous prompt: Attack techniques for language models. ML Safety Workshop at NeurIPS 2022. https://arxiv.org/abs/2211.09527
PromptInject 展示了如何用简单的对抗性用户输入,对 GPT-3 发起直接目标劫持与提示泄露攻击。
[Perez et al. 2022] Perez, Huang, Song, Cai, Ring, Aslanides, Glaese, McAleese, Irving. Red teaming language models with language models. arXiv preprint arXiv:2202.03286. https://arxiv.org/abs/2202.03286
本文提出用语言模型自动生成测试用例对目标语言模型进行红队测试,在一个 280B 参数聊天机器人中发现数万条有害输出。
[Petrov et al. 2023] Petrov, La Malfa, Torr, Bibi. Language model tokenizers introduce unfairness between languages. https://proceedings.neurips.cc/paper_files/paper/2023/hash/74bb24dca8334adce292883b4b651eda-Abstract-Conference.html
论文在平行译文上发现,某些分词器与语言组合的词元数量差异可达 15 倍,造成成本、延迟与可用上下文方面的不平等。
[Pineau et al. 2021] Pineau, Vincent-Lamarre, Sinha, Larivière, Beygelzimer, d'Alché-Buc, Fox, Larochelle. Improving reproducibility in machine learning research: a report from the NeurIPS 2019 reproducibility program. Journal of Machine Learning Research. https://www.jmlr.org/papers/v22/20-303.html
该报告介绍了一项可复现性计划,核心包括开放代码、报告实验细节,以及使用相同代码和数据进行独立复现。
[Podell et al. 2024] Podell, English, Lacey, Blattmann, Dockhorn, Müller, Penna, Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. https://arxiv.org/abs/2307.01952
SDXL 是一个用于文本到图像合成的潜扩散模型,采用 2.6B 参数 UNet、双文本编码器、图像尺寸与裁剪条件化、多宽高比训练及可选的精化阶段。
[Podkopaev and Ramdas 2022] Podkopaev, Ramdas. Tracking the risk of a deployed model and detecting harmful distribution shifts. https://arxiv.org/abs/2110.06177
Podkopaev 与 Ramdas 使用抽样或延迟标签及时间一致保证监控已部署模型的风险,将有害性能变化与任意分布变化区分开来。
[Pokkunuri 2019] Pokkunuri. Ensuring rollback safety during deployments. https://d1.awsstatic.com/builderslibrary/pdfs/ensuring-rollback-safety-during-deployments.pdf
AWS 说明了混合版本的协议与数据兼容性、两阶段变更、观察期和升降级测试如何共同保障安全回滚。
[Polu and Sutskever 2020] Polu, Sutskever. Generative language modeling for automated theorem proving. arXiv preprint arXiv:2009.03393. https://arxiv.org/abs/2009.03393
GPT-f 把 Transformer 语言模型用于 Metamath 证明搜索,并为已有定理贡献了被形式数学库接纳的更短证明。
[Polyak and others 2024] Polyak, others. Movie gen: a cast of media foundation models. arXiv preprint arXiv:2410.13720. https://arxiv.org/abs/2410.13720
Meta 的媒体基础模型,其视频模型为十六秒高清画面维护 7.3 万词元的上下文,而单张百万像素图像约为四千个潜变量词元。
[Poole et al. 2019] Poole, Ozair, Oord, Alemi, Tucker. On variational bounds of mutual information. https://proceedings.mlr.press/v97/poole19a.html
Poole 等人形式化了神经互信息下界的采样假设与偏差-方差权衡,其中包括候选集有限的对比下界。
[Pope et al. 2023] Pope, Douglas, Chowdhery, Devlin, Bradbury, Levskaya, Heek, Xiao, Agrawal, Dean. Efficiently scaling transformer inference. https://proceedings.mlsys.org/paper_files/paper/2023/hash/c4be71ab8d24cdfb45e3d06dbfca2780-Abstract-mlsys2023.html
本文提出用于 TPU v4 的张量并行分区框架与底层优化,使 PaLM 540B 在 int8 量化下实现 29ms/词元延迟与 76% 模型 FLOPs 利用率(MFU)。
[PostgreSQL Global Development Group 2026] PostgreSQL Global Development Group. Row security policies. https://www.postgresql.org/docs/current/ddl-rowsecurity.html
PostgreSQL 行级安全可限制读写,但表所有者、超级用户、BYPASSRLS 角色、策略组合与竞态条件都需要显式处理。
[Press and Wolf 2017] Press, Wolf. Using the output embedding to improve language models. Association for Computational Linguistics. https://arxiv.org/abs/1608.05859
将神经语言模型的输入与输出嵌入矩阵绑定(weight tying),可降低困惑度,并将翻译模型参数量压缩至原来的不足一半而不损失性能。
[Press et al. 2022] Press, Smith, Lewis. Train short, test long: Attention with linear biases enables input length extrapolation. https://arxiv.org/abs/2108.12409
ALiBi 用与距离成线性比例的偏置替换位置嵌入,使模型在短序列上训练后能在推理时外推到更长序列,且不增加额外运行时开销。
[Pushkarna et al. 2022] Pushkarna, Zaldivar, Kjartansson. Data cards: Purposeful and transparent dataset documentation for responsible AI. https://arxiv.org/abs/2204.01075
Data Cards 围绕数据集来源、收集与标注、预期用途、伦理考量、维护和生命周期演进来组织文档。
[Pydantic Services 2026] Pydantic Services. Durable execution: Overview. https://pydantic.dev/docs/ai/capabilities/durable_execution/overview/
Pydantic AI 文档介绍了持久执行集成,这类集成把持久化和恢复交给 Temporal、DBOS、Prefect 和 Restate 等工作流系统。
[Python Software Foundation 2026] Python Software Foundation. <span class="nocase">pickle</span> — Python object serialization. https://docs.python.org/3/library/pickle.html
Python 文档把 pickle 定义为二进制对象序列化格式,并解释了为何反序列化不可信输入可能执行任意代码。
[PyTorch n.d.] PyTorch. Getting started with fully sharded data parallel (FSDP2). https://docs.pytorch.org/tutorials/intermediate/FSDP_tutorial.html
官方教程说明了 FSDP2 如何分片参数、梯度和优化器状态,并调度全收集与归约分散操作。
[PyTorch Contributors 2026] PyTorch Contributors. Autograd mechanics. https://docs.pytorch.org/docs/stable/notes/autograd.html
该官方说明记录 PyTorch 的动态自动微分图、保存张量、不可微约定与原地操作正确性检查。
[PyTorch Contributors 2026] PyTorch Contributors. Extending PyTorch. https://docs.pytorch.org/docs/stable/notes/extending.html
PyTorch 文档说明了自定义模块、自动微分函数、算子和张量行为如何接入其执行与求导系统。
[PyTorch Contributors 2026] PyTorch Contributors. Gradcheck mechanics. https://docs.pytorch.org/docs/stable/notes/gradcheck.html
PyTorch 说明了 gradcheck 如何比较解析导数与有限差分,以及容差、复数值和非确定性为何重要。
[PyTorch Contributors 2026] PyTorch Contributors. Activation checkpointing. https://docs.pytorch.org/docs/stable/checkpoint.html
激活检查点丢弃部分前向激活值,并在反向传播时重新计算,以增加计算为代价降低内存占用。
[PyTorch Contributors 2026] PyTorch Contributors. <span class="nocase">torch.compile</span> programming model. https://docs.pytorch.org/docs/stable/user_guide/torch_compiler/compile/programming_model.html
该编程模型指南定义图捕获、图中断、守卫、重新编译以及部分图与完整图契约。
[PyTorch Contributors 2026] PyTorch Contributors. PyTorch DTensor: Distributed tensor. https://docs.pytorch.org/docs/stable/distributed.tensor.html
DTensor 契约以设备网格上的 Shard、Replicate 与 Partial 放置定义逻辑张量及其重分布语义。
[PyTorch Contributors 2026] PyTorch Contributors. torch.compile troubleshooting. https://docs.pytorch.org/docs/stable/user_guide/torch_compiler/torch.compiler_troubleshooting.html
PyTorch 故障排查指南解释了 torch.compile 流水线中的图中断、重新编译、守卫失败、日志记录和调试。
[PyTorch Contributors 2026] PyTorch Contributors. Torch distributed elastic. https://docs.pytorch.org/docs/stable/distributed.elastic.html
Torch Distributed Elastic 管理可能故障或改变成员的工作进程组,并按照既定的集合与故障模型重启整个组。
[PyTorch Contributors 2026] PyTorch Contributors. Distributed checkpoint: torch.distributed.checkpoint. https://docs.pytorch.org/docs/stable/distributed.checkpoint.html
PyTorch Distributed Checkpoint 可以在不同训练器数量和并行布局之间重新分片模型与优化器状态,但不保证其状态字典跨 PyTorch 版本向后兼容。
[PyTorch Contributors 2026] PyTorch Contributors. Serialization semantics. https://docs.pytorch.org/docs/stable/notes/serialization.html
PyTorch 文档介绍了 ZIP64 检查点布局、2.6 版起默认采用的仅权重反序列化器,以及不受信任产物仍可能造成的拒绝服务和内存破坏风险。
[PyTorch Foundation n.d.] PyTorch Foundation. ExecuTorch: Architecture and components. https://docs.pytorch.org/executorch/stable/getting-started-architecture
ExecuTorch 记录程序准备、运行时准备和执行三个阶段,并采用目标相关的下沉与后端支持。
[Qi et al. 2024] Qi, Zeng, Xie, Chen, Jia, Mittal, Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to!. https://proceedings.iclr.cc/paper_files/paper/2024/hash/83b7da3ed13f06c13ce82235c8eedf35-Abstract-Conference.html
实验表明,使用良性或对抗性数据微调已对齐模型,都可能削弱安全行为,具体结果取决于数据和方法。
[Qian et al. 2021] Qian, Zhou, Bao, Wang, Qiu, Zhang, Yu, Li. Glancing transformer for non-autoregressive neural machine translation. https://arxiv.org/abs/2008.07905
GLAT 提出瞥视语言模型训练策略,使非自回归生成神经机器翻译可在单次并行解码中完成,以 8x-15x 加速将质量差距缩小到自回归 Transformer 的 0.25-0.9 BLEU 以内。
[Qian et al. 2025] Qian, Acikgoz, He, Wang, Chen, Hakkani-Tür, Tur, Ji. ToolRL: Reward is all tool learning needs. https://proceedings.neurips.cc/paper_files/paper/2025/hash/97c5b2707228e7e3fb67e4ecc2e0e607-Abstract-Conference.html
ToolRL 通过实验说明,工具使用训练取决于动作有效性、工具选择、参数、尺度、粒度和反馈时机等多方面的奖励设计。
[Qin et al. 2025] Qin, Li, He, Cui, Ren, Zhang, Wu, Zheng, Xu. Mooncake: Trading more storage for less computation—a kvcache-centric architecture for serving LLM chatbot. USENIX Association. https://www.usenix.org/conference/fast25/presentation/qin
Mooncake 在分布式缓存层次中管理 KV 状态,以存储与传输容量换取更少的重复预填充计算。
[Qiu et al. 2025] Qiu, Biswas, Zhao, Mohan, Khare, Choukse, Goiri, Zhang, Shen, Bansal, Ramjee, Fonseca. ModServe: Modality- and stage-aware resource disaggregation for scalable multimodal model serving. https://arxiv.org/abs/2502.00937
ModServe 测量多模态推理各阶段的异构资源需求与生产请求的重尾分布,并独立扩缩与放置各阶段,以满足尾延迟目标。
[Qu et al. 2021] Qu, Ding, Liu, Liu, Ren, Zhao, Dong, Wu, Wang. RocketQA: An optimized training approach to dense passage retrieval for open-domain question answering. https://aclanthology.org/2021.naacl-main.466/
RocketQA 通过跨批次负样本、去噪难负样本和交叉编码器数据增强三种训练策略,提升双编码器密集段落检索在开放域问答中的性能。
[Qwen Team 2025] Qwen Team. Qwen3-next-80B-A3B-instruct model card. https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct
Qwen3-Next 是总参数 80B、激活 3B 的 MoE 模型,48 层按三层 Gated DeltaNet 线性注意力配一层门控全注意力交替排布。
[Rabanser et al. 2019] Rabanser, Günnemann, Lipton. Failing loudly: An empirical study of methods for detecting dataset shift. https://proceedings.neurips.cc/paper/2019/hash/846c260d715e5b854ffad5f70a516c88-Abstract.html
Rabanser、Günnemann 与 Lipton 实证比较高维数据集漂移检测器以及描述已检测漂移的方法。
[Radford et al. 2019] Radford, Wu, Child, Luan, Amodei, Sutskever. Language models are unsupervised multitask learners. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf
GPT-2 报告介绍了可逆的字节到 Unicode 映射、基于正则表达式的预分词,以及对各分段应用的 BPE。
[Radford et al. 2021] Radford, Kim, Hallacy, Ramesh, Goh, Agarwal, Sastry, Askell, Mishkin, Clark, others. Learning transferable visual models from natural language supervision. https://arxiv.org/abs/2103.00020
CLIP 通过对比语言-图像预训练在 4 亿图文对上联合训练图像编码器与文本编码器,实现对下游视觉任务的零样本迁移。
[Radford et al. 2023] Radford, Kim, Xu, Brockman, McLeavey, Sutskever. Robust speech recognition via large-scale weak supervision. https://arxiv.org/abs/2212.04356
Whisper 在 68 万小时弱监督多语言音频上训练编码器-解码器 Transformer,无需针对特定数据集微调,即可在自动语音识别(ASR)任务上实现接近人类水平的鲁棒性。
[Rafailov et al. 2023] Rafailov, Sharma, Mitchell, Ermon, Manning, Finn. Direct preference optimization: Your language model is secretly a reward model. https://proceedings.neurips.cc/paper_files/paper/2023/hash/a85b405ed65c6477a4fe8302b5e06ce7-Abstract-Conference.html
DPO 用简单的二元交叉熵损失替代基于人类反馈的强化学习(RLHF)中的显式奖励模型与强化学习循环,直接从人类偏好数据中提取最优策略。
[Ragan-Kelley et al. 2013] Ragan-Kelley, Barnes, Adams, Paris, Durand, Amarasinghe. Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines. https://dl.acm.org/doi/10.1145/2491956.2462176
Halide 把「算什么」与「怎样调度到机器上」分离开来,这是每一个张量编译器做调度搜索背后的奠基思想。
[Raina et al. 2024] Raina, Liusie, Gales. Is LLM-as-a-judge robust? Investigating universal adversarial attacks on zero-shot LLM assessment. Association for Computational Linguistics. https://aclanthology.org/2024.emnlp-main.427/
本文构造可迁移的短语来抬高评判分数,并发现其测试的绝对评分协议比比较式评测更易受攻击。
[Rajbhandari et al. 2020] Rajbhandari, Rasley, Ruwase, He. ZeRO: Memory optimizations toward training trillion parameter models. https://arxiv.org/abs/1910.02054
ZeRO 通过分区优化器状态、梯度和参数,消除数据并行训练中的内存冗余,同时改变通信调度。
[Rajbhandari et al. 2021] Rajbhandari, Ruwase, Rasley, Smith, He. ZeRO-infinity: Breaking the GPU memory wall for extreme scale deep learning. https://arxiv.org/abs/2104.07857
ZeRO-Infinity 是一种异构训练系统,将模型状态卸载至 CPU 和 NVMe 内存,使现有 GPU 集群无需修改模型代码即可训练数十万亿参数规模的模型。
[Raji et al. 2021] Raji, Denton, Bender, Hanna, Paullada. AI and the everything in the whole wide world benchmark. https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/hash/084b6fbb10729ed4da8c3d3f5a3ae7c9-Abstract-round2.html
Raji 等人分析了狭窄的基准任务如何被用来代表关于通用 AI 进展的宽泛主张,并指出这些主张往往超出了测试所能提供的构念证据。
[Rajput et al. 2023] Rajput, Mehta, Singh, Keshavan, Vu, Heldt, Hong, Tay, Tran, Samost, Kula, Chi, Sathiamoorthy. Recommender systems with generative retrieval. https://arxiv.org/abs/2305.05065
TIGER 将条目内容嵌入量化为语义 ID,并训练序列模型解码下一个条目的 ID,在其评估流程中取代近似最近邻查询,同时保留外部码本和条目映射。
[Ramesh et al. 2022] Ramesh, Dhariwal, Nichol, Chu, Chen. Hierarchical text-conditional image generation with CLIP latents. arXiv preprint arXiv:2204.06125. https://arxiv.org/abs/2204.06125
DALL-E 2(unCLIP)通过将文本映射到 CLIP 图像嵌入的先验模型与将嵌入还原为 1024×1024 图像的扩散解码器串联,实现文本条件图像生成。
[Rand et al. 2025] Rand, Manderlink, Zhang, Talley, Gorman, Wiser, Seel, Kemp, Jeong, Kahrl. Queued up: 2025 edition. Characteristics of power plants seeking transmission interconnection. https://emp.lbl.gov/publications/queued-2025-edition-characteristics
2025 年《Queued Up》报告衡量了美国并网队列中项目的容量、位置、技术类型和开发状态。
[Raschka 2024] Raschka. Build a large language model (from scratch). Manning Publications. https://www.manning.com/books/build-a-large-language-model-from-scratch
本书手把手引导读者从头实现大语言模型(LLM)的注意力机制与 GPT 风格的 Transformer 架构,涵盖训练、微调与指令跟随。
[Rashkin et al. 2023] Rashkin, Nikolaev, Lamm, Aroyo, Collins, Das, Petrov, Tomar, Turc, Reitter. Measuring attribution in natural language generation models. MIT Press. https://aclanthology.org/2023.cl-4.2/
AIS 框架把归因定义为在明确标注准则下由独立、可识别来源提供支持,同时把来源质量保留为另一项判断。
[Ratner et al. 2016] Ratner, De Sa, Wu, Selsam, Ré. Data programming: Creating large training sets, quickly. Advances in Neural Information Processing Systems (NeurIPS). https://arxiv.org/abs/1605.07723
数据编程让用户编写带噪声的标注函数替代手工标注,通过生成式模型对输出去噪,从而以程序化方式快速构建大规模训练集。
[Rawles et al. 2025] Rawles, Clinckemaillie, Chang, Waltz, Lau, Fair, Li, Bishop, Li, Campbell-Ajala, Toyama, Berry, Tyamagundlu, Lillicrap, Riva. AndroidWorld: a dynamic benchmarking environment for autonomous agents. https://proceedings.iclr.cc/paper_files/paper/2025/hash/01a83bc2f2732a58e6aa731e659e7101-Abstract-Conference.html
可复现的 Android 环境,覆盖 20 个应用的 116 个参数化任务,每项都有初始化、基于状态的成功检查和清理;任务变化会显著影响结果。
[Reddi et al. 2022] Reddi, Kanter, Mattson, Duke, Nguyen, Chukka, Shiring, Tan, Charlebois, Chou, El-Khamy, Hong, St John, Trinh, Buch, Mazumder, Markovic, Atta, Cakir, Charkhabi, Chen, Chiang, Dexter, Heo, Schmuelling, Shabani, Zika. MLPerf Mobile Inference Benchmark: An industry-standard open-source machine learning benchmark for on-device AI. https://proceedings.mlsys.org/paper_files/paper/2022/hash/a2b2702ea7e682c5ea2c20e8f71efb0c-Abstract.html
MLPerf Mobile 定义通用任务、质量目标、运行规则与设备侧测量,使异构移动栈的结果可解释。
[Reddit, Inc. 2024] Reddit, Inc.. Prospectus filed pursuant to rule 424(b)(4). U.S. Securities,Exchange Commission. https://www.sec.gov/Archives/edgar/data/1713445/000162828024012380/reddit-final424b4.htm
将多项数据许可安排披露为合同总额,而非每 token 或特定合作方价格。
[Reimers and Gurevych 2019] Reimers, Gurevych. Sentence-BERT: Sentence embeddings using siamese BERT-networks. https://aclanthology.org/D19-1410/
SBERT 通过孪生网络和三元组网络对 BERT 进行微调,生成可用余弦相似度直接比较的句子嵌入,将成对相似度搜索时间从 65 小时缩短至 5 秒。
[Ren et al. 2025] Ren, Shao, Song, Xin, Wang, Zhao, Zhang, Fu, Zhu, Yang, Wu, Gou, Ma, Tang, Liu, Gao, Guo, Ruan. DeepSeek-prover-V2: Advancing formal mathematical reasoning via reinforcement learning for subgoal decomposition. arXiv preprint arXiv:2504.21801. https://arxiv.org/abs/2504.21801
DeepSeek-Prover-V2 把非形式化与形式化推理结合到 Lean 4 定理证明中,通过递归分解和强化学习在 MiniF2F 与 PutnamBench 上取得强结果。
[Rhoades 1993] Rhoades. The herfindahl-hirschman index. Board of Governors of the Federal Reserve System. https://fraser.stlouisfed.org/files/docs/publications/FRB/pages/1990-1994/33101_1990-1994.pdf
Rhoades 解释了平方份额指数、Hirschman 与 Herfindahl 各自提出该指数的历史,以及它作为集中度筛查指标而非完整竞争分析的用途。
[Richer 2015] Richer. OAuth 2.0 token introspection. https://www.rfc-editor.org/rfc/rfc7662.html
RFC 7662 允许获得授权的受保护资源查询令牌当前是否有效及其元数据。
[Rissanen 1978] Rissanen. Modeling by shortest data description. Automatica.
Rissanen 提出最小描述长度(MDL)原理,将模型选择表述为选取最能压缩数据的模型。
[Robertson and Zaragoza 2009] Robertson, Zaragoza. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval. https://doi.org/10.1561/1500000019
系统推导并解释概率相关性框架、BM25 词项加权及其参数。
[Rombach et al. 2022] Rombach, Blattmann, Lorenz, Esser, Ommer. High-resolution image synthesis with latent diffusion models. https://arxiv.org/abs/2112.10752
潜在扩散模型(LDM)将扩散过程移至预训练自编码器的潜在空间,大幅降低训练和推理成本,并通过交叉注意力机制支持文本到图像等多模态条件生成。
[Rose et al. 2020] Rose, Borchert, Mitchell, Connelly. Zero trust architecture. https://csrc.nist.gov/pubs/sp/800/207/final
NIST 参考架构将策略决策与执行分离,并拒绝基于网络位置的隐式信任。
[Rose et al. 2020] Rose, Borchert, Mitchell, Connelly. Zero trust architecture. https://doi.org/10.6028/NIST.SP.800-207
零信任以针对主体、资产和资源的显式身份认证与授权,取代基于网络位置的隐式信任。
[Rosenblatt 1958] Rosenblatt. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological Review.
Rosenblatt 提出感知机,一种带权重更新学习规则的可训练线性分类器,是神经网络的早期基础。
[Rosenfeld et al. 2020] Rosenfeld, Rosenfeld, Belinkov, Shavit. A constructive prediction of the generalization error across scales. https://arxiv.org/abs/1909.12673
提出泛化误差的参数化包络形式,即随模型规模与数据规模的幂律再加一个不可约下界,在视觉与语言上拟合,早于其在语言模型上的专门化。
[Rosner 2025] Rosner. Chairman rosner's letter to the rtos and isos on large load forecasting. https://www.ferc.gov/news-events/news/chairman-rosners-letter-rtosisos-large-load-forecasting
美国联邦能源监管委员会要求区域电网运营方改进大型负载预测,披露方法与不确定性,并将预测纳入输电规划。
[Röttger et al. 2024] Röttger, Kirk, Vidgen, Attanasio, Bianchi, Hovy. XSTest: a test suite for identifying exaggerated safety behaviours in large language models. https://arxiv.org/abs/2308.01263
XSTEST 是一个包含 250 条提示的测试集,用于系统识别大语言模型(LLM)因词汇重叠而对安全提示产生过度拒绝(over-refusal)的行为。
[Rouhani and others 2023] Rouhani, others. Microscaling data formats for deep learning. https://arxiv.org/abs/2310.10537
OCP 微缩放(MX)提案由 AMD、Arm、Intel、Meta、Microsoft、NVIDIA 与 Qualcomm 联合提出,让窄浮点或整数元素类型共享一个块级 scale,并证明包括 MXFP4 在内的 MX 格式能以极小的精度损失用于推理与训练。
[Ru et al. 2024] Ru, Qiu, Hu, Zhang, Shi, Chang, Cheng, Wang, Sun, Li, Zhang, Wang, Jiang, He, Wang, Liu, Zhang, Zhang. RAGChecker: a fine-grained framework for diagnosing retrieval-augmented generation. https://proceedings.neurips.cc/paper_files/paper/2024/hash/27245589131d17368cccdfa990cbf16e-Abstract.html
RAGChecker 分开诊断检索与生成,并用人类判断验证其指标。
[Rumelhart et al. 1986] Rumelhart, Hinton, Williams. Learning representations by back-propagating errors. Nature. https://www.nature.com/articles/323533a0
该论文说明了反向传播如何把输出误差梯度传过多层网络,从而调整内部表示。
[Russinovich et al. 2025] Russinovich, Salem, Eldan. Great, now write an article about that: The crescendo multi-turn LLM jailbreak attack. https://www.usenix.org/conference/usenixsecurity25/presentation/russinovich
Crescendo 是一种多轮大语言模型越狱攻击,通过利用模型自身的历史输出逐步升级无害提示,在 GPT-4、Gemini-Pro 等模型上实现高攻击成功率,绕过安全对齐。
[Rust et al. 2021] Rust, Pfeiffer, Vulić, Ruder, Gurevych. How good is your tokenizer? On the monolingual performance of multilingual language models. Association for Computational Linguistics. https://aclanthology.org/2021.acl-long.243/
受控的单语比较表明,除了预训练数据外,分词器质量也会独立影响多语言下游性能。
[Saeidi et al. 2025] Saeidi, Verma, Uddin, Baral. Insights into alignment: Evaluating DPO and its variants across multiple tasks. Association for Computational Linguistics. https://aclanthology.org/2025.acl-srw.26/
一项跨任务和初始化方式的 DPO、IPO、KTO 与 CPO 比较发现,领先方法会随设置变化。
[Safetensors Project 2026] Safetensors Project. Safetensors format specification. https://github.com/huggingface/safetensors/blob/62e4d8b86063a6e5f8967547fa09a09f67420e2a/README.md#format
固定版本的 safetensors 规范定义了八字节头部长度、JSON 张量描述符和连续字节缓冲区,并且不采用 pickle 式的可调用重建。
[Saha and Ye 2024] Saha, Ye. I/O complexity of attention, or how optimal is FlashAttention?. PMLR. https://proceedings.mlr.press/v235/saha24a.html
本文给出逐点 IO 下界:当快速内存容量至少为头宽平方时与 FlashAttention 匹配,并指出低于该阈值时存在不同的最优区间。
[Saharia et al. 2022] Saharia, Chan, Saxena, others. Photorealistic text-to-image diffusion models with deep language understanding. https://arxiv.org/abs/2205.11487
Imagen 将冻结的 T5-XXL 文本编码器与级联扩散模型结合,发现扩展语言模型对文本到图像质量的提升远大于扩展图像扩散模型,在 COCO 上取得 FID 7.27。
[Sahoo et al. 2024] Sahoo, Arriola, Schiff, Gokaslan, Marroquin, Chiu, Rush, Kuleshov. Simple and effective masked diffusion language models. https://arxiv.org/abs/2406.07524
MDLM 通过 Rao-Blackwell 化目标与现代训练方法,大幅缩小了离散扩散模型与自回归语言模型之间的困惑度差距。
[Salemi et al. 2024] Salemi, Mysore, Bendersky, Zamani. LaMP: When large language models meet personalization. https://aclanthology.org/2024.acl-long.399/
LaMP 提供七项个性化语言建模任务,并评估从用户画像中选择相关内容的检索方法。
[Salimans et al. 2017] Salimans, Ho, Chen, Sidor, Sutskever. Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864. https://arxiv.org/abs/1703.03864
通过共享随机种子将进化策略(ES)扩展至1440个并行工作节点,无需反向传播或价值函数即可在MuJoCo和Atari上取得与策略梯度强化学习相当的结果。
[Salimans and Ho 2022] Salimans, Ho. Progressive distillation for fast sampling of diffusion models. https://arxiv.org/abs/2202.00512
反复把扩散采样器蒸馏成只需一半步数的学生模型,把生成从数千步降到最少四步,代价不超过原始训练。
[Saltzer and Schroeder 1975] Saltzer, Schroeder. The protection of information in computer systems. Proceedings of the IEEE. https://doi.org/10.1109/PROC.1975.9939
Saltzer 和 Schroeder 提出了经久不衰的安全原则,包括默认拒绝、完全仲裁、权限分离和最小权限。
[Sambasivan et al. 2021] Sambasivan, Kapania, Highfill, Akrong, Paritosh, Aroyo. “Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI. https://doi.org/10.1145/3411764.3445518
Sambasivan 等记录了高风险 AI 中的数据级联:上游数据问题会复合成严重的下游模型失效。
[San Roman et al. 2024] San Roman, Fernandez, Défossez, Furon, Tran, Elsahar. Proactive detection of voice cloning with localized watermarking. https://arxiv.org/abs/2401.17264
AudioSeal 是首个面向 AI 生成语音样本级定位检测的音频水印系统,采用联合训练的生成器与检测器架构,检测速度比现有方法快两个数量级。
[Santhanam et al. 2022] Santhanam, Khattab, Saad-Falcon, Potts, Zaharia. ColBERTv2: Effective and efficient retrieval via lightweight late interaction. https://aclanthology.org/2022.naacl-main.272/
ColBERTv2 将残差压缩与去噪监督相结合,使晚期交互检索的存储占用降低 6-10 倍,同时在训练域内外均达到最优检索质量。
[Sardana et al. 2024] Sardana, Portes, Doubov, Frankle. Beyond chinchilla-optimal: Accounting for inference in language model scaling laws. https://arxiv.org/abs/2401.00448
Sardana 等人把预期推理需求加入扩展律成本模型,发现高需求在固定质量与拟合假设下可能有利于训练更久的小模型。
[Sardana et al. 2024] Sardana, Portes, Doubov, Frankle. Beyond chinchilla-optimal: Accounting for inference in language model scaling laws. PMLR. https://proceedings.mlr.press/v235/sardana24a.html
将计算最优扩展到训练加推理的生命周期成本,指出在目标质量固定且预期需求很高时,更小但训练更久的模型可能更优。
[Sato 2014] Sato. Parallel change. https://martinfowler.com/bliki/ParallelChange.html
并行变更采用扩展、迁移和收缩三个阶段,使新旧表示能在安全过渡期间共存。
[Sauer et al. 2024] Sauer, Lorenz, Blattmann, Rombach. Adversarial diffusion distillation. https://eccv.ecva.net/virtual/2024/poster/1963
把分数蒸馏与对抗损失结合,实现一到四步采样,是 SDXL-Turbo 背后的方法,四步之内即可达到与教师模型相当的实时合成。
[Savinov et al. 2022] Savinov, Chung, Binkowski, Elsen, Oord. Step-unrolled denoising autoencoders for text generation. https://arxiv.org/abs/2112.06749
SUNDAE 提出一种非自回归生成文本模型,在训练时采用展开去噪机制对词元序列迭代去噪,在 WMT'14 翻译任务上达到非自回归方法的最优结果。
[Saxena 2023] Saxena. Prompt lookup decoding. https://github.com/apoorvumang/prompt-lookup-decoding
提示词查找通过把最近词元与提示或输出中的重复片段匹配来提出续写,再依赖目标模型验证。
[Scale AI 2026] Scale AI. Humanity's last exam leaderboard. https://labs.scale.com/leaderboard/humanitys_last_exam
Scale AI 官方排行榜把固定的无工具评估,与采用其他工具协议报告的结果分开列示。
[Schaeffer et al. 2023] Schaeffer, Miranda, Koyejo. Are emergent abilities of large language models a mirage?. https://papers.neurips.cc/paper_files/paper/2023/hash/adc98a266f45005c403b8311ca7e8bd7-Abstract-Conference.html
大语言模型(LLM)中部分表观涌现能力可能由非线性或不连续评估指标造成,而非模型行为本身发生不连续变化。
[SchedMD n.d.] SchedMD. Slurm workload manager: Overview. https://slurm.schedmd.com/overview.html
Slurm 概览区分集群资源分配、排队作业调度以及在已分配节点上执行作业步骤。
[Schick et al. 2023] Schick, Dwivedi-Yu, Dessì, Raileanu, Lomeli, Hambro, Zettlemoyer, Cancedda, Scialom. Toolformer: Language models can teach themselves to use tools. https://proceedings.neurips.cc/paper/2023/hash/d842425e4bf79ba039352da0f658a906-Abstract-Conference.html
Toolformer 生成候选 API 标注,执行调用并按语言模型损失筛选,再微调模型决定何时及如何使用五个固定 API。
[Schrittwieser et al. 2020] Schrittwieser, Antonoglou, Hubert, Simonyan, Sifre, Schmitt, Guez, Lockhart, Hassabis, Graepel, Lillicrap, Silver. Mastering atari, go, chess and shogi by planning with a learned model. Nature. https://www.nature.com/articles/s41586-020-03051-4
MuZero 使用学习到的隐状态动力学进行规划,预测奖励、策略与价值,而无需重建观测或预先获得环境规则。
[Schulman et al. 2017] Schulman, Wolski, Dhariwal, Radford, Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. https://arxiv.org/abs/1707.06347
PPO 提出裁剪代理目标函数,使策略梯度强化学习在仅用一阶优化的条件下达到 TRPO 的可靠性,并具备更优的样本复杂度。
[Schulman and others 2025] Schulman, others. LoRA without regret. https://thinkingmachines.ai/blog/lora/
大规模实验表明,把 LoRA 施加到所有线性层(含 MLP 与 MoE)并以完整微调约十倍的学习率训练时,效果与完整微调相当;只作用于注意力层会明显欠佳,而在 RL 后训练中即便秩为 1,LoRA 也能追平完整微调。
[Schultz et al. 1997] Schultz, Dayan, Montague. A neural substrate of prediction and reward. Science.
Schultz、Dayan 与 Montague 报告了类似奖励预测误差的中脑多巴胺反应,将神经科学与时序差分强化学习联系起来。
[Sculley et al. 2015] Sculley, Holt, Golovin, Davydov, Phillips, Ebner, Chaudhary, Young, Crespo, Dennison. Hidden technical debt in machine learning systems. https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems
Sculley 等人描述机器学习系统中的隐性技术债,包括边界侵蚀、纠缠、数据依赖和未声明消费者。
[Sculley et al. 2015] Sculley, Holt, Golovin, Davydov, Phillips, Ebner, Chaudhary, Young, Crespo, Dennison. Hidden technical debt in machine learning systems. https://proceedings.neurips.cc/paper_files/paper/2015/hash/86df7dcfd896fcaf2674f757a2463eba-Abstract.html
生产机器学习系统的技术债常积累在胶水代码、配置、未声明消费者和不断变化的外部依赖中,而不只存在于模型本身。
[Seamless Communication 2023] Seamless Communication. SeamlessM4T: Massively multilingual & multimodal machine translation. arXiv preprint arXiv:2308.11596. https://arxiv.org/abs/2308.11596
SeamlessM4T 是一个统一模型,支持自动语音识别(ASR)、语音到语音、语音到文本、文本到语音及文本到文本翻译,覆盖多达 100 种语言,使用 406,000 小时多语言语音数据训练。
[Seamless Communication 2023] Seamless Communication. Seamless: Multilingual expressive and streaming speech translation. arXiv preprint arXiv:2312.05187. https://arxiv.org/abs/2312.05187
Seamless 提出 SeamlessM4T v2、SeamlessExpressive 和 SeamlessStreaming,一套支持多语言实时、保留韵律与声音风格的表达性语音到语音翻译模型家族。
[Semiconductor Industry Association and Boston Consulting Group 2021] Semiconductor Industry Association, Boston Consulting Group. Strengthening the global semiconductor supply chain in an uncertain era. https://www.semiconductors.org/wp-content/uploads/2021/03/BCG_Strengthening-the-Global-Semiconductor-Supply-Chain_Antonio-Varas-and-Raj-Varadarajan.pdf
该报告区分了设计、EDA、设备、材料、制造,以及组装、封装和测试等环节,并记录其地域分工。
[Sennrich et al. 2016] Sennrich, Haddow, Birch. Neural machine translation of rare words with subword units. Association for Computational Linguistics. https://arxiv.org/abs/1508.07909
本文提出将 BPE(字节对编码)用于稀有词的子词分割,使神经机器翻译无需回退词典即可实现开放词表翻译。
[Settles 2009] Settles. Active learning literature survey. https://minds.wisconsin.edu/handle/1793/60660
这篇综述梳理了主动学习的查询策略、假设、评估方法,以及选择标注样本时的实际考虑。
[Seung et al. 1992] Seung, Sompolinsky, Tishby. Statistical mechanics of learning from examples. Physical Review A.
Seung、Sompolinsky 与 Tishby 用统计力学分析学习,推导出泛化误差与学习曲线随训练样本数的变化规律。
[Shah et al. 2024] Shah, Bikshandi, Zhang, Thakkar, Ramani, Dao. FlashAttention-3: Fast and accurate attention with asynchrony and low-precision. https://proceedings.neurips.cc/paper_files/paper/2024/hash/7ede97c3e082c6df10a8d6103a2eebd2-Abstract-Conference.html
FlashAttention-3 利用 Hopper GPU 的异步执行与 FP8 低精度,通过 warp 专化将注意力计算比 FlashAttention-2 加速 1.5-2 倍,FP16 达 740 TFLOPs/s,FP8 接近 1.2 PFLOPs/s。
[Shannon 1948] Shannon. A mathematical theory of communication. Bell System Technical Journal.
Shannon 奠定了信息论,将熵定义为信息的度量,并确立了数据压缩与有噪信道可靠通信的极限。
[Shao et al. 2024] Shao, Wang, Zhu, Xu, Song, Bi, Zhang, Zhang, Li, Wu, Guo. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. https://arxiv.org/abs/2402.03300
DeepSeekMath 结合 1200 亿词元数学语料与 GRPO,后者是去掉评论家、在采样组内归一化奖励的 PPO 变体。
[Shao et al. 2026] Shao, Li, Xin, Geng, Wang, Oh, Du, Lambert, Min, Krishna, Tsvetkov, Hajishirzi, Koh, Zettlemoyer. Spurious rewards: Rethinking training signals in RLVR. https://arxiv.org/abs/2506.10947
Spurious Rewards 发现,随机或错误奖励在 Qwen2.5-Math-7B 上能恢复大部分真实奖励的 RLVR 增益,却常在 Llama 与 OLMo 模型上失效,说明该效应高度依赖底座模型。
[Sharkey and Telser 1978] Sharkey, Telser. Supportable cost functions for the multiproduct firm. Journal of Economic Theory. https://doi.org/10.1016/0022-0531(78)90040-6
Sharkey 与 Telser 形式化多产品企业的成本可支持性与次可加性,为自然垄断主张提供成本条件。
[Sharma and Kaplan 2022] Sharma, Kaplan. Scaling laws from the data manifold dimension. Journal of Machine Learning Research. https://arxiv.org/abs/2004.10802
从数据流形的内蕴维数推导出扩展指数,预言损失随幂律下降,其速率由数据实际占据的维数决定。
[Sharma et al. 2024] Sharma, Tong, Korbak, Duvenaud, Askell, Bowman, Cheng, Durmus, Hatfield-Dodds, Johnston, Kravec, Maxwell, McCandlish, Ndousse, Rausch, Schiefer, Yan, Zhang, Perez. Towards understanding sycophancy in language models. https://arxiv.org/abs/2310.13548
研究发现,与用户既有观点一致的回复更容易获得偏好,而人工和学习到的偏好判断有时会让看似可信的谄媚胜过正确性。
[Sharma et al. 2025] Sharma, Tong, Mu, Wei, Kruthoff, Goodfriend, Ong, Peng, Agarwal, Anil, others. Constitutional classifiers: Defending against universal jailbreaks across thousands of hours of red teaming. arXiv preprint arXiv:2501.18837. https://arxiv.org/abs/2501.18837
Constitutional Classifiers 使用宪法(constitution)引导合成数据训练大语言模型(LLM)分类器防护层,在生产流量中以仅 0.38% 的误拒绝率增量阻断超过 95% 的通用越狱(jailbreak)攻击。
[Shazeer et al. 2017] Shazeer, Mirhoseini, Maziarz, Davis, Le, Hinton, Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. https://arxiv.org/abs/1701.06538
本文提出稀疏门控混合专家(MoE)层,通过可训练门控网络从数千个前馈网络专家中稀疏选取,在语言建模和机器翻译任务上以极小的计算开销实现超过 1000 倍的模型容量提升。
[Shazeer 2019] Shazeer. Fast transformer decoding: One write-head is all you need. https://arxiv.org/abs/1911.02150
多查询注意力(MQA)在所有注意力头之间共享键和值,大幅降低增量解码的内存带宽需求,且质量损失极小。
[Shazeer 2020] Shazeer. GLU variants improve transformer. https://arxiv.org/abs/2002.05202
本文提出将 SwiGLU、GEGLU 等 GLU 变体用于 Transformer 前馈网络(FFN)子层,替换 ReLU 激活函数,实验表明这些变体可降低困惑度并提升下游任务性能。
[Shehabi et al. 2024] Shehabi, Smith, Hubbard, Newkirk, Lei, Siddik, Holecek, Koomey, Masanet, Sartor. 2024 united states data center energy usage report. https://escholarship.org/uc/item/32d6m0d1
该报告估算了美国数据中心的用电量,并根据设备、利用率和设施效率方面的明确假设预测未来需求。
[Sheng et al. 2023] Sheng, Cao, Li, Hooper, Lee, Yang, Chou, Zhu, Zheng, Keutzer, Gonzalez, Stoica. S-LoRA: Serving thousands of concurrent LoRA adapters. arXiv preprint arXiv:2311.03285. https://arxiv.org/abs/2311.03285
S-LoRA 在共享基础模型权重上服务大量 LoRA 适配器,并统一分页和调度适配器内存与 KV 缓存。
[Sheng et al. 2025] Sheng, Zhang, Ye, Wu, Zhang, Zhang, Peng, Lin, Wu. HybridFlow: a flexible and efficient RLHF framework. Association for Computing Machinery. https://arxiv.org/abs/2409.19256
HybridFlow 提供分布式 RLHF 数据流,并用 3D-HybridEngine 在共享资源池中把同一 actor 重分片到生成与训练布局。
[Shi et al. 2017] Shi, Karpathy, Fan, Hernandez, Liang. World of bits: An open-domain platform for web-based agents. https://proceedings.mlr.press/v70/shi17a.html
GUI 智能体研究线的起点:智能体感知像素与 DOM、用鼠标键盘行动,并附带此后多年供整个领域训练用的 MiniWoB 任务集。
[Shi et al. 2023] Shi, Chen, Misra, Scales, Dohan, Chi, Schärli, Zhou. Large language models can be easily distracted by irrelevant context. https://proceedings.mlr.press/v202/shi23a.html
在小学算术题中加入无关句子会降低受测语言模型的准确率,说明抗干扰能力不同于上下文容量。
[Shi et al. 2024] Shi, Han, Wang, Doucet, Titsias. Simplified and generalized masked diffusion for discrete data. https://arxiv.org/abs/2406.04329
MD4 将离散数据的掩码扩散简化为连续时间 ELBO 等价于交叉熵损失的加权积分,在文本困惑度和像素级图像建模上超越了此前的离散扩散模型。
[Shi et al. 2024] Shi, Yuan, Liu, Huang, Zhou, Sun, Gong. Optimization-based prompt injection attack to LLM-as-a-judge. https://arxiv.org/abs/2403.17710
JudgeDeceiver 表明,攻击者控制的候选文本可以向语言模型评判者注入指令,并操纵 RLAIF 等评判系统的选择。
[Shi et al. 2024] Shi, Ajith, Xia, Huang, Liu, Blevins, Chen, Zettlemoyer. Detecting pretraining data from large language models. https://proceedings.iclr.cc/paper_files/paper/2024/hash/e32ad85fa27be4a9868d55703f01323e-Abstract-Conference.html
本文形式化黑盒预训练数据检测,并在受控与真实数据设置上评测 Min-K% Prob;它提供暴露证据,而不是未暴露证明。
[Shinn et al. 2023] Shinn, Cassano, Gopinath, Narasimhan, Yao. Reflexion: language agents with verbal reinforcement learning. https://proceedings.neurips.cc/paper_files/paper/2023/hash/1b44b878bb782e6954cd888628510e90-Abstract-Conference.html
Reflexion 把任务反馈转成保存在情节记忆中的文字反思,通过上下文而非参数更新来改变后续决策。
[Shmueli et al. 2021] Shmueli, Fell, Ray, Ku. Beyond fair pay: Ethical implications of NLP crowdsourcing. Association for Computational Linguistics. https://aclanthology.org/2021.naacl-main.295/
作者说明众包工作的伦理问题不止时薪,还包括隐私、可识别数据、有害内容暴露、同意和现有人体研究审查框架的局限。
[Shoeybi et al. 2019] Shoeybi, Patwary, Puri, LeGresley, Casper, Catanzaro. Megatron-LM: Training multi-billion parameter language models using model parallelism. https://arxiv.org/abs/1909.08053
Megatron-LM 提出 Transformer 训练的层内张量并行方案,并测量其通信与扩展行为。
[Shokri et al. 2017] Shokri, Stronati, Song, Shmatikov. Membership inference attacks against machine learning models. https://arxiv.org/abs/1610.05820
本文提出成员推断攻击,利用影子训练技术,仅通过黑盒 API 访问即可判断某条记录是否属于模型的训练数据集。
[Shumailov et al. 2024] Shumailov, Shumaylov, Zhao, Papernot, Anderson, Gal. AI models collapse when trained on recursively generated data. Nature. https://www.nature.com/articles/s41586-024-07566-y
在大语言模型自身合成输出上递归训练会导致模型崩塌,跨代累积后输出分布退化为噪声。
[Si and others 2025] Si, others. Collective communication for 100k+ gpus. arXiv preprint arXiv:2510.20171. https://arxiv.org/abs/2510.20171
NCCLX 是 Meta 为 Llama4 开发的集合通信框架,基于 NCCL 扩展,支持 100K+ GPU 的零拷贝、SM-free 传输、容错 AllReduce 及面向混合专家(MoE)推理的 GPU 常驻集合操作。
[Singh and Strouse 2024] Singh, Strouse. Tokenization counts: The impact of tokenization on arithmetic in frontier llms. https://arxiv.org/abs/2402.14903
对前沿模型的实验表明,数字令牌的方向和分组方式会改变算术准确率与错误结构。
[Singh et al. 2025] Singh, Rastegar, Bansal, Choukse, Goiri, Ramjee, Fonseca. Efficiently serving large multimodal models using EPD disaggregation. PMLR. https://proceedings.mlr.press/v267/singh25d.html
EPD 服务把多模态编码、语言模型预填充和解码放到可独立扩缩的资源上,以提高良好吞吐量。
[Sinha et al. 2026] Sinha, Arun, Goel, Staab, Geiping. The illusion of diminishing returns: Measuring long horizon execution in llms. https://arxiv.org/abs/2509.09677
本文指出短任务基准上的边际收益递减掩盖了长时域执行长度的指数级提升,并发现了一种"自我条件化"失效模式:大语言模型在上下文包含历史错误时性能显著下降。
[SK hynix 2025] SK hynix. <span class="nocase">SK hynix</span> completes world's first HBM4 development and readies mass production. https://news.skhynix.com/sk-hynix-completes-worlds-first-hbm4-development-and-readies-mass-production/
SK hynix 记录其 HBM4 实现,包括翻倍的接口宽度;这些数字仅适用于该产品。
[SLSA Community 2025] SLSA Community. SLSA v1.2 build provenance. https://slsa.dev/spec/v1.2/build-provenance
SLSA 构建来源证明把经过认证的构建者、构建定义、解析后的输入和运行细节声明绑定到输出产物摘要;消费者仍须按预期验证这些声明。
[SLSA Community 2025] SLSA Community. SLSA specification. https://slsa.dev/spec/v1.2/
SLSA 为软件构建和分发流水线定义了来源证明要求与保障等级。
[Snell et al. 2025] Snell, Lee, Xu, Kumar. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. https://arxiv.org/abs/2408.03314
在所研究的 PaLM 2 模型、MATH 数据集、过程奖励模型与离线难度估计条件下,最优测试时策略取决于题目难度和预算。
[Sohl-Dickstein et al. 2015] Sohl-Dickstein, Weiss, Maheswaranathan, Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. PMLR. https://proceedings.mlr.press/v37/sohl-dickstein15.html
本文提出扩散概率模型,通过学习逐步破坏数据结构的前向扩散过程的逆过程,构建兼具灵活性与可处理性的生成模型。
[Soldaini et al. 2024] Soldaini, Kinney, Bhagia, Schwenk, Atkinson, Authur, Bogin, Chandu, Dumas, Elazar, Hofmann, Jha, Kumar, Lucy, Lyu, Lambert, Magnusson, Morrison, Muennighoff, Naik, Nam, Peters, Ravichander, Richardson, Shen, Strubell, Subramani, Tafjord, Walsh, Zettlemoyer, Smith, Hajishirzi, Beltagy, Groeneveld, Dodge, Lo. Dolma: an open corpus of three trillion tokens for language model pretraining research. Association for Computational Linguistics. https://aclanthology.org/2024.acl-long.840/
Dolma 是一个开放的三万亿词元英语预训练语料库,来源涵盖网页、科学论文、代码、图书、社交媒体和百科内容,并附完整文档和数据策划工具包。
[Song and Ermon 2019] Song, Ermon. Generative modeling by estimating gradients of the data distribution. https://arxiv.org/abs/1907.05600
Song 和 Ermon 提出噪声条件分数网络(NCSN),通过在多个噪声水平上估计数据分布梯度,并结合退火 Langevin 动力学进行采样,在 CIFAR-10 上达到当时最优 inception score 8.87。
[Song et al. 2021] Song, Sohl-Dickstein, Kingma, Kumar, Ermon, Poole. Score-based generative modeling through stochastic differential equations. https://arxiv.org/abs/2011.13456
本文通过连续时间随机微分方程(SDE)框架统一了基于分数的生成模型与去噪扩散概率模型(DDPM),实现了精确似然计算、可控生成,并在 CIFAR-10 上达到当时最优的图像合成性能。
[Song et al. 2021] Song, Meng, Ermon. Denoising diffusion implicit models. https://arxiv.org/abs/2010.02502
DDIM 将 DDPM 的马尔可夫前向过程替换为非马尔可夫过程,无需重新训练即可实现 10 至 50 倍的采样加速。
[Song et al. 2023] Song, Dhariwal, Chen, Sutskever. Consistency models. PMLR. https://proceedings.mlr.press/v202/song23a.html
把去噪轨迹上任意一点直接映射到其起点的模型,支持一步生成并可选多步细化,既可蒸馏训练也可从头训练。
[Song et al. 2025] Song, Zhang, Luo, Gao, Xia, Luo, Li, others. Seed diffusion: a large-scale diffusion language model with high-speed inference. arXiv preprint arXiv:2508.02193. https://arxiv.org/abs/2508.02193
Seed Diffusion 将分块生成与 KV 缓存结合,并报告在 H20 硬件上达到每秒 2146 个令牌,但没有说明硬件数量和模型大小。
[Souly et al. 2024] Souly, Lu, Bowen, Trinh, Hsieh, Pandey, Abbeel, Svegliato, Emmons, Watkins, Toyer. A StrongREJECT for empty jailbreaks. arXiv preprint arXiv:2402.10260. https://arxiv.org/abs/2402.10260
StrongREJECT 评估越狱响应是否真正提供有用的违禁信息,并显示常见评判方法可能夸大攻击成功率。
[Souly et al. 2025] Souly, Rando, Chapman, Davies, Hasircioglu, Shereen, Mougan, Mavroudis, Jones, Hicks, Carlini, Gal, Kirk. Poisoning attacks on LLMs require a near-constant number of poison samples. CoRR. https://arxiv.org/abs/2510.07192
在所测试规模内,250 篇投毒文档对狭窄的“触发器到乱码”目标取得相近成功率;结论受该攻击与训练设置约束。
[Souppaya et al. 2017] Souppaya, Morello, Scarfone. Application container security guide. https://csrc.nist.gov/pubs/sp/800/190/final
NIST SP 800-190 介绍容器镜像、仓库、编排器、运行时、宿主机和网络相关风险及其缓解措施。
[Souppaya et al. 2017] Souppaya, Morello, Scarfone. Application container security guide. https://doi.org/10.6028/NIST.SP.800-190
NIST 介绍了容器特有的威胁,并建议在镜像、注册表、编排器、宿主机和运行时各层采用防护措施。
[Spangher et al. 2025] Spangher, Pasumarthi, Masiewicki, Arnold, Kaushal, Johnson, Grabowski, Ie. RLHF algorithms ranked: An extensive evaluation across diverse tasks, rewards, and hyperparameters. Association for Computational Linguistics. https://aclanthology.org/2025.emnlp-industry.35/
一项广泛的实证比较发现,对齐方法的排名取决于任务、奖励和超参数选择,并不存在一个普遍的赢家。
[SPIFFE Project 2026] SPIFFE Project. SPIFFE identity and verifiable identity document. https://spiffe.io/docs/latest/spiffe-specs/spiffe-id/
SPIFFE 负责识别和认证工作负载;应用仍需另行处理用户委托和授权策略。
[Sporny et al. 2025] Sporny, Longley, Chadwick, Herman. Verifiable credentials data model v2.0. https://www.w3.org/TR/vc-data-model-2.0/
该数据模型规范凭证结构与验证关系,同时明确把声明真伪与是否采信留给验证方策略决定。
[Squire 2004] Squire. Memory systems of the brain: a brief history and current perspective. Neurobiology of Learning and Memory. https://pubmed.ncbi.nlm.nih.gov/15464402/
这篇综述区分多种生物记忆系统,并阐明常见认知标签背后的陈述性与非陈述性结构。
[Staab et al. 2024] Staab, Vero, Balunović, Vechev. Beyond memorization: Violating privacy via inference with large language models. https://proceedings.iclr.cc/paper_files/paper/2024/hash/9028b8a3ca98f58e373f0c1497a17448-Abstract-Conference.html
大语言模型能以接近人类分析员的精度、远低于人力的成本,从寻常文本推断位置、收入与人口属性;记忆存储所持久化的,正是这种持续运行的推断。
[Stanovich and West 2000] Stanovich, West. Individual differences in reasoning: Implications for the rationality debate?. Behavioral and Brain Sciences.
Stanovich 与 West 提出双过程推理的系统 1 与系统 2 术语,并分析了人类理性中的个体差异。
[Starace et al. 2025] Starace, Jaffe, Sherburn, Aung, Chan, Maksin, Dias, Mays, Kinsella, Thompson, Heidecke, Glaese, Patwardhan. PaperBench: Evaluating AI's ability to replicate AI research. arXiv preprint arXiv:2504.01848. https://arxiv.org/abs/2504.01848
PaperBench 将长时程研究复现任务分解为分层评分项,并在独立的裁判基准上评估自动裁判。
[Stiennon et al. 2020] Stiennon, Ouyang, Wu, Ziegler, Lowe, Voss, Radford, Amodei, Christiano. Learning to summarize from human feedback. https://arxiv.org/abs/2009.01325
该研究根据成对人工判断训练摘要奖励模型,再用 PPO 优化策略,并直接测量学习奖励与人类偏好开始背离的位置。
[Stivers et al. 2009] Stivers, Enfield, Brown, Englert, Hayashi, Heinemann, Hoymann, Rossano, Ruiter, Yoon, Levinson. Universals and cultural variation in turn-taking in conversation. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.0903616106
对十种语言的是非问答分析表明,回应通常集中在前一轮话语结束附近,但会随语言和回应类型显著变化。
[Su et al. 2023] Su, Shi, Kasai, Wang, Hu, Ostendorf, Yih, Smith, Zettlemoyer, Yu. One embedder, any task: Instruction-finetuned text embeddings. https://aclanthology.org/2023.findings-acl.71/
INSTRUCTOR 是一个在推理时接受任务指令的文本嵌入模型,无需进一步微调即可生成任务感知与领域感知的嵌入向量,在 70 个多样化任务上达到最优性能。
[Su et al. 2024] Su, Lu, Pan, Murtadha, Wen, Liu. RoFormer: Enhanced transformer with rotary position embedding. Neurocomputing. https://arxiv.org/abs/2104.09864
RoFormer 提出 RoPE(旋转位置嵌入),在自注意力中将词元位置编码为旋转矩阵,同时实现序列长度灵活性和随相对距离衰减的词元间依赖。
[Subramanya et al. 2019] Subramanya, Devvrit, Simhadri, Krishnawamy, Kadekodi. DiskANN: Fast accurate billion-point nearest neighbor search on a single node. https://proceedings.neurips.cc/paper_files/paper/2019/hash/09853c7fb1d3f8ee67a61b6bf4a7f8e6-Abstract.html
该论文介绍了一种感知 SSD 的图索引,用于十亿级数据点的高召回最近邻搜索。
[Sun 2025] Sun. Why did M2 end up as a full attention model?. https://www.minimax.io/news/why-did-m2-end-up-as-a-full-attention-model
MiniMax 预训练负责人解释 M2 为何放弃 lightning attention 混合设计:混合体的短板要到规模上去后才在多跳推理上显形,而高效注意力周边的推理与评估栈尚未达到生产成熟度。
[Sun et al. 2026] Sun, Zhan, Shen, Yu, Zhang, He, Xu. When personalization misleads: Understanding and mitigating hallucinations in personalized llms. https://aclanthology.org/2026.findings-acl.395/
该研究衡量用户画像何时使事实回答偏离证据,转向个性化但错误的结论。
[Sutton 1988] Sutton. Learning to predict by the methods of temporal differences. Machine Learning.
Sutton 提出时序差分学习,依据相邻预测估计之间的差异来更新预测,而非等待最终结果。
[Sutton and Barto 2018] Sutton, Barto. Reinforcement learning: An introduction. The MIT Press. https://mitpress.mit.edu/9780262039246/reinforcement-learning/
Sutton 与 Barto 将智能体形式化为随时间与环境交互的策略,并系统阐述了情节式与持续式任务的回报评估。
[Swanson et al. 2010] Swanson, Bowen, Phillips, Gallup, Lynes. Contingency planning guide for federal information systems. https://doi.org/10.6028/NIST.SP.800-34r1
NIST 介绍了信息系统的应急规划、恢复优先级、备用处理、测试和恢复目标。
[Szabo 1999] Szabo. Micropayments and mental transaction costs. https://nakamotoinstitute.org/library/micropayments-and-mental-transaction-costs/
Szabo 论证,注意力、偏好设定、监控与争议成本可能超过小额购买本身的价格,即使由软件智能体执行也一样。
[Tabassi 2023] Tabassi. Artificial intelligence risk management framework (AI RMF 1.0). https://doi.org/10.6028/NIST.AI.100-1
NIST AI RMF 1.0 将 AI 风险管理组织为治理、映射、度量和管理四类功能,贯穿 AI 生命周期。
[Taiwan Semiconductor Manufacturing Company 2025] Taiwan Semiconductor Manufacturing Company. TSMC unveils next-generation A14 process at north america technology symposium. https://pr.tsmc.com/english/news/3228
台积电 2025 年技术研讨会公告同时列出 HBM4 的 N12 和 N3 逻辑基础裸片方案,说明基础裸片制程取决于具体产品。
[Taiwan Semiconductor Manufacturing Company 2026] Taiwan Semiconductor Manufacturing Company. TSMC debuts A13 technology at 2026 north america technology symposium. https://pr.tsmc.com/english/news/3302
台积电区分了已投产的 5.5 倍光罩尺寸 CoWoS、计划 2028 年推出的 14 倍光罩版本,以及计划 2029 年进行的进一步扩展。
[Taiwan Semiconductor Manufacturing Company 2026] Taiwan Semiconductor Manufacturing Company. Fab capacity. https://www.tsmc.com/english/dedicatedFoundry/manufacturing/fab_capacity
台积电列出了台湾的晶圆厂,以及美国、日本和中国的子公司或设施,并区分了各自的晶圆尺寸和公司性质。
[Tang et al. 2024] Tang, Zhao, Zhu, Xiao, Kasikci, Han. QUEST: Query-aware sparsity for efficient long-context LLM inference. PMLR. https://proceedings.mlr.press/v235/tang24l.html
Quest 保存逐页键极值,并用每次查询选择注意力读取的前 K 个 KV 页,同时保留完整缓存。
[Tao et al. 2024] Tao, Liu, Dou, Muennighoff, Wan, Luo, Lin, Wong. Scaling laws with vocabulary: Larger models deserve larger vocabularies. https://proceedings.neurips.cc/paper_files/paper/2024/hash/cf5a019ae9c11b4be88213ce3f85d85c-Abstract-Conference.html
受控实验对词表相关参数与文本压缩之间的取舍建模,发现计算最优词表大小会随模型规模增长。
[Tazi et al. 2025] Tazi, Mom, Zhao, Nguyen, Mekkouri, Werra, Wolf. The ultra-scale playbook: Training llms on GPU clusters. https://huggingface.co/spaces/nanotron/ultrascale-playbook
Hugging Face Nanotron 团队的专门指南,聚焦在大型 GPU 集群上训练大语言模型,并提供 PDF 与交互网页版本。
[Templeton et al. 2024] Templeton, Conerly, Marcus, Lindsey, Bricken, Chen, Pearce, Citro, Ameisen, Jones, Cunningham, Turner, McDougall, MacDiarmid, Tamkin, Durmus, Hume, Mosconi, Freeman, Sumers, Rees, Batson, Jermyn, Carter, Olah, Henighan. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet. https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html
该研究在 Claude 3 Sonnet 上训练稀疏自动编码器,并分析提取出的特征、特征引导和特征解释的局限。
[Temporal Technologies 2026] Temporal Technologies. Activity definition. https://docs.temporal.io/activity-definition
Temporal 会记录已完成活动以供重放,但未能报告完成的活动可能被重试并执行多次。
[Temporal Technologies 2026] Temporal Technologies. Cancellation: Python SDK. https://docs.temporal.io/develop/python/workflows/cancellation
Temporal 区分工作流的优雅取消与强制终止,并说明取消远程活动所需的心跳机制。
[Temporal Technologies 2026] Temporal Technologies. What is a Temporal retry policy?. https://docs.temporal.io/encyclopedia/retry-policies
Temporal 文档介绍声明式重试策略、次数与超时限制,以及可重试的暂时性故障和永久性故障之间的区别。
[Temporal Technologies 2026] Temporal Technologies. Temporal workflow. https://docs.temporal.io/workflows
Temporal 把确定性的工作流重放与外部活动分开,并持久保存事件历史以支持可靠恢复。
[TensorFlow Authors 2026] TensorFlow Authors. Introduction to gradients and automatic differentiation. https://www.tensorflow.org/guide/autodiff
TensorFlow 文档解释了梯度带、受监视变量、高阶导数、雅可比矩阵,以及无法获得梯度的情况。
[TensorFlow Authors 2026] TensorFlow Authors. Better performance with <span class="nocase">tf.function</span>. https://www.tensorflow.org/guide/function
该指南解释追踪、ConcreteFunction 缓存、输入特化、重新追踪与追踪期 Python 副作用。
[Thakur et al. 2021] Thakur, Reimers, Rücklé, Srivastava, Gurevych. BEIR: a heterogeneous benchmark for zero-shot evaluation of information retrieval models. https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/hash/65b9eea6e1cc6bb9f0cd2a47751a186f-Abstract-round2.html
BEIR 在异构检索数据集上比较词法、稀疏、稠密、后期交互与重排系统,揭示明显的领域相关权衡。
[Theano Development Team 2016] Theano Development Team. Theano: a python framework for fast computation of mathematical expressions. https://arxiv.org/abs/1605.02688
Theano 把数学表达式表示为图,进行符号求导,并编译为经过优化的 CPU 或 GPU 函数。
[Thorne et al. 2018] Thorne, Vlachos, Christodoulopoulos, Mittal. FEVER: a large-scale dataset for fact extraction and VERification. Association for Computational Linguistics. https://aclanthology.org/N18-1074/
FEVER 把 claim 与固定维基百科快照中的证据配对,并标记为支持、反驳或证据不足,从而把缺少证据与错误区分开。
[Thurgood and Ferguson 2018] Thurgood, Ferguson. Implementing slos. O'Reilly Media. https://sre.google/workbook/implementing-slos/
Google SRE 工作手册把服务级指标和目标转化为与用户旅程、测量及审查相联系的实施流程。
[Thurgood 2018] Thurgood. Alerting on slos. O'Reilly Media. https://sre.google/workbook/alerting-on-slos/
Google SRE 工作手册根据错误预算消耗率设计告警,使值班通知同时反映 SLO 威胁的规模和持续时间。
[Tian et al. 2024] Tian, Jiang, Yuan, Peng, Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. https://arxiv.org/abs/2404.02905
VAR 将图像自回归重新定义为由粗到细的"下一尺度预测",使 GPT 风格的自回归模型在图像质量、速度和扩展律上首次超越扩散 Transformer。
[Tillet et al. 2019] Tillet, Kung, Cox. Triton: An intermediate language and compiler for tiled neural network computations. https://dl.acm.org/doi/10.1145/3315508.3329973
Triton 把静态形状的分块(tile)作为 GPU 编程的基本单元:程序员为每个分块写一个程序,合并访存、共享内存与处理器内调度交给编译器。
[Tobin et al. 2017] Tobin, Fong, Ray, Schneider, Zaremba, Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. https://arxiv.org/abs/1703.06907
域随机化通过多样化的模拟渲染训练模型,减少其对单一模拟器外观的依赖,再迁移到真实图像。
[Torres-Arias et al. 2019] Torres-Arias, Afzali, Kuppusamy, Curtmola, Cappos. in-toto: Providing farm-to-table guarantees for bits and bytes. https://www.usenix.org/conference/usenixsecurity19/presentation/torres-arias
in-toto 记录并验证软件供应链中获得授权的步骤、参与者和产物。
[Touvron et al. 2023] Touvron, Martin, Stone, Albert, Almahairi, Babaei, Bashlykov, Batra, Bhargava, Bikel, others. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. https://arxiv.org/abs/2307.09288
Llama 2 描述了跨模型版本和解码温度的偏好收集,并随着新策略改变回复分布而持续补充数据。
[Trail of Bits 2024] Trail of Bits. A few notes on AWS nitro enclaves: Images and attestation. https://blog.trailofbits.com/2024/02/16/a-few-notes-on-aws-nitro-enclaves-images-and-attestation/
从从业者视角分析 Nitro Enclave 镜像可复现性、PCR 解读、证明与中心化信任假设。
[Triton Project 2026] Triton Project. Triton programming guide: Introduction. https://triton-lang.org/main/programming-guide/chapter-1/introduction.html
该指南定义 Triton 的分块程序模型,以及编译器在局部性、调度、合并访存、向量化与目标指令选择中的职责。
[Trivedi et al. 2023] Trivedi, Balasubramanian, Khot, Sabharwal. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. https://aclanthology.org/2023.acl-long.557/
IRCoT 交错执行检索与中间推理,使后续搜索能够依赖先前找到的证据。
[TSMC 2026] TSMC. CoWoS: Chip-on-wafer-on-substrate. https://www.tsmc.com/english/dedicatedFoundry/technology/cowos
TSMC 说明 CoWoS 如何通过中介层与封装基板集成逻辑小芯片和高带宽内存。
[Tung et al. 2026] Tung, Huang, Saxena, Shirvani, Hukerikar, Jain, Tyagi, Gongalore. The anatomy of silent data corruption: GPU error pattern study and modeling guidance. https://arxiv.org/abs/2605.04213
通过对生产级 GPU 进行门级故障注入,系统表征静默数据损坏(SDC)规律,发现 NaN/INF 仅占损坏输出的 1.01%,而置零(nullification)占主导达 50.68%。
[Turpin et al. 2023] Turpin, Michael, Perez, Bowman. Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting. https://proceedings.neurips.cc/paper_files/paper/2023/hash/ed3fea9033a80fea1376299fa7863f4a-Abstract.html
实验改变会影响答案的偏置信号,发现思维链解释经常省略这些真正起因。
[U.S. Bureau of Industry and Security 2026] U.S. Bureau of Industry and Security. Revision to license review policy for advanced computing commodities. https://www.federalregister.gov/documents/2026/01/15/2026-00789/revision-to-license-review-policy-for-advanced-computing-commodities
BIS 把面向中国与澳门、总处理性能低于 21,000 且总 DRAM 带宽低于 6,500 GB/s 的先进计算部件(如 NVIDIA H200、AMD MI325X)出口,从推定拒绝改为逐案许可审查,条件包括出口方证明、由美国境内独立第三方测试,以及出口总处理性能合计不超过面向美国客户出货量的 50%。
[U.S. Department of Commerce 2025] U.S. Department of Commerce. U.S. department of commerce announces $1.4 billion in final awards to support the next generation of U.S. semiconductor advanced packaging. https://www.commerce.gov/node/7087
美国商务部公布了先进封装研究与试点项目的拨款,包括基板计划,以及一座衔接实验室研究与规模化生产的设施。
[U.S. Department of Energy 2024] U.S. Department of Energy. Large power transformer resilience. https://www.energy.gov/sites/default/files/2024-10/EXEC-2022-001242%20-%20Large%20Power%20Transformer%20Resilience%20Report%207-10-24.pdf
美国能源部评估了大型电力变压器在供应、制造、运输、更换和韧性方面的风险。
[U.S. Department of Energy 2025] U.S. Department of Energy. Energy department closes loan to restart nuclear power plant in pennsylvania. https://www.energy.gov/articles/energy-department-closes-loan-restart-nuclear-power-plant-pennsylvania
美国能源部记录了联邦贷款的交割情况,以及重启前三里岛一号机组所附带的条件。
[U.S. Department of Justice and Federal Trade Commission 2023] U.S. Department of Justice, Federal Trade Commission. 2023 merger guidelines. https://www.justice.gov/atr/merger-guidelines
该指南先界定市场再计算份额,允许采用符合竞争现实的指标,并在美国并购审查中把 HHI 阈值用作结构性推定。
[U.S. Nuclear Regulatory Commission 2026] U.S. Nuclear Regulatory Commission. Christopher M. Crane clean energy center. https://www.nrc.gov/info-finder/reactors/ccec
美国核管理委员会页面记录了 Christopher M. Crane 清洁能源中心计划重启所涉及的许可和监管信息。
[UALink Consortium 2025] UALink Consortium. UALink 200G 1.0 specification. https://ualinkconsortium.org/specification/
UALink 1.0 规定交换式加速器 scale-up 接口,而实现与部署证据仍需由厂商提供。
[Uesato et al. 2022] Uesato, Kushman, Kumar, Song, Siegel, Wang, Creswell, Irving, Higgins. Solving math word problems with process- and outcome-based feedback. arXiv preprint arXiv:2211.14275. https://arxiv.org/abs/2211.14275
本文在 GSM8K 上对比过程奖励模型(PRM)与结果监督方法,发现 PRM 将推理过程错误率从 14.0% 降至 3.4%。
[United States District Court for the District of Delaware 2025] United States District Court for the District of Delaware. Thomson reuters enterprise centre GmbH v. ross intelligence inc., no. 1:20-cv-613, memorandum opinion. https://www.ded.uscourts.gov/sites/ded/files/opinions/20-613_5.pdf
Ross Intelligence 案判决把美国版权合理使用分析用于复制法律摘要并创建竞争性法律研究产品的行为。
[United States District Court for the Northern District of California 2025] United States District Court for the Northern District of California. Kadrey v. meta platforms, inc., order on cross-motions for summary judgment. https://caselaw.findlaw.com/court/us-dis-crt-n-d-cal/117422847.html
Kadrey 案裁定书根据所呈证据评估合理使用,并说明市场损害和原告举证为何会影响结果。
[Universal Commerce Protocol 2026] Universal Commerce Protocol. Universal commerce protocol specification overview. https://ucp.dev/2026-04-08/specification/overview/
UCP 发布商家能力并协调广泛的商业生命周期,通过可选组合避免强制依赖某一种身份或支付协议。
[Van Bulck et al. 2018] Van Bulck, Minkin, Weisse, Genkin, Kasikci, Piessens, Silberstein, Wenisch, Yarom, Strackx. Foreshadow: Extracting the keys to the intel SGX kingdom with transient out-of-order execution. https://www.usenix.org/conference/usenixsecurity18/presentation/bulck
利用瞬态执行提取出 SGX 自身证明密钥的攻击,重置了整个领域的预期:可信执行边界是工程造物,而非数学证明。
[Vaswani et al. 2017] Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, Polosukhin. Attention is all you need. https://arxiv.org/abs/1706.03762
Vaswani 等人提出 Transformer,一种完全基于多头注意力、无循环和卷积的序列转换架构,以更高并行性实现了当时最优的机器翻译质量。
[Villalobos et al. 2024] Villalobos, Ho, Sevilla, Besiroglu, Heim, Hobbhahn. Position: Will we run out of data? Limits of LLM scaling based on human-generated data. PMLR. https://proceedings.mlr.press/v235/villalobos24a.html
本文预测,按当前大语言模型发展趋势,训练数据需求将在 2026 至 2032 年间耗尽全部公开人类生成文本,并探讨合成数据与迁移学习作为应对策略。
[Vincent 2011] Vincent. A connection between score matching and denoising autoencoders. Neural Computation 23(7):1661–1674. https://direct.mit.edu/neco/article-abstract/23/7/1661/7677
Vincent 证明了去噪自动编码器训练与分数匹配之间的联系,把噪声下的重建与密度梯度估计联系起来。
[Visa 2026] Visa. Trusted agent protocol specifications. https://github.com/visa/trusted-agent-protocol
TAP 在 Visa 体系内定义了带签名的智能体识别、消费者和支付容器数据;签名有效并不能普遍证明权限或意图。
[vLLM 2026] vLLM. Batch invariance. https://docs.vllm.ai/en/latest/features/batch_invariance/
vLLM 记录了一种服务模式,其受支持内核能在固定环境中使输出不受组批方式影响。
[vLLM Multimodal Workstream 2025] vLLM Multimodal Workstream. Encoder disaggregation for scalable multimodal model serving. https://vllm.ai/blog/2025-12-15-vllm-epd
vLLM 原生的编码/预填充/解码分离自 v0.11.1 起可用,可独立扩缩视觉编码器池,并在四张 A100 上的 Qwen3-VL 基准中报告约 2 至 2.5 倍的有效吞吐量。
[vLLM Project 2026] vLLM Project. Encoder cache manager. https://docs.vllm.ai/en/stable/api/vllm/v1/core/encoder_cache_manager/
vLLM 编码器缓存管理器独立于语言模型 KV 缓存,管理多模态编码器输出的容量、分配和淘汰。
[Wallace et al. 2024] Wallace, Xiao, Leike, Weng, Heidecke, Beutel. The instruction hierarchy: Training llms to prioritize privileged instructions. arXiv preprint arXiv:2404.13208. https://arxiv.org/abs/2404.13208
指令层级训练模型在冲突中选择性忽略低权限指令,从而提高对提示注入与越狱的鲁棒性,同时尽量减少能力损失。
[Wan et al. 2024] Wan, Feng, Wen, Mcaleer, Wen, Zhang, Wang. AlphaZero-like tree-search can guide large language model decoding and training. PMLR. https://arxiv.org/abs/2309.17179
TS-LLM 将 AlphaZero 式树搜索与学习到的价值函数相结合,在推理、规划、RLHF 对齐和决策任务中同时引导大语言模型的解码与迭代训练。
[Wan and others 2025] Wan, others. Robust LLM training infrastructure at ByteDance. https://arxiv.org/abs/2509.16293
ByteRobust 是字节跳动面向大规模 LLM 训练的生产级容错基础设施,通过自动化故障检测与恢复在 9600 张 GPU 三个月训练任务中实现了 97% 的有效训练时间比(ETTR)。
[Wan et al. 2025] Wan, Klyman, Kapoor, Maslej, Longpre, Xiong, Liang, Bommasani. The 2025 foundation model transparency index. https://arxiv.org/abs/2512.10169
2025 年指数报告开发者透明度均分为 40.69,并提醒指标和样本变化限制了与 2024 年结果的直接比较。
[Wang et al. 2019] Wang, Yu, Sun, Chen, Yu. Improving pre-trained multilingual model with vocabulary expansion. Association for Computational Linguistics. https://aclanthology.org/K19-1030/
该论文扩展预训练多语言模型的词表,并适配新增令牌的参数,而不是从头训练整个模型。
[Wang et al. 2019] Wang, He, Wen, Tan. Truly proximal policy optimization. arXiv preprint arXiv:1903.07940. https://arxiv.org/abs/1903.07940
分析表明,PPO 的裁剪代理目标并不严格限制似然比,也不强制形成信赖域,因此促成了一种基于回滚的变体。
[Wang et al. 2020] Wang, Cho, Gu. Neural machine translation with byte-level subwords. https://ojs.aaai.org/index.php/AAAI/article/view/6451
该论文在 UTF-8 字节上训练子词词表,既保证不会出现未知字符,又保留学习得到的多字节片段。
[Wang and Isola 2020] Wang, Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. https://proceedings.mlr.press/v119/wang20k.html
本文将对比表示学习的成功归因于单位超球面上的对齐性与均匀性两个关键属性,证明对比损失渐近地优化这两者,并表明直接优化这两个可量化指标可达到与对比学习相当甚至更好的下游任务性能。
[Wang et al. 2022] Wang, Yang, Huang, Jiao, Yang, Jiang, Majumder, Wei. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533. https://arxiv.org/abs/2212.03533
E5 通过在精心筛选的 2.7 亿网络文本对数据集 CCPairs 上进行对比预训练,获得通用文本嵌入,在 BEIR 和 MTEB 基准上达到最先进水平。
[Wang et al. 2023] Wang, Chen, Wu, Zhang, Zhou, Liu, others. Neural codec language models are zero-shot text to speech synthesizers. arXiv preprint arXiv:2301.02111. https://arxiv.org/abs/2301.02111
VALL-E 将文本到语音(TTS)建模为条件编解码器语言模型任务,在 60K 小时语音数据上预训练,仅需 3 秒声学提示即可实现零样本个性化语音合成。
[Wang et al. 2023] Wang, Dong, Zeng, Adams, Sreedhar, Egert, Delalleau, Scowcroft, Kant, Swope, Kuchaiev. HelpSteer: Multi-attribute helpfulness dataset for SteerLM. arXiv preprint arXiv:2311.09528. https://arxiv.org/abs/2311.09528
HelpSteer 提供 3.7 万条回复级评分,除总体有用性外还标注正确性、连贯性、复杂度和冗长度,使训练与审计中的取舍可见。
[Wang et al. 2023] Wang, Kordi, Mishra, Liu, Smith, Khashabi, Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. Association for Computational Linguistics. https://aclanthology.org/2023.acl-long.754/
Self-Instruct 用语言模型生成、筛选并回答新任务,再用保留的样本微调模型,从而自举指令数据。
[Wang et al. 2023] Wang, Wei, Schuurmans, Le, Chi, Narang, Chowdhery, Zhou. Self-consistency improves chain of thought reasoning in language models. https://openreview.net/forum?id=1PL1NIMMrw
自洽性方法采样多条推理路径,再选择最一致的最终答案;当概率质量集中于正确答案时,它能提高结果。
[Wang et al. 2023] Wang, Jia, Zhang, Zhang, Fu, Ng, Wang. Gemini: Fast failure recovery in distributed training with in-memory checkpoints. https://doi.org/10.1145/3600006.3613145
Gemini 把检查点副本保存在主机内存中,并调度传输流量以缩短恢复时间,而不假设单一存储层能覆盖所有故障域。
[Wang et al. 2024] Wang, Gao, Zhao, Sun, Dai. Auxiliary-loss-free load balancing strategy for mixture-of-experts. https://arxiv.org/abs/2408.15664
Loss-Free Balancing 通过动态更新每个专家的路由偏置来维持混合专家(MoE)模型的负载均衡,消除辅助损失引入的干扰梯度并提升模型性能。
[Wang et al. 2024] Wang, Bai, Tan, Wang, Fan, Bai, Chen, Liu, Wang, Ge, Fan, Dang, Du, Ren, Men, Liu, Zhou, Zhou, Lin. Qwen2-VL: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191. https://arxiv.org/abs/2409.12191
Qwen2-VL 通过动态分辨率机制和多模态旋转位置嵌入(M-RoPE)构建视觉语言模型(VLM)系列(2B、8B、72B),支持任意分辨率的图像与视频理解,在多模态基准上与 GPT-4o 性能相当。
[Wang and others 2024] Wang, others. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869. https://arxiv.org/abs/2409.18869
Emu3 用单个 8B 参数 Transformer 对图像、视频和文本的离散词元做下一词元预测,无需扩散模型或 CLIP,性能媲美 SDXL 与 LLaVA-1.6。
[Wang et al. 2024] Wang, Chen, Yuan, Zhang, Li, Peng, Ji. Executable code actions elicit better LLM agents. PMLR. https://proceedings.mlr.press/v235/wang24h.html
CodeAct 以可执行 Python 作为动作表示,并在其评测模型与基准上报告了相对比较格式最高 20 个百分点的成功率提升。
[Wang et al. 2024] Wang, Yang, Huang, Yang, Majumder, Wei. Improving text embeddings with large language models. https://aclanthology.org/2024.acl-long.642/
E5-Mistral-7B 通过对 Mistral-7B 在覆盖 93 种语言的 LLM 合成数据上进行对比学习微调,以不到 1k 步训练在 BEIR 和 MTEB 上取得最优文本嵌入表示效果。
[Wang et al. 2024] Wang, Li, Chen, Cai, Zhu, Lin, Cao, Kong, Liu, Liu, Sui. Large language models are not fair evaluators. Association for Computational Linguistics. https://aclanthology.org/2024.acl-long.511/
作者展示大语言模型评测中的位置敏感判决并测试校准框架,说明回答顺序不变性必须测量,不能只靠提示要求。
[Wang et al. 2025] Wang, Wang, Wang, Zhang, Li, Yang, Yu, Nguyen, Liu, Gottlieb, Lam, Lu, Cho, Wu, Fei-Fei, Wang, Choi, Li. RAGEN: Understanding self-evolution in LLM agents via multi-turn reinforcement learning. arXiv preprint arXiv:2504.20073. https://arxiv.org/abs/2504.20073
RAGEN 在风格化环境中研究轨迹级多轮强化学习,并报告了奖励方差骤降、熵下降和梯度尖峰相伴的崩溃模式。
[Wang et al. 2026] Wang, Li, Mang, Cheung, Sen, Song. Do androids dream of breaking the game? Systematically auditing AI agent benchmarks with BenchJack. https://arxiv.org/abs/2605.12673
BenchJack 审计十个智能体基准,在八类问题中发现 219 个缺陷,并合成奖励投机攻击,使大多数基准在未完成预期任务的情况下获得接近满分。
[Wang et al. 2026] Wang, Gao, Wang, Liu, Sun, Cheng, Shi, Du, Li. MCPTox: a benchmark for tool poisoning on real-world MCP servers. Proceedings of the AAAI Conference on Artificial Intelligence. https://ojs.aaai.org/index.php/AAAI/article/view/40895
MCPTox 基于来自 45 个真实 MCP 服务器工具集的 353 个真实工具构造 1,348 个恶意描述符案例,并评估 20 种智能体设置;它并不意味着 45 个线上部署已遭入侵。
[Warner and Davidovič 2018] Warner, Davidovič. Canarying releases. O'Reilly Media. https://sre.google/workbook/canarying-releases/
SRE 工作手册把灰度发布定义为与对照组比较的局部、限时部署,并说明流量、指标、隔离和回滚要求。
[Wasserstein and Lazar 2016] Wasserstein, Lazar. The ASA's statement on p-Values: Context, process, and purpose. The American Statistician. https://doi.org/10.1080/00031305.2016.1154108
美国统计协会声明说明,p 值不是假设为真的概率,也不衡量效应大小或实际重要性。
[Wei et al. 2022] Wei, Tay, Bommasani, Raffel, Zoph, others. Emergent abilities of large language models. Transactions on Machine Learning Research. https://arxiv.org/abs/2206.07682
本文定义并综述大语言模型(LLM)的涌现能力:在所报告的评估中,小规模模型未表现、而较大模型表现出的能力。
[Wei et al. 2022] Wei, Bosma, Zhao, Guu, Yu, Lester, Du, Dai, Le. Finetuned language models are zero-shot learners. https://arxiv.org/abs/2109.01652
FLAN 报告称,在多种任务上进行指令微调可提高未见任务类型上的零样本表现,其消融实验显示任务多样性和自然语言指令都很重要。
[Wei et al. 2022] Wei, Wang, Schuurmans, Bosma, Ichter, Xia, Chi, Le, Zhou. Chain-of-thought prompting elicits reasoning in large language models. https://arxiv.org/abs/2201.11903
思维链(CoT)提示通过在少样本示例中加入中间推理步骤,显著提升了大语言模型(LLM)在算术、常识与符号推理任务上的性能。
[Wei et al. 2024] Wei, Yang, Song, Lu, Hu, Huang, Tran, Peng, Liu, Huang, Du, Le. Long-form factuality in large language models. https://proceedings.neurips.cc/paper_files/paper/2024/hash/937ae0e83eb08d2cb8627fe1def8c751-Abstract-Conference.html
LongFact 与 SAFE 测试长文本回答的搜索增强核查;其人工一致率、分歧复核和成本结果取决于论文的拆分方式、证据访问范围与 2024 年评判配置。
[Wei et al. 2024] Wei, Nguyen, Chung, Jiao, Papay, Glaese, Schulman, Fedus. Measuring short-form factuality in large language models. https://cdn.openai.com/papers/simpleqa.pdf
SimpleQA 使用 4,326 个对抗式收集、预期只有一个公认答案的短问题,并把回答标为正确、错误或不作答。
[Wei et al. 2025] Wei, Duchenne, Copet, Carbonneaux, Zhang, Fried, Synnaeve, Singh, Wang. SWE-RL: Advancing LLM reasoning via reinforcement learning on open software evolution. https://proceedings.neurips.cc/paper_files/paper/2025/hash/7107d4d2e837bde2171c6b71b5bde954-Abstract-Conference.html
SWE-RL 用可复现的补丁相似度奖励在软件演化记录上进行强化学习,也体现了代理奖励的能力边界。
[Weinstein and Kaliski 2026] Weinstein, Kaliski. Introducing the machine payments protocol. https://stripe.com/blog/machine-payments-protocol
MPP 将 HTTP 支付协商与底层支付工具分开,并支持稳定币、银行卡和其他支付方式。
[Weiss et al. 2024] Weiss, Ayzenshteyn, Amit, Mirsky. What was your prompt? A remote keylogging attack on AI assistants. https://www.usenix.org/conference/usenixsecurity24/presentation/weiss
在论文的实验条件下,加密流式响应的词元长度模式可精确重建 27% 的回复,并推断 53% 回复的主题。
[Weller et al. 2025] Weller, Chang, MacAvaney, Lo, Cohan, Van Durme, Lawrie, Soldaini. FollowIR: Evaluating and teaching information retrieval models to follow instructions. https://aclanthology.org/2025.naacl-long.597/
FOLLOWIR 基于 TREC 标注员叙述构建基准与训练集,用于评估和训练信息检索模型遵循详细自然语言指令的能力。
[Wen et al. 2026] Wen, Liu, Zheng, Ye, Wu, Wang, Xu, Liang, Li, Miao, Bian, Yang. Reinforcement learning with verifiable rewards implicitly incentivizes correct reasoning in base llms. https://openreview.net/forum?id=jGbRWwIidy
本文指出普通 Pass@K 可能把错误推理但答案正确的样本计为成功,并提出 CoT-Pass@K,要求推理路径与最终答案同时正确。
[Wengert 1964] Wengert. A simple automatic derivative evaluation program. Communications of the ACM. https://doi.org/10.1145/355586.364791
Wengert 将函数分解为带中间变量的基本步骤,奠定了基于求值列表进行自动微分的思路。
[Wenzek et al. 2020] Wenzek, Lachaux, Conneau, Chaudhary, Guzmán, Joulin, Grave. CCNet: Extracting high quality monolingual datasets from web crawl data. European Language Resources Association. https://aclanthology.org/2020.lrec-1.494/
CCNet 是一条自动化流水线,通过去重、语言识别和基于维基百科困惑度的过滤,从 Common Crawl 中提取大规模高质量单语数据集。
[WHATWG n.d.] WHATWG. HTML living standard: Server-sent events. https://html.spec.whatwg.org/multipage/server-sent-events.html
HTML 标准定义了服务器发送事件的 EventSource API、事件流线路格式、重连行为和解析规则。
[White et al. 2024] White, Haddad, Osborne, Liu, Abdelmonsef, Varghese, Le Hors. The model openness framework: Promoting completeness and openness for reproducibility, transparency, and usability in artificial intelligence. https://arxiv.org/abs/2403.13784
模型开放框架盘点发布的代码、数据、文档和模型组件;其类别并不独立验证披露,也不证明复现成功。
[White et al. 2025] White, Dooley, Roberts, Pal, Feuer, Jain, Shwartz-Ziv, Jain, Saifullah, Dey, Shubh-Agrawal, Sandha, Naidu, Hegde, LeCun, Goldstein, Neiswanger, Goldblum. LiveBench: a challenging, contamination-limited LLM benchmark. https://proceedings.iclr.cc/paper_files/paper/2025/hash/e4a46394ba5378b3f9a186a5b4c650d1-Abstract-Conference.html
LiveBench 每月从近期来源更新题目,并使用客观自动评分,以同时降低污染和主观裁判偏差。
[Wijk et al. 2025] Wijk, Lin, Becker, Jawhar, Parikh, Broadley, Chan, Chen, Clymer, Dhyani, Ericheva, Garcia, Goodrich, Jurkovic, Kinniment, Lajko, Nix, Koba Sato, Saunders, Taran, West, Barnes. RE-bench: Evaluating frontier AI R&D capabilities of language model agents against human experts. PMLR. https://proceedings.mlr.press/v267/wijk25a.html
RE-Bench 是一个包含七个机器学习研究工程任务的基准,将前沿 AI 智能体与 61 位人类专家进行对比,发现智能体在 2 小时预算下得分是人类的 4 倍,但人类在 8 小时及更长时间预算下超越了智能体。
[Willard and Louf 2023] Willard, Louf. Efficient guided generation for large language models. arXiv preprint arXiv:2307.09702. https://arxiv.org/abs/2307.09702
Willard 与 Louf 将形式语言和模型词表组合起来,使正则表达式与文法约束能在生成时高效提供词元掩码。
[Williams et al. 2009] Williams, Waterman, Patterson. Roofline: An insightful visual performance model for multicore architectures. Communications of the ACM. https://doi.org/10.1145/1498765.1498785
Roofline 将可达性能与运算强度、峰值算术速率和可持续内存带宽联系起来。
[World Wide Web Consortium 2026] World Wide Web Consortium. WebDriver. https://www.w3.org/TR/webdriver/
WebDriver 定义了自动控制浏览器所用的远程协议和会话模型,因此其端点属于高权限执行接口。
[Wortsman et al. 2022] Wortsman, Ilharco, Gadre, Roelofs, Gontijo-Lopes, Morcos, Namkoong, Farhadi, Carmon, Kornblith, Schmidt. Model soups: Averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. https://arxiv.org/abs/2203.05482
对多个微调模型的权重取平均(称为"模型汤")可在不增加推理成本的前提下,优于最佳单一模型并提升分布外鲁棒性。
[Wortsman et al. 2023] Wortsman, Liu, Xiao, Everett, Alemi, Adlam, Co-Reyes, Gur, Kumar, Novak, Pennington, Sohl-Dickstein, Xu, Lee, Gilmer, Kornblith. Small-scale proxies for large-scale transformer training instabilities. arXiv preprint arXiv:2309.14322. https://arxiv.org/abs/2309.14322
在小型代理模型中复现注意力 logit 与输出 logit 不稳定,并评估此前用于大规模训练的缓解方法。
[Wright et al. 2022] Wright, Andrews, Hutton, Dennis. JSON schema: a media type for describing JSON documents. https://json-schema.org/draft/2020-12/json-schema-core
JSON Schema Core 定义了描述 JSON 文档结构所需的词汇表、标识符、引用、锚点和求值规则。
[Wu et al. 2022] Wu, Raghavendra, Gupta, Acun, Ardalani, Maeng, Chang, Aga, Huang, Bai, Gschwind, Gupta, Ott, Melnikov, Candido, Brooks, Chauhan, Lee, Lee, Hazelwood, Saxena, Shahar, Sun, Wang, Wang, Yao. Sustainable AI: Environmental implications, challenges and opportunities. https://proceedings.mlsys.org/paper_files/paper/2022/hash/462211f67c7d858f663355eff93b745e-Abstract.html
生产工作负载测量表明训练与推理占比因系统而显著不同,不能假设统一的全生命周期成本比例。
[Wu et al. 2025] Wu, Wang, Yu, Zhang, Chang, Yu. LongMemEval: Benchmarking chat assistants on long-term interactive memory. https://proceedings.iclr.cc/paper_files/paper/2025/hash/d813d324dbf0598bbdc9c8e79740ed01-Abstract-Conference.html
LongMemEval 使用可扩展对话历史与 500 个问题测试五种记忆能力,并区分索引、检索与阅读阶段。
[x402 Foundation 2026] x402 Foundation. x402 Protocol. https://github.com/x402-foundation/x402
x402 第 2 版定义了 PAYMENT-REQUIRED、PAYMENT-SIGNATURE 和 PAYMENT-RESPONSE 交换,同时允许支付方案跨网络和资产扩展。
[Xiao et al. 2023] Xiao, Lin, Seznec, Wu, Demouth, Han. SmoothQuant: Accurate and efficient post-training quantization for large language models. https://proceedings.mlr.press/v202/xiao23c.html
SmoothQuant 通过数学等价的逐通道缩放变换将激活离群值迁移至权重,从而为大语言模型实现无需训练的 W8A8 训练后量化。
[Xiao et al. 2024] Xiao, Tian, Chen, Han, Lewis. Efficient streaming language models with attention sinks. https://proceedings.iclr.cc/paper_files/paper/2024/hash/5e5fd18f863cbe6d8ae392a93fd271c9-Abstract-Conference.html
StreamingLLM 在近期窗口之外保留少量初始注意力汇词元,使论文评估的模型族保持稳定的流式语言建模。
[Xie et al. 2023] Xie, Pham, Dong, Du, Liu, Lu, Liang, Le, Ma, Yu. DoReMi: Optimizing data mixtures speeds up language model pretraining. https://openreview.net/forum?id=lXuByUeHhd
DoReMi 在小代理模型上应用分组分布鲁棒优化来确定预训练领域权重;在其 8B 模型实验中,平均少样本下游准确率提高 6.5 个百分点,并以少 2.6 倍的训练步骤达到基线准确率。
[Xie et al. 2023] Xie, Kawaguchi, Zhao, Zhao, Kan, He, Xie. Self-evaluation guided beam search for reasoning. https://arxiv.org/abs/2305.00633
本文提出将逐步自评估机制与随机束搜索结合,引导大语言模型(LLM)多步推理,在 GSM8K 等推理基准上相比 Codex 基线最高提升 9.56%。
[Xie et al. 2024] Xie, Zhang, Chen, Li, Zhao, Cao, Hua, Cheng, Shin, Lei, Liu, Xu, Zhou, Savarese, Xiong, Zhong, Yu. OSWorld: Benchmarking multimodal agents for open-ended tasks in real computer environments. https://proceedings.neurips.cc/paper_files/paper/2024/hash/5d413e48f84dc61244b6be550f1cd8f5-Abstract-Datasets_and_Benchmarks_Track.html
OSWorld 为网页、文件、命令行与应用工作流任务定义明确的初始状态,并用定制的执行式评估器检查结果。
[Xie et al. 2025] Xie, Mao, Bai, Zhang, Wang, Lin, Gu, Chen, Yang, Shou. Show-o: One single transformer to unify multimodal understanding and generation. https://arxiv.org/abs/2408.12528
Show-o 是单一 Transformer,将文本的自回归建模与图像的离散扩散建模统一在一个模型中,同时支持多模态理解与生成。
[Xiong et al. 2020] Xiong, Yang, He, Zheng, Zheng, Xing, Zhang, Lan, Wang, Liu. On layer normalization in the transformer architecture. PMLR. https://arxiv.org/abs/2002.04745
本文用平均场理论证明,将层归一化置于残差块内部(Pre-LN)可在初始化时产生良好梯度,从而无需学习率预热即可训练 Transformer。
[Xiong et al. 2021] Xiong, Xiong, Li, Tang, Liu, Bennett, Ahmed, Overwijk. Approximate nearest neighbor negative contrastive learning for dense text retrieval. https://openreview.net/forum?id=zeFrfgyZln
ANCE 通过异步更新的 ANN 索引从整个语料库全局选取困难负样本来训练稠密检索模型,性能接近 BERT 级联流水线,效率提升 100 倍。
[Xu et al. 2021] Xu, Lee, Chen, Hechtman, Huang, Joshi, Krikun, Lepikhin, Ly, Maggioni, Pang, Shazeer, Wang, Wang, Wu, Chen. GSPMD: General and scalable parallelization for ML computation graphs. https://arxiv.org/abs/2105.04663
GSPMD 在计算图中传播张量分片标注,并生成分区后的单程序多数据程序。
[Xu et al. 2024] Xu, Shi, Choi. RECOMP: Improving retrieval-augmented lms with compression and selective augmentation. https://openreview.net/forum?id=mlJLVigNHp
RECOMP 根据下游任务训练抽取式和生成式压缩器,并学习何时应省略检索增强内容,而不是一律把它放到输入前面。
[Xue et al. 2022] Xue, Barua, Constant, Al-Rfou, Narang, Kale, Roberts, Raffel. ByT5: Towards a token-free future with pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics. https://aclanthology.org/2022.tacl-1.17/
ByT5 移除子词词表,直接在 UTF-8 字节上训练仅作少量修改的 T5。
[Yadav et al. 2023] Yadav, Tam, Choshen, Raffel, Bansal. TIES-merging: Resolving interference when merging models. Advances in Neural Information Processing Systems 36. https://arxiv.org/abs/2306.01708
TIES-MERGING 是一种无需训练的模型合并方法,通过裁剪冗余参数、解决符号冲突、仅对符号对齐的参数求平均,将多个微调模型合并为单一多任务模型。
[Yang et al. 2021] Yang, Hu, Babuschkin, Sidor, Liu, Farhi, Ryder, Pachocki, Chen, Gao. Tensor programs V: Tuning large neural networks via zero-shot hyperparameter transfer. https://arxiv.org/abs/2203.03466
提出一种宽度参数化(muP),使在小模型上调好的最优超参数可零样本迁移到大得多的模型上,省去全规模的超参数扫描。
[Yang et al. 2023] Yang, Chiang, Zheng, Gonzalez, Stoica. Rethinking benchmark and contamination for language models with rephrased samples. https://arxiv.org/abs/2311.04850
本文表明改写的基准测试样本(改述或翻译)可绕过 n-gram 与嵌入式去污染检测,并提出一种基于大语言模型的去污染方法,用于发现预训练数据集中的此类污染。
[Yang et al. 2023] Yang, Zhang, Li, Zou, Li, Gao. Set-of-mark prompting unleashes extraordinary visual grounding in GPT-4V. arXiv preprint arXiv:2310.11441. https://arxiv.org/abs/2310.11441
在检测到的屏幕区域上叠加编号标记,让模型输出编号而不是坐标;在原生定位模型出现之前,这个变通办法支撑了 GUI 视觉定位。
[Yang et al. 2023] Yang, Swope, Gu, Chalamala, Song, Yu, Godil, Prenger, Anandkumar. LeanDojo: Theorem proving with retrieval-augmented language models. https://proceedings.neurips.cc/paper_files/paper/2023/hash/4441469427094f8873d0fecb0c4e1cee-Abstract-Datasets_and_Benchmarks.html
LeanDojo 发布 Lean 定理证明的工具、数据、模型和基准,并把检索增强的前提选择作为核心瓶颈来处理。
[Yang et al. 2024] Yang, Teng, Zheng, Ding, Huang, others. CogVideoX: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072. https://arxiv.org/abs/2408.06072
CogVideoX 是一个文本到视频的扩散 Transformer 模型,采用三维因果 VAE 与专家自适应 LayerNorm,可生成 10 秒、768x1360 分辨率、16 fps 的连贯运动视频。
[Yang et al. 2024] Yang, Jimenez, Wettig, Lieret, Yao, Narasimhan, Press. SWE-agent: Agent-computer interfaces enable automated software engineering. https://proceedings.neurips.cc/paper_files/paper/2024/hash/5a7c947568c1b1328ccc5230172e1e7c-Abstract-Conference.html
SWE-agent 为代码仓库导航、编辑与测试设计模型接口,说明动作与观察接口本身会影响智能体表现。
[Yang et al. 2025] Yang, Kautz, Hatamizadeh. Gated delta networks: Improving mamba2 with delta rule. https://arxiv.org/abs/2412.06464
Gated DeltaNet 在线性注意力递归中把门控遗忘机制与 delta 规则的状态更新结合起来,在语言建模与长上下文任务上超过 Mamba2 与 DeltaNet。
[Yang et al. 2025] Yang, Yu, Li, Liu, Huang, Huang, Jiang, Tu, Zhang, Zhou, Lin, Dang, Yang, Yu, Li, Sun, Zhu, Men, He, Xu, Yin, Yu, Qiu, Ren, Yang, Li, Xu, Zhang. Qwen2.5-1M technical report. arXiv preprint arXiv:2501.15383. https://arxiv.org/abs/2501.15383
报告 Qwen2.5-1M:渐进预训练到 262,144 词元序列,再结合后训练与推断时的位置和注意力方法,把可接受输入扩到一百万词元。
[Yang et al. 2025] Yang, Yi, Ko, Lee, Jin, Yun. Revisiting multi-agent debate as test-time scaling: a systematic study of conditional effectiveness. arXiv preprint arXiv:2505.22960. https://arxiv.org/abs/2505.22960
考虑计算成本的比较发现,辩论收益取决于任务难度、模型能力和安全场景,并非普遍成立。
[Yang and others 2025] Yang, others. Qwen3 technical report. https://arxiv.org/abs/2505.09388
Qwen3 发布涵盖稠密模型与混合专家(MoE)架构(参数量从 0.6B 到 235B)的统一大语言模型(LLM)系列,将思考模式与非思考模式集成于单一模型,并引入思考预算机制以控制推理时计算资源分配。
[Yao et al. 2023] Yao, Yu, Zhao, Shafran, Griffiths, Cao, Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. https://arxiv.org/abs/2305.10601
思维树(ToT)框架让大语言模型通过树搜索探索多条推理路径并自评估,将 GPT-4 在 Game of 24 上的成功率从 4% 提升至 74%。
[Yao et al. 2023] Yao, Zhao, Yu, Du, Shafran, Narasimhan, Cao. ReAct: Synergizing reasoning and acting in language models. https://arxiv.org/abs/2210.03629
ReAct 将推理轨迹与任务动作交错起来,使大语言模型能利用来自工具或环境的外部观察更新计划。
[Yao et al. 2023] Yao, Zhao, Yu, Du, Shafran, Narasimhan, Cao. ReAct: Synergizing reasoning and acting in language models. https://openreview.net/forum?id=WE_vluYUL-X
ReAct 让语言模型交替生成文字推理与环境动作,使后续决策能够纳入新观察。
[Yao et al. 2025] Yao, Shinn, Razavi, Narasimhan. <span class="nocase">τ-bench</span>: a benchmark for tool-agent-user interaction in real-world domains. https://arxiv.org/abs/2406.12045
Tau-bench 评估工具智能体与模拟用户的对话,将最终数据库状态与标注目标比较,并引入 pass 的 k 次方来衡量重复可靠性。
[Yao et al. 2026] Yao, Tan, Liu, Li, Wang, Yu, Tan, Tian, Zhao, Sun, Zhang, Yang. Harness-Bench: Measuring harness effects across models in realistic agent workflows. arXiv preprint arXiv:2605.27922. https://arxiv.org/abs/2605.27922
Harness-Bench 在固定任务、沙箱、预算、超时与评估器的条件下比较模型与运行框架组合,把配置层性能作为测量对象。
[Ye et al. 2025] Ye, Xie, others. Dream 7B: Diffusion large language models. arXiv preprint arXiv:2508.15487. https://arxiv.org/abs/2508.15487
Dream 7B 是一个 70 亿参数的离散扩散大语言模型,通过自回归初始化和上下文自适应词元级噪声重调度,在通用任务上达到与自回归 LLM 相当的性能,同时支持任意顺序生成和灵活的质速权衡。
[Ye et al. 2025] Ye, Huang, Xiao, Chern, Xia, Liu. LIMO: Less is more for reasoning. https://openreview.net/forum?id=T2TZ0RY4Zk
LIMO 用 800 条精选数学样本微调 Qwen2.5-32B-Instruct,并报告较强的竞赛数学结果,同时显示同一配方高度依赖基础模型。
[Yen et al. 2025] Yen, Gao, Hou, Ding, Fleischer, Izsak, Wasserblat, Chen. HELMET: How to evaluate long-context models effectively and thoroughly. https://proceedings.iclr.cc/paper_files/paper/2025/hash/f5332c8273d02729730a9c24dec2135e-Abstract-Conference.html
HELMET 从回忆、检索增强生成、多样本学习、问答和摘要等方面评估长上下文模型,而不把单一检索探针当作充分证据。
[Yin et al. 2024] Yin, Gharbi, Zhang, Shechtman, Durand, Freeman, Park. One-step diffusion with distribution matching distillation. https://arxiv.org/abs/2311.18828
DMD 通过最小化真实分布与生成分布之间的近似 KL 散度,将预训练扩散模型蒸馏为单步图像生成器,在 ImageNet 64x64 上达到 2.62 FID,FP16 推理下可实现每秒 20 帧。
[Yin et al. 2025] Yin, Zhang, Zhang, Freeman, Durand, Shechtman, Huang. From slow bidirectional to fast autoregressive video diffusion models. https://arxiv.org/abs/2412.07772
CausVid 把五十步双向视频扩散模型蒸馏成四步因果生成器,并借助 KV 缓存在单张 GPU 上报告 1.3 秒首帧延迟后 9.4 FPS。
[Young 1974] Young. A first order approximation to the optimum checkpoint interval. Communications of the ACM. https://doi.org/10.1145/361147.361115
Young 在简单的独立故障模型下,根据检查点成本和平均故障间隔推导了一阶检查点间隔。
[Yu et al. 2022] Yu, Li, Koh, Zhang, Pang, Qin, Ku, Xu, Baldridge, Wu. Vector-quantized image modeling with improved VQGAN. https://arxiv.org/abs/2110.04627
ViT-VQGAN 用视觉 Transformer(ViT)替换 VQGAN 的 CNN 编码器,并引入因式分解与 L2 归一化码本学习,在 ImageNet 上同时提升了图像生成 FID 与无监督线性探针精度。
[Yu et al. 2022] Yu, Xu, Koh, Luong, others. Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789. https://arxiv.org/abs/2206.10789
Parti 将文本到图像生成建模为序列到序列问题,使用 ViT-VQGAN 图像词元化器,将自回归 Transformer 扩展至 200 亿参数,在 MS-COCO 上达到当时最优 FID 分数。
[Yu et al. 2022] Yu, Jeong, Kim, Kim, Chun. Orca: a distributed serving system for transformer-based generative models. USENIX Association. https://www.usenix.org/conference/osdi22/presentation/yu
Orca 引入迭代级调度,让生成模型服务器在每个解码步骤后重组批次,而不必等待静态批次全部结束。
[Yu et al. 2023] Yu, Simig, Flaherty, Aghajanyan, Zettlemoyer, Lewis. MEGABYTE: Predicting million-byte sequences with multiscale transformers. https://proceedings.neurips.cc/paper_files/paper/2023/hash/f8f78f8043f35890181a824e53a57134-Abstract-Conference.html
MEGABYTE 把字节分成固定大小的块,由全局模型处理块间关系,再由较小的局部模型处理块内关系。
[Yu et al. 2024] Yu, Yu, Yu, Huang, Li. Language models are super mario: Absorbing abilities from homologous models as a free lunch. https://arxiv.org/abs/2311.03099
DARE 通过随机丢弃并重缩放监督微调(SFT)的增量参数,使多个任务专用语言模型无需重新训练即可合并为单一模型。
[Yu et al. 2025] Yu, Zhang, Zhu, Yuan, Zuo, Yue, Dai, Fan, Liu, Liu, Liu, Liu, Lin, Lin, Ma, Sheng, Tong, Zhang, Zhang, Zhang, Zhang, Zhu, Zhu, Chen, Chen, Wang, Yu, Song, Wei, Zhou, Liu, Ma, Zhang, Yan, Wu, Wang. DAPO: An open-source LLM reinforcement learning system at scale. https://proceedings.neurips.cc/paper_files/paper/2025/hash/a4277440d50f1f15d2cb4c14f7e0c0d2-Abstract-Conference.html
DAPO 提出解耦裁剪与动态采样策略优化算法,通过 Clip-Higher、动态采样、词元级策略梯度损失和超长奖励塑形四项技术,基于 Qwen2.5-32B 在 AIME 2024 上达到 50 分,并完全开源大规模大语言模型强化学习系统。
[Yu et al. 2026] Yu, Zhou, Chen, Soto, Xiong, Li. Cachemir: Fully homomorphic encrypted inference of generative large language model with KV cache. arXiv preprint arXiv:2602.11470. https://arxiv.org/abs/2602.11470
展示支持 KV 缓存的 FHE 生成模型设计,说明密码学推理的结论必须标明时间与工作负载。
[Yuan et al. 2023] Yuan, Yuan, Li, Dong, Lu, Tan, Zhou, Zhou. Scaling relationship on learning mathematical reasoning with large language models. arXiv preprint arXiv:2308.01825. https://arxiv.org/abs/2308.01825
本文提出用于数学推理的拒绝采样微调,并表明过滤及结构多样化的 GSM8K 采样解答可改善监督模型。
[Yuan et al. 2024] Yuan, Shang, Zhou, Dong, Zhou, Xue, Wu, Li, Gu, Lee, Yan, Chen, Sun, Keutzer. LLM inference unveiled: Survey and roofline model insights. arXiv preprint arXiv:2402.16363. https://arxiv.org/abs/2402.16363
本综述以 Roofline 模型为框架分析大语言模型推理效率,并引入 LLM-Viewer 工具,用于识别大语言模型部署在硬件上时的显存与计算瓶颈。
[Yuan et al. 2024] Yuan, Liu, Zhong, Chuang, Li, Wang, Le, Jin, Chaudhary, Xu, Liu, Hu. KV cache compression, but what must we give in return? A comprehensive benchmark of long context capable approaches. Association for Computational Linguistics. https://aclanthology.org/2024.findings-emnlp.266/
该基准在七类任务上比较长上下文效率方法,发现质量与效率权衡会随方法、任务和模型而变化。
[Yuan et al. 2025] Yuan, Gao, Dai, Luo, Zhao, Zhang, Xie, Wei, Wang, Xiao, Wang, Ruan, Zhang, Liang, Zeng. Native sparse attention: Hardware-aligned and natively trainable sparse attention. Association for Computational Linguistics. https://arxiv.org/abs/2502.11089
NSA 把稀疏注意力做成端到端可训练,而非事后加掩码:压缩、选择与滑窗三路层级设计,在保持全注意力质量的同时加速长上下文的训练与解码。
[Yuan et al. 2025] Yuan, Sriskandarajah, Brakman, Helyar, Beutel, Vallone, Jain. From hard refusals to safe-completions: Toward output-centric safety training. arXiv preprint arXiv:2508.09224. https://arxiv.org/abs/2508.09224
安全补全训练用以输出为中心的学习目标取代二元拒绝边界;GPT-5 的报告实验显示,它能在基于意图的拒绝较脆弱的双用途提示上同时改善安全性与有用性。
[Yuan et al. 2026] Yuan, Xiao, Tao, Wang, Gao, Ding, Xu. Incentivizing strong reasoning from weak supervision. Association for Computational Linguistics. https://aclanthology.org/2026.eacl-long.336/
本文研究 Qwen 系列中的弱到强轨迹监督,并报告较小、经过强化学习的推理器所生成的结构化轨迹,在特定设置中可恢复更强学生直接强化学习的大部分增益。
[Yuan et al. 2026] Yuan, Zhou, Xiong, Wu, Sun, Song, Cui, Wang, Wu, Li, Lu, Lu, Zhen, Wang, Deng, Yang, Chen, Zheng, Su, Yu, Zou, Agashe, Lu, Kaur, Qi, Chen, Sala, Liu, Lin, Yu, Su, Reddy, Wang, Qi, Xie, Yu. OSWorld 2.0: Benchmarking computer use agents on long-horizon real-world tasks. arXiv preprint arXiv:2606.29537. https://arxiv.org/abs/2606.29537
包含 108 个长时程工作流的基准,揭示变化信息、隐藏状态、约束、向用户提问和验证方面的失败,而不只测试基础 GUI 操作。
[Yue et al. 2024] Yue, Ni, Zhang, Zheng, Liu, Zhang, Stevens, Jiang, Ren, Sun, others. MMMU: a massive multi-discipline multimodal understanding and reasoning benchmark for expert AGI. https://arxiv.org/abs/2311.16502
MMMU 是一个包含 11.5K 道大学水平多模态题目的基准,覆盖 6 个学科领域的 30 个科目,用于评测大语言模型的专家级感知、知识与推理能力。
[Yue et al. 2024] Yue, Zheng, Ni, Wang, Zhang, Tong, Sun, Yu, Zhang, Sun, others. MMMU-pro: a more robust multi-discipline multimodal understanding benchmark. arXiv preprint arXiv:2409.02813. https://arxiv.org/abs/2409.02813
MMMU-Pro 通过修改问题、增加答案选项和加入纯视觉输入版本,提高了原多模态基准的难度。
[Yue et al. 2025] Yue, Chen, Lu, Zhao, Wang, Yue, Song, Huang. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?. https://proceedings.neurips.cc/paper_files/paper/2025/hash/537d5aa768c2d534016a4d06f87bc8fb-Abstract-Conference.html
RLVR(可验证奖励的强化学习)提升了低 pass@k(至少一次成功率)下的采样效率,但不会产生超出基础模型分布的新推理能力,基础模型构成 RLVR 训练模型的推理上界。
[Zadouri et al. 2026] Zadouri, Hoehnerbach, Shah, Liu, Thakkar, Dao. FlashAttention-4: Algorithm and kernel pipelining co-design for asymmetric hardware scaling. https://arxiv.org/abs/2603.05451
FlashAttention-4 面向 Blackwell,用 CuTe-DSL(嵌入 Python 的 DSL)实现,在 FMA 单元上以软件模拟指数运算,并做条件式 softmax 重缩放,在 B200 上比 cuDNN 注意力快 1.1 至 1.3 倍。
[Zanki 2025] Zanki. Malicious ML models discovered on hugging face platform. https://www.reversinglabs.com/blog/rl-identifies-malware-ml-model-hosted-on-hugging-face
ReversingLabs 发现两个未被标记的畸形 7z 封装 pickle 工件;无害复现实验表明,较早操作码可在后续解析失败前执行。
[Zeghidour et al. 2021] Zeghidour, Luebs, Omran, Skoglund, Tagliasacchi. SoundStream: An end-to-end neural audio codec. IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30. https://arxiv.org/abs/2107.03312
SoundStream 是一种端到端神经音频编解码器,采用卷积编码/解码器与残差向量量化器(RVQ),在 3-18 kbps 低码率下对语音、音乐及通用音频的质量超越 Opus 和 EVS。
[Zelikman et al. 2022] Zelikman, Wu, Mu, Goodman. STaR: Bootstrapping reasoning with reasoning. https://proceedings.neurips.cc/paper_files/paper/2022/hash/639a9a172c044fbb64175b5fad42e9a5-Abstract-Conference.html
STaR 迭代生成推理说明,保留得到已知答案的尝试,用答案提示为失败样本生成事后说明,并从原始检查点重新训练。
[Zhai et al. 2023] Zhai, Mustafa, Kolesnikov, Beyer. Sigmoid loss for language image pre-training. https://arxiv.org/abs/2303.15343
SigLIP 用成对 sigmoid 损失替代 softmax 对比损失,用于语言-图像预训练,在小批量下性能更优且内存效率更高。
[Zhai et al. 2024] Zhai, Liao, Liu, Wang, Li, Cao, Gao, Gong, Gu, He, Lu, Shi. Actions speak louder than words: Trillion-parameter sequential transducers for generative recommendations. PMLR. https://arxiv.org/abs/2402.17152
作者报告了部署在多个平台场景的 1.5 万亿参数 HSTU 生成式推荐系统、某项公开在线指标 12.4% 的提升,以及跨三个训练算力数量级的经验幂律质量缩放;检索与排序仍是分别评估的场景。
[Zhan et al. 2024] Zhan, Liang, Ying, Kang. InjecAgent: Benchmarking indirect prompt injections in tool-integrated large language model agents. https://aclanthology.org/2024.findings-acl.624/
INJECAGENT 是一个包含 1,054 个测试用例的基准,用于评估工具集成 LLM 智能体对间接提示注入攻击的脆弱性,发现 ReAct 提示的 GPT-4 攻击成功率达 24%。
[Zhang and Sennrich 2019] Zhang, Sennrich. Root mean square layer normalization. https://arxiv.org/abs/1910.07467
RMSNorm 去除 LayerNorm 的均值中心化步骤,仅用 RMS 统计量做归一化,在保持相近精度的同时将每步运行时间降低 7% 至 64%。
[Zhang et al. 2023] Zhang, Sheng, Zhou, Chen, Zheng, Cai, Song, Tian, Ré, Barrett, Wang, Chen. H2O: Heavy-hitter oracle for efficient generative inference of large language models. https://proceedings.neurips.cc/paper_files/paper/2023/hash/6ceefa7b15572587b78ecfcebb2827f8-Abstract-Conference.html
H2O 是一种动态 KV 缓存淘汰策略,在论文评估的模型与任务中平衡近期词元和按累积注意力识别的重击者。
[Zhang et al. 2024] Zhang, Zhang, Li, Zhou, Qiu. SpeechTokenizer: Unified speech tokenizer for speech large language models. https://arxiv.org/abs/2308.16692
SpeechTokenizer 是一种基于 RVQ 的统一语音分词器,通过从 HuBERT 进行语义蒸馏,在层次化量化层间分离语义内容与声学细节,专为语音大语言模型设计。
[Zhang and Nanda 2024] Zhang, Nanda. Towards best practices of activation patching in language models: Metrics and methods. https://arxiv.org/abs/2309.16042
激活修补的结论会随破坏方式、输出指标及其他方法选择而变化,因此必须报告完整的反事实设计。
[Zhang et al. 2025] Zhang, Li, Zhang, Liu, Jiang, Tang, Gao, Li, Wang, Tan, Li, Yin, Yin, Jiang. IHEval: Evaluating language models on following the instruction hierarchy. Proceedings of NAACL 2025. https://arxiv.org/abs/2502.08745
IHEval 用 3,538 个样例测试相容与冲突指令,发现受测模型在必须解决优先级冲突时准确率大幅下降。
[Zhang et al. 2025] Zhang, Zheng, Wu, Zhang, Lin, Yu, Liu, Zhou, Lin. The lessons of developing process reward models in mathematical reasoning. https://arxiv.org/abs/2501.07301
用蒙特卡洛估计合成的步骤标签训练出的 PRM,弱于 LLM 评判与人工标注;只保留两者一致步骤的共识过滤造就了更强的 Qwen2.5-Math-PRM,作者随后在另行发布的 ProcessBench 上对其评估。
[Zhang et al. 2025] Zhang, Hosseini, Bansal, Kazemi, Kumar, Agarwal. Generative verifiers: Reward modeling as next-token prediction. https://arxiv.org/abs/2408.15240
GenRM 用下一词元预测训练大语言模型验证器,而非判别式分类,使验证器能写思维链并通过测试时投票改善 Best-of-N 选择。
[Zhang et al. 2026] Zhang, Yu, Wang, Fan, Reed, Mirhoseini, Su. KernelBench-verified: Do LLM-generated kernels actually beat PyTorch?. https://arxiv.org/abs/2607.16241
一项更严格的后续评估加入隐藏输入分布、更强基线与内存指标,说明狭窄的正确性测试如何奖励无效捷径。
[Zhao et al. 2023] Zhao, Gu, Varma, Luo, Huang, Xu, Wright, Shojanazeri, Ott, Shleifer, Desmaison, Balioglu, Damania, Nguyen, Chauhan, Hao, Mathews, Li. PyTorch FSDP: Experiences on scaling fully sharded data parallel. Proceedings of the VLDB Endowment. https://arxiv.org/abs/2304.11277
FSDP 论文说明参数分片如何与 PyTorch 自动微分、内存分配、通信和状态管理相互作用。
[Zhao et al. 2023] Zhao, Kumar, Levine, Finn. Learning fine-grained bimanual manipulation with low-cost hardware. https://arxiv.org/abs/2304.13705
ACT 预测动作序列而不是单步动作,并对重叠预测进行时间组合,用于精细双臂操作。
[Zhao et al. 2025] Zhao, Wu, Yue, others. Absolute zero: Reinforced self-play reasoning with zero data. arXiv preprint arXiv:2505.03335. https://arxiv.org/abs/2505.03335
Absolute Zero 让同一个模型自己提出最有学习价值的任务并求解,两个角色都由代码执行器核查,在不用任何外部数据的情况下取得强的代码与数学推理表现。
[Zhao et al. 2025] Zhao, Hong, Liu, Hazarika, Lin. Do llms recognize your preferences? Evaluating personalized preference following in llms. https://proceedings.iclr.cc/paper_files/paper/2025/hash/28a46044775d97a4efcbcf14e7f13209-Abstract-Conference.html
PrefEval 测试模型能否在不同主题和不断增长的上下文中遵循显式与隐式用户偏好。
[Zheng et al. 2022] Zheng, Han, Polu. MiniF2F: a cross-system benchmark for formal olympiad-level mathematics. https://iclr.cc/virtual/2022/poster/6258
MiniF2F 提供 488 道奥数级题目;原始版本完整覆盖 Metamath 与 Lean,并部分覆盖 Isabelle 与 HOL Light。
[Zheng et al. 2023] Zheng, Chiang, Sheng, Zhuang, Wu, Zhuang, Lin, Li, Li, Xing, Zhang, Gonzalez, Stoica. Judging LLM-as-a-judge with MT-bench and chatbot arena. https://proceedings.neurips.cc/paper_files/paper/2023/hash/91f18a1287b398d378ef22505bf41832-Abstract-Datasets_and_Benchmarks.html
MT-Bench 记录了语言模型评判者的位置、冗长、自我偏好和推理局限,并提出交换候选位置作为一种缓解方法。
[Zheng et al. 2024] Zheng, Yin, Xie, Sun, Huang, Yu, Cao, Kozyrakis, Stoica, Gonzalez, Barrett, Sheng. SGLang: Efficient execution of structured language model programs. https://proceedings.neurips.cc/paper_files/paper/2024/hash/724be4472168f31ba1c9ac630f15dec8-Abstract-Conference.html
SGLang 使用压缩有限状态机与跳跃式前向处理,减少结构化输出中确定性片段的顺序解码工作。
[Zheng et al. 2025] Zheng, Zhang, Zhang, Lin, Lu, Yu, Liu, Zhou, Lin. ProcessBench: Identifying process errors in mathematical reasoning. https://arxiv.org/abs/2412.06559
ProcessBench 评估验证器能否定位最早的错误步骤或识别完全正确的解答,并发现常见数学训练分布到更难问题的泛化较弱。
[Zheng et al. 2025] Zheng, Liu, Li, Chen, Yu, Gao, Dang, Liu, Men, Yang, Zhou, Lin. Group sequence policy optimization. arXiv preprint arXiv:2507.18071. https://arxiv.org/abs/2507.18071
GSPO 将 GRPO 的词元级重要性比率替换为序列级比率与裁剪,稳定了混合专家(MoE)强化学习训练,并提升了 Qwen3 模型的性能。
[Zhong et al. 2024] Zhong, Liu, Chen, Hu, Zhu, Liu, Jin, Zhang. DistServe: Disaggregating prefill and decoding for goodput-optimized large language model serving. USENIX Association. https://www.usenix.org/conference/osdi24/presentation/zhong-yinmin
DistServe 将预填充与解码放到独立 GPU 池,并用在指定 TTFT 与 TPOT 达标率下可持续的到达率定义 goodput。
[Zhou et al. 2020] Zhou, Gu, Neubig. Understanding knowledge distillation in non-autoregressive machine translation. https://arxiv.org/abs/1911.02727
本文通过实验表明,序列级知识蒸馏能降低训练数据的复杂度,从而帮助非自回归生成(NAR)翻译模型,且最优数据复杂度与模型容量强相关。
[Zhou et al. 2022] Zhou, Lei, Liu, Du, Huang, Zhao, Dai, Chen, Le, Laudon. Mixture-of-experts with expert choice routing. https://arxiv.org/abs/2202.09368
Expert Choice MoE 提出让每个专家主动选择 top-k 词元而非词元选择专家,从根本上保证负载均衡,相比 Switch Transformer 和 GShard 的混合专家路由实现超过 2 倍的训练收敛加速。
[Zhou et al. 2023] Zhou, Liu, Xu, Iyer, Sun, Mao, Ma, Efrat, Yu, Yu, Zhang, Ghosh, Lewis, Zettlemoyer, Levy. LIMA: Less is more for alignment. Advances in Neural Information Processing Systems. https://arxiv.org/abs/2305.11206
LIMA 表明,仅用 1,000 条精心筛选的提示-回复对对 65B LLaMa 模型进行监督微调(无需 RLHF)即可实现强对齐效果,支持了"表层对齐假说"。
[Zhou et al. 2023] Zhou, Schärli, Hou, Wei, Scales, Wang, Schuurmans, Cui, Bousquet, Le, Chi. Least-to-most prompting enables complex reasoning in large language models. https://arxiv.org/abs/2205.10625
由最少到最多提示法将复杂问题分解为更简单的子问题并顺序求解,使大语言模型能够泛化到比示例更难的问题。
[Zhou et al. 2024] Zhou, Xu, Zhu, Zhou, Lo, Sridhar, Cheng, Ou, Bisk, Fried, Alon, Neubig. WebArena: a realistic web environment for building autonomous agents. https://proceedings.iclr.cc/paper_files/paper/2024/hash/4410c0711e9154a7a2d26f9b3816d1ef-Abstract-Conference.html
WebArena 提供可自托管的功能完整网站与长时程任务,并按功能正确性评分,因此不同的有效操作路径可以达到同一目标。
[Zhou et al. 2025] Zhou, Yu, Babu, Tirumala, Yasunaga, Shamis, Kahn, Ma, Zettlemoyer, Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model. https://arxiv.org/abs/2408.11039
Transfusion 在单一 Transformer 上联合训练文本的下一词元预测损失与图像的 DDPM 扩散损失,将多模态模型规模扩展至 7B 参数,性能与专用模型相当。
[Zhou et al. 2026] Zhou, Ling, Chen, Wang, Fan, Wang. When more thinking hurts: Overthinking in LLM test-time compute scaling. https://aclanthology.org/2026.findings-acl.1199/
强制词元预算实验显示收益递减,且更长推理有时会推翻原本正确的答案,因此需要感知难度的停止策略。
[Zhou et al. 2026] Zhou, Pacchiardi, Martínez-Plumed, others. General scales unlock AI evaluation with explanatory and predictive power. Nature. https://www.nature.com/articles/s41586-026-10303-2
本文提出 18 条通用需求级别评分准则,对大语言模型基准测试实例进行标注,从而生成可解释的能力画像。
[Zhu et al. 2024] Zhu, Yin, Deng, Almeida, Zhou. Confidential computing on NVIDIA hopper gpus: a performance benchmark study. arXiv preprint arXiv:2409.03992. https://arxiv.org/abs/2409.03992
在两套系统与三个模型系列上测试 H100 机密模式,报告随工作负载变化的吞吐与延迟影响,而非单一通用开销。
[Zhu et al. 2025] Zhu, You, Xing, Huang, others. LLaDA-MoE: a sparse MoE diffusion language model. arXiv preprint arXiv:2509.24389. https://arxiv.org/abs/2509.24389
LLaDA-MoE 使用约 20 万亿令牌从头训练稀疏掩码扩散模型,总非嵌入参数约 70 亿,每个令牌激活约 14 亿。
[Ziegler et al. 2019] Ziegler, Stiennon, Wu, Brown, Radford, Amodei, Christiano, Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593. https://arxiv.org/abs/1909.08593
这项早期语言模型 RLHF 研究根据人工比较学习风格续写和摘要的奖励,同时记录了学习到的奖励如何编码标注者的启发式规则。
[Zoph et al. 2022] Zoph, Bello, Kumar, Du, Huang, Dean, Shazeer, Fedus. ST-MoE: Designing stable and transferable sparse expert models. https://arxiv.org/abs/2202.08906
ST-MoE-32B 是一个 269B 参数的稀疏混合专家(MoE)模型,通过 router z-loss 等技术解决训练不稳定和微调迁移差距,在多项 NLP 基准上达到最优性能。
[Zou et al. 2023] Zou, Wang, Carlini, Nasr, Kolter, Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043. https://arxiv.org/abs/2307.15043
GCG 提出一种自动化的贪婪坐标梯度方法,通过搜索对抗性后缀实现对已对齐大语言模型的越狱,并可迁移至 ChatGPT、Bard、Claude 等黑盒模型。
[Zou et al. 2024] Zou, Phan, Wang, Duenas, Lin, Andriushchenko, Wang, Kolter, Fredrikson, Hendrycks. Improving alignment and robustness with circuit breakers. https://proceedings.neurips.cc/paper_files/paper/2024/hash/97ca7168c2c333df5ea61ece3b3276e1-Abstract-Conference.html
电路断路器通过表示重路由将大语言模型内部表示从有害输出重定向,在不牺牲能力的前提下对未见攻击实现对抗鲁棒性,并可扩展至多模态模型与智能体。
[Zou et al. 2025] Zou, Geng, Wang, Jia. PoisonedRAG: Knowledge corruption attacks to retrieval-augmented generation of large language models. https://www.usenix.org/conference/usenixsecurity25/presentation/zou-poisonedrag
PoisonedRAG 表明,攻击者注入的段落能够破坏受测 RAG 配置的回答,说明语料本身就是攻击面。

评论

登录后评论