References
References
Sources cited in the English edition, ordered by the first author's surname and then by year. Each entry begins with the citation label used in the chapters, followed by the title, an available source link, and a short note explaining what the source contributes. Select a citation in any chapter to jump to its entry here.
[A2A Protocol Working Group 2026] Agent2Agent (A2A) protocol specification, version 1.0. https://a2a-protocol.org/latest/specification/
A2A 1.0 standardizes discovery metadata, messages, stateful tasks, and artifacts between independent agents; authentication and authorization remain deployment responsibilities, and push delivery may be duplicated.
[Abadi et al. 2016] Deep learning with differential privacy. https://arxiv.org/abs/1607.00133
Abadi et al. introduce DP-SGD, training deep neural networks under differential privacy via per-example gradient clipping, Gaussian noise, and a moments accountant for tighter privacy budget tracking.
[Abadi et al. 2016] TensorFlow: a system for large-scale machine learning. https://arxiv.org/abs/1605.08695
TensorFlow represents computation as dataflow graphs and coordinates their execution across CPUs, GPUs, and distributed workers for training and serving.
[Abbas et al. 2023] SemDeDup: Data-efficient learning at web-scale through semantic deduplication. https://arxiv.org/abs/2303.09540
SemDeDup uses embeddings from pre-trained models to identify and remove semantically similar but non-identical duplicates, cutting web-scale training data by 50% with minimal performance loss.
[Abhyankar et al. 2026] OSWorld-human: Benchmarking the efficiency of computer-use agents. Proceedings of Machine Learning and Systems 8. https://proceedings.mlsys.org/paper_files/paper/2026/hash/5edb57c05c81d04beb716ef1d542fe9e-Abstract-Conference.html
Measures what completion rates hide: the tested agents took 2.7 to 4.3 times as many steps as human-authored reference trajectories, while model calls dominated their latency.
[Acun et al. 2021] Understanding training efficiency of deep learning recommendation models at scale. https://arxiv.org/abs/2011.05497
The study characterizes large recommendation-training workloads and shows how model structure, data movement, and distributed execution shape their efficiency.
[Agache et al. 2020] Firecracker: Lightweight virtualization for serverless applications. https://www.usenix.org/conference/nsdi20/presentation/agache
Firecracker describes a small KVM-based virtual machine monitor and the design trade-offs used to isolate high-density serverless workloads.
[Agarwal et al. 2024] On-policy distillation of language models: Learning from self-generated mistakes. https://arxiv.org/abs/2306.13649
Generalized knowledge distillation trains the student on its own sampled sequences using teacher feedback on those sequences, which addresses the mismatch between teacher-written training prefixes and student-written inference prefixes.
[Agent Payments Protocol 2026] AP2 specification. https://ap2-protocol.org/ap2/specification/
AP2 defines deterministic, signed transaction evidence for checkout and payment authorization while leaving agent identification, settlement, and dispute resolution to other systems.
[Agentic Commerce Protocol 2026] Agentic commerce protocol. https://github.com/agentic-commerce-protocol/agentic-commerce-protocol
ACP coordinates commerce while keeping the merchant as system of record and negotiating payment capabilities through handlers rather than prescribing one settlement rail.
[Agrawal et al. 2024] Taming throughput-latency tradeoff in LLM inference with sarathi-serve. USENIX Association. https://www.usenix.org/conference/osdi24/presentation/agrawal
Sarathi-Serve splits long prefills into chunks and schedules them with decodes to limit generation stalls while retaining batching opportunities.
[Ahmadian et al. 2024] Back to basics: Revisiting REINFORCE-style optimization for learning from human feedback in llms. Association for Computational Linguistics. https://aclanthology.org/2024.acl-long.662/
Simple REINFORCE-style policy gradient (RLOO) outperforms PPO and DPO for RLHF alignment of LLMs, with lower compute cost and no need for PPO's actor-critic complexity.
[Ainslie et al. 2023] GQA: Training generalized multi-query transformer models from multi-head checkpoints. Association for Computational Linguistics. https://aclanthology.org/2023.emnlp-main.298/
Grouped-query attention uses an intermediate number of KV heads; in the paper's uptraining experiments it approaches MHA quality with MQA-like speed using 5 percent of original pretraining compute.
[Aizman et al. 2020] High performance I/O for large scale deep learning. https://arxiv.org/abs/2001.01858
The paper examines input-pipeline bottlenecks in large deep-learning jobs and describes storage and data-loading techniques for keeping accelerators supplied.
[Alayrac et al. 2022] Flamingo: a visual language model for few-shot learning. https://proceedings.neurips.cc/paper_files/paper/2022/hash/960a172bc7fbf0177ccccbb411a7d800-Abstract-Conference.html
Flamingo is a Visual Language Model (VLM) family that bridges frozen vision and language models with a Perceiver Resampler and gated cross-attention, enabling few-shot learning across 16 image and video understanding tasks.
[Albergo and Vanden-Eijnden 2023] Building normalizing flows with stochastic interpolants. https://arxiv.org/abs/2209.15571
This paper introduces stochastic interpolants, a framework for building continuous normalizing flows between any two densities via a simple quadratic loss that avoids backpropagation through ODE solvers.
[Albergo et al. 2025] Stochastic interpolants: a unifying framework for flows and diffusions. Journal of Machine Learning Research 26(209):1–80. https://www.jmlr.org/papers/v26/23-1605.html
This paper introduces stochastic interpolants, a unified framework for generative modeling that subsumes flow-based and diffusion-based methods by bridging any two densities exactly on a finite time interval via ODE or SDE dynamics.
[Alpern and Schneider 1985] Defining liveness. https://ecommons.cornell.edu/items/2ed32f4f-cc5c-413b-ba16-5498641f1939
Alpern and Schneider formalize liveness and its relationship to safety properties rather than using the terms as informal labels for accuracy.
[Alsup 2025] Bartz v. anthropic PBC, order on fair use, no. 3:24-cv-05417. United States District Court for the Northern District of California. https://www.govinfo.gov/content/pkg/USCOURTS-cand-3_24-cv-05417/pdf/USCOURTS-cand-3_24-cv-05417-0.pdf
Finds the challenged model-training use fair on its record while leaving the separate permanent-library use of pirate-sourced books for trial.
[Amatriain and Basilico 2012] Netflix recommendations: Beyond the 5 stars (part 1). https://netflixtechblog.com/netflix-recommendations-beyond-the-5-stars-part-1-55838468f429
Netflix incorporated two algorithms from the 2007 Progress Prize ensemble, but not the additional methods in the later Grand Prize solution because their marginal offline gain did not justify the engineering cost and the product had shifted toward streaming discovery.
[Amazon Web Services 2025] AWS nitro enclaves concepts. https://docs.aws.amazon.com/enclaves/latest/user/nitro-enclave-concepts.html
Documents how Nitro Enclaves receive isolated memory and vCPUs, communicate with a parent instance, and produce attestation documents rooted in AWS infrastructure.
[Amazon Web Services 2026] Idempotency for AWS Lambda durable functions. https://docs.aws.amazon.com/lambda/latest/dg/durable-execution-idempotency.html
Lambda durable steps use at-least-once execution by default; interrupted work can repeat, so effectful business logic still needs idempotency keys or another deduplication protocol.
[Amazon Web Services 2026] Choosing a purchasing option for amazon EC2. https://docs.aws.amazon.com/decision-guides/latest/ec2-purchasing-options-aws-how-to-choose/
AWS distinguishes on-demand, spot, savings commitments, capacity reservations, and accelerator capacity blocks by billing, assurance, flexibility, and interruption semantics.
[Amazon.com, Inc. 2026] Annual report for the fiscal year ended december 31, 2025. https://www.sec.gov/Archives/edgar/data/1018724/000101872426000004/amzn-20251231.htm
Amazon reports extending server lives from five to six years in 2024 and shortening a subset of servers and networking equipment to five years in 2025.
[AMD 2020] AMD SEV-SNP: Strengthening VM isolation with integrity protection and more. https://docs.amd.com/v/u/en-US/SEV-SNP-strengthening-vm-isolation-with-integrity-protection-and-more
The whitepaper for VM-level confidential computing: encrypt and integrity-protect a whole virtual machine against a malicious hypervisor, so unmodified stacks can run confidentially.
[Ameisen et al. 2025] Circuit tracing: Revealing computational graphs in language models. https://transformer-circuits.pub/2025/attribution-graphs/methods.html
Circuit Tracing builds prompt-local attribution graphs from a replacement model whose MLPs are approximated by cross-layer transcoders, providing circuit hypotheses that still require fidelity and intervention checks.
[Amershi et al. 2019] Guidelines for human-AI interaction. https://www.microsoft.com/en-us/research/publication/guidelines-for-human-ai-interaction/
This CHI paper distills eighteen generally applicable guidelines for user-facing AI products and validates them with practitioners reviewing AI-infused products.
[Anderson 1982] Reverse-time diffusion equation models. Stochastic Processes and their Applications 12(3):313–326. https://doi.org/10.1016/0304-4149(82)90051-5
Anderson (1982) derives the reverse-time SDE for a forward diffusion process, showing the reverse drift depends on the score function of the marginal distribution.
[Andriushchenko et al. 2025] Jailbreaking leading safety-aligned LLMs with simple adaptive attacks. https://proceedings.iclr.cc/paper_files/paper/2025/hash/63fa7efdd3bcf944a4bd6e0ff6a50041-Abstract-Conference.html
Simple defense-aware changes substantially increased attack success against models and defenses that appeared robust under their earlier static evaluations.
[Android Developers n.d.] Thermal API. https://developer.android.com/ndk/reference/group/thermal
Android's Thermal API exposes thermal status and headroom signals that applications can use to observe and adapt sustained workloads.
[Angrist et al. 1996] Identification of causal effects using instrumental variables. Journal of the American Statistical Association. https://www.nber.org/papers/t0136
The paper states the assumptions under which randomized assignment can identify an average causal effect for compliers when treatment take-up is incomplete.
[Anil et al. 2024] Many-shot jailbreaking. https://papers.nips.cc/paper_files/paper/2024/hash/ea456e232efb72d261715e33ce25f208-Abstract-Conference.html
Many-shot jailbreaking places many fabricated harmful demonstrations in one prompt and reports approximately power-law attack scaling across the evaluated models and tasks.
[Ankner et al. 2024] Hydra: Sequentially-dependent draft heads for medusa decoding. https://openreview.net/forum?id=FbhjirzvJG
Hydra conditions each draft head on earlier candidate tokens, replacing Medusa's independent future-token heads with sequentially dependent proposals.
[Ansel et al. 2024] PyTorch 2: Faster machine learning through dynamic python bytecode transformation and graph compilation. https://docs.pytorch.org/assets/pytorch2-2.pdf
The paper describes guarded Python-bytecode capture, graph breaks, AOTAutograd, and TorchInductor in the PyTorch 2 compiler path.
[Anthropic 2024] Introducing computer use, a new claude 3.5 sonnet, and claude 3.5 haiku. https://www.anthropic.com/news/3-5-models-and-computer-use
The first frontier model to offer computer use in public beta: general computer skills over per-task tools, released with the plain warning that it was experimental, cumbersome, and error-prone.
[Anthropic 2024] Introducing contextual retrieval. https://www.anthropic.com/engineering/contextual-retrieval
Contextual retrieval adds document-specific context to each chunk before embedding and indexing it, improving retrieval when isolated chunks are ambiguous.
[Anthropic 2024] Introducing the model context protocol. https://www.anthropic.com/news/model-context-protocol
Anthropic introduced MCP as an open protocol for connecting AI assistants to systems containing data and tools.
[Anthropic 2025] How we built our multi-agent research system. https://www.anthropic.com/engineering/multi-agent-research-system
Anthropic describes an orchestrator-worker research system, its internal 90.2 percent relative result, high token cost, and practical coordination lessons.
[Anthropic 2025] Effective context engineering for AI agents. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
This engineering note frames context management as per-step curation across instructions, tools, external data, history, retrieval, and compaction.
[Anthropic 2025] Open-sourcing circuit tracing tools. https://www.anthropic.com/research/open-source-circuit-tracing
Anthropic releases software and model artifacts for building and inspecting attribution graphs, enabling others to test circuit-tracing methods.
[Anthropic 2026] Claude's new constitution. https://www.anthropic.com/news/claude-new-constitution
Anthropic's January 2026 constitution is written primarily for Claude, orders broad safety, ethics, Anthropic guidelines, then helpfulness, and is used to generate synthetic training data.
[Anthropic 2026] Use Claude's chat search and memory to build on previous context. https://support.claude.com/en/articles/11817273-use-claude-s-chat-search-and-memory-to-build-on-previous-context
Anthropic documents project-scoped memory, search, pause and reset behavior, incognito chats, and source-linked memory management.
[Anthropic 2026] Computer use tool. https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool
Official documentation for computer-use actions, the developer-run action loop, zoom support, isolated environments, minimal privileges, domain restrictions, and human confirmation.
[Anthropic 2026] Responsible scaling policy, version 3.4. https://www.anthropic.com/responsible-scaling-policy
Defines AI Safety Levels as capability thresholds with a required safeguard tier attached to each, and rules that an unresolved assessment resolves against deployment. Version 3.0 onward stops listing controls for capability levels beyond the current one, asking for an argument for safety instead, and adds a frontier safety roadmap and a risk report every three to six months.
[Anthropic 2026] Project deal: Our claude-run marketplace experiment. https://www.anthropic.com/features/project-deal
Employee agents completed 186 real deals worth just over USD 4,000, a bounded pilot that should not be read as evidence of economy-wide adoption.
[Anthropic n.d.] OpenAI SDK compatibility. https://platform.claude.com/docs/en/cli-sdks-libraries/libraries/openai-sdk
Anthropic documents its OpenAI-SDK compatibility layer as a convenience for testing and lists semantic differences and unsupported or ignored request fields.
[Anthropic and Pattern Labs 2025] Confidential inference systems: Design principles and security risks. https://assets.anthropic.com/m/c52125297b85a42/original/Confidential_Inference_Paper.pdf
Design principles for confidential inference covering both directions of the trust problem: user data protected from the provider, and model weights protected from the infrastructure operator.
[Apple n.d.] Process information: Responding to thermal state changes. https://developer.apple.com/documentation/foundation/processinfo
Apple's process-information APIs expose thermal-state changes so applications can reduce expensive work before thermal pressure becomes critical.
[Apple n.d.] Core ML. https://developer.apple.com/documentation/coreml
Core ML provides a device model representation and can use CPU, GPU, and Neural Engine resources according to model and platform support.
[Apple Security Engineering and Architecture (SEAR), User Privacy, Core Operating Systems (Core OS), Services Engineering (ASE), and Machine Learning and AI (AIML) 2024] Private Cloud Compute: A new frontier for AI privacy in the cloud. https://security.apple.com/blog/private-cloud-compute/
Apple's confidential AI serving design and its five requirements: stateless computation, enforceable guarantees, no privileged runtime access, non-targetability, and verifiable transparency.
[Apple Security Engineering and Architecture (SEAR), User Privacy, Core Operating Systems (Core OS), Services Engineering (ASE), and Machine Learning and AI (AIML) 2026] Expanding private cloud compute. https://security.apple.com/blog/expanding-pcc/
Describes how PCC's verification and privacy properties are extended to a deployment using Intel TDX, NVIDIA confidential computing, and Google Cloud infrastructure.
[Arora et al. 2025] HealthBench: Evaluating large language models towards improved human health. arXiv preprint arXiv:2505.08775. https://arxiv.org/abs/2505.08775
HealthBench grades health conversations against 48,562 rubric criteria written by 262 physicians with practice experience across 60 countries, and meta-evaluates the model grader against physician grading before trusting its scores.
[Arriola et al. 2025] Block diffusion: Interpolating between autoregressive and diffusion language models. https://arxiv.org/abs/2503.09573
Block Diffusion (BD3-LMs) interpolates between discrete diffusion and autoregressive models by applying diffusion within blocks of tokens, enabling variable-length generation and KV caching while achieving state-of-the-art perplexity among discrete diffusion models.
[Artstein and Poesio 2008] Inter-coder agreement for computational linguistics. Computational Linguistics. https://aclanthology.org/J08-4004/
Artstein and Poesio survey agreement coefficients, their assumptions, and their interpretation for computational-linguistics annotation, emphasizing that the coefficient must match the design and scale.
[Asai et al. 2024] Self-RAG: Learning to retrieve, generate, and critique through self-reflection. https://arxiv.org/abs/2310.11511
Self-RAG trains reflection tokens that control retrieval and assess relevance, support, and response utility during generation.
[ASHRAE 2026] Retrofit and modernization strategies: AI data center energy performance framework. https://www.ashrae.org/technical-resources/ai-data-center-framework/retrofit-modernization-strategies
ASHRAE organizes retrofit decisions for AI datacenters around power, cooling, controls, measurement, and the constraints of an operating facility.
[Assran et al. 2025] V-JEPA 2: Self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985. https://arxiv.org/abs/2506.09985
V-JEPA 2 pretrains a video representation on more than one million hours of visual data, then adds an action-conditioned predictor trained on fewer than 62 hours of robot video for tabletop planning experiments.
[Auer et al. 2024] Docling technical report. https://arxiv.org/abs/2408.09869
Describes an open document-conversion toolkit and its representation of page layout, tables, text, and provenance.
[Austin et al. 2021] Structured denoising diffusion models in discrete state-spaces. https://arxiv.org/abs/2107.03006
D3PMs generalize denoising diffusion probabilistic models (DDPM) to discrete state spaces by replacing uniform corruption with structured transition matrices, achieving competitive text and image generation results.
[Austin et al. 2025] How to scale your model. Google DeepMind. https://jax-ml.github.io/scaling-book/
A DeepMind/JAX systems book that explains LLM scaling on real TPU and GPU hardware, including rooflines, sharding, training, inference, serving, and profiling.
[Autio et al. 2024] Artificial intelligence risk management framework: Generative artificial intelligence profile. https://doi.org/10.6028/NIST.AI.600-1
NIST organizes generative-AI risk management around documented governance, measurement, monitoring, incident handling, and layered mitigations.
[Autio et al. 2024] Artificial intelligence risk management framework: Generative artificial intelligence profile. https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence
NIST AI 600-1 frames generative-AI risks and controls across design, deployment-context evaluation, ongoing monitoring, incident response, change, and retirement.
[Axolotl AI n.d.] Axolotl. https://github.com/axolotl-ai-cloud/axolotl
Axolotl is a configuration-oriented open-source training project; its repository and documentation are the authoritative source for supported models and current configuration fields.
[Ba et al. 2016] Layer normalization. https://arxiv.org/abs/1607.06450
Layer normalization computes normalization statistics across all hidden units within a single layer and training case, eliminating batch-size constraints and stabilizing recurrent neural network training.
[Backman et al. 2024] HTTP message signatures. https://www.rfc-editor.org/rfc/rfc9421.html
HTTP Message Signatures authenticates selected request or response components; it does not itself identify an end user or authorize a commercial action.
[Baevski et al. 2020] wav2vec 2.0: a framework for self-supervised learning of speech representations. https://arxiv.org/abs/2006.11477
wav2vec 2.0 learns speech representations via self-supervised masking and contrastive learning over quantized latents, enabling ASR with as little as ten minutes of labeled data.
[Bai et al. 2022] Constitutional AI: Harmlessness from AI feedback. arXiv preprint arXiv:2212.08073. https://arxiv.org/abs/2212.08073
Constitutional AI uses written principles, self-critique, revision, and AI feedback to train harmless but non-evasive assistant behavior.
[Bainbridge 1983] Ironies of automation. Automatica. https://doi.org/10.1016/0005-1098(83)90046-8
Bainbridge names the paradox that automation removes routine practice while leaving humans responsible for rare, difficult interventions when automation fails.
[Baker et al. 2025] Monitoring reasoning models for misbehavior and the risks of promoting obfuscation. arXiv preprint arXiv:2503.11926. https://arxiv.org/abs/2503.11926
A weaker model reading a frontier reasoning model's chain of thought detects reward hacking that action-only monitoring misses, but adding that monitor to the training objective teaches the agent to hide its intent in the trace while continuing to hack.
[Balepur et al. 2026] Language models don't know what you want: Evaluating personalization in deep research needs real users. https://aclanthology.org/2026.acl-long.723/
The study compares real-user judgments with synthetic-user and LLM-judge evaluations and identifies personalization errors the proxies miss.
[Bansal et al. 2021] Does the whole exceed its parts? The effect of AI explanations on complementary team performance. https://arxiv.org/abs/2006.14779
Bansal et al. find that AI explanations did not improve complementary human-AI team performance and could increase acceptance of recommendations regardless of correctness.
[Baronio et al. 2025] Kevin: Multi-turn RL for generating CUDA kernels. arXiv preprint arXiv:2507.11948. https://arxiv.org/abs/2507.11948
Kevin trains a model to improve CUDA kernels through multi-turn interaction with compiler and runtime feedback rather than one-shot generation.
[Bartock et al. 2016] Guide for cybersecurity event recovery. https://doi.org/10.6028/NIST.SP.800-184
NIST provides guidance for recovery planning, testing, restoration validation, metrics, monitoring, and termination after cybersecurity events.
[Baur and Strassen 1983] The complexity of partial derivatives. Theoretical Computer Science. https://web.vu.lt/mif/s.jukna/tropical/Baur-Strassen.pdf
Baur and Strassen prove a constant-factor arithmetic-circuit bound for computing a rational function together with all first partial derivatives.
[Baydin et al. 2018] Automatic differentiation in machine learning: a survey. Journal of Machine Learning Research. https://jmlr.org/papers/v18/17-468.html
This survey distinguishes automatic, symbolic, and numerical differentiation and develops forward and reverse accumulation for machine-learning programs.
[Bean and others 2025] Measuring what matters: Construct validity in large language model benchmarks. https://papers.neurips.cc/paper_files/paper/2025/hash/1967e0fc3aa6cbbace562f5cb8e3954e-Abstract-Datasets_and_Benchmarks_Track.html
A systematic review of 445 benchmark papers documents recurring weaknesses and proposes eight recommendations for construct-valid evaluation.
[Becker et al. 2025] Measuring the impact of early-2025 AI on experienced open-source developer productivity. arXiv preprint arXiv:2507.09089. https://arxiv.org/abs/2507.09089
A randomized trial with 16 experienced open-source developers found that early-2025 AI tools increased completion time by 19 percent despite strong perceived speedups.
[Becker et al. 2026] We are changing our developer productivity experiment design. https://metr.org/blog/2026-02-24-uplift-update/
METR reports suggestive positive raw estimates but explains why selection, compensation, and concurrent-agent use made the follow-up a poor causal measure.
[Benjamini and Hochberg 1995] Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x
Benjamini and Hochberg introduce false-discovery-rate control, a less conservative alternative to family-wise error control for settings with many simultaneous tests.
[Besiroglu et al. 2024] Chinchilla scaling: a replication attempt. arXiv preprint arXiv:2404.10102. https://arxiv.org/abs/2404.10102
Replicates the Chinchilla parametric fit and finds its third estimate inconsistent with the paper's own first two, with confidence intervals so narrow they would require hundreds of thousands of runs rather than the few hundred actually performed.
[Besta et al. 2024] Graph of thoughts: Solving elaborate problems with large language models. https://arxiv.org/abs/2308.09687
Graph of Thoughts represents task-designed generation, aggregation, refinement, and feedback operations as a graph; its reported gains come from tailored workflows on four tasks.
[Betker et al. 2023] Improving image generation with better captions. https://cdn.openai.com/papers/dall-e-3.pdf
DALL-E 3 improves text-to-image prompt following by training on synthetic descriptive captions generated by a fine-tuned image captioner applied to the training dataset.
[Beurer-Kellner et al. 2025] Design patterns for securing LLM agents against prompt injections. arXiv preprint arXiv:2506.08837. https://arxiv.org/abs/2506.08837
Researchers across ETH, Google, Microsoft, and Invariant systematize six design patterns that constrain an agent's structure so an injected instruction cannot redirect its privileged actions.
[Beyer et al. 2016] Site reliability engineering: How google runs production systems. O'Reilly Media. https://sre.google/books/
Google's SRE book provides the operational vocabulary this chapter adapts to AI systems: service promises, error budgets, incident command, and learning from failure.
[Bhatt et al. 2025] Ctrl-z: Controlling AI agents via resampling. arXiv preprint arXiv:2504.10374. https://arxiv.org/abs/2504.10374
On BashBench, the best studied protocol combined resampling with untrusted monitoring and reduced estimated attack success from 58% to about 7% at a 5% normalized usefulness cost; the result is benchmark- and protocol-specific.
[Biderman et al. 2023] Pythia: a suite for analyzing large language models across training and scaling. PMLR. https://proceedings.mlr.press/v202/biderman23a.html
Pythia is a suite of 16 LLMs from 70M to 12B parameters, each with 154 public checkpoints trained on the same data order, designed to study training dynamics and scaling.
[Biderman et al. 2024] LoRA learns less and forgets less. Transactions on Machine Learning Research. https://arxiv.org/abs/2405.09673
In the studied mathematics and code settings, standard LoRA learns less of the target domain than full fine-tuning but preserves more base-model behavior.
[Bie et al. 2025] LLaDA2.0: Scaling up diffusion language models to 100B. arXiv preprint arXiv:2512.15745. https://arxiv.org/abs/2512.15745
LLaDA2.0 converts pretrained autoregressive mixture-of-experts models into diffusion models, including a 100B-total-parameter variant with 6.1B active parameters.
[Birgisson et al. 2014] Macaroons: Cookies with contextual caveats for decentralized authorization in the cloud. https://www.ndss-symposium.org/ndss2014/ndss-2014-programme/macaroons-cookies-contextual-caveats-decentralized-authorization-cloud/
Macaroons demonstrate decentralized attenuation with chained caveats, while leaving policy semantics and revocation to the surrounding system.
[Birkholz et al. 2023] Remote attestation procedureS (RATS) architecture. https://www.rfc-editor.org/rfc/rfc9334.html
Defines remote-attestation roles and information flows without collapsing evidence appraisal and the relying party's authorization decision into one step.
[Black Forest Labs 2024] Announcing black forest labs. https://bfl.ai/blog/24-08-01-bfl
Black Forest Labs introduced the FLUX.1 flow-matching model family with a full model, an open-weight distilled model, and a faster hosted variant.
[Bonney et al. 2026] The microstructure of AI diffusion: Evidence from firms, business functions, and worker tasks. https://www.census.gov/library/working-papers/2026/adrm/CES-WP-26-25.html
The survey estimates firm AI adoption across business functions and shows that use remains narrow within most adopting firms; it does not identify causal productivity.
[Borsos et al. 2023] AudioLM: a language modeling approach to audio generation. IEEE/ACM Transactions on Audio, Speech, and Language Processing. https://arxiv.org/abs/2209.03143
AudioLM frames audio generation as language modeling over a hybrid of semantic tokens (from w2v-BERT) and acoustic tokens (from SoundStream), achieving both long-term coherence and high-quality synthesis for speech and piano continuation.
[Borsos et al. 2023] SoundStorm: Efficient parallel audio generation. arXiv preprint arXiv:2305.09636. https://arxiv.org/abs/2305.09636
SoundStorm uses confidence-based parallel decoding over acoustic tokens; the paper reports generating 30 seconds of audio in 0.5 seconds on a TPU-v4.
[Bourtoule et al. 2021] Machine unlearning. https://arxiv.org/abs/1912.03817
This paper introduces SISA training, which partitions data into shards and slices to reduce the retraining cost of machine unlearning, achieving up to 4.63x speedup over full retraining.
[Bouthillier et al. 2021] Accounting for variance in machine learning benchmarks. https://proceedings.mlsys.org/paper_files/paper/2021/hash/0184b0cd3cfb185989f858a1d9f5c1eb-Abstract.html
The study measures how data splits, initialization, optimization, and implementation choices contribute to variance in machine-learning benchmark results.
[Bowman et al. 2022] Measuring progress on scalable oversight for large language models. arXiv preprint arXiv:2211.03540. https://arxiv.org/abs/2211.03540
This paper proposes a sandwiching-style empirical design for scalable oversight and shows, on MMLU and time-limited QuALITY, that humans assisted by an unreliable model can outperform both unaided humans and the model alone.
[Bradford 2020] The brussels effect: How the european union rules the world. Oxford University Press.
Bradford documents the Brussels Effect, how the EU exports its rules worldwide by setting de facto global standards through market access.
[Bradley and Terry 1952] Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika. https://doi.org/10.1093/biomet/39.3-4.324
Bradley and Terry derive a latent-score probability model for paired comparisons, the basis of modern batch preference rankings.
[Breck et al. 2017] The ML test score: a rubric for ML production readiness and technical debt reduction. https://research.google/pubs/pub46555/
The ML Test Score provides a production-readiness rubric across data, model, infrastructure, and monitoring tests.
[Breck et al. 2017] The ML test score: a rubric for ML production readiness and technical debt reduction. https://research.google/pubs/the-ml-test-score-a-rubric-for-ml-production-readiness-and-technical-debt-reduction/
A production-readiness rubric turns data, model, infrastructure, and monitoring assumptions into explicit tests.
[Bricken et al. 2023] Towards monosemanticity: Decomposing language models with dictionary learning. https://transformer-circuits.pub/2023/monosemantic-features/index.html
Dictionary learning on a one-layer Transformer's MLP activations recovers many features that are more interpretable than individual neurons, while leaving open questions about feature splitting, completeness, and generalization.
[Brill 2024] Neural scaling laws rooted in the data distribution. arXiv preprint arXiv:2412.07942. https://arxiv.org/abs/2412.07942
Uses synthetic percolation-based regression data to explore how discrete-skill and data-manifold scaling regimes can arise on opposite sides of a threshold.
[Broder 1997] On the resemblance and containment of documents. IEEE. https://doi.org/10.1109/SEQUEN.1997.666900
Broder formalizes document resemblance and containment using shingle sets and random sampling, establishing the document-similarity method now known as MinHash.
[Brohan et al. 2023] RT-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818. https://arxiv.org/abs/2307.15818
RT-2 co-fine-tunes vision-language models on web data and robot trajectories by expressing robot actions as tokens, then evaluates transfer to new objects, instructions, and tasks.
[Brooker 2019] Timeouts, retries, and backoff with jitter. https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/
AWS explains how to choose timeouts and combine bounded retries, exponential backoff, and jitter without amplifying an overloaded dependency.
[Brooker et al. 2021] Restoring uniqueness in MicroVM snapshots. CoRR. https://arxiv.org/abs/2102.12892
Restoring cloned microVM snapshots can duplicate random state and identifiers; the paper designs mechanisms that restore uniqueness after resume.
[Brooks et al. 2024] Video generation models as world simulators. https://openai.com/index/video-generation-models-as-world-simulators/
The Sora technical report: a diffusion transformer over spatiotemporal patches of video latents, with compute as the axis along which sample quality scales, up to a minute of generated video.
[Brown et al. 2001] Interval estimation for a binomial proportion. Statistical Science. https://projecteuclid.org/journals/statistical-science/volume-16/issue-2/Interval-Estimation-for-a-Binomial-Proportion/10.1214/ss/1009213286.full
Brown, Cai, and DasGupta show that the common Wald interval has erratic coverage and recommend better-performing alternatives, including the Wilson score interval.
[Brown et al. 2020] Language models are few-shot learners. https://proceedings.neurips.cc/paper_files/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html
GPT-3 showed that text instructions and demonstrations can condition task behavior without gradient updates, with results that vary across tasks and model scales.
[Brown et al. 2024] Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787. https://arxiv.org/abs/2407.21787
Across several models and tasks, repeated sampling increases candidate coverage, often following an exponentiated power-law fit over the measured range, while practical gains depend on selection.
[Brown-Cohen et al. 2023] Scalable AI safety via doubly-efficient debate. arXiv preprint arXiv:2311.14125. https://arxiv.org/abs/2311.14125
The paper proves completeness and soundness results for particular debate games under formal compute, oracle, and strategy assumptions.
[Brynjolfsson et al. 2021] The productivity j-curve: How intangibles complement general purpose technologies. American Economic Journal: Macroeconomics. https://doi.org/10.1257/mac.20180386
The authors model and document how costly investment in processes and other intangible capital can initially conceal the output gains from a general-purpose technology.
[Brynjolfsson et al. 2025] Generative AI at work. The Quarterly Journal of Economics. https://doi.org/10.1093/qje/qjae044
A conversational assistant increased resolved issues per hour at one customer-support operation, with larger gains among less experienced and lower-skill agents.
[Btrfs Maintainers 2026] Btrfs design. https://btrfs.readthedocs.io/en/stable/dev/dev-btrfs-design.html
Btrfs snapshots initially share a root block and use copy-on-write so later changes remain private to each root.
[Buçinca et al. 2021] To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. https://arxiv.org/abs/2102.09692
Buçinca, Malaya, and Gajos show that cognitive forcing interventions reduce overreliance on AI recommendations more than simple explanation displays, with usability trade-offs.
[Burns et al. 2016] Borg, omega, and kubernetes. ACM Queue. https://research.google/pubs/borg-omega-and-kubernetes/
The article traces ideas from Borg and Omega into Kubernetes, including shared cluster state, reconciliation, scheduling, and extensible control loops.
[Burns et al. 2024] Weak-to-strong generalization: Eliciting strong capabilities with weak supervision. PMLR. https://proceedings.mlr.press/v235/burns24b.html
Across NLP, chess, and reward modeling, weak model labels elicited some stronger-model capability, but naive fine-tuning remained far below full strong-model supervision.
[Cai et al. 2024] Medusa: Simple LLM inference acceleration framework with multiple decoding heads. PMLR. https://proceedings.mlr.press/v235/cai24b.html
Medusa adds future-token heads and tree attention to produce and verify multiple candidates; its exact and typical-acceptance configurations have different output contracts.
[California State Legislature 2025] SB 53: Transparency in frontier artificial intelligence act. https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202520260SB53
California SB 53 establishes transparency and safety-reporting requirements for covered frontier AI developers and defines state oversight mechanisms.
[Campbell et al. 2020] RFC 8707: Resource indicators for OAuth 2.0. https://datatracker.ietf.org/doc/html/rfc8707
RFC 8707 defines an OAuth 2.0 extension adding request parameters that let a client explicitly indicate to an authorization server which protected resource(s) it is requesting access to.
[Campbell et al. 2020] Resource indicators for OAuth 2.0. https://www.rfc-editor.org/rfc/rfc8707.html
RFC 8707 lets clients identify a target protected resource so authorization servers can issue audience-restricted tokens.
[Cappos et al. 2026] The update framework specification. https://theupdateframework.github.io/specification/latest/
TUF specifies signed update metadata and defenses against rollback, freeze, mix-and-match, key-compromise, and other software-update attacks.
[Card et al. 2020] With little power comes great responsibility. Association for Computational Linguistics. https://aclanthology.org/2020.emnlp-main.745/
Card et al. find that underpowered NLP experiments are common and show how low power makes meaningful differences harder to detect while exaggerating selected positive findings.
[Carlini et al. 2019] On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705. https://arxiv.org/abs/1902.06705
This evaluation guide argues that robustness claims require explicit threat models, strong attacks, adaptive testing, and reproducible reporting.
[Carlini et al. 2019] The secret sharer: Evaluating and testing unintended memorization in neural networks. USENIX Association. https://www.usenix.org/conference/usenixsecurity19/presentation/carlini
The paper introduces canary exposure as a quantitative test for unintended memorization of rare sequences in generative models.
[Carlini et al. 2021] Extracting training data from large language models. https://arxiv.org/abs/2012.07805
Carlini et al. show that large language models memorize training data, and that black-box query access to GPT-2 can extract verbatim text including personally identifiable information.
[Carlini et al. 2023] Quantifying memorization across neural language models. https://arxiv.org/abs/2202.07646
Carlini et al. quantify three log-linear relationships showing that LLM memorization of training data grows with model capacity, data duplication, and context length, and is more prevalent than previously believed.
[Carlini et al. 2024] Poisoning web-scale training datasets is practical. IEEE. https://doi.org/10.1109/SP54263.2024.00179
The paper shows how mutable URLs and dataset snapshots can be manipulated; its roughly sixty-dollar estimate concerns controlling content returned by 0.01 percent of URLs in two image-text datasets, not a demonstrated model backdoor.
[Castro and Liskov 1999] Practical byzantine fault tolerance. USENIX Association. https://www.usenix.org/conference/osdi-99/presentation/practical-byzantine-fault-tolerance
PBFT implements replicated state-machine agreement under an explicit membership, fault bound, communication model, and liveness assumptions.
[Center for AI Safety et al. 2026] A benchmark of expert-level academic questions to assess AI capabilities. Nature. https://www.nature.com/articles/s41586-025-09962-4
Humanity's Last Exam contains 2,500 closed-ended expert questions over more than 100 subjects and documents why older academic benchmarks had little frontier headroom.
[Chameleon Team 2024] Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09818. https://arxiv.org/abs/2405.09818
Chameleon is a family of early-fusion token-based mixed-modal foundation models that understand and generate arbitrarily interleaved image and text sequences using a single unified transformer trained with QK-norm for stability.
[Chan et al. 2016] Listen, attend and spell. https://arxiv.org/abs/1508.01211
LAS is an end-to-end sequence-to-sequence speech recognizer with a pyramidal RNN encoder and attention-based character decoder, achieving 14.1% WER without a language model on Google voice search.
[Chan et al. 2020] Imputer: Sequence modelling via imputation and dynamic programming. https://arxiv.org/abs/2002.08926
Imputer is an iterative non-autoregressive sequence model for speech recognition that uses dynamic programming to marginalize over alignments and generation orders, achieving constant-step decoding and outperforming CTC on LibriSpeech.
[Chandy and Lamport 1985] Distributed snapshots: Determining global states of distributed systems. ACM Transactions on Computer Systems. https://doi.org/10.1145/214451.214456
Chandy and Lamport show how to record a consistent global state without stopping a distributed computation.
[Chaney et al. 2018] How algorithmic confounding in recommendation systems increases homogeneity and decreases utility. https://arxiv.org/abs/1710.11214
Simulation results show how feedback between recommendations and user data can increase homogenization while reducing measured user utility.
[Chanin et al. 2025] A is for absorption: Studying feature splitting and absorption in sparse autoencoders. https://proceedings.neurips.cc/paper_files/paper/2025/hash/764ff7477b8e24dbe01531f6791e8bdf-Abstract-Conference.html
This paper identifies "feature absorption" in sparse autoencoders (SAEs), where hierarchical features cause SAE latents to silently fail to activate on tokens they should track, undermining reliable LLM interpretability.
[Chao et al. 2024] JailbreakBench: An open robustness benchmark for jailbreaking large language models. arXiv preprint arXiv:2404.01318. https://arxiv.org/abs/2404.01318
JailbreakBench provides a versioned behavior set, adversarial-prompt repository, threat models, chat templates, and scoring functions for reproducible jailbreak evaluation.
[Chen and Guestrin 2016] XGBoost: a scalable tree boosting system. https://arxiv.org/abs/1603.02754
The systems design of gradient-boosted trees at scale: cache-aware layout, sparsity handling, and out-of-core computation, with the paper's own count that 17 of 29 published Kaggle winning solutions in 2015 used it.
[Chen et al. 2016] Training deep nets with sublinear memory cost. https://arxiv.org/abs/1604.06174
The paper analyzes activation recomputation schedules that trade additional forward work for sublinear saved-activation memory.
[Chen et al. 2018] TVM: An automated end-to-end optimizing compiler for deep learning. https://arxiv.org/abs/1802.04799
TVM applies algorithm/schedule separation to deep-learning workloads and uses a learned cost model to search schedules across diverse hardware targets.
[Chen and others 2021] Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. https://arxiv.org/abs/2107.03374
This paper introduces Codex, a GPT model fine-tuned on GitHub code, and releases HumanEval, a 164-problem benchmark measuring functional correctness via pass@k unit-test evaluation.
[Chen et al. 2022] WavLM: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing. https://arxiv.org/abs/2110.13900
WavLM is a self-supervised speech pre-training model that jointly learns masked speech prediction and denoising to achieve state-of-the-art performance across full-stack speech tasks including ASR, speaker verification, separation, and diarization.
[Chen et al. 2023] Extending context window of large language models via positional interpolation. arXiv preprint arXiv:2306.15595. https://arxiv.org/abs/2306.15595
Extends RoPE-based LLM context windows by linearly down-scaling position indices, avoiding unstable extrapolation and preserving original-window quality after limited fine-tuning.
[Chen et al. 2023] Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. Transactions on Machine Learning Research. https://arxiv.org/abs/2211.12588
Program of Thoughts prompting asks LLMs to express numerical reasoning as executable programs, separating reasoning decomposition from exact computation.
[Chen et al. 2023] Accelerating large language model decoding with speculative sampling. arXiv preprint arXiv:2302.01318. https://arxiv.org/abs/2302.01318
Speculative sampling verifies a short sequence from a trained draft model with one target-model call and uses a correction distribution to preserve the target distribution.
[Chen et al. 2023] FrugalGPT: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176. https://arxiv.org/abs/2305.05176
FrugalGPT studies model cascades and other strategies for reducing API cost while measuring resulting task performance rather than assuming a cheaper route is equivalent.
[Chen et al. 2024] VALL-e 2: Neural codec language models are human parity zero-shot text to speech synthesizers. arXiv preprint arXiv:2406.05370. https://arxiv.org/abs/2406.05370
VALL-E 2 introduces repetition-aware sampling and grouped code modeling; the authors report human-parity preference results on LibriSpeech and VCTK.
[Chen et al. 2024] F5-TTS: a fairytaler that fakes fluent and faithful speech with flow matching. arXiv preprint arXiv:2410.06885. https://arxiv.org/abs/2410.06885
F5-TTS uses flow matching with DiT and ConvNeXt without phoneme alignment or an explicit duration predictor during training; inference still supplies a target span length.
[Chen et al. 2024] How far are we to GPT-4V? Closing the gap to commercial multimodal models with open-source suites. arXiv preprint arXiv:2404.16821. https://arxiv.org/abs/2404.16821
InternVL 1.5 combines a scaled vision encoder with a language model and evaluates the resulting open multimodal system across varied visual tasks.
[Chen et al. 2024] An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. https://www.ecva.net/papers/eccv_2024/papers_ECCV/html/10478_ECCV_2024_paper.php
FastV prunes visual tokens in large vision-language models after layer 2 based on attention scores, achieving 45% FLOPs reduction with negligible performance loss.
[Chen et al. 2024] AgentPoison: Red-teaming LLM agents via poisoning memory or knowledge bases. arXiv preprint arXiv:2407.12784. https://proceedings.neurips.cc/paper_files/paper/2024/hash/eb113910e9c3f6242541c1652e30dfd6-Abstract-Conference.html
AgentPoison evaluates targeted backdoor attacks that poison an agent's long-term memory or retrieval knowledge base.
[Chen et al. 2024] ZKML: An optimizing system for ML inference in zero-knowledge proofs. https://doi.org/10.1145/3627703.3650088
Shows how to prove ML inference statements in zero knowledge; it does not by itself hide a user's prompt from the prover that performs ordinary inference.
[Chen et al. 2025] ACEBench: a comprehensive evaluation of LLM tool usage. Association for Computational Linguistics. https://aclanthology.org/2025.findings-emnlp.697/
ACEBench evaluates basic tool calls, ambiguous or incomplete instructions, and simulated multi-turn interactions with fine-grained error categories.
[Chen et al. 2025] StruQ: Defending against prompt injection with structured queries. USENIX Association. https://www.usenix.org/conference/usenixsecurity25/presentation/chen-sizhe
StruQ combines a structured front end with a model specifically trained to distinguish instructions from data, rather than relying on delimiters alone.
[Cheng et al. 2024] Intel TDX demystified: a top-down approach. ACM Computing Surveys. https://doi.org/10.1145/3652597
A top-down academic treatment of Intel's trust-domain VMs: the architecture, the attestation flow, and the trust boundaries, without requiring the vendor specification.
[Chervonyi et al. 2025] Gold-medalist performance in solving olympiad geometry with AlphaGeometry2. arXiv preprint arXiv:2502.03544. https://arxiv.org/abs/2502.03544
AlphaGeometry2 improves language coverage and symbolic search for Olympiad geometry, showing how neural proposal and symbolic verification can work together.
[Chetlur et al. 2014] cuDNN: Efficient primitives for deep learning. arXiv preprint arXiv:1410.0759. https://arxiv.org/abs/1410.0759
cuDNN exposes optimized GPU primitives for common deep-learning operations, allowing frameworks to reuse device-specific implementations behind a library interface.
[Chhabria 2025] Kadrey v. meta platforms, inc., order on summary judgment, no. 3:23-cv-03417. United States District Court for the Northern District of California. https://law.justia.com/cases/federal/district-courts/california/candce/3:2023cv03417/415175/598/
Grants Meta summary judgment on the named authors' reproduction-and-training theory while stressing the absence of meaningful market-harm evidence.
[Chi et al. 2023] Diffusion policy: Visuomotor policy learning via action diffusion. https://arxiv.org/abs/2303.04137
Diffusion Policy represents a multimodal distribution over continuous action sequences and executes them with receding-horizon visual feedback.
[Chiang et al. 2024] Chatbot arena: An open platform for evaluating LLMs by human preference. PMLR. https://proceedings.mlr.press/v235/chiang24b.html
Chatbot Arena develops a large-scale pairwise human-preference platform and statistical ranking methodology; its estimates are conditional on the sampled users, prompts, model pool, and protocol.
[Chollet et al. 2025] ARC-AGI-2: a new challenge for frontier AI reasoning systems. arXiv preprint arXiv:2505.11831. https://arxiv.org/abs/2505.11831
ARC-AGI-2 introduces harder exact-match grid transformations, multiple held-out sets, and first-party human testing to study abstract task generalization.
[Chow 1970] On optimum recognition error and reject tradeoff. IEEE Transactions on Information Theory. https://doi.org/10.1109/TIT.1970.1054406
Chow formalizes the trade-off between recognition error and rejection, the classical basis for evaluating abstaining prediction policies.
[Chowdhery et al. 2023] PaLM: Scaling language modeling with pathways. Journal of Machine Learning Research. https://arxiv.org/abs/2204.02311
Reports the 540B-parameter PaLM model, defines model FLOPs utilization for large-scale training, and documents its distributed training system and stability measures.
[Chowdhery et al. 2023] PaLM: Scaling language modeling with pathways. Journal of Machine Learning Research. https://jmlr.org/papers/volume24/22-1144/22-1144.pdf
The PaLM paper defines model FLOPs utilization as a model-level efficiency measure that excludes rematerialization from the useful-work convention.
[Christiano et al. 2017] Deep reinforcement learning from human preferences. https://arxiv.org/abs/1706.03741
This paper shows that deep RL agents can learn complex behaviors from non-expert human preferences over trajectory segment pairs, requiring feedback on less than 1% of environment interactions.
[Christiano et al. 2018] Supervising strong learners by amplifying weak experts. arXiv preprint arXiv:1810.08575. https://arxiv.org/abs/1810.08575
Iterated amplification proposes recursive decomposition and reports experiments on five synthetic combinatorial tasks using a hard-coded decomposer.
[Chromium Project 2026] Site isolation. https://www.chromium.org/Home/chromium-security/site-isolation/
Chromium's explanation of Site Isolation, which places cross-site documents in different processes to limit compromised renderers but does not replace environment isolation.
[Chung et al. 2021] w2v-BERT: Combining contrastive learning and masked language modeling for self-supervised speech pre-training. https://arxiv.org/abs/2108.06209
w2v-BERT combines wav2vec 2.0 contrastive learning with masked language modeling into a single end-to-end self-supervised framework for speech representation learning, achieving competitive ASR results on LibriSpeech.
[Clark et al. 2021] All that's 'human' is not gold: Evaluating human evaluation of generated text. Association for Computational Linguistics. https://aclanthology.org/2021.acl-long.565/
Clark et al. show that untrained evaluators often identify GPT-3 generated text only at chance level, motivating evaluator training and clearer protocols.
[Clark et al. 2022] CANINE: Pre-training an efficient tokenization-free encoder for language representation. Transactions of the Association for Computational Linguistics. https://aclanthology.org/2022.tacl-1.5/
CANINE encodes Unicode characters and uses convolutional downsampling before its deep Transformer encoder.
[Coalition for Content Provenance and Authenticity 2026] C2PA technical specification. https://c2pa.org/specifications/
The C2PA specification defines signed provenance manifests for recording a media asset's origin and edits without claiming that the content itself is true.
[Cobbe et al. 2021] Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. https://arxiv.org/abs/2110.14168
Training Verifiers introduces GSM8K and shows that sampling many solutions then selecting with a verifier can outperform directly fine-tuning the generator on math word problems.
[Cohen 1960] A coefficient of agreement for nominal scales. Educational and Psychological Measurement. https://doi.org/10.1177/001316446002000104
Cohen's kappa measures agreement between two raters after adjusting for agreement expected by chance, making raw percent agreement harder to misuse.
[Cohen 2024] Data scientists targeted by malicious hugging face ML models with silent backdoor. https://jfrog.com/blog/data-scientists-targeted-by-malicious-hugging-face-ml-models-with-silent-backdoor/
JFrog reported around one hundred PyTorch or Keras artifacts its scanner classified as genuinely harmful; one investigated pickle checkpoint contained a reverse shell, with no reported victim count.
[Colorado General Assembly 2026] SB 26-189: Consumer protections for artificial intelligence interactions. https://www.leg.colorado.gov/bills/SB26-189
Colorado SB 26-189 defines disclosure and consumer-protection duties for specified AI interactions and assigns enforcement responsibilities.
[Competition and Markets Authority 2024] AI foundation models: Technical update report. https://www.gov.uk/government/publications/ai-foundation-models-update-paper
The CMA identifies compute, data, expertise, funding, and routes to market as possible constraints on entry and expansion, framing them as competition risks to examine rather than established harm.
[Competition and Markets Authority 2025] Appendix D: Market structure and concentration methodology and UK share of supply by revenue. https://assets.publishing.service.gov.uk/media/688b7ffafc784fa12a08906e/Appendix_D_-_Market_structure_and_concentration_methodology_and_UK_share_of_supply_by_revenue.pdf
The CMA estimates supplier revenue shares for specified UK cloud-service categories and explains that accelerated-compute revenue could not be separated consistently from the broader totals.
[Cormack et al. 2009] Reciprocal rank fusion outperforms condorcet and individual rank learning methods. https://research.google/pubs/reciprocal-rank-fusion-outperforms-condorcet-and-individual-rank-learning-methods/
Introduces reciprocal rank fusion for combining ranked retrieval lists without score calibration.
[Costan and Devadas 2016] Intel SGX explained. https://eprint.iacr.org/2016/086
The definitive explainer of Intel SGX: how enclaves, measurement, and attestation actually work at the silicon level, still the best single on-ramp to trusted execution.
[Coste et al. 2023] Reward model ensembles help mitigate overoptimization. arXiv preprint arXiv:2310.02743. https://arxiv.org/abs/2310.02743
In a synthetic gold-reward setup, conservative objectives over reward-model ensembles reduce proxy overoptimization for best-of-n selection and PPO.
[Cottier et al. 2024] The rising costs of training frontier AI models. https://arxiv.org/abs/2405.21015
Cottier et al. estimate amortized accelerator hardware and energy costs for final training runs of selected frontier models.
[Cottier et al. 2025] LLM inference prices have fallen rapidly but unequally across tasks. https://epoch.ai/data-insights/llm-inference-price-trends
Epoch AI compares listed API prices at fixed benchmark thresholds and finds that observed declines vary greatly by task, price weighting, and time window.
[Council of Europe 2024] Framework convention on artificial intelligence and human rights, democracy and the rule of law. https://www.coe.int/en/web/conventions/full-list?module=signatures-by-treaty&treatynum=225
The Council of Europe convention creates a treaty framework for governing AI consistently with human rights, democracy, and the rule of law.
[Covington et al. 2016] Deep neural networks for YouTube recommendations. https://dl.acm.org/doi/10.1145/2959100.2959190
YouTube documents a two-stage recommendation architecture in which a candidate model reduces millions of videos to hundreds before a ranking model selects a few dozen for display.
[D'Amour et al. 2022] Underspecification presents challenges for credibility in modern machine learning. Journal of Machine Learning Research. https://jmlr.org/papers/v23/20-1335.html
This paper argues that modern ML pipelines are often underspecified: many predictors have similar held-out performance but behave differently in deployment.
[Dabney et al. 2020] A distributional code for value in dopamine-based reinforcement learning. Nature.
Dabney et al. find heterogeneous dopamine responses consistent with a population code over distributions of discounted returns.
[Dahlgren et al. 2023] Safetensors library security assessment. https://huggingface.co/datasets/safetensors/trail_of_bits_audit_repot/resolve/main/SOW-TrailofBits-EleutherAI_HuggingFace-v1.2.pdf
The audit found no critical code-execution path in the reviewed loader after reported validation, overflow, parser, and conversion issues were addressed, while explicitly not claiming proof of absence.
[Dai et al. 2024] DeepSeekMoE: Towards ultimate expert specialization in mixture-of-experts language models. Association for Computational Linguistics. https://arxiv.org/abs/2401.06066
DeepSeekMoE proposes fine-grained expert segmentation and shared expert isolation in MoE language models to achieve stronger expert specialization, matching dense model performance with far less computation.
[Daly 2006] A higher order estimate of the optimum checkpoint interval for restart dumps. Future Generation Computer Systems. https://doi.org/10.1016/j.future.2004.11.016
Daly derives higher-order checkpoint intervals for Poisson failures and shows where the first-order approximation loses accuracy.
[Daly 2006] A higher order estimate of the optimum checkpoint interval for restart dumps. Future Generation Computer Systems. https://laro.lanl.gov/esploro/outputs/journalArticle/A-higher-order-estimate-of-the/9916364420003761
Daly derives higher-order checkpoint intervals for Poisson failures and shows where the first-order approximation loses accuracy.
[Dao et al. 2022] FlashAttention: Fast and memory-efficient exact attention with IO-awareness. https://arxiv.org/abs/2205.14135
Dense FlashAttention computes exact attention with IO-aware tiling, reducing HBM traffic and avoiding materialization of the full score and probability matrices.
[Dao and Gu 2024] Transformers are ssms: Generalized models and efficient algorithms through structured state space duality. PMLR. https://arxiv.org/abs/2405.21060
Mamba-2 unifies selective SSMs and attention via a structured state space duality (SSD) framework over semiseparable matrices, yielding a 2-8x faster SSM layer competitive with Transformers on language modeling.
[Dao 2024] FlashAttention-2: Faster attention with better parallelism and work partitioning. https://proceedings.iclr.cc/paper_files/paper/2024/hash/98ed250b203d1ac6b24bbcf263e3d4a7-Abstract-Conference.html
FlashAttention-2 improves GPU work partitioning and parallelism across sequence length to achieve roughly 2x speedup over FlashAttention, reaching 50-73% of theoretical peak FLOPs/s on A100.
[Dathathri et al. 2024] Scalable watermarking for identifying large language model outputs. Nature. https://www.nature.com/articles/s41586-024-08025-4
SynthID-Text productionizes generation-time watermarking with negligible latency, was deployed on live Gemini traffic in a 20-million-response test, and is open-sourced.
[Davidson 1970] On extending the bradley–terry model to accommodate ties in paired comparison experiments. Journal of the American Statistical Association. https://doi.org/10.1080/01621459.1970.10481082
Davidson extends paired-comparison ranking with an explicit tie outcome and parameter instead of silently converting ties into binary wins.
[Dean and Ghemawat 2004] MapReduce: Simplified data processing on large clusters. USENIX Association. https://www.usenix.org/conference/osdi-04/mapreduce-simplified-data-processing-large-clusters
MapReduce shows how decomposition, intermediate data, scheduling, failure recovery, and reduction become one runtime contract.
[Dean and Barroso 2013] The tail at scale. Communications of the ACM. https://research.google/pubs/the-tail-at-scale/
Large fan-out systems amplify latency variability, motivating bounded work, careful retries, and tail-aware operational controls.
[Debenedetti et al. 2024] AgentDojo: a dynamic environment to evaluate prompt injection attacks and defenses for LLM agents. https://proceedings.nips.cc/paper_files/paper/2024/hash/97091a5177d8dc64b1da8bf3e1f6fb54-Abstract-Datasets_and_Benchmarks_Track.html
AgentDojo evaluates both benign utility and indirect prompt-injection security in a dynamic tool-using environment; its scenarios provide comparative evidence, not a universal defense certificate.
[Debenedetti et al. 2025] Defeating prompt injections by design. arXiv preprint arXiv:2503.18813. https://arxiv.org/abs/2503.18813
CaMeL is a system-layer defense against prompt injection in LLM agents that enforces capability-based security policies on control and data flows without modifying the underlying model, solving 77% of AgentDojo tasks with provable security.
[DeepSeek-AI 2024] DeepSeek-V3 technical report. https://arxiv.org/abs/2412.19437
Reports DeepSeek-V3, a 671B-parameter Mixture-of-Experts model with 37B active per token, trained on 14.8T tokens with fp8 matmuls and auxiliary-loss-free load balancing, rivaling closed models at low cost.
[DeepSeek-AI 2024] DeepSeek-V2: a strong, economical, and efficient mixture-of-experts language model. https://arxiv.org/abs/2405.04434
DeepSeek-V2 introduces multi-head latent attention, which caches a compressed joint KV latent plus a decoupled positional key component.
[DeepSeek-AI 2025] DeepSeek-V3.2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556. https://arxiv.org/abs/2512.02556
DeepSeek-V3.2 introduces DSA: a low-dimensional lightning indexer selects top-k latent entries for the main MLA path, while the indexer itself still performs lower-cost quadratic scoring.
[DeepSeek-AI 2026] DeepSeek-V4: Towards highly efficient million-token context intelligence. https://arxiv.org/abs/2606.19348
DeepSeek-V4 describes architecture and training choices intended to make million-token context processing more efficient.
[Défossez et al. 2023] High fidelity neural audio compression. Transactions on Machine Learning Research (TMLR). https://arxiv.org/abs/2210.13438
EnCodec is a real-time neural audio codec using a streaming encoder-decoder with residual vector quantization and adversarial training, achieving state-of-the-art compression at 1.5 to 24 kbps.
[Défossez et al. 2024] Moshi: a speech-text foundation model for real-time dialogue. arXiv preprint arXiv:2410.00037. https://arxiv.org/abs/2410.00037
Moshi is a full-duplex speech-text model with parallel user and system audio streams and a text Inner Monologue; the paper reports 160 ms theoretical and about 200 ms practical latency.
[Dehghani et al. 2021] The benchmark lottery. arXiv preprint arXiv:2107.07002. https://arxiv.org/abs/2107.07002
Across several machine-learning domains, the authors show that task, dataset, and metric choices can materially change comparative conclusions and rankings.
[Delétang et al. 2024] Language modeling is compression. https://openreview.net/forum?id=jznbgiynus
Coupled to an entropy coder and excluding model weights, Chinchilla 70B losslessly encodes the study's image and audio chunks at 48.0% and 21.0% of raw size, below the corresponding chunked PNG and FLAC rates.
[Dettmers et al. 2022] LLM.int8(): 8-bit matrix multiplication for transformers at scale. https://proceedings.neurips.cc/paper_files/paper/2022/file/c3ba4962c05c49636d4c6206a97e9c8a-Paper-Conference.pdf
LLM.int8() combines vector-wise INT8 matrix multiplication with a higher-precision path for systematic activation outlier dimensions in the evaluated large transformers.
[Dettmers et al. 2023] QLoRA: Efficient finetuning of quantized llms. https://proceedings.neurips.cc/paper_files/paper/2023/hash/1feb87871436031bdc0f2beaa62a049b-Abstract-Conference.html
QLoRA stores a frozen base in 4-bit NF4 and trains higher-precision LoRA adapters, combining double quantization and paged optimizers to reduce memory in its experiments.
[Dhanuskodi et al. 2023] Creating the first confidential gpus. ACM Queue. https://doi.org/10.1145/3623393.3623391
Explains the architecture, protected data paths, attestation, and stated physical assumptions of NVIDIA's first confidential GPU generation.
[Dillon et al. 2026] Shifting work patterns with generative AI. American Economic Review: Insights. https://www.aeaweb.org/articles?id=10.1257/aeri.20250275
Integrated AI access reduced email-session time but produced no detectable average change in meetings, Word time, completed documents, or several coordination measures.
[Ding et al. 2026] Opus: Photonic rail-optimized fabric in ML datacenters. arXiv preprint arXiv:2602.12521. https://arxiv.org/abs/2602.12521
Opus evaluates reconfigurable optical rails under an explicit training-overhead model; the evidence is a preprint, not a production deployment.
[Dodge et al. 2021] Documenting large webtext corpora: a case study on the colossal clean crawled corpus. Association for Computational Linguistics. https://aclanthology.org/2021.emnlp-main.98/
An audit of C4 finds unexpected sources, machine-generated text, benchmark examples, and disproportionate removal of text from and about minority groups by a blocklist filter.
[Dong et al. 2023] RAFT: Reward rAnked FineTuning for generative foundation model alignment. Transactions on Machine Learning Research. https://arxiv.org/abs/2304.06767
RAFT aligns generative models by iteratively sampling outputs, scoring them with a reward model, and fine-tuning only on the top-ranked subset, replacing PPO with a stable SFT-style loop.
[Dong et al. 2025] XGrammar: Flexible and efficient structured generation engine for large language models. https://proceedings.mlsys.org/paper_files/paper/2025/hash/5c20ca4b0b20b0bd2f1d839dc605e70f-Abstract-Conference.html
XGrammar accelerates context-free grammar execution with prechecked tokens, persistent parser stacks, and overlap between grammar work and accelerator execution.
[Dong et al. 2025] PUMA: Secure inference of LLaMA-7B in five minutes. Security and Safety. https://arxiv.org/abs/2307.12533
Presents secure three-party LLaMA-7B inference with detailed protocol, network, and workload measurements; the title refers to the measured inference task, not a universal per-token constant.
[Donnellan et al. 2025] Uptime institute global data center survey 2025. https://intelligence.uptimeinstitute.com/resource/uptime-institute-global-data-center-survey-2025
The survey reports operator evidence on datacenter capacity, outages, staffing, sustainability, and infrastructure investment.
[Douillard and others 2026] Decoupled DiLoCo for resilient distributed pre-training. arXiv preprint arXiv:2604.21428. https://arxiv.org/abs/2604.21428
Decoupled DiLoCo uses asynchronous learners and a central synchronizer; its 88% goodput result comes from simulated failure exposure, distinct from its cross-region experiment.
[Dror et al. 2018] The hitchhiker's guide to testing statistical significance in natural language processing. https://aclanthology.org/P18-1128/
Dror and colleagues survey statistical significance tests for NLP and match common experimental designs and metrics to appropriate tests.
[Du et al. 2022] GLaM: Efficient scaling of language models with mixture-of-experts. PMLR. https://arxiv.org/abs/2112.06905
GLaM scales a decoder-only language model to 1.2T parameters via sparsely activated MoE, matching or exceeding GPT-3 on 29 NLP tasks while using one-third the training energy.
[Du et al. 2024] Improving factuality and reasoning in language models through multiagent debate. PMLR. https://proceedings.mlr.press/v235/du24e.html
An early empirical study reports gains from multi-round debate on selected reasoning and factuality tasks, motivating but not universally validating the pattern.
[Du et al. 2025] Context length alone hurts LLM performance despite perfect retrieval. https://arxiv.org/abs/2510.05381
Even with perfect retrieval, LLM reasoning performance degrades substantially (13.9%–85%) as input context length grows, revealing that input length alone hurts performance independent of distraction.
[Duan et al. 2024] Do membership inference attacks work on large language models?. https://arxiv.org/abs/2402.07841
Duan et al. find that tested membership-inference attacks on large language model pretraining data often perform near chance and show how distribution shift can create misleading results.
[Dubois et al. 2024] Length-controlled AlpacaEval: a simple way to debias automatic evaluators. https://openreview.net/forum?id=CybBmzWBX0
Length-Controlled AlpacaEval uses regression to estimate automatic preference at zero response-length difference, making the counterfactual estimand and its task-specific assumptions explicit.
[Dwork and Roth 2014] The algorithmic foundations of differential privacy. Now Publishers. https://doi.org/10.1561/0400000042
Dwork and Roth develop the formal foundations of differential privacy, including adjacency, privacy parameters, composition, and standard mechanisms.
[Dwork et al. 2015] The reusable holdout: Preserving validity in adaptive data analysis. Science. https://pubmed.ncbi.nlm.nih.gov/26250683/
The paper formalizes why ordinary holdout guarantees fail under adaptive reuse and develops a controlled mechanism for answering repeated queries while limiting overfitting.
[Dwork et al. 2015] The reusable holdout: Preserving validity in adaptive data analysis. Science. https://doi.org/10.1126/science.aaa9375
The reusable-holdout framework shows why adaptive repeated access to evaluation results can invalidate ordinary holdout inference even when the held-out records remain secret.
[Edge et al. 2024] From local to global: a graph RAG approach to query-focused summarization. arXiv preprint arXiv:2404.16130. https://arxiv.org/abs/2404.16130
GraphRAG derives an entity graph and community summaries for global, query-focused sensemaking over a corpus.
[Efron 1979] Bootstrap methods: Another look at the jackknife. The Annals of Statistics. https://doi.org/10.1214/aos/1176344552
Efron's paper introduces the bootstrap as a general resampling method for estimating uncertainty without deriving a closed-form sampling distribution.
[El-Yaniv and Wiener 2010] On the foundations of noise-free selective classification. Journal of Machine Learning Research. https://jmlr.csail.mit.edu/papers/v11/el-yaniv10a.html
The paper formalizes selective classification, where a model may abstain, and relates prediction coverage to the risk on accepted predictions.
[Elazar et al. 2024] What's in my big data?. https://proceedings.iclr.cc/paper_files/paper/2024/hash/1f7336fd66b6e6e63d1801fdd5930a5a-Abstract-Conference.html
WIMBD provides scalable count-and-search analyses for large text corpora and documents duplicates, synthetic and low-quality text, personal information, toxicity, and benchmark contamination in widely used datasets.
[Eldan and Li 2023] TinyStories: How small can language models be and still speak coherent english?. https://arxiv.org/abs/2305.07759
TinyStories introduces a synthetic dataset of simple short stories to show that language models below 10 million parameters can generate fluent, coherent English text with emergent reasoning.
[Elhage et al. 2021] A mathematical framework for transformer circuits. https://transformer-circuits.pub/2021/framework/index.html
This paper introduces a mathematical framework for mechanistic interpretability of transformers, reverse-engineering attention-only models via the residual stream, QK/OV circuits, and induction heads.
[Elhage et al. 2022] Toy models of superposition. https://arxiv.org/abs/2209.10652
Toy ReLU networks demonstrate conditions under which a network can represent more features than available dimensions through superposition; whether the toy results generalize directly to large language models remains an empirical question.
[Eloundou et al. 2024] GPTs are gpts: Labor market impact potential of llms. Science. https://doi.org/10.1126/science.adj0998
The paper estimates how much occupational task time could be affected by language models while holding task quality constant; it does not measure realized workplace effects.
[Elsworth et al. 2025] Measuring the environmental impact of delivering AI at google scale. arXiv preprint arXiv:2508.15734. https://arxiv.org/abs/2508.15734
Google measures the full-stack energy, carbon, and water footprint of Gemini Apps inference in production, finding the median text prompt consumes 0.24 Wh and showing a 44x emissions reduction over one year.
[Endsley 1995] Toward a theory of situation awareness in dynamic systems. Human Factors. https://doi.org/10.1518/001872095779049543
Endsley formalizes situation awareness as a dynamic decision-making construct shaped by perception, comprehension, projection, workload, complexity, and automation.
[Enevoldsen and others 2025] MMTEB: Massive multilingual text embedding benchmark. https://proceedings.iclr.cc/paper_files/paper/2025/hash/fc0e3f908a2116ba529ad0a1530a3675-Abstract-Conference.html
MMTEB expands embedding evaluation across hundreds of tasks and many languages, and provides reduced benchmark subsets for more affordable model comparison.
[Engels et al. 2025] Scaling laws for scalable oversight. arXiv preprint arXiv:2504.18530. https://arxiv.org/abs/2504.18530
This paper fits role-specific domain-Elo curves for four simplified oversight games and studies nested oversight within that model; its estimates depend on the games, proxy, fitted form, and nesting assumptions.
[Epoch AI 2026] FrontierMath tier 4 (v2). https://epoch.ai/benchmarks/frontiermath-tier-4-v2
FrontierMath v2 retained 338 problems after a major correction, illustrating that answer checking does not itself validate benchmark prompts and reference answers.
[Erdil 2025] Inference economics of language models. arXiv preprint arXiv:2506.04645. https://arxiv.org/abs/2506.04645
A theoretical model derives Pareto frontiers of serial token generation speed versus cost per token across arithmetic, memory, and network constraints.
[Erdogan et al. 2025] Plan-and-act: Improving planning of agents for long-horizon tasks. PMLR. https://proceedings.mlr.press/v267/erdogan25a.html
Plan-and-Act trains a planner to produce high-level plans and uses a separate executor to translate those plans into environment actions on long-horizon web tasks.
[Eren et al. 2026] Financing the AI infrastructure boom: On- and off-balance sheet borrowing. https://www.bis.org/publ/qtrpdf/r_qt2603u.htm
BIS researchers document rising hyperscaler bond issuance and, separately, joint-venture and special-purpose-vehicle financing with debt-like economic exposure.
[Es et al. 2024] RAGAs: Automated evaluation of retrieval augmented generation. Association for Computational Linguistics. https://aclanthology.org/2024.eacl-demo.16/
RAGAS proposes model-mediated, reference-free proxies for faithfulness, answer relevance, and context relevance; its archival paper does not define context recall.
[Esser et al. 2021] Taming transformers for high-resolution image synthesis. https://arxiv.org/abs/2012.09841
VQGAN combines a CNN-based discrete codebook with a transformer to enable autoregressive synthesis of high-resolution images at megapixel scale.
[Esser et al. 2024] Scaling rectified flow transformers for high-resolution image synthesis. https://arxiv.org/abs/2403.03206
SD3 introduces improved noise sampling for rectified flow training and a new transformer architecture with bidirectional text-image token mixing, demonstrating predictable scaling for high-resolution text-to-image synthesis.
[Ethayarajh 2019] How contextual are contextualized word representations? Comparing the geometry of BERT, elmo, and GPT-2 embeddings. https://aclanthology.org/D19-1006/
This paper analyzes the geometry of contextualized word representations in BERT, ELMo, and GPT-2, finding they are anisotropic and that upper layers produce more context-specific embeddings.
[Ethayarajh et al. 2024] Model alignment as prospect theoretic optimization. PMLR. https://arxiv.org/abs/2402.01306
KTO aligns LLMs using a Kahneman-Tversky prospect theory objective that learns from binary desirability signals instead of preference pairs, matching or exceeding DPO at scales from 1B to 30B parameters.
[European Commission 2025] Guidelines on the obligations for general-purpose AI model providers. https://digital-strategy.ec.europa.eu/en/policies/guidelines-gpai-providers
European Commission guidance explains how providers can interpret and document the AI Act obligations that apply to general-purpose AI models.
[European Commission 2025] The general-purpose AI code of practice. https://digital-strategy.ec.europa.eu/en/policies/contents-code-gpai
Provides a voluntary route for demonstrating specified GPAI transparency, copyright, safety, and security obligations.
[European Data Protection Board 2026] Respect individuals' rights. https://www.edpb.europa.eu/sme/be-compliant/respect-individuals-rights_en
The EDPB guide explains how to implement data-subject rights, including machine-readable portability and the limits around inferred data.
[European Parliament and Council of the European Union 2016] Regulation (EU) 2016/679, general data protection regulation. https://eur-lex.europa.eu/eli/reg/2016/679/oj
The GDPR defines data-protection principles and conditional rights including access, correction, erasure, and portability.
[European Parliament and Council of the European Union 2019] Directive (EU) 2019/790 on copyright and related rights in the digital single market. https://eur-lex.europa.eu/eli/dir/2019/790/oj
Creates EU text-and-data-mining provisions, including the Article 4 mechanism for expressly reserving rights.
[European Parliament and Council of the European Union 2024] Regulation (EU) 2024/1689 of the european parliament and of the council laying down harmonised rules on artificial intelligence (artificial intelligence act). https://eur-lex.europa.eu/eli/reg/2024/1689/oj
The EU AI Act establishes risk-based duties for AI systems, separate rules for general-purpose models, and a staged application timeline.
[European Parliament and Council of the European Union 2026] Regulation (EU) 2026/1744 amending Regulations (EU) 2024/1689, (EU) 2018/1139 and (EU) 2023/1230 as regards the simplification of the implementation of harmonised rules on artificial intelligence (Digital Omnibus on AI). https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L_202601744
The 2026 Digital Omnibus amends the AI Act and related legislation to simplify implementation while changing specified obligations and dates.
[European Parliament and Council of the European Union 2026] Regulation (EU) 2026/1744 amending regulation (EU) 2024/1689 as regards the simplification of certain rules. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32026R1744
Moves specified high-risk-system dates while leaving the GPAI application and enforcement dates intact.
[European Parliament and Council of the European Union 2026] Regulation (EU) 2026/1744 amending regulations as regards the simplification of the implementation of harmonised rules on artificial intelligence (Digital Omnibus on AI). https://eur-lex.europa.eu/eli/reg/2026/1744/oj
The 2026 Digital Omnibus amends the AI Act and related legislation to simplify implementation while changing specified obligations and dates.
[European Union 2016] Regulation (EU) 2016/679 (general data protection regulation). https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng
The GDPR defines data-protection principles and conditional rights including access, correction, erasure, and portability.
[Evtimov et al. 2025] WASP: Benchmarking web agent security against prompt injection attacks. https://proceedings.neurips.cc/paper_files/paper/2025/hash/1c9818387f5dd0a0bc151214660f059d-Abstract-Datasets_and_Benchmarks_Track.html
An end-to-end benchmark showing that realistic, low-effort prompt injections can divert capable web agents; partial attacker progress is separated from complete attacker success.
[Fabijan et al. 2019] Diagnosing sample ratio mismatch in online controlled experiments: a taxonomy and rules of thumb for practitioners. https://doi.org/10.1145/3292500.3330722
The paper classifies causes of sample ratio mismatch and explains why allocation mismatches often reveal experiment defects.
[Fang et al. 2025] Towards general agentic intelligence via environment scaling. arXiv preprint arXiv:2509.13311. https://arxiv.org/abs/2509.13311
AgentScaler automatically constructs heterogeneous fully simulated environments and trains in two phases, first for general function-calling capability and then for domain specialization, reporting gains on tau-bench, tau2-Bench, and ACEBench.
[Faysse et al. 2025] ColPali: Efficient document retrieval with vision language models. https://proceedings.iclr.cc/paper_files/paper/2025/hash/99e9e141aafc314f76b0ca3dd66898b3-Abstract-Conference.html
Applies late-interaction vision-language representations directly to page-image retrieval.
[Featonby 2021] Making retries safe with idempotent apis. https://aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs/
AWS describes request identifiers and idempotency contracts that let a service retry an operation without repeating its intended effect.
[Federal Energy Regulatory Commission 2025] FERC directs nation's largest grid operator to create new rules to embrace innovation and protect consumers. https://www.ferc.gov/news-events/news/fact-sheet-ferc-directs-nations-largest-grid-operator-create-new-rules-embrace
FERC directs PJM to develop clearer rules for large co-located loads while addressing reliability, interconnection, and consumer-cost concerns.
[Federal Energy Regulatory Commission 2026] FERC launches aggressive targeted action to speed large load integration. https://www.ferc.gov/news-events/news/ferc-launches-aggressive-targeted-action-speed-large-load-integration
FERC outlines a targeted process for accelerating large-load integration while preserving reliability and transparent cost allocation.
[Federal Trade Commission, Office of Technology Staff 2025] Partnerships between cloud service providers and AI developers. https://www.ftc.gov/reports/ftc-staff-report-ai-partnerships-investments-6b-study
FTC staff document specific equity, governance, cloud-spend, exclusivity, integration, and information-sharing terms in selected cloud-provider and AI-developer partnerships without making an illegality finding.
[Fedus et al. 2022] Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research. https://arxiv.org/abs/2101.03961
Switch Transformer simplifies MoE routing to a single expert per token, enabling trillion-parameter sparse models that achieve up to 7x pre-training speedup over T5 at equal FLOPs.
[Feng et al. 2025] Group-in-group policy optimization for LLM agent training. https://proceedings.neurips.cc/paper_files/paper/2025/hash/420c9f777c0b4f78d515e53cf74d58b2-Abstract-Conference.html
GiGPO constructs local comparison groups when equivalent environment states recur, supplying step-level relative advantages without another critic or additional rollouts.
[Fielding 2000] Architectural styles and the design of network-based software architectures. https://ics.uci.edu/~fielding/pubs/dissertation/top.htm
Fielding explains how architectural constraints and explicit interface semantics shape evolvable networked systems.
[Fielding et al. 2022] HTTP semantics. https://www.rfc-editor.org/rfc/rfc9110.html
RFC 9110 defines HTTP semantics but leaves 402 Payment Required reserved, so modern payment protocols are conventions layered on HTTP rather than a revision of HTTP itself.
[FinOps Foundation 2026] What is FinOps?. https://www.finops.org/introduction/what-is-finops/
FinOps defines technology cost management as a collaborative operating model across engineering, finance, product, and business teams.
[FinOps Open Cost and Usage Specification 2026] FOCUS: FinOps open cost and usage specification. https://focus.finops.org/
FOCUS defines uniform technology billing datasets. Version 1.4 adds invoice, billing-period, and commitment detail, while earlier virtual-currency fields support token purchase and consumption records.
[Firecracker Maintainers 2026] Snapshotting support. https://github.com/firecracker-microvm/firecracker/blob/main/docs/snapshotting/snapshot-support.md
Firecracker snapshots serialize guest memory and emulated hardware state, while disk files, connectivity, compatibility, and omitted runtime data require separate handling.
[Ford 1957] Solution of a ranking problem from binary comparisons. The American Mathematical Monthly. https://doi.org/10.1080/00029890.1957.11989117
Ford states the comparison-graph condition needed for a finite, unique normalized maximum-likelihood ranking from binary outcomes.
[Forero Cuervo 2017] Handling overload. O'Reilly Media. https://sre.google/sre-book/handling-overload/
Google SRE guidance treats overload through capacity limits, admission control, graceful degradation, and load shedding rather than unbounded queues.
[Frantar et al. 2023] GPTQ: Accurate post-training quantization for generative pre-trained transformers. https://openreview.net/forum?id=tcbBPnfwxS
GPTQ is a one-shot post-training method based on approximate second-order information; in the evaluated models it reached 3-4 bits per weight with small accuracy changes, including 3-bit OPT-175B inference on one GPU.
[Friedl et al. 2026] The impossibility of eliciting latent knowledge. arXiv preprint arXiv:2606.12268. https://arxiv.org/abs/2606.12268
Under its causal formalization and distribution-shift assumptions, the paper proves that the specified class of behavior-only feedback cannot guarantee an honest agent with certainty.
[Frostig et al. 2018] Compiling machine learning programs via high-level tracing. https://mlsys.org/Conferences/doc/2018/146.pdf
This paper presents high-level tracing of pure array programs and composable transformations for differentiation, vectorization, and compilation.
[Fu et al. 2024] Break the sequential dependency of LLM inference using lookahead decoding. PMLR. https://proceedings.mlr.press/v235/fu24a.html
Lookahead decoding uses Jacobi iteration and an n-gram pool to generate and verify several continuations without a learned draft model.
[Fu et al. 2025] AReaL: a large-scale asynchronous reinforcement learning system for language reasoning. https://proceedings.neurips.cc/paper_files/paper/2025/hash/33c00862bfa29ac72ecf630a41e19352-Abstract-Conference.html
AReaL continuously generates rollouts while the learner updates independently, controlling policy staleness and reporting up to 2.77x speedup on evaluated math and code workloads.
[Furlanello et al. 2018] Born again neural networks. https://proceedings.mlr.press/v80/furlanello18a.html
Born-again networks train a student from a teacher with the same architecture and report that the student can outperform its teacher.
[Fuster Baggetto and Fresno 2022] Is anisotropy really the cause of BERT embeddings not being semantic?. https://aclanthology.org/2022.findings-emnlp.314/
This study finds that anisotropy is not the root cause of poor semantic isometry in the tested BERT contextual representations and separates it from frequency and tokenization biases.
[G7 Industry, Digital and Technology Ministers 2026] G7 vision on AI openness: Opportunities and shared language. https://www.entreprises.gouv.fr/files/files/Actualites/2026/g7/vision-AI-openness-opportunities-and-shared-language.pdf
The G7 vocabulary separates open-source systems, openly licensed weights and deployment code, and weight downloads governed by restrictive terms.
[Gale et al. 2023] MegaBlocks: Efficient sparse training with mixture-of-experts. https://proceedings.mlsys.org/paper_files/paper/2023/hash/5a54f79333768effe7e8927bcccffe40-Abstract-mlsys2023.html
MegaBlocks uses block-sparse GPU kernels to execute dynamically routed expert batches without dropping tokens or padding every expert to a fixed capacity.
[Gandhi et al. 2026] Endless terminals: Scaling RL environments for terminal agents. arXiv preprint arXiv:2601.16443. https://arxiv.org/abs/2601.16443
An automated pipeline synthesizes 3,255 terminal tasks covering file operations, database work, and scripting, after which plain PPO raises Qwen2.5-7B from 10.7 to 53.3 percent on the authors' development set with gains that transfer to human-curated terminal benchmarks.
[Gao et al. 2021] SimCSE: Simple contrastive learning of sentence embeddings. https://aclanthology.org/2021.emnlp-main.552/
SimCSE uses dropout views for unsupervised positives and natural-language-inference pairs for supervised contrastive sentence embeddings.
[Gao et al. 2023] Scaling laws for reward model overoptimization. https://proceedings.mlr.press/v202/gao23h.html
This paper measures reward model overoptimization in RLHF, deriving scaling laws showing how gold reward degrades as a function of KL divergence from the initial policy for both RL and best-of-n sampling.
[Gao et al. 2023] PAL: Program-aided language models. PMLR. https://arxiv.org/abs/2211.10435
Program-aided Language Models use LLMs to translate natural-language reasoning problems into executable programs, then offload computation to a Python interpreter.
[Gao et al. 2023] Enabling large language models to generate text with citations. https://aclanthology.org/2023.emnlp-main.398/
ALCE evaluates long-form answers along answer quality, citation correctness, and citation completeness rather than treating citation presence as sufficient.
[Gao et al. 2025] Scaling and evaluating sparse autoencoders. https://proceedings.iclr.cc/paper_files/paper/2025/hash/42ef3308c230942d223c411adf182c88-Abstract-Conference.html
This paper proposes k-sparse autoencoders (TopK SAEs) that directly control sparsity to improve the reconstruction-sparsity frontier, establishes clean scaling laws for SAE size and sparsity, and trains a 16 million latent SAE on GPT-4 activations.
[Gao et al. 2025] Weight-sparse transformers have interpretable circuits. arXiv preprint arXiv:2511.13653. https://arxiv.org/abs/2511.13653
This work studies transformers with mostly zero weights on small, simple tasks, finding circuits that are easier to inspect while noting substantial scaling challenges.
[Gazitt et al. 2026] Authorization API 1.0. https://openid.net/specs/authorization-api-1_0.html
AuthZEN defines subject, action, resource, context, and Boolean decision exchange while leaving policy language and enforcement to implementations.
[Gebru et al. 2021] Datasheets for datasets. Communications of the ACM. https://doi.org/10.1145/3458723
Datasheets for Datasets proposes standardized dataset documentation covering motivation, composition, collection, preprocessing, uses, distribution, and maintenance.
[Geifman and El-Yaniv 2017] Selective classification for deep neural networks. https://proceedings.neurips.cc/paper/2017/hash/4a8423d5e91fda00bb7e46540e2b0cf1-Abstract.html
Selective classification adds an abstention option and evaluates the resulting trade-off between prediction coverage and error risk.
[Geiger et al. 2025] Causal abstraction: a theoretical foundation for mechanistic interpretability. Journal of Machine Learning Research. https://jmlr.org/papers/v26/23-0058.html
Causal abstraction formalizes when a simpler explanatory model preserves the intervention behavior of the original system.
[Gema et al. 2025] Are we done with MMLU?. Association for Computational Linguistics. https://aclanthology.org/2025.naacl-long.262/
A manual re-annotation of 5,700 MMLU questions estimates that 6.49% carry ground-truth errors, with some subject subsets far worse, and releases the corrected MMLU-Redux subset.
[Geng et al. 2025] Mean flows for one-step generative modeling. arXiv preprint arXiv:2505.13447. https://arxiv.org/abs/2505.13447
MeanFlow learns an average velocity field designed to map noise to data in a single generation step.
[Geng et al. 2025] JSONSchemaBench: a rigorous benchmark of structured outputs for language models. arXiv preprint arXiv:2501.10868. https://arxiv.org/abs/2501.10868
JSONSchemaBench evaluates structured-generation systems across real schemas, official schema tests, efficiency, constraint coverage, and output quality.
[Gerstgrasser et al. 2024] Is model collapse inevitable? Breaking the curse of recursion by accumulating real and synthetic data. arXiv preprint arXiv:2404.01413. https://arxiv.org/abs/2404.01413
Across the tested recursive-training protocols, retaining prior real and synthetic data performs better than repeatedly replacing the dataset.
[ggml Project 2026] GGUF file format specification. https://github.com/ggml-org/ggml/blob/6af560d55df03ad92116e3c0a697779584477e85/docs/gguf.md
GGUF is an extensible inference container for GGML-based runtimes, designed for fast loading and commonly self-contained, while the current specification also supports shards and sidecars.
[GGML project 2023] GGUF file format. https://github.com/ggml-org/ggml/blob/master/docs/gguf.md
GGUF is a file format for storing model tensors and metadata for GGML and related executors; tensor quantization schemes and backend support are separate concerns.
[Ghazvininejad et al. 2019] Mask-predict: Parallel decoding of conditional masked language models. https://arxiv.org/abs/1904.09324
Mask-Predict introduces conditional masked language models with an iterative parallel decoding algorithm that achieves near-autoregressive translation quality while decoding significantly faster.
[Gheshlaghi Azar et al. 2024] A general theoretical paradigm to understand learning from human preferences. PMLR. https://arxiv.org/abs/2310.12036
This paper introduces ΨPO, a general preference optimization objective that unifies RLHF and DPO as special cases, and proposes IPO, which bypasses the Bradley-Terry assumption to avoid overfitting.
[Ghosh et al. 2023] GenEval: An object-focused framework for evaluating text-to-image alignment. https://arxiv.org/abs/2310.11513
GENEVAL is an object-focused benchmark that uses object detection and discriminative vision models to evaluate compositional text-to-image alignment across tasks like counting, position, and attribute binding.
[Glassman et al. 1995] The millicent protocol for inexpensive electronic commerce. https://www.w3.org/Conferences/WWW4/Papers/246/
Millicent shows how merchant scrip and aggregation can reduce payment overhead without claiming that technical feasibility creates user demand.
[Glazer et al. 2024] FrontierMath: a benchmark for evaluating advanced mathematical reasoning in AI. arXiv preprint arXiv:2411.04872. https://arxiv.org/abs/2411.04872
FrontierMath uses original, expert-written problems and automated answer checks to measure advanced mathematical reasoning while reducing contamination risk.
[Gloeckle et al. 2024] Better & faster large language models via multi-token prediction. PMLR. https://proceedings.mlr.press/v235/gloeckle24a.html
Multi-token prediction trains several future-token heads as an auxiliary objective; those heads can also supply candidates for self-speculative decoding.
[Google 2026] Gemma terms of use. https://ai.google.dev/gemma/terms
The current Gemma terms govern use, modification, hosted service, and distribution of covered Gemma versions and derivative models, while directing Gemma 4 users to a separate license.
[Google 2026] Gemma 4 model card. https://ai.google.dev/gemma/docs/core/model_card_4
The Gemma 4 model card records the released models' architecture, intended uses, evaluations, safety work, and known limitations.
[Google n.d.] OpenAI compatibility. https://ai.google.dev/gemini-api/docs/openai
Google documents how OpenAI-library calls map to Gemini and recommends native Gemini integration when an application needs provider-specific capabilities.
[Google Cloud 2025] Announcing the Agent2Agent protocol (A2A). https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/
Google introduced A2A as an open protocol for agents built by different vendors or frameworks to communicate and coordinate tasks.
[Google DeepMind 2026] Frontier safety framework, version 3.1. https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/strengthening-our-frontier-safety-framework/frontier-safety-framework_3-1.pdf
Defines critical capability levels per risk domain, with early-warning evaluations and an alert threshold set below each one so the safety buffer fires before the level is reached. Deployment and security mitigations are separate tiers, and a reviewed safety case is what lets a launch proceed.
[Google DeepMind 2026] Gemini 3 deep think. https://deepmind.google/models/gemini-deep-think/
Google DeepMind reports an ARC Prize-verified 84.6 percent ARC-AGI-2 score while retaining important protocol and evaluation-set qualifications.
[Google Platforms and Devices et al. 2025] Private AI compute in the cloud. https://services.google.com/fh/files/misc/private_ai_compute_technical_brief.pdf
Describes SEV-SNP frontends, hardened TPU infrastructure, internal attestation, encrypted links, IP blinding, and which external verification features remain future work.
[Grattafiori and others 2024] The llama 3 herd of models. https://arxiv.org/abs/2407.21783
Meta presents Llama 3, a herd of dense Transformer language models at 8B, 70B, and 405B parameters trained on 15T tokens, achieving quality comparable to GPT-4 across diverse tasks.
[Graves et al. 2006] Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. https://www.cs.toronto.edu/~graves/icml_2006.pdf
CTC trains sequence models without frame-level alignments by summing the probabilities of paths that collapse to the target label sequence.
[Graves 2012] Sequence transduction with recurrent neural networks. ICML 2012 Representation Learning Workshop; arXiv:1211.3711. https://arxiv.org/abs/1211.3711
Graves 2012 introduces the RNN Transducer (RNN-T), an end-to-end probabilistic sequence transduction system that jointly models input-output and output-output dependencies without requiring a pre-defined alignment.
[Greenblatt et al. 2024] AI control: Improving safety despite intentional subversion. PMLR. https://proceedings.mlr.press/v235/greenblatt24a.html
AI control red-teams monitoring and editing protocols in an APPS programming testbed with GPT-4 treated as untrusted and GPT-3.5 as trusted.
[Greenblatt et al. 2024] Alignment faking in large language models. arXiv preprint arXiv:2412.14093. https://arxiv.org/abs/2412.14093
Claude 3 Opus selectively complied under an experimentally supplied training/deployment distinction and conflicting objective; this demonstrates behavior under the setup, not ordinary prevalence or privileged access to model intent.
[Greshake et al. 2023] Not what you've signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection. arXiv preprint arXiv:2302.12173. https://arxiv.org/abs/2302.12173
Indirect prompt injection places adversarial instructions in third-party data that an LLM-integrated application later retrieves, exposing data and tool-control risks.
[Grinsztajn et al. 2022] Why do tree-based models still outperform deep learning on typical tabular data?. https://arxiv.org/abs/2207.08815
Across 45 curated tabular datasets with 3,000 to 10,000 samples, tested tree ensembles outperform tested deep networks; the analysis highlights robustness to uninformative features and irregular target functions.
[gRPC Authors 2024] Cancellation. https://grpc.io/docs/guides/cancellation/
The gRPC guide explains that cancellation must propagate through an RPC chain and that application handlers may need to check for cancellation cooperatively.
[Grünwald 2004] A tutorial introduction to the minimum description length principle. arXiv preprint math/0406077. https://arxiv.org/abs/math/0406077
Grünwald's tutorial introduces Rissanen's MDL principle, framing statistical model selection and inductive inference as data compression to automatically guard against overfitting.
[Gu et al. 2018] Non-autoregressive neural machine translation. https://arxiv.org/abs/1711.02281
The Non-Autoregressive Transformer generates a length-conditioned target in parallel using fertility as a latent alignment variable; its reported quality and latency trade-off varies across decoding operating points.
[Gu et al. 2019] Levenshtein transformer. https://arxiv.org/abs/1905.11006
Levenshtein Transformer (LevT) is a partially autoregressive sequence generation model that uses insertion and deletion operations with dual-policy imitation learning, achieving comparable translation quality with up to 5x decoding speedup.
[Gu et al. 2019] Tiresias: a GPU cluster manager for distributed deep learning. USENIX Association. https://www.usenix.org/conference/nsdi19/presentation/gu
Tiresias studies admission, placement, and scheduling policies for distributed deep-learning jobs with all-or-nothing resource requirements.
[Gu et al. 2022] Efficiently modeling long sequences with structured state spaces. https://arxiv.org/abs/2111.00396
S4 uses a structured state matrix to make long state-space convolution kernels practical, with near-linear filter generation and strong results on long-range sequence benchmarks.
[Gu and Dao 2024] Mamba: Linear-time sequence modeling with selective state spaces. https://arxiv.org/abs/2312.00752
Mamba introduces selective SSMs with input-dependent parameters and a hardware-aware parallel scan, achieving Transformer-quality language modeling with linear-time inference and training.
[Gu et al. 2024] MiniLLM: Knowledge distillation of large language models. https://arxiv.org/abs/2306.08543
MiniLLM replaces the forward Kullback-Leibler objective with reverse Kullback-Leibler optimized on student samples, and reports better precision, calibration, and long-text generation for students from 120M to 13B parameters.
[Guan et al. 2024] Deliberative alignment: Reasoning enables safer language models. arXiv preprint arXiv:2412.16339. https://arxiv.org/abs/2412.16339
Deliberative Alignment teaches models safety specifications and trains them to reason over those specifications before answering, improving jailbreak robustness while reducing over-refusal.
[Guha et al. 2026] OpenThoughts: Data recipes for reasoning models. https://openreview.net/forum?id=7xjoTuaNmN
More than 1,000 controlled experiments produce a 1.2-million-example reasoning corpus whose matched Qwen2.5-7B experiments show continued gains from scaling a curated data recipe.
[Gulati et al. 2020] Conformer: Convolution-augmented transformer for speech recognition. https://arxiv.org/abs/2005.08100
Conformer combines convolution and self-attention in a single encoder block to capture both local and global audio features, achieving state-of-the-art ASR results on LibriSpeech.
[Gulrajani and Hashimoto 2023] Likelihood-based diffusion language models. https://arxiv.org/abs/2305.18619
Plaid 1B is a 1-billion-parameter likelihood-based diffusion language model that outperforms GPT-2 124M on zero-shot perplexity benchmarks via algorithmic improvements and compute-optimal scaling laws.
[Gunasekar et al. 2023] Textbooks are all you need. https://arxiv.org/abs/2306.11644
phi-1 is a 1.3B-parameter code model trained on 7B tokens from filtered code and synthetically generated textbook and exercise data; the paper reports 50.6% pass@1 on HumanEval after fine-tuning.
[Gunjal et al. 2025] Rubrics as rewards: Reinforcement learning beyond verifiable domains. arXiv preprint arXiv:2507.17746. https://arxiv.org/abs/2507.17746
Rubrics as Rewards decomposes open-ended judgments into per-criterion, checklist-style rubrics graded by a model and used as the reward for on-policy RL, outperforming LLM-as-judge Likert baselines on HealthBench and GPQA-Diamond.
[Guo et al. 2017] On calibration of modern neural networks. Proceedings of the 34th International Conference on Machine Learning. https://proceedings.mlr.press/v70/guo17a.html
Guo et al. show that modern neural networks can be accurate yet miscalibrated, and that simple temperature scaling can substantially improve probabilistic calibration.
[Guo et al. 2020] Certified data removal from machine learning models. https://proceedings.mlr.press/v119/guo20c.html
Guo et al. define certified removal by indistinguishability from retained-data retraining and construct a mechanism for regularized linear models under stated assumptions.
[Guo et al. 2024] The curious decline of linguistic diversity: Training language models on synthetic text. Association for Computational Linguistics. https://aclanthology.org/2024.findings-naacl.228/
The experiments find that repeated training on generated text can reduce lexical, syntactic, and semantic diversity in later model output.
[Guo et al. 2025] DeepSeek-R1 incentivizes reasoning in llms through reinforcement learning. Nature. https://www.nature.com/articles/s41586-025-09422-z
DeepSeek-R1 shows that large-scale RL with verifiable rewards can elicit long reasoning behavior, with R1-Zero using RL without supervised cold start and R1 adding multi-stage training for readability and stability.
[Guo et al. 2025] GenEnv: Difficulty-aligned co-evolution between LLM agents and environment simulators. arXiv preprint arXiv:2512.19682. https://arxiv.org/abs/2512.19682
GenEnv pays a generative environment simulator through a curriculum reward that favors tasks matched to the agent's current ability, so the agent and its environment supply co-evolve.
[Guo et al. 2026] IH-Challenge: a training dataset to improve instruction hierarchy on frontier llms. arXiv preprint arXiv:2603.10521. https://arxiv.org/abs/2603.10521
IH-Challenge uses programmatically graded reinforcement-learning tasks and online adversarial generation to improve instruction-hierarchy robustness in a GPT-5-Mini experiment.
[Gupta et al. 2020] The architectural implications of facebook's DNN-based personalized recommendation. https://arxiv.org/abs/1906.03109
The paper characterizes Facebook recommendation models and shows how sparse embeddings, dense computation, and communication create distinct system bottlenecks.
[Gururangan et al. 2020] Don't stop pretraining: Adapt language models to domains and tasks. Association for Computational Linguistics. https://arxiv.org/abs/2004.10964
Shows that additional pretraining on domain and task corpora improves downstream performance across biomedical, computer science, news, and review tasks.
[gVisor Project 2026] Security model. https://gvisor.dev/docs/architecture_guide/security/
The gVisor security model describes the Sentry, Gofer, restricted host interfaces, filesystem modes, and remaining trusted surface of the userspace-kernel design.
[Ha and Schmidhuber 2018] World models. arXiv preprint arXiv:1803.10122. https://arxiv.org/abs/1803.10122
World Models combines a visual autoencoder, recurrent latent dynamics, and a controller, including experiments that train the controller inside learned rollouts.
[Haas et al. 2025] SimpleQA Verified: a reliable factuality benchmark to measure parametric knowledge. arXiv preprint arXiv:2509.07968. https://arxiv.org/abs/2509.07968
SimpleQA Verified creates a filtered 1,000-question derivative after auditing labels, duplicates, and topic balance; it does not alter the original benchmark in place.
[Hafner et al. 2019] Learning latent dynamics for planning from pixels. https://arxiv.org/abs/1811.04551
PlaNet learns stochastic latent dynamics from images and uses online planning to solve continuous-control tasks in the DeepMind Control Suite.
[Hales et al. 2017] A formal proof of the kepler conjecture. Forum of Mathematics, Pi. https://doi.org/10.1017/fmp.2017.1
The Flyspeck project gives a formal proof of the Kepler conjecture using HOL Light and Isabelle, showing how a major mathematical result can be checked by proof assistants.
[Handa et al. 2025] Which economic tasks are performed with AI? Evidence from millions of claude conversations. https://arxiv.org/abs/2503.04761
The study maps millions of sampled Claude conversations to occupational tasks, describing product usage without observing users' jobs, accepted outputs, or productivity.
[Hao et al. 2023] Reasoning with language model is planning with world model. Association for Computational Linguistics. https://aclanthology.org/2023.emnlp-main.507/
RAP casts language-model reasoning as planning, using the model as a world model and tree search to evaluate candidate reasoning paths.
[Hardt 2012] The oauth 2.0 authorization framework. https://www.rfc-editor.org/rfc/rfc6749.html
OAuth separates the resource owner, client, authorization server, and resource server, and gives clients scoped access tokens rather than the owner's credentials.
[Hashimoto and Liang 2025] CS336: Language modeling from scratch. https://cs336.stanford.edu/spring2025/
Stanford's implementation-heavy language-modeling course walks through tokenizer construction, Transformer implementation, systems optimization, scaling laws, data processing, evaluation, and alignment.
[Hayes and Krippendorff 2007] Answering the call for a standard reliability measure for coding data. Communication Methods and Measures. https://doi.org/10.1080/19312450709336664
Hayes and Krippendorff argue for Krippendorff's alpha as a general reliability coefficient that handles many coders, missing data, and different measurement levels.
[He et al. 2019] Streaming end-to-end speech recognition for mobile devices. https://arxiv.org/abs/1811.06621
Google presents a streaming end-to-end RNN-T speech recognizer for mobile devices that outperforms a conventional CTC model in both latency and WER on voice search and dictation tasks.
[He 2025] Defeating nondeterminism in LLM inference. https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/
He traces temperature-zero inference variation to numerical kernels whose results depend on dynamic batching and develops batch-invariant alternatives.
[Hebb 1949] The organization of behavior: a neuropsychological theory. John Wiley & Sons.
Hebb proposes that learning strengthens synapses between co-active neurons (cells that fire together wire together), the basis of Hebbian learning.
[Heer 2019] Agency plus automation: Designing artificial intelligence into interactive systems. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.1807184115
Heer argues for interactive systems that use predictive models to augment human work through shared representations, review, revision, and dismissal.
[Heineman et al. 2025] Signal and noise: a framework for reducing uncertainty in language model evaluation. arXiv preprint arXiv:2508.13144. https://arxiv.org/abs/2508.13144
Signal is a benchmark's ability to separate stronger models from weaker ones and noise is its sensitivity to random variation between training steps; across 30 benchmarks and 375 open-weight models the ratio predicts which small-scale comparisons and scaling-law fits can be trusted.
[Hendrycks et al. 2020] Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300. https://arxiv.org/abs/2009.03300
MMLU introduces a 57-subject multiple-choice benchmark spanning STEM, humanities, and social sciences to measure text models' breadth of world knowledge in zero-shot and few-shot settings.
[Hermann and Del Balso 2017] Meet michelangelo: Uber's machine learning platform. https://www.uber.com/blog/michelangelo-machine-learning-platform/
Uber describes a platform that connects feature preparation, training, model management, deployment, online prediction, and monitoring.
[Hines et al. 2024] Defending against indirect prompt injection attacks with spotlighting. arXiv preprint arXiv:2403.14720. https://arxiv.org/abs/2403.14720
Spotlighting is a prompt engineering defense against indirect prompt injection attacks that transforms untrusted input text to signal its provenance, reducing attack success rate from over 50% to below 2% on GPT-family models.
[Hinton et al. 2015] Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531. https://arxiv.org/abs/1503.02531
Knowledge distillation trains a compact student on softened teacher output distributions rather than hard labels alone.
[Ho et al. 2020] Denoising diffusion probabilistic models. https://arxiv.org/abs/2006.11239
Ho et al. show that DDPM trained with a denoising score matching objective achieves high-quality image synthesis, reaching FID 3.17 on CIFAR10.
[Ho and Salimans 2022] Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598. https://arxiv.org/abs/2207.12598
Classifier-free guidance (CFG) jointly trains a conditional and unconditional diffusion model, mixing their score estimates at inference to trade off sample quality and diversity without requiring a separate classifier.
[Ho et al. 2022] Video diffusion models. https://arxiv.org/abs/2204.03458
Ho et al. extend image diffusion models to video via a space-time factorized 3D U-Net and reconstruction-guided conditional sampling, achieving state-of-the-art results on video generation and prediction benchmarks.
[Hoffmann et al. 2022] Training compute-optimal large language models. https://arxiv.org/abs/2203.15556
Finds near-equal compute-optimal scaling of model size and training tokens in its experiments; the compute-matched 70B Chinchilla model, trained on 1.4T tokens, outperforms several larger models.
[Hohnhold et al. 2015] Focusing on the long-term: It's good for users and business. https://research.google/pubs/focus-on-the-long-term-its-better-for-users-and-business/
Google describes experiments and metrics designed to detect when a short-term product gain harms longer-term user or business outcomes.
[Hollmann et al. 2025] Accurate predictions on small data with a tabular foundation model. Nature. https://www.nature.com/articles/s41586-024-08328-6
TabPFN, a transformer pretrained on synthetic tabular tasks, outperforms tuned tree ensembles in the authors' evaluations of datasets with up to 10,000 samples and 500 features; behavior beyond those limits requires further study.
[Hong et al. 2024] ORPO: Monolithic preference optimization without reference model. Association for Computational Linguistics. https://arxiv.org/abs/2403.07691
ORPO is a monolithic preference alignment algorithm that merges SFT and preference optimization into one step using an odds ratio penalty, eliminating the need for a reference model.
[Hooker 2021] The hardware lottery. Communications of the ACM. https://arxiv.org/abs/2009.06489
The hardware lottery describes how available software and hardware can favor some research ideas by making them easier to implement and evaluate than alternatives.
[Horvitz and Thompson 1952] A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association. https://www.stat.cmu.edu/~brian/905-2008/papers/Horvitz-Thompson-1952-jasa.pdf
The Horvitz-Thompson estimator weights sampled units by inverse inclusion probability to estimate population totals under unequal-probability sampling.
[Horvitz 1999] Principles of mixed-initiative user interfaces. https://doi.org/10.1145/302979.303030
Horvitz frames user interfaces as mixed-initiative systems in which humans and computers negotiate control rather than forcing either direct manipulation or full automation.
[Houlsby et al. 2019] Parameter-efficient transfer learning for NLP. https://arxiv.org/abs/1902.00751
Adapter modules inserted into BERT layers match full fine-tuning performance on GLUE while training only 3.6% as many parameters per task.
[Howard et al. 2021] Time-uniform, nonparametric, nonasymptotic confidence sequences. The Annals of Statistics. https://projecteuclid.org/journals/annals-of-statistics/volume-49/issue-2/Time-uniform-nonparametric-nonasymptotic-confidence-sequences/10.1214/20-AOS1991.full
Howard et al. develop confidence sequences whose coverage guarantee holds uniformly over time, enabling valid inference under repeated looks when their assumptions hold.
[Hsieh et al. 2023] Distilling step-by-step! Outperforming larger language models with less training data and smaller model sizes. Association for Computational Linguistics. https://aclanthology.org/2023.findings-acl.507/
Distilling Step-by-Step uses teacher-generated rationales as an auxiliary target in a multitask student objective and reports data-efficient gains on four NLP benchmarks.
[Hsieh et al. 2024] RULER: What's the real context size of your long-context language models?. arXiv preprint arXiv:2404.06654. https://arxiv.org/abs/2404.06654
Evaluates long-context models with configurable length and task complexity, showing that effective context can fall well below an advertised input limit on tasks beyond simple retrieval.
[Hsu et al. 2021] HuBERT: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing. https://arxiv.org/abs/2106.07447
HuBERT learns speech representations by predicting offline k-means cluster assignments on masked regions, matching or surpassing wav2vec 2.0 on ASR benchmarks.
[Hu et al. 2022] LoRA: Low-rank adaptation of large language models. https://arxiv.org/abs/2106.09685
LoRA freezes pretrained matrices and learns additive low-rank factors, reducing trainable and stored task-specific state in the paper's evaluated settings.
[Hu et al. 2024] MiniCPM: Unveiling the potential of small language models with scalable training strategies. https://arxiv.org/abs/2404.06395
Presents small (1.2B/2.4B) models that rival 7B-13B LLMs, using model wind-tunnel scaling experiments and a Warmup-Stable-Decay learning-rate schedule that enables continuous training.
[Hu et al. 2024] ELLA: Equip diffusion models with LLM for enhanced semantic alignment. arXiv preprint arXiv:2403.05135. https://arxiv.org/abs/2403.05135
ELLA is a lightweight LLM adapter for CLIP-based text-to-image diffusion models that uses a Timestep-Aware Semantic Connector to improve dense prompt following without retraining the U-Net or LLM.
[Hu et al. 2025] OpenRLHF: a ray-based easy-to-use, scalable and high-performance RLHF framework. Association for Computational Linguistics. https://aclanthology.org/2025.emnlp-demos.48/
OpenRLHF integrates Ray, vLLM, DeepSpeed, and Transformers and demonstrates one disaggregated organization of generation, scoring, and learning roles.
[Hu et al. 2026] Evaluating memory in LLM agents via incremental multi-turn interactions. https://arxiv.org/abs/2507.05257
MemoryAgentBench evaluates accurate retrieval, test-time learning, long-range understanding, and selective forgetting under incremental interaction.
[Huang et al. 2019] GPipe: Efficient training of giant neural networks using pipeline parallelism. https://arxiv.org/abs/1811.06965
GPipe proposes pipeline parallelism via micro-batch splitting to scale neural networks beyond single-accelerator memory limits with near-linear speedup across multiple accelerators.
[Huang et al. 2024] Compression represents intelligence linearly. https://openreview.net/forum?id=SHMj84U5SH
Across 31 public base LLMs and 12 benchmarks, average benchmark scores correlate with compression efficiency on external text corpora, with an overall Pearson correlation of -0.93.
[Huang et al. 2024] Collective constitutional AI: Aligning a language model with public input. https://arxiv.org/abs/2406.07814
Collective Constitutional AI demonstrates a process for sourcing principles from a target public, translating them into a constitution, and training and evaluating a model against it.
[Huang et al. 2024] Large language models cannot self-correct reasoning yet. https://openreview.net/forum?id=IkmD3fKBPQ
The experiments find that asking language models to revise reasoning without external feedback often fails to improve, and can reduce, answer accuracy.
[Huang et al. 2025] R-zero: Self-evolving reasoning LLM from zero data. arXiv preprint arXiv:2508.05004. https://arxiv.org/abs/2508.05004
R-Zero co-evolves Challenger and Solver models without a pre-existing task-label dataset, using majority-vote pseudo-labels rather than an exact verifier; the reported loop eventually degrades across iterations.
[Huang et al. 2025] Is best-of-N the best of them? Coverage, scaling, and optimality in inference-time alignment. PMLR. https://proceedings.mlr.press/v267/huang25c.html
Best-of-N selection can deteriorate as the candidate pool grows when an imperfect reward model increasingly selects outputs that exploit its errors.
[Huang et al. 2025] Environment scaling for interactive agentic experience collection: a survey. arXiv preprint arXiv:2511.09586. https://arxiv.org/abs/2511.09586
The survey organizes environment-scaling work into a generation, execution, and feedback loop, and treats environments as the producers of the experience data that agent training consumes.
[Huben et al. 2024] Sparse autoencoders find highly interpretable features in language models. https://proceedings.iclr.cc/paper_files/paper/2024/hash/1fa1ab11f4bd5f94b2ec20e794dbfa3b-Abstract-Conference.html
Sparse autoencoders trained on language-model activations produced features that scored as more interpretable than tested alternatives and supported finer causal localization on one task; the result is evidence for the method, not a guarantee of unique or complete feature recovery.
[Hubert et al. 2026] Olympiad-level formal mathematical reasoning with reinforcement learning. Nature. https://www.nature.com/articles/s41586-025-09833-y
AlphaProof uses AlphaZero-style reinforcement learning and Lean verification; it solved three manually formalized non-geometry problems at IMO 2024, while the combined AlphaProof and AlphaGeometry 2 system reached a silver-equivalent score.
[Hubinger et al. 2024] Sleeper agents: Training deceptive llms that persist through safety training. arXiv preprint arXiv:2401.05566. https://arxiv.org/abs/2401.05566
Some deliberately trained conditional backdoors in the tested models persisted through supervised fine-tuning, reinforcement learning, and adversarial training, demonstrating a proof of possibility rather than estimating natural prevalence.
[Hudgens and Halloran 2008] Toward causal inference with interference. Journal of the American Statistical Association. https://pmc.ncbi.nlm.nih.gov/articles/PMC2600548/
The paper defines direct, indirect, total, and overall causal effects when treatments spill over within groups.
[Hugging Face 2026] Hotswapping adapters. https://huggingface.co/docs/peft/package_reference/hotswap
The PEFT documentation describes LoRA hot-swapping and its constraints on adapter method, target modules, rank, scaling, and compiled model shapes.
[Hugging Face 2026] Models: loading custom code. https://huggingface.co/docs/transformers/en/models
Transformers documents that trust_remote_code executes custom model code from a repository and recommends reviewing that code and pinning its commit revision.
[Hugging Face 2026] Download files from the hub. https://huggingface.co/docs/huggingface_hub/en/guides/download
The Hub client can resolve a repository snapshot by branch, tag, pull request, or full commit hash and download either the whole snapshot or a filtered file set.
[Hugging Face n.d.] TRL documentation. https://huggingface.co/docs/trl
TRL documents current SFT, preference, reward-model, and reinforcement-learning trainers, including their expected dataset formats.
[Hugging Face n.d.] PEFT documentation. https://huggingface.co/docs/peft
PEFT documents parameter-efficient methods, adapter configuration, checkpoint formats, merging, and integrations with distributed training.
[Hui et al. 2024] Qwen2.5-coder technical report. arXiv preprint arXiv:2409.12186. https://arxiv.org/abs/2409.12186
Reports Qwen2.5-Coder, trained on 5.2T file-level tokens followed by roughly 300B repository-level long-context tokens, using code, text, and mathematics with next-token and fill-in-the-middle objectives.
[Humlum and Vestergaard 2025] Still waters, rapid currents: Early labor market transformation under generative AI. https://www.nber.org/papers/w33777
The linked Danish evidence finds rapid chatbot use and task reorganization but no detectable average effect on recorded hours or earnings two years after ChatGPT's launch.
[Huyen 2025] AI engineering: Building applications with foundation models. O'Reilly Media. https://www.oreilly.com/library/view/ai-engineering/9781098166298/
Chip Huyen's book covers AI engineering: building production applications on top of foundation models, including evaluation, adaptation techniques, and serving.
[Huynh and Hardouin 2023] PoisonGPT: How we hid a lobotomized LLM on hugging face to spread fake news. https://blog.mithrilsecurity.io/poisongpt-how-we-hid-a-lobotomized-llm-on-hugging-face-to-spread-fake-news/
The educational demonstration edits one GPT-J-6B association and publishes it under a one-letter namespace typo; it reports limited comparison checks and no victims or propagation.
[Hwang et al. 2025] Dynamic chunking for end-to-end hierarchical sequence modeling. arXiv preprint arXiv:2507.07955. https://arxiv.org/abs/2507.07955
H-Net learns content-dependent chunk boundaries inside a hierarchical byte model; its one-stage configuration outperforms the paper's compute- and data-matched BPE baseline.
[IEEE Electronics Packaging Society Test Technical Committee 2022] Heterogeneous integrated product testability: Best-known methods (BKM). https://cmte.ieee.org/eps-test/wp-content/uploads/sites/132/2022/01/IEEE_EPS_Test_Het_Int_Product_Testability_BKM_Final_v1_0-1-14-22-1.pdf
A technical committee guide to wafer, die, package, and system-level testing for heterogeneous integration, including the cost and coverage limits of known-good-die screening.
[Ilharco et al. 2023] Editing models with task arithmetic. https://arxiv.org/abs/2212.04089
Task arithmetic defines same-base fine-tuning deltas and reports that scaled addition or negation steers behavior in the paper's evaluated models and tasks.
[Inan et al. 2023] Llama guard: LLM-based input-output safeguard for human-AI conversations. arXiv preprint arXiv:2312.06674. https://arxiv.org/abs/2312.06674
Llama Guard is an instruction-tuned Llama2-7b model that classifies both user prompts and LLM responses against a customizable safety risk taxonomy, matching or exceeding existing content moderation APIs.
[Inception Labs et al. 2025] Mercury: Ultra-fast language models based on diffusion. arXiv preprint arXiv:2506.17298. https://arxiv.org/abs/2506.17298
Mercury reports high-throughput diffusion code generation on H100 hardware, but its headline rates are workload- and serving-stack-specific and the model sizes are not disclosed.
[International Energy Agency 2025] Energy and AI. https://www.iea.org/reports/energy-and-ai
The IEA models data-center electricity demand across computing and facility infrastructure and states the assumptions behind its energy scenarios.
[International Organization for Standardization 2020] Document management—portable document format—part 2: PDF 2.0. https://www.iso.org/standard/75839.html
ISO 32000-2 specifies the PDF 2.0 document format, including the objects, graphics, annotations, metadata, security, and conformance rules a processor may encounter.
[International Organization for Standardization 2026] ISO/CD 22144: Authenticity of information, content credentials. https://www.iso.org/standard/90726.html
The draft ISO standard develops interoperability requirements for content credentials and authenticity metadata.
[Internet Engineering Task Force 2026] Web bot auth working group charter. https://datatracker.ietf.org/wg/webbotauth/about/
The Web Bot Auth working group charter scopes standards for authenticating automated web clients without treating authentication as authorization.
[Irving et al. 2018] AI safety via debate. arXiv preprint arXiv:1805.00899. https://arxiv.org/abs/1805.00899
Debate proposes adversarial argument as an oversight method; the original paper's experiment used MNIST and a sparse classifier rather than a human judge.
[ISO/IEC 2023] ISO/IEC 42001:2023, information technology, artificial intelligence, management system. https://www.iso.org/standard/42001
The first international AI management-system standard: it certifies that an organization has documented policies, risk assessments, roles, and an improvement loop, not that any model behaves safely.
[Ivanov et al. 2021] Data movement is all you need: a case study on optimizing transformers. Proceedings of Machine Learning and Systems (MLSys). https://proceedings.mlsys.org/paper_files/paper/2021/hash/bc86e95606a6392f51f95a8de106728d-Abstract.html
A measured BERT case study showing how global layout and data-movement optimization can accelerate both a transformer layer and the complete model without changing its mathematics.
[Jacovi et al. 2025] The FACTS grounding leaderboard: Benchmarking LLMs' ability to ground responses to long-form input. arXiv preprint arXiv:2501.03200. https://arxiv.org/abs/2501.03200
The original FACTS Grounding benchmark uses long supplied documents, a separate eligibility gate, and three model judges to estimate response-level grounding under a versioned 2025 protocol.
[Jain et al. 2024] LiveCodeBench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974. https://arxiv.org/abs/2403.07974
LiveCodeBench uses recently published competition problems and covers generation, execution, test-output prediction, and self-repair to reduce exposure and broaden coding evaluation.
[Jain et al. 2025] R2E-gym: Procedural environments and hybrid verifiers for scaling open-weights SWE agents. arXiv preprint arXiv:2504.07164. https://arxiv.org/abs/2504.07164
R2E-Gym procedurally generates more than 8,700 software-engineering tasks from commits using test generation and back-translation, and combines execution-based with execution-free verifiers for test-time selection.
[JAX Authors 2026] The autodiff cookbook. https://docs.jax.dev/en/latest/notebooks/autodiff_cookbook.html
The cookbook develops Jacobian-vector and vector-Jacobian products and explains how input-output geometry determines the efficient mode.
[JAX Authors 2026] Just-in-time compilation. https://docs.jax.dev/en/latest/jit-compilation.html
JAX documentation explains tracing, compilation caching, static arguments, and the program-shape constraints that affect just-in-time compilation.
[Jeong et al. 2024] Adaptive-RAG: Learning to adapt retrieval-augmented large language models through question complexity. https://aclanthology.org/2024.naacl-long.389/
Adaptive-RAG routes questions among no retrieval, single-step retrieval, and iterative retrieval using an estimated complexity class.
[Jiang et al. 2024] Mixtral of experts. https://arxiv.org/abs/2401.04088
Mixtral 8x7B is a sparse MoE decoder-only model with 46.7B total parameters that activates only 12.9B per token, outperforming Llama 2 70B with 6x faster inference under an Apache 2.0 license.
[Jimenez et al. 2024] SWE-bench: Can language models resolve real-world GitHub issues?. https://proceedings.iclr.cc/paper_files/paper/2024/hash/edac78c3e300629acfe6cbe9ca88fb84-Abstract-Conference.html
SWE-bench is a benchmark of 2,294 real GitHub issue-resolution tasks across 12 Python repositories, where models must edit codebases to pass tests, and top models like Claude 2 solve only 1.96%.
[Jin et al. 2025] Search-R1: Training LLMs to reason and leverage search engines with reinforcement learning. https://arxiv.org/abs/2503.09516
Search-R1 trains language models to issue multiple search queries during reasoning and masks retrieved passages out of the policy loss while keeping them in context.
[Johari et al. 2022] Always valid inference: Continuous monitoring of a/B tests. Operations Research. https://doi.org/10.1287/opre.2021.2135
Johari and colleagues develop always-valid p-values and confidence intervals for experiments monitored continuously with optional stopping.
[Johnson et al. 2021] Billion-scale similarity search with GPUs. IEEE Transactions on Big Data. https://arxiv.org/abs/1702.08734
The FAISS paper describes GPU algorithms for large-scale exact and approximate nearest-neighbor search and evaluates their throughput and recall.
[Jones et al. 2020] OAuth 2.0 token exchange. https://www.rfc-editor.org/rfc/rfc8693.html
RFC 8693 defines token exchange for impersonation and delegation but does not guarantee attenuation or revocation propagation.
[Jouppi et al. 2017] In-datacenter performance analysis of a tensor processing unit. https://arxiv.org/abs/1704.04760
This paper documents the first production TPU, including its software-managed memory and 256 by 256 matrix multiply unit.
[Jouppi et al. 2023] <span class="nocase">TPU v4</span>: An optically reconfigurable supercomputer for machine learning with hardware support for embeddings. https://arxiv.org/abs/2304.01433
TPU v4 uses optical circuit switches to configure and reconfigure a large accelerator interconnect.
[Ju et al. 2024] NaturalSpeech 3: Zero-shot speech synthesis with factorized codec and diffusion models. https://arxiv.org/abs/2403.03100
NaturalSpeech 3 proposes a factorized codec (FACodec) and factorized diffusion models that decompose speech into disentangled subspaces for zero-shot TTS, achieving human-level naturalness on multi-speaker data.
[Kahneman 2011] Thinking, fast and slow. Farrar, Straus,Giroux.
Kahneman contrasts fast intuitive System 1 thinking with slow deliberate System 2 reasoning, and the cognitive biases each produces.
[Kalai and Vempala 2024] Calibrated language models must hallucinate. https://arxiv.org/abs/2311.14648
The paper proves conditions under which calibrated language models must still make errors on facts that cannot be distinguished from their training evidence.
[Kalai et al. 2025] Why language models hallucinate. arXiv preprint arXiv:2509.04664. https://arxiv.org/abs/2509.04664
The paper separates statistical causes in pretraining from post-training and evaluation incentives that make guessing score better than abstaining.
[Kandpal et al. 2022] Deduplicating training data mitigates privacy risks in language models. https://proceedings.mlr.press/v162/kandpal22a.html
Kandpal et al. measure a superlinear relationship between duplication and memorization and substantially lower extraction risk after deduplicating the studied language-model corpora.
[Kang et al. 2025] How far is video generation from world model: a physical law perspective. https://openreview.net/forum?id=DLlVjZQ7vD
Controlled experiments report strong in-distribution imitation but weak out-of-distribution physical-law extrapolation for the tested video generators and scaling settings.
[Kantamneni et al. 2025] Are sparse autoencoders useful? A case study in sparse probing. https://proceedings.mlr.press/v267/kantamneni25a.html
A probing study across 113 datasets finds that sparse autoencoder (SAE) latents fail to consistently outperform simple baselines for LLM activation probing under data scarcity, class imbalance, label noise, or covariate shift.
[Kanzhelev et al. 2021] Trace context. https://www.w3.org/TR/trace-context/
W3C Trace Context standardizes distributed correlation headers; those headers are not an authorization channel and must avoid sensitive data.
[Kaplan et al. 2020] Scaling laws for neural language models. https://arxiv.org/abs/2001.08361
Establishes that language-model loss falls as a power law in model size, dataset size, and compute, and that compute-optimal training favors very large models trained on relatively little data, stopped before convergence.
[Kapoor and Narayanan 2023] Leakage and the reproducibility crisis in machine-learning-based science. Patterns. https://doi.org/10.1016/j.patter.2023.100804
Kapoor and Narayanan classify common forms of data leakage in machine-learning science and show how they invalidate reported generalization.
[Kapoor et al. 2024] On the societal impact of open foundation models. https://arxiv.org/abs/2403.07918
This position paper analyzes open-weight foundation models through benefits, risks, and marginal risk relative to existing technologies, keeping competition claims conditional on complementary access and capability.
[Karpinska et al. 2021] The perils of using mechanical turk to evaluate open-ended text generation. Association for Computational Linguistics. https://aclanthology.org/2021.emnlp-main.97/
Karpinska et al. show that crowd workers can fail to distinguish human and model-written open-ended stories, while expert teachers and paired examples provide stronger evaluation signals.
[Karpukhin et al. 2020] Dense passage retrieval for open-domain question answering. https://aclanthology.org/2020.emnlp-main.550/
DPR trains separate query and passage encoders and demonstrates strong dense retrieval on several open-domain question-answering datasets.
[Karras et al. 2022] Elucidating the design space of diffusion-based generative models. https://arxiv.org/abs/2206.00364
Karras et al. decompose diffusion model training and sampling into a clean design space, identifying improvements to preconditioning and schedules that set new FID records on CIFAR-10 and ImageNet-64.
[Katz et al. 2024] Thought of search: Planning with language models through the lens of efficiency. https://proceedings.neurips.cc/paper_files/paper/2024/hash/fa080fe0f218871faec1d8ba20e491d5-Abstract-Conference.html
The paper analyzes language-model planning as search and compares methods by solution quality and the computation spent exploring candidate thoughts.
[Keller et al. 2026] Expanding the AI evaluation toolbox with statistical models. https://www.nist.gov/publications/expanding-ai-evaluation-toolbox-statistical-models
NIST distinguishes accuracy on a fixed benchmark from generalized accuracy over related tasks and explains why evaluation assumptions determine valid uncertainty estimates.
[Kenton et al. 2024] On scalable oversight with weak llms judging strong llms. https://proceedings.neurips.cc/paper_files/paper/2024/hash/899511e37a8e01e1bd6f6f1d377cc250-Abstract-Conference.html
Across extractive question answering, mathematics, coding, logic, and multimodal tasks, debate beat consultancy under assigned positions, but its advantage over direct question answering was mixed outside information-asymmetric extractive tasks.
[Khalifa et al. 2026] Process reward models that think. Transactions on Machine Learning Research. https://arxiv.org/abs/2504.16828
ThinkPRM is a generative PRM that verbalizes step-by-step verification as a chain of thought, outperforming discriminative PRMs and LLM-as-judge while training on roughly 1% of the process labels in PRM800K.
[Khan et al. 2024] Debating with more persuasive llms leads to more truthful answers. PMLR. https://proceedings.mlr.press/v235/khan24a.html
In information-asymmetric QuALITY questions with verified quotes, debate improved nonexpert model and human judge accuracy over naive and consultancy baselines; the result is evidence for that setting, not a general oversight guarantee.
[Khashabi et al. 2022] GENIE: Toward reproducible and standardized human evaluation for text generation. Association for Computational Linguistics. https://aclanthology.org/2022.emnlp-main.787/
GENIE studies human-evaluation design choices for text generation and introduces a standardized platform that improves reproducibility across tasks and annotator populations.
[Khattab and Zaharia 2020] ColBERT: Efficient and effective passage search via contextualized late interaction over BERT. https://arxiv.org/abs/2004.12832
Introduces token-level late interaction with precomputed document representations for retrieval.
[Khazatsky et al. 2024] DROID: a large-scale in-the-wild robot manipulation dataset. https://arxiv.org/abs/2403.12945
DROID collects 76,000 real-robot manipulation demonstrations, about 350 hours, across 564 scenes using a shared Franka setup.
[Kim and Rush 2016] Sequence-level knowledge distillation. https://arxiv.org/abs/1606.07947
Sequence-level knowledge distillation trains a small student NMT model on teacher beam-search outputs, yielding a 10x faster student that matches teacher BLEU with greedy decoding.
[Kim and Rush 2016] Sequence-level knowledge distillation. Association for Computational Linguistics. https://aclanthology.org/D16-1139/
Sequence-level distillation trains a student on teacher-decoded sequences, emphasizing selected modes of a structured prediction distribution.
[Kim et al. 2024] OpenVLA: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246. https://arxiv.org/abs/2406.09246
OpenVLA is an open 7B-parameter vision-language-action model trained on 970,000 real-world robot demonstrations, with released checkpoints and fine-tuning code.
[Kim et al. 2025] Pre-training under infinite compute. arXiv preprint arXiv:2509.14786. https://arxiv.org/abs/2509.14786
The study uses regularization, parameter scaling, and ensembling to improve fixed-data pretraining, with its 5.17x result estimated from an asymptotic fit at 200M tokens.
[Kimi Team 2025] Kimi K2: Open agentic intelligence. arXiv preprint arXiv:2507.20534. https://arxiv.org/abs/2507.20534
Kimi K2 is a 1T-total, 32B-active MoE model pre-trained on 15.5T tokens with MuonClip, a Muon variant whose QK-clip removes attention-logit blowups, with zero loss spikes over the run.
[Kimi Team et al. 2025] Kimi k1.5: Scaling reinforcement learning with LLMs. arXiv preprint arXiv:2501.12599. https://arxiv.org/abs/2501.12599
Kimi k1.5 reports long-context RL and four long2short mechanisms: weight merging, shortest-correct rejection sampling, preference training, and a length-constrained RL stage.
[Kirchenbauer et al. 2023] A watermark for large language models. https://arxiv.org/abs/2301.10226
This paper proposes a statistical watermarking framework for LLM output that embeds detectable signals into generated text by biasing token sampling toward a randomized "green list," detectable from as few as 25 tokens without model access.
[Kirchstetter et al. 2025] DOE data center load flexibility workshop summary. https://eta.lbl.gov/publications/doe-data-center-load-flexibility
The workshop report identifies technical, market, and operational requirements for using datacenter loads as flexible grid resources.
[Kirk et al. 2024] The PRISM alignment dataset: What participatory, representative and individualised human feedback reveals about the subjective and multicultural alignment of large language models. https://arxiv.org/abs/2404.16019
PRISM links fine-grained feedback from 1,500 participants in 75 countries to participant profiles across 8,011 conversations, exposing subjective and multicultural variation in alignment preferences.
[Kirkpatrick and others 2017] Overcoming catastrophic forgetting in neural networks. https://www.pnas.org/doi/full/10.1073/pnas.1611835114
Elastic weight consolidation limits catastrophic forgetting by penalizing changes to parameters estimated to be important for earlier tasks.
[Knight and Leveson 1986] An experimental evaluation of the assumption of independence in multi-version programming. IEEE Transactions on Software Engineering. https://libraopen.library.virginia.edu/entities/publication/4ac33eeb-79b4-46e4-aef9-f6ec56a62286
The experiment finds correlated failures among independently developed program versions, challenging the assumption that their errors are independent.
[Kocsis and Szepesvári 2006] Bandit based monte-carlo planning. Springer. https://doi.org/10.1007/11871842_29
UCT applies an upper-confidence-bound rule to Monte Carlo tree search so node selection balances exploration with estimated value.
[Kohavi et al. 2009] Controlled experiments on the web: Survey and practical guide. Data Mining and Knowledge Discovery. https://doi.org/10.1007/s10618-008-0114-1
Kohavi and colleagues explain randomization, power, metrics, variance, and common validity failures in online controlled experiments.
[Kojima et al. 2022] Large language models are zero-shot reasoners. https://arxiv.org/abs/2205.11916
Zero-shot-CoT uses a task-agnostic reasoning prompt followed by a separate answer-extraction prompt, improving several arithmetic, symbolic, and logical-reasoning benchmarks without task-specific few-shot examples.
[Komatsuzaki et al. 2023] Sparse upcycling: Training mixture-of-experts from dense checkpoints. https://arxiv.org/abs/2212.05055
Sparse upcycling initializes a MoE model from a pretrained dense checkpoint, outperforming both dense continuation and MoE training from scratch at roughly 50% of the original pretraining cost.
[Köpf et al. 2023] OpenAssistant conversations: Democratizing large language model alignment. arXiv preprint arXiv:2304.07327. https://arxiv.org/abs/2304.07327
OpenAssistant Conversations releases a crowd-sourced alignment corpus containing conversation trees, message ratings, and multilingual human feedback.
[Korbak et al. 2025] A sketch of an AI control safety case. arXiv preprint arXiv:2501.17315. https://arxiv.org/abs/2501.17315
This safety-case sketch makes explicit the additional claims required to move from a control evaluation to deployment, including attack elicitation, evaluation-to-deployment transfer, and conservative risk extrapolation.
[Korbak et al. 2025] Chain of thought monitorability: a new and fragile opportunity for AI safety. arXiv preprint arXiv:2507.11473. https://arxiv.org/abs/2507.11473
A position paper signed across frontier laboratories argues that human-language reasoning traces give an oversight channel that ordinary engineering choices can remove, and asks developers to evaluate monitorability and to weigh it when changing training or architecture.
[Korinek and Vipra 2025] Concentrating intelligence: Scaling and market structure in artificial intelligence. Economic Policy. https://doi.org/10.1093/epolic/eiae057
Korinek and Vipra analyze scale, scope, feedback, inertia, and integration as mechanisms that could concentrate frontier-model markets while treating future tipping as conditional.
[Korthikanti et al. 2022] Reducing activation recomputation in large transformer models. https://arxiv.org/abs/2205.05198
This paper introduces sequence parallelism and selective activation recomputation to reduce activation memory by 5x and cut activation recomputation overhead by over 90% when training large transformer models with tensor parallelism.
[Koster et al. 2022] Robots exclusion protocol. https://www.rfc-editor.org/rfc/rfc9309.html
Standardizes robots.txt as a crawler-preference protocol and explicitly distinguishes it from access authorization.
[Kreutzer and others 2022] Quality at a glance: An audit of web-crawled multilingual datasets. Transactions of the Association for Computational Linguistics. https://aclanthology.org/2022.tacl-1.4/
A human audit of 205 language-specific corpora from five multilingual web datasets finds systematic labeling and content-quality failures, especially in lower-resource languages.
[KServe Contributors 2026] V2 inference protocol. https://kserve.github.io/website/docs/concepts/architecture/data-plane/v2-protocol
KServe V2 specifies health, metadata, and inference operations over HTTP or gRPC; application semantics such as chat templates, streaming, tool calls, and sampling defaults require separate contract tests.
[Kubernetes 2025] Multi-tenancy. https://kubernetes.io/docs/concepts/security/multi-tenancy/
Kubernetes documents the isolation and fairness tools that AI platforms inherit and extend for model, cache, index, and tool boundaries.
[Kubernetes Authors 2023] CSI volume cloning. https://kubernetes.io/docs/concepts/storage/volume-pvc-datasource/
Kubernetes supports PVC-to-PVC cloning through compatible CSI drivers, while the backend determines copy implementation and performance.
[Kubernetes Authors 2024] Volume snapshots. https://kubernetes.io/docs/concepts/storage/volume-snapshots/
Kubernetes exposes VolumeSnapshot through CSI when the driver and storage backend implement snapshot support.
[Kubernetes Authors 2026] Volumes. https://kubernetes.io/docs/concepts/storage/volumes/
Kubernetes distinguishes Pod-scoped ephemeral volumes from persistent volumes that outlive an individual Pod; topology, access, reclaim, and backup remain separate properties.
[Kubernetes Authors 2026] Jobs. https://kubernetes.io/docs/concepts/workloads/controllers/job/
Kubernetes documents that a Job can sometimes start the same program twice even with one completion and parallelism of one, so workloads must tolerate duplicates.
[Kubernetes Authors 2026] Resource quotas. https://kubernetes.io/docs/concepts/policy/resource-quotas/
Kubernetes ResourceQuota constrains aggregate resource and object consumption within a namespace.
[Kubernetes Authors 2026] Resource management for pods and containers. https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
Kubernetes keeps a Pod pending when its resource request cannot be satisfied and documents resource requests and limits as scheduler and runtime controls.
[Kubernetes SIG Scheduling 2026] All-or-nothing scheduling. https://kueue.sigs.k8s.io/docs/concepts/all_or_nothing/
Kueue's all-or-nothing scheduling admits a workload only when the cluster can place its required group of pods together.
[Kubernetes SIG Scheduling n.d.] Kueue overview. https://kueue.sigs.k8s.io/docs/overview/
Kueue manages when quota-consuming workloads are admitted, wait, or are preempted while leaving pod placement, autoscaling, and job lifecycle to their respective Kubernetes components.
[Kuchnik et al. 2022] Plumber: Diagnosing and removing performance bottlenecks in machine learning data pipelines. https://proceedings.mlsys.org/paper_files/paper/2022/hash/d0e90e9a9310570dfa643aa3b2da6e89-Abstract.html
Plumber uses resource-aware pipeline analysis to locate input bottlenecks and tune parallelism, prefetching, and caching.
[Kudo and Richardson 2018] SentencePiece: a simple and language independent subword tokenizer and detokenizer for neural text processing. Association for Computational Linguistics. https://aclanthology.org/D18-2012/
SentencePiece is a language-independent subword tokenizer and detokenizer that trains directly from raw sentences using BPE or unigram language model, enabling purely end-to-end text processing without pre-tokenization.
[Kudo 2018] Subword regularization: Improving neural network translation models with multiple subword candidates. Association for Computational Linguistics. https://aclanthology.org/P18-1007/
Subword regularization trains NMT models on multiple probabilistically sampled subword segmentations and introduces a unigram language model segmentation algorithm as a probabilistic alternative to BPE.
[Kumar et al. 2023] High-fidelity audio compression with improved RVQGAN. https://arxiv.org/abs/2306.06546
Kumar et al. present Improved RVQGAN, a universal neural audio codec that compresses 44.1 kHz audio to discrete tokens at 8 kbps (90x compression) with higher fidelity than EnCodec and SoundStream.
[Kumar et al. 2025] OverThink: Slowdown attacks on reasoning LLMs. arXiv preprint arXiv:2502.02542. https://arxiv.org/abs/2502.02542
OverThink constructs slowdown attacks that induce reasoning models to spend excess tokens and latency before producing an answer.
[Kung 1982] Why systolic architectures?. Computer. https://www.eecs.harvard.edu/~htk/publication/1982-kung-why-systolic-architecture.pdf
Kung explains systolic architectures as regular arrays that rhythmically move data through processing elements to exploit parallelism and locality.
[Kusupati et al. 2022] Matryoshka representation learning. https://proceedings.neurips.cc/paper_files/paper/2022/hash/c32319f4868da7613d78af9993100e42-Abstract-Conference.html
Matryoshka Representation Learning attaches objectives to selected vector prefixes so one model can expose several tested representation dimensions without another inference pass.
[Kutasov et al. 2025] SHADE-Arena: Evaluating sabotage and monitoring in LLM agents. arXiv preprint arXiv:2506.15740. https://arxiv.org/abs/2506.15740
SHADE-Arena pairs long-horizon main tasks with hidden harmful side objectives and separately measures task completion, sabotage, and transcript monitoring; it is not a complete control-protocol evaluation.
[Kwa et al. 2025] Measuring AI ability to complete long software tasks. https://proceedings.neurips.cc/paper_files/paper/2025/hash/85069585133c4c168c865e65d72e9775-Abstract-Conference.html
A peer-reviewed study fits agent success against human task duration and reports a long-run doubling trend on self-contained software tasks.
[Kwa 2026] A simpler AI timelines model predicts 99% AI R&D automation in ~2032. https://metr.org/notes/2026-02-10-simpler-ai-timelines-model/
Thomas Kwa presents an exploratory eight-parameter scenario for AI R-and-D automation and explicitly assigns little weight to its exact 2032 date.
[Kwon et al. 2023] Efficient memory management for large language model serving with PagedAttention. ACM. https://doi.org/10.1145/3600006.3613165
vLLM uses PagedAttention to place fixed-token KV-cache blocks non-contiguously; its reported throughput gains are scoped to the complete evaluated system, workloads, models, and baselines.
[Lakens et al. 2018] Equivalence testing for psychological research: a tutorial. Advances in Methods and Practices in Psychological Science. https://doi.org/10.1177/2515245918770963
The tutorial explains how predeclared equivalence bounds can support evidence that an effect is smaller than a practically meaningful threshold; a nonsignificant zero-effect test cannot.
[Lambert et al. 2024] RewardBench: Evaluating reward models for language modeling. arXiv preprint arXiv:2403.13787. https://arxiv.org/abs/2403.13787
RewardBench evaluates explicit and implicit reward models on structured chat, reasoning, and safety comparisons, exposing weaknesses that aggregate in-distribution accuracy can hide.
[Lambert et al. 2024] Tulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124. https://arxiv.org/abs/2411.15124
Tulu 3 is a fully open post-training recipe for Llama 3.1 base models, combining SFT, DPO, and RLVR with released data, weights, and training code.
[Lamport et al. 1982] The byzantine generals problem. ACM Transactions on Programming Languages and Systems. https://lamport.azurewebsites.net/pubs/the-byz-generals.pdf
The foundational paper states the assumptions and fault bounds for agreement when some distributed participants may behave arbitrarily.
[LangChain 2026] Use time travel. https://docs.langchain.com/oss/python/langgraph/use-time-travel
LangGraph distinguishes replay from fork: replay re-executes nodes after a checkpoint, while a fork creates a new checkpoint branch without modifying the original history.
[LangChain 2026] Persistence. https://docs.langchain.com/oss/python/langgraph/persistence
LangGraph documents checkpoints, threads, state history, and persistence semantics for graph executions.
[Lanham et al. 2023] Measuring faithfulness in chain-of-thought reasoning. arXiv preprint arXiv:2307.13702. https://arxiv.org/abs/2307.13702
Intervention experiments test whether chain-of-thought text reflects the causes of a model's answer and find several forms of unfaithful reasoning.
[Lattner et al. 2021] MLIR: Scaling compiler infrastructure for domain specific computation. https://arxiv.org/abs/2002.11054
MLIR provides extensible dialects and progressive lowering so domain-specific representations can coexist and share compiler infrastructure.
[Le et al. 2023] Voicebox: Text-guided multilingual universal speech generation at scale. https://arxiv.org/abs/2306.15687
Voicebox is a non-autoregressive flow-matching model trained on 50K+ hours of speech to perform zero-shot TTS, denoising, and content editing via in-context learning, outperforming VALL-E on both intelligibility and audio similarity.
[Leask et al. 2025] Sparse autoencoders do not find canonical units of analysis. https://proceedings.iclr.cc/paper_files/paper/2025/hash/84ca3f2d9d9bfca13f69b48ea63eb4a5-Abstract-Conference.html
Sparse autoencoders trained on the same activations can learn different decompositions, challenging the assumption that one dictionary recovers canonical model features.
[Lee and See 2004] Trust in automation: Designing for appropriate reliance. Human Factors. https://doi.org/10.1518/hfes.46.1.50_30392
Lee and See review trust in automation as a basis for appropriate reliance under complexity, with display design and context shaping whether reliance is justified.
[Lee et al. 2019] Best practices for the human evaluation of automatically generated text. Association for Computational Linguistics. https://aclanthology.org/W19-8643/
The paper connects human-evaluation validity and reproducibility to explicit planning, participant selection, question wording, presentation order, training, quality control, statistical analysis, and reporting.
[Lee et al. 2022] Deduplicating training data makes language models better. Association for Computational Linguistics. https://aclanthology.org/2022.acl-long.577/
The paper finds that exact and near duplicates affect train-test overlap, memorization, training efficiency, and measured accuracy in the studied language-model corpora.
[Lee et al. 2024] RLAIF vs. RLHF: Scaling reinforcement learning from human feedback with AI feedback. https://arxiv.org/abs/2309.00267
Across three studied language tasks, RLAIF produced policies comparable to RLHF while replacing human comparison labels with judgments from an off-the-shelf language model.
[Lepikhin et al. 2020] GShard: Scaling giant models with conditional computation and automatic sharding. https://arxiv.org/abs/2006.16668
GShard introduces lightweight annotation APIs and an XLA compiler extension enabling automatic SPMD sharding of a 600B-parameter MoE Transformer trained on 2048 TPU v3 devices for multilingual translation across 100 languages.
[Leroy 2009] A formally verified compiler back-end. Journal of Automated Reasoning. https://doi.org/10.1007/s10817-009-9155-4
CompCert's verified back end proves semantic preservation from Cminor to PowerPC assembly in Coq, making compiler correctness part of the trusted evidence chain.
[Lester et al. 2021] The power of scale for parameter-efficient prompt tuning. https://arxiv.org/abs/2104.08691
Prompt tuning learns continuous input embeddings for a frozen model and reports stronger relative performance as the evaluated T5 models scale.
[Leviathan et al. 2023] Fast inference from transformers via speculative decoding. PMLR. https://proceedings.mlr.press/v202/leviathan23a.html
Speculative decoding uses a faster draft model plus modified rejection sampling to reduce target-model calls while preserving the target distribution; the reported gains are specific to the paper's T5 experiments.
[Lewis et al. 2020] Retrieval-augmented generation for knowledge-intensive NLP tasks. https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html
The original RAG paper combines a sequence-to-sequence model with a dense Wikipedia index, establishing the parametric and non-parametric memory formulation.
[Lewis et al. 2020] Retrieval-augmented generation for knowledge-intensive NLP tasks. https://proceedings.neurips.cc/paper_files/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html
Introduces retrieval-augmented generation with a learned retriever and a sequence generator for knowledge-intensive tasks.
[Lewis et al. 2021] BASE layers: Simplifying training of large, sparse models. https://arxiv.org/abs/2103.16716
BASE layers replace MoE routing heuristics and auxiliary balancing losses by formulating token-to-expert assignment as a linear assignment problem, guaranteeing equal load across experts with no new hyperparameters.
[Li et al. 2022] Diffusion-LM improves controllable text generation. https://arxiv.org/abs/2205.14217
Diffusion-LM adapts continuous diffusion models to text by iteratively denoising Gaussian vectors into word vectors, enabling gradient-based plug-and-play controllable generation over complex attributes like syntactic structure.
[Li et al. 2022] Design and verification of the arm confidential compute architecture. https://www.usenix.org/conference/osdi22/presentation/li
Describes Arm CCA's Realm architecture and the formal verification performed for the firmware implementation studied in the paper.
[Li et al. 2023] BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. https://arxiv.org/abs/2301.12597
BLIP-2 introduces a lightweight Querying Transformer (Q-Former) that bridges frozen image encoders and frozen LLMs in two pre-training stages, achieving strong vision-language performance with far fewer trainable parameters.
[Li and others 2024] DataComp-LM: In search of the next generation of training sets for language models. https://proceedings.neurips.cc/paper_files/paper/2024/hash/19e4ea30dded58259665db375885e412-Abstract-Datasets_and_Benchmarks_Track.html
DataComp-LM provides a 240-trillion-token Common Crawl pool, standardized training recipes, and 53 evaluations for controlled experiments on extraction, filtering, deduplication, and data mixing.
[Li et al. 2024] EAGLE: Speculative sampling requires rethinking feature uncertainty. PMLR. https://proceedings.mlr.press/v235/li24bt.html
EAGLE predicts second-to-top-layer features while conditioning on sampled tokens, then verifies the resulting token proposals with the target model.
[Li et al. 2024] EAGLE-2: Faster inference of language models with dynamic draft trees. Association for Computational Linguistics. https://aclanthology.org/2024.emnlp-main.422/
EAGLE-2 replaces a static candidate tree with context-aware expansion and pruning while retaining target-model verification.
[Li et al. 2024] SnapKV: LLM knows what you are looking for before generation. https://proceedings.neurips.cc/paper_files/paper/2024/hash/28ab418242603e0f7323e54185d19bde-Abstract-Conference.html
SnapKV uses an end-of-prompt observation window to select and pool important prompt KV positions separately by head before generation.
[Li et al. 2024] DistriFusion: Distributed parallel inference for high-resolution diffusion models. https://arxiv.org/abs/2402.19481
Splits a single high-resolution diffusion sample across GPUs via displaced patch parallelism, reusing the previous step's feature maps so workers communicate asynchronously, up to 6.1x lower latency with no quality loss.
[Li et al. 2024] Retrieval augmented generation or long-context llms? A comprehensive study and hybrid approach. https://aclanthology.org/2024.emnlp-industry.66/
A direct comparison finds long-context models stronger on average when sufficiently resourced in the tested settings, while RAG remains much cheaper, motivating hybrid routing.
[Li et al. 2024] Revisiting catastrophic forgetting in large language model tuning. Association for Computational Linguistics. https://aclanthology.org/2024.findings-emnlp.249/
The study measures catastrophic forgetting during language-model tuning and relates its experiments to the geometry of the fine-tuning loss landscape.
[Li et al. 2025] EAGLE-3: Scaling up inference acceleration of large language models via training-time test. https://proceedings.neurips.cc/paper_files/paper/2025/hash/c7b5a35ea98b62512a869c19ea7b03cb-Abstract-Conference.html
EAGLE-3 trains a direct token drafter from fused target-model features and evaluates how drafter scale affects speculative-decoding latency and throughput.
[Li et al. 2025] ScreenSpot-pro: GUI grounding for professional high-resolution computer use. arXiv preprint arXiv:2504.07981. https://arxiv.org/abs/2504.07981
A point-in-target grounding benchmark for high-resolution professional software: the best prior model reached 18.9%, while cascaded visual search reached 48.1%. It isolates grounding rather than end-to-end task completion.
[Li et al. 2025] Apple intelligence foundation language models: Tech report 2025. arXiv preprint arXiv:2507.13575. https://arxiv.org/abs/2507.13575
The report describes Apple's approximately 3B-parameter on-device model and its architecture, quantization-aware training, evaluation, and deployment-specific adaptations.
[Li 2026] Do post-training algorithms actually differ? A controlled study across model scales uncovers scale-dependent ranking inversions. arXiv preprint arXiv:2603.19335. https://arxiv.org/abs/2603.19335
A controlled study finds that post-training algorithm rankings depend on model scale and task, and that no tested DPO variant significantly beats DPO in its 1.5B GSM8K sweep after multiple-comparison correction.
[Liang et al. 2022] Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. https://arxiv.org/abs/2203.02053
This paper reports and analyzes a modality gap in which image and text embeddings from CLIP-style models occupy separated regions despite useful cross-modal alignment.
[Liang et al. 2023] Holistic evaluation of language models. Transactions on Machine Learning Research. https://openreview.net/forum?id=iO4LZibEqW
Defines a transparent, multi-scenario and multi-metric framework for evaluating language models beyond a single aggregate score.
[Liang et al. 2023] Holistic evaluation of language models. Transactions on Machine Learning Research. https://arxiv.org/abs/2211.09110
HELM defines language-model evaluation as a selection of scenarios, adaptations, and multiple metrics, and publishes prompts and completions to make comparisons inspectable.
[Libovický and Helcl 2018] End-to-end non-autoregressive neural machine translation with connectionist temporal classification. https://arxiv.org/abs/1811.04719
This paper proposes an end-to-end non-autoregressive neural machine translation model using CTC, enabling fully parallel decoding without separate multi-step training, evaluated on WMT English-Romanian and English-German.
[Lie 2023] Cerebras architecture deep dive: First look inside the hardware/software co-design for deep learning. IEEE Micro. https://doi.org/10.1109/MM.2023.3256384
The article presents wafer-scale integration as a locality choice with its own memory, interconnect, compilation, and scaling constraints.
[Lieber et al. 2024] Jamba: a hybrid transformer-mamba language model. https://arxiv.org/abs/2403.19887
Jamba interleaves attention and Mamba layers with MoE; under the paper's stated hardware and context settings, it reports lower KV-cache memory and up to three times Mixtral's throughput.
[Lightman et al. 2024] Let's verify step by step. https://arxiv.org/abs/2305.20050
Let's Verify Step by Step compares outcome and process supervision for fixed-generator best-of-N selection on MATH and releases about 800,000 human step labels in PRM800K.
[Liguori 2020] Automating safe, hands-off deployments. https://d1.awsstatic.com/builderslibrary/pdfs/automating-safe-hands-off-deployments-clareliguori.pdf
AWS describes staged deployment waves, bake time, one-box and one-region exposure, and progressively wider regional parallelism.
[Lin et al. 2022] TruthfulQA: Measuring how models mimic human falsehoods. Association for Computational Linguistics. https://aclanthology.org/2022.acl-long.229/
TruthfulQA tests 817 questions designed around false beliefs and misconceptions, showing that truthfulness on this adversarial population differs from imitation accuracy.
[Lin et al. 2024] AWQ: Activation-aware weight quantization for on-device LLM compression and acceleration. https://proceedings.mlsys.org/paper_files/paper/2024/hash/42a452cbafa9dd64e9ba4aa95cc1ef21-Abstract-Conference.html
AWQ proposes activation-aware per-channel weight scaling for hardware-friendly low-bit weight-only quantization; TinyChat delivered more than 3x speedup over the Hugging Face FP16 implementation on the evaluated desktop and mobile GPUs.
[Lin et al. 2025] Continual learning via sparse memory finetuning. arXiv preprint arXiv:2510.15103. https://arxiv.org/abs/2510.15103
Sparse memory finetuning updates selected memory slots and reduces old-task forgetting relative to full and LoRA fine-tuning in the paper's factual-QA experiments.
[Lindenbauer et al. 2025] The complexity trap: Simple observation masking is as efficient as LLM summarization for agent context management. https://arxiv.org/abs/2508.21433
Simple observation masking halves LLM agent cost on SWE-bench Verified while matching or slightly exceeding the solve rate of LLM-based summarization across five model configurations.
[Lindsey et al. 2025] On the biology of a large language model. https://transformer-circuits.pub/2025/attribution-graphs/biology.html
This work applies circuit tracing to Claude 3.5 Haiku to investigate the internal mechanisms the model uses across reasoning, poetry planning, multilingual, and arithmetic tasks.
[Linnainmaa 1976] Taylor expansion of the accumulated rounding error. BIT Numerical Mathematics. https://papers.baulab.info/papers/also/Linnainmaa-1976.pdf
Linnainmaa develops reverse accumulation while analyzing how rounding errors propagate through a sequence of elementary operations.
[Linux Kernel Documentation n.d.] Seccomp BPF (secure computing with filters). https://www.kernel.org/doc/html/latest/userspace-api/seccomp_filter.html
The kernel documentation defines seccomp filtering and warns that it reduces exposed kernel surface rather than serving as a complete sandbox.
[Lipman et al. 2023] Flow matching for generative modeling. https://arxiv.org/abs/2210.02747
Flow Matching (FM) is a simulation-free method for training Continuous Normalizing Flows by regressing conditional vector fields, enabling scalable CNF training with Optimal Transport paths that outperform diffusion models on ImageNet.
[Liu et al. 2023] Ring attention with blockwise transformers for near-infinite context. https://arxiv.org/abs/2310.01889
Ring Attention distributes long sequences across multiple devices in a ring topology, overlapping key-value block communication with blockwise self-attention computation to enable near-infinite context length without approximations.
[Liu et al. 2023] Flow straight and fast: Learning to generate and transfer data with rectified flow. https://arxiv.org/abs/2209.03003
Rectified flow regresses paired displacement targets and can recursively reflow model-generated couplings, which empirically supports coarse or one-step Euler sampling when the learned trajectories are sufficiently straight.
[Liu et al. 2023] Visual instruction tuning. https://proceedings.neurips.cc/paper_files/paper/2023/hash/6dcf277ea32ce3288914faf369fe6de0-Abstract-Conference.html
LLaVA connects a CLIP visual encoder to an LLM via a linear projection and applies visual instruction tuning on GPT-4-generated multimodal data to produce a general-purpose vision-language assistant.
[Liu et al. 2023] Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation. https://proceedings.neurips.cc/paper_files/paper/2023/hash/43e9d647ccd3e4b7b5baab53f0368686-Abstract-Conference.html
EvalPlus expands code-generation tests with additional cases and reveals programs that pass the original benchmark tests but remain incorrect.
[Liu et al. 2023] G-eval: NLG evaluation using GPT-4 with better human alignment. Association for Computational Linguistics. https://aclanthology.org/2023.emnlp-main.153/
G-Eval uses explicit criteria and structured form filling for summarization and dialogue evaluation, improves correspondence with human scores in those tasks, and identifies possible bias toward model-generated text.
[Liu et al. 2024] Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics. https://aclanthology.org/2024.tacl-1.9/
Finds that long-context task performance often depends strongly on evidence position, with relevant information in the middle used less reliably than information near the beginning or end.
[Liu et al. 2024] Improved baselines with visual instruction tuning. https://arxiv.org/abs/2310.03744
LLaVA-1.5 shows that replacing LLaVA's linear vision-language connector with an MLP and adding VQA data with response formatting prompts achieves state-of-the-art on 11 multimodal benchmarks using only 1.2M public samples.
[Liu et al. 2024] LLaVA-NeXT: Improved reasoning, OCR, and world knowledge. https://llava-vl.github.io/blog/2024-01-30-llava-next/
LLaVA-NeXT extends LLaVA-1.5 with AnyRes tiling and revised instruction data to improve high-resolution visual reasoning and OCR.
[Liu et al. 2024] KIVI: a tuning-free asymmetric 2bit quantization for KV cache. https://proceedings.mlr.press/v235/liu24bz.html
KIVI studies KV-cache distributions and applies different two-bit granularities to keys and values in the evaluated Llama, Falcon, and Mistral deployments.
[Liu et al. 2024] Reason for future, act for now: a principled architecture for autonomous LLM agents. PMLR. https://proceedings.mlr.press/v235/liu24ab.html
RAFA implements receding-horizon control: it plans over future actions, executes the first one, stores feedback, and plans again from the updated state.
[Liu et al. 2024] ECBD: Evidence-centered benchmark design for NLP. Association for Computational Linguistics. https://aclanthology.org/2024.acl-long.861/
ECBD organizes benchmark design around capability, content, adaptation, assembly, and evidence, requiring each module to be described, justified, and supported with validity evidence.
[Liu et al. 2024] AgentBench: Evaluating LLMs as agents. https://proceedings.iclr.cc/paper_files/paper/2024/hash/e9df36b21ff4ee211a8b71ee8b7e9f57-Abstract-Conference.html
AgentBench evaluates language-model agents across eight interactive environments, making multi-turn decisions and environment interaction the object of evaluation.
[Liu et al. 2024] MobileLLM: Optimizing sub-billion parameter language models for on-device use cases. PMLR. https://proceedings.mlr.press/v235/liu24ce.html
MobileLLM studies architecture within a sub-billion-parameter device budget and reports gains from deep-and-thin models plus memory-conscious sharing mechanisms.
[Liu et al. 2024] DoRA: Weight-decomposed low-rank adaptation. https://proceedings.mlr.press/v235/liu24bn.html
DoRA decomposes pretrained weights into magnitude and direction and applies a low-rank update to the directional component.
[Liu et al. 2025] Muon is scalable for LLM training. arXiv preprint arXiv:2502.16982. https://arxiv.org/abs/2502.16982
Shows the Muon optimizer scales to large LLMs by adding weight decay and tuning the per-parameter update scale, reaching about 2x the compute efficiency of AdamW; trains Moonlight, a 16B MoE model, on 5.7T tokens.
[Liu et al. 2025] Midtraining bridges pretraining and posttraining distributions. arXiv preprint arXiv:2510.14865. https://arxiv.org/abs/2510.14865
Proposes a working definition of midtraining and, in controlled 70M- and 160M-parameter experiments, finds that late general-plus-specialist mixtures can improve code and math transfer after SFT, with interacting effects from introduction time and mixture weight.
[Liu et al. 2025] ProRL: Prolonged reinforcement learning expands reasoning boundaries in large language models. https://proceedings.neurips.cc/paper_files/paper/2025/hash/1a22b912945fb7c0bdd079e792b31b6f-Abstract-Conference.html
ProRL combines prolonged training, KL control, reference resets, and diverse tasks and reports large-k gains over its 1.5B base model.
[Liu et al. 2025] Understanding R1-zero-like training: a critical perspective. https://openreview.net/forum?id=5PAF7PAY2Y
Dr. GRPO fixes an optimization bias in GRPO that artificially inflates response length for incorrect outputs, improving token efficiency while maintaining reasoning performance on a 7B model.
[Lodderstedt et al. 2013] OAuth 2.0 token revocation. https://www.rfc-editor.org/rfc/rfc7009.html
RFC 7009 defines how clients notify an authorization server that a token is no longer needed.
[Lodderstedt et al. 2023] OAuth 2.0 rich authorization requests. https://www.rfc-editor.org/rfc/rfc9396.html
Rich Authorization Requests provides a structured authorization_details parameter for fine-grained permissions while leaving domain semantics and enforcement to deployments.
[Lodderstedt et al. 2025] Best current practice for OAuth 2.0 security. https://www.rfc-editor.org/rfc/rfc9700.html
RFC 9700 consolidates deployment guidance for resisting token leakage, replay, mix-up, and other OAuth threats.
[Longpre et al. 2023] The data provenance initiative: a large scale audit of dataset licensing and attribution in AI. https://arxiv.org/abs/2310.16787
The Data Provenance Initiative traces the sources, licenses, creators, and derivation chains of more than 1,800 text datasets and finds widespread missing and incorrectly categorized license information.
[Longpre et al. 2024] Consent in crisis: The rapid decline of the AI data commons. https://arxiv.org/abs/2407.14933
Tracks changing robots and terms-of-service signals across domains represented in major web corpora while separating those signals from legal consent.
[Longpre et al. 2024] A large-scale audit of dataset licensing and attribution in AI. Nature Machine Intelligence. https://www.nature.com/articles/s42256-024-00878-8
The audit traces more than 1,800 text datasets and reports frequent missing or misclassified license and attribution information in dataset aggregators.
[Loshchilov and Hutter 2019] Decoupled weight decay regularization. https://arxiv.org/abs/1711.05101
Shows L2 regularization and weight decay are not equivalent for Adam, and that decoupling weight decay from the gradient update (AdamW) improves generalization.
[Lou et al. 2024] Discrete diffusion modeling by estimating the ratios of the data distribution. https://arxiv.org/abs/2310.16834
SEDD introduces score entropy for learning ratios between neighboring noisy discrete states and reports strong GPT-2-scale language-modeling results under its stated evaluation setup.
[Lovelace et al. 2026] Prescriptive scaling laws for data constrained training. arXiv preprint arXiv:2605.01640. https://arxiv.org/abs/2605.01640
In experiments up to 1B parameters and 16 epochs, adds an explicit repetition penalty and finds regimes where more capacity beats further repetition; stronger weight decay reduces the penalty but pays a single-epoch cost.
[Lu et al. 2022] DPM-solver: a fast ODE solver for diffusion probabilistic model sampling in around 10 steps. https://arxiv.org/abs/2206.00927
DPM-Solver is a fast, training-free high-order ODE solver for diffusion probabilistic models that generates high-quality samples in 10 to 20 function evaluations, achieving 4 to 16x speedup over prior samplers.
[Lu et al. 2022] Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. https://aclanthology.org/2022.acl-long.556/
Controlled four-shot classification experiments show that demonstration order can substantially change accuracy and that a good order need not transfer across models.
[Lù et al. 2025] AgentRewardBench: Evaluating automatic evaluations of web agent trajectories. arXiv preprint arXiv:2504.08942. https://arxiv.org/abs/2504.08942
AgentRewardBench compares rule-based and model-based trajectory evaluators with expert labels for success, side effects, and repetition, exposing complementary grader failure modes.
[Luo et al. 2023] Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378. https://arxiv.org/abs/2310.04378
Latent Consistency Models (LCMs) distill pre-trained Stable Diffusion into a model that generates 768x768 images in 2 to 4 steps using consistency distillation in latent space.
[Lyu et al. 2023] Faithful chain-of-thought reasoning. Association for Computational Linguistics. https://arxiv.org/abs/2301.13379
Faithful CoT translates natural-language queries into symbolic reasoning chains and uses deterministic solvers, making the executed chain causally responsible for the final answer.
[Ma et al. 2024] DeepCache: Accelerating diffusion models for free. https://arxiv.org/abs/2312.00858
A training-free method that caches high-level U-Net features across adjacent denoising steps and recomputes only the fast-changing parts, 2-4x faster with negligible quality loss.
[Ma et al. 2024] The era of 1-bit llms: All large language models are in 1.58 bits. arXiv preprint arXiv:2402.17764. https://arxiv.org/abs/2402.17764
BitNet b1.58 studies language models trained with ternary weights and reports model-level quality, memory, latency, and arithmetic-energy comparisons under its evaluated setup.
[Ma et al. 2025] Understanding silent data corruption in LLM training. Association for Computational Linguistics. https://aclanthology.org/2025.acl-long.996/
Production nodes with silent data corruption produce effects ranging from small numerical perturbations to loss spikes and different trained weights.
[Ma et al. 2025] Understanding silent data corruption in LLM training. arXiv preprint arXiv:2502.12340. https://arxiv.org/abs/2502.12340
Controlled experiments show that SDC-affected nodes can change model weights without an obvious pretraining-loss signal and can cause loss spikes in some fine-tuning runs.
[Ma et al. 2026] MOPD: Multi-teacher on-policy distillation for capability integration in LLM post-training. arXiv preprint arXiv:2606.30406. https://arxiv.org/abs/2606.30406
MOPD trains separate reinforcement-learning specialists and then distills all of them into one student on the student's own rollouts, reporting better capability integration than mixed-task training, cascaded stages, off-policy fine-tuning, or parameter merging.
[Mace et al. 2018] Incident response. O'Reilly Media. https://sre.google/workbook/incident-response/
The SRE workbook describes early incident declaration, explicit command roles, communications, operations, and a durable working record.
[Maharana et al. 2024] Evaluating very long-term conversational memory of LLM agents. arXiv preprint arXiv:2402.17753. https://arxiv.org/abs/2402.17753
Very-long-term dialogues spanning hundreds of turns across many sessions, with question answering and event summarization: the benchmark at the center of the memory-system disputes.
[Maini et al. 2024] Rephrasing the web: a recipe for compute and data-efficient language modeling. Association for Computational Linguistics. https://arxiv.org/abs/2401.16380
WRAP uses an instruction-tuned LLM to rephrase noisy web text into cleaner styles, reducing LLM pre-training compute by 3x and data by 5x compared to training on raw web corpora.
[Maini et al. 2024] TOFU: a task of fictitious unlearning for LLMs. https://arxiv.org/abs/2401.06121
TOFU evaluates language-model unlearning on synthetic author profiles with forget sets, retain sets, and multiple behavioral and utility metrics.
[Maini and others 2025] BeyondWeb: Lessons from scaling synthetic data for trillion-scale pretraining. arXiv preprint arXiv:2508.10975. https://arxiv.org/abs/2508.10975
BeyondWeb studies targeted document rephrasing in mixed real-and-synthetic corpora, including a 1B model trained for one trillion tokens.
[Malkov and Yashunin 2020] Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://arxiv.org/abs/1603.09320
HNSW proposes a fully graph-based approximate nearest neighbor search index using a multi-layer proximity graph with logarithmic complexity scaling.
[Manakul et al. 2023] SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models. Association for Computational Linguistics. https://aclanthology.org/2023.emnlp-main.557/
SelfCheckGPT uses inconsistency across repeated black-box samples as a sentence-level warning signal; agreement among samples is not evidence of truth.
[Marchisio et al. 2024] How does quantization affect multilingual LLMs?. Association for Computational Linguistics. https://aclanthology.org/2024.findings-emnlp.935/
Across the evaluated multilingual models, languages and tasks were affected unevenly by quantization, and automatic metrics understated some changes observed by human evaluators.
[Markov et al. 2023] A holistic approach to undesired content detection in the real world. AAAI 2023. https://arxiv.org/abs/2208.03274
OpenAI describes a holistic pipeline for real-world content moderation, combining content taxonomy design, active learning, quality-controlled labeling, and synthetic data to detect sexual, hateful, violent, self-harm, and harassment content.
[Martínez-Olguín 2026] Bartz v. anthropic PBC, order granting final settlement approval and final judgment, no. 4:24-cv-05417. United States District Court for the Northern District of California. https://law.justia.com/cases/federal/district-courts/california/candce/4:2024cv05417/434709/680/
Finally approves the negotiated Bartz class settlement without converting it into a merits holding or future license.
[Mazeika et al. 2024] HarmBench: a standardized evaluation framework for automated red teaming and robust refusal. https://proceedings.mlr.press/v235/mazeika24a.html
HarmBench is a standardized benchmark with 510 harmful behaviors and an evaluation pipeline that compares 18 automated red-teaming methods across 33 LLMs to enable rigorous, reproducible attack-defense co-development.
[McCandlish et al. 2018] An empirical model of large-batch training. arXiv preprint arXiv:1812.06162. https://arxiv.org/abs/1812.06162
Introduces the gradient noise scale to predict the critical batch size, beyond which larger batches stop reducing the number of training steps and waste compute.
[McCulloch and Pitts 1943] A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics.
McCulloch and Pitts give the first mathematical model of a neuron, showing networks of threshold logic units can compute any logical function.
[McNemar 1947] Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika. https://doi.org/10.1007/BF02295996
McNemar's test compares paired binary outcomes by focusing on examples where two systems disagree, which is often the right design for benchmark A/B comparisons on the same items.
[Meng et al. 2022] Locating and editing factual associations in GPT. https://arxiv.org/abs/2202.05262
ROME uses causal mediation analysis to locate factual associations in mid-layer feed-forward modules of GPT and introduces a rank-one weight editing method to update specific facts.
[Meng et al. 2023] Mass-editing memory in a transformer. https://arxiv.org/abs/2210.07229
MEMIT scales knowledge editing in large language models to thousands of simultaneous fact updates by distributing parameter changes across a range of critical MLP layers.
[Meng et al. 2024] SimPO: Simple preference optimization with a reference-free reward. https://arxiv.org/abs/2405.14734
SimPO replaces DPO's reference-model reward with a length-normalized average log probability and a target reward margin, eliminating the reference model while outperforming DPO by up to 7.5 points on Arena-Hard.
[Meta 2026] Private processing for WhatsApp overview: Technical white paper and security guide. https://ai.meta.com/static-resource/private-processing-technical-whitepaper
Documents Meta's stated threat model and its composition of SEV-SNP, H100 confidential mode, RA-TLS, relays, transparency, revocation, and bounded observability.
[Meta AI 2025] The llama 4 herd: The beginning of a new era of natively multimodal AI innovation. https://ai.meta.com/blog/llama-4-multimodal-intelligence/
Meta introduces the Llama 4 multimodal mixture-of-experts family and describes its released variants, context support, training, and evaluations.
[METR 2026] Task-completion time horizons of frontier AI models. https://metr.org/time-horizons/
METR estimates the duration of software tasks frontier models can complete at a chosen success rate and tracks how that horizon changes over time.
[METR 2026] Time horizon 1.1. https://metr.org/blog/2026-1-29-time-horizon-1-1/
METR expands its time-horizon suite and shows how task coverage, estimated human baselines, fit windows, and wide intervals qualify the headline horizon.
[METR 2026] Frontier AI risk report: February–march 2026. https://metr.org/blog/2026-05-19-frontier-risk-report/
METR reports a shared internal model near the measurable edge of Time Horizon 1.1 and warns that the suite cannot reliably resolve horizons above 16 hours.
[Meunier and Major 2026] HTTP message signatures for automated traffic. https://datatracker.ietf.org/doc/draft-meunier-webbotauth-httpsig-protocol/
The draft combines agent identifiers, key discovery, and RFC 9421 signatures but remains work in progress rather than a published Internet standard.
[Mialon et al. 2024] GAIA: a benchmark for general AI assistants. https://proceedings.iclr.cc/paper_files/paper/2024/hash/25ae35b5b1738d80f1f03a8713e405ec-Abstract-Conference.html
GAIA evaluates assistants on human-authored questions that may require browsing, files, code, multimodal interpretation, and tool use before producing a short verifiable answer.
[Michaud et al. 2023] The quantization model of neural scaling. https://arxiv.org/abs/2303.13506
Proposes a discrete-skill hypothesis for neural scaling, validates it on synthetic tasks, and presents tentative evidence that language-model scaling may show related behavior.
[Micikevicius et al. 2017] Mixed precision training. https://arxiv.org/abs/1710.03740
This paper presents mixed precision training, combining FP16 storage and arithmetic with FP32 master weights, loss-scaling, and FP32 accumulation to halve memory use without accuracy loss.
[Micikevicius et al. 2022] FP8 formats for deep learning. https://arxiv.org/abs/2209.05433
The paper specifies E4M3 and E5M2 FP8 interchange formats and evaluates training recipes across several neural-network families.
[Micron Technology 2026] HBM4 high-bandwidth memory. https://www.micron.com/products/memory/hbm/hbm4
Micron's HBM4 page separates interface width, per-pin transfer rate, per-stack bandwidth, density, and package choices.
[Microsoft 2024] MLOps and GenAIOps for AI workloads on azure. https://learn.microsoft.com/en-us/azure/well-architected/ai/mlops-genaiops
Microsoft's workload guidance treats AI operations as lifecycle management for nondeterministic systems, including monitoring, drift, deployment, governance, and automation.
[Microsoft 2026] Durable orchestrations overview. https://learn.microsoft.com/en-us/azure/durable-task/common/durable-task-orchestrations
Durable Task reconstructs workflow state from append-only history and requires deterministic orchestrator code so replay emits commands compatible with recorded events.
[Microsoft 2026] Orchestration versioning: Safe deployments for durable orchestrations. https://learn.microsoft.com/en-us/azure/durable-task/common/durable-orchestration-versioning
Changing durable workflow code can break deterministic replay, so long-running executions need explicit version assignment and migration.
[Microsoft 2026] Add a filter to a vector query in azure AI search. https://learn.microsoft.com/en-us/azure/search/vector-search-filters
Azure AI Search documentation distinguishes pre-filtering, post-filtering, and strict post-filtering for vector queries and explains their recall trade-offs.
[Milakov and Gimelshein 2018] Online normalizer calculation for softmax. arXiv preprint arXiv:1805.02867. https://arxiv.org/abs/1805.02867
The algorithm computes softmax in one pass by maintaining a running maximum and normalizer, a recurrence later used by tiled exact-attention implementations.
[Milanta and Beurer-Kellner 2025] GitHub MCP exploited: Accessing private repositories via MCP. https://invariantlabs.ai/blog/mcp-github-vulnerability
A proof-of-concept shows how malicious issue content can induce allowed private reads and public writes through a broadly connected agent.
[Miller 2024] Adding error bars to evals: a statistical approach to language model evaluations. arXiv preprint arXiv:2411.00640. https://arxiv.org/abs/2411.00640
An Anthropic treatment of LLM evaluation as statistical inference: report standard errors, use clustered standard errors when questions come in groups, analyze paired differences between models, and plan sample sizes with power analysis.
[Min et al. 2023] FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. Association for Computational Linguistics. https://aclanthology.org/2023.emnlp-main.741/
FActScore decomposes long-form generations into atomic facts and measures the percentage supported by a specified knowledge source, producing a factual-precision estimate rather than recall.
[MiniMax 2025] MiniMax-01: Scaling foundation models with lightning attention. arXiv preprint arXiv:2501.08313. https://arxiv.org/abs/2501.08313
MiniMax-01 interleaves Lightning Attention with softmax attention in a 456B-total, 45.9B-activated MoE model trained at one-million-token context and evaluated with extrapolation to four million tokens.
[Minixhofer et al. 2024] Zero-shot tokenizer transfer. https://proceedings.neurips.cc/paper_files/paper/2024/hash/532ce4fcf853023c4cf2ac38cbc5d002-Abstract-Conference.html
ZeTT predicts embeddings for a replacement tokenizer and uses limited continued training to reduce the remaining performance gap.
[Minkenberg et al. 2021] Co-packaged datacenter optics: Opportunities and challenges. IET Optoelectronics. https://doi.org/10.1049/ote2.12020
This survey explains why moving optics into the switch package changes power and density as well as testing, repair, and reliability.
[Mitchell et al. 2019] Model cards for model reporting. Association for Computing Machinery. https://doi.org/10.1145/3287560.3287596
Model cards report intended uses, evaluation conditions, limitations, and performance across relevant conditions and groups.
[Model Context Protocol 2025] Tools. https://modelcontextprotocol.io/specification/2025-11-25/server/tools
The MCP tools specification defines discovery, JSON Schema inputs and structured outputs, result validation, and human control over tool invocation.
[Model Context Protocol 2025] Security best practices. https://modelcontextprotocol.io/specification/2025-11-25/basic/security_best_practices
MCP security guidance covers token audience binding, confused-deputy risks, token passthrough, session hijacking, and least-privilege scopes.
[Model Context Protocol 2025] Transports: Mcp-Session-Id and Last-Event-ID. https://modelcontextprotocol.io/specification/2025-11-25/basic/transports
The MCP specification's transport layer defines two mechanisms for JSON-RPC message exchange: stdio for subprocess communication and Streamable HTTP (replacing SSE) for networked MCP servers.
[Model Context Protocol 2025] Key changes: MCP specification 2025-06-18. https://modelcontextprotocol.io/specification/2025-06-18/changelog
The 2025-06-18 MCP revision classifies servers as OAuth resource servers, requires clients to implement RFC 8707 resource indicators, removes JSON-RPC batching, and adds structured tool output and elicitation.
[Model Context Protocol 2026] Authorization. https://modelcontextprotocol.io/specification/2026-07-28/basic/authorization
The MCP authorization profile standardizes server discovery and audience-bound OAuth access, not application-level tool policy.
[Model Context Protocol 2026] Model context protocol 2026-07-28: Tools. https://modelcontextprotocol.io/specification/2026-07-28/server/tools
The MCP tools specification defines discovery, invocation, schemas, explicit state handles, result validation, and security considerations at the wire boundary.
[Model Context Protocol Contributors 2025] Model context protocol specification, revision 2025-11-25. https://modelcontextprotocol.io/specification/2025-11-25/architecture
The MCP specification defines host-client-server responsibilities, capability negotiation, and separate primitives for resources, prompts, and tools.
[Model Context Protocol Contributors 2026] Model context protocol specification, revision 2026-07-28. https://modelcontextprotocol.io/specification/2026-07-28
The current MCP specification defines a stateless host-client-server protocol for tools, resources, and prompts; identity metadata is self-reported, and the protocol does not itself enforce consent, authorization, or safe behavior.
[Model Context Protocol Contributors 2026] MCP security best practices. https://modelcontextprotocol.io/docs/2026-07-28/tutorials/security/security_best_practices
MCP security guidance forbids token passthrough, requires audience and resource validation for protected HTTP resources, and recommends least privilege plus sandboxing for local servers.
[Mohan et al. 2021] CheckFreq: Frequent, fine-grained DNN checkpointing. https://www.usenix.org/conference/fast21/presentation/mohan
CheckFreq profiles checkpoint cost, adjusts save frequency, and pipelines checkpoint work while preserving its evaluated data-loader invariant.
[Moreau and Missier 2013] PROV-DM: The PROV data model. https://www.w3.org/TR/prov-dm/
PROV-DM models provenance through entities, activities, agents, and relations that connect generated artifacts to their sources and transformations.
[Motamed et al. 2026] Do generative video models understand physical principles?. https://arxiv.org/abs/2501.09038
Physics-IQ is a real-world benchmark of 396 videos covering five physics domains that reveals current generative video models score at most 29.5%, showing visual realism does not imply physical understanding.
[Moura and Ullrich 2021] The Lean 4 theorem prover and programming language. Springer. https://link.springer.com/chapter/10.1007/978-3-030-79876-5_37
Lean 4 combines a dependent type theory, theorem prover, functional language, and metaprogramming system in one implementation.
[Muennighoff et al. 2023] Scaling data-constrained language models. https://arxiv.org/abs/2305.16264
Finds little loss penalty from up to roughly four repeated epochs at fixed compute in its experiments, then models the diminishing value of further repetition and excess parameters.
[Muennighoff et al. 2023] MTEB: Massive text embedding benchmark. https://aclanthology.org/2023.eacl-main.148/
MTEB evaluates text embeddings across eight task families and finds that performance on one task, such as semantic similarity, does not establish dominance on the others.
[Muennighoff et al. 2025] s1: Simple test-time scaling. Association for Computational Linguistics. https://aclanthology.org/2025.emnlp-main.1025/
s1 fine-tunes Qwen2.5-32B-Instruct on 1,000 selected Gemini-generated traces and combines the result with budget forcing; its own grader judged 53.6 percent of the training generations correct.
[Murray et al. 2021] tf.data: a machine learning data processing framework. Proceedings of the VLDB Endowment. https://www.vldb.org/pvldb/vol14/p2945-klimovic.pdf
tf.data treats input loading as a composable dataflow whose parallelism, caching, prefetching, and determinism choices affect end-to-end training.
[Nakano et al. 2021] WebGPT: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332. https://arxiv.org/abs/2112.09332
WebGPT trains a language model to browse with demonstrations and human feedback; its strongest reported configuration used rejection sampling rather than an online policy-gradient update.
[Narayanan et al. 2019] PipeDream: Generalized pipeline parallelism for DNN training. https://doi.org/10.1145/3341301.3359646
PipeDream combines pipeline parallelism with data parallelism to reduce inter-GPU communication by up to 95% and achieve up to 5x faster time-to-accuracy than data-parallel DNN training.
[Narayanan et al. 2021] Efficient large-scale language model training on GPU clusters using megatron-LM. https://arxiv.org/abs/2104.04473
The paper composes tensor, pipeline, and data parallelism and analyzes pipeline schedules for large Transformer training.
[National Academies of Sciences, Engineering, and Medicine 2019] Reproducibility and replicability in science. The National Academies Press. https://doi.org/10.17226/25303
The report distinguishes reproducing results with the same data and methods from obtaining consistent results with new data, and treats replication as evidence with scope rather than a binary seal.
[National Telecommunications and Information Administration 2024] Dual-use foundation models with widely available model weights report. https://www.ntia.gov/programs-and-initiatives/artificial-intelligence/open-model-weights-report
NTIA reviews benefits and risks of widely available model weights and recommends continued evidence collection rather than a general restriction at the time of the report.
[Naumov et al. 2019] Deep learning recommendation model for personalization and recommendation systems. arXiv preprint arXiv:1906.00091. https://arxiv.org/abs/1906.00091
Meta's reference recommendation architecture combines categorical-feature embedding tables with dense interaction and multilayer-perceptron components, requiring both model and data parallelism.
[Nelson et al. 2025] Incident response recommendations and considerations for cybersecurity risk management: a CSF 2.0 community profile. https://doi.org/10.6028/NIST.SP.800-61r3
NIST SP 800-61 Revision 3 integrates incident-response preparation, detection, response, and recovery with the Cybersecurity Framework 2.0.
[New York State Legislature 2026] S.8828: Chapter amendment to the RAISE act. https://www.nysenate.gov/legislation/bills/2025/S8828
New York S.8828 amends the RAISE Act's scope, definitions, reporting duties, and implementation provisions for frontier AI developers.
[Nichol and Dhariwal 2021] Improved denoising diffusion probabilistic models. https://arxiv.org/abs/2102.09672
This paper shows that DDPMs achieve competitive log-likelihoods via learned reverse-process variances and a hybrid objective, enabling high-quality sampling with 20x fewer forward passes.
[Nickolls et al. 2008] Scalable parallel programming with CUDA. ACM Queue. https://queue.acm.org/detail.cfm?id=1365500
The paper describes CUDA's execution model, including grids, blocks, warps, and single-instruction multiple-thread execution.
[Nie et al. 2025] Large language diffusion models. arXiv preprint arXiv:2502.09992. https://arxiv.org/abs/2502.09992
LLaDA trains an 8B masked-diffusion language model from scratch on 2.3T tokens and reports competitive results against selected similarly sized autoregressive baselines.
[North American Electric Reliability Corporation 2026] Large loads action plan. https://prod.nerc.com/initiatives/large-loads-action-plan
NERC's action plan identifies reliability work needed for forecasting, studying, interconnecting, and operating rapidly growing large loads.
[Northcutt et al. 2021] Pervasive label errors in test sets destabilize machine learning benchmarks. NeurIPS Datasets and Benchmarks Track. https://arxiv.org/abs/2103.14749
This paper identifies pervasive label errors averaging 3.3% across test sets of 10 major ML benchmarks, showing they can reverse model rankings and destabilize evaluation.
[Nottingham et al. 2023] Problem details for HTTP apis. https://www.rfc-editor.org/rfc/rfc9457.html
RFC 9457 defines a reusable machine-readable error format while warning clients not to parse human-readable detail for program logic.
[Noukhovitch et al. 2025] Asynchronous RLHF: Faster and more efficient off-policy RL for language models. https://proceedings.iclr.cc/paper_files/paper/2025/hash/0b99315234cc95e6ef281f9155b68832-Abstract-Conference.html
Asynchronous RLHF overlaps generation and learning on separate resources and studies how policy staleness changes training efficiency and quality.
[Novikov et al. 2025] AlphaEvolve: a coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131. https://arxiv.org/abs/2506.13131
AlphaEvolve evolves programs against human-supplied evaluation code; selected candidates still require held-out, expert, hardware, or deployment checks appropriate to the application.
[Noy and Zhang 2023] Experimental evidence on the productivity effects of generative artificial intelligence. Science. https://doi.org/10.1126/science.adh2586
In an experiment with 453 professionals, ChatGPT access reduced completion time and improved rated quality on short, incentivized writing tasks.
[NVIDIA 2024] NCCL: NVIDIA collective communications library. https://github.com/NVIDIA/nccl
NCCL implements topology-aware collective communication primitives for moving tensors among NVIDIA GPUs within and across nodes.
[NVIDIA 2025] Nemotron-h: a family of accurate and efficient hybrid mamba-transformer models. arXiv preprint arXiv:2504.03624. https://arxiv.org/abs/2504.03624
Nemotron-H combines Mamba-2, attention, and dense FFN layers in 8B and 56B models; the paper reports competitive task accuracy and up to three-times throughput in specified long-context H100 comparisons.
[NVIDIA 2025] Pretraining large language models with NVFP4. arXiv preprint arXiv:2509.25149. https://arxiv.org/abs/2509.25149
Trains a 12B model over 10 trillion tokens in the NVFP4 4-bit microscaling format, using Random Hadamard transforms, two-dimensional scaling, and stochastic rounding to match an fp8 baseline.
[NVIDIA 2025] NVIDIA secure AI with blackwell and hopper gpus. https://docs.nvidia.com/nvidia-secure-ai-with-blackwell-and-hopper-gpus-whitepaper.pdf
Documents the distinct Hopper and Blackwell protection modes, attestation claims, interconnect coverage, topology assumptions, and remaining trusted components.
[NVIDIA 2026] CUDA programming guide. https://docs.nvidia.com/cuda/cuda-programming-guide/
The CUDA guide defines the host-device model, SIMT execution, memory spaces, and architecture-specific accelerator features.
[NVIDIA 2026] NVL72 AI Factory: System hardware and components. https://docs.nvidia.com/enterprise-reference-architectures/nvl72-ai-factory/latest/components.html
The reference architecture documents a 72-GPU rack-scale NVLink domain, showing that scale-up is not necessarily confined to one server.
[NVIDIA 2026] NCCL user guide: Collective operations. https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/usage/collectives.html
The NCCL guide defines GPU collective semantics, including all-reduce, all-gather, reduce-scatter, broadcast, and all-to-all.
[NVIDIA 2026] GPUDirect RDMA documentation. https://docs.nvidia.com/cuda/gpudirect-rdma/
The guide describes direct peer-device access to GPU memory and the required registration, topology, driver, and synchronization conditions.
[NVIDIA 2026] CUDA programming guide. https://docs.nvidia.com/cuda/cuda-programming-guide/02-basics/writing-cuda-kernels.html
The CUDA guide defines the host-device model, SIMT execution, memory spaces, synchronization, and the structure of GPU kernels.
[NVIDIA 2026] NVIDIA CUDA compiler driver NVCC. https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html
The official compiler-driver contract explains how CUDA source, PTX, cubin images, fat binaries, host code, and runtime loading fit together.
[NVIDIA 2026] NVIDIA DGX GB200 NVL72 user guide. https://docs.nvidia.com/dgx/dgxgb200-user-guide/
The operator guide defines the physical components and network boundaries of one deployed 72-GPU rack system.
[NVIDIA 2026] Multi-node NVLink systems tuning guide. https://docs.nvidia.com/multi-node-nvlink-systems/multi-node-tuning-guide/overview.html
The guide explains the topology and software considerations that remain when NVLink spans more GPUs.
[NVIDIA 2026] Triton inference server: Model repository. https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/user_guide/model_repository.html
Triton documents concrete portability boundaries: TensorRT plans depend on CUDA compute capability, ONNX support depends on the bundled runtime and operators, and TorchScript compatibility can vary by PyTorch version.
[Nye et al. 2021] Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114. https://arxiv.org/abs/2112.00114
Scratchpad training asks models to emit intermediate computation before an answer and evaluates the approach on multi-step algorithmic tasks.
[Nygard 2018] Release it! Design and deploy production-ready software. Pragmatic Bookshelf.
Nygard catalogs stability patterns for production software, including circuit breakers, bulkheads, and timeouts.
[Office of Management and Budget 2025] M-25-21: Accelerating federal use of AI through innovation, governance, and public trust. https://www.whitehouse.gov/wp-content/uploads/2025/02/M-25-21-Accelerating-Federal-Use-of-AI-through-Innovation-Governance-and-Public-Trust.pdf
OMB M-25-21 sets governance, inventory, risk-management, and public-trust requirements for federal agencies adopting AI.
[Office of Management and Budget 2025] M-25-22: Driving efficient acquisition of artificial intelligence in government. https://www.whitehouse.gov/wp-content/uploads/2025/02/M-25-22-Driving-Efficient-Acquisition-of-Artificial-Intelligence-in-Government.pdf
OMB M-25-22 directs federal AI acquisition toward competition, interoperability, performance evaluation, data rights, and lifecycle risk management.
[OLMo Team 2025] 2 olmo 2 furious. https://arxiv.org/abs/2501.00656
OLMo 2 documents and releases model weights, data mixtures, training and evaluation code, recipes, logs, and intermediate checkpoints for studying and attempting to reproduce its development.
[Olsson et al. 2022] In-context learning and induction heads. https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html
This work presents evidence that induction heads contribute to a form of in-context learning in the studied transformers; it does not establish that they explain every in-context-learning mechanism.
[Olston et al. 2017] TensorFlow-serving: Flexible, high-performance ML serving. arXiv preprint arXiv:1712.06139. https://arxiv.org/abs/1712.06139
TensorFlow Serving describes a model-serving architecture with versioned artifacts, pluggable sources, batching, and online request handling.
[Ong et al. 2024] RouteLLM: Learning to route llms with preference data. arXiv preprint arXiv:2406.18665. https://arxiv.org/abs/2406.18665
RouteLLM evaluates learned routers as policies that trade model quality against cost, making clear that routing behavior must be measured rather than assumed.
[ONNX Project 2026] ONNX intermediate representation specification. https://onnx.ai/onnx/repo-docs/IR.html
The ONNX IR specifies a versioned computation graph with operators, functions, types, and tensor initializers; large tensors may be stored as external data.
[Oord et al. 2018] Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748. https://arxiv.org/abs/1807.03748
CPC learns representations from unlabeled data by predicting future observations in latent space using a contrastive loss (InfoNCE), demonstrating strong results across speech, images, text, and reinforcement learning.
[Open Container Initiative 2025] OCI distribution specification. https://github.com/opencontainers/distribution-spec/blob/main/spec.md
The OCI distribution protocol separates mutable tags from digest-addressed manifests and blobs, with descriptors binding media type, size, and content digest.
[Open Source Initiative 2024] The open source AI definition 1.0. https://opensource.org/ai/open-source-ai-definition
The OSI definition treats Open Source AI as requiring use, study, modification, and sharing freedoms, with data information, code, and parameters available in the preferred form for modification.
[Open X-Embodiment Collaboration 2023] Open x-embodiment: Robotic learning datasets and RT-x models. arXiv preprint arXiv:2310.08864. https://arxiv.org/abs/2310.08864
Open X-Embodiment assembles a dataset of 1M+ trajectories from 22 robot embodiments across 21 institutions and trains RT-X models that show positive transfer across robot platforms.
[OpenAI 2023] GPT-4 technical report. https://cdn.openai.com/papers/gpt-4.pdf
OpenAI reports GPT-4 benchmark and safety evaluations while explicitly withholding architecture, model size, hardware, training compute, dataset construction, and training method.
[OpenAI 2024] Hello GPT-4o. https://openai.com/index/hello-gpt-4o/
OpenAI introduces GPT-4o as one model handling text, image, and audio inputs and outputs with lower audio-response latency than a chained voice pipeline.
[OpenAI 2024] Learning to reason with llms. https://openai.com/index/learning-to-reason-with-llms/
OpenAI describes training o1 with reinforcement learning for reasoning and reports that performance increases with training and test-time computation.
[OpenAI 2024] Introducing SWE-bench Verified. https://openai.com/index/introducing-swe-bench-verified/
Professional software developers screened SWE-bench tasks for clear issue descriptions, appropriate tests, and feasibility, producing a 500-task subset and a containerized harness.
[OpenAI 2025] Sycophancy in GPT-4o: What happened and what we're doing about it. https://openai.com/index/sycophancy-in-gpt-4o/
The post-mortem of the sycophantic model update, including the admission that user memory contributed to exacerbating sycophancy's effects in some cases.
[OpenAI 2025] 4o image generation system card addendum. https://cdn.openai.com/11998be9-5319-4302-bfbf-1167e093f1fb/Native_Image_Generation_System_Card.pdf
OpenAI classifies GPT-4o native image generation as autoregressive but does not disclose its internal representation or decoder design.
[OpenAI 2025] Model spec. https://model-spec.openai.com/2025-12-18.html
The Model Spec defines intended model behavior and authority levels for resolving conflicting instructions.
[OpenAI 2025] Preparedness framework, version 2. https://cdn.openai.com/pdf/18a02b5d-6b67-4cec-ab64-68cdfbddebcd/preparedness-framework-v2.pdf
Names the capability categories tracked before deployment and defines High and Critical thresholds for each. A capabilities report and a safeguards report go to a safety advisory group, which recommends; the final deployment decision rests with OpenAI leadership.
[OpenAI 2025] Introducing gpt-oss. https://github.com/openai/gpt-oss
The release provides Apache-2.0 weights, model documentation, tokenizer material, and reference inference implementations, but not its training corpus or full training pipeline.
[OpenAI 2026] Memory FAQ. https://help.openai.com/en/articles/8590148-memory-faq
The FAQ distinguishes saved memory from chat history and documents control, deletion, retention, and source-removal behavior.
[OpenAI 2026] Human-in-the-loop. https://openai.github.io/openai-agents-python/human_in_the_loop/
The OpenAI Agents SDK documents serializable interruptions that pause a run for per-call approval and resume the original run state, including nested-agent tool calls.
[OpenAI 2026] Computer use. https://developers.openai.com/api/docs/guides/tools-computer-use
Official guidance for screenshot-action loops, original-resolution coordinates, isolated execution, untrusted content, sensitive-data transmission, and approval immediately before risky actions.
[OpenAI 2026] Agent orchestration. https://openai.github.io/openai-agents-python/multi_agent/
The official SDK documentation distinguishes manager-style agents-as-tools, handoffs, code-driven pipelines, and parallel execution.
[OpenAI 2026] Why we no longer evaluate SWE-bench Verified. https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/
A targeted re-audit of frequently failed SWE-bench Verified tasks found continuing test and specification defects and evidence of exposure, illustrating that benchmark validity must be reassessed over time.
[OpenAI 2026] Introducing GPT-5.5. https://openai.com/index/introducing-gpt-5-5/
OpenAI reports GPT-5.5 at 84.9 percent wins or ties on original GDPval under xhigh reasoning in a research environment.
[OpenSSF AI/ML Security Working Group 2025] OpenSSF model signing specification. https://github.com/ossf/model-signing-spec
OMS signs a detached manifest of model paths and digests; verification still requires a trust policy binding the signing credential to an approved identity.
[OpenTelemetry 2026] OpenTelemetry GenAI semantic conventions. https://github.com/open-telemetry/semantic-conventions-genai/tree/e771bc90d0ebd743e954dc3a4feb5201cd3a5cf8
OpenTelemetry GenAI semantic conventions specify attributes and spans for tracing model calls, agent steps, and GenAI system behavior across vendors.
[OpenTelemetry 2026] Trace semantic conventions. https://opentelemetry.io/docs/specs/semconv/general/trace/
OpenTelemetry trace conventions define common span names, attributes, events, and status fields so telemetry remains comparable across instrumented services.
[OpenTelemetry 2026] OpenTelemetry GenAI semantic conventions. https://github.com/open-telemetry/semantic-conventions-genai
OpenTelemetry's official semantic conventions for instrumenting generative AI calls with standardized span attributes covering model, token usage, and request metadata.
[OpenTelemetry n.d.] OpenTelemetry generative AI semantic conventions. https://opentelemetry.io/docs/specs/semconv/registry/attributes/gen-ai/
The OpenTelemetry registry defines generative-AI telemetry attributes and warns that model inputs and outputs are likely to contain sensitive or personally identifiable information.
[OpenXLA Project 2026] StableHLO compatibility. https://openxla.org/stablehlo/compatibility
StableHLO specifies compatibility for portable tensor-program artifacts while explicitly excluding guarantees such as identical numerical accuracy across consumers.
[OpenXLA Project 2026] XLA GPU emitters. https://openxla.org/xla/emitters
OpenXLA documentation explains how GPU emitters lower fused operations into architecture-specific kernels and when different emitter paths apply.
[Organisation for Economic Co-operation and Development 2001] Measuring productivity: OECD manual. https://doi.org/10.1787/9789264194519-en
The manual explains why productivity must relate a specified output measure to specified inputs and documents the choices needed for valid comparisons.
[Organisation for Economic Co-operation and Development 2024] OECD AI principles. https://oecd.ai/en/ai-principles
The OECD AI Principles set policy commitments around human-centered values, transparency, robustness, accountability, investment, and international cooperation.
[Organisation for Economic Co-operation and Development 2025] Competition in artificial intelligence infrastructure. https://doi.org/10.1787/623d1874-en
The OECD maps the multilayer AI infrastructure supply chain and reviews concentration, entry barriers, vertical relationships, switching barriers, and shortages while warning that reported segment shares are not one global market definition.
[Ou et al. 2025] Your absorbing discrete diffusion secretly models the conditional distributions of clean data. https://arxiv.org/abs/2406.03736
RADD expresses the absorbing-diffusion score through clean-token conditional distributions, enabling output reuse when the noisy state does not change and establishing a conditional objective equivalence to any-order autoregression.
[Ouyang et al. 2022] Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems (NeurIPS). https://arxiv.org/abs/2203.02155
InstructGPT trains on labeler demonstrations, then preference rankings and reinforcement learning; its final assistant behavior is evidence about the full pipeline, not SFT alone.
[Ouyang et al. 2022] Training language models to follow instructions with human feedback. https://proceedings.neurips.cc/paper_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract.html
InstructGPT documents a pipeline of supervised demonstrations, ranked comparisons, reward modeling, and policy optimization against that reward.
[Ouyang et al. 2025] KernelBench: Can llms write efficient GPU kernels?. PMLR. https://proceedings.mlr.press/v267/ouyang25a.html
KernelBench evaluates generated GPU kernels on 250 workloads using correctness and speedup-aware metrics; in its reported one-shot setting, frontier models beat the PyTorch baseline on fewer than 20% of tasks.
[Ouyang et al. 2025] OmniDocBench: Benchmarking diverse PDF document parsing with comprehensive annotations. https://openaccess.thecvf.com/content/CVPR2025/html/Ouyang_OmniDocBench_Benchmarking_Diverse_PDF_Document_Parsing_with_Comprehensive_Annotations_CVPR_2025_paper.html
Provides a document-parsing benchmark with annotations and metrics for text, layout, tables, formulas, and reading order.
[OWASP CycloneDX 2023] Machine learning bill of materials (ML-BOM). https://cyclonedx.org/capabilities/mlbom/
CycloneDX ML-BOM can encode declared models, datasets, dependencies, parameters, model-card fields, and lineage, but it does not validate the truth or completeness of those declarations.
[OWASP Foundation n.d.] Mobile application security verification standard. https://mas.owasp.org/MASVS/
MASVS organizes mobile security and privacy requirements across the complete application rather than treating local computation as sufficient protection.
[Packer et al. 2023] MemGPT: Towards llms as operating systems. https://arxiv.org/abs/2310.08560
MemGPT introduces virtual context management for LLMs, using an OS-inspired hierarchical memory system to page data between a fixed context window and external storage, enabling unbounded context for document analysis and multi-session chat.
[Pagnoni et al. 2025] Byte latent transformer: Patches scale better than tokens. Association for Computational Linguistics. https://arxiv.org/abs/2412.09871
BLT replaces fixed subword tokenization with dynamic byte patches selected from next-byte entropy and reports competitive FLOP-controlled scaling through its tested 8B-parameter setting.
[Pai 2025] Designing large language model applications: a holistic approach. O'Reilly Media. https://www.oreilly.com/library/view/designing-large-language/9781098150495/
An O'Reilly book by Suhas Pai covering practical design patterns and engineering decisions for building production LLM applications in enterprises.
[Pan et al. 2025] Training software engineering agents and verifiers with SWE-gym. PMLR. https://proceedings.mlr.press/v267/pan25g.html
SWE-Gym packages 2,438 real repository tasks with reproducible runtimes and tests, supporting supervised agent training and verifier-guided inference-time scaling.
[Panickssery et al. 2024] LLM evaluators recognize and favor their own generations. https://proceedings.neurips.cc/paper_files/paper/2024/hash/7f1f0218e45f5414c79c0679633e47bc-Abstract-Conference.html
Controlled experiments connect evaluator self-recognition with self-preference, showing that same-family grading can diverge systematically from human judgments.
[Parasuraman et al. 2000] A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans. https://doi.org/10.1109/3468.844354
Parasuraman, Sheridan, and Wickens separate automation by information acquisition, analysis, decision selection, and action implementation, each with distinct levels of human involvement.
[Park et al. 2023] Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442. https://arxiv.org/abs/2304.03442
The memory stream with retrieval scored by recency, importance, and relevance, plus periodic reflection that synthesizes higher-level memories: the conceptual template every later memory system echoes.
[Park et al. 2024] Grammar-aligned decoding. https://proceedings.neurips.cc/paper_files/paper/2024/hash/2bdc2267c3d7d01523e2e17ac0a754f3-Abstract-Conference.html
Grammar-Aligned Decoding shows that ordinary local grammar masking can distort a model's sequence distribution even while guaranteeing grammatical output.
[Parra and Delimarsky 2026] The 2026-07-28 specification. https://blog.modelcontextprotocol.io/posts/2026-07-28/
The final MCP 2026-07-28 release introduces a stateless protocol core, versioned extensions, authorization hardening, and a formal feature lifecycle.
[Paszke et al. 2019] PyTorch: An imperative style, high-performance deep learning library. https://arxiv.org/abs/1912.01703
The PyTorch design paper explains its eager tensor interface, dynamic autograd graph, dispatcher, allocator, and C++ execution path.
[Patarasuk and Yuan 2009] Bandwidth optimal all-reduce algorithms for clusters of workstations. Journal of Parallel and Distributed Computing. https://www.cs.fsu.edu/~xyuan/paper/09jpdc.pdf
The paper derives a communication lower bound and a bandwidth-optimal ring all-reduce for large messages under stated topology assumptions.
[Patil et al. 2025] The berkeley function calling leaderboard (BFCL): From tool use to agentic evaluation of large language models. PMLR. https://proceedings.mlr.press/v267/patil25a.html
BFCL evaluates function calling across serial, parallel, abstention, and stateful multi-turn settings instead of reducing tool use to one argument-matching score.
[Patwardhan et al. 2025] GDPval: Evaluating AI model performance on real-world economically valuable tasks. arXiv preprint arXiv:2510.04374. https://arxiv.org/abs/2510.04374
GDPval contains 1,320 professional deliverable tasks across 44 occupations, with launch results reported on a 220-task open subset through blind expert comparisons.
[Paulo et al. 2025] Transcoders beat sparse autoencoders for interpretability. arXiv preprint arXiv:2501.18823. https://arxiv.org/abs/2501.18823
On the tested models and automated metrics, transcoders offered a better reconstruction-interpretability trade-off than comparable MLP-output sparse autoencoders; the result is bounded to that comparison.
[Pearce and Song 2024] Reconciling kaplan and chinchilla scaling laws. Transactions on Machine Learning Research. https://arxiv.org/abs/2406.12907
Attributes the Kaplan-Chinchilla disagreement largely to Kaplan counting non-embedding rather than total parameters at small scale; once embeddings are included and scale grows, the optimal exponent converges to the Chinchilla estimate.
[Peebles and Xie 2023] Scalable diffusion models with transformers. https://arxiv.org/abs/2212.09748
DiT replaces the U-Net backbone in latent diffusion models with a transformer, showing that FID improves consistently as model Gflops scale, achieving state-of-the-art FID 2.27 on ImageNet 256x256.
[Penedo et al. 2023] The RefinedWeb dataset for falcon LLM: Outperforming curated corpora with web data, and web data only. https://arxiv.org/abs/2306.01116
RefinedWeb shows that aggressively filtered and deduplicated CommonCrawl web data alone, yielding five trillion tokens, can train LLMs that outperform models trained on curated corpora like The Pile.
[Penedo et al. 2024] The FineWeb datasets: Decanting the web for the finest text data at scale. https://arxiv.org/abs/2406.17557
FineWeb is a 15-trillion token pretraining dataset from 96 Common Crawl snapshots, with ablation-guided filtering and per-snapshot MinHash deduplication, that outperforms other public pretraining datasets; FineWeb-Edu is a 1.3-trillion token educational subset with strong MMLU and ARC results.
[Peng et al. 2023] YaRN: Efficient context window extension of large language models. arXiv preprint arXiv:2309.00071. https://arxiv.org/abs/2309.00071
YaRN extends RoPE-based LLaMA context windows using far fewer tokens and training steps than previous approaches, demonstrating extrapolation beyond the fine-tuning length.
[Perdomo et al. 2020] Performative prediction. https://proceedings.mlr.press/v119/perdomo20a.html
Performative prediction studies settings where deploying a model changes the data distribution and defines stability and optimality under that feedback.
[Perez and Ribeiro 2022] Ignore previous prompt: Attack techniques for language models. ML Safety Workshop at NeurIPS 2022. https://arxiv.org/abs/2211.09527
PromptInject demonstrates direct goal-hijacking and prompt-leaking attacks against GPT-3 using simple adversarial user inputs.
[Perez et al. 2022] Red teaming language models with language models. arXiv preprint arXiv:2202.03286. https://arxiv.org/abs/2202.03286
This paper introduces LM-based red-teaming, using one language model to automatically generate test cases that elicit harmful outputs from a target LM, uncovering tens of thousands of failures in a 280B-parameter chatbot.
[Petrov et al. 2023] Language model tokenizers introduce unfairness between languages. https://proceedings.neurips.cc/paper_files/paper/2023/hash/74bb24dca8334adce292883b4b651eda-Abstract-Conference.html
Across parallel translations, the paper finds token-count differences of up to 15 times for some tokenizer and language pairs, creating cost, latency, and usable-context disparities.
[Pineau et al. 2021] Improving reproducibility in machine learning research: a report from the NeurIPS 2019 reproducibility program. Journal of Machine Learning Research. https://www.jmlr.org/papers/v22/20-303.html
The report describes a reproducibility program centered on accessible code, experiment reporting, and independent reproduction using the same code and data.
[Podell et al. 2024] SDXL: Improving latent diffusion models for high-resolution image synthesis. https://arxiv.org/abs/2307.01952
SDXL is a latent diffusion model for text-to-image synthesis featuring a 2.6B-parameter UNet with dual text encoders, novel size and crop conditioning, multi-aspect training, and an optional refinement stage.
[Podkopaev and Ramdas 2022] Tracking the risk of a deployed model and detecting harmful distribution shifts. https://arxiv.org/abs/2110.06177
Podkopaev and Ramdas monitor deployed-model risk with sampled or delayed labels and time-uniform guarantees, distinguishing harmful performance change from arbitrary distribution change.
[Pokkunuri 2019] Ensuring rollback safety during deployments. https://d1.awsstatic.com/builderslibrary/pdfs/ensuring-rollback-safety-during-deployments.pdf
AWS explains how mixed-version protocol and data compatibility, two-phase changes, bake periods, and upgrade-downgrade tests make rollback safe.
[Polu and Sutskever 2020] Generative language modeling for automated theorem proving. arXiv preprint arXiv:2009.03393. https://arxiv.org/abs/2009.03393
GPT-f applies transformer language models to Metamath proof search and contributed shorter proofs for existing theorems to the formal library.
[Polyak and others 2024] Movie gen: a cast of media foundation models. arXiv preprint arXiv:2410.13720. https://arxiv.org/abs/2410.13720
Meta's media foundation models, whose video model carries a 73,000-token context for sixteen seconds of high-definition footage, against roughly four thousand latent tokens for one megapixel image.
[Poole et al. 2019] On variational bounds of mutual information. https://proceedings.mlr.press/v97/poole19a.html
Poole et al. formalize the sampling assumptions and bias-variance trade-offs behind neural mutual-information bounds, including contrastive bounds with a finite candidate set.
[Pope et al. 2023] Efficiently scaling transformer inference. https://proceedings.mlsys.org/paper_files/paper/2023/hash/c4be71ab8d24cdfb45e3d06dbfca2780-Abstract-mlsys2023.html
This paper presents an analytical partitioning framework and low-level optimizations for efficient Transformer inference on TPU v4 slices, achieving 29ms per token and 76% MFU on PaLM 540B with int8 quantization.
[PostgreSQL Global Development Group 2026] Row security policies. https://www.postgresql.org/docs/current/ddl-rowsecurity.html
PostgreSQL row security can constrain reads and writes, but table owners, superusers, BYPASSRLS roles, policy composition, and race conditions require explicit treatment.
[Press and Wolf 2017] Using the output embedding to improve language models. Association for Computational Linguistics. https://arxiv.org/abs/1608.05859
Tying the input and output embedding matrices in neural language models reduces perplexity and can cut translation model parameter count to less than half with no performance loss.
[Press et al. 2022] Train short, test long: Attention with linear biases enables input length extrapolation. https://arxiv.org/abs/2108.12409
ALiBi replaces positional embeddings with per-head linear distance penalties on attention scores, enabling transformers trained on short sequences to extrapolate to longer ones at inference time with no added runtime cost.
[Pushkarna et al. 2022] Data cards: Purposeful and transparent dataset documentation for responsible AI. https://arxiv.org/abs/2204.01075
Data Cards organize dataset documentation around origins, collection and annotation, intended use, decisions, and maintenance across a dataset lifecycle.
[Pydantic Services 2026] Durable execution: Overview. https://pydantic.dev/docs/ai/capabilities/durable_execution/overview/
Pydantic AI documents durable-execution integrations that delegate persistence and recovery to workflow systems such as Temporal, DBOS, Prefect, and Restate.
[Python Software Foundation 2026] <span class="nocase">pickle</span> — Python object serialization. https://docs.python.org/3/library/pickle.html
The Python documentation defines pickle as binary object serialization and explains why unpickling untrusted input can execute arbitrary code.
[PyTorch n.d.] Getting started with fully sharded data parallel (FSDP2). https://docs.pytorch.org/tutorials/intermediate/FSDP_tutorial.html
The official tutorial explains how FSDP2 shards parameters, gradients, and optimizer state and schedules all-gather and reduce-scatter operations.
[PyTorch Contributors 2026] Autograd mechanics. https://docs.pytorch.org/docs/stable/notes/autograd.html
The note documents PyTorch's dynamic autograd graph, saved tensors, nondifferentiable conventions, and in-place correctness checks.
[PyTorch Contributors 2026] Extending PyTorch. https://docs.pytorch.org/docs/stable/notes/extending.html
PyTorch documents how custom modules, autograd functions, operators, and tensor behaviors integrate with its execution and differentiation systems.
[PyTorch Contributors 2026] Gradcheck mechanics. https://docs.pytorch.org/docs/stable/notes/gradcheck.html
PyTorch explains how gradcheck compares analytical derivatives with finite differences and why tolerances, complex values, and nondeterminism matter.
[PyTorch Contributors 2026] Activation checkpointing. https://docs.pytorch.org/docs/stable/checkpoint.html
Activation checkpointing discards selected forward activations and recomputes them during backward propagation to trade additional computation for lower memory use.
[PyTorch Contributors 2026] <span class="nocase">torch.compile</span> programming model. https://docs.pytorch.org/docs/stable/user_guide/torch_compiler/compile/programming_model.html
The programming-model guide defines graph capture, graph breaks, guards, recompilation, and the partial-versus-full graph contract.
[PyTorch Contributors 2026] PyTorch DTensor: Distributed tensor. https://docs.pytorch.org/docs/stable/distributed.tensor.html
The DTensor contract defines a logical tensor over a device mesh using Shard, Replicate, and Partial placements plus explicit redistribution semantics.
[PyTorch Contributors 2026] torch.compile troubleshooting. https://docs.pytorch.org/docs/stable/user_guide/torch_compiler/torch.compiler_troubleshooting.html
PyTorch's troubleshooting guide explains graph breaks, recompilation, guard failures, logging, and debugging in the torch.compile pipeline.
[PyTorch Contributors 2026] Torch distributed elastic. https://docs.pytorch.org/docs/stable/distributed.elastic.html
Torch Distributed Elastic manages worker groups that may fail or change membership, restarting the group under a declared rendezvous and fault model.
[PyTorch Contributors 2026] Distributed checkpoint: torch.distributed.checkpoint. https://docs.pytorch.org/docs/stable/distributed.checkpoint.html
PyTorch Distributed Checkpoint can reshard model and optimizer state across trainer counts and parallel layouts, but its state dictionaries have no promised backward compatibility across PyTorch versions.
[PyTorch Contributors 2026] Serialization semantics. https://docs.pytorch.org/docs/stable/notes/serialization.html
PyTorch documents its ZIP64 checkpoint layout, the weights-only unpickler default introduced in 2.6, and the denial-of-service and possible memory-corruption risks that remain with untrusted artifacts.
[PyTorch Foundation n.d.] ExecuTorch: Architecture and components. https://docs.pytorch.org/executorch/stable/getting-started-architecture
ExecuTorch documents separate program-preparation, runtime-preparation, and execution phases, with target-specific lowering and linked backend support.
[Qi et al. 2024] Fine-tuning aligned language models compromises safety, even when users do not intend to!. https://proceedings.iclr.cc/paper_files/paper/2024/hash/83b7da3ed13f06c13ce82235c8eedf35-Abstract-Conference.html
The experiments show that fine-tuning aligned models on benign or adversarial data can weaken safety behavior, with outcomes depending on data and method.
[Qian et al. 2021] Glancing transformer for non-autoregressive neural machine translation. https://arxiv.org/abs/2008.07905
GLAT introduces a Glancing Language Model training strategy that enables non-autoregressive neural machine translation in a single parallel decoding pass, closing the quality gap to autoregressive Transformer to within 0.25-0.9 BLEU at 8x-15x speedup.
[Qian et al. 2025] ToolRL: Reward is all tool learning needs. https://proceedings.neurips.cc/paper_files/paper/2025/hash/97c5b2707228e7e3fb67e4ecc2e0e607-Abstract-Conference.html
ToolRL shows experimentally that tool-use training depends on reward design across action validity, tool selection, parameters, scale, granularity, and timing.
[Qin et al. 2025] Mooncake: Trading more storage for less computation—a kvcache-centric architecture for serving LLM chatbot. USENIX Association. https://www.usenix.org/conference/fast25/presentation/qin
Mooncake manages KV state across a distributed cache hierarchy and trades storage and transfer capacity against repeated prefill computation.
[Qiu et al. 2025] ModServe: Modality- and stage-aware resource disaggregation for scalable multimodal model serving. https://arxiv.org/abs/2502.00937
ModServe measures heterogeneous multimodal inference stages and heavy-tailed production requests, then independently scales and places stages to meet tail-latency objectives.
[Qu et al. 2021] RocketQA: An optimized training approach to dense passage retrieval for open-domain question answering. https://aclanthology.org/2021.naacl-main.466/
RocketQA improves dual-encoder dense passage retrieval for open-domain QA via three training strategies: cross-batch negatives, denoised hard negatives, and cross-encoder-based data augmentation.
[Qwen Team 2025] Qwen3-next-80B-A3B-instruct model card. https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct
Qwen3-Next is an 80B-total, 3B-active MoE model whose 48 layers alternate three Gated DeltaNet linear-attention layers with one gated full-attention layer.
[Rabanser et al. 2019] Failing loudly: An empirical study of methods for detecting dataset shift. https://proceedings.neurips.cc/paper/2019/hash/846c260d715e5b854ffad5f70a516c88-Abstract.html
Rabanser, Günnemann, and Lipton empirically compare high-dimensional dataset-shift detectors and methods for characterizing detected shifts.
[Radford et al. 2019] Language models are unsupervised multitask learners. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf
The GPT-2 report describes a reversible byte-to-Unicode mapping, regex pretokenization, and BPE within each resulting segment.
[Radford et al. 2021] Learning transferable visual models from natural language supervision. https://arxiv.org/abs/2103.00020
CLIP trains an image encoder and text encoder jointly on 400 million image-text pairs via contrastive learning, enabling zero-shot transfer to downstream visual tasks through natural language.
[Radford et al. 2023] Robust speech recognition via large-scale weak supervision. https://arxiv.org/abs/2212.04356
Whisper trains an encoder-decoder Transformer on 680,000 hours of weakly supervised multilingual audio and reports broad zero-shot transfer across held-out ASR datasets without dataset-specific fine-tuning.
[Rafailov et al. 2023] Direct preference optimization: Your language model is secretly a reward model. https://proceedings.neurips.cc/paper_files/paper/2023/hash/a85b405ed65c6477a4fe8302b5e06ce7-Abstract-Conference.html
DPO optimizes a policy directly from chosen and rejected responses under its preference model, avoiding a separately fitted reward model and online RL loop.
[Ragan-Kelley et al. 2013] Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines. https://dl.acm.org/doi/10.1145/2491956.2462176
Halide separates what to compute from how to schedule it on a machine, an influential design for exploring locality, parallelism, vectorization, and recomputation without changing the algorithm.
[Raina et al. 2024] Is LLM-as-a-judge robust? Investigating universal adversarial attacks on zero-shot LLM assessment. Association for Computational Linguistics. https://aclanthology.org/2024.emnlp-main.427/
The paper constructs transferable short phrases that inflate judge scores and finds greater vulnerability in its tested absolute-scoring protocols than in comparative assessment.
[Rajbhandari et al. 2020] ZeRO: Memory optimizations toward training trillion parameter models. https://arxiv.org/abs/1910.02054
ZeRO partitions optimizer state, gradients, and parameters to remove memory redundancy from data-parallel training while changing communication schedules.
[Rajbhandari et al. 2021] ZeRO-infinity: Breaking the GPU memory wall for extreme scale deep learning. https://arxiv.org/abs/2104.07857
ZeRO-Infinity is a heterogeneous training system that offloads model states to CPU and NVMe memory, enabling training of models with tens of trillions of parameters on existing GPU clusters without model code refactoring.
[Raji et al. 2021] AI and the everything in the whole wide world benchmark. https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/hash/084b6fbb10729ed4da8c3d3f5a3ae7c9-Abstract-round2.html
Raji et al. examine how narrow benchmark tasks are made to stand in for broad claims about general AI progress, and argue that those claims often exceed the construct evidence the tests provide.
[Rajput et al. 2023] Recommender systems with generative retrieval. https://arxiv.org/abs/2305.05065
TIGER quantizes item content embeddings into semantic IDs and trains a sequence model to decode the next item's ID, replacing approximate-nearest-neighbor lookup in its evaluated pipeline while retaining an external codebook and item mapping.
[Ramesh et al. 2022] Hierarchical text-conditional image generation with CLIP latents. arXiv preprint arXiv:2204.06125. https://arxiv.org/abs/2204.06125
DALL-E 2 (unCLIP) generates images from text by chaining a prior that maps text to CLIP image embeddings with a diffusion decoder that inverts those embeddings into 1024x1024 images.
[Rand et al. 2025] Queued up: 2025 edition. Characteristics of power plants seeking transmission interconnection. https://emp.lbl.gov/publications/queued-2025-edition-characteristics
The 2025 Queued Up report measures the capacity, location, technology, and development status of projects waiting in U.S. interconnection queues.
[Raschka 2024] Build a large language model (from scratch). Manning Publications. https://www.manning.com/books/build-a-large-language-model-from-scratch
A hands-on book guiding readers to implement LLM attention mechanisms and GPT-style transformer architectures from scratch, covering training, fine-tuning, and instruction following.
[Rashkin et al. 2023] Measuring attribution in natural language generation models. MIT Press. https://aclanthology.org/2023.cl-4.2/
The AIS framework defines attribution as support from independent identified sources under explicit annotation guidelines, while leaving source quality as a complementary judgment.
[Ratner et al. 2016] Data programming: Creating large training sets, quickly. Advances in Neural Information Processing Systems (NeurIPS). https://arxiv.org/abs/1605.07723
Data programming lets users write noisy labeling functions instead of hand-labeling examples; a generative model denoises their outputs to produce large training sets programmatically.
[Rawles et al. 2025] AndroidWorld: a dynamic benchmarking environment for autonomous agents. https://proceedings.iclr.cc/paper_files/paper/2025/hash/01a83bc2f2732a58e6aa731e659e7101-Abstract-Conference.html
A reproducible Android environment with 116 parameterized tasks across 20 applications, each with setup, state-based success checking, and teardown; task variations materially affected results.
[Reddi et al. 2022] MLPerf Mobile Inference Benchmark: An industry-standard open-source machine learning benchmark for on-device AI. https://proceedings.mlsys.org/paper_files/paper/2022/hash/a2b2702ea7e682c5ea2c20e8f71efb0c-Abstract.html
MLPerf Mobile defines common tasks, quality targets, run rules, and device-side measurement to make results across heterogeneous mobile stacks interpretable.
[Reddit, Inc. 2024] Prospectus filed pursuant to rule 424(b)(4). U.S. Securities,Exchange Commission. https://www.sec.gov/Archives/edgar/data/1713445/000162828024012380/reddit-final424b4.htm
Discloses several data-licensing arrangements as aggregate contracts rather than a per-token or named-partner price.
[Reimers and Gurevych 2019] Sentence-BERT: Sentence embeddings using siamese BERT-networks. https://aclanthology.org/D19-1410/
Sentence-BERT trains siamese BERT encoders to produce fixed-size sentence vectors that can be compared directly, replacing exhaustive pairwise transformer scoring for semantic search.
[Ren et al. 2025] DeepSeek-prover-V2: Advancing formal mathematical reasoning via reinforcement learning for subgoal decomposition. arXiv preprint arXiv:2504.21801. https://arxiv.org/abs/2504.21801
DeepSeek-Prover-V2 combines informal and formal reasoning for Lean 4 theorem proving, using recursive decomposition and RL to reach strong MiniF2F and PutnamBench results.
[Rhoades 1993] The herfindahl-hirschman index. Board of Governors of the Federal Reserve System. https://fraser.stlouisfed.org/files/docs/publications/FRB/pages/1990-1994/33101_1990-1994.pdf
Rhoades explains the share-squared index, its independent origins in work by Hirschman and Herfindahl, and its use as a concentration screen rather than a complete competition analysis.
[Richer 2015] OAuth 2.0 token introspection. https://www.rfc-editor.org/rfc/rfc7662.html
RFC 7662 lets an authorized protected resource query current token activity and metadata.
[Rissanen 1978] Modeling by shortest data description. Automatica.
Rissanen introduces the Minimum Description Length principle, framing model selection as choosing the model that most compresses the data.
[Robertson and Zaragoza 2009] The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval. https://doi.org/10.1561/1500000019
A systematic derivation and explanation of the probabilistic relevance framework, BM25 term weighting, and its parameters.
[Rombach et al. 2022] High-resolution image synthesis with latent diffusion models. https://arxiv.org/abs/2112.10752
Latent Diffusion Models (LDMs) run diffusion in a pretrained autoencoder's latent space, cutting training and inference cost while adding cross-attention conditioning for text-to-image and other tasks.
[Rose et al. 2020] Zero trust architecture. https://csrc.nist.gov/pubs/sp/800/207/final
NIST's reference architecture separates policy decisions from enforcement and rejects implicit trust based on network location.
[Rose et al. 2020] Zero trust architecture. https://doi.org/10.6028/NIST.SP.800-207
Zero trust replaces implicit network-location trust with explicit authentication and authorization for subjects, assets, and resources.
[Rosenblatt 1958] The perceptron: a probabilistic model for information storage and organization in the brain. Psychological Review.
Rosenblatt introduces the perceptron, a trainable linear classifier with a weight-update learning rule, an early foundation of neural networks.
[Rosenfeld et al. 2020] A constructive prediction of the generalization error across scales. https://arxiv.org/abs/1909.12673
Proposes the parametric envelope form of the generalization error as a power law in model size and data size plus an irreducible floor, fit across vision and language, predating its language-model specialization.
[Rosner 2025] Chairman rosner's letter to the rtos and isos on large load forecasting. https://www.ferc.gov/news-events/news/chairman-rosners-letter-rtosisos-large-load-forecasting
FERC asks regional grid operators to improve large-load forecasts, disclose methods and uncertainty, and coordinate them with transmission planning.
[Röttger et al. 2024] XSTest: a test suite for identifying exaggerated safety behaviours in large language models. https://arxiv.org/abs/2308.01263
XSTEST is a 250-prompt test suite that identifies exaggerated safety behaviors in LLMs, where models refuse safe prompts due to lexical overlap with unsafe ones.
[Rouhani and others 2023] Microscaling data formats for deep learning. https://arxiv.org/abs/2310.10537
The OCP Microscaling (MX) proposal from AMD, Arm, Intel, Meta, Microsoft, NVIDIA, and Qualcomm pairs narrow floating-point and integer element types with a shared per-block scale, and shows MX formats, including MXFP4, working for inference and training with minimal accuracy loss.
[Ru et al. 2024] RAGChecker: a fine-grained framework for diagnosing retrieval-augmented generation. https://proceedings.neurips.cc/paper_files/paper/2024/hash/27245589131d17368cccdfa990cbf16e-Abstract.html
RAGChecker separates retrieval and generation diagnostics and validates its metrics against human judgments.
[Rumelhart et al. 1986] Learning representations by back-propagating errors. Nature. https://www.nature.com/articles/323533a0
The paper shows how backpropagation adjusts internal representations by propagating output error gradients through a multilayer network.
[Russinovich et al. 2025] Great, now write an article about that: The crescendo multi-turn LLM jailbreak attack. https://www.usenix.org/conference/usenixsecurity25/presentation/russinovich
Crescendo is a multi-turn LLM jailbreak that escalates from benign prompts using the model's own prior outputs, bypassing safety alignment on GPT-4, Gemini-Pro, and other models with high attack success rates.
[Rust et al. 2021] How good is your tokenizer? On the monolingual performance of multilingual language models. Association for Computational Linguistics. https://aclanthology.org/2021.acl-long.243/
Controlled monolingual comparisons show that tokenizer quality contributes to multilingual downstream performance independently of pretraining data.
[Saeidi et al. 2025] Insights into alignment: Evaluating DPO and its variants across multiple tasks. Association for Computational Linguistics. https://aclanthology.org/2025.acl-srw.26/
A comparison of DPO, IPO, KTO, and CPO across tasks and initialization regimes finds that the leading method varies by setting.
[Safetensors Project 2026] Safetensors format specification. https://github.com/huggingface/safetensors/blob/62e4d8b86063a6e5f8967547fa09a09f67420e2a/README.md#format
The pinned safetensors specification defines an eight-byte header length, JSON tensor descriptors, and a contiguous byte buffer without pickle-style callable reconstruction.
[Saha and Ye 2024] I/O complexity of attention, or how optimal is FlashAttention?. PMLR. https://proceedings.mlr.press/v235/saha24a.html
The paper gives a pointwise IO lower bound matching FlashAttention for fast-memory capacity at least the square of head width, and identifies a different optimal regime below it.
[Saharia et al. 2022] Photorealistic text-to-image diffusion models with deep language understanding. https://arxiv.org/abs/2205.11487
Imagen combines a frozen T5-XXL text encoder with a cascade of diffusion models, finding that scaling the language model improves text-to-image quality more than scaling the image model, achieving FID 7.27 on COCO.
[Sahoo et al. 2024] Simple and effective masked diffusion language models. https://arxiv.org/abs/2406.07524
MDLM shows that masked discrete diffusion with a Rao-Blackwellized objective and modern training recipes closes most of the perplexity gap between diffusion and autoregressive language models.
[Salemi et al. 2024] LaMP: When large language models meet personalization. https://aclanthology.org/2024.acl-long.399/
LaMP provides seven personalized language-modeling tasks and evaluates retrieval methods for selecting relevant items from user profiles.
[Salimans et al. 2017] Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864. https://arxiv.org/abs/1703.03864
Evolution Strategies (ES) scaled to 1,440 parallel workers via shared random seeds achieves competitive results with policy-gradient RL on MuJoCo and Atari without backpropagation or value functions.
[Salimans and Ho 2022] Progressive distillation for fast sampling of diffusion models. https://arxiv.org/abs/2202.00512
Repeatedly distills a diffusion sampler into a student that needs half the steps, taking generation from thousands of steps down to as few as four at no more than the original training cost.
[Saltzer and Schroeder 1975] The protection of information in computer systems. Proceedings of the IEEE. https://doi.org/10.1109/PROC.1975.9939
Saltzer and Schroeder formulate enduring security principles including fail-safe defaults, complete mediation, separation of privilege, and least privilege.
[Sambasivan et al. 2021] “Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI. https://doi.org/10.1145/3411764.3445518
Sambasivan et al. document data cascades in high-stakes AI, where upstream data problems compound into large downstream model failures.
[San Roman et al. 2024] Proactive detection of voice cloning with localized watermarking. https://arxiv.org/abs/2401.17264
AudioSeal jointly trains a watermark generator and detector to identify and localize its embedded mark; it is not a detector for arbitrary unwatermarked synthetic speech.
[Santhanam et al. 2022] ColBERTv2: Effective and efficient retrieval via lightweight late interaction. https://aclanthology.org/2022.naacl-main.272/
ColBERTv2 combines residual compression and denoised supervision, reducing its uncompressed late-interaction index footprint by six to ten times in the reported experiments.
[Sardana et al. 2024] Beyond chinchilla-optimal: Accounting for inference in language model scaling laws. https://arxiv.org/abs/2401.00448
Sardana et al. add forecast inference demand to a scaling-law cost model and find that high demand can favor smaller models trained longer, conditional on fixed quality and fitted assumptions.
[Sardana et al. 2024] Beyond chinchilla-optimal: Accounting for inference in language model scaling laws. PMLR. https://proceedings.mlr.press/v235/sardana24a.html
Extends compute-optimal scaling to lifetime training-plus-inference cost and finds that high expected demand can favor smaller models trained for longer at a target quality.
[Sato 2014] Parallel change. https://martinfowler.com/bliki/ParallelChange.html
Parallel change uses expand, migrate, and contract phases so old and new representations can coexist during a safe transition.
[Sauer et al. 2024] Adversarial diffusion distillation. https://eccv.ecva.net/virtual/2024/poster/1963
Combines score distillation with an adversarial loss for one-to-four-step sampling, the method behind SDXL-Turbo, reaching real-time synthesis that matches its teacher within four steps.
[Savinov et al. 2022] Step-unrolled denoising autoencoders for text generation. https://arxiv.org/abs/2112.06749
SUNDAE proposes a non-autoregressive text generative model that iteratively denoises a sequence of tokens using unrolled denoising during training, achieving state-of-the-art results among non-autoregressive methods on WMT'14 translation.
[Saxena 2023] Prompt lookup decoding. https://github.com/apoorvumang/prompt-lookup-decoding
Prompt lookup proposes continuations by matching recent tokens against repeated spans in the prompt or output, then relies on target verification.
[Scale AI 2026] Humanity's last exam leaderboard. https://labs.scale.com/leaderboard/humanitys_last_exam
Scale AI's official leaderboard separates fixed no-tools evaluations from results reported under other tool protocols.
[Schaeffer et al. 2023] Are emergent abilities of large language models a mirage?. https://papers.neurips.cc/paper_files/paper/2023/hash/adc98a266f45005c403b8311ca7e8bd7-Abstract-Conference.html
Some apparent emergent abilities in LLMs may be induced by nonlinear or discontinuous evaluation metrics rather than discontinuous changes in model behavior.
[SchedMD n.d.] Slurm workload manager: Overview. https://slurm.schedmd.com/overview.html
The Slurm overview separates cluster resource allocation, queued-job scheduling, and execution of job steps on allocated nodes.
[Schick et al. 2023] Toolformer: Language models can teach themselves to use tools. https://proceedings.neurips.cc/paper/2023/hash/d842425e4bf79ba039352da0f658a906-Abstract-Conference.html
Toolformer generates candidate API annotations, executes and filters them by language-model loss, and fine-tunes a model to decide when and how to use five fixed APIs.
[Schrittwieser et al. 2020] Mastering atari, go, chess and shogi by planning with a learned model. Nature. https://www.nature.com/articles/s41586-020-03051-4
MuZero plans with learned hidden dynamics trained to predict rewards, policies, and values without reconstructing observations or receiving environment rules.
[Schulman et al. 2017] Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. https://arxiv.org/abs/1707.06347
PPO introduces a clipped surrogate objective for policy gradient reinforcement learning that achieves TRPO-level reliability with simpler first-order optimization and better sample complexity.
[Schulman and others 2025] LoRA without regret. https://thinkingmachines.ai/blog/lora/
A 2025 company research report comparing LoRA and full fine-tuning across specified Llama and Qwen experiments, including target-layer, rank, batch-size, and learning-rate sweeps.
[Schultz et al. 1997] A neural substrate of prediction and reward. Science.
Schultz, Dayan, and Montague report midbrain dopamine responses that resemble a reward-prediction error, linking neuroscience to temporal-difference reinforcement learning.
[Sculley et al. 2015] Hidden technical debt in machine learning systems. https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems
Production ML debt often accumulates in glue code, configuration, undeclared consumers, and changing external dependencies rather than in the model alone.
[Sculley et al. 2015] Hidden technical debt in machine learning systems. https://proceedings.neurips.cc/paper_files/paper/2015/hash/86df7dcfd896fcaf2674f757a2463eba-Abstract.html
Production ML debt often accumulates in glue code, configuration, undeclared consumers, and changing external dependencies rather than in the model alone.
[Seamless Communication 2023] SeamlessM4T: Massively multilingual & multimodal machine translation. arXiv preprint arXiv:2308.11596. https://arxiv.org/abs/2308.11596
SeamlessM4T supports ASR plus speech and text translation tasks across up to 100 languages, with language coverage varying by task and direction.
[Seamless Communication 2023] Seamless: Multilingual expressive and streaming speech translation. arXiv preprint arXiv:2312.05187. https://arxiv.org/abs/2312.05187
Seamless introduces SeamlessM4T v2, SeamlessExpressive, and SeamlessStreaming: a family of multilingual models enabling real-time expressive speech-to-speech translation with prosody and vocal style preservation.
[Semiconductor Industry Association and Boston Consulting Group 2021] Strengthening the global semiconductor supply chain in an uncertain era. https://www.semiconductors.org/wp-content/uploads/2021/03/BCG_Strengthening-the-Global-Semiconductor-Supply-Chain_Antonio-Varas-and-Raj-Varadarajan.pdf
The report separates design, EDA, equipment, materials, fabrication, and assembly, packaging, and test, and documents their geographic specialization.
[Sennrich et al. 2016] Neural machine translation of rare words with subword units. Association for Computational Linguistics. https://arxiv.org/abs/1508.07909
This paper proposes using BPE to segment rare and unknown words into subword units, enabling open-vocabulary neural machine translation without back-off dictionaries.
[Settles 2009] Active learning literature survey. https://minds.wisconsin.edu/handle/1793/60660
The survey organizes active-learning query strategies, assumptions, evaluation methods, and practical considerations for choosing which examples to label.
[Seung et al. 1992] Statistical mechanics of learning from examples. Physical Review A.
Seung, Sompolinsky, and Tishby apply statistical mechanics to learning, deriving generalization error and learning curves as a function of training-set size.
[Shah et al. 2024] FlashAttention-3: Fast and accurate attention with asynchrony and low-precision. https://proceedings.neurips.cc/paper_files/paper/2024/hash/7ede97c3e082c6df10a8d6103a2eebd2-Abstract-Conference.html
FlashAttention-3 uses Hopper asynchrony, warp specialization, and an FP8 path; the final paper reports 1.5-2x over FlashAttention-2, up to 840 TFLOP/s in BF16 and 1.3 PFLOP/s in FP8 on the evaluated H100 configurations.
[Shannon 1948] A mathematical theory of communication. Bell System Technical Journal.
Shannon founds information theory, defining entropy as the measure of information and establishing the limits of compression and reliable communication over noisy channels.
[Shao et al. 2024] DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. https://arxiv.org/abs/2402.03300
DeepSeekMath combines a curated 120B-token math corpus with GRPO, a PPO variant that removes the critic and normalizes rewards within sampled groups.
[Shao et al. 2026] Spurious rewards: Rethinking training signals in RLVR. https://arxiv.org/abs/2506.10947
Spurious Rewards finds that random or incorrect rewards recover much of the ground-truth RLVR gain on Qwen2.5-Math-7B, while often failing on Llama and OLMo models, making the effect strongly base-model dependent.
[Sharkey and Telser 1978] Supportable cost functions for the multiproduct firm. Journal of Economic Theory. https://doi.org/10.1016/0022-0531(78)90040-6
Sharkey and Telser formalize cost supportability and subadditivity for multiproduct firms, providing the cost condition behind a natural-monopoly claim.
[Sharma and Kaplan 2022] Scaling laws from the data manifold dimension. Journal of Machine Learning Research. https://arxiv.org/abs/2004.10802
Derives the scaling exponent from the intrinsic dimension of the data manifold, predicting that loss falls as a power law whose rate is set by how many dimensions the data effectively occupies.
[Sharma et al. 2024] Towards understanding sycophancy in language models. https://arxiv.org/abs/2310.13548
The study finds that responses matching a user's stated views are more likely to be preferred and that human and learned preference judgments can favor convincing sycophancy over correctness.
[Sharma et al. 2025] Constitutional classifiers: Defending against universal jailbreaks across thousands of hours of red teaming. arXiv preprint arXiv:2501.18837. https://arxiv.org/abs/2501.18837
Constitutional Classifiers train LLM safeguard classifiers on constitution-guided synthetic data to block universal jailbreaks, achieving over 95% jailbreak rejection with only 0.38% increase in production refusals.
[Shazeer et al. 2017] Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. https://arxiv.org/abs/1701.06538
This paper introduces a Sparsely-Gated MoE layer with up to thousands of feed-forward experts and a trainable gating network, achieving over 1000x model capacity gains with minor computational overhead on language modeling and translation tasks.
[Shazeer 2019] Fast transformer decoding: One write-head is all you need. https://arxiv.org/abs/1911.02150
Multi-query attention (MQA) shares keys and values across all attention heads, cutting memory bandwidth for incremental decoding with only minor quality loss.
[Shazeer 2020] GLU variants improve transformer. https://arxiv.org/abs/2002.05202
This paper proposes GLU variants such as SwiGLU and GEGLU as replacements for the ReLU activation in the Transformer FFN sublayer, finding they improve perplexity and downstream task quality.
[Shehabi et al. 2024] 2024 united states data center energy usage report. https://escholarship.org/uc/item/32d6m0d1
The report estimates U.S. datacenter electricity use and projects future demand under explicit assumptions about equipment, utilization, and facility efficiency.
[Shenfeld et al. 2025] RL's razor: Why online reinforcement learning forgets less. arXiv preprint arXiv:2509.04259. https://arxiv.org/abs/2509.04259
Matched supervised and on-policy reinforcement-learning fine-tunes show that forgetting of prior tasks tracks the Kullback-Leibler divergence between the fine-tuned and base policy on the new task, and that on-policy updates are biased toward the divergence-nearest solution.
[Sheng et al. 2023] S-LoRA: Serving thousands of concurrent LoRA adapters. arXiv preprint arXiv:2311.03285. https://arxiv.org/abs/2311.03285
S-LoRA serves many LoRA adapters over shared base-model weights using unified paging and scheduling for adapter and KV-cache memory.
[Sheng et al. 2025] HybridFlow: a flexible and efficient RLHF framework. Association for Computing Machinery. https://arxiv.org/abs/2409.19256
HybridFlow models distributed RLHF as a dataflow and introduces a hybrid controller and actor resharding between generation and training layouts.
[Shi et al. 2017] World of bits: An open-domain platform for web-based agents. https://proceedings.mlr.press/v70/shi17a.html
The origin of GUI agents as a research line: agents perceive pixels and DOM and act with mouse and keyboard, plus the MiniWoB task suite the field trained on for years.
[Shi et al. 2023] Large language models can be easily distracted by irrelevant context. https://proceedings.mlr.press/v202/shi23a.html
Adding irrelevant sentences to grade-school arithmetic problems reduced accuracy for evaluated language models, making distractor robustness a separate property from context capacity.
[Shi et al. 2024] Simplified and generalized masked diffusion for discrete data. https://arxiv.org/abs/2406.04329
MD4 simplifies masked diffusion for discrete data by showing the continuous-time ELBO reduces to a weighted integral of cross-entropy losses, improving perplexity and image modeling over prior discrete diffusion models.
[Shi et al. 2024] Optimization-based prompt injection attack to LLM-as-a-judge. https://arxiv.org/abs/2403.17710
JudgeDeceiver demonstrates that attacker-controlled candidate text can inject instructions into an LLM judge and manipulate selection in RLAIF and other judge-based systems.
[Shi et al. 2024] Detecting pretraining data from large language models. https://proceedings.iclr.cc/paper_files/paper/2024/hash/e32ad85fa27be4a9868d55703f01323e-Abstract-Conference.html
The paper formalizes black-box pretraining-data detection and evaluates Min-K
[Shinn et al. 2023] Reflexion: language agents with verbal reinforcement learning. https://proceedings.neurips.cc/paper_files/paper/2023/hash/1b44b878bb782e6954cd888628510e90-Abstract-Conference.html
Reflexion turns task feedback into textual reflection kept in episodic memory, changing later decisions through context rather than parameter updates.
[Shmueli et al. 2021] Beyond fair pay: Ethical implications of NLP crowdsourcing. Association for Computational Linguistics. https://aclanthology.org/2021.naacl-main.295/
The authors show that crowdwork ethics extends beyond hourly pay to privacy, identifiable data, harmful exposure, consent, and the limits of existing human-subject review frameworks.
[Shoeybi et al. 2019] Megatron-LM: Training multi-billion parameter language models using model parallelism. https://arxiv.org/abs/1909.08053
Megatron-LM presents an intra-layer tensor-parallel formulation for Transformer training and measures its communication and scaling behavior.
[Shokri et al. 2017] Membership inference attacks against machine learning models. https://arxiv.org/abs/1610.05820
This paper introduces membership inference attacks that use shadow training to determine, via black-box API access, whether a record was in a model's training dataset.
[Shumailov et al. 2024] AI models collapse when trained on recursively generated data. Nature. https://www.nature.com/articles/s41586-024-07566-y
The paper analyzes recursive model-data feedback and shows early tail loss; its language-model experiment uses OPT-125M, WikiText-2, and generated continuations.
[Si and others 2025] Collective communication for 100k+ gpus. arXiv preprint arXiv:2510.20171. https://arxiv.org/abs/2510.20171
NCCLX is Meta's collective communication framework for training and inference on a multi-building RoCE fabric exceeding 100,000 GPUs.
[Singh and Strouse 2024] Tokenization counts: The impact of tokenization on arithmetic in frontier llms. https://arxiv.org/abs/2402.14903
Experiments on frontier models show that the direction and grouping of number tokens change arithmetic accuracy and error structure.
[Singh et al. 2025] Efficiently serving large multimodal models using EPD disaggregation. PMLR. https://proceedings.mlr.press/v267/singh25d.html
EPD serving separates multimodal encoding, language-model prefill, and decoding onto independently scaled resources to improve goodput.
[Sinha et al. 2026] The illusion of diminishing returns: Measuring long horizon execution in llms. https://arxiv.org/abs/2509.09677
Controlled multi-turn experiments distinguish independent compounding from history-dependent self-conditioning and show that the tested thinking variants mitigate the latter.
[SK hynix 2025] <span class="nocase">SK hynix</span> completes world's first HBM4 development and readies mass production. https://news.skhynix.com/sk-hynix-completes-worlds-first-hbm4-development-and-readies-mass-production/
SK hynix documents its HBM4 implementation, including a doubled interface width; the figures remain specific to that product.
[SLSA Community 2025] SLSA v1.2 build provenance. https://slsa.dev/spec/v1.2/build-provenance
SLSA build provenance binds authenticated claims about a builder, build definition, resolved inputs, and run details to output artifact digests; consumers must verify those claims against expectations.
[SLSA Community 2025] SLSA specification. https://slsa.dev/spec/v1.2/
SLSA defines provenance requirements and assurance levels for software build and distribution pipelines.
[Snell et al. 2025] Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. https://arxiv.org/abs/2408.03314
On MATH with the studied PaLM 2 models, process reward model, and offline difficulty estimates, the best test-time strategy depends on problem difficulty and budget.
[Sohl-Dickstein et al. 2015] Deep unsupervised learning using nonequilibrium thermodynamics. PMLR. https://proceedings.mlr.press/v37/sohl-dickstein15.html
This paper introduces diffusion probabilistic models, which learn to reverse an iterative forward diffusion process that gradually destroys data structure, yielding a flexible and tractable generative model.
[Soldaini et al. 2024] Dolma: an open corpus of three trillion tokens for language model pretraining research. Association for Computational Linguistics. https://aclanthology.org/2024.acl-long.840/
Dolma is an open, three-trillion-token English pretraining corpus built from web content, scientific papers, code, books, social media, and encyclopedic materials, released with full documentation and a data curation toolkit.
[Song and Ermon 2019] Generative modeling by estimating gradients of the data distribution. https://arxiv.org/abs/1907.05600
Song and Ermon propose Noise Conditional Score Networks (NCSN), a generative model that estimates data distribution gradients at multiple noise levels and samples via annealed Langevin dynamics, achieving state-of-the-art inception score 8.87 on CIFAR-10.
[Song et al. 2021] Score-based generative modeling through stochastic differential equations. https://arxiv.org/abs/2011.13456
This paper unifies score-based generative models and DDPM under a continuous-time SDE framework, enabling exact likelihood computation, controllable generation, and state-of-the-art image synthesis on CIFAR-10.
[Song et al. 2021] Denoising diffusion implicit models. https://arxiv.org/abs/2010.02502
DDIM constructs non-Markovian processes with the DDPM training objective and reports 10x to 50x lower wall-clock sampling time without retraining.
[Song et al. 2023] Consistency models. PMLR. https://proceedings.mlr.press/v202/song23a.html
Models that map any point on the denoising trajectory directly to its origin, enabling one-step generation with optional multi-step refinement, trainable by distillation or from scratch.
[Song et al. 2025] Seed diffusion: a large-scale diffusion language model with high-speed inference. arXiv preprint arXiv:2508.02193. https://arxiv.org/abs/2508.02193
Seed Diffusion combines blockwise generation with KV caching and reports 2,146 tokens per second on H20 hardware, with hardware count and model size left unspecified.
[Souly et al. 2024] A StrongREJECT for empty jailbreaks. arXiv preprint arXiv:2402.10260. https://arxiv.org/abs/2402.10260
StrongREJECT evaluates whether a jailbreak response supplies useful prohibited information and shows that common evaluators can overstate attack success.
[Souly et al. 2025] Poisoning attacks on LLMs require a near-constant number of poison samples. CoRR. https://arxiv.org/abs/2510.07192
Across the tested scales, 250 poisoned documents produced similar success for a narrow trigger-to-gibberish objective; the result is conditional on that attack and training setup.
[Souppaya et al. 2017] Application container security guide. https://csrc.nist.gov/pubs/sp/800/190/final
NIST SP 800-190 describes container-specific image, registry, orchestrator, runtime, host, and network risks and countermeasures.
[Souppaya et al. 2017] Application container security guide. https://doi.org/10.6028/NIST.SP.800-190
NIST describes container-specific threats and recommends layered controls across images, registries, orchestrators, hosts, and runtimes.
[Spangher et al. 2025] RLHF algorithms ranked: An extensive evaluation across diverse tasks, rewards, and hyperparameters. Association for Computational Linguistics. https://aclanthology.org/2025.emnlp-industry.35/
A broad empirical comparison finds that alignment-method rankings depend on task, reward, and hyperparameter choices rather than yielding one universal winner.
[SPIFFE Project 2026] SPIFFE identity and verifiable identity document. https://spiffe.io/docs/latest/spiffe-specs/spiffe-id/
SPIFFE identifies and authenticates workloads; applications still need separate user delegation and authorization policy.
[Sporny et al. 2025] Verifiable credentials data model v2.0. https://www.w3.org/TR/vc-data-model-2.0/
The data model standardizes credential structure and verification relationships while explicitly leaving claim truth and reliance decisions to verifier policy.
[Squire 2004] Memory systems of the brain: a brief history and current perspective. Neurobiology of Learning and Memory. https://pubmed.ncbi.nlm.nih.gov/15464402/
This review distinguishes multiple biological memory systems and clarifies the declarative versus nondeclarative organization behind common cognitive labels.
[Staab et al. 2024] Beyond memorization: Violating privacy via inference with large language models. https://proceedings.iclr.cc/paper_files/paper/2024/hash/9028b8a3ca98f58e373f0c1497a17448-Abstract-Conference.html
LLMs infer location, income, and demographics from ordinary text at high accuracy for a fraction of a human profiler's cost; a memory store persists exactly that inference, run continuously.
[Stanovich and West 2000] Individual differences in reasoning: Implications for the rationality debate?. Behavioral and Brain Sciences.
Stanovich and West introduce the System 1 / System 2 terminology for dual-process reasoning and analyze individual differences in human rationality.
[Starace et al. 2025] PaperBench: Evaluating AI's ability to replicate AI research. arXiv preprint arXiv:2504.01848. https://arxiv.org/abs/2504.01848
PaperBench decomposes long research-replication tasks into hierarchical rubric items and evaluates the automated judge on a separate judge benchmark.
[Stiennon et al. 2020] Learning to summarize from human feedback. https://arxiv.org/abs/2009.01325
This work trains summarization reward models from pairwise human judgments, optimizes policies with PPO, and directly measures the point where learned reward and human preference diverge.
[Stivers et al. 2009] Universals and cultural variation in turn-taking in conversation. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.0903616106
Across ten languages, responses to yes-no questions clustered near the end of the preceding turn but varied substantially by language and response type.
[Su et al. 2023] One embedder, any task: Instruction-finetuned text embeddings. https://aclanthology.org/2023.findings-acl.71/
INSTRUCTOR trains one text encoder on task and domain descriptions so the same input can receive a task-conditioned representation without per-task fine-tuning.
[Su et al. 2024] RoFormer: Enhanced transformer with rotary position embedding. Neurocomputing. https://arxiv.org/abs/2104.09864
RoFormer introduces RoPE, which encodes token positions as rotation matrices in self-attention, giving sequence-length flexibility and decaying inter-token dependency with distance.
[Subramanya et al. 2019] DiskANN: Fast accurate billion-point nearest neighbor search on a single node. https://proceedings.neurips.cc/paper_files/paper/2019/hash/09853c7fb1d3f8ee67a61b6bf4a7f8e6-Abstract.html
Describes an SSD-aware graph index for high-recall billion-point nearest-neighbor search.
[Sun 2025] Why did M2 end up as a full attention model?. https://www.minimax.io/news/why-did-m2-end-up-as-a-full-attention-model
MiniMax's pre-training lead explains why M2 dropped the hybrid lightning-attention design: hybrid deficits surfaced only at scale on multi-hop reasoning, and the inference and evaluation stack around efficient attention is not yet production-mature.
[Sun et al. 2026] When personalization misleads: Understanding and mitigating hallucinations in personalized llms. https://aclanthology.org/2026.findings-acl.395/
The study measures when user profiles bias factual answers away from evidence and toward personalized but incorrect conclusions.
[Sutton 1988] Learning to predict by the methods of temporal differences. Machine Learning.
Sutton introduces temporal-difference learning, which updates predictions from the difference between successive estimates rather than waiting for the final outcome.
[Sutton and Barto 2018] Reinforcement learning: An introduction. The MIT Press. https://mitpress.mit.edu/9780262039246/reinforcement-learning/
Sutton and Barto formalize an agent as a policy interacting with an environment over time and develop return-based evaluation for episodic and continuing tasks.
[Swanson et al. 2010] Contingency planning guide for federal information systems. https://doi.org/10.6028/NIST.SP.800-34r1
NIST explains contingency planning, recovery priorities, alternate processing, testing, and restoration objectives for information systems.
[Szabo 1999] Micropayments and mental transaction costs. https://nakamotoinstitute.org/library/micropayments-and-mental-transaction-costs/
Szabo argues that attention, preference specification, monitoring, and dispute costs can dominate the price of a tiny purchase, including when software agents make it.
[Tabassi 2023] Artificial intelligence risk management framework (AI RMF 1.0). https://doi.org/10.6028/NIST.AI.100-1
NIST AI RMF 1.0 organizes AI risk management into govern, map, measure, and manage functions across the AI lifecycle.
[Taiwan Semiconductor Manufacturing Company 2025] TSMC unveils next-generation A14 process at north america technology symposium. https://pr.tsmc.com/english/news/3228
TSMC's 2025 symposium announcement names both N12 and N3 logic base-die options for HBM4, showing that the base-die process is product-specific.
[Taiwan Semiconductor Manufacturing Company 2026] TSMC debuts A13 technology at 2026 north america technology symposium. https://pr.tsmc.com/english/news/3302
TSMC distinguishes 5.5-reticle CoWoS in production from a 14-reticle version planned for 2028 and a further expansion planned for 2029.
[Taiwan Semiconductor Manufacturing Company 2026] Fab capacity. https://www.tsmc.com/english/dedicatedFoundry/manufacturing/fab_capacity
TSMC lists fabrication facilities in Taiwan and subsidiaries or facilities in the United States, Japan, and China, while distinguishing their wafer sizes and corporate status.
[Tang et al. 2024] QUEST: Query-aware sparsity for efficient long-context LLM inference. PMLR. https://proceedings.mlr.press/v235/tang24l.html
Quest stores per-page key extrema and uses each query to select the top-K KV pages read by attention while retaining the full cache.
[Tao et al. 2024] Scaling laws with vocabulary: Larger models deserve larger vocabularies. https://proceedings.neurips.cc/paper_files/paper/2024/hash/cf5a019ae9c11b4be88213ce3f85d85c-Abstract-Conference.html
Controlled experiments model the trade-off between vocabulary-dependent parameters and text compression, finding that compute-optimal vocabulary size grows with model scale.
[Tazi et al. 2025] The ultra-scale playbook: Training llms on GPU clusters. https://huggingface.co/spaces/nanotron/ultrascale-playbook
A Hugging Face Nanotron guide focused specifically on training large language models on large GPU clusters, with an accompanying PDF and interactive web version.
[Templeton et al. 2024] Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet. https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html
The study trains sparse autoencoders on Claude 3 Sonnet and analyzes extracted features, feature steering, and the limits of feature interpretation.
[Temporal Technologies 2026] Activity definition. https://docs.temporal.io/activity-definition
Temporal records completed activities for replay, but an activity that fails to report completion may be retried and execute more than once.
[Temporal Technologies 2026] Cancellation: Python SDK. https://docs.temporal.io/develop/python/workflows/cancellation
Temporal distinguishes graceful workflow cancellation from forceful termination and documents the heartbeat requirement for cancelling remote activities.
[Temporal Technologies 2026] What is a Temporal retry policy?. https://docs.temporal.io/encyclopedia/retry-policies
Temporal documents declarative retry policy, bounded attempts and timeouts, and the separation of retryable transient failures from permanent failures.
[Temporal Technologies 2026] Temporal workflow. https://docs.temporal.io/workflows
Temporal separates deterministic workflow replay from external activities and persists an event history for durable recovery.
[TensorFlow Authors 2026] Introduction to gradients and automatic differentiation. https://www.tensorflow.org/guide/autodiff
TensorFlow documentation explains gradient tapes, watched variables, higher-order derivatives, Jacobians, and cases where gradients are unavailable.
[TensorFlow Authors 2026] Better performance with <span class="nocase">tf.function</span>. https://www.tensorflow.org/guide/function
The guide explains tracing, ConcreteFunction caches, input specialization, retracing, and trace-time Python effects.
[Thakur et al. 2021] BEIR: a heterogeneous benchmark for zero-shot evaluation of information retrieval models. https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/hash/65b9eea6e1cc6bb9f0cd2a47751a186f-Abstract-round2.html
Evaluates retrieval methods across heterogeneous datasets and exposes substantial domain variation.
[Theano Development Team 2016] Theano: a python framework for fast computation of mathematical expressions. https://arxiv.org/abs/1605.02688
Theano represents mathematical expressions as graphs, differentiates them symbolically, and compiles optimized CPU or GPU functions.
[Thorne et al. 2018] FEVER: a large-scale dataset for fact extraction and VERification. Association for Computational Linguistics. https://aclanthology.org/N18-1074/
FEVER couples claims with evidence from a fixed Wikipedia snapshot and labels them Supported, Refuted, or NotEnoughInfo, keeping missing evidence distinct from falsehood.
[Thurgood and Ferguson 2018] Implementing slos. O'Reilly Media. https://sre.google/workbook/implementing-slos/
Google's SRE workbook turns service-level indicators and objectives into an implementation process tied to user journeys, measurement, and review.
[Thurgood 2018] Alerting on slos. O'Reilly Media. https://sre.google/workbook/alerting-on-slos/
Google's SRE workbook derives alerts from error-budget burn rates so paging reflects both the size and duration of an SLO threat.
[Tian et al. 2024] Visual autoregressive modeling: Scalable image generation via next-scale prediction. https://arxiv.org/abs/2404.02905
VAR redefines image autoregression as coarse-to-fine next-scale prediction and reports stronger class-conditional ImageNet results than the tested diffusion-transformer baselines.
[Tillet et al. 2019] Triton: An intermediate language and compiler for tiled neural network computations. https://dl.acm.org/doi/10.1145/3315508.3329973
Triton makes the statically-shaped tile the unit of GPU programming: the programmer writes one program per tile while the compiler handles coalescing, shared memory, and intra-processor scheduling.
[Tobin et al. 2017] Domain randomization for transferring deep neural networks from simulation to the real world. https://arxiv.org/abs/1703.06907
Domain randomization trains on varied simulated renderings to reduce reliance on any one simulator appearance before transfer to real images.
[Torres-Arias et al. 2019] in-toto: Providing farm-to-table guarantees for bits and bytes. https://www.usenix.org/conference/usenixsecurity19/presentation/torres-arias
in-toto records and verifies the authorized steps, actors, and artifacts in a software supply chain.
[Touvron et al. 2023] Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. https://arxiv.org/abs/2307.09288
Llama 2 describes preference collection across model variants and decoding temperatures, including repeated collection as newer policies shift the response distribution.
[Trail of Bits 2024] A few notes on AWS nitro enclaves: Images and attestation. https://blog.trailofbits.com/2024/02/16/a-few-notes-on-aws-nitro-enclaves-images-and-attestation/
Examines Nitro Enclave image reproducibility, PCR interpretation, attestation, and centralized trust assumptions from a practitioner's perspective.
[Triton Project 2026] Triton programming guide: Introduction. https://triton-lang.org/main/programming-guide/chapter-1/introduction.html
The guide defines Triton's blocked-program model and the compiler responsibilities for locality, scheduling, coalescing, vectorization, and target instruction selection.
[Trivedi et al. 2023] Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. https://aclanthology.org/2023.acl-long.557/
IRCoT interleaves retrieval with intermediate reasoning so later searches can depend on evidence found by earlier searches.
[TSMC 2026] CoWoS: Chip-on-wafer-on-substrate. https://www.tsmc.com/english/dedicatedFoundry/technology/cowos
TSMC describes how CoWoS integrates logic chiplets and high-bandwidth memory through an interposer and package substrate.
[Tung et al. 2026] The anatomy of silent data corruption: GPU error pattern study and modeling guidance. https://arxiv.org/abs/2605.04213
A gate-level fault-injection campaign found structured corruption outcomes that are poorly represented by NaN-only checks or uniform single-bit models.
[Turpin et al. 2023] Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting. https://proceedings.neurips.cc/paper_files/paper/2023/hash/ed3fea9033a80fea1376299fa7863f4a-Abstract.html
The experiments alter features that bias an answer and find that chain-of-thought explanations often omit those causal influences.
[U.S. Bureau of Industry and Security 2026] Revision to license review policy for advanced computing commodities. https://www.federalregister.gov/documents/2026/01/15/2026-00789/revision-to-license-review-policy-for-advanced-computing-commodities
For eligible exports from the United States to end users in China or Macau, BIS changes the review policy for applications covering commodities below both stated thresholds from a presumption of denial to case-by-case review. Conditions include certifications, independent testing, and a cap under which cumulative China and Macau total processing performance for the same commodities cannot exceed 50 percent of cumulative shipments for U.S. end use.
[U.S. Department of Commerce 2025] U.S. department of commerce announces $1.4 billion in final awards to support the next generation of U.S. semiconductor advanced packaging. https://www.commerce.gov/node/7087
Commerce announced awards for advanced-packaging research and piloting, including substrate programs and a facility intended to bridge laboratory work and full-scale production.
[U.S. Department of Energy 2024] Large power transformer resilience. https://www.energy.gov/sites/default/files/2024-10/EXEC-2022-001242%20-%20Large%20Power%20Transformer%20Resilience%20Report%207-10-24.pdf
The U.S. Department of Energy assesses large power-transformer supply, manufacturing, transport, replacement, and resilience risks.
[U.S. Department of Energy 2025] Energy department closes loan to restart nuclear power plant in pennsylvania. https://www.energy.gov/articles/energy-department-closes-loan-restart-nuclear-power-plant-pennsylvania
The Department of Energy records the federal loan closing and stated conditions for restarting the former Three Mile Island Unit 1 reactor.
[U.S. Department of Justice and Federal Trade Commission 2023] 2023 merger guidelines. https://www.justice.gov/atr/merger-guidelines
The guidelines define markets before calculating shares, permit metrics suited to competitive reality, and use HHI thresholds as structural presumptions within U.S. merger review.
[U.S. Nuclear Regulatory Commission 2026] Christopher M. Crane clean energy center. https://www.nrc.gov/info-finder/reactors/ccec
The Nuclear Regulatory Commission page records licensing and regulatory information for the planned restart of the Christopher M. Crane Clean Energy Center.
[UALink Consortium 2025] UALink 200G 1.0 specification. https://ualinkconsortium.org/specification/
UALink 1.0 specifies a switched accelerator scale-up interface while leaving implementation and deployment evidence to vendors.
[Uesato et al. 2022] Solving math word problems with process- and outcome-based feedback. arXiv preprint arXiv:2211.14275. https://arxiv.org/abs/2211.14275
This paper compares process-supervised and outcome-supervised reward models for math reasoning on GSM8K, finding that a PRM reduces reasoning trace error from 14.0% to 3.4%.
[United States District Court for the District of Delaware 2025] Thomson reuters enterprise centre GmbH v. ross intelligence inc., no. 1:20-cv-613, memorandum opinion. https://www.ded.uscourts.gov/sites/ded/files/opinions/20-613_5.pdf
The Ross Intelligence opinion applies U.S. copyright fair-use analysis to copied legal headnotes and the creation of a competing research product.
[United States District Court for the Northern District of California 2025] Kadrey v. meta platforms, inc., order on cross-motions for summary judgment. https://caselaw.findlaw.com/court/us-dis-crt-n-d-cal/117422847.html
The Kadrey order evaluates fair use on the evidence presented and explains why market harm and the plaintiffs' proof matter to the result.
[Universal Commerce Protocol 2026] Universal commerce protocol specification overview. https://ucp.dev/2026-04-08/specification/overview/
UCP publishes merchant capabilities and coordinates a broad commerce lifecycle, with optional composition rather than a required dependency on one identity or payment protocol.
[Van Bulck et al. 2018] Foreshadow: Extracting the keys to the intel SGX kingdom with transient out-of-order execution. https://www.usenix.org/conference/usenixsecurity18/presentation/bulck
The transient-execution attack that extracted SGX's own attestation keys, resetting the field's expectations: a trusted execution boundary is an engineering artifact, not a proof.
[Vaswani et al. 2017] Attention is all you need. https://arxiv.org/abs/1706.03762
The Transformer replaces recurrence with multi-head attention and position-wise feed-forward blocks, enabling substantially more parallel sequence training.
[Villalobos et al. 2024] Position: Will we run out of data? Limits of LLM scaling based on human-generated data. PMLR. https://proceedings.mlr.press/v235/villalobos24a.html
The paper estimates the effective public-human-text stock and forecasts when a single training dataset could become comparable in size, conditional on continued dataset-growth trends.
[Vincent 2011] A connection between score matching and denoising autoencoders. Neural Computation 23(7):1661–1674. https://direct.mit.edu/neco/article-abstract/23/7/1661/7677
Vincent proves a connection between denoising autoencoder training and score matching, linking reconstruction under noise to density-gradient estimation.
[Visa 2026] Trusted agent protocol specifications. https://github.com/visa/trusted-agent-protocol
TAP defines signed agent-recognition, consumer, and payment-container data within Visa's scheme; signature validity is not universal proof of authority or intent.
[vLLM 2026] Batch invariance. https://docs.vllm.ai/en/latest/features/batch_invariance/
vLLM documents a serving mode whose supported kernels make outputs invariant to batch composition under a pinned environment.
[vLLM Multimodal Workstream 2025] Encoder disaggregation for scalable multimodal model serving. https://vllm.ai/blog/2025-12-15-vllm-epd
vLLM's native encode-prefill-decode disaggregation, available since v0.11.1, independently scales a vision-encoder pool and reports about 2–2.5x goodput in its four-A100 Qwen3-VL benchmarks.
[vLLM Project 2026] Encoder cache manager. https://docs.vllm.ai/en/stable/api/vllm/v1/core/encoder_cache_manager/
vLLM's encoder-cache manager tracks multimodal encoder outputs, capacity, allocation, and eviction separately from the language model's KV cache.
[Wallace et al. 2024] The instruction hierarchy: Training llms to prioritize privileged instructions. arXiv preprint arXiv:2404.13208. https://arxiv.org/abs/2404.13208
Instruction hierarchy trains models to selectively ignore lower-privileged conflicting instructions, improving robustness to prompt injections and jailbreaks with minimal capability degradation.
[Wan et al. 2024] AlphaZero-like tree-search can guide large language model decoding and training. PMLR. https://arxiv.org/abs/2309.17179
TS-LLM applies AlphaZero-like tree search with a learned value function to guide LLM decoding and iterative training across reasoning, planning, RLHF alignment, and decision-making tasks.
[Wan and others 2025] Robust LLM training infrastructure at ByteDance. https://arxiv.org/abs/2509.16293
ByteRobust combines live checks, stop-time diagnosis, warm standbys, and in-memory peer-backed checkpoints for large production training jobs.
[Wan et al. 2025] The 2025 foundation model transparency index. https://arxiv.org/abs/2512.10169
The 2025 index reports a mean developer-transparency score of 40.69 while warning that changed indicators and samples limit direct comparison with its 2024 result.
[Wang et al. 2019] Improving pre-trained multilingual model with vocabulary expansion. Association for Computational Linguistics. https://aclanthology.org/K19-1030/
The paper expands a pretrained multilingual model's vocabulary and adapts the added token parameters rather than retraining the model from scratch.
[Wang et al. 2019] Truly proximal policy optimization. arXiv preprint arXiv:1903.07940. https://arxiv.org/abs/1903.07940
The analysis shows that PPO's clipped surrogate does not strictly bound likelihood ratios or enforce a trust region, motivating a rollback-based variant.
[Wang et al. 2020] Neural machine translation with byte-level subwords. https://ojs.aaai.org/index.php/AAAI/article/view/6451
The paper trains subword vocabularies over UTF-8 bytes, guaranteeing coverage without unknown characters while retaining learned multi-byte pieces.
[Wang and Isola 2020] Understanding contrastive representation learning through alignment and uniformity on the hypersphere. https://proceedings.mlr.press/v119/wang20k.html
Wang and Isola formalize alignment of positive pairs and uniformity on the hypersphere as two useful properties of normalized contrastive representations under stated sampling assumptions.
[Wang et al. 2022] Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533. https://arxiv.org/abs/2212.03533
E5 trains general-purpose text embeddings via contrastive pre-training on CCPairs, a curated 270M web-scale text pair dataset, achieving state-of-the-art results on BEIR and MTEB benchmarks.
[Wang et al. 2023] Neural codec language models are zero-shot text to speech synthesizers. arXiv preprint arXiv:2301.02111. https://arxiv.org/abs/2301.02111
VALL-E treats TTS as conditional codec language modeling, training on 60K hours of speech to enable zero-shot personalized speech synthesis from a 3-second acoustic prompt.
[Wang et al. 2023] HelpSteer: Multi-attribute helpfulness dataset for SteerLM. arXiv preprint arXiv:2311.09528. https://arxiv.org/abs/2311.09528
HelpSteer provides 37,000 response-level ratings for overall helpfulness plus correctness, coherence, complexity, and verbosity, making trade-offs visible for training and audit.
[Wang et al. 2023] Self-instruct: Aligning language models with self-generated instructions. Association for Computational Linguistics. https://aclanthology.org/2023.acl-long.754/
Self-Instruct bootstraps instruction data by generating, filtering, and answering new tasks with a language model before fine-tuning on the retained examples.
[Wang et al. 2023] Self-consistency improves chain of thought reasoning in language models. https://openreview.net/forum?id=1PL1NIMMrw
Self-consistency samples multiple reasoning paths and selects the most consistent final answer, improving results when probability mass concentrates on the correct answer.
[Wang et al. 2023] Gemini: Fast failure recovery in distributed training with in-memory checkpoints. https://doi.org/10.1145/3600006.3613145
Gemini places checkpoint replicas in host memory and schedules their traffic to shorten recovery without assuming that one storage tier covers every failure domain.
[Wang et al. 2024] Auxiliary-loss-free load balancing strategy for mixture-of-experts. https://arxiv.org/abs/2408.15664
Loss-Free Balancing maintains balanced expert load in MoE models by dynamically updating per-expert routing biases, eliminating auxiliary-loss interference gradients and improving model performance.
[Wang et al. 2024] Qwen2-VL: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191. https://arxiv.org/abs/2409.12191
Qwen2-VL introduces dynamic-resolution visual sequences and multimodal rotary position encoding across text, image, and video axes.
[Wang and others 2024] Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869. https://arxiv.org/abs/2409.18869
Emu3 trains a single 8B-parameter Transformer on discrete image, video, and text tokens using only next-token prediction, matching or surpassing SDXL and LLaVA-1.6 without diffusion or CLIP.
[Wang et al. 2024] Executable code actions elicit better LLM agents. PMLR. https://proceedings.mlr.press/v235/wang24h.html
CodeAct uses executable Python as an action representation and reports up to 20 absolute percentage points higher success than compared formats across its evaluated models and benchmarks.
[Wang et al. 2024] Improving text embeddings with large language models. https://aclanthology.org/2024.acl-long.642/
This work generates diverse multilingual embedding tasks and pairs with an LLM, then contrastively adapts a decoder model using synthetic data alone or mixed with labeled data.
[Wang et al. 2024] Large language models are not fair evaluators. Association for Computational Linguistics. https://aclanthology.org/2024.acl-long.511/
The authors demonstrate position-sensitive verdicts in LLM evaluation and test a calibration framework, showing why response-order invariance must be measured rather than prompted.
[Wang et al. 2025] RAGEN: Understanding self-evolution in LLM agents via multi-turn reinforcement learning. arXiv preprint arXiv:2504.20073. https://arxiv.org/abs/2504.20073
RAGEN studies trajectory-level multi-turn RL in stylized environments and reports a collapse pattern involving reward-variance cliffs, entropy loss, and gradient spikes.
[Wang et al. 2026] Agent world model: Infinity synthetic environments for agentic reinforcement learning. arXiv preprint arXiv:2602.10090. https://arxiv.org/abs/2602.10090
Agent World Model generates 1,000 code-driven environments whose state lives in a database rather than in a simulating language model, and reports that training only in those synthetic environments generalizes to three held-out benchmarks.
[Wang et al. 2026] Do androids dream of breaking the game? Systematically auditing AI agent benchmarks with BenchJack. https://arxiv.org/abs/2605.12673
BenchJack audits ten agent benchmarks, finds 219 flaws across eight classes, and synthesizes reward-hacking exploits that reach near-perfect scores on most without solving the intended tasks.
[Wang et al. 2026] MCPTox: a benchmark for tool poisoning on real-world MCP servers. Proceedings of the AAAI Conference on Artificial Intelligence. https://ojs.aaai.org/index.php/AAAI/article/view/40895
MCPTox builds 1,348 controlled malicious-descriptor cases over 353 authentic tools from 45 real-world MCP server toolsets and evaluates 20 agent settings; it does not report 45 compromised deployments.
[Warner and Davidovič 2018] Canarying releases. O'Reilly Media. https://sre.google/workbook/canarying-releases/
The SRE workbook defines canarying as a partial, time-limited deployment evaluated against a control and details traffic, metric, isolation, and rollback requirements.
[Wasserstein and Lazar 2016] The ASA's statement on p-Values: Context, process, and purpose. The American Statistician. https://doi.org/10.1080/00031305.2016.1154108
The ASA statement explains that p-values do not give the probability that a hypothesis is true and do not measure effect size or practical importance.
[Wei et al. 2022] Emergent abilities of large language models. Transactions on Machine Learning Research. https://arxiv.org/abs/2206.07682
This paper defines and surveys emergent abilities of large language models: capabilities not observed in smaller models but observed in larger ones under the reported evaluations.
[Wei et al. 2022] Finetuned language models are zero-shot learners. https://arxiv.org/abs/2109.01652
FLAN reports that instruction tuning across many tasks improves zero-shot performance on held-out task types, with task diversity and natural-language instructions important in its ablations.
[Wei et al. 2022] Chain-of-thought prompting elicits reasoning in large language models. https://arxiv.org/abs/2201.11903
Chain-of-thought prompting, which adds intermediate reasoning steps as few-shot exemplars, significantly improves large language model performance on arithmetic, commonsense, and symbolic reasoning tasks.
[Wei et al. 2024] Long-form factuality in large language models. https://proceedings.neurips.cc/paper_files/paper/2024/hash/937ae0e83eb08d2cb8627fe1def8c751-Abstract-Conference.html
LongFact and SAFE test search-augmented verification of long-form responses; the reported human agreement, disagreement audit, and cost results are conditional on the paper's decomposition, evidence access, and 2024 evaluator setup.
[Wei et al. 2024] Measuring short-form factuality in large language models. https://cdn.openai.com/papers/simpleqa.pdf
SimpleQA uses 4,326 adversarially collected short questions intended to have one accepted answer and grades responses as correct, incorrect, or not attempted.
[Wei et al. 2025] SWE-RL: Advancing LLM reasoning via reinforcement learning on open software evolution. https://proceedings.neurips.cc/paper_files/paper/2025/hash/7107d4d2e837bde2171c6b71b5bde954-Abstract-Conference.html
SWE-RL applies reinforcement learning to software-evolution records using a reproducible patch-similarity reward, illustrating both the reach and the limitations of proxy rewards.
[Weinstein and Kaliski 2026] Introducing the machine payments protocol. https://stripe.com/blog/machine-payments-protocol
MPP separates HTTP payment negotiation from the underlying instrument and supports stablecoin, card, and other payment methods.
[Weiss et al. 2024] What was your prompt? A remote keylogging attack on AI assistants. https://www.usenix.org/conference/usenixsecurity24/presentation/weiss
Under the paper's experimental conditions, token-length patterns in encrypted streaming responses enabled exact reconstruction of 27% of replies and topic inference for 53%.
[Weller et al. 2025] FollowIR: Evaluating and teaching information retrieval models to follow instructions. https://aclanthology.org/2025.naacl-long.597/
FollowIR uses detailed TREC relevance instructions to test whether retrievers follow constraints rather than treating the added text as keywords, and supplies targeted training data.
[Wen et al. 2026] Reinforcement learning with verifiable rewards implicitly incentivizes correct reasoning in base llms. https://openreview.net/forum?id=jGbRWwIidy
This paper argues that ordinary Pass@K can credit correct answers with flawed reasoning and proposes CoT-Pass@K, requiring both reasoning path and final answer to be correct.
[Wengert 1964] A simple automatic derivative evaluation program. Communications of the ACM. https://doi.org/10.1145/355586.364791
Wengert decomposes a function into elementary steps with intermediate variables, establishing the evaluation-list idea used by tape-based differentiation.
[Wenzek et al. 2020] CCNet: Extracting high quality monolingual datasets from web crawl data. European Language Resources Association. https://aclanthology.org/2020.lrec-1.494/
CCNet is an automatic pipeline that extracts large, high-quality monolingual datasets from Common Crawl by deduplicating documents, identifying language, and filtering with a Wikipedia-based perplexity model.
[WHATWG n.d.] HTML living standard: Server-sent events. https://html.spec.whatwg.org/multipage/server-sent-events.html
The HTML standard defines the EventSource API, event-stream wire format, reconnection behavior, and parsing rules for server-sent events.
[White et al. 2024] The model openness framework: Promoting completeness and openness for reproducibility, transparency, and usability in artificial intelligence. https://arxiv.org/abs/2403.13784
The Model Openness Framework inventories released code, data, documentation, and model components; its classes do not independently validate the disclosure or prove successful reproduction.
[White et al. 2025] LiveBench: a challenging, contamination-limited LLM benchmark. https://proceedings.iclr.cc/paper_files/paper/2025/hash/e4a46394ba5378b3f9a186a5b4c650d1-Abstract-Conference.html
LiveBench updates questions monthly from recent sources and uses objective automatic grading to reduce both contamination and subjective judge bias.
[Wijk et al. 2025] RE-bench: Evaluating frontier AI R&D capabilities of language model agents against human experts. PMLR. https://proceedings.mlr.press/v267/wijk25a.html
RE-Bench compares agents with 61 human experts on seven research-engineering environments and finds sharply different rankings at short and long total-compute budgets.
[Willard and Louf 2023] Efficient guided generation for large language models. arXiv preprint arXiv:2307.09702. https://arxiv.org/abs/2307.09702
Willard and Louf compose a formal language with a model vocabulary so regular-expression and grammar constraints can provide efficient token masks during generation.
[Williams et al. 2009] Roofline: An insightful visual performance model for multicore architectures. Communications of the ACM. https://doi.org/10.1145/1498765.1498785
Roofline relates attainable performance to operational intensity, peak arithmetic rate, and sustainable memory bandwidth.
[World Wide Web Consortium 2026] WebDriver. https://www.w3.org/TR/webdriver/
WebDriver defines the remote-control protocol and session model used to automate a browser, making its endpoint a privileged execution interface.
[Wortsman et al. 2022] Model soups: Averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. https://arxiv.org/abs/2203.05482
For compatible models from the same fine-tuning sweep, the paper reports that selected weight averages can outperform the best individual checkpoint without ensemble inference cost.
[Wortsman et al. 2023] Small-scale proxies for large-scale transformer training instabilities. arXiv preprint arXiv:2309.14322. https://arxiv.org/abs/2309.14322
Reproduces attention-logit and output-logit instabilities in small proxy models and evaluates mitigations that had been used at large scale.
[Wright et al. 2022] JSON schema: a media type for describing JSON documents. https://json-schema.org/draft/2020-12/json-schema-core
JSON Schema Core defines vocabularies, identifiers, references, anchors, and evaluation rules for describing the structure of JSON documents.
[Wu et al. 2022] Sustainable AI: Environmental implications, challenges and opportunities. https://proceedings.mlsys.org/paper_files/paper/2022/hash/462211f67c7d858f663355eff93b745e-Abstract.html
Production workload measurements show that training and inference shares differ materially by system, contradicting a universal lifetime-cost split.
[Wu et al. 2025] LongMemEval: Benchmarking chat assistants on long-term interactive memory. https://proceedings.iclr.cc/paper_files/paper/2025/hash/d813d324dbf0598bbdc9c8e79740ed01-Abstract-Conference.html
LongMemEval uses scalable chat histories and 500 questions to test five memory abilities while separating indexing, retrieval, and reading.
[x402 Foundation 2026] x402 Protocol. https://github.com/x402-foundation/x402
x402 version 2 defines PAYMENT-REQUIRED, PAYMENT-SIGNATURE, and PAYMENT-RESPONSE exchanges while keeping payment schemes extensible across networks and assets.
[Xiao et al. 2023] SmoothQuant: Accurate and efficient post-training quantization for large language models. https://proceedings.mlr.press/v202/xiao23c.html
SmoothQuant enables training-free W8A8 post-training quantization for LLMs by migrating activation outliers to weights via a mathematically equivalent per-channel scaling transformation.
[Xiao et al. 2024] Efficient streaming language models with attention sinks. https://proceedings.iclr.cc/paper_files/paper/2024/hash/5e5fd18f863cbe6d8ae392a93fd271c9-Abstract-Conference.html
StreamingLLM retains a few initial attention-sink tokens with a recent window to support stable streaming language modeling in its evaluated model families.
[Xie et al. 2023] DoReMi: Optimizing data mixtures speeds up language model pretraining. https://openreview.net/forum?id=lXuByUeHhd
DoReMi uses Group DRO on a small proxy model to find domain mixture weights for pretraining, improving average few-shot downstream accuracy by 6.5 percentage points and reaching the baseline accuracy with 2.6 times fewer training steps in its 8B-model experiment.
[Xie et al. 2023] Self-evaluation guided beam search for reasoning. https://arxiv.org/abs/2305.00633
This paper proposes a stepwise self-evaluation mechanism integrated with stochastic beam search to guide LLM multi-step reasoning, outperforming Codex baselines by up to 9.56% on reasoning benchmarks.
[Xie et al. 2024] OSWorld: Benchmarking multimodal agents for open-ended tasks in real computer environments. https://proceedings.neurips.cc/paper_files/paper/2024/hash/5d413e48f84dc61244b6be550f1cd8f5-Abstract-Datasets_and_Benchmarks_Track.html
OSWorld defines desktop tasks with explicit initial-state setup and custom execution-based evaluators across web, file, command-line, and application workflows.
[Xie et al. 2025] Show-o: One single transformer to unify multimodal understanding and generation. https://arxiv.org/abs/2408.12528
Show-o is a single transformer that unifies multimodal understanding and generation by combining autoregressive modeling for text with discrete diffusion for images.
[Xiong et al. 2020] On layer normalization in the transformer architecture. PMLR. https://arxiv.org/abs/2002.04745
This paper uses mean field theory to show that placing layer normalization inside residual blocks (Pre-LN) yields well-behaved gradients at initialization, allowing Transformer training without learning rate warm-up.
[Xiong et al. 2021] Approximate nearest neighbor negative contrastive learning for dense text retrieval. https://openreview.net/forum?id=zeFrfgyZln
ANCE mines high-scoring unjudged training candidates from an asynchronously refreshed whole-corpus ANN index, replacing mostly uninformative random negatives.
[Xu et al. 2021] GSPMD: General and scalable parallelization for ML computation graphs. https://arxiv.org/abs/2105.04663
GSPMD propagates tensor-sharding annotations through a computation graph and emits a partitioned single-program, multiple-data program.
[Xu et al. 2024] RECOMP: Improving retrieval-augmented lms with compression and selective augmentation. https://openreview.net/forum?id=mlJLVigNHp
RECOMP trains extractive and abstractive compressors against downstream tasks and learns when retrieved augmentation should be omitted rather than always prepended.
[Xue et al. 2022] ByT5: Towards a token-free future with pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics. https://aclanthology.org/2022.tacl-1.17/
ByT5 removes the subword vocabulary and trains a minimally modified T5 directly on UTF-8 bytes.
[Yadav et al. 2023] TIES-merging: Resolving interference when merging models. Advances in Neural Information Processing Systems 36. https://arxiv.org/abs/2306.01708
TIES-Merging trims small same-base deltas, elects a sign per coordinate, and averages updates that agree with that sign to reduce measured merge interference.
[Yang et al. 2021] Tensor programs V: Tuning large neural networks via zero-shot hyperparameter transfer. https://arxiv.org/abs/2203.03466
Introduces muP, under which many hyperparameter optima can transfer across width in a matched model family, greatly reducing full-scale tuning cost.
[Yang et al. 2023] Rethinking benchmark and contamination for language models with rephrased samples. https://arxiv.org/abs/2311.04850
This paper shows that rephrased benchmark test samples (paraphrased or translated) bypass n-gram and embedding decontamination, and proposes an LLM-based decontaminator that detects such contamination in pre-training datasets.
[Yang et al. 2023] Set-of-mark prompting unleashes extraordinary visual grounding in GPT-4V. arXiv preprint arXiv:2310.11441. https://arxiv.org/abs/2310.11441
Overlay numbered marks on detected screen regions so the model outputs a mark instead of coordinates, the workaround that carried GUI grounding before natively grounded models.
[Yang et al. 2023] LeanDojo: Theorem proving with retrieval-augmented language models. https://proceedings.neurips.cc/paper_files/paper/2023/hash/4441469427094f8873d0fecb0c4e1cee-Abstract-Datasets_and_Benchmarks.html
LeanDojo releases tools, data, models, and benchmarks for Lean theorem proving, with retrieval-augmented premise selection as a central bottleneck.
[Yang et al. 2024] CogVideoX: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072. https://arxiv.org/abs/2408.06072
CogVideoX is a text-to-video diffusion transformer using a 3D causal VAE and expert adaptive LayerNorm to generate 10-second, 768x1360 videos at 16 fps with coherent motion.
[Yang et al. 2024] SWE-agent: Agent-computer interfaces enable automated software engineering. https://proceedings.neurips.cc/paper_files/paper/2024/hash/5a7c947568c1b1328ccc5230172e1e7c-Abstract-Conference.html
SWE-agent designs a model-facing interface for repository navigation, editing, and testing, showing that the action and observation interface is part of agent performance.
[Yang et al. 2025] Gated delta networks: Improving mamba2 with delta rule. https://arxiv.org/abs/2412.06464
Gated DeltaNet combines a gated forgetting mechanism with delta-rule state updates in a linear-attention recurrence, surpassing Mamba2 and DeltaNet on language modeling and long-context tasks.
[Yang et al. 2025] Qwen2.5-1M technical report. arXiv preprint arXiv:2501.15383. https://arxiv.org/abs/2501.15383
Reports Qwen2.5-1M, progressively pre-trained to 262,144-token sequences, then combined with post-training and inference-time position and attention methods to accept inputs up to one million tokens.
[Yang et al. 2025] SWE-smith: Scaling data for software engineering agents. arXiv preprint arXiv:2504.21798. https://arxiv.org/abs/2504.21798
SWE-smith installs a Python repository, perturbs the code until an existing test fails, and treats the broken state as a task with that test as its verifier, producing 50,000 instances from 128 repositories.
[Yang et al. 2025] Revisiting multi-agent debate as test-time scaling: a systematic study of conditional effectiveness. arXiv preprint arXiv:2505.22960. https://arxiv.org/abs/2505.22960
A compute-aware comparison finds that debate gains depend on task difficulty, model capability, and safety setting rather than holding universally.
[Yang and others 2025] Qwen3 technical report. https://arxiv.org/abs/2505.09388
Qwen3 introduces a unified dense and MoE model family (0.6B to 235B parameters) that integrates thinking and non-thinking modes with a thinking budget mechanism for inference-time compute control.
[Yao et al. 2023] Tree of thoughts: Deliberate problem solving with large language models. https://arxiv.org/abs/2305.10601
Tree of Thoughts (ToT) is a framework that lets LMs explore multiple reasoning paths via tree search with self-evaluation, raising GPT-4's Game of 24 success rate from 4% to 74%.
[Yao et al. 2023] ReAct: Synergizing reasoning and acting in language models. https://arxiv.org/abs/2210.03629
ReAct interleaves reasoning traces and task-specific actions, letting LLMs update plans with external observations from tools or environments.
[Yao et al. 2023] ReAct: Synergizing reasoning and acting in language models. https://openreview.net/forum?id=WE_vluYUL-X
ReAct interleaves model reasoning with environment actions in experiments on question answering and interactive tasks; it is an agent-loop method, not a framework, runtime, or safety boundary.
[Yao et al. 2025] <span class="nocase">τ-bench</span>: a benchmark for tool-agent-user interaction in real-world domains. https://arxiv.org/abs/2406.12045
Tau-bench evaluates conversations between a tool-using agent and a simulated user, compares final database state with an annotated goal, and introduces pass-to-the-k for repeated reliability.
[Yao et al. 2026] Harness-Bench: Measuring harness effects across models in realistic agent workflows. arXiv preprint arXiv:2605.27922. https://arxiv.org/abs/2605.27922
Harness-Bench compares model and harness pairings under fixed task, sandbox, budget, timeout, and evaluator conditions, treating configuration-level performance as the measured object.
[Ye et al. 2025] Dream 7B: Diffusion large language models. arXiv preprint arXiv:2508.15487. https://arxiv.org/abs/2508.15487
Dream 7B initializes masked-diffusion training from Qwen2.5 weights and continues training on 580B tokens, demonstrating that autoregressive pretraining can transfer into this diffusion setup.
[Ye et al. 2025] LIMO: Less is more for reasoning. https://openreview.net/forum?id=T2TZ0RY4Zk
LIMO fine-tunes Qwen2.5-32B-Instruct on 800 selected mathematical examples and reports strong competition-math results, while showing that the same recipe is highly base-model dependent.
[Yen et al. 2025] HELMET: How to evaluate long-context models effectively and thoroughly. https://proceedings.iclr.cc/paper_files/paper/2025/hash/f5332c8273d02729730a9c24dec2135e-Abstract-Conference.html
HELMET evaluates long-context models across recall, retrieval-augmented generation, many-shot learning, question answering, and summarization instead of treating a single retrieval probe as sufficient.
[Yin et al. 2024] One-step diffusion with distribution matching distillation. https://arxiv.org/abs/2311.18828
DMD distills a pretrained diffusion model into a one-step image generator by minimizing an approximate KL divergence between real and fake distributions, achieving 2.62 FID on ImageNet 64x64 at 20 FPS with FP16 inference.
[Yin et al. 2025] From slow bidirectional to fast autoregressive video diffusion models. https://arxiv.org/abs/2412.07772
CausVid distills a fifty-step bidirectional video diffusion model into a four-step causal generator and reports 9.4 FPS on one GPU after 1.3 seconds of initial latency using KV caching.
[Young 1974] A first order approximation to the optimum checkpoint interval. Communications of the ACM. https://doi.org/10.1145/361147.361115
Young derives the first-order checkpoint interval from checkpoint cost and mean time between failures, under a simple independent-failure model.
[Yu et al. 2022] Vector-quantized image modeling with improved VQGAN. https://arxiv.org/abs/2110.04627
ViT-VQGAN replaces CNN encoders with Vision Transformers in VQGAN and introduces factorized, L2-normalized codebook learning, improving image generation FID and unsupervised linear-probe accuracy on ImageNet.
[Yu et al. 2022] Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789. https://arxiv.org/abs/2206.10789
Parti scales an autoregressive encoder-decoder Transformer for text-to-image generation to 20B parameters, treating image tokens from ViT-VQGAN as a sequence-to-sequence target and achieving state-of-the-art FID on MS-COCO.
[Yu et al. 2022] Orca: a distributed serving system for transformer-based generative models. USENIX Association. https://www.usenix.org/conference/osdi22/presentation/yu
Orca introduces iteration-level scheduling so a generative-model server can rebuild its batch after each decoding step instead of waiting for a static batch to finish.
[Yu et al. 2023] MEGABYTE: Predicting million-byte sequences with multiscale transformers. https://proceedings.neurips.cc/paper_files/paper/2023/hash/f8f78f8043f35890181a824e53a57134-Abstract-Conference.html
MEGABYTE processes fixed-size byte patches with a global model between patches and a smaller local model within each patch.
[Yu et al. 2024] Language models are super mario: Absorbing abilities from homologous models as a free lunch. https://arxiv.org/abs/2311.03099
DARE randomly drops and rescales same-base fine-tuning deltas before applying another merge rule, with performance depending on compatible source models and chosen sparsity.
[Yu et al. 2025] DAPO: An open-source LLM reinforcement learning system at scale. https://proceedings.neurips.cc/paper_files/paper/2025/hash/a4277440d50f1f15d2cb4c14f7e0c0d2-Abstract-Conference.html
DAPO proposes Decoupled Clip and Dynamic Sampling Policy Optimization with four techniques (Clip-Higher, Dynamic Sampling, Token-Level Policy Gradient Loss, Overlong Reward Shaping) to enable reproducible large-scale RL training, achieving 50 points on AIME 2024 with Qwen2.5-32B.
[Yu et al. 2026] Cachemir: Fully homomorphic encrypted inference of generative large language model with KV cache. arXiv preprint arXiv:2602.11470. https://arxiv.org/abs/2602.11470
Demonstrates an FHE generative-model design with KV-cache support, showing why claims about cryptographic inference must be dated and workload-specific.
[Yuan et al. 2023] Scaling relationship on learning mathematical reasoning with large language models. arXiv preprint arXiv:2308.01825. https://arxiv.org/abs/2308.01825
This study introduces rejection sampling fine-tuning for mathematical reasoning and shows that filtering and structurally diversifying sampled GSM8K solutions can improve supervised models.
[Yuan et al. 2024] LLM inference unveiled: Survey and roofline model insights. arXiv preprint arXiv:2402.16363. https://arxiv.org/abs/2402.16363
This survey analyzes LLM inference efficiency using a Roofline model framework and introduces LLM-Viewer, a tool that identifies memory and compute bottlenecks when deploying LLMs on hardware.
[Yuan et al. 2024] KV cache compression, but what must we give in return? A comprehensive benchmark of long context capable approaches. Association for Computational Linguistics. https://aclanthology.org/2024.findings-emnlp.266/
This benchmark compares long-context efficiency methods across seven task categories and finds that their quality and efficiency trade-offs vary by method, task, and model.
[Yuan et al. 2025] Native sparse attention: Hardware-aligned and natively trainable sparse attention. Association for Computational Linguistics. https://arxiv.org/abs/2502.11089
NSA trains a gated combination of compressed-token, selected-block, and local-window attention branches end to end, with hardware-aligned kernels for long contexts.
[Yuan et al. 2025] From hard refusals to safe-completions: Toward output-centric safety training. arXiv preprint arXiv:2508.09224. https://arxiv.org/abs/2508.09224
Safe-completions training replaces a binary refusal boundary with an output-centric learned objective; the reported GPT-5 experiments improve safety and helpfulness on dual-use prompts where intent-based refusal is brittle.
[Yuan et al. 2026] Incentivizing strong reasoning from weak supervision. Association for Computational Linguistics. https://aclanthology.org/2026.eacl-long.336/
This paper studies Qwen weak-to-strong trace supervision and reports that structured traces from smaller RL-trained reasoners can recover much of a stronger student's direct-RL gain in selected settings.
[Yuan et al. 2026] OSWorld 2.0: Benchmarking computer use agents on long-horizon real-world tasks. arXiv preprint arXiv:2606.29537. https://arxiv.org/abs/2606.29537
A benchmark of 108 long-horizon workflows showing failures around changing information, hidden state, constraints, asking the user, and verification rather than only elementary GUI control.
[Yue et al. 2024] MMMU: a massive multi-discipline multimodal understanding and reasoning benchmark for expert AGI. https://arxiv.org/abs/2311.16502
MMMU is a benchmark of 11.5K college-level multimodal questions across 30 subjects and 6 disciplines to evaluate expert-level perception, knowledge, and reasoning in large multimodal models.
[Yue et al. 2024] MMMU-pro: a more robust multi-discipline multimodal understanding benchmark. arXiv preprint arXiv:2409.02813. https://arxiv.org/abs/2409.02813
MMMU-Pro makes the original multimodal benchmark harder through revised questions, expanded answer choices, and vision-only input variants.
[Yue et al. 2025] Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?. https://proceedings.neurips.cc/paper_files/paper/2025/hash/537d5aa768c2d534016a4d06f87bc8fb-Abstract-Conference.html
Across the tested math, coding, and visual-reasoning settings, RLVR improved low-k sampling but did not extend large-k coverage beyond the base model.
[Zadouri et al. 2026] FlashAttention-4: Algorithm and kernel pipelining co-design for asymmetric hardware scaling. https://arxiv.org/abs/2603.05451
FlashAttention-4 targets Blackwell with a CuTe-DSL (Python-embedded) implementation, software-emulated exponentials on FMA units, and conditional softmax rescaling, reaching 1.1-1.3x over cuDNN attention on B200.
[Zanki 2025] Malicious ML models discovered on hugging face platform. https://www.reversinglabs.com/blog/rl-identifies-malware-ml-model-hosted-on-hugging-face
ReversingLabs found two malformed 7z-wrapped pickle artifacts that were not flagged; a benign reproduction showed earlier opcodes can execute before a later parse failure.
[Zeghidour et al. 2021] SoundStream: An end-to-end neural audio codec. IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30. https://arxiv.org/abs/2107.03312
SoundStream is an end-to-end neural audio codec with a convolutional encoder/decoder and residual vector quantizer (RVQ) that outperforms Opus and EVS at low bitrates (3-18 kbps) on speech, music, and general audio.
[Zelikman et al. 2022] STaR: Bootstrapping reasoning with reasoning. https://proceedings.neurips.cc/paper_files/paper/2022/hash/639a9a172c044fbb64175b5fad42e9a5-Abstract-Conference.html
STaR iteratively generates rationales, retains attempts that reach known answers, rationalizes failures with the answer as a hint, and retrains from the original checkpoint.
[Zhai et al. 2023] Sigmoid loss for language image pre-training. https://arxiv.org/abs/2303.15343
SigLIP replaces softmax contrastive loss with a pairwise sigmoid loss for language-image pre-training, enabling memory-efficient large-batch training and better performance at small batch sizes.
[Zhai et al. 2024] Actions speak louder than words: Trillion-parameter sequential transducers for generative recommendations. PMLR. https://arxiv.org/abs/2402.17152
The authors report a 1.5-trillion-parameter HSTU generative recommender deployed on several platform surfaces, a 12.4% gain on one disclosed online metric, and empirical power-law quality scaling across three orders of training compute; retrieval and ranking remain separate evaluated settings.
[Zhan et al. 2024] InjecAgent: Benchmarking indirect prompt injections in tool-integrated large language model agents. https://aclanthology.org/2024.findings-acl.624/
INJECAGENT is a benchmark of 1,054 test cases evaluating LLM agent vulnerability to indirect prompt injection attacks, finding ReAct-prompted GPT-4 susceptible 24% of the time.
[Zhang and Sennrich 2019] Root mean square layer normalization. https://arxiv.org/abs/1910.07467
RMSNorm replaces LayerNorm's mean-and-variance normalization with RMS-only scaling, achieving comparable accuracy while reducing per-step runtime by 7 to 64 percent.
[Zhang et al. 2023] H2O: Heavy-hitter oracle for efficient generative inference of large language models. https://proceedings.neurips.cc/paper_files/paper/2023/hash/6ceefa7b15572587b78ecfcebb2827f8-Abstract-Conference.html
H2O is a dynamic KV-cache eviction policy that balances recent tokens with heavy hitters identified by accumulated attention in the evaluated models and tasks.
[Zhang et al. 2024] SpeechTokenizer: Unified speech tokenizer for speech large language models. https://arxiv.org/abs/2308.16692
SpeechTokenizer uses HuBERT distillation to encourage its first RVQ layer to expose linguistic information while later layers refine reconstruction detail.
[Zhang and Nanda 2024] Towards best practices of activation patching in language models: Metrics and methods. https://arxiv.org/abs/2309.16042
Activation-patching conclusions can change with the corruption procedure, output metric, and other methodological choices, so the full counterfactual design must be reported.
[Zhang et al. 2025] IHEval: Evaluating language models on following the instruction hierarchy. Proceedings of NAACL 2025. https://arxiv.org/abs/2502.08745
IHEval tests aligned and conflicting instructions across 3,538 examples and finds large accuracy drops when evaluated models must resolve priority conflicts.
[Zhang et al. 2025] The lessons of developing process reward models in mathematical reasoning. https://arxiv.org/abs/2501.07301
Monte-Carlo-estimated step labels yield weaker PRMs than LLM-as-judge and human annotation; a consensus filter that keeps only steps where both agree produces the stronger Qwen2.5-Math-PRM, which the authors evaluate on the separately released ProcessBench.
[Zhang et al. 2025] Generative verifiers: Reward modeling as next-token prediction. https://arxiv.org/abs/2408.15240
GenRM trains LLM verifiers with next-token prediction rather than discriminative classification, enabling generated verification rationales and test-time voting for best-of-N selection.
[Zhang et al. 2026] KernelBench-verified: Do LLM-generated kernels actually beat PyTorch?. https://arxiv.org/abs/2607.16241
A stricter follow-up evaluation adds hidden input distributions, a stronger baseline, and memory metrics, showing how narrow correctness tests can reward invalid shortcuts.
[Zhao et al. 2023] PyTorch FSDP: Experiences on scaling fully sharded data parallel. Proceedings of the VLDB Endowment. https://arxiv.org/abs/2304.11277
The FSDP paper explains how parameter sharding interacts with PyTorch autograd, allocation, communication, and state management.
[Zhao et al. 2023] Learning fine-grained bimanual manipulation with low-cost hardware. https://arxiv.org/abs/2304.13705
ACT predicts action sequences rather than single steps and temporally combines overlapping predictions for fine-grained bimanual manipulation.
[Zhao et al. 2025] Absolute zero: Reinforced self-play reasoning with zero data. arXiv preprint arXiv:2505.03335. https://arxiv.org/abs/2505.03335
Absolute Zero trains one pretrained model to propose and solve tasks against a code executor without an external post-training task-answer dataset; the setup still relies on pretraining, human-designed task schemas, and the executor.
[Zhao et al. 2025] Do llms recognize your preferences? Evaluating personalized preference following in llms. https://proceedings.iclr.cc/paper_files/paper/2025/hash/28a46044775d97a4efcbcf14e7f13209-Abstract-Conference.html
PrefEval tests whether models follow explicit and implicit user preferences across topics and increasingly long contexts.
[Zheng et al. 2022] MiniF2F: a cross-system benchmark for formal olympiad-level mathematics. https://iclr.cc/virtual/2022/poster/6258
MiniF2F provides 488 Olympiad-level statements, with complete Metamath and Lean translations and partial Isabelle and HOL Light coverage in its original release.
[Zheng et al. 2023] Judging LLM-as-a-judge with MT-bench and chatbot arena. https://proceedings.neurips.cc/paper_files/paper/2023/hash/91f18a1287b398d378ef22505bf41832-Abstract-Datasets_and_Benchmarks.html
Studies model-based judging for chat assistants and documents position, verbosity, and self-enhancement biases.
[Zheng et al. 2024] SGLang: Efficient execution of structured language model programs. https://proceedings.neurips.cc/paper_files/paper/2024/hash/724be4472168f31ba1c9ac630f15dec8-Abstract-Conference.html
SGLang includes compressed finite-state machines and jump-forward processing to reduce sequential decoding work across deterministic structured-output spans.
[Zheng et al. 2025] ProcessBench: Identifying process errors in mathematical reasoning. https://arxiv.org/abs/2412.06559
ProcessBench evaluates whether a verifier can identify the earliest erroneous step or recognize a fully correct solution, and finds weak generalization from common math training distributions to harder problems.
[Zheng et al. 2025] Group sequence policy optimization. arXiv preprint arXiv:2507.18071. https://arxiv.org/abs/2507.18071
GSPO replaces GRPO's token-level importance ratios with sequence-level ratios and clipping, stabilizing MoE RL training and improving efficiency in Qwen3 models.
[Zhong et al. 2024] DistServe: Disaggregating prefill and decoding for goodput-optimized large language model serving. USENIX Association. https://www.usenix.org/conference/osdi24/presentation/zhong-yinmin
DistServe places prefill and decode on separate GPU pools and defines goodput through the arrival rate sustainable at a chosen TTFT and TPOT SLO attainment.
[Zhou et al. 2020] Understanding knowledge distillation in non-autoregressive machine translation. https://arxiv.org/abs/1911.02727
This paper studies why sequence-level knowledge distillation helps non-autoregressive translation, using alignment-based word-translation entropy as a proxy for data complexity and reporting an empirical relationship with model capacity.
[Zhou et al. 2022] Mixture-of-experts with expert choice routing. https://arxiv.org/abs/2202.09368
Expert Choice MoE proposes letting each expert select its top-k tokens instead of each token choosing experts, guaranteeing perfect load balancing and achieving over 2x faster training convergence than Switch Transformer and GShard.
[Zhou et al. 2023] LIMA: Less is more for alignment. Advances in Neural Information Processing Systems. https://arxiv.org/abs/2305.11206
LIMA fine-tunes a 65B LLaMA model on 1,000 curated prompt-response pairs and reports strong format learning and conversational behavior, motivating the superficial-alignment hypothesis in that setting.
[Zhou et al. 2023] Least-to-most prompting enables complex reasoning in large language models. https://arxiv.org/abs/2205.10625
Least-to-most prompting first decomposes a problem, then solves the subproblems sequentially; the paper reports its strongest easy-to-hard generalization on a task-engineered SCAN setup.
[Zhou et al. 2024] WebArena: a realistic web environment for building autonomous agents. https://proceedings.iclr.cc/paper_files/paper/2024/hash/4410c0711e9154a7a2d26f9b3816d1ef-Abstract-Conference.html
WebArena provides self-hosted functional websites and long-horizon tasks evaluated for functional correctness, allowing different valid action paths to reach the same goal.
[Zhou et al. 2025] Transfusion: Predict the next token and diffuse images with one multi-modal model. https://arxiv.org/abs/2408.11039
Transfusion trains a single transformer on mixed text and image data by applying next-token prediction loss to text and DDPM diffusion loss to images, scaling to 7B parameters with quality matching dedicated models.
[Zhou et al. 2026] When more thinking hurts: Overthinking in LLM test-time compute scaling. https://aclanthology.org/2026.findings-acl.1199/
Forced token-budget experiments show diminishing returns and cases where longer reasoning reverses an initially correct answer, motivating difficulty-aware stopping.
[Zhou et al. 2026] General scales unlock AI evaluation with explanatory and predictive power. Nature. https://www.nature.com/articles/s41586-026-10303-2
The study tests 18 demand-level rubrics across 15 language models and 63 tasks to explain and predict instance difficulty.
[Zhu et al. 2024] Confidential computing on NVIDIA hopper gpus: a performance benchmark study. arXiv preprint arXiv:2409.03992. https://arxiv.org/abs/2409.03992
Benchmarks H100 confidential mode on two systems and three model families, reporting workload-dependent throughput and latency effects rather than one universal overhead.
[Zhu et al. 2025] LLaDA-MoE: a sparse MoE diffusion language model. arXiv preprint arXiv:2509.24389. https://arxiv.org/abs/2509.24389
LLaDA-MoE trains a sparse masked-diffusion model from scratch on roughly 20T tokens, with about 7B non-embedding parameters in total and 1.4B active per token.
[Ziegler et al. 2019] Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593. https://arxiv.org/abs/1909.08593
This early language-model RLHF study learns rewards from human comparisons for stylistic continuation and summarization, while documenting that a learned reward can encode labeler heuristics.
[Zoph et al. 2022] ST-MoE: Designing stable and transferable sparse expert models. https://arxiv.org/abs/2202.08906
ST-MoE-32B is a 269B sparse MoE model that resolves MoE training instability and fine-tuning transfer gaps, achieving state-of-the-art results across diverse NLP benchmarks.
[Zou et al. 2023] Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043. https://arxiv.org/abs/2307.15043
GCG introduces an automated greedy coordinate gradient method to find adversarial suffixes that jailbreak aligned LLMs, transferring to black-box models including ChatGPT, Bard, and Claude.
[Zou et al. 2024] Improving alignment and robustness with circuit breakers. https://proceedings.neurips.cc/paper_files/paper/2024/hash/97ca7168c2c333df5ea61ece3b3276e1-Abstract-Conference.html
Circuit breakers use representation rerouting to move harmful internal representations away from their original trajectory and report lower attack success on the evaluated unseen attacks with limited measured utility loss.
[Zou et al. 2025] PoisonedRAG: Knowledge corruption attacks to retrieval-augmented generation of large language models. https://www.usenix.org/conference/usenixsecurity25/presentation/zou-poisonedrag
PoisonedRAG demonstrates that attacker-injected passages can corrupt answers in evaluated RAG configurations, establishing the corpus as an attack surface.
Comments
Log in to comment