The Model Landscape
A model name is not a deployment option. The same organization may publish one model's weights, sell another through an API, and attach different terms to a third. Even one model can appear as a research checkpoint, a quantized community build, and a hosted service. Those versions may differ in behavior, permitted use, data handling, and support.
The durable map is not a single open-to-closed score and does not rank laboratories. Read each release as a release contract: which artifact access it provides, which legal permission it grants, which operational access it promises, and what evidence quality supports its claims. These are independent questions. A downloadable model can have restrictive terms; a well-documented model can be unavailable for download; an API can have stronger operational guarantees than a self-hosted checkpoint. Every answer also needs an exact version and a snapshot date.
Start with precise nouns
Several words that sound interchangeable name different objects.
- An AI system includes the model and the surrounding software, data flows, tools, policies, and human processes that make it useful.
- A model consists, in the Open Source AI Definition's terminology, of the model architecture, learned parameters, and inference code needed to run it.
- Weights are the learned parameters. They are an important artifact, but they do not reveal the training corpus, training procedure, or deployment controls.
- A hosted service lets a customer send requests to a provider-operated system. It exposes behavior through an interface, not necessarily the underlying model artifact.
- In this book, open weights is a descriptive claim that weights can be obtained under terms broad enough for local use. Industry usage is inconsistent. A 2026 G7 vocabulary instead reserves “Open Weights AI” for weights and deployment code under open-source terms, and calls restricted downloads “Weights Available AI” (G7 Industry, Digital and Technology Ministers 2026). Whichever vocabulary a team adopts, it must record the actual artifacts and terms rather than rely on the label.
- Open Source AI is a stronger, defined claim. Version 1.0 of the Open Source AI Definition requires freedom to use, study, modify, and share the system, plus access to the preferred form for modification: data information, complete training and inference code, and parameters under qualifying terms (Open Source Initiative 2024).
One nuance prevents a common misunderstanding. The definition requires enough data information for a skilled person to build a substantially equivalent system, including provenance, scope, selection, processing, and locations of available data. It does not require publication of every training example when the data cannot legally be shared. That is broader than a bare weight release, but it is not identical to releasing the complete corpus. The Model Openness Framework uses three completeness classes over a component inventory rather than assuming that one downloadable file settles every openness question. Its classification is self-reported and concerns disclosed components and stated licenses; it does not independently confirm the claims or the success of a reproduction (White et al. 2024).
These definitions describe what is supplied and permitted. They do not certify that a model is accurate, safe, unbiased, inexpensive, or suitable for a particular deployment.
A release contract has five fields
Here, a candidate release is described as a versioned evidence record:
For a set of hard requirements, the release gate is
where:
r : the candidate release under review
v : its exact version identifier, revision, or artifact hash
t : the observation date for mutable pages, terms, and services
R_r : the release profile for r at version v and time t
A_r : available access channels: download, gated download, hosted API, or none
X_r : released artifacts and documentation
P_r : legal permissions, obligations, restrictions, and incorporated policies
O_r : operational constraints and service commitments
E_r : the evidence bundle supporting claims about the release
Q : the set of hard deployment requirements
q : one requirement in Q
evidence(R_r, q)
: confirmed, refuted, or unknown from documented evidence
1[...] : 1 when the enclosed condition is true and 0 otherwise
⋀ : logical AND across every hard requirement
This is not an openness metric and should not become a weighted leaderboard. If local execution is mandatory, an excellent API score cannot compensate for absent weights. If redistribution is mandatory, benchmark quality cannot compensate for a prohibition in the terms. Unknown does not satisfy a release gate; it creates an unresolved item that needs evidence or an explicit risk decision.
Artifact access: what can you actually obtain?
Record artifacts separately instead of writing “open” in one cell. Depending on the release, the list may include:
- final weights, architecture and tokenizer files, and inference code;
- training code and configuration, including hyperparameters and software versions;
- training-data information, a data mixture, or the training data itself;
- intermediate checkpoints, optimizer state, data order, and run telemetry;
- evaluation code, prompts, raw outputs, model card, and safety documentation.
Each artifact enables a different investigation. Final weights and compatible runtime code can support local inference. Training code can expose the procedure, but it cannot recover a withheld corpus. A dataset list does not recover the exact data order or filtering state. Intermediate checkpoints help study learning dynamics, while optimizer state can matter when continuing a run.
The word reproducible also needs a target. Re-running published evaluation code on the same weights is computational reproduction of an evaluation. To repeat the published run, a team needs the data, order, code, configuration, environment, and resources. Independent replication asks whether a new team or new data can recover the substantive result. None of these, by itself, proves that a deployment is safe or that a model card is complete (Mitchell et al. 2019).
Legal permission: what may you do with it?
Possession and permission are separate. Before adopting downloadable weights, have the responsible reviewer inspect the actual license and every policy it incorporates. The checklist normally includes the copyright grant, patent grant, redistribution conditions, use restrictions, treatment of a derivative model, attribution and notice duties, trademark rules, and termination clauses. A custom agreement may allow commercial inference but constrain redistribution or require downstream users to inherit a use policy. The word “community” does not answer any of those questions.
Apache 2.0 is often called permissive because it grants broad copyright and patent rights subject to stated conditions. A model-specific agreement can be broader in one respect and narrower in another. The relevant question is not whether a press release says “open,” but whether the exact terms authorize the planned product and distribution path. This chapter provides an engineering checklist, not legal advice.
Operational access: what happens when someone else runs it?
For a hosted service, the API agreement and product documentation are part of the release contract. Check the acceptable-use policy, input and output data retention, training on customer data, region availability, rate limits, version pinning, deprecation notice, support path, and service-level agreement. Also record which claims are contractual and which are merely current product behavior.
These questions can matter more than weight availability. A regulated workload may prefer a hosted service with an acceptable region and enforceable retention terms. A product that must work offline may reject the same service immediately. Conversely, a downloadable checkpoint removes provider-side version drift but makes the operator responsible for capacity, patching, monitoring, and incident response.
Evidence quality: what can you verify?
A technical report is evidence, but not all reports answer the same questions. Record whether a claim is backed by executable code, released raw results, an artifact hash, an independent reproduction, or only a provider summary. Document the evaluation scope: model revision, prompt format, sampling settings, tool access, scorer, and comparison baseline. Model cards were proposed precisely because an aggregate score without intended uses, limitations, and disaggregated evaluation is too thin for deployment decisions (Mitchell et al. 2019).
Evidence quality is not the same as model quality. A modest research model can have excellent provenance. A highly capable service can have strong uptime commitments and weak training disclosure. Keep those facts separate so that one attractive property does not silently stand in for another.
Release examples, not permanent tiers
The following examples are evidence snapshots as of 7 August 2026. They show why classification belongs to a release, not a laboratory. Terms and repositories can change, so a production review must capture the exact documents it relied on.
Pythia: designed for training-dynamics research. The ICML paper describes 16 language models trained on public data in the same data order. It releases 154 checkpoints per model and tools to reconstruct the training dataloader (Biderman et al. 2023). That combination supports studies that a final checkpoint alone cannot. It does not make Pythia suitable for every product or independently replicate every reported result.
OLMo 2: a broad training release. OLMo 2 publishes weights, training code, training data and mixtures, training logs, intermediate checkpoints, and its recipe, including changes made for stability and a two-stage curriculum (OLMo Team 2025). “Fully open” here describes the released development stack. It is not a universal assurance about outputs, downstream fine-tunes, or a particular deployment.
Llama 3, Qwen3, and DeepSeek-V3: weights plus substantial reports. Their reports document architecture, training choices, and evaluations at useful depth (Grattafiori and others 2024; Yang and others 2025; DeepSeek-AI 2024). Their complete pretraining corpora are not published as reconstructable datasets. Their legal profiles also differ: Llama 3 uses versioned custom community terms, Qwen3 weights use Apache 2.0, and DeepSeek-V3's reference code and original weights have different licenses. A user can run and adapt a release only as its particular terms, format, and hardware allow, and cannot repeat the original training run from the publication alone.
gpt-oss: permissive weights inside a mixed portfolio. OpenAI released gpt-oss-120b and gpt-oss-20b on 5 August 2025 under Apache 2.0, with downloadable weights, a model card, tokenizer material, and reference inference implementations (OpenAI 2025). The page gives a high-level description of the mostly English, text-only training data, not the corpus itself. The same company separately operates models available only through hosted products. The company name therefore cannot be the classification.
Gemma: the version changes the legal answer. Gemma 1–3 are downloadable weights under custom terms that incorporate use restrictions and impose conditions on distribution and derivative models (Google 2026). Gemma 4 instead uses Apache 2.0 (Google 2026). Writing only “Gemma” in an approval record would hide the fact that changes the legal review.
GPT-4 technical report: evidence without a reproducible training release. It describes evaluations and safety work but explicitly does not disclose architecture, model size, hardware, training compute, dataset construction, or training method; the weights are not released (OpenAI 2023). The report is still evidence for the claims it actually tests. It is not evidence for the withheld training details.
The point is not to award a permanent tier. A later release from any of these organizations can have a different artifact set, license, or service contract. The decision is release-specific.
What public reports can and cannot teach
Public releases and papers create a valuable public evidence base. They let other teams inspect an idea, reproduce a result when the required artifacts exist, and adapt a method to new systems. Three historical questions must remain separate: who introduced a method, who validated it at a stated scale, and who released enough artifacts to repeat the experiment. Publications cannot establish who used a method first inside an undisclosed project. A method appearing with open weights also does not show that a technique originated because the weights were open, or that every closed system uses it.
Read the record as dated, scoped contributions:
- The FP8 formats paper proposes the E4M3 and E5M2 formats and evaluates language models up to 175 billion parameters (Micikevicius et al. 2022). DeepSeek-V3 later supplies a detailed case study of FP8 training at its reported scale and deploys auxiliary-loss-free expert-routing biases (DeepSeek-AI 2024; Wang et al. 2024).
- DeepSeek-V2 reports multi-head latent attention for compressed key-value state (DeepSeek-AI 2024). The ICML multi-token prediction paper introduces an objective with independent prediction heads; DeepSeek-V3 uses a sequential variant (Gloeckle et al. 2024; DeepSeek-AI 2024). These are separate mechanisms, not one attention family.
- The grouped-query attention paper defines the intermediate design between multi-head and multi-query attention and demonstrates uptraining from existing checkpoints (Ainslie et al. 2023). The publication is evidence for that experiment, not for all later implementations.
- Chinchilla reports compute-optimal training under its experimental setup; inference-aware scaling work studies why serving demand can favor a smaller model trained on more tokens (Hoffmann et al. 2022; Sardana et al. 2024).
- Tensor Programs V reports zero-shot hyperparameter transfer with the maximal update parametrization, while MiniCPM describes a warmup-stable-decay schedule (Yang et al. 2021; Hu et al. 2024).
- Megatron-LM documents 3D parallelism across tensor, pipeline, and data dimensions. CheckFreq studies frequent checkpointing with asynchronous persistence (Narayanan et al. 2021; Mohan et al. 2021).
This history crosses organizational access models. Chinchilla came from a lab whose deployed frontier models were not released as weights; GQA came from Google; the DeepSeek reports accompanied downloadable weights. “Publicly documented” is the claim supported by a paper. “Invented by open models” is not.
Turn requirements into a gate
Start from the intended use, not from a catalog. A local private assistant may require local inference and private inputs. A regulated audit may additionally require training-data provenance. A redistributable product needs an affirmative redistribution right. These are hard predicates, not preferences to average away.
The small example below evaluates three use cases against one hypothetical release.
True means documented evidence confirms a requirement, False means the release
refutes it, and None means the review has not resolved it. The example models only
gate mechanics; it is not legal analysis and it does not score a real model.
release = {
"local_inference": True,
"private_inputs": True,
"training_data_provenance": None,
"redistribution": False,
}
use_cases = {
"local-private": ["local_inference", "private_inputs"],
"regulated-audit": ["local_inference", "training_data_provenance"],
"redistributable-product": ["local_inference", "redistribution"],
}
def classify(requirements):
refuted = [name for name in requirements if release[name] is False]
unknown = [name for name in requirements if release[name] is None]
if refuted:
return f"ineligible ({', '.join(refuted)})"
if unknown:
return f"unresolved ({', '.join(unknown)})"
return "eligible"
for use_case, requirements in use_cases.items():
print(f"{use_case}: {classify(requirements)}")
Expected output:
local-private: eligible
regulated-audit: unresolved (training_data_provenance)
redistributable-product: ineligible (redistribution)
Keep a release ledger
The decision should be reviewable after the model, terms, or provider changes. Store at least:
- exact model and revision, artifact hashes, access channel, and snapshot date;
- license version, terms version, incorporated policies, and their archived copies;
- required permissions and the reviewer responsible for confirming them;
- provenance evidence, released artifacts, and unresolved disclosure gaps;
- evaluation scope, raw results, and deployment-specific acceptance thresholds;
- hosting dependencies, data path, capacity owner, and incident contact;
- an exit test: whether weights, adapters, prompts, and data can move elsewhere;
- decision status (eligible, ineligible, or unresolved), plus owner and review date;
- an expiry trigger such as a new revision, changed terms, provider migration, material capability change, or security incident.
Do not copy provider claims into the ledger without their source. A URL alone is also fragile: mutable terms need a retrieved date and an archived or hashed copy.
A release that passes the access and permission review is still an untrusted software supply-chain input. Downloadable weights may arrive in an unsafe serialization format, a repository can move tags, and neither a checksum nor a license detects a backdoor in model behavior. Chapter 74 therefore begins where this chapter ends: pin the exact bytes, inspect the format, verify provenance, scan what can be scanned, and evaluate behavior before promotion.
Making capable weights widely available can lower barriers to research, local use, competition, adaptation, and privacy-preserving deployment. It can also make a capability difficult to recall and can reduce a developer's ability to enforce provider-side safeguards. The relevant policy comparison is the marginal benefit and risk of releasing a particular model's weights relative to realistic alternatives, including existing closed services and already available weights. After public consultation, the US National Telecommunications and Information Administration found the evidence insufficient in July 2024 either to justify current restrictions in general or to conclude that restrictions would never be appropriate; it recommended continued evidence collection and monitoring (National Telecommunications and Information Administration 2024). That is a reason to state assumptions and capability thresholds, not to treat either “open” or “closed” as a universal safety verdict.
From landscape to selection
The model landscape is a collection of changing release contracts, not a race with one winner. Separate artifacts, permissions, operations, and evidence; pin the version and date; then ask whether every hard requirement is confirmed. This approach survives changes in rankings because it records what the product can actually use and what the organization must operate.
The surrounding chapters carry the decision forward. Chapter 79 examines whether training and deployment rights are documented. Chapter 76 compares the costs of provider-operated and self-hosted paths. Chapter 81 turns requirements and evaluations into a shortlist, while Chapter 89 handles version drift after adoption.
Further reading
- Grattafiori & others, “The Llama 3 Herd of Models” (open-weights training report, 4D parallelism, public reliability data), 2024. arXiv:2407.21783Meta presents Llama 3, a herd of dense Transformer language models at 8B, 70B, and 405B parameters trained on 15T tokens, achieving quality comparable to GPT-4 across diverse tasks.
- OpenAI, “GPT-4 Technical Report” (no architecture, size, hardware, data, or method disclosed), 2023. cdn.openai.comOpenAI reports GPT-4 benchmark and safety evaluations while explicitly withholding architecture, model size, hardware, training compute, dataset construction, and training method.
- OpenAI, “Introducing gpt-oss” (gpt-oss-120b and gpt-oss-20b under Apache 2.0, the first open weights from OpenAI since GPT-2), 2025. github.comThe release provides Apache-2.0 weights, model documentation, tokenizer material, and reference inference implementations, but not its training corpus or full training pipeline.
- DeepSeek-AI, “DeepSeek-V3 Technical Report” (open-weights report, frontier-scale FP8, auxiliary-loss-free MoE balancing, multi-token prediction), 2024. arXiv:2412.19437Reports DeepSeek-V3, a 671B-parameter Mixture-of-Experts model with 37B active per token, trained on 14.8T tokens with fp8 matmuls and auxiliary-loss-free load balancing, rivaling closed models at low cost.
- OLMo Team, “2 OLMo 2 Furious” (fully open: training-stability recipe and two-stage curriculum), 2025. arXiv:2501.00656OLMo 2 documents and releases model weights, data mixtures, training and evaluation code, recipes, logs, and intermediate checkpoints for studying and attempting to reproduce its development.
- Biderman et al., “Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling” (research suite with public checkpoints and reconstructable dataloader order), 2023. proceedings.mlr.pressPythia is a suite of 16 LLMs from 70M to 12B parameters, each with 154 public checkpoints trained on the same data order, designed to study training dynamics and scaling.
- Open Source Initiative, “The Open Source AI Definition 1.0” (defines the freedoms and preferred form for Open Source AI), 2024. opensource.orgThe OSI definition treats Open Source AI as requiring use, study, modification, and sharing freedoms, with data information, code, and parameters available in the preferred form for modification.
- White et al., “The Model Openness Framework: Promoting Completeness and Openness for Reproducibility, Transparency, and Usability in Artificial Intelligence” (three self-reported completeness classes over released components and their stated licenses), 2024. arXiv:2403.13784The Model Openness Framework inventories released code, data, documentation, and model components; its classes do not independently validate the disclosure or prove successful reproduction.
- G7 Industry, Digital and Technology Ministers, “G7 Vision on AI Openness: Opportunities and Shared Language” (shared vocabulary for Open Source AI, Open Weights AI, and Weights Available AI), 2026. entreprises.gouv.frThe G7 vocabulary separates open-source systems, openly licensed weights and deployment code, and weight downloads governed by restrictive terms.
- Mitchell et al., “Model Cards for Model Reporting” (structured reporting of intended use, evaluation context, and limitations), 2019. doi.orgModel cards report intended uses, evaluation conditions, limitations, and performance across relevant conditions and groups.
- Google, “Gemma Terms of Use” (custom terms for Gemma 1–3, including use restrictions and downstream distribution duties), 2026. ai.google.devThe current Gemma terms govern use, modification, hosted service, and distribution of covered Gemma versions and derivative models, while directing Gemma 4 users to a separate license.
- National Telecommunications and Information Administration, “Dual-Use Foundation Models with Widely Available Model Weights Report” (evidence review of marginal benefits, risks, and policy options for widely available weights), 2024. ntia.govNTIA reviews benefits and risks of widely available model weights and recommends continued evidence collection rather than a general restriction at the time of the report.
Comments
Log in to comment