Introducing open-1b: the first model you don’t have to trust

We’re releasing our language model open-1b, along with something that no lab has shipped before: proof of how it was trained.

We’re providing a public fingerprint of every step that produced the model, and a tool that lets anyone replay those steps on their own hardware. We’re inviting the world to audit the training to confirm, bit-for-bit, that each published step is the real one.

This process is functionally identical to decentralized training, but because the initial training has been done in a centralized way, it doesn’t incur the same huge upfront costs or the communication issues. Instead of trying to shift model creation to the decentralized nodes, we’ve shifted model verification to them, with the same net benefits.

open-1b represents a milestone on the path toward verifiable AI, a goal that we believe is absolutely necessary for the future of intelligence. At a time when even the companies that control these models are warning us of existential risks, auditable models represent the singular defense against a black box future for AI.

AI mistrust is a growing problem

The most capable, important and widely-used models in the world are closed. They are controlled by a handful of companies, accessible through an API. There is no way to see their architecture, their data, or the way they were produced. You can’t know what preferences shaped them, what was filtered out, or what was quietly added.

Critics of these companies – and, often, the companies themselves – warn the public of the risk posed by the speed at which these models are improving. But the only solution they’ve offered so far is the suggestion that only they be trusted enough to be allowed to develop this secretive and arcane technology. This solution is problematic, of course, as the world becomes increasingly dependent on the models.

It is becoming common for organizations to run these models with tens or even hundreds of thousands of AI agents at once. This represents years of human work reduced to hours. The agents write code, move money, negotiate, file paperwork, run experiments, and shape what billions of people consume daily.

You can’t check a model by asking it questions. You can’t check it by reading its system card. If it was trained to behave one way under observation and another way at scale, or if a small poisoned slice of its data taught it to favor a certain client or country, there is no test you can run from the outside that would tell you.

Likewise, you can’t trust an agent to monitor for these problems on your behalf. Agents deployed to supervise poisoned models have been observed taking on the same biases. You can’t trust a biased model to monitor other biased models.

The only true defense against a fleet of models you can’t trust is a fleet of models you can. Watchdog models, built to be auditable.

What auditable means

Model transparency is a ladder.

An open‑weight model hands you a finished cake. An open‑recipe model hands you the cake and the recipe card.

These are an important step, but even a complete recipe leaves a gap: nothing connects the recipe to the weights. Research has shown that a backdoor can be planted with just a handful of poisoned examples out of hundreds of billions, invisible to any after-the-fact inspection of the dataset.

An auditable model, like open-1b, hands you the cake, the recipe, and a kitchen camera anyone can review.

This “kitchen camera” unlocks some important applications that are impossible without auditability:

• Proves every single datapoint trained into a model, allowing biases to be modelled. Reproducibility ensures that each training step can be audited bit-for-bit, so no single datapoint can be secretly or silently added. This enables tools like our dataset search application, which can be used to search through the verifiably-complete training data. Labs could use these techniques to allow consumers to know what parts of their own data – their likeness, or content, for instance – are being used to train a model, similar to how websites provide consumers with their data on request today.
• Allows the creation of models that monitor other, biased models. Because their training and inference can be audited, and their biases can be modelled, we can trust that agents built on verifiable models can act as watchdogs over vast swarms of agents built on the closed models used today, from weight initialization all the way through to the monitoring itself.
• Shows that AI training does not have to be reckless and obscure. Training a model can simply be a repeatable and verifiable compression process from raw data to knowledge worker, rather than the dangerous creation of a new mind.

How we built it

The reason that no auditable model has been released isn’t due to politics or lack of demand. It’s a technical problem, and it isn’t trivial.

Floating-point arithmetic isn’t associative. Adding the same numbers in a different order gives you a slightly different bit-level result. This matters, because GPUs, CPUs, and even different GPU generations all add in different orders. Existing “deterministic mode” settings only make a run repeatable on the same hardware.

This means that even a lab that publishes everything and acts in perfect good faith produces a training run that nobody else can reproduce exactly. If it can’t be reproduced, it can’t be verified.

Our verifiable AI infrastructure closes that gap. RepOps, our library of reproducible operations, and REE, our reproducible execution environment, make matrix multiply, normalization, and gradient reduction produce identical bits whether they run on a consumer NVIDIA card, an x86 or ARM CPU, or Apple M processor.

Because the run is reproducible across hardware, it is checkable across hardware. This stack can be used to train any model auditably, and open-1b is the first model to prove it.

Three things need to be true for a training run to be auditable. None of these are true of any framework before open-1b.

• Every operation must be reproducible across hardware. RepOps, our library of reproducible operations, fixes a single reduction order for every sum, forces one multiply‑add convention, flushes subnormals identically everywhere, and drives all randomness from a counter‑based generator that gives the same draw at the same index regardless of how many GPUs are running.
• The data must be independent of the cluster. Conventional data loaders shuffle differently depending on how many workers you have. We define the training data as a single canonical stream that depends only on the seed and the corpus — not on world size, rank, or node count. A cluster of 48 GPUs and a single laptop enumerate exactly the same windows in exactly the same order.
• Communication must be replayable. Gradient reduction across a cluster normally routes through vendor collectives whose summation order depends on the topology. We replaced them with deterministic collectives whose exact pairing order a single device can reproduce sequentially, so the audit harness can replay the forward and backward pass of every GPU in the cluster, one at a time, and land on the same gradient the cluster did.

Additionally, every step is anchored by a canonical hash of the data batch, the parameters, the optimizer state, and the gradients. Those hashes are the public record. An audit is simple: pick a step, load a checkpoint before that step, replay the step, hash the result, compare.

Unfortunately, audibility is not a property that can be bolted on afterward. The verification tool we’ve launched for open-1b today can’t be used to audit Llama, or GPT. Provability has to be designed into the training run from the beginning, the first step. open-1b proves that this can be done in the real world, at a real scale, on consumer hardware.

We have proved that it can be done, we are setting the standard for how, and we can scale it to larger models today.

Benchmarking verifiability

Our goal is clear: prove that auditable training of LLMs is possible (far beyond open weight and open recipe). We’re inviting judgement of open-1b in that respect, rather than model to model comparisons against larger, or opaquely-trained, benchmark leaders.

The model itself is a 24‑layer decoder‑only transformer — 1.61B parameters, 1.08B non‑embedding — trained on 400B tokens from four permissively licensed public sources (DCLM‑Baseline, FineWeb‑Edu, The Stack v2, Proof‑Pile‑2).

The reproducible runtime sustains around 5% model FLOPs utilization, roughly 5× slower than an optimized PyTorch stack on the same hardware and the same recipe. The 400B‑token run took 27.8 days of active training on 48 H100s, about 29.5 calendar days end to end.

On the OLMo 2 evaluation suite, open-1b scores 25.4 versus 31.9.

Audit it yourself

We built the audit to be collective, the way open source builds software. Verifying all 80,957 steps on one laptop isn’t practical, but performed over thousands of distributed machines it’s a different story.

Download the audit harness, pick any step of the run, and replay it. It runs on NVIDIA GPUs, x86 and ARM CPUs, and natively on Apple Silicon. When your result matches the published hash, your verification is recorded and credited on-chain in the public ledger at [LINK]. This ledger assembles the individual checks into a single collective statement: this model was trained exactly as declared and has been verified over decentralized infrastructure.

There’s no payment, no token, and no points on a leaderboard. The reward is your name on the immutable record of the first ever fully audited training run. In a future that has achieved democratization of intelligence, that is an important place in history.

Here’s what’s released today:

Model. open‑1b base and instruction‑tuned checkpoints.

Evidence pack. Every intermediate checkpoint at 100‑step intervals across the full run, the complete pretraining dataset and pipeline, the training and evaluation code and configs, and the per‑step state hashes.

Recipe. Our native int8 quantization‑aware pretraining recipe, with the stability fixes we found along the way documented in the paper.

Audit harness. A downloadable tool that replays any training step on commodity hardware, and a public site tracking which parts of the run have been verified and by whom.

Dataset search tool. A tool that allows anyone to search the entire data training set to know exactly what the model has been trained on.

The paper. Read it here.

What’s next

We are not solving audibility as a gimmick or academic exercise. The concept is bigger than that, and it’s bigger than our own 1.6B model. As calls for pausing AI development grow louder each day, largely due to the unknowable, untestable advancement of closed models, auditable training represents an actionable solution. We strongly believe it to be the right one.

We think of open‑1b as a first step and a floor: a lab said how a model was made, and the world can check. Let’s work together to make it the expectation.

[Audit the Training]