The Collapse of the Closed AI Moat and the Supply-Chain Paradox of Unverifiable Weights

Two years ago, frontier artificial intelligence was the private property of a handful of firms. Astronomical training costs, proprietary data, and weights hidden behind an API formed a triple moat that looked impossible for latecomers to cross. That moat is draining faster than almost anyone predicted. When DeepSeek released a model with reinforcement-learning-trained reasoning capability as downloadable weights, the very thing closed labs had been selling at a premium — the ability to deliberate, to reason step by step — became a public good anyone could pull down and run on their own hardware. Projects like Llamafile, which package an entire model into a single executable that runs across operating systems without setup, dismantled even the distribution moat. A capable model now fits on a thumb drive and travels offline, hand to hand.

How Fast the Moat Drains

The real shift is not that there are more free models. It is that the axis of competition has moved. The window during which a closed model holds a clear performance lead keeps shrinking, and the cadence at which open models close that gap has compressed to a matter of quarters. Once reasoning — the most expensive differentiator of all — went open, value migrated rapidly away from the model itself and toward how an organization integrates, fine-tunes, and operates it. For enterprises, the ability to run inference inside their own infrastructure rather than piping sensitive data to an external endpoint stopped being a cost question and became a question of regulatory compliance and sovereignty. The moat the closed camp built is increasingly revealed as a temporary barricade that buys time, not a durable technical advantage.

But this is precisely where the shadow of openness appears. The claim that anyone can download and run a set of weights is synonymous with the admission that the recipient cannot meaningfully verify where those weights came from or what they learned. With software you can read the source, reproduce the build, and trace dependencies. A blob of billions of floating-point numbers permits none of that transparency. We can observe a model's outputs, but we cannot see what sleeps inside it.

The Warning in 250 Samples

Recent research turned that unease into a concrete number. Regardless of total model size, a few hundred poisoned training samples can be enough to implant a backdoor that makes the model behave maliciously whenever a specific trigger appears in its input. The unsettling part is that the absolute count of samples needed does not scale up with model size. The intuition that bigger, more capable models are inherently safer simply breaks. A pinch of toxic data folded into a vast training corpus does not get statistically diluted into harmlessness; it survives as a latent switch that fires only under a chosen condition.

When this finding meets the open-distribution trend, the problem descends from the abstract into the operational. The common practice of judging a model by its star count and download numbers on a hub, then wiring its weights into internal systems, is no different from shipping an unverified binary of unknown provenance straight to production. Unable to reproduce the training run or prove the innocence of the data, we are effectively outsourcing trust. To the same degree that openness democratized competition, it democratized the supply-chain attack surface.

The question the open-source path poses, then, is not the tired binary of open versus closed. In a world where openness has already won, the question is how we are supposed to trust artificial intelligence we cannot verify. Without infrastructure to sign the provenance of weights, trace the lineage of training data, and audit models at the level of behavior before deployment, openness's victory over the closed moat risks becoming a sandcastle built atop a deeper trap: verification debt. If open is the way forward, it becomes a safe way only when we lay down, alongside it, the infrastructure to prove trust in code.