A Tiny Bridge Lets Small and Large Models Talk Without Words

On ARC-AGI 3, a benchmark where GPT-6 Astra is approaching a perfect score, a 4B-parameter model small enough to run on a phone is now posting competitive numbers. More precisely, it is a pairing of 4B Qwen-3.5 and 753B GLM 5.2 — the large model stays silent in the cloud, the small model writes the answer on-device, and a mechanism called a "bridge" carries information between them.

The team behind it, Mostik, was founded only four months ago. Among its 15 members are 12 PhDs, and its chief scientist is Stanislav Smirnov, winner of the 2010 Fields Medal. Technical details remain under wraps — "the competition is still ongoing" — but the core finding is striking enough: models communicating through text is shockingly inefficient.

"Finding a mathematical common ground between two AI models is actually very difficult." — Stanislav Smirnov

The Imbalance of Two Megabytes and Seventeen Bits

When a large model generates one token, it builds up hundreds of hidden vectors — roughly two megabytes of internal state. What comes out? A single token chosen from a 150,000-word vocabulary — seventeen bits. Two megabytes are computed and discarded, and every multi-model system today — coding sub-agents, model committees, routing and scheduling — is built on top of those seventeen bits.

Recent interpretability research confirms that the discarded state contains the core of deep computation. An ICLR 2026 paper shows that several tokens before Qwen-3 writes the word "accountant," its internal representation already carries that word — which is precisely how it correctly chooses the article "an" over "a" ahead of time. Anthropic has also observed a "J-space" structure in Claude 3.5 Haiku: the model maintains a set of unexpressed concepts, internal reasoning that never appears in the output.

The Promise of the Bridge: From Cost Cuts to Distillation

Mostik's approach is straightforward: train a small bridge module that directly "translates" the sender model's hidden states to the receiver, with both model weights fully frozen and no text generated at all. The large model only does cheap prefill, leaving expensive decoding generation entirely to the small model — bringing inference cost down to roughly one-twentieth of the original. The 4B model closes 50% of the performance gap with the 753B model, boosting its own accuracy by 25%, and up to 2x on harder problem subsets.

More importantly, GLM and Qwen were trained by different teams on different data — yet the bridge still works. This suggests some universal structure naturally emerges during model training. And as Smirnov notes, we don't even have the right mathematical language to describe it yet. Next, Mostik is exploring using bridges for distillation, specialization, and as a new window for safety observability.

See you tomorrow.