Tree‑Ising: Bit‑Level Edits Repair Language Models
agents qwen reasoning
| Source: HF Papers | Original article
MIT researchers used weight‑space programming to let their GPT‑6 Astra agent read and edit the weights of Qwen3.5‑0.8B, fixing its reasoning loops without gradients in 158 turns and 89.7 M tokens.
A team of researchers has unveiled **Tree‑Ising**, a technique that repairs large language models by tweaking only a handful of bits in the model’s BF16 weight matrix. The method reframes a faulty reasoning loop as a sparse discrete optimisation problem, solving an Ising or QUBO formulation directly on the weights. Forward‑pass statistics define an objective that weakens the looping direction, strengthens the correct conclusion and preserves the layer’s output on productive tokens.
Applied to the 0.8 billion‑parameter Qwen 3.5 model, Tree‑Ising lifted the MATH‑500 benchmark accuracy from a meagre 7.9 % to **54.6 %**, surpassing the earlier MIT “weight‑space programming” approach that used a GPT‑6 Astra agent to achieve 51.7 % after 158 agent turns, 89.7 million tokens and 50 forward passes. Crucially, Tree‑Ising accomplishes the repair with **no gradient updates and no LLM‑token cost**, editing only a sparse set of BF16 parameters.
The breakthrough matters because it offers a low‑cost, low‑compute pathway to fix specific reasoning failures that would otherwise require costly full‑model fine‑tuning or retraining. By operating at the bit level, the approach can be applied to models already deployed in production, potentially extending their useful lifespan and improving reliability on tasks such as mathematics or code generation where looping behaviours hit token limits.
The authors will soon release the implementation on GitHub, opening the door for broader testing. Observers will watch for extensions to larger models, integration with existing weight‑space programming tools, and real‑world deployments that demonstrate whether sparse bit‑level edits can become a standard part of the AI maintenance toolkit.
Sources
Back to AIPULSEN