Recurrent Looped Transformer (RLT) Eliminates Forgetting
| Source: Mastodon | Original article
The Recurrent Looped Transformer (RLT) lets language models retain context across 48 layers, removing the need to forget.
A technical report released on 12 September 2026 by Princeton researcher Yifan Zhang introduces the **Recurrent Looped Transformer (RLT)**, a new architecture that claims to give language models “infinite reasoning depth.” In a companion GitHub repository posted the same day, Zhang demonstrates how the model closes the feedback gap that exists in most decoder‑only LLMs: instead of passing only cached attention keys and values from token t to token t + 1, the RLT feeds the full hidden state from the last layer of one token back into the first layer of the next.
The design splits memory from computation. A causal encoder builds a global key‑value store once, then a recurrent decoder repeatedly runs the same block of layers, carrying the complete state forward until a stopping criterion is met. By looping the same stack, the architecture can apply arbitrarily many reasoning steps without adding new parameters, addressing the “forgetting” problem that limits conventional transformers.
The claim of “infinite reasoning depth” has sparked rapid interest across the AI community. It builds on the same recurrent‑depth ideas we highlighted in our September 2 coverage of OpenAI’s Astra, which also reuses hidden states across passes. If the RLT delivers deeper, more consistent reasoning with modest compute, it could reshape how future LLMs are built, offering efficiency gains for both research prototypes and commercial deployments. At the same time, the ability to run unbounded loops raises safety questions about controllability and resource consumption—issues already surfacing in debates over AI scaling.
What to watch next: peer‑review validation of the technical report, benchmark results comparing RLT‑based models with standard transformers, and responses from major labs on whether to adopt the looped approach. The community is also likely to scrutinise the safety implications, potentially prompting new guidelines for architectures that permit unbounded recurrent depth.
Sources
Back to AIPULSEN