Puro-2B: Poor Lab's Qwen2‑1.5B Trained on RTX 5090 at $5090
open-source qwen training
| Source: HF Papers | Original article
Researchers at Poor Lab trained the Qwen2‑1.5B model on a single RTX 5090 GPU for under $5,090, demonstrating a low‑cost path for language model pretraining.
A team calling itself Poor Lab has released a new research paper showing that a 1.5 billion‑parameter language model, built on the Qwen architecture, can be trained from scratch on consumer‑grade hardware for less than US $5,090. The authors used a single Nvidia RTX 5090 GPU, running FP8 precision, to process up to 1.4 trillion tokens and produce a family of “Puro‑2B” models. The work, posted on arXiv as 2608.27370, details the training recipe and the hardware‑efficiency tricks that keep the total expense comparable to the price of the GPU itself.
The result matters because pre‑training large language models has become synonymous with multi‑million‑dollar budgets, effectively excluding most academic labs and open‑source developers. By demonstrating a viable sub‑$5k pathway, Poor Lab lowers the entry barrier and could accelerate experimentation, model diversification, and community‑driven innovation. The approach also showcases the practical value of FP8 arithmetic for large‑scale training, hinting that future cost reductions may rely more on precision engineering than on raw compute power.
The next steps to watch include whether other research groups adopt the Puro‑2B recipe, and if the community releases publicly available checkpoints or downstream tools built on the model. Follow‑up work may explore scaling the method to larger parameter counts, integrating the technique into existing open‑source training frameworks, or applying it to domain‑specific corpora. If the approach gains traction, it could reshape the economics of LLM development and broaden participation beyond well‑funded corporate labs.
Sources
Back to AIPULSEN