Hugo Vergnes trains 3.8B LLM to 0.384 CORE for $998
training
| Source: HN | Original article
A 3.8‑billion‑parameter language model was pretrained from scratch on 65 billion tokens, reaching 0.384 CORE in 43 hours at a cost of $998.
An independent developer has demonstrated that a modern, 3.8‑billion‑parameter language model can be trained from scratch for under $1,000. Hugo Vergnes used rented GPU instances to run 65 billion token pre‑training steps, reaching a CORE benchmark score of 0.384 in just 43 hours. The resulting model, dubbed “little‑lm 3.8B,” reportedly outperforms OpenAI’s older GPT‑2 on standard evaluation suites.
The achievement matters because it challenges the prevailing assumption that multi‑billion‑parameter models require massive cloud budgets and weeks of compute. By compressing the entire pre‑training pipeline into a single evening’s work and a sub‑$1,000 expense, Vergnes shows that high‑quality LLMs are becoming accessible to individual researchers and small teams. The low cost also lowers the barrier for open‑weight experiments, potentially accelerating community‑driven innovation and diversification of model architectures.
What to watch next is whether the little‑lm model will be released publicly and how the broader AI community responds. If the weights and training recipe are shared, other developers may replicate or extend the approach, creating a new class of affordable, open LLMs. Benchmark leaders will likely reassess performance baselines, especially as cost‑effective models begin to eclipse legacy systems like GPT‑2. Finally, the experiment may inspire cloud providers to offer more granular pricing or specialized instances aimed at hobbyist‑scale training, further democratizing large‑scale AI research.
Sources
Back to AIPULSEN