Alex Cheema (@alexocheema) on X
llama microsoft qwen
| Source: Mastodon | Original article
Alex Cheema, co‑founder of the AI‑focused start‑up EXO Labs, used his X account on 1 April to publish a compact but potent bibliography of the latest tools for running large language models (LLMs) locally. The post links to Ollama’s new MLX backend, Microsoft’s BitNet B1.58 2‑billion‑parameter 4‑tensor model, and the TurboQuant research paper, among other sources. Cheema framed the list as a “quick reference for tracking lightweight local LLMs and quantisation techniques”.
The curation arrives at a moment when the AI community is racing to shrink model footprints without sacrificing performance. Ollama’s MLX backend promises to harness Apple’s silicon‑optimised MLX library, enabling faster inference on Mac‑based hardware—a platform Cheema has repeatedly showcased, from his four‑Mac‑Mini M4 cluster that runs Qwen 2.5 Coder 32B at 18 tokens s⁻¹ to two‑Mac Studio rigs that host DeepSeek R1. Microsoft’s BitNet, meanwhile, is a publicly released 2‑billion‑parameter model that demonstrates competitive quality at a fraction of the compute cost of larger systems. TurboQuant, a recent quantisation method, claims to halve memory usage while preserving accuracy, a claim that could make 4‑bit inference viable on consumer laptops.
For Nordic developers and enterprises, the shared resources lower the barrier to experimenting with on‑premise AI, reducing reliance on costly cloud credits and easing data‑privacy concerns. The links also signal that the ecosystem around open‑source quantisation and hardware‑aware backends is coalescing, a trend that could accelerate the adoption of AI in sectors ranging from fintech to media production across the region.
What to watch next: Ollama is expected to release a stable MLX‑based client later this quarter, and Microsoft has hinted at a follow‑up to BitNet with a 4‑billion‑parameter variant. The TurboQuant paper is already sparking forks on GitHub; early benchmarks from EXO Labs’ Mac‑Mini clusters will likely surface on X and in upcoming conference talks. Monitoring these rollouts will reveal how quickly truly local, high‑quality LLMs become a mainstream tool for Nordic AI innovators.
Sources
Back to AIPULSEN