GitHub - AlexsJones/llmfit: Hundreds of models & providers. One command to find what runs on your hardware.
| Source: Mastodon | Original article
GitHub has seen a fresh addition to the toolbox for developers who want to run large language models (LLMs) locally: llmfit, a command‑line utility that scans a machine’s RAM, CPU cores and GPU VRAM, then returns a shortlist of models that will actually fit. Created by Alex Jones, the open‑source project aggregates metadata for hundreds of models across providers such as Meta, Mistral and Cohere, and can be queried by name, size or use‑case with simple commands like `llmfit search 'llama 8b'` or `llmfit recommend --use-case coding --limit 3`. The tool also outputs JSON for easy integration into scripts or CI pipelines.
The relevance of llmfit lies in the accelerating shift toward on‑device AI. As we reported on April 18, leading LLMs have become “nearly indistinguishable” in performance, prompting developers to experiment with smaller, self‑hosted variants to cut cloud costs and protect data privacy. Yet the sheer number of open‑source models—ranging from 1 B‑parameter whisper‑size nets to 70 B‑parameter behemoths—makes manual selection a guessing game. By automating the compatibility check, llmfit lowers the barrier for hobbyists, startups and enterprises that lack deep‑learning expertise, potentially widening adoption of edge AI in the Nordics and beyond.
Watch for community contributions that expand the model catalog and add support for emerging hardware accelerators such as Intel’s Gaudi or Apple’s M‑series chips. The author hints at future integration with package managers like crates.io, which could enable one‑click installation of the recommended model and its runtime. If the tool gains traction, we may see IDE plugins or CI extensions that automatically pin the optimal model for a given build, turning llmfit from a convenience script into a core piece of the local‑LLM workflow.
Sources
Back to AIPULSEN