Llama.cpp Lets You Run Any GGUF Model Natively on a Jailbroken PS5
gemma gpu llama mistral qwen
| Source: HN | Original article
Developers have ported llama.cpp to a jailbroken PlayStation 5, allowing the console to run any GGUF model natively.
A developer has turned a jail‑broken PlayStation 5 into a private language‑model server by porting the open‑source llama.cpp inference engine to the console. The new project, PS5LM, runs any GGUF‑format model directly on the PS5’s Zen 2 CPU and its custom GPU via Vulkan, eliminating the need for a PC, cloud service or external account.
The port builds on earlier PS5 LLM experiments that required hand‑written GPU kernels for a single model. By integrating the full llama.cpp codebase, PS5LM inherits support for every architecture and quantisation scheme the engine offers – from 1‑bit to 8‑bit, and models such as Qwen, Llama, Mistral and Gemma. Benchmarks on a PS5 Slim show a Qwen 3.5 0.8 B model (Q4_K_M) generating roughly 207 tokens per second, a 9 B model at 55 t/s, and a 27 B model quantised to UD‑Q2_K_XL delivering about 20 t/s with a 64 k context while using 11.4 GB of the shared GPU memory.
The development matters because it demonstrates that high‑performance LLM inference can be achieved on mainstream consumer hardware, expanding the pool of devices that can host private AI services. By serving OpenAI‑style and Anthropic‑style APIs over a local network, PS5LM offers a privacy‑preserving alternative to cloud‑based inference and showcases the untapped compute potential of gaming consoles for edge AI workloads.
Watchers should monitor whether the community refines the Vulkan implementation for higher throughput, expands support to newer console revisions, or adapts the approach to other popular platforms such as the Xbox Series X. Security implications of running AI services on jail‑broken hardware may also attract attention from both hobbyists and console manufacturers as the line between gaming and AI computation continues to blur.
Sources
Back to AIPULSEN