HN Demonstrates 80B Qwen Model Running in 4.3 GB of RAM on Mac, and 35B Model on iPhone
apple inference qwen
| Source: HN | Original article
Large AI models now run on Macs and iPhones with limited RAM. Qwen models can be run locally on various devices.
A significant breakthrough has been achieved in running large AI models on consumer devices. It is now possible to run an 80B Qwen model in just 4.3 GB of RAM on a Mac, and a 35B model on an iPhone. This development matters because it demonstrates the rapid progress being made in optimizing AI models for local execution, reducing the need for cloud services and enabling more private and efficient use of AI.
As we previously reported, open AI models like Qwen have been gaining attention for their potential to democratize access to AI technology. The ability to run these models on everyday devices like Macs and iPhones takes this trend a step further, opening up new possibilities for developers and users alike.
What to watch next is how these advancements will be utilized in real-world applications, and whether they will pave the way for even more powerful AI models to be run on consumer hardware. With the ongoing evolution of AI technology, it will be interesting to see how this development impacts the broader AI landscape.
Sources
Back to AIPULSEN