GitHub Unveils slvDev/esp32-ai Innovation
chips
| Source: Mastodon | Original article
AI model runs on $8 microcontroller. GitHub hosts the project, a 28.9M parameter LLM.
A significant breakthrough has been achieved in running large language models on tiny, low-cost hardware. Developer slvDev has successfully run a 28.9M parameter LLM on an $8 microcontroller, the ESP32-S3. This feat is impressive given the chip's small size and low cost. The model generates text on the chip itself, without sending any data to a server, and can write each word to a small screen at a rate of roughly 9 tokens per second.
This development matters because it demonstrates the potential for AI to be run on extremely low-cost, low-power devices, which could have significant implications for a wide range of applications, from IoT devices to edge computing. The use of Per-Layer Embeddings, a technique that allows for more efficient use of model parameters, is a key factor in achieving this breakthrough.
As this project continues to evolve, it will be worth watching to see how the community responds and whether others are able to build on this achievement. The fact that the project is open-source and hosted on GitHub means that developers can access and contribute to the code, which could lead to further innovations and advancements in this area.
Sources
Back to AIPULSEN