Only a handful of experts can tackle HRM-Text, researchers claim, as they train a foundation model from scratch
apple training
| Source: Mastodon | Original article
Researchers train foundation model from scratch for $1,500.
Researchers at Sapient have made a significant breakthrough in training a foundation model from scratch, reportedly spending only around $1,500. This achievement challenges the conventional wisdom that training such models requires massive investments, often in the millions, and vast amounts of data. The key to their success lies in their development of HRM-Text, a brain-inspired foundation model that replaces standard Transformers with a more efficient architecture.
This development matters because it could democratize access to foundation models, allowing more organizations to develop their own AI capabilities without breaking the bank. Currently, the high costs and data requirements associated with training foundation models from scratch limit their adoption to mostly large tech companies. Sapient's innovation could change this landscape, enabling smaller enterprises and researchers to participate in the development of AI models.
As we watch this space, it will be interesting to see how Sapient's HRM-Text model performs in real-world applications and whether its efficiency and cost-effectiveness can be replicated by others. Additionally, the potential impact on the AI research community and the broader industry will be worth monitoring, as this breakthrough could pave the way for more diverse and innovative AI developments.
Sources
Back to AIPULSEN