Stability AI Launches LLM for Portable Smart Devices
Plus: An AI-powered necklace, How to deploy LLMs in streaming applications.
Hello Engineering Leaders and AI Enthusiasts!
Welcome to the 117th edition of The AI Edge newsletter. This edition brings you Stability AI’s LLM for portable digital devices.
And a huge shoutout to our incredible readers. We appreciate you😊
In today’s edition:
🤖 Stability AI launches LLM for portable smart devices
✨Introducing Rewind Pendant, a personalized AI-powered wearable
🚀 StreamingLLM for efficient deployment of LLMs in streaming applications
📚 Knowledge Nugget: Vector database is not a separate database category by
Let’s go!
Stability AI launches LLM for portable smart devices
Stability AI has launched an experimental version of Stable LM 3B, its latest in the suite of high-performance generative AI solutions. At 3 billion parameters (vs. the 7 to 70 billion parameters typically used by the industry), Stable LM 3B is a compact language model designed to operate on portable digital devices like handhelds and laptops.
Its key is its smaller size and efficiency. But despite its size, it is highly performant– it outperforms the previous state-of-the-art 3B parameter language models and even some of the best open-source language models at the 7B parameter scale.
Why does this matter?
Stable LM 3B broadens the range of applications that are viable on the edge or on home PCs. It means that individuals and companies can now develop cutting-edge technologies with strong conversational capabilities (like creative writing assistance) while keeping costs low and performance high.
Introducing Rewind Pendant, a personalized AI-powered wearable
Rewind Pendant is a wearable necklace that captures what you say and hear in the real world and then transcribes, encrypts, and stores it entirely locally on your phone. It is like a wearable AI assistant who you can then ask any question using AI.
So no more forgetting what your spouse asked you to pick up at the grocery store, eh?😉
Why does this matter?
Meta announced AI camera glasses, Tab announced their AI pendant, and Humane announced the AI pin. It seems consumer wearables that can transform our daily lives may be the hottest new thing in AI.
It may also be a significant step forward in the intersection of AI and IoT and local AI processing.
StreamingLLM for efficient deployment of LLMs in streaming applications
Deploying LLMs in streaming applications is urgently needed but comes with challenges due to efficiency limitations and reduced performance with longer texts. Window attention provides a partial solution, but its performance plummets when initial tokens are excluded.
Recognizing the role of these tokens as “attention sinks", new research by Meta AI (and others) has introduced StreamingLLM– a simple and efficient framework that enables LLMs to handle unlimited texts without fine-tuning. By adding attention sinks with recent tokens, it can efficiently model texts of up to 4M tokens. It further shows that pre-training models with a dedicated sink token can improve the streaming performance.
Here’s an illustration of StreamingLLM vs. existing methods. It firstly decouples the LLM’s pre-training window size and its actual text generation length, paving the way for the streaming deployment of LLMs.
Why does this matter?
The ability to deploy LLMs for infinite-length inputs without sacrificing efficiency and performance opens up new possibilities and efficiencies in various AI applications.
Enjoying the daily updates?
Refer your pals to subscribe to our daily newsletter and get exclusive access to 400+ game-changing AI tools.
When you use the referral link above or the “Share” button on any post, you'll get the credit for any new subscribers. All you need to do is send the link via text or email or share it on social media with friends.
Knowledge Nugget: Vector database is not a separate database category
In this interesting article,
suggests that the distinction between specialized "vector databases" and other databases is blurring, particularly in the context of generative AI and Retrieval Augmented Generation (RAG) workloads.It discusses how the concept of vector databases is evolving and becoming integrated into various types of databases, including graph, relational, document, and key-value databases, as well as caches. Moreover, it discusses why it would make sense for incumbent database players to offer vector search and what might be its consequences.
Why does this matter?
As AI becomes more data-intensive and evolved, understanding how databases can adapt to handle these workloads is crucial. It can lead to more cost-effective AI implementations, making AI more accessible to a broader range of applications and businesses.
What Else Is Happening❗
🏛National Security Agency (NSA) to open an AI Security Center
This new entity will become the focal point for developing best practices, evaluation methodology, and risk frameworks to promote the secure adoption of new AI capabilities across the national security enterprise and the defense industrial base. (Link)
🏠Google Home’s new AI feature helps you create custom Routines
It is introducing a “help me script” feature in the script editor, which will generate code for you with the help of generative AI so you can easily create advanced, custom automations. (Link)
🤖Samsung to manufacture chips for AI chip startup Tenstorrent
Tenstorrent has selected Samsung's Foundry Design Service to manufacture versatile AI chiplets for various applications. CEO Jim Keller cited Samsung's commitment to semiconductor technology as an ideal fit for their vision of advancing RISC-V and AI. (Link)
💰Visa earmarks $100M to invest in generative AI companies
It plans to invest, through Visa Ventures, in companies developing generative AI technologies and applications that will impact the future of commerce and payments. (Link)
🧠Google Chromebooks’ new category Plus now has powerful AI capabilities
Google has introduced a new category, Chromebook Plus, which offers built-in Google apps and powerful AI capabilities. It also offers Google Photos Magic Eraser and Adobe Photoshop on web to boost productivity and creativity. (Link)
That's all for now!
Subscribe to The AI Edge and join the impressive list of readers that includes professionals from Moody’s, Vonage, Voya, WEHI, Cox, INSEAD, and other reputable organizations.
Thanks for reading, and see you tomorrow. 😊