Deploy State-of-the-Art LLMs on Your Phone
Plus: AI detects cancer. Amazon building a new LLM.
Hello, Engineering Leaders and AI enthusiasts,
Welcome to the ninth edition of The AI Edge newsletter. In today’s edition, we bring you LLM that’s deployable on your smartphone. Thank you everyone who is reading this.
In today’s edition:
📱 MLC LLM lets you run language models on your smartphone.
🔬 AI can now detect cancer.
🚀 EVA relational databases lead to 10-100x faster AI pipelines.
🗣️ Amazon is developing a new LLM to power Alexa.
⚙️ How ChatGPT works technically by
Let’s go
MLC LLM: Deploy language models like GPT/Llama natively on your phone and laptop.
MLC LLM offers a versatile platform for deploying language models on various hardware and native apps (including iPhones), providing an efficient framework for custom performance optimization. They use machine learning compilation (MLC) to effectively deploy AI models, utilizing open-source tools like HuggingFace and Google tokenizers, and LLMs such as Llama, Vicuna, Dolly. The main workflow relies on the developing Apache TVM Unity platform
Why does this matter?
The platform has the potential to allow everyone to deploy AI models locally on browsers, laptops, and phones. The simpler and (supposedly) cheaper deployment could lead to a fresh wave of AI assistants for specialized use cases, among other innovations.
AI model outperforms traditional methods in identifying cancerous nodules
An artificial intelligence model developed by experts at the Royal Marsden NHS foundation trust, the Institute of Cancer Research, London, and Imperial College London can accurately identify cancer, potentially speeding up diagnosis and treatment. The algorithm, which analyzes CT scans to determine if abnormal growths are cancerous, reportedly performs more efficiently and effectively than current methods.
According to a study published in the Lancet's eBioMedicine journal, the AI model identified each nodule's risk of cancer with an area under the curve (AUC) of 0.87, outperforming the Brock score (0.67) and performing comparably with the Herder score (0.83).
Why does this matter?
The AI tool may help doctors make faster decisions about patients with abnormal growths that are currently deemed medium-risk. The model, which is still in its early stages, will require further testing before it can be introduced in healthcare systems. However, researchers hope the AI tool will eventually speed up cancer detection by fast-tracking patients to treatment.
EVA: AI Relational database system for 10-100X faster AI pipeline
Lately trending, EVA is a database application support system that accelerates AI pipelines for analyzing structured (tables, feature vectors) and unstructured data (videos, podcasts, PDFs, etc.) using deep learning models. It optimizes performance with function caching, sampling, and cost-based predicate reordering. EVA features an AI-focused SQL-like query language and offers various models for tasks like image classification, object detection, and sentiment analysis. It is written in Python and available under the Apache license.
Why does this matter?
The relational database could significantly reduce the threshold and cost of building simple AI-powered apps using short SQL-like queries. Since the user can compose multiple models in a single query, it can lead to 10-100 times faster AI pipelines.
Amazon boosting Alexa's intelligence & capabilities with an improved LLM
Amazon is working on an upgraded LLM that could make Alexa more capable. The new LLM will help Amazon in its goal of creating “the world’s best personal assistant.” However, the CEO -Andy Jassy, acknowledged this would be difficult to achieve across many domains.
Why does this matter?
Interestingly, Amazon isn't the only big player investing in LLMs, as Apple, Google, Alphabet, Microsoft, and Meta are all racing to keep up with the fast-paced AI space. However, the competition should only lead to cheaper and better AI products for general users for the foreseeable future.
Knowledge Nugget: How ChatGPT works technically?
In this video,
breaks the architecture of ChatGPT down to its bones. He explains what is LLM, how ChatGPT was trained, what were the data sources, how it carries a conversation and more.Why does this matter?
ChatGPT played a significant role in kickstarting the AI revolution. You should know what is the underlying framework powering it. The future AI implementations are only going to be slightly sophisticated/modified versions of ChatGPT’s technical framework.
What Else Is Happening
![[video-to-gif output image]](https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff208badc-42ea-481f-8e30-57beb8e726d6_600x338.gif "[video-to-gif output image]")
🔍 Video object tracking and segmentation with Track Anything. (Link)
🩺 AI is better than physicians when it comes to dealing with patients. (Link)
💬 Stability AI releases it's own chatbot: StableVicuna. (Link)
💡 ChatGPT is back in action in Italy. (Link)
🎖️ Demo released how AI can be used in the military and warfare. (Link)
Trending Tools
Guidde AI: Generative AI platform for teams to deliver know-how 11x faster to customers or employees.
Landing AI: Generative AI for unique landing pages with 29 themes, 31 languages, selling copy, logo & illustrations.
Style AI: Levi, your new AI assistant for fully customized websites with custom requests. Free trial, no signup.
CraftAI: GPT-powered Admin Panel Generator for custom interfaces in 5 minutes. Codeless and free.
Dubly.io - Video translator: Web app using AI to translate and dub videos automatically in any language. User-friendly.
FlexGPT: Unlimited GPT-4 access with long-term memory, internet access and per-use billing. No subscriptions.
Score: Chrome extension using AI to find hidden deals and affordable alternatives while shopping.
Please CLI: OSS AI helper script to create CLI commands. Works with OpenAI API key.
That's all for now!
If you are new to ‘The AI Edge’ newsletter. Subscribe to receive the ‘Ultimate AI tools and ChatGPT Prompt guide’ specifically designed for Engineering Leaders and AI enthusiasts.
Thanks for reading, and see you tomorrow.