Meta's New Approach to Faster, Smarter AI
Plus: Anthropic launches an iOS app and a new plan for teams, Google's AI advancements urged Microsoft's billion-dollar OpenAI investment in 2019.
Hello Engineering Leaders and AI Enthusiasts!
Welcome to the 266th edition of The AI Edge newsletter. This edition features Meta's new approach to faster and better AI: multi-token prediction.
And a huge shoutout to our amazing readers. We appreciate you😊
In today’s edition:
🤖 Better and faster LLMs via multi-token prediction: New research
and
📱 Anthropic launches an iOS app and a new plan for teams
💸 Google's AI advancements urged Microsoft's billion-dollar OpenAI investment
📚 Knowledge Nugget: AI leaderboards are no longer useful. It's time to switch to Pareto curves by
Let’s go!
Better and faster LLMs via multi-token prediction: New research
New research, apparently from Meta, has proposed a novel approach to training language models (LMs). It suggests that training LMs to predict multiple future tokens at once instead of predicting only the next token in a sequence results in higher sample efficiency. The architecture is simple, with no train time or memory overhead.
Figure: Overview of multi-token prediction
The research also provides experimental evidence that this training paradigm is increasingly useful for larger models and in particular, shows strong improvements for code tasks. Multi-token prediction also enables self-speculative decoding, making models up to 3 times faster at inference time across a wide range of batch sizes.
Why does it matter?
LLMs such as GPT and Llama rely on next-token prediction. Despite their recent impressive achievements, next-token prediction remains an inefficient way of acquiring language, world knowledge, and reasoning capabilities. It latches on local patterns and overlooks “hard” decisions.
Perhaps, multi-token prediction could bring a shift in how LMs learn. It could equip LLMs with deeper understanding and complex problem-solving capabilities. (or Meta just wasted their compute.)
Anthropic launches an iOS app and a new plan for teams
Anthropic, the creator of the Claude 3 AI models, released a new iOS app named Claude. The app enables users to access AI models, chat with them, and analyze images by uploading them.
Anthropic also introduced a paid team plan, offering enhanced features like more chat queries and admin control for groups of five or more. The app is free for all users of Claude AI models, including free users, Claude Pro subscribers, and team plan members. The company will also roll out an Android version soon.
Why does it matter?
Though a little late with its mobile app, Anthropic has caught up with its competitors like OpenAI and Google, who have apps running for quite a while. The company decided to offer an app version because many users have been accessing its AI models through the web.
Google's AI advancements may have urged Microsoft's billion-dollar OpenAI investment
Internal emails have revealed that Microsoft invested $1 billion in OpenAI in 2019 out of fear that Google was significantly ahead in its AI efforts.
Microsoft CTO Kevin Scott sent a lengthy email to CEO Satya Nadella and Bill Gates stating Google’s AI-powered “auto complete in Gmail” was getting “scarily good” and added that Microsoft was years behind in terms of ML scale.
The emails, with the subject line “Thoughts on OpenAI,” were made public on Tuesday as part of the Department of Justice's antitrust case against Google. A large section of Scott's email was redacted. Check out the email here.
Why does it matter?
While some might call it paranoia, the well-timed move has undeniably paid off– the initial $1 billion has now turned into a multi-billion-dollar partnership with OpenAI.
While the email-surfacing highlights the growing scrutiny of competition in the tech industry, it also makes me wonder if Microsoft's investment in OpenAI could have influenced the overall direction of AI research and development.
Enjoying the daily updates?
Refer your pals to subscribe to our daily newsletter and get exclusive access to 400+ game-changing AI tools.
When you use the referral link above or the “Share” button on any post, you'll get the credit for any new subscribers. All you need to do is send the link via text or email or share it on social media with friends.
Knowledge Nugget: AI leaderboards are no longer useful. It's time to switch to Pareto curves
In this article, the authors
and suggest that AI leaderboards are no longer useful and propose using Pareto curves instead. They highlight the importance of considering the accuracy-cost trade-off when evaluating AI systems, particularly in code generation.The authors compare complex agent architectures with simpler baselines and find that simpler baselines can achieve similar accuracy at lower costs. They also criticize the lack of reproducibility and standardization in agent evaluations and question the effectiveness of certain approaches. They argue against using proxies for cost and suggest directly measuring dollar costs for downstream evaluation.
Why does it matter?
This advocacy calls for more rigorous and transparent evaluation practices, promoting a practical, cost-conscious approach to selecting and deploying AI systems for real-world applications.
What Else Is Happening❗
🤖 Sanctuary AI teams up with Microsoft to advance general-purpose robot AI
Sanctuary AI has announced a collaboration with Microsoft to develop AI models for general-purpose humanoid robots. The partnership will leverage Microsoft's Azure cloud computing platform and AI technologies to enhance the capabilities of Sanctuary AI's robots. (Link)
🗣️ Nvidia's ChatRTX now supports voice queries and Google's Gemma model
Nvidia has updated its ChatRTX chatbot to support Google's Gemma model, voice queries, and additional AI models. The chatbot, which runs locally on a PC, enables users to search personal documents and YouTube videos using various AI models, including ChatGLM3 and OpenAI's CLIP model. (Link)
🤝 Atlassian launches Rovo: An AI assistant for enhanced teamwork
Atlassian has launched Rovo, an AI assistant designed to improve teamwork and productivity. Rovo integrates with Atlassian's products and offers features such as AI-powered search, workflow automation, and integration with third-party tools like Google Drive, Microsoft SharePoint, and Slack. (Link)
📊 MongoDB launches an AI app-building toolkit to help businesses use gen AI
It has launched the MongoDB AI Applications Program, or MAAP, to help companies accelerate building and deployment of AI-powered applications. It brings consultancies and foundation models providers, cloud infrastructure, generative AI frameworks, and model hosting together with MongoDB Atlas to develop solutions for business problems. (Link)
🎨 Ideogram introduces Pro Tier: 12,000 fast AI image generations monthly
Ideogram has launched a paid Pro tier for its AI image generation platform, allowing users to generate up to 12,000 images per month at faster speeds. The platform utilizes AI algorithms to create high-quality images for various applications, including design, marketing, and content creation. (Link)
New to the newsletter?
The AI Edge keeps engineering leaders & AI enthusiasts like you on the cutting edge of AI. From machine learning to ChatGPT to generative AI and large language models, we break down the latest AI developments and how you can apply them in your work.
Thanks for reading, and see you tomorrow. 😊