Salesforce’s XGen Can Replace Meta’s LLaMA
Plus: Databricks launches LakehouseIQ and other AI tools. Microsoft's Professional Certificate on Gen AI.
Hello Engineering Leaders and AI Enthusiasts!
Welcome to the 52nd edition of The AI Edge newsletter. This edition brings you “Salesforce’s XGen Can Replace Meta’s LLaMA.”
And a big thanks to all our incredible readers! 😊😊
In today’s edition:
🚀 Salesforce’s XGen Can Replace Meta’s LLaMA
🔥 Databricks launches LakehouseIQ and Lakehouse AI tools
🎯 Gen AI is now a Bankable Skill backed by Industry Titans
📚 Knowledge Nugget: Accelerating PyTorch Model Training by
Let’s go!
Salesforce’s XGen Can Replace Meta’s LLaMA
Salesforce Introduces XGen-7B, a new 7B LLM trained on up to 8K sequence length for 1.5 Trillion tokens. It is open-sourced under Apache License 2.0.
On standard NLP benchmarks, it achieves comparable or better results when compared with leading open-source LLMs- MPT, Falcon, LLaMA, Redpajama, and OpenLLaMA in a similar size.
It archives equally strong results both in text (e.g., MMLU, QA) and code (HumanEval) tasks.
The targeted evaluation on long sequence modeling benchmarks shows the benefits of 8K-seq models over 2K- and 4K-seq models.
XGen has the same architecture as Meta’s LLaMA models except for a different tokenizer. However, LLaMA 7B is trained on one trillion tokens compared to XGen-7B’s 1.5T tokens.
Why does this matter?
Small models trained on more tokens are easier to retrain and fine-tune for specific use cases. Thus, could XGen be a superior alternative for commercial use? Moreover, a large context allows pre-trained LLMs to utilize customer data not used in training and give better responses. XGen-7B has a larger dataset(1.5T) and context(8K-seq) than LLaMA, MPT, and Falcon, allowing it to outperform them despite the same size.
Databricks launches LakehouseIQ and Lakehouse AI tools
Databricks launched LakehouseIQ, a generative AI tool democratizing access to data insights. It allows anyone in an organization to search, understand and query internal corporate data by simply asking questions in plain English. No Python, SQL or data querying skills needed.
It also announced new Lakehouse AI innovations aimed at making it easier for its customers to build and govern their own LLMs on the lakehouse.
This move follows Databricks’ $1.3 billion acquisition of MosaicML and comes at a time when Snowflake– its main competitor– continues to make its own generative AI push.
Why does this matter?
It looks like Databricks is doing everything possible to put AI at the heart of its data lakehouse. Infusing AI may solve many challenges businesses face, such as easing the burden of data management, freeing time-strapped data engineers, enhancing data analysis, and empowering employees to take advantage of the AI revolution.
Gen AI is now a Bankable Skill backed by Industry Titans
Microsoft has announced the launch of its new AI Skills Initiative aimed at addressing the technical skills gap in the global workforce. The initiative includes a grant challenge, free online courses, and a teacher training toolkit. Microsoft Philanthropies' corporate vice president believes AI drives efficiency and revolutionizes learning. The company has introduced the first Professional Certificate on Generative AI as part of the initiative.
Additionally, Microsoft will provide grants to nonprofit organizations, social enterprises, and academic institutions that use GenAI for social and economic benefits. The exact size of the grants and Microsoft's investment in the programs have not been disclosed.
Why does this matter?
Microsoft calls its AI Skill Initiative the first Professional Certificate on Generative AI in online learning. However, it's worth noting that other notable players in the industry, such as Google, Nvidia, and Stanford University, also offer courses and resources related to Generative AI.
While Microsoft's program seems to have a focused approach tailored to specific AI skills, Google's AI courses offer a broader range of topics encompassing the wider AI landscape. This highlights the competitive nature of the industry, with various players aiming to provide comprehensive education and training in the rapidly evolving field of AI.
Knowledge Nugget: Accelerating PyTorch Model Training
This informative read by
explores techniques for efficiently scaling PyTorch model training with minimal modifications to the existing code. The main emphasis is on harnessing mixed-precision methods and multi-GPU training approaches rather than delving into low-level machine optimizations. For demonstration purposes, the base model is a straightforward Vision Transformer (ViT) designed for image classification.
Why does this matter?
Efficiently scaling PyTorch model training with minimal code changes has significant implications. It enables faster training, better resource utilization, cost savings, and improved scalability for larger models and datasets. These benefits accelerate experimentation, model development, and real-world deployment, ultimately driving progress in deep learning.
Additionally, ViT models support transfer learning and pre-training, facilitating the application of pre-trained models to various computer vision tasks.
What Else Is Happening❗
🎨 Playground’s new feature let’s you edit images as you imagine! (Link)
🔥 OpenAI announces its first international expansion with a new office in London (Link)
💼 Oracle adds new generative AI features to its HCM offering for streamlined HR workflows (Link)
💁♂️ Microsoft Clippy gets major upgrade with ChatGPT, thanks to the new Windows 11 app (Link)
💰 Salesforce to invest $4 billion in UK on AI innovation over the next five years (Link)
💡 Celestial AI raises $100M to transfer data using light-based interconnects (Link)
💸 AI startup Typeface valued at $1 billion after Salesforce-led fundraising (Link)
🛠️ Trending Tools
Journalist AI: Generate high-quality articles easily using Journalist, the AI Article Generator for your business.
Torq AI: Boost your productivity with advanced AI assistance from Torq AI, powered by ChatGPT.
Knibble AI: Access information, collaborate, and share knowledge effortlessly with Knibble.AI, your intelligent knowledge assistant.
Scriptreader AI: Revolutionize screenplay analysis with ScriptReader.ai, providing detailed AI-powered feedback to refine your script.
Every: Empower your bookkeepers with AI for higher efficiency and accuracy, ensuring compliant and cost-effective bookkeeping.
Krater AI: The ultimate tool for content creators! Detect authenticity easily, avoid penalties. Try the free detector now.
OpenAI expenses checker: Track expenses directly from your browser with the "OpenAI Expenses Checker" Chrome extension, saving time and effort.
Demands AI: AI-powered solution for quick, accurate demand letters. Achieve higher settlements, win cases, and save time and money.
That's all for now!
Take your knowledge to the next level! Join The AI Edge community that consists of industry leaders from Moody’s, Vonage, Voya, WEHI, Cox, INSEAD, and more.
Thanks for reading, and see you tomorrow. 😊