NVIDIA's ACE brings NPCs to life
Plus: BiomedGPT for multiple biomedical modalities, Break-A-Scene from Google Research
Hello, Engineering Leaders and AI Enthusiasts,
Welcome to the 29th edition of The AI Edge newsletter. In today’s edition, we bring you NVIDIA’s powerful Avatar Cloud Engine (ACE) for Games and more. Thank you everyone who is reading this. 😊
In today’s edition:
🚀NVIDIA launches generative AI that sparks life into virtual characters
🔬BiomedGPT: First generalist AI model for various biomedicine modalities
📸Break-A-Scene: Extracting multiple concepts from a single image
📚How to finetune GPT-like LLMs on a custom dataset
Let’s go!
NVIDIA uses AI to bring NPCs to life
NVIDIA has announced the NVIDIA Avatar Cloud Engine (ACE) for Games. This cloud-based service provides developers access to various AI models, including natural language processing (NLP) models, facial animation models, and motion capture models.
ACE for Games can create NPCs that can have intelligent, unscripted, and dynamic conversations with players, express emotions, and realistically react to their surroundings.
It can help developers in many ways:
To create more realistic and believable NPCs with more natural and engaging conversations with players.
To save time and money by providing them access to various AI models.
Why does this matter?
ACE for Games is a powerful tool that has the potential to change the way that games are created. The service could lead to the creation of more immersive and engaging gaming experiences. Use of AI in gaming will substantially elevate gaming experiences
BiomedGPT: The most sophisticated AI medical model?
BiomedGPT is a unified and generalist Biomedical Generative Pre-trained Transformer model. BiomedGPT utilizes self-supervision on diverse datasets to handle multi-modal inputs and perform various downstream tasks.
Extensive experiments show that BiomedGPT surpasses most previous state-of-the-art models in performance across 5 distinct tasks with 20 public datasets spanning over 15 biomedical modalities.
The study also demonstrates the effectiveness of the multi-modal and multi-task pretraining approach in transferring knowledge to previously unseen data.
Why does this matter?
This research represents a significant advancement in developing unified and generalist models for biomedicine, holding promising implications for enhancing healthcare outcomes, and it could lead to discoveries in biomedical research.
In addition to its potential benefits for healthcare, BiomedGPT could also be used in drug discovery & medical education.
Break-A-Scene: AI breaks down single image into multiple concepts
If given a photo of a ceramic artwork depicting a creature seated on a bowl, humans can effortlessly imagine the same creature in various poses and locations or envision the same bowl in a new setting. However, today's generative models struggle to do this type of task.
This research from Google (and others) introduces a new approach to textual scene decomposition. Given a single image of a scene that may contain multiple concepts of different kinds, it extracts a dedicated text token for each concept (handles) and enables fine-grained control over the generated scenes. The approach uses textual prompts in natural language for creating novel images featuring individual concepts or combinations of multiple concepts, as demonstrated in the video below.
Why does this matter?
The model provides a foundation for developing more advanced algorithms and models to understand and generate diverse visual content by addressing the current limitations of models in handling complex tasks. And it opens up avenues for developing more sophisticated image-generation systems for practical applications in graphic design, advertising, VR, gaming, etc.
Knowledge Nugget: How to finetune GPT-like LLMs on a custom dataset
The AI community’s effort has led to the development of many high-quality open-source LLMs. This tutorial will tell you how to fine-tune models on a custom instruction dataset to adapt to your specific task, such as training a chatbot to answer financial questions.
It walks you through the process, starting with installing Lit-Parrot, an implementation based on GPT-NeoX, which supports StableLM, Pythia, and RedPajama-INCITE model weights. It aims to provide the AI/ML community with a clean, solid, and optimized implementation of LLMs with pretraining and fine-tuning support using LoRA and Adapter.
Why does this matter?
This comprehensive tutorial helps gain a solid understanding of how to effectively fine-tune GPT-like LLMs on custom datasets, empowering developers to create tailored AI solutions for specific applications and harness their full potential.
What Else Is Happening
🎥 Watch how CoffeeVectors brings NYC to life with Google's Photorealistic 3D map (Link)
💼 JPMorgan developing a ChatGPT-like service to provide investment advice to customers (Link)
🔬AI to help scientists predict whether breast cancer spread risk (Link)
💡 IBM consulting launches generative AI center of excellence (Link)
🐼 PandaGPT: The all-in-one model for instruction-following (Link)
Trending Tools
Desku: Transform your business with AI automations, shared inboxes, and WhatsApp integration for better customer support.
Character AI: Chat with AI that feels alive and understands you. Create your own characters powered by LLMs.
Codefy: Speed up workflow with 15+ customized coding widgets for writing, explaining, debugging, and translating code.
Coachify: Achieve personal goals with the help of AI coaches for improving diet, fitness, and overall well-being.
Simulai: Increase website traffic with 1000s of AI-generated posts for SEO promotion of your business.
Coindive: Stay updated on crypto projects and track more than prices with community data and AI for effortless understanding.
WP Wand: Create high-quality content 10X faster and 50X cheaper with AI content generation plugin for WordPress.
Workflos: Boost productivity and save money with private Enterprise AI Assistant and AI SaaS finder.
That's all for now!
If you are new to ‘The AI Edge’ newsletter. Subscribe to receive the ‘Ultimate AI tools and ChatGPT Prompt guide’ specifically designed for Engineering Leaders and AI enthusiasts.
Thanks for reading, and see you tomorrow.