Hugging Face + Databricks → 🦾

Plus: Microsoft's next AI tool and JP Morgan's AI implementation

Hello, Engineering Leaders and AI enthusiasts,

Welcome to the eighth edition of The AI Edge newsletter. In today’s edition, we bring you the 40% faster AI development method. Thank you everyone who is reading this.

In today’s edition:

🤖 Databricks and Hugging Face integration boosts AI training
💹 JPMorgan's ChatGPT Tool scours fed speeches for trading clues
🎨 Microsoft launches AI-Driven design tool: Designer
🧠 Knowledge Nugget: The Anatomy of Autonomy by

swyx

Let’s go

---

40% faster AI model building by Hugging Face and Databricks

Databricks and Hugging Face have collaborated to integrate Apache Spark with Hugging Face datasets, allowing users to create Hugging Face datasets from an Apache Spark data frame. The integration simplifies the process of loading and transforming data for AI model training and fine-tuning.

By using the "from_spark" function, users can efficiently load and transform their data for training, significantly reducing data processing time and costs. Databricks claims that the new Spark integration has reduced the processing time for a 16GB dataset by more than 40%, dropping from 22 to 12 minutes.

Why does this matter?

This integration simplifies data loading and transformation processes for model training and fine-tuning, improving efficiency and reducing the costs associated with handling large AI datasets. By harnessing the parallelization capabilities of Spark, processing times can be dramatically reduced, contributing to faster AI model development.

Source

---

AI meets finance: JPMorgan's ChatGPT Tool scours fed speeches for trading clues

JPMorgan has developed an AI tool that utilizes OpenAI's ChatGPT to analyze Federal Reserve speeches and statements for potential trading signals. The tool rates these policy signals on a Hawk-Dove Score, which assesses the likelihood of interest rate changes. By predicting shifts in central bank tightening, the AI tool can provide early indications of trading signals, assisting the investment bank in making informed decisions.

JPMorgan continues to explore AI use cases, with over 300 AI applications currently in production, as revealed by CEO Jamie Dimon in an annual letter to shareholders.

Why does this matter?

The introduction of JPMorgan's AI tool to analyze Federal Reserve statements highlights the growing significance of AI in the financial sector. As a leading financial institution, JPMorgan's adoption of AI for detecting trading signals demonstrates the potential for AI-driven solutions to reshape decision-making processes within the industry. This development is likely to encourage other banks and financial institutions to explore and implement AI applications in various areas, ultimately enhancing efficiency, accuracy, and the overall competitive landscape.

Source

---

Microsoft launches AI-Driven design tool: Designer

Microsoft launches Designer, which utilizes the latest version of OpenAI's Dall-E to generate content from user prompts. Similar to Canva, the Designer app allows users to write a description of their desired output, and the AI responds by creating a graphic design.

The Designer app, which was previously available only through a waitlist, will now be integrated into the Microsoft Edge browser sidebar for easy access. Users can try the AI tool for free, while Microsoft 365 subscribers will have access to additional premium features. More AI-powered features, such as Fill, Expand background, Erase, and Replace backgrounds, are expected to be added to the app over time.

Why does this matter?

Microsoft Designer has the potential to attract a large user base of creators and establish a monopoly eventually. Other efficient text-to-image generators like Midjourney require a subscription, while the free tools aren’t as good as users want them to be.

Source

---

Knowledge Nugget: The Anatomy of Autonomy

Shawn(

swyx

) provides a detailed breakdown of the rise of autonomous AI agents and draws a picture of what the future might look like. He discusses the viability of using existing LLM APIs and tool selection prompt patterns in an infinite loop to achieve high-level goals.

He mentions how autonomous AI has progressed through several key autonomy capabilities, namely

1. Metacognition (self-improvement of pure reasoning)

2. External Memory (reading from mostly static external data)

3. Browser Automation (sandboxed read-and-write in a browser)

4. Tool making and Tool use (server-side, hooked up to everything), and

5. Planning, reflexion, and prioritization

He mentions as we progress through various levels of autonomy, we'll eventually shift to a paradigm with many AI agents per human, aiming to augment human agency rather than replace it. However, there will be two schools of thought on autonomous AI: the Jobs School, focusing on AI agents that augment human agency, and the Zuck School, focusing on AI algorithms that replace human agency.

Why does this matter?

Autonomous AI agents have brought about concrete applications of AI in our daily lives. With AI agents, we can now foresee how AI will play a bigger role in multiple human activities, and it’s essential to understand the underlying principles and features to leverage the technology.

The article takes you on a deep dive into the world of AI and autonomous agents, giving you much-needed clarity on what the future holds.

---

What Else Is Happening

⚽ Bipedal robots learn agile soccer skills using AI. (Link)

📜 EU to soon impose copyright laws for generative AI. (Link)

🔬 Researchers use AI to predict a material’s internal structure. (Link)

💬 GigaChat launched in Russia. (Link)

🎓 OpenAI partners for free ChatGPT prompt engineering course for devs. (Link)

💰 Walmart’s AI bot is now negotiating prices with vendors. (Link)

💊 CancerGPT to predict synergies of drug pairs on specific tissues. (Link)

🛠️ Copilot: The ChatGPT for hardware design. (Link)

---

Trending Tools

• Merlin 2.0: ChatGPT extension to finish tasks on any website in seconds. Boost productivity with a sleek new chat interface!

• Microsoft Designer: Type in ideas to get AI-powered designs & suggestions for social media & digital needs.

• Copymate: Create high-quality, SEO-optimized articles in minutes! Publish directly to WordPress.

• Lettria: Transform your business with AI and Large Language Models without compromising privacy.

• Virtuo: Access ChatGPT from any website instead of switching the tabs.

• W&B Prompts: Suite of LLMops tools to improve prompt engineering workflows and unlock a deeper understanding of LLMs.

• AI LogoBrainstorm: AI-driven tool generates 200 unique logo concepts based on your brand’s identity.

• HeyGen: Dress up your avatar with Generative Outfit powered by ChatGPT for any scenario with diverse styles!

---

That's all for now!

If you are new to ‘The AI Edge’ newsletter. Subscribe to receive the ‘Ultimate AI tools and ChatGPT Prompt guide’ specifically designed for Engineering Leaders and AI enthusiasts.

Thanks for reading, and see you tomorrow.

---