Meta’s 3D AI for Everyday Devices
Plus: ByteDance presents DiffPortrait3D, Can a SoTA LLM run on a phone without internet?
Hello Engineering Leaders and AI Enthusiasts!
Welcome to the 175th edition of The AI Edge newsletter. This edition brings you Meta’s new method for 3D reconstruction on everyday devices.
And a huge shoutout to our incredible readers. We appreciate you😊
In today’s edition:
🎥 Meta’s 3D AI for everyday devices
💻ByteDance presents DiffPortrait3D for zero-shot portrait view
🚀 Can a SoTA LLM run on a phone without internet?
🧠 Knowledge Nugget: Seven Predictions For Tech in 2024 by
Let’s go!
Meta’s 3D AI for everyday devices
Meta research and Codec Avatars Lab (with MIT) have proposed PlatoNeRF, a method to recover scene geometry from a single view using two-bounce signals captured by a single-photon lidar. It reconstructs lidar measurements with NeRF, which enables physically-accurate 3D geometry to be learned from a single view.
The method outperforms related work in single-view 3D reconstruction, reconstructs scenes with fully occluded objects, and learns metric depth from any view. Lastly, the research demonstrates generalization to varying sensor parameters and scene properties.
Why does this matter?
The research is a promising direction as single-photon lidars become more common and widely available in everyday consumer devices like phones, tablets, and headsets.
ByteDance presents DiffPortrait3D for zero-shot portrait view
ByteDance research presents DiffPortrait3D, a novel conditional diffusion model capable of generating consistent novel portraits from sparse input views.
Given a single portrait as reference (left), DiffPortrait3D is adept at producing high-fidelity and 3d-consistent novel view synthesis (right). Notably, without any finetuning, DiffPortrait3D is universally effective across a diverse range of facial portraits, encompassing, but not limited to, faces with exaggerated expressions, wide camera views, and artistic depictions.
Why does this matter?
The framework opens up possibilities for accessible 3D reconstruction and visualization from a single picture.
Can a SoTA LLM run on a phone without internet?
Amidst the rapid evolution of generative AI, on-device LLMs offer solutions to privacy, security, and connectivity challenges inherent in cloud-based models.
New research at Haltia, Inc. explores the feasibility and performance of on-device large language model (LLM) inference on various Apple iPhone models. Leveraging existing literature on running multi-billion parameter LLMs on resource-limited devices, the study examines the thermal effects and interaction speeds of a high-performing LLM across different smartphone generations. It presents real-world performance results, providing insights into on-device inference capabilities.
It finds that newer iPhones can handle LLMs, but achieving sustained performance requires further advancements in power management and system integration.
Why does this matter?
Running LLMs on smartphones or even other edge devices has significant advantages. This research is pivotal for enhancing AI processing on mobile devices and opens avenues for privacy-centric and offline AI applications.
We need your help!
We are working on a Gen AI survey and would love your input.
It takes just 2 minutes.
The survey insights will help us both.
And hey, you might also win a $100 Amazon gift card!
Every response counts. Thanks in advance!
Knowledge Nugget: Seven Predictions For Tech in 2024
What’s next for generative AI?
2023 was easily the most interesting on the tech beat in over a decade. We witnessed AI breakthroughs, the breaking and reformation of OpenAI, NVIDIA’s ascent, and much more.
2024 might top it.
The tech industry has poured so much money and effort into generative AI that new models and products will soon flood the market. Some will retain information longer than today’s bots, and others might start to reason. This evolution of LLMs leads
‘s list of predictions for 2024, covering what’s next for AI, self-driving, mixed reality, and more. Check out his article for the full list of predictions!Why does this matter?
We are nowhere near AI’s final destination. In 2024, we’re going to see what better AI looks like. Startups will make progress on getting AI agents to call Ubers, file expenses, etc. A new obsession will emerge around LLM’s ability to reason vs. predict the next word.
What Else Is Happening❗
📰Apple reportedly wants to use the news to help train its AI models.
Apple is talking with some big news publishers about licensing their news archives and using that information to help train its generative AI systems in “multiyear deals worth at least $50M. It has been in touch with publications like Condé Nast, NBC News, and IAC. (Link)
🤖Sam Altman-backed Humane to ship ChatGPT-powered AI Pin starting March 2024.
Humane plans to prioritize the dispatch of products to customers with priority orders. Orders will be shipped in chronological order by whoever placed their order first. The Ai Pin, with the battery booster, will cost $699. A monthly charge of $24 for a Humane subscription offers cellular connectivity, a dedicated number, and data coverage. (Link)
💰OpenAI seeks fresh funding round at a valuation at or above $100 billion.
Investors potentially involved have been included in preliminary discussions. Details like the terms, valuation, and timing of the funding round are yet to finalize and could still change. If the round happens, OpenAI would become the second-most valuable startup in the US, behind Elon Musk’s SpaceX. (Link)
🔍AI companies are required to disclose copyrighted training data under a new bill.
Two lawmakers filed a bill requiring creators of foundation models to disclose sources of training data so copyright holders know their information was taken. The AI Foundation Model Transparency Act– filed by Reps. Anna Eshoo (D-CA) and Don Beyer (D-VA) – would direct the Federal Trade Commission (FTC) to work with the NIST to establish rules. (Link)
🔬AI discovers a new class of antibiotics to kill drug-resistant bacteria.
AI has helped discover a new class of antibiotics that can treat infections caused by drug-resistant bacteria. This could help in the battle against antibiotic resistance, which was responsible for killing more than 1.2 million people in 2019– a number expected to rise in the coming decades. (Link)
That's all for now!
Subscribe to The AI Edge and join the impressive list of readers that includes professionals from Moody’s, Vonage, Voya, WEHI, Cox, INSEAD, and other reputable organizations.
Thanks for reading, and see you tomorrow. 😊