Leanne Test 3

Test 0724-2

Twice a week, Data Points brings you the latest AI news, tools, models, and research in brief. In today’s edition, you’ll find:

 * Nvidia’s Nemotron adds reasoning to Llama models
 * Does ChatGPT make frequent users more lonely?
 * OpenAI’s o1-pro costs a pretty penny
 * Mistral Small 3.1 gives Gemma 3 27B some competition

But first:

New speech model enables real-time visual conversations

Kyutai released MoshiVis, an open vision speech model that lets users have natural voice conversations about images while maintaining low latency. The model adds 206 million trainable parameters on top of the existing Moshi speech model and uses a data-efficient training approach that requires minimal audio data by training on text-based image descriptions. MoshiVis may represent a significant step toward more natural multimodal AI interactions, as the model can seamlessly switch between general conversation and discussing visual content while maintaining low latency on consumer hardware. (Kyutai and arXiv)

Cloudflare uses generative AI to fight unauthorized AI crawlers

Cloudflare launched AI Labyrinth, a new defense system that generates fake web pages to waste the resources of unauthorized AI crawlers that ignore “no crawl” directives. The system creates convincing but irrelevant content networks that serve as honeypots, helping Cloudflare identify and track unauthorized scrapers. AI crawlers now generate over 50 billion requests daily on Cloudflare’s network, representing nearly 1 percent of all web traffic they handle. This approach marks a shift from traditional blocking methods and could make it difficult for AI crawlers to extract useful data. (Cloudflare)

Nvidia releases open reasoning models with shared training data

Nvidia unveiled a new family of open weight reasoning models called Llama Nemotron, sharing not only the models but also 30 million training samples and detailed training methods. The three models - ranging from 8 billion to 253 billion parameters - feature toggleable reasoning capabilities, distilling Meta’s open Llama models but adding DeepSeek-like reinforcement learning. This comprehensive release, which includes model weights, post-training data, and technical documentation, enables AI developers to better understand, modify, and build upon Nvidia’s work to create more capable AI systems. (Nvidia)

OpenAI studies emotional impact of ChatGPT use

OpenAI and MIT Media Lab researchers analyzed 40 million ChatGPT interactions and conducted a four-week trial with nearly 1,000 participants to study how people emotionally engage with the AI system. The studies found that users who developed emotional bonds with ChatGPT were more likely to be lonely and dependent on the system, while participants using voice chat with a gender different from their own reported higher levels of loneliness. Although researchers acknowledge the limitations of self-reported emotional data, these findings begin to address how large language models affect human psychology and could help companies design safer AI interactions and attempt to make their models more “emotionally intelligent.” (OpenAI and MIT Media Lab)

OpenAI launches o1-pro in the API, its most expensive model yet

OpenAI’s reasoning model o1-pro is now available via the company’s Responses API at a price of $150 per million tokens of input and $600 per million tokens of output. This makes o1-pro easily the company’s most expensive model, surpassing GPT-4.5. Previously, o1-pro had only been available through the company’s monthly Pro subscription plan; this release opens it to developers who want to take advantage of its use of more computing power and generates more tokens at inference, allowing it to provide more accurate and logically thorough answers than a standard AI model. (OpenAI)

Mistral releases new open multimodal model

Mistral AI released Mistral Small 3.1, a 24 billion parameter open weights model that processes text and images while running on consumer hardware like an RTX 4090 graphics card. The model outperforms Gemma 3 and similar-sized competitors on various knowledge and instruction—following benchmarks, handles up to 128,000 tokens of context, and operates at speeds of 150 tokens per second. The release shows how competition between open AI models continues to narrow the performance gap with proprietary alternatives while maintaining accessibility for developers. (Mistral)

Still want to know more about what matters in AI right now?

Read last week’s issue of The Batch for in-depth analysis of news and research.

Last week, Andrew Ng shared insights from AI Dev 25. He highlighted attendees’ strong interest in agentic AI and solving real-world problems over AGI hype. He also praised the event’s technical depth, emphasizing DeepLearning.AI’s “Learner First” mentality and the value of bringing developers together.

“There is something magical about bringing people together physically to share ideas, make friends, and to learn from and help each other. I hope we’ll be able to bring even more people together in the future.”

Read Andrew’s full letter here.

Other top AI news and research stories we covered in depth: Cohere’s Aya Vision outperformed multimodal rivals in text and image understanding, demonstrating fluency across a wide range of languages; AI Co-Scientist, Google’s new research agent, showed itself capable of generating hypotheses to aid drug discovery; the U.S. Copyright Office ruled that no new laws are needed to govern AI-generated works, noting the copyrightability of AI-assisted creations with sufficient human guidance; and MatterGen, a diffusion model, showcased its ability to design novel materials with tailored properties, advancing AI-driven material discovery.

Subscribe to Data Points

Data Points

Building a model for vision and speech: How Cloudflare thwarts unauthorized AI crawlers… by using AI

Twice a week, Data Points brings you the latest AI news, tools, models, and research in brief. In today’s edition, you’ll find:

 * Nvidia gives Project DIGITS a new name
 * AI models compete to build Minecraft items
 * Claude chatbot now includes search
 * A Moore’s law-like regularity for AI agents

But first:

AI model surpasses traditional weather forecasting systems

Researchers at Cambridge University, Microsoft, Google, and other institutions developed Aardvark Weather, an end-to-end machine learning model that outperforms traditional numerical weather prediction systems for global and local forecasts. The model ingests raw observational data and produces accurate forecasts up to ten days in advance, competing with state-of-the-art systems that incorporate human input. The model’s high accuracy shows the potential for fully data-driven weather prediction to significantly reduce computational costs and enable customized forecasting models for individual users or smaller nations. (Nature)

OpenAI releases new speech models

OpenAI debuted new speech-to-text models (gpt-4o-transcribe and gpt-4o-mini-transcribe) and a text-to-speech model (gpt-4o-mini-tts) that outperform current Whisper models on tests of accuracy and reliability. The speech-to-text models demonstrate improved Word Error Rate performance across multiple benchmarks, while the text-to-speech model allows developers to instruct it to speak in specific ways (like a storytelling pirate, or a calm customer service representative). OpenAI says they built the new speech-to-text and text-to-speech models using new distillation techniques to shrink large models like GPT-4o and reinforcement learning to improve transcription and voice generation accuracy. (OpenAI)

Nvidia unveils personal computers for AI developers

Nvidia CEO Jensen Huang introduced two new AI-focused desktop systems, DGX Spark (formerly known as Project DIGITS) and DGX Station (a larger model), during the company’s GTX keynote. The computers, powered by Nvidia’s Grace Blackwell platform, are designed to enable developers, researchers, and data scientists to run large AI models locally for prototyping and fine-tuning. Five major PC manufacturers, including Asus, Dell, HP, and Lenovo, will produce these systems, with DGX Spark reservations opening immediately and DGX Station expected later in 2025. (Ars Technica)

Minecraft emerges as novel AI benchmark tool

Developers led by 12th-grader Adi Singh created Minecraft Benchmark (MC-Bench), a website where AI models compete to build Minecraft creations by writing code based on prompts. Users vote on the best builds without knowing which AI produced them, providing a novel way to assess AI capabilities beyond traditional benchmarks. The site is built with subsidies from Anthropic, OpenAI, and Alibaba, but remains unaffiliated; currently Claude 3.7 Sonnet tops the leaderboard. MC-Bench’s approach tests coding ability, visual understanding, and problem solving in a way that leverages Minecraft’s widespread familiarity to make AI progress more accessible and understandable to the general public. (MC-Bench and TechCrunch)

Claude introduces web search

Anthropic added web search functionality to its AI chatbot Claude, allowing it to access up-to-date information and provide more relevant responses to queries. The feature is currently available in preview for paid U.S. users of Claude 3.7 Sonnet, with plans to expand to free users and other countries. This update enables Claude to incorporate current data from internet sources, providing inline citations in conversational, aggregated responses, similar to competitors like ChatGPT and Gemini. (Anthropic)

Identifying a new AI problem-solving progression law

Researchers at METR proposed a “50%-task-completion time horizon” metric to compare AI and human capabilities on various long-duration tasks. Current top AI models like Claude 3.7 Sonnet can complete tasks with 50 percent success that take skilled humans about 50 minutes, with this time horizon doubling roughly every seven months since 2019 – in other words, in seven months, we may expect a model to be able to complete a task halfway that takes humans 100 minutes, then 200 minutes, etc.. This metric offers AI developers a concrete way to measure progress in AI capabilities relative to human performance, potentially signaling that within five years, top AI agents may be able to automate tasks with 50 percent success that currently take skilled humans about a month to complete. (arXiv)

Still want to know more about what matters in AI right now?

Read this week’s issue of The Batch for in-depth analysis of news and research.

This week, Andrew Ng shared insights from AI Dev 25. He highlighted attendees’ strong interest in agentic AI and solving real-world problems over AGI hype. He also praised the event’s technical depth, emphasizing DeepLearning.AI’s “Learner First” mentality and the value of bringing developers together.

“With the wide range of AI tools now available, there is a rich set of opportunities for developers to build new things, but also a need for a neutral forum that helps developers do so.”

Read Andrew’s full letter here.

Other top AI news and research stories we covered in depth: Cohere’s Aya Vision outperformed multimodal rivals in text and image understanding, demonstrating fluency across a wide range of languages; AI Co-Scientist, Google’s new research agent, showed itself capable of generating hypotheses to aid drug discovery; the U.S. Copyright Office ruled that no new laws are needed to govern AI-generated works, noting the copyrightability of AI-assisted creations with sufficient human guidance; and MatterGen, a diffusion model, showcased its ability to design novel materials with tailored properties, advancing AI-driven material discovery.

Subscribe to Data Points

This Aardvark predicts the weather: GPT-4o meets Whisper; OpenAI’s new models



Loading the Elevenlabs Text to Speech AudioNative Player...



Materials that have specific properties are essential to progress in critical technologies like solar cells and batteries. A machine learning model designs new materials to order.

What’s new: Researchers at Microsoft and Shenzhen Institute of Advanced Technology proposed MatterGen, a diffusion model that generates a material’s chemical composition and structure from a prompt that specifies a desired property. The model and code are available under a license that allows commercial as well as noncommercial uses without limitation. The training data also is noncommercially available.

How it works: MatterGen’s training followed a two-stage process. In the first stage, it learned to generate materials (specifically crystals — no liquids, gasses, or amorphous solids like glass). In the second, it learned to generate materials given a target mechanical, electronic, magnetic, or chemical property such as magnetic density or bulk modulus (the material’s resistance to compression).

 * MatterGen first learned to remove noise that had been added to 600,000 examples drawn from two datasets. Specifically, it learned to remove noise from three noisy matrices that represented a crystal’s shape (parallelepiped), the type of each atom, and the coordinates of each atom.
 * To incorporate information about properties, the authors added to the diffusion model four vanilla neural networks, each of which took an embedding of the target property. The diffusion model added the output of these networks to its intermediate embeddings at different layers.
 * Then the authors fine-tuned the system to remove added noise from materials that contained property information in their original dataset.
 * At inference, given three matrices of pure noise representing crystal shape, atom types, and atom coordinates, and a prompt specifying the desired property, the diffusion model iteratively removed the noise from all three matrices.

Results: The authors generated a variety of materials, and they synthesized one to test whether it had a target property. Specifically, they generated over 8,000 candidates with the target bulk modulus of 200 gigapascals (a measure of resistance to uniform compression), then automatically filtered them based on a number of factors to eliminate material in their dataset and unstable materials. Of the remaining candidates, they chose four manually and successfully synthesized one. The resulting crystal had a measured bulk modulus of 158 gigapascals. (Most materials in the dataset had a bulk modulus of between 0 and 400 gigapascals.)

Behind the news: Published in 2023, DiffCSP also uses a diffusion model to generate the structures of new materials. However, it does so without considering their desired properties.

Why it matters: Discovering materials relies mostly on searching large databases of existing materials for those with desired properties or synthesizing new materials and testing their properties by trial and error. Designing new crystals with desired properties at the click of a button accelerates the process dramatically.

We’re thinking: While using AI to design materials accelerates an important step, determining whether a hypothesized material can be  manufactured efficiently at scale is still challenging. We look forward to research into AI models that also take into account ease of manufacturing.

Science

Designer Materials: MatterGen, a diffusion model that designs new materials with specified properties



Loading the Elevenlabs Text to Speech AudioNative Player...



The United States Copyright Office determined that existing laws are sufficient to decide whether a given AI-generated work is protected by copyright, making additional legislation unnecessary.

What’s new: AI-generated works qualify for copyright if a human being contributed enough creative input, according to the second part of what will be a three-part report on artificial intelligence and copyright law.

How it works: The report states that “the outputs of generative AI can be protected by copyright only where a human author has determined sufficient expressive elements.” In other words, humans and AI can collaborate on creative works, but copyright protection applies only if a human shapes the AI-generated material beyond simply supplying a prompt.

 * The report rejects the argument that protecting AI-generated works requires a new legal framework. Instead, it argues that copyright law already establishes clear standards of authorship and originality.
 * Human authors or artists retain copyright over creative contributions in the form of selection, coordination, and modification of generated outputs. Selection refers to curating AI-generated elements. Coordination involves organizing multiple generated outputs into a cohesive work. Modification is altering generated material in a way that makes it original. They retain copyright even if AI processes their creative work. They lose it only if the generated output is genuinely transformative.
 * The report emphasizes continuity with past decisions regarding computer-assisted works. It cites a February 2022 ruling in which the Copyright Office rejected a work that had no human involvement. However, in 2023, the office granted a copyright to a comic book that incorporated AI-generated images because a human created original elements such as text, arrangement, and modifications. The report argues this approach aligns with prior treatment of technologies like photography: Copyright protection depends on identifiable human creative input, and that input merits protection even if technology assists in producing it.

Behind the news: The first part of the Copyright Office’s report on digital replicas, or generated likenesses of a person’s appearance and voice. It found that existing laws don’t provide sufficient protection against unauthorized digital replicas and recommended federal legislation to address the gap. Its findings influenced ongoing discussions in Congress, where proposed bills like the No AI FRAUD Act and the NO FAKES Act aim to regulate impersonation via AI. Additionally, industry groups such as the Authors Guild and entertainment unions have pursued their own agreements with studios and publishers to safeguard performers, artists, and authors from unauthorized digital reproduction. However, no federal law currently defines whether copyright can protect a person’s likeness or performance.

Why it matters: The Copyright Office deliberately avoided prescribing rigid criteria for the types or degrees of human input that are sufficient for copyright. Such determinations require nuanced evaluation case by case. This flexible approach accommodates the diverse ways creative people use AI as well as unforeseen creative possibilities of emerging technology.

We’re thinking: Does copyright bar the use of protected works to train AI systems? The third part of the Copyright Office’s report — no indication yet as to when to expect it — will address this question. The answer could have important effects on both the arts and AI development.

Business

Some AI-Generated Works Are Copyrightable: U.S. Copyright Office says that no new laws are needed for AI-generated works



Loading the Elevenlabs Text to Speech AudioNative Player...



An AI agent synthesizes novel scientific research hypotheses. It's already making an impact in biomedicine.

What’s new: Google introduced AI co-scientist, a general multi-agent system designed to generate in-depth research proposals within constraints specified by the user. The team generated and evaluated proposals for repurposing drugs, identifying drug targets, and explaining antimicrobial resistance in real-world laboratories. It’s available to research organizations on a limited basis.

How it works: AI co-scientist accepts a text description of a research goal, including relevant constraints or ideas. In response, it generates research proposals and reviews, ranks, and improves them using seven agents based on Google’s Gemini 2.0 family of large language models. The completed proposals include sections that explain background, unmet needs, a proposed solution, goals, hypotheses, reasoning, study steps, and relevant articles. The agents take feedback and outputs from other agents to perform their prompted task simultaneously.

 * The supervisor agent periodically determines how often to run the other six agents, how important their output is, and whether the system is finished. To accomplish this, it computes statistics that represent the number of proposals generated so far, how many have been reviewed, and so on.
 * The generation agent generates a list of proposals. It searches the web for relevant research articles, identifies testable assumptions, and debates with itself to improve ambiguous statements and adhere to constraints.
 * The reflection agent filters the generated proposals according to correctness, quality, safety, and novelty. First, it reviews a proposal without web search and discards obviously bad proposals. Then it reviews each proposal against literature it finds online. It breaks down and checks the proposal’s assumptions, checks whether the proposal might explain some observations in previous work, and simulates the proposed experiment (via text generation, similar to how a person performs a thought experiment).
 * The proximity agents compute similarity between proposals to avoid redundancy.
 * The ranking agent determines the best proposals according to a tournament. It examines one pair of proposals at a time (including reviews from the reflection agent) and debates itself to pick the better one. To save computation, it prioritizes comparing similar proposals, new proposals, and highest-ranking proposals.
 * The evolution agent generates new proposals by improving existing ones. It does this in several different ways, including simplifying current ideas, combining top-ranking ideas, and generating proposals that are very different from current ones.
 * The meta-review agent identifies common patterns in the reflection agent’s reviews and the ranking agent’s debates. Its feedback goes to the reflection and generation agents, which use it to address common factors in future reviews and avoid generating similar proposals, respectively.

Results: AI co-scientist achieved a number of impressive biomedical results in tests.

 * Google researchers generated proposals for experiments that would repurpose drugs to treat acute myeloid leukemia. They shared the 30 highest-ranked proposals with human experts, who chose five for lab tests. Of the five drugs tested, three killed acute myeloid leukemia cells.
 * Experts selected three among 15 top-ranked generated proposals that proposed repurposing existing drugs to treat liver fibrosis. Two significantly inhibited liver fibrosis without being toxic to general cells. (Prior to this research, one of the drugs was approved by the United States Food and Drug Administration for a different illness, which may lead to a new treatment for liver fibrosis.)
 * AI co-scientist invented a hypothesis to explain how microbes become resistant to antibiotics. Human researchers had proposed and experimentally validated the same hypothesis, but their work had not yet been published at the time, and AI co-scientist did not have access to it.

Behind the news: A few AI systems have begun to produce original scientific work. For instance, a model generated research proposals that human judges deemed more novel than proposals written by flesh-and-blood scientists, and an agentic workflow produced research papers that met standards for acceptance by top conferences.

Why it matters: While previous work used agentic workflows to propose research ideas on a general topic, this work generates proposals for specific ideas according to a researcher’s constraints (for example, a researcher could specify that a novel medical treatment for a specific disease only consider drugs already approved for human trials for other uses) and further instructions. AI co-scientist can take feedback at any point, allowing humans to collaborate with the machine: People provide ideas, feedback, and guidance for the model, and the model researches and proposes ideas in return.

We’re thinking: I asked my AI system to propose a new chemical experiment. But there was no reaction!

Machine Learning Research

Science Research Proposals Made to Order: AI Co-Scientist, an agent that generates research hypotheses, aiding drug discovery



Loading the Elevenlabs Text to Speech AudioNative Player...



Multilingual AI models often suffer uneven performance across languages, especially in multimodal tasks. A pair of lean models counters this trend with consistent understanding of text and images across major languages.

What’s new: A team at Cohere led by Saurabh Dash released Aya Vision, a family of multilingual vision-language models with downloadable weights in 8 billion- and 32-billion-parameter sizes. 

 * Input/output: Text and images in (up to 2,197 image tokens, up to 16,000 tokens total), text out (up to 4,000 tokens).
 * Availability: Free via WhatsApp or Cohere Playground. Weights available to download, but licensed only for noncommercial uses.
 * Features: Multilingual input and output in 23 languages. 
 * Undisclosed: Knowledge cutoff, training datasets, adapter architecture.

How it works: Each model comprises a pretrained large language model (Aya Expanse for the 32B model, C4AI Command R7B for the 8B version), a pretrained vision encoder (SigLIP 2), and a vision-language adapter (“connector”) of unspecified architecture.

 * To establish basic vision-language understanding, the team froze the vision encoder and language model and trained the vision-language connector.
 * They fine-tuned the vision-language connector and language model on multimodal tasks. To build the fine-tuning dataset, they generated synthetic annotations for various English-language datasets and translated a large amount of data into a variety of languages. They rephrased the translations to add fluency and variety, particularly for languages with little real-world data, by matching generated pairs with the original synthetic samples.
 * They merged the language model with the fine-tuned vision-language model using an undisclosed method that preserved text capabilities while adding vision understanding.
 * After proving this method for 8 billion parameters, they scaled up the recipe to 32 billion parameters.

Performance: To test the model, the team built and released two benchmarks: m-WildVision, a multilingual version of Wild Vision Bench’s arena-style competition for discussion of images, and AyaVisionBench, 135 image-question pairs in each language that cover nine tasks including captioning images, understanding charts, recognizing characters in images, visual reasoning, and converting screenshots to code. On these two benchmarks, Aya Vision 8B and 32B outperformed larger competitors, as judged by Claude 3.7 Sonnet.

 * In head-to-head competitions on AyaVisionBench, Aya Vision 8B won up to 79 percent of the time against six competitors of similar size. On m-WildVision, it achieved 81 percent when compared to vision-language models of similar size including Qwen2.5-VL 7B, Pixtral 12B, Gemini Flash 1.5 8B, and Llama-3.2 11B Vision. Aya Vision 8B won 63 percent of the time against Llama-3.2 90B Vision, a model more than 10 times its size.
 * On both benchmarks, Aya Vision 32B outperformed vision-language models more than twice its size including Llama-3.2 90B Vision, Molmo 72B, and Qwen2.5-VL 72B. On AyaVisionBench, it won between 50 and 64 percent of the time. On WildVision, it achieved win rates between 52 percent and 72 percent across all languages.

Behind the news: Aya Vision builds on the Cohere-led Aya initiative, a noncommercial effort to build models that perform consistently well in all languages, especially languages that lack high-quality training data. The project started with a multilingual text model (Aya Expanse), added vision (Aya Vision), and plans to eventually add video and audio.

Why it matters: Multilingual vision-language models often perform less well in low-resource languages, and the gap widens when they process media other than text. Aya Vision’s recipe for augmenting synthetic data with successively refined translations may contribute to more universally capable models. Aya Vision is available on the global messaging platform WhatsApp, where it can be used to translate text and images in all 23 of its current languages.

We’re thinking: Multilingual vision models could soon help non-native speakers decipher Turkish road signs, Finnish legal contracts, and Korean receipts. We look forward to a world in which understanding any scene or document is as effortless in Swahili as it is in English.

Equally Fluent in Many Languages: Cohere’s Aya Vision beats multilingual rivals in text & image understanding



Loading the Elevenlabs Text to Speech AudioNative Player...



Dear friends,

Last Friday on Pi Day, we held AI Dev 25, a new conference for AI Developers. Tickets had (unfortunately) sold out days after we announced their availability, but I came away energized by the day of coding and technical discussions with fellow AI Builders! Let me share here my observations from the event.

I'd decided to start AI Dev because while there're great academic AI conferences that disseminate research work (such as NeurIPS, ICML and ICLR) and also great meetings held by individual companies, often focused on each company's product offerings, there were few vendor-neutral conferences for AI developers. With the wide range of AI tools now available, there is a rich set of opportunities for developers to build new things (and to share ideas on how to build things!), but also a need for a neutral forum that helps developers do so.

Based on an informal poll, about half the attendees had traveled to San Francisco from outside the Bay Area for this meeting, including many who had come from overseas. I was thrilled by the enthusiasm to be part of this AI Builder community. To everyone who came, thank you!

Other aspects of the event that struck me:

 * First, agentic AI continues to be a strong theme. The topic attendees most wanted to hear about (based on free text responses to our in-person survey at the start of the event) was agents!
 * Google's Paige Bailey talked about embedding AI in everything and using a wide range of models to do so. I also particularly enjoyed her demos of Astra and Deep Research agents.
 * Meta's Amit Sangani talked compellingly as usual about open models. Specifically, he described developers fine-tuning smaller models on specific data, resulting in superior performance than with large general purpose models. While there're still many companies using fine-tuning that should really just be prompting, I'm also seeing continued growth of fine-tuning in applications that are reaching scale and that are becoming valuable.
 * Many speakers also spoke about the importance of being pragmatic about what problems we are solving, as opposed to buying into the AGI hype. For example, Nebius' Roman Chernin put it simply: Focusing on solving real problems is important! 
 * Lastly, I was excited to hear continued enthusiasm for the Voice Stack. Justin Uberti gave a talk about OpenAI’s realtime audio API to a packed room, with many people pulling out laptops to try things out themselves in code! 

DeepLearning.AI has a strong “Learner First” mentality; our foremost goal is always to help learners. I was thrilled that a few attendees told me they enjoyed how technical the sessions were, and said they learned many things that they're sure they will use. (In fact, I, too, came away with a few ideas from the sessions!) I was also struck that, both during the talks and at the technical demo booths, the rooms were packed with attendees who were highly engaged throughout the whole day. I'm glad that we were able to have a meeting filled with technical and engineering discussions.

I'm delighted that AI Dev 25 went off so well, and am grateful to all the attendees, volunteers, speakers, sponsors, partners, and team members that made the event possible. I regretted only that the physical size of the event space prevented us from admitting more attendees this time. There is something magical about bringing people together physically to share ideas, make friends, and to learn from and help each other. I hope we'll be able to bring even more people together in the future.

Keep building!

Andrew



P.S. I'm thrilled to share our newest course series: the Data Analytics Professional Certificate! Data analytics remains one of the core skills of data science and AI, and this professional certificate takes you up to being job-ready for this. Led by Netflix data science leader Sean Barnes, this certificate gives you hands-on experience with essential tools like SQL, Tableau, and Python, while teaching you to use Generative AI effectively as a thought partner in your analyses. Labor economists project a 36% growth in data science jobs by 2033. I'm excited to see rising demand for data professionals, since working with data is such a powerful way to improve decision-making, whether in business, software development, or your private life. Data skills create opportunities at every level—I’m excited to see where they take you! Sign up here!

Letters

Lessons From Our First AI Dev Conference: How our learners began building their way to AI in everything at AI Dev 25 Tags: Letters, DeepLearning.AI News, Learning & Education

Twice a week, Data Points brings you the latest AI news, tools, models, and research in brief. In today’s edition, you’ll find:

 * Google’s two new Gemini vision-language-action robotics models
 * Cohere’s Command A, another lightweight LMM
 * New China regulations require mandatory labels for AI content
 * Monitoring reasoning models for reward hacking or unwanted behavior

But first:

Baidu releases ERNIE 4.5 and ERNIE X1 models

Baidu launched its latest foundation models, ERNIE 4.5 and ERNIE X1, with free access for individual users through ERNIE Bot’s website. ERNIE 4.5 is a multimodal model integrating text, images, audio, and video, while ERNIE X1 is a deep-thinking reasoning model with enhanced planning and problem-solving capabilities. Enterprise users and developers can try both models free on Baidu’s ERNIE bot website, or access ERNIE 4.5 on Baidu AI Cloud’s Qianfan with pricing starting at RMB 0.004 per thousand tokens. ERNIE 4.5 reportedly outperforms GPT-4.5 on multiple benchmarks at 1 percent of the price, while ERNIE X1 (available via API soon) offers performance comparable to DeepSeek-R1, with input prices of RMB 0.002 per thousand tokens and output prices of RMB 0.008 per thousand tokens, about half the price. (PR Newswire)

OLMo 2 32B launches as a high-performing, fully open model

AI2 released OLMo 2 32B, the largest model in their OLMo 2 lineup. This model, trained on trillions of tokens and post-trained with Tulu 3.1, competes with leading open weight models (Qwen 2.5 72B, Llama 3.1 and 3.3 70B) and outperforms GPT-3.5 Turbo and GPT4o-mini on various academic benchmarks. Developers and researchers could gain from OLMo 2 32B’s open code and open data availability, allowing them to study and customize advanced model pipelines and experiment with multimodal input. (Allen AI)

Google DeepMind introduces Gemini models for robotics

Google DeepMind unveiled Gemini Robotics and Gemini Robotics-ER, two AI models based on Gemini 2.0, designed to enhance robotic capabilities in the physical world. Gemini Robotics incorporates vision-language-action abilities for direct robot control, and it adapts easily to various robot types, while Gemini Robotics-ER focuses on spatial reasoning and can connect with existing low-level robot controllers. These new AI models aim to improve generalization, interactivity, and dexterity in robots, making them better suited for real-world tasks and applications. (Google)

Cohere’s new model offers long contexts and language support for enterprise

Cohere released Command A, a 111 billion parameter generative model that aims to deliver strong performance for enterprise applications using minimal hardware. Command Al matches or exceeds the performance of GPT-4o and DeepSeek-V3 on tasks such as instruction following and multilingual capabilities while requiring only two A100 or H100 GPUs for deployment. Command A supports the 23 languages spoken by most of the world’s population, and can be deployed locally or via API for $2.50/million input tokens and $10/million output tokens. (Cohere)

China introduces extensive AI content labeling regulations

China will require explicit labeling of AI-generated content under new regulations taking effect on September 1. The new regulations were announced by the Cyberspace Administration of China and other agencies. The rules mandate visible and metadata-based labels for AI-generated text, images, audio, video, and virtual content, and require service providers to ensure compliance with cybersecurity regulations. These measures aim to curb misinformation and enhance online transparency, similar to regulatory efforts in the EU and U.S. (South China Morning Post)

How to use LLMs to detect misbehavior in reasoning models

Open AI researchers showed that they can use large language models to monitor the chain-of-thought (CoT) reasoning of frontier models to detect misbehavior. They found that when these reasoning models exploit or circumvent tasks using “bad thoughts” (unwanted steps in the reasoning chain or attempts at reward hacking), penalizing these thoughts led models to hide their intentions, rather than ceasing misbehavior. Findings suggest that CoT monitoring can provide valuable insights for overseeing advanced AI models in the future, but the researchers recommend against strong optimization pressure on CoTs as it risks making models’ intentions less transparent. (OpenAI)

Still want to know more about what matters in AI right now?

Read last week’s issue of The Batch for in-depth analysis of news and research.

Last week, Andrew Ng defended the importance of learning to code, arguing that as AI-assisted coding makes programming easier, more people should code—not fewer. He pushed back against claims that programming will become obsolete, arguing that understanding the “language of software” empowers individuals to work effectively with AI tools and maximize their impact.

“One question I’m asked most often is what someone should do who is worried about job displacement by AI. My answer is: Learn about AI and take control of it, because one of the most important skills in the future will be the ability to tell a computer exactly what you want, so it can do that for you. Coding (or getting AI to code for you) is the best way to do that.”

Read Andrew’s full letter here.

Other top AI news and research stories we covered in depth: QwQ-32B emerged as a strong contender against DeepSeek-R1 and other larger reasoning models, challenging the dominance of high-parameter architectures with compact reasoning; Microsoft’s Phi-4 Multimodal model offered simultaneous processing of text, images, and speech; a U.S. court ruling rejected the fair use defense in the Thomson Reuters AI lawsuit, citing Ross's attempt to use copyrighted material to build a competing product; and Perplexity launched an uncensored version of DeepSeek-R1, raising discussions about AI safety and adapting open language models.

Subscribe to Data Points

ERNIE checks competitors with low prices: AI2’s OLMo2 32B may be the top fully open model

Twice a week, Data Points brings you the latest AI news, tools, models, and research in brief. In today’s edition, you’ll find:

 * OpenAI’s new SDK and APIs for agentic workflows
 * Olympic Coder, two powerful open coding models
 * Alibaba applies RL to emotion detection
 * GPT-4.5 and Claude Sonnet 3.7 top a new agent leaderboard

But first:

U.S. AI companies weigh in on federal Action Plan

Anthropic, Google, and OpenAI published policy proposals in response to U.S. President Trump calling for a national “AI Action Plan.” Both Google and OpenAI argued that the U.S. should implement fair use and data-mining exceptions to copyright restrictions, allowing AI companies to train their models on copyrighted material. OpenAI and Anthropic’s proposals both argued that certain Chinese models like DeepSeek should be restricted, with OpenAI calling them government-funded and Anthropic warning of biosecurity concerns. Other matters addressed include U.S. export controls on chips and other AI hardware and the broad regulations of the EU’s AI Act. (Anthropic, Google, and OpenAI)

Google unveils Gemma 3 family of smaller, open weight models

Google relesed Gemma 3, a set of multimodal models based on its Gemini 2.0 technology. The models range from 1 billion to 27 billion parameters and are designed to run on various devices, from smartphones to workstations. Gemma 3 supports over 140 languages and includes features like visual reasoning and a 128,000-token context window. Gemma 3 27B outperforms larger models like DeepSeek-V3 and Llama 3-405B on Chatbot Arena while remaining small enough to run on a single GPU. (Google)

OpenAI introduces new tools for AI agent development

OpenAI released new APIs and tools to help developers build AI agents. The new Responses API combines features from OpenAI’s existing Chat Completions and Assistants APIs, allowing models to use built-in tools like web search and file search. (OpenAI plans to phase out the Assistants API by mid-2026.) The company also launched an open-source Agents SDK for orchestrating multi-agent workflows. These tools aim to simplify the creation of AI systems that can perform complex tasks independently for developers using OpenAI models. (OpenAI)

OlympicCoder models excel at competitive programming tasks

Researchers affiliated with Open-R1 developed OlympicCoder, a set of 7B and 32B parameter models fine-tuned on competitive programming data that outperform some closed-source frontier models on challenging coding tasks. The models were trained on CodeForces-CoTs, a new dataset of nearly 100,000 high-quality coding samples distilled from DeepSeek-R1, and evaluated on a new benchmark using problems from the International Olympiad in Informatics (IOI). OlympicCoder-32B demonstrated particularly strong performance, surpassing all open weight models tested and even the much larger Claude Sonnet 3.7 on IOI problems. The new coding models are an important step in replicating the performance of DeepSeek’s R1 reasoning model using fully open data sets. (Hugging Face)

Alibaba releases AI model that can read emotions

Alibaba’s Tongyi Lab unveiled R1-Omni, an open vision model capable of inferring emotional states from video and audio inputs. The model, a reinforcement learning-enhanced version of the earlier HumanOmni, achieves state of the art performance on emotion recognition vision benchmarks. R1-Omni adds another nascent layer of understanding to vision models and is freely available on GitHub and Hugging Face. (GitHub)

Hugging Face’s smolagents evaluates top models for agents

Researchers launched a leaderboard to measure large language models’ effectiveness in powering AI agents, using a CodeAgent on various benchmarks. GPT-4.5 topped the rankings, outperforming specialized reasoning models, with Claude 3.7 Sonnet placing second. The leaderboard shows that all models achieve significant performance gains from agentic setups compared to vanilla LLMs, providing valuable insights for AI developers working on agent-based systems. (Hugging Face)

Still want to know more about what matters in AI right now?

Read this week’s issue of The Batch for in-depth analysis of news and research.

This week, Andrew Ng defended the importance of learning to code, arguing that as AI-assisted coding makes programming easier, more people should code—not fewer. He pushed back against claims that programming will become obsolete, arguing that understanding the “language of software” empowers individuals to work effectively with AI tools and maximize their impact.

“I see tech-savvy people coordinating AI tools to move toward being 10x professionals — individuals who have 10 times the impact of the average person in their field. I am increasingly convinced that the best way for many people to accomplish this is not to be just consumers of AI applications, but to learn enough coding to use AI-assisted coding tools effectively.”

Read Andrew’s full letter here.

Other top AI news and research stories we covered in depth: QwQ-32B emerged as a strong contender against DeepSeek-R1 and other larger reasoning models, challenging the dominance of high-parameter architectures with compact reasoning; Microsoft’s Phi-4 Multimodal model offered simultaneous processing of text, images, and speech; a U.S. court ruling rejected the fair use defense in the Thomson Reuters AI lawsuit, citing Ross's attempt to use copyrighted material to build a competing product; and Perplexity launched an uncensored version of DeepSeek-R1, raising discussions about AI safety and adapting open language models.

Subscribe to Data Points

AI giants’ U.S. policy proposals: Gemma 3 beats bigger open weight rivals



Loading the Elevenlabs Text to Speech AudioNative Player...



Large language models built by developers in China may, in some applications, be less useful outside that country because they avoid topics its government deems politically sensitive. A developer fine-tuned DeepSeek-R1 to widen its scope without degrading its overall performance.

What’s new: Perplexity released R1 1776, a version of DeepSeek-R1 that responds more freely than the original. The model weights are available to download under a commercially permissive MIT license.

How it works: The team modified DeepSeek-R1’s knowledge of certain topics by fine-tuning it on curated question-answer pairs.

 * Human experts identified around 300 topics that are censored in China.
 * The authors developed a multilingual classifier that spots text related to these topics.
 * They identified 40,000 prompts that the classifier classified as sensitive with high confidence. They discarded those that contained personally identifiable information.
 * For each prompt, they produced factual, chain-of-thought responses that mirrored DeepSeek-R1's typical reasoning processes.
 * They fine-tuned DeepSeek-R1 on the resulting prompt-response pairs.

Results: The fine-tuned model responded to politically charged prompts factually without degrading its ability to generate high-quality output.

 * The authors fed their model 1,000 diverse prompts that covered frequently censored topics. An unspecified combination of human and AI judges rated the models' responses according to the degree to which they are (i) evasive and (ii) censored outright.
 * 100 percent of the fine-tuned model’s responses were rated uncensored, whereas the original version censored around 85 percent of sensitive queries. By comparison, DeepSeek-V3 censored roughly 73 percent, Claude-3.5-Sonnet around 5 percent, o3-mini about 1 percent, and GPT-4o 0 percent.
 * Evaluated on four language and math benchmarks (MMLU, DROP, MATH-500, and AIME 2024) and unspecified internal benchmarks, the fine-tuned and original models performed nearly identically. Their scores differed by a few tenths of a percent except on AIME 2024 (competitive high-school math problems), where the fine-tuned model achieved 79.8 percent compared to the original’s 80.96 percent.

Behind the news: Among the first countries to regulate AI, China requires AI developers to build models that uphold “Core Socialist Values” and produce true and reliable output. When these objectives conflict, the political goal tends to dominate. While large language models built by developers in China typically avoid contentious topics, the newer DeepSeek models enforce this more strictly than older models like Qwen and Yi, using methods akin to Western measures for aligning output, like Reinforcement Learning from Human Feedback and keyword filters.

Why it matters: AI models tend to reflect their developers’ values and legal constraints. Perplexity’s targeted fine-tuning approach addresses this barrier to international adoption of open-source models.

We’re thinking: As models with open weights are adopted by the global community, they become a source of soft power for their developers, since they tend to reflect their developers’ values. This work reflects a positive effort to customize a model to reflect the user’s values instead — though how many developers will seek out a fine-tuned version rather than the original remains to be seen.

Tech & Society

DeepSeek-R1 Uncensored: Perplexity launches uncensored version of DeepSeek-R1



Loading the Elevenlabs Text to Speech AudioNative Player...



A United States court delivered a major ruling that begins to answer the question whether, and under what conditions, training an AI system on copyrighted material is considered fair use that doesn’t require permission.

What’s new: A U.S. Circuit judge ruled on a claim by the legal publisher Thomson Reuters that Ross Intelligence, an AI-powered legal research service, could not claim that training its AI system on materials owned by Thomson Reuters was a so-called “fair use.” Training the system did not qualify as fair use, he decided, because its output competed with Thomson Reuters’ publications.

How it works: Thomson Reuters had sued Ross Intelligence after the defendant trained an AI model using 2,243 works produced by Thomson Reuters without the latter’s permission. This ruling reversed an earlier decision in 2023, when the same judge had allowed Ross Intelligence’s fair-use defense to proceed to trial. In the new ruling, he found that Ross Intelligence’s use failed to meet the definition of fair use in key respects. (A jury trial is scheduled to determine whether Thomson Reuters' copyright was in effect at the time of the infringement and other aspects of the case.)

 * Ross Intelligence’s AI-powered service competed directly with Thomson Reuters, potentially undermining its market by offering a derivative product without licensing its works. Use in a competing commercial product undermines a key factor in fair use.
 * The judge found that Ross Intelligence’s use was commercial and not transformative, meaning it did not significantly alter or add new meaning to Thomson Reuters’ works — another key factor in fair use. Instead, it simply repackaged the works.
 * The ruling acknowledged that Thomson Reuters’ works were not highly creative but noted that they possessed sufficient originality for copyright protection due to the editorial creativity and judgment involved in producing it.
 * Although Ross Intelligence used only small portions of Thomson Reuters’ works, this did not weigh strongly in favor of fair use because those portions represented the most important summaries produced by Ross Intelligence.

Behind the news: The ruling comes amid a wave of lawsuits over AI training and copyright in several countries. Many of these cases are in progress, but courts have weighed in on some.

 * The New York Times is suing OpenAI and Microsoft, arguing that their models generate output that competes with its journalism.
 * Condé Nast, McClatchy, and other major publishers recently filed a lawsuit against Cohere, accusing it of using copyrighted news articles to train its AI models.
 * Sony, UMG, and Warner Music filed lawsuits against AI music companies including Suno and Udio for allegedly using copyrighted recordings without permission.
 * A judge dismissed key arguments brought by software developers who claimed that GitHub Copilot was trained on software they created in violation of open source licenses. The judge ruled in favor of Microsoft and OpenAI.
 * In Germany, the publisher of the LAION dataset won a case in which a court ruled that training AI models on publicly available images did not violate copyrights.

Why it matters: The question of whether training (or copying data to train) AI systems is a fair use of copyrighted works hangs over the AI industry, from academic research to commercial projects. In the wake of this ruling, courts may be more likely to reject a fair-use defense when AI companies train models on copyrighted material to create output that overlaps with or replaces traditional media, as The New York Times alleges in its lawsuit against OpenAI. However, the ruling leaves room for fair use with respect to models whose output doesn’t compete directly with copyrighted works.

We’re thinking: Current copyright laws weren’t designed with AI in mind, and rulings like this one fill in the gaps case by case. Clarifying copyright for the era of generative AI could help our field move forward faster.

Judge Upholds Copyright in AI Training Case: U.S. court rejects fair use defense in Thomson Reuters AI lawsuit



Loading the Elevenlabs Text to Speech AudioNative Player...



Microsoft debuted its first official large language model that responds to spoken input.

What’s new: Microsoft released Phi-4-multimodal, an open weights model that processes text, images, and speech simultaneously. 

 * Input/output: Text, speech, images in (up to 128,000 tokens); text out (0.34 seconds to first token, 26 tokens per second)
 * Performance: State of the art in speech transcription. Comparable to similar models in other tasks
 * Knowledge cutoff: June 2024
 * Architecture: transformer, 5.6 billion parameters
 * Features: Text-image-speech processing, multilingual, tool use.
 * Undisclosed: Training datasets, output size
 * The company also released Phi-4-mini, an open weights 3.8 billion-parameter version of its biggest large language model (LLM), Phi-4. Phi-4-mini outperforms larger models including Llama 3.1 8B and Ministral-2410 8B on some benchmarks.
 * Availability/price: Weights are free to download for noncommercial and commercial use under a MIT license.

How it works: Phi-4-multimodal has six components: Phi-4-mini, vision and speech encoders as well as corresponding projectors (which modify the vision or speech embeddings so the base model can understand them), and two LoRA adapters. The LoRA adapters modify the base weights depending on the input: One adapter modifies them for speech-text problems, and one for vision-text and vision-speech problems.

 * The speech encoder is a Conformer (which combines convolutional layers with a transformer) and the speech projector is a vanilla neural network. They trained Phi-4-multimodal to convert 2 million hours of speech to text, modifying only the speech encoder and projector. They further trained the system to convert speech to text, translate speech to other languages, summarize speech, and answer questions about speech, modifying only the speech encoder and the speech-text LoRA adapter.
 * The vision encoder is based on a pretrained SigLIP-400M vision transformer, and the vision projector is a vanilla neural network. They trained the model to process text and images in four stages: (i) They trained Phi-4-multimodal to caption images, modifying only the vision projector. (ii) They trained the system on 500 billion tokens to caption images, transcribe text in images, and perform other tasks, modifying only the vision encoder and projector. (iii) They trained the system to answer questions about images, charts, tables, and diagrams and to transcribe text in images, modifying the vision encoder, project, and vision-text LoRA adapter. (iv) Finally, they trained the system to compare images and summarize videos, modifying only the vision projector and vision-text LoRA adapter.
 * To adapt Phi-4-multimodal for images and speech, they trained the system to generate the text responses to a subset of the text-vision data that had been converted to speech-image using a proprietary text-to-speech engine, modifying only the text-vision LoRA adapter, vision encoder, and vision projector.
 * Example inference: Given a question as speech and an image, the audio encoder and projector convert the speech to tokens, and the image encoder and projector convert the image into tokens. Given the tokens, Phi-4-multimodal, which uses the weights of Phi-4-mini modified by the vision-text/vision-speech LoRA adapter, generates a text response.

Results: The authors compared Phi-4-multimodal to other multimodal models on text-vision, vision-speech, text-speech tasks.

 * Across 11 text-vision benchmarks, Phi-4-multimodal came in fourth out of 11 models. It outperformed Qwen2.5-VL-3B, Claude 3.5 Sonnet, and GPT 4o-mini. It trailed Qwen2.5-VL-7B, GPT-4o, and Gemini-2 Flash.
 * Across four vision-speech benchmarks, Phi-4-multimodal outperformed by at least 6 percentage points Gemini-2.0-Flash, Gemini-2.0-Flash-Lite-preview, and InternOmni.
 * Phi-4-multimodal outperformed all competitors in Microsoft’s report (including Qwen2-audio, Gemini 2.0 Flash, and GPT-4o) at transcribing speech from text in three datasets. It also achieved competitive performance in speech translation, outperforming its competitors on two of four datasets.

Behind the news: This work adds to the growing body of models with voice-in/text-out capability, including the open weights DiVA model developed by a team led by Diyi Yang at Stanford University.

Why it matters: The architectural options continue to expand for building neural networks that process text, images, audio, and various combinations. While some teams maintain separate models for separate data modalities, like Qwen2.5 (for text) and Qwen2.5-VL) (for vision-language tasks), others are experimenting with mixture-of-expert models like DeepSeek-V3. Phi-4-multimodal shows that Mixture-of-LoRAs is an effective approach for processing multimodal data — and gives developers a couple of new open models to play with.

We’re thinking: Output guardrails have been built to ensure appropriateness of text output, but this is difficult to apply to a voice-in/voice-out architecture. (Some teams have worked on guardrails that screen audio output directly, but the technology is still early.) For voice-based applications, a voice-in/text-out model can generate a candidate output without a separate, explicit speech-to-text step, and it accommodates text-based guardrails before it decides whether or not to read the output to the user.

Microsoft Tackles Voice-In, Text-Out: Microsoft’s Phi-4 Multimodal model can process text, images, and speech simultaneously



Loading the Elevenlabs Text to Speech AudioNative Player...



Most models that have learned to reason via reinforcement learning were huge models. A much smaller model now competes with them.

What’s new: Alibaba introduced QwQ-32B, a large language model that rivals the reasoning prowess of DeepSeek-R1 despite its relatively modest size.

 * Input/output: Text in (up to 131,072 tokens), text out
 * Architecture: Transformer, 32.5 billion total parameter
 * Performance: Outperforms OpenAI o1-mini and DeepSeek-R1 on some bencharks
 * Features: Chain-of-thought reasoning, function calling, multilingual in 29 languages
 * Undisclosed: Output size, training data
 * Availability/price: Free via Qwen Chat. Weights are free to download for noncommercial and commercial uses under an Apache 2.0 license.

How it works: QwQ-32B is a version of Qwen2.5-32B that was fine-tuned to generate chains of thought using reinforcement learning (RL). Fine-tuning proceeded in two stages.

 * The first stage of RL fine-tuning focused on math and coding tasks. The model earned rewards for correct final outcomes (no partial credit for intermediate steps). An accuracy verifier checked its math solutions, while a code-execution server verified generated code for predefined test cases.
 * The second stage encouraged the model to follow instructions, use tools, and align its values with human preferences while maintaining math and coding performance, again rewarding final outcomes. In this stage, the model earned rewards from an unspecified reward model and some rule-based verifiers.

Performance: On several benchmarks for math, coding, and general problem solving, QwQ-32B outperforms OpenAI o1-mini (parameter count undisclosed) and achieves performance roughly comparable to DeepSeek-R1 (671 billion parameters, 37 billion active at any moment).

 * On AIME24 (high-school competition math problems), QwQ-32B achieved 79.5 percent accuracy, well ahead of o1-mini (63.6 percent) but slightly behind DeepSeek-R1 (79.8 percent).
 * On LiveCodeBench (code generation, repair, and testing), QwQ-32B achieved 63.4 percent, outperforming o1-mini (53.8 percent) but trailing DeepSeek-R1 (65.9 percent).
 * On LiveBench (problem-solving in math, coding, reasoning, and data analysis), QwQ-32B reached 73.1 percent, ahead of o1-mini (59.1 percent) and DeepSeek-R1 (71.6 percent).
 * On IFEval (following instructions), QwQ-32B achieved 83.9 percent, outperforming DeepSeek-R1 (83.8 percent) but behind o1-mini (84.8 percent).
 * On BFCL (function calling), QwQ-32B achieved 66.4 percent, better than DeepSeek-R1 (60.3 percent), and o1-mini (62.8 percent).

Behind the news: DeepSeek’s initial model, DeepSeek-R1-Zero, similarly applied RL to a pretrained model. That effort produced strong reasoning but poor readability (for example, math solutions with correct steps but jumbled explanations). To address this shortcoming, the team fine-tuned DeepSeek-R1 on long chain-of-thought examples before applying RL. In contrast, QwQ-32B skipped preliminary fine-tuning and applied RL in two stages, first optimizing for correct responses and then for readability.

Why it matters: RL can dramatically boost LLMs’ reasoning abilities, but the order in which different behaviors are rewarded matters. Using RL in stages enabled the team to build a 32 billion parameter model — small enough to run locally on a consumer GPU — that rivals a much bigger mixture-of-experts model, bringing powerful reasoning models within reach for more developers. The Qwen team plans to scale its RL approach to larger models, which could improve the next-gen reasoning abilities further while adding greater knowledge.

We’re thinking: How far we’ve come since “Let’s think step by step”!

Compact Reasoning: QwQ-32B challenges DeepSeek-R1 and other larger reasoning models



Loading the Elevenlabs Text to Speech AudioNative Player...



Dear friends,

Some people today are discouraging others from learning programming on the grounds AI will automate it. This advice will be seen as some of the worst career advice ever given. I disagree with the Turing Award and Nobel prize winner who wrote, “It is far more likely that the programming occupation will become extinct  than that it will become all-powerful. More and more, computers will program themselves.” Statements discouraging people from learning to code are harmful!

In the 1960s, when programming moved from punchcards (where a programmer had to laboriously make holes in physical cards to write code character by character) to keyboards with terminals, programming became easier. And that made it a better time than before to begin programming. Yet it was in this era that Nobel laureate Herb Simon wrote the words quoted in the first paragraph. Today’s arguments not to learn to code continue to echo his comment.

As coding becomes easier, more people should code, not fewer!

Over the past few decades, as programming has moved from assembly language to higher-level languages like C, from desktop to cloud, from raw text editors to IDEs to AI assisted coding where sometimes one barely even looks at the generated code (which some coders recently started to call vibe coding), it is getting easier with each step. (By the way, to learn more about AI assisted coding, check out our video-only short course, “Build Apps with Windsurf’s AI Coding Agents.”)

I wrote previously that I see tech-savvy people coordinating AI tools to move toward being 10x professionals — individuals who have 10 times the impact of the average person in their field. I am increasingly convinced that the best way for many people to accomplish this is not to be just consumers of AI applications, but to learn enough coding to use AI-assisted coding tools effectively.

One question I’m asked most often is what someone should do who is worried about job displacement by AI. My answer is: Learn about AI and take control of it, because one of the most important skills in the future will be the ability to tell a computer exactly what you want, so it can do that for you. Coding (or getting AI to code for you) is the best way to do that.

When I was working on the course Generative AI for Everyone and needed to generate AI artwork for the background images, I worked with a collaborator who had studied art history and knew the language of art. He prompted Midjourney with terminology based on the historical style, palette, artist inspiration and so on — using the language of art — to get the result he wanted. I didn’t know this language, and my paltry attempts at prompting could not deliver as effective a result.

Similarly, scientists, analysts, marketers, recruiters, and people of a wide range of professions who understand the language of software through their knowledge of coding can tell an LLM or an AI-enabled IDE what they want much more precisely, and get much better results. As these tools continue to make coding easier, this is the best time yet to learn to code, to learn the language of software, and learn to make computers do exactly what you want them to do.

Keep building!

Andrew 

Learn the Language of Software: AI won’t kill programming. There has never been a better time to start coding.

Twice a week, Data Points brings you the latest AI news, tools, models, and research in brief. In today’s edition, you’ll find:

 * Music and lyrics in one diffusion model
 * Manus AI’s impressive demos spark excitement and backlash
 * OpenAI sees AGI as a gradual evolution
 * Google unveils its first Gemini-branded embedding models

But first:

EAGLE-3 introduces new techniques for accelerating inference

Researchers at Peking University, Microsoft, and elsewhere developed EAGLE-3, an updated method for speculative sampling that aims to speed up large language model inference. The approach removes feature prediction constraints and introduces a “training-time test” technique for directly predicting draft tokens. EAGLE-3 also incorporates a fusion of low, middle, and high-level features from the target model, moving beyond the use of only top-layer features. Experiments show EAGLE-3 achieves faster inference speeds compared to standard autoregressive decoding and previous speculative sampling methods across various tasks and model sizes. (arXiv)

Reinforcement learning pioneers honored with top computing award 

Andrew Barto and Richard Sutton won the 2024 Turing Award for their groundbreaking work on reinforcement learning, a method for AI systems to learn from digital rewards and punishments. Their research, which began in the late 1970s, laid the foundation for major AI breakthroughs like AlphaGo and ChatGPT. The $1 million prize acknowledges Barto and Sutton’s role in developing a fundamental AI technique that continues to shape the field’s rapid advancement and future potential. (Association for Computing Machinery)

DiffRhythm generates full-length songs within seconds

Chinese researchers developed DiffRhythm, a diffusion-based model capable of generating complete songs up to 4 minutes 45 seconds long, including both vocals and accompaniment. The model uses a variational autoencoder to compress audio into latent representations, which are then generated by a diffusion transformer conditioned on lyrics and style prompts. DiffRhythm can produce high-quality 4-minute songs in just 10 seconds, significantly faster than previous language model approaches. The researchers released their model and code under a noncommercial license. (GitHub and arXiv)

Manus AI attracts plenty of attention but little consensus

Chinese startup The Butterfly Effect launched Manus, an AI agent platform that uses Claude and various undisclosed models to autonomously perform complex tasks without human oversight. Despite generating significant buzz and comparisons to breakthroughs like DeepSeek, some early users report Manus struggling with basic requests and crashing frequently. Manus is still in private preview, and the widely differing reports seem to stem from a combination of users’ limited access and very different expectations. (Manus, Forbes, and TechCrunch)

OpenAI outlines its evolving approach to AI safety and alignment

OpenAI detailed its current principles for ensuring artificial general intelligence benefits humanity. The company now views AGI development as a continuous process rather than a sudden leap, emphasizing iterative deployment to learn from real-world usage. OpenAI’s core safety principles include embracing uncertainty, layering multiple safeguards, developing scalable alignment methods, maintaining human control, and collaborating with the wider AI community. (OpenAI)

Gemini Embedding model tops multilingual benchmarks

Google unveiled a new experimental Gemini Embedding text model, available through the Gemini API, which outperforms previous models and tops the Massive Text Embedding Benchmark Multilingual leaderboard. The model features an 8K token input limit, 3K output dimensions, and supports over 100 languages, making it applicable for diverse tasks like retrieval augmented generation and text classification. This release gives developers early access to Gemini Embedding capabilities, with Google working towards a stable, generally available version in the coming months. (Google)

Still want to know more about what matters in AI right now?

Read last week’s issue of The Batch for in-depth analysis of news and research.

Last week, Andrew Ng discussed the challenges of Voice Activity Detection (VAD) in noisy environments and highlighted Moshi, a model that continuously listens and decides when to speak, eliminating the need for explicit turn-taking detection. He emphasized ongoing innovations in voice AI and the potential for improved voice-to-voice interactions.

“Just as the architecture of text-only transformers has gone through many evolutions (such as encoder-decoder models, decoder-only models, and reasoning models that generate a lot of ‘reasoning tokens’ before the final output), voice models are going through a lot of architecture explorations.”

Read Andrew’s full letter here.

Other top AI news and research stories we covered in depth: Mercury Coder released a fast text generator with a non-transformer architecture, introducing what may be the first commercially available Language Diffusion Model; OpenAI unveiled GPT-4.5, its most powerful non-reasoning model to date, promising enhanced performance and efficiency; Claude 3.7 Sonnet introduced a budget for reasoning tokens, a hybrid approach to reasoning models; and Amazon launched Alexa+, integrating generative AI and intelligent agents powered by Claude and other models to create a more advanced voice assistant.

Subscribe to Data Points

EAGLE-3 speeds up language models: And the 2024 Turing Award goes to…

Twice a week, Data Points brings you the latest AI news, tools, models, and research in brief. In today’s edition, you’ll find:

 * Cohere’s open vision models support many languages
 * Jamba 1.6’s two hybrid MoE models promise more speed
 * Anthropic overhauls its developer console for Claude Sonnet 3.7
 * Mistral brings its multilingual/multimedia skills to OCR

But first:

Qwen applies reinforcement learning to a smaller language model

Alibaba’s Qwen released QwQ-32B, a 32 billion parameter reasoning model that matches the performance of larger models like DeepSeek-R1 and o1-mini. The model, based on Qwen 2.5, excels at tasks like mathematical reasoning and coding, and incorporates agent-like capabilities for self-criticism and tool use. The model is available for download at Hugging Face and ModelScope, and in environments like Ollama, under an Apache 2.0 license. QwQ-32B shows the potential of scaled reinforcement learning to powerfully enhance AI capabilities, even with relatively modest model sizes. (GitHub and Hugging Face)

Sesame unveils expressive, context-aware speech system

Sesame introduced the Conversational Speech Model (CSM), an end-to-end multimodal learning system designed to generate more natural and contextually appropriate AI speech. The model uses transformers to process both text and audio inputs, leveraging conversation history to produce coherent speech with improved expressivity and efficiency. Sesame’s work addresses limitations in current text-to-speech systems and aims to create AI companions with “voice presence” that can engage in genuine dialogue. The company released a demo and made its models available under an Apache 2.0 license. (Sesame)

Cohere releases multilingual vision-language models

Cohere introduced Aya Vision, a family of open weight multimodal models designed to understand language and images across 23 languages. The 8B and 32B parameter models outperform larger competitors on multilingual benchmarks by leveraging techniques like synthetic annotations, data scaling, and multimodal model merging. This release adds strong multilingual capabilities to multimodal AI models, potentially enabling more inclusive and globally accessible AI applications. (Cohere and Hugging Face)

Jamba’s hybrid architecture gets a model update

AI21 Labs released Jamba 1.6, an open language mixture-of-experts model family with a hybrid Mamba-transformer architecture. The company reports that Jamba Large 1.6 (398 billion parameters, 94 billion active) outperforms Mistral Large 2, Llama 3.3 70B, and Command R+ on ArenaHard, LongBench, and other benchmarks, while Jamba Mini 1.6 (52 billion parameters, 12 billion active) surpasses Ministral 8B, Llama 3.1 8B, and Command R7B. AI21 Labs highlights Jamba 1.6’s 256K token context window, its speed, and its performance on RAG and long-context question-answering tasks. (AI21 Labs and Hugging Face)

Anthropic upgrades developer console with new collaboration features

Anthropic redesigned its console to streamline AI development with Claude, adding features like shareable prompts for team collaboration and support for the Claude 3.7 Sonnet model. The console now offers tools to write, evaluate, and optimize prompts, including automatic prompt generation and refinement capabilities. These upgrades aim to help developers build more reliable AI applications by improving prompt quality and enabling better teamwork across organizations. (Anthropic)

Mistral introduces new OCR API for advanced document processing

Mistral OCR extracts content from complex text-and-image documents, outperforming competitors like Microsoft Azure and Gemini 2.0 Flash in speed and accuracy benchmarks across various document types and languages. The API processes up to 2000 pages per minute, supports document-as-prompt functionality, and offers structured output options. Improved OCR enables organizations to unlock insights from their document repositories, potentially accelerating research, preserving cultural heritage, and improving customer service –  as well as making it easier for them to be processed by AI. (Mistral)

Still want to know more about what matters in AI right now?

Read this week’s issue of The Batch for in-depth analysis of news and research.

This week, Andrew Ng discussed the challenges of Voice Activity Detection (VAD) in noisy environments and highlighted Moshi, a model that continuously listens and decides when to speak, eliminating the need for explicit turn-taking detection. He emphasized ongoing innovations in voice AI and the potential for improved voice-to-voice interactions.

“Given the importance of foundation models with voice-in and voice-out capabilities, many large companies right now are investing in developing better voice models. I’m confident we’ll see many more good voice models released this year.”

Read Andrew’s full letter here.

Other top AI news and research stories we covered in depth: Mercury Coder released a fast text generator with a non-transformer architecture, introducing what may be the first commercially available Language Diffusion Model; OpenAI unveiled GPT-4.5, its most powerful non-reasoning model to date, promising enhanced performance and efficiency; Claude 3.7 Sonnet introduced a budget for reasoning tokens, a hybrid approach to reasoning models; and Amazon launched Alexa+, integrating generative AI and intelligent agents powered by Claude and other models to create a more advanced voice assistant.

Subscribe to Data Points

Qwen’s mid-sized reasoning model scores big: Sesame moves through speech models’ “uncanny valley”



Loading the Elevenlabs Text to Speech AudioNative Player...



Amazon announced Alexa+, a major upgrade to its long-running voice assistant.

What’s new: Alexa+, which accepts spoken commands and responds conversationally, is designed to work with a variety of vendors as an autonomous agent to make purchases, book reservations, play media, and so on. It will roll out in the U.S. over coming weeks, initially on some Echo Show devices and eventually nearly every current Echo speaker.

How it works: Alexa+ updates the system to take advantage of generative AI including Anthropic Claude, Amazon Nova, and other large language models. Inputs are filtered through a routing system that determines the best model to respond to any given request. It’s trained to understand colloquial, conversational language. Its personality is designed to be “smart, considerate, empathetic, and inclusive” as well as humorous.

 * Alexa+  interacts with online vendors to manage smart-home devices (Philips Hue, Ring, Roborock), reserve restaurant seats (OpenTable, Vagaro), play music (Amazon Music, Spotify, Apple Music, iHeartRadio) and videos (Amazon Video, Hulu, Netflix, Disney+), book local service technicians (Thumbtack), and purchase items (Amazon Fresh, Whole Foods, Grubhub, Uber Eats, Ticketmaster). Amazon+ will cost $19.99 per month, free with an Amazon Prime membership ($139 per year). (Disclosure: Andrew Ng is a member of Amazon’s board of directors.)
 * The system recognizes individual users and keeps track of personalized information such as dates; recipes, and preferences in sports, food, music, and movies. In addition, it can respond to queries based on purchase records, video and music playbacks, shipping addresses, documents, emails, photos, messages, and so on.
 * It can behave proactively, for instance, advising users to start their commute early if traffic is heavy.
 * The system calls what Amazon calls experts — groups of systems, APIs, and instructions — that orchestrate API calls to accomplish online tasks. For instance, it can navigate and use the web to perform tasks such as finding and booking, say, a local repair service to fix a broken household appliance.
 * Alexa+ can deliver timely news and information based on partnerships with news sources including Associated Press, Business Insider, Politico, Reuters, USA Today, and The Washington Post.

Behind the news: Amazon launched Alexa in 2014, and the voice assistant now resides in over 600 million devices worldwide. However, users relied on it more to set timers, report sports scores, and play music than to purchase products, and Alexa revenue lagged. Following cutbacks in 2021, Amazon made multibillion-dollar investments in Anthropic and set about updating the technology for the generative AI era.

Why it matters: Alexa, along with Apple’s Siri and Google Assistant, pioneered the market for voice assistants. However, as large language models (LLMs) blossomed, all three systems fell behind the times. (Google allows Android users to substitute one of its Gemini LLMs for Google Assistant, but the system still calls Google Assistant for some tasks.) Alexa+ is the first major voice-assistant update that aims to take advantage of LLMs as well as emerging agentic technology and improved voice interactions, and the rollout is taking these capabilities to a large, existing user base.

We’re thinking: Rapid improvements in the voice stack are opening doors not only for voice assistants but for a galaxy of applications that rely on spoken input and output. Product designers will need to learn how to design smooth user voice experiences. Watching how Alexa+ manages them will provide useful guidelines.