Big Bot Makes Small Talk: A research summary of Facebook's Generative BST chatbot
Transformer

Big Bot Makes Small Talk: A research summary of Facebook's Generative BST chatbot

Facebook recently rolled out its entry in the World’s Biggest Chatbot sweepstakes. In keeping with the company’s social-networking dominance, the bot is designed to excel at chitchat on any subject.

May 27, 20202 min read
Toward Open-Domain Chatbots: Meena Scores High on System for Grading NLP Chatbots
Transformer

Toward Open-Domain Chatbots: Meena Scores High on System for Grading NLP Chatbots

Progress in language models is spawning a new breed of chatbots and, unlike their narrow-domain forebears, they have the gift of gab. Recent research tests the limits of conversational AI.

May 27, 20202 min read
Toward Open-Domain Chatbots: Meena Scores High on System for Grading NLP Chatbots
Transformer

Toward Open-Domain Chatbots: Meena Scores High on System for Grading NLP Chatbots

Progress in language models is spawning a new breed of chatbots and, unlike their narrow-domain forebears, they have the gift of gab. Recent research tests the limits of conversational AI.

May 27, 20202 min read
Roll Over, Beyoncé: How OpenAI's Jukebox generates synthetic music
Transformer

Roll Over, Beyoncé: How OpenAI's Jukebox generates synthetic music

A new generative model croons like Elvis and raps like Eminem. It might even make you think you’re listening to a lost demo by the Beatles. OpenAI released Jukebox, a deep learning system that has generated thousands of songs in styles from country to metal and soul.

May 6, 20202 min read
Roll Over, Beyoncé: How OpenAI's Jukebox generates synthetic music
Transformer

Roll Over, Beyoncé: How OpenAI's Jukebox generates synthetic music

A new generative model croons like Elvis and raps like Eminem. It might even make you think you’re listening to a lost demo by the Beatles. OpenAI released Jukebox, a deep learning system that has generated thousands of songs in styles from country to metal and soul.

May 6, 20202 min read
Transformers Transformed: Research improves transformer efficiency with Reformer.
Transformer

Transformers Transformed: Research improves transformer efficiency with Reformer.

Transformer networks have revolutionized natural language processing, but they hog processor cycles and memory. New research demonstrates a more frugal variation.

March 4, 20202 min read
Transformers Transformed: Research improves transformer efficiency with Reformer.
Transformer

Transformers Transformed: Research improves transformer efficiency with Reformer.

Transformer networks have revolutionized natural language processing, but they hog processor cycles and memory. New research demonstrates a more frugal variation.

March 4, 20202 min read
Bot Comic: How Google's Meena chatbot developed a sense of humor
Transformer

Bot Comic: How Google's Meena chatbot developed a sense of humor

Androids may not dream of electric sheep, but some crack jokes about horses and cows. Meena, a 2.6-billion parameter chatbot developed by Google Brain, showed impressive conversational ability, discussing a variety of topics.

February 5, 20202 min read
Bot Comic: How Google's Meena chatbot developed a sense of humor
Transformer

Bot Comic: How Google's Meena chatbot developed a sense of humor

Androids may not dream of electric sheep, but some crack jokes about horses and cows. Meena, a 2.6-billion parameter chatbot developed by Google Brain, showed impressive conversational ability, discussing a variety of topics.

February 5, 20202 min read
Neural Networks Study Math: A sequence to sequence model for solving math problems.
Transformer

Neural Networks Study Math: A sequence to sequence model for solving math problems.

In tasks that involve generating natural language, neural networks often map an input sequence of words to an output sequence of words. Facebook researchers used a similar technique on sequences of mathematical symbols, training a model to map math problems to math solutions.

January 15, 20202 min read
Here Be Dragons: AI Dungeon 2 generated the script for a short movie.
Transformer

Here Be Dragons: AI Dungeon 2 generated the script for a short movie.

AI is contributing to paintings, music, and now a whimsical fantasy video. The Squire is an amateur romp through a snowy realm of knights in armor and damsels in distress. The script was composed by AI Dungeon 2, an interactive text-adventure game based on the GPT-2 language model.

January 15, 20201 min read
Neural Networks Study Math: A sequence to sequence model for solving math problems.
Transformer

Neural Networks Study Math: A sequence to sequence model for solving math problems.

In tasks that involve generating natural language, neural networks often map an input sequence of words to an output sequence of words. Facebook researchers used a similar technique on sequences of mathematical symbols, training a model to map math problems to math solutions.

January 15, 20202 min read
Here Be Dragons: AI Dungeon 2 generated the script for a short movie.
Transformer

Here Be Dragons: AI Dungeon 2 generated the script for a short movie.

AI is contributing to paintings, music, and now a whimsical fantasy video. The Squire is an amateur romp through a snowy realm of knights in armor and damsels in distress. The script was composed by AI Dungeon 2, an interactive text-adventure game based on the GPT-2 language model.

January 15, 20201 min read
Language Modeling on One GPU: Single-headed attention competes with transformers.
Transformer

Language Modeling on One GPU: Single-headed attention competes with transformers.

The latest large, pretrained language models rely on trendy layers based on transformer networks. New research shows that these newfangled layers may not be necessary.

January 8, 20202 min read
Language Modeling on One GPU: Single-headed attention competes with transformers.
Transformer

Language Modeling on One GPU: Single-headed attention competes with transformers.

The latest large, pretrained language models rely on trendy layers based on transformer networks. New research shows that these newfangled layers may not be necessary.

January 8, 20202 min read

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox