Chatbots Against Depression: The Trevor Project used GPT-2 to train crisis counselors.
Transformer

Chatbots Against Depression: The Trevor Project used GPT-2 to train crisis counselors.

A language model is helping crisis-intervention volunteers practice their suicide-prevention skills. The Trevor Project, a nonprofit organization that operates a 24-hour hotline for LGBTQ youth, uses a “crisis contact simulator” to train its staff in how to talk with troubled teenagers.

March 24, 20211 min read
Transformer Variants Head to Head: A benchmark for comparing different AI transformers.
Transformer

Transformer Variants Head to Head: A benchmark for comparing different AI transformers.

The transformer architecture has inspired a plethora of variations. Yet researchers have used a patchwork of metrics to evaluate their performance, making them hard to compare. New work aims to level the playing field.

March 3, 20212 min read
Transformer Variants Head to Head: A benchmark for comparing different AI transformers.
Transformer

Transformer Variants Head to Head: A benchmark for comparing different AI transformers.

The transformer architecture has inspired a plethora of variations. Yet researchers have used a patchwork of metrics to evaluate their performance, making them hard to compare. New work aims to level the playing field.

March 3, 20212 min read
Sharper Eyes For Vision+Language: AI research shows improved image and text matching.
Transformer

Sharper Eyes For Vision+Language: AI research shows improved image and text matching.

Models that interpret the interplay of words and images tend to be trained on richer bodies of text than images. Recent research worked toward giving such models a more balanced knowledge of the two domains.

February 24, 20212 min read
Sharper Eyes For Vision+Language: AI research shows improved image and text matching.
Transformer

Sharper Eyes For Vision+Language: AI research shows improved image and text matching.

Models that interpret the interplay of words and images tend to be trained on richer bodies of text than images. Recent research worked toward giving such models a more balanced knowledge of the two domains.

February 24, 20212 min read
Bigger, Faster Transformers: Increasing parameters without slowing down transformers
Transformer

Bigger, Faster Transformers: Increasing parameters without slowing down transformers

Performance in language tasks rises with the size of the model — yet, as a model’s parameter count rises, so does the time it takes to render output. New work pumps up the number of parameters without slowing down the network.

February 24, 20212 min read
Bigger, Faster Transformers: Increasing parameters without slowing down transformers
Transformer

Bigger, Faster Transformers: Increasing parameters without slowing down transformers

Performance in language tasks rises with the size of the model — yet, as a model’s parameter count rises, so does the time it takes to render output. New work pumps up the number of parameters without slowing down the network.

February 24, 20212 min read
Better Zero-Shot Translations: A method for improving transformer NLP translation
Transformer

Better Zero-Shot Translations: A method for improving transformer NLP translation

Train a multilingual language translator to translate between Spanish and English and between English and German, and it may be able to translate directly between Spanish and German as well. New work proposes a simple path to better machine translation between languages.

February 17, 20212 min read
Better Zero-Shot Translations: A method for improving transformer NLP translation
Transformer

Better Zero-Shot Translations: A method for improving transformer NLP translation

Train a multilingual language translator to translate between Spanish and English and between English and German, and it may be able to translate directly between Spanish and German as well. New work proposes a simple path to better machine translation between languages.

February 17, 20212 min read
Better Language Through Vision: Study improved Bert performance using visual tokens.
Transformer

Better Language Through Vision: Study improved Bert performance using visual tokens.

For children, associating a word with a picture that illustrates it helps them learn the word’s meaning. Research aims to do something similar for machine learning models. Researchers improved a BERT model’s performance on some language tasks by training it on a large dataset of image-word pairs.

February 10, 20212 min read
Better Language Through Vision: Study improved Bert performance using visual tokens.
Transformer

Better Language Through Vision: Study improved Bert performance using visual tokens.

For children, associating a word with a picture that illustrates it helps them learn the word’s meaning. Research aims to do something similar for machine learning models. Researchers improved a BERT model’s performance on some language tasks by training it on a large dataset of image-word pairs.

February 10, 20212 min read
Language Models Want to Be Free: How EleutherAI is developing a GPT-3 clone.
Transformer

Language Models Want to Be Free: How EleutherAI is developing a GPT-3 clone.

A grassroots research collective aims to make a GPT-3 clone that’s available to everyone. EleutherAI, a loose-knit group of independent researchers, is developing GPT-Neo, an open source, free-to-use version of OpenAI’s gargantuan language model.

February 3, 20211 min read
Language Models Want to Be Free: How EleutherAI is developing a GPT-3 clone.
Transformer

Language Models Want to Be Free: How EleutherAI is developing a GPT-3 clone.

A grassroots research collective aims to make a GPT-3 clone that’s available to everyone. EleutherAI, a loose-knit group of independent researchers, is developing GPT-Neo, an open source, free-to-use version of OpenAI’s gargantuan language model.

February 3, 20211 min read
Quake Watch: AI model detects earthquakes and estimates epicenters.
Transformer

Quake Watch: AI model detects earthquakes and estimates epicenters.

Detecting earthquakes is an important step toward warning surrounding communities that damaging seismic waves may be headed their way. A new model detects tremors and provides clues to their epicenter.

January 27, 20212 min read
Quake Watch: AI model detects earthquakes and estimates epicenters.
Transformer

Quake Watch: AI model detects earthquakes and estimates epicenters.

Detecting earthquakes is an important step toward warning surrounding communities that damaging seismic waves may be headed their way. A new model detects tremors and provides clues to their epicenter.

January 27, 20212 min read

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox