Outing Hidden Hatred: How Facebook built a hate speech detector
BERT

Outing Hidden Hatred: How Facebook built a hate speech detector

Facebook uses automated systems to block hate speech, but hateful posts can slip through when seemingly benign words and pictures combine to create a nasty message. The social network is tackling this problem by enhancing AI’s ability to recognize context.

June 17, 20202 min read
Choosing Words Carefully: BLUERT trains language models to be better translators.
BERT

Choosing Words Carefully: BLUERT trains language models to be better translators.

The words “big” and “large” have similar meanings, but they aren’t always interchangeable: You wouldn’t refer to an older, male sibling as your “large brother” (unless you meant to be cheeky). Choosing among words with similar meanings is critical in language tasks like translation.

June 17, 20202 min read
Choosing Words Carefully: BLUERT trains language models to be better translators.
BERT

Choosing Words Carefully: BLUERT trains language models to be better translators.

The words “big” and “large” have similar meanings, but they aren’t always interchangeable: You wouldn’t refer to an older, male sibling as your “large brother” (unless you meant to be cheeky). Choosing among words with similar meanings is critical in language tasks like translation.

June 17, 20202 min read
Language Modeling on One GPU: Single-headed attention competes with transformers.
BERT

Language Modeling on One GPU: Single-headed attention competes with transformers.

The latest large, pretrained language models rely on trendy layers based on transformer networks. New research shows that these newfangled layers may not be necessary.

January 8, 20202 min read
Language Modeling on One GPU: Single-headed attention competes with transformers.
BERT

Language Modeling on One GPU: Single-headed attention competes with transformers.

The latest large, pretrained language models rely on trendy layers based on transformer networks. New research shows that these newfangled layers may not be necessary.

January 8, 20202 min read
Yann LeCun — Learning From Observation: The power of self-supervised learning
BERT

Yann LeCun — Learning From Observation: The power of self-supervised learning

How is it that many people learn to drive a car fairly safely in 20 hours of practice, while current imitation learning algorithms take hundreds of thousands of hours, and reinforcement learning algorithms take millions of hours? Clearly we’re missing something big.

January 1, 20202 min read
Yann LeCun — Learning From Observation: The power of self-supervised learning
BERT

Yann LeCun — Learning From Observation: The power of self-supervised learning

How is it that many people learn to drive a car fairly safely in 20 hours of practice, while current imitation learning algorithms take hundreds of thousands of hours, and reinforcement learning algorithms take millions of hours? Clearly we’re missing something big.

January 1, 20202 min read
Natural Language Processing Models Get Literate: Why 2019 was a breakthrough year for NLP
BERT

Natural Language Processing Models Get Literate: Why 2019 was a breakthrough year for NLP

Earlier language models powered by Word2Vec and GloVe embeddings yielded confused chatbots, grammar tools with middle-school reading comprehension, and not-half-bad translations. The latest generation is so good, some people consider it dangerous.

December 24, 20192 min read
Natural Language Processing Models Get Literate: Why 2019 was a breakthrough year for NLP
BERT

Natural Language Processing Models Get Literate: Why 2019 was a breakthrough year for NLP

Earlier language models powered by Word2Vec and GloVe embeddings yielded confused chatbots, grammar tools with middle-school reading comprehension, and not-half-bad translations. The latest generation is so good, some people consider it dangerous.

December 24, 20192 min read
Inside AI’s Muppet Empire: Why Are So Many NLP Models Named After Muppets?
BERT

Inside AI’s Muppet Empire: Why Are So Many NLP Models Named After Muppets?

As language models show increasing power, a parallel trend has received less notice: The vogue for naming models after characters in the children’s TV show Sesame Street.

December 18, 20191 min read
Inside AI’s Muppet Empire: Why Are So Many NLP Models Named After Muppets?
BERT

Inside AI’s Muppet Empire: Why Are So Many NLP Models Named After Muppets?

As language models show increasing power, a parallel trend has received less notice: The vogue for naming models after characters in the children’s TV show Sesame Street.

December 18, 20191 min read
Keeping the Facts Straight: NLP system FactCC fact checks texts.
BERT

Keeping the Facts Straight: NLP system FactCC fact checks texts.

Automatically generated text summaries are becoming common in search engines and news websites. But existing summarizers often mix up facts. For instance, a victim’s name might get switched for the perpetrator’s.

December 11, 20192 min read
Keeping the Facts Straight: NLP system FactCC fact checks texts.
BERT

Keeping the Facts Straight: NLP system FactCC fact checks texts.

Automatically generated text summaries are becoming common in search engines and news websites. But existing summarizers often mix up facts. For instance, a victim’s name might get switched for the perpetrator’s.

December 11, 20192 min read
Hidden Findings Revealed
BERT

Hidden Findings Revealed

Drugs undergo rigorous experimentation and clinical trials to gain regulatory approval, while dietary supplements get less scrutiny. Even when a drug study reveals an interaction with supplements, the discovery tends to receive little attention.

October 9, 20192 min read
Hidden Findings Revealed
BERT

Hidden Findings Revealed

Drugs undergo rigorous experimentation and clinical trials to gain regulatory approval, while dietary supplements get less scrutiny. Even when a drug study reveals an interaction with supplements, the discovery tends to receive little attention.

October 9, 20192 min read

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox