The Transformation Continues: Technique boosts transformer performance on long sequences.
Machine Learning Research

The Transformation Continues: Technique boosts transformer performance on long sequences.

Transformer networks are gaining popularity as a high-accuracy alternative to recurrent neural networks. But they can run slowly when they’re applied to long sequences.

September 2, 20202 min read
The Transformation Continues: Technique boosts transformer performance on long sequences.
Machine Learning Research

The Transformation Continues: Technique boosts transformer performance on long sequences.

Transformer networks are gaining popularity as a high-accuracy alternative to recurrent neural networks. But they can run slowly when they’re applied to long sequences.

September 2, 20202 min read
Dropout With a Difference: Reduce neural net overfitting without impacting accuracy
Machine Learning Research

Dropout With a Difference: Reduce neural net overfitting without impacting accuracy

The technique known as dropout discourages neural networks from overfitting by deterring them from reliance on particular features. A new approach reorganizes the process to run efficiently on the chips that typically run neural network calculations.

September 2, 20202 min read
Experience Counts: Research proposes an upgrade to experience replay.
Machine Learning Research

Experience Counts: Research proposes an upgrade to experience replay.

If the world changes every second and you take a picture every 10 seconds, you won’t have enough pictures to observe the changes clearly, and storing a series of pictures won’t help. On the other hand, if you take a picture every tenth of a second, then storing a history will help model the world.

August 26, 20202 min read
Experience Counts: Research proposes an upgrade to experience replay.
Machine Learning Research

Experience Counts: Research proposes an upgrade to experience replay.

If the world changes every second and you take a picture every 10 seconds, you won’t have enough pictures to observe the changes clearly, and storing a series of pictures won’t help. On the other hand, if you take a picture every tenth of a second, then storing a history will help model the world.

August 26, 20202 min read
All Examples Are Not Equal: An algorithm for improved semi-supervised learning
Machine Learning Research

All Examples Are Not Equal: An algorithm for improved semi-supervised learning

Semi-supervised learning — a set of training techniques that use a small number of labeled examples and a large number of unlabeled examples — typically treats all unlabeled examples the same way. But some examples are more useful for learning than others.

August 19, 20202 min read
Same Job, Different Scenery: A reinforcement learning technique for visual changes
Machine Learning Research

Same Job, Different Scenery: A reinforcement learning technique for visual changes

People who take driving lessons during daytime don’t need instruction in driving at night. They recognize that the difference doesn’t disturb their knowledge of how to drive. Similarly, a new reinforcement learning method manages superficial variations in the environment without re-training.

August 19, 20202 min read
Same Job, Different Scenery: A reinforcement learning technique for visual changes
Machine Learning Research

Same Job, Different Scenery: A reinforcement learning technique for visual changes

People who take driving lessons during daytime don’t need instruction in driving at night. They recognize that the difference doesn’t disturb their knowledge of how to drive. Similarly, a new reinforcement learning method manages superficial variations in the environment without re-training.

August 19, 20202 min read
All Examples Are Not Equal: An algorithm for improved semi-supervised learning
Machine Learning Research

All Examples Are Not Equal: An algorithm for improved semi-supervised learning

Semi-supervised learning — a set of training techniques that use a small number of labeled examples and a large number of unlabeled examples — typically treats all unlabeled examples the same way. But some examples are more useful for learning than others.

August 19, 20202 min read
When Optimization is Suboptimal: How gradient descent can sometimes lead to model bias
Machine Learning Research

When Optimization is Suboptimal: How gradient descent can sometimes lead to model bias

Bias arises in machine learning when we fit an overly simple function to a more complex problem. A theoretical study shows that gradient descent itself may introduce such bias and render algorithms unable to fit data properly.

August 12, 20202 min read
Hidden in Plain Sight: Researchers make clothes that fool face recognition.
Machine Learning Research

Hidden in Plain Sight: Researchers make clothes that fool face recognition.

With the rise of AI-driven surveillance, anonymity is in fashion. Researchers are working on clothing that evades face recognition systems and designed a t-shirt that tricks a variety of object detection models into failing to spot people.

August 12, 20202 min read
Hidden in Plain Sight: Researchers make clothes that fool face recognition.
Machine Learning Research

Hidden in Plain Sight: Researchers make clothes that fool face recognition.

With the rise of AI-driven surveillance, anonymity is in fashion. Researchers are working on clothing that evades face recognition systems and designed a t-shirt that tricks a variety of object detection models into failing to spot people.

August 12, 20202 min read
When Optimization is Suboptimal: How gradient descent can sometimes lead to model bias
Machine Learning Research

When Optimization is Suboptimal: How gradient descent can sometimes lead to model bias

Bias arises in machine learning when we fit an overly simple function to a more complex problem. A theoretical study shows that gradient descent itself may introduce such bias and render algorithms unable to fit data properly.

August 12, 20202 min read
Transforming Pixels: An image generation model using the GPT architecture
Machine Learning Research

Transforming Pixels: An image generation model using the GPT architecture

Language models like Bert, Ernie, and Elmo have achieved spectacular results based on clever pre-training approaches. New research applies some of those Sesame Street lessons into image processing.

August 5, 20202 min read
Transforming Pixels: An image generation model using the GPT architecture
Machine Learning Research

Transforming Pixels: An image generation model using the GPT architecture

Language models like Bert, Ernie, and Elmo have achieved spectacular results based on clever pre-training approaches. New research applies some of those Sesame Street lessons into image processing.

August 5, 20202 min read

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox