It is Google’s neural network-based technique for natural language processing (NLP) pre-training. BERT stands for Bidirectional Encoder Representations from Transformers.

It was opened-sourced last year and written about in more detail on the Google AI blog. In short, BERT can help computers understand language a bit more like humans do.

https://ai.googleblog.com/2018/11/open-sourcing-bert-state-of-art-pre.html

BERT has the highest score on superglue language tests and is being deployed to google search english first. The superglue test version had 11 billion features.

A super powerful improving neural network is powering google's search now and interfacing with human requests.