It obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE score to 80.5% (7.7% point absolute improvement), MultiNLI accuracy to 86.7% (4.6% absolute improvement), SQuAD v1.1 question a…

BERT is conceptually simple and empirically powerful. Jacob Devlin, et al., "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" https://arxiv.org/abs/1810.04805 自然言語処理のあらゆるタスクに適用できる汎用的…

As a result, the pre-trained BERT model can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specif…

Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. Jacob Devlin, et al., "BERT: Pre-train…

We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Jacob Devlin, et al., "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding…

Models and code are available at https://github.com/hujie-frank/SENet. Jie Hu, Li Shen, et al., "Squeeze-and-Excitation Networks" https://arxiv.org/abs/1709.01507 CNNの汎化性能を改善するために、特徴マップのチャンネルごとに重み付けして次の…

Squeeze-and-Excitation Networks formed the foundation of our ILSVRC 2017 classification submission which won first place and reduced the top-5 error to 2.251%, surpassing the winning entry of 2016 by a relative improvement of ∼25%. Jie Hu,…

We further demonstrate that SE blocks bring significant improvements in performance for existing state-of-the-art CNNs at slight additional computational cost. Jie Hu, Li Shen, et al., "Squeeze-and-Excitation Networks" https://arxiv.org/ab…

We show that these blocks can be stacked together to form SENet architectures that generalise extremely effectively across different datasets. Jie Hu, Li Shen, et al., "Squeeze-and-Excitation Networks" https://arxiv.org/abs/1709.01507 CNN…

In this work, we focus instead on the channel relationship and propose a novel architectural unit, which we term the “Squeeze-and-Excitation” (SE) block, that adaptively recalibrates channel-wise feature responses by explicitly modelling i…

A broad range of prior research has investigated the spatial component of this relationship, seeking to strengthen the representational power of a CNN by enhancing the quality of spatial encodings throughout its feature hierarchy. Jie Hu, …

The central building block of convolutional neural networks (CNNs) is the convolution operator, which enables networks to construct informative features by fusing both spatial and channel-wise information within local receptive fields at e…

We then demonstrate the effectiveness of MobileNets across a wide range of applications and use cases including object detection, finegrain classification, face attributes and large scale geo-localization. Andrew G. Howard, et al., "Mobile…

We present extensive experiments on resource and accuracy tradeoffs and show strong performance compared to other popular models on ImageNet classification. Andrew G. Howard, et al., "MobileNets: Efficient Convolutional Neural Networks for…

These hyper-parameters allow the model builder to choose the right sized model for their application based on the constraints of the problem. Andrew G. Howard, et al., "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision …

We introduce two simple global hyperparameters that efficiently trade off between latency and accuracy. Andrew G. Howard, et al., "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications" https://arxiv.org/abs/17…

MobileNets are based on a streamlined architecture that uses depthwise separable convolutions to build light weight deep neural networks. Andrew G. Howard, et al., "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Appl…

We present a class of efficient models called MobileNets for mobile and embedded vision applications. Andrew G. Howard, et al., "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications" https://arxiv.org/abs/1704…