NLP

RoBERTa: A Robustly Optimized BERT Pretraining Approach Soon after BERT got released in late 2018, A floodgate of transformer-based networks got opened. Full capabilities of BERT was going unnoticed until RoBERTa. In this paper, the authors question and improve the hyperparameters and training paradigm of BERT with carefully crafted experiments and come up with a robust and better performing network without changing the core architecture of BERT. Please feel free to read along with the paper with my notes and highlights. ...

Few-Shot Named Entity Recognition: A Comprehensive Study A lesser-known albeit important paper in my opinion. This paper highlights a key problem in the industry that does not always appear in research making it all the more impressive. In this paper, the authors talk about the problem of less data for NER in industry and experimentally try the effects of three key approaches on few-shot NER: Meta-Learning: Construct prototypes for different entities Supervised pre-training on huge noisy data Self Training Please feel free to read along with the paper with my notes and highlights. ...

Fine-Tune BERT for Text Classification with TensorFlow Figure 1: BERT Classification Model We will be using GPU accelerated Kernel for this tutorial as we would require a GPU to fine-tune BERT. Prerequisites: Willingness to learn: Growth Mindset is all you need Some basic idea about Tensorflow/Keras Some Python to follow along with the code Initial Set Up Install TensorFlow and TensorFlow Model Garden import tensorflow as tf print(tf.version.VERSION) Cloning the Github Repo for tensorflow models ...

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding The revolutionary paper by Google that increased the State-of-the-art performance for various NLP tasks and set the stepping stone for many other revolutionary architectures. This paper leads the way and sets a direction for the entire domain. It shows clear benefits of using pre-trained models(trained on huge datasets) and transfer learning independent of the downstream tasks. Please feel free to read along with the paper with my notes and highlights. ...

PICK: Processing Key Information Extraction from Documents using Improved Graph Learning-Convolutional Networks This paper talks KIE(Key Information Extraction), which is to extract texts of a number of key fields from given documents, and save the texts to structured documents. It proposes a solution which overcomes the existing problems fully and efficiently exploiting both textual and visual features of documents to get a richer semantic representation that is crucial for extraction. ...