Introduction to Transformers for NLP : With the Hugging Face Library and Models to Solve Problems

Get a hands-on introduction to Transformer architecture using the Hugging Face library. This book explains how Transformers are changing the AI domain, particularly in the area of natural language processing. This book covers Transformer architecture and its relevance in natural language processing (NLP). It starts with an introduction to NLP and a progression of language models from n-grams to a Transformer-based architecture. Next, it offers some basic Transformers examples using the Google colab engine. Then, it introduces the Hugging Face ecosystem and the different libraries and models provided by it. Moving forward, it explains language models such as Google BERT with some examples before providing a deep dive into Hugging Face API using different language models to address tasks such as sentence classification, sentiment analysis, summarization, and text generation. After completing Introduction to Transformers for NLP , you will understand Transformer concepts and be able to solve problems using the Hugging Face library. What You Will Learn Understand language models and their importance in NLP and NLU (Natural Language Understanding) Master Transformer architecture through practical examples Use the Hugging Face library in Transformer-based language models Create a simple code generator in Python based on Transformer architecture Who This Book Is For Data Scientists and software developers interested in developing their skills in NLP and NLU (Natural Language Understanding) Table of Contents About the Author About the Technical Reviewer Introduction Chapter 1: Introduction to Language Models History of NLP Bag of Words n-grams Recurrent Neural Networks What Exactly Is a Recurrent Neural Network (RNN)? How RNNs Work Language Models What Advantages Does Using a Language Model Give Us? Neural Network–Based Language Models Summary Chapter 2: Introduction to Transformers What Is a Seq2Seq Neural Network? The Transformer Transformers Encoder Input Embeddings Multi-headed Attention The Residual Connections, Layer Normalization, and Feed-Forward Network Decoder Summary Chapter 3: BERT Workings of BERT Masked LM (MLM) Next Sentence Prediction Inference in NSP BERT Pretrained Models BERT Input Representations Use Cases for BERT Sentiment Analysis on Tweets Performance of BERT on a Variety of Common Language Tasks Summary Chapter 4: Hugging Face Features of the Hugging Face Platform Components of Hugging Face Pipelines Tokenizer Padding Truncation AutoModel Summary Chapter 5: Tasks Using the Hugging Face Library Gradio: An Introduction Creating a Space on Hugging Face Hugging Face Tasks Question and Answering Translation Summary Zero-Shot Learning Zero-Shot Text Classification Why We Need Zero-Shot Text Generation Task/Models Text-to-Text Generation English-to-German Using T5 Sentiment Analysis Task Sentence Paraphrasing Task Chatbot/Dialog Bot Code and Code Comment Generation Code Comment Generator Summary Untitled Chapter 6: Fine-Tuning Pretrained Models Datasets Fine-Tuning a Pretrained Model Training for Fine-Tuning Inference Summary Appdindix A: Vision Transformers Self-Attention and Vision Transformers Summary Index