Natural Language Processing on Oracle Cloud Infrastructure: Building Transformers-Based NLP Solutions Using Oracle AI and Hugging Face

This book demonstrates how to use Oracle Cloud Infrastructure (OCI) and Hugging Face technologies to develop advanced NLP solutions. Through a practical case study, it addresses common NLP challenges and offers strategies for creating efficient, cost-effective transformer-based models. By the end of this book, you will have the skills and knowledge to create cutting-edge NLP solutions on OCI, customized to meet the needs of various industries and projects. The book takes you through the complete NLP solution life cycle--covering data preparation, model fine-tuning, deployment, and monitoring--while highlighting key topics such as cost-effectiveness and responsible AI for NLP implementations. Drawing from real-world experience and offering practical insights, it bridges the gap between theory and practice, equipping you to design and deploy scalable, cost-efficient NLP solutions. What You Will Learn Master key NLP concepts and the OCI ecosystem Create high-quality datasets using Hugging Face and OCI Data Labeling Service Fine-tune domain-specific pre-trained models from Hugging Face using OCI Data Science Notebook Sessions Deploy and operationalize your models with OCI Data Science Model Deployments Automate the NLP life cycle with OCI Data Science Pipelines Implement cost-effective strategies throughout the entire NLP life cycle, from dataset preparation to model training and deployment Who This Book Is For A diverse audience interested in implementing NLP solutions on Oracle Cloud Infrastructure: NLP practitioners, data scientists, and machine learning engineers who want to learn how to leverage Oracle AI and Hugging Face to implement an end-to-end NLP solution life cycle, from data preparation to model deployment; Oracle practitioners who want to expand their Oracle expertise by exploring OCI's advanced capabilities for building and scaling cutting-edge NLP solutions in enterprise environments; business decision makers who want to discover the strategic benefits of NLP solutions on OCI, including cost-effectiveness and responsible AI, while driving business value Table of Contents About the Author About the Technical Reviewers Acknowledgments Introduction Chapter 1: NLP Essentials Introduction to Natural Language Processing NLP Tasks NLP Key Concepts Common Challenges Transformers for NLP Transformer Architecture Transformer Taxonomy Transfer Learning Hugging Face Ecosystem Strategic Considerations for NLP Adoption Models Data Team Summary References Chapter 2: Oracle Cloud for NLP Introduction to Oracle Cloud Infrastructure (OCI) History Core Concepts and Terminology Regions and Realms Tenancy/Compartment Core OCI Resources OCI Networking OCI Compute OCI Storage Identity and Access Management (IAM) Oracle’s AI Overview AI Strategy AI Stack OCI AI Services OCI ML Services AI Infrastructure OCI for NLP OCI Language Use Cases OCI Data Science AI Quick Actions OCI Data Labeling AI Samples High-Level Flow for Building NLP Models Using OCI Summary References Chapter 3: Healthcare NLP Case Study MedTALN Inc. Case Study Company Background Healthcare NLP Business Drivers Healthcare NER Initiative What Is Named Entity Recognition (NER) Healthcare NER Benefits Use Cases Healthcare NER Inception Scope and Requirements Requirements Assembling the Team Engaging the NLP Consultant Healthcare NER Elaboration Architectural Design Methodology Preselection of Candidate Solution Options OCI Language-Based Models Option LLMs and OCI Data Science AI Quick Actions Fully Custom Healthcare NER Model Selection of the Optimal Approach Solution Blueprint High-Level Architecture High-Level Approach Project Preparation OCI Account Defining Roles and Responsibilities Summary Reference Chapter 4: Tenancy Preparation Getting Started Cost-Saving Strategies OCI Tenancy Preparation Compartment Creation Network Configuration Storage Identity and Security IAM Setup for Data Scientists Users and Groups Dynamic Groups Policies IAM Setup for Data Labelers Data Science Environment Setup Project Notebook Sessions CPU-Based Notebook Session Setup Conda Installation Setup Check GPU-Based Notebook Session Summary Chapter 5: Dataset Preparation Preliminaries Labeled Datasets Cost Saving Off-the-Shelf Datasets Cost Comparative Analysis Dataset Life Cycle Framing the Problem (Step 1) Dataset Selection (Step 2) Selecting Datasets on Hugging Face Candidate Healthcare NER Dataset Training Dataset Preparation Dataset Collection and Wrangling (Steps 3 and 4) Dataset Preparation Notebook Loading Wrangling Steps Dataset Labeling (Step 5) OCI Data Labeling Service (DLS) Dataset Import Dataset Import Notebook Initialization Dataset Import Dataset Labeling Quality Assurance (QA) Dataset Creation (Step 6) Additional Notes Dataset Import Using DLS UI Record Count Limit Summary References Chapter 6: Model Fine-Tuning Preliminaries Language Models (LMs) Evolution of LMs Neural Language Models (2003) Word Embeddings: Word2Vec and GloVe (2013–2014) Transformers (2017) Pretrained Language Models (2018–2019) Large Language Models (LLMs) (2020s) Acronyms Taxonomy of Pretrained Language Models Healthcare-Specific Pretrained Language Models Why Domain-Specific Models for Healthcare Why Open Pretrained Models Cost-Saving Strategies for the Training Phase Transfer Learning–Based Fine-Tuning Workflow Pretrained Model Selection Framing the Problem (Step 1) MLM Model Selection from Hugging Face (Step 2) Pretrained Model Selection Notebook Identify a List of Candidate MLM Models from Hugging Face Hub Search MLM Models Check the Model Configuration Retrieve Mask Tokens Evaluate and Rank Models Based on Entity Prediction Healthcare NER Model Fine-Tuning Training Dataset Creation Notebook Declare Helper Functions Create HF Dataset from CoNLL File Create Splits for the HF Dataset Save Dataset Training Notebook Loading Training Dataset Training Initialization Set Pretrained Models for Fine-Tuning Declare Helper Functions Initialize the Training Objects Starting the Training Analyzing Training and Evaluation Losses Visual Analysis Automated Checkpoint Selection Healthcare NER Model Evaluation Evaluation Notebook Initialization Load the Training Dataset Define Helper Functions Evaluate Load Best Checkpoints Evaluate All the Models’ Best Checkpoints Select the Best Model Save the Best Model Test the Best Model Prepare the Test Examples Load and Use the Best Model Generate Predictions Summary References Chapter 7: Model Deployment and Monitoring Model Inference Preliminaries Understanding Inference vs. Training Cost-Saving Strategies for the Inference Phase Preparing the Environment Setting Up Policies Setting Up Logging Publish Custom Conda Env. Deployment Process Oracle Data Science Model Catalog Oracle Data Science Model Deployment Oracle ADS HuggingFacePipelineModel Deployment Process Notebook Initializing the ADS Class “HuggingFacePipelineModel” Authenticate Initialize Hugging Face Pipeline Prepare Model Artifact Manually Correct score.py Run Introspection Call Model Summary Verify the Generated Model Artifacts Save the Model to the Model Catalog Create a Model Version Set Save the Model Deploy and Invoke Deploy and Generate Endpoint Run Prediction Against Endpoint Monitoring and Maintenance Logs Metrics Summary References Chapter 8: MLOps and Conclusion MLOps with OCI Data Science OCI Data Science Pipelines Pipeline Example Pipeline Creation Step-by-Step Pipeline Creation Prerequisites Create Pipeline Step Artifacts Data Science Dynamic Group Rule Create Pipeline Journey Through NLP: From Theory to Practice Healthcare NER Model Life Cycle Summary Data Preparation Model Training and Evaluation Model Deployment and Monitoring Deploy Monitor Responsible AI Summary Reference rima o poi torno è una raccolta di racconti con protagonisti giovani italiani emigrati a Bruxelles, storie di vita di "cervelli in fuga", che vivono e lavorano nella capitale europea. Il filo conduttore dei racconti è il desiderio, comune a molti di loro, di tornare alle proprie radici, nonostante l'Italia sia ancora incapace di trattenerli come "teste pensanti". L'obiettivo dell'autrice è quello di dare una speranza ai suoi coetanei, mostrando che all'estero, se si ha talento, ce la si può fare. Basta essere determinati e credere in un sogno. Allo stesso tempo, il libro descrive la realtà di Bruxelles, con cui la maggior parte degli italiani ha un rapporto "conflittuale", essendo una città di passaggio, con dinamiche a sé, rispetto a quelle che in genere contraddistinguono le altre capitali d'Europa. Nel libro si riscontra una forte componente autobiografica. La visione dell'autrice di Bruxelles fa da sfondo a tutte le storie, che nascono da lunghe interviste con i personaggi protagonisti. Cronaca, racconto e autobiografia si intrecciano fino a creare una sorta di "reportage narrativo". All'interno del libro si ritrova l'amore dell'autrice per i grandi cantautori italiani, tanto che a ogni personaggio, all'inizio e alla fine di ciascun racconto, sono dedicate due canzoni.