La Matemática: su contenido, métodos y significado Vol 3

Gain both a firm practical understanding and sufficient theoretical insight into the inner workings of Linux kernel internals, learn to write high-quality kernel module code, understand the complexities of kernel synchronization Purchase of the print or Kindle book includes a free eBook in PDF format. Key Features Discover how to write Linux kernel and module code for real-world products Implement industry-grade techniques in real-world scenarios for fast, efficient memory allocation and data synchronization Understand and exploit kernel architecture, CPU scheduling, and kernel synchronization techniques Book Description The 2nd Edition of Linux Kernel Programming is an updated, comprehensive guide for new programmers to the Linux kernel. This book uses the recent 6.1 Long-Term Support (LTS) Linux kernel series, which will be maintained until Dec 2026, and also delves into its many new features. Further, the Civil Infrastructure Project has pledged to maintain and support this 6.1 Super LTS (SLTS) kernel right until August 2033, keeping this book valid for years to come! You’ll begin this exciting journey by learning how to build the kernel from source. In a step by step manner, you will then learn how to write your first kernel module by leveraging the kernel’s powerful Loadable Kernel Module (LKM) framework. With this foundation, you will delve into key kernel internals topics including Linux kernel architecture, memory management, and CPU (task) scheduling. You’ll finish with understanding the deep issues of concurrency, and gain insight into how they can be addressed with various synchronization/locking technologies (e.g., mutexes, spinlocks, atomic/refcount operators, rw-spinlocks and even lock-free technologies such as per-CPU and RCU). By the end of this book, you’ll have a much better understanding of the fundamentals of writing the Linux kernel and kernel module code that can straight away be used in real-world projects and products. What you will learn Configure and build the 6.1 LTS kernel from source Write high-quality modular kernel code (LKM framework) for 6.x kernels Explore modern Linux kernel architecture Get to grips with key internals details regarding memory management within the kernel Understand and work with various dynamic kernel memory alloc/dealloc APIs Discover key internals aspects regarding CPU scheduling within the kernel, including cgroups v2 Gain a deeper understanding of kernel concurrency issues Learn how to work with key kernel synchronization primitives Who this book is for This book is for beginner Linux programmers and developers looking to get started with the Linux kernel, providing a knowledge base to understand required kernel internal topics and overcome frequent and common development issues. A basic understanding of Linux CLI and C programming is assumed. Table of Contents Linux Kernel Programming – A Quick Introduction Building the 6.x Linux Kernel from Source - Part 1 Building the 6.x Linux Kernel from Source - Part 2 Writing Your First Kernel Module - Part 1 Writing Your First Kernel Module - Part 2 Kernel Internals Essentials - Processes and Threads Memory Management Internals - Essentials Kernel Memory Allocation for Module Authors - Part 1 Kernel Memory Allocation for Module Authors - Part 2 The CPU Scheduler - Part 1 The CPU Scheduler - Part 2 Kernel Synchronization - Part 1 Kernel Synchronization - Part 2 Cover Copyright Contributors Table of Contents Preface Chapter 1: Linux Kernel Programming – A Quick Introduction Kernel workspace setup Technical requirements Cloning this book’s code repository Chapter 2: Building the 6.x Linux Kernel from Source – Part 1 Technical requirements Preliminaries for the kernel build Understanding the Linux kernel release nomenclature Fingers-and-toes releases Kernel development workflow – understanding the basics Viewing the kernel’s Git log via the command line Viewing the kernel’s Git log via its GitHub page The kernel dev workflow in a nutshell Exercise Exploring the types of kernel source trees LTS kernels – the new mandate Which kernel should I run? Steps to build the kernel from source Step 1 – Obtaining a Linux kernel source tree Downloading a specific kernel tree Cloning a Git tree Step 2 – Extracting the kernel source tree A brief tour of the kernel source tree Step 3 – Configuring the Linux kernel Minimally understanding the Kconfig/Kbuild build system How the Kconfig+Kbuild system works – a minimal take Arriving at a default configuration Obtaining a good starting point for kernel configuration Kernel config using distribution config as a starting point Tuned kernel config via the localmodconfig approach Kernel config for typical embedded Linux systems Seeing all available config options Getting going with the localmodconfig approach Tuning our kernel configuration via the make menuconfig UI Sample usage of the make menuconfig UI Kernel config – exploring a bit more Searching within the menuconfig UI Looking up the differences in configuration Using the kernel’s config script to view/edit the kernel config Configuring the kernel for security Miscellaneous tips – kernel config Customizing the kernel menu, Kconfig, and adding our own menu item Understanding the Kconfig* files Creating a new menu item within the General Setup menu A few details on the Kconfig language Summary Exercise Questions Further reading Chapter 3: Building the 6.x Linux Kernel from Source – Part 2 Technical requirements Step 4 – building the kernel image and modules Getting over a cert config issue on Ubuntu Step 5 – installing the kernel modules Locating the kernel modules within the kernel source Getting the kernel modules installed Overriding the default module installation location Step 6 – generating the initramfs image and bootloader setup Generating the initramfs image – under the hood Understanding the initramfs framework Why the initramfs framework? Understanding the basics of the boot process on the x86 More on the initramfs framework Peeking into the initramfs image Step 7 – customizing the GRUB bootloader Customizing GRUB – the basics Selecting the default kernel to boot into Booting our VM via the GNU GRUB bootloader Experimenting with the GRUB prompt Verifying our new kernel’s configuration Kernel build for the Raspberry Pi Step 1 – cloning the Raspberry Pi kernel source tree Step 2 – installing an x86_64-to-AArch64 cross-toolchain Step 3 – configuring and building the Raspberry Pi AArch64 kernel Miscellaneous tips on the kernel build Minimum version requirements Building a kernel for another site Watching the kernel build run A shortcut shell syntax to the build procedure Dealing with missing OpenSSL development headers How can I check which distro kernels are installed? Summary Questions Further reading Chapter 4: Writing Your First Kernel Module – Part 1 Technical requirements Understanding the kernel architecture – part 1 User space and kernel space Library and system call APIs Kernel space components Exploring LKMs The LKM framework Kernel modules within the kernel source tree Writing our very first kernel module Introducing our Hello, world LKM C code Breaking it down Kernel headers Module macros Entry and exit points Return values Common operations on kernel modules Building the kernel module Running the kernel module A quick first look at the kernel printk() Listing the live kernel modules Unloading the module from kernel memory Our lkm convenience script Understanding kernel logging and printk Using the kernel memory ring buffer Kernel logging and systemd’s journalctl Using printk log levels The pr_ convenience macros Writing to the console Writing output to the Raspberry Pi console Turning on debug-level kernel messages Rate limiting the printk instances Rate-limiting macros to use Generating kernel messages from user space Standardizing printk output via the pr_fmt macro Portability and the printk format specifiers Understanding the new printk indexing feature Understanding the basics of a kernel module Makefile Summary Questions Further reading Chapter 5: Writing Your First Kernel Module – Part 2 Technical requirements A “better” Makefile template for your kernel modules Configuring a “debug” kernel Cross-compiling a kernel module Setting up the system for cross-compilation Attempt 1 – setting the ARCH and CROSS_COMPILE environment variables Attempt 2 – pointing the Makefile to the correct kernel source tree for the target Attempt 3 – cross-compiling our kernel module Examining Linux kernel ABI compatibility issues Attempt 4 – cross-compiling our kernel module Summarizing what went wrong with the module cross-buildd/load and how it was fixed Gathering minimal system information Being a bit more security-aware Licensing kernel modules Licensing of inline kernel code Licensing of out-of-tree kernel modules Emulating “library-like” features for kernel modules Performing library emulation via linking multiple source files Understanding function and variable scope in a kernel module Understanding module stacking Trying out module stacking Emulating ‘library-like’ features – summary and conclusions Passing parameters to a kernel module Declaring and using module parameters Getting/setting module parameters after insertion Learning module parameter data types and validation Validating kernel module parameters Overriding the module parameter’s name Hardware-related kernel parameters Floating point not allowed in the kernel Forcing FP in the kernel Auto-loading modules on system boot Module auto-loading – additional details Kernel modules and security – an overview Proc filesystem tunables affecting the system log A quick word on the dmesg_restrict sysctl A quick word on the kptr_restrict sysctl Understanding the cryptographic signing of kernel modules The two module-signing modes Disabling kernel modules altogether The kernel lockdown LSM – an introduction Coding style guidelines for kernel developers Contributing to the mainline kernel Getting started with contributing to the kernel Summary Questions Further reading Chapter 6: Kernel Internals Essentials – Processes and Threads Technical requirements Understanding process and interrupt contexts Understanding the basics of the process Virtual Address Space (VAS) Organizing processes, threads, and their stacks – user and kernel space Running a small script to see the number of processes and threads alive User space organization Kernel space organization Summarizing the kernel with respect to threads, task structures, and stacks Viewing the user and kernel stacks Traditional approach to viewing the stacks eBPF – the modern approach to viewing both stacks The 10,000-foot view of the process VAS Understanding and accessing the kernel task structure Looking into the task structure Accessing the task structure with current Determining the context Working with the task structure via ‘current’ Built-in kernel helper methods and optimizations Trying out the kernel module to print process context info Seeing that the Linux OS is monolithic Coding for security with printk Iterating over the kernel’s task lists Iterating over the task list I – displaying all processes Iterating over the task list II – displaying all threads Differentiating between the process and thread – the TGID and the PID Iterating over the task list III – the code Summary Questions Further reading Chapter 7: Memory Management Internals – Essentials Technical requirements Understanding the VM split Looking under the hood – the Hello, world C program Going beyond the printf() API Virtual addressing and address translation VM split on 64-bit Linux systems Common VM splits Understanding the process VAS – the full view Examining the process VAS Examining the user VAS in detail Directly viewing the process memory map using procfs Frontends to view the process memory map Understanding VMA basics Examining the kernel VAS High memory on 32-bit systems Writing a kernel module to show information about the kernel VAS Macros and variables describing the kernel VAS layout Trying it out – viewing kernel VAS details The kernel VAS via procmap Trying it out – the user segment Viewing kernel documentation on the memory layout Randomizing the memory layout – KASLR User memory randomization with ASLR Kernel memory layout randomization with KASLR Querying/setting KASLR status with a script Understanding physical memory organization Physical RAM organization Nodes and NUMA Zones within a node Direct-mapped RAM and address translation An introduction to physical memory models Understanding the sparsemem[-vmemmap] memory model in brief Summary Questions Further reading Chapter 8: Kernel Memory Allocation for Module Authors – Part 1 Technical requirements Introducing kernel memory allocators Understanding and using the kernel page allocator (or BSA) The fundamental workings of the page allocator Understanding the organization of the page allocator freelist The workings of the page allocator Working through a few scenarios Page allocator internals – a few more details Learning how to use the page allocator APIs Dealing with the GFP flags Freeing pages with the page allocator A few guidelines to observe when (de)allocating kernel memory Writing a kernel module to demo using the page allocator APIs Deploying our lowlevel_mem_lkm kernel module The page allocator and internal fragmentation (wastage) The GFP flags – digging deeper Never sleep in interrupt or atomic contexts Page allocator – pros and cons Understanding and using the kernel slab allocator The object caching idea A few FAQs regarding (slab) memory usage and their answers Learning how to use the slab allocator APIs Allocating slab memory Freeing slab memory Data structures – a few design tips The actual slab caches in use for kmalloc Writing a kernel module to use the basic slab APIs Size limitations of the kmalloc API Testing the limits – memory allocation with a single call Checking via the /proc/buddyinfo pseudofile Slab allocator – a few additional details Using the kernel’s resource-managed memory allocation APIs Additional slab helper APIs Control groups and memory Caveats when using the slab allocator Background details and conclusions Testing slab allocation with ksize() – case 1 Testing slab allocation with ksize() – case 2 Interpreting the output from case 2 Graphing it Finding internal fragmentation (wastage) within the kernel The easy way with slabinfo More details with alloc_traces and a custom script Slab layer - pros and cons Slab layer – a word on its implementations within the kernel Summary Questions Further reading Chapter 9: Kernel Memory Allocation for Module Authors – Part 2 Technical requirements Creating a custom slab cache Creating and using a custom slab cache within a kernel module Step 1 – creating a custom slab cache Step 2 – using our custom slab cache’s memory Step 3 – destroying our custom cache Custom slab – a demo kernel module Extracting useful information regarding slab caches Understanding slab shrinkers Summarizing the slab allocator pros and cons Debugging kernel memory issues – a quick mention Understanding and using the kernel vmalloc() API Learning to use the vmalloc family of APIs Trying out vmalloc() A brief note on user-mode memory allocations and demand paging Friends of vmalloc() Is this vmalloc-ed (or module region) memory? Unsure which API to use? Try kvmalloc() Miscellaneous helpers – vmalloc_exec() and vmalloc_user() Specifying memory protections The kmalloc() and vmalloc() APIs – a quick comparison Memory allocation in the kernel – which APIs to use when Visualizing the kernel memory allocation API set Selecting an appropriate API for kernel memory allocation A word on DMA and CMA Memory reclaim – a key kernel housekeeping task Zone watermarks and kswapd The new multi-generational LRU (MGLRU) lists feature Trying it out – seeing histogram data from MGLRU A quick introduction to DAMON – the Data Access Monitoring feature Running a memory workload and visualizing it with DAMON’s damo front-end Stayin’ alive – the OOM killer Deliberately invoking the OOM killer Invoking the OOM killer via Magic SysRq Invoking the OOM killer with a crazy allocator program Understanding the three VM overcommit_memory policies VM overcommit from the viewpoint of the __vm_enough_memory() code Case 1: vm.overcommit_memory == 0 (the default, OVERCOMMIT_GUESS) Case 2: vm.overcommit_memory == 2 (VM overcommit turned off, OVERCOMMIT_NEVER) and vm.overcommit_ratio == 50 Demand paging and OOM The optimized (unmapped) read Understanding the OOM score Closing thoughts on the OOM killer and cgroups Cgroups and memory bandwidth – a note Summary Questions Further reading Chapter 10: The CPU Scheduler – Part 1 Technical requirements Learning about the CPU scheduling internals – part 1 – essential background What is the KSE on Linux? The Linux process state machine The POSIX scheduling policies Thread priorities Visualizing the flow Using the gnome-system-monitor GUI to visualize the flow Using perf to visualize the flow Trying it out – the command-line approach Trying it out – the graphical approach Visualizing the flow via alternate approaches Learning about the CPU scheduling internals – part 2 Understanding modular scheduling classes A conceptual example to help understand scheduling classes Asking the scheduling class The workings of the Completely Fair Scheduling (CFS) class in brief Scheduling statistics Querying a given thread’s scheduling policy and priority Learning about the CPU scheduling internals – part 3 Preemptible kernel The dynamic preemptible kernel feature Who runs the scheduler code? When does schedule() run? Minimally understanding the thread_info structure The timer interrupt housekeeping – setting TIF_NEED_RESCHED The process context part – checking TIF_NEED_RESCHED CPU scheduler entry points – a summary The core scheduler code in brief Summary Questions Further reading Chapter 11: The CPU Scheduler – Part 2 Technical requirements Understanding, querying, and setting the CPU affinity mask Querying and setting a thread’s CPU affinity mask Using taskset to perform CPU affinity Setting the CPU affinity mask on a kernel thread Querying and setting a thread’s scheduling policy and priority Setting the policy and priority within the kernel – on a kernel thread A real-world example – threaded interrupt handlers An introduction to cgroups Cgroup controllers Exploring the cgroups v2 hierarchy Enabling or disabling controllers The cgroups within the hierarchy Systemd and cgroups Our cgroups v2 explorer script Trying it out – constraining the CPU resource via cgroups v2 Leveraging systemd to set up CPU resource constraints on a service The manual way – a cgroups v2 CPU controller Running Linux as an RTOS – an introduction Pointers to building RTL for the mainline 6.x kernel (on x86_64) Miscellaneous scheduling related topics A few small (kernel space) routines to check out The ghOSt OS Summary Questions Further reading Chapter 12: Kernel Synchronization – Part 1 Technical requirements Critical sections, exclusive execution, and atomicity What is a critical section? A classic case – the global i ++ Concepts – the lock Critical sections – a summary of key points Data races – a more formal definition Concurrency concerns within the Linux kernel Multicore SMP systems and data races Preemptible kernels, blocking I/O, and data races Hardware interrupts and data races Locking guidelines and deadlock Mutex or spinlock? Which to use when Determining which lock to use – in theory Determining which lock to use – in practice Using the mutex lock Initializing the mutex lock Correctly using the mutex lock Mutex lock and unlock APIs and their usage Mutex lock – via [un]interruptible sleep? Mutex locking – an example driver The mutex lock – a few remaining points Mutex lock API variants The semaphore and the mutex Priority inversion and the RT-mutex Internal design Using the spinlock Spinlock – simple usage Spinlock – an example driver Test – sleep in an atomic context Testing the buggy module on a 6.1 debug kernel Locking and interrupts Scenario 1 – driver method and hardware interrupt handler run serialized, sequentially Scenario 2 – driver method and hardware interrupt handler run interleaved Scenario 2 on a single-core (UP) system Scenario 2 on a multicore (SMP) system Solving the issue on UP and SMP with the spin_[un]lock_irq() API variant Scenario 3 – some interrupts masked, driver method and hardware interrupt handler run interleaved Interrupt handling, bottom halves, and locking Interrupt handling on Linux – a summary of key points Bottom halves and locking Using spinlocks – a quick summary Locking – common mistakes and guidelines Common mistakes Locking guidelines Solutions Summary Questions Further reading Chapter 13: Kernel Synchronization – Part 2 Technical requirements Using the atomic_t and refcount_t interfaces The newer refcount_t versus older atomic_t interfaces The simpler atomic_t and refcount_t interfaces Examples of using refcount_t within the kernel codebase 64-bit atomic integer operators A note on internal implementation Using the RMW atomic operators RMW atomic operations – operating on device registers Using the RMW bitwise operators Using bitwise atomic operators – an example Efficiently searching a bitmask Using the reader-writer spinlock Reader-writer spinlock interfaces Trying out the reader-writer spinlock Performance issues with reader-writer spinlocks The reader-writer semaphore Understanding CPU caching basics, cache effects, and false sharing An introduction to CPU caches The risks – cache coherency, performance issues, and false sharing What is the cache coherency problem? The false sharing issue Lock-free programming with per-CPU and RCU Per-CPU variables Working with per-CPU variables Per-CPU – an example kernel module Per-CPU usage within the kernel Understanding and using the RCU (Read-Copy-Update) lock-free technology – a primer How does RCU work? Trying out RCU RCU: detailed documentation RCU usage within the kernel Lock debugging within the kernel Configuring a debug kernel for lock debugging The lock validator lockdep – catching locking issues early Catching deadlock bugs with lockdep – a few examples Example 1 – catching a self deadlock bug with lockdep Example 2 – catching an AB-BA deadlock with lockdep Brief notes on lockdep – annotations and issues A note on lockdep annotations A note on lockdep – known issues Kernel lock statistics Viewing and interpreting the kernel lock statistics Introducing memory barriers An example of using memory barriers in a device driver A note on marked accesses Summary Questions Further reading Packt Page Other Books You May Enjoy Index Gain a firm practical understanding of the fundamentals of Linux kernel internals, learn to write high-quality kernel module code, and kernel synchronization The 2nd Edition of Linux Kernel Programming is an updated, comprehensive introduction for new programmers to the Linux kernel. This book uses the recent 5.10 Long-Term Support (LTS) Linux kernel series, which will be maintained until 2026, showing 5.X/6.X new features. In this new edition, youll start the journey by learning how to build the kernel from source, including doing a kernel build for the popular Raspberry Pi. Youll then learn, step by step, how to write your first kernel module by leveraging the kernels powerful Loadable Kernel Module (LKM) framework. Youll then get to delve into key kernel internals topics including Linux kernel architecture, memory management, and CPU scheduling. Youll understand the deep issues of concurrency, and gain insight into how they can be addressed with various locking technologies (mutexes, spinlocks, atomic / refcount operators and even lock-free technologies). By the end of this 2nd Edition book, youll have a better understanding of the fundamentals of writing Linux kernel and kernel module code that can be used in real-world projects and products. This book is for beginner Linux programmers and developers looking to get started with Linux kernel, providing a knowledge base to understand required kernel internal topics and overcome frequent and common development issues. A basic understanding of Linux CLI and C programming is assumed