Mastering Proxmox - Third Edition : Discover Real World Scenarios for Proxmox Troubleshooting and Become an Expert Cloud Builder

Preface What this book covers What you need for this book Who this book is for Conventions Reader feedback Customer support Downloading the color images of this book Errata Piracy Questions Understanding Proxmox VE and Advanced Installation Understanding Proxmox features It is free! Built-in firewall Open vSwitch The graphical user interface KVM virtual machines Linux containers, or LXC Storage plugins Vibrant culture The basic installation of Proxmox The advanced installation option Debugging the Proxmox installation Proxmox subscription and repositories Proxmox VE Enterprise repository Type Subscription key Status Server ID Sockets Last checked Next due date Proxmox VE No-Subscription repository Proxmox VE Test repository Summary Creating a Cluster and Exploring the Proxmox GUI Creating a Proxmox cluster Exploring the Proxmox GUI The GUI menu system Cluster tree view Server View Folder View Storage View Pool View The Datacenter menu Datacenter | Search Datacenter | Summary Datacenter | Options Datacenter | Storage Datacenter | Backup Datacenter | Permissions Datacenter | Permissions | Users Datacenter | Permissions | Groups Datacenter | Permissions | Pools Datacenter | Permissions | Roles Datacenter | Permissions | Authentication Datacenter | HA Datacenter | Firewall Datacenter | Support Node-specific menus Node | Search Node | Summary Node | Shell Node | System Node | Network Node | DNS Node | Time Node | Syslog Node | Updates Node | Firewall Node | Disks Node | Ceph Node | Task History Node | Subscription KVM menu KVM VM | Summary KVM | Console KVM | Hardware KVM | Options KVM VM | Task History KVM | Monitor KVM | Backup KVM VM | Snapshot KVM | Firewall KVM | Permissions LXC container menu LXC container | Summary LXC container | Resources LXC container | Network LXC container | DNS LXC container | Options LXC container | Task History LXC container | Backup LXC container | Snapshots LXC container | Firewall LXC container | Permissions Pool menu Pool | Summary Pool | Members Pool | Permissions Summary Proxmox under the Hood The Proxmox cluster file system Proxmox directory structure Dissecting the configuration files The cluster configuration file logging { } nodelist { } quorum { } totem { } interface { } Storage configuration file User configuration files The password configuration file KVM virtual machine configuration file Arguments in the KVM configuration file LXC container configuration file Version configuration file Member nodes Virtual machine list file The cluster log file Ceph configuration files Firewall configuration file Summary Storage Systems Local storage versus shared storage Live migration of a virtual machine Seamless expansion of multinode storage space Centralized backup Multilevel data tiering Central storage management Local and shared storage comparison A virtual disk image Supported image formats The .qcow2 images The .raw image type The .vmdk image type Virtual device types Managing disk images Resizing a virtual disk image Moving a virtual disk image Throttling a virtual disk image Caching a virtual disk image VirtIO bus type for Windows VMs Installing VirtIO drivers during Windows installation Installing VirtIO drivers after Windows installation Storage types in Proxmox Directory iSCSI Logical Volume Management NFS ZFS Ceph RBD GlusterFS Noncommercial/commercial storage options Summary Installing and Configuring Ceph Ceph components A physical node as cluster member Maps A cluster map A CRUSH map Monitor OSD OSD journal Metadata server PG Pools Ceph components summary Virtual Ceph for training Installing a Ceph cluster Installing Ceph on Proxmox Preparing a Proxmox node for Ceph Installing Ceph Creating mons from the Proxmox GUI Creating OSDs from Proxmox GUI Managing a Ceph pool using Proxmox GUI Creating a Ceph pool using Proxmox GUI Connecting Ceph to Proxmox Ceph command list Summary KVM Virtual Machines Exploring KVM Creating a KVM Creating a KVM using an ISO image General tab Node VM ID Name Resource Pool Help The OS tab The CD/DVD tab The Hard Disk tab Bus/Device Storage Disk size (GB) Format Cache No backup Discard IO thread The CPU tab Sockets Cores Enabling NUMA Type The Memory tab The Network tab Bridged mode Firewall NAT mode No network device Model MAC address Rate limit (MB/s) Multiqueues Disconnect Creating VM by cloning Creating VMs from a template Target node Mode Advanced configuration options for VMs Configuring a sound device Configuring PCI passthrough Configuring GPU passthrough Preparing for hotplug Configuring VMs with hotplug Hotplugging vCPUs Hotplugging memory Hotplugging disks/vNICs Migrating KVM virtual machines Summary LXC Virtual Machines Exploring LXC virtual machines Understanding container templates Creating an LXC container General tab Node CT ID Hostname Unprivileged container Resource Pool The Template tab The Root Disk tab Storage ACLs Enable quota The CPU tab Cores The Memory tab The Network tab Name MAC address Bridge The VLAN Tag Rate limit Firewall IPv4/IPv6 The DNS tab The Confirm tab Managing an LXC container Adjusting resources using the GUI Adjusting resources using the CLI Adjusting resources using direct modification Migrating an LXC container Accessing an LXC container The noVNC console Direct shell through the CLI Converting OpenVZ to LXC Summary Network of Virtual Networks Exploring virtual networks Physical networks versus virtual networks A physical network A virtual network Networking components in Proxmox Virtual Network Interface Cards Adding/removing vNIC A virtual bridge Adding a virtual bridge through the GUI Name IP information Bridge ports VLAN-aware Adding a virtual bridge through CLI Extra bridge options bridge_stp bridge_fd Virtual LAN Adding a VLAN Network Address Translation/Translator Adding NAT/masquerading Network bonding Adding a bonding interface The layer 2 hash policy The layer 2+3 hash policy The layer 3+4 hash policy Multicast Configuring multicast on Netgear Open vSwitch Features of Open vSwitch Adding an Open vSwitch bridge Adding the Open vSwitch bond Adding Open vSwitch IntPort CLI for Open vSwitch Practicing Open vSwitch Configuration requirements Solutions Sample virtual networks Network #1 – Proxmox in its simplest form Network #2 – the multi-tenant environment Network #3 – academic institution A multi-tenant virtual environment A multi-tenant network diagram Summary The Proxmox VE Firewall Exploring the Proxmox VE firewall Components of the Proxmox firewall Zones Security groups IPSet Rules Protocols Macros The pve-firewall and pvefw-logger services Configuration files of a firewall Configuring the data center-specific firewall Configuring the Datacenter firewall through the GUI Creating the Datacenter firewall rules Creating the Datacenter IPSet Creating aliases Configuring the Datacenter firewall through the CLI [OPTIONS] [ALIASES] [IPSET ] [RULES] [group ] Configuring a host-specific firewall Creating host firewall rules Options for the host zone firewall Enable a firewall The SMURFS filter The TCP flags filter NDP nf_conntrack_max nf_conntrack_tcp_timeout_established log_level_in/out tcp_flags_log_level smurf_log_level Configuring the host firewall through the CLI Configuring a VM-specific firewall Creating VM firewall rules Creating aliases Creating IPSets Options for a VM zone firewall Enable DHCP The MAC filter Input/output policy Configuring a VM-specific firewall through the CLI Integrating a Suricata IDS/IPS Installing/configuring Suricata Limitations of Suricata in Proxmox Summary Proxmox High Availability Understanding HA HA in Proxmox How Proxmox HA works Requirements for HA setup At least three nodes Shared storage Fencing BIOS power-on feature Configuring Proxmox HA The HA menu Status The Resources menu The Groups menu ID Node The restricted checkbox The nofailback checkbox The Fencing menu Testing Proxmox HA configuration The Proxmox HA simulator Configuring the Proxmox HA simulator Summary Monitoring the Proxmox Cluster An introduction to monitoring Proxmox built-in monitoring Datacenter Status Node Status Zabbix as a monitoring solution Installing Zabbix Configuring Zabbix Configuring a host to monitor Displaying data using a graph Configuring the disk health notification Installing smart monitor tools Configuring the Zabbix agent Creating a Zabbix item in the GUI Creating a trigger in the GUI Creating graphs in the GUI Configuring SNMP in Proxmox Object Identifiers Management Information Base Adding an SNMP device in Zabbix Monitoring the Ceph cluster with the Proxmox GUI Monitoring a Ceph cluster with third-party options Summary Proxmox Production-Level Setup Defining the production level Key components Stable and scalable hardware Redundancy Node level Utility level Network level HVAC level Storage level Current load versus future growth Budget Simplicity Tracking hardware inventory Hardware selection Sizing CPU and memory Single socket versus multi-socket Hyper-threading – enable versus disable Start small with VM resources Balancing node resources Ceph cluster production Forget about hardware RAID Solid State Drive for Ceph Journal Network bandwidth Liquid cooling Total immersion in oil Total immersion in 3M Novec Direct contact liquid cooling Real-world Proxmox scenarios Scenario 1 – an academic institution Scenario 2 – multi-tier storage cluster with a Proxmox cluster Scenario 3 - Virtual infrastructure for a multi-tenant cloud service provider Scenario 4 – nested virtual environment for a software development company Scenario 5 – virtual infrastructure for a public library Scenario 6 – multi-floor office virtual infrastructure with virtual desktops Scenario 7 – virtual infrastructure for the hotel industry Scenario 8 – virtual infrastructure for geological survey organization Summary Back Up and Restore Virtual Machines Proxmox backup options A full backup Full backup modes Snapshot Suspend Stop Backup compression None LZO GZIP Snapshots Configuring backup storage Show VM configuration from backup Configuring full backup Creating a schedule for backup Node Storage Day of week Start Time Selection mode Send email to Email notification Compression Mode Enable Creating a manual backup Creating snapshots Restoring a virtual machine Backup/restore through the CLI Backup using the CLI Restore using the CLI Unlocking a VM after a backup error Virtual machine replication Creating a replication task through the GUI Target Schedule Rate limit (MB/s) Enabled Creating a replication task through the CLI Replication process Backup configuration file The bwlimit option The lockwait option The stopwait option The stdexcludes option The mailto option The script option The exclude-path option The pigz option Summary Updating/Upgrading Proxmox Introducing Proxmox updates Updating Proxmox through the GUI Updating Proxmox through the CLI Difference between upgrade and dist-upgrade Recovering from the grub2 update issue Updating after a subscription change Rebooting dilemma after Proxmox updates Applying update without reboot Summary Proxmox Troubleshooting Proxmox node issues Issue – fresh Proxmox install stuck with /dev to be a fully populated error during node reboot Issue – rejoining a node to a Proxmox node with the same old IP address Issue – Proxmox installation completed but grub is in an endless loop after reboot Issue – LSI MegaRAID 9240-8i/9240-4i causes an error during booting of the Proxmox node Downloading and updating the LSI driver Updating the Supermicro BIOS Issue – the Upgrade button is disabled on the Proxmox GUI, which prevents the node upgrade Issue – Proxmox cannot start due to the getpwnam error Issue – cannot log in to the GUI as root after reinstalling Proxmox on the same node The main cluster issues Issue – Proxmox virtual machines are running, but the Proxmox GUI shows that everything is offline Issue – kernel panic when disconnecting USB devices, such as a keyboard, mouse, or UPS Issue – virtual machines on Proxmox will not shut down if shutdown is initiated from the Proxmox GUI Issue – kernel panic with HP NC360T (Intel 82571EB chipset) only in Proxmox VE 3.2 Issue – the Proxmox cluster is out of quorum and cluster filesystem is in read-only mode Issue – VM will not respond to shutdown or restart Issue – Proxmox GUI not responding after Firefox update Issue – the Proxmox GUI is not showing RRD graphs Storage issues Issue – deleting a damaged LVM from Proxmox with the error read failed from 0 to 4096 Issue – Proxmox cannot mount NFS share due to the timing out error Issue – how to delete leftover NFS shares in Proxmox or what to do when the NFS stale file handle error occurs? Issue – Proxmox issues --mode session exit code 21 errors while trying to access the iSCSI target Issue – cannot read an iSCSI target even after it has been deleted from Proxmox storage Issue – a Ceph node is removed from the Proxmox cluster, but OSDs still show up in PVE Issue – the no such block device error during creation of an OSD through the Proxmox GUI Issue – the fstrim command does not trim unused blocks for the Ceph storage Issue – the RBD couldn't connect to cluster (500) error when connecting Ceph with Proxmox Issue – changing the storage type from IDE to VirtIO after the VM has been set up and the OS has been installed Issue – the pveceph configuration not initialized (500) error when you click on the Ceph tab in the Proxmox GUI Issue – the CephFS storage disappears after a Proxmox node reboots Issue – VM cloning does not parse in the Ceph storage Issue – VM disk images stored on ZFS is extremely slow Network connectivity issues Issue – no connectivity on Realtek RTL8111/8411 rev. 06 network interfaces Issue – network performance is slower with the E1000 virtual network interfaces Issue – patch port for Open vSwitch in Proxmox not working Issue – trying to add a node to a newly created Proxmox cluster when nodes do not form quorum Issue – implemented IPv6 but firewall rules do not get applied KVM virtual machine issues Issue – Windows 7/XP machine converted to Proxmox KVM hangs during boot Issue – Windows 7 VM does not reboot, instead it shuts down, requiring a manual boot from Proxmox Issue – the qemu-img command does not convert the .vmdk image files created with the .ova template in Proxmox VE 5.0 Issue – online migration of a virtual machine fails with a failed to sync data error Issue – no audio in Windows KVM Issue – the VirtIO virtual disk is not available during the Windows Server installation LXC container issues Issue – a Proxmox node hangs when trying to stop or restart an LXC container Issue – the noVNC console only shows a cursor for LXC containers Backup/restore issues Issue – a Proxmox VM is locked after backup crashes unexpectedly Issue – how can Proxmox back up only the primary OS virtual disk instead of all the virtual disks for a VM? Issue – backup of virtual machines stops prematurely with an operation not permitted error Issue – a backup task takes a very long time to complete, or it crashes when multiple nodes are backing up to the same backup storage Issue – backup of virtual machines aborts a backup task prematurely Issue – backup storage has a lot of .dat files and .tmp folders using the storage space VNC/SPICE console issues Issue – the mouse pointer is not shared with SPICE (virt-viewer) on Windows 8 VM Issue – remote viewer is unable to connect to a SPICE-enabled virtual machine on the Windows OS Firewall issues Issue – rules are created and a firewall is enabled for vNIC, but rules do not get applied Issue – a firewall is enabled for a VM and the necessary rules are created, but nothing is being filtered for that VM Summary Rescuing Proxmox Recovering from OS drive failure Physical drive failure OS data corruption Migrating VMs from a faulty node Reinstalling Proxmox Recovering from a quorum failure Recovering from a node failure Recovering from a network failure Loss of connectivity between Proxmox nodes Loss of connectivity between Proxmox nodes and users Loss of connectivity between Proxmox and storage nodes Recovering from Ceph failure Best practices for a healthy Ceph cluster Stuck inconsistent PGs in Ceph Stuck inactive incomplete PGs in Ceph Error while moving a Ceph journal to another drive Ceph node running out of resources during recovery Summary Discover real world scenarios for Proxmox troubleshooting and become an expert cloud builderAbout This Book\* Formulate Proxmox-based solutions and set up virtual machines of any size while gaining expertise even on the most complex multi-cluster setups\* Master the skills needed to analyze, monitor, and troubleshoot real-world virtual environments\* This is the most up-to-date title on mastering Proxmox, with examples based on the new Linux Kernel 4.10.15 and Debian Stretch (9.x)Who This Book Is ForThis book is for Linux and system administrators and professionals working in IT teams who would like to design and implement an enterprise-quality virtualized environment using Proxmox. Some knowledge of networking and virtualization concepts is assumed.What You Will Learn\* Install basic Proxmox VE nodes and get to know the Proxmox GUI\* Get to know Proxmox's internal structure and mechanics\* Create and manage KVM or LXC-based virtual machines\* Understand advanced virtual networks\* Configure high availability Proxmox nodes\* Integrate Ceph big data storage with the Proxmox hypervisor\* Plan a large virtual environment for cloud-based services\* Discover real-world scenarios for Proxmox troubleshootingIn DetailProxmox is an open source server virtualization solution that has enterprise-class features for managing virtual machines, for storage, and to virtualize both Linux and Windows application workloads.You'll begin with a refresher on the advanced installation features and the Proxmox GUI to familiarize yourself with the Proxmox VE hypervisor. Then, you'll move on to explore Proxmox under the hood, focusing on storage systems, such as Ceph, used with Proxmox. Moving on, you'll learn to manage KVM virtual machines, deploy Linux containers fast, and see how networking is handled in Proxmox. You'll also learn how to protect a cluster or a VM with a firewall and explore the new high availability features introduced in Proxmox VE 5.0. Next, you'll dive deeper into the backup/restore strategy and see how to properly update and upgrade a Proxmox node. Later, you'll learn how to monitor a Proxmox cluster and all of its components using Zabbix. Finally, you'll discover how to recover Promox from disaster strikes through some real-world examples.By the end of the book, you'll be an expert at making Proxmox work in production environments with minimal downtime.Style and approachThis book walks you through every aspect of virtualization using Proxmox using a practical, scenario-based approach that features best practices and all the weaponry you need to succeed when building virtual environments with Proxmox 5.0. Discover real world scenarios for Proxmox troubleshooting and become an expert cloud builder About This Book Formulate Proxmox-based solutions and set up virtual machines of any size while gaining expertise even on the most complex multi-cluster setups Master the skills needed to analyze, monitor, and troubleshoot real-world virtual environments This is the most up-to-date title on mastering Proxmox, with examples based on the new Linux Kernel 4.10.15 and Debian Stretch (9.x) Who This Book Is For This book is for Linux and system administrators and professionals working in IT teams who would like to design and implement an enterprise-quality virtualized environment using Proxmox. Some knowledge of networking and virtualization concepts is assumed. What You Will Learn Install basic Proxmox VE nodes and get to know the Proxmox GUI Get to know Proxmox's internal structure and mechanics Create and manage KVM or LXC-based virtual machines Understand advanced virtual networks Configure high availability Proxmox nodes Integrate Ceph big data storage with the Proxmox hypervisor Plan a large virtual environment for cloud-based services Discover real-world scenarios for Proxmox troubleshooting In Detail Proxmox is an open source server virtualization solution that has enterprise-class features for managing virtual machines, for storage, and to virtualize both Linux and Windows application workloads. You'll begin with a refresher on the advanced installation features and the Proxmox GUI to familiarize yourself with the Proxmox VE hypervisor. Then, you'll move on to explore Proxmox under the hood, focusing on storage systems, such as Ceph, used with Proxmox. Moving on, you'll learn to manage KVM virtual machines, deploy Linux containers fast, and see how networking is handled in Proxmox. You'll also learn how to protect a cluster or a VM with a firewall and explore the new high availability features introduced in Proxmox VE 5.0. Next, you'll dive deeper into the backup/restore strategy and see how to properly update and upgrade a Proxmox node. Later, you'll learn how to monitor a Proxmox cluster and all of its components using Zabbix. Finally, you'll discover how to recover Promox from disaster strikes through some real-world examples. By the end of the book, you'll be an expert at making Proxmox work in production environments with minimal downtime. Style and approach This book walks you through every aspect of virtualization using Proxmox using a practi ..