Generative Adversarial Networks for Image-to-Image Translation

Generative Adversarial Networks (GAN) have started a revolution in Deep Learning, and today GAN is one of the most researched topics in Artificial Intelligence. Generative Adversarial Networks for Image-to-Image Translation provides a comprehensive overview of the GAN (Generative Adversarial Network) concept starting from the original GAN network to various GAN-based systems such as Deep Convolutional GANs (DCGANs), Conditional GANs (cGANs), StackGAN, Wasserstein GANs (WGAN), cyclical GANs, and many more. The book also provides readers with detailed real-world applications and common projects built using the GAN system with respective Python code. A typical GAN system consists of two neural networks, i.e., generator and discriminator. Both of these networks contest with each other, similar to game theory. The generator is responsible for generating quality images that should resemble ground truth, and the discriminator is accountable for identifying whether the generated image is a real image or a fake image generated by the generator. Being one of the unsupervised learning-based architectures, GAN is a preferred method in cases where labeled data is not available. GAN can generate high-quality images, images of human faces developed from several sketches, convert images from one domain to another, enhance images, combine an image with the style of another image, change the appearance of a human face image to show the effects in the progression of aging, generate images from text, and many more applications. GAN is helpful in generating output very close to the output generated by humans in a fraction of second, and it can efficiently produce high-quality music, speech, and images. Introduces the concept of Generative Adversarial Networks (GAN), including the basics of Generative Modelling, Deep Learning, Autoencoders, and advanced topics in GAN Demonstrates GANs for a wide variety of applications, including image generation, Big Data and data analytics, cloud computing, digital transformation, E-Commerce, and Artistic Neural Networks Includes a wide variety of biomedical and scientific applications, including unsupervised learning, natural language processing, pattern recognition, image and video processing, and disease diagnosis Provides a robust set of methods that will help readers to appropriately and judiciously use the suitable GANs for their applications Front matter Copyright Contributors Super-resolution-based GAN for image processing: Recent advances and future trends Introduction Train the discriminator Train the generator Organization of the chapter Background study SR-GAN model for image processing Architecture of SR-GAN Network architecture Perceptual loss Content loss Adversarial loss Case study Case study 1: Application of EE-GAN to enhance object detection Case study 2: Edge-enhanced GAN for remote sensing image Case study 3: Application of SRGAN on video surveillance and forensic application Case study 4: Super-resolution of video using SRGAN Open issues and challenges Conclusion and future scope References GAN models in natural language processing and image translation Introduction Variational auto encoders Drawback of VAE Brief introduction to GAN Basic GAN model classification based on learning Unsupervised learning Vanilla GAN WGAN WGAN-GP Info GAN BEGAN Unsupervised sequential GAN Parallel GAN Cycle GAN Semisupervised learning Semi GAN Supervised learning CGAN BiGAN ACGAN Supervised seq-GAN Comparison of GAN models Pros and cons of the GAN models GANs in natural language processing Application of GANs in natural language processing Generation of semantically similar human-understandable summaries using SeqGAN with policy gradient Semantic similarity discriminator Generation of quality language descriptions and ranking using RankGAN Dialogue generation using reinforce GAN Text style transfer using UGAN Tibetan question-answer corpus generation using Qu-GAN Generation of the sentence with lexical constraints using BFGAN Short-spoken language intent classification with cSeq-GAN Recognition of Chinese characters using TH-GAN NLP datasets GANs in image generation and translation Applications of GANs in image generation and translation Ensemble learning GANs in face forensics Spherical image generation from the 2D sketch using SGANs Generation of radar images using TsGAN Generation of CT from MRI using MCRCGAN Generation of scenes from text using text-to-image GAN Gastritis image generation using PG-GAN Image-to-image translation using quality-aware GAN Generation of images from ancient text using encoder-based GAN Generation of footprint images from satellite images using IGAN Underwater image enhancement using a multiscale dense generative adversarial network Image datasets Evaluation metrics Precision Recall F1 score Accuracy Fréchet inception distance Inception score IoU score Sensitivity Specificity BELU score ROUGE score Tools and languages used for GAN research Python R programming MatLab Julia Open challenges for future research Conclusion References Generative adversarial networks and their variants Introduction of generative adversarial network (GAN) Generative model (GM) Discriminator model (DM) Related work Deep-learning methods Convolutional neural network Recurrent neural network (RNN) Deep belief network (DBN) Long short-term memory Variants of GAN Vari GAN TGAN Laplacian pyramid of generative adversarial network (LAPGAN) Video generative adversarial network (VGAN) Superresolution GAN (SRGAN) Face conditional generative adversarial network (FCGAN) Applications of GAN Conclusion References Comparative analysis of filtering methods in fuzzy C-means: Environment for DICOM image segmentation Introduction Organization of chapter Related works Methodology Proposed algorithm Evaluation metrics Morphological operations 2D median filter Imguided filter Imfilter Wiener 2 filtering Gaussian filter Research design Experimental analysis Performance analysis Results and discussion Conclusion References A review of the techniques of images using GAN Introduction to GANs Need for GANs GAN architectures Fully connected GANs Conditional GANs Adversarial autoencoders Deep convolution GANs StackGANs CycleGANs Wasserstein GANs Discussion on research gaps GAN applications Conclusion References A review of techniques to detect the GAN-generated fake images Introduction DeepFake DeepFake challenges GAN-based techniques for generating DeepFake Image-to-image translation StarGAN: Unified generative adversarial networks for multidomain image-to-image translation Toward multimodal image-to-image translation U-GAT-IT: Unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-i ... Image-to-image translation with conditional adversarial networks Multichannel attention selection GAN with cascaded semantic guidance for cross-view image translation Cross-view image synthesis using geometry-guided CGANs Cross-view image synthesis using CGANs WarpGAN: Automatic caricature generation CariGANs: Unpaired photo-to-caricature translation Unpaired photo-to-caricature translation on faces in the wild Text-to-image synthesis Generative adversarial text-to-image synthesis StackGAN: Text to photo-realistic image synthesis with stacked generative adversarial networks MC-GAN: Multiconditional generative adversarial network for image synthesis MirrorGAN: Learning text-to-image generation by redescription StackGAN++: Realistic image synthesis with stacked generative adversarial networks Conditional image generation and manipulation for user-specified content Controllable text-to-image generation DM-GAN: Dynamic memory generative adversarial networks for text-to-image synthesis Object-driven text-to-image synthesis via adversarial training AttnGAN: Fine-grained text-to-image generation with attentional generative adversarial networks Cycle text-to-image GAN with BERT Dualattn-GAN: Text-to-image synthesis with dual attentional generative adversarial network Artificial intelligence-based methods to detect DeepFakes Can forensic detectors identify GAN-generated images? Detection of deep network-generated images using disparities in color components Detecting and simulating artifacts in GAN fake images Detecting GAN-generated fake images using cooccurrence matrices Detecting GAN-generated imagery using color cues Attributing fake images to GANs: Analyzing fingerprints in generated images FakeSpotter: A simple baseline for spotting AI-synthesized fake faces Incremental learning for the detection and classification of GAN-generated images Unmasking DeepFakes with simple features DeepFake detection by analyzing convolutional traces Face X-ray for more general face forgery detection DeepFake image detection based on pairwise learning Comparative study of artificial intelligence-based techniques to detect the face manipulation in GAN-generated fake ... Techniques for detecting the construction of a new face Techniques for detecting the swapping of the facial identity Techniques for detecting the manipulated of facial features Techniques for detecting the manipulated facial expressions Legal and ethical considerations Conclusion and future scope References Synthesis of respiratory signals using conditional generative adversarial networks from scalogram representation Introduction Related work GAN for signal synthesis Simple GAN Conditional generative adversarial networks Conditional GAN for respiratory sound synthesis System model Time-scale representation using CWT Generator and discriminator network architecture of cGAN Algorithm Steps Results and discussion Dataset Data augmentation using conditional GAN Samples of generated scalogram images for different classes Synthesis of respiratory sounds using inverse CWT Performance results Analysis Conclusion and future scope References Visual similarity-based fashion recommendation system Introduction Related works Vanilla GAN InfoGAN CNN-based architectures Fashion recommendation system Deep network architectures Proposed network State-of-the-art CNNs Experiments and results Experimental setup Comparative results Web interface for visual inspection Conclusion and future works References Deep learning-based vegetation index estimation Introduction Related work Vegetation index: Formulations and applications Deep learning-based approaches Proposed approach Cycle generative adversarial networks Residual learning model (ResNet) Proposed architecture Loss functions Least-square GAN's loss Results and discussions Datasets for training and testing Data augmentation Evaluation metrics Experimental results Conclusions References Image generation using generative adversarial networks Introduction to deep learning Generative deep learning Variational autoencoder Introduction to GAN Nash equilibrium GAN and Nash equilibrium Nash equilibrium proof Training problems VAE-GAN Applications Image-to-image translation using {c, cycle}-GAN Face generation using StarGAN Photo-realistic images using SRGAN and Art2Real Image animation and scene generation using monkey net, first-order motion, and StackGAN Future of GANs References Generative adversarial networks for histopathology staining Introduction Generative adversarial networks Improvements to vanilla GAN Deep convolutional GANs Variations in optimization functions Image-quality metrics The image-to-image translational problem Histology and medical imaging Histology as different feature spaces Network architecture and dataset ANHIR dataset Dataset preparation Network architectures Results and discussions Conclusions Appendix: Network architectures References Analysis of false data detection rate in generative adversarial networks using recurrent neural network Introduction Contributions Related works Methods GAN-RNN architecture Optimization of GAN using RNN Performance evaluation Dataset collection Performance metrics Discussions Conclusions References WGGAN: A wavelet-guided generative adversarial network for thermal image translation Introduction Related work Infrared image translation GANs in image translation Wavelet-guided generative adversarial network Overall architecture Wavelet-guided variational autoencoder Reparameterization in latent space Discrete wavelet transformation for pooling Objective functions in adversarial training Experiments Data description Evaluation methods Baselines Experimental setup Translation results Qualitative analysis Quantitative analysis Conclusion References Generative adversarial network for video analytics Introduction Building blocks of GAN Training process Objective functions GAN variations for video analytics GAN variations for video generation and prediction GAN variations for video recognition GAN variations for video summarization PoseGAN Discussion Advantages of GAN Disadvantages of GAN Conclusion References Multimodal reconstruction of retinal images over unpaired datasets using cyclical generative adversarial networks Introduction Related research Multimodal reconstruction of retinal images Cyclical GAN methodology Paired SSIM methodology Network architectures Experiments and results Datasets Qualitative evaluation of the reconstruction Quantitative evaluation of the reconstruction Ablation analysis of the generated images Structural coherence of the generated images Discussion and conclusions References Generative adversarial network for video anomaly detection Introduction Anomaly detection for surveillance videos A broader view of generative adversarial network for anomaly detection in videos Literature review The basic structure of generative adversarial network The literature of video anomaly detection based on generative adversarial network Cross-channel generative adversarial networks Future frame prediction based on generative adversarial network Cross-channel adversarial discriminators Training a generative adversarial network Using generative adversarial network based on the image-to-image translation Unsupervised learning of generative adversarial network for video anomaly detection System overview Feature collection Spatiotemporal translation model Anomaly detection Experimental results Dataset UCSD dataset UMN dataset CHUK Avenue dataset Implementation details Evaluation criteria Receiver operating characteristic (ROC) Area under curve (AUC) Equal error rate (EER) Frame-level and pixel-level evaluations for anomaly detection Pixel accuracy Structural similarity index (SSIM) Performance of DSTN The comparison of generative adversarial network with an autoencoder Advantages and limitations of generative adversarial network for video anomaly detection Summary References Index