Computer Vision – ACCV 2020: 15th Asian Conference on Computer Vision, Kyoto, Japan, November 30 – December 4, 2020, Revised Selected Papers, Part V (Lecture Notes in Computer Science)

The six volume set of LNCS 12622-12627 constitutes the proceedings of the 15th Asian Conference on Computer Vision, ACCV 2020, held in Kyoto, Japan, in November/ December 2020.* The total of 254 contributions was carefully reviewed and selected from 768 submissions during two rounds of reviewing and improvement. The papers focus on the following topics: Part I: 3D computer vision; segmentation and grouping Part II: low-level vision, image processing; motion and tracking Part III: recognition and detection; optimization, statistical methods, and learning; robot vision Part IV: deep learning for computer vision, generative models for computer vision Part V: face, pose, action, and gesture; video analysis and event recognition; biomedical image analysis Part VI: applications of computer vision; vision for X; datasets and performance analysis *The conference was held virtually. Preface Organization Contents – Part V Face, Pose, Action, and Gesture Video-Based Crowd Counting Using a Multi-scale Optical Flow Pyramid Network 1 Introduction 2 Related Work 2.1 Counting in Static Images 2.2 Video-Based Counting Methods 2.3 Optical Flow Pyramid 3 Technical Approach 3.1 Crowd Counting Baseline Network 3.2 Multi-scale Optical Flow Pyramid Network (MOPN) 3.3 Training Details 4 Experiments 4.1 Evaluation Metric 4.2 Crowd Counting in Images 4.3 Crowd Counting in Videos 4.4 Qualitative Results 4.5 Transfer Learning 4.6 Ablation Studies 5 Conclusion References RealSmileNet: A Deep End-to-End Network for Spontaneous and Posed Smile Recognition 1 Introduction 2 Related Work 3 Our Approach 3.1 Preliminaries 3.2 Architecture 3.3 Analysis 4 Experiment 4.1 Setup 4.2 Recognition Performance 4.3 Discussion 5 Conclusion References Decoupled Spatial-Temporal Attention Network for Skeleton-Based Action-Gesture Recognition 1 Introduction 2 Related Work 3 Methods 3.1 Spatial-Temporal Attention Module 3.2 Decoupled Position Encoding 3.3 Spatial Global Regularization 3.4 Complete Attention Module 3.5 Overall Architecture 3.6 Data Decoupling 4 Experiments 4.1 Datasets 4.2 Training Details 4.3 Ablation Studies 4.4 Comparison with Previous Methods 5 Conclusion References Unpaired Multimodal Facial Expression Recognition 1 Introduction 2 Related Work 2.1 Learning with Privileged Information 2.2 Facial Expression Recognition 2.3 Multimodal Facial Expression Recognition 3 Problem Statement 4 Proposed Method 4.1 Basic Classification Loss for Two Views 4.2 Adversarial Learning in Feature and Label Levels 4.3 Visible Reconstruction Loss 4.4 Adaptive Classification Loss Adjustment 4.5 Overall Loss Function 4.6 Optimization 5 Experiment 5.1 Experimental Conditions 5.2 Experimental Results and Analysis 5.3 Comparison to Related Methods 5.4 Evaluation of Adversarial Learning 5.5 Visualization of the Decoder Network 6 Conclusions References Gaussian Vector: An Efficient Solution for Facial Landmark Detection 1 Introduction 2 Related Work 3 Our Approach 3.1 Overview 3.2 Vector Label 3.3 Band Pooling Module 3.4 Beyond the Box 4 Experiments 4.1 Datasets 4.2 Evaluation Metrics 4.3 Implementation Details 4.4 Evaluation on Different Benchmarks 4.5 Ablation Study 4.6 Efficiency Analysis 5 Conclusion References A Global to Local Double Embedding Method for Multi-person Pose Estimation 1 Introduction 2 Related Works 3 Double Embedding Method 4 Experiments 4.1 Experiment Setup 4.2 Results on COCO Dataset 4.3 Results on MPII Dataset 4.4 Results on CrowdPose Dataset 5 Conclusion References Semi-supervised Facial Action Unit Intensity Estimation with Contrastive Learning 1 Introduction 2 Related Work 2.1 Action Unit Modeling 2.2 Self-supervised Learning in Computer Vision 3 Method 3.1 Problem Statement and Notation 3.2 Network 3.3 Unsupervised Pre-training 3.4 Learning with Partially Labeled Data 4 Experimental Results 4.1 Ablation Study 4.2 Comparison with State-of-the-Art 5 Conclusion References MMD Based Discriminative Learning for Face Forgery Detection 1 Introduction 2 Related Work 2.1 Face Manipulation Methods 2.2 Face Forgery Detection 2.3 Domain Adaptation 3 Method 3.1 Overview 3.2 MMD Based Domain Generalization 3.3 Triplet Constraint 3.4 Center Loss 4 Experiments 4.1 Implementation Details 4.2 Evaluation Results 4.3 Analysis 4.4 Ablation Study 5 Conclusions References RE-Net: A Relation Embedded Deep Model for AU Occurrence and Intensity Estimation 1 Introduction 2 Related Works 3 Proposed Method 3.1 Problem Formulation 3.2 Preliminary: Batch Normalization 3.3 AU Relationship Graph 3.4 AU Recognition with Graph Constraint 4 Experiments 4.1 Data 4.2 Evaluation Metrics 4.3 Implementation Details 4.4 AU Detection Results 4.5 AU Intensity Estimation 4.6 Ablation Study 4.7 Conclusion References Learning 3D Face Reconstruction with a Pose Guidance Network 1 Introduction 2 Related Work 3 Method 3.1 Preliminaries 3.2 Pose Guidance Network 3.3 Learning from Multiple Frames 3.4 Training Loss 4 Experimental Evaluation 5 Conclusion References Self-supervised Multi-view Synchronization Learning for 3D Pose Estimation 1 Introduction 2 Prior Work 3 Unsupervised Learning of 3D Pose-Discriminative Features 3.1 Classifying Synchronized and Flipped Views 3.2 Static Backgrounds Introduce Shortcuts 3.3 Implementation 3.4 Transfer to 3D Human Pose Estimation 4 Experiments 4.1 Ablations 4.2 Comparison to Prior Work 4.3 Evaluation of the Synchronization Task 5 Conclusions References Faster, Better and More Detailed: 3D Face Reconstruction with Graph Convolutional Networks 1 Introduction 2 Related Work 3 Method 3.1 Overview of Our Framework 3.2 Coarse 3D Face Reconstruction with GCN 3.3 Unsupervised Detailed Reconstruction 3.4 Network Architecture and Training Details 4 Experiments 4.1 Evaluation Databases 4.2 Evaluation Protocol 4.3 Ablation Study 4.4 3D Face Alignment Results 4.5 3D Face Reconstruction Results 4.6 Qualitative Evaluation 4.7 Comparisons of Inference Speed and Model Size 5 Conclusions References Localin Reshuffle Net: Toward Naturally and Efficiently Facial Image Blending 1 Introduction 2 Related Works 2.1 Image Blending 2.2 Neural Style Transfer 2.3 Generative Adversarial Networks (GAN) 3 Facial Pairs to Blend (FPB) Dataset 3.1 Dataset Generation Algorithm 3.2 Dataset Implementation Details 4 Localin Reshuffle Network (LRNet) 4.1 LocalIN 4.2 Local Reshuffle 4.3 Semantic Correspondence Constraint 4.4 Network Architecture 4.5 Loss Functions 5 Experiments 5.1 Implementation Details 5.2 Ablation Study 5.3 Comparison with Baselines 6 Conclusions References Rotation Axis Focused Attention Network (RAFA-Net) for Estimating Head Pose 1 Introduction 2 Related Work 3 Proposed Approach (RAFA-Net) 3.1 Attentional Spatial Pooling 3.2 Learning 4 Experiments 4.1 Implementation 4.2 Datasets and Evaluation Strategies 4.3 Data Augmentation 4.4 Comparison with the State-of-the-Art (SotA) Methods 4.5 Ablation Studies 5 Conclusion References Unified Application of Style Transfer for Face Swapping and Reenactment 1 Introduction 2 Related Works 2.1 Generative Models 2.2 Face Reenactment 2.3 Face Swapping 3 Proposed Method 3.1 Disentanglement Property and Vector Computations 3.2 Architecture 3.3 Face Reenactment and Swapping 3.4 Losses 3.5 Training Details 4 Experiments 4.1 Ablation Study 4.2 Latent Space Interpolation 4.3 Face Swap and Reenactment State-of-the-Art Comparison 5 Limitations 6 Conclusion References Multiple Exemplars-Based Hallucination for Face Super-Resolution and Editing 1 Introduction 2 Related Work 3 Methodology 3.1 Architecture 3.2 Objective Functions 4 Evaluation 4.1 Quality and Quantity Results 4.2 User Study 4.3 Qualitative Comparison with Respect to the State-of-the-art 4.4 Ablation Study 4.5 Facial Features Editing via Exemplars 5 Conclusion References Imbalance Robust Softmax for Deep Embeeding Learning 1 Introduction 2 Related Work 3 Methodology 3.1 Motivation 3.2 Imbalance Robust Softmax (IR-Softmax) 3.3 Relation to Metric Learning 4 Experiments 4.1 Face Verification 4.2 Person Re-identification 5 Conclusion References Domain Adaptation Gaze Estimation by Embedding with Prediction Consistency 1 Introduction 2 Related Work 3 Proposed Method 3.1 Target Domain Gaze Representation 3.2 Embedding with Prediction Consistency 3.3 Loss Function 3.4 Training 4 Experiments 4.1 Datasets 4.2 Data Pre-processing 4.3 Comparison with Appearance-Based Methods 4.4 Ablation Study 4.5 Visual Results 5 Conclusion References Speech2Video Synthesis with 3D Skeleton Regularization and Expressive Body Poses 1 Introduction 2 Related Work 3 Methods 3.1 Speech2Video Dataset 3.2 Body Model Fitting 3.3 Dictionary Building and Key Pose Insertion 3.4 Train LSTM 3.5 Train Video Generative Network 4 Results 4.1 Evaluation and Analysis 4.2 Ablation Study 5 Conclusion References 3D Human Motion Estimation via Motion Compression and Refinement 1 Introduction 2 Related Works 2.1 Recovering 3D Human Pose and Shape from a Single Image 2.2 Recovering 3D Human Pose and Shape from Video 2.3 Human Pose and Motion Prior 2.4 Human Motion Representation 3 Approach 3.1 Problem Formulation 3.2 Spatio-Temporal Feature Extractor (STE) 3.3 Variational Motion Estimator (VME) 3.4 Motion Residual Regressor (MRR) 3.5 Training and Losses 4 Experiments 4.1 Datasets 4.2 Evaluation Results and Analysis 4.3 Ablation Experiments 5 Conclusion References Spatial Temporal Attention Graph Convolutional Networks with Mechanics-Stream for Skeleton-Based Action Recognition 1 Introduction 2 Related Work 3 Proposed Method 3.1 Spatial-Temporal Graph Convolutional Block 3.2 Attention Branch 3.3 Perception Branch 3.4 Learning Method 3.5 Mechanics-Stream 4 Experiment 4.1 Datasets 4.2 Implementation Details 4.3 Adaptation of Attention Graph to Conventional Method 4.4 Accuracy with Multi-modal Learning 4.5 Comparison with State-of-the-Arts 4.6 Visualization of Attention Graph 4.7 Ablation Study 5 Conclusions References DiscFace: Minimum Discrepancy Learning for Deep Face Recognition 1 Introduction 2 Related Works 3 Minimum Discrepancy Learning 3.1 Discrepancy in Face Recognition Schemes 3.2 Learning Discrepancy-Free Representations 4 Analytical Study 4.1 Feature Visualization 4.2 A Recognition Task Under Extreme Class-Imbalanced Dataset 4.3 Effects on Process Discrepancy 5 Experiments on Various Benchmark Datasets 5.1 Experimental Setup 5.2 QMUL-SurvFace Dataset 5.3 Comparison Results 5.4 Ablation Study on 6 Conclusion References Uncertainty Estimation and Sample Selection for Crowd Counting 1 Introduction 2 Related Work 3 Uncertainty Estimation for Crowd Counting 3.1 Crowd Transformer Network 3.2 Training Objective 4 Uncertainty Guided Sample Selection 4.1 Aleatoric Uncertainty Based Sample Selection 4.2 Ensemble Disagreement Based Sample Selection 5 Experiments 5.1 Crowd Density Prediction 5.2 Uncertainty Guided Image Annotation 5.3 Qualitative Results 6 Conclusions References Multi-task Learning for Simultaneous Video Generation and Remote Photoplethysmography Estimation 1 Introduction 2 Related Work 3 Proposed Method 3.1 Overview 3.2 RPPG Network 3.3 Image-to-Video Network 3.4 Video-to-Video Network 3.5 Overall Framework 4 Experiments 4.1 Datasets 4.2 Implementation Setting 4.3 Evaluation Metrics 4.4 Ablation Study 4.5 Results and Comparison 5 Conclusions References Video Analysis and Event Recognition Interpreting Video Features: A Comparison of 3D Convolutional Networks and Convolutional LSTM Networks 1 Introduction 2 Related Work 3 Approach 3.1 Temporal Masks 3.2 Grad-CAM 4 Experiments 4.1 Datasets 4.2 Architecture Details 4.3 Comparison Method 5 Results 5.1 Interpretability Results on Something-Something 5.2 Interpretability Results on the KTH Actions Dataset 5.3 Discussion 6 Conclusions and Future Work References Encode the Unseen: Predictive Video Hashing for Scalable Mid-stream Retrieval 1 Introduction 2 Related Work 2.1 Video Activity Understanding 2.2 Early Activity Recognition 2.3 Video Hashing 3 Methods 3.1 Overview 3.2 Sampling and Feature Extraction 3.3 Binary RNN Encoder 3.4 Data-Augmentated Encoder via Truncated Training Duplicates 3.5 Augmented Codebook with Truncated Database Duplicates 3.6 Look-Ahead Distillation for Encoders 3.7 Experimental Setup 4 Results 4.1 Qualitative Results 5 Conclusion References Active Learning for Video Description with Cluster-Regularized Ensemble Ranking 1 Introduction 2 Related Work 3 Methods 3.1 Active Learning Methods 3.2 Improving Diversity with Clustering 3.3 Comparison with Coreset Active Learning 3.4 Models 3.5 Datasets 3.6 Experimental Setup 3.7 Evaluation 4 Results and Discussion 5 Conclusion and Future Work References Condensed Movies: Story Based Retrieval with Contextual Embeddings 1 Introduction 2 Related Work 3 Condensed Movie Dataset 3.1 Dataset Collection Pipeline 3.2 Story Coverage 4 Text-to-Video Retrieval 4.1 Model Architecture 5 Experiments 5.1 Experimental Set-Up 5.2 Baselines 5.3 Implementation Details 5.4 Results 6 Plot Alignment 7 Conclusion References Play Fair: Frame Attributions in Video Models 1 Introduction 2 Element Attribution in Sequences 2.1 Element Attribution and the Shapley Value 2.2 Element Attribution in Variable-Length Sequences 2.3 Tractable Approach to Element Shapley Values 3 Experiments 3.1 Analysing Element Shapley Values 3.2 ESV Approximation Evaluation 4 Related Work 5 Conclusion References Transforming Multi-concept Attention into Video Summarization 1 Introduction 2 Related Work 3 Proposed Method 3.1 Problem Formulation and Notation 3.2 Overview of MC-VSA 3.3 Video Self-attention for Summarization 3.4 Self-learning of Video Semantic Consistency for Video Summarization 3.5 Full Objectives 4 Experiment 4.1 Datasets 4.2 Protocols and Implementation Details 4.3 Comparison with Supervised Approaches 4.4 Comparisons with Unsupervised Approaches 4.5 Qualitative Results 4.6 Ablation Studies 5 Conclusion References Learning to Adapt to Unseen Abnormal Activities Under Weak Supervision 1 Introduction 2 Related Work 2.1 Anomaly Detection 2.2 Meta-learning 3 Method 3.1 Overview 3.2 Weakly Supervised Anomaly Detector 3.3 Meta-Training 3.4 Meta-testing 4 Experiments 4.1 Datasets 4.2 Evaluation Metric and Protocol 4.3 Experimental Settings and Implementation Details 4.4 Quantitative Results 4.5 Qualitative Results 4.6 Further Analysis 5 Conclusion References TSI: Temporal Scale Invariant Network for Action Proposal Generation 1 Introduction 2 Related Work 3 Our Approach 3.1 Problem Definition and Video Representation 3.2 Scale-Imbalance Analysis in Proposal Generation 3.3 Temporal Scale Invariant Network 4 Training and Inference 4.1 Training of TSI 4.2 Inference of TSI 5 Experiments 5.1 Datasets and Settings 5.2 Temporal Action Proposal Generation 5.3 Temporal Action Proposal Detection 6 Conclusion References Discovering Multi-label Actor-Action Association in a Weakly Supervised Setting 1 Introduction 2 Related Work 3 Multi-label Action Detection and Recognition 4 Multi-instance and Multi-label (MIML) Learning 5 Actor-Action Association 5.1 Power Set of Actions 5.2 Actor-Action Association 5.3 Training 6 Experiments 6.1 Dataset and Implementation Details 6.2 Experimental Results 7 Conclusion References Reweighted Non-convex Non-smooth Rank Minimization Based Spectral Clustering on Grassmann Manifold 1 Introduction 2 Notation and Definition of Grasssmann Manifold 2.1 Notation 2.2 Definition of Grassmann Manifold 3 Related Works 4 Reweighted Non-convex and Non-smooth Rank Minimization Model on Grassmann Manifold 4.1 Model Formulation 4.2 Optimization of G-RNNRM 4.3 Converge and Complexity Analysis 5 Experimental Results 5.1 Data Setting 5.2 Parameters Setting 5.3 Results Analysis 6 Conclusion References Biomedical Image Analysis Descriptor-Free Multi-view Region Matching for Instance-Wise 3D Reconstruction 1 Introduction 2 Related Works 3 Multi-view Instance Segmentation (MVIS) 3.1 Epipolar Region Matching 3.2 Application: Epipolar Region Matching for Region Proposals 3.3 Application: Instance-Wise 3D Reconstruction 3.4 Implementation Details 4 Experiments 4.1 Dataset 4.2 Multi-view Matching Results 4.3 Instance-Wise 3D Reconstruction Results 5 Discussion References Hierarchical X-Ray Report Generation via Pathology Tags and Multi Head Attention 1 Introduction 2 Related Work 3 Proposed Methodology 3.1 Chest Region Feature Extractor Net (CRFENet) 3.2 Abnormality Detection Net (ADNet) 3.3 Tag Classification Net (TCNet) 3.4 Report Generation Net (RGNet) 3.5 Modular Training and Hyper-parameterization 4 Experimental Analysis 4.1 Dataset Used and Evaluation Metric 4.2 Ablation Study 4.3 Comparative Analysis 4.4 Qualitative Analysis 5 Conclusion References Self-guided Multiple Instance Learning for Weakly Supervised Disease Classification and Localization in Chest Radiographs 1 Introduction 2 Related Work 3 Methodology 3.1 Preliminaries of Multiple-Instance Learning 3.2 Self-guiding Loss 3.3 MIL for Chest Radiograph Diagnosis 4 Experiments 4.1 Datasets 4.2 Implementation Details 4.3 Evaluation Metrics 4.4 Results 5 Conclusion References MBNet: A Multi-task Deep Neural Network for Semantic Segmentation and Lumbar Vertebra Inspection on X-Ray Images 1 Introduction 1.1 Background 2 Multi-Task Deep Neural Network (MBNet) 2.1 Semantic Segmentation Branch 2.2 Inspected Values Branch 2.3 Multiple Loss Function 3 Experiments 3.1 Semantic Segmentation Evaluation 3.2 Lumbar Vertebra Inspection Results 4 Conclusion References Attention-Based Fine-Grained Classification of Bone Marrow Cells 1 Introduction 2 Related Work 2.1 Bone Marrow Cell Classification 2.2 Gradient-Boosting Cross Entropy Loss 3 Proposed Method 3.1 Framework 3.2 ASAE Layer 3.3 Comparison with Similar Methods 3.4 GMMCE Loss 3.5 Preprocessing and Sampling Strategies 4 Experiments 4.1 Dataset 4.2 Implementation Details 4.3 Ablation Study 4.4 Comparison with Other Methods 5 Conclusion References Learning Multi-instance Sub-pixel Point Localization 1 Introduction 2 Related Work 3 Model 3.1 Dense Offset Prediction 3.2 Continuous Heatmap Matching 3.3 Detection Sparsity Through Counting Regularization 4 Experiments 4.1 Single Molecule Localization Microscopy 4.2 Checkerboard Corner Detection 4.3 Sub-frame Temporal Event Detection in Videos 5 Conclusion References Utilizing Transfer Learning and a Customized Loss Function for Optic Disc Segmentation from Retinal Images 1 Introduction 2 Related Work 3 Proposed Method 3.1 Network Architecture 3.2 Transfer Learning 3.3 Loss Function 3.4 Implementation Details 4 ORDS Dataset 5 Experimental Results 5.1 Datasets 5.2 Evaluation Methods 5.3 Effectiveness of TL and IA 5.4 Effectiveness of Loss Functions 5.5 Comparing with Other Approaches 5.6 Leave One Out Experiment 6 Conclusion and Future Work References Author Index