Human and Robot Hands: Sensorimotor Synergies to Bridge the Gap Between Neuroscience and Robotics (Springer Series on Touch and Haptic Systems)

This book looks at the common problems both human and robotic hands encounter when controlling the large number of joints, actuators and sensors required to efficiently perform motor tasks such as object exploration, manipulation and grasping. The authors adopt an integrated approach to explore the control of the hand based on sensorimotor synergies that can be applied in both neuroscience and robotics. Hand synergies are based on goal-directed, combined muscle and kinematic activation leading to a reduction of the dimensionality of the motor and sensory space, presenting a highly effective solution for the fast and simplified design of artificial systems. Presented in two parts, the first part, __Neuroscience,__ provides the theoretical and experimental foundations to describe the synergistic organization of the human hand. The second part, __Robotics, Models and Sensing Tools__, exploits the framework of hand synergies to better control and design robotic hands and haptic/sensing systems/tools, using a reduced number of control inputs/sensors, with the goal of pushing their effectiveness close to the natural one. __Human and Robot Hands__ provides a valuable reference for students, researchers and designers who are interested in the study and design of the artificial hand. Series Editors' Foreword 6 Contents 7 Contributors 9 1 Introduction 12 References 15 Part I Neuroscience 17 2 Dexterous Manipulation: From High-Level Representation to Low-Level Coordination of Digit Forces and Positions 18 2.1 Introduction 18 2.2 Materials and Methods 20 2.3 Experiment 1: Digit Force and Position Coordination in Unconstrained Grasping 23 2.4 Experiment 2: Transfer of Learned Manipulation Between Different Grip Types 28 2.5 Discussion 31 2.5.1 Redundancy of Kinematic and Kinetic Solutions Through Digit Force-to-Position Modulation 32 2.5.2 High-Level Representation of Learned Manipulation 33 2.5.3 Open Questions and Future Research 34 References 35 3 Digit Position and Force Synergies During Unconstrained Grasping 37 3.1 Introduction 38 3.2 Methods 39 3.2.1 Participants 39 3.2.2 Hardware 39 3.2.3 Procedure 40 3.3 Data Processing and Analysis 41 3.4 Results 42 3.4.1 Center of Pressure for Individual Participants 42 3.4.2 Digit Normal Forces Versus CoPs 43 3.4.3 Digit Forces Synergies 44 3.5 Discussion 46 References 47 4 The Motor Control of Hand Movements in the Human Brain: Toward the Definition of a Cortical Representation of Postural Synergies 49 4.1 Introduction 49 4.2 Action Processing in the Brain 50 4.3 A Cortical Network for Hand Posture Control 51 4.4 The Network for Hand Control in Humans: fMRI Evidences 52 4.5 Somatotopic Control of Hand Muscles 54 4.6 ``Languages'' of Hand Control in Primary Motor Cortex 56 4.7 Synergies and Their Brain Correlates 57 4.8 Alternative Hypotheses: A Revised Somatotopy? 58 4.9 Techniques for Hand Movement Recordings: Motion Capture and EMG 59 4.10 Encoding Techniques: Integrating Behavioral and fMRI Data 60 4.11 Combining Techniques to ``Decode'' Hand Posture 61 4.12 Description and Preliminary Results 62 4.13 Conclusions and Future Directions 63 References 64 5 Synergy Control in Subcortical Circuitry: Insights from Neurophysiology 69 5.1 Introduction 69 5.2 State-of-the-Art 70 5.3 Problem Framing 70 5.4 Synergy Control in Subcortical Circuitry 71 5.5 Conclusions 75 References 75 6 Neuronal ``Op-amps'' Implement Adaptive Control in Biology and Robotics 77 6.1 Introduction 77 6.1.1 Two Central (Nervous System) Problems 77 6.1.2 Main Objective 79 6.1.3 Scope and Assumptions 79 6.1.4 Outline 80 6.2 The Neuronal Op-amp 80 6.2.1 Plasticity of Neuronal Op-amps 80 6.2.2 Internal Model Control Using Neuronal Op-amps 81 6.3 Experiment: Neuronal Op-amps in Biology 83 6.3.1 Setup 83 6.3.2 Execution 84 6.3.3 Results 86 6.4 Experiment: Neuronal Op-amps in Engineering 86 6.4.1 Setup 88 6.4.2 Execution 89 6.4.3 Results 89 6.5 Discussion and Conclusions 91 References 93 7 Sensorymotor Synergies: Fusion of Cutaneous Touch and Proprioception in the Perceived Hand Kinematics 95 7.1 Introduction 96 7.2 Contact Area 97 7.2.1 Methods 98 7.2.2 Results 99 7.3 Slip Motion 100 7.3.1 Methods 100 7.3.2 Results 102 7.4 Discussion 103 References 105 Part II Robotics, Models and Sensing Tools 107 8 From Soft to Adaptive Synergies: The Pisa/IIT SoftHand 108 8.1 Introduction 109 8.2 Hand Actuation, Synergies and Adaptation 111 8.2.1 Fully Actuated Hands 111 8.2.2 Approaches to Simplification 113 8.2.3 Soft Synergies 115 8.2.4 Adaptive Synergies 116 8.2.5 From Soft to Adaptive Synergies 117 8.3 The Pisa/IIT SoftHand 119 8.4 Experimental Results 123 8.4.1 Force and Torque Measurements 123 8.4.2 Grasp Experiments 124 8.5 A New Set of Possibilities 126 8.6 Conclusion 130 References 131 9 A Learn by Demonstration Approach for Closed-Loop, Robust, Anthropomorphic Grasp Planning 133 9.1 Introduction 134 9.2 Apparatus and Kinematic Models 136 9.2.1 Mitsubishi PA 10 DLR/HIT II Robot Arm Hand System 136 9.2.2 Tactile Sensors 137 9.2.3 Motion Capture Systems 138 9.2.4 Kinematic Model of the Human Arm Hand System 139 9.3 Learn by Demonstration for Closed Loop, Anthropomorphic Grasp Planning 139 9.3.1 Learn by Demonstration Experiments 139 9.3.2 Mapping Human to Robot Motion with Functional Anthropomorphism 140 9.3.3 Learning Navigation Function Models in the Anthropomorphic Robot Low-D Space 142 9.3.4 A Vision System Based on RGB-D Cameras 143 9.4 Task Specific, Robust Grasping with Tactile Sensing 144 9.4.1 A Scheme for Deriving Task Specific Grasping Postures 144 9.4.2 A Scheme that Provides Optimal Force Transmission and Robustness Against Positioning Inaccuracies 145 9.4.3 A Grasping Force Optimization Scheme Utilizing Tactile Sensing 146 9.5 Results and Experimental Validation 148 9.5.1 Closed-Loop, Anthropomorphic Grasp Planning Scenario 149 9.5.2 Task-Specific, Robust Grasping Scenario 149 9.6 Conclusions and Discussion 152 References 153 10 Teleimpedance Control: Overview and Application 156 10.1 Teleimpedance Control 157 10.2 Application 159 10.2.1 Teleimpedance Control of a Robotic Arm 159 10.2.2 Teleimpedance Control of a Robotic Hand 167 References 173 11 Incremental Learning of Muscle Synergies: From Calibration to Interaction 175 11.1 Introduction 175 11.2 Background 177 11.2.1 Muscle Activations in Prosthetic Control 178 11.2.2 Unreliability 179 11.2.3 Building a More Detailed Model or Learning More? 181 11.2.4 Incremental/Interactive Learning 182 11.3 A Practical Method of Incremental Learning 184 11.3.1 Monolithic Learning in the Linear Case 184 11.3.2 Extension to the Non-linear Case 185 11.3.3 Incrementality 187 11.3.4 Obtaining Ground Truth 188 11.3.5 Applications 190 11.4 Discussion 191 11.4.1 On the Capacity of Incremental Learning 192 11.4.2 Relation to Muscle Synergies as Traditionally Defined 193 11.5 Conclusions 193 References 194 12 How to Map Human Hand Synergies onto Robotic Hands Using the SynGrasp Matlab Toolbox 198 12.1 Introduction 199 12.2 The SynGrasp Toolbox 200 12.2.1 How to Use SynGrasp 201 12.2.2 Hand Modelling 202 12.2.3 Grasp Definition 203 12.2.4 Grasp Analysis 206 12.3 Object-Based Mapping Using SynGrasp 208 12.4 Conclusion 211 References 211 13 Quasi-Static Analysis of Synergistically Underactuated Robotic Hands in Grasping and Manipulation Tasks 213 13.1 Introduction 214 13.2 System Modeling 216 13.2.1 Object Equations 217 13.2.2 Hand Equations 219 13.2.3 Hand/Object Interaction Model 219 13.2.4 Soft Synergy Underactuation Model 220 13.2.5 The Fundamental Grasp Equation 221 13.3 Controllable System Configuration Variations 222 13.3.1 The Canonical Form of the Fundamental Grasp Equation 222 13.3.2 Relevant Properties of the Canonical Form of the Fundamental Grasp Matrix 223 13.3.3 GEROME-B: A Specialized Gauss Elimination Method for Block Partitioned Matrices 224 13.4 Solution Space Decomposition 226 13.4.1 Relevant Types of System Solutions 227 13.4.2 Discovering (Non-)Nullity Patterns in the Solution Space 228 13.5 Geometrical Interpretation of the Fundamental Grasp Equation 229 13.6 Other Types of (Under-)Actuation 230 13.7 Numerical Results 231 13.7.1 Power Grasp 231 13.8 Conclusions 233 References 234 14 A Simple Model of the Hand for the Analysis of Object Exploration 236 14.1 Introduction 236 14.2 Model of the Hand 239 14.2.1 Sensors 240 14.2.2 Calibration 240 14.2.3 Calculation of a Point from a Sensor 242 14.2.4 Joint Positions 243 14.2.5 Distal (1st) phalanx 245 14.2.6 Middle (2nd) phalanx 245 14.2.7 Proximal (3rd) phalanx 246 14.2.8 Hand 247 14.3 Application Example: Contact Analysis 247 14.4 Experimental Evaluation of the Model 249 14.4.1 Participants and Apparatus 249 14.4.2 Task and Procedure 250 14.4.3 Analysis 250 14.4.4 Results 250 14.5 Discussion 252 14.5.1 Comparison with Other Models 254 14.5.2 Applications 256 14.5.3 Conclusion 257 References 258 15 Synergy-Based Optimal Sensing Techniques for Hand Pose Reconstruction 260 15.1 Introduction 261 15.2 Biology and Artificial Systems: A Mutual Inspiration 262 15.3 Performance Enhancement 264 15.3.1 The Hand Posture Estimation Algorithm 264 15.3.2 Data Acquisition 266 15.3.3 Experimental Results 267 15.4 Optimal Design 270 15.4.1 Problem Definition 272 15.4.2 Continuous Sensing Design 273 15.4.3 Discrete Sensing Design 276 15.4.4 Hybrid Sensing Design 277 15.4.5 Continuous and Discrete Sensing Optimal Distribution 279 15.4.6 Estimation Results with Optimal Discrete Sensing Devices 280 15.5 Conclusions and Future Works 282 References 283 This Book Looks At The Common Problems Both Human And Robotic Hands Encounter When Controlling The Large Number Of Joints, Actuators And Sensors Required To Efficiently Perform Motor Tasks Such As Object Exploration, Manipulation And Grasping. The Authors Adopt An Integrated Approach To Explore The Control Of The Hand Based On Sensorimotor Synergies That Can Be Applied In Both Neuroscience And Robotics. Hand Synergies Are Based On Goal-directed, Combined Muscle And Kinematic Activation Leading To A Reduction Of The Dimensionality Of The Motor And Sensory Space, Presenting A Highly Effective Solution For The Fast And Simplified Design Of Artificial Systems. Presented In Two Parts, The First Part, Neuroscience, Provides The Theoretical And Experimental Foundations To Describe The Synergistic Organization Of The Human Hand. The Second Part, Robotics, Models And Sensing Tools, Exploits The Framework Of Hand Synergies To Better Control And Design Robotic Hands And Haptic/sensing Systems/tools, Using A Reduced Number Of Control Inputs/sensors, With The Goal Of Pushing Their Effectiveness Close To The Natural One. Human And Robot Hands Provides A Valuable Reference For Students, Researchers And Designers Who Are Interested In The Study And Design Of The Artificial Hand. Introduction -- Part I: Neuroscience -- Dexterous Manipulation: From High-level Representation To Low-level Co-ordination Of Digit Force And Position -- Digit Position And Force Synergies During Unconstrained Grasping -- The Motor Control Of Hand Movements In The Human Brain: Toward The Definition Of A Cortical Representation Of Postural Synergies -- Synergy Control In Subcortical Circuitry: Insights From Neurophysiology -- Neuronal Op-amps Implement Adaptive Control In Biology And Robotics -- Sensorimotor Synergies:fusion Of Cutaneous Touch And Proprioception In The Perceived Hand Kinematics -- Part Ii: Robotics, Models And Sensing Tools -- From Soft To Adaptive Synergies:the Pisa/iit Softhand -- A Learn By Demonstration Approach For Closed-loop Robust Anthromorphic Grasp Planning -- Teleimpudance Control: Overview And Application -- Incremental Learning Of Muscle Synergies:from Calibrating A Prothesis To Interacting With It -- How To Map Human Hand Synergies Onto Robotic Hands Using The syngrasp Matlab Toolbox -- Quasi-static Analysis Of Synergistically Underactuated Robotic Hands In Grasp And Manipulation Tasks -- A Simple Model Of The Hand For The Analysis Of Object Exploration -- Synergy-based Optimal Sensing Techniques For Hand Pose Reconstruction. Edited By Matteo Bianchi, Alessandro Moscatelli. Front Matter....Pages i-xi Introduction....Pages 1-5 Front Matter....Pages 7-7 Dexterous Manipulation: From High-Level Representation to Low-Level Coordination of Digit Forces and Positions....Pages 9-27 Digit Position and Force Synergies During Unconstrained Grasping....Pages 29-40 The Motor Control of Hand Movements in the Human Brain: Toward the Definition of a Cortical Representation of Postural Synergies....Pages 41-60 Synergy Control in Subcortical Circuitry: Insights from Neurophysiology....Pages 61-68 Neuronal “Op-amps” Implement Adaptive Control in Biology and Robotics....Pages 69-86 Sensorymotor Synergies: Fusion of Cutaneous Touch and Proprioception in the Perceived Hand Kinematics....Pages 87-98 Front Matter....Pages 99-99 From Soft to Adaptive Synergies: The Pisa/IIT SoftHand....Pages 101-125 A Learn by Demonstration Approach for Closed-Loop, Robust, Anthropomorphic Grasp Planning....Pages 127-149 Teleimpedance Control: Overview and Application....Pages 151-169 Incremental Learning of Muscle Synergies: From Calibration to Interaction....Pages 171-193 How to Map Human Hand Synergies onto Robotic Hands Using the SynGrasp Matlab Toolbox....Pages 195-209 Quasi-Static Analysis of Synergistically Underactuated Robotic Hands in Grasping and Manipulation Tasks....Pages 211-233 A Simple Model of the Hand for the Analysis of Object Exploration....Pages 235-258 Synergy-Based Optimal Sensing Techniques for Hand Pose Reconstruction....Pages 259-283 This volume looks at the common problems both human and robotic hands encounter when controlling the large number of joints, actuators and sensors required to efficiently perform motor tasks such as object exploration, manipulation and grasping