Development of Professional Expertise : Toward Measurement of Expert Performance and Design of Optimal Learning Environments

Development of Professional Expertise is perhaps the finest book I've ever read on how to generate great individual and group performances, and I've read more than 400 books on the topic. Indirectly it shows how ineffective traditional talent management really is at many organizations. Using terms like "work-sample testing, engagement simulation, and after-activity reviews," it provides the reader with a toolbox of approaches that can be applied in any type of organization. The variety of authors and case studies provides extraordinary breadth and depth on the topics of objectively measuring performance and ways to improve it. I can't think of a better investment for any business manager who wants to improve performance throughout his or her organization, or part of an organization. My favorite parts of this book were the actual methods for improving performance, which include engagement simulation, cognitive apprenticeship, processing activities, modeling, and problem-based methods. My on-going professional quest is to find ways to improve group performance because I believe groups are the fundamental unit of any organization. No individual creates and delivers value to customers in a vacuum. That value is created and delivered by groups of people. Organizations don't create new products or services. It is groups within those organizations that create the new products and services. Consequently any methods that can be applied to improve group performance are very valuable in my opinion. Even though on page three, it says, "The primary focus of this book is on individual performance," I still believe these methods for improving performance can be applied to groups as well even though it may require some tweaking of the methods. An additional strength of the book is the breadth of examples and authors. This theme of developing professional expertise was looked at from the eyes of more than 30 authors, all of whom seem to know the content of their chapters very well. This enriched the overall perspective that the reader gained from the book. It would be impossible for any one person to have that much breadth of knowledge and experience. Cover 1 Half-title 3 Title 5 Copyright 6 Contents 7 Figures 10 Tables 13 Contributors 15 1 The Measurement and Development of Professional Performance: An Introduction to the Topic and a Background to the Design and Origin of This Book 19 A Brief History of Research on Objective Measurement of Professional Performance 22 The Expert Performance Approach to the Study of Superior Performance 25 The Process of Generating This Book 27 The Outline of This Book 28 Retrospective Comments 38 References 38 Section 1 CHALLENGES IN PAST AND CONTEMPORARY EFFORTS TO MEASURE AND TRAIN THE OBJECTIVE PERFORMANCE OF PROFESSIONALS 43 2 The 20th-Century Revolution in Military Training 45 Declaration of a Revolution 46 Training Counts 46 Th e Military Training Problem 47 A Revolution in Training 47 A Story to Set Context 48 A Cultural Revolution, Too 50 Connecting Training to Military Proficiency 50 A Training Assessment Conundrum 51 Measuring Military Capability by Weapons Assessments 51 Counting Systems 51 Components of Readiness 53 The Learning Curve and Skills Training 53 Skill Atrophy: The Unlearning Curve 57 Summary: Skills Training and Individual Proficiency 59 Warfare and Mission-Level Profi ciency 59 You Fight Like You Train: A Personal Example 60 A Hierarchy of Learning Curves 61 Quantifying the Relationship Between Training Performance and Warfare Performance 64 The Measurement Problem 64 Measurable War: Air-to-Air Combat and the Existence of Aces 65 Simulating Combat: Creating Aces Without Bloodshed 67 Top Gun and the Air War Over Viet Nam 67 Engagement Simulation and Training 67 Summary of Engagement Simulation 70 Implications of the Success of Engagement Simulation 71 It Is Not Just About the Military 71 Training Surprise: Other Countries Could Use It, Too 71 Limitations 73 The Process Doesn’t Touch Everyone 73 The Interdeployment Readiness Bathtub Curve 73 Training Counts but We Don’t Count Training 74 Conclusions 74 Training Counts 74 Training versus Effectiveness Data Appear Contradictory 75 Resolving the Contradiction: A Hierarchy of Training 75 Intelligence Implications 76 Admonition to the Defense Department 76 References 77 3 Developing Professional Expertise with a Cognitive Apprenticeship Model: Examples from Avionics and Medicine 79 Assessing Professional Competence 80 Cognitive Apprenticeship as an Instructional Model 82 Avionics Troubleshooting 83 Background 84 The Solution 85 Evidence of Effectiveness 89 Reasons for Success 90 Extending Sherlock’s Success to New Domains: From Avionics Troubleshooting to Medical Problem Solving 92 Using Cognitive Models in the Design of Computer-Based Learning Environments 93 Lessons Learned and Future Directions 96 References 98 4 Leadership Development and Assessment: Describing and Rethinking the State of the Art 102 What Do Leaders Do? 103 Leader Behavior 103 Measurement 104 Model of Individual Differences of Leaders’ Thinking 107 Knowledge Structures 107 Processes 108 Support for the Model 111 Leader Assessment 112 Processes 113 Leader Development 114 Knowledge 115 Processes 117 Conclusions 118 References 120 5 Revolutions, Leaders, and Diagnosticians : Reflections on the Themes in Chapters 2–4 126 Chatham’s “First Revolution” in Military Training: (Hopefully) Some Constructive Comments 127 Sherlock and Its Heirs 136 Is Leadership a Natural Kind? 138 Summary 144 References 145 Section 2 PAST AND CONTEMPORARY EFFORTS TO DESIGN INSTRUCTION, TRAIN, AND MAINTAIN PROFESSIONAL PERFORMANCE 147 6 Research on Past and Current Training in Professional Domains: The Emerging Need for a Paradigm Shift 149 The Changing Goals of Professional Training 149 Instructional Systems Design 151 Social-Constructivist Design 152 Whole-Task Design Models 154 State of the Art and Outlook 155 Learning Technologies 155 The Computer 156 The Internet 156 Learning Objects and Standards 157 State of the Art and Outlook 158 Technology Enhanced Learning Environments 159 Computer-Based Training 159 Intelligent Tutoring Systems 160 Dynamic Visual Representations and Animations 162 Hypertext and Hypermedia 163 Computer Simulations and Virtual Reality 164 State of the Art and Outlook 165 The Emerging Need for a Paradigm Shift 167 References 170 7 Designing Training for Professionals Based on Subject Matter Experts and Cognitive Task Analysis 175 The Advantages of Task Analysis 176 The Disadvantages of Task Analysis 177 Task Analysis for Instructional Systems Development: A Consensual View 179 Cognitive Task Analysis and the Use of Subject-Matter Experts 180 Training Structured Troubleshooting: A Case Study 183 Structured Troubleshooting as a One-Week Add-On 186 Modifying Existing Courses Based on Structured-Troubleshooting Principles 191 Conclusions 192 References 194 8 How to Help Professionals Maintain and Improve Their Knowledge and Skills: Triangulating Best Practices in Medicine 198 The Gap in Clinical Care: From Best Evidence to Best Practice 200 Causes of the Gap Between Current and Best Practices 200 The Iterative Goal of Approaching “Best Medical Care” 202 Self-Assessment 202 Competency Assessment 203 Synthesis: Toward a Broader Model of Best Practice 204 Background and History of CME: The Currency of Credit 205 The Use of CME Credits by State Medical Boards, Specialty Boards, and Others 205 Premise of CME Credit 206 The Effect of CME Credits: Indirect Effects, Indirect Benefits 207 Pressures for Change 209 Toward a New Best Practice Model for the Professions 210 Some Thoughts About the Future 211 New Competencies 212 New Technologies 212 Methods of Feedback 213 A Picture of the Future 213 References 215 9 Advances in Specifying What Is to Be Learned: Reflections on the Themes in Chapters 6–8 221 Introduction 221 Advances in Instructional Design 221 Applying the Science of Instruction 221 Phase 1: Specifying Compartmentalized Behaviors 223 Phase 2: Specifying Compartmentalized Knowledge 224 Phase 3: Specifying Integrated Knowledge 224 Phase 4: Specifying Individualized Knowledge 226 Where Do We Go From Here? 226 References 228 Section 3 THE ASSESSMENT AND TRAINING OF SKILLED AND EXPERT PERFORMERS IN THE MILITARY 231 10 Toward a Second Training Revolution: Promise and Pitfalls of Digital Experiential Learning 233 The First Training Revolution Does Not Scale 233 Recent Changes at the National Training Center 234 Observations on Individual Training 235 What You Don’t Measure Can Hurt You 236 Kirkpatrick’s Assessment Categories 236 The Proximity Effect 236 Toward a Second Training Revolution 237 Simulation to the Rescue? 237 Can Computer Games Help? 238 Common Assumptions About Games for Training 238 Bait: Some DARWARS Training Projects and a Transfer Study 239 Tactical Language and Culture Trainer 239 Game from the Start: DARWARS Ambush! 243 Why a Game, and Why That Specific Game 247 Distribution by Diffusion 247 Why DARWARS Ambush! Was a Success 248 A Lesson or Two about Trainer-Less Training 250 DARWARS Ambush! Is an Environment, Not a Trainer 250 Th ere Is No Golden Disk 251 Information Technology Troubleshooting Digital Tutor: Immersive Experiential Training Without a Game 252 Gorman’ s Gambit 254 Does Training Transfer From Lightweight PC Simulations Into Real-World Competence? 255 Ending With a Whimper, Not a Bang 259 You Can’t Do This Cheaply 259 What Is Missing? 259 Where We Might Go from Here: People Engines in a Simulation of a Mindscape 260 Final Words and the Promised Whimper 262 References 263 11 Evaluating Pilot Performance 265 A Brief Historical Overview of Flight Simulation 265 Distributed Mission Operations 268 Distributed Simulation Development: Challenges for Assessment 268 The DMO Community’s Assumptions 271 Prior Human Performance DMO Data 272 Limitations on Relying on Subjective Assessment in Complex Environments 273 Metric Generation and Validation 275 Conceptual Application/Transition of Measures for Adaptive Training 281 Additional Air Force Performance Measurement Needs and Uses 282 Performance Measurement for Pilot Selection 282 Performance Measurement for Cockpit Design and Testing 283 Performance Measurement for Cost Estimation for Trainee in a Pilot Training Course 284 Conclusion 284 Acronyms 285 References 285 12 Contrasting Submarine Specialty Training: Sonar and Fire Control 289 The Submarine World 290 Task Descriptions: ST and FT Expertise 291 Sonar Technicians 291 Fire Control Technicians 292 The Problem with Military Training 296 Training and Performance Measurement: Old Model 297 Training and Performance Measurement: Training Continuum Model 299 Conclusions 302 References 303 13 Training Complex Cognitive Skills: A Theme-Based Approach to the Development of Battlefield Skills 304 Adaptive Performance and Adaptive Thinking 305 Training Adaptive Performance and Mental Models 306 Training Thinking Behaviors 308 The Expert Themes of Battlefield Thinking 309 Deliberate Practice 312 TLAC Training 314 Evaluation of the TLAC Training 318 Applying the Theme-Based Training Method to Crisis Response Training for Military and Civilian Organizations 322 Conclusion 324 References 325 14 Structuring the Conditions of Training to Achieve Elite Performance: Reflections on Elite Training Programs and Related Themes in Chapters 10–13 330 Basic Considerations in Structuring Training to Achieve Elite Performance 330 The Need to Distinguish Between Learning and Performance 331 Learning Without Performance 331 Performance Without Learning 332 The Potential for Instructors and Learners to Be Misled 332 The Potential to Misunderstand the Meaning and Role of Errors 333 Top Gun, the National Training Center (NTC), and Similar Programs as Counterexamples 334 After-Action Reviews 334 The Prevalence of Forgetting and Its Role in Relearning 335 The Importance of Overlearning and Relearning 336 Forgetting as a Necessary Condition for Optimal Relearning 336 Implications for Sustaining Elite Performance 337 Specific Comments on the Preceding Chapters 337 Quantifying Warfighter Performance in Distributed Simulation Environments 337 Introducing Desirable Difficulties 338 Engineering Fidelity Versus Psychological Fidelity 339 Learning Versus Relearning 339 Contrasting Submarine Specialty Training: Sonar and Fire Control 339 The Old Training Model Versus the New Training Model 340 Learning Versus Performance 340 Career-Long Learning 341 Training Complex Cognitive Skills: A Th eme-Based Approach 341 The Generality of Training Principles 342 The Interaction of Experience and Training 342 Deliberate Practice and Learning Versus Performance Revisited 343 Toward a Second Training Revolution: Promise and Pitfalls of Digital Experiential Training 344 Games as Tools for Training 344 Performance Gains Versus Learning Gains 345 User Authoring, Generation Effects, and Cooperative Learning 345 Conclusion 345 References 346 Section 4 THE DEVELOPMENT OF EXPERTISE AND EXPERT PERFORMANCE 349 15 The Influence of Learning Research on the Design and Use of Assessment 351 Review of Assessment Parameters 352 Challenges to the Integration of Learning and Measurement 356 A Brief History of Testing: So How Did We Get Here? 356 The Conflict Between Measuring Stable Constructs and Learning 356 Learning from Procedures 357 Assessment Purposes (or Clarifying Functions of Results) 358 A Status Check on Assessment Design 360 Cognitive Demands and Domain Description 361 Content Representation 363 Examples of Assessment Design Methodology 365 Developed Ontologies in Other Domains 366 Criteria Needed for Excellent Performance: More Validity Built In 368 Validation and Measures of Technical Quality 369 Conclusion 370 References 371 16 Acquiring Conceptual Expertise from Modeling: The Case of Elementary Physics 374 What Knowledge Comprises Conceptual Expertise in Physics? 378 What Happened to the Novices’ Misconceptions? 383 How Does Expertise Affect Quantitative Problem Solving? 384 How Can Conceptual Expertise Be Learned? 385 Learning Description Phase Knowledge 385 Learning Applicability Conditions 386 Learning Confluences 387 Conclusion 391 References 393 17 Teaching for Expertise: Problem-Based Methods in Medicine and Other Professional Domains 397 Traditional Professional Academic Education 399 Horizontal and Vertical Integration 400 Curriculum Overload 400 Accountability and Risk Avoidance 401 Task and Role of the Professional 401 Implications 402 Problem-Based Learning as an Instructional Method and as a Curriculum Strategy 403 Problems 403 Courses and Curricula 407 Different Goals and Different Implementations of Problem-Based Learning 408 PBL and Expertise Research 410 Medical Expertise Development Research 410 Does It Work? 413 Do Students Learn Better? 413 Have Students Learned Better? 416 Do Graduates Have Better Fitting Jobs? 417 Do Problem-Based Learning Graduates Do a Better Job? 417 Conclusion 418 References 419 18 Enhancing the Development of Professional Performance: Implications from the Study of Deliberate Practice 423 The Traditional View of Skill Acquisition and Professional Development: History and Some Recent Criticisms 424 Capturing Reproducibly Superior Performance Under Standardized Conditions 426 The Necessity of Domain-Specific Experience for Attaining Reproducibly Superior Performance 429 From Experience to Designed Practice 430 Applying the Framework to Professional Training and Practice 439 Conclusion 443 References 445 19 It Takes Expertise to Make Expertise: Some Thoughts About Why and How and Reflections on the Themes in Chapters 15–18 450 Two Kinds of Expertise 451 Learning Expertise 451 Teaching Expertise 452 Feedback for Bi-directional Learning 454 Problems with Low-Quality Feedback Cycles 455 Research with Clinicians 456 Research with Language Therapists 456 Problem-Based Learning in Medicine 457 Work in Education 457 Standards of Expert Performance 459 Knowledge of Performance Conditions 459 Stable and Variable Performance Conditions 460 Conclusion 463 References 464 20 The Value of Expertise and Expert Performance: A Review of Evidence from the Military 467 Themes 467 Self-Assessment and Self-Directed Learning 467 Deliberate Practice in Developing Expertise 468 Agility in Expertise and Professional Performance 468 Assessment of Professional Growth Toward Expertise 468 Centrality of Cognition in Expertise 469 Design of Learning Environments to Promote Expertise 469 Military Value 470 Human Ability and Military Task Performance 471 Expertise and Military Task Performance 473 Success in Military Operations 477 Top Gun 478 73 Easting 480 Chapter Themes Revisited 481 Deliberate Practice 481 Assessment and Self-Assessment 482 Agility 483 Centrality of Cognition 483 Design of Learning Environments 483 Military Value of Training 484 Conclusion 484 References 485 21 Expertise in the Management of People: A New Frontier for Research on Expert Performance 488 The Tradition of Management Training 489 Case-Based Instruction for Management and Leadership 490 How Effective Is Case-Based Instruction? 491 Expertise in Case-Based Discussion Teaching 493 Expertise in College and Graduate Level Teaching, Generally 495 Toward a Research Agenda for the Future 496 Teachers’ Mental Models of Students 496 Characterizing Skill in Discussion Teaching 498 Evaluating the Effectiveness of MBA Training 500 The Role of Mental Models in the Management of People 500 The Myers-Briggs Cult: A System for Mental Modeling of Employees 502 How Adequate Are Myers-Briggs Models of People? 504 Studying Management and Leadership Behavior 505 Reprise 507 References 509 Name Index 513 Subject Index 530 Cover......Page 1 Half-title......Page 3 Title......Page 5 Copyright......Page 6 Contents......Page 7 Figures......Page 10 Tables......Page 13 Contributors......Page 15 1 The Measurement and Development of Professional Performance: An Introduction to the Topic and a Background to the Design and Origin of This Book......Page 19 A Brief History of Research on Objective Measurement of Professional Performance......Page 22 The Expert Performance Approach to the Study of Superior Performance......Page 25 The Process of Generating This Book......Page 27 The Outline of This Book......Page 28 References......Page 38 Section 1 CHALLENGES IN PAST AND CONTEMPORARY EFFORTS TO MEASURE AND TRAIN THE OBJECTIVE PERFORMANCE OF PROFESSIONALS......Page 43 2 The 20th-Century Revolution in Military Training......Page 45 Training Counts......Page 46 A Revolution in Training......Page 47 A Story to Set Context......Page 48 Connecting Training to Military Proficiency......Page 50 Counting Systems......Page 51 The Learning Curve and Skills Training......Page 53 Skill Atrophy: The Unlearning Curve......Page 57 Warfare and Mission-Level Profi ciency......Page 59 You Fight Like You Train: A Personal Example......Page 60 A Hierarchy of Learning Curves......Page 61 The Measurement Problem......Page 64 Measurable War: Air-to-Air Combat and the Existence of Aces......Page 65 Engagement Simulation and Training......Page 67 Summary of Engagement Simulation......Page 70 Training Surprise: Other Countries Could Use It, Too......Page 71 The Interdeployment Readiness Bathtub Curve......Page 73 Training Counts......Page 74 Resolving the Contradiction: A Hierarchy of Training......Page 75 Admonition to the Defense Department......Page 76 References......Page 77 3 Developing Professional Expertise with a Cognitive Apprenticeship Model: Examples from Avionics and Medicine......Page 79 Assessing Professional Competence......Page 80 Cognitive Apprenticeship as an Instructional Model......Page 82 Avionics Troubleshooting......Page 83 Background......Page 84 The Solution......Page 85 Evidence of Effectiveness......Page 89 Reasons for Success......Page 90 Extending Sherlock’s Success to New Domains: From Avionics Troubleshooting to Medical Problem Solving......Page 92 Using Cognitive Models in the Design of Computer-Based Learning Environments......Page 93 Lessons Learned and Future Directions......Page 96 References......Page 98 4 Leadership Development and Assessment: Describing and Rethinking the State of the Art......Page 102 Leader Behavior......Page 103 Measurement......Page 104 Knowledge Structures......Page 107 Processes......Page 108 Support for the Model......Page 111 Leader Assessment......Page 112 Processes......Page 113 Leader Development......Page 114 Knowledge......Page 115 Processes......Page 117 Conclusions......Page 118 References......Page 120 5 Revolutions, Leaders, and Diagnosticians : Reflections on the Themes in Chapters 2–4......Page 126 Chatham’s “First Revolution” in Military Training: (Hopefully) Some Constructive Comments......Page 127 Sherlock and Its Heirs......Page 136 Is Leadership a Natural Kind?......Page 138 Summary......Page 144 References......Page 145 Section 2 PAST AND CONTEMPORARY EFFORTS TO DESIGN INSTRUCTION, TRAIN, AND MAINTAIN PROFESSIONAL PERFORMANCE......Page 147 The Changing Goals of Professional Training......Page 149 Instructional Systems Design......Page 151 Social-Constructivist Design......Page 152 Whole-Task Design Models......Page 154 Learning Technologies......Page 155 The Internet......Page 156 Learning Objects and Standards......Page 157 State of the Art and Outlook......Page 158 Computer-Based Training......Page 159 Intelligent Tutoring Systems......Page 160 Dynamic Visual Representations and Animations......Page 162 Hypertext and Hypermedia......Page 163 Computer Simulations and Virtual Reality......Page 164 State of the Art and Outlook......Page 165 The Emerging Need for a Paradigm Shift......Page 167 References......Page 170 7 Designing Training for Professionals Based on Subject Matter Experts and Cognitive Task Analysis......Page 175 The Advantages of Task Analysis......Page 176 The Disadvantages of Task Analysis......Page 177 Task Analysis for Instructional Systems Development: A Consensual View......Page 179 Cognitive Task Analysis and the Use of Subject-Matter Experts......Page 180 Training Structured Troubleshooting: A Case Study......Page 183 Structured Troubleshooting as a One-Week Add-On......Page 186 Modifying Existing Courses Based on Structured-Troubleshooting Principles......Page 191 Conclusions......Page 192 References......Page 194 8 How to Help Professionals Maintain and Improve Their Knowledge and Skills: Triangulating Best Practices in Medicine......Page 198 Causes of the Gap Between Current and Best Practices......Page 200 Self-Assessment......Page 202 Competency Assessment......Page 203 Synthesis: Toward a Broader Model of Best Practice......Page 204 The Use of CME Credits by State Medical Boards, Specialty Boards, and Others......Page 205 Premise of CME Credit......Page 206 The Effect of CME Credits: Indirect Effects, Indirect Benefits......Page 207 Pressures for Change......Page 209 Toward a New Best Practice Model for the Professions......Page 210 Some Thoughts About the Future......Page 211 New Technologies......Page 212 A Picture of the Future......Page 213 References......Page 215 Applying the Science of Instruction......Page 221 Phase 1: Specifying Compartmentalized Behaviors......Page 223 Phase 3: Specifying Integrated Knowledge......Page 224 Where Do We Go From Here?......Page 226 References......Page 228 Section 3 THE ASSESSMENT AND TRAINING OF SKILLED AND EXPERT PERFORMERS IN THE MILITARY......Page 231 The First Training Revolution Does Not Scale......Page 233 Recent Changes at the National Training Center......Page 234 Observations on Individual Training......Page 235 The Proximity Effect......Page 236 Simulation to the Rescue?......Page 237 Common Assumptions About Games for Training......Page 238 Tactical Language and Culture Trainer......Page 239 Game from the Start: DARWARS Ambush!......Page 243 Distribution by Diffusion......Page 247 Why DARWARS Ambush! Was a Success......Page 248 DARWARS Ambush! Is an Environment, Not a Trainer......Page 250 Th ere Is No Golden Disk......Page 251 Information Technology Troubleshooting Digital Tutor: Immersive Experiential Training Without a Game......Page 252 Gorman’ s Gambit......Page 254 Does Training Transfer From Lightweight PC Simulations Into Real-World Competence?......Page 255 What Is Missing?......Page 259 Where We Might Go from Here: People Engines in a Simulation of a Mindscape......Page 260 Final Words and the Promised Whimper......Page 262 References......Page 263 A Brief Historical Overview of Flight Simulation......Page 265 Distributed Simulation Development: Challenges for Assessment......Page 268 The DMO Community’s Assumptions......Page 271 Prior Human Performance DMO Data......Page 272 Limitations on Relying on Subjective Assessment in Complex Environments......Page 273 Metric Generation and Validation......Page 275 Conceptual Application/Transition of Measures for Adaptive Training......Page 281 Performance Measurement for Pilot Selection......Page 282 Performance Measurement for Cockpit Design and Testing......Page 283 Conclusion......Page 284 References......Page 285 12 Contrasting Submarine Specialty Training: Sonar and Fire Control......Page 289 The Submarine World......Page 290 Sonar Technicians......Page 291 Fire Control Technicians......Page 292 The Problem with Military Training......Page 296 Training and Performance Measurement: Old Model......Page 297 Training and Performance Measurement: Training Continuum Model......Page 299 Conclusions......Page 302 References......Page 303 13 Training Complex Cognitive Skills: A Theme-Based Approach to the Development of Battlefield Skills......Page 304 Adaptive Performance and Adaptive Thinking......Page 305 Training Adaptive Performance and Mental Models......Page 306 Training Thinking Behaviors......Page 308 The Expert Themes of Battlefield Thinking......Page 309 Deliberate Practice......Page 312 TLAC Training......Page 314 Evaluation of the TLAC Training......Page 318 Applying the Theme-Based Training Method to Crisis Response Training for Military and Civilian Organizations......Page 322 Conclusion......Page 324 References......Page 325 Basic Considerations in Structuring Training to Achieve Elite Performance......Page 330 Learning Without Performance......Page 331 The Potential for Instructors and Learners to Be Misled......Page 332 The Potential to Misunderstand the Meaning and Role of Errors......Page 333 After-Action Reviews......Page 334 The Prevalence of Forgetting and Its Role in Relearning......Page 335 Forgetting as a Necessary Condition for Optimal Relearning......Page 336 Quantifying Warfighter Performance in Distributed Simulation Environments......Page 337 Introducing Desirable Difficulties......Page 338 Contrasting Submarine Specialty Training: Sonar and Fire Control......Page 339 Learning Versus Performance......Page 340 Training Complex Cognitive Skills: A Th eme-Based Approach......Page 341 The Interaction of Experience and Training......Page 342 Deliberate Practice and Learning Versus Performance Revisited......Page 343 Games as Tools for Training......Page 344 Conclusion......Page 345 References......Page 346 Section 4 THE DEVELOPMENT OF EXPERTISE AND EXPERT PERFORMANCE......Page 349 15 The Influence of Learning Research on the Design and Use of Assessment......Page 351 Review of Assessment Parameters......Page 352 The Conflict Between Measuring Stable Constructs and Learning......Page 356 Learning from Procedures......Page 357 Assessment Purposes (or Clarifying Functions of Results)......Page 358 A Status Check on Assessment Design......Page 360 Cognitive Demands and Domain Description......Page 361 Content Representation......Page 363 Examples of Assessment Design Methodology......Page 365 Developed Ontologies in Other Domains......Page 366 Criteria Needed for Excellent Performance: More Validity Built In......Page 368 Validation and Measures of Technical Quality......Page 369 Conclusion......Page 370 References......Page 371 16 Acquiring Conceptual Expertise from Modeling: The Case of Elementary Physics......Page 374 What Knowledge Comprises Conceptual Expertise in Physics?......Page 378 What Happened to the Novices’ Misconceptions?......Page 383 How Does Expertise Affect Quantitative Problem Solving?......Page 384 Learning Description Phase Knowledge......Page 385 Learning Applicability Conditions......Page 386 Learning Confluences......Page 387 Conclusion......Page 391 References......Page 393 17 Teaching for Expertise: Problem-Based Methods in Medicine and Other Professional Domains......Page 397 Traditional Professional Academic Education......Page 399 Curriculum Overload......Page 400 Task and Role of the Professional......Page 401 Implications......Page 402 Problems......Page 403 Courses and Curricula......Page 407 Different Goals and Different Implementations of Problem-Based Learning......Page 408 Medical Expertise Development Research......Page 410 Do Students Learn Better?......Page 413 Have Students Learned Better?......Page 416 Do Problem-Based Learning Graduates Do a Better Job?......Page 417 Conclusion......Page 418 References......Page 419 18 Enhancing the Development of Professional Performance: Implications from the Study of Deliberate Practice......Page 423 The Traditional View of Skill Acquisition and Professional Development: History and Some Recent Criticisms......Page 424 Capturing Reproducibly Superior Performance Under Standardized Conditions......Page 426 The Necessity of Domain-Specific Experience for Attaining Reproducibly Superior Performance......Page 429 From Experience to Designed Practice......Page 430 Applying the Framework to Professional Training and Practice......Page 439 Conclusion......Page 443 References......Page 445 19 It Takes Expertise to Make Expertise: Some Thoughts About Why and How and Reflections on the Themes in Chapters 15–18......Page 450 Learning Expertise......Page 451 Teaching Expertise......Page 452 Feedback for Bi-directional Learning......Page 454 Problems with Low-Quality Feedback Cycles......Page 455 Research with Language Therapists......Page 456 Work in Education......Page 457 Knowledge of Performance Conditions......Page 459 Stable and Variable Performance Conditions......Page 460 Conclusion......Page 463 References......Page 464 Self-Assessment and Self-Directed Learning......Page 467 Assessment of Professional Growth Toward Expertise......Page 468 Design of Learning Environments to Promote Expertise......Page 469 Military Value......Page 470 Human Ability and Military Task Performance......Page 471 Expertise and Military Task Performance......Page 473 Success in Military Operations......Page 477 Top Gun......Page 478 73 Easting......Page 480 Deliberate Practice......Page 481 Assessment and Self-Assessment......Page 482 Design of Learning Environments......Page 483 Conclusion......Page 484 References......Page 485 21 Expertise in the Management of People: A New Frontier for Research on Expert Performance......Page 488 The Tradition of Management Training......Page 489 Case-Based Instruction for Management and Leadership......Page 490 How Effective Is Case-Based Instruction?......Page 491 Expertise in Case-Based Discussion Teaching......Page 493 Expertise in College and Graduate Level Teaching, Generally......Page 495 Teachers’ Mental Models of Students......Page 496 Characterizing Skill in Discussion Teaching......Page 498 The Role of Mental Models in the Management of People......Page 500 The Myers-Briggs Cult: A System for Mental Modeling of Employees......Page 502 How Adequate Are Myers-Briggs Models of People?......Page 504 Studying Management and Leadership Behavior......Page 505 Reprise......Page 507 References......Page 509 Name Index......Page 513 Subject Index......Page 530 Professionals Such As Medical Doctors, Aeroplane Pilots, Lawyers, And Technical Specialists Find That Some Of Their Peers Have Reached High Levels Of Achievement That Are Difficult To Measure Objectively. In Order To Understand To What Extent It Is Possible To Learn From These Expert Performers For The Purpose Of Helping Others Improve Their Performance, We First Need To Reproduce And Measure This Performance. This Book Is Designed To Provide The First Comprehensive Overview Of Research On The Acquisition And Training Of Professional Performance As Measured By Objective Methods Rather Than By Subjective Ratings By Supervisors. In This Collection Of Articles, The World's Foremost Experts Discuss Methods For Assessing The Experts' Knowledge And Review Our Knowledge On How We Can Measure Professional Performance And Design Training Environments That Permit Beginning And Experienced Professionals To Develop And Maintain Their High Levels Of Performance, Using Examples From A Wide Range Of Professional Domains. List Of Figures -- List Of Tables -- List Of Contributors -- 1. Challenges In Past And Contemporary Efforts To Measure And Train The Objective Performance Of Professionals -- 2. Past And Contemporary Efforts To Design Instruction, Train, And Maintain Professional Performance -- 3. The Assessment And Training Of Skilled And Expert Performers In The Military -- 4. The Development Of Expertise And Expert Performance -- Name Index -- Subject Index Edited By K. Anders Ericsson. Includes Bibliographical References And Indexes. Professionals such as medical doctors, airplane pilots, lawyers, and technical specialists find that some of their peers have reached high levels of achievement that are difficult to measure objectively. In order to understand to what extent it is possible to learn from these expert performers for the purpose of helping others improve their performance, we first need to reproduce and measure this performance. This book is designed to provide the first comprehensive overview of research on the acquisition and training of professional performance as measured by objective methods rather than by subjective ratings by supervisors. In this collection of articles, the worlds foremost experts discuss methods for assessing the experts knowledge and review our knowledge on how we can measure professional performance and design training environments that permit beginning and experienced professionals to develop and maintain their high levels of performance, using examples from a wide range of professional domains.