تحقیق تجربی کاربر کمی: اطلاع‌رسانی به تصمیمات محصول با درک کاربران در مقیاس بزرگ

This book is your definitive guide to the rapidly growing role of Quantitative User Experience (Quant UX) Research in product development. The book provides an overview of the skills you need on the job, presents hands-on projects with reusable code, and shares advice on starting and developing a career. The book goes beyond basic skills to focus on what is unique to Quant UX. The authors are two of the most widely recognized practitioners in Quant UX research, and this book shares insights from their combined decades of experience. Organizations today have more data about user needs and behaviors than ever before. With this large-scale data, Quant UX researchers work to understand usage patterns, measure the impact of design changes, and inform strategic decisions. In the Quant UX role, interdisciplinary researchers apply analytical skills to uncover user needs, inform engineering and design, answer strategic business questions, and optimize software and hardware products for human interaction. This book provides guidance around customer satisfaction surveys, understanding user behavior from log analysis, and the statistical methods that are commonly used to assess user outcomes. What You Will Learn Discover the role of Quantitative User Experience (Quant UX) research Understand how Quant UX research differs from other disciplines such as data science Plan common research projects and know how to achieve success Position Quant UX activities in product development, engineering, and UX organizations Apply the HEART framework to measure user experience outcomes Evaluate your skills and potential to be hired as a Quant UX researcher Know what to expect during job interviews Find examples of common Quant UX projects with shared R code and data sets Who This Book Is For Practitioners and managers who seek a comprehensive guide to the new field of Quantitative User Experience Research. Readers will understand the Quant UX role, build research skills, find examples of hands-on code and analyses, learn about UX organizations and stakeholders, and receive advice on job interviews and career paths. Data scientists, social scientists, and other researchers will learn how their skills transfer to Quant UX, where they can help teams build better, more successful products. Table of Contents About the Authors About the Technical Reviewers Acknowledgments Introduction Part I: User Experience and Introduction to Part I Chapter 1: Getting Started 1.1 Who Are We? Why Should You Listen to Us? 1.2 What Is Different About This Book? 1.3 Who Is Our Audience? 1.3.1 A Quick Check on Your Interests 1.4 What Will You Learn? 1.5 How to Use This Book 1.5.1 Assumptions 1.5.2 A Note About Jargon 1.5.3 End of Chapter Exercises 1.6 Online Materials 1.6.1 Code and Data Sources 1.6.2 Help! Updates and Errata 1.7 Key Points Chapter 2: User Experience and UX Research 2.1 User Experience 2.1.1 UX Roles 2.1.2 UX Design and Software Engineering 2.1.3 Product Management 2.2 UX Research 2.2.1 Categories of UX Researchers 2.2.2 The Research Lifecycle for UXRs 2.2.3 Typical Research Projects in the Product Lifecycle 2.3 Key Points 2.4 Learning More Chapter 3: Quantitative UX Research: Overview 3.1 Quantitative UX Research 3.2 Week-to-Week Practice of Quant UX Research 3.2.1 Typical Activities in a Week 3.2.2 Common Research Questions for Quant UXRs 3.2.3 Stakeholder Questions 3.3 Varieties of Quant UXRs 3.4 Quant UXR Differences from Other Roles 3.4.1 Quant UXR vs. General UXR 3.4.2 Quant UXR vs. Mixed Methods UXR 3.4.3 Quant UXR vs. Survey Scientist 3.4.4 Quant UXR vs. Marketing Researcher 3.4.5 Quant UXR vs. Data Scientist 3.4.6 Quant UXR vs. Business or Product Analyst 3.4.7 Quant UXR vs. Research Scientist 3.4.8 Quant UXR vs. Academic Research 3.5 Will You Like a Quant UXR Role? 3.6 Key Points 3.7 Learning More Part II: Core Introduction to Part II Chapter 4: UX Research 4.1 Foundational and In-Depth Skills for Quant UXR 4.1.1 “T-Shape” Skills 4.2 Focus on the User 4.2.1 Adopt the User’s Perspective 4.2.2 Assess User-Centric Variables and Outcomes 4.2.3 Answer “Why?” with a Cognitive Approach 4.2.4 Focus on Unmet Needs 4.2.5 Relate to UX Actions and Stakeholders 4.3 Research Validity 4.4 Assessing Users and Assessing Products 4.5 Research Ethics 4.5.1 Research Risks and Benefits 4.5.2 Privacy and Legal Requirements 4.5.3 Minimum Collection 4.5.4 Scientific Standards 4.5.5 Impact on Society 4.5.6 The Newspaper Test 4.6 Research Planning 4.7 Key Points 4.8 Learning More Chapter 5: Statistics 5.1 Why Statistics? 5.1.1 Statistics vs. Machine Learning 5.2 The Foundation: Sampling and Data Quality 5.3 Core Statistical Analysis Skills 5.3.1 Exploratory Data Analysis and Visualization 5.3.2 Descriptive Statistics 5.3.3 Inferential Tests and Practical Significance 5.3.4 Fundamentals of A/B Testing 5.3.5 Linear Models 5.4 Frequently Observed Issues 5.4.1 Bad or Biased Data 5.4.2 Focusing on Discovery, Losing Sight of Decisions 5.4.3 Prematurely Assuming an Outcome of Interest 5.4.4 Interpreting Statistical Significance 5.4.5 Applying Fancy Models 5.5 Key Points 5.6 Learning More 5.7 Questions and an Exercise Chapter 6: Programming 6.1 Overview 6.1.1 Is Programming Required? 6.1.2 What Language? 6.2 Procedural Programming Basics 6.2.1 Algorithms 6.2.1.1 Logical Steps 6.2.1.2 Control Structures 6.2.1.3 Functions 6.2.2 Data Structures 6.2.2.1 Vectors 6.2.2.2 Quick Check: What Do You Think? 6.2.2.3 Arrays and Data Frames 6.2.2.4 Hash Tables 6.3 SQL 6.4 Other Coding Topics 6.4.1 Reproducibility of Code 6.4.2 Performance and Scale 6.5 Key Points 6.6 Learning More 6.7 Exercises Part III: Tools and Introduction to Part III Chapter 7: Metrics of User Experience 7.1 The HEART Framework 7.1.1 Happiness 7.1.2 Engagement 7.1.3 Adoption 7.1.4 Retention 7.1.5 Task Success 7.2 The Goals-Signals-Metrics Process 7.2.1 Goals 7.2.2 Signals 7.2.3 Metrics 7.3 Applying the Methods Together 7.4 Example: Redesigning Labels in Gmail 7.5 Lessons Learned From Experience 7.5.1 Individual Pitfalls 7.5.1.1 Not Enough Team Involvement 7.5.1.2 Starting Too Big 7.5.1.3 Underestimating the Next Steps 7.5.1.4 Too Many Metrics 7.5.2 Organizational Issues 7.5.2.1 Unwillingness to be Evaluated 7.5.2.2 Optimizing for a Single Metric 7.5.2.3 Failure to Consider Ethical Consequences 7.6 Key Points 7.7 Learning More 7.8 Exercises Chapter 8: Customer Satisfaction Surveys 8.1 Goals of a Customer Satisfaction Program 8.2 The Components of Listening to Customers 8.2.1 Customer Population and Sample 8.2.2 Survey Mechanism 8.2.3 Ordinal Ratings 8.2.3.1 Top 2 Box and Proportional Scores 8.2.3.2 What About Net Promoter Scores? 8.2.4 Open-Ended Comments 8.2.5 Demographic and Behavioral Information 8.2.6 Don’t Compare Groups, Compare Over Time 8.2.7 Follow-up with Stakeholders and Customers 8.2.7.1 What to Report 8.2.7.2 Closing the Loop 8.3 Common Problems in CSat Analysis 8.4 Example Analysis in R 8.4.1 Initial Data Inspection 8.4.2 CSat for One Time Period 8.4.3 CSat over Time 8.4.4 Top 2 Box Proportions 8.4.5 Is CSat Changing? Initial Analysis 8.4.6 Examination by Country 8.4.7 A Better Model of CSat Change in These Data 8.5 Key Points 8.6 Learning More 8.7 Exercises Chapter 9: Log Sequence Visualization 9.1 Example Sequence Data 9.1.1 Sunburst Chart for the Buffet Data 9.2 Sunburst Visualization of Website Data 9.2.1 Transforming the Logs to Sequences 9.2.1.1 Loading and Sessionizing the Data 9.2.1.2 Creating Sequences from the Sessions 9.2.2 Sunburst Visualization of the EPA Data 9.2.3 Next Steps in Analysis 9.3 Key Points 9.4 Learning More 9.5 Exercises Chapter 10: MaxDiff: Prioritizing Features and User Needs 10.1 Overview of MaxDiff 10.1.1 Illustration of MaxDiff Analysis 10.1.2 Calculating Pizza Demand 10.1.3 Summary of MaxDiff Advantages 10.2 Detailed Introduction to MaxDiff Estimation 10.2.1 Common UX Topics for MaxDiff Surveys 10.2.2 Writing and Fielding a MaxDiff Survey 10.2.2.1 Writing the Question and Column Headers 10.2.2.2 Developing an Item List Simple Items Length Commonality of Items Developing the Item List Check the Trade-offs Maximum Number of Items 10.2.2.3 Prohibitions: Items That Can’t Appear Together 10.2.2.4 Number of Tasks Survey Length for Average, Sample-Level Results Survey Length for Individual-Level Results 10.2.2.5 Sample Size 10.2.2.6 Survey Fielding Methods 10.2.2.7 Mixed Qualitative-Quantitative Group Interview 10.2.3 Survey Authoring Platforms 10.2.3.1 MaxDiff in Qualtrics 10.2.4 MaxDiff and Accessibility 10.2.5 MaxDiff Statistical Models 10.2.5.1 Counts and Difference Scores 10.2.5.2 Multinomial Logit Model 10.2.5.3 Hierarchical Bayes Model 10.2.5.4 Using and Reporting the Scores 10.3 Example: Information Seeking Use Cases 10.3.1 Overview: MaxDiff for Information Seeking 10.3.2 Survey Format 10.3.3 Data Format 10.3.3.1 Data Sets in Other Formats 10.3.4 Estimation with the choicetools Package 10.3.4.1 Setup for Estimation 10.3.4.2 Check the Data 10.3.4.3 Load the Data 10.3.4.4 Estimate the Model 10.3.4.5 Plot the Results More on HB Iterations 10.3.5 Next Steps 10.4 Key Points 10.5 Learning More 10.6 Exercises Part IV: Organizations and Introduction to Part IV Chapter 11: UX Organizations 11.1 Typical UX Organization Models 11.1.1 Role-Centric Organization 11.1.1.1 Quant UXRs in Role-Centric Organizations 11.1.1.2 Notes on Success with Role-Centric Organizations 11.1.2 Product-Centric Organization 11.1.2.1 Quant UXRs in Product-Centric Organizations 11.1.2.2 Notes on Success with Product-Centric Organizations 11.2 Other Organizational Models for Quant UXRs 11.2.1 Centralized Quant UX Research Teams 11.2.1.1 Recommendations for Centralized Quant UX Research Teams Managing a Centralized Quant Team Recommendations to Individual Quant UXRs 11.2.2 Quant UX in a Data Science or Analytics Team 11.2.2.1 Recommendations for Quant UX in a Data Science or Analytics Team 11.3 Advice for Managers of Quant UXRs 11.3.1 Access to Stakeholders and Data 11.3.2 Shield from Immediate Requests 11.3.3 Growth Opportunity 11.3.4 Help with Determining Impact 11.3.5 Stay Out of the Way 11.4 Key Points 11.5 Learning More Chapter 12: Interviews and Job Postings 12.1 General Quant UXR Interview Process 12.2 Two Formats for Interview Panels 12.2.1 Format 1: Interview Loops 12.2.2 Format 2: Hands-On Interviews 12.2.3 What Happens Among Interviewers? 12.2.4 Who Makes the Hiring Decision? 12.3 Before, During, and After an Onsite Interview 12.3.1 Before: What Happens at the Company 12.3.1.1 Panel Membership 12.3.2 Before: Your Preparation 12.3.2.1 Research Presentations 12.3.2.2 Requests for Analyses in Advance 12.3.2.3 Do and Don’t 12.3.3 During Interviews 12.3.3.1 A General Approach to Questions 12.3.3.2 Bring a List of Questions 12.3.4 Afterward 12.3.4.1 Thank You 12.3.4.2 Fit Calls 12.3.4.3 What You Can Negotiate 12.3.4.4 Red Flags 12.3.4.5 When the Answer is “No” 12.4 Job Postings and Applications 12.4.1 Finding Jobs 12.4.2 Additional Suggestions for Applications 12.4.2.1 Informational Interviews 12.4.2.2 Referrals and References 12.4.2.3 Cover Letter 12.4.2.4 CV vs. Résumé 12.4.2.5 Personal Websites and Open Source Projects 12.5 Key Points 12.6 Learning More Chapter 13: Research Processes, Reporting, and Stakeholders 13.1 Initial Engagement 13.1.1 What Stakeholders Want...and What They Need 13.1.2 Focus on Decisions 13.1.3 Work Backward 13.2 Delivering Results 13.2.1 Stakeholders Are the Users of Your Research 13.2.2 Two Models: Presentations and Documents 13.2.2.1 Presentation Slide Decks Advantages of Slides Disadvantages of Slides 13.2.2.2 Research Report Documents Advantages of Documents Disadvantages of Documents 13.2.2.3 Recommendation for Reporting 13.3 Principles of Good Deliverables 13.3.1 Short and Focused on Action 13.3.2 Minimally Technical Reports 13.3.3 Remain Unbiased 13.3.4 Reproducible and Generalizable 13.4 Research Archives 13.5 Common Problems with Stakeholders 13.5.1 Lack of a Decision Criterion 13.5.2 Ad Hoc Projects 13.5.3 Opportunity Cost 13.5.4 Validation Research 13.5.5 Statistical Significance 13.5.6 Cherry Picking Results 13.5.7 Conflicting Results 13.5.8 Challenge Only If Negative (COIN) 13.6 Finding a Great Stakeholder 13.7 Key Points 13.8 Learning More Chapter 14: Career Development for Quant UX Researchers 14.1 Elements of Career Paths in Industry 14.1.1 Job Levels 14.1.1.1 Levels, Responsibility, and Expertise 14.1.1.2 Levels and Compensation 14.1.2 Career Ladder 14.1.3 Tracks: Individual Contributor and Manager 14.1.4 Distribution of Levels 14.1.5 The Choice of IC vs. Manager 14.2 The Problems with Levels 14.3 Performance Reviews and Promotion 14.3.1 Performance Reviews 14.3.2 Impact 14.3.3 Promotion 14.4 Personal Styles and Goals 14.4.1 Maximizing vs. Satisficing 14.4.2 Builder vs. Explorer 14.5 Building Skills Throughout a Career 14.5.1 Areas for Skills Development 14.5.1.1 Quant or UX? 14.5.1.2 Qualitative Research Skills 14.5.1.3 General UX Research...and UX Generally 14.5.1.4 Programming 14.5.1.5 Statistics 14.5.2 Find Mentors 14.6 Paths for Senior ICs 14.6.1 Staff Level Pattern 1: Tech Lead 14.6.2 Staff Level Pattern 2: Evangelist 14.6.3 Staff Level Pattern 3: Strategic Partner 14.7 Key Points 14.8 Learning More Chapter 15: Future Directions for Quant UX 15.1 Future 1: UX Data Science 15.2 Future 2: Computational Social Science 15.3 Future 3: Mixed Methods UX 15.4 Future 4: Quant UX Evolution 15.5 Learning More 15.6 Finally Appendix A: Example Quant UX Job Description Quantitative User Experience Researcher Appendix B: Example Quant UX Hiring Rubrics Rubrics to Assess Quant UXR Candidates Appendix C: References Index