تسلط بر رابط‌های صوتی: ایجاد برنامه‌های صوتی عالی برای کاربران واقعی

Build great voice apps of any complexity for any domain by learning both the how's and why's of voice development. In this book you'll see how we live in a golden age of voice technology and how advances in automatic speech recognition (ASR), natural language processing (NLP), and related technologies allow people to talk to machines and get reasonable responses. Today, anyone with computer access can build a working voice app. That democratization of the technology is great. But, while it's fairly easy to build a voice app that runs, it's still remarkably difficult to build a great one, one that users trust, that understands their natural ways of speaking and fulfills their needs, and that makes them want to return for more. We start with an overview of how humans and machines produce and process conversational speech, explaining how they differ from each other and from other modalities. This is the background you need to understand the consequences of each design and implementation choice as we dive into the core principles of voice interface design. We walk you through many design and development techniques, including ones that some view as advanced, but that you can implement today. We use the Google development platform and Python, but our goal is to explain the reasons behind each technique such that you can take what you learn and implement it on any platform. Readers of Mastering Voice Interfaces will come away with a solid understanding of what makes voice interfaces special, learn the core voice design principles for building great voice apps, and how to actually implement those principles to create robust apps. We've learned during many years in the voice industry that the most successful solutions are created by those who understand both the human and the technology sides of speech, and that both sides affect design and development. Because we focus on developing task-oriented voice apps for real users in the real world, you'll learn how to take your voice apps from idea through scoping, design, development, rollout, and post-deployment performance improvements, all illustrated with examples from our own voice industry experiences. What You Will Learn Create truly great voice apps that users will love and trust See how voice differs from other input and output modalities, and why that matters Discover best practices for designing conversational voice-first applications, and the consequences of design and implementation choices Implement advanced voice designs, with real-world examples you can use immediately. Verify that your app is performing well, and what to change if it doesn't Who This Book Is For Anyone curious about the real how's and why's of voice interface design and development. In particular, it's aimed at teams of developers, designers, and product owners who need a shared understanding of how to create successful voice interfaces using today's technology. We expect readers to have had some exposure to voice apps, at least as users. Table of Contents About the Authors About the Technical Reviewers Acknowledgments Introduction Part I: Conversational Voice System Foundations Chapter 1: Say Hello to Voice Systems Voice-First, Voice-Only, and Conversational Everything Claims About Voice Technology Claim: Everyone Has a Smart Voice Device and Uses It All the Time Claim: You Can Simply “Add Voice” to an Existing GUI or Touch Interface Claim: Voice or Chatbot, Both Are Conversational So Basically the Same Claim: I Speak the Language; What More Is There to VUI Design? Claim: Every Voice Solution Needs a Strong Personality Claim: Hire a Scriptwriter; They Can Write Conversations Claim: Recognition Is a Solved Problem; It’s Basically Perfect Today Claim: AI Takes Care of Understanding What People Say Claim: IVRs Are Irrelevant Today, Nothing to Be Learned from Them Claim: That Ship Has Sailed; Alexa Is the Winner Claim: Everyone Needs a Voice Solution; Voice Is Great for Everything Introduction to Voice Technology Components Speech-to-Text In-Depth: Is STT Better Than Human Recognition and Understanding? Natural Language Understanding Dialog Management Natural Language Generation Text-to-Speech In-Depth: TTS Synthesis—Is It Like Human Speech? The House That Voice Built: The Phases of Voice Development Success Plan Design Build Test Deploy and Assess Iterate What Next? Summary Chapter 2: Keeping Voice in Mind Why Voice Is Different Hands-On: A Precoding Thought Experiment Voice Dialog and Its Participants The Human: Spoken Natural Language The Computer: Voice Recognition and Interpretation Human-Computer Voice Dialog What’s Next? Summary Chapter 3: Running a Voice App—and Noticing Issues Hands-On: Preparing the Restaurant Finder Choosing Voice Platforms Hands-On: Implementing the Restaurant Finder Basic Setup Step 1: Respond to an Invocation Step 2: Specify What the User Says Step 3: Specify What the VUI Says Step 4: Connect Dialogflow to Actions on Google Step 5: Test Your VUI Testing Using Dialogflow Testing Using the Actions console Testing with Assistant, Home, or Nest Devices Step 6: Save, Export, or Import Your Work Why We’re Using Actions on Google and Assistant Google’s Voice Development Ecosystem Actions on Google Dialogflow The Pros and Cons of Relying on Tools Hands-On: Making Changes Adding More Phrases with the Same Meaning Pushing the Limits Rejection Fixing Rejection Expanding Information Coverage Adding Granularity Preview: Static vs. Dynamic Responses What’s Next? Summary Part II: The Planning Phase: Requirements Discovery and High-Level Design Definition Chapter 4: Define Through Discovery: Building What, How, and Why for Whom General Functionality: What Are You Building? User Needs: Who’ll Use It and What Do They Want? Demographics and Characteristics Engagement Patterns Mental Models and Domain Knowledge Environment and State of Mind From General to Detailed Functionality Step 1: Define the Core Features Step 2: From Features to Intents and Slots Step 3: Identify Potential Complications Business Requirements: Why Are You Building It? General Purpose and Goal Primary and Secondary Goals Success Metrics Domain, Device, and Other Modalities Underlying Service and Existing Automation Branding and Terminology Prompt Format: Text-to-Speech or Recordings Terminology and Pronunciation Data Needs Access and Availability Privacy and Security Concerns Access Limitations Based on Time or Place Special Announcements Legal and Business Constraints Compliance Identification and Authentication Error Handling and Navigation Behaviors System Requirements: How Will You Build It? Recognizer, Parser, and Interpreter External Data Sources Data Storage and Data Access Other System Concerns What’s Next? Summary Chapter 5: From Discovery to UX and UI: Tools of Voice Design Where to Find Early User Data on Any Budget Online Research and Crowdsourcing Options Blogs and Articles Industry Studies Customer Reviews Supporting Materials Crowdsourced Data Dialog Participant Observation Actual or Potential Primary User Observation Secondary Dialog Participant Observation Expert Reviews and Heuristic Evaluation Focus Groups, Interviews, and Surveys In-Depth Interviews and Focus Groups Subject Expert Panel Targeted Surveys In-App Data Collection How Discovery Informs VUI Design Decisions Discovery Informing Design: An Example Dialog Manager Graph, Yes; Hierarchical Decision Tree, No Capturing and Documenting VUI Design Dialog Flows Sample Dialogs Detailed Design Specifications VUI Design Documentation Approaches Self-Contained Design Tools Open Platform-Specific Tools Proprietary or Closed Platform-Specific Tools Standard Documentation Tools Prototyping and Testing Your Assumptions Voice UX and Prototyping Approaches Hallway “Guerilla” Studies Low-Fidelity Mockups (Voice Version of “Wireframes”) Wizard of Oz (“WOZ”) Functional Prototypes What’s Next? Summary Part III: The Building Phase: Design-Informed Development Chapter 6: Applying Human “Rules of Dialog” to Reach Conversation Resolution Dialog Acts, Games, and Turns—and Grice In-Depth: Reaching the End of a Dialog Through Grice’s Cooperative Principle Question Answering Building Out the Question Answering Example In-Depth: Entities, Parameters, and Slots Design Notes on Question Answering Action Requests Building Action Requests Design Notes on Action Requests Task Completion Requests Building Task Completion Example 1 Building Task Completion Example 2 Design Notes on Task Completion Fully Specified Requests Single-Slot Requests Implementing Single-Slot Requests Design Notes on Single-Slot Requests Multi-slot Requests Implementing Multi-slot Requests Design Notes on Multi-slot Requests Determining Dialog Acts Based on Feature Discovery Dialog Completion Responding to “Goodbye” and “Thanks” Implementing the End of a Conversation What’s Next? Summary Chapter 7: Resolving Incomplete Requests Through Disambiguation Incomplete Requests Reaching Completeness Through Dialog Management Implementing Request Completion Notes on Implementing Request Completion Ambiguous Requests Disambiguation Methods Logic-Based Assumptions Yes/No Questions Implementing Yes/No Questions Notes on Implementing Yes/No Questions A/B Sets Implementing A/B Sets Notes on Implementing A/B Sets Static Lists Implementing Static Lists Notes on Implementing Static Lists Dynamic Lists Implementing Dynamic Lists Notes on Implementing Dynamic Lists Open Sets Implementing Open Sets Notes on Implementing Open Sets Menus Implementing Menus Notes on Implementing Menus Testing on the Device to Find and Solve Issues Two Big Lessons Webhooks 1: Toward Code Independence Fulfillment and Webhooks Webhook Overview Webhook in Depth Contexts, Context Parameters, and Follow-Up Intents What’s Next Summary Chapter 8: Conveying Reassurance with Confidence and Confirmation Conveying Reassurance and Shared Certainty Setting Expectations with Your Implications Webhooks 2 JSON The Webhook Request The Webhook Response Custom Payloads Implementing the Webhook Confirmation Methods Non-verbal Confirmation Code for Non-verbal Confirmation Notes on Non-verbal Confirmation Generic Acknowledgment Implicit Confirmation Code for Implicit Confirmation What If the User Says No? Notes on Implicit Confirmation Explicit Confirmation Code for Explicit Confirmation Dialogflow, App Engine, and Statelessness Notes on Explicit Confirmation Confirmation Placement: Slots vs. Intents Disconfirmation: Dealing with “No” Code for Disconfirmation Notes on Disconfirmation Additional Reassurance Techniques and Pitfalls System Pronunciation Backchannels Discourse Markers VUI Architecture Choosing the Right Reassurance Method Summary Chapter 9: Helping Users Succeed Through Consistency Universals Providing Clarification and Additional Information Providing a Do-Over Providing an Exit Coding Universals Step 1: Creating Intents Step 2: Coding the Webhook Step 3: Adding Repeat Step 4: Checking Context Robustness Step 5: Ending the Conversation Sidebar: Cleaning Up the Code Navigation Landmarks Non-verbal Audio Code for NVA Content Playback Navigation List Navigation Consistency, Variation, and Randomization Code for Randomization Working with Built-In Global Intents Consistency and Standards Across VUIs and Frameworks What’s Next? Summary Chapter 10: Creating Robust Coverage for Speech-to-Text Resolution Recognition Is Speech-to-Text Interpretation Inside the STT Box Creating Sentences with Smaller Pieces Using STT to Build Up Sentences Recognition Engines Grammar Concepts Coverage Recognition Space Static or Dynamic, Large or Small End-Pointing Multiple Hypotheses Types of Grammars Rule-Based Grammars Statistical Models Hot Words, Wake Words, and Invocations Working with Grammars Writing Rule-Based Regular Expressions Regular Expressions Defined Grammars and Training Phrases Sub-grammars Big Sub-grammars Pronunciations How to Succeed with Grammars Bootstrap Normalize Punctuation and Spellings Handle Unusual Pronunciations Use Domain Knowledge Understand the Strengths and Limitations of STT Hands-On: Taking Control over Coverage Changes in Dialogflow Remove Existing Sample Phrases Add an Event Regular Expressions in the Webhook Handling the Phrase If No Other Intent Does Why Do It This Way? Programming Sidebar: Debugging the Webhook Limitations on Grammar Creation and Use What’s Next Summary Chapter 11: Reaching Understanding From Words to Meaning NLP NLU In-Depth: What’s “Smart,” AI, and Understanding? Parsing Machine Learning and NLU Ontologies, Knowledge Bases, and Content Databases Intents Intent Tagging and Tagging Guides Middle Layers: Semantic Tags vs. System Endpoints Putting It All Together Matching Wide or Narrow Multiple Grammars, Multiple Passes The Stanford Parser Revisited Determining Intent Introducing spaCy Loading spaCy Calling spaCy Using spaCy’s Dependency Parse Dialogflow Changes In-Depth: Understanding, Parsing, and Knowledge in Practice Revisiting Intents Machine Learning and Using Knowledge The Not-So-Distant Future? What’s Next? Summary Chapter 12: Applying Accuracy Strategies to Avoid Misunderstanding Accuracy Robustness Concepts Accuracy Robustness Strategies Examples Implementing Examples Providing Help Notes on Designing for Help Just-in-Time Information Notes on Designing for JIT Hidden Options and “None of Those” Recognize-and-Reject One-Step Correction Implementing One-Step Correction Notes on One-Step Correction Tutorials Spelling Narrowing the Recognition Space Advanced Techniques Multi-tier Behavior and Confidence Scores Implementing Multi-tier Confirmation Notes on Multi-tier Confidence-Based Design N-Best and Skip Lists Implementing Skip Lists Notes on N-Best and Skip List Design Probabilities Timeout and Latency What’s Next Summary Chapter 13: Choosing Strategies to Recover from Miscommunication Recovery from What? Recognition, Intent, or Fulfillment Errors Recovery Strategies Meaningful Contextual Prompts Escalating Prompts Code for Escalating Prompts Tapered Prompts Rapid Reprompt Backoff Strategies When to Stop Trying Max Error Counts Code for Keeping Track of Max Errors Transfers Choosing a Recovery Strategy What’s Next Summary Chapter 14: Using Context and Data to Create Smarter Conversations Why There’s No Conversation Without Context Hands-On: Context Complexity Thought Experiment Data Access External Accounts and Services Persistence Tracking Context-Aware and Context-Dependent Dialogs Discourse Markers and Acknowledgments Code for Context-Aware Dialogs Anaphora Resolution Follow-Up Dialogs and Linked Requests Code for Follow-Up Dialogs Proactive Behaviors Code for Proactive Behaviors Topic, Domain, and World Knowledge Geolocation-Based Behavior Proximity and Relevance Number and Type of Devices Time and Day User Identity, Preferences, and Account Types User Utterance Wording System Conditions Tracking Context in Modular and Multi-turn Dialogs Fulfillment What’s Next? Summary Chapter 15: Creating Secure Personalized Experiences The Importance of Knowing Who’s Talking Individualized Targeted Behaviors Concepts in Personalization and Customization Implementing Individualized Experiences Code for Personalized Experiences In-Depth: Context or History? Authorized Secure Access Approaches to Identification and Authentication Implementing Secure Gated Access Notes on Implementing Access Privacy and Security Concerns System Persona Defining and Implementing a System Persona How Persona Affects Dialogs System Voice Audio TTS Synthesis or Voice Talent, Generated or Recorded Finding and Working with Voice Talents One or Several Voices Prompt Management Emotion and Style Voice for Specific User Groups What’s Next? Summary Part IV: The Launch Phase: Verifying and Tuning Chapter 16: Testing and Measuring Voice System Performance Testing Voice System Performance Recognition Testing Dialog Traversal: Functional End-to-End Testing Wake Word and Speech Detection Testing Other Types of System Integration Testing Testing Usability and Task Completion Voice Usability Testing Concepts Study Methodology: Task Scenarios and Surveys Study Environment: In-Lab or Remote Study Administration: Moderated or Unmoderated Results: Quantitative or Qualitative Wizard of Oz Studies Developing the WOZ Tracking and Measuring Performance Recognition Performance Metrics Task Completion Metrics Audio and TTS Testing User Satisfaction Metrics Functional System Task Scenario Studies Administered and Crowdsourced Surveys In-App Automated Surveys Business Metrics What’s Next? Summary Chapter 17: Tuning and Deploying Voice Systems Tuning: What Is It and Why Do You Do It? Why Recognition Accuracy Isn’t Enough Analyzing Causes of Poor System Performance Tuning Types and Approaches Log-Based vs. Transcription-Based Tuning Coverage Tuning Ambiguity Tuning Recognition Accuracy Tuning Acoustic Confusability Tuning Dictionary Tuning Confidence Threshold Timeout and Probability Parameters In-Depth: Building Accuracy from Sample Phrases Finding and Using Recognition Accuracy Data Task Completion Tuning Code for Task Completion Analysis Dialog Tuning Intent Tuning How to Prioritize Your Tuning Efforts Mapping Observations to the Right Remedy Reporting On and Using Tuning Results How to Maximize Deployment Success Know When to Tune Understand Tuning Complexities to Avoid Pitfalls What’s Next? Summary Index We live in a golden age of voice technology. Advances in automatic speech recognition (ASR), natural language processing (NLP) and other technologies have made it extremely viable for people to be talking to machines and getting reasonable answers. Platforms like Amazon Alexa and Google Home, and the associated tools, have made it so anyone can build a voice app, and this is excellent. What we have seen though is that it's fairly easy to build a voice app, but still remarkably difficult to build a great app, one that gets the user what they need, and hopefully the user comes back for more. In Mastering Voice Interfaces we want to show you how to build great voice apps. We start with the basics of voice interfaces, and how they are different from others, then dive into basic design principles that we've learned in many years building these apps in the industry. As we cover a design principle, we'll also demonstrate how to implement it with one of the established voice platforms (Google Home), and show how, though the tools are great, you don't have to go too far to have to do some custom work to get what you really want. We'll walk through many design and development techniques that some would view as advanced, but that can make a huge difference in the quality of the app. Readers of Mastering Voice Interfaces will come away with a very good understanding of what makes voice interfaces so special, learn the basic design principles are for building great voice apps, and how to actually implement those principles and create working apps. What you will learn: What makes voice special and different from other input and output modalities, and why that matters. What the best practices for the various components of the voice-first creation process What are the consequences of design and implementation choices How to create truly great voice apps that users will love Who this book is for We expect readers to have had some exposure to voice apps, at least as users. The book is written for anyone who wants a deeper understanding of the how's and why's of voice interface design and development. For team of developers, designers, product owners who need a shared understanding of voice interfaces We live in a golden age of voice technology. Advances in automatic speech recognition (ASR), natural language processing (NLP) and other technologies have made it extremely viable for people to be talking to machines and getting reasonable answers. Platforms like Amazon Alexa and Google Home, and the associated tools, have made it so anyone can build a voice app, and this is excellent. What we have seen though is that it's fairly easy to build a voice app, but still remarkably difficult to build a great app, one that gets the user what they need, and hopefully the user comes back for more. In Mastering Voice Interfaces we want to show you how to build great voice apps. We start with the basics of voice interfaces, and how they are different from others, then dive into basic design principles that we're learned in many years building these apps in the industry. As we cover a design principle, we'll also demonstrate how to implement it with one of the established voice platforms (Google Home), and show how, though the tools are great, you don't have to go too far to have to do some custom work to get what you really want. Well l walk through many design and development techniques that some would view as advanced, but that can make a huge difference in the quality of the app. Readers of Mastering Voice Interfaces will come away with a very good understanding of what makes voice interfaces so special, learn the basic design principles are for building great voice apps, and how to actually implement those principles and create working apps. What you will learn: What makes voice special and different from other input and output modalities, and why that matters. What the best practices for the various components of the voice-first creation process What are the consequences of design and implementation choices How to create truly great voice apps that users will love Who this book is for We expect readers to have had some exposure to voice apps, at least as users. The book is written for anyone who wants a deeper understanding of the how and why of voice interface design and development. For team of developers, designers, product owners who need a shared understanding of voice interfaces