تسلط بر کینکت مایکروسافت: ردیابی بدن، شناسایی اشیاء و خدمات ابری آژور

Know how to program the Microsoft Kinect and use the device for applications that interact directly with humans through gestures and motion. This book covers the mathematics and theoretical background needed for depth sensing, motion tracking, and object recognition while maintaining a practical focus on getting things done. You will learn to track the human body in three-dimensional space, analyze the human motion, and remove the background to isolate the person being tracked. You will see how to recognize objects and voice, and transform between the three-dimensional physical space and a computer’s two-dimensional screen. The book is written with real-world applications in mind. It provides step-by-step tutorials and source code for common use cases. The author has worked with startups and Fortune 500 companies, and all of the examples are taken directly from the industry. The book’s practical focus simplifies the core principles, removes the clutter, and allows developers to start writing code right away. Also covered is the use of Azure Cognitive Services on Microsoft’s cloud platform and their use for object and voice recognition, enabling your applications to “see” objects and respond to their environment. What You Will Learn Visualize color, depth, and body data Calculate angles between different body joints Analyze human motion and create fitness applications Recognize objects and voice using cloud-powered artificial intelligence Remove the background from a scene to create virtual worlds Who This Book Is For Developers who want to build demanding Kinect apps and games, and those who are looking for a careful balance between theoretical knowledge and practical application that favors the practical. Readers should have a basic knowledge of C# and some familiarity with the Unity3D engine. Table of Contents About the Author About the Technical Reviewer Acknowledgments Introduction Part I: Meet the Kinect Chapter 1: Mixed Reality and Kinect A Brief History of Mixed Reality Natural User Interfaces Kinect Sensor Anatomy The RGB Video Camera The Depth Sensor The Microphone Arrays The IMU (Accelerometer and Gyroscope) The External Sync Pins Enhancing Kinect with Azure Key Points Chapter 2: The Developer Toolbox Overview of the Microsoft Azure SDK Azure Kinect Sensor SDK Sensor SDK System Requirements The Kinect Viewer Azure Kinect Body Tracking SDK Body Tracking SDK System Requirements The Kinect Body Tracking Viewer Setting Up the Development Environment Unity3D and Visual Studio Bringing It All Together Key Points Part II: The Basics Chapter 3: Configuring the Device Adding the Kinect SDKs in Unity3D The Azure Kinect Binaries Managed C# Binaries Native C++ Binaries The Machine Learning Models Running in the Editor Deploying Your Unity Application Mastering Azure Kinect: Source Code Starting and Stopping the Device Basic Device Properties Kinect Device Configuration Handling Invalid Configurations Receiving Captures Reading Data Key Points Chapter 4: Color Data Structure of a Color Frame The BGRA32 Color Format The MJPG Color Format Displaying Color Data in Unity3D Specifying the Color Configuration Reading Kinect Color Data as BGRA32 Reading Kinect Color Data as MJPG Key Points Chapter 5: Depth Data Structure of a Depth Frame Narrow and Wide Fields of View Displaying Depth Data in Unity3D Depth Configuration and Data Grayscale Depth Visualization Jet Depth Visualization Ambiguous Depth Cases Too Close to the Camera or Too Far from the Camera Object Edges Moving Objects Corners Key Points Chapter 6: Body Tracking The Technology of Body Tracking The New Azure Kinect Approach Structure of a Human Body Joint ID Joint Confidence Level Joint Position Acquiring Body Data Tracker Initialization Tracker Update Constructing Body Objects Tracker Release Displaying Body Data in Unity3D Key Points Part III: The Magic Chapter 7: Streaming Data in the Background Creating a Data Package Creating the Streaming Class Starting and Stopping the Device Streaming Data in a Background Thread Updating the Main Thread Using the KinectSensor Class Key Points Chapter 8: Coordinate Mapping Coordinate Systems 2D Coordinates 3D Coordinates Coordinate Mapping Coordinate Transformations World to Color World to Depth Color to World Depth to World Color to Depth Depth to Color Using the Coordinate Mapper Class The Complete Coordinate Mapper Class Key Points Chapter 9: Augmented Reality: Removing the Background of the Users Mixing the Physical and the Digital Worlds Background Removal The Body Index Map Mapping Color to Depth A Background Removal Game in Unity3D Key Points Chapter 10: Motion Analysis Motion Analysis 1) Skeleton Data 2) Measurements 3) Feedback Measuring Physical Distances Example: Evaluating a Squat Measuring Angles Example: Counting Bicep Curls Establishing a Threshold Increasing the Counter Measuring Speed Example: How Fast Are You Walking? Key Points Part IV: The “Azure” in Kinect Chapter 11: Azure Cognitive Services There Is an API for That! Azure Cognitive Services Offline Approach Online Approach Understanding Azure in Kinect Creating a Computer Vision API Key Points A Demo Service for You Chapter 12: Computer Vision and Object Detection Do You Need a Kinect After All? The Azure Cognitive Services SDK The Computer Vision NuGet Package Importing the Packages in Unity3D Creating a New Unity Scene Referencing the Azure Computer Vision SDK Computer Vision in Action Input: Kinect Color Frames Output: Object Rectangles Coordinate Mapping Visualizing the Results Closing the Service Key Points Index