VALIDATION ANALYSIS OF HUMAN TARGET

• US 20130251204A1
• Filed: 05/17/2013
• Published: 09/26/2013
• Est. Priority Date: 12/17/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Pulsipher, Jon D., Mohadjer, Parham, ElDirghami, Nazeeh Amin, Liu, Shao, Cook, Patrick Orville, Foster, James Chadon, Forbes, Ronald Omega Jr., Stachniak, Szymon P., Leyvand, Tommer, Bertolami, Joseph, Janney, Michael Taylor, Huynh, Kien Toan, Marais, Charles Claudius, Perreault, Spencer Dean, Fitzgerald, Robert John, Bisson, Wayne Richard, Peeper, Craig Carroll, Johnson, Michael
• Granted Patent: US 8,775,916 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 10/776 Validation; Performance evaluation
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

Technology for testing a target recognition, analysis, and tracking system is provided. A searchable repository of recorded and synthesized depth clips and associated ground truth tracking data is provided. Data in the repository is used by one or more processing devices each including at least one instance of a target recognition, analysis, and tracking pipeline to analyze performance of the tracking pipeline. An analysis engine provides at least a subset of the searchable set responsive to a request to test the pipeline and receives tracking data output from the pipeline on the at least subset of the searchable set. A report generator outputs an analysis of the tracking data relative to the ground truth in the at least subset to provide an output of the error relative to the ground truth.

Claims

1. A method for verifying accuracy of a target recognition, analysis, and tracking system, comprising:

providing depth data and associated ground truth data, a deviation between the ground truth data and depth data providing an error thereby determining the accuracy of a tracking pipeline;

responsive to a request to test the pipeline,

generating an analysis of the tracking data relative to the ground truth in at least a subset of the depth data to provide an output of the error relative to the ground truth.

2. The method of claim 1 wherein the method further includes searching the depth data, creating a test suite, and outputting the depth data to a plurality of processing devices, each of the processing devices including an instance of the tracking pipeline.

3. The method of claim 1 wherein the step of generating an analysis includes detecting errors between a processed version of the depth data created by instances of the pipeline and analyzing the errors according to one or more computational metrics, the one or more computational metrics providing an indication of tracking accuracy of the pipeline.

4. The method of claim 1 further including testing metrics measure overall progress and regression in tracking accuracy for one or more of:

an entire scene, a single frame, a single subject in a frame or specific body parts of a user.

5. The method of claim 1 further including testing metrics measure overall progress and regression in tracking accuracy for one or more of:

background removal, retargeting to simulated player representations, body part classification, and detection of scene components.

6. The method of claim 1 wherein manual intervention is allowed to correct ground truth to a known good state.

7. The method of claim 1 further including creating a plurality of test data clips by associating sensor recordings with metadata to create the ground truth.

8. The method of claim 7 wherein the depth data is provided in a searchable set including both recorded depth data and synthesized depth data.

9. The method of claim 8 further including generating synthetic composite test clips by projection of a depth clip into a coordinate space of another depth clip using a computed floor plane.

10. The method of claim 9 further including a process allowing users to transform depth clips from a source depth file into a composite depth file coordinate space.

11. The method of claim 9 further including a process allowing users to load three dimensional synthetic objects into a scene, and composite them into a synthetic data clip.

12. The method of claim 9 further including a process allowing users to load sequence input clips to occur at defined times and to play for defined durations in a synthetic composite test data clips.

13. The method of claim 9 further including registering a motion capture coordinate space to a human tracking pipeline space prior to creating a motion capture data file.

14. The method of claim 7 further including calibrating motion capture sensors to expected ground truth joint positions after creating a test data file.

15. A system for verifying accuracy of a target recognition, analysis, and tracking system, comprising:

a repository of depth clips and associated ground truth for each depth clip, the ground truth comprising an association of joint positions of a human skeletal information detected in the depth clip which has been verified to be accurate;

one or more processing devices each including at least one instance of a target recognition, analysis, and tracking pipeline;

an analysis engine controlling providing at least a subset of the repository to a request to test the pipeline; and

a report generator outputting an analysis of the tracking data relative to the ground truth in the at least subset to provide an output of tracking error relative to the ground truth.

16. The system of claim 16 wherein the analysis engine includes one or more metrics engines providing computational analysis of errors determined between a processed subset and known ground truth for the subset.

17. The system of claim 16 further including a visualization interface outputting reports provided by the report generator to a user.

18. A method for verifying accuracy of a target recognition, analysis, and tracking system, comprising:

storing test data recorded and synthesized and associated ground truth tracking data in a searchable repository,

responsive to a request to test a target recognition, analysis, and tracking system, returning at least a subset of the searchable set;

outputting the subset and code for processing the subset to one or more processing devices, the devices process the subset to provide skeletal tracking information using a pipeline;

receiving tracking data information from the devices;

generating an analysis of the tracking data in the subset relative to the ground truth in the at least subset to provide an output of the error relative to the ground truth; and

creating a summary report of the analysis providing a representation of whether the pipeline performed better than previous versions of the pipeline.

19. The method of claim 18 wherein the method further includes searching the depth data, creating a test suite, and outputting the depth data to a plurality of processing devices, each of the processing devices including an instance of the tracking pipeline.

20. The method of claim 18 wherein the method further includes detecting errors between a processed version of the depth data created by instances of the pipeline and analyzing the errors according to one or more computational metrics, the one or more computational metrics providing an indication of tracking accuracy of the pipeline