Depth-based feature systems for classification applications

• US 10,304,002 B2
• Filed: 02/08/2016
• Issued: 05/28/2019
• Est. Priority Date: 02/08/2016
• Status: Active Grant

• Current Assignee: Hasivision, LLC
• Original Assignee: YouSpace, Inc.
• Inventors: Brunner, Ralph
• Primary Examiner(s): Nilsson, Eric
• Assistant Examiner(s): Hoover, Brent Johnston
• Application Number:
  US15/018,303
• Publication Number:
  US 20170228647A1
• Time in Patent Office: 1,205 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06N 5/025 Extracting rules from data
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 10/421 by analysing segments intersecting the pattern
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

Human Computer Interfaces (HCI) may allow a user to interact with a computer via a variety of mechanisms, such as hand, head, and body gestures. Various of the disclosed embodiments allow information captured from a depth camera on an HCI system to be used to recognize such gestures. Particularly, the HCI system'"'"'s depth sensor may capture depth frames of the user'"'"'s movements over time. To discern gestures from these movements, the system may group portions of the user'"'"'s anatomy represented by the depth data into classes. “Features” which reflect distinguishing features of the user'"'"'s anatomy may be used to accomplish this classification. Some embodiments provide improved systems and methods for generating and/or selecting these features. Features prepared by various of the disclosed embodiments may be less susceptible to overfitting training data and may more quickly distinguish portions of the user'"'"'s anatomy.

Claims

1. A computer system configured to recognize a user gesture from depth data, the computer system comprising:

at least one processor;

at least one memory comprising instructions configured to cause the computer system to perform a method comprising;

receiving depth data captured while a user performed a gesture;

applying a template to the depth data to generate a plurality of features; and

identifying the gesture based upon the plurality of features, wherein

the template was generated, at least in part, by;

determining a Gaussian function;

sampling the Gaussian function to generate a first distribution; and

generating a second distribution by iteratively sampling the first distribution and selecting points of the first distribution using a metric, wherein selecting points of the first distribution using a metric comprises determining a minimum of a plurality of Sub_Metric values, each Sub_Metric value determined using the formula

2. The computer system of claim 1, wherein the generated second distribution is a Well-Constructed Gaussian distribution and the template comprises offsets corresponding to the generated second distribution.

3. The computer system of claim 1, wherein the template was generated by the computer system.

4. The computer system of claim 1, wherein the template comprises a plurality of offsets corresponding to the second distribution and wherein identifying the gesture based upon the plurality of features comprises applying the features to a forest.

5. The computer system of claim 1, the method further comprising:

receiving a plurality of training images; and

generating a plurality of trees from the training images and the template.

6. The computer system of claim 1, wherein the generated second distribution is a Well-Constructed Gaussian distribution, the template comprises offsets corresponding to the generated second distribution, and wherein identifying the gesture based upon the plurality of features comprises applying the features to a forest generated using the template.

7. The computer system of claim 1, wherein determining a minimum of a plurality of Sub_Metric values for points in the first distribution comprises determining a minimum of Sub_Metric values for a point in the first distribution and each existing point in the second distribution.

8. A computer-implemented method to recognize a user gesture from depth data, the method comprising:

receiving depth data captured while a user performed a gesture;

applying a template to the depth data to generate a plurality of features; and

identifying the gesture based upon the plurality of features, wherein

the template was generated, at least in part, by;

determining a Gaussian function;

sampling the Gaussian function to generate a first distribution; and

generating a second distribution by iteratively sampling the first distribution and selecting points of the first distribution using a metric, wherein selecting points of the first distribution using a metric comprises determining a minimum of a plurality of Sub_Metric values, each Sub_Metric value determined using the formula

9. The computer-implemented method of claim 8, wherein the generated second distribution is a Well-Constructed Gaussian distribution and the template comprises offsets corresponding to the generated second distribution.

10. The computer-implemented method of claim 8, wherein the template was generated by the computer.

11. The computer-implemented method of claim 8, wherein the template comprises a plurality of offsets corresponding to the second distribution and wherein identifying the gesture based upon the plurality of features comprises applying the features to a forest.

12. The computer-implemented method of claim 8, the method further comprising:

receiving a plurality of training images; and

generating a plurality of trees from the training images and the template.

13. The computer-implemented method of claim 8, wherein the generated second distribution is a Well-Constructed Gaussian distribution, the template comprises offsets corresponding to the generated second distribution, and wherein identifying the gesture based upon the plurality of features comprises applying the features to a forest generated using the template.

14. The computer-implemented method of claim 8, wherein determining a minimum of a plurality of Sub_Metric values for points in the first distribution comprises determining a minimum of Sub_Metric values for a point in the first distribution and each existing point in the second distribution.

15. A non-transitory computer-readable medium comprising instructions configured to cause a computer system to perform a method to recognize a user gesture from depth data, the method comprising:

receiving depth data captured while a user performed a gesture;

applying a template to the depth data to generate a plurality of features; and

identifying the gesture based upon the plurality of features, wherein

the template was generated, at least in part, by;

determining a Gaussian function;

sampling the Gaussian function to generate a first distribution; and

generating a second distribution by iteratively sampling the first distribution and selecting points of the first distribution using a metric, wherein selecting points of the first distribution using a metric comprises determining a minimum of a plurality of Sub_Metric values, each Sub_Metric value determined using the formula

16. The non-transitory computer-readable medium of claim 15, wherein the generated second distribution is a Well-Constructed Gaussian distribution and the template comprises offsets corresponding to the generated second distribution.

17. The non-transitory computer-readable medium of claim 15, wherein the template was generated by the computer system.

18. The non-transitory computer-readable medium of claim 15, wherein the template comprises a plurality of offsets corresponding to the second distribution and wherein identifying the gesture based upon the plurality of features comprises applying the features to a forest.

19. The non-transitory computer-readable medium of claim 15, the method further comprising:

receiving a plurality of training images; and

generating a plurality of trees from the training images and the template.

20. The non-transitory computer-readable medium of claim 15, wherein determining a minimum of a plurality of Sub_Metric values for points in the first distribution comprises determining a minimum of Sub_Metric values for a point in the first distribution and each existing point in the second distribution.