FLOOR ESTIMATION FOR HUMAN COMPUTER INTERFACES

• US 20170227353A1
• Filed: 02/08/2016
• Published: 08/10/2017
• Est. Priority Date: 02/08/2016
• Status: Active Grant

• Current Assignee: Hasivision, LLC
• Original Assignee: YouSpace, Inc.
• Inventors: Brunner, Ralph
• Granted Patent: US 10,030,968 B2
• Time in Patent Office: Days
• CPC Class Codes: G01B 11/22 for measuring depth
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06V 10/267 by performing operations on regions, e.g. growing, shrinking or watersheds
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Human Computer Interfaces (HCI) may allow a user to interact with a computer via a variety of mechanisms, such as hand, head, and body gestures. Various of the disclosed embodiments allow information captured from a depth camera on an HCI system to be used to recognize such gestures. Particularly, the HCI system'"'"'s depth sensor may capture depth frames of the user'"'"'s movements over time. To discern gestures from these movements, the system may group portions of the user'"'"'s anatomy represented by the depth data into classes. This grouping may require that the relevant depth data be extracted from the depth frame. Such extraction may itself require that appropriate clipping planes be determined. Various of the disclosed embodiments better establish floor planes from which such clipping planes may be derived.

Claims

1. A device configured to determine a floor plane estimation from depth frame data comprising:

a depth sensor device configured to acquire a frame of depth data;

at least one processor;

at least one memory comprising instructions configured to cause the at least one processor to perform a method comprising;

receiving a frame of depth data from the depth sensor;

determining an initial floor plane;

generating a plurality of candidate floor planes by;

rotating a normal associated with a reference floor plane;

determining a metric value for each of the plurality of candidate floor planes; and

selecting the candidate floor plane associated with the best metric value as the determined floor plane.

2. The device of claim 1, wherein the reference floor plane is the initial floor plane.

3. The device of claim 1, wherein the reference floor plane is a previously determined candidate floor plane.

4. The device of claim 1, wherein rotating the normal associated with the reference floor plane comprises rotating the normal by a random amount within a range.

5. The device of claim 1, wherein determining a metric for a candidate floor plane comprises:

determining a first plurality of distances between points in the frame of depth data and the candidate floor plane.

6. The device of claim 5, wherein determining a metric for a candidate floor plane further comprises:

determining an origin of the candidate floor plane by selecting a depth point from a first percentage of points in the depth frame data corresponding to the smallest of the first plurality of distances.

7. The device of claim 6, wherein the distances include negative values.

8. The device of claim 6, wherein the first percentage is 5 percent.

9. The device of claim 6, wherein determining a metric for a candidate floor plane further comprises:

determining a second plurality of distances between a second percentage of points in the frame of depth data corresponding to the smallest of the first plurality of distances and the candidate floor plane; and

summing the second plurality of distances.

10. The device of claim 9, wherein the second percentage is 10 percent.

11. A computer-implemented method for determining a floor plane estimation from depth frame data comprising:

receiving a frame of depth data from a depth sensor;

determining an initial floor plane;

generating a plurality of candidate floor planes by;

rotating a normal associated with a reference floor plane;

determining a metric value for each of the plurality of candidate floor planes; and

selecting the candidate floor plane associated with the best metric value as the determined floor plane.

12. The computer-implemented method of claim 11, wherein the reference floor plane is the initial floor plane in a first iteration and a previously determined candidate floor plane in a subsequent iteration.

13. The computer-implemented method of claim 11, wherein rotating the normal associated with the initial floor plane comprises rotating the normal by a random amount within a range.

14. The computer-implemented method of claim 11, wherein determining a metric for a candidate floor plane comprises:

determining a first plurality of distances between points in the frame of depth data and the candidate floor plane.

15. The computer-implemented method of claim 11, wherein determining a metric for a candidate floor plane further comprises:

determining an origin of the candidate floor plane by selecting a depth point from a first percentage of points in the depth frame data corresponding to the smallest of the first plurality of distances.

16. The computer-implemented method of claim 15, wherein determining a metric for a candidate floor plane further comprises:

determining a second plurality of distances between a second percentage of points in the frame of depth data corresponding to the smallest of the first plurality of distances and the candidate floor plane; and

summing the second plurality of distances.

17. The computer-implemented method of claim 16, wherein the second percentage is 10 percent.

18. A non-transitory computer readable medium comprising instructions configured to cause one or more processors to cause a method to be performed comprising:

receiving a frame of depth data from a depth sensor;

determining an initial floor plane;

generating a plurality of candidate floor planes by;

rotating a normal associated with a reference floor plane;

determining a metric value for each of the plurality of candidate floor planes; and

selecting the candidate floor plane associated with the best metric value as the determined floor plane.

19. The non-transitory computer readable medium of claim 18, wherein determining a metric for a candidate floor plane further comprises:

determining a second plurality of distances between a second percentage of points in the frame of depth data corresponding to the smallest of the first plurality of distances and the candidate floor plane; and

summing the second plurality of distances.

20. The non-transitory computer readable medium of claim 18, wherein determining a metric for a candidate floor plane further comprises:

determining an origin of the candidate floor plane by selecting a depth point from a first percentage of points in the depth frame data corresponding to the smallest of the first plurality of distances.