User interface, means of movement, and methods for recognizing a user's hand

• US 10,261,593 B2
• Filed: 04/13/2017
• Issued: 04/16/2019
• Est. Priority Date: 04/13/2016
• Status: Active Grant

• Current Assignee: Volkswagen AG
• Original Assignee: Volkswagen AG
• Inventors: Tendyck, Christian, Theimer, Wolfgang, Larson, Gordon
• Primary Examiner(s): Watko, Julie Anne
• Application Number:
  US15/487,373
• Publication Number:
  US 20170300120A1
• Time in Patent Office: 733 Days
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/22 Display screens
• CPC Class Codes: B60R 2300/103 using camera systems provided with artificial illumination device, e.g. IR light source
• CPC Class Codes: B60R 2300/107 using stereoscopic cameras
• CPC Class Codes: B60R 2300/108 using 'non-standard' camera systems, e.g. camera sensor used for additional purposes i.a. rain sensor, camera sensor split in multiple image areas
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/11 Hand-related biometrics; Hand pose recognition
• CPC Class Codes: G06V 40/113 Recognition of static hand signs

Abstract

A hand of a user may be detected in free space, where a plurality of surface points are determined and include a center area surface point and at least two surface points of the plurality of surface points located on a periphery of the surface of the hand. A curve extending through the plurality of surface points may be determined based on a position of a curvature. The plurality of surface points are processed to determine if the plurality of surface points of the detected hand are arranged in at least one of a substantially concave area relative to the sensor, and/or a substantially convex area relative to the sensor. The detected hand may be identified as a palm or back of the hand based on the processing of the plurality of surface points.

Claims

1. A system for detecting a hand orientation, comprising:

a sensor for detecting a hand of a user in free space;

a processing apparatus, operatively coupled to the sensor, the processing apparatus being configured to

detect a plurality of surface points of the detected hand;

process the plurality of surface points to determine if the plurality of surface points of the detected hand are arranged in at least one of a substantially concave area relative to the sensor, and/or a substantially convex area relative to the sensor;

identify the detected hand as a palm of the hand directed towards the sensor if the processing apparatus determines the plurality of surface points of the detected hand are arranged in a substantially concave area relative to the sensor; and

identify the detected hand as a back of the hand directed towards the sensor if the processing apparatus determines the plurality of surface points of the detected hand are arranged in a substantially convex area relative to the sensor.

2. The system according to claim 1, wherein the processing apparatus is configured to:

determine a position of a thumb on the hand; and

identify if the hand is a left hand or a right hand in response to determining the position of the thumb on the hand.

3. The system according to claim 2, wherein the position of the thumb is determined by the processing apparatus using at least one of a radius and/or size assessment of a shape of the surface points of the hand.

4. The system according to claim 1, wherein the processing apparatus is configured to:

determine a surface point from the plurality of surface points located in a center area of the surface of the hand; and

determine at least two other of the surface points of the plurality of surface points located on a periphery of the surface of the hand.

5. The system according to claim 4, wherein the processing apparatus is further configured to determine a curve extending through the surface point located in the surface area to the at least two other of the surface points, based on a position of a curvature.

6. The system according to claim 1, wherein the processing apparatus is further configured to determine an equalization plane placed through the plurality of surface points of the hand as a reference for identifying the at least one of the substantially concave and/or convex area(s).

7. The system according to claim 1, wherein the sensor comprises one of a stereo-camera, an infrared sensor, and a time-of-flight camera.

8. A method for detecting a hand orientation, comprising:

detecting, via a sensor, a hand of a user in free space;

determining, via a processor, a plurality of surface points of the detected hand;

processing, via the processor, the plurality of surface points to determine if the plurality of surface points of the detected hand are arranged in at least one of a substantially concave area relative to the sensor, and/or a substantially convex area relative to the sensor;

identifying, via the processor, the detected hand as a palm of the hand directed towards the sensor if the processor determined the plurality of surface points of the detected hand are arranged in a substantially concave area relative to the sensor; and

identifying, via the processor, the detected hand as a back of the hand directed towards the sensor if the processor determined the plurality of surface points of the detected hand are arranged in a substantially convex area relative to the sensor.

9. The method according to claim 8, further comprising:

determining, via the processor, a position of a thumb on the hand; and

identifying, via the processor, if the hand is a left hand or a right hand in response to determining the position of the thumb on the hand.

10. The method according to claim 9, wherein the position of the thumb is determined using at least one of a radius and/or size assessment of a shape of the surface points of the hand.

11. The method according to claim 8, wherein processing the plurality of surface points comprises:

determining, via the processor, a surface point from the plurality of surface points located in a center area of the surface of the hand; and

determining, via the processor, at least two other of the surface points of the plurality of surface points located on a periphery of the surface of the hand.

12. The method according to claim 11, further comprising determining, via the processor, a curve extending through the surface point located in the surface area to the at least two other of the surface points, based on a position of a curvature.

13. The method according to claim 8, further comprising determining, via the processor, an equalization plane placed through the plurality of surface points of the hand as a reference for identifying the concave and/or convex area(s).

14. The method according to claim 8, wherein the sensor comprises one of a stereo-camera, an infrared sensor, and a time-of-flight camera.

15. A method for detecting a hand orientation, comprising:

detecting, via a sensor, a hand of a user in free space;

determining, via a processor, a plurality of surface points of the detected hand, the surface points comprising a center area surface point and at least two surface points of the plurality of surface points located on a periphery of the surface of the hand;

determining, via the processor, a curve extending through the plurality of surface points, based on a position of a curvature;

processing, via the processor, the plurality of surface points to determine if the plurality of surface points of the detected hand are arranged in at least one of a substantially concave area relative to the sensor, and/or a substantially convex area relative to the sensor;

identifying, via the processor, the detected hand as a palm of the hand directed towards the sensor if the processor determined the plurality of surface points of the detected hand are arranged in a substantially concave area relative to the sensor; and

identifying, via the processor, the detected hand as a back of the hand directed towards the sensor if the processor determined the plurality of surface points of the detected hand are arranged in a substantially convex area relative to the sensor.

16. The method according to claim 15, further comprising:

determining, via the processor, a position of a thumb on the hand; and

identifying, via the processor, if the hand is a left hand or a right hand in response to determining the position of the thumb on the hand.

17. The method according to claim 16, wherein the position of the thumb is determined using at least one of a radius and/or size assessment of a shape of the surface points of the hand.

18. The method according to claim 15, further comprising determining, via the processor, an equalization plane placed through the plurality of surface points of the hand as a reference for identifying the at least one of the substantially concave and/or convex area(s).

19. The method according to claim 15, wherein the sensor comprises one of a stereo-camera, an infrared sensor, and a time-of-flight camera.