Method circuit and system for human to machine interfacing by hand gestures

• US 8,872,899 B2
• Filed: 01/03/2011
• Issued: 10/28/2014
• Est. Priority Date: 07/30/2004
• Status: Expired due to Fees

• Current Assignee: Extreme Reality Ltd.
• Original Assignee: Extreme Reality Ltd.
• Inventors: Givon, Dor
• Primary Examiner(s): PHILIPPE, GIMS S
• Application Number:
  US12/983,335
• Publication Number:
  US 20110129124A1
• Time in Patent Office: 1,394 Days
• Field of Search: 348/49, 348/77, 345/419, 703/2, 382115-118, 382/128, 382/154, 382/206
• US Class Current: 348/49
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 40/113 Recognition of static hand signs

Abstract

Disclosed are systems, methods and circuits for human to machine interfacing using one or more 3D hand skeleton models, comprising an image acquisition circuit to acquire a 2D image of a hand, a hand feature identifier module to identify hand features in an acquired image and an image skeleton builder module to generate a 2D skeleton dataset of the hand using identified features. A 3D skeleton builder module is adapted to generate a 3D hand skeleton dataset by determining a configuration of some or all of the one or more 3D hand skeleton models whose 2D projection substantially corresponds to the generated 2D skeleton dataset.

Claims

1. A system for human to machine interfacing using one or more three dimensional (3D) hand skeleton models, said system comprising:

receiving circuitry adapted to receive a two dimensional (2D) image including a hand;

a hand feature identifier module comprising processing circuitry adapted to identify hand features in the received image, wherein identifying hand features includes identifying skin wrinkles indicative of locations of joints of the hand;

an image skeleton builder module comprising processing circuitry adapted to generate a 2D skeleton dataset of the hand using the identified features; and

a 3D skeleton builder module comprising processing circuitry adapted to generate a 3D hand skeleton dataset of the hand by determining a configuration of a 3D hand skeleton model matching the generated 2D skeleton dataset, by comparing 2D projections of configurations of the 3D hand skeleton model to the generated 2D skeleton dataset to determine the matching combination;

wherein said 3D skeleton builder module selects configurations of the 3D model to compare with the 2D skeleton dataset based upon a first set of locations of joints of the 2D skeleton dataset, which first set of locations of joints has been determined based on the identified skin wrinkles.

2. The system according to claim 1, wherein the 3D hand skeleton model includes anatomic relationships and constraints between elements of the 3D hand skeleton model, which constraints and relationships are factored when selecting configurations of the 3D model to compare with the 2D skeleton dataset.

3. The system according to claim 1, wherein the image skeleton builder module is further adapted to determine a quality attribute for one or more elements of the 2D skeleton dataset.

4. The system according to claim 1, further comprising a skeleton database builder module adapted to store one or more generated 3D hand skeleton datasets;

and

wherein the 3D skeleton builder module is further adapted to generate the 3D hand skeleton dataset at least partially based on one or more 3D hand skeleton datasets previously stored by said database builder module.

5. The system according to claim 1, further comprising a prediction module adapted to predict a predicted 3D hand skeleton dataset based on two or more preceding 3D hand skeleton datasets to which motion dynamics physics rules are applied;

and

wherein said 3D skeleton builder is further adapted to generate the 3D hand skeleton dataset at least partially based on the predicted 3D hand skeleton dataset.

6. A method for human to machine interfacing using one or more three dimensional (3D) hand skeleton models comprising:

receiving, by processing circuitry, a two dimensional (2D) image including a hand;

identifying, by processing circuitry, one or more hand features in the received image, wherein identifying hand features includes identifying skin wrinkles indicative of locations of joints of the hand;

generating, by processing circuitry, a 2D skeleton dataset of the hand using the identified features; and

generating, by processing circuitry, a 3D hand skeleton dataset of the hand by determining a configuration of a 3D hand skeleton model matching the generated 2D skeleton dataset, by comparing 2D projections of configurations of the 3D hand skeleton model to the generated 2D skeleton dataset to determine the matching combination;

wherein selecting configurations of the 3D model to compare with the 2D skeleton dataset is based upon a first set of locations of joints of the 2D skeleton dataset, which first set of locations of joints has been determined based on the identified skin wrinkles.

7. The method according to claim 6, wherein the 3D hand skeleton model includes anatomic relationships and constraints between elements of the 3D hand skeleton model, which constraints and relationships are factored when selecting configurations of the 3D model to compare with the 2D skeleton dataset.

8. The method according to claim 6, wherein generating a 2D skeleton dataset further comprises determining a quality attribute for one or more elements of the 2D skeleton dataset.

9. The method according to claim 6, further comprising storing one or more 3D hand skeleton datasets, and generating the 3D hand skeleton dataset at least partially based on the previously stored 3D hand skeleton datasets.

10. The method according to claim 6, wherein generating a 3D hand skeleton dataset further comprises predicting a predicted 3D hand skeleton dataset by applying motion dynamics physics rules to two or more previously generated 3D hand skeleton datasets, and generating the 3D hand skeleton dataset at least partially based on the predicted 3D hand skeleton dataset.

11. A system for generating a three dimensional (3D) hand skeleton dataset by using one or more 3D hand skeleton models, said system comprising:

a hand feature identifier module comprising processing circuitry adapted to identify hand features in a two dimensional (2D) image including a hand, wherein identifying features includes extrapolating tangents to segments of the hand and identifying intersections of the tangents as locations of joints of the hand;

an image skeleton builder module comprising processing circuitry adapted to generate a 2D skeleton dataset of the hand using the identified features; and

a 3D skeleton builder comprising processing circuitry adapted to determine a configuration of one of the one or more 3D hand skeleton models matching the generated 2D skeleton dataset by comparing 2D projections of configurations of the 3D hand skeleton model to the generated 2D skeleton dataset to determine the matching combination;

wherein said 3D skeleton builder module selects configurations of the 3D model to compare with the 2D skeleton dataset based upon a first set of locations of joints of the 2D skeleton dataset, which first set of locations of joints has been determined based on the identified intersections.

12. The system according to claim 11, wherein the 3D hand skeleton model includes anatomic relationships and constraints between elements of the 3D hand skeleton model, which constraints and relationships are factored when selecting configurations of the 3D model to compare with the 2D skeleton dataset.

13. The system according to claim 11, wherein the image skeleton builder module is further adapted to determine a quality attribute for one or more elements of the 2D skeleton dataset.

14. The system according to claim 11, further comprising a skeleton database builder module adapted to store one or more generated 3D hand skeleton datasets;

and

wherein the 3D skeleton builder module is further adapted to generate the 3D hand skeleton dataset at least partially based on one or more 3D hand skeleton datasets previously stored by said database builder module.

15. The system according to claim 11, further comprising a prediction module adapted to predict a predicted 3D hand skeleton dataset based on two or more preceding 3D hand skeleton datasets to which motion dynamics physics rules are applied;

and

wherein said 3D skeleton builder is further adapted to generate the 3D hand skeleton dataset at least partially based on the predicted 3D hand skeleton dataset.