Apparatus system and method for human-machine-interface

• US 8,681,100 B2
• Filed: 03/21/2013
• Issued: 03/25/2014
• Est. Priority Date: 07/30/2004
• Status: Expired due to Fees

• Current Assignee: Extreme Reality Ltd.
• Original Assignee: Extreme Realty Limited
• Inventors: Givon, Dor
• Primary Examiner(s): Nguyen, Kimbinh T
• Application Number:
  US13/848,148
• Publication Number:
  US 20130215028A1
• Time in Patent Office: 369 Days
• Field of Search: 345/419, 345/158, 345/157, 345/426, 345/427, 345/473, 382/107, 382/154
• US Class Current: 345/158
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

There is provided a 3D human machine interface (“3D HMI”), which 3D HMI may include: (1) an image acquisition assembly, (2) an initializing module, (3) an image segmentation module, (4) a segmented data processing module, (5) a scoring module, (6) a projection module, (7) a fitting module, (8) a scoring and error detection module, (9) a recovery module, (10) a three dimensional correlation module, (11) a three dimensional skeleton prediction module, (12) an output module and (13) a depth extraction module.

Claims

1. A human machine interface comprising:

an image acquisition assembly to acquire a series of substantially consecutive sets of two-dimensional image data of a user via multiple optical paths, said image acquisition assembly comprising;

a. an image sensor array having an image sensing area; and

b. two or more optical paths directing optical image information of the user onto an at least partially overlapping segment of said image sensing area;

a first processing unit to;

(1) derive, based on the optical image information of the user from two or more optical paths, estimated three dimensional coordinates of elements of the user body, during the acquisition of at least two of the substantially consecutive sets of two-dimensional image data;

(2) determine a movement of one or more body parts of the user between the acquisition of the at least two of the substantially consecutive sets of two-dimensional image data, based on a difference between the estimated three dimensional coordinates of the elements of the user body, during the acquisition of each of the at least two of the substantially consecutive sets of two-dimensional image data; and

a second processing unit to;

(1) correlate the determined movement of one or more body parts of the user to a user input; and

(2) transmit a signal representing the user input.

2. The human machine interface according to claim 1, wherein said image sensor array is a webcam.

3. The human machine interface according to claim 1, wherein deriving, based on the optical image information of the user from two or more optical paths, estimated three dimensional coordinates of elements of the user body, during the acquisition of at least two of the substantially consecutive sets of two-dimensional image data, includes comparing appearances of the elements of the user body from the different optical paths.

4. The human machine interface according to claim 3, wherein comparing appearances of the elements of the user body from the different optical paths includes triangulation.

5. The human machine interface according to claim 1, wherein each of said two or more optical paths comprises a distinct optical element.

6. The human machine interface according to claim 5, wherein said distinct element is a distinct filter.

7. The human machine interface according to claim 5, wherein said distinct element is a distinct grid.

8. The human machine interface according to claim 1, wherein said optical paths direct optical image information of the user onto a common segment of said image sensing area.

9. A method of human machine interfacing, comprising:

acquiring, by an image acquisition assembly, a series of at least two substantially consecutive sets of two-dimensional image data of a user, via multiple optical paths, wherein said image acquisition assembly comprises;

an image sensor array having an image sensing area; and

two or more optical paths directing optical image information of the user onto an at least partially overlapping segment of said image sensing area;

deriving, by processing circuitry and based on the optical image information of the user from two or more optical paths, estimated three dimensional coordinates of elements of the user body, during said acquisition of at least two of the substantially consecutive sets of two-dimensional image data;

determining, by the processing circuitry and based on a difference between the estimated three dimensional coordinates of the elements of the user body, during the acquisition of each of the at least two of the substantially consecutive sets of two-dimensional image data, a movement of one or more body parts of the user, between the acquisition of the at least two of the substantially consecutive sets of two-dimensional image data;

correlating, by the processing circuitry, the determined movement of one or more body parts of the user to a user input; and

transmitting a signal representing the user input.

10. The method according to claim 9, wherein said image sensor array is a webcam.

11. The method according to claim 9, wherein deriving, based on the optical image information of the user from two or more optical paths, estimated three dimensional coordinates of elements of the user body, during the acquisition of at least two of the substantially consecutive sets of two-dimensional image data, includes comparing appearances of the elements of the user body from the different optical paths.

12. The method according to claim 11, wherein comparing appearances of the elements of the user body from the different optical paths includes triangulation.

13. The method according to claim 9, wherein each of said two or more optical paths comprises a distinct optical element.

14. The method according to claim 13, wherein said distinct element is a distinct filter.

15. The method according to claim 13, wherein said distinct element is a distinct grid.

16. The method according to claim 13, wherein said optical paths direct optical image information of the user onto a common segment of said image sensing area.