Three-Dimensional User Interface

• US 20110164032A1
• Filed: 01/07/2010
• Published: 07/07/2011
• Est. Priority Date: 01/07/2010
• Status: Abandoned Application

• Current Assignee: Apple Inc.
• Original Assignee: PrimeSense Ltd (Apple Inc.)
• Inventors: Shadmi, Avraham
• Application Number:
  US12/683,452
• Publication Number:
  US 20110164032A1
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Methods and systems for interfacing a computer system are provided, which include capturing three-dimensional image maps over time of at least a part of a control element such as the body of a human subject, generating a three-dimensional representation of scene elements by driving a three-dimensional display with a second sequence of three-dimensional maps of scene elements. The two sequences of maps are correlated to detect a direction and speed of movement of the part of the body with respect to the scene elements. A relationship of the direction and speed of movement to at least one of the scene elements is established. A computer application is controlled according to the relationship that is established.

Claims

1. A method of interfacing a computer system, comprising the steps of:

capturing a first sequence of three-dimensional maps over time of a control entity that is situated external to the computer system;

generating a three-dimensional representation of scene elements by driving a three-dimensional display with a second sequence of three-dimensional maps of scene elements;

correlating the first sequence with the second sequence in order to detect a spatial relationship between the control entity and the scene elements; and

controlling a computer application responsively to the spatial relationship.

2. The method according to claim 1, wherein the spatial relationship is an overlap of the control entity in a frame of the first sequence with one of the scene elements in frame of the second sequence wherein a control entity.

3. The method according to claim 1, wherein generating the three-dimensional representation comprises producing an image of the scene elements in free space.

4. The method according to claim 1, wherein generating the three-dimensional representation comprises extending a two-dimensional representation of the scene elements on a display screen to another representation having three perceived spatial dimensions.

5. A method of interfacing a computer system, comprising the steps of:

capturing a first sequence of three-dimensional maps over time of at least a part of a control entity;

generating a three-dimensional representation of scene elements by driving a three-dimensional display with a second sequence of three-dimensional maps of scene elements;

correlating the first sequence with the second sequence in order to detect a direction and speed of movement of the part of the control entity with respect to the scene elements; and

controlling a computer application responsively to the direction and speed of movement with respect to at least one of the scene elements.

6. The method according to claim 5, wherein controlling the computer application comprises updating at least a portion of the scene elements.

7. The method according to claim 5, wherein generating the three-dimensional representation comprises producing an image of the scene elements in free space.

8. The method according to claim 5, wherein generating the three-dimensional representation comprises extending a two-dimensional representation of the scene elements on a display screen to another representation having three perceived spatial dimensions.

9. The method according to claim 5, further comprising the steps of:

deriving a viewing distance of a human subject from the first sequence of three-dimensional maps; and

adjusting the second sequence of three-dimensional maps according to the viewing distance.

10. The method according to claim 5, further comprising the steps of:

deriving a viewing angle of a human subject from the first sequence of three-dimensional maps; and

adjusting the second sequence of three-dimensional maps according to the viewing angle.

11. A computer program product for interfacing a computer system, including a computer-readable storage medium in which computer program instructions are stored, which instructions, when executed by a computer, cause the computer to perform the steps of:

capturing a first sequence of three-dimensional maps over time of at least a part of a control entity;

generating a three-dimensional representation of scene elements by driving a three-dimensional display with a second sequence of three-dimensional maps of scene elements;

correlating the first sequence with the second sequence in order to detect a direction and speed of movement of the part of the control entity with respect to the scene elements; and

controlling a computer application responsively to the direction and speed of movement with respect to at least one of the scene elements.

12. The computer program product according to claim 11, wherein controlling the computer application comprises updating at least a portion of the scene elements.

13. The computer program product according to claim 11, wherein the three-dimensional maps of scene elements comprise interactive regions that include respective ones of the scene elements and controlling the computer application comprises detecting a motion of the part of the control entity within respective interactive regions.

14. The computer program product according to claim 11, further comprising the steps of:

deriving a viewing distance of a human subject from the first sequence of three-dimensional maps; and

adjusting the second sequence of three-dimensional maps according to the viewing distance.

15. The computer program product according to claim 11, further comprising the steps of:

deriving a viewing angle of a human subject from the first sequence of three-dimensional maps; and

adjusting the second sequence of three-dimensional maps according to the viewing angle.

16. A user interface apparatus, comprising:

a sensing device, which is configured to capture a first sequence of three-dimensional maps over time of at least a part of a control entity;

a three-dimensional display module, which is adapted for generating a three-dimensional representation of scene elements; and

a processor;

a memory accessible to the processor having a computer application stored therein, wherein the processor is configured to execute the computer application and cooperatively therewith perform the steps of;

constructing a second sequence of three-dimensional maps of scene elements;

driving the three-dimensional display module with the second sequence;

correlating the first sequence with the second sequence in order to detect a direction and speed of movement of the part of the control entity with respect to the scene elements; and

controlling the computer application responsively to the direction and speed of movement with respect to at least one of the scene elements.

17. The apparatus according to claim 16, wherein controlling the computer application comprises updating at least a portion of the scene elements.

18. The apparatus according to claim 16, wherein the three-dimensional maps of scene elements comprise interactive regions that include respective ones of the scene elements and controlling the computer application comprises detecting a motion of the part of the control entity within respective interactive regions.

19. The apparatus according to claim 16, wherein generating the three-dimensional representation comprises producing an image of the scene elements in free space.

20. The apparatus according to claim 16, wherein generating the three-dimensional representation comprises extending a two-dimensional representation of the scene elements on a display screen to another representation having three perceived spatial dimensions.