Adaptive Projector

• US 20130106692A1
• Filed: 12/23/2012
• Published: 05/02/2013
• Est. Priority Date: 07/20/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: PrimeSense Ltd (Apple Inc.)
• Inventors: Shpunt, Alexander, Berliner, Tamir, Maizels, Aviad
• Granted Patent: US 9,201,501 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/363 using image projection screens volumetric display H04N13/388
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes

Abstract

An apparatus for processing data includes a projector, which is configured to project content onto at least a part of a scene. A processor is configured to detect a location of an eye of a person in the scene and to control the projector so as to reduce an intensity of the projected content in an area of the eye.

Claims

1. A method for augmented interaction with a data processing system, comprising the steps of:

receiving a depth map of a scene containing 3-dimensional objects meeting first criteria for projection of images thereon;

receiving a 2-dimensional image of the scene;

using a digital processor, processing the depth map to locate the objects;

executing a computer application to analyze the 2-dimensional image to identify a most suitable one of the 3-dimensional objects according to second criteria;

projecting a user interface to the computer application as a content-containing image onto the most suitable one of the 3-dimensional objects;

using the digital processor, recognizing a gesture;

interpreting the gesture as an interaction with the user interface; and

controlling the computer application responsively to the interaction.

2. The method according to claim 1, wherein processing the depth map comprises identifying a position of the most suitable one of the 3-dimensional objects with six degrees of freedom with respect to a reference system of coordinates, and projecting a user interface comprises compensating for scale, pitch, yaw and angular rotation thereof.

3. The method according to claim 1, wherein processing the depth map comprises referencing a database of 3-dimensional object definitions and comparing the 3-dimensional objects with the definitions in the database.

4. The method according to claim 1, wherein one of the 3-dimensional objects in the depth map is a portion of a humanoid form.

5. A method for augmented interaction with a data processing system, comprising the steps of:

receiving a depth map of a scene containing 3-dimensional objects meeting first criteria for projection of images thereon;

using a digital processor, processing the depth map to locate the objects;

executing a computer application to identify a most suitable one of the objects according to second criteria;

projecting an image of the one object onto a virtual surface;

using the digital processor, recognizing a gesture;

interpreting the gesture as an interaction with the image of the one object; and

controlling the computer application responsively to the interaction.

6. The method according to claim 5, wherein processing the depth map comprises identifying a position of the one object with six degrees of freedom with respect to a reference system of coordinates, and projecting the image of the one object comprises compensating for scale, pitch, yaw and angular rotation of the one object.

7. The method according to claim 5, wherein processing the depth map comprises referencing a database of 3-dimensional object definitions and comparing the 3-dimensional objects in the depth map with the definitions in the database.

8. The method according to claim 5, wherein the image of the one object further comprises a user interface for control of the computer application.

9. A method for augmented interaction with a data processing system, comprising the steps of:

receiving a depth map of a scene containing 3-dimensional objects meeting first criteria for projection of images thereon;

using a digital processor, processing the depth map to locate the objects;

executing a computer application to identify a most suitable one of the objects according to second criteria;

projecting an image of the one object onto a wearable monitor;

using the digital processor, recognizing a gesture of a user;

interpreting the gesture as an interaction with the image of the one object; and

controlling the computer application responsively to the interaction.

10. The method according to claim 9, wherein processing the depth map comprises identifying a position of the one object with six degrees of freedom with respect to a reference system of coordinates, and projecting the image of the one object comprises compensating for scale, pitch, yaw and angular rotation of the one object.

11. The method according to claim 9, wherein processing the depth map comprises referencing a database of 3-dimensional object definitions and comparing the 3-dimensional objects with the definitions in the database.

12. The method according to claim 9, wherein the image of the one object further comprises a user interface for control of the computer application.

13. An apparatus for processing data, comprising:

a projector, which is configured to project content onto at least a part of a scene; and

a processor, which is configured to detect a location of an eye of a person in the scene and to control the projector so as to reduce an intensity of the projected content in an area of the eye.

14. The apparatus according to claim 13, and comprising a sensing device, which is configured to capture an image of the scene, wherein the processor is configured to process the image in order to identify the area of the eye.

15. The apparatus according to claim 14, wherein the projector comprises:

a first radiation source, which emits an infrared beam, which is modulated to create a pattern of spots on the scene;

a second radiation source, which emits a visible light beam, which is modulated to form the content that is projected onto at least the part of the scene; and

scanning optics configured to project both the infrared beam and the visible light beam onto the scene simultaneously,

wherein the processor is configured to derive a 3-dimensional map of the scene from the pattern of the spots appearing in the image and to identify the area of the eye using the 3-dimensional map.

16. A method for processing data, comprising:

projecting content onto at least a part of a scene;

detecting a location of an eye of a person in the scene; and

controlling projection of the content so as to reduce an intensity of the projected content in an area of the eye.

17. The method according to claim 16, and comprising capturing an image of the scene, processing the image automatically in order to identify the area of the eye.

18. The method according to claim 17, wherein projecting the content comprises scanning a visible light beam over the scene, while modulating the visible light beam to form the content that is projected onto at least the part of the scene, and

wherein capturing the image comprises scanning an infrared beam over the scene simultaneously with the visible light beam, while modulating the infrared beam to create a pattern of spots on the scene, and deriving a 3-dimensional map of the scene from the pattern of the spots appearing in the image, and

wherein controlling the projection comprises identifying the area of the eye using the 3-dimensional map.