Augmented reality in a field of view including a reflection

• US 9,933,855 B2
• Filed: 03/31/2016
• Issued: 04/03/2018
• Est. Priority Date: 03/31/2016
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Ferens, Ron, Hurwitz, Barak, Dorin, Camila, Kamhi, Gila
• Primary Examiner(s): Williams, Kimberly A
• Application Number:
  US15/087,478
• Publication Number:
  US 20170285345A1
• Time in Patent Office: 733 Days
• Field of Search: 345633
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 2210/16 Cloth
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

Systems, apparatuses, and/or methods to augment reality. An object identifier may identify an object in a field of view of a user that includes a reflection of the user from a reflective surface, such as a surface of a traditional mirror. In addition, a reality augmenter may generate an augmented reality object based on the identification of the object. In one example, eyeglasses including a relatively transparent display screen may be coupled with an image capture device on the user and the augmented reality object may be observable by the user on the transparent display screen when the user wears the eyeglasses. A localizer may position the augmented reality object on the transparent display screen relative to the reflection of the user that passes though the transparent display screen during natural visual perception of the reflection by the user.

Claims

1. A system comprising:

eyeglasses including a transparent display screen,

a range camera on a user coupled with the eyeglasses to capture image data and depth data for a virtual object in a field of view of the user that is included in a reflection of the user from a reflective surface, wherein the depth data is based on a location of the reflective surface in the field of view of the user that is set as an image surface and a distance of a center line between the image surface and an intersection of the virtual object with a plurality of perspective lines that extend from the image surface to the virtual object,

a reality augmenter to automatically generate an augmented reality object based on an identification of the virtual object, and

a localizer to position the augmented reality object on the transparent display screen relative to the reflection of the user that is to pass though the transparent display screen to allow the augmented reality object to be observable by the user on the transparent display screen when the user wears the eyeglasses.

2. The system of claim 1, further including,

an image capture device that is to face the user, and

a synchronizer to synchronize data from the image capture device that is to face the user with data from the range camera on the user.

3. The system of claim 2, wherein the image capture device that is to face the user is to include a range camera that is to face the user.

4. An apparatus comprising:

an object identifier to automatically make an identification of a virtual object in a field of view of a user that is included in a reflection of the user from a reflective surface,

an image data identifier to identify image data for the virtual object from a range camera on the user,

a depth data identifier to identify depth data for the virtual object from the range camera, wherein the depth data is based on a location of the reflective surface in the field of view of the user that is set as an image surface and a distance of a center line between the image surface and an intersection of the virtual object with a plurality of perspective lines that extend from the image surface to the virtual object,

a reality augmenter to automatically generate an augmented reality object based on the identification of the virtual object, and

a localizer to position the augmented reality object on a transparent display screen of eyeglasses relative to the reflection of the user that is to pass though the transparent display screen to allow the augmented reality object to be observable by the user on the transparent display screen when the user wears the eyeglasses.

5. The apparatus of claim 4, further including,

a device identifier to identify a device on the user,

a skeletal identifier to identify a body position of the user,

a facial identifier to identify a face of the user, and

a gesture identifier to identify a gesture by the user.

6. The apparatus of claim 4, further including,

a mapper to generate a map from image data of a scene, and

a feature extractor to extract a feature from the image data of the scene, wherein the localizer is to,

position the user in the map based on a displacement of the feature in the map, and

position the augmented reality object in the map based on the position of the user.

7. The apparatus of claim 4, further including a synchronizer to synchronize data from an image capture device that is to face the user with data from an range camera on the user.

8. At least one computer readable storage medium comprising a set of instructions, which when executed by a processor, cause the processor to:

automatically make an identification of a virtual object in a field of view of a user that is included in a reflection of the user from a reflective surface;

automatically identify image data for the virtual object from a range camera on the user;

automatically identify depth data for the virtual object from the range camera, wherein the depth data is based on a location of the reflective surface in the field of view of the user that is set as an image surface and a distance of a center line between the image surface and an intersection of the virtual object with a plurality of perspective lines that extend from the image surface to the virtual object;

automatically generate an augmented reality object based on the identification of the virtual object; and

automatically position the augmented reality object on a transparent display screen of eyeglasses relative to the reflection of the user that is to pass though the transparent display screen to allow the augmented reality object to be observable by the user on the transparent display screen when the user wears the eyeglasses.

9. The at least one computer readable storage medium of claim 8, wherein the instructions, when executed, cause the processor to:

identify a device on the user;

identify a body position of the user;

identify a face of the user; and

identify a gesture by the user.

10. The at least one computer readable storage medium of claim 8, wherein the instructions, when executed, cause the processor to:

generate a map from image data of a scene;

extract a feature from the image data of the scene;

position the user in the map based on a displacement of the feature in the map; and

position the augmented reality object in the map based on the position of the user.

11. The at least one computer readable storage medium of claim 8, wherein the instructions, when executed, cause the processor to synchronize data from an image capture device that is to face the user with data from the range camera on the user.

12. A method comprising:

automatically making an identification of a virtual object in a field of view of a user that is included in a reflection of the user from a reflective surface;

automatically identifying image data for the virtual object from a range camera on the user;

automatically identifying depth data for the virtual object from the range camera, wherein the depth data is based on a location of the reflective surface in the field of view of the user that is set as an image surface and a distance of a center line between the image surface and an intersection of the virtual object with a plurality of perspective lines that extend from the image surface to the virtual object;

automatically generating an augmented reality object based on the identification of the virtual object; and

automatically positioning the augmented reality object on a transparent display screen of eyeglasses relative to the reflection of the user that is to pass though the transparent display screen to allow the augmented reality object to be observable by the user on the transparent display screen when the user wears the eyeglasses.

13. The method of claim 12, further including:

identifying a device on the user;

identifying a body position of the user;

identifying a face of the user; and

identifying a gesture by the user.

14. The method of claim 12, further including:

generating a map from image data of a scene;

extracting a feature from the image data of the scene;

positioning the user in the map based on a displacement of the feature in the map; and

positioning the augmented reality object in the map based on the position of the user.

15. The method of claim 12, further including synchronizing data from an image capture device that faces the user with data from the range camera on the user.