Augmented reality apparatus

• US 8,922,589 B2
• Filed: 10/15/2013
• Issued: 12/30/2014
• Est. Priority Date: 04/07/2013
• Status: Expired due to Fees

• Current Assignee: Laor Consulting LLC, A Colorado Limited Liability Corporation
• Original Assignee: Laor Consulting, LLC
• Inventors: Laor, Herzel
• Primary Examiner(s): McDowell, Jr., Maurice L
• Assistant Examiner(s): Zhai, Kyle
• Application Number:
  US14/053,607
• Publication Number:
  US 20140300632A1
• Time in Patent Office: 441 Days
• Field of Search: 345/440, 345/633
• US Class Current: 345/633
• CPC Class Codes: G02B 2027/011 comprising device for correcting geometrical aberrations, distortion
• CPC Class Codes: G02B 2027/0116 comprising devices for correcting chromatic aberration
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

An embodiment of the invention relates to an apparatus and related methods for providing a person with an augmented reality, the apparatus comprising: a projector that projects light that generates an augmenting visual image (AVI); optics that directs to a camera a portion of the projected light and a portion of light arriving from a scene so that a combined image comprising an image of the scene and the AVI are generated in a photosensor in the camera; and a controller that processes the combined image to compare positions of homologous features in the image of the scene with positions of corresponding location markers in the AVI, and renders an adjusted AVI based on the comparison, so that the location markers and the corresponding homologous features are substantially coincident in the combined image.

Claims

1. An apparatus for providing a person with an augmented reality responsive to a scene that the person views, the apparatus comprising:

a photosensor that images light incident on the photosensor;

a light projector;

a controller that controls the projector to project light that generates an augmenting virtual image (AVI) that is not projected onto the scene and having markers corresponding to homologous features in the scene;

optics that directs light from the scene and light projected by the projector to form a first combined image on the person'"'"'s retina comprising an image of the scene overlaid by the AVI, and a second combined image comprising an image of the scene overlaid by the AVI on the photosensor;

wherein the controller configures the AVI responsive to the second combined image so that the markers in the first combined image are substantially coincident with the corresponding homologous features of the scene comprised in the first combined image.

2. The apparatus of claim 1, wherein the apparatus is substantially free of one or more sources of relative dimensional variance between the first combined image and the second combined image selected from the group consisting of:

parallax error due to the positioning of the retina of the person and the photosensor, differential chromatic magnification and optical aberrations.

3. The apparatus of claim 1 wherein the optics comprises a mirror that is partially transmitting and partially reflecting.

4. The apparatus of claim 1, wherein the AVI includes an image representing an object within the scene, the image being rendered from a computer code stored in a memory and encoding a three-dimensional representation of the object.

5. The apparatus of claim 4, wherein the computer code is a reconstruction based on one or more captured images of the object.

6. The apparatus of claim 1, wherein the homologous features in the scene are native to the scene.

7. The apparatus of claim 1, wherein the homologous features comprise one or more artificial indicators placed on the scene.

8. The apparatus of claim 1, wherein the optics is configured to make the relative luminance between the image of the scene and the AVI directed to the person substantially the same as relative luminance between the image of the scene and the AVI directed to the photosensor.

9. The apparatus of claim 1, wherein the optics is configured to have light in the combined image reflecting from the retina be directed to the photosensor.

10. The apparatus of claim 1 configured to be incorporated into a head-mounted apparatus.

11. The apparatus of claim 1 configured to be combined with an optical instrument that provides magnification or reduction of the scene.

12. A method for providing a person with an augmented reality, the method comprising:

rendering an augmenting visual image (AVI) comprising location markers corresponding to selected homologous features in a scene;

projecting the AVI, wherein the AVI is not projected to the scene;

directing the AVI and an image of the scene to the person'"'"'s retina and a photosensor;

capturing the AVI and the image of the scene;

comparing positions of the homologous features in the image of the scene with positions of the corresponding location markers in the AVI;

rendering an adjusted AVI in which the location markers are substantially coincident with the corresponding homologous features in the capture scene image; and

projecting the adjusted AVI, wherein the AVI is not projected to the scene; and

directing the adjusted AVI and the image of the scene to the person'"'"'s retina and the photosensor.

13. The method of claim 12, wherein following directing the adjusted AVI and the image of the scene, the method is repeated from capturing the AVI and the image of the scene.

14. The method of claim 12, wherein the AVI includes an image representing an object within the scene, the image being rendered from a computer code stored in a memory and encoding a three-dimensional representation of the object.

15. The method of claim 14, preceded by creating a new computer code stored in a memory and encoding the three-dimensional representation of the object.

16. The method of claim 15, wherein, following directing the adjusted AVI and the image of the scene, the method is repeated from creating the new computer code encoding the three-dimensional representation of the object.

17. The method of claim 14, wherein the computer code encoding the three-dimensional representation of the object is a reconstruction based on one or more captured images of the object.

18. The method of claim 12, wherein the homologous features in the scene are native to the scene.

19. The method of claim 12, wherein the homologous features in the scene comprise one or more artificial markers placed on the scene.

20. An apparatus for providing a person with an augmented reality responsive to a scene that the person views, the apparatus comprising:

a photosensor;

a light projector that projects light that generates an augmenting virtual image (AVI) that is not projected onto the scene;

optics comprising a partially reflecting flat mirror situated between a scene and the person'"'"'s retina and also between the projector and the photosensor, the partially reflecting flat mirror being configured to direct portions of the light from the scene to form an image of the scene on the person'"'"'s retina and on the photosensor, and to direct portions of the light from the projector to form the AVI on the person'"'"'s retina and on the photosensor so that the image of the scene is overlaid by the AVI on the retina and on the photosensor.

21. The apparatus of claim 20 further comprising an occluder situated between the scene and the optics.