Display system

• US 8,823,740 B1
• Filed: 08/15/2011
• Issued: 09/02/2014
• Est. Priority Date: 08/15/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Amirparviz, Babak, Miao, Xiaoyu, Wong, Adrian
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): CHEN, FRANK S
• Application Number:
  US13/209,706
• Time in Patent Office: 1,114 Days
• US Class Current: 345/633
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 5/30 Polarising elements light-modulating devices with active elements G02F1/00
• CPC Class Codes: G02C 11/10 Electronic devices other than hearing aids
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

A display system includes an image generator configured to generate a virtual image and a first beam splitter coupled to the image generator. The virtual image and a real-world view are viewable through the first beam splitter from a viewing location. The display system also includes a second beam splitter coupled to the first beam splitter and a camera coupled to the second beam splitter. The camera is configured to image an eye of a user located at the viewing location. Further, a controller is coupled to the image generator and the camera. The controller is configured to process the image of the eye of the user, to determine a location in at least one of the real-world view and the virtual image where the eye of the user is focused, and to control an operation of the display system in response to the determined location.

Claims

1. A display system, comprising:

an image generator configured to generate a virtual image;

a first beam splitter coupled to the image generator, wherein the virtual image and a real-world view are viewable through the first beam splitter from a viewing location;

a second beam splitter coupled to the first beam splitter;

a camera coupled to the second beam splitter, wherein the camera is configured to obtain an image of the viewing location through the first and second beam splitters, and wherein the camera is further configured to obtain an image of the real-world view through the first and second beam splitters; and

a controller coupled to the image generator and the camera and configured to control an operation of the display system based on the image of the viewing location.

2. The display system of claim 1, wherein the camera is configured to image an eye of a user positioned at the viewing location, and wherein the controller is configured to process the image of the eye of the user, to determine a location in at least one of the real-world view and the virtual image where the eye of the user is focused, and to control an operation of the display system in response to the determined location.

3. The display system of claim 2, wherein the controller is configured to determine a location in the real-world view where the eye of the user is focused and to control the generation of the virtual image in response to the determined location.

4. The display system of claim 2, wherein the controller is configured to determine a location in the virtual image where the eye of the user is focused and to control an interaction with a virtual user interface in response to the determined location.

5. The display system of claim 1, further comprising a light source coupled to the second beam splitter, and wherein the image generator includes a display panel coupled to the second beam splitter and a reflector coupled to the first beam splitter.

6. The display system of claim 5, wherein the display panel is configured to spatially modulate light emitted by the light source, and wherein the first and second beam splitters are configured to transmit the spatially modulated light to the reflector, the reflector is configured to reflect the spatially modulated light toward a beam-splitting interface of the first beam splitter, and the beam-splitting interface is configured to reflect the spatially modulated light toward the viewing location as the virtual image.

7. The display system of claim 5, wherein the light source is configured to emit light that is reflected from the viewing location and received by the camera to obtain the image of the viewing location.

8. The display system of claim 5, wherein the first beam splitter is a polarizing beam splitter and the image generator further includes a quarter-wave plate coupled between the reflector and the first beam splitter.

9. The display system of claim 5, wherein the reflector is a curved reflector and the first and second beam splitters include first and second beam-splitting interfaces, respectively, and wherein at least one of the first and second beam-splitting interfaces is a curved interface.

10. The display system of claim 1, wherein the camera is disposed at a distal surface of the second beam splitter and an image sensing portion of the camera is directed toward the second beam splitter.

11. The display system of claim 1, wherein at least one of the first and second beam splitters is a polarizing beam splitter.

12. The display system of claim 1, further comprising a light pipe coupled between the first and second beam splitters.

13. A display system, comprising:

a display panel configured to generate a light pattern;

one or more optical components coupled to the display panel, and configured to transmit the light pattern, external light from a real-world environment, and reflected light from a viewing location, wherein the light pattern is viewable from the viewing location through the one or more optical components as a virtual image superimposed over the real-world environment;

an optical sensor coupled to the one or more optical components and configured to receive the external light transmitted by the one or more optical components to obtain an image of the real-world environment and to receive the reflected light transmitted by the one or more optical components to obtain an image of the viewing location; and

a processor coupled to the display panel and the optical sensor and configured to process the image of the real-world environment and the image of the viewing location.

14. The display system of claim 13, wherein the one or more optical components are configured to transmit reflected light from an eye of a user positioned at the viewing location, and wherein the optical sensor is configured to receive the reflected light to obtain an image of the eye of the user.

15. The display system of claim 14, wherein the processor is configured to process the image of the real-world environment and the image of the eye of the user, to track the eye of the user with respect to the real-world environment, and to control the generation of the light pattern in response to the tracking.

16. The display system of claim 14, wherein the processor is configured to track the eye of the user with respect to the virtual image and to control an operation of the display system in response to the tracking.

17. The display system of claim 16, wherein the processor is configured to control an interaction with a virtual user interface of the display system in response to the processing.

18. The display system of claim 16, wherein the processor is configured to control one or more input/output components coupled to the processor in response to the tracking of the eye of the user with respect to the virtual image.

19. The display system of claim 18, wherein the one or more input/output components include at least one of a speaker, a transmitter and receiver for communicating with other devices, and a microphone.

20. A method, comprising:

generating a light pattern using a display panel;

forming a computer generated image from the light pattern utilizing one or more optical components, wherein the computer generated image is viewable from a viewing location;

receiving external light from a real-world environment through the one or more optical components and incident on an optical sensor, wherein the real-world environment is viewable from the viewing location;

obtaining, by the optical sensor, an image of the real-world environment from the received external light;

receiving light reflected from the viewing location through the one or more optical components and incident on the optical sensor;

obtaining, by the optical sensor, an image of the viewing location from the received reflected light; and

controlling the generation of the light pattern based on the image of the viewing location.

21. The method of claim 20, further comprising calibrating the image of the viewing location with at least one of the computer generated image and the image of the real-world environment.

22. The method of claim 20, wherein the optical components include a first beam splitter with a first beam-splitting interface and a second beam splitter with a second beam-splitting interface, wherein the first beam splitter is positioned adjacent the viewing location, and wherein the display panel, the optical sensor, and a light source are coupled to the second beam splitter.

23. The method of claim 22, wherein receiving external light from a real-world environment through the one or more optical components comprises:

reflecting the external light at the first beam-splitting interface toward the second beam splitter and reflecting the external light at the second beam-splitting interface toward the optical sensor.

24. The method of claim 22, wherein the generating and forming comprise:

emitting light from the light source, reflecting the light at the second beam-splitting interface toward the display panel, spatially modulating the light utilizing the display panel, transmitting the spatially modulated light through the first and second beam splitters, reflecting the spatially modulated light utilizing a reflector coupled to the first beam splitter, and reflecting the spatially modulated light at the first beam-splitting interface toward the viewing location.

25. The method of claim 22, wherein receiving light reflected from the viewing location through the one or more optical components comprises:

reflecting light from the view location toward the first beam splitter, reflecting the reflected light at the first beam-splitting interface toward a reflector coupled to the first beam splitter, reflecting the reflected light at the reflector through the first beam splitter toward the second beam splitter, and reflecting the reflected light at the second beam-splitting interface toward the optical sensor.