System and Method for Providing Augmented Reality

• US 20090244097A1
• Filed: 03/25/2008
• Published: 10/01/2009
• Est. Priority Date: 03/25/2008
• Status: Abandoned Application

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Estevez, Leonardo William
• Application Number:
  US12/055,116
• Publication Number:
  US 20090244097A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06F 1/1613 for portable computers cooling arrangements therefor G06F1/203; constructional details or arrangements for pocket calculators, electronic agendas or books G06F15/0216; constructional details of portable telephone sets: with several bodies H04M1/0202
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: H04N 5/2224 related to virtual studio applications
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

A system and method for providing augmented reality. A method comprises retrieving a specification of an environment of the electronic device, capturing optical information of the environment of the electronic device, and computing the starting position/orientation from the captured optical information and the specification. The use of optical information in addition to positional information from a position sensor to compute the starting position may improve a viewer'"'"'s experience with a mobile augmented reality system.

Claims

1. A method for calculating a starting position/orientation of an electronic device, the method comprising:

retrieving a specification of an environment of the electronic device;

capturing optical information of the environment of the electronic device; and

computing the starting position/orientation from the captured optical information and the specification.

2. The method of claim 1, wherein retrieving the specification comprises retrieving the specification from an information server.

3. The method of claim 2, wherein retrieving the specification further comprises prior to retrieving the specification from the information server, detecting the presence of the information server.

4. The method of claim 1, wherein capturing optical information comprises:

panning the electronic device about the environment; and

capturing optical information as the electronic device pans.

5. The method of claim 4, wherein panning the electronic device comprises:

bringing the electronic device into a specified position;

initiating a capturing sequence; and

panning the electronic device between a first specified position and a second specified position.

6. The method of claim 1, wherein capturing optical information comprises retrieving luminosity information from an image sensor.

7. The method of claim 6, wherein retrieving the luminosity information comprises retrieving automatic gain control information from the image sensor.

8. The method of claim 6, wherein computing the starting position comprises:

locating high luminosity objects in the environment;

processing the luminosity information from the image sensor; and

computing the starting position/orientation from a difference between the specification and the processed luminosity information.

9. The method of claim 8, wherein processing the luminosity information comprises applying a Hough transform to the luminosity information.

10. The method of claim 1, wherein capturing optical information comprises capturing a sequence of optical images with an optical sensor in the electronic device.

11. The method of claim 10, wherein computing the starting position comprises:

creating a unified image from the sequence of optical images;

computing a first angle between the electronic device and a first pair of objects in the environment from the unified image;

computing a second angle between the electronic device and a second pair of objects in the environment from the unified image; and

computing the starting position/orientation from the first angle, the second angle, and the specification.

12. The method of claim 1, wherein capturing optical information comprises retrieving hyperspectral information from a hyperspectral sensor in the electronic device.

13. The method of claim 12, wherein computing the starting position comprises:

locating objects of known hyperspectral signature in the environment;

processing the hyperspectral information from the hyperspectral sensor; and

computing the starting position/orientation from a difference between the specification and the processed hyperspectral information.

14. The method of claim 1, wherein computing the starting position/orientation also makes use of position information from a positional sensor.

15. A method for displaying an image using a portable display device, the method comprising:

computing a position/orientation for the portable display device;

rendering the image using the computed position/orientation for the portable display device;

displaying the image; and

in response to a determining that the portable display device has changed position/orientation,

computing a new position/orientation for the portable display device, wherein the computing makes use of optical position information captured by an optical sensor in the portable display device, and

repeating the rendering and the displaying using the computed new position/orientation.

16. The method of claim 15, further comprising after displaying the image, continuing to display the image in response to a determining that the portable display device has not changed position/orientation.

17. The method of claim 15, wherein rendering the image comprises adjusting the image to correct for a point of view determined by the computed position/orientation.

18. The method of claim 15, wherein computing the new position/orientation also makes use of position/orientation information from a positional sensor.

19. The method of claim 15, wherein the optical position information is selected from the group consisting of:

luminosity information, visual image of a specified object, hyperspectral image information, and combinations thereof.

20. An electronic device comprising:

a projector configured to display an image;

a position sensor configured to provide position and orientation information of the electronic device;

an optical sensor configured to capture optical information for use in computing a position and orientation of the electronic device; and

a processor coupled to the projector, to the position sensor, and to the optical sensor, the processor configured to process the optical information and the position and orientation information to compute the position and orientation of the electronic device and to render the image using the position and orientation of the electronic device.

21. The electronic device of claim 20, wherein a scan mirror device is used to display the image and to redirect optical information to the optical sensor.

22. The electronic device of claim 20, wherein the projector utilizes the optical sensor to display the image, and wherein the projector is not displaying the image when the optical sensor captures optical information for use in computing the position and orientation of the electronic device.