Method, system and device for augmented reality

• US 20020167536A1
• Filed: 03/29/2002
• Published: 11/14/2002
• Est. Priority Date: 03/30/2001
• Status: Abandoned Application

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Thomason, Graham G., Valdes, Armando S.
• Application Number:
  US10/109,771
• Publication Number:
  US 20020167536A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G01S 13/74 Systems using reradiation of radio waves, e.g. secondary radar systems; Analogous systems
• CPC Class Codes: G02B 27/01 Head-up displays

Abstract

A portable electronic device comprises augmented reality viewing apparatus for viewing a real scene and a superimposed computer generated overlay scene. In one embodiment the viewing apparatus comprises a display screen (2) and a semitransparent mirror (3). The semitransparent mirror (3) is pivotally mounted on the device and may be rotated between a position for viewing augmented reality and a position for viewing a displayed image alone. In another embodiment the real scene is viewed through a transparent display screen. When viewing augmented reality, the user aligns the overlay scene with the real scene by means of an alignment indicator (13,15, not shown in FIG. 5) in the overlay scene which corresponds to a predetermined element of the real scene. The device may be equipped with location determining means (50), the selection of a displayed image thereby being dependent on the location of the device, whether the images for display are stored locally in the device or transmitted by radio from a remote server. The device may also be equipped with an orientation sensor so that the selection of a displayed images is dependent on orientation of the device.

Claims

1. A method of preparing an overlay scene for display on an augmented reality viewing apparatus, characterised by generating an alignment indicator corresponding to a predetermined element of a real scene for inclusion in the overlay scene, the alignment indicator in use being aligned with the predetermined element of the real scene.

2. An overlay scene suitable for combining with a real scene to form an augmented reality scene, comprising an alignment indicator corresponding to a predetermined element of the real scene, the alignment indicator in use being aligned with the predetermined element of the real scene.

3. A portable electronic device equipped with augmented reality viewing apparatus suitable for viewing a real scene and an overlay scene having an alignment indicator corresponding to a predetermined element of the real scene, the augmented reality viewing apparatus comprising a display screen, wherein the device has a first mode wherein the display screen displays an overlay scene and a second mode wherein the display screen displays a non-overlay scene.

4. A device as claimed in claim 3, wherein the viewing apparatus further comprises a pivotally mounted semitransparent mirror arrangeable in a first position in which a user can view superimposed on the real scene an overlay scene displayed on the display screen and in a second position in which the user can view the display screen without viewing the real scene.

5. A device as claimed in claim 4, wherein pivotal rotation of the semitransparent mirror is motor driven.

6. A device as claimed in claim 4, wherein the display screen is pivotally mounted.

7. A device as claimed in claim 6, wherein pivotal rotation of the display screen is motor driven.

8. A device as claimed in claim 4, wherein adoption of the first mode is responsive to a first pivotal position of the semitransparent mirror and adoption of the second mode is responsive to a second pivotal position of the semitransparent mirror.

9. A device as claimed in claim 6, wherein adoption of the first mode is responsive to a first pivotal position of the display screen and adoption of the second mode is responsive to a second pivotal position of the display screen.

10. A device as claimed in claim 3, wherein in the first mode the display screen is transparent and the real scene may be viewed through the display screen and in the second mode the real scene may not be viewed through the display screen.

11. A device as claimed in any one of claims 3 to 10, comprising storage means wherein the storage means contains a plurality of overlay scenes each corresponding to a different real scene and each comprising an alignment indicator corresponding to a predetermined element of their respective real scene, and selection means for selecting which of the plurality of overlay scenes is displayed, wherein the selection means is responsive to an indication of the location of the portable electronic device.

12. A device as claimed in claim 11, comprising means to determine location and means to supply to the selection means the indication of location.

13. A device as claimed in claim 12, comprising an orientation sensor and means to supply to the selection means an indication of orientation, wherein the selection means is responsive to the indication of orientation.

14. A device as claimed in any one of claims 3 to 10, comprising orientation sensing means for generating an indication of orientation, location determining means for generating an indication of location, and storage means wherein the storage means contains a plurality of overlay scenes each corresponding to a different real scene, and selection means for selecting which of the plurality of overlay scenes is displayed, wherein the selection means is responsive to the indications of location and orientation of the portable electronic device.

15. A device as claimed in any one of claims 3 to 10, further comprising means to receive over a radio link an overlay scene for display.

16. A device as claimed in claim 15, further comprising means to determine location and means to transmit an indication of location over a radio link.

17. A device as claimed in claim 16, further comprising an orientation sensor and means to transmit an indication of orientation over a radio link.

18. A device as claimed in claim 3, comprising an orientation sensor, wherein adoption of the first mode is responsive to a first orientation of the device and adoption of the second mode is responsive to a second orientation of the device.

19. An augmented reality system comprising a portable electronic device as claimed in claim 15, serving means comprising storage means for storing a plurality of overlay scenes each corresponding to a different real scene and each overlay scene comprising an alignment indicator corresponding to a predetermined element of its respective real scene, the serving means further comprising selection means for selecting one of the plurality of overlay scenes for display on the portable electronic device, wherein the selection means is responsive to an indication of the location of the portable electronic device, and means for transmitting the selected overlay scene to the portable electronic device.

20. An augmented reality system as claimed in claim 19, the portable electronic device further comprising means to determine location and means to transmit an indication of location over a radio link to the serving means.

21. An augmented reality system as claimed in claim 20, the portable electronic device further comprising an orientation sensor and means to transmit an indication of orientation over the radio link to the serving means, and wherein the selection means is responsive to the indication of orientation of the portable electronic device.

22. An augmented reality system comprising a portable electronic device as claimed in claim 17, serving means comprising storage means for storing a plurality of overlay scenes each corresponding to a different real scene, the serving means further comprising selection means for selecting one of the plurality of overlay scenes for display on the portable electronic device, wherein the selection means is responsive to the indications of the location and orientation of the portable electronic device, and means for transmitting the selected overlay scene to the portable electronic device.