Information providing apparatus, information providing method, storage medium, and computer program

• US 20030160867A1
• Filed: 01/15/2003
• Published: 08/28/2003
• Est. Priority Date: 01/17/2002
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Ohto, Yasunori, Ohki, Mitsuharu
• Granted Patent: US 7,119,831 B2
• Time in Patent Office: Days
• US Class Current: 348/135
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A display device including a camera is used by inserting it in the direction of a user'"'"'s gaze. Alternatively, an image of the same area as that of a scene from a windshield of an automobile may be obtained and the image may be displayed by using an information providing apparatus installed in the automobile. A system provides information to the user by adding a two-dimensional or three-dimensional computer graphics object to an image taken by the camera. The information is directly added to a scene in the direction of the user'"'"'s gaze and is displayed to the user. Thus the user can easily understand the relationship between the information and the position to which the information is added.

Claims

1. An information providing apparatus for providing additional information associated with position information so that the user can understand the relationship between the additional information and the position, the apparatus comprising:

real image obtaining means for obtaining a real image;

three-dimensional position obtaining means for obtaining the three-dimensional position where the real image is obtained;

building data obtaining means for obtaining data of each building included in the real image based on the obtained three-dimensional position;

additional information obtaining means for obtaining additional information based on the obtained three-dimensional position;

image synthesizing means for synthesizing the additional information and the real image; and

display means for displaying the real image together with the additional information by using a predetermined display window.

2. The apparatus according to claim 1, wherein the three-dimensional position obtaining means obtains the three-dimensional position by using at least one of a gyroscope and a GPS.

3. The apparatus according to claim 1, further comprising:

three-dimensional position correcting means for correcting the three-dimensional position by performing matching of the building data and a characteristic point sequence extracted from the real image.

4. The apparatus according to claim 1, wherein, when the real image is obtained by using the image obtaining means and when the image is distorted due to camera shake, the display means moves the display window in accordance with the movement in the camera direction based on an optical flow of the image.

5. The apparatus according to claim 1, wherein, when the real image is obtained by using the image obtaining means and when the image obtaining position significantly moves so that the display window is outside the range of the display frame, the display means continuously presents the obtained real image to the user until the movement becomes sufficiently slight.

6. The apparatus according to claim 1, wherein the additional information obtaining means obtains a predetermined number of pieces of information from among information about objects near the user.

7. The apparatus according to claim 1, wherein the additional information obtaining means gives priority to obtaining information required by the user.

8. An information providing method for providing additional information associated with position information so that the user can understand the relationship between the additional information and the position, the method comprising:

a real image obtaining step for obtaining a real image;

a three-dimensional position obtaining step for obtaining the three-dimensional position where the real image is obtained;

a building data obtaining step for obtaining data of each building included in the real image based on the obtained three-dimensional position;

an additional information obtaining step for obtaining additional information based on the obtained three-dimensional position;

an image synthesizing step for synthesizing the additional information and the real image; and

a display step for displaying the real image together with the additional information by using a predetermined display window.

9. The method according to claim 8, wherein the three-dimensional position is obtained by using at least one of a gyroscope and a GPS in the three-dimensional position obtaining step.

10. The method according to claim 8, further comprising:

a three-dimensional position correcting step for correcting the three-dimensional position by performing matching of the building data and a characteristic point sequence extracted from the real image.

11. The method according to claim 8, wherein, when the real image is obtained in the image obtaining step and when the image is distorted due to camera shake, the display window is moved in accordance with the movement in the camera direction based on an optical flow of the image in the display step.

12. The method according to claim 8, wherein, when the real image is obtained in the image obtaining step and when the image obtaining position significantly moves so that the display window is outside the range of the display frame, the obtained real image is continuously presented to the user until the movement becomes sufficiently slight in the display step.

13. The method according to claim 8, wherein a predetermined number of pieces of information are obtained from among information about objects near the user in the additional information obtaining step.

14. The method according to claim 8, wherein priority is given to obtaining information required by the user in the additional information obtaining step.

15. A storage medium for physically storing computer-readable software which performs in a computer system a process for providing additional information associated with position information so that the user can understand the relationship between the additional information and the position, the software comprising:

a real image obtaining step for obtaining a real image;

a three-dimensional position obtaining step for obtaining the three-dimensional position where the real image is obtained;

a building data obtaining step for obtaining data of each building included in the real image based on the obtained three-dimensional position;

an additional information obtaining step for obtaining additional information based on the obtained three-dimensional position;

an image synthesizing step for synthesizing the additional information and the real image; and

a display step for displaying the real image together with the additional information by using a predetermined display window.

16. A computer program for performing in a computer system a process for providing additional information associated with position information so that the user can understand the relationship between the additional information and the position, the computer program comprising:

a real image obtaining step for obtaining a real image;

a three-dimensional position obtaining step for obtaining the three-dimensional position where the real image is obtained;

a building data obtaining step for obtaining data of each building included in the real image based on the obtained three-dimensional position;

an additional information obtaining step for obtaining additional information based on the obtained three-dimensional position;

an image synthesizing step for synthesizing the additional information and the real image; and

a display step for displaying the real image together with the additional information by using a predetermined display window.