INFORMATION DISPLAYING APPARATUS AND INFORMATION DISPLAYING METHOD

• US 20110254861A1
• Filed: 12/22/2009
• Published: 10/20/2011
• Est. Priority Date: 12/25/2008
• Status: Active Grant

• Current Assignee: Sinkitech Limited
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Ueda, Junko, Inatomi, Yasuaki, Emura, Koichi, Kanehara, Toshiki
• Granted Patent: US 9,204,050 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/42202 environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 5/445 for displaying additional information H04N5/50 takes precedence

Abstract

An information displaying apparatus capable of providing an easy-to-see display of information that the user wants to know out of information related to objects seen in a captured real-world image. The information displaying apparatus (100) comprises: an image input unit (300) for inputting a captured real-world image; a distance acquiring unit (650) for acquiring the distance between an object seen in the input captured real-world image and a view point of the captured real-world image; an object superimposing unit (710) for outputting an image in which information related to the object is superimposed on the captured real-world image; and an information output control unit (690) for controlling the information outputted by the object superimposing unit (710). The information output control unit (690) decides, in accordance with the acquired distance, how to present the information outputted by the object superimposing unit (710).

Claims

1. An information displaying apparatus comprising:

an image input section that receives as input a captured real-world image;

a distance acquisition section that acquires a distance between an object shown in the input captured real-world image and a viewpoint of the captured real-world image;

an object superimposition section that outputs an image in which information relating to the object is superimposed on the captured real-world image; and

an information output control section that controls the information output from the object superimposition section,

wherein the information output control section decides a way of presenting the information output from the object superimposition section, according to the acquired distance.

2. The information displaying apparatus according to claim 1, wherein, the shorter the distance, the information output control section decides upon at least one of a greater amount of information and a more emphasized presentation mode for the information output from the object superimposition section.

3. The information displaying apparatus according to claim 1, further comprising:

a subject position and orientation acquisition section that acquires a position of a viewpoint of the captured real-world image; and

an object positioning section that acquires a position in the real world of an object shown in the input captured real-world image,

wherein the object positioning section acquires the object position based on the acquired viewpoint position, and a distance, direction, and angle of elevation of an object with reference to the viewpoint.

4. The information displaying apparatus according to claim 3, further comprising an image recognition section that recognizes from the input captured real-world image an object shown in the captured real-world image,

wherein the information output control section makes the object superimposition section output information corresponding to the decided way of presenting information among information relating to the recognized object.

5. The information displaying apparatus according to claim 3, wherein the object positioning section acquires the object position based on an area in the captured real-world image of the recognized object and a viewpoint and field of view of the captured real-world image.

6. The information displaying apparatus according to claim 2, further comprising a lens control section that changes a zoom factor of the captured real-world image input by the image input section, wherein, the larger a zoom factor of the captured real-world image is, the information output control section decides upon a greater amount of information.

7. The information displaying apparatus according to claim 2, wherein, the smaller a number of objects that are information display objects is, the information output control section decides upon a greater amount of information for the information per object.

8. The information displaying apparatus according to claim 1, wherein the information output control section decides upon an object that is an information display object from among objects shown in the input captured real-world image.

9. The information displaying apparatus according to claim 8, wherein the information output control section decides upon an object that is an information display object within a range of a predetermined plurality of objects.

10. The information displaying apparatus according to claim 8, further comprising a lens control section that changes a focus position of the captured real-world image input by the image input section, wherein the information output control section decides upon an object near the focus position as an information display object.

11. The information displaying apparatus according to claim 10, wherein:

the lens control section fixes receives a predetermined operation and fixes the focus position; and

the object superimposition section changes a way of presenting information that is presented when the focus position has been fixed.

12. The information displaying apparatus according to claim 8, further comprising a lens control section that changes a depth of field of the captured real-world image input by the image input section, wherein the information output control section decides upon an object positioned within a range of the depth of field as an information display object.

13. The information displaying apparatus according to claim 3, wherein the object superimposition section displays information relating to the recognized object, associated with an area in the captured real-world image of that object.

14. The information displaying apparatus according to claim 4, further comprising an information position decision section that, when information of a plurality of objects is displayed, decides display positions of information of the plurality of objects as positions corresponding to real-world positional relationships of the plurality of objects.

15. The information displaying apparatus according to claim 14, wherein the information position decision section detects a depth direction of input the captured real-world image, and decides display positions of information of a plurality of objects so as to be aligned in the depth direction when the information of a plurality of objects is displayed.

16. The information displaying apparatus according to claim 1, further comprising:

an operation input section that receives a user operation; and

an operation control section that switches among and sets a mode in which the user operation is reflected in the input captured real-world image, a mode in which the user operation is reflected in information of the presented captured real-world image, and a mode in which the user operation is reflected in the input captured real-world image and the object information is changed in conjunction with the input captured real-world image.

17. The information displaying apparatus according to claim 1, further comprising an operation input section that receives a user operation by means of a touch panel operation for the captured real-world image displayed by the object superimposition section.

18. An information displaying method comprising the steps of:

receiving as input a captured real-world image;

acquiring a distance between an object shown in the input captured real-world image and a viewpoint of the captured real-world image;

deciding a way of presenting information relating to the object according to the acquired distance; and

outputting the information superimposed on the captured real-world image so as to be presented in a decided way of presenting.