Augmented reality information displaying device and augmented reality information displaying method

• US 10,083,546 B2
• Filed: 03/16/2017
• Issued: 09/25/2018
• Est. Priority Date: 04/11/2016
• Status: Expired due to Fees

• Current Assignee: Fujitsu Ten Limited (DENSO Corporation)
• Original Assignee: Fujitsu Ten Limited (DENSO Corporation)
• Inventors: Soda, Tomofumi, Iino, Masahiro, Yamamoto, Chiharu, Ikeda, Satoru
• Primary Examiner(s): Hoang, Phi
• Application Number:
  US15/460,747
• Publication Number:
  US 20170294046A1
• Time in Patent Office: 558 Days
• CPC Class Codes: B60K 2360/167 Vehicle dynamics information
• CPC Class Codes: B60K 2360/177 Augmented reality
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2360/27 using semi-transparent optical elements
• CPC Class Codes: B60K 2360/334 Projection means
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/23 Head-up displays [HUD] optical aspects of head-up displays G02B27/01
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/81 for controlling displays
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/107 using stereoscopic cameras
• CPC Class Codes: B60R 2300/20 characterised by the type of display used
• CPC Class Codes: B60R 2300/308 by overlaying the real scene, e.g. through a head-up display on the windscreen
• CPC Class Codes: G01S 19/13 Receivers
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2354/00 : Aspects of interface with display user
• CPC Class Codes: G09G 3/003 : to produce spatial visual effects
• CPC Class Codes: G09G 5/00 : Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: G09G 5/026 : Control of mixing and/or overlay of colours in general G09G5/022 and G09G5/024 take precedence

Abstract

An augmented reality information displaying device according to the embodiment includes a storage, a display-position deciding unit, and a display controller. The storage stores augmented reality information associated with a real position. The display-position deciding unit decides a display position of the augmented reality information on a display screen of a transmission-type display device based on a real position of a user that visually recognizes the display screen, a real position of the display screen, the real position in the scene through the display screen associated with the augmented reality information. The display controller controls to display, at a display position decided by the display-position deciding unit, the augmented reality information corresponding to a mode according to positional relationship between the real position of the user and the real position associated with the augmented reality information in the scene.

Claims

1. An augmented reality information displaying device comprising:

a storage that stores augmented reality information associated with a real position, the augmented reality information including three-dimensional image information; and

a processor operatively coupled to the storage, the processor being programmed to;

specify a real position of a user that is in a moving body and visually recognizes a display screen of a transmission-type display device of the moving body;

specify a real position of the display screen;

specify a real position in a scene seen through the display screen associated with the augmented reality information;

decide a display position of the augmented reality information on the display screen based on the real position of the user, the real position of the display screen, and the real position of the augmented reality information;

in response to deciding the display position of the augmented reality information, display, at the display position on the display screen, the augmented reality information;

determine a moving direction of the moving body based on a positional relationship between the real position of the user and the real position of the augmented reality information; and

in response to determining the moving direction of the moving body, change at least one of a direction and a size of the three-dimensional augmented reality information displayed on the display screen in accordance with the moving direction of the moving body.

2. The augmented reality information displaying device according to claim 1, wherein, in accordance with a movement of the display screen, the processor is further programmed to gradually reduce a thickness of an image of the augmented reality information from a side where the blocking of the visual recognition caused by the blocking object is started, and gradually increase the thickness of the image of the augmented reality information from a side where the blocking of the visual recognition caused by the blocking object is terminated.

3. The augmented reality information displaying device according to claim 1, wherein the processor is further programmed to adjust a brightness of the augmented reality information to be displayed in accordance with a brightness of the real position in the scene associated with the augmented reality information.

4. The augmented reality information displaying device according to claim 1, wherein the processor is further programmed to adjust a color of the augmented reality information to be displayed in accordance with a hue of the scene.

5. The augmented reality information displaying device according to claim 1, wherein, in response to determining that the moving body moves from side to side with respect to the real position of the augmented reality information, the processor is further programmed to change a direction and a size of the three-dimensional augmented reality information displayed on the display screen.

6. The augmented reality information displaying device according to claim 5, wherein, in response to determining that the moving body moves back and forth with respect to the real position of the augmented reality information, the processor is further programmed to change only the size of the three-dimensional augmented reality information displayed on the display screen.

7. An augmented reality information displaying device comprising:

a storage that stores augmented reality information associated with a real position; and

a processor operatively coupled to the storage, the processor being programmed to;

decide a display position of the augmented reality information on a display screen of a transmission-type display device based on a real position of a user that visually recognizes the display screen, a real position of the display screen, and the real position in a scene seen through the display screen associated with the augmented reality information;

display, at the display position, the augmented reality information corresponding to a mode according to a positional relationship between the real position of the user and the real position in the scene associated with the augmented reality information;

detect a blocking object blocking visual recognition, from the display position, of the real position in the scene associated with the augmented reality information;

cause the display screen not to display the augmented reality information when the visual recognition is blocked by the blocking object;

acquire a moving speed of the display screen;

compute a duration time during which the visual recognition is blocked by the blocking object based on the acquired moving speed; and

switch between display and non-display of the augmented reality information in accordance with the computed duration time.

8. The augmented reality information displaying device according to claim 7, wherein

the processor is further programmed to display a transmitted image of the augmented reality information when the visual recognition is blocked by the blocking object.

9. An augmented reality information displaying method that causes a computer to execute a process comprising:

storing augmented reality information associated with a real position, the augmented reality information including three-dimensional image information; and

specifying a real position of a user that is in a moving body and visually recognizes a display screen of a transmission-type display device of the moving body;

specifying a real position of the display screen;

specifying a real position in a scene seen through the display screen associated with the augmented reality information;

deciding a display position of the augmented reality information on the display screen based on the real position of the user, the real position of the augmented reality information;

in response to deciding the display position of the augmented reality information, displaying, at the display position on the display screen, the augmented reality information;

determining a moving direction of the moving body based on a positional relationship between the real position of the user and the real position of the augmented reality information; and

in response to determining the moving direction of the moving body, change at least one of a direction and a size of the three-dimensional augmented reality information displayed on the display screen in accordance with the moving direction of the moving body.