IN-VEHICLE DEVICE, CONTROL METHOD OF IN-VEHICLE DEVICE, AND COMPUTER- READABLE STORAGE MEDIUM

• US 20160052394A1
• Filed: 08/18/2015
• Published: 02/25/2016
• Est. Priority Date: 08/22/2014
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: YAMADA, Yoshinori
• Granted Patent: US 9,649,936 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60K 2360/334 Projection means
• CPC Class Codes: B60K 2370/193 for improving awareness
• CPC Class Codes: B60K 2370/334 Projection means
• CPC Class Codes: B60K 2370/733 Left or right steering
• CPC Class Codes: B60K 2370/736 the user being the driver
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/50 Instruments characterised by their means of attachment to or integration in the vehicle B60K35/231 takes precedence
• CPC Class Codes: B60K 35/652 for left- or right-hand drive
• CPC Class Codes: B60K 35/654 the user being the driver
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: B60R 2300/308 by overlaying the real scene, e.g. through a head-up display on the windscreen
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/42 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2422/00 Indexing codes relating to the special location or mounting of sensors
• CPC Class Codes: B60W 2520/10 Longitudinal speed
• CPC Class Codes: B60W 2540/18 Steering angle
• CPC Class Codes: B60W 2540/20 Direction indicator values
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4029 : Pedestrians
• CPC Class Codes: B60W 2554/80 : Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2556/45 : External transmission of data to or from the vehicle
• CPC Class Codes: B60W 2556/50 : of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 50/14 : Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/0053 : from vehicle to occupant
• CPC Class Codes: B60W 60/0059 : Estimation of the risk associated with autonomous or manual driving, e.g. situation too complex, sensor failure or driver incapacity
• CPC Class Codes: G01C 21/365 : Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2380/10 : Automotive applications
• CPC Class Codes: G09G 5/38 : with means for controlling the display position

Abstract

An in-vehicle device includes: a drive assist unit configured to perform drive assist processing; a display unit configured to display an image in a display area with the image overlapped on a scene in a real space ahead of a vehicle; and a display control unit configured to control the display unit to display, in a normal state while the drive assist processing is performed, an image of a first marker at a vehicle-corresponding position as a position in the display area corresponding to a position where the vehicle is estimated to be positioned in the real space in a predetermined time. The first marker has a first color, the first marker has a first shape that has a smooth outline and fluctuates, and a display position of the first marker varies periodically from the vehicle-corresponding position.

Claims

1. An in-vehicle device comprising:

a drive assist unit configured to perform drive assist processing that assists an occupant in driving a vehicle;

a display unit configured to display an image in a display area with the image overlapped on a scene in a real space ahead of the vehicle; and

a display control unit configured to control the display unit to display, in a normal state while the drive assist processing is performed by the drive assist unit, an image of a first marker at a vehicle-corresponding position as a position in the display area corresponding to a position where the vehicle is estimated to be positioned in the real space in a predetermined time, wherein

the first marker has a first color,

the first marker has a first shape that has a smooth outline and fluctuates, and

a display position of the first marker varies periodically from the vehicle-corresponding position.

2. The in-vehicle device according to claim 1, wherein:

the display control unit is configured to control the display unit to display an image of a second marker when there is a need to issue an alert to the occupant;

the second marker has a second color that differs from the first color;

the second marker has a second shape that differs from the first shape; and

a display position of the second marker moves from the vehicle-corresponding position to a position in the display area corresponding to a location where an attention of the occupant is required, at a speed higher than a speed at which the display position of the first marker varies.

3. The in-vehicle device according to claim 2, wherein the second shape has an outline composed of a plurality of acute projections.

4. The in-vehicle device according to claim 2, wherein:

the display control unit is configured to control the display unit to display an image of a third marker when a failure is occurred in the drive assist unit or in the drive assist processing performed by the drive assist unit;

the third marker has a third color that differs from the first color and the second color; and

the third marker has a third shape that differs from the first shape and the second shape and that changes indeterminately.

5. The in-vehicle device according to claim 4, wherein the display control unit is configured to control the display unit to display a message indicating termination of the drive assist processing and erase the display of the third marker, when a type of the occurred failure is a predetermined type.

6. The in-vehicle device according to claim 1, wherein the display control unit is configured to control the display unit to display, while the vehicle is traveling, a track mark that indicates a movement track of the first marker.

7. The in-vehicle device according to claim 6, wherein the display control unit is configured to control the display unit to display, as the track mark, a plurality of marks arranged in such a manner that the marks are continuously flowing from the first marker into an area in the display area corresponding to a road surface in the real space.

8. The in-vehicle device according to claim 2, wherein the display control unit is configured to control the display unit to display, while the vehicle is traveling, a track mark that indicates a movement track of the second marker.

9. The in-vehicle device according to claim 8, wherein the display control unit is configured to control the display unit to display, as the track mark, a plurality of marks arranged in such a manner that the marks are continuously flowing from the second marker into an area in the display area corresponding to a road surface in the real space.

10. The in-vehicle device according to claim 4, wherein the display control unit is configured to control the display unit to display, while the vehicle is traveling, a track mark that indicates a movement track of the third marker.

11. The in-vehicle device according to claim 10, wherein the display control unit is configured to control the display unit to display, as the track mark, a plurality of marks arranged in such a manner that the marks are continuously flowing from the third marker into an area in the display area corresponding to a road surface in the real space.

12. The in-vehicle device according to claim 1, wherein the display control unit is configured to control the display unit to display an alert mark that alerts the occupant, when a simple alert is issued to the occupant or a message to the occupant is displayed.

13. The in-vehicle device according to claim 12, wherein the display control unit is configured to control the display unit to display, as the alert mark, a plurality of marks arranged in such a manner that the marks are scattered from a predetermined position into a surrounding area of the predetermined position.

14. The in-vehicle device according to claim 1, wherein:

the drive assist unit is configured to perform, as the drive assist processing, vehicle-following processing in which the vehicle is controlled to follow a specific vehicle that is traveling ahead of the vehicle in the same direction as a direction of the vehicle; and

the display control unit is configured to control the display unit to display, while the vehicle-following processing is performed by the drive assist unit, a following-target vehicle mark indicating that the specific vehicle is a following target vehicle at a position in the display area corresponding to a position of the specific vehicle in the real space.

15. The in-vehicle device according to claim 1, wherein the display control unit is configured to control the display unit to display, when the vehicle makes one of a lane change and a right or left turn, a direction mark indicating a direction in which the vehicle is to make the one of the lane change and the right or left turn.

16. The in-vehicle device according to claim 1, further comprising a surrounding situation acquisition unit configured to acquire a surrounding situation around the vehicle, wherein

the display control unit controls the display unit to display a surrounding situation diagram that indicates the surrounding situation around the vehicle based on the surrounding situation around the vehicle, acquired by the surrounding situation acquisition unit.

17. A control method for an in-vehicle device including a drive assist unit configured to perform drive assist processing that assists an occupant in driving a vehicle and a display unit configured to display an image in a display area with the image overlapped on a scene in a real space ahead of the vehicle, the control method comprising

controlling the display unit to display, in a normal state while the drive assist processing is performed by the drive assist unit, an image of a first marker at a vehicle-corresponding position as a position in the display area corresponding to a position where the vehicle is estimated to be positioned in the real space in a predetermined time, wherein

the first marker has a first color,

the first marker has a first shape that has a smooth outline and fluctuates, and

a display position of the first marker varies periodically from the vehicle-corresponding position.

18. A non-transitory computer-readable storage medium that stores computer-executable instructions for performing a control method for an in-vehicle device including a drive assist unit configured to perform drive assist processing that assists an occupant in driving a vehicle and a display unit configured to display an image in a display area with the image overlapped on a scene in a real space ahead of the vehicle, the control method comprising

controlling the display unit to display, in a normal state while the drive assist processing is performed by the drive assist unit, an image of a first marker at a vehicle-corresponding position as a position in the display area corresponding to a position where the vehicle is estimated to be positioned in the real space in a predetermined time, wherein

the first marker has a first color,

the first marker has a first shape that has a smooth outline and fluctuates, and

a display position of the first marker varies periodically from the vehicle-corresponding position.