Image display system, image display method, and recording medium

• US 10,796,507 B2
• Filed: 07/26/2019
• Issued: 10/06/2020
• Est. Priority Date: 02/23/2017
• Status: Active Grant

• Current Assignee: Panasonic Automotive Systems Company of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Inventors: Motomura, Hideto, Kourkouss, Sahim, Murata, Hisaji, Emura, Koichi, Ohdachi, Eriko, Tsuji, Masanaga
• Primary Examiner(s): Foxx, Chico A
• Application Number:
  US16/523,822
• Publication Number:
  US 20190347879A1
• Time in Patent Office: 438 Days
• Field of Search: None
• CPC Class Codes: B60K 2360/00 Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
• CPC Class Codes: B60K 2360/166 Navigation
• CPC Class Codes: B60K 2360/167 Vehicle dynamics information
• CPC Class Codes: B60K 2360/175 Autonomous driving
• CPC Class Codes: B60K 2360/178 Warnings
• CPC Class Codes: B60K 2360/1868 according to driving situations
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/29 Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 21/00 Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks safety belts or body harnesses in vehicles B60R22/00; devices, apparatus or methods for life-saving in general A62B; safety devices for propulsion unit control specially adapted for, or arranged in, vehicles B60K28/00; seats constructed to protect the occupant from the effect of abnormal g-forces, e.g. crash or safety seats, B60N2/42; energy-absorbing arrangements for hand wheels for steering vehicles B62D1/11; energy-absorbing arrangements for vehicle steering columns B62D1/19; harnessing in aircraft B64D25/00
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/80 characterised by the intended use of the viewing arrangement
• CPC Class Codes: B60W 40/10 related to vehicle motion
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G07C 5/0825 using optical means
• CPC Class Codes: G08G 1/16 Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08

Abstract

An image display system includes a first processor, a second processor, and a comparator. The first processor acquires a behavior estimation result of a vehicle, and generates future position information after a predetermined time passes of the vehicle based on the behavior estimation result. The second processor acquires present information about the vehicle, and generates present position information on the vehicle and a peripheral object based on the acquired information. The comparator compares the future position information on the vehicle and the present position information on the vehicle and the peripheral object, and generates present image data indicating present positions of the vehicle and the peripheral object and future image data indicating future positions of the vehicle and the peripheral object. Further, the comparator allows a notification device to display a present image based on the present image data and a future image based on the future image data together.

Claims

1. An image display system comprising:

a processor; and

a memory coupled to the processor, the memory storing a program that, when executed by the processor, causes the image display system to;

acquire a behavior estimation result of a vehicle, and generate future position information of the vehicle after a predetermined time passes from a present time based on the behavior estimation result;

acquire present information about the vehicle, and generate present position information of the vehicle and an object around the vehicle based on the acquired present information;

compare the future position information of the vehicle and the present position information of the vehicle and the object around the vehicle, and generate present image data indicating present positions of the vehicle and the object around the vehicle and future image data indicating future positions of the vehicle and the object around the vehicle; and

allow a display device to display a present image based on the present image data and a future image based on the future image data together,

wherein;

the future image includes a parameter value indicating a future running state of the vehicle, the parameter value being based on the behavior estimation result and acquired present speed information and operation information of the vehicle, or

the behavior estimation result is a result based on a behavior instruction acquired from a driver of the vehicle.

2. The image display system according to claim 1, wherein the program, when executed by the processor, causes the image display system to:

allow the display device to display the present image and the future image side by side.

3. The image display system according to claim 1, wherein

the present information about the vehicle includes position information and speed information of the vehicle, map information on a position of the vehicle, and position information and speed information of the object around the vehicle, and

the program, when executed by the processor, causes the image display system to:

display, in the present image and the future image, the vehicle and the object around the vehicle on a map showing the position of the vehicle constructed based on the present information about the vehicle.

4. The image display system according to claim 2, wherein

the present information about the vehicle includes position information and speed information of the vehicle, map information on a position of the vehicle, and position information and speed information of the object around the vehicle, and

the program, when executed by the processor, causes the image display system to:

display, in the present image and the future image, the vehicle and the object around the vehicle on a map showing the position of the vehicle constructed based on the present information about the vehicle.

5. The image display system according to claim 1, wherein

the behavior estimation result is a result estimated, by using machine learning, from at least one of information on a peripheral condition of the vehicle and information on a running state of the vehicle.

6. The image display system according to claim 1, wherein the program, when executed by the processor, causes the image display system to:

acquire the present speed information and the operation information of the vehicle from the vehicle,

calculate the parameter value indicating the future running state of the vehicle using the behavior estimation result and the present speed information and the operation information of the vehicle, and

display the parameter value in the future image.

7. The image display system according to claim 1, wherein the program, when executed by the processor, causes the image display system to:

receive the behavior instruction from the driver of the vehicle, and

process the behavior instruction as the behavior estimation result.

8. The image display system according to claim 1, wherein

when the behavior estimation result is the result based on the behavior instruction acquired from the driver of the vehicle, the future image includes the parameter value indicating the future running state of the vehicle, the parameter value being based on the behavior estimation result and the acquired present speed information and operation information of the vehicle.

9. The image display system according to claim 1, wherein

when the future image includes the parameter value indicating the future running state of the vehicle, the behavior estimation result is the result based on the behavior instruction acquired from the driver of the vehicle.

10. An image display method comprising:

acquiring present information about a vehicle, and generating present position information of the vehicle and an object around the vehicle based on the acquired present information;

acquiring a behavior estimation result that is a behavior estimation result of the vehicle, and generating future position information of the vehicle after a predetermined time passes from a present time based on the behavior estimation result;

comparing the future position information of the vehicle and the present position information of the vehicle and the object around the vehicle, and generating present image data indicating present positions of the vehicle and the object around the vehicle and future image data indicating future positions of the vehicle and the object around the vehicle; and

displaying a present image based on the present image data and a future image based on the future image data together,

wherein;

the future image includes a parameter value indicating a future running state of the vehicle, the parameter value being based on the behavior estimation result and acquired present speed information and operation information of the vehicle, or

the behavior estimation result is a result based on a behavior instruction acquired from a driver of the vehicle.

11. A non-transitory recording medium describing a program for causing a computer to execute an image display method, the image display method comprising:

acquiring present information about a vehicle, and generating present position information of the vehicle and an object around the vehicle based on the acquired present information;

acquiring a behavior estimation result that is a behavior estimation result of the vehicle, and generating future position information after a predetermined time passes of the vehicle based on the behavior estimation result;

comparing the future position information of the vehicle and the present position information of the vehicle and the object around the vehicle, and generating present image data indicating present positions of the vehicle and the object around the vehicle and future image data indicating future positions of the vehicle and the object around the vehicle; and

displaying a present image based on the present image data and a future image based on the future image data together,

wherein;

the future image includes a parameter value indicating a future running state of the vehicle, the parameter value being based on the behavior estimation result and acquired present speed information and operation information on the vehicle, or

the behavior estimation result is a result based on a behavior instruction acquired from a driver of the vehicle.