Vehicle display system and method of controlling vehicle display system

• US 10,336,188 B2
• Filed: 02/16/2018
• Issued: 07/02/2019
• Est. Priority Date: 03/15/2017
• Status: Active Grant

• Current Assignee: Subaru Corp.
• Original Assignee: Subaru Corp.
• Inventors: Torii, Takeshi, Sato, Noeru, Sumikawa, Ryuichi
• Primary Examiner(s): Lim, Steven
• Assistant Examiner(s): Adnan, Muhammad
• Application Number:
  US15/898,254
• Publication Number:
  US 20180264940A1
• Time in Patent Office: 501 Days
• Field of Search: None
• CPC Class Codes: B60K 2360/186 Displaying information according to relevancy
• CPC Class Codes: B60K 2360/1868 according to driving situations
• CPC Class Codes: B60K 28/02 responsive to conditions relating to the driver
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/29 Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60Q 1/268 on windscreens or windows B60Q1/302 takes precedence; lighting devices for vehicle interior on or for windows B60Q3/208
• CPC Class Codes: B60Q 1/5035 electronic displays
• CPC Class Codes: B60Q 1/525 automatically indicating risk of collision between vehicles in traffic or with pedestrians, e.g. after risk assessment using the vehicle sensor data
• CPC Class Codes: B60Q 1/544 for indicating other states or conditions of the vehicle occupants, e.g. for indicating disabled occupants
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 2300/107 using stereoscopic cameras
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/161 : Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/18 : Eye characteristics, e.g. of the iris
• CPC Class Codes: G08G 1/09623 : Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle G08G1/0967 takes precedence
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

A vehicle display system includes a display device, a driver state determiner, and a display controller. The display device is configured to provide a display on a window of a vehicle. The driver state determiner is configured to determine whether a driver of the vehicle is in a normal state. The display controller is configured to control displaying of the display device, depending on whether the driver is in the normal state. The display controller is configured to control the displaying of the display device to provide a display for the driver when the driver is determined by the driver state determiner as being in the normal state, and to provide a display for an outside of the vehicle when the driver is determined by the driver state determiner as not being in the normal state.

Claims

1. A vehicle display system comprising:

a display device configured to display an image on a window of a vehicle;

a controller configured to;

determine whether a driver of the vehicle is in a normal state;

responsive to determining that the driver is in the normal state;

provide for display, on the display device, a first image towards inside the vehicle for the driver;

acquire, from one or more sensors, information of an environment outside the vehicle, wherein the information of the environment indicates that a target is present outside the vehicle;

determine whether the driver is aware of the presence of the target outside the vehicle; and

responsive to determining that the driver is aware of the presence of the target, provide for display, on the display device, a second image for the target outside the vehicle;

responsive to determining that the driver is not in the normal state;

provide for display, on the display device, a third image towards outside the vehicle; and

provide for display, on the display device concurrently with the second image, a fourth image towards inside the vehicle for an occupant present inside the vehicle based on determining that the occupant other than the driver is present inside the vehicle.

2. The vehicle display system according to claim 1, wherein responsive to determining that the driver is not aware of the presence of the target outside the vehicle, the controller displays, on the display device, a fifth image so that the fifth image on the display device overlaps the target outside the vehicle when viewed by the driver through the window of the vehicle.

3. The vehicle display system according to claim 2, wherein the fifth image is a frame that surrounds the target when viewed by the driver through the window of the vehicle.

4. The vehicle display system according to claim 2, wherein the controller determines, using the one or more sensors, a distance from the vehicle to the target outside the vehicle, and

wherein the controller displays, on the display device, the fifth image when the distance is equal to or less than a preset value.

5. The vehicle display system according to claim 1, further comprising a vehicle-exterior sensor configured to perform detection of a state of the outside of the vehicle, wherein

the controller acquires the information on the environment from the vehicle-exterior sensor.

6. The vehicle display system according to claim 5, wherein the vehicle-exterior sensor comprises a stereo camera.

7. The vehicle display system according to claim 1, further comprising:

a camera configured to capture an image of the driver,

wherein the controller determines whether the driver is in the normal state based on the image of the driver.

8. The vehicle display system according to claim 7, wherein the controller determines, based on the image of the driver, an awakening level of the driver is in the normal state.

9. A method of controlling a vehicle display system, the method comprising:

determining whether a driver of a vehicle is in a normal state;

displaying, on a window of the vehicle, a first image for inside the vehicle based on determining that the driver is in the normal state;

displaying, on the window of the vehicle, a second image for a target outside the vehicle based on determining that

     1) the driver is in the normal state and

     2) the driver is aware of a presence of the target outside the vehicle;

displaying, on the window of the vehicle, a third image for outside the vehicle based on determining that the driver is not in the normal state;

displaying, on the window of the vehicle concurrently with the third image, a fourth image for an occupant present inside the vehicle based on determining that the occupant other than the driver is present inside the vehicle.

10. A vehicle display system comprising:

a display device configured to display an image on a window of a vehicle;

a controller configured to;

provide for display, on the display device, the image with a first display orientation so that the image is displayed towards inside the vehicle;

acquire, from one or more sensors, information of an environment outside the vehicle, the information of the environment indicates that a target is present outside the vehicle; and

switch a display orientation of the image from the first display orientation to a second display orientation so that the image is displayed only towards outside the vehicle based on determining that a driver is aware of the presence of the target.

11. A method of controlling a vehicle display system, the method comprising:

displaying, on a window of a vehicle, an image with a first display orientation so that the image is displayed towards inside the vehicle;

acquiring, from one or more sensors, information of an environment outside the vehicle, the information of the environment indicates that a target is present outside the vehicle; and

switching a display orientation of the image from the first display orientation to a second display orientation so that the image is displayed only towards outside the vehicle based on determining that a driver is aware of the presence of the target.

12. A vehicle display system comprising:

a display device configured to display an image on a window of a vehicle; and

circuitry configured to

determine whether a driver of a vehicle is in a normal state;

provide for display, on the display device, a first image for inside the vehicle based on determining that the driver is in the normal state;

provide for display, on the display device, a second image for a target outside the vehicle based on determining that

     1) the driver is in the normal state and

     2) the driver is aware of a presence of the target outside the vehicle;

provide for display, on the display device, a third image for outside the vehicle based on determining that the driver is not in the normal state;

provide for display, on the display device concurrently with the third image, a fourth image for an occupant present inside the vehicle based on determining that the occupant other than the driver is present inside the vehicle.

13. A vehicle display system comprising:

a display device configured to display an image on a window of a vehicle; and

circuitry configured to

provide for display, on the display device, the image with a first display orientation so that the image is displayed towards inside the vehicle;

acquire, from one or more sensors, information of an environment outside the vehicle, the information of the environment indicates that a target is present outside the vehicle; and

switch a display orientation of the image from the first display orientation to a second display orientation so that the image is displayed only towards outside the vehicle based on determining that a driver is aware of the presence of the target.