Driver assistance apparatus and method for controlling the same

• US 10,296,083 B2
• Filed: 08/09/2016
• Issued: 05/21/2019
• Est. Priority Date: 08/26/2015
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Sung, Sinji
• Primary Examiner(s): Bukowski, Kenneth
• Application Number:
  US15/232,484
• Publication Number:
  US 20170060234A1
• Time in Patent Office: 1,015 Days
• CPC Class Codes: B60K 2360/149 Instrument input by detecting viewing direction not otherwise provided for
• CPC Class Codes: B60K 2360/171 Vehicle or relevant part thereof displayed
• CPC Class Codes: B60K 2360/178 Warnings
• CPC Class Codes: B60K 2360/186 Displaying information according to relevancy
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/23 Head-up displays [HUD] optical aspects of head-up displays G02B27/01
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/29 Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60K 37/00 Dashboards as road-vehicle superstructure sub-units B62D25/14
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/006 Side-view mirrors, e.g. V-shaped mirrors located at the front or rear part of the vehicle combined with rear-view mirrors B60R1/08; side-view periscopes B60R1/10
• CPC Class Codes: B60R 1/008 allowing the driver to see passengers, e.g. for busses combined with rear-view mirrors B60R1/08
• CPC Class Codes: B60R 1/04 mounted inside vehicle B60R1/025, B60R1/08 take precedence
• CPC Class Codes: B60R 1/24 in front of the vehicle
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: B60R 2300/70 characterised by an event-triggered choice to display a specific image among a selection of captured images
• CPC Class Codes: B60R 2300/8006 for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
• CPC Class Codes: B60R 2300/8026 in addition to a rear-view mirror system
• CPC Class Codes: B60R 2300/804 : for lane monitoring
• CPC Class Codes: B60W 2050/146 : Display means
• CPC Class Codes: B60W 2420/403 : Image sensing, e.g. optical camera
• CPC Class Codes: B60W 50/14 : Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G06F 2203/0381 : Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/013 : Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 : Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/038 : Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/1423 : controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 3/1431 : using a single graphics controller
• CPC Class Codes: G06F 3/147 : using display panels
• CPC Class Codes: G06F 3/167 : Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 10/25 : Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/597 : Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/20 : Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08G 1/167 : Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: G09G 2320/0261 : in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 2340/04 : Changes in size, position or resolution of an image
• CPC Class Codes: G09G 2340/10 : Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2380/10 : Automotive applications
• CPC Class Codes: G09G 5/003 : Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092

Abstract

A driver assistance apparatus can include a display unit, a camera configured to capture an image of an interior of a vehicle, and a processor configured to sense a direction of at least one of a gaze and a gesture of a driver based on images provided from the camera, detect a region corresponding to the direction among a plurality of predetermined regions of the vehicle, and control the display unit to output information related to the detected region in response to an occurrence of an event related to the detected region among a plurality of predetermined events.

Claims

1. A driver assistance apparatus comprising:

a display unit;

a camera configured to capture an image of an interior of a vehicle; and

a processor configured to;

sense a direction of at least one of a gaze and a gesture of a driver based on images provided from the camera,

detect a region corresponding to the direction from among a plurality of predetermined regions of the vehicle, and

control the display unit to output information related to the detected region in response to an occurrence of an event related to the detected region among a plurality of predetermined events,

wherein the processor is further configured to;

detect at least one part of an upper body of the driver in the images provided from the camera of the interior of the vehicle,

determine whether the upper body of the driver has moved forward based on a change of a size of the upper body of the driver shown in the images provided from the camera,

extract a partial image containing an object of interest from an image related to the detected region, and

control the display unit to display the partial image containing the object of interest based on whether the upper body of the driver has moved forward, and

wherein the processor is further configured to;

control the display unit to display information about the object of interest along with the partial image, and

adjust a size of the partial image displayed on the display unit according to a distance between the vehicle and the object of interest.

2. The driver assistance apparatus according to claim 1, wherein the display unit comprises at least one of a head up display (HUD) and a transparent display, the HUD being disposed on one side of a dashboard to project information related to the detected region and the transparent display being disposed on a windshield of the vehicle.

3. The driver assistance apparatus according to claim 1, wherein the plurality of predetermined regions comprises a first region containing a left side-view mirror of the vehicle,

wherein an event related to the first region comprises turning on a left turn signal light of the vehicle.

4. The driver assistance apparatus according to claim 3, wherein information related to the first region comprises a left view image of the vehicle provided from a left camera of the vehicle.

5. The driver assistance apparatus according to claim 3, wherein, when the direction points to the first region, the processor activates a lane change support function of the vehicle in response to an occurrence of the event related to the first region.

6. The driver assistance apparatus according to claim 1, wherein the plurality of predetermined regions comprises a second region containing a right side-view mirror of the vehicle, and

wherein an event related to the second region comprises turning on a right turn signal light of the vehicle.

7. The driver assistance apparatus according to claim 6, wherein information related to the second region comprises a right view image of the vehicle provided from a right camera of the vehicle.

8. The driver assistance apparatus according to claim 6, wherein, when the direction points to the second region, the processor activates a lane change support function of the vehicle in response to an occurrence of the event related to the second region.

9. The driver assistance apparatus according to claim 1, wherein the plurality of predetermined regions comprises a third region containing a rearview mirror of the vehicle, and

wherein an event related to the third region comprises another vehicle approaching within a predetermined distance from a rear of the vehicle.

10. The driver assistance apparatus according to claim 9, wherein information related to the third region comprises a rearview image of the vehicle provided from a rear camera of the vehicle.

11. The driver assistance apparatus according to claim 1, wherein the plurality of predetermined regions comprises a third region containing a rearview mirror of the vehicle, and

wherein an event related to the third region comprises receiving a voice of a passenger in the vehicle.

12. The driver assistance apparatus according to claim 11, wherein information related to the third region comprises an image of the passenger.

13. The driver assistance apparatus according to claim 1, wherein the plurality of predetermined regions comprises a fourth region containing at least one part of a center console of the vehicle, and

wherein an event related to the fourth region comprises a movement of a portable device in the vehicle.

14. The driver assistance apparatus according to claim 13, wherein information related to the fourth region comprises an image displayed by the portable device.

15. The driver assistance apparatus according to claim 1, wherein the plurality of predetermined regions comprises a fifth region containing at least one part of a windshield of the vehicle, and

wherein an event related to the fifth region comprises receiving a voice of the driver commanding displaying of a specific exterior image of the vehicle.

16. The driver assistance apparatus according to claim 15, wherein information related to the firth region comprises the specific exterior image.

17. The driver assistance apparatus according to claim 1, wherein the plurality of predetermined regions of the vehicle comprises a fourth region containing at least one part of a center console of the vehicle and a fifth region containing at least one part of a windshield of the vehicle, and

wherein an event related to the fourth region and the fifth region comprises the vehicle approaching within a predetermined distance from a predetermined object of interest.

18. The driver assistance apparatus according to claim 17, wherein information related to the fourth region comprises route guide information, and

wherein information related to the fifth region comprises information about the object of interest.

19. The driver assistance apparatus according to claim 1, wherein the processor is further configured to:

control the display unit to display an indicator guiding information previously output by the display unit,

determine whether the gaze of the driver is directed to the indicator displayed by the display unit, and

control the display unit to output the information guided by the indicator.