System and method to provide driving assistance

• US 10,181,266 B2
• Filed: 09/11/2015
• Issued: 01/15/2019
• Est. Priority Date: 09/11/2015
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Gupta, Manish, Chintalapoodi, Pramod, Tanaka, Sho
• Primary Examiner(s): Mahne, Kevin P
• Application Number:
  US14/851,201
• Publication Number:
  US 20170076606A1
• Time in Patent Office: 1,222 Days
• Field of Search: None
• CPC Class Codes: B60K 2360/186 Displaying information according to relevancy
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2360/334 Projection means
• CPC Class Codes: B60K 2360/347 Optical elements for superposition of display information
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/29 Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60Q 1/0023 the element being a sensor, e.g. distance sensor, camera
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

Various aspects of a system and method to provide driving assistance are disclosed herein. The system comprises one or more circuits in an electronic control unit used in a vehicle configured to detect one or more actions associated with the vehicle based on one or more signals generated by one or more sensors embedded in the vehicle. The electronic control unit may be further configured to control the activation of an imaging device located on the vehicle at an opposite side of a driver of the vehicle, based on the detected one or more actions. The imaging device is operable to capture one or more images in a forward-facing direction of the vehicle.

Claims

1. A system for driving assistance, said system comprising:

circuitry in an electronic control unit in a first vehicle, said circuitry configured to;

detect an action associated with said first vehicle based on a signal generated by a sensor embedded in said first vehicle; and

control activation of a camera located on said first vehicle based on said detected action,

wherein said camera captures an image in a forward-facing direction of said first vehicle,

wherein said camera is located on said first vehicle at a first side opposite to a second side of a driver of said first vehicle,

wherein said activation of said camera corresponds to rotation of said camera from a rearward-facing direction of said first vehicle to said forward-facing direction of said first vehicle based on said detected action, and

wherein said detected action corresponds to a first change in a steering angle of said first vehicle above a first threshold angle.

2. The system according to claim 1, wherein said circuitry is further configured to control said activation of said camera in an operational mode,

wherein said operational mode corresponds to one of a first mode or a second mode,

wherein said first mode and said second mode are based on at least one of said detected action or said image captured by said camera, and

wherein

in said first mode, said first vehicle surpasses a second vehicle in front of said first vehicle, and

in said second mode, said first vehicle turns at a first intersection in a direction of motion of said first vehicle.

3. The system according to claim 2, wherein said circuitry is further configured to:

control said activation of said camera to capture said image in said forward-facing direction for said first mode; and

detect said second vehicle in front of said first vehicle based on said detected action,

wherein said detected action comprises at least one of a deceleration of said first vehicle below a speed threshold for activation of said first mode, selection of a manual button for said activation of said first mode, lateral displacement of said first vehicle above a threshold distance for said activation of said first mode, push of a brake pedal of said first vehicle, activation of a turn signal of said first vehicle, a presence of a third vehicle within a specific distance from said first vehicle, a blockage in a view in said forward-facing direction of said first vehicle for said driver of said first vehicle, reduction in a speed of said second vehicle to a value below said speed threshold for said activation of said first mode, or said first change in said steering angle of said first vehicle above said first threshold angle for said activation of said first mode.

4. The system according to claim 2, wherein said circuitry is further configured to:

control said activation of said camera to capture said image in at least one of said rearward-facing direction or a sideward-facing direction of said first vehicle for said second mode; and

detect at least one of an obstacle or a life-form based on said detected action,

wherein said detected action comprises at least one of activation of a turn signal of said first vehicle, selection of a manual button for activation of said second mode, detection of a lane change by said first vehicle, detection of said first intersection in said direction of motion of said first vehicle, detection of a turn at a second intersection on an upcoming driving route set in a navigation application for said first vehicle, detection of reduction in a speed of said first vehicle to a value below a speed threshold for said activation of said second mode, detection of a second change in said steering angle of said first vehicle above a second threshold angle for said activation of said second mode, or detection of a traffic signal.

5. The system according to claim 1, wherein said circuitry is further configured to at least one of control a display of said image on a display screen of said first vehicle or display an alert indication with said image,

wherein said alert indication is displayed based on detection of at least one of a second vehicle or a life-form, wherein said second vehicle or said life-form is detected based on said image.

6. The system according to claim 1, wherein said activation of said camera further corresponds to adjustment of a line-of-sight of said camera.

7. The system according to claim 1, wherein said circuitry is further configured to control adjustment of a line-of-sight of said activated camera based on road surface characteristics of a road surface beneath said first vehicle,

wherein said road surface characteristics comprise at least one of an upward slope, a downward slope, a bank angle, a curvature, a boundary, a speed limit, a road texture, a pothole, or a lane marking.

8. The system according to claim 1, wherein said circuitry is further configured to control said activation of said camera based on at least one of a voice based command, a gesture-based command, or selection of a manual button on one of a panel or a steering wheel of said first vehicle.

9. The system according to claim 1, wherein said circuitry is further configured to control display of said image based on one of an infotainment display, a heads-up display (HUD), an augmented reality (AR)-HUD, a driver information console (DIC), a see-through display, or a smart-glass display.

10. The system according to claim 1, wherein said circuitry is further configured to deactivate said activated camera based on at least one of an acceleration of said first vehicle above a speed threshold for a mode deactivation, a lateral displacement of said first vehicle below a threshold distance for said mode deactivation, an absence of a fifth vehicle from said image, a second change in said lateral displacement with respect to a time of activation of said camera, a third change in said steering angle of said first vehicle below a third threshold angle for said mode deactivation, detection of completion of a turning action of said first vehicle at an intersection, deactivation of a turn signal, or selection of a manual button for said deactivation.

11. The system according to claim 1, wherein said circuitry is further configured to generate information indicating whether said first vehicle is capable to safely pass a second vehicle in front of said first vehicle,

wherein said information is generated based on said activation of said camera,

wherein said generated information comprises at least one of visual information, haptic information, or audio information.

12. The system according to claim 1, wherein said circuitry is further configured to detect a third vehicle within a specific distance from said first vehicle based on at least one of said camera, a radio wave-based object detection device, a laser-based object detection device, or a wireless communication device.

13. The system according to claim 1, wherein said camera is one of mounted on or integrated into one of a passenger side rear-view mirror or an external body of said first vehicle.

14. The system according to claim 1, wherein said camera is adjustably mounted on one of a passenger side rear-view mirror or an external body of said first vehicle, wherein said camera captures said image in at least one of said rearward-facing direction or a sideward-facing direction of said first vehicle.

15. The system according to claim 1, wherein said camera one of extends outward from an external body of said first vehicle or retracts into said external body of said first vehicle, based on said first change in said steering angle of said first vehicle above said first threshold angle.

16. A method for driving assistance, said method comprising:

detecting, by an electronic control unit in a first vehicle, an action associated with said first vehicle based on a signal generated by a sensor embedded in said first vehicle; and

controlling, by said electronic control unit in said first vehicle, activation of a camera located on said first vehicle based on said detected action,

wherein said camera captures an image in a forward-facing direction of said first vehicle,

wherein said camera is located on said first vehicle at a first side opposite to a second side of a driver of said first vehicle,

wherein said activation of said camera corresponds to rotation of said camera from a rearward-facing direction of said first vehicle to said forward-facing direction of said first vehicle based on said detected action, and

wherein said detected action corresponds to a change in a steering angle of said first vehicle above a threshold angle.

17. The method according to claim 16, further comprising:

controlling said activation of said camera in an operational mode,

wherein said operational mode corresponds to one of a first mode or a second mode, and

wherein said first mode and said second mode are based on at least one of said detected action or said image captured by said camera,

wherein in said first mode, said first vehicle surpasses a second vehicle in front of said first vehicle, and

in said second mode, said first vehicle turns at a first intersection in a direction of motion of said first vehicle.

18. The method according to claim 16, further comprising at least one of controlling a display of said image on a display screen of said first vehicle or displaying an alert indication with said image,

wherein said alert indication is displayed based on detection of at least one of a third vehicle or a life-form, wherein said third vehicle or said life-form is detected based on said image.

19. The method according to claim 16, wherein said activation of said camera further corresponds to adjustment of a line-of-sight of said camera.

20. A non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed by a processor, cause a computer to execute operations, said operations comprising:

detecting, by an electronic control unit in a vehicle, an action associated with said vehicle based on a signal generated by a sensor embedded in said vehicle; and

controlling, by said electronic control unit in said vehicle, activation of a camera located on said vehicle based on said detected action,

wherein said camera captures an image in a forward-facing direction of said vehicle,

wherein said camera is located on said vehicle at a first side opposite to a second side of a driver of said vehicle,

wherein said activation of said camera corresponds to rotation of said camera from a rearward-facing direction of said vehicle to said forward-facing direction of said vehicle based on said detected action, and

wherein said detected action corresponds to a change in a steering angle of said vehicle above a threshold angle.

21. A vehicle, comprising:

a battery;

a sensor configured to generate a signal associated with said vehicle;

a camera configured to be powered by said battery and capture an image in a forward-facing direction of said vehicle,

wherein said camera is located on said vehicle at a first side opposite to a second side of a driver of said vehicle, and

wherein activation of said camera corresponds to rotation of said camera from a rearward-facing direction of said vehicle to said forward-facing direction of said vehicle based on an action associated with said vehicle, and

wherein said action corresponds to a change in a steering angle of said vehicle above a threshold angle;

a display configured to render said image; and

circuitry in an electronic control unit (ECU), said circuitry configured to;

detect said action associated with said vehicle based on said signal generated by said sensor embedded in said vehicle; and

control said activation of said camera located on said vehicle based on at least one of said detected action or said image.