Handover procedure for driver of autonomous vehicle

• US 10,496,090 B2
• Filed: 09/27/2017
• Issued: 12/03/2019
• Est. Priority Date: 09/29/2016
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Latotzki, Ralph, Goseberg, Frank
• Primary Examiner(s): Elchanti, Hussein
• Assistant Examiner(s): Berns, Michael A
• Application Number:
  US15/716,718
• Publication Number:
  US 20180088574A1
• Time in Patent Office: 797 Days
• CPC Class Codes: B60G 2400/823 Obstacle sensing
• CPC Class Codes: B60W 2540/043 Identity of occupants
• CPC Class Codes: B60W 2540/045 Occupant permissions
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 2540/221 Physiology, e.g. weight, heartbeat, health or special needs
• CPC Class Codes: B60W 2555/20 Ambient conditions, e.g. wind or rain
• CPC Class Codes: B60W 30/095 Predicting travel path or likelihood of collision
• CPC Class Codes: B60W 60/0053 from vehicle to occupant
• CPC Class Codes: B60W 60/0057 Estimation of the time available or required for the handover
• CPC Class Codes: B60W 60/007 Emergency override Handing over between remote control and on-board control or handing over between remote control arrangements G05D1/227
• CPC Class Codes: G05D 1/0061 for transition from automatic pilot to manual pilot and vice versa
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/2435 Extracting 3D information
• CPC Class Codes: G05D 1/249 from positioning sensors located off-board the vehicle, e.g. from cameras
• CPC Class Codes: G05D 1/81 Handing over between on-board automatic and on-board manual control

Abstract

A control system that is operable to control a vehicle in an autonomous or semi-autonomous mode includes a processor that processes data captured by a plurality of exterior sensing sensors. When the control system is operating in the autonomous or semi-autonomous mode and responsive to a determination of an upcoming event that requires the system to hand over control of the vehicle to a driver before the vehicle encounters the event, the control determines (i) a total action time until the vehicle encounters the event, (ii) an estimated time for the driver to take over control and (iii) an estimated handling time for the vehicle to be controlled before the vehicle encounters the event. Responsive to the determinations, the control system (i) allows the driver to take over control of the vehicle or (ii) controls the vehicle to slow down and stop the vehicle before the vehicle encounters the event.

Claims

1. A vehicle control system for a vehicle that is operable to control the vehicle in an autonomous or semi-autonomous mode, said vehicle control system comprising:

a plurality of sensors disposed at the vehicle and having fields of sensing exterior of the vehicle;

a processor operable to process data captured by said sensors;

wherein, when said vehicle control system is operating in the autonomous or semi-autonomous mode and responsive to a determination of a triggering event that requires the vehicle control system to hand over control of the vehicle to a driver of the vehicle before the vehicle encounters an event point, said vehicle control system determines (i) a total action time until the vehicle encounters the event point, (ii) an estimated time for the driver to take over control of the vehicle and (iii) an estimated handling time for the vehicle to be controlled before the vehicle encounters the event point; and

wherein, responsive to the determined total action time, estimated driver take over time and estimated handling time, said vehicle control system one of (i) allows the driver to take over control of the vehicle and (ii) controls the vehicle to slow down and stop the vehicle before the event point.

2. The vehicle control system of claim 1, wherein, responsive to the estimated driver takeover time being greater than the difference between the total action time and the estimated handling time, said vehicle control system controls the vehicle to slow down and stop the vehicle.

3. The vehicle control system of claim 2, wherein a location where the vehicle is slowing down and stopping at is an emergency stopping lane or a shoulder of a road the vehicle is travelling along.

4. The vehicle control system of claim 2, wherein a location where the vehicle is slowing down and stopping at is a parking bay or parking lot.

5. The vehicle control system of claim 1, wherein, responsive to the estimated driver takeover time being less than the difference between the total action time and the estimated handling time, said vehicle control system allows the driver to take over control of the vehicle.

6. The vehicle control system of claim 5, wherein, responsive to the driver not taking over control of the vehicle by the difference between the total action time and the estimated handling time, said vehicle control system controls the vehicle to slow down and stop the vehicle.

7. The vehicle control system of claim 6, wherein a location where the vehicle is slowing down and stopping at is an emergency stopping lane or a shoulder of a road the vehicle is travelling along.

8. The vehicle control system of claim 6, wherein a location where the vehicle is slowing down and stopping at is a parking bay or parking lot.

9. The vehicle control system of claim 1, wherein the event point comprises an event that cannot be handled by said vehicle control system when said vehicle control system is operating in the autonomous or semi-autonomous mode.

10. The vehicle control system of claim 1, wherein the triggering event comprises detection of an object ahead of the vehicle and in a current path of travel of the vehicle, and wherein the event point comprises the detected object.

11. The vehicle control system of claim 1, wherein the triggering event is a determination of a hazardous driving condition ahead of the vehicle, and wherein the event point is a location of the determined hazardous driving condition.

12. The vehicle control system of claim 1, wherein the triggering event is a determination of an error in said vehicle control system that may affect said control system'"'"'s ability to control the vehicle in the autonomous or semi-autonomous mode, and the event point is a time or location by which the vehicle should be stopped.

13. A vehicle control system for a vehicle that is operable to control the vehicle in an autonomous or semi-autonomous mode, said vehicle control system comprising:

a plurality of sensors disposed at the vehicle and having fields of sensing exterior of the vehicle, wherein said plurality of sensors comprises a plurality of cameras disposed at the vehicle and a plurality of radar sensors disposed at the vehicle;

a processor operable to process data captured by said sensors;

wherein said vehicle control system determines an upcoming event that cannot be handled by said vehicle control system when said vehicle control system is operating in the autonomous or semi-autonomous mode;

wherein, when said vehicle control system is operating in the autonomous or semi-autonomous mode and responsive to a determination of the upcoming event that requires the vehicle control system to hand over control of the vehicle to a driver of the vehicle before the vehicle encounters the event, said control determines (i) a total action time until the vehicle encounters the event, (ii) an estimated time for the driver to take over control of the vehicle and (iii) an estimated handling time for the vehicle to be controlled before the vehicle encounters the event; and

wherein, responsive to the determined total action time, estimated driver take over time and estimated handling time, said vehicle control system one of (i) allows the driver to take over control of the vehicle and (ii) controls the vehicle to slow down and stop the vehicle before the vehicle encounters the event.

14. The vehicle control system of claim 13, wherein, responsive to the estimated driver takeover time being greater than the difference between the total action time and the estimated handling time, said vehicle control system controls the vehicle to slow down and stop the vehicle.

15. The vehicle control system of claim 13, wherein, responsive to the estimated driver takeover time being less than the difference between the total action time and the estimated handling time, said vehicle control system allows the driver to take over control of the vehicle.

16. The vehicle control system of claim 15, wherein, responsive to the driver not taking over control of the vehicle by the difference between the total action time and the estimated handling time, said vehicle control system controls the vehicle to slow down and stop the vehicle.

17. The vehicle control system of claim 13, wherein the event comprises a hazardous driving condition ahead of the vehicle.

18. The vehicle control system of claim 13, wherein the event comprises an imminent failure in said vehicle control system that may affect said control system'"'"'s ability to control the vehicle in the autonomous or semi-autonomous mode.

19. A vehicle control system for a vehicle that is operable to control the vehicle in an autonomous or semi-autonomous mode, said vehicle control system comprising:

a plurality of sensors disposed at the vehicle and having fields of sensing exterior of the vehicle, wherein said plurality of sensors comprises a plurality of cameras disposed at the vehicle and a plurality of radar sensors disposed at the vehicle;

a processor operable to process data captured by said sensors;

wherein said vehicle control system determines an upcoming event that cannot be handled by said vehicle control system when said vehicle control system is operating in the autonomous or semi-autonomous mode;

wherein, when said vehicle control system is operating in the autonomous or semi-autonomous mode and responsive to a determination of the upcoming event that requires the vehicle control system to hand over control of the vehicle to a driver of the vehicle before the vehicle encounters the event, said control determines (i) a total action time until the vehicle encounters the event, (ii) an estimated time for the driver to take over control of the vehicle and (iii) an estimated handling time for the vehicle to be controlled before the vehicle encounters the event;

wherein, responsive to the determined total action time, estimated driver take over time and estimated handling time, said vehicle control system one of (i) allows the driver to take over control of the vehicle and (ii) controls the vehicle to slow down and stop the vehicle before the vehicle encounters the event;

wherein, responsive to the estimated driver takeover time being greater than the difference between the total action time and the estimated handling time, said vehicle control system controls the vehicle to slow down and stop the vehicle;

wherein, responsive to the estimated driver takeover time being less than the difference between the total action time and the estimated handling time, said vehicle control system allows the driver to take over control of the vehicle; and

wherein, responsive to said control allowing the driver to take over control of the vehicle and responsive to the driver not taking over control of the vehicle by the difference between the total action time and the estimated handling time, said vehicle control system controls the vehicle to slow down and stop the vehicle.

20. The vehicle control system of claim 19, wherein the event comprises one of (i) a hazardous driving condition ahead of the vehicle and (ii) an imminent failure in said vehicle control system that may affect said control system'"'"'s ability to control the vehicle in the autonomous or semi-autonomous mode.