SENSOR FUSION FOR AUTONOMOUS DRIVING TRANSITION CONTROL

• US 20180107214A1
• Filed: 10/17/2016
• Published: 04/19/2018
• Est. Priority Date: 10/17/2016
• Status: Active Grant

• Current Assignee: Steering Solutions IP Holding Corporation (Nexteer Automotive Group Ltd.)
• Original Assignee: Steering Solutions IP Holding Corporation (Nexteer Automotive Group Ltd.)
• Inventors: Chandy, Ashok
• Granted Patent: US 10,481,602 B2
• Time in Patent Office: Days
• CPC Class Codes: B60K 2360/175 Autonomous driving
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2510/20 Steering systems
• CPC Class Codes: B60W 2510/202 Steering torque
• CPC Class Codes: B60W 2510/205 Steering speed
• CPC Class Codes: B60W 30/10 Path keeping cruise control for automatically following a preceding vehicle B60W30/165
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: B62D 1/286 Systems for interrupting non-mechanical steering due to driver intervention
• CPC Class Codes: B62D 15/025 Active steering aids, e.g. helping the driver by actively influencing the steering system after environment evaluation B62D1/28 takes precedence
• CPC Class Codes: B62D 5/0424 the axes of motor and final driven element of steering gear, e.g. rack, being parallel
• CPC Class Codes: B62D 6/10 characterised by means for sensing or determining torque
• CPC Class Codes: G05D 1/0061 for transition from automatic pilot to manual pilot and vice versa

Abstract

A system for sensor fusion for autonomous driving transition control includes a sensor fusion module and a decision making module. The sensor fusion module fuses a plurality of steering sensor data from one or more sensors of a steering system with a plurality of driver state data from a plurality of driver state sensors as a plurality of fused state data aligned in time. The decision making module determines whether to transition from an autonomous driving mode to a manual driving mode based on the fused state data.

Claims

1. A system for sensor fusion for autonomous driving transition control, the system comprising:

a sensor fusion module that fuses a plurality of steering sensor data from one or more sensors of a steering system with a plurality of driver state data from a plurality of driver state sensors as a plurality of fused state data aligned in time; and

a decision making module that determines whether to transition from an autonomous driving mode to a manual driving mode based on the fused state data.

2. The system of claim 1, wherein the steering sensor data comprise sensed or derived data from one or more of a handwheel torque sensor and a handwheel position/velocity sensor.

3. The system of claim 1, wherein the driver state data comprise sensed or derived data from one or more of handwheel touch sensor and a camera.

4. The system of claim 3, wherein the driver state data sensed or derived from the handwheel touch sensor comprises one or more of:

an area of contact with a handwheel, a width of contact with the handwheel, a force of contact with the handwheel, and a position of contact with the handwheel, and wherein the driver state data sensed or derived from the camera comprises one or more of;

a driver body posture, a driver head pose, a driver eye gaze, and a driver hand position.

5. The system of claim 1, further comprising a recent data store of the fused state data and a history data store of the fused state data, wherein the history data store is operable to collect older values of the fused state data than collected in the recent data store.

6. The system of claim 5, wherein the decision making module is operable to average multiple samples of the fused state data from the history data store as time-averaged fused state data and determine whether to transition from the autonomous driving mode to the manual driving mode based on the fused state data from the recent data store in combination with the time-averaged fused state data.

7. The system of claim 1, wherein the decision making module is operable to continue monitoring the fused state data after the transition from the autonomous driving mode to the manual driving mode has been initiated and confirm whether to revert to the autonomous driving mode based on transition validity criteria.

8. A steering system comprising:

one or more sensors operable to produce a plurality of steering sensor data;

a plurality of driver state sensors operable to produce a plurality of driver state data;

a steering actuator motor; and

a control module operable to operate the steering actuator motor in an autonomous driving mode and in a manual driving mode, the control module further operable to fuse the steering sensor data with the driver state data as a plurality of fused state data aligned in time and determine whether to transition from the autonomous driving mode to the manual driving mode based on the fused state data.

9. The steering system of claim 8, wherein the steering sensor data comprise sensed or derived data from one or more of a handwheel torque sensor and a handwheel position/velocity sensor.

10. The steering system of claim 8, wherein the driver state data comprise sensed or derived data from one or more of handwheel touch sensor and a camera.

11. The steering system of claim 10, wherein the driver state data sensed or derived from the handwheel touch sensor comprises one or more of:

an area of contact with a handwheel, a width of contact with the handwheel, a force of contact with the handwheel, and a position of contact with the handwheel, and wherein the driver state data sensed or derived from the camera comprises one or more of;

a driver body posture, a driver head pose, a driver eye gaze, and a driver hand position.

12. The steering system of claim 8, wherein the control module comprises a recent data store of the fused state data and a history data store of the fused state data, wherein the history data store is operable to collect older values of the fused state data than collected in the recent data store.

13. The steering system of claim 12, wherein the control module is operable to average multiple samples of the fused state data from the history data store as time-averaged fused state data and determine whether to transition from the autonomous driving mode to the manual driving mode based on the fused state data from the recent data store in combination with the time-averaged fused state data.

14. The steering system of claim 8, wherein the control module is operable to continue monitoring the fused state data after the transition from the autonomous driving mode to the manual driving mode has been initiated and confirm whether to revert to the autonomous driving mode based on transition validity criteria.

15. A method for sensor fusion for autonomous driving transition control, the method comprising:

acquiring, by a control module, a plurality of steering sensor data from one or more sensors of a steering system;

acquiring a plurality of driver state data from a plurality of driver state sensors;

fusing the steering sensor data with the driver state data as a plurality of fused state data aligned in time; and

determining, by the control module, whether to transition from an autonomous driving mode to a manual driving mode based on the fused state data.

16. The method of claim 15, wherein the steering sensor data comprise sensed or derived data from one or more of a handwheel torque sensor and a handwheel position/velocity sensor.

17. The method of claim 15, wherein the driver state data comprise sensed or derived data from one or more of handwheel touch sensor and a camera, the driver state data sensed or derived from the handwheel touch sensor comprises one or more of:

an area of contact with a handwheel, a width of contact with the handwheel, a force of contact with the handwheel, and a position of contact with the handwheel, and wherein the driver state data sensed or derived from the camera comprises one or more of;

a driver body posture, a driver head pose, a driver eye gaze, and a driver hand position.

18. The method of claim 15, further comprising:

storing the fused state data in a recent data store; and

storing the fused state data in the history data store, wherein the history data store collects older values of the fused state data than collected in the recent data store.

19. The method of claim 18, further comprising:

averaging multiple samples of the fused state data from the history data store as time-averaged fused state data; and

determining whether to transition from the autonomous driving mode to the manual driving mode based on the fused state data from the recent data store in combination with the time-averaged fused state data.

20. The method of claim 15, further comprising:

continuing to monitor the fused state data after the transition from the autonomous driving mode to the manual driving mode has been initiated; and

confirming whether to revert to the autonomous driving mode based on transition validity criteria.