Autonomous vehicle interface system

• US 9,915,950 B2
• Filed: 12/31/2014
• Issued: 03/13/2018
• Est. Priority Date: 12/31/2013
• Status: Active Grant

• Current Assignee: VAY Technology GmbH
• Original Assignee: Polysync Technologies, Inc.
• Inventors: Hartung, Joshua John, Lamb, Jonathan, Miller, David Paul, Hambrick, Robert Dean
• Primary Examiner(s): Melton, Todd
• Assistant Examiner(s): Roberson, Jason R
• Application Number:
  US14/588,313
• Publication Number:
  US 20150331422A1
• Time in Patent Office: 1,168 Days
• Field of Search: None
• CPC Class Codes: G05D 1/02 Control of position or course in two dimensions
• CPC Class Codes: G05D 1/021 specially adapted to land vehicles
• CPC Class Codes: G05D 1/0231 using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G06F 1/10 Distribution of clock signals , e.g. skew
• CPC Class Codes: G06F 1/12 Synchronisation of different clock signals provided by a plurality of clock generators
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: H04L 12/40 Bus networks
• CPC Class Codes: H04L 12/40013 Details regarding a bus controller
• CPC Class Codes: H04Q 9/00 Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom

Abstract

In embodiments of an autonomous vehicle interface system, system nodes are each implemented as a distributed node for independent data processing of low-level sensor data and/or high-level system data. The high-level system data is abstracted from the low-level sensor data, providing invariance to system configuration in higher-level processing algorithms. The autonomous vehicle interface system includes at least one real-time bus for data communications of the low-level sensor data and the high-level system data between the system nodes. The system also includes an application programming interface (API) configured for access by the system nodes to the low-level sensor data and the high-level system data that is published on the real-time bus and accessible via the API.

Claims

1. An autonomous vehicle comprising:

a plurality of sensors configured to detect conditions in an environment of the autonomous vehicle and generate low-level sensor data indicative of the detected conditions;

an autonomous vehicle interface system comprising;

system nodes of the autonomous vehicle that are each a distributed node of the autonomous vehicle interface system, each of the system nodes having processing hardware and memory to independently process the low-level sensor data generated by one or more of the sensors or process high-level system data, the low-level sensor data abstracted to generate the high-level system data as a sensor data type of the respective one or more sensors, the abstracted sensor data type being invariant to specific sensor implementations; and

at least one on-vehicle real-time bus that connects the system nodes of the autonomous vehicle for data communications of the low-level sensor data and the high-level system data, each of the system nodes configured to publish the low-level sensor data or the high-level system data from a respective system node on the real-time bus, the published low-level sensor data and the high-level system data being exposed to each of the system nodes over the on-vehicle real-time bus for processing and vehicle control of the autonomous vehicle without an acting central managing computer managing the published low-level sensor data and the high-level system data that is exposed to the system nodes; and

one or more autonomous vehicle control systems configured to control operations of the autonomous vehicle within the environment based on the low-level sensor data and the high-level system data published on the real-time bus.

2. The autonomous vehicle as recited in claim 1, wherein:

the processing hardware of each of the system nodes is further configured to implement a node control module to;

detect a new system node implemented in the autonomous vehicle interface system; and

add the new system node as an independent, distributed node of the autonomous vehicle interface system.

3. The autonomous vehicle as recited in claim 2, wherein the new system node is a proprietary application of a vehicle manufacturer, and the new system node is connected to the on-vehicle real-time bus for access to the data communications of the low-level sensor data and the high-level system data.

4. The autonomous vehicle as recited in claim 1, wherein the autonomous vehicle interface system is configured according to an open source transport protocol that supports the data communications of the low-level sensor data and the high-level system data over the on-vehicle real-time bus.

5. The autonomous vehicle as recited in claim 1, wherein the system nodes of the autonomous vehicle are further configured to execute applications, each application is associated with at least one of a plurality of security levels, including at least one of:

a first security level that provides the applications in the first security level control over at least one vehicle actuator;

a second security level that provides the applications in the second security level chassis control access for path planning though the environment;

a third security level that provides the applications in the third security level control over at least one of an onboard sensing driver or a sensing application;

or

a fourth security level that provides the applications in the fourth security level control over at least one of onboard entertainment or driver information for output via the autonomous vehicle.

6. The autonomous vehicle as recited in claim 1, wherein:

one or more of the plurality of sensors is configured to detect targets in the environment; and

the processing hardware of at least one of the system nodes is further configured to implement a node control module to;

receive the low-level sensor data from the one or more sensors;

abstract the received low-level sensor data to generate the high-level system data; and

publish the high-level system data via the on-vehicle real-time bus.

7. The autonomous vehicle as recited in claim 6, wherein the node control module is further configured to:

receive timestamps associated with the low-level sensor data from the respective sensors; and

correct all of the received timestamps to correlate with a synchronized system time for a unified time domain.

8. The autonomous vehicle as recited in claim 6, wherein the node control module is further configured to:

detect a new sensor connected to the autonomous vehicle;

determine a type and model of the new sensor based on a database of system supported sensors; and

add the new sensor as a component of the autonomous vehicle.

9. The autonomous vehicle as recited in claim 1, wherein:

one or more of the plurality of sensors is configured to detect targets in the environment; and

the processing hardware of at least one of the system nodes is further configured to implement a node control module to;

receive the high-level system data from one or more of the system nodes via the on-vehicle real-time bus, the high-level system data representative of the low-level sensor data in abstracted data form;

determine by data fusion the targets in the environment surrounding the autonomous vehicle based on the high-level system data; and

publish the determined targets via the on-vehicle real-time bus.

10. The autonomous vehicle as recited in claim 1, wherein:

one or more of the plurality of sensors is configured to detect targets in the environment; and

the processing hardware of at least one of the system nodes is further configured to implement a node control module to;

receive the low-level sensor data from the one or more sensors; and

log to maintain the received low-level sensor data.

11. The autonomous vehicle as recited in claim 1, wherein:

one or more of the plurality of sensors is configured to detect targets in the environment;

the memory of each of the system nodes is configured to maintain a complete copy of information enabling redistribution of the autonomous vehicle interface system across different said system nodes; and

the processing hardware of each of the system nodes is further configured to implement a node control module to;

detect a failure of the one or more sensors resulting in a loss of target detection by the failed sensor; and

determine whether the sensors that remain operational can replace the loss of target detection by the failed sensor based on the complete copy of the autonomous vehicle interface system.

12. The autonomous vehicle as recited in claim 1, wherein:

the memory of each of the system nodes is configured to maintain a complete copy of information enabling redistribution of the autonomous vehicle interface system across different said system nodes; and

the processing hardware of each of the system nodes is further configured to implement a node control module to;

detect a failure of one of the system nodes resulting in a loss of processing operations by the failed system node; and

determine at least one of the system nodes that remains operational as a replacement system node for the failed system node, the replacement system node configured to perform the processing operations of the failed system node based on the complete copy of the autonomous vehicle interface system.

13. The autonomous vehicle as recited in claim 1, wherein the processing hardware of each of the system nodes is further configured to implement a node control module to:

detect a communication failure of the data via a communication link of the on-vehicle real-time bus;

determine a multi-path solution to re-route the data within the autonomous vehicle interface system; and

said re-route the data via an alternate communication link of the on-vehicle real-time bus.

14. A method implemented by an autonomous vehicle, the method comprising:

detecting conditions in an environment of the autonomous vehicle by a plurality of sensors of the autonomous vehicle;

generating low-level sensor data by the plurality of sensors that is indicative of the detected conditions;

abstracting the low-level sensor data to generate high-level system data, the low-level sensor data being abstracted to generate the high-level system data as a sensor data type by system nodes that are each a distributed node of an autonomous vehicle interface system of the autonomous vehicle, the abstracted sensor data type being invariant to specific sensor implementations;

publishing the low-level sensor data and the high-level system data to an on-vehicle real time bus that connects the system nodes of the autonomous vehicle for data communications, the low-level sensor data and the high-level system data that is published by a respective system node over the on-vehicle real-time bus being exposed to the other system nodes for processing and vehicle control of the autonomous vehicle without an acting central managing computer managing the published low-level sensor data and the high-level system data that is exposed; and

controlling operations of the autonomous vehicle within the environment based on the low-level sensor data and the high-level system data published to the on-vehicle real-time bus.

15. The method as recited in claim 14, further comprising:

detecting a new system node implemented in the autonomous vehicle interface system; and

adding the new system node as an independent, distributed node of the autonomous vehicle interface system.

16. The method as recited in claim 15, further comprising connecting a proprietary application of a vehicle manufacturer as the new system node to the on-vehicle real-time bus for exposure to the data communications of the low-level sensor data and the high-level system data.

17. The method as recited in claim 15, further comprising connecting a path planning algorithm as the new system node to the on-vehicle real-time bus for exposure to the data communications of the high-level system data to determine an autonomous vehicle path based on the high-level system data.

18. The method as recited in claim 14, further comprising:

receiving timestamps associated with the low-level sensor data from the respective sensors; and

correcting all of the received timestamps to correlate with a synchronized system time for a unified time domain.

19. The method as recited in claim 14, further comprising:

receiving the high-level system data from one or more of the system nodes via the on-vehicle real-time bus, the high-level system data representative of the low-level sensor data in abstracted data form;

determining by data fusion targets in the environment surrounding the autonomous vehicle based on the high-level system data; and

communicating the determined targets to all of the system nodes via the real time bus.

20. An autonomous vehicle interface system node that is onboard an autonomous vehicle having a plurality of sensors to detect conditions in an environment surrounding the autonomous vehicle, the autonomous vehicle interface system node comprising:

a memory and a processing system to implement a node control module to;

receive low-level sensor data from one or more of the sensors;

abstract the low-level sensor data to generate high-level system data, the low-level sensor data from a sensor being abstracted to generate the high-level system data as a sensor data type, the abstracted sensor data type being invariant to specific sensor implementations; and

publish the low-level sensor data and the high-level system data over an on-vehicle real time bus of the autonomous vehicle by the system node, the on-vehicle real-time bus configured to expose data communications published by the system node to additional, distributed system nodes of the autonomous vehicle interface system without an acting central managing computer managing the published low-level sensor data and the high-level system data that is exposed, the operations of the autonomous vehicle controlled within the environment based, in part, on the low-level sensor data and the high-level system data published to the on-vehicle real-time bus.

21. The autonomous vehicle interface system node as recited in claim 20, wherein the node control module is further configured to independently process the low-level sensor received from the one or more sensors or process the high-level system data.

22. The autonomous vehicle interface system node as recited in claim 20, wherein the node control module is further configured to:

detect a new system node implemented in the autonomous vehicle interface system; and

add the new system node as an independent, distributed node of the autonomous vehicle interface system.

23. The autonomous vehicle interface system node as recited in claim 22, wherein the new system node is a proprietary application for an automobile, and the new system node is connected to the on-vehicle real-time bus for access to the data communications of the low-level sensor data and the high-level system data.

24. The autonomous vehicle interface system node as recited in claim 20, wherein:

one or more of the plurality of sensors is configured to detect targets in the environment; and

the node control module is further configured to;

determine by data fusion the targets in the environment surrounding the autonomous vehicle based on the high-level system data; and

publish the determined targets via the on-vehicle real-time bus.

25. The autonomous vehicle interface system node as recited in claim 20, wherein:

one or more of the plurality of sensors is configured to detect targets in the environment;

the memory is further configured to maintain a complete copy of information enabling redistribution of the autonomous vehicle interface system across different system nodes; and

the node control module is further configured to;

detect a failure of the one or more sensors resulting in a loss of target detection by the failed sensor; and

determine whether the sensors that remain operational can replace the loss of target detection by the failed sensor based on the complete copy of the autonomous vehicle interface system.