Systems and methods for implementing multimodal safety operations with an autonomous agent

• US 10,803,745 B2
• Filed: 01/30/2020
• Issued: 10/13/2020
• Est. Priority Date: 07/24/2018
• Status: Active Grant

• Current Assignee: May Mobility, Inc.
• Original Assignee: May Mobility, Inc.
• Inventors: Vozar, Steve, Olson, Edwin, Messenger, Sean M., Johnson, Collin
• Primary Examiner(s): Schneider, Paula L
• Application Number:
  US16/776,993
• Publication Number:
  US 20200168085A1
• Time in Patent Office: 257 Days
• Field of Search: 701 24
• CPC Class Codes: B60W 2050/0016 State machine analysis
• CPC Class Codes: B60W 2050/0292 Fail-safe or redundant systems, e.g. limp-home or backup systems
• CPC Class Codes: B60W 2556/45 External transmission of data to or from the vehicle
• CPC Class Codes: B60W 50/0205 Diagnosing or detecting failures; Failure detection models
• CPC Class Codes: B60W 50/029 Adapting to failures or work around with other constraints, e.g. circumvention by avoiding use of failed parts
• CPC Class Codes: B60W 60/00186 related to the vehicle
• CPC Class Codes: G05D 1/0055 with safety arrangements
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G08G 1/0116 from roadside infrastructure, e.g. beacons
• CPC Class Codes: G08G 1/0133 for classifying traffic situation
• CPC Class Codes: G08G 1/096725 where the received information generates an automatic action on the vehicle control
• CPC Class Codes: G08G 1/096783 where the origin of the information is a roadside individual element

Abstract

A system and method includes an autonomous agent having a communication interface that enables the autonomous agent to communicate with a plurality of infrastructure sensing devices; a plurality of distinct health monitors that monitor distinct operational aspects of the autonomous agent; an autonomous state machine that computes a plurality of allowed operating states of the autonomous agent based on inputs from the plurality of distinct health monitors; a plurality of distinct autonomous controllers that generate a plurality of distinct autonomous control instructions; and an arbiter of autonomous control instructions that: collects, as a first input, the plurality of autonomous control instructions generated by each of the plurality of distinct autonomous controllers; collects, as a second input, data relating to the plurality of allowed operating state of the autonomous agent; and selectively enables only a subset of the autonomous control instructions to pass to driving components of the autonomous agent.

Claims

1. A system for intelligently implementing an autonomous agent, the system comprising:

the autonomous agent, wherein a computing system of the autonomous agent computes a set of allowed operating states of the autonomous agent;

a set of autonomous controllers that each generate a set of autonomous control instructions; and

an arbiter of autonomous control instructions comprising selection circuitry, wherein the arbiter of autonomous control instructions;

collects, as a first input, the set of autonomous control instructions generated by each of the set of autonomous controllers;

collects, as a second input, a level of operability of the autonomous agent, wherein the level of operability is determined based on the set of allowed operating states of the autonomous agent;

switches between the sets of autonomous control instructions based on the level of operability; and

selectively enables only a subset of the set of autonomous control instructions to pass to one or more driving components of the autonomous agent thereby controlling operations of the one or more driving components.

2. The system of claim 1, wherein the autonomous agent further comprises a set of health monitors.

3. The system of claim 2, wherein each of the set of health monitors monitors one or more distinct operational aspects of the autonomous agent.

4. The system of claim 1, wherein the autonomous agent further comprises a communication interface.

5. The system of claim 4, wherein the communication interface enables the autonomous agent to communicate with at least one of a set of infrastructure sensing devices.

6. The system of claim 4, wherein the computing system computes the set of allowed operating states of the autonomous agent based on inputs from one or more of a set of distinct health monitors.

7. The system of claim 1, further comprising a set of infrastructure sensing devices.

8. The system of claim 7, wherein each of the set of infrastructure sensing devices generates subjective health status data.

9. The system of claim 8, wherein the subjective health status data is generated based on introspective evaluations of at least one of:

one or more operations of one or more systems of a respective infrastructure sensing device of the set of infrastructure devices.

10. The system of claim 1, wherein each of the set of autonomous controllers:

collects data streams of observation data from a set of distinct data sources;

operates asynchronously to simulate one or more trajectory outcomes for the autonomous agent based on the observation data; and

generates autonomous control instructions based on a simulation having a highest probability of occurring.

11. The system of claim 1, wherein the set of allowed operating states of the autonomous agent includes a computed value that indicates an extent to which capabilities of the autonomous agent are operable or not operable.

12. The system of claim 1, wherein the arbiter of autonomous control instructions blocks a portion of the set of autonomous control instructions from passing to the one or more driving components of the autonomous agent based on the level of operability.

13. A method for intelligently implementing an autonomous agent, the method comprising:

collecting streams of data from one or more data sources in communication with a computing system of an autonomous agent;

generating, by a set of health monitors, autonomous health intelligence data indicating a quality of operation of one or more operational aspects of the autonomous agent, wherein each of the set of health monitors is mapped to expected health values, wherein the expected health values specify criteria that the one or more data sources should satisfy for a given level of operability of the autonomous agent;

identifying a level of operability of the autonomous agent based on the autonomous health intelligence data;

computing by a set of autonomous control planning modules a set of autonomous control instructions; and

controlling the autonomous agent by selectively passing a subset of the set of autonomous control instructions based on the identified level of operability.

14. The method of claim 13, wherein the level of operability relates to one of a set of differentiated levels of autonomous operation identified within an autonomous operation gradient.

15. The method of claim 14, wherein the autonomous operation gradient includes a range of distinct levels of autonomous operation of an autonomous agent ranging from fully autonomous operation to at least one of limited autonomous operation and no autonomous operation.

16. The method of claim 14, wherein each of the set of differentiated levels of autonomous operation is mapped to a predetermined set of minimum capabilities of an autonomous agent.

17. The method of claim 16, wherein the predetermined set of minimum capabilities indicates the minimally required capabilities for maintaining an associated level of autonomous operation.

18. The method of claim 13, wherein controlling the autonomous agent includes deciding which of the set of autonomous control instructions from each of the set of autonomous control planning modules to allow as command instructions for controlling at least one of driving components and driving processes of the autonomous agent based on the identified level of operability.

19. The method of claim 13, wherein the one or more data sources include one or more onboard sensors of the autonomous agent and one or more offboard infrastructure devices arranged along a route and in proximity of an area of operation of the autonomous agent.

20. The method of claim 13, wherein generating the autonomous health intelligence data includes evaluating the streams of data against one or more failure modes, wherein the one or more failure modes identify circumstances in which it is likely that a data source associated with the autonomous agent is in a failing state.