Swarm system including an operator control section enabling operator input of mission objectives and responses to advice requests from a heterogeneous multi-agent population including information fusion, control diffusion, and operator infusion agents that controls platforms, effectors, and sensors

• US 11,699,083 B2
• Filed: 06/10/2019
• Issued: 07/11/2023
• Est. Priority Date: 06/08/2018
• Status: Active Grant

• Current Assignee: the united states of america as represented by the secretary of the navy
• Original Assignee: the united states of america as represented by the secretary of the navy
• Inventors: Cruise, Robert Bruce
• Primary Examiner(s): Gonzales, Vincent
• Application Number:
  US16/436,910
• Publication Number:
  US 20200134491A1
• Time in Patent Office: 1,492 Days
• Field of Search: None
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06N 5/043 Distributed expert systems; Blackboards
• CPC Class Codes: G06N 5/048 Fuzzy inferencing
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

Systems and methods are provided relating to a complex adaptive command guided swarm system including an operator section comprising a first command and control section and a plurality of networked swarm of semi-autonomously agent controlled system of systems platforms (SAASoSPs). The first command and control section includes a user interface, computer system, network interface, and plurality of command and control systems executed or running on the computer system. The networked SAASoSPs each include a second command and control section, wherein the second command and control section utilizes artificial intelligence (AI) configured with a combination of both symbolic and probabilistic machine learning for various functions including pattern recognition and new pattern identification. The AI is also configured to combine advice-based learning with active learning, wherein the AI solicits advice from a domain expert user via the first command and control section as necessary during both training and operational stages of the system.

Claims

1. A complex adaptive command guided swarm system comprising:

a first command and control section comprising a user interface, a computer system, a network interface, and a plurality of command and control systems executable on the computer system; and

a plurality of networked swarm of semi-autonomously agent controlled system of systems platforms (SAASoSPs), each of the plurality of SAASoSPs comprising;

a second command and control section;

one or more sensors;

a network communication section configured to communicate with at least the first command and control section and one or more other SAASoSPs of the plurality of SAASoSPs; and

one or more equipment systems configured to create a physical effect on an object or an environment external to the SAASoSP;

wherein the second command and control section includes;

at least one information fusion agent configured to mine and process information that originates in the environment external to the SAASoSP, the information including features including patterns recognized within a set of pixel vectors, objects including patterns recognized within a set of feature vectors, situations including patterns recognized within a set of object state vectors, and impacts or threats including patterns recognized within a set of situation vectors;

at least one control diffusion agent configured to generate control actions based, in part, on the information collected and processed by the at least one information fusion agent and disseminate the control actions within the SAASoSP; and

at least one operator infusion agent configured to receive a range of operator inputs via the first command and control section including at least a range of high level mission objective data;

wherein the second control and command system is configured to operate using active advice-seeking learning logic using a combination of active learning and advice-seeking learning, wherein each SAASoSP of the plurality of SAASoSPs is configured to solicit or query for advice from a human operator via the first control and command system during either training of the complex adaptive command guided swarm system or execution of a mission, including configuration to receive input of advice from the human operator concerning one or more complex domains of interest and/or concerning broad characterizations of entire regions of one or more data input spaces.

2. The complex adaptive command guided swarm system of claim 1, wherein the at least one information fusion agent includes a combination of symbolic machine learning (ML) and probabilistic ML configured for data mining and pattern recognition.

3. The complex adaptive command guided swarm system of claim 2, wherein the combination of symbolic machine learning (ML) and probabilistic ML further includes a combination of Bayesian graphical models with first-order logic.

4. The complex adaptive command guided swarm system of claim 1, wherein the plurality of SAASoSPs are configured to share multiple memories or data stores in a distributed database among the plurality of SAASoSPs using respective parallel processing units, wherein the distributed database includes a plurality of storage devices that are not all attached to a common processor.

5. The complex adaptive command guided swarm system of claim 1, wherein the network communication section is further configured to interact using at least one single instruction, multiple data (SIMD) processing agent across one or more of the plurality of SAASoSPs.

6. The complex adaptive command guided swarm system of claim 1, wherein the at least one information fusion agent is configured to utilize at least one SIMD processing agent to determine a threat or impact assessment based, in part, on the mined and processed information.

7. The complex adaptive command guided swarm system of claim 1, wherein the at least one information fusion agent or at least one control diffusion agent is configured to use statistical relational learning (SRL).

8. The complex adaptive command guided swarm system of claim 1, wherein the SRL includes use of a Markov Logic Network (MLN).

9. The complex adaptive command guided swarm system of claim 1, wherein each SAASoSP includes one or more movement and propulsion systems.

10. The complex adaptive command guided swarm system of claim 9, wherein the movement and propulsion systems includes one or more of wings, a quadcopter, a mobile floating platform, an underwater movement platform, or a space movement platform.

11. The complex adaptive command guided swarm system of claim 1, wherein the one or more equipment systems include one more of a weapon, an actuator, a gripper, a laser, a projectile firing system, a capture system, a drilling system, mining equipment, a measuring system, a sensor, or a material handling system.

12. The complex adaptive command guided swarm system of claim 1, wherein the active advice-seeking learning logic includes statistical relational learning (SRL).

13. The complex adaptive command guided swarm system of claim 1, wherein second control and command system is further configured to received multiple input information and respond to the multiple input information.

14. The complex adaptive command guided swarm system of claim 13, wherein the response by the second control and command system to the multiple input information includes responding to broad characterization by the human operator concerning entire regions of input space by targeting the entire regions of input space using the active advice-seeking learning logic.

15. A method of operating a complex adaptive command guided swarm system comprising:

supplying a first command and control section comprising a user interface, a computer system, a network interface, and a plurality of command and control systems executable on the computer system; and

supplying a plurality of networked swarm of semi-autonomously agent controlled system of systems platforms (SAASoSPs), each of the plurality of SAASoSPs comprising;

a second command and control section;

one or more sensors;

a network communication section configured to communicate with at least the first command and control section and one or more other SAASoSPs of the plurality of SAASoSPs; and

one or more equipment systems configured to create a physical effect on an object or an environment external to the SAASoSP;

wherein the second command and control section is configured to implement;

at least one information fusion agent configured to mine and process information that originates in the environment external to the SAASoSP, the information including features including patterns recognized within a set of pixel vectors, objects including patterns recognized within a set of feature vectors, situations including patterns recognized within a set of object state vectors, and impacts or threats including patterns recognized within a set of situation vectors;

at least one control diffusion agent configured to generate control actions based, in part, on the information collected and processed by the at least one information fusion agent and disseminate the control actions within the SAASoSP; and

at least one operator infusion agent configured to receive a range of operator inputs via the first command and control section including at least a range of high level mission objective data;

wherein the second control and command system is configured to operate using active advice-seeking learning logic using a combination of active learning and advice-seeking learning, wherein each SAASoSP of the plurality of SAASoSPs is configured to solicit or query for advice from a human operator via the first control and command system during either training of the complex adaptive command guided swarm system or execution of a mission, including configuration to receive input of advice from the human operator concerning one or more complex domains of interest and/or concerning broad characterizations of entire regions of one or more data input spaces.

16. The method of claim 15, wherein the at least one information fusion agent implements a combination of symbolic machine learning (ML) and probabilistic ML configured for data mining and pattern recognition.

17. The method of claim 16, wherein the combination of symbolic machine learning (ML) and probabilistic ML further includes a combination of Bayesian graphical models with first-order logic.

18. The method of claim 15, wherein the second control and command system is further configured to receive multiple input information and respond to the multiple input information.