Bio-inspired actionable intelligence method and system

• US 8,515,160 B1
• Filed: 12/17/2008
• Issued: 08/20/2013
• Est. Priority Date: 10/04/2007
• Status: Active Grant

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Khosla, Deepak, Chelian, Suhas E.
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): Dunphy, David F
• Application Number:
  US12/316,941
• Time in Patent Office: 1,707 Days
• Field of Search: None
• US Class Current: 382/156
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/451 with interaction between the filter responses, e.g. cortical complex cells

Abstract

A bio-inspired actionable intelligence method and system is disclosed. The actionable intelligence method comprises recognizing entities in an imagery signal, detecting and classifying anomalous entities, and learning new hierarchal relationships between different classes of entities. A knowledge database is updated after each new learning experience to aid in future searches and classification. The method can accommodate incremental learning via Adaptive Resonance Theory (ART).

Claims

1. A method of actionable intelligence for detecting anomalous entities, comprising an act of initializing one or more processors to perform operations of:

receiving an input signal;

selecting a class of entities to be recognized;

recognizing a set of entities of the selected class in the input signal using an Adaptive Resonance Theory (ART)-based neural network;

selecting a set of threshold criteria by which a set of anomalous entities can be detected within the set of recognized entities;

detecting the set of anomalous entities by comparing the set of recognized entities against the set of threshold criteria;

alerting an operator to the presence of the set of anomalous entities, whereby anomalous entities are detected;

prompting the operator to assign new labels to the set of anomalous entities;

discovering underlying hierarchical relationships between the new labels assigned by the operator; and

updating a knowledge database with the new labels and hierarchical relationships,

whereby anomalous entities are classified and hierarchically related.

2. The method of claim 1, where in the operation of receiving an input signal, the input signal is a signal selected from the group consisting of a video signal, an audio signal, a radar signal, and a sonar signal.

3. The method of claim 2, where in the operation of recognizing a set of entities, the entities recognized are selected from the group consisting of objects, spatial patterns, events and behaviors.

4. The method of claim 1, where in the operation of receiving an input signal, the input signal is a signal selected from the group consisting of a video signal, an audio signal, a radar signal, and a sonar signal.

5. The method of claim 1, where in the operation of recognizing a set of entities, the entities recognized are selected from the group consisting of objects, spatial patterns, events and behaviors.

6. The method of claim 1, wherein in recognizing the set of entities, the one or more processors further perform operations of:

using a Scene Recognition Engine (SCE) to recognize spatial relationships between objects that compose a particular scene category in static imagery as captured from the input signal, where the input signal is a video signal; and

using an Event Recognition Engine (ERE) to recognize spatio-temporal sequences and, thereby, recognize events and behaviors in the video signal by maintaining and recognizing ordered spatio-temporal sequences.

7. An actionable-intelligence system for detecting anomalous entities, comprising one or more processors configured to perform operations of:

receiving an input signal;

selecting a class of entities to be recognized;

recognizing a set of entities of the selected class in the input signal using an Adaptive Resonance Theory (ART)-based neural network;

selecting a set of threshold criteria by which a set of anomalous entities can be detected within the set of recognized entities;

detecting the set of anomalous entities by comparing the set of recognized entities against the set of threshold criteria;

alerting an operator to the presence of the set of anomalous entities, whereby anomalous entities are detected;

prompting the operator to assign new labels to the set of anomalous entities;

discovering underlying hierarchical relationships between the new labels assigned by the operator; and

updating a knowledge database with the new labels and hierarchical relationships,

whereby anomalous entities are classified and hierarchically related.

8. The system of claim 7, where the input signal is a signal selected from the group consisting of a video signal, an audio signal, a radar signal, and a sonar signal.

9. The system of claim 8, where the set of entities recognized are selected from the group consisting of objects, spatial patterns, events and behaviors.

10. The system of claim 7, where the input signal is a signal selected from the group consisting of a video signal, an audio signal, a radar signal, and a sonar signal.

11. The system of claim 7, where the set of entities recognized is selected from the group consisting of objects, spatial patterns, events and behaviors.

12. The system of claim 7, further configured to perform operations of:

using a Scene Recognition Engine (SCE) to recognize spatial relationships between objects that compose a particular scene category in static imagery as captured from the input signal, where the input signal is a video signal; and

using an Event Recognition Engine (ERE) to recognize spatio-temporal sequences and, thereby, recognize events and behaviors in the video signal by maintaining and recognizing ordered spatio-temporal sequences.

13. An actionable intelligence computer program product for detecting anomalous entities, the computer program product comprising computer-readable instruction means stored on a non-transitory computer-readable medium that are executable by a computer having a processor for causing the processor to perform operations of:

receiving an input signal;

selecting a class of entities to be recognized;

recognizing a set of entities of the selected class in the input signal using an Adaptive Resonance Theory (ART)-based neural network;

selecting a set of threshold criteria by which a set of anomalous entities can be detected within the set of recognized entities;

detecting the set of anomalous entities by comparing the set of recognized entities against the set of threshold criteria;

alerting an operator to the presence of the set of anomalous entities, whereby anomalous entities are detected;

prompting the operator to assign new labels to the set of anomalous entities;

discovering underlying hierarchical relationships between the new labels assigned by the operator; and

updating a knowledge database with the new labels and hierarchical relationships,

whereby anomalous entities are classified and hierarchically related.

14. The computer program product of claim 13, where the input signal is a signal selected from the group consisting of a video signal, an audio signal, a radar signal, and a sonar signal.

15. The computer program product of claim 14, where the set of entities recognized are selected from the group consisting of objects, spatial patterns, events and behaviors.

16. The computer program product of claim 13, where the input signal is a signal selected from the group consisting of a video signal, an audio signal, a radar signal, and a sonar signal.

17. The computer program product of claim 13, where the set of entities recognized is selected from the group consisting of objects, spatial patterns, events and behaviors.

18. The computer program product of claim 13, further comprising computer-readable instructions to cause one or more processors to perform operations of:

using a Scene Recognition Engine (SCE) to recognize spatial relationships between objects that compose a particular scene category in static imagery as captured from the input signal, where the input signal is a video signal; and

using an Event Recognition Engine (ERE) to recognize spatio-temporal sequences and, thereby, recognize events and behaviors in the video signal by maintaining and recognizing ordered spatio-temporal sequences.