Cognitive-neural method for image analysis

• US 8,214,309 B1
• Filed: 12/16/2008
• Issued: 07/03/2012
• Est. Priority Date: 12/16/2008
• Status: Active Grant

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Khosla, Deepak, Daily, Michael J.
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Pellett, Daniel
• Application Number:
  US12/316,779
• Time in Patent Office: 1,295 Days
• Field of Search: None
• US Class Current: 706/14
• CPC Class Codes: G06F 18/2185 the supervisor being an automated module, e.g. intelligent oracle
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06N 3/042 Knowledge-based neural networks; Logical representations of neural networks
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/7788 the supervisor being a human, e.g. interactive learning with a human teacher
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition

Abstract

A self adapting cognitive-neural method and system for image analysis is disclosed. The method comprises detecting a set of potential items of interest in an image using a cognitive algorithm, refining the set of potential items of interest using a neural analysis, and adapting the cognitive algorithm based on information gained from the results of the neural analysis. The method may further comprise fusing the results of the cognitive and neural algorithms with a fusion classifier. The neural analysis can include eye tracked EEG, RSVP, or presentation to an operator for visual inspection.

Claims

1. A cognitive-neural method for detecting and prioritizing items of interest in visual imagery, comprising an act of initializing one or more processors to perform operations of:

detecting a set of potential items of interest in an image using a cognitive algorithm;

prioritizing the set of potential items of interest using neural analysis to produce a set of prioritized items of interest, whereby items of interest in the image are detected and prioritized; and

adapting the cognitive algorithm based on the prioritized items of interest from the neural analysis.

2. The method of claim 1, wherein the cognitive algorithm is adapted using a technique selected from a group consisting of:

feature-based optimal biasing via learning;

combining categorizing items of interest using visual swarm technology with feature-based optimal biasing via learning; and

gist-based learning.

3. The method of claim 2, wherein the cognitive algorithm is selected from a group consisting of an object-based attention algorithm, a feature-based attention algorithm, an object-based recognition algorithm, and a spatio-temporal pattern-based recognition algorithm.

4. The method of claim 3, wherein the cognitive algorithm is configured to detect potential items of interest selected from a group consisting of single objects, multiple objects, and spatio-temporal patterns.

5. The method of claim 4, wherein the neural analysis is selected from a group consisting of Rapid Serial Visual Presentation (RSVP), real-time eye tracked Electro-Encephalography (EEG), and presenting the set of potential items of interest to an operator for operator-controlled visual inspection and validation.

6. The method of claim 1, wherein the cognitive algorithm is selected from a group consisting of an object-based attention algorithm, a feature-based attention algorithm, an object-based recognition algorithm, and a spatio-temporal pattern-based recognition algorithm.

7. The method of claim 1, wherein the neural analysis is selected from a group consisting of Rapid Serial Visual Presentation (RSVP), real-time eye tracked Electro-Encephalography (EEG), and presenting the set of potential items of interest to an operator for operator-controlled visual inspection and validation.

8. The method of claim 1, further comprising operations of

fusing the potential items of interest from the cognitive algorithm with the prioritized items of interest from neural analysis using a fusion classifier to generate a set of fused items of interest;

and

adapting the cognitive algorithm based on the prioritized items of interest from the neural analysis and the fused items of interest from the fusion classifier.

9. The method of claim 8, wherein the cognitive algorithm is adapted using a technique selected from a group consisting of:

feature-based optimal biasing via learning;

combining categorizing items of interest using visual swarm technology with feature-based optimal biasing via learning; and

gist-based learning.

10. A data processing system for detecting and prioritizing items of interest in visual imagery comprising one or more processors configured to perform operations of:

detecting a set of potential items of interest in an image by using a cognitive algorithm;

prioritizing the set of potential items of interest using neural analysis to produce a set of prioritized items of interest, whereby items of interest in the image are detected and prioritized; and

adapting the cognitive algorithm based on the prioritized items of interest from the neural analysis.

11. The data processing system of claim 10, wherein the cognitive algorithm is adapted using a technique selected from a group consisting of:

feature-based optimal biasing via learning;

combining categorizing items of interest using visual swarm technology with feature-based optimal biasing via learning; and

gist-based learning.

12. The data processing system of claim 11, wherein the cognitive algorithm is selected from a group consisting of an object-based attention algorithm, a feature-based attention algorithm, an object-based recognition algorithm, and a spatio-temporal pattern, based recognition algorithm.

13. The data processing system of claim 12, wherein the cognitive algorithm is configured to detect potential items of interest selected from a group consisting of single objects, multiple objects, and spatio-temporal patterns.

14. The data processing system of claim 13, wherein the neural analysis is selected from a group consisting of Rapid Serial Visual Presentation (RSVP), real-time eye tracked Electro-Encephalography (EEG), and presenting the set of potential items of interest to an operator for operator-controlled visual inspection and validation.

15. The data processing system of claim 10, wherein the cognitive algorithm is selected from a group consisting of an object-based attention algorithm, a feature-based attention algorithm, an object-based recognition algorithm, and a spatio-temporal pattern-based recognition algorithm.

16. The data processing system of claim 10, wherein the neural analysis is selected from a group consisting of Rapid Serial Visual Presentation (RSVP), real-time eye tracked Electro-Encephalography (EEG), and presenting the set of potential items of interest to an operator for operator-controlled visual inspection and validation.

17. The data processing system of claim 10, further configured to perform operations of:

fusing the potential items of interest from the cognitive algorithm with the prioritized items of interest from neural analysis using a fusion classifier to generate a set of fused items of interest; and

adapting the cognitive algorithm based on the prioritized items of interest from the neural analysis and the fused items of interest from the fusion classifier.

18. The data processing system of claim 17, wherein the cognitive algorithm is adapted using a technique selected from a group consisting of:

feature-based optimal biasing via learning;

combining categorizing items of interest using visual swarm technology with feature-based optimal biasing via learning; and

gist-based learning.

19. A computer program product for detecting and prioritizing items of interest in visual imagery, the computer program product comprising computer-readable instruction means stored on a computer-readable medium that are executable by a computer having a processor for causing the processor to perform operations of:

detecting a set of potential items of interest in an image by using a cognitive algorithm;

prioritizing the set of potential items of interest using neural analysis to produce a set of prioritized items of interest, whereby items of interest in the image are detected and prioritized; and

adapting the cognitive algorithm based on the prioritized items of interest from the neural analysis.

20. The computer program product of claim 19, wherein the cognitive algorithm is adapted using a technique selected from a group consisting of:

feature-based optimal biasing via learning;

combining categorizing items of interest using visual swarm technology with feature-based optimal biasing via learning; and

gist-based learning.

21. The computer program product of claim 20, wherein the cognitive algorithm is selected from a group consisting of an object-based attention algorithm, a feature-based attention algorithm, an object-based recognition algorithm, and a spatio-temporal pattern-based recognition algorithm.

22. The computer program product of claim 21, wherein the cognitive algorithm is configured to detect potential items of interest selected from a group consisting of single objects, multiple objects, and spatio-temporal patterns.

23. The computer program product of claim 22, wherein the neural analysis is selected from a group consisting of Rapid Serial Visual Presentation (RSVP), real-time eye tracked Electro-Encephalography (EEG), and presenting the set of potential items of interest to an operator for operator-controlled visual inspection and validation.

24. The computer program product of claim 19, wherein the cognitive algorithm is selected from a group consisting of an object-based attention algorithm, a feature-based attention algorithm, an object-based recognition algorithm, and a spatio-temporal pattern-based recognition algorithm.

25. The computer program product of claim 19, wherein the neural analysis is selected from a group consisting of Rapid Serial Visual Presentation (RSVP), real-time eye tracked. Electro-Encephalography (EEG), and presenting the set of potential items of interest to an operator for operator-controlled visual inspection and validation.

26. The computer program product of claim 19, further comprising instruction means to cause the processor to perform operations of:

fusing the potential items of interest from the cognitive algorithm with the prioritized items of interest from neural analysis using a fusion classifier to generate a set of fused items of interest; and

adapting the cognitive algorithm based on the prioritized, items of interest from the neural analysis and the fused items of interest from the fusion classifier.

27. The computer program product of claim 26, wherein the cognitive algorithm is adapted using a technique selected from a group consisting of:

feature-based optimal biasing via learning;

combining categorizing items of interest using visual swarm technology with feature-based optimal biasing via learning; and

gist-based learning.