Multi-stage method for object detection using cognitive swarms and system for automated response to detected objects

• US 8,515,126 B1
• Filed: 06/18/2009
• Issued: 08/20/2013
• Est. Priority Date: 05/03/2007
• Status: Active Grant

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Medasani, Swarup, Owechko, Yuri, Daily, Michael, Azuma, Ronald
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): BROOKS, JULIAN D
• Application Number:
  US12/456,558
• Time in Patent Office: 1,524 Days
• Field of Search: 382/103, 382/227, 382/100, 382/155
• US Class Current: 382/103
• CPC Class Codes: G06F 18/2111 by using evolutionary computational techniques, e.g. genetic algorithms
• CPC Class Codes: G06V 10/771 Feature selection, e.g. selecting representative features from a multi-dimensional feature space
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A multi-stage method of visual object detection is disclosed. The method was originally designed to detect humans in specific poses, but is applicable to generic detection of any object. A first stage comprises acts of searching for members of a predetermined general-class of objects (such as humans) in an image using a cognitive swarm, detecting members of the general-class of objects in the image, and selecting regions of the image containing detected members of the general-class of objects. A second stage comprises acts of searching for members of a predetermined specific-class of objects (such as humans in a certain pose) within the selected regions of the image using a cognitive swarm, detecting members of the specific-class of objects within the selected regions of the image, and outputting the locations of detected objects to an operator display and optionally to an automatic response system.

Claims

1. A multi-stage method of object detection, comprising:

a first stage, the first stage comprising acts of;

searching for members of a predetermined general-class of objects in an image using cognitive swarm;

detecting members of the general-class of objects in the image; and

selecting regions of the image containing detected members of the general-class of objects; and

a second stage, the second stage comprising acts of;

searching for members of a predetermined specific-class of objects within the selected regions; and

detecting members of the specific-class of objects within the selected regions of the image, whereby members of the specific-class of objects are located in the image.

2. The method of claim 1, wherein when searching for members of a predetermined general-class of objects, the general-class of objects is humans, and when searching for members of a predetermined specific-class of objects, the specific-class of objects is humans in a specific pose.

3. The method of claim 2, further comprising an act of cueing an operator with the locations of detected members of the specific-class of objects.

4. The method of claim 3, further comprising an act of cueing an automatic response system with the locations of detected members of the specific-class of objects.

5. The method of claim 4, wherein searching, for members of the predetermined specific-class is performed by a search method selected from a group consisting of a cognitive swarm and a scanning search.

6. The method of claim 1, further comprising art act of cueing an operator with the locations of detected members of the specific-class of objects.

7. The method of claim 1, further comprising an act of cueing an automatic response system with the locations of detected members of the specific-class of objects.

8. The method of claim 1, wherein searching for members of the predetermined specific-class is performed by a search method selected from a group consisting of a cognitive swarm and a scanning search.

9. A data processing system for multi-stage object detection, the system having one or more processors configured to:

search for members of a predetermined general-class of objects in an image using, a cognitive swarm;

detect members of the general-class of objects in the image;

select regions of the image containing detected members of the general-class of objects;

search for members of a predetermined specific-class of objects within, the selected regions; and

detect members of the specific-class of objects within the selected regions of the image.

10. The data processing system of claim 9, where the general-class of objects is humans, and where the specific-class of objects is humans in a specific pose.

11. The data processing system of claim 10, further configured to output locations of detected members of the specific-class of objects to an operator display unit.

12. The data processing system of claim 11, further configured to output locations of detected members of the specific-class of objects to an automatic response subsystem.

13. The data processing system of claim 12, wherein the search for members of the predetermined specific-class is performed by a search method selected from a group consisting of a cognitive swarm and a scanning search.

14. The data processing system of claim 9, further configured to output locations of detected members of the specific-class of objects to an operator display unit.

15. The data processing system of claim 9, further configured to output locations of detected members of the specific-class of objects to an automatic response subsystem.

16. The data processing system of claim 9, wherein the search for members of the predetermined specific-class is performed by a search method selected from a group consisting of a cognitive swarm and a scanning search.

17. A system for multi-stage object detection, comprising:

at least one optical sensor for imaging a scene;

a data processing sub-system having one or more processors configured to receive at least one image from the at least one optical sensor and being further configured to;

search for members of a predetermined general-class of objects in an image using a cognitive swarm;

detect members of the general-class of objects in the image;

select regions of the image containing detected members of the general-class of objects;

search for members of a predetermined specific-class of objects within the selected regions; and

detect members of the specific-class of objects within the selected regions of the image; and

an operator display sub-system configured to receive locations of detected members of the specific-class of objects from the data processing sub-system and alert an operator to the locations of the detected members of the specific-class of objects.

18. The system of claim 17, where the general-class of objects is humans, and where the specific-class of objects is humans in a specific pose.

19. The system of claim 18, wherein the operator display sub-system is operable in a manner selected from a group consisting of illuminating, the object, displaying the object location through personal head-worn displays, and displaying the object location, on a flat panel display.

20. The system of claim 19, wherein the at least one optical sensor comprises a plurality of optical sensors, the data processing subsystem comprises a network of data processors configured to process the images from the plurality of optical sensors in parallel, and the network of data processors are connected with a master processor for coordinating results from the network of data processors.

21. The system of claim 20, further comprising an automatic response sub-system configured to receive locations of detected members of the specific-class of objects from the data processing sub-system.

22. The system of claim 21, wherein the search for members of the predetermined specific-class is performed by a search method selected from a group consisting of a cognitive swarm and a scanning search.

23. The system of claim 17, wherein the operator display sub-system is operable in a manner selected from a group consisting of illuminating the object, displaying the object location through personal head-worn displays, and displaying the object location on a flat panel display.

24. The system of claim 17, wherein the at least one optical sensor comprises a plurality of optical sensors, the data processing subsystem comprises a network of data processors configured to process the images from the plurality of optical sensors in parallel, and the network of data processors are connected with a master processor for coordinating results from the network of data processors.

25. The system of claim 17, further comprising an automatic response sub-system configured to receive locations of detected members of the specific-class of objects from the data processing, sub-system.

26. The system of claim 17, wherein the search for members of the predetermined specific-class is performed by a search method selected from a group consisting of a cognitive swarm and a scanning search.

27. A computer program product for multi-stage object detection, the computer program product comprising computer-readable instruction means stored on a non-transitory computer-readable medium that are executable by a computer for causing the computer to:

search for members of a predetermined general-class of objects in an image using, a cognitive swarm;

detect members of the general-class of objects in the image;

select regions of the image containing detected members of the general-class of objects;

search for members of a predetermined specific-class of objects within the selected regions; and

detect members of the specific-class of objects within the selected regions of the image.

28. The computer program product of claim 27, where the general-class of objects is humans, and where the specific-class of objects is humans in a specific pose.

29. The computer program product of claim 28, further including instruction means for causing the computer to output locations of detected members of the specific-class of objects to an operator display unit.

30. The computer program product of claim 29, further including, instruction means for causing the computer to output locations of detected members of the specific-class of objects to an automatic response subsystem.

31. The computer program product of claim 30, wherein the search for members of the predetermined specific-class is performed by a search method selected from a group consisting of a cognitive swarm and a scanning search.

32. The computer program product of claim 27, further including instruction means for causing the computer to output locations of detected members of the specific-class of objects to an operator display unit.

33. The computer program product of claim 27, further including instruction means for causing the computer to output locations of detected members of the specific-class of objects to an automatic response subsystem.

34. The computer program product of claim 27, wherein the search for members of the predetermined specific-class is performed by a search method selected from a group consisting of a cognitive swarm and a scanning search.