APPARATUS, SYSTEM, AND METHOD FOR OBJECT DETECTION AND IDENTIFICATION

• US 20120143808A1
• Filed: 12/02/2010
• Published: 06/07/2012
• Est. Priority Date: 12/02/2010
• Status: Active Grant

• Current Assignee: Pukoa Scientific LLC
• Original Assignee: Pukoa Scientific LLC
• Inventors: Karins, James P., Mills, Stuart A.
• Granted Patent: US 8,527,445 B2
• Time in Patent Office: Days
• US Class Current: 706/46
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

An apparatus, system, and method are disclosed for identifying a target object. An object detection module detects objects by matching data from one or more sensors to known data of a target object and determining one or more correlation metrics for each object. An object tracking module tracks geographic locations for detected objects over time using subsequent data from the one or more sensors. A contextual data module determines one or more contextual indicators for detected objects based on the data from the one or more sensors. An artificial intelligence module estimates probabilities that detected objects comprise the target object based on the correlation metrics, the geographic locations, the contextual indicators, and one or more target contextual indicators associated with the target object. The artificial intelligence module estimates the probabilities using an artificial intelligence model, such as a Bayesian network.

Claims

1. An apparatus to identify a target object, the apparatus comprising:

an object detection module that detects a plurality of objects by matching data from one or more sensors to known data of a target object, the object detection module determining one or more correlation metrics for each object;

an object tracking module that tracks geographic locations for at least a subset of the plurality of objects over time using subsequent data from the one or more sensors;

a contextual data module that determines one or more contextual indicators for at least a subset of the plurality of objects based on the data from the one or more sensors; and

an artificial intelligence module that estimates probabilities that each of at least a subset of the plurality of objects comprises the target object based on the correlation metrics, the geographic locations, the contextual indicators, and one or more target contextual indicators associated with the target object, the artificial intelligence module estimating the probabilities using an artificial intelligence model.

2. The apparatus of claim 1, further comprising a target definition module that determines the known data of the target object and the one or more target contextual indicators for the target object based on user input.

3. The apparatus of claim 2, wherein the known data comprises one or more spatial correlation filters corresponding to the target object, wherein the one or more target contextual indicators each indicate an expected physical property of the target object and the one or more contextual indicators for the plurality of objects each indicate a detected physical property of an object of the plurality of objects.

4. The apparatus of claim 1, wherein the target definition module updates one or more of the target contextual indicators based on the data from the one or more sensors.

5. The apparatus of claim 1, further comprising an object removal module that removes an object from the plurality of objects in response to the estimated probability for the object satisfying a removal threshold.

6. The apparatus of claim 1, further comprising an identified target action module that performs an identified target action corresponding to an object in response to the estimated probability for the object satisfying an identification threshold.

7. The apparatus of claim 1, further comprising a data conditioning module that conditions the data from the one or more sensors to prepare the data for the object detection module, the object tracking module, and the contextual data module.

8. The apparatus of claim 1, wherein the data from the one or more sensors comprises an image of terrain in which the plurality of objects are located and the subsequent data from the one or more sensors comprises a subsequent image of at least a portion of the terrain.

9. The apparatus of claim 8, wherein the object detection module comprises,

a filter selection module that estimates a scale of the image of the terrain, estimates an angle of view for the image of the terrain, and selects one or more spatial correlation filters corresponding to the target object based on the estimated scale and the estimated angle of view;

a correlation module that correlates the one or more selected spatial correlation filters and the image of the terrain to form one or more correlation planes;

a correlation metric module that determines the one or more correlation metrics for each point in the one or more correlation planes; and

an object selection module that selects the plurality of detected objects based on the correlation metrics.

10. The apparatus of claim 9, wherein the one or more correlation planes comprise a plurality of correlation planes and the object detection module further comprises an object merge module that detects multiple detected objects within a predefined distance of each other in different correlation planes and combines the multiple detected objects into a single detected object.

11. The apparatus of claim 9, wherein the correlation module correlates the one or more selected spatial correlation filters and the image of the terrain by performing a Fourier transform on the image of the terrain and on the one or more selected spatial correlation filters and, for each transformed spatial correlation filter, multiplying the transformed image of the terrain by the transformed spatial correlation filter and performing an inverse Fourier transform on a result of the multiplication to form one of the one or more correlation planes.

12. The apparatus of claim 9, wherein at least one of the one or more correlation metrics is selected from the group consisting of a peak-to-correlation energy (“

PCE”

), a peak-to-sidelobe ratio (“

PSR”

), a peak magnitude of a point in a correlation plane, and a standard deviation over a region of a correlation plane.

13. The apparatus of claim 8, further comprising a motion module that compensates for motion of the one or more sensors between the image of the terrain and the subsequent image of at least a portion of the terrain.

14. The apparatus of claim 1, further comprising an action module that estimates an action state for one or more of the plurality of objects based on the data from the one or more sensors and on the subsequent data from the one or more sensors, the action module estimating the action state using the artificial intelligence model.

15. The apparatus of claim 1, wherein the target object comprises a mobile object capable of independent physical motion.

16. The apparatus of claim 1, wherein the artificial intelligence module further selects a subset of the plurality of objects for the object tracking module to track.

17. The apparatus of claim 1, wherein the artificial intelligence model that the artificial intelligence module uses to estimate the probabilities comprises a Bayesian network model that represents a set of random variables and conditional dependencies of the set of random variables as a directed acyclic graph (“

DAG”

).

18. The apparatus of claim 17, wherein the set of random variables includes random variables representing the correlation metrics, the geographic locations, and the contextual indicators.

19. The apparatus of claim 17, wherein the conditional dependencies of the set of random variables comprise one or more probability functions that define an estimated probability that a detected object comprises the target object based on values of the set of random variables, the artificial intelligence module estimating the probabilities using the one or more probability functions.

20. The apparatus of claim 19, wherein at least one of the one or more probability functions comprises a probability table with an entry for possible combinations of values for random variables associated with the at least one probability function from the set of random variables.

21. The apparatus of claim 1, wherein the object tracking module further estimates multiple hypothesis tracks for an object of the plurality of objects in response to the object being obscured in the subsequent data from the one or more sensors and tracks multiple hypothesis geographic locations for the object along the estimated multiple hypothesis tracks.

22. The apparatus of claim 21, wherein the object tracking module selects one of the multiple hypothesis tracks for the object as a most likely track and determines the geographic location for the object based on the most likely track.

23. The apparatus of claim 1, wherein the object detection module iteratively updates the correlation metrics based on the subsequent data from the one or more sensors, the object tracking module iteratively updates the geographic locations using the subsequent data from the one or more sensors, the contextual data module iteratively updates the one or more contextual indicators based on the subsequent data from the one or more sensors, and the artificial intelligence module iteratively updates the estimated probabilities that each of at least a subset of the plurality of objects comprises the target object based on the updated correlation metrics, the updated geographic locations, and the updated contextual indicators.

24. The apparatus of claim 1, wherein the geographic location that the object tracking module tracks for an object comprises geographic coordinates of the object, an estimated velocity of the object, and an estimated direction of the object.

25. The apparatus of claim 1, wherein at least one of the contextual indicators is selected from the group consisting of a color of an object, a temperature of an object, an estimated destination of an object, and an identifying marking on an object.

26. The apparatus of claim 1, wherein the artificial intelligence module further determines a confidence metric corresponding to each estimated probability, each confidence metric representing a strength of an estimated probability.

27. A system to identify a target object, the system comprising:

one or more sensors;

an object detection module that detects a plurality of objects by matching data from the one or more sensors to known data of a target object, the object detection module determining one or more correlation metrics for each object;

an object tracking module that tracks geographic locations for at least a subset of the plurality of objects over time using subsequent data from the one or more sensors;

a contextual data module that determines one or more contextual indicators for at least a subset of the plurality of objects based on the data from the one or more sensors; and

an artificial intelligence module that estimates probabilities that each of at least a subset of the plurality of objects comprises the target object based on the correlation metrics, the geographic locations, the contextual indicators, and one or more target contextual indicators associated with the target object, the artificial intelligence module estimating the probabilities using an artificial intelligence model.

28. The system of claim 27, further comprising one or more processors and a computer readable storage medium, the computer readable storage medium storing computer executable code of the object detection module, the object tracking module, the contextual data module, and the artificial intelligence module for execution on the one or more processors.

29. The system of claim 28, wherein the one or more processors comprise a plurality of processing cores and the object detection module matches the data from the one or more sensors to the known data in parallel on the plurality of processing cores.

30. The system of claim 29, wherein the plurality of processing cores comprise compute unified device architecture (“

CUDA”

) cores.

31. The system of claim 28, further comprising an unmanned vehicle housing the one or more sensors, the one or more processors, and the computer readable storage medium.

32. A method for identifying a target object, the method comprising:

detecting a plurality of objects by matching data from one or more sensors to known data of a target object and determining one or more correlation metrics for each object;

tracking geographic locations for at least a subset of the plurality of objects over time using subsequent data from the one or more sensors;

determining one or more contextual indicators for at least a subset of the plurality of objects based on the data from the one or more sensors; and

estimating, using an artificial intelligence model, probabilities that each of at least a subset of the plurality of objects comprises the target object based on the correlation metrics, the geographic locations, the contextual indicators, and one or more target contextual indicators associated with the target object.

33. The method of claim 32, further comprising determining the known data of the target object and the one or more target contextual indicators for the target object based on user input.

34. The method of claim 32, further comprising removing an object from the plurality of objects in response to the estimated probability for the object satisfying a removal threshold.

35. The method of claim 32, further comprising performing an identified target action corresponding to an object in response to the estimated probability for the object satisfying an identification threshold.

36. The method of claim 32, wherein the data from the one or more sensors comprises an image of terrain in which the plurality of objects are located and the subsequent data from the one or more sensors comprises a subsequent image of at least a portion of the terrain, the method further comprising,

estimating a scale of the image of the terrain;

estimating an angle of view for the image of the terrain;

selecting one or more spatial correlation filters corresponding to the target object based on the estimated scale and the estimated angle of view;

correlating the one or more selected spatial correlation filters and the image of the terrain to form one or more correlation planes;

determining the one or more correlation metrics for each point in the one or more correlation planes; and

selecting the plurality of detected objects based on the correlation metrics.

37. The method of claim 36, further comprising detecting multiple detected objects within a predefined distance of each other in different correlation planes from the one or more correlation planes and combining the multiple detected objects into a single detected object.

38. The method of claim 32, further comprising compensating for motion of the one or more sensors between the data from the one or more sensors and the subsequent data from the one or more sensors.

39. The method of claim 32, further comprising estimating an action state for one or more of the plurality of objects, using the artificial intelligence model, based on the data from the one or more sensors and on the subsequent data from the one or more sensors.

40. A computer program product comprising a computer readable medium having computer usable program code executable to perform operations for identifying a target object, the operations of the computer program product comprising:

detecting a plurality of objects by matching data from one or more sensors to known data of a target object and determining one or more correlation metrics for each object;

tracking geographic locations for at least a subset of the plurality of objects over time using subsequent data from the one or more sensors;

determining one or more contextual indicators for at least a subset of the plurality of objects based on the data from the one or more sensors; and

estimating, using an artificial intelligence model, probabilities that each of at least a subset of the plurality of objects comprises the target object based on the correlation metrics, the geographic locations, the contextual indicators, and one or more target contextual indicators associated with the target object.

41. The computer program product of claim 40, the operations further comprising determining the known data of the target object and the one or more target contextual indicators for the target object based on user input, wherein the known data comprises one or more spatial correlation filters corresponding to the target object, the one or more target contextual indicators each indicate an expected physical property of the target object, and the one or more contextual indicators for the plurality of objects each indicate a detected physical property of an object of the plurality of objects.

42. The computer program product of claim 40, the operations further comprising compensating for motion of the one or more sensors between the data from the one or more sensors and the subsequent data from the one or more sensors.

43. The computer program product of claim 40, the operations further comprising estimating an action state for one or more of the plurality of objects, using the artificial intelligence model, based on the data from the one or more sensors and on the subsequent data from the one or more sensors.