System and method of chaining algorithms for global object recognition to improve probability of correctness and reduce processing load

• US 10,460,252 B2
• Filed: 02/25/2016
• Issued: 10/29/2019
• Est. Priority Date: 07/25/2014
• Status: Active Grant

• Current Assignee: Raytheon Company (Rtx Corporation)
• Original Assignee: Raytheon Company (Rtx Corporation)
• Inventors: Hershey, Paul C., Radcliffe, Richard, Hoppes, Brianne R.
• Primary Examiner(s): Corrielus, Jean M
• Application Number:
  US15/053,959
• Publication Number:
  US 20170200090A1
• Time in Patent Office: 1,342 Days
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A system and method improves the probability of correctly detecting an object from a collection of source data and reduces the processing load. A plurality of algorithms for a given data type are selected and ordered based on a cumulative trained probability of correctness (Pc) that each of the algorithms, which are processed in a chain and conditioned upon the result of the preceding algorithms, produce a correct result and a processing. The algorithms cull the source data to pass forward a reduced subset of source data in which the conditional probability of detecting the object is higher than the a priori probability of the algorithm detecting that same object. The Pc and its confidence interval is suitably computed and displayed for each algorithm and the chain and the final object detection.

Claims

1. A computer-implemented method for global object recognition comprising:

receiving, by the one or more hardware processors, object metadata including a plurality of characteristics that define an object to be detected;

receiving, by one or more hardware processors, search metadata including a plurality of context parameters that define a search for the object;

retrieving, based on the object and search metadata, a plurality of source data of a given data type;

selecting, from a plurality of algorithms, a subset of algorithms to be used in processing the retrieved source data based on a cumulative trained probability of correctness (Pc) that each of the algorithms, which are processed in a chain and conditioned upon the result of the preceding algorithms, produce a correct result;

ordering the algorithms in the subset based on algorithm metadata including a plurality of algorithm characteristics to reduce an expected processing load of the retrieved source data; and

processing the retrieved source data in order according to the chain of the selected subset of algorithms to obtain a plurality of results and to reduce the number of source data that is processed by the next algorithm in the chain, at least one result indicating whether the object was detected in corresponding source data output from the last algorithm in the chain.

2. The method of claim 1, further comprising:

computing a cumulative object Pc and confidence interval representing whether the object was detected in the plurality of source data.

3. The method of claim 2, further comprising:

computing a cumulative algorithm Pc and confidence interval representing whether the result of each algorithm in the chain was correct in the plurality of source data.

4. The method of claim 1, further comprising configuring the algorithms in the subset to reduce the number of retrieved source data by a nominal culling percentage.

5. The method of claim 4, further comprising:

receiving, by one or more hardware processors, a mission time critical targeting (TCT) requirement; and

reconfiguring one or more of the image processing algorithms to adjust the nominal culling percentage based on the mission TCT requirement.

6. The method of claim 1, wherein the steps of selecting and ordering the subset of algorithms further comprises:

for a plurality of objects, non-real-time evaluation of a plurality of candidate subsets of different chained algorithms configured based on object and algorithm metadata, selection of a subset based on its cumulative trained Pc and expected processing load for each said object and storing of selected subsets in a repository; and

real-time selection of the subset from the repository based on the object metadata.

7. The method of claim 6, wherein a candidate subset'"'"'s cumulative trained Pc is evaluated against a threshold after each image processing algorithm in the chain and disqualified if the trained Pc does not exceed the threshold.

8. The method of claim 7, wherein the plurality of candidate subsets'"'"' cumulate trained Pc are evaluated against a different threshold at each level of a multi-path tree to disqualify algorithms within the subset chain at that level and to identify one or more candidate algorithms to replace the disqualified algorithm at that level to continue the chain, complete the subset, and ensure that at least one subset'"'"'s cumulative Pc exceeds a final threshold.

9. The method of claim 6, further comprising using a stochastic math model (SMM) to compute the cumulative trained Pc and using a discrete event simulator (DES) to implement the SMM and perform a Monte Carlo simulation on multiple instances of the object in the source data to generate the confidence interval corresponding to the Pc at each level.

10. The method of claim 6, wherein the expected processing load is based on both the processing resources required to run each algorithm and a nominal culling percentage for that algorithm.

11. The method of claim 6, further comprising:

selecting multiple subsets each configured to detect the object, each said subset'"'"'s algorithms configured to process a different data type of source data;

retrieving multiple source data for each data type;

processing the source data according to the selected subset for the data type to obtain one or more results for each subset indicating whether the object was detected; and

fusing the one or more results for each subset to obtain one or more results indicating whether the object was detected.

12. The method of claim 11, further comprising:

computing and displaying a cumulative fused Pc and confidence interval for the detected object.

13. The method of claim 6, wherein the algorithms in the candidate subsets are configured to reduce the number of retrieved source data by a nominal culling percentage in total.

14. The method of claim 13, further comprising:

receiving a mission time critical targeting (TCT) requirement; and

reconfiguring one or more of the algorithms to adjust the nominal culling percentage based on the mission TCT requirement.

15. The method of claim 6, further comprising:

computing and displaying a cumulative object Pc and confidence interval for the detected object.

16. The method of claim 15, further comprising:

for each algorithm in the chain, computing and displaying a cumulative algorithm Pc and confidence interval.

17. The method of claim 6, further comprising:

if the object metadata is not a match for a subset, selecting in real-time a subset of algorithms based on the object and algorithm metadata and ordering the algorithms to reduce an expected processing load.

18. The method of claim 6, further comprising:

receiving operator feedback as to whether the detected object was correct or incorrect.

19. The method of claim 6, further comprising:

receiving operator feedback as to whether the result for each said algorithm was correct or incorrect.

20. The method of claim 6, further comprising using a non-real-time evaluation global object recognition server and cluster of processing nodes to perform the non-real-time evaluation and using a client device to perform the real-time processing.

21. The method of claim 1, wherein processing the retrieved source data in order according to the chain comprises:

for each of the first N−

1 algorithms in the chain, determining at least one probability and an associated confidence value representing whether the object was located in one or more of the retrieved one or more source data and whether data representing the object is correctly passed to the next algorithm in the chain based on the at least one result; and

for the Nth algorithm in the chain, determining a probability and associated confidence value for a geographic location of the object in the one or more remaining source data.

22. The method of claim 21, further comprising:

generating a visualization in the form of a heatmap of the at least one probability and the associated confidence value mapped to the geographic location corresponding to the at least one source data.

23. The method of claim 1, wherein the chain is an image chain configured to detect a land or water object, said image chain including a plurality of image size, cloud cover, sun quality, land/water, land object detection and water objection detection algorithms.

24. The method of claim 1, wherein the chain is an audio chain configured to detect a voice recognition object, said audio data chain including a plurality of source, language, quality, key words and frequency of key words algorithms.

25. The method of claim 1, wherein the chain is a text chain configured to extract text relevant to an object, said text data chain including a plurality of text format, language, topic, key words and frequency of key words algorithms.

26. The method of claim 1, wherein a signals chain is configured to locate an object that is the source of signals, said signals chain including a plurality of signal type, waveform, positioning and location algorithms.

27. A computer-implemented method for global object recognition comprising:

(a) receiving, by the one or more hardware processors, object metadata including a plurality of characteristics that define an object to be located;

(b) receiving, by one or more hardware processors, for each of a plurality of algorithms configured to process source data of a given data type selected from one of image, audio, text and signals, algorithm metadata including a plurality of algorithm characteristics that describe the algorithm;

(c) retrieving a plurality of training source data of the given data type;

(d) selecting, from the plurality of algorithms, based on the object and algorithm metadata a plurality of candidate subsets of algorithms to be used in processing the retrieved source data;

(e) for each candidate subset, ordering the algorithms in a chain based on algorithm metadata to reduce an expected processing load, wherein an image chain is configured to detect a land or water object, said image chain including a plurality of image size, cloud cover, sun quality, land/water, land object detection and water objection detection algorithms, wherein an audio chain is configured to detect a voice recognition object, said audio chain including a plurality of source, language, quality, key words and frequency of key words algorithms, wherein a text chain is configured to extract text relevant to an object, said text chain including a plurality of text format, language, topic, key words and frequency of key words algorithms and wherein a signals chain is configured to locate an object that is the source of signals, said signals chain including a plurality of signal type, waveform, positioning and location algorithms;

(f) for each candidate subset, processing the retrieved source data in order according to the chain of algorithms to obtain a plurality of results and to reduce the number of training source data that is processed by the next algorithm in the chain, at least one result indicating whether the object was identified in corresponding source data output from the last algorithm in the chain;

(g) for each candidate subset, computing a cumulative trained probability of correctness (Pc) and corresponding confidence interval that each of the algorithms, which are processed in the chain and conditioned upon the result of the preceding algorithms, produce a correct result;

(h) selecting a candidate subset based on its trained Pc and corresponding confidence interval and expected processing load;

(i) pairing the selected subset of algorithms with the object to be detected; and

(j) repeating the steps (a) through (i) for either the same object with a different data type or for different objects.

28. The method of claim 27, further comprising

selecting, from a plurality of algorithms for a different data type, a subset of algorithms to be used in processing the retrieved source data based on a cumulative trained probability of correctness (Pc) that each of the algorithms, which are processed in a different chain, ordering the algorithms to reduce the expected processing load and processing the retrieved source data of the different data type;

and

fusing the results of the different chains to provide a fused result indicating whether the object was detected.

29. A computer-implemented method for global object recognition comprising:

receiving, by the one or more hardware processors, object metadata including a plurality of characteristics that define the object to be detected;

receiving, by one or more hardware processors, search metadata including a plurality of context parameters that define a search for the object;

receiving, by one or more hardware processors, a plurality algorithms, algorithm metadata including a plurality of algorithm characteristics that describe each algorithm, and a plurality of defined subsets of chained algorithms configured to detect different objects, each said defined subset selected based on a cumulative trained probability of correctness (Pc) and corresponding confidence interval that each one of the algorithms, which are processed in the chain and conditioned upon the result of the preceding algorithms, produce a correct result and an expected processing load of the chain;

selecting, from the plurality of defined subsets, based on the object metadata one of the defined subsets, said algorithms in the selected subsets configured to process source data of a given data type selected from one of image, audio, text and signals;

if none of the defined subsets match the object to be located, based on the object and algorithm metadata selecting and ordering in a chain a plurality of algorithms, configured to process source data of a given data type, to define a selected subset, wherein an image chain is configured to detect a land or water object, said image chain including a plurality of image size, cloud cover, sun quality, land/water, land object detection and water objection detection algorithms, wherein an audio chain is configured to detect a voice recognition object, said audio chain including a plurality of source, language, quality, key words and frequency of key words algorithms, wherein a text chain is configured to extract text relevant to an object, said text chain including a plurality of text format, language, topic, key words and frequency of key words algorithms and wherein a signals chain is configured to locate an object that is the source of signals, said signals chain including a plurality of signal type, waveform, positioning and location algorithms;

retrieving, based on the plurality of context parameters, a plurality of source data of the given data type;

processing the retrieved one or more source data in order according to the chain of the selected subset of algorithms to obtain a plurality of results and to reduce the number of source data that is processed by the next algorithm in the chain, at least one result indicating whether the object was detected in corresponding source data output from the last algorithm in the chain; and

determining a cumulative object Pc and confidence interval representing whether the object was detected in one or more of the retrieved source data output from the last algorithm based on the at least one result.

30. A computer-implemented method for global object recognition comprising:

receiving, by the one or more hardware processors, object metadata including a plurality of characteristics that define an object to be detected;

receiving, by one or more hardware processors, search metadata including a plurality of context parameters that define a search for the object;

retrieving, based on the object and search metadata, a plurality of source data of a given data type selected from one of image, audio, text and signals;

selecting, from a plurality of algorithms, a subset of algorithms to be used in processing the retrieved source data based on a cumulative trained probability of correctness (Pc) that each of the algorithms, which are processed in a chain and conditioned upon the result of the preceding algorithms, produce a correct result, wherein an image chain is configured to detect either land or water objects, said image chain including a plurality of image size, cloud cover, sun quality, land/water, land object detection and water objection detection algorithms, wherein an audio chain is configured to detect a voice recognition object, said audio chain including a plurality of source, language, quality, key words and frequency of key words algorithms, wherein a text chain is configured to extract text relevant to an object, said text chain including a plurality of text format, language, topic, key words and frequency of key words algorithms, and for a signals chain is configured to locate an object that is the source of signals, said signals chain including a plurality of signal type, waveform, positioning and location algorithms;

ordering the algorithms in the subset based on algorithm metadata including a plurality of algorithm characteristics to reduce an expected processing load of the retrieved source data; and

processing the retrieved source data in order according to the chain of the selected subset of algorithms to obtain a plurality of results and to reduce the number of source data that is processed by the next algorithm in the chain, at least one result indicating whether the object was detected in corresponding source data output from the last algorithm in the chain.