Scaleable object recognition with a belief model

• US 20020103776A1
• Filed: 06/18/2001
• Published: 08/01/2002
• Est. Priority Date: 06/16/2000
• Status: Active Grant

• Current Assignee: Leidos Innovations Technology, Inc.
• Original Assignee: Lockheed Martin Mission Systems (Martin Marietta Corporation)
• Inventors: Hood, Christopher, Bella, Ivan
• Granted Patent: US 7,184,994 B2
• Time in Patent Office: Days
• US Class Current: 706/49
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation

Abstract

An exemplary embodiment of the present invention is directed to a system, method and computer program product for providing an object recognition blackboard system architecture. The system for recognizing objects in content can include: a blackboard comprising a plurality of experts, and data comprising original input data and data created by processing of any of the plurality of experts, and a controller operative to control the experts; a belief model, coupled to the controller, comprising a set of beliefs and probabilities associated with each belief of the set of beliefs; a belief network, coupled to the controller; and a relations subsystem, coupled to the controller.

Claims

1. A system operative to recognize objects in content comprising:

a blackboard comprising

a plurality of experts, and

data comprising original input data and data created by processing of any of said plurality of experts, and

a controller operative to control said experts;

a belief model, coupled to said controller, comprising a set of beliefs and probabilities associated with each belief of said set of beliefs;

a belief network, coupled to said controller; and

a relations subsystem, coupled to said controller.

2. The system according to claim 1, wherein said experts comprise expert object recognizers comprising at least one of:

region identification experts;

color region experts;

a corner recognizer;

a closed curve recognizer;

a roof recognizer;

a text recognizer;

simulated experts;

microphone recognizer;

space suit recognizer;

satellite recognizer;

a geometric shape recognizer;

a building recognizer;

an egg recognizer;

a dice recognizer;

a person recognizer;

a face recognizer; and

a product recognizer.

3. The system according to claim 1, wherein said data comprises at least one of:

relations data;

expert status data;

image subsection data; and

said belief model.

4. The system according to claim 1, wherein said controller is at least one of:

operative to choose chosen experts from said plurality of experts which are to be executed;

operative to schedule execution of said chosen experts; and

operative to execute said chosen experts.

5. The system according to claim 1, wherein said blackboard further comprises at least one of:

storage for receiving an input image; and

a reporter operative to output results of processing.

6. The system according to claim 1, wherein said belief model comprises:

a set of rules deduced from a learning system which describes how different classes recognized by the system are related to each other spatially and physically.

7. The system according to claim 1, wherein said belief model is operative to predict existence of a shadow object in an image even if there are no specific experts capable of recognizing said shadow object.

8. The system according to claim 1, wherein said belief network is operative to combine beliefs in output data output by said experts and probabilities drawn from said belief model into a single belief for a given object.

9. The system according to claim 1, wherein said relations subsystem is operative to determine how returned objects returned by said experts are related to each other.

10. The system according to claim 1, wherein said relations subsystem is operative to determine spatial relations.

11. The system according to claim 10, wherein said spatial relations include types comprising at least one of:

a north type,

a south type,

an east type,

a west type,

a contains type,

a contained by type, and

an adjacent to type.

12. The system according to claim 1, wherein said relations subsystem is operative to determine temporal relations.

13. The system according to claim 12, wherein said temporal relations include types comprising at least one of:

a before type,

an after type, and

an exists with type.

14. The system according to claim 1, wherein the content comprises at least one of:

video;

an image;

digitized content; and

a frame.

15. The system of claim 1, wherein said belief model is generated by a learning system.

16. The system of claim 15, wherein said learning system comprises:

truth data files for deducing beliefs, probabilities and shadow objects;

a learning system controller; and

a statistics space controlled by said controller.

17. The system according to claim 15, wherein said learning system is operative to assist in integrating a new expert wherein said new expert has been created, encapsulated, compiled, a stub function has been added to said blackboard, if output is new has been added to the belief model, and a blackboard rule has been added to control when said new expert will be executed.

18. The system of claim 1, wherein said belief network is at least one of:

a Bayesian Network;

a mean probability; and

a Dempster-Shafer Network.

19. The system according to claim 1, wherein said belief model comprises:

rules operative to be used to make a determination whether or not one of said experts should be executed by search of said belief model to determine whether an adaptable threshold of supporting evidence has been exceeded for an execution supportability rule that evaluates outputs of currently executing experts.

20. The system according to claim 1, wherein said belief model is operative to model expected object associations, to weigh relative object positions, and to tie a probability or belief value to those associations.

21. The system according to claim 1, wherein said belief network is operative to combine the belief model with hypotheses generated by said experts to form belief values for hypothesized objects.

22. A method of recognizing objects comprising:

identifying classes of objects specified by a user using a plurality of cooperative object recognition experts;

achieving higher accuracy from using in parallel said plurality of cooperative object recognition experts than is achievable using in serial said plurality of cooperative object recognition experts;

supporting scaleability of performance including supporting multiple processors;

developing a belief model including

specifying specified associations among said objects,

learning learned associations among said objects,

representing said specified and learned associations, and

forming a belief network

wherein said belief network is at least one of a Bayesian Network and a Dempster Shafer Network; and

deducing shadow objects from said belief model.

23. A method for adding a new expert to a blackboard comprising:

creating an expert;

encapsulating said expert;

compiling said expert;

adding a stub function to a blackboard;

determining if output of said expert is new and if new, then

adding the output'"'"'s class to said blackboard, and

updating a belief model by providing truth data file data to a learning system; and

creating a rule to control when said new expert is to be executed when supporting evidence is found to exceed an adaptable threshold.