Scaleable object recognition with a belief model

• US 7,184,994 B2
• Filed: 06/18/2001
• Issued: 02/27/2007
• Est. Priority Date: 06/16/2000
• Status: Expired due to Term

• Current Assignee: Leidos Innovations Technology, Inc.
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Bella, Ivan, Hood, Christopher
• Primary Examiner(s): Knight; Anthony
• Assistant Examiner(s): Brown, Jr.; Nathan H.
• Application Number:
  US09/881,746
• Publication Number:
  US 20020103776A1
• Time in Patent Office: 2,080 Days
• Field of Search: 706 20- 21, 706/25, 706 59- 61, 706/1, 706/57, 706/12, 706/13, 706/16, 706/24, 706 46- 49, 706 51- 55, 382/156, 382/159, 382/228, 382/103, 382/140, 382/149, 382/151, 382164-165, 382/170, 382/172, 382/174, 382/181, 382/224, 382/227, 382/229, 382/231, 600/408
• US Class Current: 706/49
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation

Abstract

An exemplary embodiment of the present invention is directed to a system, method and computer program product for providing an object recognition blackboard system architecture. The system for recognizing objects in content can include: a blackboard comprising a plurality of experts, and data comprising original input data and data created by processing of any of the plurality of experts, and a controller operative to control the experts; a belief model, coupled to the controller, comprising a set of beliefs and probabilities associated with each belief of the set of beliefs; a belief network, coupled to the controller; and a relations subsystem, coupled to the controller.

Claims

1. A computer readable medium encoded with instructions for causing a computer to recognize objects in content by following the instructions to implement:

a blackboard comprising

a plurality of experts adapted to process data on a computer, and said data comprising at least one of original input data and/or data created by processing of any of said plurality of experts, and

a controller operative to control said experts;

a belief model, coupled to said controller, comprising a set of beliefs and probabilities associated with each belief of said set of beliefs, wherein said belief model comprises a set of rules deduced from a computer-implemented learning system, said learning system comprising truth data files for deducing said set of beliefs, said probabilities and shadow objects, a learning system controller and a statistics space controlled by said learning system controller, wherein said set of rules describes how different classes recognized by said learning system are related to each other spatially and physically;

a belief network, coupled to said controller; and

a relations subsystem, coupled to said controller.

2. The computer-readable medium according to claim 1, wherein said experts comprise expert object recognizers comprising at least one of:

region identification experts;

color region experts;

a corner recognizer;

a closed curve recognizer;

a roof recognizer;

a text recognizer;

simulated experts;

microphone recognizer;

space suit recognizer;

satellite recognizer;

a geometric shape recognizer;

a building recognizer;

an egg recognizer;

a dice recognizer;

a person recognizer;

a face recognizer; and

/or

a product recognizer.

3. The computer-readable medium according to claim 1, wherein said data comprises at least one of:

relations data;

expert status data;

image subsection data; and

/or

said belief model.

4. The computer-readable medium according to claim 1, wherein said controller is at least one of:

operative to choose chosen experts from said plurality of experts which are to be executed;

operative to schedule execution of said chosen experts; and

/or

operative to execute said chosen experts.

5. The computer-readable medium according to claim 1, wherein said blackboard further comprises at least one of:

storage for receiving an input image; and

/or

a reporter operative to output results of processing.

6. The computer-readable medium according to claim 1, wherein said belief model is operative to predict existence of a shadow object in an image even if there are no specific experts capable of recognizing said shadow object.

7. The computer-readable medium according to claim 1, wherein said belief network is operative to combine beliefs in output data output by said experts and probabilities drawn from said belief model into a single belief for a given object.

8. The computer-readable medium according to claim 1, wherein said relations subsystem is operative to determine how returned objects returned by said experts are related to each other.

9. The computer-readable medium according to claim 1, wherein said relations subsystem is operative to determine spatial relations.

10. The computer-readable medium according to claim 9, wherein said spatial relations include types comprising at least one of:

a north type, a south type, an east type, a west type,

a contains type,

a contained by type, and/or

an adjacent to type.

11. The computer-readable medium according to claim 1, wherein said relations subsystem is operative to determine temporal relations.

12. The computer-readable medium according to claim 11, wherein said temporal relations include types comprising at least one of:

a before type,

an after type, and/or

an exists with type.

13. The system-computer-readable medium according to claim 1, wherein the content comprises at least one of:

video;

an image;

digitized content; and

/or a frame.

14. The computer-readable medium according to claim 1, wherein said learning system is operative to assist in integrating a new expert, said new expert being adapted to create, encapsulate and compile said new expert;

to add a stub function to said blackboard;

if output of said new expert is new, to add the output to said belief model; and

to add a blackboard rule to control when said new expert is to be executed.

15. The computer-readable medium of claim 1, wherein said belief network is at least one of:

a Bayesian Network;

a mean probability; and

/or

a Dempster-Shafer Network.

16. The computer-readable medium according to claim 1, wherein said belief model comprises:

rules operative to be used to make a determination whether or not one of said experts should be executed by search of said belief model to determine whether an adaptable threshold of supporting evidence has been exceeded for an execution supportability rule that evaluates outputs of currently executing experts.

17. The computer-readable medium according to claim 1, wherein said belief model is operative to model expected object associations, to weigh relative object positions, and to tie a probability or belief value to those associations.

18. The computer-readable medium according to claim 1, wherein said belief network is operative to combine the belief model with hypotheses generated by said experts to form belief values for hypothesized objects.

19. A computer-readable medium encoded with instructions for causing a computer to implement a method of recognizing objects comprising:

identifying classes of objects specified by a user using a plurality of cooperative object recognition experts;

achieving higher accuracy from using in parallel said plurality of cooperative object recognition experts than is achievable using in serial said plurality of cooperative object recognition experts;

supporting scaleability of performance including supporting multiple computer processors;

developing a belief model by deducing a set of rules from a computer-implemented learning system, said learning system comprising truth data files for deducing beliefs, probabilities and shadow objects, a learning system controller and a statistics space controlled by said learning system controller, said set of rules describing how different classes recognized by said learning system are related to each other spatially and physically, the developing of said belief model including

specifying specified associations among said objects,

learning learned associations among said objects,

representing said specified and learned associations, and

forming a belief network

wherein said belief network is at least one of a Bayesian Network and a Dempster Shafer Network; and

deducing said shadow objects from said belief model.

20. A computer-readable medium encoded with instructions for causing a computer to implement a method for adding a new expert to a blackboard comprising:

creating an expert adapted for processing data on a computer;

encapsulating said expert;

compiling said expert;

adding a stub function to a blackboard;

determining if output of said expert is new and if new, then

adding the output'"'"'s class to said blackboard, and

updating a belief model by providing truth data file data to a computer-implemented learning system, said learning system comprising truth data files for deducing beliefs, probabilities and shadow objects, a learning system controller and a statistics space controlled by said learning system controller;

creating a rule to control when said new expert is to be executed when supporting evidence is found to exceed an adaptable threshold; and

deducing a set of rules from said learning system, said set of rules describing how different classes recognized by said learning system are related to each other spatially and physically.