Predicate logic based image grammars for complex visual pattern recognition

• US 8,548,231 B2
• Filed: 03/16/2010
• Issued: 10/01/2013
• Est. Priority Date: 04/02/2009
• Status: Expired due to Fees

• Current Assignee: Siemens Corp. (Siemens AG)
• Original Assignee: Siemens Corp. (Siemens AG)
• Inventors: Shet, Vinay Damodar, Singh, Maneesh Kumar, Bahlmann, Claus, Ramesh, Visvanathan, Masticola, Stephen P., Neumann, Jan, Parag, Toufiq, Gall, Michael A., Suarez, Roberto Antonio
• Primary Examiner(s): Zarka, David
• Assistant Examiner(s): Potts, Ryan P
• Application Number:
  US12/724,954
• Publication Number:
  US 20100278420A1
• Time in Patent Office: 1,295 Days
• Field of Search: None
• US Class Current: 382/156
• CPC Class Codes: G06F 18/24765 Rule-based classification
• CPC Class Codes: G06V 10/765 using rules for classification or partitioning the feature space
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

First order predicate logics are provided, extended with a bilattice based uncertainty handling formalism, as a means of formally encoding pattern grammars, to parse a set of image features, and detect the presence of different patterns of interest implemented on a processor. Information from different sources and uncertainties from detections, are integrated within the bilattice framework. Automated logical rule weight learning in the computer vision domain applies a rule weight optimization method which casts the instantiated inference tree as a knowledge-based neural network, to converge upon a set of rule weights that give optimal performance within the bilattice framework. Applications are in (a) detecting the presence of humans under partial occlusions and (b) detecting large complex man made structures in satellite imagery (c) detection of spatio-temporal human and vehicular activities in video and (c) parsing of Graphical User Interfaces.

Claims

1. A method for a detection of a pattern having one or more features from image data of a scene, comprising:

defining a set of rules to detect the pattern based on the one or more features using a plurality of first order logic bilattice predicates;

obtaining the image data related to the scene;

a processor processing the image data with one or more detectors to detect the one or more features;

the processor executing the set of rules to detect the pattern based on a presence or an absence of each of the one or more features;

the processor generating data related to;

a justification that the set of rules detected the pattern;

a location in the scene where the pattern occurs, and

a measure of uncertainty related to the detection of the pattern;

wherein the set of rules is implemented as a knowledge-based artificial neural network;

wherein the measure of uncertainty related to the rule is expressed as a link weight in the knowledge-based artificial neural network; and

wherein

the link weight is optimized by applying a change being expressed as;

2. The method as claimed in claim 1, wherein a bilattice is a set of 2-tuples, quantifying a measure of belief for an assertion and a measure of belief against it.

3. The method as claimed in claim 1, wherein the one or more rules include a measure of beliefs related to a rule.

4. The method as claimed in claim 1, further comprising a set of composition rules which encodes the pattern as a hierarchy of parts.

5. The method as claimed in claim 1, further comprising one or more embodiment rules which encode a geometric layout of the pattern.

6. The method as claimed in claim 1, further comprising one or more context rules which encode a context of the pattern.

7. The method as claimed in claim 1, wherein the method is applied to detect an object.

8. The method as claimed in claim 1, wherein the method is applied to detect a human.