Apparatus, method and program storage medium for image interpretation

• US 20080037904A1
• Filed: 07/16/2007
• Published: 02/14/2008
• Est. Priority Date: 08/14/2006
• Status: Abandoned Application

• Current Assignee: OKI Electric Industry Company Limited
• Original Assignee: OKI Electric Industry Company Limited
• Inventors: Hiramoto, Michiyo, Ohkuma, Yoshinori, Murata, Shinichi
• Application Number:
  US11/826,467
• Publication Number:
  US 20080037904A1
• Time in Patent Office: Days
• US Class Current: 382/306
• CPC Class Codes: G06F 16/535 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06V 10/225 based on a marking or identifier characterising the area
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors

Abstract

An image interpretation apparatus including a registration section, an image search section, and an image interpretation section is provided. The registration section registers an object image in an object database. The image search section searches a type, an attribute, and an arrangement, or a combination of the object images included in an input image. The image interpretation section interprets semantics of the input image based on the arrangement, the combination, or the like. Due to this configuration, plural pieces of semantics can be given to a single image, and a complex subsequent-stage process can be performed according to an image interpretation result.

Claims

1. An image interpretation apparatus comprising:

a registration image information storage section which includes an object database in which an object image expressing a single object, at least one feature being able to specify a type of the object image, and semantic information corresponding to the object image are registered in correlation with one another;

an image obtaining section which obtains an input image to be a subject of interpretation of semantics;

an object image extraction section which scans the input image to detect its features, and extracts the registered object image included in the input image and the semantic information corresponding to the object image;

an arrangement information obtaining section which obtains arrangement information indicating a relationship between the input image and the object image;

a grammatical rule information storage section which includes a grammatical rule database in which at least one grammatical rule is registered for adding additional semantics to the input image corresponding to the relationship between the input image and the object image; and

an image interpretation section which retrieves the at least one grammatical rule based on the arrangement information, and interprets the semantics of the input image based on the semantic information on the object image and the at least one grammatical rule.

2. The image interpretation apparatus of claim 1, wherein:

the arrangement information includes positional information indicating a position of each object image in the input image;

the at least one grammatical rule is a rule for selecting a single piece of semantic information from the semantic information corresponding to the object image, according to the positional information; and

the image interpretation section interprets the semantic information of the object image selected according to the at least one grammatical rule, as the semantics of the input image.

3. The image interpretation apparatus of claim 1, wherein:

the arrangement information includes morphological information on a size and/or a gradient of the object image;

the at least one grammatical rule defines a method of computing an evaluation value, the parameters of which are based on the morphological information; and

the image interpretation section interprets the semantics of the input image by adding the evaluation value computed according to the at least one grammatical rule.

4. The image interpretation apparatus of claim 1, wherein:

the arrangement information obtaining section comprises a combination information obtaining section which, when a plurality of object images are extracted by the object image extraction section, further obtains combination information indicating a relationship between one of the extracted object images and the other extracted object images;

the grammatical rule information storage section comprises a combination rule information storage section which includes a combination rule database in which at least one grammatical rule is registered for adding additional semantics to the input image corresponding to the relationship between the object images; and

the image interpretation section retrieves the at least one grammatical rule based on the arrangement information and the combination information, and interprets the semantics of the input image based on the semantic information of the object image and the at least one grammatical rule.

5. The image interpretation apparatus of claim 1, wherein:

the arrangement information includes at least one of;

(a) positional information indicating a position of each object image in the input image;

(b) combination information indicating, when a plurality of object images are extracted, a relationship between one of the extracted object images and the other extracted object images;

or

(c) missing information on a missing region of the extracted object image.

6. The image interpretation apparatus of claim 4, wherein:

the combination information includes positional information indicating a relative positional relationship of the plurality of extracted object images;

the at least one grammatical rule defines a joining relationship of the semantic information corresponding to each object image according to the positional information; and

the image interpretation section interprets the semantic information on the plurality of object images, which are joined according to the at least one grammatical rule, as the semantics of the input image.

7. The image interpretation apparatus of claim 1, wherein:

the arrangement information obtaining section comprises a missing information obtaining section which detects missing information on a missing region of the extracted object image;

the grammatical rule information storage section comprises a missing rule information storage section which includes a combination rule database in which at least one grammatical rule is registered for adding additional semantics to the input image based on a missing percentage of the object images; and

the image interpretation section retrieves the at least one grammatical rule based on the missing information, and interprets the semantics of the input image based on the semantic information on the object image and the at least one grammatical rule.

8. The image interpretation apparatus of claim 7, wherein:

the missing information includes missing area information indicating an area ratio of an area of the detected missing region to an area of the object image;

the at least one grammatical rule defines a computation method in which a quantitative value included in the semantic information corresponding to the object image is changed according to the area ratio; and

the image interpretation section interprets a quantitative value of the object image, which is computed according to the at least one grammatical rule, as the semantics of the input image.

9. An image interpretation method comprising:

registering an object image expressing a single object, at least one feature being able to specify a type of the object image, and semantic information corresponding to the object image in an object database, wherein the object image, the at least one feature, and the semantic information are correlated with one another;

obtaining an input image to be a subject for interpretation of semantics;

scanning the input image to detect its features, and extracting the registered object image included in the input image and the semantic information corresponding to the object image;

obtaining arrangement information indicating a relationship between the input image and the object image;

registering in an arrangement rule database at least one grammatical rule for adding additional semantics to the input image, corresponding to the relationship between the input image and the object image; and

retrieving the at least one grammatical rule based on the arrangement information, and interpreting the semantics of the input image based on the semantic information on the object image and the at least one grammatical rule.

10. The image interpretation method of claim 9, further comprising:

extracting a plurality of registered object images included in the input image, and the semantic information corresponding to the object images;

obtaining combination information indicating a relationship between one of the extracted object images and the other extracted object images;

registering in a combination rule database at least one grammatical rule for adding additional semantics to the input image, corresponding to the relationship between the object images; and

retrieving the at least one grammatical rule based on the combination information, and interpreting the semantics of the input image based on the semantic information on the object image and the at least one grammatical rule.

11. The image interpretation method of claim 9, further comprising:

detecting missing information on a missing region of the extracted object image;

registering in a combination rule database at least one grammatical rule for adding additional semantics to the input image, corresponding to a missing percentage of the object image; and

retrieving the at least one grammatical rule based on the missing information, and interpreting the semantics of the input image based on the semantic information on the object image and the at least one grammatical rule.

12. A machine-readable storage medium storing a program for causing a computer to execute an image interpretation process, the process comprising:

registering an object image expressing a single object, at least one feature being able to specify a type of the object image, and semantic information corresponding to the object image in an object database, wherein the object image, the at least one feature, and the semantic information are correlated with one another;

obtaining an input image to be a subject for interpretation of semantics;

scanning the input image to detect its features, and extracting the registered object image included in the input image and the semantic information corresponding to the object image;

obtaining arrangement information indicating a relationship between the input image and the object image;

registering in an arrangement rule database at least one grammatical rule for adding additional semantics to the input image, corresponding to the relationship between the input image and the object image; and

retrieving the at least one grammatical rule based on the arrangement information, and interpreting the semantics of the input image based on the semantic information on the object image and the at least one grammatical rule.

13. The machine-readable storage medium of claim 12, the process further comprising:

scanning the input image to detect its features, and extracting a plurality of registered object images included in the input image and the semantic information corresponding to the object images;

obtaining combination information indicating a relationship between one of the extracted input images and the other extracted object images;

registering in a combination rule database at least one grammatical rule for adding additional semantics to the input image, corresponding to the relationship between the object images; and

retrieving the at least one grammatical rule based on the combination information, and interpreting the semantics of the input image based on the semantic information on the object image and the at least one grammatical rule.

14. The machine-readable storage medium of claim 12, the process further comprising:

detecting missing information on a missing region of the extracted object image;

registering in a combination rule database at least one grammatical rule for adding additional semantics to the input image, corresponding to a missing percentage of the object image; and

retrieving the at least one grammatical rule based on the missing information, and interpreting the semantics of the input image based on the semantic information on the object image and the at least one grammatical rule.