Smart image filtering method with domain rules application

• US 10,169,649 B2
• Filed: 07/28/2016
• Issued: 01/01/2019
• Est. Priority Date: 07/28/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kochura, Nadiya, Lu, Fang
• Primary Examiner(s): Thomas, Mia M
• Application Number:
  US15/221,718
• Publication Number:
  US 20180032803A1
• Time in Patent Office: 887 Days
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 10/762 using clustering, e.g. of similar faces in social networks
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

An image filtering system for compliance assessment is built and modified dynamically. The system defines a vocabulary for a domain by extracting entities from corpus documents and grouping entities into image categories. An image catalog is built by identifying objects in images from a system image catalog and associating the objects with entities in the image categories. Compliance rules are generated by identifying and using image categories related to topics in corpus data sources. An image classification pipeline is then trained to classify images according to the compliance rules until performance metrics are met. The training process may be repeated to extract new and updated knowledge in corpus document and/or corpus data sources. For compliance assessment of an image, entities in related image categories are identified. The image is then classified according to actions associated with any applicable compliance rules.

Claims

1. A method for building computing components of an image filtering system prior to a runtime assessment of images for compliance by the image filtering system, comprising:

defining, by the image filtering system, a vocabulary for a given domain, comprising;

extracting a set of entities from a set of corpus documents related to the given domain; and

grouping the set of entities into a set of image categories based on probabilities of co-occurrence of the set of entities;

building, by the image filtering system, an image catalog, comprising;

identifying one or more objects in one or more images from the image catalog; and

associating the one or more objects with the set of entities in the set of image categories;

generating, by the image filtering system, a set of compliance rules, comprising;

identifying a given image category of the set of image categories that relate to topics in corpus data sources; and

generating the set of compliance rules, using the set of entities grouped in the given image category; and

training, by the image filtering system, an image classification pipeline, comprising;

identifying one or more entities in a given image of the one or more images from the image catalog;

classifying the given image according to one or more of the set of compliance rules; and

repeating the identifying of the one or more entities in the given image and the classifying of additional images until a set of performance metrics are met.

2. The method of claim 1, further comprising:

repeating the defining of the vocabulary, the building of the image catalog, the generating of the set of compliance rules, and the training of the image classification pipeline to incorporate new or updated corpus documents or corpus data sources.

3. The method of claim 1, wherein the defining of the vocabulary for the given domain further comprises building a set of anchor keywords,

wherein the identifying of the image categories related to the topics in the corpus data sources comprises identifying the topics in the corpus data sources using the set of anchor keywords.

4. The method of claim 1, wherein the generating of the set of compliance rules using the one or more image categories related to the topics comprises:

retrieving the set of entities grouped in the given image category;

searching within the topics using the set of entities grouped in the given image category;

creating a given compliance rule of the set of compliance rules using entities found in the corpus data sources matching the set of entities grouped in the given image category; and

associating an action with the given compliance rule.

5. The method of claim 1, wherein the classifying of the given image according to one or more of the set of compliance rules comprises:

comparing the one or more entities in the given image with entities in the set of compliance rules;

determining whether any of the set of compliance rules apply to the given image based on the comparison; and

classifying the given image based on an action associated with each applicable compliance rule.

6. The method of claim 1, further comprising:

receiving a second given image for compliance assessment;

identifying related image categories from the set of image categories using image metadata of the second given image;

identifying one or more entities in the second given image matching entities in the related image categories;

creating a list of the entities found in the second given image;

determining whether any of the set of compliance rules apply to the list of the entities found in the second given image;

classifying the second given image according to an action associated with each applicable compliance rule; and

performing the action associated with each applicable compliance rule.

7. The method of claim 6, wherein the performing of the action associated with each applicable compliance rule comprises:

altering the second given image based on the action; and

returning the altered second given image as a result of the compliance assessment.

8. A computer program product for building computing components of an image filtering system prior to a runtime assessment of images for compliance by the image filtering system, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:

define a vocabulary for a given domain, comprising;

extracting a set of entities from a set of corpus documents related to the given domain; and

grouping the set of entities into a set of image categories based on probabilities of co-occurrence of the set of entities;

build an image catalog, comprising;

identifying one or more objects in one or more images from the image catalog; and

associating the one or more objects with the set of entities in the set of image categories;

generate a set of compliance rules, comprising;

identifying a given image category of the set of image categories that relating to topics in corpus data sources; and

generating the set of compliance rules, using the set of entities grouped in the given image category; and

train an image classification pipeline, comprising;

identifying one or more entities in a given image of the one or more images from the image catalog;

classifying the given image according to one or more of the set of compliance rules; and

repeating the identifying of the one or more entities in the given image and the classifying of additional images until a set of performance metrics are met.

9. The computer program product of claim 8, further cause the processor to:

repeat the defining of the vocabulary, the building of the image catalog, the generating of the set of compliance rules, and the training of the image classification pipeline to incorporate new or updated corpus documents or corpus data sources.

10. The computer program product of claim 8, wherein the defining of the vocabulary for the given domain further comprises building a set of anchor keywords,

wherein the identifying of the image categories related to the topics in the corpus data sources comprises identifying the topics in the corpus data sources using the set of anchor keywords.

11. The computer program product of claim 8, wherein the generating of the set of compliance rules using the one or more image categories related to the topics comprises:

retrieve the set of entities grouped in the given image category;

search within the topics using the set of entities grouped in the given image category;

create a given compliance rule of the set of compliance rules using entities found in the corpus data sources matching the set of entities grouped in the given image category; and

associate an action with the given compliance rule.

12. The computer program product of claim 8, wherein the classifying of the given image according to one or more of the set of compliance rules comprises:

compare the one or more entities in the given image with entities in the set of compliance rules;

determine whether any of the set of compliance rules apply to the given image based on the comparison; and

classify the given image based on an action associated with each applicable compliance rule.

13. The computer program product of claim 8, further causing the processor to:

receive a second given image for compliance assessment;

identify related image categories from the set of image categories using image metadata of the second given image;

identify one or more entities in the second given image matching entities in the related image categories;

create a list of the entities found in the second given image;

determine whether any of the set of compliance rules apply to the list of the entities found in the second given image;

classify the second given image according to an action associated with each applicable compliance rule; and

perform the action associated with each applicable compliance rule.

14. The computer program product of claim 13, wherein the performing of the action associated with each applicable compliance rule comprises:

alter the second given image based on the action; and

return the altered second given image as a result of the compliance assessment.

15. An image filtering system, comprising:

a processor; and

a computer readable storage medium having program instructions embodied therewith for building computing components of the image filtering system prior to a runtime assessment of images for compliance by the image filtering system, the program instructions executable by the processor to cause the processor to;

define a vocabulary for a given domain, comprising;

extracting a set of entities from a set of corpus documents related to the given domain; and

grouping the set of entities into a set of image categories based on probabilities of co-occurrence of the set of entities;

build an image catalog, comprising;

identifying one or more objects in one or more images from the image catalog; and

associating the one or more objects with the set of entities in the set of image categories;

generate a set of compliance rules, comprising;

identifying a given image category of the set of image categories that relate to topics in corpus data sources; and

generating the set of compliance rules, using the set of entities grouped in the given image category; and

train an image classification pipeline, comprising;

identifying one or more entities in a given image of the one or more images from the image catalog;

classifying the given image according to one or more of the set of compliance rules; and

repeating the identifying of the one or more entities in the given image and the classifying of additional images until a set of performance metrics are met.

16. The system of claim 15, further cause the processor to:

repeat the defining of the vocabulary, the building of the image catalog, the generating of the set of compliance rules, and the training of the image classification pipeline to incorporate new or updated corpus documents or corpus data sources.

17. The system of claim 15, wherein the generating of the set of compliance rules using the one or more image categories related to the topics comprises:

retrieve the set of entities grouped in the given image category;

search within the topics using the set of entities grouped in the given image category;

create a given compliance rule of the set of compliance rules using entities found in the corpus data sources matching the set of entities grouped in the given image category; and

associate an action with the given compliance rule.

18. The system of claim 15, wherein the classifying of the given image according to one or more of the set of compliance rules comprises:

compare the one or more entities in the given image with entities in the set of compliance rules;

determine whether any of the set of compliance rules apply to the given image based on the comparison; and

classify the given image based on an action associated with each applicable compliance rule.

19. The system of claim 15, further causing the processor to:

receive a second given image for compliance assessment;

identify related image categories from the set of image categories using image metadata of the second given image;

identify one or more entities in the second given image matching entities in the related image categories;

create a list of the entities found in the second given image;

determine whether any of the set of compliance rules apply to the list of the entities found in the second given image;

classify the second given image according to an action associated with each applicable compliance rule; and

perform the action associated with each applicable compliance rule.

20. The system of claim 19, wherein the performing of the action associated with each applicable compliance rule comprises:

alter the second given image based on the action; and

return the altered second given image as a result of the compliance assessment.