Proactive input selection for improved image analysis and/or processing workflows

• US 10,311,330 B2
• Filed: 08/17/2016
• Issued: 06/04/2019
• Est. Priority Date: 08/17/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Anglin, Debbie A., Calcaterra, Jeffrey A., Liu, Su, Xu, Cheng
• Primary Examiner(s): Niu, Feng
• Application Number:
  US15/239,716
• Publication Number:
  US 20180053068A1
• Time in Patent Office: 1,021 Days
• Field of Search: None
• CPC Class Codes: G06V 10/96 Management of image or video recognition tasks
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

Systems, methods, and computer program products for proactively improving image analysis and/or processing in real time are presented, in accordance with various embodiments. In one embodiment, a computer-implemented method includes: receiving an analysis and/or processing request from a mobile device; and either generating or retrieving analysis and/or processing information based on a result of either or both of: determining whether the analysis and/or processing request includes image data suitable for generating an analysis and/or processing result; and determining whether the analysis and/or processing request includes metadata suitable for generating an estimated analysis and/or processing result. In another embodiment, a computer-implemented method includes: capturing image data; evaluating one or more conditions corresponding to the captured image data; generating metadata corresponding to the captured image data; and generating and submitting the analysis and/or processing request to an analysis and/or processing platform. Corresponding systems and computer program products are also described.

Claims

1. A computer-implemented method for proactively improving image analysis and/or processing in real time by processing an analysis and/or processing request, the method comprising:

receiving the analysis and/or processing request from a mobile device, the analysis and/or processing request including metadata and condition data each corresponding to an image for which analysis and/or processing is requested, and the analysis and/or processing request excluding the image;

determining, based on the metadata, whether an analysis and processing result corresponding to the image exists in a database of analysis and processing results; and

in response to determining the analysis and processing result corresponding to the image exists in the database of analysis and processing results;

retrieving, without performing any analysis or processing of the image, the analysis and processing result corresponding to the image;

or

generating, without performing any analysis or processing of the image, an estimated analysis and/or processing result corresponding to the image, wherein the generating is based at least in part on the metadata.

2. The computer-implemented method as recited in claim 1, wherein the analysis and/or processing information comprises information selected from the group consisting of:

the analysis and/or processing result, the estimated analysis and/or processing result, and contextual information; and

the computer-implemented method further comprising;

retrieving the contextual information in response to determining both;

the analysis and/or processing request does not include image data characterized by an image quality which permits accurately generating an analysis and/or processing result; and

the analysis and/or processing request does not include metadata which permits accurately generating an estimated analysis and/or processing result.

3. The computer-implemented method as recited in claim 1, comprising, in response to determining the analysis and/or processing request includes image data characterized by an image quality which permits accurately generating an analysis and/or processing result, generating the analysis and/or processing information, wherein generating the analysis and/or processing information comprises:

extracting information from the image data;

analyzing and/or processing some or all of the extracted information using at least one of a plurality of analysis and/or processing models; and

for each of the plurality of analysis and/or processing models used to analyze and/or process some or all of the extracted information, storing a corresponding analysis and/or processing result in an analysis and/or processing result repository.

4. The computer-implemented method as recited in claim 1, comprising, in response to determining the analysis and/or processing request includes image data characterized by an image quality which permits accurate analysis and/or processing, generating the analysis and/or processing information, wherein generating the analysis and/or processing information comprises:

determining whether an image corresponding to the image data included in the analysis and/or processing request exists in an image data and/or metadata repository;

in response to determining the image corresponding to the image data included in the analysis and/or processing request exists in the image data and/or metadata repository, determining a corresponding image identifier;

identifying previously analyzed and/or processed data based on the corresponding image identifier; and

retrieving the analysis and/or processing information from the previously analyzed and/or processed data.

5. The computer-implemented method as recited in claim 1, comprising generating the estimated analysis and/or processing result without performing analysis and/or processing in response to determining both:

the analysis and/or processing request does not include image data characterized by an image quality which permits accurate analysis and/or processing; and

the analysis and/or processing request does include image metadata which permits accurately generating the estimated analysis and/or processing result; and

wherein generating the estimated analysis and/or processing result comprises;

determining whether an image corresponding to the image data included in the analysis and/or processing request exists in an image data and/or metadata repository based at least in part on the image metadata;

in response to determining the corresponding image exists in the image data and/or metadata repository;

identifying previously analyzed and/or processed data associated with the corresponding image; and

generating the estimated analysis and/or processing result based on the previously analyzed and/or processed data.

6. The computer-implemented method as recited in claim 1, comprising, in response to determining the analysis and/or processing request does not include image data suitable for analysis and/or processing,

determining whether an image corresponding to the image data included in the analysis and/or processing request exists in an image data and/or metadata repository, wherein the corresponding image is characterized by an image quality which permits accurately generating the analysis and/or processing result;

in response to determining the corresponding image exists in the image data and/or metadata repository,

extracting information from the corresponding image;

analyzing and/or processing some or all of the extracted information using at least one of a plurality of analysis and/or processing models; and

for each of the plurality of analysis and/or processing models used to analyze and/or process some or all of the extracted information, storing a corresponding analysis and/or processing result in an analysis and/or processing result repository.

7. The computer-implemented method as recited in claim 1, comprising one or more of establishing and updating:

an image data and/or metadata repository; and

an analysis and/or processing result repository; and

wherein either or both of the establishing and the updating the image data and/or metadata repository comprises;

receiving one or more images;

receiving and/or generating metadata descriptive of each of the one or more of images, and associating the received and/or generated metadata with the image of which the metadata are descriptive;

classifying at least some of the one or more of images, wherein the classifying of each one of the one or more images is optionally based at least in part on the respective metadata associated with the image;

associating a different image identifier with each classified image; and

storing each classified image and the associated image identifier in the image data and/or metadata repository.

8. The computer-implemented method as recited in claim 7, comprising normalizing at least some of the classified images.

9. The computer-implemented method as recited in claim 8, wherein the normalizing comprises generating a composite image from a plurality of images in the image data and/or metadata repository, wherein the composite image either:

includes textual information or a portion of an object depicted in only some of the plurality of images;

excludes distortion present in at least some of the plurality of images;

or

includes the textual information or the portion of the object depicted in only some of the plurality of images and excludes the distortion present in at least some of the plurality of images.

10. The computer-implemented method as recited in claim 8, wherein the normalizing comprises ranking a plurality of images in the image data and/or metadata repository according to a quality of each of the plurality of images;

designating a top ranked one of the plurality of images for use in generating the analysis and/or processing information for future analysis and/or processing requests corresponding to the top ranked one of the plurality of images with respect to at least one capture characteristic selected from the group consisting of;

capture time, capture location, camera position, camera orientation, and camera azimuth; and

designating analysis and/or processing information corresponding to the top ranked one of the plurality of images for use in retrieving the analysis and/or processing information for future analysis and/or processing requests corresponding to the top ranked one of the plurality of images with respect to at least one capture characteristic selected from the group consisting of;

capture time, capture location, camera position, camera orientation, and camera azimuth; and

wherein each of the plurality of images depicts an image characteristic selected from the group consisting of;

a same subject, a same location, and a same set of textual information.

11. The computer-implemented method as recited in claim 10, wherein the ranking is based on analyzing metadata associated with each respective one of the plurality of images, the metadata comprising:

one or more indicia of the quality of the respective one of the plurality of images; and

capture parameters relevant to analyzing and/or processing the respective one of the plurality of images, or both.

12. The computer-implemented method as recited in claim 11, wherein the metadata are selected from the group consisting of:

capture time, capture location, camera position, camera orientation, camera azimuth, environmental conditions at a time of capturing an image, and processing results achieved in response to previously analyzing and/or processing the image.

13. The computer-implemented method as recited in claim 10, comprising generating the analysis and/or processing information using the top ranked one of the plurality of images in response to determining image for which analysis and/or processing is requested is characterized by an image quality which precludes accurately generating an analysis and/or processing result.

14. The computer-implemented method as recited in claim 10, comprising retrieving the analysis and/or processing information corresponding to the top ranked one of the plurality of images in response to determining both of:

the analysis and/or processing request does not include image data from which an accurate analysis and/or processing result can be generated; and

the analysis and/or processing request does not include metadata from which an accurate estimated analysis and/or processing result can be generated.

15. The computer-implemented method as recited in claim 1, comprising one or more of establishing and updating:

an image data and/or metadata repository; and

an analysis and/or processing result repository;

wherein either or both of the establishing and updating, the analysis and/or processing result repository comprises;

receiving one or more images;

extracting information depicted in each of the one or more images;

analyzing and/or processing the information extracted from each respective one of the one or more images according to one or more of a plurality of analysis and/or processing models;

associating a different analysis and/or processing result identifier with each result of analyzing and/or processing the information according to the one or more of the plurality of analysis and/or processing models; and

storing each analysis and/or processing result and the associated analysis and/or processing result identifier in the analysis and/or processing result repository.

16. The computer-implemented method as recited in claim 15, comprising:

associating a different extraction identifier with the information extracted from each respective one of the one or more images; and

storing the information extracted from each respective one of the one or more images in association with the respective different extraction identifier in an extracted information repository.

17. The computer-implemented method as recited in claim 1, wherein the condition data are selected from the group consisting of:

a time of day corresponding to capturing the image, environmental conditions corresponding to capturing the image, and network characteristics.

18. The computer-implemented method as recited in claim 1, wherein the metadata are descriptive of image capture conditions selected from the group consisting of:

capture device position;

capture device orientation;

capture azimuth;

image resolution;

capture flash setting; and

image illumination level.

19. The computer-implemented method as recited in claim 1, wherein the image is characterized by an image quality that precludes accurate analysis or processing.

20. The computer-implemented method as recited in claim 1, wherein the analysis and processing request is a request to translate text depicted in the image.

21. A computer program product for proactively improving image analysis and/or processing in real time, said computer program product comprising:

a computer readable storage medium having stored thereon first program instructions executable by a processor to cause the processor to retrieve, without translating any text, a translation of textual information depicted in image data associated with a translation request, the retrieving being performed in response to;

receiving the translation request, wherein the translation request excludes the image data; and

either

determining, using the processor, the image data is characterized by an image quality which precludes accurate, direct translation of the textual information;

or

determining, using the processor, the translation request includes metadata identifying a second image associated with a previous translation result corresponding to the textual information, the previous translation result being determined from analyzing the second image corresponding to the image data associated with the translation request; and

wherein the metadata comprise a capture device position;

a capture device orientation; and

a capture azimuth at the time of capturing the image data using the capture device.