Identifying updated content in images

• US 9,607,216 B2
• Filed: 03/30/2015
• Issued: 03/28/2017
• Est. Priority Date: 03/30/2015
• Status: Expired due to Fees

• Current Assignee: AirBNB Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Bostick, James E., Ganci, John M. Jr., Keen, Martin G., Rakshit, Sarbajit K., Trim, Craig M.
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): RIVERA-MARTINEZ, GUILLERMO M
• Application Number:
  US14/672,441
• Publication Number:
  US 20160292502A1
• Time in Patent Office: 729 Days
• US Class Current: 1/1
• CPC Class Codes: G06V 10/768 using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/39 Urban scenes
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition

Abstract

Aspects of the present invention disclose a method, computer program product, and system for analyzing an image. The method includes one or more processors identifying a document that includes (i) at least one image and (ii) unstructured text that is situated proximate to the image, wherein both the image and the unstructured text are concurrently displayed in the document. The method further includes identifying one or more potential locations depicted in the image included in the identified document, based on an analysis of the unstructured text that is situated proximate to the image. The method further includes identifying a set of images of a first potential location from the identified one or more potential locations. The method further includes determining whether an image in the set of images of the first potential location substantially matches the image included in the identified document.

Claims

1. A method for analyzing an image, the method comprising:

identifying, by one or more processors, a document that includes (i) at least one image and (ii) unstructured text that is situated proximate to the image, wherein both the image and the unstructured text are concurrently displayed in the document;

identifying, by one or more processors, one or more potential locations depicted in the image included in the identified document, based on an analysis of the unstructured text that is situated proximate to the image;

identifying, by one or more processors, a set of images of a first potential location from the identified one or more potential locations;

responsive to determining that an image in the set of images of the first potential location substantially matches the image included in the identified document, determining, by one or more processors, that the location depicted in the image that substantially matches the image included in the identified document is the location depicted in the image included in the identified document;

identifying, by one or more processors, an updated image that includes the location depicted in the image included in the identified document, wherein the updated image is associated with a date that is more recent than a date associated with the image included in the identified document; and

initiating, by one or more processors, display of the identified updated image within the identified document.

2. The method of claim 1, wherein identifying the updated image that includes the location depicted in the image included in the identified document further comprises:

responsive to determining that an image in the set of images of the first potential location substantially matches the image included in the identified document, identifying, by one or more processors, a set of tagged images that are tagged with metadata indicating location information that corresponds to metadata associated with the determined matching image in the set of images of the first potential location; and

determining, by one or more processors, an updated image of the image included in the identified document from the identified set of tagged images, wherein the determined updated image is the image of the identified set of tagged images that includes metadata indicating a most recent date.

3. The method of claim 1, further comprising:

responsive to determining that no image in the set of images of the first potential location substantially matches the image included in the identified document, identifying, by one or more processors, a set of images of a second potential location from the identified one or more potential locations; and

determining, by one or more processors, whether an image in the set of images of the second potential location substantially matches the image included in the identified document.

4. The method of claim 1, wherein identifying a set of images of a first potential location of the identified one or more potential locations, comprises:

determining, by one or more processors, a confidence value for at least one of the identified one or more potential locations that indicates a likelihood that a potential location is represented in the image included in the identified document, wherein the confidence values are derived from the unstructured text that is situated proximate to the image; and

identifying, by one or more processors, the set of images of the first potential location of the identified one or more potential locations, wherein the first potential location is associated with highest determined confidence value.

5. The method of claim 1 wherein, the identified updated image is displayed as a thumbnail image in proximity to the image included in the identified document and in proximity to the unstructured text that is situated proximate to the image.

6. The method of claim 1, wherein determining whether an image in the set of images of the first potential location substantially matches the image included in the identified document, further comprises:

removing, by one or more processors, objects from foreground of the image included in the identified document and objects from foreground of images of the identified set of images of the first potential location; and

comparing, by one or more processors, the images of the identified set of images of the first potential location with their foreground objects removed to the image included in the identified document with its foreground objects removed.

7. The method of claim 1, wherein identifying one or more potential locations depicted in the image included in the identified document, based on an analysis of the unstructured text that is situated proximate to the image, further comprises:

analyzing, by one or more processors, the unstructured text that is situated proximate to the image to identify text in the identified document indicative of names of locations and a respective distance from the image to said text in the identified document indicative of the names of locations.

8. The method of claim 1, wherein the image included in the identified document is not tagged with metadata that indicates a location depicted in the image.

9. A computer program product for analyzing an image, the computer program product comprising:

one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising;

program instructions to a document that includes (i) at least one image and (ii) unstructured text that is situated proximate to the image, wherein both the image and the unstructured text are concurrently displayed in the document;

program instructions to one or more potential locations depicted in the image included in the identified document, based on an analysis of the unstructured text that is situated proximate to the image;

program instructions to identify a set of images of a first potential location from the identified one or more potential locations;

responsive to determining that an image in the set of images of the first potential location substantially matches the image included in the identified document, program instructions to determine that the location depicted in the image that substantially matches the image included in the identified document is the location depicted in the image included in the identified document;

program instructions to identify an updated image that includes the location depicted in the image included in the identified document, wherein the updated image is associated with a date that is more recent than a date associated with the image included in the identified document; and

program instructions to initiate display of the identified updated image within the identified document.

10. The computer program product of claim 9, wherein identifying the updated image that includes the location depicted in the image included in the identified document further comprises:

responsive to determining that an image in the set of images of the first potential location substantially matches the image included in the identified document, identify a set of tagged images that are tagged with metadata indicating location information that corresponds to metadata associated with the determined matching image in the set of images of the first potential location; and

determine an updated image of the image included in the identified document from the identified set of tagged images, wherein the determined updated image is the image of the identified set of tagged images that includes metadata indicating a most recent date.

11. The computer program product of claim 9, further comprising program instructions, stored on the one or more computer readable storage media, to:

responsive to determining that no image in the set of images of the first potential location substantially matches the image included in the identified document, identify a set of images of a second potential location from the identified one or more potential locations; and

determine whether an image in the set of images of the second potential location substantially matches the image included in the identified document.

12. The computer program product of claim 9, wherein the program instructions to identify a set of images of a first potential location of the identified one or more potential locations, comprises program instructions to:

determine a confidence value for at least one of the identified one or more potential locations that indicates a likelihood that a potential location is represented in the image included in the identified document, wherein the confidence values are derived from the unstructured text that is situated proximate to the image; and

identify the set of images of the first potential location of the identified one or more potential locations, wherein the first potential location is associated with highest determined confidence value.

13. The computer program product of claim 9, wherein the identified updated image is displayed as a thumbnail image in proximity to the image included in the identified document and in proximity to the unstructured text that is situated proximate to the image.

14. The computer program product of claim 9, wherein the program instruction to determine whether an image in the set of images of the first potential location substantially matches the image included in the identified document, further comprise program instructions to:

remove objects from foreground of the image included in the identified document and objects from foreground of images of the identified set of images of the first potential location; and

compare the images of the identified set of images of the first potential location with their foreground objects removed to the image included in the identified document with its foreground objects removed.

15. The computer program product of claim 9, wherein the program instruction to identify one or more potential locations depicted in the image included in the identified document, based on an analysis of the unstructured text that is situated proximate to the image, further comprise program instructions to:

analyze the unstructured text that is situated proximate to the image to identify text in the identified document indicative of names of locations and a respective distance from the image to said text in the identified document indicative of the names of locations.

16. A computer system for analyzing an image, the computer system comprising:

one or more computer processors;

one or more computer readable storage media; and

program instructions stored on the computer readable storage media for execution by at least one of the one or more processors, the program instructions comprising;

program instructions to a document that includes (i) at least one image and (ii) unstructured text that is situated proximate to the image, wherein both the image and the unstructured text are concurrently displayed in the document;

program instructions to one or more potential locations depicted in the image included in the identified document, based on an analysis of the unstructured text that is situated proximate to the image;

program instructions to identify a set of images of a first potential location from the identified one or more potential locations;

responsive to determining that an image in the set of images of the first potential location substantially matches the image included in the identified document, program instructions to determine that the location depicted in the image that substantially matches the image included in the identified document is the location depicted in the image included in the identified document;

program instructions to identify an updated image that includes the location depicted in the image included in the identified document, wherein the updated image is associated with a date that is more recent than a date associated with the image included in the identified document; and

program instructions to initiate display of the identified updated image within the identified document.

17. The computer system of claim 16, wherein identifying the updated image that includes the location depicted in the image included in the identified document further comprises

responsive to determining that an image in the set of images of the first potential location substantially matches the image included in the identified document, identify a set of tagged images that are tagged with metadata indicating location information that corresponds to metadata associated with the determined matching image in the set of images of the first potential location;

and

determine an updated image of the image included in the identified document from the identified set of tagged images, wherein the determined updated image is the image of the identified set of tagged images that includes metadata indicating a most recent date.

18. The computer system of claim 16, further comprising program instructions, stored on the computer readable storage media for execution by at least one of the one or more processors, to:

responsive to determining that no image in the set of images of the first potential location substantially matches the image included in the identified document, identify a set of images of a second potential location from the identified one or more potential locations; and

determine whether an image in the set of images of the second potential location substantially matches the image included in the identified document.

19. The computer system of claim 16, wherein the program instructions to identify a set of images of a first potential location of the identified one or more potential locations, comprises program instructions to:

determine a confidence value for at least one of the identified one or more potential locations that indicates a likelihood that a potential location is represented in the image included in the identified document, wherein the confidence values are derived from the unstructured text that is situated proximate to the image; and

identify the set of images of the first potential location of the identified one or more potential locations, wherein the first potential location is associated with highest determined confidence value.

20. The computer system of claim 16, wherein the identified updated image is displayed as a thumbnail image in proximity to the image included in the identified document and in proximity to the unstructured text that is situated proximate to the image.