Generating a reference digital image based on an indicated time frame and searching for other images using the reference digital image

• US 9,836,669 B2
• Filed: 02/22/2016
• Issued: 12/05/2017
• Est. Priority Date: 02/22/2016
• Status: Active Grant

• Current Assignee: AirBNB Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Abou Mahmoud, Alaa, Bastide, Paul R., Lu, Fang
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): TORRES, JOSE
• Application Number:
  US15/049,918
• Publication Number:
  US 20170243081A1
• Time in Patent Office: 652 Days
• Field of Search: None
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06F 40/20 Natural language analysis semantic analysis of natural language G06F40/30
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations
• CPC Class Codes: G06V 30/274 Syntactic or semantic context, e.g. balancing

Abstract

A digital image and a text string is received. The text string can be processed to identify at least a time frame and determine whether the time frame is a future time frame or a past time frame. How at least one element of the first digital image will change or has changed during the time frame can be predicted. At least one reference digital image can be generated, the reference digital image including at least one change to the at least one element corresponding to how the at least one element will change or has changed during the time frame. The reference digital image to each of a plurality of other digital images. A correlation parameter can be assigned to each of the plurality of other digital images. A portion of the plurality of other digital images having highest correlation parameters can be output for presentation to a user.

Claims

1. A method comprising:

receiving a first digital image;

receiving a text string indicating at least a time frame;

processing the text string to identify at least the time frame and determine whether the time frame is a future time frame or a past time frame;

responsive to determining whether the time frame is the future time frame or the past time frame, predicting how at least one element of the first digital image will change or has changed during the time frame;

responsive to predicting how the at least one element of the first digital image will change or has changed during the time frame, generating, using a processor, at least one reference digital image that is a revised version of the first digital image, the reference digital image including at least one change to the at least one element corresponding to how the at least one element will change or has changed during the time frame;

comparing the reference digital image to each of a plurality of other digital images;

based on comparing the reference digital image to each of the plurality of other digital images, assigning a correlation parameter to each of the plurality of other digital images, each correlation parameter indicating how closely a respective other digital image matches the reference digital image; and

outputting for presentation to a user a portion of the plurality of other digital images having highest correlation parameters.

2. The method of claim 1, wherein:

the text string further indicates a name of a person; and

the correlation parameter is assigned to a respective other digital image is based, at least in part, on whether the other digital image is associated with the name of the person indicated in the text string.

3. The method of claim 1, wherein:

the text string further indicates a name of a location; and

the correlation parameter is assigned to a respective other digital image is based, at least in part, on whether the other digital image is associated with location indicated in the text string.

4. The method of claim 1, wherein processing the text string to identify at least the time frame and determine whether the time frame is a future time frame or a past time frame comprises the text string using natural language processing.

5. The method of claim 1, wherein outputting for presentation to the user the portion of the plurality of other digital images having highest correlation parameters comprises presenting the portion of the plurality of other digital images having highest correlation parameters in an order based the correlation parameters.

6. The method of claim 1, further comprising:

assigning a ranking to each of the portion of the plurality of other digital images having highest correlation parameters;

wherein outputting for presentation to the user the portion of the plurality of other digital images having highest correlation parameters comprises presenting the portion of the plurality of other digital images having highest correlation parameters in an order based the assigned rankings.

7. The method of claim 1, wherein the other digital images are accessed from a social networking system.

8. A system, comprising:

a processor programmed to initiate executable operations comprising;

receiving a first digital image;

receiving a text string indicating at least a time frame;

processing the text string to identify at least the time frame and determine whether the time frame is a future time frame or a past time frame;

responsive to determining whether the time frame is the future time frame or the past time frame, predicting how at least one element of the first digital image will change or has changed during the time frame;

responsive to predicting how the at least one element of the first digital image will change or has changed during the time frame, generating at least one reference digital image that is a revised version of the first digital image, the reference digital image including at least one change to the at least one element corresponding to how the at least one element will change or has changed during the time frame;

comparing the reference digital image to each of a plurality of other digital images;

based on comparing the reference digital image to each of the plurality of other digital images, assigning a correlation parameter to each of the plurality of other digital images, each correlation parameter indicating how closely a respective other digital image matches the reference digital image; and

outputting for presentation to a user a portion of the plurality of other digital images having highest correlation parameters.

9. The system of claim 8, wherein:

the text string further indicates a name of a person; and

the correlation parameter is assigned to a respective other digital image is based, at least in part, on whether the other digital image is associated with the name of the person indicated in the text string.

10. The system of claim 8, wherein:

the text string further indicates a name of a location; and

the correlation parameter is assigned to a respective other digital image is based, at least in part, on whether the other digital image is associated with location indicated in the text string.

11. The system of claim 8, wherein processing the text string to identify at least the time frame and determine whether the time frame is a future time frame or a past time frame comprises the text string using natural language processing.

12. The system of claim 8, wherein outputting for presentation to the user the portion of the plurality of other digital images having highest correlation parameters comprises presenting the portion of the plurality of other digital images having highest correlation parameters in an order based the correlation parameters.

13. The system of claim 8, the executable operations further comprising:

assigning a ranking to each of the portion of the plurality of other digital images having highest correlation parameters;

wherein outputting for presentation to the user the portion of the plurality of other digital images having highest correlation parameters comprises presenting the portion of the plurality of other digital images having highest correlation parameters in an order based the assigned rankings.

14. The system of claim 8, wherein the other digital images are accessed from a social networking system.

15. A computer program product comprising a computer readable storage medium having program code stored thereon, the program code executable by a processor to perform a method comprising:

receiving, by the processor, a first digital image;

receiving, by the processor, a text string indicating at least a time frame;

processing, by the processor, the text string to identify at least the time frame and determine whether the time frame is a future time frame or a past time frame;

responsive to determining whether the time frame is the future time frame or the past time frame, predicting, by the processor, how at least one element of the first digital image will change or has changed during the time frame;

responsive to predicting how the at least one element of the first digital image will change or has changed during the time frame, generating, by the processor, at least one reference digital image that is a revised version of the first digital image, the reference digital image including at least one change to the at least one element corresponding to how the at least one element will change or has changed during the time frame;

comparing, by the processor, the reference digital image to each of a plurality of other digital images;

based on comparing the reference digital image to each of the plurality of other digital images, assigning, by the processor, a correlation parameter to each of the plurality of other digital images, each correlation parameter indicating how closely a respective other digital image matches the reference digital image; and

outputting, by the processor, for presentation to a user a portion of the plurality of other digital images having highest correlation parameters.

16. The computer program product of claim 15, wherein:

the text string further indicates a name of a person; and

the correlation parameter is assigned to a respective other digital image is based, at least in part, on whether the other digital image is associated with the name of the person indicated in the text string.

17. The computer program product of claim 15, wherein:

the text string further indicates a name of a location; and

the correlation parameter is assigned to a respective other digital image is based, at least in part, on whether the other digital image is associated with location indicated in the text string.

18. The computer program product of claim 15, wherein processing the text string to identify at least the time frame and determine whether the time frame is a future time frame or a past time frame comprises the text string using natural language processing.

19. The computer program product of claim 15, wherein outputting for presentation to the user the portion of the plurality of other digital images having highest correlation parameters comprises presenting the portion of the plurality of other digital images having highest correlation parameters in an order based the correlation parameters.

20. The computer program product of claim 15, the method further comprising:

assigning a ranking to each of the portion of the plurality of other digital images having highest correlation parameters;

wherein outputting for presentation to the user the portion of the plurality of other digital images having highest correlation parameters comprises presenting the portion of the plurality of other digital images having highest correlation parameters in an order based the assigned rankings.