Juxtaposing contextually similar cross-generation images

• US 10,678,845 B2
• Filed: 04/02/2018
• Issued: 06/09/2020
• Est. Priority Date: 04/02/2018
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hwang, Inseok, Liu, Su, Rozner, Eric, Yoo, Chungkuk
• Primary Examiner(s): Chen, Xuemei G
• Application Number:
  US15/942,635
• Publication Number:
  US 20190303458A1
• Time in Patent Office: 799 Days
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A contemporary image is stored, is analyzed, and an associated data structure is generated. An image data structure may include image meta data, image classification data, image segment data, image caption data, or the like. The storage of the contemporary image is used as a triggering event and its data structure is used as an input to determine contextually relevant ancestral images. The contemporary image data structure is compared to respective data structures of ancestral images. A series of ancestral images are ranked based upon contextual similarly of their data structures to the contemporary image data structure. The most similar ancestral image is simultaneously displayed with the contemporary image. Interactions may be monitored, a prompt may be provided, a next ranked ancestral image may be displayed in place of the previous ancestral image, and/or a prompt to store a new contemporary image may be provided.

Claims

1. A method of simultaneously displaying an ancestral image and a contemporary image comprising:

analyzing, with a first client computer, a plurality of ancestral images stored within an ancestral dataspace local to first client computer to generate a plurality of ancestral image data structures, wherein one ancestral data structure is linked to one ancestral image, and wherein each of the ancestral image data structures comprises contextual data of one or more entities depicted within the linked ancestral image;

subsequently detecting, with a server computer, that a contemporary image has been stored within a contemporary image dataspace local to a second computer connected to the first client computer and to the server computer by a network;

resultantly obtaining, with the first client computer, the contemporary image;

analyzing, with the first client computer, the contemporary image to generate a linked contemporary image data structure comprising contextual data of one or more entities depicted within the contemporary image;

ranking, with the first client computer, the plurality of ancestral images based upon degree of similarity between the ancestral image data structures and the contemporary image data structure;

identifying, with the first client computer, an ancestral image linked to an ancestral image data structure with the highest degree of similarity to the contemporary image data structure;

simultaneously displaying the contemporary image and the ancestral image within a graphical user interface upon a screen of the first client computer;

generating, with the first client computer, a description prompt comprising a description of one or more entities depicted within contemporary image from the contemporary image data structure;

generating, with the first client computer, an interaction prompt comprising a query about one or more entities depicted within the ancestral image from the ancestral image data structure;

providing, with the first client computer, the description prompt and the interaction prompt;

subsequent to providing the interaction prompt, monitoring, with a voice user interface (VUI) of the first client computer, vocal audio of a user of the first client computer that is received by a microphone of the first client computer to determine whether the user of the first client computer vocally responds to the interaction prompt query about the one or more entities depicted within the ancestral image; and

determining, with the VUI of the first client computer, that the user of the first client computer does not adequately vocally respond to the interaction prompt query about the one or more entities depicted within the ancestral image and resultantly displaying, within a graphical user interface upon a screen of the second client computer, a prompt to store a new contemporary image within the contemporary image dataspace.

2. The method of claim 1, wherein each of the ancestral image data structures comprise a plurality of contextual data scene tags that identify an entity within the linked ancestral image and wherein the contemporary image data structure comprises a plurality of contextual data scene tags that identify an entity within the linked contemporary image.

3. The method of claim 1, wherein each of the ancestral image data structures comprise an image caption tag that captions the linked ancestral image and wherein the contemporary image data structure comprises an image caption tag that captions the linked contemporary image.

4. The method of claim 1, wherein each of the ancestral image data structures comprise a facial analysis tag that identifies a person depicted in the linked ancestral image and wherein the contemporary image data structure comprises facial analysis tag that identifies a person depicted in the ancestral image.

5. The method of claim 1, wherein each of the ancestral image data structures comprise a metadata entry comprising metadata local to the linked ancestral image and wherein the contemporary image data structure comprises a metadata entry comprising metadata local to the linked contemporary image.

6. The method of claim 1, further comprising:

generating, with the first client computer, a relationship profile data structure comprising a first entry associated with a first person, a second entry associated with a second person, and a pointer between the first entry and the second entry that identifies a relation between the first person to the second person.

7. The method of claim 1, further comprising:

identifying, with the first client computer, a next ancestral image linked to an ancestral image data structure with a next highest degree of similarity to the contemporary image data structure; and

displaying, within the graphical user interface upon the screen of the first client computer, the next ancestral image in place of the ancestral image.

8. A computer program product for simultaneously displaying an ancestral image and a contemporary image, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions readable by:

a first client computer to analyze a plurality of ancestral images stored within an ancestral dataspace local to the first client computer and generate a plurality of ancestral image data structures, wherein one ancestral data structure is linked to one ancestral image, and wherein each of the ancestral image data structures comprises contextual data of one or more entities depicted within the linked ancestral image;

a server computer to subsequently detect that a contemporary image has been stored within a contemporary image dataspace local to a second client computer connected to the server computer and to the first client computer by a network;

the first client computer to resultantly obtain the contemporary image;

the first client computer to analyze the contemporary image and generate a linked contemporary image data structure comprising contextual data of one or more entities depicted within the contemporary image;

the first client computer to rank the plurality of ancestral images based upon degree of similarity between the ancestral image data structures and the contemporary image data structure;

the first client computer to identify an ancestral image linked to an ancestral image data structure with the highest degree of similarity to the contemporary image data structure;

the first client computer to simultaneously display the contemporary image and the ancestral image within a graphical user interface upon a screen of the first client computer;

the first client computer to generate a description prompt comprising a description of one or more entities depicted within contemporary image from the contemporary image data structure;

the first client computer to generate an interaction prompt comprising a query about one or more entities depicted within the ancestral image from the ancestral image data structure;

the first client computer to provide the description prompt and the interaction prompt;

the first client computer to monitor, with a voice user interface (VUI) of the first client computer, vocal audio of a user of the first client computer that is received by a microphone of the first client computer to determine whether the user of the first client computer vocally responds to the interaction prompt query about the one or more entities depicted within the ancestral image;

the first client computer to determine that the user of the first client computer does not adequately vocally respond to the interaction prompt query about the one or more entities depicted within the ancestral image; and

the second client computer to resultantly display, within a graphical user interface upon a screen of the second client computer, a prompt to store a new contemporary image within the contemporary image dataspace.

9. The computer program product of claim 8, wherein each of the ancestral image data structures comprise a plurality of contextual data scene tags that identify an entity within the linked ancestral image and wherein the contemporary image data structure comprises a plurality of contextual data scene tags that identify an entity within the linked contemporary image.

10. The computer program product of claim 8, wherein each of the ancestral image data structures comprise an image caption tag that captions the linked ancestral image and wherein the contemporary image data structure comprises an image caption tag that captions the linked contemporary image.

11. The computer program product of claim 8, wherein each of the ancestral image data structures comprise a facial analysis tag that identifies a person depicted in the linked ancestral image and wherein the contemporary image data structure comprises facial analysis tag that identifies a person depicted in the ancestral image.

12. The computer program product of claim 8, wherein each of the ancestral image data structures comprise a metadata entry comprising metadata local to the linked ancestral image and wherein the contemporary image data structure comprises a metadata entry comprising metadata local to the linked contemporary image.

13. The computer program product of claim 8, wherein the program instructions are readable by:

the first client computer to generate a relationship profile data structure comprising a first entry associated with a first person, a second entry associated with a second person, and a pointer between the first entry and the second entry that identifies a relation between the first person to the second person.

14. The computer program product of claim 8, wherein the program instructions are readable by:

the first client computer to identify a next ancestral image linked to an ancestral image data structure with a next highest degree of similarity to the contemporary image data structure; and

the first client computer to display the next ancestral image in place of the ancestral image.

15. A computer system comprising:

a first client computer that analyzes a plurality of ancestral images stored within an ancestral dataspace local to the first client computer and that generates a plurality of ancestral image data structures, wherein one ancestral data structure is linked to one ancestral image, and wherein each of the ancestral image data structures comprises contextual data of one or more entities depicted within the linked ancestral image;

a server computer that detects that a contemporary image has been stored within a contemporary image dataspace local to a second client computer connected to the server computer and to the first client computer by a network;

the first client computer that obtains the contemporary image;

the first client computer that analyzes the contemporary image and that generates a linked contemporary image data structure comprising contextual data of one or more entities depicted within the contemporary image;

the first client computer that ranks the plurality of ancestral images based upon degree of similarity between the ancestral image data structures and the contemporary image data structure;

the first client computer that identifies an ancestral image linked to an ancestral image data structure with the highest degree of similarity to the contemporary image data structure;

the first client computer that simultaneously displays the contemporary image and the ancestral image within a graphical user interface upon a screen of the first client computer;

the first client computer that generates a description prompt comprising a description of one or more entities depicted within contemporary image from the contemporary image data structure;

the first client computer that generates an interaction prompt comprising a query about one or more entities depicted within the ancestral image from the ancestral image data structure;

the first client computer that provides the description prompt and the interaction prompt;

the first client computer that monitors, with a voice user interface (VUI) of the first client computer, vocal audio of a user of the first client computer that is received by a microphone of the first client computer to determine whether the user of the first client computer vocally responds to the interaction prompt query about the one or more entities depicted within the ancestral image;

the first client computer that determines that the user of the first client computer does not adequately vocally respond to the interaction prompt query about the one or more entities depicted within the ancestral image; and

the second client computer that resultantly displays, within a graphical user interface upon a screen of the second client computer, a prompt to store a new contemporary image within the contemporary image dataspace.

16. The computer system of claim 15, wherein each of the ancestral image data structures comprise a plurality of contextual data scene tags that identify an entity within the linked ancestral image and wherein the contemporary image data structure comprises a plurality of contextual data scene tags that identify an entity within the linked contemporary image.

17. The computer system of claim 15, wherein each of the ancestral image data structures comprise an image caption tag that captions the linked ancestral image and wherein the contemporary image data structure comprises an image caption tag that captions the linked contemporary image.

18. The computer system of claim 15, wherein each of the ancestral image data structures comprise a facial analysis tag that identifies a person depicted in the linked ancestral image and wherein the contemporary image data structure comprises facial analysis tag that identifies a person depicted in the ancestral image.

19. The computer system of claim 15, wherein each of the ancestral image data structures comprise a metadata entry comprising metadata local to the linked ancestral image and wherein the contemporary image data structure comprises a metadata entry comprising metadata local to the linked contemporary image.

20. The computer system of claim 15, further comprising:

the first client computer that generates a relationship profile data structure comprising a first entry associated with a first person, a second entry associated with a second person, and a pointer between the first entry and the second entry that identifies a relation between the first person to the second person.