EMPATHETIC IMAGE SELECTION

• US 20190332342A1
• Filed: 07/12/2019
• Published: 10/31/2019
• Est. Priority Date: 10/20/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chun, Travis W., Flores, Romelia H., Johnson, Marlentae A., Lau, Michael A., Rodriguez, Roberto R., Rutkowski, Ronald J.
• Granted Patent: US 10,929,080 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2360/144 the light being ambient light
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 2370/04 Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller
• CPC Class Codes: G09G 2380/16 Digital picture frames

Abstract

An approach is provided for selecting and displaying one or more images. A determination is made that a user is in a proximity to a digital picture frame. Weather information is received from a weather sensor coupled to the digital picture frame. The weather information specifies a current weather of a geographic location that includes the digital picture frame. Based on the current weather, an emotional state of the user is determined. Based on user profile information corresponding to the user, image(s) included in multiple images are determined to convey the emotional state of the user. Based on the emotional state of the user and the image(s) conveying the emotional state, the image(s) are displayed on a display included in the digital picture frame.

Claims

1. A method of selecting and displaying one or more images, the method comprising the steps of:

a computer determining that a user is in a proximity to a digital picture frame that includes the computer;

in response to the step of determining that the user is in the proximity to the digital picture frame, the computer receiving weather information from a weather sensor coupled to the digital picture frame, the weather information specifying a current weather of a geographic location that includes the digital picture frame;

in response to the step of determining that the user is in the proximity to the digital picture frame, and based on the current weather of the geographic location that includes the digital picture frame, the computer determining an emotional state of the user;

based on user profile information corresponding to the user, the computer determining that one or more images included in a plurality of images convey the emotional state of the user; and

based on the emotional state of the user and the one or more images conveying the emotional state of the user, the computer displaying the one or more images on a display included in the digital picture frame.

2. The method of claim 1, wherein the step of determining the emotional state of the user includes the computer receiving a measurement of ambient lighting of a room in which the digital picture frame is located, the measurement being received from a light sensor coupled to the digital picture frame, wherein the step of determining the emotional state of the user is further based on the measurement of the ambient lighting of the room in which the digital picture frame is located.

3. The method of claim 1, further comprising the step of the computer collecting schedules of events of interest to the user from a calendar service, wherein the step of determining the emotional state of the user is based on the schedules of events of interest to the user collected from the calendar service.

4. The method of claim 1, further comprising the steps of:

the computer determining that one or more additional users are in the proximity to the digital picture frame;

in response to the one or more additional users being in the proximity to the digital picture frame, the computer determining one or more emotional states of the one or more additional users, respectively;

based on the emotional state of the user and the one or more emotional states of the one or more additional users, the computer determining one emotional state of a majority of users in a group of users consisting of the user and the one or more additional users;

the computer determining an association between the one emotional state of the majority of the users and one sentiment included in a plurality of sentiments conveyed by the plurality of images;

the computer determining that an image included in the plurality of images conveys the one sentiment;

based on the association between the one emotional state of the majority of the user and the one sentiment and based on the image conveying the one sentiment, the computer selecting the image from the plurality of images; and

the computer displaying the image on the display included in the digital picture frame.

5. The method of claim 1, further comprising the step of the computer automatically generating metadata tags for the plurality of images, the metadata tags specifying respective contents of corresponding images included in the plurality of images, wherein the step of determining that the one or more images convey the emotional state of the user includes:

identifying contents of the one or more images based on the one or more metadata tags generated for the one or more images, the one or more metadata tags being included in the generated metadata tags; and

identifying the emotional state of the user based on the identified contents of the one or more images.

6. The method of claim 1, further comprising the step of the computer automatically generating metadata tags for the plurality of images, the metadata tags specifying respective sentiments of corresponding images included in the plurality of images, wherein the step of determining that the one or more images convey the emotional state of the user includes identifying the emotional state based on one or more metadata tags generated for the one or more images, the one or more metadata tags being included in the generated metadata tags.

7. The method of claim 1, further comprising the step of the computer identifying a new post authored by the user via a social media service, wherein the step of determining the emotional state of the user includes determining a sentiment based in part on contents of the new post, and wherein the step of displaying the one or more images is further based on the sentiment and the contents of the new post.

8. The method of claim 1, further comprising the step of:

providing at least one support service for at least one of creating, integrating, hosting, maintaining, and deploying computer-readable program code in the computer, the program code being executed by a processor of the computer to implement the steps of determining that the user is in the proximity to the digital picture frame, receiving the weather information, determining the emotional state of the user, determining that the one or more images convey the emotional state of the user, and displaying the one or more images on the display.

9. A computer program product, comprising:

a computer-readable storage device; and

a computer-readable program code stored in the computer-readable storage device, the computer-readable program code containing instructions that are executed by a central processing unit (CPU) of a computer system to implement a method of selecting and displaying one or more images, the method comprising the steps of;

the computer system determining that a user is in a proximity to a digital picture frame that includes the computer system;

in response to the step of determining that the user is in the proximity to the digital picture frame, the computer system receiving weather information from a weather sensor coupled to the digital picture frame, the weather information specifying a current weather of a geographic location that includes the digital picture frame;

in response to the step of determining that the user is in the proximity to the digital picture frame, and based on the current weather of the geographic location that includes the digital picture frame, the computer system determining an emotional state of the user;

based on user profile information corresponding to the user, the computer system determining that one or more images included in a plurality of images convey the emotional state of the user; and

based on the emotional state of the user and the one or more images conveying the emotional state of the user, the computer system displaying the one or more images on a display included in the digital picture frame.

10. The computer program product of claim 9, wherein the step of determining the emotional state of the user includes the computer system receiving a measurement of ambient lighting of a room in which the digital picture frame is located, the measurement being received from a light sensor coupled to the digital picture frame, wherein the step of determining the emotional state of the user is further based on the measurement of the ambient lighting of the room in which the digital picture frame is located.

11. The computer program product of claim 9, wherein the method further comprises the step of the computer system collecting schedules of events of interest to the user from a calendar service, wherein the step of determining the emotional state of the user is based on the schedules of events of interest to the user collected from the calendar service.

12. The computer program product of claim 9, wherein the method further comprises the steps of:

the computer system determining that one or more additional users are in the proximity to the digital picture frame;

in response to the one or more additional users being in the proximity to the digital picture frame, the computer system determining one or more emotional states of the one or more additional users, respectively;

based on the emotional state of the user and the one or more emotional states of the one or more additional users, the computer system determining one emotional state of a majority of users in a group of users consisting of the user and the one or more additional users;

the computer system determining an association between the one emotional state of the majority of the users and one sentiment included in a plurality of sentiments conveyed by the plurality of images;

the computer system determining that an image included in the plurality of images conveys the one sentiment;

based on the association between the one emotional state of the majority of the user and the one sentiment and based on the image conveying the one sentiment, the computer system selecting the image from the plurality of images; and

the computer system displaying the image on the display included in the digital picture frame.

13. The computer program product of claim 9, wherein the method further comprises the step of the computer system automatically generating metadata tags for the plurality of images, the metadata tags specifying respective contents of corresponding images included in the plurality of images, wherein the step of determining that the one or more images convey the emotional state of the user includes:

identifying contents of the one or more images based on the one or more metadata tags generated for the one or more images, the one or more metadata tags being included in the generated metadata tags; and

identifying the emotional state of the user based on the identified contents of the one or more images.

14. The computer program product of claim 9, wherein the method further comprises the step of the computer system automatically generating metadata tags for the plurality of images, the metadata tags specifying respective sentiments of corresponding images included in the plurality of images, wherein the step of determining that the one or more images convey the emotional state of the user includes identifying the emotional state based on one or more metadata tags generated for the one or more images, the one or more metadata tags being included in the generated metadata tags.

15. A computer system comprising:

a central processing unit (CPU);

a memory coupled to the CPU; and

a computer readable storage device coupled to the CPU, the computer readable storage device containing instructions that are executed by the CPU via the memory to implement a method of selecting and displaying one or more images, the method comprising the steps of;

the computer system determining that a user is in a proximity to a digital picture frame that includes the computer system;

in response to the step of determining that the user is in the proximity to the digital picture frame, the computer system receiving weather information from a weather sensor coupled to the digital picture frame, the weather information specifying a current weather of a geographic location that includes the digital picture frame;

in response to the step of determining that the user is in the proximity to the digital picture frame, and based on the current weather of the geographic location that includes the digital picture frame, the computer system determining an emotional state of the user;

based on user profile information corresponding to the user, the computer system determining that one or more images included in a plurality of images convey the emotional state of the user; and

based on the emotional state of the user and the one or more images conveying the emotional state of the user, the computer system displaying the one or more images on a display included in the digital picture frame.

16. The computer system of claim 15, wherein the step of determining the emotional state of the user includes the computer system receiving a measurement of ambient lighting of a room in which the digital picture frame is located, the measurement being received from a light sensor coupled to the digital picture frame, wherein the step of determining the emotional state of the user is further based on the measurement of the ambient lighting of the room in which the digital picture frame is located.

17. The computer system of claim 15, wherein the method further comprises the step of the computer system collecting schedules of events of interest to the user from a calendar service, wherein the step of determining the emotional state of the user is based on the schedules of events of interest to the user collected from the calendar service.

18. The computer system of claim 15, wherein the method further comprises the steps of:

the computer system determining that one or more additional users are in the proximity to the digital picture frame;

in response to the one or more additional users being in the proximity to the digital picture frame, the computer system determining one or more emotional states of the one or more additional users, respectively;

based on the emotional state of the user and the one or more emotional states of the one or more additional users, the computer system determining one emotional state of a majority of users in a group of users consisting of the user and the one or more additional users;

the computer system determining an association between the one emotional state of the majority of the users and one sentiment included in a plurality of sentiments conveyed by the plurality of images;

the computer system determining that an image included in the plurality of images conveys the one sentiment;

based on the association between the one emotional state of the majority of the user and the one sentiment and based on the image conveying the one sentiment, the computer system selecting the image from the plurality of images; and

the computer system displaying the image on the display included in the digital picture frame.

19. The computer system of claim 15, wherein the method further comprises the step of the computer system automatically generating metadata tags for the plurality of images, the metadata tags specifying respective contents of corresponding images included in the plurality of images, wherein the step of determining that the one or more images convey the emotional state of the user includes:

identifying contents of the one or more images based on the one or more metadata tags generated for the one or more images, the one or more metadata tags being included in the generated metadata tags; and

identifying the emotional state of the user based on the identified contents of the one or more images.

20. The computer system of claim 15, wherein the method further comprises the step of the computer system automatically generating metadata tags for the plurality of images, the metadata tags specifying respective sentiments of corresponding images included in the plurality of images, wherein the step of determining that the one or more images convey the emotional state of the user includes identifying the emotional state based on one or more metadata tags generated for the one or more images, the one or more metadata tags being included in the generated metadata tags.