Generating a mood log based on user images

• US 10,133,918 B1
• Filed: 04/20/2015
• Issued: 11/20/2018
• Est. Priority Date: 04/20/2015
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Chang, Sheldon
• Primary Examiner(s): Ahmed, Samir
• Application Number:
  US14/691,249
• Time in Patent Office: 1,310 Days
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/175 Static expression
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: G06V 40/179 metadata assisted face recognition

Abstract

A system and method for generate a mood log based on user images. In one embodiment, a system includes an image module that receives images taken by a user'"'"'s mobile computing device and determines that a face of the user is included in the image, a mood module that determines a mood level of the user based on the face, and a log module that stores the mood level in a log of mood levels for the user.

Claims

1. A method comprising:

iteratively performing, in response to interaction events between a user and another user;

receiving, by one or more hardware processors, an image taken by a mobile computing device during an interaction event;

determining, using facial recognition and in response to an image being taken, that a face of the user is included in the image;

in response to determining that the image includes the face of the user;

identifying points for two or more features of the face of the user,

comparing a distribution of the points to a database of point distributions, determining a mood from the database associated with the distribution in the database that most closely matches the distribution of points,

determining an intensity level of the mood based on the distribution in the database that most closely matches the distribution of points; and

storing an association between the mood and intensity level with the other user associated with the interaction event;

causing, by a server, display of a visual indicator of the moods on the mobile computing device.

2. The method of claim 1, wherein determining the mood level comprises defining points for two or more features of face of the user, comparing a distribution of the points to a database of point distributions, and selecting the mood level from the database associated with the distribution in the database that most closely matches the distribution of points.

3. The method of claim 1, wherein determining a mood level comprises transmitting the image to a remote server and receiving the mood level from the remote server.

4. The method of claim 1, wherein the images comprise a video and determining the mood level is further based on physical movements by the user in the video.

5. The method of claim 1, further comprising requesting that the user provide an image of the user in response to no images being taken in a threshold period of time.

6. The method of claim 1, further comprising associating a mood level with an event experienced by the user and notifying the user that the event is associated with the mood level.

23. The method of claim 1, wherein the interaction events include at least one of a text message exchange with one of the other users, a phone call with one of the other users, or an email exchange with one of the other users.

7. A system comprising:

one or more hardware processors, configured to;

iteratively perform, in response to interaction events between a user and another user;

receive an image taken by a mobile computing device during an interaction event,

determining, using facial recognition and in response to the image being received, that a face of the user is included in the received image;

in response to determining that the image includes the face of the user;

identifying points for two or more features of the face of the user,

comparing a distribution of the points to a database of point distributions, determining a mood from the database associated with the distribution in the database that most closely matches the distribution of points,

determining an intensity level of the mood based on the distribution in the database that most closely matches the distribution of points,

storing an association between the mood and intensity level with the other user associated with the interaction event, and

causing, by a server, display of a visual indicator of the moods on the mobile computing device.

8. The system of claim 7, wherein the log module further generates a graphical representation of the log of mood levels.

9. The system of claim 7, wherein the mood module determines the mood level by transmitting the image to a remote server and receiving the mood level from the remote server.

10. The system of claim 7, wherein the images comprise a video, the mood module further determines the mood level based on physical movements by the user in the video.

11. The system of claim 7, wherein the image module further requests that the user provide an image of the user in response to no images being taken in a threshold period of time.

12. The system of claim 7, wherein the log module further associates a mood level with an interaction between the user and another user, the log module determines an aggregate mood level associated with the other user based on the associated mood levels.

13. The system of claim 12, wherein the log module further notifies the user of the aggregate mood level associated with the other user.

14. The system of claim 12, wherein the log module further selects a classification for the other user from a set of predefined classifications according to the aggregate mood level associated with the other user.

15. A machine-readable medium having no transitory signals and storing instructions that, when executed by at least one processor of a machine, cause the machine to perform operations comprising:

iteratively performing, in response to interaction events between a user and another user;

receiving, an image taken by the mobile computing device during an interaction event;

determining, using facial recognition and in response to an image being taken, that a face of the user is included in the image;

in response to determining that the image includes the face of the user;

identifying points for two or more features of the face of the user,

comparing a distribution of the points to a database of point distributions, determining a mood from the database associated with the distribution in the database that most closely matches the distribution of points,

determining an intensity level of the mood based on the distribution in the database that most closely matches the distribution of points; and

storing an association between the mood and intensity level with the other user associated with the interaction event;

causing, by a server, display of a visual indicator of the moods on the mobile computing device.

16. The machine-readable medium of claim 15, wherein the operations further comprise generating a graphical representation of the log of mood levels.

17. The machine-readable medium of claim 15, wherein determining a mood level comprises transmitting the image to a remote server and receiving the mood level from the remote server.

18. The machine-readable medium of claim 15, wherein images comprise a video and determining the mood level is further based on physical movements by the user in the video.

19. The machine-readable medium of claim 15, wherein the operations further comprise requesting that the user provide an image of the user in response to no images being received in a threshold period of time.

20. The machine-readable medium of claim 15, wherein the operations further comprise associating a mood level with an interaction between the user and another user, and determining an aggregate mood level associated with the other user based on the associated mood levels.

21. The machine-readable medium of claim 20, wherein the operations further comprise notifying the user of the aggregate mood level associated with the other user.

22. The machine-readable medium of claim 20, wherein the operations further comprise selecting a classification for the other user from a set of predefined classifications according to the aggregate mood level associated with the other user.