EMOJI-BASED COMMUNICATIONS DERIVED FROM FACIAL FEATURES DURING GAME PLAY

• US 20190126152A1
• Filed: 11/01/2017
• Published: 05/02/2019
• Est. Priority Date: 11/01/2017
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Taylor, Michael, Black, Glenn
• Granted Patent: US 10,870,056 B2
• Time in Patent Office: Days
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/87 Communicating with other players during game play, e.g. by e-mail or chat
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 51/52 for supporting social networking services

Abstract

Techniques for emoji-based communications derived from facial features during game play provide a communication channel accessible by players associated with gameplay hosted by the online game platform and capture an image of a user using a controller that includes a camera. The techniques further determine facial features of the user based on the image, generate an emoji based on the facial features of the user, and transmit the emoji to the players associated with the online game platform over the communication channel.

Claims

1. A method for emoji-based communications derived from facial features during game play, the method comprising:

providing a communication channel accessible by players associated with gameplay hosted by a game platform;

capturing an image of a user by a controller that includes a camera;

determining facial features of the user based on the image;

generating an emoji based on the facial features of the user; and

transmitting the emoji to the players associated with the gameplay over the communication channel.

2. The method of claim 1, wherein capturing the image of the user further comprises:

capturing the image of the user based on one or more of an image capture command, a scheduled time, or a gameplay status.

3. The method of claim 1, wherein the image is a selected image, wherein capturing the selected image of the user further comprises:

capturing multiple images of the user;

determining a change in the facial features of the user from the multiple images; and

selecting one of the multiple images as the selected image based on the change.

4. The method of claim 1, wherein determining the facial features of the user further comprises determining landmarks in the image based on a facial recognition algorithm.

5. The method of claim 1, wherein the emoji is a modified model emoji, wherein generating the modified model emoji further comprises:

adjusting portions of a model emoji based on the facial features to form the modified model emoji.

6. The method of claim 1, wherein the emoji is a selected emoji, wherein generating the selected emoji further comprises:

designating one emoji of a plurality of emoji as the selected emoji based on a comparison between the facial features of the user and one or more facial features mapped to the plurality of emoji.

7. The method of claim 1, wherein the emoji is a selected emoji, wherein generating the selected emoji based on the facial features further comprises:

determining an expression or an emotion associated with the facial features of the user; and

designating one emoji of a plurality of emoji as the selected emoji based on the expression or the emotion.

8. The method of claim 1, wherein the emoji is a selected emoji, wherein generating the selected emoji further comprises:

determining an expression or an emotion associated with the facial features of the user based on a proximity in a vector-space between a query vector mapped to the image of the user and one or more position vectors assigned the expression or the emotion, wherein one feature corresponds to one dimension in the vector-space; and

designating one emoji of a plurality of emoji as the selected emoji based on the expression or the emotion.

9. The method of claim 8, further comprising:

mapping a plurality of images to position vectors in the vector-space based on features extracted from each image;

assigning one or more position vectors to a group that represents a respective emotion or a respective expression based on proximity in the vector-space.

10. The method of claim 1, wherein the emoji includes one or more of an ideogram, a smiley, a pictograph, or an emoticon.

11. A system for emoji-based communications derived from facial features during game play, the system comprising:

a controller having a camera to capture images of a user;

a network interface to communicate over a communication channel accessible by players associated with gameplay hosted by a game platform;

a processor coupled to the network interface and adapted to execute one or more processes; and

a memory configured to store a process executable by the processor, the process when executed operable to;

capture an image of the user by the controller;

determine facial features of the user based on the image;

generate an emoji based on the facial features of the user; and

transmit the emoji to the players associated with the gameplay over the communication channel.

12. The system of claim 11, wherein the process to capture the image of the user is further operable to:

capture the image of the user based on one or more of an image capture command, a scheduled time, or a gameplay status.

13. The system of claim 11, wherein the image is a selected image, wherein the process to capture the selected image of the user is further operable to:

capture multiple images of the user;

determine a change in the facial features of the user from the multiple images; and

select one of the multiple images as the selected image based on the change.

14. The system of claim 11, wherein determining the facial features for the user is further operable to determine landmarks in the image based on a facial recognition algorithm.

15. The system of claim 11, wherein the emoji is a modified model emoji, wherein the process to generate the modified model emoji is further operable to adjust portions of a model emoji based on the facial features to form the modified model emoji.

16. The system of claim 11, wherein the emoji is a selected emoji, wherein the process to generate the selected emoji is further operable to:

designate one emoji of a plurality of emoji as the selected emoji based on a comparison between the facial features of the user and one or more facial features mapped to the plurality of emoji.

17. The system of claim 11, wherein the emoji is a selected emoji, wherein the process to generate the selected emoji is further operable to:

determine an expression or an emotion associated with the facial features of the user; and

designate one emoji of a plurality of emoji as the selected emoji based on the expression or the emotion.

18. The system of claim 11, wherein the emoji is a selected emoji, wherein the process to generate the selected emoji is further operable to:

determine an expression or an emotion associated with the facial features of the user based on a proximity in a vector-space between a query vector mapped to the image of the user and one or more position vectors assigned the expression or the emotion, wherein one feature corresponds to one dimension in the vector-space;

designate one emoji of a plurality of emoji as the selected emoji based on the expression or the emotion.

19. A tangible, non-transitory, computer-readable media having instructions for emoji-based communications encoded thereon, the instructions, when executed by a processor, are operable to:

provide a communication channel accessible by players associated with gameplay hosted by a game platform;

capture an image of a user by a controller that includes a camera;

determine facial features of the user based on the image;

generate an emoji based on the facial features of the user; and

transmit the emoji to the players associated with the gameplay over the communication channel.

20. The computer-readable media of claim 19, wherein the emoji is a selected emoji, wherein the instructions, when executed by the processor to generate the selected emoji are further operable to:

designate one emoji of a plurality of emoji as the selected emoji based on a comparison between the facial features of the user and one or more facial features mapped to the plurality of emoji.