SYSTEM AND METHOD FOR GENERATING EMOJI MASHUPS WITH MACHINE LEARNING

• US 20190122403A1
• Filed: 10/23/2017
• Published: 04/25/2019
• Est. Priority Date: 10/23/2017
• Status: Active Grant

• Current Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Original Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Inventors: Woo, Titus, Kuo, Wei Ting, Lu, Xueguang, Komma, Venkatesh
• Granted Patent: US 10,593,087 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 40/109 Font handling; Temporal or kinetic typography
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components

Abstract

Aspects of the present disclosure involve systems, methods, devices, and the like for emoji mashup generation. The system and method introduce a method and model that can generate emoji mashups representative of contextual information received by a user at an application. The emoji mashup may come in the form of two or more emojis coherently combined to represent the contextual idea or emotion being conveyed.

Claims

1. A system comprising:

a non-transitory memory storing instructions; and

a processor configured to execute instructions to cause the system to;

receive, over a network connection, user input text from a user device;

analyze, using a first learning model, the user input text;

extract, based on the analyze, emoji sequences related to the user input text;

process, using a second learning model, the emoji sequences to obtain coordinate information for each emoji in the emoji sequences; and

combine at least two of the emojis in the emoji sequence using the coordinate information to generate an emoji mashup.

2. The system of claim 1, executing instructions further causes the system to:

transmit, over the network connection, the emoji mashup to the user device, wherein the emoji mashup relates to the user input text.

3. The system of claim 1, wherein the analyze includes converting the user input text from words to vectors.

4. The system of claim 3, wherein the analyze includes filtering the converted vectors into matrices to generate new emojis.

5. The system of claim 1, wherein the analyze includes training a text to emoji sequence model and wherein the text to emoji sequence model is used for converting the user input text to emojis.

6. The system of claim 1, wherein the second learning model uses object recognition to obtain the coordinate information.

7. The system of claim 1, executing instructions further causes the system to:

transmit, over the network connection, one or more of the emojis in the emoji sequence to the user device for user selection.

8. A method comprising:

receiving, over a network connection, user input text from a user device;

analyzing, using a first learning model, the user input text;

extracting, based on the analyze, emoji sequences related to the user input text;

processing, using a second learning model, the emoji sequences to obtain coordinate information for each emoji in the emoji sequences; and

combining at least two of the emojis in the emoji sequence using the coordinate information to generate an emoji mashup.

9. The method of claim 8, further comprising:

transmitting, over the network connection, the emoji mashup to the user device, wherein the emoji mashup relates to the user input text.

10. The method of claim 8, wherein the analyze includes converting the user input text from words to vectors.

11. The method of claim 10, wherein the analyze includes a making a sentiment analysis to determine a purpose and context of the user text input.

12. The method of claim 8, wherein the analyze includes training a text to emoji sequence model and wherein the text to emoji sequence model is used for converting the user input text to emojis.

13. The method of claim 8, wherein the second learning model uses object recognition to obtain the coordinate information.

14. The method of claim 8, further comprising:

transmitting, over the network connection, one or more of the emojis in the emoji sequence to the user device for user selection.

15. A non-transitory machine readable medium having stored thereon machine readable instructions executable to cause a machine to perform operations comprising:

receiving, over a network connection, user input text from a user device;

analyzing, using a first learning model, the user input text;

extracting, based on the analyze, emoji sequences related to the user input text;

processing, using a second learning model, the emoji sequences to obtain coordinate information for each emoji in the emoji sequences; and

combining at least two of the emojis in the emoji sequence using the coordinate information to generate an emoji mashup.

16. The non-transitory medium of claim 15, further comprising:

transmitting, over the network connection, the emoji mashup to the user device, wherein the emoji mashup relates to the user input text.

17. The non-transitory medium of claim 15, wherein the analyze includes converting the user input text from words to vectors.

18. The non-transitory medium of claim 17, wherein the analyze includes filtering the converted vectors into matrices to generate new emojis.

19. The non-transitory medium of claim 15, wherein the analyze includes training a text to emoji sequence model and wherein the text to emoji sequence model is used for converting the user input text to emojis.

20. The non-transitory medium of claim 15, wherein the second learning model uses object recognition to obtain the coordinate information.