SYSTEMS AND METHODS FOR RECOMMENDING RESPONSES

• US 20150243279A1
• Filed: 02/26/2015
• Published: 08/27/2015
• Est. Priority Date: 02/26/2014
• Status: Abandoned Application

• Current Assignee: Chatterbox Capital, LLC (Apple Inc.)
• Original Assignee: ToyTalk, Inc. (Apple Inc.)
• Inventors: Tinio, Aurelio, Chalfant, James, Morse, Benjamin, Reddy, Martin
• Application Number:
  US14/632,187
• Publication Number:
  US 20150243279A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models

Abstract

Various of the disclosed embodiments concern systems and methods for identifying and recommending interesting user responses that are obtained by an interactive device (e.g., audio responses to a virtual character as part of a virtual interaction). In some embodiments, a user may interact with one or more virtual characters via a mobile device, tablet, desktop computer, or the like. During the interaction, the user may respond to one or more questions posed by the virtual characters or to contexts presented by the interactive device. The system may record these user responses, analyze the audio data to extract one or more features, and prepare a ranking of the user responses. The extracted features can be augmented with human-generated metadata or ground truth values. A reviewer can review, share, etc., the user response.

Claims

1. A computer-implemented method for recommending interesting user responses produced by a user and obtained by an interactive device comprising:

receiving, from the interactive device, a user response including an audio waveform;

computing a textual hypothesis of the audio waveform, the textual hypothesis including a transcription of words identified in the audio waveform;

extracting a feature from the audio waveform, the textual hypothesis, or both;

generating a metric value for the feature, the metric value representing interest level of the feature;

weighting the metric value based on;

a general language model that includes a generic corpus of ground truth feature values that indicate how user responses should be analyzed;

a public language model that includes a public corpus of ground truth feature values derived from user responses produced by other users;

a personal language model that includes a personal corpus of ground truth feature values derived from user responses previously produced by the user; and

contextual factors that indicate whether the user response should be characterized as interesting; and

summing the weighted metric value with all other weighted metric values associated with features extracted from the user response, thereby generating a cumulative metric value that represents interest level of the user response as a whole.

2. The computer-implemented method of claim 1, wherein the generic corpus of ground truth feature values, the public corpus of ground truth feature values, the personal corpus of ground truth feature values, and the contextual factors are weighted with varying degrees of relevance.

3. The computer-implemented method of claim 1, wherein the user response is obtained by the interactive device when the user interacts with a virtual character via a user interface.

4. The computer-implemented method of claim 1, wherein the feature includes a determination of user response duration, total word count, individual word count, a fitted commonality score, a flag indicating a tagged question, a peak volume, average volume deviation, average duration deviation, average total word count deviation, or any combination thereof.

5. The computer-implemented method of claim 1, further comprising:

generating at least one supplemental feature derived from a behavior of a reviewer, the behavior including examining the entirety of the user response, reviewing the user response multiple times, electing to share the user response, or any combination thereof.

6. The computer-implemented method of claim 1, wherein generating the cumulative metric value for the user response includes evaluating a stored feature of a previous user response.

7. The computer-implemented method of claim 6, wherein the previous user response is associated with the user or a distinct user.

8. The computer-implemented method of claim 1, wherein the method is executed by a supervised machine learning system that determines an appropriate weighting of the feature, the appropriate weighting based on an analysis of a corpus of ground truth values provided by a plurality of reviewers.

9. The computer-implemented method of claim 1, wherein the method is executed by an unsupervised machine learning system that determines an appropriate weighting of the feature, the appropriate weighting based on an analysis of previous user responses obtained from the user.

10. A system for identifying and recommending interesting user responses comprising:

a recommendation engine configured to;

receive a plurality of user responses obtained by one or more interactive devices, the plurality of user responses associated with a user;

extract a feature from each user response;

assign a metric value to each extracted feature, the metric value representing interest level of the feature; and

determine a cumulative metric value for each user response, wherein the cumulative metric value is determined by summing the metric values of all extracted features identified in each user response;

a retrieval application program interface configured to;

receive, from an initiating device, a request for interesting user responses;

identify an interesting user response from the plurality of user responses, the interesting user response identified based on cumulative metric value; and

transmit at least a portion of the interesting user response to the initiating device; and

a database configured to store the plurality of user responses, the extracted features, the metric value for each extracted feature, the cumulative metric value for each user response, or any combination thereof.

11. The system of claim 10, wherein the recommendation engine is further configured to:

normalize the metric value to a common score; and

weight the metric value based on importance of the feature to interest level of the user response.

12. The system of claim 11, wherein the metric value is weighted based on one or more of:

a general language model that includes a generic corpus of ground truth feature values that indicate how user responses should be analyzed;

a public language model that includes a public corpus of ground truth feature values derived from user responses produced by other users;

a personal language model that includes a personal corpus of ground truth feature values derived from user responses previously produced by the user; and

contextual factors that indicate whether the user response should be characterized as interesting.

13. The system of claim 10, wherein the retrieval application program interface is further configured to:

implement a false positive directive that errs on the side of characterizing more user responses as interesting;

or

implement a false negative directive that errs on the side of characterizing fewer user responses as interesting.

14. The system of claim 10, wherein the recommendation engine is further configured to:

perform natural language processing on, and generate a textual hypothesis for, each user response, the textual hypothesis including a transcription of words identified in each user response.

15. The system of claim 11, wherein the recommendation engine is further configured to:

order the plurality of user responses by cumulative metric value, such that interesting user responses are ranked higher.

16. The system of claim 10, wherein the retrieval application program interface is further configured to:

identify a top “

N”

set of interesting user responses, wherein “

N”

is a predetermined integer; and

transmit the top “

N”

set to the initiating device associated with a requester.

17. The system of claim 16, wherein the top “

N”

set is ordered by cumulative metric value.

18. The system of claim 16, wherein the predetermined integer is determined by the requester.

19. The system of claim 10, wherein the initiating device is one of the one or more interactive devices.

20. The system of claim 19, wherein the recommendation engine, the retrieval application program interface, the database, or any combination thereof are stored on each of the one or more interactive devices.

21. The system of claim 10, wherein the recommendation engine, the retrieval application program interface, the database, or any combination thereof are stored on a remote storage medium communicatively coupled to each of the one or more interactive devices and the initiating device.

22. A user interface configured to:

permit a requester to specify a search parameter indicating desired characteristics of user responses to be retrieved;

send, to a processor, a request for interesting user responses, wherein the request includes the search parameter;

cause the processor to identify an interesting user response from a plurality of user responses stored in a storage medium, wherein each of the plurality of user responses includes an image of a speaker, an audio waveform, and a contextual indication;

receive, from the processor, the interesting user response; and

present the interesting user responses to the requester, wherein the user interface comprises a playback mechanism for reviewing the interesting user response.

23. The user interface of claim 22, wherein the processor identifies the interesting user response by:

computing, for each of the plurality of user responses, a textual hypothesis of the audio waveform, wherein the textual hypothesis includes a transcription of words identified in the audio waveform;

extracting a feature from the audio waveform, the textual hypothesis, or both;

determining a metric value for the feature, the metric value representing interest level of the feature;

weighting the metric value based on importance of the feature to interest level of the user response; and

summing the weighted metric value with all other weighted metric values associated with features extracted from the user response, thereby generating a cumulative metric value that represents interest level of the user response as a whole.

24. The request interface of claim 23, wherein the metric value is weighted based on one or more of:

a general language model that includes a generic corpus of ground truth feature values that indicate how user responses should be analyzed;

a public language model that includes a public corpus of ground truth feature values derived from user responses produced by other users;

a personal language model that includes a personal corpus of ground truth feature values derived from user responses previously produced by the user; and

contextual factors that indicate whether the user response should be characterized as interesting.

25. The request interface of claim 22, wherein the user interface is presented to the requester via an email, a web application, a web browser, or a mobile application adapted for one or more of a cellular device, a personal digital assistant, a tablet, and a personal computer.