Digital companions for human users

• US 9,704,103 B2
• Filed: 12/16/2014
• Issued: 07/11/2017
• Est. Priority Date: 12/16/2014
• Status: Active Grant

• Current Assignee: The Affinity Project, Inc.
• Original Assignee: The Affinity Project, Inc.
• Inventors: Suskind, Ronald Steven, Suskind, H. Owen, Suskind, Cornelia Kennedy
• Primary Examiner(s): Smith, Paulinho E
• Application Number:
  US14/571,472
• Publication Number:
  US 20160171387A1
• Time in Patent Office: 938 Days
• Field of Search: None
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2021/0135 Voice conversion or morphing
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

A succession of conversations are facilitated between a user of a device and a non-human companion portrayed on the device, to develop a relationship between the user and the non-human companion over a time period that spans the successive conversations. The relationship is developed between the user and the non-human companion to cause a change in a state of the user over the time period. A conversation is facilitated by presenting a segment of speech of the non-human companion to the user and detecting a segment of speech of the user, the segments including a portion of the conversation. At the device, information is received from an intelligent agent about a next segment of speech to be presented to the user, as determined by the intelligent agent based on intelligent processes applied to the segment of speech of the user and to the change in state to be caused.

Claims

1. A computer-implemented method comprising

providing affinity therapy through a mobile telephone to improve a diagnosed psychological state of an autistic child,

the providing of the affinity therapy comprising

through the mobile telephone developing a relationship between the autistic child and a consistently presented animated character portrayed on the mobile telephone, the developing of the relationship comprising

facilitating successive spoken conversations and non-spoken interactions through the mobile telephone between the autistic child and the animated character portrayed on the phone, at least some of the successive spoken conversations and the non-spoken interactions being interleaved, the relationship being developed over a time period that spans the successive spoken conversations and non-spoken interactions,

the facilitating of a current one of the successive spoken conversations comprising:

presenting to the autistic child a segment of speech of the animated character, the speech of the segment conforming to speech qualities that are characteristic of the animated character and are presented consistently with the speech qualities of other speech segments of the current spoken conversation and of others of the successive spoken conversations, the segments of speech comprising segments of morphed versions of human speech of two different humans,

detecting a segment of speech of the autistic child,

the autistic child'"'"'s segment of speech and the segment of speech presented to the autistic child comprising a portion of the current spoken conversation, and

presenting to the autistic child a subsequent segment of speech of the animated character, the subsequent segment of speech being determined by a choice made by an intelligent agent comprising a human, the choice having been made by the human in response to an automated presentation through a user interface of a list of suggested responses, the list of suggested responses being produced by intelligent processes applied to the detected segment of speech of the autistic child.

2. The method of claim 1 in which the animated character is portrayed through speech and images or video.

3. The method of claim 1 in which the relationship is caused to change over time.

4. The method of claim 1 in which characteristics or behavior or both of the animated character changes over time based on intelligent processes of the intelligent agent.

5. The method of claim 1 in which the time period over which the relationship is developed is at least a year.

6. The method of claim 1 in which the non-spoken interactions comprise presenting information to the autistic child.

7. The method of claim 6 in which the information is presented on a display of the mobile telephone.

8. The method of claim 1 in which the non-spoken interactions comprise receiving input from the autistic child.

9. The method of claim 1 in which the segment of speech of the autistic child and information about non-spoken interactions are provided in real-time to a human.

10. The method of claim 1 in which the intelligent agent comprises an automated process.

11. The method of claim 1 in which the intelligent agent comprises both an automated process and the human.

12. The method of claim 1 in which the autistic child is unaware of the operation of the intelligent agent.

13. The method of claim 1 comprising the intelligent agent explicitly or implicitly eliciting from the autistic child information about the autistic child'"'"'s beliefs, opinions, emotions, goals, or desires.

14. The method of claim 1 comprising the intelligent agent deriving personal and demographic information about the autistic child.

15. The method of claim 1 comprising the intelligent agent acquiring knowledge from public online sources of general information.

16. The method of claim 1 comprising forming a knowledge graph for the autistic child.

17. The method of claim 1 comprising maintaining a log of the successive conversations and interactions between the autistic child and the device.

18. The method of claim 17 comprising making the log available to a third party for study.

19. The method of claim 1 in which the animated character is portrayed as having certain characteristics that include at least one of the following:

responsiveness, initiative, moodiness, emotion, opinions, beliefs, correct ability, and self-awareness.

20. The method of claim 1 in which facilitating successive spoken conversations and non-spoken interactions comprises correcting the intelligent agent when it makes a mistake.

21. The method of claim 1 comprising stereotyping the autistic child and other users who share common characteristics, and operating the intelligent agent to take advantage of the stereotyping.

22. The method of claim 1 comprising representing concepts used in the successive spoken conversations and non-spoken interactions by a uniform knowledge representation.

23. The method of claim 1 in which the morphing comprises morphing to the vocabulary, grammar, and knowledge of the animated character.

24. The method of claim 1 in which the intelligent agent comprises an automated intelligent agent, and the automated intelligent agent learns from scratch rather than being pre-trained.