Guided personal companion

• US 9,710,613 B2
• Filed: 02/04/2016
• Issued: 07/18/2017
• Est. Priority Date: 12/16/2014
• Status: Active Grant

• Current Assignee: The Affinity Project, Inc.
• Original Assignee: The Affinity Project, Inc.
• Inventors: Nguyen, John, Patterson, Stuart R., Springer, Stephen R., Fanty, Mark Alan, Suskind, Ronald Steven
• Primary Examiner(s): Smith, Paulinho E
• Application Number:
  US15/015,891
• Publication Number:
  US 20160171971A1
• Time in Patent Office: 530 Days
• Field of Search: None
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
• CPC Class Codes: G09B 5/14 with provision for individual teacher-student communication
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G16H 20/40 relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 20/70 relating to mental therapies, e.g. psychological therapy or autogenous training
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: H04M 1/72427 for supporting games or graphical animations

Abstract

The operation of an application on a first device may be guided by a user operating a second device. The application on the first device may present a character on a display of the first device and obtain an audio signal of speech of a user of the first device. Audio data may be transmitted to the second device and corresponding audio may be played from speakers of the second device. The second device may present suggestions of phrases to be spoken by the character displayed on the first device. A user of the second device may select a phrase to be spoken by the character. Phrase data may be transmitted to the first device, and the first device may generate audio of the character speaking the phrase using a text-to-speech voice associated with the character.

Claims

1. A system for guiding operation of an application on a first device, the system comprising the first device and a second device, wherein:

the application on the first device is configured to;

enter into a session with the second device, wherein, during the session, a user of the second device specifies a phrase that is spoken by a character to a user of the first device and wherein the user of the first device and the user of the second device participate simultaneously in the session;

present the character on a display of the first device, wherein the character is associated with a text-to-speech voice,

obtain an audio signal from a microphone of the first device, wherein the audio signal comprises speech of the user of the first device, and

transmit audio data to the second device, wherein the audio data is generated from the audio signal;

the second device is configured to;

enter into the session with the first device;

receive the audio data from the first device,

cause audio to be played using the audio data,

present a plurality of phrases as suggestions of phrases to be spoken by the character,

receive an input from the user of the second device that specifies a selected phrase to be spoken by the character, and

cause phrase data, corresponding to the selected phrase, to be transmitted to the first device; and

the application on the first device is configured to;

receive the phrase data corresponding to the selected phrase, and

cause audio to be played from the first device corresponding to the selected phrase, wherein the audio is generated using the text-to-speech voice of the character.

2. The system of claim 1, wherein:

the user of the second device is a parent or caretaker of the user of the first device; and

the user of the second device selects the selected phrase to assist the user of the first device.

3. The system of claim 1, wherein the plurality of phrases comprises the selected phrase.

4. The system of claim 1, wherein the input from the user of the second device corresponds to receiving keyboard input or receiving an audio signal from a microphone connected to the second device.

5. The system of claim 1, wherein the input from the user of the second device comprises selecting a first phrase of the plurality of phrases and editing the first phrase to generate the selected phrase.

6. The system of claim 1, wherein:

the application on the first device is configured to transmit video data of the user of the first device to the second device during the session, wherein the video data is generated from a camera of the first device; and

the second device is configured to receive the video data from the first device and present video on a display using the video data.

7. The system of claim 1, wherein:

the second device is configured to;

receive an input from the user of the second device corresponding to an instruction to play an audio or video clip on the first device, and

send an instruction to the first device to play the audio or video clip; and

the first device is configured to;

receive the instruction to play the audio or video clip, and

cause the audio or video clip to be presented by the first device.

8. The system of claim 1, wherein the system comprises a server computer and wherein the application on the first device is configured to transmit the audio data to the second device during the session by transmitting the audio data via the server computer, and wherein the server computer is configured to transmit the audio data to the second device.

9. The system of claim 8, wherein the server computer is configured to:

receive preferences for communications with the user of the first device;

select the plurality of phrases using the preferences; and

transmit the plurality of phrases to the second device.

10. The system of claim 8, wherein the server computer is configured to:

receive a request from the first device to initiate the session;

send a message to the user of the second device;

wherein the second device enters the session after the user of the second device receives the message.

11. The system of claim 8, wherein the server computer is configured to:

receive a request from the second device to initiate the session;

send a message to the user of the first device;

wherein the first device enters the session after the user of the first device receives the message.

12. One or more non-transitory computer-readable media comprising computer executable instructions that, when executed, cause at least one processor to perform actions comprising:

receiving a request from a user of a first device to speak with a character, wherein the character is associated with a text-to-speech voice;

sending a message to a user of a second device corresponding to the request;

entering into a session with the second device, wherein, during the session, the user of the second device specifies a phrase that is spoken by the character to the user of the first device and wherein the user of the first device and the user of the second device participate simultaneously in the session;

presenting the character on a display of the first device;

obtaining an audio signal from a microphone of the first device;

transmitting audio data to the second device, wherein the audio data is generated from the audio signal;

receiving, from the second device, phrase data corresponding to the phrase to be spoken by the character;

generating an audio signal using the phrase data corresponding to the phrase and the text-to-speech voice of the character; and

causing audio to played corresponding to the audio signal.

13. The one or more computer-readable media of claim 12, the actions comprising receiving, from the user of the first device, a selection of the character from among a plurality of characters.

14. The one or more computer-readable media of claim 12, the actions comprising presenting text of the phrase on the display of the first device.

15. The one or more computer-readable media of claim 12, the actions comprising:

obtaining text of speech of the user of the first device; and

presenting the text of the speech of the user of the first device on the display of the first device.

16. The one or more computer-readable media of claim 12, the actions comprising:

receiving a request from the second device to play an audio or video clip; and

causing the audio or video clip to be presented by the first device.

17. The one or more computer-readable media of claim 12, wherein obtaining an audio signal and transmitting audio data to the second device commences after opening an application on the first device, receiving a request to speak with the character, or a selection of the character from among a plurality of characters.

18. A method for guiding operation of an application on a first device, the method performed by a second device and comprising:

entering into a session with the first device, wherein, during the session, a user of the second device specifies a phrase that is spoken by a character to a user of the first device and wherein the user of the first device and the user of the second device participate simultaneously in the session;

receiving audio data, wherein the audio data represents speech of the user of the first device;

causing audio to be played using the audio data;

receiving a plurality of phrases as suggestions of phrases to be spoken by the character displayed on the first device;

presenting the plurality of phrases on a display of the second device;

receiving an input from the user of the second device that specifies a selected phrase; and

causing phrase data to be transmitted to the first device, wherein the phrase data corresponds to the selected phrase.

19. The method of claim 18, comprising:

receiving from the user of the second device an input requesting a second plurality of phrases as suggestions of phrases to be spoken by the character displayed on the first device;

transmitting to a server computer a request for the second plurality of phrases;

receiving the second plurality of phrases; and

presenting the second plurality of phrases on the display of the second device.

20. The method of claim 18, comprising:

receiving video data, wherein the video data represents video captured by a camera of the first device; and

causing video to be presented on the display of the second device.

21. The method of claim 18, wherein the plurality of phrases comprises the selected phrase.

22. The method of claim 18, wherein the audio data is received in real time.