On demand guided virtual companion

• US 9,802,125 B1
• Filed: 05/27/2016
• Issued: 10/31/2017
• Est. Priority Date: 05/27/2016
• Status: Active Grant

• Current Assignee: The Affinity Project, Inc.
• Original Assignee: The Affinity Project, Inc.
• Inventors: Suskind, Ronald Steven, Patterson, Stuart R., Nguyen, John, Fanty, Mark Alan, Springer, Stephen R., Gregory, Erik Martin
• Primary Examiner(s): Grant, Michael
• Application Number:
  US15/166,333
• Time in Patent Office: 522 Days
• Field of Search: 434236
• CPC Class Codes: A63F 13/67 adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
• CPC Class Codes: A63F 13/847 Cooperative playing, e.g. requiring coordinated actions from several players to achieve a common goal
• CPC Class Codes: A63F 13/87 Communicating with other players during game play, e.g. by e-mail or chat

Abstract

A first user of a first device may interact with a character presented on the first device where the character presented on the first device is guided by a second user of a second device. To begin a communications session, the first user may request to communicate with the character. A server may process the request to speak with the character and identify the second user from a plurality of guiding users, where the second user is identified by comparing profile information of the first user with profile information of the second user. The server may then send a request to the second device of the second user to guide the character presented by the first device. The first user may then communicate with the character with the guidance of the second user.

Claims

1. A system for providing a communications session using a guided character wherein the communications session comprises a first device streaming audio and video of a first user to be presented to a second user of a second device and the second device transmitting a phrase to the first device to be played as audio by the first device using a text-to-speech voice of the guided character, the system comprising the first device, a server computer, and the second device, wherein:

the first device is configured to;

present the character on a display of the first device,

receive an input from the first user of the first device indicative of a request to start the communications session with the character, and

transmit information indicative of the request to start the communications session with the character to the server computer;

the server computer is configured to;

receive the information indicative of the request to start the communications session with the character,

obtain profile information of the first user,

identify the second user in a data store of guiding users, wherein (i) the data store comprises an availability indicator for each of the guiding users, (ii) the data store comprises profile information for each of the guiding users, and (iii) the second user is identified by comparing the profile information of the first user with profile information of the second user;

send, to the second device of the second user, information indicative of a request for the second user to guide the character presented by the first device in the communications session,

receive, from the second device, an acceptance of the request, and

after the acceptance of the request has been received, start the communications session wherein, during the communications session, the server computer (i) streams an audio signal from the first device to the second device in real time, (ii) streams a video signal from the first device to the second device in real time, (iii) receives information corresponding to the phrase to be spoken by the character from the second device, and (iv) causes the first device to present audio of the character speaking the phrase using the text-to-speech voice of the character, and wherein the communications session comprises multiple audio communications of the first user and multiple phrases spoken by the character;

the first device is configured to join the communications session after the communications session has been started by the server computer, wherein during the communications session, the first device (i) streams the audio signal corresponding to audio captured by a microphone of the first device to the server computer, (ii) streams the video signal corresponding to video captured by a camera of the first device to the server computer, (iii) receives information corresponding to the phrase from the server computer, and (iv) presents audio of the phrase using the text-to-speech voice of the character;

the second device is configured to join the communications session after the communications session has been started by the server computer, wherein during the communications session, the second device (i) presents audio corresponding to the streamed audio signal, (ii) presents video corresponding to the streamed video signal, (iii) receives the information corresponding to the phrase from the second user, and (iv) transmits the information corresponding to the phrase to the server computer; and

the server computer is configured to;

evaluate the performance of the second user,

determine that the performance level of the second user is lower than a performance threshold, and

request assistance of a third user.

2. The system of claim 1, wherein:

the first user is a child with autism; and

the second user has been trained to communicate with autistic children.

3. The system of claim 1, wherein the second user specifies the phrase by at least one of:

selecting the phrase from a list of phrases, typing the phrase, or speaking the phrase.

4. The system of claim 1, wherein the second user is a family member of the first user or a medical professional who is treating the first user.

5. The system of claim 1, wherein:

the second device is configured to present, during the communications session, a list of video clips to the second user, receive a selection of a video clip by the second user, and transmit information indicative of the selected video clip to the server computer; and

the server computer is configured to receive, during the communications session, the information indicative of the selected video clip and cause the video clip to be presented by the first device and the second device.

6. The system of claim 1, wherein the server computer is configured to:

create a summary of the communications session, wherein the summary comprises at least one of;

a transcript of at least a portion of the communications session, audio of at least a portion of the communications session, or video of at least a portion of the communications session; and

transmit the summary to a person other than the first user.

7. The system of claim 1, wherein the server computer is configured to:

compute a suitability score using the profile of the first user and the profile of the second user; and

select the second user using the suitability score.

8. A computer-implemented method for providing a communications session using a guided character wherein the communications session comprises a first device streaming audio and video of a first user to be presented to a second user of a second device and the second device transmitting a phrase to the first device to be played as audio by the first device using a text-to-speech voice of the guided character, the method comprising:

receiving, from the first device, information indicative of a request of the first user of the first device to start the communications session with the character;

identifying the second user among possible guiding users by comparing profile information of the first user with profile information of the second user, wherein the profile information of the second user is retrieved from a data store comprising profile information of the possible guiding users;

sending, to the second device of the second user, information indicative of a request for the second user to guide the character presented by the first device in the communications session;

receiving, from the second device, an acceptance of the request;

after the acceptance of the request has been received, starting, by a computer, the communications session between the first device and the second device wherein, during the communications session, the computer (i) streams an audio signal from the first device to the second device in real time, (ii) streams a video signal from the first device to the second device in real time, (iii) receives information corresponding to the phrase to be spoken by the character from the second device, and (iv) causes the first device to present audio of the character speaking the phrase using the text-to-speech voice of the character, and wherein the communications session comprises multiple audio communications of the first user and multiple phrases spoken by the character;

adding a third device of a third user to the communications session, wherein the third user is a guiding user;

streaming the audio signal to the third device in real time;

streaming the video signal to the third device in real time;

receiving information about a second phrase to be spoken by the character from the third device; and

causing the first device to present audio of the character speaking the second phrase using the text-to-speech voice of the character.

9. The computer-implemented method of claim 8, comprising:

computing a suitability score for each of the guiding users, wherein each suitability score is computed using a profile of the first user and a profile of a guiding user; and

wherein the second user is identified using the suitability scores.

10. The computer-implemented method of claim 9, comprising:

identifying a third user using the suitability scores;

sending, to a third device of the third user, information indicative of a request for the third user to guide the character of the first device in the communications session; and

selecting the second user using a response time of the second user and a response time of the third user.

11. The computer-implemented method of claim 10, wherein the second user responded before the third user.

12. The computer-implemented method of claim 8, wherein the third device is added to the communications session in response to an end of a work shift of the second user or in response to the first user needing additional assistance.

13. The computer-implemented method of claim 8, comprising facilitating communications between the second user and the third user and wherein the first user does not receive the communications between the second user and the third user.

14. The computer-implemented method of claim 8, wherein the third user is at least one of:

more highly skilled than the second user, a medical professional who is treating the first user, or a family member of the first user.

15. One or more non-transitory computer-readable media comprising computer executable instructions that, when executed, cause at least one processor to perform actions comprising:

receiving, from a first device, information indicative of a request of a first user of the first device to start a communications session with a character presented by the first device;

identifying a second user among possible guiding users by comparing profile information of the first user with profile information of the second user, wherein the profile information of the second user is retrieved from a data store comprising profile information of the possible guiding users;

sending, to a second device of the second user, information indicative of a request for the second user to guide the character presented by the first device in the communications session;

receiving, from the second device, an acceptance of the request;

after the acceptance of the request has been received, starting, by a computer, the communications session between the first device and the second device wherein, during the communications session, the computer (i) streams an audio signal from the first device to the second device in real time, (ii) streams a video signal from the first device to the second device in real time, (iii) receives a information about a phrase to be spoken by the character from the second device, and (iv) causes the first device to present audio of the character speaking the phrase using a text-to-speech voice of the character, and wherein the communications session comprises multiple audio communications of the first user and multiple phrases spoken by the character;

adding a third device of a third user to the communications session, wherein the third user is a guiding user;

streaming the audio signal to the third device in real time;

streaming the video signal to the third device in real time; and

disconnecting (i) the second device from the communications session while the third device is still part of the communications session or (ii) the third device from the communications session while the second device is still part of the communications session.

16. The one or more non-transitory computer-readable media of claim 15, the actions comprising:

determining a state and severity level of the first user using audio or video or both received from the first device; and

selecting the second user using the state and the severity level.

17. The one or more non-transitory computer-readable media of claim 15 in which the actions comprise:

receiving an evaluation of the second user by a person other than the first user; and

updating a profile of the second user with the evaluation.

18. The one or more non-transitory computer-readable media of claim 15 in which the actions comprise:

evaluating the second user by at least one of (1) determining an amount of time for the first user to respond to the phrase presented by the character or (2) a number of words in a response of the first user; and

updating a profile of the second user with the evaluation.

19. The one or more non-transitory computer-readable media of claim 15, wherein audio comprising speech of the second user is not presented to the first user and video of the second user is not presented to the first user.