Methods and systems of handling a dialog with a robot

• US 10,008,196 B2
• Filed: 04/17/2015
• Issued: 06/26/2018
• Est. Priority Date: 04/17/2014
• Status: Active Grant

• Current Assignee: SoftBank Robotics Europe SAS
• Original Assignee: SoftBank Robotics Europe SAS
• Inventors: Maisonnier, Bruno, Houssin, David, Pataillot, Rmy, Barbieri, Gabriele
• Primary Examiner(s): ALBERTALLI, BRIAN LOUIS
• Application Number:
  US15/300,715
• Publication Number:
  US 20170125008A1
• Time in Patent Office: 1,166 Days
• CPC Class Codes: B25J 11/0005 Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means
• CPC Class Codes: B25J 11/003 Manipulators for entertainment
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G10L 13/0335 Pitch control
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

A computer-implemented method of handling an audio dialog between a robot and a human user comprises: during the audio dialog, receiving audio data and converting the audio data into text data; in response to the verification of one or more dialog mode execution rules of the text data, selecting a modified dialog mode; wherein a dialog mode comprises one or more dialog contents and one or more dialog voice skins; wherein a dialog content comprises a collection of predefined sentences, the collection comprising question sentences and answer sentences; and wherein a dialog voice skin comprises voice rendering parameters comprising frequency, tone, velocity and pitch. Described developments comprise modifying dialog contents and/or dialog voice skin, using dialog execution rules (for example depending on the environment perceived by the robot) and moderating dialog contents.

Claims

1. A computer-implemented method of handling an audio dialog between a companion robot and a human user, the method comprising:

during said audio dialog, receiving audio data and converting said audio data into text data;

in response to verification of one or more dialog mode execution rules of said text data, selecting a modified dialog mode;

wherein a dialog mode comprises one or more dialog contents and one or more dialog voice skins;

wherein a dialog content comprises a collection of predefined sentences, said collection comprising question sentences and answer sentences;

wherein a dialog voice skin comprises voice rendering parameters comprising frequency, tone, velocity and pitch;

wherein said one or more dialog contents and/or voice skins are authored or edited online using a web platform;

wherein one or more predefined dialog contents and/or voice skins are modified by multiple parties;

wherein one or more dialog contents or a selection thereof are moderated by application of one or more filters, said filters comprising blacklists of one or more words, white lists of one or more words and/or dialog mode execution rules; and

wherein a moderation of use of said one or more dialog content and/or voice skins to a final dialog expressed by the companion robot comprise the use of secured boot methods.

2. The method of claim 1, wherein an edition of one or more dialog contents and/or voice skins comprises one or more steps comprising mixing predefined voice skins and/or providing a web platform user with creation controls to fine tune voice skins and/or uploading and sharing sounds or recorded sentences or combinations thereof.

3. The method of claim 1, wherein one or more predefined dialog contents are extracted from the Internet.

4. The method of claim 1, further comprising modifying one or more dialog contents, receiving one or more user feedbacks and propagating said one or more modified dialog contents to another robot.

5. The method of claim 1, further comprising regulating the use of one or more dialog contents, said regulating step comprising one or more steps comprising modulating, filtering attenuating, amplifying, increasing, encouraging, decreasing, inhibiting, limiting, avoiding or forbidding the use of one or more dialog contents and/or voice skins and/or associated execution rules.

6. The method of claim 1, wherein a manufacturer of the robot has control over a final audio rendering or latest point before analog audio restitution.

7. The method of claim 1, further comprising executing said selected modified dialog mode.

8. The method of claim 1, wherein the modified dialog mode is obtained by modifying the current dialog content and/or to the current dialog voice skin of the current dialog.

9. The method of claim 8, wherein modifying the current dialog content comprises using synonyms of words of said dialog content and applying syntactic modifications comprising insertion or permutation or substitution of one or more words of said dialog content.

10. The method of claim 8, wherein modifying the current dialog voice skin comprises modifying frequency and/or tone and/or velocity and/or pitch of the current dialog voice skin.

11. The method of claim 1, wherein the modified dialog mode is obtained by activating a predefined dialog mode, said predefined dialog having been selected by the one or more dialog execution rules and said predefined dialog mode comprising a predefined dialog content and/or a predefined dialog voice skin.

12. The method of claim 1, wherein a dialog mode execution rule is dependent on an environment perceived by the robot.

13. The method of claim 12, wherein a dialog mode execution rule comprises parameters selected from the list comprising age of a user, gender of a user, mood of a user, emotion of a user, number of users, interaction history with a user, user preferences, spatial placement of the robot and/or of a user, gesture or combination of gestures of the robot and/or a user, detected event in the environment of the robot, local weather, geolocation, date, time and combinations thereof.

14. The method of claim 1, wherein a dialog mode execution rule is dynamically retrieved from the Internet and/or is user-configurable.

15. The method of claim 1, wherein a dialog content and/or a dialog voice skin is selected by voice command or user request.

16. The method of claim 1, further comprising identifying a missing dialog mode, retrieving and installing said missing dialog mode in the robot during the dialog with the user.

17. The method of claim 1, further comprising receiving the confirmation of the user before executing a selected dialog mode.

18. A non-transitory computer program comprising instructions for carrying out the steps of the method according to claim 1 when said non-transitory computer program is executed on a suitable computer device.

19. A system comprising means adapted to carry out the steps of the method according to claim 1.