Automated speech pronunciation attribution

• US 10,013,971 B1
• Filed: 12/29/2016
• Issued: 07/03/2018
• Est. Priority Date: 12/29/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Lewis, Justin, Takehana, Lisa
• Primary Examiner(s): Azad, Abul
• Application Number:
  US15/394,104
• Publication Number:
  US 20180190262A1
• Time in Patent Office: 551 Days
• Field of Search: 704258-271
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/54 Presence management, e.g. monitoring or registration for receipt of user log-on information, or the connection status of the users

Abstract

Methods, systems, and apparatus for determining candidate user profiles as being associated with a shared device, and identifying, from the candidate user profiles, candidate pronunciation attributes associated with at least one of the candidate user profiles determined to be associated with the shared device. The methods, systems, and apparatus are also for receiving, at the shared device, a spoken utterance; determining a received pronunciation attribute based on received audio data corresponding to the spoken utterance; comparing the received pronunciation attribute to at least one of the candidate pronunciation attributes; and selecting a particular pronunciation attribute from the candidate pronunciation attributes based on a result of the comparison of the received pronunciation attribute to at least one of the candidate pronunciation attributes. With the methods, systems, and apparatus, the particular pronunciation attribute, selected from the candidate pronunciation attributes, is provided for outputting audio associated with the spoken utterance.

Claims

1. A computer-implemented method comprising:

determining candidate user profiles that are associated with a shared digital assistant device;

determining that a mobile computing device that is associated with a particular candidate user profile, from among the candidate user profiles that are associated with the shared digital assistant device, is indicated as being proximate to the shared digital assistant device;

identifying, from the candidate user profiles, candidate pronunciation attributes associated with the particular candidate user profile that is associated with the mobile computing device that is indicated as being proximate to the shared digital assistant device;

receiving, at the shared digital assistant device, a spoken utterance;

determining a received pronunciation attribute based on received audio data corresponding to the spoken utterance;

comparing the received pronunciation attribute to at least one of the candidate pronunciation attributes that are associated with the particular candidate user profile that is associated with the mobile computing device that is indicated as being proximate to the shared digital assistant device;

selecting a particular pronunciation attribute from the candidate pronunciation attributes that that are associated with the particular candidate user profile that is associated with the mobile computing device that is indicated as being proximate to the shared digital assistant device based on a result of the comparison of the received pronunciation attribute to at least one of the candidate pronunciation attributes; and

providing the particular pronunciation attribute, selected from the candidate pronunciation attributes, for outputting audio associated with the spoken utterance.

2. The computer-implemented method of claim 1, wherein determining candidate user profiles that are associated with a shared digital assistant device comprises:

determining a relationship between each of a plurality of user profiles and the shared digital assistant device;

determining, for each user profile, whether the relationship is indicative of an association between the user profile and the shared digital assistant device; and

identifying, for each user profile having a relationship indicative of an association with the shared digital assistant device, the user profile as being one of the candidate user profiles associated with the shared digital assistant device.

3. The computer-implemented method of claim 2, wherein, for each of the plurality of user profiles, the relationship comprises a record of whether the user profile has been logged-in to the shared digital assistant device or whether at least one user device associated with the user profile has communicated with the shared digital assistant device.

4. The computer-implemented method of claim 2, wherein, for each of the plurality of user profiles, the relationship comprises a geographical proximity of at least one user device associated with the user profile to the shared digital assistant device.

5. The computer-implemented method of claim 2, wherein, for each of the plurality of user profiles, the relationship comprises a social connectivity, the social connectivity being based on at least one social connectivity metric.

6. The computer-implemented method of claim 1, wherein each user profile of the candidate user profiles comprises one or more pronunciation attributes associated with a canonical identifier, the canonical identifier representing a particular pronunciation.

7. The computer-implemented method of claim 1, further comprising:

providing an audio response to the spoken utterance, the audio response comprising the particular pronunciation selected from the candidate pronunciation attributes.

8. A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

determining candidate user profiles that are associated with a shared digital assistant device;

determining that a mobile computing device that is associated with a particular candidate user profile, from among the candidate user profiles that are associated with the shared digital assistant device, is indicated as being proximate to the shared digital assistant device;

identifying, from the candidate user profiles, candidate pronunciation attributes associated with the particular candidate user profile that is associated with the mobile computing device that is indicated as being proximate to the shared digital assistant device;

receiving, at the shared digital assistant device, a spoken utterance;

determining a received pronunciation attribute based on received audio data corresponding to the spoken utterance;

comparing the received pronunciation attribute to at least one of the candidate pronunciation attributes that are associated with the particular candidate user profile that is associated with the mobile computing device that is indicated as being proximate to the shared digital assistant device;

selecting a particular pronunciation attribute from the candidate pronunciation attributes that that are associated with the particular candidate user profile that is associated with the mobile computing device that is indicated as being proximate to the shared digital assistant device based on a result of the comparison of the received pronunciation attribute to at least one of the candidate pronunciation attributes; and

providing the particular pronunciation attribute, selected from the candidate pronunciation attributes, for outputting audio associated with the spoken utterance.

9. The system of claim 8, wherein determining candidate user profiles that are associated with a shared digital assistant device comprises:

determining a relationship between each of a plurality of user profiles and the shared digital assistant device;

determining, for each user profile, whether the relationship is indicative of an association between the user profile and the shared digital assistant device; and

identifying, for each user profile having a relationship indicative of an association with the shared digital assistant device, the user profile as being one of the candidate user profiles associated with the shared digital assistant device.

10. The system of claim 9, wherein, for each of the plurality of user profiles, the relationship comprises a record of whether the user profile has been logged-in to the shared digital assistant device or whether at least one user device associated with the user profile has communicated with the shared digital assistant device.

11. The system of claim 9, wherein, for each of the plurality of user profiles, the relationship comprises a geographical proximity of at least one user device associated with the user profile to the shared digital assistant device.

12. The system of claim 9, wherein, for each of the plurality of user profiles, the relationship comprises a social connectivity, the social connectivity being based on at least one social connectivity metric.

13. The system of claim 8, wherein each user profile of the candidate user profiles comprises one or more pronunciation attributes associated with a canonical identifier, the canonical identifier representing a particular pronunciation.

14. The system of claim 8, further comprising:

providing an audio response to the spoken utterance, the audio response comprising the particular pronunciation selected from the candidate pronunciation attributes.

15. A computer-readable storage device storing instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

determining candidate user profiles that are associated with a shared digital assistant device;

determining that a mobile computing device that is associated with a particular candidate user profile, from among the candidate user profiles that are associated with the shared digital assistant device, is indicated as being proximate to the shared digital assistant device;

identifying, from the candidate user profiles, candidate pronunciation attributes associated with the particular candidate user profile that is associated with the mobile computing device that is indicated as being proximate to the shared digital assistant device;

receiving, at the shared digital assistant device, a spoken utterance;

determining a received pronunciation attribute based on received audio data corresponding to the spoken utterance;

comparing the received pronunciation attribute to at least one of the candidate pronunciation attributes that are associated with the particular candidate user profile that is associated with the mobile computing device that is indicated as being proximate to the shared digital assistant device;

selecting a particular pronunciation attribute from the candidate pronunciation attributes that that are associated with the particular candidate user profile that is associated with the mobile computing device that is indicated as being proximate to the shared digital assistant device based on a result of the comparison of the received pronunciation attribute to at least one of the candidate pronunciation attributes; and

providing the particular pronunciation attribute, selected from the candidate pronunciation attributes, for outputting audio associated with the spoken utterance.

16. The computer-readable storage device of claim 15, wherein determining candidate user profiles that are associated with a shared digital assistant device comprises:

determining a relationship between each of a plurality of user profiles and the shared digital assistant device;

determining, for each user profile, whether the relationship is indicative of an association between the user profile and the shared digital assistant device; and

identifying, for each user profile having a relationship indicative of an association with the shared digital assistant device, the user profile as being one of the candidate user profiles associated with the shared digital assistant device.

17. The computer-readable storage device of claim 16, wherein, for each of the plurality of user profiles, the relationship comprises a record of whether the user profile has been logged-in to the shared digital assistant device or whether at least one user device associated with the user profile has communicated with the shared digital assistant device.

18. The computer-readable storage device of claim 16, wherein, for each of the plurality of user profiles, the relationship comprises a geographical proximity of at least one user device associated with the user profile to the shared digital assistant device.

19. The computer-readable storage device of claim 16, wherein, for each of the plurality of user profiles, the relationship comprises a social connectivity, the social connectivity being based on at least one social connectivity metric.

20. The computer-readable storage device of claim 15, wherein each user profile of the candidate user profiles comprises one or more pronunciation attributes associated with a canonical identifier, the canonical identifier representing a particular pronunciation.