SYSTEM AND METHOD FOR SPEAKER RECOGNITION ON MOBILE DEVICES

• US 20130225128A1
• Filed: 12/12/2012
• Published: 08/29/2013
• Est. Priority Date: 02/24/2012
• Status: Active Grant

• Current Assignee: Cirrus Logic International Semiconductor Ltd. (Cirrus Logic Incorporated)
• Original Assignee: Agnitio s.l. (Microsoft Corporation)
• Inventors: Gomar, Marta Garcia
• Granted Patent: US 9,042,867 B2
• Time in Patent Office: Days
• US Class Current: 455/411
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: H04L 63/0861 using biometrical features, e.g. fingerprint, retina-scan cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231
• CPC Class Codes: H04W 12/06 Authentication

Abstract

A speaker recognition system for authenticating a mobile device user includes an enrollment and learning software module, a voice biometric authentication software module, and a secure software application. Upon request by a user of the mobile device, the enrollment and learning software module displays text prompts to the user, receives speech utterances from the user, and produces a voice biometric print. The enrollment and training software module determines when a voice biometric print has met at least a quality threshold before storing it on the mobile device. The secure software application prompts a user requiring authentication to repeat an utterance based at least on an attribute of a selected voice biometric print, receives a corresponding utterance, requests the voice biometric authentication software module to verify the identity of the second user using the utterance, and, if the user is authenticated, imports the voice biometric print.

Claims

1. A speaker recognition system operable to recognize or authenticate a user of a mobile device, the system comprising:

an enrollment and learning software module stored and operable on a mobile device and adapted to receive speech utterances from a user via a microphone on the mobile device;

a voice biometric authentication software module stored and operable on the mobile device; and

a secure software application operable on the mobile device;

wherein, upon request by a first user of the mobile device, the enrollment and learning software module displays a plurality of text prompts to the first user and receives speech utterances from the first user corresponding substantially to at least one of the plurality of text prompts and produces a voice biometric print therefrom;

wherein the enrollment and training software module determines when a voice biometric print has met at least a quality threshold before storing it on the mobile device;

further wherein the secure software application, on request, receives a list of all available voice biometric prints from the enrollment and learning software module, displays at least a portion of the received list to a second user of the mobile device, receives a selection from the second user of a particular voice biometric print to be used by the secure software application, prompts the second user to repeat an utterance based at least on an attribute of the selected voice biometric print, receives an utterance from the second user, requests the voice biometric authentication software module to verify the identity of the second user based on the received utterance, and, if the voice biometric authentication software module verifies the identity of the second user, imports the voice biometric print.

2. The system of claim 1, wherein the imported voice biometric print is stored on the mobile device by the secure software application;

and

wherein the secure software application, when authentication of a third user of the secure software application is required, displays a plurality of text prompts to the third user and receives a plurality of speech utterances from the third user corresponding substantially to at least one of the plurality of text prompts and produces a voice biometric print therefrom, the voice biometric authentication software module analyzes the plurality of speech utterances to obtain thereby a new voice biometric print corresponding to the third user and compares the new voice biometric print corresponding to the third user to the first biometric voice print and, if an indicia of correspondence between the new voice biometric print and the first voice biometric print exceeds a threshold, authenticates the third user and provides privileged access to the third user.

3. The system of claim 2, wherein the enrollment and learning software module comprises a feature extraction module that extracts statistical data from a plurality of speech utterances received from an enrolling user using a universal background model, generates a most-valuable Gaussian based at least on the extracted statistical data, prunes a first set of statistics comprising one or more of a total variability matrix T, the extracted statistical data, or the universal background model in order to reduce data storage and processing required for speaker recognition, and generates a voice biometric print corresponding to the enrolling user;

and

wherein the voice biometric authentication software module comprises a feature extraction module that extracts statistical data from a plurality of speech utterances received from the third user using a universal background model, generates a most-valuable Gaussian based at least on the extracted statistical data, prunes a second set of statistics comprising one or more of a total variability matrix T, the extracted statistical data, or the universal background model using the same pruning in order to reduce data storage and processing required for speaker recognition, wherein the second set of statistics is identical to the first set of statistics pruned by the enrollment and learning module, and generates a voice biometric print corresponding to the third user.

4. The system of claim 3, wherein the voice biometric authentication software module determines when a voice biometric print has met at least a quality threshold before storing it on the mobile device.

5. The system of claim 1, wherein the imported voice biometric print received by the secure software application is transmitted by the secure software application over a network to a remote voice authentication service associated with and accessible by the secure software application;

wherein the enrollment and learning software module comprises a feature extraction module that extracts statistical data from a plurality of speech utterances received from an enrolling user using a universal background model, generates a most-valuable Gaussian based at least on the extracted statistical data, prunes a first set of statistics comprising one or more of a total variability matrix T, the extracted statistical data, or the universal background model in order to reduce data storage and processing required for speaker recognition, and generates a voice biometric print corresponding to the enrolling user;

wherein the voice biometric authentication software module comprises a feature extraction module that extracts statistical data from a plurality of speech utterances received from the third user using a universal background model, generates a most-valuable Gaussian based at least on the extracted statistical data, prunes a second set of statistics comprising one or more of a total variability matrix T, the extracted statistical data, or the universal background model using the same pruning in order to reduce data storage and processing required for speaker recognition, wherein the second set of statistics is identical to the first set of statistics pruned by the enrollment and learning module, generates a voice biometric print corresponding to the third user, and the secure software application sends the voice biometric print corresponding to the third user to the remote voice authentication service;

and wherein the secure software application receives an authentication result from the remote voice authentication service that is based at least on the voice biometric print corresponding to the third user.

6. The system of claim 1, wherein the imported voice biometric print received by the secure software application is transmitted by the secure software application over a network to a remote voice authentication service associated with and accessible by the secure software application;

wherein the secure software application sends the plurality of speech utterances received from the third user to the remote voice authentication service; and

and wherein the secure software application receives an authentication result from the remote voice authentication service that is based at least on the voice biometric print corresponding to the third user.

7. The system of claim 1, wherein the enrollment and learning software module determines whether a voice biometric print has met a quality threshold by one of:

(a) determining whether the plurality of speech utterances from which the voice biometric print is generated comprises at least a specific total speech time, (b) determining whether the plurality of speech utterances from which a voice biometric print is generated displays at least a minimum level of either vocal intensity or vocal variability, or both, and (c) determining whether a noise level present in the plurality of speech utterances from which a voice biometric print is generated exceeds a maximum peak or average value.

8. The system of claim 7, wherein a visual indication is provided to a user of whether more speech time is needed to obtain a voice biometric print of sufficient quality.

9. The system of claim 4 wherein the voice biometric authentication software module determines whether a voice biometric print has met a quality threshold by one of:

(a) determining whether the plurality of speech utterances from which the voice biometric print is generated comprises at least a specific total speech time, (b) determining whether the plurality of speech utterances from which a voice biometric print is generated displays at least a minimum level of either vocal intensity or vocal variability, or both, (c) determining whether a noise level present in the plurality of speech utterances from which a voice biometric print is generated exceeds a maximum peak or average value, and (d) determining whether a signal to noise ratio present in the plurality of speech utterances exceeds some minimum threshold.

10. The system of claim 9, wherein a visual indication is provided to a user of whether more speech time is needed to obtain a voice biometric print of sufficient quality.

11. A method for recognizing or authenticating a user of a mobile device, the method comprising the steps of:

(a) upon receipt of a request from a first user of a mobile device to enroll in a speaker recognition system, the request being received at an enrollment and learning software module stored and operating on the mobile device, the enrollment and learning module displays a first plurality of text prompts to the first user;

(b) receiving a first plurality of speech utterances from the first user corresponding substantially to at least one of the first plurality of text prompts;

(c) extracting one or more voice feature data components from the first plurality of speech utterances in a feature extraction software module operating with or a component of the enrollment and learning module;

(d) generating a set of statistics pertaining to the voice feature data components using a universal background model;

(e) executing, in the enrollment and learning software module, one or more pruning operations to reduce the size of intermediate data objects to reduce any of processor usage, memory usage, or bandwidth usage;

(f) extracting a voice biometric print from the set of statistics using at least a total variability matrix;

(g) determining when an extracted voice biometric print has met at least a quality threshold; and

(h) if the quality threshold step (g) was met, storing the voice biometric print on the mobile device.

12. The method of claim 11, further comprising the steps of:

(i) receiving, at a secure software application stored and operating on the mobile device, a list all available voice biometric prints from the enrollment and learning software module;

(j) displaying at least a portion of the received list to a second user of the mobile device;

(k) receiving from the second user a selection of a particular voice biometric print to be used by the secure software application;

(l) prompting the second user to repeat an utterance based at least on an attribute of the selected voice biometric print;

(m) receiving an utterance from the second user;

(n) requesting a voice biometric authentication software module stored and operating on the mobile device to verify the identity of the second user based on the received utterance; and

(o) if the voice biometric authentication software module verifies the identity of the second user, importing the voice biometric print into the secure software application.

13. The method of claim 12, further comprising the steps of:

(m) storing the imported voice biometric print on the mobile device by the secure software application;

(n) when authentication of a third user of the secure software application is required, displaying a plurality of text prompts to the third user;

(o) receiving a plurality of speech utterances from the third user corresponding substantially to at least one of the plurality of text prompts;

(p) producing a voice biometric print from the received plurality of speech utterances;

(q) analyzing the plurality of speech utterances to obtain thereby a new voice biometric print corresponding to the third user;

(r) comparing the new voice biometric print corresponding to the third user to the first biometric voice print; and

(s) if an indicia of correspondence between the new voice biometric print and the first voice biometric print exceeds a threshold, authenticating the third user and providing privileged access to the third user.

14. The method of claim 13, wherein the enrollment and learning software module comprises a feature extraction module that extracts statistical data from a plurality of speech utterances received from an enrolling user using a universal background model, generates a most-valuable Gaussian based at least on the extracted statistical data, prunes a first set of statistics comprising one or more of a total variability matrix T, the extracted statistical data, or the universal background model in order to reduce data storage and processing required for speaker recognition, and generates a voice biometric print corresponding to the enrolling user;

and

wherein the voice biometric authentication software module comprises a feature extraction module that extracts statistical data from a plurality of speech utterances received from the third user using a universal background model, generates a most-valuable Gaussian based at least on the extracted statistical data, prunes a second set of statistics comprising one or more of a total variability matrix T, the extracted statistical data, or the universal background model using the same pruning in order to reduce data storage and processing required for speaker recognition, wherein the second set of statistics is identical to the first set of statistics pruned by the enrollment and learning module, and generates a voice biometric print corresponding to the third user.

15. The method of claim 11, further comprising the steps of:

(m) transmitting the imported voice biometric print received by the secure software application over a network to a remote voice authentication service associated with and accessible by the secure software application;

(n) sending the plurality of speech utterances received by the secure software application from the third user to the remote voice authentication service; and

(o) receiving, at the secure software application, an authentication result from the remote voice authentication service that is based at least on the voice biometric print corresponding to the third user.

16. The method of claim 11, wherein the enrollment and learning software module determines whether a voice biometric print has met a quality threshold by one of:

(a) determining whether the plurality of speech utterances from which the voice biometric print is generated comprises at least a specific total speech time, (b) determining whether the plurality of speech utterances from which a voice biometric print is generated displays at least a minimum level of either vocal intensity or vocal variability, or both, and (c) determining whether a noise level present in the plurality of speech utterances from which a voice biometric print is generated exceeds a maximum peak or average value.

17. The method of claim 16, wherein a visual indication is provided to a user of whether more speech time is needed to obtain a voice biometric print of sufficient quality.

18. The method of claim 13, wherein the voice biometric authentication software module determines whether a voice biometric print has met a quality threshold by one of:

(a) determining whether the plurality of speech utterances from which the voice biometric print is generated comprises at least a specific total speech time, (b) determining whether the plurality of speech utterances from which a voice biometric print is generated displays at least a minimum level of either vocal intensity or vocal variability, or both, (c) determining whether a noise level present in the plurality of speech utterances from which a voice biometric print is generated exceeds a maximum peak or average value, and (d) determining whether a signal to noise ratio present in the plurality of speech utterances exceeds some minimum threshold.

19. The method of claim 18, wherein a visual indication is provided to a user of whether more speech time is needed to obtain a voice biometric print of sufficient quality.

20. A mobile computing device comprising:

a microphone;

an enrollment and learning software module adapted to receive speech utterances from a user via the microphone;

a voice biometric authentication software module; and

a secure software application with a multimodal interface;

wherein, upon request by a user of the mobile computing device, the enrollment and learning software module displays a plurality of text prompts to the user and receives speech utterances from the user corresponding substantially to at least one of the plurality of text prompts and produces a voice biometric print therefrom;

further wherein the enrollment and training software module determines when a voice biometric print has met at least a quality threshold before exporting the voice biometric print to the secure software application;

wherein the enrollment and learning software module comprises a feature extraction module that extracts statistical data from a plurality of speech utterances received from an enrolling user using a universal background model, generates a most-valuable Gaussian based at least on the extracted statistical data, prunes one or more of a total variability matrix T, the extracted statistical data, or the universal background model in order to reduce data storage and processing required for speaker recognition, and generates a voice biometric print corresponding to the enrolling user; and

further wherein the secure software application, on request, receives a list all available voice biometric prints from the enrollment and learning software module, displays at least a portion of the received list to a second user of the mobile device, receives a selection from the second user of a particular voice biometric print to be used by the secure software application, prompts the second user to repeat an utterance based at least on an attribute of the selected voice biometric print, receives an utterance from the second user, requests the voice biometric authentication software module to verify the identity of the second user based on the received utterance, and, if the voice biometric authentication software module verifies the identity of the second user, imports the voice biometric print.