System and method for recognizing speech securely using a secure multi-party computation protocol

• US 7,937,270 B2
• Filed: 01/16/2007
• Issued: 05/03/2011
• Est. Priority Date: 01/16/2007
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Shashanka, Madhusudana, Smaragdis, Paris
• Primary Examiner(s): Smits; Talivaldis Ivars
• Assistant Examiner(s): Roberts; Shaun
• Application Number:
  US11/623,522
• Publication Number:
  US 20080172233A1
• Time in Patent Office: 1,568 Days
• Field of Search: 704/256, 704/273, 380/28
• US Class Current: 704/256
• CPC Class Codes: G10L 15/142 Hidden Markov Models [HMMs]
• CPC Class Codes: G10L 15/144 Training of HMMs
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: H04K 1/00 Secret communication
• CPC Class Codes: H04L 2209/46 Secure multiparty computation, e.g. millionaire problem
• CPC Class Codes: H04L 2209/50 Oblivious transfer
• CPC Class Codes: H04L 9/008 involving homomorphic encryption

Abstract

A system and method recognizes speech securely using a secure multi-party computation protocol. The system includes a client and a server. The client is configured to provide securely speech in a form of an observation sequence of symbols, and the server is configured to provide securely a multiple trained hidden Markov models (HMMs), each trained HMM including a multiple states, a state transition probability distribution and an initial state distribution, and each state including a subset of the observation symbols and an observation symbol probability distribution. The observation symbol probability distributions are modeled by mixtures of Gaussian distributions. Also included are means for determining securely, for each HMM, a likelihood the observation sequence is produced by the states of the HMM, and means for determining a particular symbol with a maximum likelihood of a particular subset of the symbols corresponding to the speech.

Claims

1. A method for recognizing a speech unit stored at a client using hidden Markov models (HMMs) stored at a server, each HMM corresponds to a unit of recognizable speech, wherein the speech unit is represented as feature vectors, and wherein each feature vector is partitioned into two random additive shares such that the server receives only one random additive share from the client, the method comprising the steps of:

determining iteratively by the server, in response to receiving a random additive share of each feature vector, an additive share of a likelihood of the speech unit with respect to the each HMM using at least one secure multi-party computation protocol to produce additive shares of likelihoods of units of recognizable speech, wherein the secure multi-party computation protocol uses as inputs the two random additive shares of a corresponding feature vector, each of the two random additive shares is provided by the client and by the server respectively; and

transmitting the additive shares of the likelihoods for recognizing the speech unit.

2. The method of claim 1, wherein the secure multi-party computation protocol is selected from a group consisting of a secure inner products protocol , a secure maximum index protocol, a secure maximum value protocol , a secure logsum protocol and a Gaussian mixture likelihood protocol.

3. The method of claim 1, further comprising:

determining by the server, an additive share of an index of a maximum likelihood of the unit of speech using a secure maximum index protocol; and

transmitting the additive share of the index of the maximum likelihood to the client.

4. The method of claim 1, wherein the unit of speech includes at least some of letters, words, phonemes, syllables, and phrases, or combinations thereof.

5. The method of claim 1, wherein the likelihoods of the units of recognizable speech given a sequence of states of each HMM, is a single multivariate Gaussian distribution.

6. The method of claim 1, wherein the likelihood of the units of recognizable speech given a sequence of states of each HMM, is a mixture of multiple weighted multivariate Gaussian distributions.

7. The method of claim 1, wherein the cryptographic protocol is a secure logsum protocol (SLOG), and wherein the SLOG comprises:

determining an additive share of an inner product of exponents of the two random additive shares using a secure inner products protocol.

8. A server configured for recognizing a speech unit stored at a client using hidden Markov models (HMMs) stored at a server, each HMM corresponds to a unit of recognizable speech, wherein the speech unit is represented as feature vectors, and wherein each feature vector is partitioned into two random additive shares such that the server receives only one random additive share from the client, comprising:

means for determining iteratively by the server, in response to receiving a random additive share of each feature vector, an additive share of a likelihood of the speech unit with respect to the each HMM using at least one secure multi-party computation protocol to produce additive shares of likelihoods of units of recognizable speech, wherein the secure multi-party computation protocol uses as inputs the two random additive shares of a corresponding feature vector, each of the two random additive shares is provided by the client and by the server respectively; and

means for transmitting the additive shares of the likelihoods for recognizing the speech unit.

9. The server of claim 8, wherein the secure multi-party computation protocol is selected from a group of a secure inner products protocol (SIP), a secure maximum index protocol (SMAX), a secure maximum value protocol (SVAL), a secure logsum protocol (SLOG) and a Gaussian mixture likelihood protocol (GML).

10. The server of claim 8, further comprising:

means for determining by the server, an additive share of an index of a maximum likelihood of the unit of speech using a secure maximum index protocol (SMAX); and

means for transmitting the additive share of the index of the maximum likelihood to the client.

11. The server of claim 8, wherein the unit of speech includes at least some of letters, words, phonemes, syllables, and phrases, or combinations thereof.

12. The server of claim 8, wherein the likelihood of the unit of speech given a sequence of states of each HMM is a single multivariate Gaussian distribution.

13. The server of claim 8, wherein the likelihood of the unit of speech given a sequence of states of each HMM is a mixture of multiple weighted multivariate Gaussian distributions.

14. The server of claim 8, wherein the cryptographic protocol is a secure logsum protocol (SLOG), and wherein the SLOG comprises:

means for determining an additive share of an inner product of exponents of the two random additive shares using a secure inner products protocol (SIP).