METHOD AND APPARATUS FOR UNITERM DISCOVERY AND VOICE-TO-VOICE SEARCH ON MOBILE DEVICE

• US 20090164218A1
• Filed: 12/21/2007
• Published: 06/25/2009
• Est. Priority Date: 12/21/2007
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: MA, CHANGXUE
• Granted Patent: US 8,019,604 B2
• Time in Patent Office: Days
• US Class Current: 704/254
• CPC Class Codes: G06F 16/632 Query formulation
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit

Abstract

A method, system and communication device for enabling uniterm discovery from audio content and voice-to-voice searching of audio content stored on a device using discovered uniterms. Received audio/voice input signal is sent to a uniterm discovery and search (UDS) engine within the device. The audio data may be associated with other content that is also stored within the device. The UDS engine retrieves a number of uniterms from the audio data and associates the uniterms with the stored content. When a voice search is initiated at the device, the UDS engine generates a statistical latent lattice model from the voice query and scores the uniterms from the audio database against the latent lattice model. Following a further refinement, the best group of uniterms is then determined and segments of the stored audio data and/or other content corresponding to the best group of uniterms are outputted.

Claims

1. In an electronic device, a method comprising:

generating one or more first phoneme lattices from audio data stored within an audio database;

determining one or more best paths from the one or more first phoneme lattices;

extracting one or more uniterms from the one or more first phoneme lattices; and

storing the one or more uniterms and the one or more best paths in a uniterm index database.

2. The method of claim 1, further comprising:

receiving a voice query for retrieval of stored content;

generating a latent statistical lattice model from one or more second phoneme lattices generated from the voice query;

scoring a plurality of uniterms against the latent statistical lattice model to determine a set of best scoring uniterms; and

retrieving content associated with the set of best scoring uniterms as a response to the voice query.

3. The method of claim 1, further comprising:

storing the one or more uniterms in a uniterms phoneme tree structure; and

forwarding the uniterms phoneme tree structure and the one or more best paths to a coarse search function that scores the one or more uniterms of the uniterms phoneme tree structure against the statistical latent lattice model.

4. The method of claim 2, wherein:

generating one or more first phoneme lattices further comprises forwarding the audio data from the audio database to a speech recognizer, which speech recognizer evaluates received audio and generates the one or more phoneme lattices from the received audio.

5. The method of claim 1, wherein extracting one or more uniterms comprises:

generating a next latent statistical lattice model from the one or more phoneme lattice(s) generated from the audio data;

extracting phoneme strings with a length that is at least equal to a pre-set minimum length from the phoneme lattice(s) as candidates for the one or more uniterms;

scoring the candidates for the one or more uniterms against the next latent statistical lattice model; and

identifying a preset number of candidates with best scores as the one or more uniterms selected to represent the phoneme lattice.

6. The method of claim 2, wherein said generating further comprises:

forwarding the voice query to a speech recognizer, which speech recognizer evaluates received audio and generates one or more phoneme lattices from the received audio;

generating the phoneme lattice from the received audio; and

wherein the statistical latent lattice model represents an application of a series of statistical probabilities to the phoneme lattice.

7. The method of claim 2, wherein said scoring further comprises:

performing a coarse search of the statistical latent lattice model with the plurality of uniterms and the one or more best paths to generate a plurality of coarse search candidates;

performing a fine search on the coarse search candidates, which fine search involves comparison of the coarse search candidates against the phoneme lattice generated from the voice query to generate a fine search output from among the coarse search candidates.

8. The method of claim 7, wherein performing the coarse search further comprises:

retrieving a uniterm phoneme tree from a uniterm index database, wherein the uniterm phoneme tree is a tree that includes substantially all the uniterms discovered from the audio database;

scoring the uniterms of the uniterm phoneme tree against the statistical latent lattice model, wherein a statistical probability of a match of the uniterms and branches of the uniterm phoneme tree to the latent lattice model is provided;

evaluating a resulting score to determine which branches of the uniterm phoneme tree are the top branches, having one of (a) a highest score relative to other branches and (b) a score above a pre-set minimum score; and

identifying the top branches as a result of the coarse search, representing coarse search candidates for utilization as inputs for performing the fine search.

9. The method of claim 8, wherein performing the fine search further comprises:

matching the top branches resulting from the coarse search against the one or more second phoneme lattices of the voice query; and

outputting a top set of audio segments resulting from the fine search as the response to the voice query.

10. The method of claim 9, wherein outputting the top set of audio segments further comprises:

retrieving non-audio content associated with the top set of audio segments; and

outputting the non-audio content as the response to the voice query.

11. A device comprising:

a processor;

an audio input device for receiving audio data, including voice input data and voice queries;

a storage mechanism for storing content, including the audio data;

a uniterm discovery and search (UDS) engine executing on the processor and having functional components for completing the following functions;

generating one or more first phoneme lattices from audio data stored within an audio database;

determining one or more best paths from the one or more first phoneme lattices;

extracting one or more uniterms from the one or more first phoneme lattices; and

storing the one or more uniterms and the one or more best paths in a uniterm index database.

12. The device of claim 11, said UDS engine further comprising functional components for performing the functions of:

receiving a voice query for retrieval of stored content;

generating a latent statistical lattice model from one or more second phoneme lattices generated from the voice query;

scoring a plurality of uniterms against the latent statistical lattice model to determine a set of best scoring uniterms; and

retrieving content associated with the set of best scoring uniterms as a response to the voice query.

13. The device of claim 11, wherein the functional component for generating one or more first phoneme lattices further performs the function of forwarding the audio data from the audio database to a speech recognizer, which speech recognizer evaluates received audio and generates the one or more phoneme lattices from the received audio.

14. The device of claim 11, wherein the functional component for extracting one or more uniterms, further performs the functions of:

generating a next latent statistical lattice model from the one or more phoneme lattice(s) generated from the audio data;

extracting phoneme strings with a length that is at least equal to a pre-set minimum length from the phoneme lattice(s) as candidates for the one or more uniterms;

scoring the candidates for the one or more uniterms against the next latent statistical lattice model;

identifying a preset number of candidates with best scores as the one or more uniterms selected to represent the phoneme lattice;

storing the one or more uniterms in a uniterms phoneme tree structure; and

forwarding the uniterms phoneme tree structure and the one or more best paths to a coarse search function that scores the one or more uniterms of the uniterms phoneme tree structure against the statistical latent lattice model.

15. The device of claim 12, wherein said functional component for generating further performs the functions of:

forwarding the voice query to a speech recognizer, which speech recognizer evaluates received audio and generates one or more phoneme lattices from the received audio;

generating the phoneme lattice from the received audio; and

wherein the statistical latent lattice model represents an application of a series of statistical probabilities to the phoneme lattice.

16. The device of claim 12, wherein said functional component for scoring further performs the functions of:

performing a coarse search of the statistical latent model with the plurality of uniterms and the one or more best paths to generate a plurality of coarse search candidates;

performing a fine search on the coarse search candidates, which fine search involves comparison of the coarse search candidates against the phoneme lattice generated from the voice query to generate a fine search output from among the coarse search candidates.

17. The device of claim 16, wherein the functional component for performing the coarse search further performs the functions of:

retrieving a uniterm phoneme tree from a uniterm index database, wherein the uniterm phoneme tree is a tree that includes substantially all the uniterms discovered from the audio database;

scoring the uniterms of the uniterm phoneme tree against the statistical latent lattice model, wherein a statistical probability of a match of the uniterms and branches of the uniterm phoneme tree to the latent lattice model is provided;

evaluating a resulting score to determine which branches of the uniterm phoneme tree are the top branches, having one of (a) a highest score relative to other branches and (b) a score above a pre-set minimum score; and

identifying the top branches as a result of the coarse search, representing coarse search candidates for utilization as inputs for performing the fine search.

18. The device of claim 17, wherein the functional component for performing the fine search further performs the functions of:

matching the top branches resulting from the coarse search against the one or more second phoneme lattices of the voice query; and

outputting a top set of audio segments resulting from the fine search as the response to the voice query.

19. The method of claim 18, wherein the functional component for outputting the top set of audio segments further performs the functions of:

retrieving non-audio content associated with the top set of audio segments; and

outputting the non-audio content as the response to the voice query.

20. The device of claim 11, wherein the device is a mobile communication device.