Method and Apparatus for Voice Searching for Stored Content Using Uniterm Discovery

• US 20090210226A1
• Filed: 02/15/2008
• Published: 08/20/2009
• Est. Priority Date: 02/15/2008
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Motorola Mobility, Inc. (Lenovo Group Ltd.)
• Inventors: Ma, Changxue
• Granted Patent: US 8,015,005 B2
• Time in Patent Office: Days
• US Class Current: 704/236
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: G10L 2015/088 Word spotting

Abstract

A method, system and communication device for enabling voice-to-voice searching and ordered content retrieval via audio tags assigned to individual content, which tags generate uniterms that are matched against components of a voice query. The method includes storing content and tagging at least one of the content with an audio tag. The method further includes receiving a voice query to retrieve content stored on the device. When the voice query is received, the method completes a voice-to-voice search utilizing uniterms of the audio tag, scored against the phoneme latent lattice model generated by the voice query to identify matching terms within the audio tags and corresponding stored content. The retrieved content(s) associated with the identified audio tags having uniterms that score within the phoneme lattice model are outputted in an order corresponding to an order in which the uniterms are structured within the voice query.

Claims

1. In an electronic device, a method comprising:

storing one or more content, wherein said one or more content includes one or more of text, images, audio, videos, and multimedia content;

tagging at least one of the one or more content with an audio tag;

receiving a voice query to retrieve content from the one or more content stored on the device;

completing a voice-to-voice search utilizing uniterms of the audio tag and a phoneme latent lattice model generated from the voice query to identify audio tags tagged to stored content among the audio tags of the one or more content, which audio tags provide one or more uniterms that scores within the phoneme lattice model; and

outputting retrieved content associated with the identified audio tags having uniterms that score within the phoneme lattice model, wherein the retrieved content is outputted in an order corresponding to an order in which the uniterms are structured within the voice query.

2. The method of claim 1, further comprising:

retrieving one or more keywords from the voice query;

identifying an maintaining a keyword order of the one or more keywords within the voice query as an output order for the retrieved content; and

when the retrieved content is identified, outputting the retrieved content in the keyword order.

3. The method of claim 1, wherein said completing further comprises:

generating one or more first phoneme lattices from audio tags;

determining one or more best paths from the one or more first phoneme lattices;

extracting one or more uniterms from the one or more first phoneme lattices;

storing the one or more uniterms and the one or more best paths in a uniterm index database; and

re-associating the one or more uniterms with corresponding, stored content with the associated audio tag from which the uniterm was generated.

4. The method of claim 3, wherein extracting one or more uniterms comprises:

generating a next latent statistical lattice model from the one or more phoneme lattice(s) generated from the audio tags;

extracting phoneme strings with a length that is at least equal to a pre-set minimum length from the phoneme lattice(s) as the one or more best paths;

scoring the one or more best paths against the next latent statistical lattice model; and

identifying a preset number of best strings as the uniterms selected to represent the phoneme lattice.

5. The method of claim 4, wherein said generating further comprises:

forwarding the voice query to a speech recognizer, which speech recognizer evaluates received audio and generates one or more phoneme lattices from the received audio;

generating the phoneme lattice from the received audio; and

wherein the statistical latent lattice model represents an application of a series of statistical probabilities to the phoneme lattice.

6. The method of claim 3, wherein said completing further comprises:

generating a latent statistical lattice model from one or more second phoneme lattices generated from the voice query;

scoring a plurality of uniterms from the first phoneme lattices against the latent statistical lattice model to determine a set of best scoring uniterms; and

retrieving content associated with the set of best scoring uniterms as a response to the voice query.

7. The method of claim 6, wherein said scoring further comprises:

performing a coarse search of the statistical latent lattice model with the plurality of uniterms and the one or more best paths to generate a plurality of coarse search candidates; and

performing a fine search on the coarse search candidates, which fine search involves comparison of the coarse search candidates against the phoneme lattice generated from the voice query to generate a fine search output from among the coarse search candidates.

8. The method of claim 7, wherein:

performing the coarse search further comprises;

retrieving a uniterm phoneme tree from a uniterm index database, wherein the uniterm phoneme tree is a tree that includes substantially all the uniterms discovered from the audio database;

scoring the uniterms of the uniterm phoneme tree against the statistical latent lattice model, wherein a statistical probability of a match of the uniterms and branches of the uniterm phoneme tree to the latent lattice model is provided;

evaluating a resulting score to determine which branches of the uniterm phoneme tree are the top branches, having one of (a) a highest score relative to other branches and (b) a score above a pre-set minimum score; and

identifying the top branches as a result of the coarse search, representing coarse search candidates for utilization as inputs for performing the fine search; and

wherein performing the fine search further comprises;

matching the top branches resulting from the coarse search against the one or more second phoneme lattices of the voice query; and

outputting a top set of audio segments resulting from the fine search as the response to the voice query.

9. The method of claim 1, wherein the audio tag is a voice identifier (ID).

10. The method of claim 1, further comprising:

combining two or more different content retrieved by the search in the keyword order to generate a combined content;

outputting the combined content as a single content; and

enabling storing of the combined content with a new audio tag within the device.

11. A device comprising:

a processor;

an audio input device for receiving audio data, including audio tags and voice queries;

a storage mechanism for storing content, and corresponding audio tags;

an output mechanism for outputting stored content based on a search of tags associated with the content;

a voice search and content ordering (VSCO) utility executing on the processor and having logic for completing the following functions;

storing one or more content, wherein said one or more content includes one or more of text, images, audio, videos, and multimedia content;

tagging at least one of the one or more content with an audio tag;

receiving a voice query to retrieve content from the one or more content stored on the device;

triggering completion of a voice-to-voice search utilizing uniterms of the audio tag and a phoneme latent lattice model generated from the voice query to identify audio tags tagged to stored content among the audio tags of the one or more content, which audio tags provide one or more uniterms that scores within the phoneme lattice model; and

outputting retrieved content associated with the identified audio tags having uniterms that score within the phoneme lattice model, wherein the retrieved content is outputted in an order corresponding to an order in which the uniterms are structured within the voice query.

12. The device of claim 11, further comprising:

a uniterm discovery and search (UDS) engine executing on the processor and having functional logic for completing the following functions;

generating one or more first phoneme lattices from audio data stored within an audio database;

determining one or more best paths from the one or more first phoneme lattices;

extracting one or more uniterms from the one or more first phoneme lattices; and

storing the one or more uniterms and the one or more best paths in a uniterm index database;

wherein said triggering activates the UDS engine to complete the voice-to-voice search.

13. The device of claim 11, said logic of the VSCO utility for triggering completion of a voice-to-voice search further comprises functional logic for performing the functions of:

generating one or more first phoneme lattices from audio tags;

determining one or more best paths from the one or more first phoneme lattices;

extracting one or more uniterms from the one or more first phoneme lattices;

storing the one or more uniterms and the one or more best paths in a uniterm index database; and

re-associating the one or more uniterms with corresponding, stored content with the associated audio tag from which the uniterm was generated.

14. The device of claim 13, wherein said logic for extracting one or more uniterms comprises logic for performing the functions of:

generating a next latent statistical lattice model from the one or more phoneme lattice(s) generated from the audio tags;

extracting phoneme strings with a length that is at least equal to a pre-set minimum length from the phoneme lattice(s) as the one or more best paths;

scoring the one or more best paths against the next latent statistical lattice model; and

identifying a preset number of best strings as the uniterms selected to represent the phoneme lattice.

15. The device of claim 12, wherein said functional logic for generating further comprises logic for performing the functions of:

forwarding the voice query to a speech recognizer, which speech recognizer evaluates received audio and generates one or more phoneme lattices from the received audio;

generating the phoneme lattice from the received audio; and

wherein the statistical latent lattice model represents an application of a series of statistical probabilities to the phoneme lattice.

16. The device of claim 13, wherein said functional logic for completing further comprises logic for performing the functions of:

generating a latent statistical lattice model from one or more second phoneme lattices generated from the voice query;

scoring a plurality of uniterms from the first phoneme lattices against the latent statistical lattice model to determine a set of best scoring uniterms; and

retrieving content associated with the set of best scoring uniterms as a response to the voice query.

17. The device of claim 16, wherein the functional logic for scoring comprises logic for performing the functions of:

performing a coarse search of the statistical latent lattice model with the plurality of uniterms and the one or more best paths to generate a plurality of coarse search candidates; and

performing a fine search on the coarse search candidates, which fine search involves comparison of the coarse search candidates against the phoneme lattice generated from the voice query to generate a fine search output from among the coarse search candidates.

18. The device of claim 17, wherein:

the functional logic for performing the coarse search further comprises logic for;

retrieving a uniterm phoneme tree from a uniterm index database, wherein the uniterm phoneme tree is a tree that includes substantially all the uniterms discovered from the audio database;

scoring the uniterms of the uniterm phoneme tree against the statistical latent lattice model, wherein a statistical probability of a match of the uniterms and branches of the uniterm phoneme tree to the latent lattice model is provided;

evaluating a resulting score to determine which branches of the uniterm phoneme tree are the top branches, having one of (a) a highest score relative to other branches and (b) a score above a pre-set minimum score; and

identifying the top branches as a result of the coarse search, representing coarse search candidates for utilization as inputs for performing the fine search; and

the functional logic for performing the fine search further comprises logic for;

matching the top branches resulting from the coarse search against the one or more second phoneme lattices of the voice query; and

outputting a top set of audio segments resulting from the fine search as the response to the voice query.

19. The device of claim 11, wherein the audio tag is a voice identifier (ID) and the device is a mobile communication device.

20. The device of claim 11, wherein said VSCO utility further comprises functional logic for:

combining two or more different content retrieved by the search in the keyword order to generate a combined content;

outputting the combined content as a single content; and

enabling storing of the combined content with a new audio tag within the device.