Voice query extension method and system

• US 8,155,956 B2
• Filed: 03/10/2008
• Issued: 04/10/2012
• Est. Priority Date: 12/18/2007
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Cho, Jeong Mi, Kwak, Byung-Kwan, Kim, Nam Hoon, Han, Ick Sang
• Primary Examiner(s): JACKSON, JAKIEDA R
• Application Number:
  US12/045,138
• Publication Number:
  US 20090157383A1
• Time in Patent Office: 1,492 Days
• Field of Search: 704/235, 704/8
• US Class Current: 704/235
• CPC Class Codes: G06F 16/332 Query formulation
• CPC Class Codes: G06F 16/3337 Translation of the query language, e.g. Chinese to English
• CPC Class Codes: G06F 16/3343 using phonetics
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: G10L 2015/088 Word spotting

Abstract

A voice query extension method and system. The voice query extension method includes: detecting voice activity of a user from an input signal and extracting a feature vector from the voice activity; converting the feature vector into at least one phoneme sequence and generating the at least one phoneme sequence; matching the at least one phoneme sequence with words registered in a dictionary, extracting a string of the matched words with a linguistic meaning, and selecting the string of the matched words as a query; determining whether the query is in a predetermined first language, and when the query is not in the first language as a result of the determining, converting the query using a phoneme to grapheme rule, and generating a query in the first language; and searching using the query in the first language.

Claims

1. A voice query extension method, comprising:

detecting voice activity of a user from an input signal and extracting a feature vector from the voice activity using a computer;

converting the feature vector into at least one phoneme sequence and generating the at least one phoneme sequence;

matching the at least one phoneme sequence with words registered in a language dictionary, extracting a string of the matched words with a linguistic meaning, and selecting the string of the matched words as a query;

determining whether the query is in a predetermined first language, and when the query is not in the first language as a result of the determining, converting the query using a phoneme to grapheme rule, and generating a query in the first language; and

searching using the query in the first language.

2. The voice query extension method of claim 1, wherein the extracting and selecting matches the at least one phoneme sequence with words registered in a first language dictionary, and when a string of the matched words with a value greater than a predetermined threshold value exists as a result of the matching, selects the string of the matched words as the query, and when the string of the matched words with the value greater than the predetermined threshold value does not exist as a result of the matching, matches the at least one phoneme sequence with words registered in a second language dictionary and selects a string of the matched words as the query.

3. The voice query extension method of claim 2, wherein the extracting and selecting matches the at least one phoneme sequence with the words registered in the second language dictionary, and when the string of the matched words with the value greater than the predetermined threshold value exists as a result of the matching, selects the string of the matched words as the query, and when the string of the matched words with the value greater than the predetermined threshold value does not exist as a result of the matching, selects a string of the matched words with a greatest value as the query.

4. The voice query extension method of claim 1, wherein the extracting and selecting matches the at least one phoneme sequence with words registered in a universal language dictionary including the first language and the second language dictionary, and when a string of the matched words with a value greater than a predetermined threshold value exists as a result of the matching, extracts the string of the matched words as the query, and when the string of the matched words with the value greater than the predetermined threshold value does not exist as a result of the matching, selects a string of the matched words with a greatest value as the query.

5. The voice query extension method of claim 1, wherein the converting and generating applies an acoustic model, converts the feature vector into the at least one phoneme sequence, and generates the at least one phoneme sequence.

6. The voice query extension method of claim 1, wherein the matching, extracting, and selecting applies a phone confusion matrix, and matches the at least one phoneme sequence with the words registered in the dictionary.

7. The voice query extension method of claim 1, wherein the matching, extracting, and selecting extracts the string of the matched words with the linguistic meaning using a language model, and selects the string of the matched words as the query.

8. At least one non-transitory computer-readable medium comprising computer readable instructions, that when executed, implements a voice query extension method, comprising:

detecting a voice activity of a user from an input signal and extracting a feature vector from the voice activity using a computer;

converting the feature vector into at least one phoneme sequence and generating the at least one phoneme sequence;

matching the at least one phoneme sequence with words registered in a language dictionary, extracting a string of the matched words with a linguistic meaning, and selecting the string of the matched words as a query;

determining whether the query is in a predetermined first language, and when the query is not in the first language as a result of the determining, converting the query using a phoneme to grapheme rule, and generating a query in the first language; and

searching using the query in the first language.

9. A voice query extension system, comprising:

a signal processing unit to detect a voice activity of a user from an input signal and extracting a feature vector from the voice activity using a computer;

a phonetic decoder to convert the feature vector into at least one phoneme sequence and to generate the at least one phoneme sequence;

a lexical decoder to match the at least one phoneme sequence with words registered in a dictionary, to extract a string of the matched words with a linguistic meaning, and to select the string of the matched words as a query;

a multilingual query extension unit to determine whether the query is in a predetermined first language, and when the query is not in the first language as a result of the determining, to convert the query using a phoneme to grapheme rule, and to generate a query in the first language; and

a search unit to search using the query in the first language.

10. The voice query extension system of claim 9, wherein the lexical decoder matches the at least one phoneme sequence with words registered in a first language dictionary, and when a string of the matched words with a value greater than a predetermined threshold value exists as a result of the matching, selects the string of the matched words as the query, and when the string of the matched words with the value greater than the predetermined threshold value does not exist as a result of the matching, matches the at least one phoneme sequence with words registered in a second language dictionary and selects a string of the matched words as the query.

11. The voice query extension system of claim 10, wherein the lexical decoder matches the at least one phoneme sequence with the words registered in the second language dictionary, and when the string of the matched words with the value greater than the predetermined threshold value exists as a result of the matching, selects the string of the matched words as the query, and when the string of the matched words with the value greater than the predetermined threshold value does not exist as a result of the matching, selects a string of the matched words with a greatest value as the query.

12. The voice query extension system of claim 9, wherein the lexical decoder matches the at least one phoneme sequence with words registered in a universal language dictionary including the first language and the second language dictionary, and when a string of the matched words with a value greater than a predetermined threshold value exists as a result of the matching, selects the string of the matched words as the query, and when the string of the matched words with the value greater than the predetermined threshold value does not exist as a result of the matching, selects a string of the matched words with a greatest value as the query.

13. The voice query extension system of claim 9, wherein the phonetic decoder applies an acoustic model, converts the feature vector into the at least one phoneme sequence, and generates the at least one phoneme sequence.

14. The voice query extension system of claim 9, wherein the lexical decoder applies a phone confusion matrix, and matches the at least one phoneme sequence with the words registered in the dictionary.

15. The voice query extension system of claim 9, wherein the lexical decoder extracts the string of the matched words with the linguistic meaning using a language model, and selects the string of the matched words as the query.