Method and apparatus for searching multimedia data using speech recognition in mobile device

• US 20070208561A1
• Filed: 02/09/2007
• Published: 09/06/2007
• Est. Priority Date: 03/02/2006
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Nam Hoon, Choi, In Jeong, Jeong, Sang Bae, Han, Ick Sano
• Granted Patent: US 8,200,490 B2
• Time in Patent Office: Days
• US Class Current: 704/231
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

A method of searching music using speech recognition in a mobile device, the method including: recognizing a speech signal uttered by a user as a phoneme sequence; and searching music information by performing partial symbol matching between the recognized phoneme sequence and a standard pronunciation sequence.

Claims

1. A method of searching music using speech recognition, the method comprising:

recognizing as a phoneme sequence a speech signal uttered by a user; and

searching music information by performing partial symbol matching between the recognized phoneme sequence and a standard pronunciation sequence.

2. The method of claim 1, wherein the searching music information comprises:

performing the partial symbol matching between the recognized phoneme sequence and the standard pronunciation sequence;

calculating a match score according to a result of the partial symbol matching; and

displaying a music information search result according to the match score.

3. The method of claim 1, wherein the recognizing as a phoneme sequence a speech signal uttered by a user comprises:

extracting a feature of the speech signal uttered by the user; and

decoding a phoneme according to the extracted feature of the speech signal.

4. The method of claim 2, wherein the match score is calculated by a phoneme confusion matrix.

5. The method of claim 2, wherein, in the displaying a music information search result according to the match score, only a music information search result having the match score greater than a predetermined reference value is displayed.

6. The method of claim 1, further comprising extracting a recognition target vocabulary from a predetermined music file and generating the music information with respect to the extracted recognition target vocabulary.

7. The method of claim 6, further comprising:

generating a pronunciation dictionary with the recognition target vocabulary; and

sorting the generated pronunciation dictionary.

8. A computer-readable recording medium in which a program for executing a method of searching music using speech recognition is recorded, the method comprising:

recognizing as a phoneme sequence a speech signal uttered by a user; and

searching music information by performing partial symbol matching between the recognized phoneme sequence and a standard pronunciation sequence.

9. A music search apparatus comprising:

a music database storing a pronunciation dictionary with respect to music and music information;

a phoneme decoding unit decoding a speech signal into a candidate phoneme sequence;

a matching unit matching the candidate phoneme sequence with a reference phoneme pattern in the pronunciation dictionary with respect to the music information;

a calculation unit calculating a match score according to a result of the matching; and

a display unit displaying a music information search result according to the calculated match score.

10. The apparatus of claim 9, wherein the matching unit matches the candidate phoneme sequence with the reference phoneme pattern in the pronunciation dictionary, with respect to the music information, using a phoneme confusion matrix and language boundary information.

11. The apparatus of claim 9, wherein the matching unit converts a pronunciation sequence of a part of the candidate phoneme sequence exhibiting an effect of palatalization into an original pronunciation sequence in an isolated speech form and matches the converted pronunciation sequence with the reference phoneme pattern of the pronunciation dictionary.

12. The apparatus of claim 9, wherein the display unit displays only music information search results having the match score greater than a predetermined reference value.

13. The apparatus of claim 9, wherein the display unit arranges and displays music information search results according to a predetermined criteria when the match score of the music information search result is the same as another match score of another search.

14. The apparatus of claim 9, further comprising a feature extraction unit extracting a speech feature from the speech signal before the phoneme decoding unit decodes the speech signal into the candidate phoneme sequence.

15. The apparatus of claim 9, further comprising a music information generation unit extracting a recognition target vocabulary from a predetermined music file, and generating the music information with respect to the extracted recognition target vocabulary.

16. A music search apparatus comprising:

a feature extraction unit extracting a feature vector sequence of a speech signal of an input speech query;

a phoneme decoding unit decoding the extracted feature vector sequence into at least one candidate phoneme sequences;

a matching unit partially matching a candidate phoneme sequence with a reference pattern included in a stored lexicon by matching the candidate phoneme sequence with the reference pattern using a phoneme confusion matrix and linguistic constraints and, after the partial matching, matching a converted pronunciation sequence with a reference phoneme pattern of the lexicon so as to overcome an inconsistency due to a difference in pronunciation caused by palatalization; and

a calculation unit calculating a match score according to the match score using a probability value of the phoneme confusion matrix and considering probabilities of insertion and deletion of the phoneme.

17. The apparatus of claim 16, further comprising a music database storing music, music information, and the lexicon, the lexicon being for the music information and corresponding to a reference pronunciation pattern for comparing a speech query with a recognized phoneme sequence.

18. The apparatus of claim 16, wherein the feature extraction unit extracts a feature vector sequence of a speech signal of an input speech query by reducing background noise of the speech signal of the speech query, extracting a speech interval from the speech signal, and extracting a feature vector sequence usable in speech recognition from the detected speech interval

19. The apparatus of claim 16, wherein the phoneme decoding unit decodes the extracted feature vector sequence into the at least one candidate phoneme sequence using a phoneme or a tri-phoneme acoustic model and applies connectivity between contexts when using the tri-phoneme acoustic model.

20. The apparatus of claim 16, wherein the phoneme decoding unit applies a phoneme-level grammar when converting the extracted feature vector sequence into the at least one candidate phoneme sequence.

21. The apparatus of claim 16, wherein the matching unit obtains the converted pronunciation sequence used to overcome an inconsistency due to a difference in pronunciation caused by palatalization by converting a pronunciation sequence of a part of the candidate phoneme sequence exhibiting an effect of palatalization into an original pronunciation sequence in an isolated speech form.

22. The apparatus of claim 16, wherein the conversion of the pronunciation sequence into the original pronunciation sequence enables regularization by back-tracking from a pronunciation rule.

23. The apparatus of claim 16, wherein the matching a converted pronunciation sequence with a reference phoneme pattern of the lexicon is achieved by Viterbi alignment with respect to a matched phoneme segment of a candidate recognition list obtained from the partial matching.