Multi-layered speech recognition apparatus and method

• US 8,370,159 B2
• Filed: 05/04/2005
• Issued: 02/05/2013
• Est. Priority Date: 10/08/2004
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Jaewon, Cho, Jeongmi, Hwang, Kwangil, Lee, Yongbeom, Kim, Jeongsu
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US11/120,983
• Publication Number:
  US 20060080105A1
• Time in Patent Office: 2,834 Days
• Field of Search: 704/219, 704/275, 704/246, 704/231, 704/270, 704/235, 704/277, 704/270.1, 704/257, 704/2, 700/94, 370/428, 370/204, 455/444, 455/422.1, 382/239, 382/166, 375/340, 375/295
• US Class Current: 704/270.1
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

A multi-layered speech recognition apparatus and method, the apparatus includes a client checking whether the client recognizes the speech using a characteristic of speech to be recognized and recognizing the speech or transmitting the characteristic of the speech according to a checked result; and first through N-th servers, wherein the first server checks whether the first server recognizes the speech using the characteristic of the speech transmitted from the client, and recognizes the speech or transmits the characteristic according to a checked result, and wherein an n-th (2≦n≦N) server checks whether the n-th server recognizes the speech using the characteristic of the speech transmitted from an (n−1)-th server, and recognizes the speech or transmits the characteristic according to a checked result.

Claims

1. A multi-layered speech recognition apparatus comprising:

a client checking whether the client recognizes speech and recognizing the speech when the client itself can recognize the speech and transmitting the speech when the client itself cannot recognize the speech, according to a first checked result; and

first through N-th (where N is a positive integer equal to or greater than

     1) servers,

wherein the first server receives the speech transmitted from the client, checks whether the first server recognizes the speech, and recognizes the speech when the first server itself can recognize the speech or transmits the speech when the first server itself cannot recognize the speech according to a second checked result,

wherein an n-th (2≦

n≦

N) server receives the speech transmitted from an (n−

1)-th server, checks whether the n-th server recognizes the speech, and recognizes the speech when the n-th server itself can recognize the speech or transmits the speech when the n-th server itself cannot recognize the speech according to a third checked result, and

wherein the first server has a larger capacity of resource than the client and the n-th server has a larger capacity of resource than the (n−

1)-th server.

2. The apparatus of claim 1, wherein the client comprises:

a speech characteristic extraction unit extracting a characteristic of the speech to be recognized;

recognized;

a client speech recognition unit recognizing the speech from the extracted characteristic of the speech; and

a client adjustment unit checking whether the client recognizes the speech using the extracted characteristic of the speech, and transmitting the characteristic of the speech to the first server or outputting the characteristic of the speech to the client speech recognition unit according to the first checked result.

3. The apparatus of claim 2, wherein the client adjustment unit comprises:

a client topic-checking portion detecting a topic that is most similar to the extracted characteristic of the speech and calculating a score of the detected most similar topic;

a client comparison portion comparing the detected score with a client threshold value; and

a client output-controlling portion outputting the extracted characteristic of the speech to the client speech recognition unit or transmitting the extracted characteristic of the speech to the first server according to the compared result.

4. The apparatus of claim 3, wherein the client further comprises:

a client compression transmission unit compressing the extracted characteristic of the speech according to the compared result and according to a transmission format signal, and transmitting the compressed characteristic of the speech to the first server via a loss channel, and

the first server comprises a client restoration receiving unit receiving the compressed characteristic of the speech transmitted from the client compression transmission unit and restoring the received compressed characteristic of the speech, and

the first server checks whether the speech is recognized using the restored characteristic of the speech, and recognizes the speech or transmits the restored characteristic of the speech according to the second checked result.

5. The apparatus of claim 3, wherein the client speech recognition unit comprises:

first through P-th (where P is the number of topics checked by the client topic-checking portion) speech recognition decoders; and

a client decoder selection portion being activated according to the compared result and selectively outputting the extracted characteristic of the speech to a speech recognition decoder corresponding to the detected most similar topic input from the client topic-checking portion from the first through P-th speech recognition decoders, and

a p-th (1≦

p≦

P) speech recognition decoder recognizes the speech from the extracted characteristic of the speech.

6. The apparatus of claim 2, wherein the client further comprises:

a speech input unit inputting the speech from an outside and detecting a valid speech section from the input speech, and

the client extracts the characteristic of the speech from the valid speech section.

7. The apparatus of claim 1, wherein a q-th (1≦

q≦

N) server comprises;

a server speech recognition unit recognizing the speech from the received speech; and

a server adjustment unit checking whether the q-th server recognizes the speech using the received speech and transmitting the received speech to a different server or outputting the received speech to the server speech recognition unit according to a fourth checked result.

8. The apparatus of claim 7, wherein the server adjustment unit comprises:

a server topic-checking portion detecting a topic that is most similar to the received speech and calculating a score of the detected most similar topic;

a server comparison portion comparing the score detected by the server topic-checking portion with a server threshold value; and

a server output-controlling portion outputting the received speech to the server speech recognition unit or transmitting the received speech to the different server in response to a result compared by the server comparison portion.

9. The apparatus of claim 8, wherein the server speech recognition unit comprises:

first through R-th (where R is the number of topics checked by the server topic-checking portion) speech recognition decoders; and

a server decoder selection portion being activated in response to the result compared by the server comparison portion and selectively outputting the received speech to a speech recognition decoder corresponding to the detected most similar topic input from the server topic-checking portion from the first through R-th speech recognition decoders, and

an r-th (1≦

r≦

R) speech recognition decoder recognizes the speech from the received speech.

10. The apparatus of claim 3, wherein the client topic-checking portion comprises:

a keyword storage portion storing a plurality of keywords;

a keyword search portion searching words having a characteristic similar to the characteristic of the speech from the plurality of keywords and outputting the searched words in a list format; and

a score calculation portion calculating scores according to each topic from the list, outputting a largest score as a score of the most similar topic and outputting a topic having the largest score as the most similar topic, and

each of the keywords has unique speech characteristics and scores according to each topic.

11. The apparatus of claim 8, wherein the server topic-checking portion comprises:

a keyword storage portion storing a plurality of keywords;

a keyword search portion searching keywords having a characteristic similar to the speech from the plurality of keywords and outputting the searched words in a list format; and

a score calculation portion calculating scores according to each topic from the list, outputting a largest score as a score of the most similar topic and outputting a topic having the largest score as the most similar topic, and

each of the keywords has unique speech characteristics and scores according to each topic.

12. The apparatus of claim 8, wherein the q-th server comprises:

a server compression transmission unit compressing the received speech in response to the result compared by the server comparison portion and in response to a transmission format signal and transmitting the compressed characteristic of the speech to the different server via a loss channel, and

the different server comprises a server restoration receiving unit receiving the compressed characteristic of the speech transmitted from the server compression transmission unit and restoring the received compressed characteristic of the speech, and

the different server checks whether the different server recognizes the speech using the restored characteristic of the speech and recognizes the speech or transmitting the restored characteristic of the speech according to the fourth checked result.

13. The apparatus of claim 2, wherein the client further comprises a speech input unit through which speech is input from an outside, detects a valid speech section from the input speech, and outputs the detected valid speech section to the speech characteristic extraction unit.

14. The apparatus of claim 6, wherein the speech characteristic extraction unit extracts the characteristic of the speech in a vector format from the valid speech section from the input speech.

15. The apparatus of claim 2, wherein the client adjustment unit determines whether the characteristic extracted by the speech characteristic extraction unit is to be transmitted via a loss channel or a lossless channel.

16. The apparatus of claim 12, wherein the different server further comprises a server adjustment unit, a server speech recognition unit, a server application unit and a server compression transmission unit.

17. The apparatus of claim 3, wherein the first server comprises a server adjustment unit directly receiving the characteristic of the speech transmitted from the client and the first server checks whether the speech is recognized using restored characteristics of the speech according to the first checked result.

18. The apparatus of claim 1, wherein the n-th server comprises a server adjustment unit directly receiving the speech transmitted from the first server and the n-th server checks whether the speech is recognized according to the third checked result.

19. The apparatus of claim 1, wherein the first through N-th (where N is a positive integer equal to or greater than 1) servers comprise ubiquitous robot companions.

20. A multi-layered speech recognition method performed in a multi-layered speech recognition apparatus having a client and first through N-th (where N is a positive integer equal to or greater than 1) servers, the method comprising:

checking whether the client recognizes speech and recognizing the speech when the client itself can recognize the speech and transmitting the speech when the client itself cannot recognize the speech according to a first checked result; and

receiving the speech transmitted from the client,

checking whether the first server recognizes the speech and recognizing the speech when the first server itself can recognize the speech or transmitting the speech when the first server itself cannot recognize the speech according to the first checked result, and

receiving the speech transmitted from an (n−

1)-th (2≦

n≦

N) server, checking whether an n-th server recognizes the speech and recognizing the speech when the n-th server itself can recognize the speech or transmitting the speech when the n-th server itself cannot recognize the speech according to a second checked result,

wherein the receiving of the speech transmitted from the client is performed by the first server, the receiving of the speech transmitted from the (n−

1)-th server is performed by the n-th server, and

wherein the first server has a larger capacity of resource than the client and the n-th server has a larger capacity of resource than the (n−

1)-th server.

21. The method of claim 20, wherein the receiving of the speech transmitted from the (n−

1)-th server includes receiving one characteristic of the speech after another by several servers until the speech is recognized.

22. The method of claim 21, wherein the extracting of the characteristic of the speech to be recognized comprises:

extracting the characteristic of the speech to be recognized;

determining whether the client recognizes the speech using the extracted characteristic of the speech;

if determined that the client recognizes the speech, recognizing the speech using the extracted characteristic of the speech; and

if determined that the client cannot recognize the speech, transmitting the extracted characteristic of the speech to the first server.

23. The method of claim 22, wherein the determining whether the client recognizes the speech comprises:

detecting a topic that is most similar to the extracted characteristic of the speech and calculating scores of the detected most similar topic; and

determining whether the calculated scores are larger than a client threshold value, and when the client recognizes the speech, the calculated scores are larger than the client threshold value.

24. The method of claim 22, wherein the transmitting of the extracted characteristic of the speech to the first server comprises:

determining whether the extracted characteristic of the speech is to be transmitted via a loss channel or a lossless channel;

if determined that the extracted characteristic of the speech is to be transmitted via the lossless channel, transmitting the extracted characteristic of the speech to the first server via the lossless channel; and

if determined that the extracted characteristic of the speech is to be transmitted via the loss channel, compressing the extracted characteristic of the speech and transmitting the compressed characteristic of the speech to the first server via the loss channel,

the first server includes receiving the compressed characteristic of the speech transmitted via the loss channel and restoring the received compressed characteristic of the speech, and

the first server performs the receiving of the characteristic of the speech transmitted from the client using the restored characteristic of the speech.

25. The method of claim 22, wherein the extracting of the characteristic of the speech to be recognized further comprises inputting the speech from an outside and detecting a valid speech section from the input speech, and the client extracts the characteristic of the speech from the valid speech section.

26. The method of claim 20, wherein the receiving of the speech transmitted from the client or the receiving of the speech transmitted from a (n−

1)-th server comprises;

determining whether a q-th (1≦

q≦

N) server recognizes the speech;

if determined that the q-th server recognizes the speech, recognizing the speech; and

if determined that the q-th server cannot recognize the speech, transmitting the speech to a different server.

27. The method of claim 26, wherein the determining whether the q-th server recognizes the speech comprises:

detecting a topic that is most similar to a received characteristic of the speech and calculating scores of the detected most similar topic; and

determining whether the calculated scores are larger than a server threshold value, and

when the q-th server recognizes the speech, the calculated scores are larger than the server threshold value.

28. The method of claim 23, wherein the calculating of the scores of the detected most similar topic comprises:

searching words having a characteristic similar to the characteristic of the speech from a plurality of keywords that have been previously stored; and

calculating the scores of the detected most similar topic according to each topic from the searched keywords and determining a largest score as the score of the most similar topic,

each of the words has unique speech characteristics and scores according to each topic.

29. The method of claim 27, wherein the calculating of the scores of the detected most similar topic comprises:

searching words having a characteristic similar to the characteristic of the speech from a plurality of keywords that have been previously stored; and

calculating the scores according to each topic from the searched words and determining a largest score as the score of the most similar topic,

each of the words has unique speech characteristics and scores according to each topic.

30. The method of claim 27, wherein the transmitting of the received characteristic of the speech to the different server comprises:

determining whether the received characteristic of the speech is to be transmitted via a loss channel or a lossless channel;

if determined that the received characteristic of the speech is to be transmitted via the lossless channel, transmitting the extracted characteristic of the speech to the different server via the lossless channel; and

if determined that the received characteristic of the speech is to be transmitted via the loss channel, compressing the received characteristic of the speech and transmitting the compressed characteristic of the speech to the different server via the loss channel,

wherein the different server includes receiving the compressed characteristic of the speech transmitted via the loss channel and restoring the received compressed characteristic of the speech, and

wherein the different server checks whether the different server recognizes the speech using the restored characteristic of the speech and recognizes the speech or transmits the restored characteristic of the speech according to the checked result.