Multi-layered speech recognition apparatus and method

• US 20060080105A1
• Filed: 05/04/2005
• Published: 04/13/2006
• Est. Priority Date: 10/08/2004
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Yongbeom, Lee, Jaewon, Kim, Jeongsu, Hwang, Kwangil, Cho, Jeongmi
• Granted Patent: US 8,370,159 B2
• Time in Patent Office: Days
• US Class Current: 704/270.100
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

A multi-layered speech recognition apparatus and method, the apparatus includes a client checking whether the client recognizes the speech using a characteristic of speech to be recognized and recognizing the speech or transmitting the characteristic of the speech according to a checked result; and first through N-th servers, wherein the first server checks whether the first server recognizes the speech using the characteristic of the speech transmitted from the client, and recognizes the speech or transmits the characteristic according to a checked result, and wherein an n-th (2≦n≦N) server checks whether the n-th server recognizes the speech using the characteristic of the speech transmitted from an (n−1)-th server, and recognizes the speech or transmits the characteristic according to a checked result.

Claims

1. A multi-layered speech recognition apparatus comprising:

a client extracting a characteristic of speech to be recognized, checking whether the client recognizes the speech using the extracted characteristic of the speech and recognizing the speech or transmitting the characteristic of the speech, according to a first checked result; and

first through N-th (where N is a positive integer equal to or greater than

     1) servers,

wherein the first server receives the characteristic of the speech transmitted from the client, checks whether the first server recognizes the speech, using the received characteristic of the speech, and recognizes the speech or transmits the characteristic of the speech according to a second checked result, and

wherein an n-th (2≦

n≦

N) server receives the characteristic of the speech transmitted from an (n−

1)-th server, checks whether the n-th server recognizes the speech, using the received characteristic of the speech, and recognizes the speech or transmits the characteristic of the speech according to a third checked result.

2. The apparatus of claim 1, wherein the client comprises:

a speech characteristic extraction unit extracting the characteristic of the speech to be recognized;

a client speech recognition unit recognizing the speech from the extracted characteristic of the speech; and

a client adjustment unit checking whether the client recognizes the speech using the extracted characteristic of the speech, and transmitting the characteristic of the speech to the first server or outputting the characteristic of the speech to the client speech recognition unit according to the first checked result.

3. The apparatus of claim 2, wherein the client adjustment unit comprises:

a client topic-checking portion detecting a topic that is most similar to the extracted characteristic of the speech and calculating a score of the detected most similar topic;

a client comparison portion comparing the detected score with a client threshold value; and

a client output-controlling portion outputting the extracted characteristic of the speech to the client speech recognition unit or transmitting the extracted characteristic of the speech to the first server according to the compared result.

4. The apparatus of claim 3, wherein the client further comprises:

a client compression transmission unit compressing the extracted characteristic of the speech according to the compared result and according to a transmission format signal, and transmitting the compressed characteristic of the speech to the first server via a loss channel, and

the first server comprises a client restoration receiving unit receiving the compressed characteristic of the speech transmitted from the client compression transmission unit and restoring the received compressed characteristic of the speech, and

the first server checks whether the speech is recognized using the restored characteristic of the speech, and recognizes the speech or transmits the restored characteristic of the speech according to the second checked result.

5. The apparatus of claim 3, wherein the client speech recognition unit comprises:

first through P-th (where P is the number of topics checked by the client topic-checking portion) speech recognition decoders; and

a client decoder selection portion being activated according to the compared result and selectively outputting the extracted characteristic of the speech to a speech recognition decoder corresponding to the detected most similar topic input from the client topic-checking portion from the first through P-th speech recognition decoders, and

a p-th (1≦

p≦

P) speech recognition decoder recognizes the speech from the extracted characteristic of the speech.

6. The apparatus of claim 2, wherein the client further comprises:

a speech input unit inputting the speech from an outside and detecting a valid speech section from the input speech, and

the client extracts the characteristic of the speech from the valid speech section.

7. The apparatus of claim 1, wherein a q-th (1≦

q≦

N) server comprises;

a server speech recognition unit recognizing the speech from the received characteristic of the speech; and

a server adjustment unit checking whether the q-th server recognizes the speech using the received characteristic of the speech and transmitting the received characteristic of the speech to a different server or outputting the received characteristic of the speech to the server speech recognition unit according to a fourth checked result.

8. The apparatus of claim 7, wherein the server adjustment unit comprises:

a server topic-checking portion detecting a topic that is most similar to the received characteristic of the speech and calculating a score of the detected most similar topic;

a server comparison portion comparing the score detected by the server topic-checking portion with a server threshold value; and

a server output-controlling portion outputting the received characteristic of the speech to the server speech recognition unit or transmitting the received characteristic of the speech to the different server in response to a result compared by the server comparison portion.

9. The apparatus of claim 8, wherein the server speech recognition unit comprises:

first through R-th (where R is the number of topics checked by the server topic-checking portion) speech recognition decoders; and

a server decoder selection portion being activated in response to the result compared by the server comparison portion and selectively outputting the received characteristic of the speech to a speech recognition decoder corresponding to the detected most similar topic input from the server topic-checking portion from the first through R-th speech recognition decoders, and

an r-th (1≦

r≦

R) speech recognition decoder recognizes the speech from the received characteristic of the speech.

10. The apparatus of claim 3, wherein the client topic-checking portion comprises:

a keyword storage portion storing a plurality of keywords;

a keyword search portion searching words having a characteristic similar to the characteristic of the speech from the plurality of keywords and outputting the searched words in a list format; and

a score calculation portion calculating scores according to each topic from the list, outputting a largest score as a score of the most similar topic and outputting a topic having the largest score as the most similar topic, and

each of the keywords has unique speech characteristics and scores according to each topic.

11. The apparatus of claim 8, wherein the server topic-checking portion comprises:

a keyword storage portion storing a plurality of keywords;

a keyword search portion searching keywords having a characteristic similar to the characteristic of the speech from the plurality of keywords and outputting the searched words in a list format; and

a score calculation portion calculating scores according to each topic from the list, outputting a largest score as a score of the most similar topic and outputting a topic having the largest score as the most similar topic, and

each of the keywords has unique speech characteristics and scores according to each topic.

12. The apparatus of claim 8, wherein the q-th server comprises:

a server compression transmission unit compressing the received characteristic of the speech in response to the result compared by the server comparison portion and in response to a transmission format signal and transmitting the compressed characteristic of the speech to the different server via a loss channel, and

the different server comprises a server restoration receiving unit receiving the compressed characteristic of the speech transmitted from the server compression transmission unit and restoring the received compressed characteristic of the speech, and

the different server checks whether the different server recognizes the speech using the restored characteristic of the speech and recognizes the speech or transmitting the restored characteristic of the speech according to the fourth checked result.

24. The apparatus of claim 2, wherein the client further comprises a speech input unit through which speech is input from an outside, detects a valid speech section from the input speech, and outputs the detected valid speech section to the speech characteristic extraction unit.

25. The apparatus of claim 6, wherein the speech characteristic extraction unit extracts the characteristic of the speech in a vector format from the valid speech section from the input speech.

26. The apparatus of claim 2, wherein the client adjustment unit determines whether the characteristic extracted by the speech characteristic extraction unit is to be transmitted via a loss channel or a lossless channel.

27. The apparatus of claim 12, wherein the different server further comprises a server adjustment unit, a server speech recognition unit, a server application unit and a server compression transmission unit.

28. The apparatus of claim 3, wherein the first server comprises a server adjustment unit directly receiving the characteristic of the speech transmitted from the client and the first server checks whether the speech is recognized using restored characteristics of the speech according to the first checked result.

29. The apparatus of claim 1, wherein the n-th server comprises a server adjustment unit directly receiving the characteristic of the speech transmitted from the first server and the n-th server checks whether the speech is recognized using restored characteristics of the speech according to the third checked result.

13. A multi-layered speech recognition method performed in a multi-layered speech recognition apparatus having a client and first through N-th (where N is a positive integer equal to or greater than 1) servers, the method comprising:

extracting a characteristic of speech to be recognized, checking whether the client recognizes the speech using the extracted characteristic of the speech, and recognizing the speech or transmitting the characteristic of the speech according to a first checked result; and

receiving the characteristic of the speech transmitted from the client,

checking whether the first server recognizes the speech using the received characteristic of the speech, and recognizing the speech or transmitting the characteristic of the speech according to the first checked result, and

receiving the characteristic of the speech transmitted from an (n−

1)-th (2≦

n≦

N) server, checking whether an n-th server recognizes the speech using the received characteristic of the speech, and recognizing the speech or transmitting the characteristic of the speech according to a second checked result,

wherein the extracting of the characteristic of the speech to be recognized is performed by the client, the receiving of the characteristic of the speech transmitted from the client is performed by the first server, and the receiving of the characteristic of the speech transmitted from the (n−

1)-th server is performed by the n-th server.

14. The method of claim 13, wherein the receiving of the characteristic of the speech transmitted from the (n−

1)-th server includes receiving one characteristic of the speech after another by several servers until the speech is recognized.

15. The method of claim 14, wherein the extracting of the characteristic of the speech to be recognized comprises:

extracting the characteristic of the speech to be recognized;

determining whether the client recognizes the speech using the extracted characteristic of the speech;

if determined that the client recognizes the speech, recognizing the speech using the extracted characteristic of the speech; and

if determined that the client cannot recognize the speech, transmitting the extracted characteristic of the speech to the first server.

16. The method of claim 15, wherein the determining whether the client recognizes the speech comprises:

detecting a topic that is most similar to the extracted characteristic of the speech and calculating scores of the detected most similar topic; and

determining whether the calculated scores are larger than a client threshold value, and when the client recognizes the speech, the calculated scores are larger than the client threshold value.

17. The method of claim 15, wherein the transmitting of the extracted characteristic of the speech to the first server comprises:

determining whether the extracted characteristic of the speech is to be transmitted via a loss channel or a lossless channel;

if determined that the extracted characteristic of the speech is to be transmitted via the lossless channel, transmitting the extracted characteristic of the speech to the first server via the lossless channel; and

if determined that the extracted characteristic of the speech is to be transmitted via the loss channel, compressing the extracted characteristic of the speech and transmitting the compressed characteristic of the speech to the first server via the loss channel,

the first server includes receiving the compressed characteristic of the speech transmitted via the loss channel and restoring the received compressed characteristic of the speech, and

the first server performs the receiving of the characteristic of the speech transmitted from the client using the restored characteristic of the speech.

18. The method of claim 15, wherein the extracting of the characteristic of the speech to be recognized further comprises inputting the speech from an outside and detecting a valid speech section from the input speech, and the client extracts the characteristic of the speech from the valid speech section.

19. The method of claim 13, wherein the receiving of the characteristic of the speech transmitted from the client or the receiving of the characteristic of the speech transmitted from a (n−

1)-th server comprises;

determining whether a q-th (1≦

q≦

N) server recognizes the speech using the received characteristic of the speech;

if determined that the q-th server recognizes the speech, recognizing the speech using the received characteristic of the speech; and

if determined that the q-th server cannot recognize the speech, transmitting the received characteristic of the speech to a different server.

20. The method of claim 19, wherein the determining whether the q-th server recognizes the speech comprises:

detecting a topic that is most similar to the received characteristic of the speech and calculating scores of the detected most similar topic; and

determining whether the calculated scores are larger than a server threshold value, and

when the q-th server recognizes the speech, the calculated scores are larger than the server threshold value.

21. The method of claim 16, wherein the calculating of the scores of the detected most similar topic comprises:

searching words having a characteristic similar to the characteristic of the speech from a plurality of keywords that have been previously stored; and

calculating the scores of the detected most similar topic according to each topic from the searched keywords and determining a largest score as the score of the most similar topic,

each of the words has unique speech characteristics and scores according to each topic.

22. The method of claim 20, wherein the calculating of the scores of the detected most similar topic comprises:

searching words having a characteristic similar to the characteristic of the speech from a plurality of keywords that have been previously stored; and

calculating the scores according to each topic from the searched words and determining a largest score as the score of the most similar topic,

each of the words has unique speech characteristics and scores according to each topic.

23. The method of claim 20, wherein the transmitting of the received characteristic of the speech to the different server comprises:

determining whether the received characteristic of the speech is to be transmitted via a loss channel or a lossless channel;

if determined that the received characteristic of the speech is to be transmitted via the lossless channel, transmitting the extracted characteristic of the speech to the different server via the lossless channel; and

if determined that the received characteristic of the speech is to be transmitted via the loss channel, compressing the received characteristic of the speech and transmitting the compressed characteristic of the speech to the different server via the loss channel,

wherein the different server includes receiving the compressed characteristic of the speech transmitted via the loss channel and restoring the received compressed characteristic of the speech, and

wherein the different server checks whether the different server recognizes the speech using the restored characteristic of the speech and recognizes the speech or transmits the restored characteristic of the speech according to the checked result.

30. A speech recognition apparatus comprising:

a client and at least one server connected to each other via a network, wherein the client recognizes the speech continuously and sequentially in a multi-layered manner, optimally dispersing a work load of the speech recognition in the client and in the at least one server, improving speech recognition.

31. A speech recognition apparatus comprising:

a client capable of interacting with a server connected to each other via a network, wherein the client recognizes the speech continuously and sequentially in a multi-layered manner, optimally dispersing a work load of the speech recognition in the client and in the server, improving speech recognition.