METHOD, A SYSTEM AND A DEVICE FOR CONVERTING SPEECH

• US 20090319267A1
• Filed: 04/27/2006
• Published: 12/24/2009
• Est. Priority Date: 04/27/2006
• Status: Active Grant

• Current Assignee: Dicta-Direct LLC (2S Ventures, LLC)
• Original Assignee: MUSEOKATU 8 A 6
• Inventors: Kurki-Suonio, Risto
• Granted Patent: US 9,123,343 B2
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

An arrangement for converting speech into text comprises a mobile device (202) and a server entity (208) configured to perform the conversion and additional optional processes in co-operation. The user of the mobile device (202) may locally edit the speech signal prior to or between the execution of the actual speech recognition tasks. Task sharing details can be negotiated dynamically based on a number of parameters.

Claims

1-23. -23. (canceled)

24. A mobile device operable in a wireless communications network comprising

a transducer for receiving speech and converting the speech into a representative digital speech signal (616),

a user interface for communicating an edit command relating to the digital speech signal (608),

a processing unit for performing a digital speech signal editing task responsive to the received edit command (602),

at least part of a speech recognition engine for carrying out tasks of the digital speech signal to text conversion (612), and

a transceiver for exchanging information relating to the digital speech signal and speech to text conversion thereof with an external entity functionally connected to said wireless communications network (614).

25. The mobile device according to claim 24, further comprising display means for visualizing at least part of the digital speech signal, whereupon said user interface is configured to communicate an edit command relating to said visualized part.

26. The mobile device according to claim 25, wherein the visualization of the signal comprises at least one element selected from the group consisting of:

a time-domain representation of the signal, a frequency-domain representation of the signal, a parameterization of the signal, a zoom or unzoom operation targeted to the visualized signal, a numeric value determined from a user-defined portion of the signal, a pointer to a user-defined location in the visualized signal, and highlighting of a user-defined sub-area of the visualized signal.

27. The mobile device according to claim 25, wherein at least a portion of the text resulted from the conversion is visualized as aligned in relation to the corresponding visualized portion of the signal.

28. The mobile device according to claim 24, wherein said editing task is selected from the group consisting of:

a deletion of a portion of the signal, an insertion of a speech portion in the signal, re-recording of a portion of the signal, replacement of a portion of the signal, change in the amplitude of the signal, change in the spectral content of the signal, change of the dynamics of the signal, and execution of a noise reduction algorithm.

29. The mobile device according to claim 24, wherein said at least part of the speech recognition engine comprises an element selected from the group consisting of:

preprocessor for dividing the digital speech signal into frames of a predetermined length, cepstral analyser, acoustic classifier, neural network classifier, best path decoder, HMM (Hidden Markov Model) decoder, lexical language model, grammatical language model, user-specific settings, and vocabulary.

30. The mobile device according to claim 24, wherein said processing unit is configured to compress the digital speech signal by encoding.

31. The mobile device according to claim 24, wherein said exchanged information includes an element selected from the group consisting of:

digital form speech, digital encoded speech, device status information, message acknowledgment, control information, edit command, task sharing negotiation data, parameter value related to task sharing, task status, service down notice, load figure, intermediary speech to text conversion result.

32. The mobile device according to claim 24, wherein said information is exchanged by utilizing at least one communication practise selected from the group consisting of:

an SMS (Short Message Service) message, an MMS (Multimedia Message Service) message, an e-mail, a data call, a GPRS (Global Packet Radio Service) connection, and a voice call.

33. The mobile device according to claim 24, configured to share the execution of tasks required for carrying out the speech to text conversion with the external entity.

34. The mobile device according to claim 33, configured to share the execution of tasks so as to optimise a factor according to predetermined criteria, said factor being selected from the group consisting of:

execution time of the speech to text conversion, conversion costs, amount of required data transfer, processing load, and memory load.

35. The mobile device according to claim 33, wherein the exchanged information includes data for allocating or performing the tasks of the speech to text conversion.

36. The mobile device according to claim 33, wherein said exchanged information is selected from the group consisting of:

processing load, memory load, a battery status, a battery capacity, information about tasks running with higher priority, available transmission bandwidth, data transmission rate, external entity usage cost per speech data size or duration, size or duration of the digital speech signal, available encoding/decoding method, conversion status, task status, device unavailability notice, intermediary speech to text conversion result, digital speech, digital encoded speech, speech recognition parameter, and text.

37. The mobile device according to claim 33, configured to utilize intermediary results of the speech to text conversion provided by both the device and the external entity in order to produce the text.

38. The mobile device according to claim 33, configured to transmit intermediary results of the speech to text conversion to another entity, for example said external entity, so as to enable the another entity to perform at least one of the following options:

to combine the intermediary results acquired from the mobile device with locally obtained results to produce the text, subject the intermediary results to additional processing in order to produce the text.

39. The mobile device according to claim 24, configured to transmit text resulting from the speech to text conversion to another entity for further processing selected from the group consisting of:

spell-checking, machine translation, human translation, translation verification, and text to speech synthesis.

40. A server operable in a communications network comprising

a data interface for receiving a digital data signal sent by a mobile device, said digital data signal representing speech or at least part thereof (714),

at least part of a speech recognition engine for carrying out tasks of a digital data signal to text conversion (712),

a controlling unit for exchanging control information with the mobile device and determining, based on the control information, the tasks to be performed on the received digital data signal by said at least part of the speech recognition engine (702), and

a data output interface for communicating at least part of the output of the performed tasks to an external entity (718).

41. A system for converting speech into text comprising a mobile device (202) operable in a wireless communications network and a server (208) functionally connected to said wireless communications network, wherein

said mobile device (202) is configured to receive speech and convert the speech into a representative digital speech signal, to exchange information relating to the digital speech signal and speech to text conversion thereof with the server (208), and to execute part of the tasks required for carrying out a digital speech signal to text conversion, and

said server (208) is configured to receive information relating to the digital speech signal and speech to text conversion thereof, and to execute, based on the exchanged information, the remaining part of the tasks required for carrying out a digital speech signal to text conversion.

42. A method for converting speech into text having the steps of

receiving, in a mobile device operable in a wireless network, a speech source and converting the speech source into a representative digital speech signal (304),

receiving an edit command relating to the digital speech signal (308),

processing the digital speech signal in accordance with the edit command (310),

exchanging information relating to the digital speech signal and speech to text conversion thereof (312), and

executing on the basis of the exchanged information at least part of the tasks required for carrying out a speech to text conversion of the digital speech signal (314).

43. The method according to claim 42, further comprising a step of visualizing at least part of the digital speech signal on a display of the mobile device, whereupon the received edit command optionally relates to said visualized part.

44. A computer executable program comprising code means adapted, when run on a computer, to carry out the method steps as defined by claim 42.

45. A carrier medium comprising the computer executable program of claim 44.

46. The carrier medium according to claim 45, selected from the group consisting of:

a memory card, a floppy disc, a CD-ROM, and a hard drive.