PROVIDING TEXT INPUT USING SPEECH DATA AND NON-SPEECH DATA

• US 20090216531A1
• Filed: 02/22/2008
• Published: 08/27/2009
• Est. Priority Date: 02/22/2008
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Yanagihara, Kazuhisa
• Granted Patent: US 8,065,143 B2
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

Systems, methods, and computer readable media providing a speech input interface. The interface can receive speech input and non-speech input from a user through a user interface. The speech input can be converted to text data and the text data can be combined with the non-speech input for presentation to a user.

Claims

1. A system comprising:

a user interface operable to receive speech input and non-speech input from a user;

a communications interface operable to communicate the speech input and non-speech input to a speech to text composition module, the speech to text composition module being operable to convert the speech input to text data and to supplement the text data with the non-speech input to produce combined text data; and

a presentation module operable to receive the combined text data and to present the combined text data to the user.

2. The system of claim 1, wherein the speech input and non-speech input include sequence indicators associated with the respective input.

3. The system of claim 1, wherein the speech to text composition module is located remotely from the communications interface.

4. The system of claim 1, wherein the presentation module comprises a text editor.

5. The system of claim 1, further comprising a text editor operable to facilitate keyboard editing of the combine text data subsequent to receipt of the speech input.

6. The system of claim 1, wherein the user interface, communications interface and presentation module reside on a mobile device, and the communications interface comprises a high-speed wireless connection to the speech to text composition module.

7. The system of claim 1, wherein the communications interface is operable to use a real-time streaming protocol to communicate the speech input and the non-speech input to the speech to text composition module.

8. The system of claim 1, wherein the non-speech input includes typeface data, symbols or punctuation data modifying the speech input into the system.

9. The system of claim 1, wherein the communications interface is operable to multiplex the speech input and the non-speech input into a single communication stream to the speech to text composition module.

10. A computer-implemented method comprising:

receiving speech input and non-speech input from a user, the speech input and non-speech input comprising respective sequence indicators;

providing the speech input and non-speech input to a speech to text composition server;

receiving text data from the speech to text composition server, the text data comprising a textual representation of the provided speech input combined with the provided non-speech input; and

presenting the text data to the user.

11. The computer-implemented method of claim 10, further comprising receiving non-speech input using a touchscreen interface.

12. The computer-implemented method of claim 11, wherein the non-speech input comprises punctuation, symbols or typeface data entered by the user using the touchscreen interface.

13. The computer-implemented method of claim 12, further comprising receiving speech input using a microphone.

14. The computer-implemented method of claim 10, further comprising:

receiving edit input from the user; and

modifying the text data based upon the edit input.

15. The computer-implemented method of claim 10, further comprising providing the speech input and non-speech input using a real-time streaming protocol, and wherein the sequence indicators comprise time-stamps.

16. One or more computer readable media having software program code operable to provide an editing interface, operable to perform steps comprising:

receiving speech input and non-speech input from a user through a user interface;

multiplexing the speech input with the non-speech input to produce a data stream;

transmitting the data stream to a speech recognition module;

receiving text data from the speech recognition module, the text data comprising a textual representation of the data stream; and

presenting the text data to the user.

17. The computer readable media of claim 16, further comprising:

receiving edit input from the user through the user interface; and

editing the text data based upon the edit input.

18. A system comprising:

an interface operable to receive speech data and non-speech data from a mobile device, the speech data and non-speech data including sequence information;

a speech recognition module operable to analyze the speech data to derive text data, the text data comprising sequence information associated with each of a plurality of words associated with the speech data; and

a text composition module operable to receive the text data and to combine the text data with the non-speech data based upon the sequence information associated with the non-speech data and the text data, the text composition module being operable to produce combined text data derived from the text data and the non-speech data;

wherein the interface is operable to transmit the combined text data to the mobile device for presentation to a user of the mobile device.

19. A text editor comprising:

a non-speech editing environment displayed during a non-speech editing mode, the non-speech editing environment being operable to receive keyboard data from a user and to present text data related to the keyboard input to the user, the non-speech editing environment comprising a first escape sequence to enter a speech editing mode; and

a speech input environment displayed during the speech editing mode, the speech input environment operable to receive speech input and non-speech input from the user, and to present text data derived from the speech input and non-speech input to the user, the speech input environment comprising a second escape sequence to return to the non-speech editing mode.

20. The text editor of claim 19, wherein the text editor is associated with an application comprising any of an electronic mail application, a web browsing application, a word processing application, a data entry application, a text messaging application, or an instant messaging application.

21. The text editor of claim 20, wherein the speech input environment comprises a keyboard entry interface comprising symbols, punctuation or typeface data.

22. The text editor of claim 19, wherein the first and second escape sequence comprise selection of one or more button representations.

23. A method comprising:

obtaining speech input;

obtaining non-speech input;

converting the inputs into text data; and

transmitting the text data to a device for presentation on the device.

24. An apparatus comprising:

a user interface operable to receive speech input and non-speech input; and

a processor operable to time stamp the inputs, to convert the time-stamped inputs to text data, and to present the text data on the apparatus.