Personalized agent for portable devices and cellular phone

• US 6,895,257 B2
• Filed: 02/18/2002
• Issued: 05/17/2005
• Est. Priority Date: 02/18/2002
• Status: Expired due to Term

• Current Assignee: Sovereign Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Boman, Robert, Stoimenov, Kirill, Kuhn, Roland, Junqua, Jean-claude
• Primary Examiner(s): Urban, Edward F.
• Assistant Examiner(s): Jackson, Blane J.
• Application Number:
  US10/077,904
• Publication Number:
  US 20030157968A1
• Time in Patent Office: 1,184 Days
• Field of Search: 455/563, 455/466, 455/414.1, 455/556.1, 455/556.2, 709/206, 379/88.13, 379/88.14, 379/88.22, 379/88.23, 379/88.25, 379/92.03, 379/88.26, 379/88.17
• US Class Current: 455/556.1
• CPC Class Codes: H04M 1/271 controlled by voice recognition
• CPC Class Codes: H04M 1/7243 with interactive means for internal management of messages
• CPC Class Codes: H04M 1/72436 for text messaging, e.g. short messaging services [SMS] or e-mails
• CPC Class Codes: H04M 2201/60 Medium conversion
• CPC Class Codes: H04M 2203/4536 Voicemail combined with text-based messaging
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML
• CPC Class Codes: H04M 3/53366 Message disposing or creating aspects

Abstract

Personalized agent services are provided in a personal messaging device, such as a cellular telephone or personal digital assistant, through services of a speech recognizer that converts speech into text and a text-to-speech synthesizer that converts text to speech. Both recognizer and synthesizer may be server-based or locally deployed within the device. The user dictates an e-mail message which is converted to text and stored. The stored text is sent back to the user as text or as synthesized speech, to allow the user to edit the message and correct transcription errors before sending as e-mail. The system includes a summarization module that prepares short summaries of incoming e-mail and voice mail. The user may access these summaries, and retrieve and organize email and voice mail using speech commands.

Claims

1. A personal messaging device comprising:

a hand held mobile device having an audio receiver and an audio transmitter, said audio receiver operable to receive input audio signals, and said audio transmitter operable to transmit output audio signals;

a speech recognizer operable to convert said input audio signals into recognized words;

a mail interface module coupled to said speech recognizer for managing mail messages based on said recognized words;

a speech editor, said speech editor providing a means for editing a segment of said output audio signals or said input audio signals;

said speech editor allowing a user to select a segment of said audio signals; and

said speech editor further allowing a user to alter said segment of said audio signals.

2. The apparatus of claim 1 wherein said mail messages are e-mail text messages.

3. The apparatus of claim 1 wherein said mail messages are voice messages.

4. The apparatus of claim 1 wherein said speech recognizer resides in a remote location independent of said hand held mobile device.

5. The apparatus of claim 1 wherein:

a user selects said segment of said audio signal by indicating a beginning and an end point of said segment; and

said speech editor provides a user interface communicating with said hand held mobile device through which a user selects said beginning and end points of said segment by keypad or voice inputs.

6. The apparatus of claim 5 wherein said speech editor includes means for providing a user with a modified audio signal, said modified audio signals identifying said segment that the user selected for editing, said modified audio signal identifying said beginning and end points of said segment with short audible tones.

7. The apparatus of claim 1 wherein said speech editor provides the contents of said audio signal to the hand held mobile device as a text message based on said recognized words.

8. The apparatus of claim 7 wherein a user edits said text message through keypad inputs.

9. The apparatus of claim 1 wherein the input audio signal is a voice input from a user.

10. The apparatus of claim 1 wherein the input audio signal is an audio conversation communicated using the hand held mobile device.

11. The apparatus of claim 1 wherein the output audio signal is synthesized speech.

12. A personal messaging device comprising:

a hand held mobile device having an audio receiver and an audio transmitter, said audio receiver operable to receive input audio signals, and said audio transmitter operable to transmit output audio signals, said hand held mobile device in communication with a mail storage database;

a speech recognizer operable to convert said input audio signals into text;

a text to speech synthesizer operable to convert said text into output audio signals;

a search engine interface in communication with said speech recognizer, said search engine interface operable to receive verbal or textual search queries from a user, said search engine interface further operable to retrieve a text or voice message from said mail storage database based on said search queries; and

a message summarizer, said summarizer operable to receive said retrieved messages from said search engine interface, said summarizer further operable to extract a summary of the contents of said messages.

13. The apparatus of claim 12 wherein said search engine interface is operable to return said retrieved message to a user in the form of text or synthesized speech.

14. The apparatus of claim 12 wherein summaries are provided to the user in the form of a text message or as synthesized speech.

15. The apparatus of claim 12 wherein retrieved message is an electronic mail or text message.

16. The apparatus of claim 12 wherein said retrieved message is a voice message or a recorded voice conversation.

17. A method for inputting speech into a hand held mobile device having an audio receiver and an audio transmitter, wherein the input speech is returned to the device in the form of text or synthesized speech for the purpose of review and correction by the user, the process comprising steps of:

transmitting said input speech to a speech recognizer;

converting said input speech into a text message;

converting said text message into synthesized speech;

transmitting said synthesized speech to a user;

selecting a segment of the message based on voice or keypad inputs from a user, the inputs indicating the beginning and end points of the desired segment;

replaying the message to the user with audible tones inserted at the points selected by the user;

receiving an indication from the user that the selected segment of the message is correct; and

editing the selected segment by allowing the user to input speech to replace the selected segment.

18. The method according to claim 17 further comprising the step of providing the message to a user in the form of a text message.

19. The method of claim 17 further comprising the step of editing the selected segment by allowing the user to input text to replace the selected segment.

20. The method of claim 17 further comprising the step of transmitting the edited message as synthesized speech or text.

21. A method for retrieving a summary of a stored electronic mail or text message using a hand held mobile device having an audio receiver and an audio transmitter, wherein the message is returned to the device based on an audio search query input by a user, the process comprising steps of:

transmitting the audio search query to a speech recognizer;

converting audio query into a text query;

interfacing with a search engine;

searching a mail database with the search engine;

retrieving an electronic mail or text message from the mail database based on the search query;

extracting a summary of the contents of the message;

converting the summary of the message into synthesized speech; and

transmitting the summary of the message to the hand held mobile device in the form of synthesized speech.

22. The method according to claim 21 further comprising the step of transmitting the summary of a message to the hand held mobile device in the form of a text message.

23. A method for retrieving a summary of a stored voice conversation or voice message using a hand held mobile device having an audio receiver and an audio transmitter, wherein the message is returned to the device based on an audio search query input by a user, the process comprising steps of:

transmitting the audio search query to a speech recognizer;

converting audio query into a text query;

interfacing with a search engine;

searching an audio database with the search engine;

retrieving a voice conversation or voice message from the audio database based on the search query;

extracting a summary of the contents of the voice conversation or message;

converting the summary of the message into synthesized speech; and

transmitting the summary of the message to the hand held mobile device in the form of synthesized speech.

24. The method according to claim 23 further comprising the step of transmitting the message to the hand held mobile device in the form of a text message.