Global speech user interface

• US 8,407,056 B2
• Filed: 07/08/2011
• Issued: 03/26/2013
• Est. Priority Date: 10/03/2001
• Status: Expired due to Term

• Current Assignee: Promptu Systems Corporation
• Original Assignee: Promptu Systems Corporation
• Inventors: Jordan, Adam, Maddux, Scott Lynn, Plowman, Tim, Stanbach, Victoria, Williams, Jody
• Primary Examiner(s): Han, Qi
• Application Number:
  US13/179,294
• Publication Number:
  US 20110270615A1
• Time in Patent Office: 627 Days
• Field of Search: 704/270.1, 704/270, 704/231, 704/251, 704/275
• US Class Current: 704/270.1
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06Q 30/0271 Personalized advertisement
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/221 Announcement of recognition results
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4221 Dedicated function buttons, e.g. for the control of an EPG, subtitles, aspect ratio, picture-in-picture or teletext
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/472 End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content end-user interfaces for retrieving video data from a database G06F16/739; network services for supporting unicast streaming H04L65/612
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand
• CPC Class Codes: H04N 21/47211 for requesting pay-per-view content payment schemes payment architectures or payment protocols G06Q20/00, G07F
• CPC Class Codes: H04N 21/47214 for content reservation or setting reminders; for requesting event notification, e.g. of sport results or stock market stock exchange data over packet-switching network H04L12/1804; push services over packet-switching network H04L12/1859; notification of incoming messages in packet switching networks H04L51/224
• CPC Class Codes: H04N 21/475 End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data
• CPC Class Codes: H04N 21/478 Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/4781 Games
• CPC Class Codes: H04N 21/4782 : Web browsing , e.g. WebTV
• CPC Class Codes: H04N 21/4788 : communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/482 : End-user interface for program selection broadcast systems using EPGs H04H60/72
• CPC Class Codes: H04N 21/4826 : using recommendation lists, e.g. of programs or channels sorted out according to their score
• CPC Class Codes: H04N 21/4828 : for searching program descriptors retrieval of video data G06F16/739
• CPC Class Codes: H04N 21/4852 : for modifying audio parameters, e.g. switching between mono and stereo
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8173 : End-user applications, e.g. Web browser, game

Abstract

A global speech user interface (GSUI) comprises an input system to receive a user'"'"'s spoken command, a feedback system along with a set of feedback overlays to give the user information on the progress of his spoken requests, a set of visual cues on the television screen to help the user understand what he can say, a help system, and a model for navigation among applications. The interface is extensible to make it easy to add new applications.

Claims

1. A non-transitory computer readable storage medium encoded with instructions, which when executed by a communications system, configures a processor to establish a global speech user interface (GSUI) system comprising modules for:

performing speech recognition to transcribe spoken commands into commands acceptable by said communications system;

using the transcribed spoken commands to navigate among applications hosted on said communications system; and

displaying a set of visual cues to guide a user in issuing proper spoken commands, said visual cues comprising;

a set of immediate speech feedback overlays, each of which provides non-textual feedback information about a state of said communications system;

a set of help overlays, each of which provides a context-sensitive list of frequently used speech-activated commands for each screen of every speech-activated application;

a set of feedback overlays, each of which provides information about a problem that said communications system is experiencing; and

a main menu overlay that shows a list of services available to the user, each of said services being accessible by spoken command;

further comprising a user center that provides any of;

training and tutorials on how to use said communications system;

more help with specific speech-activated applications;

user account management; and

user settings and preferences for said communications system;

wherein said set of feedback overlays comprises any of;

a set of recognition feedback overlays that informs the user of a situation related to recognition; and

a set of application overlays that informs the user of an error or a problem related to an application used in said GSUI; and

wherein said set of recognition feedback overlays, in responding to unsuccessful recognitions that immediately follow one another, is displayed in three different modes comprising;

a first mode wherein said immediate speech feedback indicator changes to a question mark in responding to the first unsuccessful recognition;

a second mode wherein a textual message and a link to said help overlay are displayed in responding to the second unsuccessful recognition; and

a third mode wherein a textual message, a link to said help overlay, and a link to said more help overlay are displayed in responding to the third and subsequent unsuccessful recognition.

2. The medium of claim 1, wherein each of said immediate speech feedback overlays provides non-textual feedback information about a state of said communications system, said state being any of:

listening to the user'"'"'s spoken command;

non-speech enabled alert;

speech recognition processing;

application alert;

positive speech recognition; and

speech recognition unsuccessful.

3. The medium of claim 1, wherein each of said help overlays is accessible at all times.

4. The medium of claim 1, wherein said list of speech-activated commands provided by said help overlay comprises any of:

a set of application-specific commands;

a command associated with the user center for more help;

a command associated to said main menu display; and

a command to make said overlay disappear.

5. The medium of claim 1, wherein said visual cues further comprise a treatment of on-screen text which can be activated by a spoken command.

6. The medium of claim 5, wherein said treatment is an overlay in round shape and green color.

7. The medium of claim 5, wherein said treatment can be turned on or off by the user.

8. The medium of claim 5, wherein said on-screen text comprises any of:

a static text used in labels for on-screen graphics or in virtual buttons that may be selected by a cursor; and

a dynamic text used in content wherein one or more words can be activated by a spoken command.

9. The medium of claim 1, wherein any of said help overlays, feedback overlays and main menu overlay is implemented in a dialog box, said dialog box comprising any of:

one or more text box for textual information; and

one or more virtual buttons.

10. The medium of claim 9, wherein said dialog box further comprises an identity indicator.

11. The medium of claim 10, wherein said dialog box has either of an approximately transparent background, and an opaque background.

12. The medium of claim 11, wherein said approximately transparent background is incorporated with any of a dynamic image to enhance said identity indicator and a static image to enhance said identity indicator.

13. The medium of claim 11, wherein said text box is overlaid on said approximately transparent background.

14. The medium of claim 1, further comprising a speaker personalization and identification mechanism that allows users to train said communications system to identify respective users by voice.

15. The medium of claim 1, further including a remote control to relay user speech directly or indirectly for performing speech recognition, the remote control including a push-to-talk button;

where the help overlays include a volume indicator in the same color as the push-to-talk button and on-screen graphics indicating speech-enabled user interface elements.

16. The medium of claim 1, wherein said displaying provides immediate real-time visual feedback indicating various states of speech recognition activities.

17. The medium of claim 16, said real-time visual feedback comprises a set of overlays, each of which provides non-textual feedback information indicating state information selected from a predetermined group speech recognition activity states, the group including all of:

receiving spoken utterance;

processing utterance;

successful recognition;

unsuccessful recognition; and

command not allowed.

18. The medium of claim 1, wherein said displaying allows the user to initiate, via spoken command, an overlay display which indicates selectable user interface elements.

19. The medium of claim 18, wherein said selectable user interface elements comprise any of:

numeric identifications;

navigation options; and

application control options.

20. The medium of claim 1, wherein when the user'"'"'s spoken command is not recognized with a predefined degree of confidence, said displaying presents a list of predicted commands prompting the user to select from said list.

21. The medium of claim 1, further comprising:

navigating one or more on-screen lists based information via spoken commands.

22. The medium of claim 21, wherein said navigating enables the user to perform any of:

direct said on-screen list based information scroll up or scroll down by speaking a corresponding command;

select an item from said on-screen list based information by speaking a letter or a number identifying said item; and

select an item from said on-screen list based information by speaking the name of said item.

23. The medium of claim 1, further comprising any of:

allowing the user to navigate directly between applications via spoken command or a speech enabled menu; and

allowing the user to navigate directly to previously book-marked pages via spoken command.

24. The medium of claim 23, wherein said direct navigation to previously book-marked pages operates within and between applications.

25. The medium of claim 1, further including providing an interactive program guide that the user can access via spoken command.

26. The medium of claim 1, further including providing an interactive video on demand service, from which the user can order any video program contained in a list.

27. The medium of claim 26, wherein said video on demand service comprises any of:

allowing the user to, via spoken command, sort video programs by categories;

allowing the user to, via spoken command, search video programs by properties;

allowing the user to, via spoken command, set parental control with which children are blocked from accessing controlled video programs; and

allowing the user to obtain automatic recommendation based on voice identification.

28. The medium of claim 1, further including:

allowing the user to complete all aspects of a transaction via spoken commands.

29. The medium of claim 1, further including:

allowing the user to exercise control, via spoken commands, over home services and devices.