Interactive user interface for an intelligent assistant

• US 9,292,254 B2
• Filed: 05/15/2014
• Issued: 03/22/2016
• Est. Priority Date: 05/15/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Maluuba Inc. (Microsoft Corporation)
• Inventors: Simpson, James, Pasupalak, Sam, Musbah, Mohamed, Lee, Jennifer
• Primary Examiner(s): Cordero, Marivelisse Santiago
• Assistant Examiner(s): Brinich, Stephen
• Application Number:
  US14/278,283
• Publication Number:
  US 20140343950A1
• Time in Patent Office: 677 Days
• Field of Search: 704/235, 704/215, 704/E15.043, 704/E15.04, 704/275, 704/9, 704/E21.019, 704/257, 704/E15.001, 704/251
• US Class Current: 1/1
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/221 Announcement of recognition results

Abstract

A system, method and computer program for performing voice commands on a mobile device and presenting the results on an interactive timeline is disclosed. A user may utter a voice command into the microphone of their mobile device while an application is running. The voice command is processed to derive the intention of the user, specifically by determining the domain, at least one task and at least one parameter for the task from the voice command. A services component performs the task identified and presents the results on the mobile device screen. In various embodiments, the results are presented on a timeline and may be grouped together by domains and sorted by the time that the results were obtained. A search history view may also be viewed that includes search results sorted chronologically each of which is represented graphically by an icon that represents the category of each search. When a user utters a voice command, the text representation is displayed together with an edit button, a resay button, and a progress bar. A user may modify the text representation at any time while the natural language processing is being performed.

Claims

1. A computer-implemented method for making a voice input to a computing device, the voice input for instructing the computing device to perform at least one task, the method being performed by one or more processors of the computing device and comprising:

receiving at the computing device one or more user inputs to display a voice input interface;

displaying the voice input interface;

receiving the voice input at the voice input interface;

providing the voice input to an automatic speech recognition engine for processing the voice input and producing a text representation of the voice input;

displaying the text representation of the voice input on the voice input interface, wherein the voice input interface is configured to process the text representation and further configured to allow a user to modify the text representation before a results screen is displayed to the user; and

displaying a progress representation on the voice input interface,

wherein the voice input interface is configured to allow the user to modify the text representation to generate a modified text representation after processing, and

wherein processing the modified text representation includes performing natural language processing on the modified text representation to identify at least one domain, the at least one task, and at least one parameter to which the modified text representation relates.

2. The method of claim 1, wherein the voice input interface is configured to allow the user to cancel the processing of the text representation while the progress representation is being displayed.

3. The method of claim 2, wherein the processing of the text representation comprises performing one or more commands.

4. The method of claim 1, wherein processing the modified text representation includes performing the at least one task.

5. A non-transitory computer readable storage medium for storing instructions for making a voice input to a computing device, the voice input for instructing the computing device to perform at least one task, the instructions when executed by a processor cause the processor to:

display a voice input interface when one or more predetermined inputs is received by the computing device;

receive the voice input uttered by a user at the voice input interface;

provide the voice input to an automatic speech recognition engine for processing the voice input and for producing a text representation of the voice input;

display the text representation of the voice input on the voice input interface, wherein the voice input interface is configured to process the text representation and further configured to allow the user to modify the text representation before a results screen is displayed to the user; and

display a progress representation on the voice input interface,

wherein the voice input interface is configured to allow the user to modify the text representation to generate a modified text representation after processing, and

wherein processing the modified text representation includes performing natural language processing on the modified text representation to identify at least one domain, the at least one task, and at least one parameter to which the modified text representation relates.

6. A system for making a voice input to a computing device, the voice input for instructing the computing device to perform at least one task, the system including at least one computer processor and comprising:

a voice input interface configured to receive the voice input uttered by a user;

a user interface for receiving at the computing device, one or more user inputs to display the voice input interface; and

an automatic speech recognition engine for processing the voice input and producing a text representation of the voice input,

the voice input interface further configured to display the text representation of the voice input on the voice input user interface and display a progress representation on the voice input user interface,

wherein the voice input user interface is configured to process the text representation and is configured to allow the user to modify the text representation before a results screen is displayed to the user,

wherein the voice input interface is configured to allow the user to modify the text representation to generate a modified text representation after processing, and

wherein the voice input interface is configured to process the modified text representation by performing natural language processing on the modified text representation to identify at least one domain, the at least one task, and at least one parameter to which the modified text representation relates.

7. The system of claim 6, wherein the voice input interface is configured to allow the user to cancel the processing of the text representation while the progress representation is being displayed.

8. The system of claim 7, wherein the voice input interface is configured to process of the text representation by performing one or more commands.

9. The system of claim 6, wherein the voice input interface is configured to process the modified text representation by performing the at least one task.