INTERACTIVE TIMELINE FOR PRESENTING AND ORGANIZING TASKS

• US 20140012574A1
• Filed: 06/21/2013
• Published: 01/09/2014
• Est. Priority Date: 06/21/2012
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Maluuba Inc. (Microsoft Corporation)
• Inventors: Pasupalak, Sam, Petrescu, Adrian, Ismail, Tareq, Simpson, James, Lee, Jennifer
• Application Number:
  US13/923,771
• Publication Number:
  US 20140012574A1
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G06F 16/243 Natural language query formulation
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/285 Clustering or classification

Abstract

A system, method and computer program for performing voice commands and presenting results on an interactive timeline is disclosed. A user may utter a voice command (e.g. into a mobile device) which is processed to derive the intention, specifically by determining the domain, at least one task and at least one parameter for the task. A services component performs the task identified and presents the results. In various embodiments, the results are presented on a timeline and may be grouped together by domains and presented chronologically. A search history view may also be viewed that includes search results sorted chronologically each of which is represented graphically by an icon that represents a search domain. A voice command may be presented by a text representation with an edit button, a resay button, and a progress bar. The text representation may be modified while the natural language processing is being performed.

Claims

1. A computer-implemented method for searching and presenting search results on a history timeline of a mobile computing device, the method being performed by one or more processors of the mobile computing device and comprising:

receiving a command at the mobile computing device to initiate a search, the command comprising at least one parameter and being related to at least one domain and at least one task;

providing the command or the at least one domain, at least one task, and at least one parameter from the command for execution by at least one service configured to execute the command;

receiving results generated by the at least one service and generating a results user interface (UI) summarizing the results;

organizing the results provided by the at least one service for presenting on a results history UI configured as a timeline, defining an event entry for the results history UI for each said result, each respective event entry comprising a graphical representation identifying the domain and a summary of details for the for each respective result, wherein each event entry is configured to present the respective results UI when the event entry is invoked; and

presenting the results history UI on the mobile computing device.

2. The method of claim 1 further comprising, in response to receiving an input for invoking an event entry on the results history UI, presenting the respective results UI on the mobile computing device.

3. The method of claim 1 comprising presenting the results UI on the mobile computing device in response to receiving the command and prior to presenting the results history UI.

4. The method of claim 1, wherein the command is a voice command uttered by the user.

5. The method of claim 4, wherein the domain, at least one task, and at least one parameter related to the voice command are identified by performing statistical natural language processing on the voice command.

6. The method of claim 1, wherein organizing the results provided by the at least one service comprises presenting the results on the results history UI in chronological order.

7. The method of claim 1, wherein organizing the results provided by the at least one service comprises presenting the results on the results history page grouped by domain.

8. A non-transitory computer readable storage medium for storing instructions for searching and presenting searched data entries on a mobile computing device, the instructions when executed by a processor cause the processor to:

receive a command at the mobile computing device to initiate a search, the command comprising at least one parameter and being related to at least one domain and at least one task;

provide the command or the at least one domain, at least one task, and at least one parameter from the command for execution by at least one service configured to execute the command;

receive results generated by the at least one service and generate a results user interface (UI) summarizing the results;

organize the results provided by the at least one service for presenting on a results history UI configured as a timeline, defining an event entry for the results history UI for each said result, each respective event entry comprising a graphical representation identifying the domain and a summary of details for the for each respective result, wherein each event entry is configured to present the results UI when the event entry is invoked; and

present the results history UI on the mobile computing device.

11. The method of claim 8, wherein the position of the group on the timeline is determined according to the time that a last event in the group was performed.

12. The method of claim 8, wherein the position of the group on the timeline is determined according to the time that a first event in the group was performed.

13. The method of claim 8, wherein each event within a group is provided with one of two statuses according to whether a user has reviewed the action performed.

14. The method of claim 8 wherein the plurality of predefined categories are defined to represent respective types of search events, time-organization events and communication events.

9. A system for searching and presenting search results on a history timeline of a mobile computing device, the system including one or more processors of the mobile computing device and comprising:

receive a command at the mobile computing device to initiate a search, the command comprising at least one parameter and being related to at least one domain and at least one task;

provide the command or the at least one domain, at least one task, and at least one parameter from the command for execution by at least one service configured to execute the command;

receive results generated by the at least one service and generate a results user interface (UI) summarizing the results;

organize the results provided by the at least one service for presenting on a results history UI configured as a timeline, defining an event entry for the results history UI for each said result, each respective event entry comprising a graphical representation identifying the domain and a summary of details for the for each respective result, wherein each event entry is configured to present the results UI when the event entry is invoked; and

present the results history UI on the mobile computing device.

10. A computer-implemented method for grouping events and presenting the grouped events on an interactive timeline of a mobile computing device, the method being performed by one or more processors of the mobile computing device and comprising:

performing a plurality of events on the mobile computing device, each respective event associated with a time;

identifying a predefined category of a plurality of predefined categories to which an event relates and further associating the event with the predefined category;

forming a group of related events where each event in the group is associated with the same predefined category; and

presenting the group on the interactive timeline, the position in the timeline being based on the time that at least one of the related events was performed and wherein the group is indicated graphically in the timeline according in accordance with the predefined category of the related events of the group.

15. A non-transitory computer readable storage medium for storing instructions grouping events and presenting the grouped events on an interactive timeline of a mobile computing device, the instructions when executed cause a computer processor to:

perform a plurality of events on the mobile computing device, each respective event associated with a time;

identify a predefined category of a plurality of predefined categories to which an event relates and further associating the event with the predefined category,

form a group of related events where each event in the group is associated with the same predefined category; and

present the group on the interactive timeline, the position in the timeline being based on the time that at least one of the related events was performed; and

wherein each said group is indicated graphically in the timeline according in accordance with the predefined category of the related events of the group.

16. A system for grouping events and presenting the grouped events on an interactive timeline of a mobile computing device, the system comprising:

a services component configured to perform at least one event on the mobile computing device;

a timeline component configured to identify at least one predefined category to which the at least one event relates and further associating the at least one event with the at least one predefined category;

the timeline module configured to form a group of related events in the at least one predefined category; and

a user interface component for presenting the group on the interactive timeline, the position of the group on the timeline being based on the time that at least one of the related actions was performed; and

wherein each said group is indicated graphically according to the at least one predefined category.

17. A computer-implemented method for modifying a voice input received by a computing device, the voice input instructing the computing device to perform one or more tasks, the method being performed by one or more processors of the computing device and comprising:

providing the voice input to an automatic speech recognition engine for processing the voice input and producing a text representation of the voice input;

while continuing to process the text representation and before a results screen for the one or more tasks is displayed to the user, displaying the text representation of the voice input on a voice input user interface configured to receive input to modify the text representation thereby to modify the one or more tasks to be performed;

receiving input to modify the text representation; and

processing the text representation as modified.

18. The method of claim 17 comprising displaying a progress representation on the voice input user interface representing progress of the processing.

19. The method of claim 18, wherein the voice user interface is configured to receive input to cancel the processing of the text representation while the progress representation is being displayed.

20. The method of claim 17, wherein the processing of the text representation comprises performing the one or more tasks.

21. The method of claim 17, wherein processing the text representation as modified includes performing natural language processing to identify at least one domain, at least one task, and at least one parameter to which the text representation as modified relates.

22. The method of claim 21, wherein processing the text representation as modified includes performing the at least one task.

23. A non-transitory computer readable storage medium for storing instructions for modifying a voice input received at a computing device, the voice input for instructing the computing device to perform one or more tasks, the instructions when executed by a processor cause the processor to:

provide the voice input to an automatic speech recognition engine for processing the voice input and producing a text representation of the voice input;

while continuing to process the text representation and before a results screen for the one or more tasks is displayed to the user, display the text representation of the voice input on a voice input user interface configured to receive input to modify the text representation thereby to modify the one or more tasks to be performed;

receive input to modify the text representation; and

process the text representation as modified.

24. A system for modifying a voice input received at a computing device, the voice input for instructing the mobile computing device to perform one or more tasks, the system including at least one computer processor and comprising:

a user interface for receiving at a computing device one or more user inputs to display a voice input interface;

a voice input interface configured to receiving the voice input uttered by a user;

an automatic speech recognition engine for processing the voice input and producing a text representation of the voice input, the voice input interface further configured to display the text representation of the voice input on the voice input user interface and display a progress representation on the voice input user interface; and

wherein the voice input user interface is configured to receive input to modify the text representation before a results screen is displayed.