Task creation and completion with bi-directional user interactions

• US 10,614,030 B2
• Filed: 06/02/2017
• Issued: 04/07/2020
• Est. Priority Date: 06/02/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ferrydiansyah, Reza, Virdi, Gurpreet, Prasad, Richa, Seixeiro, Steve, Howard, III, Robert
• Primary Examiner(s): Tran, Loc
• Application Number:
  US15/612,727
• Publication Number:
  US 20180349389A1
• Time in Patent Office: 1,040 Days
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

A digital assistant supported on a local device and/or a remote digital assistant service is disclosed herein. The device may interact with a user and perform various functions or tasks based on the user'"'"'s input (e.g., web search, set calendar appointments, capture images or video). The digital assistant may identify and store artifacts associated with each action or function. Artifacts may include, for example, people in an image, search results from a web search, participants in a meeting, etc. Subsequently, the user may query the digital assistant for information about one of the artifacts, such as who the participants were in a meeting conducted the prior Thursday. The digital assistant can sift through the various artifacts and provide the user with the relevant information. Additionally, the digital assistant can suggest actions for the user based on the retrieved artifacts, such as to message one of the identified participants.

Claims

1. A method performed on a computing device having an operating system and hardware components, comprising:

receiving an initial input from a user of the computing device among a plurality of other computing devices that are each associated with the user, the initial input pertaining to content or events and implemented using one or more applications that execute on the computing device;

performing an action for the content or the events using the computing device that is responsive to the received initial input;

analyzing context for the performed action, the context including configuration and operational state of each of the applications, operating system, and hardware components;

generating one or more artifacts for the performed action using the analyzed context, wherein the one or more artifacts comprise information or data that describe the performed action;

storing the one or more artifacts in a remote archive, in which the remote archive stores multiple artifacts that describe a respective plurality of different actions performed by one or more of the plurality of other computing devices;

after storing the one or more artifacts, receiving a subsequent input from the user about the content or the events;

parsing the remote archive to identify artifacts for the content or the events that are responsive to the received subsequent input;

retrieving the identified artifacts from the remote archive in response to the parsing;

providing the retrieved artifacts to the user for selection;

receiving a user selection of an artifact from the retrieved artifacts; and

exposing a user experience on the computing device using at least the selected artifact,

wherein the initial input, the received subsequent input and the user selection are receivable at each of the plurality of other computing devices so that artifacts in the remote archive are retrievable at each of the plurality of other computing devices.

2. The method of claim 1, wherein the performed action based on the received initial input is associated with one or more of searching a network, searching the World Wide Web, capturing an image, capturing video, communicating with other users or devices, setting calendar appointments, setting reservations, or interacting with a map application.

3. The method of claim 2, wherein, when the performed action is the Internet search, the generated one or more artifacts include one or more of a portion of text from a search query, search results, URL (Uniform Resource Locator) for websites selected by a user, content associated with the selected URL, or the user'"'"'s actions with the search results.

4. The method of claim 2, wherein, when the performed action is capturing an image or capturing video, the generated one or more artifacts include at least one of landmarks, people, public establishments, private establishments, user emotions, identifiable objects, date of image or video capture, or time of image or video capture.

5. The method of claim 2, wherein the communicating with other users includes one or more of a video call, audio call, text message, or e-mail, and when the performed action is communicating with other users, the generated one or more artifacts include one or more of meeting participants, duration of meeting, date of meeting, time of meeting, topics discussed, or objects or data transferred or referenced during the communicating.

6. The method of claim 2, wherein, when the performed action is interacting with the map application, the generated one or more artifacts include one or more of location of the computing device, directions history, search history, nearby public establishments, nearby private establishments, nearby contacts, date associated with map application actions, or time associated with map application actions.

7. The method of claim 1, wherein the analysis is performed in real time or using one or more offline processes.

8. The method of claim 1, wherein the computing device includes a digital assistant that interoperates with multiple applications exposed on the computing device,

wherein the one or more artifacts are generated while the user uses an application, and

wherein the subsequent inputs are directed to the digital assistant.

9. The method of claim 1, wherein the computing device indexes artifacts associated with actions or functions performed on multiple devices associated with the user, and the indexed artifacts are stored for a predetermined time period.