Action broker

• US 9,558,275 B2
• Filed: 12/13/2012
• Issued: 01/31/2017
• Est. Priority Date: 12/13/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Viegas, Evelyne, Mulwad, Varish, Pantel, Patrick
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Le, Thuykhanh
• Application Number:
  US13/713,197
• Publication Number:
  US 20140172412A1
• Time in Patent Office: 1,510 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 9/547 Remote procedure calls [RPC]; Web services

Abstract

Among other things, one or more techniques and/or systems are provided for building an action catalogue, generating an action frame for an action within the action catalogue, and/or executing an action. In an example, an action may be included within the action catalogue based upon descriptive text associated with an application indicating that the application is capable of performing the action (e.g., a movie app may be capable of performing an order movie tickets action). A parameter (e.g., a movie name) and/or an execution endpoint (e.g., a uniform resource identifier used to access movie ticket ordering functionality) may be used to generate an action frame for the action. In this way, user intent to perform an action may be identified from user input (e.g., a spoken command), and the action may be performed (e.g., on behalf of the user with minimal additional user input) by using the action frame.

Claims

1. A computer system, comprising:

one or more processors; and

one or more computer-readable media having stored thereon computer-executable instructions that are executable by the one or more processors, and that configure the computer system to provide an action frame utilizing functionality from a plurality of different network-connected computer-executable applications, including computer-executable instructions that configure the computer system to perform at least the following;

identify a plurality of different network-connected computer-executable applications that are accessible to the computer system;

generate an action catalog identifying, for each of the plurality of different computer-executable applications, one or more corresponding actions, including, for at least a particular computer-executable application of the plurality of different computer-executable applications;

parsing one or more descriptive texts corresponding to the particular computer-executable application, to identify at least one particular action that may be provided by the particular computer-executable application; and

populating the action catalog with the at least one particular action in association with the particular computer-executable application;

based at least on having generated the action catalog, generate an action frame for each of the one or more actions, each action frame identifying how to invoke a fillable form at a corresponding computer-executable application to carry out a corresponding action, including, for the at least one particular action;

identifying at least one fillable form of the particular computer-executable application for carrying out the at least one particular action, the at least one fillable form including one or more parameters for receiving user-supplied values to use as part of carrying out the at least one particular action;

extracting the one or more parameters from the at least one fillable form;

identifying at least one execution endpoint that is usable for invoking the at least one fillable form of the particular computer-executable application; and

populating a particular action frame with the one or more parameters and with the at least one execution endpoint;

subsequent to generating the action frame, identify a user intent to perform the least one particular action;

based at least on identifying the user intent to perform the at least one particular action, identify the particular action frame; and

based at least on identifying the particular action frame, invoke the least one execution endpoint over a network using at least one user-supplied value for at least one of the one or more parameters as input to the at least one fillable form.

2. The computer system of claim 1, also including computer-executable instructions that configure the computer system to generate a one or more feature vectors from the one or more actions.

3. The computer system of claim 2, also including computer-executable instructions that configure the computer system to generate a feature vector from an action based at least on there being one or more content sources that reference the action.

4. The computer system of claim 3, also including computer-executable instructions that configure the computer system to generate the feature vector from the action based at least on one or more features extracted from the one or more content sources.

5. The computer system of claim 4, wherein the one or more features are selected from the group comprising a number feature, a category feature, and a binary feature.

6. The computer system of claim 3, wherein the one or more content sources are selected from the group comprising a website, an application marketplace description, a web service, a cloud service, an application, an email, a calendar entry, and a task entry.

7. The computer system of claim 2, also including computer-executable instructions that configure the computer system to utilize a classifier function to determine whether each action is valid or invalid based upon the one or more feature vectors.

8. The computer system of claim 7, also including computer-executable instructions that configure the computer system to determine that an action is valid based upon one or more of (i) a number of features and (ii) a number of feature occurrences associated with one or more content source being above a defined threshold.

9. The computer system of claim 1, also including computer-executable instructions that configure the computer system to identify one or more parts of speech tags selected from the group comprising a noun, a verb, and an adjective when parsing the descriptive text corresponding to each computer-executable application to identify the one or more actions.

10. The computer system of claim 1, also including computer-executable instructions that configure the computer system to identify noun phrases when parsing the descriptive text corresponding to each computer-executable application to identify the one or more actions.

11. The computer system of claim 1, wherein at least one of the computer-executable applications comprises a website.

12. The computer system of claim 1, wherein each of the one or more actions comprise an identified action verb and an identified action object.

13. The computer system of claim 1, wherein an execution endpoint comprises a Uniform Resource Identifier (URI).

14. The computer system of claim 1, also including computer-executable instructions that configure the computer system to rank execution endpoints.

15. The computer system of claim 14, wherein execution endpoints are ranked based on one or more of reliability, functionality robustness, user reviews, trust, marketplace registration, and user preference.

16. A method, implemented at a computer system that includes one or more processors, for providing an action frame utilizing functionality from a plurality of different network-connected computer-executable applications, the method comprising:

identify a plurality of different network-connected computer-executable applications that are accessible to the computer system;

generating an action catalog identifying, for each of the plurality of different computer-executable applications, one or more corresponding actions, including, for at least a particular computer-executable application of the plurality of different computer-executable applications;

parsing one or more descriptive texts corresponding to the particular computer-executable application, to identify at least one particular action to be provided by the particular computer-executable application; and

populating the action catalog with the at least one particular action in association with the particular computer-executable application;

based at least on having generated the action catalog, generating an action frame for each of the one or more actions, each action frame identifying how to invoke a fillable form at a corresponding computer-executable application to carry out a corresponding action, including, for the at least one particular action;

identifying at least one fillable form of the particular computer-executable application for carrying out the at least one particular action, the at least one fillable form including one or more parameters for receiving user-supplied values to use as part of carrying out the at least one particular action;

extracting the one or more parameters from the at least one fillable form;

identifying at least one execution endpoint that is usable for invoking the at least one fillable form of the particular computer-executable application; and

populating a particular action frame with the one or more parameters and with the at least one execution endpoint;

subsequent to generating the action frame, identify a user intent to perform the least one particular action;

based at least on identifying the user intent to perform the at least one particular action, identify the particular action frame; and

based at least on identifying the particular action frame, invoke the least one execution endpoint over a network using at least one user-supplied value for at least one of the one or more parameters as input to the at least one fillable form.

17. The method of claim 16, further comprising generating a one or more feature vectors from the one or more actions.

18. The method of claim 17, further comprising utilizing a classifier function to determine whether each action is valid or invalid based upon the one or more feature vectors.

19. The method of claim 16, wherein at least one of the computer-executable applications comprises a website.

20. A computer program product comprising one or more hardware storage devices having stored thereon computer-executable instructions that are executable by one or more processors of a computer system, and that configure the computer system to provide an action frame utilizing functionality from a plurality of different network-connected computer-executable applications, including computer-executable instructions that configure the computer system to perform at least the following:

identify a plurality of different network-connected computer-executable applications that are accessible to the computer system;

generate an action catalog identifying, for each of the plurality of different computer-executable applications, one or more corresponding actions, including, for at least a particular computer-executable application of the plurality of different computer-executable applications;

parsing one or more descriptive texts corresponding to the particular computer-executable application, to identify at least one particular action to be provided by the particular computer-executable application; and

populating the action catalog with the at least one particular action in association with the particular computer-executable application;

based at least on having generated the action catalog, generate an action frame for each of the one or more actions, each action frame identifying how to invoke a fillable form at a corresponding computer-executable application to carry out a corresponding action, including, for the at least one particular action;

identifying at least one fillable form of the particular computer-executable application for carrying out the at least one particular action, the at least one fillable form including one or more parameters for receiving user-supplied values to use as part of carrying out the at least one particular action;

extracting the one or more parameters from the at least one fillable form;

identifying at least one execution endpoint that is usable for invoking the at least one fillable form of the particular computer-executable application; and

populating a particular action frame with the one or more parameters and with the at least one execution endpoint;

subsequent to generating the action frame, identify a user intent to perform the least one particular action;

based at least on identifying the user intent to perform the at least one particular action, identify the particular action frame; and

based at least on identifying the particular action frame, invoke the least one execution endpoint over a network using at least one user-supplied value for at least one of the one or more parameters as input to the at least one fillable form.