Flexible speech-activated command and control

• US 8,620,667 B2
• Filed: 10/17/2005
• Issued: 12/31/2013
• Est. Priority Date: 10/17/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Andrew, Felix
• Primary Examiner(s): Shah, Paras D
• Assistant Examiner(s): Serrou, Abdelali
• Application Number:
  US11/251,681
• Publication Number:
  US 20070088556A1
• Time in Patent Office: 2,997 Days
• Field of Search: 704/257, 704/251, 704/231, 704/275, 704/270.1
• US Class Current: 704/275
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

A collection of human language terms is obtained. The terms describe a system resource. At least one term in the collection does not describe the identify of the resource. The collection of human language terms is incorporated into a grammar. The grammar is utilized as a basis for identifying spoken user commands.

Claims

1. A computer-implemented method of supporting speech-activated user commands, the method comprising:

activating a speech-activated command and control application;

launching speech support based at least in part upon the activation of the speech-activated command and control application;

performing a plurality of queries against a search engine interface, comprising a web service, to obtain a collection of human language terms from a remote source through the web service, the human language terms pertaining to a system resource, the plurality of queries being directed locally and through the web service to a plurality of different sources, one of the plurality of different sources including a central database having application capability and characteristic information, a second one of the plurality of different sources including a descriptive information file that includes application description and explanation information, a third one of the plurality of different sources including directly obtaining information from a plurality of individual applications, wherein at least one term in the collection does not describe the identity of the system resource, wherein the plurality of queries include specifications of what should be included in the collection of human language terms, wherein the collection of human language terms includes query results received in response to the plurality of queries, and wherein content of the collection of human language terms is automatically determined, by a computer processor that is a functional component of a computer, to be consistent with said specifications;

incorporating the collection of human language terms into a grammar, wherein the grammar includes action words including run, launch, load and start, wherein the grammar defines a collection of terms that have a logical connection to the system resource, wherein the grammar also defines other terms that have logical connections to other system resources, wherein the other terms that have the logical connections to the other system resources include names of the other system resources, the names being derived based on analysis of one or more user accessible file systems;

receiving a spoken user command;

identifying a particular word included in the spoken command;

determining that the particular word included in the spoken command is included in the grammar as part of the collection of terms that have a logical connection to the system resource, and wherein the particular word included in the spoken command is one of the collection of human language terms received in response to the query;

transmitting an indication of the particular word to the speech-activated command and control application;

utilizing the speech-activated command and control application to execute one of a plurality of functions related to the system resource, wherein executing the one of the plurality of functions comprises executing the one function at least partially in response to the determination that the particular word included in the spoken command is included in the grammar as part of the collection of terms that have a logical connection to the system resource;

receiving a notification that a new system resource has been added; and

reconstructing the grammar to account for the new system resource.

2. The method of claim 1, wherein the system resource and the other system resources include network resources, hardware devices, and databases.

3. The method of claim 1, wherein:

the system resource is a software application;

obtaining the collection of terms further comprises obtaining a collection of terms that include attributes of the software application;

the collection of terms includes words that describe functions, from the perspective of a user of the software application, that the software application is configured to perform; and

the attributes of the software application include a capability of the software application.

4. The method of claim 3, further comprising:

obtaining words that describe capabilities of other software applications; and

wherein the capability of the software application and the capabilities of the other software applications include a television capability, a photo editing capability, a music player capability, a burner capability, a spreadsheet capability, and a video conferencing capability.

5. The method of claim 1, wherein:

the system resource is a particular database;

obtaining the collection of terms further comprises obtaining a collection of terms that include attributes of the particular database; and

the attributes of the particular database include a set of terms that describes content of the particular database.

6. The method of claim 2, wherein the network resources include network data resources, wherein the hardware devices include a camera, a telephone, and an external peripheral device, and wherein the databases include an external database, a network-based database, and a local database.

7. The method of claim 1, wherein:

the system resource is a remotely accessible system resource;

obtaining the collection of terms further comprises obtaining a collection of terms that describe attributes of the remotely accessible system resource; and

identifying a particular word included in the spoken command comprises capturing sound waves of a user'"'"'s voice, digitizing the captured sound waves, and converting the digitized sound waves into basic language units.

8. The method of claim 1, wherein obtaining the collection of terms comprises obtaining a collection of terms from a remotely accessed central database that contains terms associated with a plurality of different system resources, the plurality of different system resources including a plurality of different software applications, wherein the plurality of different software applications are separate from the central database, and wherein, for each of the plurality of different software applications, the central database exposes a collection of extra metadata for the application.

9. The method of claim 1, wherein the system resource is a hardware device, and wherein obtaining the collection of terms further comprises obtaining a collection of terms that include attributes of the hardware device.

10. A computer-implemented, speech-activated user command system, the system comprising:

a search engine component configured to receive queries and respond by directing at least some of the queries locally and against a web service and obtaining collections of human language terms that describe system resources, and further by obtaining, through the web service, application and capability information from a remote source, wherein each of the system resources has a corresponding collection of human language terms, wherein at least one term in the collections does not describe the identity of the corresponding resource;

a command and control grammar configured to reflect the collections of terms obtained by the search engine component and including action words comprising run, launch, load and start;

a speech-activated command application configured to receive user input and analyze it based at least in part on the command and control grammar, the speech-activated command application making a determination as to how many of the system resources correspond to the user input, wherein the speech-activated command application initiates a conflict resolution process upon a determination that the user input does not correspond to any of the system resources, wherein the speech-activated command application launches a particular system resource upon a determination that the user input corresponds to one of the system resources, and wherein the speech-activated command application interacts with a user upon a determination that the user input corresponds to more than one of the system resources, the interaction with the user including presenting the user with a list of the more than one of the system resources on a user interface, the list being sortable based on application capability information and each of the more than one of the system resources in the list being capable of being selected by the user, the user interface being configured to enable the user to eliminate system resources from the list and to add system resources to the list, and the speech-activated command application further performing a confirmation step to confirm a user-selected one of the system resources before launching the system resource; and

a computer processor with memory, that is a functional part of the system and activated by the search engine component to facilitate receiving the queries and obtaining the collection of human terms and running the speech-activated command application.

11. The system of claim 10, wherein the search engine component is implemented as a web service, and wherein the system further comprises a speech recognition engine that converts the user input into phonemes.

12. The system of claim 11, wherein the collection of human language terms obtained by the search engine is a collection of terms that includes attributes of a software application, and wherein the system further comprises an operating system having a speech application program interface, the speech application program interface facilitating communication between the speech-activated command application and the speech recognition engine.

13. The system of claim 12, wherein the collection of human language terms obtained by the search engine is a collection of terms that include characteristics of a software application, and wherein the speech recognition engine incorporates a speaker profile, the speaker profile enabling the speech recognition engine to accommodate a particular user'"'"'s speech pattern and accent.

14. The system of claim 10, wherein the collections of human language terms obtained by the search engine include terms that correspond to attributes of databases, and wherein the queries include requests for attributes of the system resources, requests for capabilities of the system resources, requests for characteristics of the system resources, requests for names of the system resources, and requests for files that should be run upon a proper user request.

15. The system of claim 14, wherein the queries are performed against a search engine application programming interface, and wherein the search engine application programming interface is a software layer that allows the speech-activated command application to communicate with a speech recognition engine.

16. A computer-implemented method of supporting speech-activated user commands, the method comprising:

receiving a spoken input;

utilizing a computer processor that is a functional component of the computer to apply a grammar to the content of the spoken input so as to identify an attribute that corresponds to a plurality of system resources, wherein the attribute defines a characteristic of the plurality of system resources without identifying the plurality of system resources by their names, one of the plurality of system resources corresponding to a first database that is associated with a first type of content and a second one of the plurality of system resources corresponding to a second database that is associated with a second type of content, each of the first and second databases being associated with particular activation key words and particular identifiers that follow the activation key words, wherein content of the grammar includes the activation key words and the identifiers, including run, launch, load and start, wherein each of the first and second databases is associated with a particular query to be executed by a search engine application programming interface to identify the associated activation key words and identifiers, wherein the content is determined at least in part based on results of execution of the queries against the search engine application programming interface (API) to a centralized database that contains the attribute of the plurality of system resources, and wherein the query identifies the plurality of system resources, the search engine API comprising at least in part a web service interface, the content being determined at least in part based on results obtained from a local source, and a remote source through the web service interface;

receiving from a user an indication that the user does not want to run one of the system resources in the identified plurality of system resources;

graphically presenting on a display device a list to the user, the list not including the one of the system resources that the user does not want to run, the list including the other system resources in the identified plurality of system resources;

interacting with the user to determine one of the other system resources in the identified plurality of system resources from the list; and

utilizing a command and control application to execute a function relative to the determined one of the other system resources.

17. The method of claim 16, further comprising:

executing a user confirmation step prior to executing the function.

18. The method of claim 17, wherein utilizing a grammar as a basis for identifying an attribute that corresponds to a plurality of system resources comprises utilizing a grammar as a basis for identifying a characteristic of software applications, and wherein the list of the plurality of identified system resources is narrowed by the user based upon a hierarchy.