Mapping an audio utterance to an action using a classifier

• US 8,484,025 B1
• Filed: 10/04/2012
• Issued: 07/09/2013
• Est. Priority Date: 10/04/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Moreno Mengibar, Pedro J., Jansche, Martin, Biadsy, Fadi
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US13/644,986
• Time in Patent Office: 278 Days
• Field of Search: 704/231, 704/241, 704/232, 704/255, 704/236, 704/239, 704/244, 704/256, 704/259, 704/257, 704/256.2, 704/233, 704/9, 704/8, 704/243, 704/240, 704/270, 704/10, 706/20, 706/12
• US Class Current: 704/255
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/0638 Interactive procedures
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Disclosed embodiments relate to mapping an utterance to an action using a classifier. One illustrative computing device includes a user interface having an input component. The computing device further includes a processor and a computer-readable storage medium, having stored thereon program instructions that, upon execution by the processor, cause the computing device to perform a set of operations including: receiving an audio utterance via the input component; determining a text string based on the utterance; determining a string-feature vector based on the text string; selecting a target classifier from a set of classifiers, wherein the target classifier is selected based on a determination that a string-feature criteria of the target classifier corresponds to at least one string-feature of the string-feature vector; and initiating a target action that corresponds to the target classifier.

Claims

1. A method, comprising:

receiving, by a computing device, an audio utterance;

determining a text string based on the utterance;

determining a string-feature vector based on the text string;

receiving sensor data;

determining a sensor-feature vector based on the sensor data;

selecting a target classifier from a set of classifiers, wherein the target classifier is selected based on a determination that a string-feature criteria of the target classifier corresponds to at least one string-feature of the string-feature vector and a sensor-feature criteria of the target classifier corresponds to at least one sensor-feature in the sensor-feature vector; and

initiating, by the computing device, a target action that corresponds to the target classifier.

2. The method of claim 1, wherein the string-feature criteria comprises at least one criteria in a set of criteria comprising a term-identity criteria, a part-of-speech identity criteria, and a lexical-class identity criteria.

3. The method of claim 1, wherein the sensor-feature criteria comprises at least one criteria in a set of criteria comprising a travel-speed criteria and a user-identity criteria.

4. The method of claim 1, further comprising:

determining a non-target classifier from the set of classifiers;

wherein a string-feature criteria of the non-target classifier corresponds to at least one string in the string-feature vector; and

wherein the non-target classifier is determined based on a determination that a sensor-feature criteria of the non-target classifier does not correspond to any sensor-feature of the sensor-feature vector.

5. The method of claim 4, wherein the non-target classifier corresponds to a non-target action, and wherein the text string has at least two semantic meanings, a first semantic-meaning that corresponds to the target action and a second semantic-meaning that corresponds to the non-target action.

6. The method of claim 5, wherein receiving sensor data comprises receiving sensor data via one of an accelerometer and a GPS receiver included in the computing device;

wherein the sensor-feature criteria of the target classifier comprises a first travel-speed criteria and the target action comprises launching a mapping application and providing mapping information related to a point-of-interest associated with at least a portion of the text string; and

wherein the sensor-feature criteria of the non-target classifier comprises a second travel-speed criteria and the non-target action comprises launching a non-mapping application.

7. The method of claim 6, wherein launching a non-mapping application comprises launching a web-browsing application, and wherein the non-target action further comprises loading a website that corresponds to the at least a portion of the text string.

8. The method of claim 6, wherein launching a non-mapping application comprises launching an application associated with the at least a portion of the text string.

9. The method of claim 4, wherein receiving sensor data comprises receiving sensor data via an image-capturing sensor included in the computing device;

wherein the sensor-feature criteria of the target classifier comprises a first user-identity criteria and the target action comprises sending an email;

wherein the sensor-feature criteria of the non-target classifier comprises a second user-identity criteria and the non-target action comprises sending an SMS message; and

wherein determining a sensor-feature vector based on the sensor data comprises determining a user-identity sensor-feature based on a facial-recognition technique and the sensor data.

10. The method of claim 1, the method further comprising:

determining an argument template that corresponds to the target classifier; and

parsing the text string for an argument based on the argument template, wherein the target action is based on the argument.

11. The method of claim 1, wherein the set of classifiers is a set of sub-classifiers arranged as children to a parent super-classifier in a hierarchical arrangement.

12. The method of claim 1, wherein the utterance is a first utterance, the text string is a first text-string, the string-feature vector is a first string-feature vector, the sensor data is first sensor-data, and the sensor-feature vector is a first sensor-feature vector, the method further comprising:

receiving a second utterance;

determining a second text-string based on the second utterance;

determining a second string-feature vector based on the second text-string, wherein the first string-feature vector and the second string-feature vector are different;

receiving second sensor-data;

determining a second sensor-feature vector based on the second sensor-data, wherein the first sensor-feature vector and the second sensor-feature vector are different;

selecting the target classifier from the set of classifiers, wherein the target classifier is selected based on a determination that the string-feature criteria corresponds to at least one string-feature of the second string-feature vector and the sensor-feature criteria corresponds to at least one sensor-feature of the second sensor-feature vector; and

initiating the target action.

13. A computer-readable storage medium, having stored thereon program instructions that, upon execution by a computing device, cause the computing device to perform a set of operations comprising:

receiving an audio utterance;

determining a text string based on the utterance;

determining a string-feature vector based on the text string;

receiving sensor data;

determining a sensor-feature vector based on the sensor data;

selecting a target classifier from a set of classifiers, wherein the target classifier is selected based on a determination that a string-feature criteria of the target classifier corresponds to at least one string-feature of the string-feature vector and a sensor-feature criteria of the target classifier corresponds to at least one sensor-feature in the sensor-feature vector; and

initiating a target action that corresponds to the target classifier.

14. The computer-readable storage medium of claim 13, wherein the string-feature criteria comprises at least one criteria in a set of criteria comprising a term-identity criteria, a part-of-speech identity criteria, and a lexical-class identity criteria.

15. The computer-readable storage medium of claim 13, wherein the sensor-feature criteria comprises at least one criteria in a set of criteria comprising a travel-speed criteria and a user-identity criteria.

16. The computer-readable storage medium of claim 13, the set further comprising:

determining a non-target classifier from the set of classifiers;

wherein a string-feature criteria of the non-target classifier corresponds to at least one string in the string-feature vector; and

wherein the non-target classifier is determined based on a determination that a sensor-feature criteria of the non-target classifier does not correspond to any sensor-feature of the sensor-feature vector.

17. The computer-readable storage medium of claim 16, wherein the non-target classifier corresponds to a non-target action, and wherein the text string has at least two semantic meanings, a first semantic-meaning that corresponds to the target action and a second semantic-meaning that corresponds to the non-target action.

18. A computing device comprising:

a user interface having an input component;

a processor; and

a computer-readable storage medium, having stored thereon program instructions that, upon execution by the processor, cause the computing device to perform a set of operations comprising;

receiving an audio utterance via the input component;

determining a text string based on the utterance;

determining a string-feature vector based on the text string;

selecting a target classifier from a set of classifiers, wherein the target classifier is selected based on a determination that a string-feature criteria of the target classifier corresponds to at least one string-feature of the string-feature vector; and

initiating a target action that corresponds to the target classifier.

19. The computing device of claim 18, wherein the target classifier is selected based further on an action previously initiated by the computing device.

20. The computing device of claim 18, wherein the utterance is a first utterance, the text string is a first text-string, the string-feature vector is a first string-feature vector, the set of operations further comprising:

receiving a second utterance;

determining a second text-string based on the second utterance;

determining a second string-feature vector based on the second text-string, wherein the first string-feature vector and the second string-feature vector are different;

selecting the target classifier from the set of classifiers, wherein the target classifier is selected based on a determination that the string-feature criteria corresponds to at least one string-feature of the second string-feature vector and the sensor-feature criteria corresponds to at least one sensor-feature of the second sensor-feature vector; and

initiating the target action.

21. A method, comprising:

receiving, by a computing device, an audio utterance;

determining a text string based on the utterance;

determining a string-feature vector based on the text string;

receiving sensor data;

determining a sensor-feature vector based on the sensor data;

determining an identified action; and

generating a classifier, wherein the classifier has a string-feature criteria that corresponds to at least one string-feature in the string-feature vector, has a sensor-feature criteria that corresponds to at least one sensor-feature in the sensor-feature vector, and corresponds to the identified action.

22. The method of claim 21, wherein the string-feature criteria comprises at least one of a term-identity criteria, a part-of-speech identity criteria, and a lexical-class identity criteria.

23. The method of claim 21, wherein receiving sensor data comprises receiving sensor data via one of an accelerometer and a GPS receiver, included in the computing device, and wherein the sensor-feature criteria comprises a travel-speed criteria.

24. The method of claim 22, wherein receiving sensor data comprises receiving sensor data via an image-capturing sensor included in the computing device, and wherein the sensor-feature criteria comprises a user-identity criteria.

25. The method of claim 21, wherein the utterance is a first utterance, the text string is a first text-string, the string-feature vector is a first string-feature vector, the method further comprising:

receiving a second utterance;

determining a second text-string based on the second utterance;

determining a second string-feature vector based on the second text-string; and

modifying the classifier such that the string-feature criteria further corresponds to at least one string-feature in the second string-feature vector.