VEHICLE CONTROL

• US 20090083034A1
• Filed: 09/19/2008
• Published: 03/26/2009
• Est. Priority Date: 09/21/2007
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: LOPEZ, JAVIER FERREIROS, HERNANDEZ, RUBEN SAN-SEGUNDO, SCARLATTI, DAVID, VILLAR, VICTOR PEREZ, MOLINA, ROBERTO
• Granted Patent: US 8,311,827 B2
• Time in Patent Office: Days
• US Class Current: 704/251
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals

Abstract

The present invention relates to voice-activated vehicle control, and to the control of UAVs (unmanned air vehicles) using speech in particular. A method of controlling a vehicle is provided that includes receiving one or more instructions issued as speech and analyzing the speech using speech recognition software to provide a sequence of words and a word confidence measure for each word so recognized. The sequence of words is analyzed to identify a semantic concept corresponding to an instruction based on the analysis, and a semantic confidence level for the semantic concept identified derived at least in part with reference to the word confidence measures of the words associated with the semantic concept. A spoken confirmation of the semantic concept so identified based on the semantic confidence level is provided, and the semantic concept is used to provide a control input for the vehicle.

Claims

1. A computer-implemented method of controlling a vehicle, comprising:

receiving one or more instructions issued as speech;

analyzing the speech using speech recognition software to provide a sequence of words and a word confidence measure for each word so recognized;

analyzing the sequence of words to identify a semantic concept corresponding to an instruction based on the analysis, and a semantic confidence level for the semantic concept identified derived at least in part with reference to the word confidence measures of the words associated with the semantic concept;

providing a spoken confirmation of the semantic concept so identified based on the semantic confidence level; and

using the semantic concept so identified to provide a control input for the vehicle.

2. The method of claim 1, wherein the step of analyzing the speech to provide a sequence of words comprises using (continuous) hidden Markov models.

3. The method of claim 1, wherein the step of analyzing the speech to provide a sequence of words comprises using a language model that provides information about the most likely sequences of words expected to be found.

4. The method of claim 3, wherein the step of analyzing the speech to provide a sequence of words comprises using the language 30 model to provide a 2-gram statistical model.

5. The method of claim 1, wherein the step of analyzing the sequence of words comprises analyzing the sequence of words and the associated word confidence levels to identify the one or more semantic concepts.

6. The method of claim 5, wherein the semantic concept comprises an identifier and a value.

7. The method of claim 5, wherein the step of analyzing the sequence of words comprises:

identifying the one or more semantic concepts using a bottom-up approach starting with an analysis of each word identified individually and then extending the analysis to neighboring words.

8. The method of claim 5, wherein the step of analyzing the sequence of words to identify the one or more semantic concepts comprises:

tagging each word according to its type, and looking for known patterns from the sequence of tags, thereby allowing deduction of a semantic concept.

9. The method of claim 1, wherein the step of providing a spoken confirmation of the semantic concept comprises an indication that the instruction was not understood when the semantic confidence level is below a threshold.

10. The method of claim 9, further comprising providing a spoken confirmation of the semantic concept based on the semantic confidence level and an indicated verbosity level.

11. The method of claim 10, comprising providing a spoken confirmation of the semantic concept identified with at least one of a speaking rate and a pitch that is increased as the indicated verbosity level decreases.

12. The method of claim 11, further comprising:

providing a spoken confirmation of the semantic concept identified based on the semantic confidence level and an urgency level, the verbosity of the spoken confirmation decreasing with increasing urgency level.

13. The method of claim 12, wherein the providing a spoken confirmation step comprises providing a spoken confirmation of the semantic concept identified with at least one of a speaking rate and a pitch that is increased as the urgency level increases.

14. The method of claim 1, wherein the step of using the semantic concept comprises providing a control input for the vehicle only when the semantic confidence level exceeds a threshold.

30. A computer program comprising program instructions that, when executed, cause a computer to operate in accordance with the method of claim 1.

31. A computer program product carrying the computer program of claim 30.

15. An apparatus for controlling a vehicle, comprising:

an input arranged to receive one or more instructions issued as speech;

a memory arranged to store the received speech;

a speech recognition module arranged to analyze the speech to provide a sequence of words and a word confidence measure for each word recognized;

a natural language understanding module arranged to receive the sequence of words and the word confidence measures, and analyze the sequence of words to identify a semantic concept corresponding to an instruction based on the analysis and a semantic confidence level for the semantic concept identified derived at least in part with reference to the word confidence measures of the words associated with the semantic concept;

a response generation module arranged to provide a spoken confirmation of the semantic concept identified based on the semantic confidence level; and

a command generation module arranged to use the semantic concept identified to provide a control input for the vehicle.

16. The apparatus of claim 15, further comprising an acoustic model database operably coupled to the speech recognition module, wherein the acoustic models database has hidden Markov models stored therein, and additionally has optionally continuous hidden Markov models stored therein.

17. The apparatus of claim 15, further comprising a language model database operably coupled to the speech recognition module, wherein the language model database has data stored therein that provides information about the most likely sequences of words expected to be found.

18. The apparatus of claim 17, wherein the language model database has a 2-gram statistical model stored therein.

19. The apparatus of claim 15, wherein the natural language understanding module is arranged to analyze the sequence of words and the associated word confidence levels to identify the one or more semantic concepts.

20. The apparatus of claim 15, wherein the natural language understanding module is arranged to provide the semantic concept as an identifier and a value

21. The apparatus of claim 15, wherein the natural language understanding module is arranged to analyze the sequence of words to identify the one or more semantic concepts using a bottom-up approach starting with an analysis of each word identified individually and then extending the analysis to neighbouring words.

22. The apparatus of claim 15, wherein the natural language understanding module is arranged to analyze the sequence of words to identify the one or more semantic concepts by tagging each word according to at least one of its type, command, digit, and letter, and to look for known patterns from the sequence of tags, thereby allowing deduction of a semantic concept.

23. The apparatus of claim 15, wherein the response generation module is arranged to provide a spoken confirmation of the semantic concept identified to include an indication that the instruction was not understood when the semantic confidence level is below a threshold.

24. The apparatus of claim 15, wherein the response generation module is arranged to provide a spoken confirmation of the semantic concept identified based on the semantic confidence level and an indicated verbosity level.

25. The apparatus of claim 24, wherein the response generation module is arranged to provide a spoken confirmation of the semantic concept identified with at least one of a speaking rate and a pitch that is increased as the indicated verbosity level decreases.

26. The apparatus of claim 15, wherein the response generation module is arranged to provide a spoken confirmation of the semantic concept identified based on the semantic confidence level and an urgency level, the verbosity of the spoken confirmation decreasing with increasing urgency level.

27. The apparatus of claim 26, wherein the response generation module is arranged to provide a spoken confirmation of the semantic concept identified with at least one of a speaking rate and a pitch that is increased as the urgency level increases.

28. The apparatus of claim 15, wherein the response generation module has associated with it a database containing response templates for use in constructing the spoken confirmation.

29. The apparatus of claim 15, wherein the command generation module is arranged to use the semantic concept identified to provide a control input for the vehicle only when the semantic confidence level exceeds a threshold.