Relative delta computations for determining the meaning of language inputs

• US 8,630,856 B2
• Filed: 07/08/2009
• Issued: 01/14/2014
• Est. Priority Date: 10/01/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Balchandran, Rajesh, Boyer, Linda M.
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): Serrou, Abdelali
• Application Number:
  US12/499,544
• Publication Number:
  US 20100010805A1
• Time in Patent Office: 1,651 Days
• Field of Search: 704/251, 704/257, 704/254, 704/9, 704/E15.001, 704/E15.004
• US Class Current: 704/251
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A method for processing language input can include the step of determining at least two possible meanings for a language input. For each possible meaning, a probability that the possible meaning is a correct interpretation of the language input can be determined. At least one relative data computation can be computed based at least in part upon the probabilities. At least one irregularity within the language input can be detected based upon the relative delta computation. The irregularity can include mumble, ambiguous input, and/or compound input. At least one programmatic action can be performed responsive to the detection of the irregularity.

Claims

1. A computer implemented method for processing language input comprising the steps of:

using at least one processor to perform;

determining at least two possible meanings for a language input;

determining a first probability that a first possible meaning of the at least two possible meanings is a correct interpretation of said language input;

determining a second probability that a second possible meaning of the at least two possible meanings is a correct interpretation of said language input

computing at least one relative delta computation comprising a value derived at least in part on a difference between at least the first probability and the second probability, the difference divided by a denominator based on the first probability;

detecting at least one irregularity within said language input based upon said relative delta computation; and

performing at least one programmatic action responsive to detecting said irregularity.

2. The method of claim 1, wherein said irregularity comprises mumble.

3. The method of claim 1, wherein said irregularity comprises ambiguous input.

4. The method of claim 1, wherein said irregularity comprises compound input.

5. The method of claim 1, wherein said at least one irregularity detected by said method comprises two or more irregularities, said method further comprises the step of:

differentiating between different ones of said irregularities such that different programmatic actions are performed responsive to detecting different irregularities.

6. The method of claim 1, wherein said programmatic action comprises the step of prompting a user for further input to resolve said irregularity.

7. The method of claim 1, wherein, in computing said at least one relative delta computation;

said difference is between the first probability and the second probability; and

the denominator is equal to the first probability.

8. The method of claim 1, further comprising the step of receiving a top-n list of possible meanings for said language input, wherein said determining of possible meanings further comprises selecting at least two possible meanings contained within said top-n list.

9. The method of claim 8, wherein said possible meanings of said top-n list are each associated with a probability that said possible meaning is a correct interpretation of said language input, said selecting step further comprises the step of selecting said two possible meanings having the highest probabilities.

10. The method of claim 1, wherein said at least two possible meanings comprise at least three possible meanings, said method further comprises ordering said possible meanings according to the first and second probabilities and a third probability that a third possible meaning of the at least three possible meanings is a correct interpretation of said language input, wherein said computing step further comprises the steps of:

computing a first relative delta computation based upon two sequentially ordered ones of said possible meanings; and

computing a second relative delta computation based upon two different sequentially ordered ones of said ordered meanings, wherein said detecting of said irregularity is based upon said first relative delta computation and said second relative delta computation.

11. The method of claim 1, wherein said first and second probabilities are confidence values, said method further comprising the steps of:

receiving a plurality of language inputs;

for each language input, determining at least two possible meanings and associated confidence values;

plotting at least a portion of said confidence values on a graph; and

determining at least one threshold from said graph, wherein said relative delta computation is compared against said threshold when detecting said irregularity.

12. The method of claim 11, further comprising the step of:

presenting said graph within a graphical user interface; and

automatically adjusting said threshold using controls of said graphical user interface.

13. A system for processing language input comprising:

a computer processor configured to convert a language input into a plurality of possible meanings and to assign a confidence value to each of the plurality of possible meanings to obtain a plurality of confidence values; and

an anomaly detector configured to detect the occurrence of at least one irregularity within said language input using at least one relative delta computation, wherein said relative delta computation comprises a value derived at least in part on a difference between a first confidence value and a second confidence value from the plurality of confidence values, the difference divided by a denominator based on the first confidence value.

14. The system of claim 13, further comprising:

a training engine configured to generate thresholds used by said anomaly detector to detect said irregularities.

15. The system of claim 13, wherein said anomaly detector further comprises a relative delta score calculator configured to calculate relative delta computations.

16. The system of claim 13, wherein said anomaly detector further comprises a relative delta score interpreter configured to compare relative delta computations to threshold values to detect said irregularities.

17. The system of claim 13, wherein said anomaly detector further comprises a learning module configured to adjust threshold values base upon operational behavior.

18. Hardware having stored thereon, a computer program having a plurality of code sections, said code sections executable by a machine for causing the machine to perform the steps of:

determining at least two possible meanings for a language input;

determining a first probability that a first possible meaning of the at least two possible meanings is a correct interpretation of said language input;

determining a second probability that a second possible meaning of the at least two possible meanings is a correct interpretation of said language input;

computing at least one relative delta computation comprising a value derived at least in part on a difference between at least the first probability and the second probability, the difference divided by a denominator based on the first probability;

detecting at least one irregularity within said language input based upon said relative delta computation; and

performing at least one programmatic action responsive to detecting said irregularity.

19. The hardware of claim 18, wherein said irregularity comprises mumble.

20. The hardware of claim 18, wherein said irregularity comprises ambiguous input.

21. The hardware of claim 18, wherein said irregularity comprises compound input.

22. The hardware of claim 18, wherein said at least one irregularity detected by said method comprises two or more irregularities, said method further comprises the step of:

differentiating between different ones of said irregularities such that different programmatic actions are performed responsive to detecting different irregularities.

23. The hardware of claim 18, wherein said programmatic action comprises the step of prompting a user for further input to resolve said irregularity.

24. The hardware of claim 18, wherein, in computing said at least one relative delta computation:

said difference is between the first probability and the second probability; and

the denominator is equal to the first probability.

25. The hardware of claim 18, further comprising the step of receiving a top-n list of possible meanings for said language input, wherein said determining of possible meanings further comprises selecting at least two possible meanings contained within said top-n list.

26. The hardware of claim 25, wherein said possible meanings of said top-n list are each associated with a probability that said possible meaning is a correct interpretation of said language input, said selecting step further comprises the step of selecting said two possible meanings having the highest probabilities.

27. The hardware of claim 18, wherein said at least two possible meanings comprise at least three possible meanings, said method further comprises ordering said possible meanings according to the first and second probabilities and a third probability that a third possible meaning of the at least three possible meanings is a correct interpretation of said language input, wherein said computing step further comprises the steps of:

computing a first relative delta computation based upon two sequentially ordered ones of said possible meanings; and

computing a second relative delta computation based upon two different sequentially ordered ones of said ordered meanings, wherein said detecting of said irregularity is based upon said first relative delta computation and said second relative delta computation.

28. The hardware of claim 18, wherein said first and second probabilities are confidence values, said method further comprising the steps of:

receiving a plurality of language inputs;

for each language input, determining at least two possible meanings and associated confidence values;

plotting at least a portion of said confidence values on a graph; and

determining at least one threshold from said graph, wherein said relative delta computation is compared against said threshold when detecting said irregularity.

29. The hardware of claim 28, further comprising the step of:

presenting said graph within a graphical user interface; and

automatically adjusting said threshold using controls of said graphical user interface.

30. A system for processing language input comprising:

means for determining at least two possible meanings for a language input;

means for determining a first probability, that a first possible meaning of the at least two possible meanings is a correct interpretation of said language input;

means for determining a second probability that a second possible meaning of the at least two possible meanings is a correct interpretation of said language input;

means for computing at least one relative delta computation comprising a value derived at least in part on a difference between at least the first probability and the second probability, the difference divided by a denominator based on the first probability;

means for detecting at least one irregularity within said language input based upon said relative delta computation; and

means for performing at least one programmatic action responsive to detecting said irregularity.