Creating and editing grammars for speech recognition graphically

• US 6,434,523 B1
• Filed: 04/23/1999
• Issued: 08/13/2002
• Est. Priority Date: 04/23/1999
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Monaco, Peter C.
• Primary Examiner(s): Knepper, David D.
• Application Number:
  US09/299,173
• Time in Patent Office: 1,208 Days
• Field of Search: 704/230-235, 704/243-245, 704/255-257, 704/270, 704/275, 704/276, 704/278, 345/348, 345/349, 707/5, 707/100, 707/103
• US Class Current: 704/257
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A graphical tool is provided for allowing a developer of a speech-enabled application to create and edit grammars for speech recognition. The tool provides a user interface that has two editing modes, which the user can quickly and easily switch between. In the first editing mode, the user can specifying a grammar by writing or editing a Grammar Specification Language (GSL) listing. In response to such user inputs, the tool automatically generates or modifies a set of displayable graphical objects representing the grammar. In the second editing mode, the displayable objects are displayed, such that each object represents one or more speech expressions, and the logical relationships between the expressions are graphically represented. The user may edit the set of displayed objects in the second editing mode, including adding or deleting objects, modifying an expression represented by an object, or altering the logical relationships between the expressions. User inputs directed to the objects in the second editing mode are automatically reflected in the GSL listing when the user switches back to the first editing mode.

Claims

1. A computer-implemented method of providing a user interface for allowing a user to edit grammars for speech recognition, the method comprising:

operating in a first editing mode for allowing the user to enter first inputs to specify a first grammar, such that in response to the first inputs, a first set of graphical objects and a corresponding first grammar specification language (GSL) sequence representing the first grammar are generated; and

operating in a second editing mode for allowing the user to enter second inputs to specify a second GSL sequence, the second GSL sequence including a second grammar, such that in response to the second inputs, a second set of graphical objects representing the second GSL sequence is generated.

2. A method as recited in claim 1, wherein each said grammar may comprise a plurality of expressions and logical relationships between the expressions.

3. A method as recited in claim 1, further comprising providing the first and second editing modes such that, in response to a user input specifying a change to the first grammar, both the first GSL sequence and the first set of graphical objects are automatically updated.

4. A method as recited in claim 1, further comprising allowing the user to select either of the first editing mode and the second editing mode.

5. A method as recited in claim 4, further comprising allowing the user to repeatedly switch between the first editing mode and the second editing mode to edit a particular grammar.

6. A computer-implemented method of providing a user interface for allowing a user to create and edit grammars for speech recognition, the method comprising:

receiving first user inputs specifying a first grammar for speech recognition;

in response to the first user inputs, generating a first set of graphical objects representing the first grammar and generating a corresponding first grammar specification language (GSL) sequence representing the first grammar;

receiving second user inputs specifying a second GSL sequence representing a second grammar for speech recognition; and

in response to the second user inputs, generating data representing a second set of graphical objects, the second of graphical objects representing the second grammar.

7. A method as recited in claim 6, wherein each said grammar may comprise a plurality of expressions and logical relationships between the expressions.

8. A computer-implemented method of providing a user interface for allowing a user to create and edit grammars for speech recognition, the method comprising:

receiving first user inputs specifying a modification to a displayed set of graphical objects, the displayed set of graphical objects representing a grammar; and

in response to the first user inputs, concurrently modifying the displayed set of graphical objects and a grammar specification language (GSL) sequence textually representing the grammar.

9. A method as recited in claim 8, further comprising:

receiving second user inputs specifying a modification to the GSL sequence; and

in response to the second user inputs, modifying the GSL sequence and the set of graphical objects.

10. A method as recited in claim 8, wherein the grammar may comprise a plurality of expressions and logical relationships between the expressions.

11. A computer-implemented method of providing a user interface for allowing a user to edit grammars for speech recognition, the method comprising:

receiving first user inputs specifying a modification to a displayed grammar specification language (GSL) sequence, the displayed GSL sequence including a grammar; and

in response to the first user inputs, modifying the displayed GSL sequence and data representing a set of displayable graphical objects, the displayable graphical objects representing the grammar.

12. A method as recited in claim 11, further comprising:

receiving second user inputs specifying a modification to the GSL sequence; and

in response to the second user inputs, modifying the GSL sequence and the set of graphical objects.

13. A method as recited in claim 11, wherein the grammar may comprise a plurality of expressions and logical relationships between the expressions.

14. A computer-implemented method of providing a user interface for allowing a user to create and edit grammars for speech recognition, the method comprising:

receiving first user inputs specifying a first grammar;

in response to the first user inputs, generating a first set of graphical objects representing the first grammar and generating a corresponding first grammar specification language (GSL) sequence representing the first grammar;

displaying the first set of graphical objects;

receiving second user inputs directed to the displayed first set of graphical objects, the second user inputs specifying a modification to the first grammar;

in response to the second user inputs, concurrently modifying the first set of graphical objects and the first GSL sequence according to the second user inputs;

receiving third user inputs directed to the first GSL sequence, the third user inputs specifying a modification to the first GSL sequence;

in response to the third user inputs, concurrently modifying the first GSL sequence and the first set of graphical objects according to the third user inputs.

15. A method as recited in claim 14, wherein each said grammar comprises expressions and logical relationships between the expressions.

16. A method of operating an editor for allowing a user to create and edit grammars for speech recognition, the method comprising:

receiving first user inputs specifying a first grammar specification language (GSL) sequence, the first GSL sequence representing a first set of expressions and logical relationships associated with the first set of expressions;

in response to the first user inputs

displaying the first GSL sequence, and

defining a first set of graphical objects representing the first set of expressions and the logical relationships associated with the first set of expressions; and

displaying the first set of graphical objects according to the logical relationships associated with the first set of expressions.

17. A method as recited in claim 16, further comprising:

receiving second user inputs specifying a modification to the first set of graphical objects; and

in response to the second user inputs, modifying the first GSL sequence.

18. A method as recited in claim 16, further comprising:

receiving second user inputs specifying a modification to the first GSL sequence; and

in response to the second user inputs, modifying the first set of graphical objects.

19. A method as recited in claim 16, further comprising:

receiving second user inputs specifying a modification to the first set of graphical objects;

in response to the second user inputs, modifying the first GSL sequence;

receiving third user inputs specifying a modification to the first GSL sequence; and

in response to the third user inputs, modifying the first set of graphical objects.

20. A computer-implemented method of operating an editor for allowing a user to create and edit grammars for speech recognition, the method comprising:

receiving first user inputs, the first user inputs specifying a first set of expressions for speech recognition and logical relationships associated with the first set of expressions;

in response to the first user inputs

displaying a first set of graphical objects according to the logical relationships associated with the first set of expressions, and

generating a first grammar specification language (GSL) sequence, the first GSL sequence including the first set of expressions and the logical relationships associated with the first set of expressions;

receiving second user inputs specifying second GSL sequence, the second GSL sequence including a second set of expressions for speech recognition and logical relationships associated with the second set of expressions;

in response to the second user inputs

displaying the second GSL sequence, and

defining a second set of graphical objects representing the second set of expressions and the logical relationships associated with the second set of expressions;

displaying the first GSL sequence; and

displaying the second set of graphical objects according to the logical relationships associated with the second set of expressions.

21. A method as recited in claim 20, further comprising:

receiving third user inputs specifying a modification to the first set of graphical objects; and

in response to the third user inputs, modifying the first GSL sequence.

22. A method as recited in claim 20, further comprising:

receiving third user inputs specifying a modification to the first GSL sequence; and

in response to the third user inputs, modifying the first set of graphical objects.

23. A method as recited in claim 20, further comprising:

receiving third user inputs specifying a modification to the first set of graphical objects;

in response to the third user inputs, modifying the first GSL sequence;

receiving fourth user inputs specifying a modification to the first GSL sequence; and

in response to the fourth user inputs, modifying the first set of graphical objects.

24. A method of enabling a remote computer system to provide a user interface to allow a user to create and edit grammars for speech recognition, the method comprising transmitting to a remote computer system sequences of instructions for execution on the remote computer system, the sequences of instructions for configuring the remote computer system to perform the steps of:

receiving first user inputs specifying a first grammar;

generating, in response to the first user inputs, a first set of graphical objects representing the first grammar and generating a corresponding first grammar specification language (GSL) sequence representing the first grammar;

receiving second user inputs specifying a script representing a second grammar; and

generating, in response to the second user inputs, data representing a second set of graphical objects, the second of graphical objects representing the second grammar.

25. A method as recited in claim 24, wherein each said grammar may comprise expressions and logical relationships between the expressions.

26. A device for providing a user interface to allow a user to create and edit grammars for speech recognition, the device comprising:

a GSL editor configured to receive user inputs specifying a listing of a first grammar specification language (GSL) sequence representing a grammar;

a first GSL parser configured to parse the first GSL sequence according to a first GSL syntax to generate a first representation of the grammar, the first GSL parser further configured to modify the first representation in response to inputs specifying a modification to the listing;

a first GSL generator configured to receive the representation and configured to generate the listing of the first GSL sequence based on the first representation, the first GSL generator further configured to automatically update the listing of the first GSL sequence in response to a modification to the first representation;

a graphical object generator configured to receive the first representation and configured to generate a set of graphical objects based on the first representation, the graphical objects for graphically representing the first GSL sequence when displayed, the graphical object generator further configured to automatically update the set of graphical objects in response to a modification to the first representation; and

a graphical editor configured to receive user inputs directed to the set of graphical objects and specifying a modification to the grammar, the first editor further configured to modify the first representation according to the user inputs.

27. A device as recited in claim 26, further comprising:

a second GSL parser configured to parse a second GSL sequence according to a second GSL syntax to generate a second representation of a grammar, the second GSL syntax different from the first GSL syntax, the second GSL parser further configured to modify the second representation in response to inputs specifying a modification to a listing of the second GSL sequence; and

a second GSL generator configured to receive the second representation and to generate the listing of the second GSL sequence based on the second representation, the second GSL generator further configured to automatically update the listing of the second GSL sequence in response to a modification to the second representation.

28. A method as recited in claim 27, wherein:

the second GSL generator is further configured to receive the first representation and to generate the listing of the first GSL sequence based on the first representation according to the second GSL syntax; and

the first GSL generator is further configured to receive the second representation and to generate the listing of the second GSL sequence based on the second representation according to the first GSL syntax.

29. A computer system comprising:

a processor;

a display device coupled to the processor; and

a storage device coupled to the processor and having instructions stored therein which configure the computer system for

operating in a first editing mode for allowing the user to enter first inputs to specify a first grammar, such that in response to the first inputs, a first set of graphical objects representing the first grammar and a corresponding first grammar specification language (GSL) sequence representing the first grammar are generated; and

operating in a second editing mode for allowing the user to enter second inputs to specify second GSL sequence, the second GSL sequence including a second grammar, such that in response to the second inputs, a second set of graphical objects representing the second GSL sequence is automatically generated.

30. A computer system as recited in claim 29, wherein each said grammar comprises expressions and logical relationships between the expressions.

31. A computer system as recited in claim 29, wherein the instructions further comprise instructions which configure the computer system for providing the first and second editing modes such that, in response to a user input specifying a change to the first grammar, both the first GSL sequence and the first set of graphical objects are automatically updated.

32. A computer system as recited in claim 29, wherein the instructions further comprise instructions which configure the computer system for allowing the user to select either of the first editing mode and the second editing mode.

33. A computer system as recited in claim 32, wherein the instructions further comprise instructions which configure the computer system for allowing the user to repeatedly switch between the first editing mode and the second editing mode to edit a particular grammar.

34. A device as recited in claim 33, wherein each said grammar comprises expressions and logical relationships between the expressions.

35. A device for providing a user interface to allow a user to create and edit grammars for speech recognition, the device comprising:

means for receiving first user inputs specifying a first grammar;

means for generating, in response to the first user inputs, a first set of graphical objects representing the first grammar and generating a corresponding first grammar specification language (GSL) sequence representing the first grammar;

means for receiving second user inputs specifying a script representing a second grammar; and

means for generating, in response to the second user inputs, data representing a second set of graphical objects, the second of graphical objects representing the second grammar.

36. A machine readable program storage medium having stored therein sequences of instructions which, when executed by a computer system, cause the computer system to:

receive first user inputs specifying a first grammar for speech recognition;

generate, in response to the first user inputs, a first set of graphical objects representing the first grammar and generating a corresponding first grammar specification language (GSL) sequence representing the first grammar;

receive second user inputs specifying a script representing a second grammar for speech recognition; and

generate, in response to the second user inputs, data representing a second set of graphical objects, the second of graphical objects representing the second grammar.

37. A machine readable program storage medium as recited in claim 36, wherein each said grammar comprises expressions and logical relationships between the expressions.

38. A computer system comprising:

a processor;

a display device coupled to the processor; and

a storage device coupled to the processor and having instructions stored therein which configure the computer system for

receiving first user inputs, the first user inputs specifying a first set of expressions and logical relationships associated with the first set of expressions;

in response to the first user inputs

displaying a first set of graphical objects according to the logical relationships associated with the first set of expressions, and

generating a first grammar specification language (GSL) sequence, the first GSL sequence including the first set of expressions and the logical relationships associated with the first set of expressions;

receiving second user inputs specifying second GSL sequence, the second GSL sequence including a second set of expressions and logical relationships associated with the second set of expressions;

in response to the second user inputs

displaying the second GSL sequence, and

defining a second set of graphical objects representing the second set of expressions and the logical relationships associated with the second set of expressions;

displaying the first GSL sequence; and

displaying the second set of graphical objects according to the logical relationships associated with the second set of expressions.

39. A computer system as recited in claim 38, wherein the instructions further comprise instructions which configure the computer system for:

receiving third user inputs specifying a modification to the first set of graphical objects; and

in response to the third user inputs, modifying the first GSL sequence.

40. A computer system as recited in claim 38, wherein the instructions further comprise instructions which configure the computer system for:

receiving third user inputs specifying a modification to the first GSL sequence; and

in response to the third user inputs, modifying the first set of graphical objects.

41. A computer system as recited in claim 38, wherein the instructions further comprise instructions which configure the computer system for:

receiving third user inputs specifying a modification to the first set of graphical objects;

in response to the third user inputs, modifying the first GSL sequence;

receiving fourth user inputs specifying a modification to the first GSL sequence; and

in response to the fourth user inputs, modifying the first set of graphical objects.