Middleware layer between speech related applications and engines

• US 7,177,813 B2
• Filed: 12/15/2004
• Issued: 02/13/2007
• Est. Priority Date: 07/20/2000
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Schmid, Philipp Heinz, Lipe, Ralph, Chambers, Robert, Connell, Edward
• Primary Examiner(s): Hudspeth; David
• Assistant Examiner(s): Sked; Matthew J.
• Application Number:
  US11/012,547
• Publication Number:
  US 20050096911A1
• Time in Patent Office: 790 Days
• Field of Search: None
• US Class Current: 704/270
• CPC Class Codes: G06F 9/4488 Object-oriented
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: H04L 65/1101 Session protocols

Abstract

The present invention provides an application-independent and engine-independent middleware layer between applications and engines. The middleware provides speech-related services to both applications and engines, thereby making it far easier for application vendors and engine vendors to bring their technology to consumers.

Claims

1. A computer-implemented method for facilitating communication between a speech-related application and a speech-related engine, comprising:

providing a middleware layer having a speech component that includes at least one processing component, an application-independent interface, and an engine-independent interface, wherein the application-independent interface and the engine-independent interface are different interfaces;

coupling the application-independent interface to the application so as to facilitate data flow across the application-independent interface between the application and the processing component;

coupling the engine-independent interface to the engine so as to facilitate data flow across the engine-independent interface between the engine and the processing component;

utilizing the processing component to perform speech related services for the application and the engine; and

wherein providing a middleware layer further comprises;

providing a middleware layer having a speech component that includes a format negotiation component and an interface configured to be coupled to an audio device; and

configuring the format negotiation component to negotiate a data format of data used by the audio device and data used by the engine.

2. The method of claim 1 wherein providing a middleware layer further comprises:

providing a middleware layer having a speech component that includes a marshaling component and a plurality of processing components associated with a plurality of different processes; and

configuring the marshaling component to access at least one processing component in each process and to marshal information transfer among the processes.

3. The method of claim 1 wherein providing a middleware layer further comprises:

providing a middleware layer having a speech component that includes a processing component having a lexicon container object; and

configuring the lexicon container object to contain a plurality of lexicons and to provide a lexicon interface to the engine to represent the plurality of lexicons as a single lexicon to the engine.

4. The method of claim 1 wherein providing a middleware layer further comprises:

providing a middleware layer having a speech component that includes a processing component having a site object having an interface configured to receive result information from the engine.

5. The method of claim 1 wherein the engine comprises a text-to-speech (TTS) engine and wherein providing a middleware layer further comprises:

providing a middleware layer having a speech component that includes a processing component having a first object having an application interface and an engine interface.

6. The method of claim 1 wherein the engine comprises a speech recognition (SR) engine and wherein providing a middleware layer further comprises:

providing a middleware layer having a speech component that includes a processing component having a first object having an application interface and an engine interface.

7. The method of claim 6 wherein providing a middleware layer further comprises:

providing a middleware layer having a speech component that includes a processing component having a first object having an application interface that exposes a method configured to receive recognition attributes from the application and instantiate a specific speech recognition engine based on the engine attributes received.

8. The method of claim 6 wherein providing a middleware layer further comprises:

providing a middleware layer having a speech component that includes a processing component having a first object having an application interface that exposes a method configured to receive audio device attributes from the application and instantiate a specific audio device based on the audio device attributes received.

9. The method of claim 6 wherein providing a middleware layer further comprises:

providing a middleware layer having a speech component that includes a processing component having a first object having an application interface that exposes a method configured to receive an alternate request from the application and to configure the speech component to retain alternates provided by the SR engine for transmission to the application based on the alternate request.

10. The method of claim 6 wherein providing a middleware layer further comprises:

providing a middleware layer having a speech component that includes a processing component having a first object having an application interface that exposes a method configured to receive an audio information request from the application and to configure the speech component to retain audio information recognized by the SR engine based on the audio information request.

11. The method of claim 6 wherein providing a middleware layer further comprises:

providing a middleware layer having a speech component that includes a processing component having a first object having an application interface that exposes a method configured to receive bookmark information from the application identifying a position in an input data stream being recognized and to notify the application when the SR engine reaches the identified position.

12. The method of claim 6 wherein providing a middleware layer further comprises:

providing a middleware layer having a speech component that includes a processing component having a first object having an engine interface configured to call the SR engine to set acoustic profile information in the SR engine.

13. The method of claim 6 wherein providing a middleware layer further comprises:

providing a middleware layer having a speech component that includes a processing component having a first object having an engine interface configured to call the SR engine to load a grammar in the SR engine.

14. The method of claim 6 wherein providing a middleware layer further comprises:

providing a middleware layer having a speech component that includes a processing component having a first object having an engine interface configured to call the SR engine to load a language model in the SR engine.

15. The method of claim 6 wherein providing a middleware layer further comprises:

providing a middleware layer having a speech component that includes a processing component having a first object having an application interface that exposes a method configured to receive a grammar request from the application and to instantiate a grammar object based on the grammar request.

16. The method of claim 1 wherein providing a middleware layer further comprises:

providing a middleware layer having a speech component that includes at least one processing component having a site object exposing an engine interface configured to receive information from the engine.

17. A computer-readable medium having instructions embedded therein that, when executed, cause a computing device to execute a series of steps that format data for use by a speech engine and an audio device, the steps comprising:

obtaining, at a middleware layer which facilitates communication between the speech engine and an application, a data format for data used by the engine;

obtaining, at the middleware layer, a data format of data used by the audio device;

determining, at the middleware layer, whether the engine data format and the audio data format are consistent; and

if not, utilizing the middleware layer to attempt to change the data format of the data used by at least one of the engine and the audio device.

18. The computer-readable medium of claim 17 further comprising a step of:

if the attempt to change the data format used by the at least one of the engine and the audio device is unsuccessful, invoking a format converter to change data format for data between the engine and the audio device to ensure the data formats are consistent.

19. A computer-implemented method for facilitating communication between a speech-related application and a speech-related engine, comprising:

providing a middleware layer having a speech component that includes at least one processing component, an application-independent interface, and an engine-independent interface, wherein the application-independent interface and the engine-independent interface are different interfaces;

coupling the application-independent interface to the application so as to facilitate data flow across the application-independent interface between the application and the processing component;

coupling the engine-independent interface to the engine so as to facilitate data flow across the engine-independent interface between the engine and the processing component;

utilizing the processing component to perform speech related services for the application and the engine; and

wherein providing a middleware layer further comprises;

providing a middleware layer having a speech component that includes a processing component having a lexicon container object; and

configuring the lexicon container object to contain a plurality of lexicons and to provide a lexicon interface to the engine to represent the plurality of lexicons as a single lexicon to the engine.

20. A computer-implemented method for facilitating communication between a speech-related application and a speech-related engine, comprising:

providing a middleware layer having a speech component that includes at least one processing component, an application-independent interface, and an engine-independent interface, wherein the application-independent interface and the engine-independent interface are different interfaces;

coupling the application-independent interface to the application so as to facilitate data flow across the application-independent interface between the application and the processing component;

coupling the engine-independent interface to the engine so as to facilitate data flow across the engine-independent interface between the engine and the processing component;

utilizing the processing component to perform speech related services for the application and the engine; and

wherein the engine comprises a speech recognition (SR) engine and wherein providing a middleware layer further comprises;

providing a middleware layer having a speech component that includes a processing component having a first object having an application interface and an engine interface; and

providing a middleware layer having a speech component that includes a processing component having a first object having an application interface that exposes a method configured to receive audio device attributes from the application and instantiate a specific audio device based on the audio device attributes received.