Systems and Methods for Generating Markup-Language Based Expressions from Multi-Modal and Unimodal Inputs

• US 20100100509A1
• Filed: 12/22/2009
• Published: 04/22/2010
• Est. Priority Date: 10/24/2002
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: AT&T Corporation (AT&T, Inc.)
• Inventors: Bangalore, Srinivas, Johnston, Michael J.
• Granted Patent: US 8,433,731 B2
• Time in Patent Office: Days
• US Class Current: 706/11
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06F 40/151 Transformation
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

When using finite-state devices to perform various functions, it is beneficial to use finite state devices representing regular grammars with terminals having markup-language-based semantics. By using markup-language-based symbols in the finite state devices, it is possible to generate valid markup-language expressions by concatenating the symbols representing the result of the performed function. The markup-language expression can be used by other applications and/or devices. Finite-state devices are used to convert strings of words and gestures into valid markup-language, for example, XML, expressions that can be used, for example, to provide an application program interface to underlying system applications.

Claims

1. A computing device that generates markup-language-based expressions from input, the computing device comprising:

a module configured to receive at least one data input in at least one input mode of a recognition device;

a module configured to generate at least one input lattice representing the at least one data input;

a module configured to use at least one finite-state device to perform a function using the at least one generated input lattice, the finite-state device having symbols based on markup-language semantics, wherein performing the function comprises mapping a first markup-language expression to a second markup-language expression using a transducer relating the first-markup-language expression to the second-markup-language expression;

a module configured to generate a result finite-state device representing a result of the performed function, the result finite-state device relating at least one set of input symbols to at least one set of output symbols;

a module configured to generate the at least one markup-language expression; and

a module configured to output the at least one markup-language expression to a processor.