Method and apparatus for transducer-based text normalization and inverse text normalization

• US 20060069545A1
• Filed: 09/10/2004
• Published: 03/30/2006
• Est. Priority Date: 09/10/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Jiang, Li, Wu, Qiang, Morton, Rachel I.
• Granted Patent: US 7,630,892 B2
• Time in Patent Office: Days
• US Class Current: 704/8
• CPC Class Codes: G06F 40/151 Transformation
• CPC Class Codes: G06F 40/205 Parsing

Abstract

A method and apparatus are provided that perform text normalization and inverse text normalization using a single grammar. During text normalization, a finite state transducer identifies a second string of symbols from a first string of symbols it receives. During inverse text normalization, the context free transducer identifies the first string of symbols after receiving the second string of symbols.

Claims

1. A method for performing text normalization and inverse text normalization, the method comprising:

receiving a first string of symbols;

using a grammar to identify a second string of symbols from the first string of symbols;

providing the second string of symbols as the text normalization of the first string of symbols;

receiving the second string of symbols;

using the grammar to identify the first string of symbols from the second string of symbols; and

providing the first string of symbols as the inverse text normalization of the second string of symbols.

2. The method of claim 1 wherein the grammar provides an entry that associates the first string of symbols with the second string of symbols.

3. The method of claim 1 wherein using a grammar to identify a second string of symbols comprises forming a context free transducer network from the grammar and using the network to identify the second string of symbols from the first string of symbols.

4. The method of claim 3 wherein using the grammar to identify the first string of symbols from the second string of symbols comprises using the network to identify the first string of symbols from the second string of symbols.

5. The method of claim 1 wherein using the grammar to identify a second string of symbols comprises using the grammar to identify a plurality of strings of symbols that include the second string of symbols.

6. The method of claim 5 wherein providing the second string of symbols comprises providing the plurality of strings of symbols, each string of symbols in the plurality of strings of symbols forming a possible text normalization of the first string of symbols.

7. The method of claim 6 further comprising:

receiving a third string of symbols that is in the plurality of strings of symbols;

using the grammar to identify the first string of symbols from the third string of symbols; and

providing the first string of symbols as the inverse text normalization of the third string of symbols.

8. A computer-readable medium having computer-executable instructions for performing steps comprising:

performing text normalization using a context free transducer and a grammar;

performing inverse text normalization using a context free transducer and the grammar.

9. The computer-readable medium of claim 8 wherein the grammar includes an entry comprising a first string of symbols and a second string of symbols.

10. The computer-readable medium of claim 9 wherein performing text normalization comprises matching a text string to the first string of symbols and providing the second string of symbols as output.

11. The computer-readable medium of claim 10 wherein performing inverse text normalization comprises matching a text string to the second string of symbols and providing the first string of symbols as output.

12. The computer-readable medium of claim 8 wherein performing text normalization comprises generating a lattice of output strings of symbols for an input string of symbols.

13. The computer-readable medium of claim 8 wherein performing inverse text normalization comprises generating a lattice of output strings of symbols for an input string of symbols.

14. The computer-readable medium of claim 8 wherein the grammar comprises a set of rules and wherein at least one rule references another rule in the set of rules.

15. A computer-readable medium having computer-executable instructions for performing steps comprising:

providing an input text and a context free transducer network to a context free transducer that is capable of performing text normalization or inverse text normalization using the context free transducer network; and

instructing the context free transducer to perform one of text normalization or inverse text normalization.

16. The computer-readable medium of claim 15 wherein the context free transducer network comprises a network of transitions, each transition having at least one spoken form of a text string and at least one display form of a text string.

17. The computer-readable medium of claim 16 wherein instructing the context free transducer to perform text normalization causes the context free transducer to return the spoken form of the text string if at least a portion of the input text matches the display form of the text string.

18. The computer-readable medium of claim 17 wherein instructing the context free transducer to perform inverse text normalization causes the context free transducer to return the display form of the text string if at least a portion of the input text matches the spoken form of the text string.

19. The computer-readable medium of claim 16 wherein the context free transducer produces multiple spoken forms of the text string when performing text normalization.

20. The computer-readable medium of claim 17 wherein the context free transducer produces multiple display forms of the text string when performing inverse text normalization.

21. The computer-readable medium of claim 16 wherein a transition further comprises a property.

22. The computer-readable medium of claim 21 wherein the property comprises a name entity property.