Phonetic data processing system and method

• US 20010049601A1
• Filed: 03/23/2001
• Published: 12/06/2001
• Est. Priority Date: 03/24/2000
• Status: Active Grant

• Current Assignee: Eliza Corporation (Gainwell Technologies)
• Original Assignee: Eliza Corporation (Gainwell Technologies)
• Inventors: Kroeker, John, Boulanov, Oleg, Yelpatov, Andrey
• Granted Patent: US 6,895,377 B2
• Time in Patent Office: Days
• US Class Current: 704/254
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/14 using statistical models, e.g. Hidden Markov Models [HMMs] G10L15/18 takes precedence
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/193 Formal grammars, e.g. finite state automata, context free grammars or word networks
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

A phonetic data processing system processes phonetic stream data to produce a set of semantic data, using a context-free rich semantic grammar database (RSG DB) that includes a grammar tree, comprised of sub-trees, representing words and phrases. A phonetic searcher accepts the phonetic estimates and searches the RSG DB to produce a best word list, which is processed by a semantic parser, using the RSG DB, to produce a semantic tree instance, including all valid interpretations of the phonetic stream. An application accesses a semantic tree evaluator to interpret the semantic tree instance according to a context to produce a final linguistic interpretation of the phonetic stream, which is returned to the application.

Claims

1. A phonetic data processing system comprising:

A. a computer processing device having access to a memory;

B. a rich semantic grammar (RSG) stored in said memory and comprising one or more grammars comprising syntactic information and semantic information; and

C. a phonetic data processing module, executable by said processing device, said module comprising;

(1) a phonetic searcher, configured to generate, as a function of said RSG and a received phonetic stream comprised of phonetic estimates, a set of sequences comprising a set of best words from said RSG corresponding to said phonetic estimates; and

(2) a semantic parser module, configured to generate a set of semantic data from said sequences and said RSG, wherein said set of semantic data includes all valid interpretations of the sequences.

2. A system as in

claim 1

, wherein each of said sequences comprises set of words combined to define word paths.

3. A system as in

claim 1

, wherein said phonetic searcher is configured to extract a context free grammar (CFG) comprising syntactic information from said RSG and is further configured to access said CFG to generate said set of best words.

4. A system as in

claim 1

, wherein each of said phonetic estimates has a fixed start time and a plurality of end times, and wherein there is a score associated with each end time corresponding to the likelihood that a given phonetic estimate is a word or a syllable in said RSG.

5. A system as in

claim 4

, wherein said phonetic searcher includes a grammar builder configured to selectively combine words from said set of best words into sequences, as a function of the start time and end times of said phonetic estimates corresponding to said words.

6. A system as in

claim 5

, said phonetic searcher is configured to combine said words using dynamic programming.

7. A system as in

claim 1

, wherein said RSG is a grammar tree comprising nodes having certain of said syntactic and semantic information associated with each of a plurality of said nodes.

8. A system as in

claim 1

, wherein said semantic information includes one or more categories, and each category dictates an interpretation of a corresponding word.

9. A system as in

claim 1

, wherein said semantic information includes one or more operators, and each operator defines a manner of combing a plurality of words.

10. A system as in

claim 1

, further comprising:

(3) an application program, configured to receive said set of semantic data and to define context information associated with said phonetic stream;

(4) a semantic evaluator, configured to interpret said set of semantic data in accordance with said context information and to derive a linguistic result therefrom.

11. A system as in

claim 10

, wherein said set of semantic data is represented as a tree of nodes representing all valid interpretations of said word sequences and said semantic evaluator is configured to determine a category at each node, as a function of said context information, and to apply to values at each node a corresponding category to determine said linguistic result.

12. A system as in

claim 1

, wherein said set of semantic data is a semantic tree comprised of a set of nodes representing all valid interpretations of said sequences.

13. A method of processing phonetic data, comprising:

A. defining in a memory a context free rich semantic grammar (RSG) comprising syntactic and semantic information;

B. receiving a phonetic stream comprising phonetic estimates;

C. generating a set of sequences comprised of best words, as a function of said RSG and said phonetic stream;

D. generating, from said sequences and said RSG, a set of semantic data including all valid interpretations of said sequences.

14. A method as in

claim 13

, wherein each of said sequences is comprised of a set of words combined to define word paths.

15. A method as in

claim 13

, wherein said phonetic searching includes extracting a context free grammar comprising syntactic information from said RSG.

16. A method as in

claim 13

, wherein said phonetic searching includes:

(1) determining for each of said phonetic estimates a fixed start time and a plurality of end times; and

(2) determining a score associated with each end time corresponding to the likelihood that a given phonetic estimate is a word or a syllable in said RSG.

17. A method as in

claim 16

, wherein said phonetic searching further includes:

(3) combining said words from said set of best words into said sequences, as a function of the start time and the end times of said phonetic estimates corresponding to said words.

18. A method as in

claim 16

, wherein said phonetic searching includes:

(3) combining said words using dynamic programming.

19. A method as in

claim 13

, wherein said RSG is a grammar tree comprising nodes having certain of said syntactic and semantic information associated with each of said nodes.

20. A method as in

claim 13

, further including:

E. defining a context associated with said phonetic stream by an application program; and

F. interpreting said set of semantic data with a semantic evaluator, in accordance with said context, and deriving a linguistic result therefrom.

21. A method as in

claim 20

, wherein interpreting said set of semantic data includes, for each node, determining a category at a given node, as a function of said context, and applying said category to interpret values at said given node.

22. A method as in

claim 13

, wherein generating said set of semantic data includes generating a semantic tree instance comprised of a set of nodes representing all valid interpretations of said sequences.

23. A phonetic searcher, coupled to a database comprising a context free grammar including syntactic information, and configured to receive a phonetic stream of data and to generate a word list representing all valid words represented by said phonetic stream, as a function of said RSG.

24. A grammar builder coupled to a database comprising a context free grammar, including syntactic information, and coupled to a database comprising a set of valid words corresponding to a stream of phonetic estimates, wherein each of said phonetic estimates is represented as having a fixed start time and a plurality of end times, and wherein said grammar builder is configured to selectively combine words from said set of words as a function of said fixed start time and said end times of phonetic estimates corresponding said words to be combined.

25. A grammar builder as in

claim 24

, wherein for each phonetic estimate, a score is associated with each of said end times, and wherein said score is related to a probability that said phonetic estimate is a certain word or syllable from said context free grammar.

26. A semantic parser coupled to a database having a context free rich semantic grammar (RSG) and configured to generate a set of semantic data as a function of a set of word sequences and said RSG, wherein said sequences include words derived from a phonetic stream and said RSG, and said set of semantic data includes all valid interpretations of the sequences.

27. A semantic parser as in

claim 26

, wherein the RSG is a grammar tree comprised of a plurality of nodes, one or more of said nodes including syntactic information and semantic information.

28. A semantic parser as in

claim 26

, wherein said set of semantic data is a semantic tree.

29. A semantic tree evaluation tool coupled to an application program that defines a context and coupled to a memory including a semantic tree representing all valid interpretations of a phonetic stream, said evaluation tool configured to generate a linguistic result as a single valid interpretation of said phonetic stream, in accordance with said context.

30. A semantic evaluation tool as in

claim 29

, wherein said semantic tree is comprised of a plurality of nodes including semantic information, and wherein said semantic evaluation tool is configured to determine at each of said nodes a category, as a function of said context, and to apply to values at each node a corresponding category to determine said linguistic result.