Computer method and apparatus for translating text to sound

• US 6,076,060 A
• Filed: 05/01/1998
• Issued: 06/13/2000
• Est. Priority Date: 05/01/1998
• Status: Expired due to Term

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Compaq Computer Corporation (HP Inc.)
• Inventors: Kopec, Thomas, Lin, Ginger Chun-Che
• Primary Examiner(s): Zele, Krista
• Assistant Examiner(s): Opsasnick, Michael N.
• Application Number:
  US09/071,441
• Time in Patent Office: 774 Days
• Field of Search: 704/258, 704/260, 704/269
• US Class Current: 704/260
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination

Abstract

A computer method and apparatus provide fast and efficient conversion (translation) of text to phonemes. The method and apparatus employ a plurality of rule sets, each formed of rules designed for specific portions of an input text string. A suffix rule set is used to match substrings from the end of an input text string to suffix rules. A prefix rule set is used to match substrings from the beginning of the input text to prefix rules. And an infix rule set is provided to match substrings taken from the middle of the input text or any remaining text not matched by either the suffix of prefix rules. Phonemic data is produced for any portion of the input text that matches a particular rule. The phonemic data may be used by a speech synthesizer to vocalize or read aloud the input text. Dictionary lookup of any portions of the input text string in conjunction with rule matching is also provided.

Claims

1. In a digital processing system, a method for creating phonemic data from text, comprising the steps of:

receiving input text;

providing a plurality of rule sets, one rule set for processing one portion of the input text and different rule sets for processing respective different portions of the input text, each rule set having one or more rules for processing a respective portion of the input text;

for each rule set, iteratively applying the rules of the rule set to the input text by comparing the input text with at least one of the rules of the rule set to produce respective phonemic data portions corresponding to portions of the input text, different rule sets producing different phonemic data portions; and

combining the produced phonemic data portions to form a phonemic data sequence corresponding to the input text.

2. The method of claim 1, wherein:

the plurality of rule sets includes a suffix rule set having a multiplicity of suffix rules, and wherein the applying step iteratively compares the input text with the suffix rules in the suffix rule set to produce ending portions of the phonemic data sequence; and

wherein a remainder of the input text is compared with a remainder of the plurality of rule sets to produce a remainder portion of the phonemic data sequence.

3. The method of claim 2 wherein in the applying step, suffix rules are compared with the input text beginning at a rightmost part of the input text and the comparison with each suffix rule is performed in a right to left direction with respect to the input text.

4. The method of claim 2 further comprising the step of performing a dictionary lookup on the remainder of the input text after the input text has been iteratively compared with the suffix rules.

5. The method of claim 1, wherein:

the plurality of rule sets includes a prefix rule set having multiple prefix rules, and wherein the applying step iteratively compares the input text with prefix rules in the prefix rule set to produce beginning portions of the phonemic data sequence; and

wherein a remainder of the input text is compared with a remainder of the plurality of rule sets to produce a remainder portion of the phonemic data sequence.

6. The method of claim 5 wherein in the applying step, prefix rules are compared with the input text beginning at a leftmost part of the input text and the comparison with each prefix rule is performed in a left to right direction with respect to the input text.

7. The method of claim 5 further comprising the step of performing a dictionary lookup on the remainder of the input text after the text input has been compared with the prefix rules.

8. The method of claim 1, wherein:

the plurality of rule sets includes an infix rule set having a multiplicity of infix rules, and wherein the applying step iteratively compares the input text with infix rules in the infix rule set to produce middle portions of the phonemic data sequence; and

wherein a remainder of the input text is compared with a remainder of the plurality of rule sets to produce a remainder portion of the phonemic data sequence.

9. The method of claim 8 wherein in the applying step, infix rules are compared with the input text beginning at a rightmost part of the input text and the comparison with each infix rule is performed in a right to left direction with respect to the input text, and wherein infix rules are compared with the input text beginning at a leftmost part of the input text and the comparison with each infix rule is performed in a left to right direction with respect to the input text.

10. The method of claim 8 further comprising the step of performing a dictionary lookup on the remainder part of the input text after the input text has been compared with the infix rules.

11. The method of claim 1 further comprising the step of:

performing a dictionary lookup on the input text after the receiving input text step.

12. The method of claim 1, wherein the step of providing a plurality of rule sets includes providing a suffix rule set, a prefix rule set and an infix rule set, the suffix rule set having text-to-phonemic data rules for ending portions of input text, the prefix rule set having text-to-phonemic data rules for beginning portions of the input text, and the infix rule set having text-to-phonemic data rules for intermediate portions of the input text;

and

wherein the step of applying the rules further comprises the steps of;

iteratively comparing the input text to the rules in the suffix rule set to ultimately produce phonemic data ending portions based on ending portions of the input text and to produce a first remainder text excluding the ending portions of the input text;

iteratively comparing the first remainder text to the rules in the prefix rule set to produce phonemic data beginning portions based on beginning portions of the first remainder text and to ultimately produce a second remainder text excluding the beginning portions of the first remainder text;

iteratively comparing the second remainder text to the rules in the infix rule set to produce phonemic data middle portions based on intermediate portions of the input text; and

such that the step of combining combines the phonemic data beginning portions, the phonemic data middle portions and the phonemic data ending portions to produce the phonemic data sequence which phonetically represents the input text.

13. The method of claim 12, wherein the step of comparing the input text to the rules in the suffix rule set compares the input text beginning at a rightmost part of the input text and compares in a right to left direction, with respect to the input text, against each rule of the suffix rule set.

14. The method of claim 13, wherein the step of comparing the first remainder text to the rules in the prefix rule set compares the first remainder text beginning at a leftmost part of the first remainder text and compares in a left to right direction, with respect to the first remainder text, against each rule of the prefix rule set.

15. The method of claim 14, wherein the step of comparing the second remainder text to the rules in the infix rule set compares the second remainder text with each rule of the infix rule set beginning from a rightmost part of the second remainder text and compares in a right to left direction to obtain the phonemic data middle portions, and compares the second remainder text with each rule of the infix rule set beginning from a leftmost part of the second remainder text and compares in a left to right direction to obtain the phonemic data middle portions, wherein the right to left and left to right comparisons are performed in parallel.

16. The method of claim 15, wherein:

the step of providing a plurality of rule sets further includes, for each of the rule sets, arranging the rules of the rule set in order according to length of text to which the rule applies, from largest in length to smallest in length, and arranging in a predefined order the rules applying to equivalent lengths of text; and

wherein each comparing step compares one of input text and remainder text against a respective rule set beginning with a rule of the rule set that applies to a largest length of text encompassed by length of the input text or remainder text being compared.

17. The method of claim 13, wherein the step of comparing the second remainder text to the infix rule set compares the second remainder text with each rule of the infix rule set beginning from a rightmost part of the second remainder text and compares in a right to left direction to obtain the phonemic data middle portions, and compares the second remainder text with each rule of the infix rule set beginning from a leftmost part of the second remainder text and compares in a left to right direction to obtain the phonemic data middle portions, wherein the right to left and left to right comparisons are performed in parallel.

18. The method of claim 12, wherein the step of comparing the first remainder text to the prefix rule set compares the first remainder text beginning at a leftmost part of the first remainder text and compares in a left to right direction, with respect to the first remainder text, against each rule of the prefix rule set.

19. The method of claim 18, wherein the step of comparing the second remainder text to the infix rule set compares the second remainder text with each rule of the infix rule set beginning from a rightmost part of the second remainder text and compares in a right to left direction to obtain the phonemic data middle portions, and compares the second remainder text with each rule of the infix rule set beginning from a leftmost part of the second remainder and compares in a left to right direction to obtain the phonemic data middle portions, wherein the right to left and left to right comparisons are performed in parallel.

20. The method of claim 12, wherein the step of comparing the second remainder text to the infix rule set compares the second remainder text with each rule of the infix rule set beginning from a rightmost part of the second remainder text and compares in a right to left direction to obtain the phonemic data middle portions, and compares the second remainder text with each rule of the infix rule set beginning from a leftmost part of the second remainder text and compares in a left to right direction to obtain the phonemic data middle portions, wherein the right to left and left to right comparisons are performed in parallel.

21. The method of claim 12, wherein:

the step of providing a plurality of rule sets further includes, for each of the rule sets, arranging the rules of the rule set in order according to length of text to which the rule applies from largest in length to smallest in length, and arranging in a predefined order the rules applying to equivalent lengths of text; and

wherein each comparing step compares one of input text and remainder text against a respective rule set beginning with a rule of the rule set that applies to a largest length of text encompassed by length of the input text or remainder text being compared.

22. The method of claim 1, wherein:

the step of providing a plurality of rule sets includes, for each of the rule sets, arranging the rules of a rule set in order according to length of text which the rule applies from largest in length to smallest in length, and arranging in alphabetical order the rules applying to equivalent lengths of text; and

wherein the step of applying the rules of a respective rule set includes comparing the input text against the rule set beginning with a rule of the rule set that applies to a largest length of text encompassed by length of the input text.

38. The method of claim 12, wherein before performing the step of iteratively comparing the second remainder text to rules in the infix rule set to produce phonemic data middle portions, the method performs the steps of:

detecting and delimiting a largest group of infix rule matching characters existing within the second remainder text to create at least one third remainder; and

comparing the at least one third remainder to rules of the infix rule set to produce phonemic data middle portions before comparing the largest group of infix rule matching characters to produce phonemic data middle portions, to avoid splitting the second remainder portion into multiple third remainders.

23. A method of translating an incoming text string to corresponding phonemic data representing the input text string for use in a speech synthesizer, comprising the steps of:

in a computer medium, providing a suffix rule set, a prefix rule set and an infix rule set, each rule set having a respective plurality of rules for specifying phonemes for respective text strings;

in a digital processor, comparing substrings of an incoming text string to rules of the suffix rule set and when a match is found, placing phonemes specified by a matching rule into a work storage area and modifying the incoming text string by effectively removing from the incoming text string the substring of the incoming text string which matched the rule in the suffix rule set, such that a modified incoming text string is formed;

for each time the incoming text string is modified, repeating the comparing step with the modified incoming text string as last modified and the suffix rule set until there are no more matches for the suffix rule set in the modified incoming text string as last modified;

comparing substrings of the incoming text string to rules of the prefix rule set and when a match is found, placing phonemes specified by a matching rule into the work storage area and modifying the incoming text string by effectively removing from the incoming text string the substring of the incoming text string which matched the rule in the prefix rule set such that a modified incoming text string is formed;

for each time the incoming text string is modified, repeating the comparing step with the modified incoming text string as last modified and the prefix rule set until there are no more matches for the prefix rule set in the modified incoming text string as last modified;

comparing substrings of the incoming text string to rules of the infix rule set and when a match is found comparing substrings of the incoming text string to rules of the infix rule set and when a match is found, placing phonemes specified by a matching rule into the work storage area and modifying the incoming text string by effectively removing from the incoming text string the substring of the incoming text string which matched the rule in the infix rule set such that a modified incoming text string is formed;

for each time the incoming text string is modified, repeating the comparing step with the modified incoming text string as last modified and the infix rule set until there are no more matches for the infix rule set in the modified incoming text string as last modified;

until each substring of the incoming text string has a corresponding phonemes stored in the work storage area, such that a phonemic data sequence representing the incoming text string is formed and held in the work storage area for use in a speech synthesizer.

24. The method of claim 23, wherein:

the steps of comparing substrings of the incoming text string to rules of the suffix rule set and repeating the comparing step with respect to the suffix rule set forms a first pair of steps;

the steps of comparing substrings of the incoming text string to rules of the prefix rule set and repeating the comparing step with respect to the prefix rule set forms a second pair of steps;

the steps of comparing substrings of the incoming text string to rules of the infix rule set and repeating the comparing step with respect to the infix rule set forms a third pair of steps; and

the digital processor further performs the first, second and third pairs of steps in one of;

(a) first, second, third pair order;

(b) second, first, third pair order;

(c) first, third, second pair order;

(d) second, third, first pair order;

(e) third, first, second pair order; and

(f) third, second, first pair order.

25. The method of claim 23, wherein:

the step of comparing substrings to rules of the suffix rule set includes beginning comparing substrings at an ending portion of the incoming text string, and comparing substrings in an end to beginning direction of the substring;

the step of comparing substrings to rules of the prefix rule set includes beginning comparing substrings at a beginning portion of the incoming text string, and comparing substrings in a beginning to end direction of the substring; and

the step of comparing substrings to rules of the infix rule set includes comparing substrings at a beginning portion of the incoming text string and comparing substrings in a beginning to end direction of the substring, and comparing substrings at an end portion of the incoming text string and compares substrings in an end to beginning direction of the substring.

26. The method of claim 25, further including the step of employing a dictionary look up of at least one of the incoming text string and modified incoming text string as last modified.

27. In a data processing system having a digital processor, an apparatus for translating incoming text to phoneme data, the apparatus comprising:

a source of incoming text;

a letter-to-sound processor executable by the digital processor and coupled to receive incoming text from the source, the letter-to-sound processor including a rule engine and a plurality of rule sets, each rule set including rules encoding translation of portions of incoming text to corresponding portions of phoneme data for representing the incoming text;

an input device for obtaining the incoming text;

in response to the letter-to-sound processor receiving the incoming text, the rule engine comparing portions of the incoming text to the rules in each of the plurality of rule sets, for different portions of the incoming text the rule engine comparing to different rule sets depending on location of the portion in the incoming text, and upon the rule engine determining a match between a rule and a subject portion of the incoming text, the rule engine producing a corresponding portion of the phoneme data according to the rule that was matched, such that the letter-to-sound processor combines the portions of the phoneme data and on output, provides the phoneme data representing the incoming text.

28. The apparatus of claim 27 wherein:

the plurality of rule sets includes a suffix rule set, a prefix rule set, and an infix rule set;

the rule engine first comparing ending portions of the incoming text to rules of the suffix rule set to produce ending portions of the phoneme data and a first remainder portion of the incoming text;

the rule engine comparing the first remainder portion of the incoming text with rules of the prefix rule set to produce beginning portions of the phoneme data and a second remainder portion of the incoming text; and

the rule engine further comparing the second remainder portion of the incoming text to rules of the infix rule set to produce intermediate portions of the phoneme data.

29. The apparatus of claim 28 further comprising:

a dictionary coupled to the letter-to-sound processor formed of a multiplicity of entries, each entry cross referencing a respective text string to a corresponding phoneme data string for representing the text string;

the letter-to-sound processor using the dictionary to perform a dictionary lookup on at least one of the incoming text, the first remainder portion and the second remainder portion to produce the phoneme data representing the incoming text.

30. The apparatus of claim 29, wherein:

the rule engine iteratively compares each rule of the suffix rule set beginning at a rightmost position in the ending portions of the incoming text and compares the rules in a right to left direction with respect to the incoming text;

the rule engine iteratively compares the rules from the prefix rule set beginning at a leftmost position in the first remainder portion of the incoming text and compares the rules in a left to right direction with respect to the first remainder portion of the incoming text; and

the rule engine iteratively compares each rule of the infix rule set beginning at a rightmost position of the second remainder portion such that the comparison of the second remainder portion with the rules of the infix rule set is in a right to left direction with respect to the incoming text, and the rule engine compares the rules of the infix rule set against the second remainder portion beginning at a leftmost position of the second remainder portion such that the comparison of the second remainder portion with the rules in the infix rule set is in a left to right direction with respect to the second remainder portion of the incoming text.

31. The apparatus as claimed in claim 28 wherein:

the rule engine compares each rule of the suffix rule set beginning at a rightmost position in the ending portions of the incoming text and compares the rules in a right to left direction with respect to the incoming text;

the rule engine compares the rules from the prefix rule set beginning at a leftmost position in the first remainder portion of the incoming text and compares the rules in a left to right direction with respect to the first remainder portion of the incoming text; and

the rule engine compares each rule of the infix rule set beginning at a rightmost position of the second remainder portion such that the comparison of the second remainder portion with the rules of the infix rule set is in a right to left direction with respect to the incoming text, and the rule engine compares the rules of the infix rule set against the second remainder portion beginning at a leftmost position of the second remainder portion such that the comparison of the second remainder portion with the rules in the infix rule set is in a left to right direction with respect to the second remainder portion of the incoming text.

32. The apparatus as claimed in claim 31 wherein:

in each of the suffix rule set, prefix rule set and infix rule set, the respective rules are arranged in order according to length of text to which the rule applies, from longest in length to shortest in length, and in a predefined order for rules applying to a same length text; and

the rule engine comparing one of the ending portions of the incoming text, the first remainder portion and the second remainder portion against a respective rule set beginning with a rule of the rule set that applies to a longest length of text encompassed by length of the portion of incoming text being compared.

33. The apparatus as claimed in claim 28 wherein:

in each of the suffix rule set, prefix rule set and infix rule set, the respective rules are arranged in order according to length of text to which the rule applies, from longest in length to shortest in length, and in a predefined order for rules applying to a same length text; and

the rule engine comparing one of the ending portions of the incoming text, the first remainder portion and the second remainder portion against a respective rule set beginning with a rule of the rule set that applies to a longest length of text encompassed by length of the portion of incoming text being compared.

34. The apparatus as claimed in claim 27 wherein:

in each of the rule sets, the respective rules are arranged in order according to length of text to which the rule applies, from longest in length to shortest in length, and in alphabetical order for rules applying to a same length text;

the rule engine comparing one of the ending portions of the incoming text, the first remainder portion and the second remainder portion against a respective rule set beginning with a rule of the rule set that applies to a longest length of text encompassed by length of the portion of incoming text being compared; and

the rule engine comparing portions of the incoming text against a respective rule set beginning with a rule of the rule set that applies to a longest length of text encompasses by length of the portion of incoming text being compared.

35. The apparatus of claim 27, wherein:

the data processing system is incorporated into a navigation system, the navigation system including an output sound system coupled to receive phoneme data from the letter-to-sound processor, the output sound system translating the phoneme data to sound signals and providing a spoken characterization of the incoming text in an audible fashion.

36. The apparatus as claimed in claim 27, wherein the letter-to-sound processor is incorporated into a speech synthesizer used to translate text to speech.

37. The apparatus as claimed in claim 28, wherein during the rule engine comparing the second remainder portion of the incoming text to rules of the infix rule set to produce intermediate portions of the phoneme data, the rule engine avoids splitting the second remainder into multiple third remainders by first detecting and delimiting a largest group of infix rule matching characters existing within the second remainder portion of the incoming text to create at least one third remainder, and comparing the at least one third remainder to rules of the infix rule set to produce intermediate portions of the phoneme data before comparing the largest group of infix rule matching characters to produce phoneme data.