System, method to generate transliteration and method for generating decision tree to obtain transliteration

• US 20080270111A1
• Filed: 06/18/2007
• Published: 10/30/2008
• Est. Priority Date: 04/30/2007
• Status: Active Grant

• Current Assignee: Tachyon Technologies Pvt., Ltd.
• Original Assignee: Tachyon Technologies Pvt., Ltd.
• Inventors: Hanumanthappa, Ram Prakash
• Granted Patent: US 8,005,664 B2
• Time in Patent Office: Days
• US Class Current: 704/3
• CPC Class Codes: G06F 40/129 Handling non-Latin characters, e.g. kana-to-kanji conversion
• CPC Class Codes: G06F 40/151 Transformation

Abstract

The transliteration scheme developed as the present invention is contingent upon two languages, namely the keyboard language and the target language. While the invention is independent of the actual languages in question, save to the extent that they must be phonetic or pseudo-phonetic and the keyboard language must possess a script, it is nonetheless convenient to cast the invention within the framework of two specific languages. In this regard, English has been chosen as the keyboard language, and Hindi (a language widely spoken in India) has been chosen as the target language. An understanding of the current invention is possible without any knowledge of Hindi by a practitioner skilled in the art.

Claims

1. A method to generate transliteration of source language script into target language script using decision tree based technique with automated supervised learning, said method comprising steps of

i. selecting text to identify predetermined character to train particular instance of usage of identified character;

ii. representing various possible phonetic and/or pseudo-phonetic translations of identified character to generate a multitude of transliterated words;

iii. grouping generated words followed by annotating them with predetermined information;

iv. analyzing annotated words to represent possible decision questions to map the characters under the focus; and

v. applying predetermined rules on analyzed words to generate transliteration of the source language into the target language.

2. The method as claimed in claim 1, wherein the method used to generate transliteration is independent of the languages.

3. The method as claimed in claim 1, wherein the method provides predetermined rules for reverse transliteration from the target language script to the source language script.

4. The method as claimed in claim 1, wherein the source language is a keyboard language preferably English and target language is any predetermined language preferably selected from a group comprising Kannada, Hindi, Telugu, Tamil, Marathi, Malayalam and Gujarati.

5. The method as claimed in claim 1, wherein the character is identified by means of any representational scheme such as Unicode.

6. The method as claimed in claim 1, wherein predetermined properties/characteristics corresponding to different characters of the target language, independent of the method itself are used in forming the decision question.

7. The method as claimed in claim 1, wherein the multitude of possible transliterations is used to train the transliteration engine.

8. The method as claimed in claim 1, wherein the information for annotating the words is position of the character within a word under focus and generated target language character.

9. The method as claimed in claim 1, wherein the method provides frequencies of the words from the text by maintaining a running counter for each unique word.

10. The method as claimed in claim 1, wherein the position of the character is indexed beginning with the number zero.

11. The method as claimed in claim 1, wherein the predetermined rules are selected for a character depending upon the semantic-groups to which it belongs.

12. The method as claimed in claim 1, wherein the method automatically learns the particular language word patterns by parsing web pages and/or predetermined documents containing language content.

13. The method as claimed in claim 1, wherein the method automatically predicts the transliteration of each letter depending on context around the letter.

14. The method as claimed in claim 1, wherein the method operates by applying patterns from its learnt knowledge base rather than matching words in any database.

15. The method as claimed in claim 1, wherein the method captures frequency of each letter in a context specific manner to determine the mapping.

16. The method as claimed in claim 1, wherein the method performs reverse transliteration from predetermined language to the source language.

17. The method as claimed in claim 1, wherein the method uses semantic context rather than character level context to identify the transliteration.

18. The method as claimed in claim 1, wherein the method automatically transliterates source language words to target language based on native pronunciation of the target language.

19. The method as claimed in claim 1, wherein the method shows available multiple transliterations for a particular input text to choose the most appropriate transliteration.

20. The method as claimed in 1, wherein the method, with heuristics to decide whether the input is for predictive or non-predictive rule-based transliteration, provides options for rule-based input as well without the user explicitly having to choose a user interface element to switch between the predictive and non-predictive modes.

21. A method for generating decision tree to obtain transliteration of source language script into target language script, said method comprising steps of

i. generating all possible tuples of semantic-groups along with a focus number;

ii. applying rules representing generated tuples to set of training words;

iii. dividing the training words into two sets depending upon whether the words satisfies the query posed by the tuples or not; and

iv. adding the tuple as a node for two sets of trained words to generate the decision tree.

22. The method as claimed in claim 21, wherein the method is repeated until the entire set of words is found to be segregated or none of the rules is able to decide between the words at leaves of the tree.

23. The method as claimed in claim 21, wherein the focus number specifies context around the particular letter on which rules are to be applied.

24. The method as claimed in claim 21, wherein the focus number is a predetermined number preferably between −

4 and 4.

25. The method as claimed in claim 21, wherein first set of the two sets comprises the words for which the question asked by the tuple is answered “

yes” and

other is answered “

no”

.

26. The method as claimed in claim 21, wherein the method used to generate decision tree is language independent.

27. The method as claimed in claim 21, wherein the semantic-groups are language/pronunciation related groupings like vowel, consonant, long vowel, short vowel, aspirated, unaspirated or non-language related predetermined groups like groupA, groupB etc.

28. A system to generate transliteration of source language script into target language script using decision tree based technique with automated supervised learning, said system comprising of

i. a device having memory;

ii. an input device for entering text;

iii. transliteration engine to maintain patterns and predetermined rules used in transliteration of source language script into target language script;

iv. a display device for displaying entered text and transliterated text; and

v. an interface to enable typing in any language and optionally to integrate the transliteration system into existing web-pages.

29. The system as claimed in claim 28, wherein the device having source language alphabet layout like English is used for typing target language words phonetically.

31. The system as claimed in claim 28, wherein the transliteration engine is completely independent of the web-server setup or the application with which the transliteration engine is integrated.

32. The system as claimed in claim 28, wherein the device is any predetermined system having an input device for inputting source language alphabet like computer systems with keyboards with complete English alphabet and/or with hand-writing recognition, mobile phones and telephones with numeric keypad layouts and/or hand-writing recognition.

33. The system as claimed in claim 28, wherein the display device uses single and/or dual panel interface for displaying entered text, wherein only the target language content is displayed while taking input from the user in the source language in case of a single panel interface or both the source language representation and the target language output are displayed in case of a dual panel interface.

34. The system as claimed in claim 28, wherein the interface is an AJAX interface, which keeps entire knowledge base on the server side.

35. The system as claimed in claim 28 adapts to low resource environments such as mobile phones, where the amount of memory available is a constraint, by having a disk based decision tree without having to load the entire tree into memory.

36. The system as claimed in claim 28, wherein the server is platform-independent.

30. The system as claimed in 28, wherein the interface for using the transliteration mechanism is placed in any existing web pages or web applications or non-web applications in general and specifically as input interface for search engines, composition of emails and blogs and text editing applications.