Recognizing transliterated words using suffix and/or prefix outputs

• US 10,558,748 B2
• Filed: 11/01/2017
• Issued: 02/11/2020
• Est. Priority Date: 11/01/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bostick, James E., Ganci, Jr., John M., Keen, Martin G., Trim, Craig M.
• Primary Examiner(s): Roberts, Shaun
• Application Number:
  US15/800,261
• Publication Number:
  US 20190129935A1
• Time in Patent Office: 832 Days
• Field of Search: 704 2
• CPC Class Codes: G06F 16/31 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/3332 Query translation
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 40/129 Handling non-Latin characters, e.g. kana-to-kanji conversion
• CPC Class Codes: G06F 40/232 Orthographic correction, e.g. spell checking or vowelisation
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/268 Morphological analysis
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams

Abstract

A computer-implemented method includes: receiving, by a computing device, an input file defining correct spellings of one or more transliterated words; generating, by the computing device, suffix outputs based on the one or more transliterated words; generating, by the computing device, a dictionary that maps the suffix outputs to the one or more transliterated words; recognizing, by the computing device, an alternatively spelled transliterated word included in a document as one of the one or more correctly spelled transliterated words using the dictionary; and outputting, by the computing device, information corresponding to the recognized transliterated word.

Claims

1. A computer-implemented method comprising:

receiving, by a computing device, an input file defining one or more correctly spelled transliterated words;

generating, by the computing device, a suffix output file of suffix outputs present in the one or more correctly spelled transliterated words;

generating, by the computing device, a dictionary that maps the suffix outputs in the suffix output file to the one or more correctly spelled transliterated words;

recognizing, by the computing device, an alternatively spelled transliterated word included in a document as one of the one or more correctly spelled transliterated words using the dictionary; and

outputting, by the computing device, information corresponding to the recognized transliterated word.

2. The method of claim 1, further comprising generating candidate suffixes for each of the one or more transliterated words included in the input file, wherein the generating the suffix outputs is based on the generating the candidate suffixes.

3. The method of claim 2, wherein the candidate suffixes include characters that follow an initial character of the one or more transliterated words.

4. The method of claim 3, wherein the candidate suffixes include a configurable minimum number of characters.

5. The method of claim 2, wherein the suffix outputs include suffixes that are included in more than one transliterated word in the input file.

6. The method of claim 1, further comprising generating a trie based on the suffix outputs, wherein the generating the dictionary is based on generating the trie.

7. The method of claim 1, wherein the outputting the information corresponding to the recognized transliterated word includes autocorrecting the alternatively spelled transliterated word in a word processor application.

8. The method of claim 1, wherein the outputting the information corresponding to the recognized transliterated word includes highlighting the alternatively spelled transliterated word in a word processor application.

9. The method of claim 1, wherein the recognizing the alternatively spelled transliterated word comprises:

identifying that a suffix in the alternatively spelled transliterated word is included in the dictionary;

identifying a prefix in the alternatively spelled transliterated word; and

mapping the prefix to a particular one of the one or more transliterated words in the input file.

10. The method of claim 1, wherein a service provider at least one of creates, maintains, deploys and supports the computing device.

11. The method of claim 1, wherein the receiving the input file, the generating the suffix outputs, the generating the dictionary, the recognizing the alternatively spelled transliterated word, and the outputting the information corresponding to the recognized transliterated word are provided by a service provider on a subscription, advertising, and/or fee basis.

12. The method of claim 1, wherein the computing device includes software provided as a service in a cloud environment.

13. The method of claim 1, further comprising deploying a system for recognizing alternatively spelled transliterated words in a document, comprising providing a computer infrastructure operable to perform the receiving the input file, the generating the suffix outputs, the generating the dictionary, the recognizing the alternatively spelled transliterated word, and the outputting the information corresponding to the recognized transliterated word.

14. A computer program product for recognizing alternatively spelled transliterated words in a document, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:

receive input including an alternatively spelled transliterated word;

identify that the alternatively spelled transliterated word includes a suffix in a transliterated word recognition dictionary that maps the suffix to a suffix output file of suffix outputs present in a plurality of correctly spelled transliterated words;

identify a particular one of the plurality of correctly spelled transliterated words that corresponds to the alternatively spelled transliterated word; and

output information corresponding to the particular one of the plurality of correctly spelled transliterated words.

15. The computer program product of claim 14, wherein the identifying the particular one of the plurality of correctly spelled transliterated comprises:

identifying a prefix in the alternatively spelled transliterated word; and

mapping the identified prefix with a prefix of the particular one of the plurality of correctly spelled transliterated words.

16. The computer program product of claim 14, wherein the outputting the information corresponding to the particular one of the plurality of correctly spelled transliterated words includes at least one selected from the group consisting of:

autocorrecting alternatively spelled transliterated word;

highlighting the alternatively spelled transliterated word and displaying the particular one of the plurality of correctly spelled transliterated words; and

performing data analysis based on the particular one of the plurality of correctly spelled transliterated words.

17. The computer program product of claim 14, wherein the program instructions further cause the computing device to generate the transliterated word recognition dictionary, wherein the generating the transliterated word recognition dictionary comprises:

receiving an input file identifying the plurality of correctly spelled transliterated words;

generating suffix outputs based on the plurality of correctly spelled transliterated words;

generating the transliterated word recognition dictionary that maps the suffix outputs to the one or more transliterated words.

18. The computer program product of claim 17, wherein the generating the transliterated word recognition dictionary further comprises generating candidate suffixes for each of the plurality of correctly spelled transliterated words included in the input file, wherein the generating the suffix outputs is based on the generating the candidate suffixes.

19. A system comprising:

a CPU, a computer readable memory and a computer readable storage medium associated with a computing device;

program instructions to receive an input file identifying one or more correctly spelled transliterated words;

program instructions to generate prefix outputs present in the one or more correctly spelled transliterated words;

program instructions to generate a dictionary that maps the prefix outputs to the one or more correctly spelled transliterated words;

program instructions to recognize an alternatively spelled transliterated word included in a document as one of the one or more correctly spelled transliterated words using the dictionary; and

program instructions to output information corresponding to the recognized transliterated word,

wherein the program instructions are stored on the computer readable storage medium for execution by the CPU via the computer readable memory.

20. The system of claim 19, wherein the program instructions to recognize the alternatively spelled transliterated word comprise:

program instructions to identify that a prefix in the alternatively spelled transliterated word is included in the dictionary;

program instructions to identify a suffix in the alternatively spelled transliterated word; and

program instructions to map the suffix to a particular one of the one or more transliterated words in the input file.

21. A computer-implemented method comprising:

receiving, by a computing device, an input file of one or more correctly spelled transliterated words;

generating, by the computing device, a dictionary that maps suffix outputs or prefix outputs, respectively, to a suffix output file or a prefix output file of suffix outputs or prefix outputs present in the one or more correctly spelled transliterated words;

recognizing, by the computing device, an alternatively spelled transliterated word included in a document as one of the one or more correctly spelled transliterated words using the dictionary; and

outputting, by the computing device, information corresponding to the recognized transliterated word.

22. The method of claim 21, wherein the outputting the information corresponding to the recognized transliterated word includes autocorrecting the alternatively spelled transliterated word in a word processor application.

23. The method of claim 21, wherein the outputting the information corresponding to the recognized transliterated word includes highlighting the alternatively spelled transliterated word in a word processor application.

24. The method of claim 21, wherein the outputting the information corresponding to the recognized transliterated word includes performing data analysis based on the recognized transliterated word instead of based on the alternatively spelled transliterated word.

25. A computer program product for recognizing alternatively spelled transliterated words in a document, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:

receive input including an alternatively spelled transliterated word;

identify that the alternatively spelled transliterated word includes a prefix in a transliterated word recognition dictionary that maps the prefix to a prefix output file of prefix outputs present in a plurality of correctly spelled transliterated words;

identify a particular one of the plurality of correctly spelled transliterated words that corresponds to the alternatively spelled transliterated word; and

output information corresponding to the particular one of the plurality of correctly spelled transliterated words.