Method and system for approximate string matching

• US 20060004744A1
• Filed: 06/16/2005
• Published: 01/05/2006
• Est. Priority Date: 06/19/2004
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Volkov, Pavel, Nevidomski (Alex Nevidomski), Alexei
• Granted Patent: US 7,809,744 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/90344 by using string matching techniques

Abstract

A method and system are provided for approximate string matching of a target string to a trie data structure. The trie data structure has a root node and generations of child nodes each node representing at least one character in an alphabet to provide a lexicon of words and word fragments. The method involves traversing the trie data structure starting from the root node by comparing each node of a branch of the trie data structure to characters in the target string and adding characters traversed in a branch of the trie data structure to a gathered string to provide suggestions of approximate matches. If the method reaches a node flagged as a node for a word or a word fragment and, if the target string is longer than the gathered string, the method loops back to the root node, and continues the traverse from the root node. This enables the trie data structure to use word fragments for compound words and to split non-delimited words where appropriate. The method also includes, at each node, determining if there is a correction rule for one or more characters in the remainder of the target string from the current node, and if so, applying the correction rule to the target string to obtain a modified target string.

Claims

1. A method of approximate string matching of a target string to a trie data structure, the trie data structure having a root node and generations of child nodes each node representing at least one character in an alphabet, the method comprising:

traversing a trie data structure starting from the root node by comparing each node of a branch of the trie data structure to characters in the target string;

adding characters traversed in a branch of the trie data structure to a gathered string;

reaching a node flagged as a node for a word or a word fragment and, if the target string is longer than the gathered string, looping back to the root node, and continuing the traverse from the root node.

2. A method as claimed in claim 1, wherein at the step of reaching a node flagged as a node for a word or a word fragment, if the node is flagged as a word and if the target string is longer than the gathered string, the method includes adding a word break to the gathered string before looping back to the root node.

3. A method as claimed in claim 1, wherein at the step of reaching a node flagged as a node for a word or a word fragment, if the node is flagged as a word fragment and if the target string is longer than the gathered string, the method includes looping back to the root node and continuing to add characters to the gathered string.

4. A method as claimed in claim 1, wherein the method includes:

at each node, determining if there is a correction rule for one or more characters in the remainder of the target string from the current node;

if so, applying the correction rule to the target string to obtain a modified target string.

5. A method as claimed in claim 4, wherein a correction rule is a sequence to sequence character substitution.

6. A method as claimed in claim 4, wherein the method includes continuing the traverse for the modified target string as well as for the original target string.

7. A method as claimed in claim 4, wherein no additional modifications of the modified target string are allowed within its already modified parts.

8. A method as claimed in claim 1, wherein the method includes at each node:

accepting a match for a character by appending the character to a gathered string and incrementing the current character in the target string;

or

applying an edit operation to a gathered string and incrementing a count of an error value for the approximation.

9. A method as claimed in claim 8, wherein the method includes predefining edit operations to be skipped depending on the edit operations carried out in previous steps of the traverse.

10. A method as claimed in claim 1, wherein the traverse is a single, non-deterministic pass which gathers entries which conform to an error tolerance.

11. A system for approximate string matching of a target string to a trie data structure, the system comprising:

a trie data structure having a root node and generations of child nodes each node representing at least one character in an alphabet;

means for traversing a trie data structure starting from the root node by comparing each node of a branch of the trie data structure to characters in the target string and adding characters traversed in a branch of the trie data structure to a gathered string;

means for determining if a node is flagged as a node for a word or a word fragment and means for comparing the length of the target string to the length of the gathered string;

means for looping back to the root node when the means for determining determines a node flagged as a node for a word or a word fragment and the means for comparing determines that the target string is longer than the gathered string.

12. A system as claimed in claim 11, wherein the system includes means for providing a set of suggestions from the trie data structure.

13. A system as claimed in claim 11, wherein the system includes:

a set of correction rules for applying a sequence to sequence substitution within a target string.

14. A system as claimed in claim 11, wherein the system includes means for indicating if a node is a word or a word fragment and, if a word fragment, the position of the word fragment in a word.

15. A computer program product stored on a computer readable storage medium, comprising computer readable program code means for performing the steps of:

traversing a trie data structure starting from the root node by comparing each node of a branch of the trie data structure to characters in the target string;

adding characters traversed in a branch of the trie data structure to a gathered string;

reaching a node flagged as a node for a word or a word fragment and, if the target string is longer than the gathered string, looping back to the root node, and continuing the traverse from the root node.

16. A method of approximate string matching of a target string to a trie data structure, the trie data structure having a root node and generations of child nodes each node representing a character in an alphabet, the method comprising:

traversing a trie data structure starting from a root node by comparing each node of a branch of the trie data structure to characters in the target string;

at each node, determining if there is a correction rule for one or more characters in the remainder of the target string from the current node;

if so, applying the correction rule to the target string to obtain a modified target string.

17. A method as claimed in claim 16, wherein a correction rule is a sequence to sequence character substitution.

18. A method as claimed in claim 16, wherein the method includes continuing the traverse for the modified target string as well as for the original target string.

19. A method as claimed in claim 16, wherein no additional modifications of the modified target string are allowed within its already modified parts.

20. A method as claimed in claim 16, wherein the correction rules are stored in the trie data structure.

21. A method as claimed in claim 16, wherein the correction rules are stored in a separate data structure.

22. A method as claimed in claim 16, wherein the method includes:

adding characters traversed in a branch of the trie data structure to a gathered string;

reaching a node flagged as a node for a word or a word fragment and, if the target string is longer than the gathered string, looping back to the root node, and continuing the traverse from the root node.

23. A method as claimed in claim 22, wherein at the step of reaching a node flagged as a node for a word or a word fragment, if the node is flagged as a word and if the target string is longer than the gathered string, the method includes adding a word break to the gathered string before looping back to the root node.

24. A method as claimed in claim 22, wherein at the step of reaching a node flagged as a node for a word or a word fragment, if the node is flagged as a word fragment and if the target string is longer than the gathered string, the method includes looping back to the root node and continuing to add characters to the gathered string.

25. A method as claimed in claim 16, wherein the method includes at each node:

accepting a match for a character by appending the character to a gathered string and incrementing the current character in the target string;

or

applying an edit operation to a gathered string and incrementing a count of an error value for the approximation.

26. A method as claimed in claim 25, wherein the method includes predefining edit operations to be skipped depending on the edit operations carried out in previous steps of the traverse.

27. A method as claimed in claim 16, wherein the traverse is a single, non-deterministic pass which gathers entries which conform to an error tolerance.

28. A system for approximate string matching of a target string to a trie data structure, the system comprising:

a trie data structure having a root node and generations of child nodes each node representing at least one character in an alphabet;

means for traversing a trie data structure starting from the root node by comparing each node of a branch of the trie data structure to characters in the target string and adding characters traversed in a branch of the trie data structure to a gathered string; and

a set of correction rules for applying a sequence to sequence substitution within a target string.

29. A system as claimed in claim 28, wherein the set of correction rules are stored in the trie data structure.

30. A system as claimed in claim 28, wherein the set of correction rules are stored in a separate data structure.

31. A system as claimed in claim 28, wherein the system includes:

means for determining if a node is flagged as a node for a word or a word fragment and means for comparing the length of the target string to the length of the gathered string;

means for looping back to the root node when the means for determining determines a node flagged as a node for a word or a word fragment and the means for comparing determines that the target string is longer than the gathered string.

32. A system as claimed in claim 28, wherein the system includes means for providing a set of suggestions from the trie data structure.

33. A system as claimed in claim 28, wherein the system includes means for indicating if a node is a word or a word fragment and, if a word fragment, the position of the word fragment in a word.

34. A computer program product stored on a computer readable storage medium, comprising computer readable program code means for performing the steps of:

traversing a trie data structure starting from a root node by comparing each node of a branch of the trie data structure to characters in the target string;

at each node, determining if there is a correction rule for one or more characters in the remainder of the target string from the current node;

if so, applying the correction rule to the target string to obtain a modified target string.