Methods and systems for improving text segmentation

• US 20070124301A1
• Filed: 09/30/2004
• Published: 05/31/2007
• Est. Priority Date: 09/30/2004
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Elbaz, Gilad, Mandelson, Jacob
• Granted Patent: US 7,680,648 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates

Abstract

Methods and systems for improving text segmentation are disclosed. In one embodiment, at least a first segmented result and a second segmented result are determined from a string of characters, a first frequency of occurrence for the first segmented result and a second frequency of occurrence for the second segmented result are determined, and an operable segmented result is identified from the first segmented result and the second segmented result based at least in part on the first frequency of occurrence and the second frequency of occurrence.

Claims

1. A method, comprising:

determining at least a first segmented result and a second segmented result from a string of characters;

determining a first frequency of occurrence for the first segmented result and a second frequency of occurrence for the second segmented result; and

identifying an operable segmented result from the first segmented result and the second segmented result based at least in part on the first frequency of occurrence and the second frequency of occurrence.

2. The method of claim 1, wherein the first segmented result comprises a first combination of tokens and the second segmented result comprises a second combination of tokens.

3. The method of claim 1, wherein determining the first frequency of occurrence comprises determining a number of articles containing the first segmented result.

4. The method of claim 3, wherein determining the number of articles containing the first segmented result comprises determining a number of article identifiers in a search result set generated in response to a search query comprising the first segmented result.

5. The method of claim 3, wherein determining the number of articles containing the first segmented result comprises accessing an index of articles.

6. The method of claim 1, wherein determining the first frequency of occurrence comprises determining a number of occurrences of the first segmented result in a plurality of search queries.

7. The method of claim 1, wherein the string of characters comprises a domain name.

8. The method of claim 1, further comprising selecting an article based at least in part on the operable segmented result.

9. The method of claim 8, wherein the article comprises an advertisement.

10. The method of claim 1, further comprising determining whether to filter a domain name comprising the string of characters based at least in part on the operable segmented result.

11. The method of claim 1, wherein determining the first segmented result and the second segmented result comprises:

determining a plurality of segmented results from the string of characters; and

identifying the first segmented result and the second segmented result from the plurality of segmented results.

12. The method of claim 11, wherein identifying the first segmented result and the second segmented result comprises calculating a probability value for each of the plurality of segmented results.

13. The method of claim 12, wherein a first probability value associated with the first segmented result is based at least in part on a frequency of each token within the first segmented result.

14. The method of claim 1, wherein identifying the operable segment from the first segment result and the second segment result based at least in part on the first frequency of occurrence and the second frequency of occurrence comprises selecting the first segment result if the first frequency of occurrence is greater than the second frequency of occurrence.

15. The method of claim 1, wherein the second segmented result is a spelling corrected version of the first segmented result.

16. A method, comprising:

determining a plurality of segmented results from the string of characters;

identifying at least a first segmented result and a second segmented result from the plurality of segmented results;

generating a first search results set comprising a first number of article identifiers associated with the first segmented result and a second search results set comprising a second number of article identifiers associated with the second segmented result; and

selecting the first segmented result when the first number of article identifiers is greater than the second number of article identifiers and selecting the second segmented result when the second number of article identifiers is greater than the first number of article identifiers.

17. The method of claim 16, wherein generating the first search result set comprises generating a first search query comprising the first segmented result and generating the second search result set comprises generating a second search query comprising the second segmented result.

18. A computer-readable medium containing program code, comprising:

program code for determining at least a first segmented result and a second segmented result from a string of characters;

program code for determining a first frequency of occurrence for the first segmented result and a second frequency of occurrence for the second segmented result; and

program code for identifying an operable segmented result from the first segmented result and the second segmented result based at least in part on the first frequency of occurrence and the second frequency of occurrence.

19. The computer-readable medium of claim 18, wherein the first segmented result comprises a first combination of tokens and the second segmented result comprises a second combination of tokens.

20. The computer-readable medium of claim 18, wherein program code for determining the first frequency of occurrence comprises program code for determining a number of articles containing the first segmented result.

21. The computer-readable medium of claim 20, wherein program code for determining the number of articles containing the first segmented result comprises program code for determining a number of article identifiers in a search result set generated in response to a search query comprising the first segmented result.

22. The computer-readable medium of claim 20, wherein program code for determining the number of articles containing the first segmented result comprises program code for accessing an index of articles.

23. The computer-readable medium of claim 18, wherein program code for determining the first frequency of occurrence comprises program code for determining a number of occurrences of the first segmented result in a plurality of search queries.

24. The computer-readable medium of claim 18, wherein the string of characters comprises a domain name.

25. The computer-readable medium of claim 18, further comprising program code for selecting an article based at least in part on the operable segmented result.

26. The computer-readable medium of claim 25, wherein the article comprises an advertisement.

27. The computer-readable medium of claim 18, further comprising program code for determining whether to filter a domain name comprising the string of characters based at least in part on the operable segmented result.

28. The computer-readable medium of claim 27, wherein program code for determining the first segmented result and the second segmented result comprises:

program code for determining a plurality of segmented results from the string of characters; and

program code for identifying the first segmented result and the second segmented result from the plurality of segmented results.

29. The computer-readable medium of claim 28, wherein program code for identifying the first segmented result and the second segmented result comprises program code for calculating a probability value for each of the plurality of segmented results.

30. The computer-readable medium of claim 29, wherein a first probability value associated with the first segmented result is based at least in part on a frequency of each token within the first segmented result.

31. The computer-readable medium of claim 18, wherein program code for identifying the operable segment from the first segment result and the second segment result based at least in part on the first frequency of occurrence and the second frequency of occurrence comprises program code for selecting the first segment result if the first frequency of occurrence is greater than the second frequency of occurrence.

32. The computer readable medium of claim 18, wherein the second segmented result is a spelling corrected version of the first segmented result.

33. A computer-readable medium containing program code, comprising:

program code for determining a plurality of segmented results from the string of characters;

program code for identifying at least a first segmented result and a second segmented result from the plurality of segmented results;

program code for generating a first search results set comprising a first number of article identifiers associated with the first segmented result and a second search results set comprising a second number of article identifiers associated with the second segmented result; and

program code for selecting the first segmented result when the first number of article identifiers is greater than the second number of article identifiers and selecting the second segmented result when the second number of article identifiers is greater than the first number of article identifiers.

34. The computer-readable medium of claim 33, wherein program code for generating the first search result set comprises program code for generating a first search query comprising the first segmented result and generating the second search result set comprises generating a second search query comprising the second segmented result.

35. A method, comprising:

determining at least a first segmented result and a second segmented result from a domain name;

determining a first frequency of occurrence for the first segmented result in at least one of an article index, a text index, and a search result set;

determining a second frequency of occurrence for the second segmented result;

if the first frequency of occurrence is greater than the second frequency of occurrence, then selecting the first segmented result as an operable segmented result;

if the second frequency of occurrence is greater than the first frequency of occurrence, then selecting the second segmented result as the operable segmented result; and

selecting an advertisement based at least in part on the operable segmented result, wherein the advertisement includes text associated with the operable segmented result; and

causing a display of the advertisement in association with a web page associated with the domain name.