IDENTIFYING CONTENT OF INTEREST

• US 20110010393A1
• Filed: 07/12/2010
• Published: 01/13/2011
• Est. Priority Date: 06/08/2006
• Status: Abandoned Application

• Current Assignee: Arthur R. Crivella, Clayton Darwin, Wayne J. West
• Original Assignee: Arthur R. Crivella, Clayton Darwin, Wayne J. West
• Inventors: Darwin, Clayton, Crivella, Arthur R., West, Wayne J.
• Application Number:
  US12/834,533
• Publication Number:
  US 20110010393A1
• Time in Patent Office: Days
• US Class Current: 707/780
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: G06F 16/353 into predefined classes

Abstract

Methods of identifying content of interest within a corpus are disclosed. The methods may comprise the step of applying a first marker set to the corpus, where the first marker set comprises at least one marker identifying a first type of text. For a first textual unit included in the corpus, the methods may comprise generating a score for the first marker set and comparing the score to a reference score. The score may indicate a number of instances of the at least one marker in the first textual unit.

Claims

1-22. -22. (canceled)

23. A computer-implemented method of identifying content of interest within a corpus, the method comprising:

identifying with a computer a textual unit in the corpus that includes an instance of an anchor marker set, wherein the computer comprises a processor circuit and operatively associated memory;

generating with the computer a plurality of scores for the textual unit, wherein each of the plurality of scores indicates a number of instances in the textual unit of one of a plurality of marker sets;

comparing with the computer the plurality of scores to a plurality of reference scores;

calculating with the computer an offset between the instance of the anchor marker set and an instance of an instance of one of the plurality of marker sets;

and

determining with the computer whether the textual unit comprises content of interest considering the comparing and the offset.

24. The method of claim 23, further comprising identifying at least one textual unit in the corpus that has external metadata consistent with a first set of criteria.

25. The method of claim 24, wherein the first set of criteria includes at least one criterion selected from the group consisting of:

an owner, a type, a creation date and a modification date.

26. The method of claim 23, further comprising identifying at least one textual unit in the corpus that has language metadata consistent with a second set of criteria.

27. The method of claim 23, wherein the second set of criteria includes at least criterion selected from the group consisting of:

a total number of tokens, an average number of characters per token, a number of tokens in a particular category.

28. The method of claim 23, further comprising generating a second plurality of scores for the textual unit, wherein each of the second plurality of scores indicates a number of instances in the textual unit of one of a second plurality of marker sets.

29. The method of claim 28, wherein at least one of the second plurality of marker sets includes markers that indicate a first language.

38. The method of claim 23, further comprising generating a marker set selected from the group consisting of the anchor marker set and a first marker set selected from the plurality of marker sets, wherein generating the marker set comprises:

selecting a seed marker set comprising at least one seed marker;

generating a seed corpus from a first reference corpus, wherein the seed corpus comprises a plurality of textual units, and wherein each of the plurality of textual units included in the seed corpus comprises at least one instance of a seed marker included in the seed marker set;

generating a statistical value describing the seed marker set and the seed corpus; and

generating a revised seed marker set.

39. The method of claim 38, wherein generating the marker set further comprises analyzing a plurality of instances of markers included in the seed marker set in context.

40. The method of claim 38, wherein generating the marker set further comprises:

generating a revised seed corpus from the seed corpus, wherein the revised seed corpus comprises a second plurality of textual units, and wherein each of the second plurality of textual units included in the revised seed corpus comprises at least one instance of a second seed marker included in the revised seed marker set; and

generating a second statistical value describing the revised seed marker set and the revised seed corpus.

41. The method of claim 38, wherein the statistical value indicates at least one of a rarity of a seed marker in the seed corpus and a saliency of the seed marker in the seed corpus.

42. The method of claim 38, wherein generating the statistical value comprises:

generating a first frequency of instances of a seed marker in the seed corpus; and

comparing the first frequency to an expected first frequency of instances of the seed marker in the seed corpus.

43. The method of claim 23, wherein each of the plurality of reference scores represents an average score for one of the plurality of marker sets in a reference corpus.

44. The method of claim 23, wherein the anchor marker set comprises a single marker, the method further comprising:

for an instance of one of the plurality of marker sets that occurs within a predetermined range of an instance of the anchor market set, displaying tokens comprising the instance of the one of the plurality of marker sets, the tokens comprising the anchor marker set and an intervening token.

45. The method of claim 44, wherein the instance of the one of the plurality of marker sets occurs within the predetermined range of the instance of the anchor marker set upon at least one condition selected from the group consisting of:

the instance of the one of the plurality of marker sets and the instance of the anchor marker set occurring within a predetermined number of tokens; and

the instance of the one of the plurality of marker sets and the instance of the anchor marker set occurring within a common textual unit.

30. A system for identifying content of interest within a corpus, the system comprising a processor circuit and an operatively associated memory, wherein the processor circuit is programmed to:

identify a textual unit in the corpus that includes an instance of an anchor marker set;

generate a plurality of scores for the textual unit, wherein each of the plurality of scores indicates a number of instances in the textual unit of one of a plurality of marker sets;

compare the plurality of scores to a plurality of reference scores;

calculate an offset between the instance of the anchor marker set and an instance of an instance of one of the plurality of marker sets; and

determine whether the textual unit comprises content of interest considering the comparing and the offset.

46. The system of claim 30, wherein the processor circuit is further programmed to identify at least one textual unit in the corpus that has external metadata consistent with a first set of criteria.

47. The system of claim 30, wherein the first set of criteria includes at least one criterion selected from the group consisting of:

an owner, a type, a creation date and a modification date.

48. The system of claim 30, wherein the processor circuit is further programmed to identify at least one textual unit in the corpus that has language metadata consistent with a second set of criteria.

49. The system of claim 30, wherein the second set of criteria includes at least criterion selected from the group consisting of:

a total number of tokens, an average number of characters per token, a number of tokens in a particular category.

50. The system of claim 30, wherein the processor circuit is further programmed to generate a second plurality of scores for the textual unit, wherein each of the second plurality of scores indicates a number of instances in the textual unit of one of a second plurality of marker sets.

51. The system of claim 50, wherein at least one of the second plurality of marker sets includes markers that indicate a first language.

52. The system of claim 30, wherein the processor circuit is further programmed to generate a marker set selected from the group consisting of the anchor marker set and a first marker set selected from the plurality of marker sets, wherein generating the marker set comprises:

selecting a seed marker set comprising at least one seed marker;

generating a seed corpus from a first reference corpus, wherein the seed corpus comprises a plurality of textual units, and wherein each of the plurality of textual units included in the seed corpus comprises at least one instance of a seed marker included in the seed marker set;

generating a statistical value describing the seed marker set and the seed corpus; and

generating a revised seed marker set.

53. The system of claim 52, wherein the generating the marker set further comprises analyzing a plurality of instances of markers included in the seed marker set in context.

54. The system of claim 52, wherein the generating the marker set further comprises:

generating a revised seed corpus from the seed corpus, wherein the revised seed corpus comprises a second plurality of textual units, and wherein each of the second plurality of textual units included in the revised seed corpus comprises at least one instance of a second seed marker included in the revised seed marker set; and

generating a second statistical value describing the revised seed marker set and the revised seed corpus.

55. The system of claim 52, wherein the statistical value indicates at least one of a rarity of a seed marker in the seed corpus and a saliency of the seed marker in the seed corpus.

56. The system of claim 52, wherein generating the statistical value comprises:

generating a first frequency of instances of a seed marker in the seed corpus; and

comparing the first frequency to an expected first frequency of instances of the seed marker in the seed corpus.

57. The system of claim 30, wherein each of the plurality of reference scores represents an average score for one of the plurality of marker sets in a reference corpus.

58. The system of claim 30, wherein the anchor marker set comprises a single marker, the method further comprising:

for an instance of one of the plurality of marker sets that occurs within a predetermined range of an instance of the anchor market set, displaying tokens comprising the instance of the one of the plurality of marker sets, the tokens comprising the anchor marker set and an intervening token.

59. The system of claim 58, wherein the instance of the one of the plurality of marker sets occurs within the predetermined range of the instance of the anchor marker set upon at least one condition selected from the group consisting of:

the instance of the one of the plurality of marker sets and the instance of the anchor marker set occurring within a predetermined number of tokens; and

the instance of the one of the plurality of marker sets and the instance of the anchor marker set occurring within a common textual unit.

31. A computer readable medium comprising instructions that when executed by a processor, cause the processor to perform the steps of:

identifying a textual unit in the corpus that includes an instance of an anchor marker set;

generating a plurality of scores for the textual unit, wherein each of the plurality of scores indicates a number of instances in the textual unit of one of a plurality of marker sets;

comparing the plurality of scores to a plurality of reference scores;

calculating an offset between the instance of the anchor marker set and an instance of an instance of one of the plurality of marker sets; and

determining whether the textual unit comprises content of interest considering the comparing and the offset.

32-37. -37. (canceled)