Real time single-sweep detection of key words and content analysis

• US 9,519,704 B2
• Filed: 05/14/2012
• Issued: 12/13/2016
• Est. Priority Date: 05/18/2011
• Status: Active Grant

• Current Assignee: NetSpark Ltd.
• Original Assignee: NetSpark Ltd.
• Inventors: Korenwaitz, Elyasaf, Doron, Moshe
• Primary Examiner(s): Chen, Susan
• Application Number:
  US13/885,164
• Publication Number:
  US 20140095515A1
• Time in Patent Office: 1,674 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/335 Filtering based on additional data, e.g. user or group profiles filtering in web context G06F16/9535, G06F16/9536
• CPC Class Codes: G06F 40/237 Lexical tools

Abstract

A system and method are provided for real-time analysis of text. During a single sweep through the text, a detection tree is used to identify a sequence of characters in the text from a large dictionary of keywords. When a keyword is detected a rule tally database is updated. An intermediate score may be available during the sweep and a final score of the text may be available substantially immediately upon finishing the single sweep. A second text may be analyzed immediately using the same score buffer and rule tally database without updating the rule tally database.

Claims

1. A system for analyzing of a plurality of texts, the system comprising:

A) a plurality of keywords stored in a non-transitory computer readable structure, each keyword of said plurality of keywords stored as a respective site of a plurality of sites, said structure including a plurality of branches, each branch having plurality of sub branches, a unique ordered combination of a branch and one or more sub branches leading to each said respective site;

each branch of said plurality of branches and each sub branch of said plurality of sub branches being associated with one respective character string, each said keyword being the ordered combination of respective character strings associated with the unique ordered combination of branch and sub branches leading to said respective site and each branch of said plurality of branches being included in more than one said unique combination of a branch and one or more sub branches leading to more than one respective site;

B) a plurality of databases including a respective database corresponding to each text of the plurality of texts;

each said respective database including a plurality of counters including a respective incidence corresponding to each of said plurality of keywords;

C) a computer configured for said analyzing text of said plurality of texts including

a) reading a current character string from said one text starting at a start character and selecting from said plurality of branches a current branch associated with said character string,

b) recursively navigating said structure according to said one text subsequent to said first character string;

said recursively navigating including,

i) further reading from said one text a sequential character string sequential to said current character string,

ii) selecting from a plurality of sub branches of said current branch, a next branch associated with said sequential character string and

iii) replacing said current character string with said sequential character string and replacing said current branch with said next branch and repeating said further reading and said selecting and said replacing until a current branch leads to an encountered site of said plurality of sites,

c) incrementing in said respective database of said one text said respective incidence counter of a keyword when said keyword is found in said text starting at said start character by incrementing in said respective database of said one text said respective incidence counter of a keyword stored as said encountered site,

e) designating a new start character in the one text and repeating said reading, said navigating, said incrementing and

f) repeating said designating until for each of said plurality of keywords, said respective incidence counter indicates the number of times that said keyword appears in the one text;

D) a plurality of score buffers including at least one respective score buffer for each said text;

each said score buffer storing a computer modifiable value; and

wherein said computer is further configured for updating said value stored in a respective score buffer for said one text dependent on a limitation on a value of said respective incidence counter of at least two of said plurality of keywords

E) a rule counter and wherein said computer is further configured to set a value said rule counter dependent on at least two of said incidence counters, and wherein said updating is dependent on a value of said of rule counter.

2. The system of claim 1, wherein each character of said one text is designated as a start character at most once.

3. The system of claim 2, wherein each said respective database further includes an associated scan counter storing a scan id indicating whether said incidence counter was incremented during a current scan.

4. The system of claim 3, wherein said incrementing of said incidence counter further includes updating said scan id associated with said respective scan counter to a scan id of said one text, said computer further configured for:

f) further navigating said structure according to another text with another scan id subsequent to the analyzing of the one text and wherein during said further navigating said incrementing includes zeroing said respective incidence counter when a stored scan id associated with said incidence counter does not equal said another scan id and wherein said incrementing said incidence counter of said further navigating includes updating said stored scan id associated said respective scan counter to said another scan id.

5. The system of claim 1, further wherein said computer is further configured for simultaneously analyzing another text of said plurality of texts.

6. The system of claim 1, further including

a lookup table configured for looking up at least two letters of the text simultaneously during said navigating.

7. The system of claim 1, further comprising

a further sub-branch of said current branch leading to said encountered site and wherein said recursively navigating, continues with said further sub branch as said current branch.

8. The system of claim 7, further comprising:

further reading at least one even more character from the text subsequent to said incrementing, and

selecting a further sub-branch of said encountered site, said further sub-branch matching said at least one even more character.

9. The system of claim 1, wherein said computer is configured to perform said updating only when a value of said respective incidence counter of a second keyword in said respective database of said one text is less than a threshold value.

10. The system of claim 9, wherein said threshold value is one.