System, Method and Computer Program Product for Automatic Topic Identification Using a Hypertext Corpus

• US 20130204876A1
• Filed: 03/14/2013
• Published: 08/08/2013
• Est. Priority Date: 09/07/2011
• Status: Active Grant

• Current Assignee: Venio, Inc.
• Original Assignee: Venio, Inc.
• Inventors: Szucs, John Joseph, Warner, Kurtis Lee, Paris, Thomas Carl, Moye, Charles David
• Granted Patent: US 9,442,930 B2
• Time in Patent Office: Days
• US Class Current: 707/738
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: G06F 16/374 Thesaurus
• CPC Class Codes: G06F 16/94 Hypermedia Hyperlinking G06F40/134

Abstract

A system, method, and/or computer program product for automatic topic identification using a hypertext corpus may include a) receiving a content document(s); b) identifying or lexically scoring candidate topic(s) in the received content document based on label(s) used in a corpus to link to or relate to the candidate topics; c) evaluating or semantically scoring the candidate topic(s) of the received document based on a relationship between two or more candidate topics in the corpus; and d) weighting candidate topics for relevance based on algorithmic or statistical analysis of links or relationships in the corpus.

Claims

1. A method for performing workflow process on at least one content document comprising:

a) receiving, by at least one processor, the at least one document content;

b) organizing, by the at least one processor, the at least one document content;

c) discovering, by the at least one processor, at least one metadata relating to the at least one document content; and

d) taking action, by the at least one processor, on the at least one document content, based on said (a), (b), and (c).

2. The method according to claim 1, wherein at least one of said (b) or said (c) comprises at least one of:

i) analyzing, by the at least one processor, a topic of the at least one content document;

or

ii) organizing, by the at least one processor, based on a topic of the at least one content document.

3. The method according to claim 2, wherein said (i) comprises:

a) identifying, by the at least one processor, at least one candidate topic in at least one received content document of the at least one content document based on at least one label used in a corpus to link to or relate to said at least one candidate topic;

b) evaluating, by the at least one processor, the at least one candidate topic of the at least one received content document as a whole based on at least one relationship between at least two of said at least one candidate concept from at least one relationship in the corpus; and

c) weighting, by the at least one processor, the candidate content documents for relevance based on at least one of algorithmic analysis, or statistical analysis of the at least one relationship in the corpus.

4. The method according to claim 1, wherein said content document comprises at least one of:

an email;

a message;

a Simple Message System (SMS) message;

a Multimedia Message System (MMS) message;

a facsimile;

a Really Simple Syndication (RSS) feed;

an electronic communication;

a social-networking communication, post, or comment;

a social media communication, post, or comment;

a micro-blog;

an image;

audio content;

video content;

text-based content;

or

optical character recognized (OCR) content.

5. A method comprising:

a) receiving, by at least one processor, at least one content document;

b) identifying or lexically scoring, by the at least one processor, at least one candidate topic in said at least one received content document based on at least one label used in a corpus to link to or relate to said at least one candidate topic;

c) evaluating or semantically scoring, by the at least one processor, the at least one candidate topic of the at least one received document based on at least one relationship between at least two of said at least one candidate topics in the corpus; and

d) weighting, by the at least one processor, said at least one candidate topics for relevance based on algorithmic analysis or statistical analysis of the at least one hypertext link in the corpus.

6. The method according to claim 5, wherein said corpus comprises at least one of:

a source of at least one of lexical information or semantic information;

a Wiki;

a Wikipedia;

an online encyclopedia;

an electronic encyclopedia;

a world wide web in whole or in part;

an enterprise intranet;

a dictionary or lexicon;

a semantic database;

a professional database;

a source of lexical and/or semantic information;

a dictionary;

a lexicon;

a database;

at least one online encyclopedia;

a personalized corpus;

a field specific corpus;

or

at least one online encyclopedia supplemented by at least one dictionary.

7. The method according to claim 5, wherein said at least one content document comprises at least one of:

a textual document;

a recognized speech;

an electronic mail (email) message;

an electronic communication;

a text message;

a Simple Message System (SMS) message;

a multimedia message;

a Multimedia Message System (MMS) message;

a social media post or posting;

a social network post or posting;

an information stream;

a Really Simple Syndication (RSS) feed;

a syndicated content stream;

an Optically Recognized Characters (OCR) from content;

hypermedia;

audio content;

video content;

a web page;

image content;

textual content;

an Internet content page;

an intranet content page;

another hypermedia document;

a recognized speech converted document;

an optically recognized character (OCR) content document from at least one of;

a still image, or

a motion image;

an electronic communication comprising at least one of;

electronic mail,

text message,

multimedia message,

social media post or comment,

social network post or comment, or

forum posting or comment;

or

an information stream comprising at least one of;

a Really Simple Syndication (RSS) feed,

an Atom feed,

another feed, or

syndicated content.

8. The method according to claim 5, wherein said at least one label comprises at least one of:

i) an anchor term in the corpus;

ii) a single word;

iii) a phrase including a plurality of words;

iv) a word or phrase in a lexicon;

or

v) an identifying name, word or phrase associated with an entity in a database, wherein said database comprises at least one of a contact list, an inventory, a product database, another database).

9. The method according to claim 5, wherein said identifying or lexically scoring of said (b) comprises at least one of:

i) finding a plurality of said at least one label used in the corpus, appearing in said at least one received document in a lexical stage,

ii) filtering said plurality of said at least one label appearing in said at least one received document;

iii) calculating a score for each of said at least one label appearing in said at least one received document;

or

iv) identifying said at least one candidate topic based on at least one topic to which a given label may refer in the corpus.

10. The method according to claim 9, wherein said (i) comprises:

wherein said finding comprises at least one of;

1) searching for said at least one label using case insensitive matching;

2) searching for said at least one label using case sensitive matching;

3) searching for said at label using a full text search algorithm;

4) indexing of keywords;

5) synonyms;

6) word stemming or conjugating;

7) searching for said at least one label using a text search algorithm, which may include at least one of;

indexing,

word stemming or conjugating,

stop-word filtering,

case folding, and

pattern matching, or

regular expression matching.

11. The method according to claim 9, wherein said (ii) comprises:

wherein said filtering comprises;

excluding based on criteria comprising at least one of;

i. excluding a most commonly used words list for a given language;

or

ii. another exclusion.

12. The method according to claim 9, wherein said (iii) comprises:

wherein said calculating comprising at least one of;

1) counting a number of occurrences of said at least one label in said at least one document;

or

2) adjusting the calculated score of a given one of said at least one label based on the frequency of occurrence of said given one of said at least one label in a given language.

13. The method according to claim 9, wherein said (iv) comprises:

wherein said identifying comprises at least one of;

1) selecting at least one candidate topic in the corpus to which a given label refers;

or

2) calculating a score for each said at least one candidate topic from at least one factor,

wherein said calculating said score based on said at least one factor comprises at least one of;

i) calculating said score based on a calculated score for said at least one label associated with said at least one candidate topic;

or

ii) calculating said score based on a probability that a given label refers to a given topic in the corpus.

14. The method according to claim 5, wherein said evaluating or semantically scoring of said (c) comprises:

wherein said at least one relationship is based on at least one of;

at least one link,

at least one hypertext link,

at least one link in a database,

at least one link in a semantic database,

at least one relationship in a relational database,

at least one relationship in a social network,

at least one relationship between data elements, or

at least one other relationship.

15. The method according to claim 5, wherein said evaluating or semantically scoring of said (c) comprises:

wherein said semantically scoring comprises at least one of;

i) adjusting said score based on presence of a relationship between said at least two candidate topics;

ii) adjusting said score based on a weighted strength of a relationship between said at least two candidate topics;

or

iii) adjusting said score based on algorithmic analysis or statistical analysis of the relatedness between said at least two candidate topics.

16. The method according to claim 5, wherein said (d) comprises:

wherein said algorithmic analysis comprises at least one of;

i) a page ranking relevancy analysis;

or

ii) a link analysis algorithm (an algorithm for evaluating relative importance of at least two of said at least one candidate topics in the corpus); and

wherein said statistical analysis comprises at least one of;

i) vector space models;

or

ii) probabilistic models.

17. A method, according to claim 15, wherein said (c) (iii) comprises:

wherein said algorithmic analysis comprises at least one of;

1) evaluating, by the at least one computer, relatedness between a plurality of said at least one topic in a hypermedia database;

2) evaluating, by the at least one computer, relatedness between a plurality of adjacent vertices in a graph representation of a hypermedia database;

3) evaluating, by the at least one computer, relatedness between a plurality of adjacent vertices based on a number of other vertices to which said plurality of vertices are coupled;

4) evaluating, by the at least one computer, information content of an edge in a graph;

wherein said information content comprises at least one of;

i) self-information;

ii) surprisal;

iii) other measures of information content based on information theory;

or

5) evaluating, by the at least one computer, relatedness between a plurality of non-adjacent vertices;

wherein said evaluating of (5) comprises at least one of;

i) finding a path between said plurality of non-adjacent vertices;

or

ii) incorporating a cost function into path finding of (i) based on said (2), (3), or (4).

20. The method according to claim 5, wherein said corpus comprises:

a hypertext corpus comprising

at least one graph, wherein said graph comprises;

a plurality V of vertices, wherein each individual vertex v represents a page; and

a plurality of edges, each edge representing a link.

21. The method according to claim 20, wherein said each edge comprises:

information content H,

wherein said information content H comprises

22. The method according to claim 21, further comprising:

iterating through at least one of;

each candidate topic, or

a subset of candidate topics, identified by a lexical stage; and

calculating a semantic score S, for each link from a first said candidate topic being evaluated that connects to another said candidate topic,

wherein S=Σ

(L_x×

H),

wherein L_x describes the lexical score for each said candidate topic in a collection of said candidate topics.

23. The method according to claim 5, wherein said weighting comprises:

calculating a weighting u, for a given term based on an occurrence of said given term in an input document,

wherein

18. A system comprising:

at least one memory; and

at least one processor coupled to said at least one memory, said at least one processor adapted to;

receive at least one content document;

identify or lexically score at least one candidate topic in said at least one received content document based on at least one label used in a corpus to link to or relate to said at least one candidate topic;

evaluate or semantically score the at least one candidate topic of the at least one received document based on at least one relationship between at least two of said at least one candidate topics in the corpus; and

weight said at least one candidate topics for relevance based on algorithmic analysis or statistical analysis of the at least one hypertext link in the corpus.

19. A nontransitory computer program product embodied on a computer readable medium, said computer program product comprising program logic, which when executed on at least one computer processor performs a method comprising:

a) receiving, by at least one processor, at least one content document;

b) identifying or lexically scoring, by the at least one processor, at least one candidate topic in said at least one received content document based on at least one label used in a corpus to link to or relate to said at least one candidate topic;

c) evaluating or semantically scoring, by the at least one processor, the at least one candidate topic of the at least one received document based on at least one relationship between at least two of said at least one candidate topics in the corpus; and

d) weighting, by the at least one processor, said at least one candidate topics for relevance based on algorithmic analysis or statistical analysis of the at least one hypertext link in the corpus.