System and method for online information analysis

• US 7,685,091 B2
• Filed: 02/14/2006
• Issued: 03/23/2010
• Est. Priority Date: 02/14/2006
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services GmbH (Accenture PLC)
• Inventors: Boone, Gary N., Sheth, Neil Vijay, Richter, Owen E.
• Primary Examiner(s): Vital; Pierre M
• Assistant Examiner(s): Vo; Truong V
• Application Number:
  US11/354,668
• Publication Number:
  US 20070198459A1
• Time in Patent Office: 1,498 Days
• Field of Search: None
• US Class Current: 707/737
• CPC Class Codes: G06F 16/34 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data

Abstract

The present disclosure includes systems and techniques relating to online information analysis. In general, in one implementation, a system includes a collection engine configured to accumulate document information retrieved from publicly accessible network resources according to predefined subjects, and an analysis engine configured to analyze the accumulated document information to identify change over a time period in general discussion of a topic within a selected subject of the predefined subjects, the analysis engine further configured to normalize the identified change over the time period based on change in a total number of documents found for the selected subject during the time period.

Claims

1. A computer-implemented method comprising:

accumulating multiple historical collections of blog entries, the blog entries each having been retrieved from a network and each satisfying predefined source and subject matter criteria;

scraping, then tokenizing, then stoplist filtering, then vectorizing, then weighting, then normalizing the historical collections of the blog entries;

receiving a first user input identifying a subject of which a user intends to analyze prevalence over time in the historical collections of blog entries, the subject being associated with a collection of query terms;

determining the query terms associated with the identified subject;

selecting a single historical collection of blog entries corresponding to the identified subject, from among the multiple, normalized historical collections of blog entries;

determining the prevalence of the query terms over time within the single selected historical collection;

generating a line chart illustrating a change in the determined prevalence of the query terms over time, a line of the line chart including points which define hyperlinks to one or more blog entries of the selected single historical collection;

receiving a second user input selecting one of the hyperlinks; and

presenting the blog entry associated with the selected hyperlink.

2. The method of claim 1, wherein:

generating the line chart further comprises generating a line chart illustrating the change in the determined prevalence over a time period specified by the user.

3. The method of claim 1, wherein the historical collection is weighted using an Okapi BM25 or Term frequency inverse document frequency (TFIDF) approach.

4. The method of claim 1, wherein the historical collection is vectorized using n-dimensional sparse vector arrays.

5. The method of claim 1, wherein the historical collection is weighted based on performing a dot product of a reverse index of the vectorized historical collection, and weights associated with the query terms.

6. The method of claim 1, further comprising identifying one or more weights associated with the collection of query terms.

7. The method of claim 1, wherein determining the prevalence of the query terms further comprises:

determining whether a univariate linear-regression of the prevalence of the query terms exhibits a shape associated with an idealized novelty event.

8. The method of claim 7, wherein the shape associated with the idealized novelty event comprises a horizontal linear shape followed by a sharply upward linear shape.

9. A computer-readable medium encoded with a computer program comprising instructions that, when executed, operate to cause a computer to perform operations comprising:

accumulating multiple historical collections of blog entries, the blog entries each having been retrieved from a network and each satisfying predefined source and subject matter criteria;

scraping, then tokenizing, then stoplist filtering, then vectorizing, then weighting, then normalizing the historical collections of the blog entries;

receiving a first user input identifying a subject of which a user intends to analyze prevalence over time in the historical collections of blog entries, the subject being associated with a collection of query terms;

determining the query terms associated with the identified subject;

selecting a single historical collection of blog entries corresponding to the identified subject, from among the multiple, normalized historical collections of blog entries;

determining the prevalence of the query terms over time within the single selected historical collection;

generating a line chaff illustrating a change in the determined prevalence of the query terms over time, a line of the line chart including points which define hyperlinks to one or more blog entries of the selected single historical collection;

receiving a second user input selecting one of the hyperlinks; and

presenting the blog entry associated with the selected hyperlink.

10. The computer readable medium of claim 9, wherein:

generating the line chart further comprises generating a line chart illustrating the change in the determined prevalence over a time period specified by the user.

11. The computer-readable medium of claim 9, wherein the historical collection is weighted using an Okapi BM25 or Term frequency inverse document frequency (TFIDF) approach.

12. The computer-readable medium of claim 9, wherein the historical collection is vectorized using n-dimensional sparse vector arrays.

13. The computer-readable medium of claim 9, wherein the historical collection is weighted based on performing a dot product of a reverse index of the vectorized historical collection, and weights associated with the query terms.

15. The system of claim 10, wherein:

generating the line chart further comprises generating a line chart illustrating the change in the determined prevalence over a time period specified by the user.

16. The system of claim 10, wherein the historical collection is weighted using an Okapi BM25 or Term frequency inverse document frequency (TFIDF) approach.

17. The system of claim 10, wherein the historical collection is vectorized using n-dimensional sparse vector arrays.

18. The system of claim 10, wherein the historical collection is weighted based on performing a dot product of a reverse index of the vectorized historical collection, and weights associated with the query terms.

19. The system of claim 10, wherein the operations further comprise identifying one or more weights associated with the collection of query terms.

20. The system of claim 10, wherein determining the prevalence of the query terms further comprises:

determining whether a univariate linear-regression of the prevalence of the query terms exhibits a shape associated with an idealized novelty event.

21. The system of claim 20, wherein the shape associated with the idealized novelty event comprises a horizontal linear shape followed by a sharply upward linear shape.

14. A system comprising:

one or more computers; and

a computer-readable medium coupled to the one or more computers having instructions stored thereon which, when executed by the one or more computers, causes the one or more computers to perform operations comprising;

accumulating multiple historical collections of blog entries, the blog entries each having been retrieved from a network and each satisfying predefined source and subject matter criteria;

scraping, then tokenizing, then stoplist filtering, then vectorizing, then weighting, then normalizing the historical collections of the blog entries;

receiving a first user input identifying a subject of which a user intends to analyze prevalence over time in the historical collections of blog entries, the subject being associated with a collection of query terms;

determining the query terms associated with the identified subject;

selecting a single historical collection of blog entries corresponding to the identified subject, from among the multiple, normalized historical collections of blog entries;

determining the prevalence of the query terms over time within the single selected historical collection;

generating a line chart illustrating a change in the determined prevalence of the query terms over time, a line of the line chart including points which define hyperlinks to one or more blog entries of the selected single historical collection;

receiving a second user input selecting one of the hyperlinks; and

presenting the blog entry associated with the selected hyperlink.