Method and system for information discovery and text analysis

• US 9,256,667 B2
• Filed: 07/10/2009
• Issued: 02/09/2016
• Est. Priority Date: 01/10/2007
• Status: Active Grant

• Current Assignee: Meltwater News US, Inc. (Meltwater NV)
• Original Assignee: Sysomos Incorporated (Nasdaq, Inc.)
• Inventors: Koudas, Nick, Bansal, Nilesh
• Primary Examiner(s): Hutton, Nan
• Application Number:
  US12/501,324
• Publication Number:
  US 20090319518A1
• Time in Patent Office: 2,405 Days
• Field of Search: 707/602, 707/730, 707/737, 360/2, 373/0, 373/7
• US Class Current: 1/1
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

A method for searching text sources including temporally-ordered data objects, such as a blog, is provided including the steps of: (i) providing access to text sources, each text source including temporally-ordered data objects; (ii) obtaining or generating a search query based on terms and time intervals; (iii) obtaining or generating time data associated with the data objects; (iv) identifying data objects based on the search query; and (v) generating popularity curves based on the frequency of data objects corresponding to one or more of the search terms in the one or more time intervals. A system and computer program for text source searching is also provided.

Claims

1. A method for searching one or more text sources including temporally-ordered data objects, the method comprising the steps of:

providing access to the one or more text sources, each text source including one or more temporally-ordered data objects;

obtaining or generating a search query based on one or more search tokens and one or more time intervals;

obtaining or generating time data associated with the temporally-ordered data objects included in the one or more text sources;

identifying one or more data objects related to the search query in the one or more text sources;

generating one or more popularity curves based on frequency of the identified data objects corresponding to one or more of the search tokens in the one or more time intervals and the time data, the popularity curve exhibiting fluctuations of popularity over the one or more time intervals;

identifying a burst in the one or more popularity curves, the burst occurring in one or more burst time intervals;

identifying data objects related to the search query in the one or more burst time intervals as burst data objects;

for each of the burst data objects, assigning an authoritative index to the each burst data objects, the authoritative index being computed from a frequency of content related to the search query contributed by an author of the each burst data object;

ranking the burst data objects according to the authoritative index to obtain the top-i ranked burst data objects, i being an integer; and

providing the top-i ranked burst data objects as authoritative data objects for user selection and viewing;

wherein each data object comprises at least one of a blog, document, posting, article, email and message.

2. The method of claim 1, the method further comprising the steps of:

receiving a request from a user of searching with a drill-down parameter, the drill-down parameter being selected from;

a drilled-down time interval within any one of the one or more time intervals, a geographic location associated with at least one of the identified data objects, an identifiable origin of the one or more text sources, or any combination thereof;

restricting the search query to the drill-down parameter to obtain a restricted search query;

identifying one or more drilled-down data objects related to the restricted search query;

obtaining or generating drilled-down time data associated with the one or more drilled-down data objects; and

generating one or more drilled-down popularity curves based on frequency of the drilled-down data objects and the drilled-down time data.

3. The method of claim 1, the method further comprising the steps of:

generating one or more additional search tokens associated with the one or more data objects, the one or more additional search tokens having a low popularity value and a high number of occurrences in the one or more data objects.

4. The method of claim 1, wherein the search query includes one or more of:

one or more geographical search tokens, one or more demographic search tokens.

5. The method of claim 1, the method further comprising the steps of:

assigning user sentiment data extracted from each of the identified data objects to the each data objects, optionally the sentiment data being one of positive, neutral or negative,

segregating the popularity curves into regions generated from the identified data objects based on assigned sentiment data.

6. The method of claim 5, wherein the regions include a positive region generated from data objects associated with positive sentiment data, a neutral region generated from data objects associated with neutral sentiment data and a negative region generated from data objects associated with negative sentiment data.

7. The method of claim 1 further comprising determining at least one of the search tokens to be of interest if the at least one of the search tokens has a deviation of popularity above an expected value of popularity of a given day, the expected value of popularity computed based on popularity values from a predetermined number of days before the given day.

8. The method of claim 7, wherein the expected value of popularity is computed by a regression of the popularity values from the predetermined number of days before the given day.

9. The method of claim 1, wherein the authoritative index is further computed from a readability index assigned to the each burst data object.

10. The method of claim 1, wherein the authoritative index is further computed from at least one of:

geographic information of the author, and demographic information of the author.

11. A system for searching one or more text sources including temporally-ordered data objects, the system comprising:

a computer connected to the one or more text sources; and

a search term definition utility linked to the computer or executing on the computer;

the computer and the search term definition utility being configured to cooperate with each other and being operable to;

provide access to the one or more text sources, each text source including one or more temporally-ordered data objects;

obtain or generate a search query based on one or more search tokens and one or more time intervals;

obtain or generate time data associated with the temporally-ordered data objects included in the one or more text sources;

identify one or more data objects related to the search query in the one or more text sources;

generate one or more popularity curves based on the frequency of the identified data objects corresponding to one or more of the search tokens in the one or more time intervals and the time data;

identify a burst in the one or more popularity curves, the burst occurring in one or more burst time intervals;

identify data objects related to the search query in the one or more burst time intervals as burst data objects;

for each of the burst data objects, assign an authoritative index to the each burst data objects, the authoritative index being computed from a frequency of content related to the search query contributed by an author of the each burst data object;

rank the burst data objects according to the authoritative index to obtain the top-i ranked burst data objects, i being an integer; and

provide the top-i ranked burst data objects as authoritative data objects for user selection and viewing;

wherein each data object comprises at least one of a blog, document, posting, article, email and message.

12. The system of claim 11, further comprising:

a graphical user interface operably connected to the computer and the search term definition utility, the graphical user interface receiving a request from a user of searching with a drill-down parameter, the drill-down parameter being selected from;

a drilled-down time interval within any one of the one or more time intervals, a geographic location associated with at least one of the identified data objects, an identifiable origin of the one or more text sources, or any combination thereof,

wherein the computer and the search term definition utility are further configured and operable to;

restrict the search query to the drill-down parameter to obtain a restricted search query;

identify one or more drilled-down data objects related to the restricted search query;

obtain or generate drilled-down time data associated with the one or more drilled-down data objects; and

generate one or more drilled-down popularity curves based on frequency of the drilled-down data objects and the drilled-down time data.

13. A computer software product for use on a computer system, the computer software product comprising:

a computer readable non-transitory storage medium,

computer program code means stored on the computer readable non-transitory storage medium, the computer program code means comprising encoded instructions, wherein the encoded instructions comprise;

providing access to the one or more text sources, each text source including one or more temporally-ordered data objects;

obtaining or generating a search query based on one or more search tokens and one or more time intervals;

obtaining or generating time data associated with the temporally-ordered data objects included in the one or more text sources;

identifying one or more data objects related to the search query in the one or more text sources;

generating one or more popularity curves based on frequency of the identified data objects corresponding to one or more of the search tokens in the one or more time intervals and the time data, the popularity curve exhibiting fluctuations of popularity over the one or more time intervals;

identifying a burst in the one or more popularity curves, the burst occurring in one or more burst time intervals;

identifying data objects related to the search query in the one or more burst time intervals as burst data objects;

for each of the burst data objects, assigning an authoritative index to the each burst data objects, the authoritative index being computed from a frequency of content related to the search query contributed by an author of the each burst data object;

ranking the burst data objects according to the authoritative index to obtain the top-i ranked burst data objects, i being an integer; and

providing the top-i ranked burst data objects as authoritative data objects for user selection and viewing;

wherein each data object comprises at least one of a blog, document, posting, article, email and message.

14. The computer software product of claim 13 wherein the encoded instructions further comprise:

receiving a request from a user of searching with a drill-down parameter, the drill-down parameter being selected from;

a drilled-down time interval within any one of the one or more time intervals, a geographic location associated with at least one of the identified data objects, an identifiable origin of the one or more text sources, or any combination thereof;

restricting the search query to the drill-down parameter to obtain a restricted search query;

identifying one or more drilled-down data objects related to the restricted search query;

obtaining or generating drilled-down time data associated with the one or more drilled-down data objects; and

generating one or more drilled-down popularity curves based on frequency of the drilled-down data objects and the drilled-down time data.

15. The computer software product of claim 13 wherein the encoded instructions further comprise generating one or more additional search tokens associated with the one or more data objects, the one or more additional search tokens having a low popularity value and a high number of occurrences in the one or more data objects.

16. The computer software product of claim 13 wherein the search query includes one or more of:

one or more geographical search tokens, one or more demographic search tokens.

17. The computer software product of claim 13 wherein the encoded instructions further comprise:

assigning user sentiment data extracted from each of the identified data objects to the each data objects, optionally the sentiment data being one of positive, neutral or negative; and

segregating the popularity curves into regions generated from the identified data objects based on assigned sentiment data.

18. The computer software product of claim 17 wherein the regions include a positive region generated from data objects associated with positive sentiment data, a neutral region generated from data objects associated with neutral sentiment data and a negative region generated from data objects associated with negative sentiment data.

19. The computer software product of claim 13, wherein the encoded instructions further comprise determining at least one of the search tokens to be of interest if the at least one of the search tokens has a deviation of popularity above an expected value of popularity of a given day, the expected value of popularity computed based on popularity values from a predetermined number of days before the given day.

20. The computer software product of claim 19 wherein the expected value of popularity is computed by a regression of popularity values from the predetermined number of days before the given day.

21. The computer software product of claim 13, wherein the authoritative index is further computed from a readability index assigned to the each burst data object.

22. The computer software product of claim 13, wherein the authoritative index is further computed from at least one of:

geographic information of the author, and demographic information of the author.