Detecting novel document content

• US 8,521,665 B1
• Filed: 03/19/2012
• Issued: 08/27/2013
• Est. Priority Date: 03/20/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kumar, M. Bharath, Bharat, Krishna
• Primary Examiner(s): Holmes, Michael B
• Application Number:
  US13/423,718
• Time in Patent Office: 526 Days
• Field of Search: 706/12
• US Class Current: 706/12
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

A system determines an ordered sequence of documents and determines an amount of novel content contained in each document of the ordered sequence of documents. The system assigns a novelty score to each document based on the determined amount of novel content.

Claims

1. A method comprising:

identifying, by one or more processors, a group of documents related to a particular topic;

identifying, by the one or more processors, in a first document, of the group of documents, first content that is different from second content in other documents of the group of documents;

determining, by the one or more processors, a degree of difference between the first content and the second content;

determining, by the one or more processors, a score for the first document based on the degree of difference; and

modifying, by the one or more processors, a ranking of the first document relative to the other documents based on the score.

2. The method of claim 1, where determining the degree of difference includes:

identifying a string of characters included in the first content,

determining a number of occurrences, of the string of characters, within the first document, and

determining the degree of difference based on the number of occurrences.

3. The method of claim 2, where the string of characters includes a word sequence that is included in a title of the first document.

4. The method of claim 1, where determining the degree of difference includes:

identifying one or more pairs of two or more textual sequences that occur within a paragraph of the first document,

estimating an indication of importance associated with the one or more pairs relative to the first document, and

determining the degree of difference based on the indication of importance.

5. The method of claim 2, where the string of characters includes information identifying a particular person.

6. The method of claim 1, where identifying the first content includes:

identifying first information related to the topic,

identifying second information related to the topic,

the second information being different from the first information,

determining a proximity of the first information relative to the second information based on a position of the first information and a position of the second information within the first document, and

identifying the first content based on the proximity.

7. The method of claim 2, where the string of characters includes information identifying a named entity.

8. A non-transitory computer-readable medium storing instructions, the instructions comprising:

one or more instructions which, when executed by one or more processors, cause the one or more processors to identify a first document and a second document,

the first document and the second document being related to a topic, and the topic being received from a client;

one or more instructions which, when executed by the one or more processors, cause the one or more processors to identify, in the first document, first content that is different from second content in the second document;

one or more instructions which, when executed by the one or more processors, cause the one or more processors to determine a degree of difference between the first content and the second content;

one or more instructions which, when executed by the one or more processors, cause the one or more processors to determine a score for the first document based on the degree of difference; and

one or more instructions which, when executed by the one or more processors, cause the one or more processors to rank the first document relative to the second document based on the score.

9. The non-transitory computer-readable medium of claim 8, where the one or more instructions to determine the degree of difference include:

one or more instructions to identify a string of characters included in the first content,

one or more instructions to determine a quantity of occurrences, of the string of characters, within the first document, and

one or more instructions to determine the degree of difference based on the quantity of occurrences.

10. The non-transitory computer-readable medium of claim 9, where the string of characters includes a word sequence that is included in a title of the first document.

11. The non-transitory computer-readable medium of claim 8, where the one or more instructions to determine the degree of difference include:

one or more instructions to identify one or more pairs of two or more textual sequences that occur within a paragraph in the first document,

one or more instructions to estimate an indication of importance associated with the one or more pairs relative to the first document, and

one or more instructions to estimate the degree of difference based on the indication of importance.

12. The non-transitory computer-readable medium of claim 8, where the one or more instructions to identify the first content include:

one or more instructions to determine that a sequence of characters, included in the first document, identifies a particular person, and

one or more instructions to identify the first content based on the sequence of characters.

13. The non-transitory computer-readable medium of claim 8,

where the topic is included in a search query received from the client, and

where the one or more instructions to identify the first content include:

one or more instructions to identify first information related to the search query,

the first information being located at a first position within the first document,

one or more instructions to identify second information related to the search query,

the second information being different from the first information, and

the second information being located at a second position within the first document,

one or more instructions to determine a proximity of the first information relative to the second information based on the first position and the second position, and

one or more instructions to identify the first content based on the proximity.

14. The non-transitory computer-readable medium of claim 8, where the first content includes information identifying a particular person.

15. A system comprising:

one or more processors to;

identify a plurality of documents related to a search query,

identify, in a first document of the plurality of documents, first content that is different from second content in a second document of the plurality of documents,

determine a degree of difference between the first content and the second content,

determine a score for the first document based on the degree of difference, and

rank, based on the score, the first document relative to the second document and among the plurality of documents.

16. The system of claim 15, where, when determining the degree of difference, the one or more processors are to:

identify a string of characters included the content, and

determine a quantity of occurrences, of the string of characters, within the first document, and

determine the degree of difference based on the quantity.

17. The system of claim 15,

where one or more documents, of the plurality of documents, comprise documents posted by one or more users, and

where the one or more processors are further to:

obtain the one or more documents from a server that stores the one or more documents.

18. The system of claim 15, where, when determining the degree of difference, the one or more processors are to:

determine that a sequence of characters, included in the first document, identify a particular person associated with the search query, and

determine the degree of difference based on the sequence of characters.

19. The system of claim 15, where, when determining the degree of difference, the one or more processors are to:

identify first information in the first document,

the first information being located at a first position,

identify second information in the first document,

the second information being different from the first information and located at a second position,

determine a proximity of the first information relative to the second information based on the first position and the second position,

determine an interaction between the first information and the second information based on the proximity, and

determine the degree of difference based on the interaction.

20. The system of claim 19,

where the first information includes information identifying a particular person, and

where the second information includes information identifying a particular place.