Systems and methods for identifying key phrase clusters within documents

• US 9,535,974 B1
• Filed: 12/23/2014
• Issued: 01/03/2017
• Est. Priority Date: 06/30/2014
• Status: Active Grant

• Current Assignee: Palantir Technologies Incorporated
• Original Assignee: Palantir Technologies Incorporated
• Inventors: Wadhar, Hem, Kesin, Max
• Primary Examiner(s): Chbouki, Tarek
• Application Number:
  US14/581,920
• Time in Patent Office: 742 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 40/205 Parsing

Abstract

Systems and methods are disclosed for key phrase clustering of documents. In accordance with one implementation, a method is provided for key phrase clustering of documents. The method includes obtaining a first plurality of documents based at least on a user input, obtaining a statistical model based at least on the user input, and obtaining, from content of the first plurality of documents, a plurality of segments. The method also includes identifying a plurality of clusters of segments from the plurality of segments, determining statistical significance of the plurality of clusters based at least on the statistical model and the content, and providing for display a representative cluster from the plurality of tokens, the representative cluster being determined based at least on the statistical significance. The method further includes determining a label for the representative cluster based at least on the plurality of clusters and the statistical significance.

Claims

1. An electronic device comprising:

a computer display;

one or more computer-readable storage media configured to store instructions; and

one or more processors configured to execute the instructions to cause the electronic device to at least;

obtain a first plurality of documents based at least in part on a user input;

obtain a statistical model based at least in part on the user input;

obtain, from content of the first plurality of documents, a plurality of segments;

determine statistical significance for the obtained plurality of segments based at least in part on the obtained statistical model;

determine, for each document in the first plurality of documents, representative segments from the obtained plurality of segments, the representative segments being determined based at least in part on the determined statistical significance;

cluster documents from the obtained first plurality of documents based at least in part on the determined representative segments;

receive a selection of a date range;

for a cluster of documents associated with a date within the date range, automatically associate a label with the cluster of documents based at least in party on the determined representative segments; and

display within a graphical user interface on the computer display a representation of the date range, the label, and contents of and/or links to documents in the cluster of documents.

2. The electronic device of claim 1, wherein the user input identifies an entity, and the obtained statistical model was generated based on at least one of:

the first plurality of documents;

a second plurality of documents associated with the entity; and

a third plurality of documents associated with an industry associated with the entity.

3. The electronic device of claim 1, wherein determining the label is automatically associated with the cluster of documents based at least in part on a frequency of appearances of the representative segments in the first plurality of documents.

4. The electronic device of claim 1, wherein the one or more processors are further configured to execute instructions to cause the electronic device to:

receive a selection input associated with the cluster of documents; and

responsive to the selection input, provide for display contents of one or more documents within the cluster of documents.

5. A method performed by one or more processors, the method comprising:

obtaining a first plurality of documents based on at least a user input;

obtaining a statistical model based at least on the user input;

obtaining, from content of the first plurality of documents, a plurality of segments;

determining statistical significance for the obtained plurality of segments based at least on the obtained statistical model;

determining representative segments from the obtained plurality of segments for each document in the first plurality of documents, the representative segments being determined based at least in part on the determined statistical significance;

clustering documents from the obtained first plurality of documents based at least in part on the determined representative segments;

receiving a selection of a date range;

for a cluster of documents associated with a date within the date range, automatically associating a label with the cluster of documents based at least in party on the determined representative segments; and

providing for display within a graphical user interface a representation of the date range, the label, and at least one of contents of and links to documents in the cluster of documents.

6. The method of claim 5, wherein the user input identifies an entity, and the obtained statistical model was generated based on at least one of:

the first plurality of documents;

a second plurality of documents associated with the entity; and

a third plurality of documents associated with an industry associated with the entity.

7. The method of claim 5, wherein the automatically associating the label with the cluster of documents is further based on a frequency of appearances of the representative segments in the first plurality of documents.

8. The method of claim 5 further comprising:

receiving a selection input associated with the cluster of documents; and

responsive to the selection input, providing for display contents of one or more documents within the cluster of documents.

9. A non-transitory computer-readable medium storing a set of instructions that are executable by one or more electronic devices, each having one or more processors, to cause the one or more electronic devices to perform a method, the method comprising:

obtaining a first plurality of documents associated with a user input;

obtaining a statistical model associated with the user input;

obtaining, from content of the first plurality of documents, a plurality of segments;

determining statistical significance for the plurality of segments based at least on the statistical model;

determining, for each document in the first plurality of documents, representative segments from the plurality of segments, the representative segments being determined based at least in part on the statistical significance;

clustering documents from the first plurality of documents based at least in part on the representative segments;

receiving a selection of a date range;

for a cluster of documents associated with a date within the date range, automatically associating a label with the cluster of documents based at least in part on the determined representative segments; and

providing for display within a graphical user interface a representation of

the date range,

the label, and

contents of documents in the cluster of documents, or links to documents in the cluster of documents, or a combination thereof.

10. The non-transitory computer-readable medium of claim 9, wherein the user input identifies an entity, and the statistical model was generated based on at least one of:

the first plurality of documents;

a second plurality of documents associated with the entity; and

a third plurality of documents associated with an industry associated with the entity.

11. The non-transitory computer-readable medium of claim 9, wherein the automatically associating the label with the cluster of documents is further based on a frequency of appearances of the representative segments in the first plurality of documents.

12. The non-transitory computer-readable medium of claim 9, the method further comprising:

receiving a selection input associated with the cluster of documents; and

responsive to the selection input, providing for display contents of one or more documents within the cluster of documents.