System and method for news events detection and visualization

• US 9,256,664 B2
• Filed: 07/03/2014
• Issued: 02/09/2016
• Est. Priority Date: 07/03/2014
• Status: Active Grant

• Current Assignee: Palantir Technologies Incorporated
• Original Assignee: Palantir Technologies Incorporated
• Inventors: Thompson, James, Cai, Allen, Chakerian, John, Fink, Robert, Schafer, Mark, Sum, Marvin
• Primary Examiner(s): MOSER, BRUCE M
• Application Number:
  US14/323,935
• Publication Number:
  US 20160004764A1
• Time in Patent Office: 586 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/287 Visualization; Browsing
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing

Abstract

Systems and methods are disclosed for news events detection and visualization. In accordance with one implementation, a method is provided for news events detection and visualization. The method includes, for example, obtaining a document, obtaining from the document a plurality of tokens, obtaining a document vector based on a plurality of frequencies associated with the plurality of tokens, obtaining one or more clusters of documents, each cluster associated with a plurality of documents and a cluster vector, determining a matching cluster from the one or more clusters based at least on the similarity between the document vector and the cluster vector of the matching cluster, and updating a database to associate the document with the matching cluster.

Claims

1. An electronic device comprising:

one or more computer-readable storage media configured to store instructions; and

one or more processors configured to execute the instructions to cause the electronic device to;

obtain a document;

obtain from the document a plurality of tokens;

obtain a document vector based on a plurality of frequencies associated with the plurality of tokens;

obtain one or more clusters of documents, each cluster being associated with a plurality of documents, a cluster vector, a cluster weight, and a score;

mark a cluster of the one or more clusters as inactive if the cluster weight of the cluster is below a predetermined weight;

determine a matching cluster from the one or more clusters based at least on the similarity between the document vector and the cluster vector of the matching cluster;

update a database to associate the document with the matching cluster; and

update the score of the matching cluster based at least on a plurality of documents associated with the matching cluster.

2. The electronic device of claim 1, wherein each of the obtained one or more clusters is associated with an entity, and wherein the one or more processors are further configured to execute the instructions to cause the electronic device to:

after receiving a user input identifying the entity, display cluster information of the matching cluster and document information associated with the document.

3. The electronic device of claim 1, wherein the determination of the matching cluster is further based on the cluster weight of the matching cluster, and wherein the one or more processors are further configured to execute the instructions to cause the electronic device to:

periodically decrease the cluster weights of each of the one or more clusters.

4. The electronic device of claim 1, wherein the one or more processors are further configured to execute the instructions to cause the electronic device to:

update the cluster vector of the matching cluster based on the document vector.

5. The electronic device of claim 1, wherein the one or more processors are further configured to execute the instructions to cause the electronic device to:

obtain a plurality of megaclusters, each megacluster associated with one or more clusters;

determine a matching megacluster from the plurality of megaclusters based at least on the similarities between the cluster vector of the matching cluster and cluster vectors of the matching megacluster.

6. The electronic device of claim 5, wherein each of the obtained one or more clusters is associated with an entity, and wherein the one or more processors are further configured to execute the instructions to cause the electronic device to:

after receiving a user input identifying the entity, display megacluster information of the matching megacluster, cluster information of the matching cluster, and document information associated with the document.

7. The electronic device of claim 1, wherein the one or more processors are further configured to execute the instructions to cause the electronic device to:

after receiving a user input, display cluster information of the matching cluster and document information associated with the document, wherein the cluster information of the matching cluster includes the score of the matching cluster.

8. The electronic device of claim 1, wherein the one or more clusters are stored in the database and marking a cluster of the one or more clusters as inactive includes removing the cluster from the database.

9. A method performed by one or more processors, the method comprising:

obtaining a document;

obtaining from the document a plurality of tokens;

obtaining a document vector based on a plurality of frequencies associated with the plurality of tokens;

obtaining one or more clusters of documents, each cluster associated with a plurality of documents, a cluster vector, a cluster weight, and a score;

marking a cluster of the one or more clusters as inactive if the cluster weight of the cluster is below a predetermined weight;

determining a matching cluster from the one or more of clusters based at least on the similarity between the document vector and the cluster vector of the matching cluster;

updating a database to associate the document with the matching cluster; and

updating the score of the matching cluster based at least on a plurality of documents associated with the matching cluster.

10. The method of claim 9, wherein each of the obtained one or more clusters is associated with an entity, and wherein the method further comprises:

after receiving a user input identifying the entity, displaying cluster information of the matching cluster and document information associated with the document.

11. The method of claim 10, wherein the one or more clusters are stored in the database and marking a cluster of the one or more clusters as inactive includes removing the cluster from the database.

12. The method of claim 9, wherein the determination of the matching cluster is further based on the cluster weight of the matching cluster, and wherein the method further comprises:

periodically decreasing the cluster weights of each of the one or more clusters.

13. The method of claim 9, further comprising:

updating the cluster vector of the matching cluster based on the document vector.

14. The method of claim 9, further comprising:

obtaining a plurality of megaclusters, each megacluster associated with one or more clusters;

determining a matching megacluster from the plurality of megaclusters based at least on the similarities between the cluster vector of the matching cluster and cluster vectors of the matching megacluster.

15. The method of claim 14, wherein each of the obtained one or more clusters is associated with an entity, and wherein the method further comprises:

after receiving a user input identifying the entity, displaying megacluster information of the matching megacluster, cluster information of the matching cluster, and document information associated with the document.

16. The method of claim 9, wherein the method further comprises:

after receiving a user input, displaying cluster information of the matching cluster and document information associated with the document, wherein the cluster information of the matching cluster includes the score of the matching cluster.

17. A non-transitory computer-readable medium storing a set of instructions that are executable by one or more processors of one or more electronic devices to cause the one or more electronic devices to perform a method, the method comprising:

obtaining a document;

obtaining from the document a plurality of tokens;

obtaining a document vector based on a plurality of frequencies associated with the plurality of tokens;

obtaining one or more clusters of documents, each cluster associated with a plurality of documents, a cluster vector, a cluster weight, and a score;

marking a cluster of the one or more clusters as inactive if the cluster weight of the cluster is below a predetermined weight;

determining a matching cluster from the one or more clusters based at least on the similarity between the document vector and the cluster vector of the matching cluster;

providing an update to a database to associate the document with the matching cluster; and

updating the score of the matching cluster based at least on a plurality of documents associated with the matching cluster.

18. The non-transitory computer-readable medium of claim 17, wherein each of the obtained one or more clusters is associated with an entity, the non-transitory computer-readable medium further comprising instructions executable by the one or more electronic devices to cause the one or more electronic devices to perform:

after receiving a user input identifying the entity, displaying cluster information of the matching cluster and document information associated with the document.

19. The non-transitory computer-readable medium of claim 17, wherein the determination of the matching cluster is further based on the cluster weight of the matching cluster, the non-transitory computer-readable medium further comprising instructions executable by the one or more electronic devices to cause the one or more electronic devices to perform:

periodically decreasing the cluster weights of each of the one or more clusters.

20. The non-transitory computer-readable medium of claim 19, wherein the one or more clusters are stored in the database and marking a cluster of the one or more clusters as inactive includes removing the cluster from the database.

21. The non-transitory computer-readable medium of claim 17, further comprising instructions executable by the one or more electronic devices to cause the one or more electronic devices to perform:

updating the cluster vector of the matching cluster based on the document vector.

22. The non-transitory computer-readable medium of claim 17, further comprising instructions executable by the one or more electronic devices to cause the one or more electronic devices to perform:

obtaining a plurality of megaclusters, each megacluster associated with one or more clusters;

determining a matching megacluster from the plurality of megaclusters based at least on the similarities between the cluster vector of the matching cluster and cluster vectors of the matching megacluster.

23. The non-transitory computer-readable medium of claim 22, wherein each of the obtained one or more clusters is associated with an entity, the non-transitory computer-readable medium further comprising instructions executable by the one or more electronic devices to cause the one or more electronic devices to perform:

after receiving a user input identifying the entity, displaying megacluster information of the matching megacluster, cluster information of the matching cluster, and document information associated with the document.