Method, computer product program and system for analysis of data

• US 8,990,236 B2
• Filed: 10/08/2010
• Issued: 03/24/2015
• Est. Priority Date: 10/09/2009
• Status: Active Grant

• Current Assignee: Qrative Ltd.
• Original Assignee: Qrative Ltd.
• Inventors: Mizrahy, Moshe, Golan, Ronen, Kenneth, Nadav
• Primary Examiner(s): VO, TRUONG V
• Application Number:
  US13/499,926
• Publication Number:
  US 20120284301A1
• Time in Patent Office: 1,628 Days
• Field of Search: 707/759
• US Class Current: 707/759
• CPC Class Codes: G06F 16/367 Ontology

Abstract

A system, computer program product and a system for analysis of data, the method includes: extracting a concise representation of a content a sample of a first dataset, wherein the extracting is executed without being mainly based on a-priory knowledge of a content of the sample of the first dataset; querying a queried dataset using at least a portion of the concise representation of the content of the sample of the first dataset, to provide query results; and generating a concise representation of a content of the query results.

Claims

1. A computer implemented method for analysis of data, the method comprising:

extracting without manual intervention a concise representation of a content a sample of a first dataset, wherein the extracting is executed without being based on a-priory knowledge of the content of the sample of the first dataset;

wherein the first dataset is either a biological dataset or a textual dataset;

wherein when the first dataset is a biological dataset it comprises least one out of sequence out of a DNA sequence and a RNA sequence;

wherein when the first dataset is a textual dataset then the sample of the first dataset comprises multiple words;

querying without manual intervention a queried dataset using at least a portion of the concise representation of the content of the sample of the first dataset, to provide query results; and

generating without manual intervention a concise representation of a content of the query results.

2. The method according to claim 1, wherein the extracting of the concise representation comprises extracting clusters of key subunits from the sample of the first dataset, and wherein the querying of the queried dataset comprises using at least one cluster of key subunits, to provide query results.

3. The method according to claim 2, comprising selecting key subunits based on a relationship between a frequency of appearance of subunits in the sample of the first dataset and a frequency of appearance of the subunits in a language of the subunits.

4. The method according to claim 3, comprising selecting up to a predefined number of key subunits that have a highest ratio between the frequency of appearance of the subunits in the sample of the first dataset and the frequency of appearance of the subunits in a language of the subunits.

5. The method according to claim 2, comprising clustering the key subunits based on a Jensen Shannon distance.

6. The method according to claim 2, comprising extracting the key subunits by an extraction module;

querying the queried dataset by a query module; and

generating the concise representation by concise representation generator.

7. The method according to claim 2, comprising querying a deep web site that belongs to the queried dataset;

wherein the deep web site comprises deep web content and surface web content;

wherein the surface web content is exposed to certain types of search engines and the deep web content is not exposed to certain types of search engines;

wherein the querying of the deep web site comprises;

exposing at least a portion of the deep web content by using a probing query; and

querying the exposed deep web content using the at least one cluster of key subunits.

8. The method according to claim 7, wherein the probing query consists of at least one highly popular word.

9. The method according to claim 2, wherein the key subunits are nucleotides.

10. The method according to claim 2, further comprising querying a queried dataset using multiple clusters of key subunits, to provide multiple query results;

consolidating the multiple query results to provide a consolidated dataset; and

extracting clusters of key subunits from the consolidated dataset.

11. The method according to claim 2, wherein an aggregate size of the clusters of key subunits is smaller than a size of the sample of a first dataset;

and wherein a size of the concise representation of the content of the query results is smaller than an aggregate size of documents that form the query results.

12. The method according to claim 1, comprising generating a concise representation of the content of the query results and of the content of the sample of the first dataset.

13. The method according to claim 1, wherein the concise representation of the query results is an associative data structure.

14. The method according to claim 1, wherein the query results comprise at least one group of files;

and wherein the generating of the concise representation comprises generating at least one group associative structure, wherein each group associative structure represents a content of a group of files.

15. The method according to claim 1 wherein the concise representation of the content of the query results is an associative ontology.

16. The method according to claim 1, wherein the concise representation of the content of the query results is a vertical ontology.

17. The method according to claim 1, wherein the queried dataset differs from the first dataset.

18. The method according to claim 1, wherein the queried dataset is the first dataset.

19. The method according to claim 1, comprising repeating the stages of extracting, querying and generating at different points in time to provide multiple concise representations of the queried dataset at different points in time and comparing between the multiple concise representations obtained at the different points in time.

20. The method according to claim 1, comprising generating the first dataset by monitoring documents retrieved by a user.

21. The method according to claim 1, comprising generating multiple concise representations of the first dataset, and querying the queried dataset using at least a portion of each of the multiple concise representations of the first dataset to provide query results.

22. The method according to claim 21, further comprising querying the queried dataset using at least a combination of at least a portion of at least two different concise representations of the first dataset to provide query results.

23. The method according to claim 21, comprising generating multiple sets of clusters key subunits of the first dataset, and querying the queried dataset using at least a portion of each set of the clusters of key subunits to provide query results.

24. The method according to claim 23, further comprising querying the queried dataset using at least a combination of clusters key subunits of different sets to provide query results.

25. The method according to claim 21 further comprising consolidating the multiple query results to provide a consolidated dataset;

and generating a concise representation of the consolidated dataset.

26. The method according to claim 21 comprising generating a concise representation of each query result;

and clustering the concise representations of the multiple query result.

27. The method according to claim 1 further comprising consolidating the multiple query results to provide a consolidated dataset;

and generating a concise representation of the consolidated dataset.

28. The method according to claim 1 comprising generating a concise representation of each query result;

and clustering the concise representations of the multiple query result.

29. A computer program product that comprises a non-tangible computer readable medium that stores instructions for:

extracting without manual intervention a concise representation of a content of a sample of a first dataset, wherein the extracting is executed without being based on a-priory knowledge of a content of the sample of the first dataset;

wherein the first dataset is either a biological dataset or a textual dataset;

wherein when the first dataset is a biological dataset it comprises least one out of sequence out of a DNA sequence and a RNA sequence;

wherein when the first dataset is a textual dataset then the sample of the first dataset comprises multiple words;

querying without manual intervention a queried dataset using at least a portion of the concise representation of the sample of the first dataset, to provide query results; and

generating without manual intervention a concise representation of a content of the query results.

30. The computer program product according to claim 29, wherein the non-tangible computer readable medium stores instructions for extracting of the concise representation by extracting clusters of key subunits from at least a sample of a first dataset, and querying of the queried dataset comprises by using at least one cluster of key subunits, to provide query results.

31. The computer program product according to claim 30, wherein the non-tangible computer readable medium stores instructions for selecting key subunits based on a relationship between a frequency of appearance of subunits in the sample of the first dataset and a frequency of appearance of the subunits in a language of the subunits.

32. The computer program product according to claim 31, wherein the non-tangible computer readable medium stores instructions for selecting up to a predefined number of key subunits that have a highest ratio between the frequency of appearance of the subunits in the sample of the first dataset and the frequency of appearance of the subunits in a language of the subunits.

33. The computer program product according to claim 30, wherein the non-tangible computer readable medium stores instructions for clustering the key subunits based on a Jensen Shannon distance.

34. The computer program product according to claim 30, wherein the non-tangible computer readable medium stores instructions for extracting the key subunits by an extraction module;

querying the queried dataset by a query module; and

generating the concise representation by concise representation generator.

35. The computer program product according to claim 30, wherein the non-tangible computer readable medium stores instructions for querying a deep web site that belongs to the queried dataset;

wherein the deep web site comprises deep web content and surface web content;

wherein the surface web content is exposed to certain types of search engines and the deep web content is not exposed to certain types of search engines;

wherein the querying of the deep web site comprises;

exposing at least a portion of the deep web content by using a probing query; and

querying the exposed deep web content using the at least one cluster of key subunits.

36. The computer program product according to claim 35, wherein the non-tangible computer readable medium stores instructions for using a probing query that consists of at least one highly popular word.

37. The computer program product according to claim 30, wherein the key subunits are nucleotides.

38. The computer program product according to claim 30, wherein the non-tangible computer readable medium stores instructions for querying a queried dataset using multiple clusters of key subunits, to provide multiple query results;

consolidating the multiple query results to provide a consolidated dataset; and

extracting clusters of key subunits from the consolidated dataset.

39. The computer program product according to claim 30, wherein an aggregate size of the clusters of key subunits is smaller than a size of the sample of a first dataset;

and wherein a size of the concise representation of the content of the query results is smaller than an aggregate size of documents that form the query results.

40. The computer program product according to claim 29, wherein the non-tangible computer readable medium stores instructions for generating a concise representation of the content of the query results and of the content of the sample of the first dataset.

41. The computer program product according to claim 29, wherein the concise representation of the query results is an associative data structure.

42. The computer program product according to claim 29, wherein the query results comprise at least one group of files;

and wherein the non-tangible computer readable medium stores instructions for generating the concise representation by generating at least one group associative structure, wherein each group associative structure represents a content of a group of files.

43. The computer program product according to claim 29, wherein the concise representation of the content of the query results is an associative ontology.

44. The computer program product according to claim 29, wherein the concise representation of the content of the query results is a vertical ontology.

45. The computer program product according to claim 29, wherein the queried dataset differs from the first dataset.

46. The computer program product according to claim 29, wherein the queried dataset is the first dataset.

47. The computer program product according to claim 29, wherein the non-tangible computer readable medium stores instructions for repeating the stages of extracting, querying and generating at different points in time to provide multiple concise representations of the queried dataset at different points in time and comparing between the multiple concise representations obtained at the different points in time.

48. The computer program product according to claim 29, wherein the non-tangible computer readable medium stores instructions for generating the first dataset by monitoring documents retrieved by a user.

49. The computer program product according to claim 29, wherein the non-tangible computer readable medium stores instructions for generating multiple concise representations of the first dataset, and querying the queried dataset using at least a portion of each of the multiple concise representations of the first dataset to provide query results.

50. The computer program product according to claim 49, wherein the non-tangible computer readable medium stores instructions for querying the queried dataset using at least a combination of at least a portion of at least two different concise representations of the first dataset to provide query results.

51. The computer program product according to claim 49, wherein the non-tangible computer readable medium stores instructions for generating multiple sets of clusters key subunits of the first dataset, and querying the queried dataset using at least a portion of each set of the clusters of key subunits to provide query results.

52. The computer program product according to claim 49, wherein the non-tangible computer readable medium stores instructions for querying the queried dataset using at least a combination of clusters key subunits of different sets to provide query results.

53. The computer program product according to claim 49, wherein the non-tangible computer readable medium stores instructions for consolidating the multiple query results to provide a consolidated dataset;

and generating a concise representation of the consolidated dataset.

54. The computer program product according to claim 49, wherein the non-tangible computer readable medium stores instructions for generating a concise representation of each query result;

and clustering the concise representations of the multiple query result.

55. The computer program product according to claim 29, wherein the non-tangible computer readable medium stores instructions for consolidating the multiple query results to provide a consolidated dataset;

and generating a concise representation of the consolidated dataset.

56. The computer program product according to claim 29, wherein the non-tangible computer readable medium stores instructions for generating a concise representation of each query result;

and to cluster the concise representations of the multiple query result.

57. A system, comprising:

a query module arranged to query without manual intervention a queried dataset using at least a portion of a concise representation of a content of at least a sample of a first dataset, to provide query results;

a memory module for storing the query results; and

an extraction module arranged to;

extract without manual intervention a concise representation of a content of the sample of a first dataset, wherein the extraction module is arranged to extract the concise representation of the content of the sample of the first dataset without a-priory knowledge of a content of the sample of the first dataset;

wherein the first dataset is either a biological dataset or a textual dataset;

wherein when the first dataset is a biological dataset it comprises least one out of sequence out of a DNA sequence and a RNA sequence;

wherein when the first dataset is a textual dataset then the sample of the first dataset comprises multiple words; and

generate without manual intervention a concise representation of a content of the query results.