Clustered information processing and searching with structured-unstructured database bridge

• US 9,092,504 B2
• Filed: 04/05/2013
• Issued: 07/28/2015
• Est. Priority Date: 04/09/2012
• Status: Active Grant

• Current Assignee: Vivek Ventures, LLC
• Original Assignee: Vivek Ventures, LLC
• Inventors: Doshi, Sundeep
• Primary Examiner(s): Le, Hung
• Application Number:
  US13/857,816
• Publication Number:
  US 20130268532A1
• Time in Patent Office: 844 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/3331 Query processing
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Systems and methods for indexing information and for performing searches are disclosed. In these systems and methods information is “ingested” into the system by clustering the information using a clustering algorithm such as k-means or k-medoids clustering. During the clustering process, a hybrid distance measurement is used that allows the systems and methods to determine similarity across a number of different types of information. Once the information is clustered, it is stored and “mirrored” both in a structured (e.g., relational) data repository and in an unstructured data repository. Methods according to the invention allow the retrieval of both direct search results and search results including related concepts. After clustered information is stored, future searches can be performed by searching the stored results in whichever data repository is most appropriate for the context.

Claims

1. A method for indexing and classifying related information, comprising:

using a computing system, measuring the similarity of or distance between a plurality of individual resources using a hybrid distance measurement, at least some of the plurality of individual resources being of different types and originating from different data sources;

clustering the plurality of individual resources into a plurality of clusters using the hybrid distance measurement; and

storing the plurality of clusters in both a structured and an unstructured data repository on the computing system or another computing system, such that the structured and the unstructured data repositories contain essentially the same information.

2. The method of claim 1, wherein said clustering comprises k-means or k-medoids clustering.

3. The method of claim 1, wherein said clustering comprises hierarchical clustering.

4. The method of claim 1, wherein the structured data repository comprises a relational database.

5. The method of claim 1, further comprising:

receiving a set of search terms at a search server via a communication network; and

searching a defined set of locations for the set of search terms to locate the plurality of individual resources.

6. The method of claim 1, wherein the hybrid distance measurement comprises a linearly weighted composite distance measurement based, at least in part, on cosine similarity.

7. A method for indexing, searching, and retrieving information, comprising:

using a first computing system, measuring the similarity of or distance between a plurality of individual resources using a hybrid distance measurement;

clustering the plurality of individual resources into a plurality of clusters using the hybrid distance measurement;

storing the plurality of clusters in both a structured and an unstructured data repository on the computing system or another computing system, such that the structured and the unstructured data repositories contain essentially the same information;

receiving a query at the first computing system or a second computing system in communication with a first computing system;

automatically directing the query to either the structured repository or an unstructured repository;

searching the structured or the unstructured repositories in accordance with said automatically directing; and

returning a result set to the query including at least a portion of the contents of at least one of the clusters.

8. The method of claim 7, wherein the structured repository comprises a relational database.

9. The method of claim 7, wherein the unstructured repository comprises an unstructured database.

10. The method of claim 7, wherein said automatically directing the query comprises determining whether searching the structured repository or searching the unstructured repository would be faster in view of the query and a measure of available resources.

11. The method of claim 7, wherein the hybrid distance measurement comprises a linearly weighted composite distance measurement based, at least in part, on cosine similarity.