Method and apparatus for indexing, searching and displaying data

• US 7,840,524 B2
• Filed: 04/17/2006
• Issued: 11/23/2010
• Est. Priority Date: 06/14/1993
• Status: Expired due to Fees

• Current Assignee: Daniel Egger
• Original Assignee: Software Rights Archive, LLC (Altitude Capital Partners)
• Inventors: Cannon, Shawn, Egger, Daniel, Sauers, Ronald D.
• Primary Examiner(s): Robinson; Greta L
• Assistant Examiner(s): Chang; Jeffrey
• Application Number:
  US11/404,824
• Publication Number:
  US 20060242564A1
• Time in Patent Office: 1,681 Days
• Field of Search: 382/225, 707/7, 707/205, 707/737
• US Class Current: 707/709
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2457 with adaptation to user needs
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9532 Query formulation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 16/9558 Details of hyperlinks; Management of linked annotations
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A computer research tool for indexing, searching and displaying data is disclosed. Specifically, a computer research tool for performing computerized research of data including textual objects in a database or a network and for providing a user interface that significantly enhances data presentation is described. Textual objects and other data in a database or network is indexed by creating a numerical representation of the data. The indexing technique called proximity indexing generates a quick-reference of the relations, patterns and similarity found among the data in the database. Proximity indexing indexes the data by using statistical techniques and empirically developed algorithms.

Claims

1. A method related to cluster analysis of a world wide web having identifiable web pages and hyperlink relationships made up of Universal Resource Locaters with pointers, wherein objects are related to the world wide web, direct non-semantic relationships relate to hyperlink relationships, and indirect non-semantic relationships relate to a series of hyperlink relationships between objects, comprising:

crawling webpages on the world wide web for information used to define a set of objects to be indexed and to collect information about the direct non-semantic relationships, wherein Universal Resource Locators that either point to or point away from one or more of the web pages are crawled;

defining the set of objects to be indexed, wherein each object in the set of objects has an identification and wherein a plurality of the objects in the set of objects have direct and indirect non-semantic relationships;

generating, using a computer processor, a numerical representation for the set of objects in the form of a series of arrays representing each of said objects in the set of objects based upon each of said object'"'"'s direct non-semantic relationships, if any, with other of said objects in the set of objects,

wherein generating the numerical representation for the set of objects accounts for a plurality of direct non-semantic relationships and includes quantifying the accounted for direct non-semantic relationships, wherein the quantifying includes weighting some of the direct non-semantic relationships differently than others;

generating, using a computer processor, a scalar value for each of said objecting the set of objects, wherein said scalar value accounts for direct and indirect non-semantic relationships that exist with other said objects in the set of objects and generating the scalar value includes;

quantifying, for each of said objects in the set of objects that has one or more of the indirect non-semantic relationships, said object'"'"'s indirect non-semantic relationships with other objects in the set of objects, wherein a.) some of the indirect non-semantic relationships contribute greater value to the scalar value than others, b.) a plurality of different types of indirect relationships, when present, contribute to the scalar value, and c.) quantifying said object'"'"'s indirect non-semantic relationships includes accounting for at least the following three indirect non-semantic relationship patterns for a given object A when present;

i) B cites f and f cites A,

ii) B cites f, f cites e, and e cites A, and

iii) B cites f, f cites e, e cites d, and d cites A, wherein B, d, e, and fare objects in the set of objects and said accounting for indirect non-semantic relationships uses weights that are calculated using one or more of said objects'"'"'quantity of outbound direct relationships;

storing the generated scalar values in one or more computer memories as an index;

receiving search commands wherein the search commands are received from an input device, wherein the received search commands include one or more search terms;

identifying a resultant set of said objects that are associated with one or more search terms using at least a word index and the received search commands;

determining a rank for objects in the resultant set of objects using said scalar values as a factor in determining the rank; and

sending, for use by a display device, information for displaying identities of two or more objects in the resultant set of objects using the rank as a factor in determining an order of display.

2. The method of claim 1 wherein generating a scalar value further comprises accounting for shape characteristics of an image.

3. The method of claim 1 further comprising:

using one or more robots or spiders to crawl the world wide web for information, wherein the one or more robots or spiders reads HTML instructions.

4. The method of claim 1 wherein quantifying said object'"'"'s indirect non-semantic relationships further includes:

using a damping factor so that indirect relationships of greater link lengths have less impact on the scalar value;

examining, for each object that has indirect non-semantic relationships, a subset of objects in the set of objects, wherein the examined subset of objects is less than the total number objects in the set of objects and the examined subset includes all of the objects in the set of objects that have an indirect non-semantic relationship within a certain number of link lengths within a plurality of relationship patterns which includes at least the three indirect non-semantic relationship patterns i), ii) and iii) when present; and

analyzing at least the three indirect non-semantic relationship patterns i), ii) and iii) when present and combining results to generate the scalar.

5. The method of claim 4 wherein examining the subset of objects comprises:

examining a value associated with each object in the subset of objects that has an indirect non-semantic relationship to another object in the set of objects, wherein the value associated with each object is a function of at least influence links, source links and a damping factor based upon the number of link lengths between an identified object and an object in the subset of objects having an indirect non-semantic relationship with the identified object.

6. The method of claim 1 wherein the set of objects relates to a plurality of webpages and the generating a numerical representation comprises using weights based on the importance of one or more of the plurality of webpages.

7. The method of claim 1 wherein the set of objects relates to a plurality of webpages and wherein the step of generating a scalar value accounts for a number of times a webpage has been visited.

8. The method of claim 1 wherein a plurality of the objects in the set of objects relates to webpages, more than one of the pages include source links and each source link is a direct non-semantic relationship, and wherein generating a scalar value accounts for a quality of source links included within a webpage.

9. The method of claim 1 wherein the step of sending information for display includes providing one or more result nodes that include embedded active links in text associated with the one or more result nodes and said embedded active links provide options for displaying the content of an object in the resultant set of objects and displaying a pool of objects that have non-semantic link relationships with an object in the resultant set of objects.

10. The method of claim 1, wherein generating a scalar value is done iteratively, and each iteration includes analyzing indirect non-semantic relationships at least one additional link length from the previous iteration and a value from each of the iterations contributes to the scalar value.

11. The method of claim 1 wherein quantifying said object'"'"'s indirect non-semantic relationships comprises recursively analyzing indirect non-semantic relationships, wherein said recursive analysis accounts for indirect non-semantic relationships that a first object has with a second object based upon the second object'"'"'s direct relationships with other objects in the set of objects, and wherein recursively analyzing includes an iterative calculation wherein the number of iterative calculations is based upon the number of link lengths in the indirect non-semantic relationships to be analyzed.

12. The method of claim 11 wherein the generating a scalar value includes damping, wherein a damping factor is used to lessen a contribution to the scalar value of indirect non-semantic relationships as the number of link lengths in the indirect non-semantic relationship increases;

and wherein the number of iterative calculations are limited and quantified indirect non-semantic relationships are limited to relationships within a number of link lengths from said object, and wherein indirect non-semantic relationships not quantified for a given object would have an insignificant effect on the scalar value.

13. The method of claim 1 wherein the use of the one or more of said object'"'"'s quantity of outbound direct relationships is as a divisor in a ratio.

14. The method of claim 1 wherein storing the generated scalar values is performed prior to receiving the search commands and wherein the scalar values are stored in a database.

15. The method of claim 1 wherein scalar values are generated independent from the search commands.

16. The method of claim 1 wherein quantifying said object'"'"'s indirect non-semantic relationships further includes weighing, wherein some types of indirect relationships are weighed more heavily than others.

17. The method of claim 1 wherein said rank is an importance rank.

18. A data processing system for use in cluster analysis of a world wide web having identifiable web pages and hyperlink relationships made up of Universal Resource Locaters with pointers, wherein objects are related to the world wide web, direct non-semantic relationships relate to hyperlink relationships, and indirect non-semantic relationships relate to a series of hyperlink relationships between objects, comprising:

one or more computer processors for producing results and sending results for display configured to execute instructions to;

control a crawl of the web pages for Universal Resource Locators, wherein the Universal Resource Locators either point to or point away from one or more of the web pages;

define, using an identification, each object in a first set objects, wherein the objects are individually identified by the identification;

generate a numerical representation of direct non-semantic relationships in the first set of objects wherein the numerical representation quantifies into a plurality of numbers the direct non-semantic relationships between objects in the first set of objects and the quantification of the direct non-semantic relationships includes according a weight using an objects quantity of outbound direct non-semantic relationships;

calculate a scalar value for each object in the first set of objects, the scalar value valuing the direct and indirect non-semantic relationships existing among objects, wherein the calculation values at least the following indirect non-semantic relationship patterns for a given object A when present;

i) B cites f and f cites A,

ii) B cites f, f cites e, and e cites A, and

iii) B cites f, f cites e, e cites d, and d cites A, wherein B, d, e and f are objects in the first set of objects and certain indirect non-semantic relationships contribute greater value to the scalar value than other indirect non-semantic relationships;

receive search input including one or more search terms emanating from an input device;

identify, using the one or more search terms, a word index and the assigned identifiers, a second set of objects, wherein the second set of objects is a subset of the first set of objects having fewer objects;

rank a plurality of the objects in the second set of objects, wherein the scalar value is used as a factor in performing the ranking; and

send search results about the ranked objects for display, wherein identifying information for two or more of the ranked objects is sent to be displayed in a ranked order display;

one or more computer memory devices that store data including;

the identification for each object in the first set of objects,

the numerical representation,

the scalar values, and

the word index,

wherein the scalar values are stored in the one or more computer memory devices before the one or more computer processors process the received search input; and

a network for use by the one or more computer processors.

19. The system of claim 18 further comprising:

one or more robots or spiders to crawl the world wide web, wherein the one or more robots or spiders read HTML instructions;

and wherein the one or more computer processors are further configured to;

use a damping factor so that indirect relationships of greater link lengths have less impact on the scalar value;

examine, for each object that has an indirect non-semantic relationship, a subset of objects in the first set of objects, wherein the subset of objects is less than the total number objects in the first set of objects, wherein determining the subset comprises examining all of the objects in the first set of objects that have an indirect relationship within a certain number of link lengths within a plurality of relationship patterns;

wherein a quantity of outbound direct relationships of B is used as a weight; and

wherein a value for at least the three indirect non-semantic relationship patterns i), ii), and iii) when present are combined when calculating the scalar value.

20. The system of claim 18 wherein the search results that are sent for display include one or more result nodes that include embedded active links in text associated with the one or more result nodes and said embedded active links provide options for displaying content of one of the ranked objects and displaying a second pool of objects that have non-semantic link relationships with one of the ranked objects.

21. The system of claim 18 wherein the one or more computer processors are further configured to recursively calculate, wherein the scalar value is recursively calculated by valuing for each object in the first set of objects indirect non-semantic relationships and wherein the recursively calculating accounts for indirect non-semantic relationships that a first object in the first set of objects has with a second object in the first set of objects based upon the second object'"'"'s direct relationships with other objects in the first set of objects, wherein the recursively calculating includes an iterative calculation wherein the number of iterative calculations is based upon the number of link lengths in the indirect non-semantic relationships to be analyzed, wherein the iterative calculation includes damping, and wherein a damping factor is used to lessen a contribution to the scalar value from indirect non-semantic relationships as the number of link lengths in the indirect non-semantic relationship increases.

22. The system of claim 21 wherein the number of iterative calculations are limited and valued indirect relationships are limited to relationships within a number of link lengths of a given object, and wherein the indirect non-semantic relationships not valued for a given object would have an insignificant effect on the scalar value.