System and method providing automated margin tree analysis and processing of sampled data

• US 7,672,939 B2
• Filed: 08/07/2006
• Issued: 03/02/2010
• Est. Priority Date: 06/25/2002
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Burges, Christopher J. C.
• Primary Examiner(s): Trujillo; James
• Assistant Examiner(s): SOMERS, MARC S
• Application Number:
  US11/462,932
• Publication Number:
  US 20060271512A1
• Time in Patent Office: 1,303 Days
• Field of Search: 707/100, 707/101, 707/104.1, 707/10
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2246 Trees, e.g. B+trees
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation

Abstract

The present invention relates to a system and methodology to facilitate database processing in accordance with a plurality of various applications. In one aspect, a large database of objects is processed, wherein the objects can be represented as points in a vector space, and two or more objects are deemed ‘close’ if a Euclidean distance between the points is small. This can apply for substantially any type of object, provided a suitable distance measure can be defined. In another aspect, a ‘test’ object having a vector x, is processed to determine if there exists an object y in the database such that the distance between x and y falls below a threshold t. If several objects in the database satisfy this criteria, a list of objects can be returned, together with their corresponding distances. If no objects were to satisfy the criterion, an indication of this condition can also be provided, but in addition, the condition or information relating to the condition can be provided.

Claims

1. A computer related database lookup system embodied on a computer-readable storage medium, comprising:

a quantizer that forms a quantized vector having M components, M being an integer, the M components of the quantized vector having at least one of three values (0, 1, and q), wherein the quantizer assigns a value q to at least one of the M components;

a threshold function T(x) processing the quantized vector to control at least one of a backtrack, lookup accuracy, lookup load balancing between systems, or noise margins;

a component that performs the lookup in a database in accordance with the values, where q directs the component to perform the backtrack during the lookup, when encountered; and

one or more controls to adjust the threshold function T(x).

2. The computer related system of claim 1, further comprising a vector generator that computes the quantized vector from a second vector that is derived from at least one data source.

3. The computer related system of claim 2, wherein the second vector includes N components, N being an integer, comprising at least one of a floating point value and an integer value, and wherein further the floating point value and the integer values are stored in an array.

4. The computer related system of claim 1, further comprising an orthonormal direction generator to compute a third vector that is employed in the lookup.

5. The computer related system of claim 1, further comprising a binary tree generator to map all quantized vectors into a binary tree, for those vectors that occur in the database.

6. A computer related database lookup system embodied on a computer-readable storage medium, comprising:

a quantizer that forms a quantized vector having M components, M being an integer, the M components of the quantized vector having at least one of three values (0, 1, and q), wherein the quantizer assigns a value q to at least one of the M components;

a threshold function T(x) processing the quantized vector to control at least one of a backtrack, lookup accuracy, lookup load balancing between systems, or noise margins;

a component that performs a lookup in a database in accordance with the values, where q directs the component to perform a backtrack during the lookup, when encountered;

at least one of a local system or a remote system having associated databases processed in accordance with a lookup tree;

the quantized vector employed to navigate through the lookup tree to determine if a test vector that is derived from the quantized vector matches portions of the database; and

one or more user inputs to facilitate user adiustments of the threshold function T(x).

7. The computer related system of claim 6, wherein the local system directs an optimized request via a network to facilitate searching in a database associated with the remote system, wherein further the optimized request including at least one index and a vector.

8. A computer related database lookup method, comprising:

receiving a test vector x having d components;

creating a quantized vector {circumflex over (x)} based on the test vector x that maps at least one of the d components to each of three values respectively indicating performing a left side traversal of a database, a right side traversal of the database, or a backtrack operation on the database;

looking up data in the database via the quantized vector {circumflex over (X)};

determining the quantized values by testing the d components against threshold parameters t₂ and −

t₂ to determine quantized values, the threshold parameters relating to a threshold function T(x); and

configuring threshold parameters t₁ and t₃ to control backtracking complexity, determining a maximum of n backtrack symbols q for a given test point x, and analyzing if the number of components x_i for which −

t₁<

x_i≦

t₁ is greater than or equal to n, then rejecting that point.

9. The computer related method of claim 8, further comprising rescaling and shifting the components of x so that each component has zero mean and unit variance in a representative dataset.

10. The computer related method of claim 8, further comprising testing a selected d-component as x_i in accordance with the threshold function T(x),

determining x_i≧

t₂ then setting T(x_i)=1;

determining x_i<

−

t₂ then setting T(x_i)=0, and

determining if −

t₂≦

x_i<

t₂ then setting T(x_i)=q.

11. The computer related method of claim 8, further comprising:

determining a value for t₂ such that t₁<

t₂<

t₃ and such that, for n values of i, −

t₂≦

x_i≦

t₂ is established; and

mapping x₁to q in order that n locations in the test vector require backtracking.

12. The computer related method of claim 11, further comprising:

setting t₂=t₃ if no n values exist which satisfy −

t₂≦

x_i≦

t₂;

computing whether d<

n components satisfy −

t₂≦

x_i≦

t₂; and

setting selected components to q in order to flag that the selected components require backtracking.

13. The computer related method of claim 8, further comprising:

defining a set of N random orthonormal direction vectors s_j; and

generating a feature vector at whose j'"'"'th component is a dot product of x with s_j in order to mitigate noise generating large shifts in individual components of the vector x.