SYSTEM AND METHOD PROVIDING AUTOMATED MARGIN TREE ANALYSIS AND PROCESSING OF SAMPLED DATA

• US 20060271512A1
• Filed: 08/07/2006
• Published: 11/30/2006
• Est. Priority Date: 06/25/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Burges, Christopher J.C.
• Granted Patent: US 7,672,939 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2246 Trees, e.g. B+trees
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation

Abstract

The present invention relates to a system and methodology to facilitate database processing in accordance with a plurality of various applications. In one aspect, a large database of objects is processed, wherein the objects can be represented as points in a vector space, and two or more objects are deemed ‘close’ if a Euclidean distance between the points is small. This can apply for substantially any type of object, provided a suitable distance measure can be defined. In another aspect, a ‘test’ object having a vector x, is processed to determine if there exists an object y in the database such that the distance between x and y falls below a threshold t. If several objects in the database satisfy this criteria, a list of objects can be returned, together with their corresponding distances. If no objects were to satisfy the criterion, an indication of this condition can also be provided, but in addition, the condition or information relating to the condition can be provided.

Claims

1. A computer related database lookup system, comprising:

a quantizer to form a vector having M components, M being an integer, the M components of the vector having at least one of three values (0, 1, and q); and

a component to perform a lookup in a database in accordance with the values, where q directs a backtrack on the lookup, if encountered.

2. The computer related system of claim 1, further comprising a vector generator that computes the vector from a second vector that is derived from at least one data source.

3. The computer related system of claim 2, the second vector includes N components, N being an integer, the N components being at least one of a floating point value and an integer value, the values stored in an array.

4. The computer related system of claim 1, the quantizer further comprises a threshold function T(x) to control at least one of the backtrack, lookup accuracy, lookup load balancing between systems, and noise margins with respect to processing the quantized vector.

5. The computer related system of claim 4, further comprising one or more controls to adjust the threshold function T(x).

6. The computer related system of claim 1, further comprising at least one of a local system and a remote system having associated databases processed in accordance with a lookup tree, the quantized vector employed to navigate through the lookup tree to determine if a test vector that is derived from the quantized vector matches portions of the database.

7. The computer related system of claim 6, the local system directs an optimized request via a network to facilitate searching in a database associated with the remote system, the optimized request including at least one index and a vector.

8. The computer related system of claim 6, further comprising one or more user inputs to facilitate user adjustments of the threshold function T(x).

9. The computer related system of claim 1, further comprising an orthonormal direction generator to compute a third vector that is employed in the lookup.

10. The computer related system of claim 1, further comprising a binary tree generator to map all quantized vectors into a binary tree, for those vectors that occur in the database.

11. A computer related database lookup method, comprising:

receiving a test vector x having d components;

creating a quantized vector {circumflex over (x)} that maps the d components to at least one of three values of 0, 1, and q, q denoting a database backtrack operation; and

looking up data in the database via the quantized vector {circumflex over (x)}.

12. The computer related method of claim 11, further comprising resealing and shifting the components of x so that each component has zero mean and unit variance in a representative dataset.

13. The computer related method of claim 11, further comprising testing the d components against threshold parameters t₂ and −

t₂ to determine quantized values, the threshold parameters relate to a threshold function T(x).

14. The computer related method of claim 13, testing a selected d-component as x_i in accordance with the threshold function T(x),

determining if x_i≧

t₂ then setting T(x_i)=1;

determining if x_i<

−

t₂ then setting T(x_i)=0, and

determining if −

t₂≦

x_i<

t₂ then setting T(x_i)=q.

15. The computer related method of claim 13, further comprising determining threshold parameters t₁, and t₃ to control backtracking complexity, determining a maximum of n backtrack symbols q for a given test point x, determining if the number of components x_i for which −

t₁<

x_i≦

t₁ is greater than or equal to n, then rejecting that point.

16. The computer related method of claim 15, further comprising:

determining a value for t₂ such that t₁<

t₂<

t₃ and such that, for n values of i, −

t₂≦

x_i≦

t₂ is established; and

mapping x_i to q in order that n locations in the test vector require backtracking.

17. The computer related method of claim 16, further comprising:

setting t₂=t₃ if no n values exist which satisfy t₂≦

x_i≦

t₂;

computing whether d<

n components satisfy t₂≦

x_i≦

t₂; and

setting selected components to q in order to flag that the selected components require backtracking.

18. The computer related method of claim 11, further comprising:

defining a set of N random orthonormal direction vectors s_j; and

generating a feature vector at whose j'"'"'th component is a dot product of x with s_j in order to mitigate noise generating large shifts in individual components of the vector x.

19. A computer related lookup method to mitigate computational costs when accessing a database, comprising:

constructing d binary trees having a depth less than or equal to d;

computing a number of edges for the trees;

selecting the tree having at least a lowest number of edges that must be traversed, for a given x; and

employing the selected tree to mitigate computational costs when accessing the database.

20. The computer related method of claim 19, further comprising the following expression to determine the lowest number of edges that must be traversed for a given x, given a simplifying assumption that the tree is complete:

$\underset{i=1}{\overset{d}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>2^{\underset{j=1}{\overset{i}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>r_j}\equiv <br><br>N<br><br>\left(z\right)$

wherein N is a number of edges computed for a respective binary tree, i and j are integers, d is the depth of the tree, z is a d-vector having elements specified in {0, 1, q), and r_i=1 if z_i=q, otherwise set r_i to 0.