Systems and Related Methods of User-Guided Searching

• US 20090019036A1
• Filed: 07/09/2008
• Published: 01/15/2009
• Est. Priority Date: 07/10/2007
• Status: Active Grant

• Current Assignee: Asim Roy, Patrick Mackin
• Original Assignee: Asim Roy, Patrick Mackin
• Inventors: Roy, Asim, Mackin, Patrick
• Granted Patent: US 8,713,001 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3326 using relevance feedback from the user, e.g. relevance feedback on documents, documents sets, document terms or passages
• CPC Class Codes: G06F 16/337 Profile generation, learning or modification
• CPC Class Codes: G06F 16/358 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

Systems and related methods of user-guided searching using preference feedback from user searching to arrive at user-preferred ordered results from a large collection of objects.

Claims

1. ) A method, relating to refining at least one search by at least one searcher, relating to finding relevant such at least one searcher'"'"'s relevant objects, comprising the step(s) of:

a) presenting to such at least one searcher at least one large collection of objects comprising at least one set of object characteristics, each such object comprising at least one subset of such at least one set of object characteristics;

b) gathering searcher overall evaluating relating to each single object of at least one sample subset of such at least one large collection of objects;

c) wherein such searcher overall evaluating comprises searcher-setting of at least one relevancy-value relating to each object of such at least one sample subset;

d) using such gathered searcher overall evaluating relating to each single object of at least one sample subset of such at least one large collection of objects, identifying at least one searcher evaluation subset of relevancy-ordered characteristics relating to such sampled objects;

e) using such identified at least one searcher evaluation subset of relevancy-ordered characteristics relating to such sampled objects, identifying at least one relevancy order of such at least one large collection of objects; and

f) presenting, to such at least one searcher, at least one first relevancy-ordered hierarchy comprising such at least one large collection of objects.

2. ) The method according to claim 1 wherein such searcher overall evaluating comprises searcher-setting of at least one scalar relevancy-value and/or of at least one comparative relevancy-value relating to each object of such at least one sample subset.

3. ) The method according to claim 1 further comprising:

a) after such presenting, to such at least one searcher, such at least one relevancy-ordered hierarchy comprising such at least one large collection of objects, repeating steps (b) through (e) of claim 1; and

b) performing a second presenting to such at least one searcher of at least one second relevancy-ordered hierarchy comprising such at least one large collection of objects.

4. ) The method according to claim 3 further comprising n repeats of claim 1 steps by:

a) after such performing an nth presenting to such at least one searcher of at least one nth relevancy-ordered hierarchy comprising such at least one large collection of objects, performing an (n+1)th presenting to such at least one searcher of at least one (n+1)th relevancy-ordered hierarchy comprising such at least one large collection of objects.

5. ) The method according to claim 2 wherein such searcher-setting of at least one scalar relevancy-value relating to each object of such at least one sample subset comprises the step(s) of:

a) searcher-assigning of at least one scalar such relevancy value to at least one such single object of such at least one sample subset;

b) wherein such at least one scalar such relevancy value represents at least one measure of relative relevancy value, of such at least one such single object of such at least one sample subset, relative to such searcher'"'"'s ideal most-relevant object being sought.

6. ) The method according to claim 5 wherein such at least one scalar such relevancy value, relative to such searcher'"'"'s ideal most-relevant object being sought, comprises a value-scale including negative values.

7. ) The method according to claim 5 wherein such at least one scalar such relevancy value, relative to such searcher'"'"'s ideal most-relevant object being sought, comprises at least one choice among at least one natural-language positive expression and at least one natural-language negative expression.

8. ) The method according to claim 1 wherein the step of, using such identified at least one searcher evaluation subset of relevancy-ordered characteristics relating to such sampled objects, identifying at least one relevancy order of such at least one large collection of objects, comprises the step(s) of:

a) determining at least one individual relevancy weight value of each at least one object relating to such sampled objects;

b) applying such at least one individual relevancy weight value, of each at least one object relating to such sampled objects, to each such object-included characteristic, thereby setting individual relevancy weight values of each such object-included characteristic; and

c) obtaining at least one overall relevancy weight value relating to each valued object characteristic by combination of each such at least one individual relevancy weight value of each such object-included characteristic.

9. ) The method according to claim 8 wherein the step of identifying at least one relevancy order of such at least one large collection of objects further comprises the step(s) of applying such overall relevancy weight value relating to each valued object characteristic to the object characteristics within each object of such at least one large collection of objects so that, if a selected characteristic occurs at least once in such collection object, such overall relevancy weight value relating to such characteristic becomes an addend in the total relevancy score assigned to such collection of objects.

10. ) The method according to claim 8 wherein the step of obtaining at least one overall relevancy weight value relating to each valued object characteristic by combination of each such at least one individual relevancy weight value of each such object-included characteristic comprises the step of determining at least one overall relevancy weight value of each such valued object characteristic by using both additive and multiplicative weighting.

11. ) The method according to claim 10 wherein such step of determining at least one overall relevancy weight value of each such valued object characteristic by using both additive and multiplicative weighting comprises:

a) applying the algorithm

maximize ε

,

subject to

$\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>w_i<br><br>D_{\mathrm{ki}}-\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>w_i<br><br>D_{\mathrm{ji}}\ge <br><br>\mathrm{\varepsilon <br><br>},<br><br>\forall <br><br>k\in <br><br>S_t,j\in <br><br>S_{t+1},t=1,\mathrm{\dots <br><br>}<br><br>,T-1$ $\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>w_i<br><br>D_{\mathrm{ji}}^{P_A}-\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>w_i<br><br>D_{\mathrm{ji}}^{P_B}\ge <br><br>\mathrm{\varepsilon <br><br>},<br><br>j=1,\mathrm{\dots <br><br>}<br><br>,R,<br><br>0\le <br><br>w_i\le <br><br>1,i=1,\mathrm{\dots <br><br>}<br><br>,M$ $\mathrm{\varepsilon <br><br>}\ge <br><br>0;<br><br>$

b) wherein

i) i represents such valued object characteristic,

ii) w represents such at least one overall relevancy weight value,

iii) D represents an object described by an array of such valued object characteristics,

iv) ε

represents the difference between the weighted values of all objects in such at least one large collection of objects,

v) S_t represents the set of objects within a scalar category t,

vi) T represents the total number of scalar categories,

vii) M represents the total number of such valued object characteristics within such at least one large collection of objects,

viii) P_A and P_B indicate a pair of relatively compared objects with P_A indicating the more preferred, and

ix) R represents the total number of direct preference statements.

12. ) The method according to claim 11 wherein the step of applying algorithm maximize ε

is iteratively performed to approximate such searcher value function.

13. ) The method according to claim 1 wherein such searcher-setting of at least one comparative relevancy-value relating to each object of such at least one sample subset comprises the step(s) of:

a) searcher-assigning of at least one comparative such relevancy value to at least two such single objects of such at least one sample subset;

b) wherein such at least one comparative such relevancy value represents at least one measure of relative relevancy value, of such at least one first of such at least two single objects of such at least one sample subset, relative to such at least one second of such at least two single objects.

14. ) The method according to claim 13 wherein such at least one comparative such relevancy value comprises at least one choice among at least one natural-language expression indicating more-relevancy-than and at least one natural-language expression indicating less-relevancy-than.

15. ) The method according to claim 14 wherein each such natural language choice for each object is given a relative numerical scalar value;

and each such relative numerical scalar value is given a position in an overall numerical scalar value among objects evaluated.

16. ) The method according to claim 1 wherein the step of presenting to such at least one searcher at least one large collection of objects comprising at least one set of object characteristics, each such object comprising at least one subset of such at least one set of object characteristics, comprises

a) relating to such at least one set of object characteristics, determining kinds of similarities among such at least one large collection of objects;

b) clustering at least some of such at least one large collections of objects by at least some of such kinds of similarities; and

c) presenting multiple resulting clusters of such objects to such at least one searcher;

d) wherein such at least one searcher is enabled to value a potentially larger variety of such objects.

17. ) The method according to claim 1 further comprising, prior to such step of presenting to such at least one searcher at least one large collection of objects comprising at least one set of object characteristics, each such object comprising at least one subset of such at least one set of object characteristics, the steps(s) of:

a) relating to such at least one set of object characteristics, determining kinds of similarities among such at least one large collection of objects;

b) clustering at least some of such at least one large collections of objects by at least some of such kinds of similarities; and

c) presenting multiple resulting clusters of such objects to such at least one searcher;

d) wherein such at least one searcher is enabled to value a potentially larger variety of such objects.

18. ) The method according to claim 17 wherein the step of determining kinds of similarities among such at least one large collection of objects comprises the steps of:

a) identifying at least one lesser-relevancy subset of such object characteristics having less relevancy in determining relevant similarities among objects within such multiple resulting clusters of such objects; and

b) performing the step of clustering at least some of such at least one large collections of objects by at least some of such kinds of similarities, only with respect to similarities relating to such object characteristics not within such identified at least one lesser-relevancy subset.

19. ) The method according to claim 13:

a) wherein such objects comprise documents and such object characteristics comprise natural-language strings within such documents; and

b) wherein such at least one lesser-relevancy subset of such object characteristics comprise such natural-language strings less relevant to human searching for “

topic”

similarity.

20. ) The method according to claim 1 further comprising the step of defining at least one set of object characteristics assignable among such large collection of objects;

wherein such step of defining at least one set of object characteristics is performed prior to the step of presenting to such at least one searcher at least one large collection of objects comprising at least one set of object characteristics, each such object comprising at least one subset of such at least one set of object characteristics.

21. ) A method, relating to refining at least one document search by at least one searcher, relating to finding relevant such at least one searcher'"'"'s relevant documents, comprising the step(s) of:

a) presenting to such at least one searcher at least one large collection of documents comprising at least one set of document natural-language strings, each such document comprising at least one subset of such at least one set of document natural-language strings;

b) gathering searcher overall evaluating relating to each single document of at least one sample subset of such at least one large collection of documents;

c) wherein such searcher overall evaluating comprises searcher-setting of at least one scalar relevancy-value and/or of at least one comparative relevancy-value for each document of such at least one sample subset;

d) using such gathered searcher overall evaluating relating to each single document of at least one sample subset of such at least one large collection of documents, identifying at least one searcher evaluation subset of relevancy-ordered document natural-language strings relating to such sampled documents;

e) using such identified at least one searcher evaluation subset of relevancy-ordered document natural-language strings relating to such sampled documents, identifying at least one relevancy order of such at least one large collection of documents; and

f) presenting, to such at least one searcher, at least one relevancy-ordered hierarchy of such at least one large collection of documents.

22. ) The method according to claim 21 wherein such at least one large collection of documents relates to vehicles.

23. ) The method according to claim 21 wherein such at least one large collection of documents relates to romantic interests.

24. ) A computer-implemented method comprising the steps of:

a) gathering preference feedback, relating to at least one object from at least one large collection of objects, from at least one user searcher;

b) approximating a value function of the at least one user searcher based on such gathered preference feedback; and

c) ordering such at least one large collection of objects based on such approximation of the value function of the at least one user searcher.

25. ) The computer-implemented method according to claim 24 wherein the step of approximating a value function of the at least one user searcher comprises iterative application of the algorithm

maximize ε

,

subject to

$\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>w_i<br><br>D_{\mathrm{ki}}-\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>w_i<br><br>D_{\mathrm{ji}}\ge <br><br>\mathrm{\varepsilon <br><br>},<br><br>\forall <br><br>k\in <br><br>S_t,j\in <br><br>S_{t+1},t=1,\mathrm{\dots <br><br>}<br><br>,T-1$ $\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>w_i<br><br>D_{\mathrm{ji}}^{P_A}-\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>w_i<br><br>D_{\mathrm{ji}}^{P_B}\ge <br><br>\mathrm{\varepsilon <br><br>},<br><br>j=1,\mathrm{\dots <br><br>}<br><br>,R,<br><br>0\le <br><br>w_i\le <br><br>1,i=1,\mathrm{\dots <br><br>}<br><br>,M$ $\mathrm{\varepsilon <br><br>}\ge <br><br>0;<br><br>$

a) wherein

i) i represents such valued object characteristic,

ii) w represents such at least one overall relevancy weight value,

iii) D represents an object described by an array of such valued object characteristics,

iv) ε

represents the difference between the weighted values of all objects in such at least one large collection of objects,

v) S_t represents the set of objects within a scalar category t,

vi) T represents the total number of scalar categories,

vii) M represents the total number of such valued object characteristics within such at least one large collection of objects,

viii) P_A and P_B indicate a pair of relatively compared objects with P_A indicating the more preferred, and

ix) R represents the total number of direct preference statements;

b) wherein after such iterative application of such algorithm maximize ε

, ordering such at least one large collection of objects based on such approximation of the value function of the at least one user searcher may occur.

26. ) A computer system, relating to refining at least one search by at least one searcher, relating to finding relevant such at least one searcher'"'"'s relevant objects, comprising:

a) at least one presenting computer interface adapted to present to such at least one searcher at least one large collection of objects comprising at least one set of object characteristics, each such object comprising at least one subset of such at least one set of object characteristics;

b) at least one gathering computer interface adapted to gather searcher overall evaluating relating to each single object of at least one sample subset of such at least one large collection of objects;

c) wherein such at least one gathering computer interface searcher overall evaluating comprises at least one gathering computer processor adapted to gather searcher-setting of at least one scalar relevancy-value and/or of at least one comparative relevancy-value relating to each object of such at least one sample subset;

d) at least one characteristic-evaluating computer processor, using such gathered searcher overall evaluating relating to each single object of at least one sample subset of such at least one large collection of objects, adapted to identify at least one searcher evaluation subset of relevancy-ordered characteristics relating to such sampled objects;

e) at least one object-evaluating computer processor, using such identified at least one searcher evaluation subset of relevancy-ordered characteristics relating to such sampled objects, adapted to identify at least one relevancy order of such at least one large collection of objects; and

f) at least one re-presenting computer interface adapted to present, to such at least one searcher, at least one first relevancy-ordered hierarchy comprising such at least one large collection of objects.

27. ) The computer system according to claim 26 further comprising:

a) at least one computer processor structured and arranged to repeat the following steps

i) gathering searcher overall evaluating relating to each single object of at least one sample subset of such at least one large collection of objects;

ii) wherein such searcher overall evaluating comprises searcher-setting of at least one relevancy-value relating to each object of such at least one sample subset;

iii) using such gathered searcher overall evaluating relating to each single object of at least one sample subset of such at least one large collection of objects, identifying at least one searcher evaluation subset of relevancy-ordered characteristics relating to such sampled objects;

iv) using such identified at least one searcher evaluation subset of relevancy-ordered characteristics relating to such sampled objects, identifying at least one relevancy order of such at least one large collection of objects;

b) at least one second re-presenting interface adapted to present, to such at least one searcher, at least one second relevancy-ordered hierarchy comprising such at least one large collection of objects.

28. ) A computer system, relating to refining at least one search by at least one searcher, relating to finding relevant such at least one searcher'"'"'s relevant objects, comprising:

a) computer means for presenting to such at least one searcher at least one large collection of objects comprising at least one set of object characteristics, each such object comprising at least one subset of such at least one set of object characteristics;

b) computer means for gathering searcher overall evaluating relating to each single object of at least one sample subset of such at least one large collection of objects;

c) wherein such searcher overall evaluating comprises searcher-setting of at least one relevancy-value relating to each object of such at least one sample subset;

d) computer means for using such gathered searcher overall evaluating relating to each single object of at least one sample subset of such at least one large collection of objects, identifying at least one searcher evaluation subset of relevancy-ordered characteristics relating to such sampled objects;

e) computer means for, using such identified at least one searcher evaluation subset of relevancy-ordered characteristics relating to such sampled objects, identifying at least one relevancy order of such at least one large collection of objects; and

f) computer means for presenting, to such at least one searcher, at least one first relevancy-ordered hierarchy comprising such at least one large collection of objects.

29. ) The computer system according to claim 28 wherein such searcher overall evaluating comprises computer means for searcher-setting of at least one scalar relevancy-value and/or of at least one comparative relevancy-value relating to each object of such at least one sample subset.

30. ) The computer system according to claim 28 further comprising:

a) computer means for, after such presenting, to such at least one searcher, such at least one relevancy-ordered hierarchy comprising such at least one large collection of objects, repeating computer processing of elements (b) through (e) of claim 28; and

b) computer means for performing a second presenting to such at least one searcher of at least one second relevancy-ordered hierarchy comprising such at least one large collection of objects.

31. ) The computer system according to claim 30, further computer processing n repeats of claim 28, further comprising:

a) computer means for performing an (n+1)th presenting to such at least one searcher of at least one (n+1)th relevancy-ordered hierarchy comprising such at least one large collection of objects, after an nth presenting to such at least one searcher of at least one nth relevancy-ordered hierarchy comprising such at least one large collection of objects.

32. ) The computer system according to claim 29 wherein such computer means for searcher-setting of at least one scalar relevancy-value relating to each object of such at least one sample subset comprises:

a) computer means for searcher-assigning of at least one scalar such relevancy value to at least one such single object of such at least one sample subset;

b) wherein such at least one scalar such relevancy value represents at least one measure of relative relevancy value, of such at least one such single object of such at least one sample subset, relative to such searcher'"'"'s ideal most-relevant object being sought.

33. ) The computer system according to claim 32 wherein such at least one scalar such relevancy value, relative to such searcher'"'"'s ideal most-relevant object being sought, comprises a value-scale including negative values.

34. ) The computer system according to claim 32 wherein such at least one scalar such relevancy value, relative to such searcher'"'"'s ideal most-relevant object being sought, comprises at least one choice among at least one natural-language positive expression and at least one natural-language negative expression.

35. ) The computer system according to claim 28 wherein the computer means for, using such identified at least one searcher evaluation subset of relevancy-ordered characteristics relating to such sampled objects, identifying at least one relevancy order of such at least one large collection of objects, comprises:

a) computer means for determining at least one individual relevancy weight value of each at least one object relating to such sampled objects;

b) computer means for applying such at least one individual relevancy weight value, of each at least one object relating to such sampled objects, to each such object-included characteristic, thereby setting individual relevancy weight values of each such object-included characteristic; and

c) computer means for obtaining at least one overall relevancy weight value relating to each valued object characteristic by combination of each such at least one individual relevancy weight value of each such object-included characteristic.

36. ) The computer system according to claim 35 wherein the computer means for identifying at least one relevancy order of such at least one large collection of objects further comprises:

a) computer means for applying such overall relevancy weight value relating to each valued object characteristic to the object characteristics within each object of such at least one large collection of objects so that, if a selected characteristic occurs at least once in such collection object, such overall relevancy weight value relating to such characteristic becomes an addend in the total relevancy score assigned to such collection of objects.

37. ) The computer system according to claim 36 wherein the computer means for obtaining at least one overall relevancy weight value relating to each valued object characteristic by combination of each such at least one individual relevancy weight value of each such object-included characteristic comprises:

a) computer means for determining at least one overall relevancy weight value of each such valued object characteristic by using both additive and multiplicative weighting.

38. ) The computer system according to claim 37 wherein such computer means for determining at least one overall relevancy weight value of each such valued object characteristic by using both additive and multiplicative weighting comprises computer means for applying the algorithm

maximize ε

,

subject to

$\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>w_i<br><br>D_{\mathrm{ki}}-\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>w_i<br><br>D_{\mathrm{ji}}\ge <br><br>\mathrm{\varepsilon <br><br>},<br><br>\forall <br><br>k\in <br><br>S_t,j\in <br><br>S_{t+1},t=1,\mathrm{\dots <br><br>}<br><br>,T-1$ $\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>w_i<br><br>D_{\mathrm{ji}}^{P_A}-\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>w_i<br><br>D_{\mathrm{ji}}^{P_B}\ge <br><br>\mathrm{\varepsilon <br><br>},<br><br>j=1,\mathrm{\dots <br><br>}<br><br>,R,<br><br>0\le <br><br>w_i\le <br><br>1,i=1,\mathrm{\dots <br><br>}<br><br>,M$ $\mathrm{\varepsilon <br><br>}\ge <br><br>0;<br><br>$

a) wherein

i) i represents such valued object characteristic,

ii) w represents such at least one overall relevancy weight value,

iii) D represents an object described by an array of such valued object characteristics,

iv) ε

represents the difference between the weighted values of all objects in such at least one large collection of objects,

v) S_t represents the set of objects within a scalar category t,

vi) T represents the total number of scalar categories,

vii) M represents the total number of such valued object characteristics within such at least one large collection of objects,

viii) P_A and P_B indicate a pair of relatively compared objects with P_A indicating the more preferred, and

ix) R represents the total number of direct preference statements.

39. ) The computer system according to claim 38 comprising computer means for iteratively applying algorithm maximize ε

to approximate a value function of such searcher.

40. ) The computer system according to claim 29 wherein such computer means for searcher-setting of at least one comparative relevancy-value relating to each object of such at least one sample subset comprises:

a) computer means for searcher-assigning of at least one comparative such relevancy value to at least two such single objects of such at least one sample subset;

b) wherein such at least one comparative such relevancy value represents at least one measure of relative relevancy value, of such at least one first of such at least two single objects of such at least one sample subset, relative to such at least one second of such at least two single objects.

41. ) The computer system according to claim 40 wherein such at least one comparative such relevancy value comprises at least one choice among at least one natural-language expression indicating more-relevancy-than and at least one natural-language expression indicating less-relevancy-than.

42. ) The computer system according to claim 41 wherein each such natural language choice for each object is given a relative numerical scalar value;

and each such relative numerical scalar value is given a position in an overall numerical scalar value among objects evaluated.

43. ) The computer system according to claim 28 wherein the computer means for presenting to such at least one searcher at least one large collection of objects comprising at least one set of object characteristics, each such object comprising at least one subset of such at least one set of object characteristics, comprises

a) computer means for, relating to such at least one set of object characteristics, determining kinds of similarities among such at least one large collection of objects;

b) computer means for clustering at least some of such at least one large collections of objects by at least some of such kinds of similarities; and

c) computer means for presenting multiple resulting clusters of such objects to such at least one searcher;

d) wherein such at least one searcher is enabled to value a potentially larger variety of such objects.

44. ) The computer system according to claim 28 further comprising, prior to operation of such computer means for presenting to such at least one searcher at least one large collection of objects comprising at least one set of object characteristics, each such object comprising at least one subset of such at least one set of object characteristics:

a) computer means for, relating to such at least one set of object characteristics, determining kinds of similarities among such at least one large collection of objects;

b) computer means for clustering at least some of such at least one large collections of objects by at least some of such kinds of similarities; and

c) computer means for presenting multiple resulting clusters of such objects to such at least one searcher;

d) wherein such at least one searcher is enabled to value a potentially larger variety of such objects.

45. ) The computer system according to claim 44 wherein such computer means for determining kinds of similarities among such at least one large collection of objects comprises:

a) computer means for identifying at least one lesser-relevancy subset of such object characteristics shaving less relevancy in determining relevant similarities among objects within such multiple resulting clusters of such objects; and

b) computer means for performing the step, of clustering at least some of such at least one large collections of objects by at least some of such kinds of similarities, only with respect to similarities relating to such object characteristics not within such identified at least one lesser-relevancy subset.

46. ) The computer system according to claim 45:

a) wherein such objects comprise documents and such object characteristics comprise natural-language strings within such documents; and

b) wherein such at least one lesser-relevancy subset of such object characteristics comprise such natural-language strings less relevant to human searching for “

topic”

similarity.

47. ) A computer system, relating to refining at least one document search by at least one searcher, relating to finding relevant such at least one searcher'"'"'s relevant documents, comprising:

a) computer means for presenting to such at least one searcher at least one large collection of documents comprising at least one set of document natural-language strings, each such document comprising at least one subset of such at least one set of document natural-language strings;

b) computer means for gathering searcher overall evaluating relating to each single document of at least one sample subset of such at least one large collection of documents;

c) wherein such searcher overall evaluating comprises searcher-setting of at least one scalar relevancy-value and/or of at least one comparative relevancy-value for each document of such at least one sample subset;

d) computer means for, using such gathered searcher overall evaluating relating to each single document of at least one sample subset of such at least one large collection of documents, identifying at least one searcher evaluation subset of relevancy-ordered document natural-language strings relating to such sampled documents;

e) computer means for, using such identified at least one searcher evaluation subset of relevancy-ordered document natural-language strings relating to such sampled documents, identifying at least one relevancy order of such at least one large collection of documents; and

f) computer means for presenting, to such at least one searcher, at least one relevancy-ordered hierarchy of such at least one large collection of documents.

48. ) A method, relating to refining at least one search by at least one searcher, relating to finding relevant such at least one searcher'"'"'s relevant objects, comprising the step(s) of:

a) presenting to such at least one searcher at least one large collection of objects comprising at least one set of object characteristics, each such object comprising at least one subset of such at least one set of object characteristics;

b) gathering searcher overall evaluating relating to each single object of at least one sample subset of such at least one large collection of objects;

c) wherein such searcher overall evaluating comprises searcher-setting of at least one scalar relevancy-value and/or of at least one comparative relevancy-value relating to each object of such at least one sample subset;

d) using such gathered searcher overall evaluating relating to each single object of at least one sample subset of such at least one large collection of objects, identifying at least one searcher evaluation subset of relevancy-ordered characteristics relating to such sampled objects;

e) using such identified at least one searcher evaluation subset of relevancy-ordered characteristics relating to such sampled objects, identifying at least one relevancy order of such at least one large collection of objects; and

f) presenting, to such at least one searcher, at least one first relevancy-ordered hierarchy comprising such at least one large collection of objects.