Reduction of seach ambiguity with multiple media references

• US 20060074861A1
• Filed: 09/30/2002
• Published: 04/06/2006
• Est. Priority Date: 09/30/2002
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Wilensky, Gregg D.
• Granted Patent: US 9,684,675 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/56 having vectorial format
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/24133 Distances to prototypes
• CPC Class Codes: G06T 7/40 Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects

Abstract

Methods and apparatus implementing a technique for searching media objects. In general, in one aspect, the technique includes receiving user input specifying a plurality of reference objects (2), defining a set of features for them, and combining the features to generate composite reference information (4) defining criteria for search (6). In general, in another aspect, the technique includes combining object information for a plurality of reference objects to produce composite reference information, comparing (10) the composite reference information to object information for media objects in a collection of media objects, and selecting a media object based upon the comparison

Claims

1. A method for searching a collection of media objects, comprising:

combining information pertaining to a feature common to a plurality of reference objects to produce composite reference information representing criteria for a search; and

comparing the composite reference information to information pertaining to the same feature for each respective one of a plurality of media objects in a collection of media objects to identify one or more media objects;

wherein a non-Euclidian function is used either to combine the information pertaining to the feature, or to compare the composite reference information to information pertaining to the same feature.

2. The method of claim 1, further comprising:

receiving user input specifying the plurality of reference objects.

3. The method of claim 1, further comprising:

selecting a media object in the collection of media objects based upon the comparison of the information pertaining to the feature for each media object and the composite reference information.

4. The method of claim 1, wherein:

the plurality of reference objects includes one or more objects having a type selected from;

audio, image, text, CD, or video.

5. The method of claim 4, wherein:

combining information pertaining to a feature common to a plurality of reference objects includes combining information for different types of objects.

6. The method of claim 1, wherein:

combining information pertaining to a feature common to a plurality of reference objects to produce composite reference information includes determining the intersection of the information for the reference objects.

7. The method of claim 1, wherein:

combining information pertaining to a feature common to a plurality of reference objects to produce composite reference information includes determining the union of the information for the reference objects.

8. The method of claim 1, further comprising:

combining information pertaining to a second feature common to the plurality of reference objects to produce additional composite reference information representing criteria for the search; and

comparing the additional composite reference information to information pertaining to the second feature for each respective one of the plurality of media objects in the collection of media objects to identify one or more media objects.

9. The method of claim 8, wherein:

the information pertaining to a feature and the information pertaining to a second feature is weighted to specify a relative importance of the features.

10. The method of claim 8, further comprising:

receiving user input indicating the relative importance of the feature and the second feature.

11. The method of claim 8, wherein:

the feature and the second feature are each represented by a relative frequency of occurrence of a feature value.

12. The method of claim 8, wherein:

information pertaining to the feature and information pertaining to the second feature includes color information describing the relative frequency of occurrence of colors in an object.

13. The method of claim 8, wherein:

information pertaining to the feature is mapped to information pertaining to the second feature.

14. The method of claim 1, further comprising:

combining information pertaining to the feature for an additional reference object with the composite reference information to revise the composite reference information.

15. The method of claim 14, wherein:

the additional reference is a media object identified by comparing the composite reference information to information pertaining to the feature for each respective one of the plurality of media objects.

16. The method of claim 14, further comprising:

comparing the revised composite reference information to information for the feature for each of a second plurality of media objects in the collection of media objects.

17. The method of claim 1, wherein:

comparing the composite reference information to information pertaining to the same feature for each respective one of a plurality of media objects in a collection of media objects includes assigning a similarity value to each respective one of the media objects in the collection of media objects, each similarity value indicating the similarity of the information for the media object and the composite reference information.

18. The method of claim 17, wherein:

each similarity value of each of the media objects in the collection of media objects is less than or equal to a similarity value calculated for each reference object.

19. The method of claim 17, further comprising:

ranking the media objects according to their similarity values; and

selecting one or more media objects in the collection of media objects based upon its rank.

20. The method of claim 8, wherein:

for each reference and media object, the information pertaining to the feature and the information pertaining to the second feature is expressed as a feature vector of components.

21. The method of claim 20, wherein:

combining information pertaining to a feature and combining information pertaining to a second feature common to a plurality of reference objects includes combining the feature vectors of the plurality of reference objects to produce a composite reference vector.

22. The method of claim 21, wherein:

each feature vector includes one or more components representing metadata associated with the corresponding reference or media object; and

combining information pertaining to a feature and combining information pertaining to a second feature common to a plurality of reference objects includes combining components representing the feature or the second feature according to a first combination function and combining the one or more components representing metadata associated with each reference object according to a second combination function.

23. The method of claim 21, further comprising:

defining a weighting vector for the feature and the second feature, the weighting vector specifying a relative importance for the corresponding features;

wherein combining the feature vectors includes using the weighting vector to specify a relative importance of the features.

24. The method of claim 21, wherein:

combining the feature vectors includes using a Min or Max function.

25. The method of claim 21, wherein:

comparing the composite reference information to information pertaining to the same feature for each respective one of a plurality of media objects in a collection of media objects includes comparing the composite reference vector to a feature vector of each of the plurality of media objects in the collection of media objects.

26. The method of claim 25, wherein:

comparing the composite reference vector to the feature vectors of each of a plurality of media objects includes using a Min or Max function.

27. The method of claim 1, wherein:

combining information pertaining to a feature common to a plurality of reference objects includes using a combination function;

comparing the composite reference information to information pertaining to the same feature for each respective one of a plurality of media objects in a collection of media objects includes using a comparison function that is based upon the combination function.

28. A method for searching a collection of media objects, comprising:

combining information pertaining to two or more features common to a plurality of reference objects to produce composite reference information representing criteria for a search, wherein the information is expressed as a feature vector of components and combining includes combining the feature vectors of the plurality of reference objects using a Min or Max function to produce a composite reference vector; and

comparing the composite reference information to information pertaining to the same feature for each respective one of a plurality of the media objects in the collection of media objects, wherein comparing includes comparing the composite reference vector to the feature vectors of each media object in the collection of media objects using a Min or Max function and assigning a similarity value to each media object in the collection of media objects, the similarity value indicating the similarity of the feature vector of the media object to the composite reference vector, where the similarity value of each of the media objects in the collection of media objects is less than or equal to a similarity value calculated for each reference object.

29. A computer program product on a computer-readable medium, for searching a collection of media objects, the computer program comprising instructions operable to cause a programmable processor to:

combine information pertaining to a feature common to a plurality of reference objects to produce composite reference information representing criteria for a search;

compare the composite reference information to information pertaining to the same feature for each respective one of a plurality of media objects in a collection of media objects to identify one or more media objects;

wherein a non-Euclidian function is used either to combine the information pertaining to the feature, or to compare the composite reference information to information pertaining to the same feature.

30. The computer program product of claim 29, further comprising instructions operable to cause a programmable processor to:

receive user input specifying the plurality of reference objects.

31. The computer program product of claim 29, further comprising instructions operable to cause a programmable processor to:

select a media object in the collection of media objects based upon the comparison of the information pertaining to the feature for each media object and the composite reference information.

32. The computer program product of claim 29, wherein:

the plurality of reference objects includes one or more objects having a type selected from;

audio, image, text, CD, or video.

33. The computer program product of claim 32, wherein:

instructions to combine information pertaining to a feature common to a plurality of reference objects include instructions to combine information for different types of objects.

34. The computer program product of claim 29, wherein:

instructions to combine information pertaining to a feature common to a plurality of reference objects to produce composite reference information include instructions to determine the intersection of the information for the reference objects.

35. The computer program product of claim 29, wherein:

instructions to combine object information pertaining to a feature common to a plurality of reference objects to produce composite reference information include instructions to determine the union of the information for the reference objects.

36. The computer program product of claim 29, further comprising:

instructions to combine information pertaining to a second feature common to the plurality of reference objects to produce additional composite reference information representing criteria for the search; and

instructions to compare the additional composite reference information to information pertaining to the second feature for each respective one of the plurality of media objects in the collection of media objects to identify one or more media objects.

37. The computer program product of claim 36, wherein:

the information pertaining to a feature and the information pertaining to a second feature is weighted to specify a relative importance of the features.

38. The computer program product of claim 36, further comprising instructions operable to cause a programmable processor to:

receive user input indicating the relative importance of the feature and the second feature.

39. The computer program product of claim 36, wherein:

the feature and the second feature are each represented by a relative frequency of occurrence of a feature value.

40. The computer program product of claim 36, wherein:

information pertaining to the feature and information pertaining to the second feature includes color information describing the relative frequency of occurrence of colors in an object.

41. The computer program product of claim 36, wherein:

information pertaining to the feature is mapped to information pertaining to the second feature.

42. The computer program product of claim 29, further comprising instructions operable to cause a programmable processor to:

combine information pertaining to the feature for an additional reference object with the composite reference information to revise the composite reference information.

43. The computer program product of claim 42, wherein:

the additional reference is a media object identified by comparing the composite reference information to information pertaining to the feature for each respective one of the plurality of media objects.

44. The computer program product of claim 42, further comprising instructions operable to cause a programmable processor to:

compare the revised composite reference information to information for the feature for each of a second plurality of media objects in the collection of media objects.

45. The computer program product of claim 29, wherein:

instructions to compare the composite reference information to information pertaining to the same feature for each respective one of a plurality of media objects in a collection of media objects include instructions to assign a similarity value to each respective one of the media objects in the collection of media objects, each similarity value indicating the similarity of the information for the media object and the composite reference information.

46. The computer program product of claim 45, wherein:

each similarity value of each of the media objects in the collection of media objects is less than or equal to a similarity value calculated for each reference object.

47. The computer program product of claim 45, further comprising instructions operable to cause a programmable processor to:

rank the media objects according to their similarity values; and

select one or more media objects in the collection of media objects based upon its rank.

48. The computer program product claim 36, wherein:

for each reference and media object, the information pertaining to the feature and the information pertaining to the second feature is expressed as a feature vector of components.

49. The computer program product of claim 48, wherein:

instructions to combine information pertaining to a feature and instructions to combine information pertaining to a second feature common to a plurality of reference objects include instructions to combine the feature vectors of the plurality of reference objects to produce a composite reference vector.

50. The computer program product of claim 49, wherein:

each feature vector includes one or more components representing metadata associated with the corresponding reference or media object; and

combining information pertaining to a feature and combining information pertaining to a second feature common to a plurality of reference objects includes combining components representing the feature or the second feature according to a first combination function and combining the one or more components representing metadata associated with each reference object according to a second combination function.

51. The computer program product of claim 49, further comprising instructions operable to cause a programmable processor to:

define a weighting vector for the feature and the second feature, the weighting vector specifying a relative importance for the corresponding features;

wherein instructions to combine the feature vectors include instructions to use the weighting vector to specify a relative importance of the features.

52. The computer program product of claim 49, wherein:

instructions to combine the feature vectors include instructions to use a Min or Max function.

53. The computer program product of claim 49, wherein:

instructions to compare the composite reference information to information pertaining to the same feature for each respective one of a plurality of media objects in a collection of media objects include instructions to compare the composite reference vector to a feature vector of each of the plurality of media objects in the collection of media objects.

54. The computer program product of claim 53, wherein:

instructions to compare the composite reference vector of the reference object to the feature vectors of each of a plurality of media objects include instructions to use a Min or Max function.

55. The computer program product of claim 29, wherein:

instructions to combine information pertaining to a feature common to a plurality of reference objects include instructions to use a combination function;

instructions to compare the composite reference information to information pertaining to the same feature for each respective one of a plurality of media objects in a collection of media objects include instructions to use a comparison function that is based on the combination function.

56. A computer program product on a computer-readable medium, for searching a collection of media objects, the computer program comprising instructions operable to cause a programmable processor to:

combine information pertaining to two or more features common to a plurality of reference objects to produce composite reference information representing criteria for a search, wherein the information is expressed as a feature vector of components and the instructions to combine include instructions to combine the feature vectors of the plurality of reference objects using a Min or Max function to produce a composite reference vector; and

compare the composite reference information to information pertaining to the same feature for each respective one of a plurality of the media objects in the collection of media objects includes comparing the composite reference vector to the feature vectors of each media object in the collection of media objects using a Min or Max function and assigning a similarity value to each media object in the collection of media objects, the similarity value indicating the similarity of the feature vector of the media object to the composite reference vector, where the similarity value of each of the media objects in the collection of media objects is less than or equal to a similarity value calculated for each reference object.

57. A system for searching a collection of media objects, comprising:

means for combining information pertaining to a feature common to a plurality of reference objects to produce composite reference information representing criteria for a search;

means for comparing the composite reference information to information pertaining to the same feature for each respective one of a plurality of media objects in a collection of media objects to identify one or more media objects;

wherein a non-Euclidian function is used either to combine the information pertaining to the feature, or to compare the composite reference information to information pertaining to the same feature.

58. The system of claim 57, wherein:

means for comparing the composite reference information to information pertaining to the same feature for each respective one of a plurality of media objects in the collection of media objects includes means for assigning a similarity value to each respective one of the media objects in the collection of media objects, each similarity value indicating the similarity of the information for the media object and the composite reference information, wherein the similarity value of each of the media objects in the collection of media objects is less than or equal to a similarity value calculated for each reference object.

59. A system for searching a collection of media objects, comprising:

means for combining information pertaining to two or more features common to a plurality of reference objects to produce composite reference information representing criteria for a search, wherein the information is expressed as a feature vector of components and means for combining includes means for combining the feature vectors of the plurality of reference objects to produce a composite reference vector; and

means for comparing the composite reference information to information pertaining to the same two or more features for each respective one of a plurality of media objects in a collection of media objects, wherein the means for comparing includes means for comparing the composite reference vector to the feature vectors of each of the media objects in the collection of media objects.