Managing media objects in a database

• US 7,076,503 B2
• Filed: 12/19/2001
• Issued: 07/11/2006
• Est. Priority Date: 03/09/2001
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Attias, Hagai, Kagle, Jonathan, Milton, Victoria Elizabeth, Platt, John Carlton
• Primary Examiner(s): Rimell, Sam
• Assistant Examiner(s): Chojnacki, Mellissa M.
• Application Number:
  US10/021,255
• Publication Number:
  US 20030009469A1
• Time in Patent Office: 1,665 Days
• Field of Search: 345/433, 395/604, 395/605, 395/615, 707 1- 10, 707100-1041, 725/91, 725/97, 725/117
• US Class Current: 1/1
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/535 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure

Abstract

A method and apparatus are provided for organizing media objects in a database using contextual information for a media object and known media objects, categories, indexes and searches, to arrive at an inference for cataloging the media object in a database. The media object may then be cataloged in the database according to the inference. A method and apparatus are provided for clustering media objects by forming groups of unlabeled data and applying a distance metric to said group.

Claims

1. A method of organizing media objects in a database, comprising:

capturing a media object;

comparing the captured media object with media objects that are stored in the database;

identifying the stored media objects in the database that include features in common with the captured media object;

inferring organization information for the captured media object based upon information, obtained from each of the media objects including features in common with the captured media object, representing organization in the database; and

organizing the captured media object in the database based upon the inferred organization information,

wherein the step of inferring includes a step of determining whether data associated with the captured media object exceeds an adaptive threshold, wherein the adaptive threshold is based on a running average of log time gaps between media objects,

wherein the step of inferring includes a step of determining into which group of stored media objects to store the captured media object.

2. The method of claim 1, wherein the adaptive threshold is computed based on the formula:

$\log <br><br>\left(X_n\right)=\log <br><br>\left(17\right)+\frac{1}{20}<br><br>\underset{i=n-10}{\overset{n+9}{\sum <br><br>}}<br><br><br><br>\log <br><br>\left(t_i+1-t_i\right),$

wherein t_i is a creation time of a media object i and X_n is a threshold that is compared to a difference between a creation time of media object n and media object n+1.

19. The method according to claim 1, wherein the detecting step comprises:

detecting attributes of the captured media object generated when the media object was captured.

20. The method according to claim 1, wherein the organizing step includes storing the captured media object in the database, and wherein the method further comprises assigning at least one attribute to the metadata for the captured media object prior to storing the captured media object.

21. The method according to claim 1, further comprising:

assigning at least one attribute to the metadata for the captured media object based upon the inferred organization information.

22. The method according to claim 1, further comprising:

detecting common features of the stored media objects;

identifying the stored media objects that have common features; and

eliminating the stored media objects that are not identified prior to inferring the organization information.

23. The method according to claim 22, further comprising:

adding information to the attributes of the metadata of the captured media object based upon the common features of the stored media objects.

24. The method according to claim 23, further comprising:

adding information to the metadata of the captured media object indicating that the inferred organization information for the captured media object was determined based upon an inference.

25. The method according to claim 1, further comprising:

adding information to the metadata of the captured media object indicating that the inferred organization information for the captured media object was determined based upon an inference.

26. The method of claim 1, wherein the adaptive threshold is a temporal designation and the group of stored media objects is a collection.

27. The method of claim 1, wherein the adaptive threshold is computed using the formula:

$\log <br><br>\left(X_n\right)=\log <br><br>\left(17\right)+\frac{1}{20}<br><br>\underset{i=n-10}{\overset{n+9}{\sum <br><br>}}<br><br><br><br>\log <br><br>\left(t_i+1-t_i\right),$

wherein t_i is a creation time of a media object i and X_n is a threshold that is compared to a difference between a creation time of media object n and media object n+1.

3. A method of organizing media objects in a database, comprising:

capturing a media object;

identifying a feature of the captured media object;

comparing the feature of the captured media object with stored media objects that are stored in the database;

identifying the stored media objects having the identified feature;

inferring organization information for the captured media object based upon information obtained from each of the stored media objects having the identified feature of the captured media object; and

organizing the captured media object in the database based upon the inferred organization information,

wherein the step of inferring includes a step of determining whether the identified feature of the captured media object exceeds an adaptive threshold, wherein the adaptive threshold is based on a running average of log time gaps between media objects,

wherein the step of inferring includes a step of determining into which group of stored media objects to store the captured media object.

4. The method of claim 3, wherein the adaptive threshold is computed based on the formula:

$\log <br><br>\left(X_n\right)=\log <br><br>\left(17\right)+\frac{1}{20}<br><br>\underset{i=n-10}{\overset{n+9}{\sum <br><br>}}<br><br><br><br>\log <br><br>\left(t_i+1-t_i\right),$

wherein t_i is a creation time of a media object i and X_n is a threshold that is compared to a difference between a creation time of media object n and media object n+1.

5. A method of organizing media objects in a database, comprising:

capturing a media object;

identifying a feature of the captured media object;

performing an inexact search to detect stored media objects that are stored in the database having the identified feature of the captured media object;

identifying the stored media objects having the identified feature of the captured media object;

inferring organization information for the captured media object based upon information obtained from each of the stored media objects having the feature identified in the captured media object; and

organizing the captured media object in the database based upon the inferred organization information,

wherein the step of inferring includes a step of determining whether the identified feature of the captured media object exceeds an adaptive threshold, wherein the adaptive threshold is based on a running average of log time gaps between media objects,

wherein the step of inferring includes a step of determining into which group of stored media objects to store the captured media object.

6. The method of claim 5, wherein the adaptive threshold is computed based on the formula:

$\log <br><br>\left(X_n\right)=\log <br><br>\left(17\right)+\frac{1}{20}<br><br>\underset{i=n-10}{\overset{n+9}{\sum <br><br>}}<br><br><br><br>\log <br><br>\left(t_i+1-t_i\right),$

wherein t_i is a creation time of a media object i and X_n is a threshold that is compared to a difference between a creation time of media object n and media object n+1.

7. A method of organizing media objects in a database, comprising:

capturing a media object;

identifying media objects stored in a database that are related to the captured media object;

obtaining information from each of the stored media objects that are related to the captured media object;

determining where the captured media object is to be stored with respect to the stored media objects that are related to the captured media object based upon the obtained information; and

storing the captured media object in the database,

wherein the step of determining where the captured media object is to be stored further comprises the step of computing an adaptive threshold, wherein the adaptive threshold is based on a running average of log time gaps between media objects.

8. The method of claim 7, wherein the adaptive threshold is computed based on the formula:

$\log <br><br>\left(X_n\right)=\log <br><br>\left(17\right)+\frac{1}{20}<br><br>\underset{i=n-10}{\overset{n+9}{\sum <br><br>}}<br><br><br><br>\log <br><br>\left(t_i+1-t_i\right),$

wherein t_i is a creation time of a media object i and X_n is a threshold that is compared to a difference between the creation time of media object n and media object n+1.

9. A computer-readable medium having computer-executable instructions for performing the steps of:

capturing a media object;

detecting attributes of the captured media object indicated in metadata for the captured media object;

identifying media objects stored in the database that are related to the captured media object;

inferring organization information for the captured media object based upon information obtained from each of the stored media objects that are related to the captured media object; and

organizing the captured media object in the database based upon the inferred organization information,

wherein the step of inferring includes a step of determining whether data associated with the captured media object exceeds an adaptive threshold, wherein the adaptive threshold is based on a running average of log time gaps between media objects,

wherein the step of inferring includes a step of determining into which group of stored media objects to store the captured media object.

10. The computer-readable medium according to claim 9, wherein the detecting step comprises:

detecting attributes of the captured media object generated when the captured media object was captured.

11. The computer-readable medium according to claim 9, wherein the organizing step includes storing the captured media object in the database, the computer-readable medium having further computer-executable instructions for performing the step of assigning at least one attribute to the metadata for the captured media object prior to storing the captured media object.

12. The computer-readable medium according to claim 9, having further computer-executable instructions for performing the step of:

assigning at least one attribute to the metadata for the captured media object based upon the inferred organization information.

13. The computer-readable medium according to claim 9, having further computer-executable instructions for performing the steps of:

detecting common features of the stored media objects;

identifying the stored media objects that have common features; and

eliminating the stored media objects that are not identified prior to inferring the organizing information.

14. The computer-readable medium according to claim 13, having further computer-executable instructions for performing the step of:

adding information to the attributes of the metadata of the captured media object based upon the common features of the stored media objects.

15. The computer-readable medium according to claim 14, having further computer-executable instructions for performing the step of:

adding information to the metadata of the captured media object indicating that the inferred organization information for the captured media object was determined based upon an inference.

16. The computer-readable medium according to claim 9, having further computer-executable instructions for performing the step of:

adding information to the metadata of the captured media object indicating that the inferred organization information for the captured media object was determined based upon an inference.

17. The computer-readable medium having computer-executable instructions of claim 9, wherein the adaptive threshold is computed based on the formula:

$\log <br><br>\left(X_n\right)=\log <br><br>\left(17\right)+\frac{1}{20}<br><br>\underset{i=n-10}{\overset{n+9}{\sum <br><br>}}<br><br><br><br>\log <br><br>\left(t_i+1-t_i\right),$

wherein t_i is a creation time of a media object i and X_n is a threshold that is compared to a difference between a creation time of media object n and media object n+1.

18. A method of organizing media objects in a database, comprising:

capturing a media object;

detecting attributes of the captured media object indicated in metadata for the captured media object;

identifying media objects stored in the database that are related to the captured media object;

inferring organization information for the captured media object based upon information obtained from each of the stored media objects that are related to the captured media object; and

organizing the captured media object in the database based upon the inferred organization information,

wherein the step of inferring includes a step of determining whether data associated with the captured media object exceeds an adaptive threshold, wherein the adaptive threshold is based on a running average of log time gaps between media objects,

wherein the step of inferring includes a step of determining into which group of stored media objects to store the captured media object.

28. A method of organizing media objects in a database, comprising:

capturing a media object;

detecting attributes of the captured media object indicated in metadata for the captured media object;

performing an inexact search of the database based upon at least one of the attributes of the captured media object to identify media objects stored in the database that are related to the captured media object;

inferring organization information for the captured media object based upon information obtained from each of the stored media objects that are related to the captured media object; and

organizing the captured media object in the database based upon the inferred organization information,

wherein the step of inferring includes a step of determining whether data associated with the captured media object exceeds an adantive threshold, wherein the adaptive threshold is based on a running average of log time gaps between media objects,

wherein the step of inferring includes a step of determining into which group of stored media objects to store the captured media object.

29. The method according to claim 28, wherein the inexact search step comprises:

performing an inexact search of the database based upon a date on which the captured media object was captured, wherein the date comprises one of the attributes of the captured media object.

30. The method according to claim 28, wherein the inexact search step comprises:

performing an inexact search of the database based upon a location where the captured media object was captured, wherein the location comprises one of the attributes of the captured media object.

31. The method of claim 28, wherein the adaptive threshold is computed based on the formula:

$\log <br><br>\left(X_n\right)=\log <br><br>\left(17\right)+\frac{1}{20}<br><br>\underset{i=n-10}{\overset{n+9}{\sum <br><br>}}<br><br><br><br>\log <br><br>\left(t_i+1-t_i\right),$

wherein t_i is a creation time of a media object i and X_n is a threshold that is compared to a difference between a creation time of media object n and media object n+1.

32. A computer-readable medium having computer-executable instructions for performing the steps of:

capturing a media object;

detecting attributes of the captured media object indicated in metadata for the captured media object;

performing an inexact search of the database based upon at least one of the attributes of the captured media object to identify stored media objects stored in the database that are related to the captured media object;

inferring organization information for the captured media object based upon information obtained from each of the stored media objects that are related to the captured media object; and

organizing the captured media object in the database based upon the inferred organization information,

wherein the step of inferring includes a step of determining whether data associated with the captured media object exceeds an adaptive threshold, wherein the adaptive threshold is based on a running average of log time gaps between media objects,

wherein the step of inferring includes a step of determining into which group of stored media objects to store the captured media object.

33. The computer-readable medium according to claim 32, wherein the inexact search step comprises:

performing an inexact search of the database based upon a date on which the captured media object was captured, wherein the date comprises one of the attributes of the captured media object.

34. The computer-readable medium according to claim 32, wherein the inexact search step comprises:

performing an inexact search of the database based upon a location where the captured media object was captured, wherein the location comprises one of the attributes of the captured media object.

35. The computer-readable medium of claim 32, wherein the adaptive threshold is computed based on the formula:

$\log <br><br>\left(X_n\right)=\log <br><br>\left(17\right)+\frac{1}{20}<br><br>\underset{i=n-10}{\overset{n+9}{\sum <br><br>}}<br><br><br><br>\log <br><br>\left(t_i+1-t_i\right),$

wherein t_i is a creation time of a media object i and X_n is a threshold that is compared to a difference between a creation time of media object n and media object n+1.

36. A computer-readable medium having computer-executable instructions for performing the steps of:

capturing a media object;

comparing the captured media object with stored media objects that are stored in the database;

identifying the stored media objects in the database that include features in common with the captured media object;

inferring organization information for the captured media object based upon information obtained from each of the media objects including features in common with the captured media object; and

organizing the captured media object in the database based upon the inferred organization information,

wherein the step of inferring includes a step of determining whether data associated with the captured media object exceeds an adaptive threshold, wherein the adaptive threshold is based on a running average of log time gaps between media objects,

wherein the step of inferring includes a step of determining into which group of stored media objects to store the captured media object.

37. A computer-readable medium having computer-executable instructions for performing the steps of:

capturing a media object;

identifying a feature of the captured media object;

comparing the feature of the captured media object with stored media objects that are stored in the database;

identifying the stored media objects having the identified feature of the captured media object;

inferring organization information for the captured media object based upon information obtained from each of the stored media objects having the identified feature of the captured media object; and

organizing the captured media object in the database based upon the inferred organization information,

wherein the step of inferring includes a step of determining whether the identified feature of the captured media object exceeds an adaptive threshold, wherein the adantive threshold is based on a running average of log time gaps between media objects,

wherein the step of inferring includes a step of determining into which group of stored media objects to store the captured media object.

38. A computer-readable medium having computer-executable instructions for performing the steps of:

capturing a media object;

identifying a feature of the captured media object;

performing an inexact search to detect stored media objects that are stored in the database having the identified feature of the captured media object;

identifying the stored media objects having the identified feature of the captured media object;

inferring organization information for the captured media object based upon information obtained from each of the stored media objects having the feature identified in the captured media object; and

organizing the captured media object in the database based upon the inferred organization information,

wherein the step of inferring includes a step of determining whether the identified feature of the captured media object exceeds an adaptive threshold, wherein the adaptive threshold is based on a running average of log time gaps between media objects,

wherein the step of inferring includes a step of determining into which group of stored media objects to store the captured media object.

39. The computer-readable medium of claim 38, wherein the adaptive threshold is computed based on the formula:

$\log <br><br>\left(X_n\right)=\log <br><br>\left(17\right)+\frac{1}{20}<br><br>\underset{i=n-10}{\overset{n+9}{\sum <br><br>}}<br><br><br><br>\log <br><br>\left(t_i+1-t_i\right),$

wherein t_i is a creation time of a media object i and X_n is a threshold that is compared to a difference between the creation time of media object n and media object n+1.

40. A method of organizing media objects in a database, comprising:

capturing a media object;

determining attributes of the captured media object indicated in metadata for the captured media object;

determining a date on which the captured media object was captured, wherein the date comprises one of the attributes of the captured media object;

comparing the date with adaptive threshold date information, wherein the adaptive threshold date information is based on a running average of log time gaps between photographs;

identifying media objects stored in the database that are related to the captured media object based upon the comparison;

inferring organization information for the captured media object based upon information obtained from each of the stored media objects related to the captured media object; and

organizing the captured media object in the database based upon the inferred organization information,

wherein the step of inferring includes a step of determining whether the date is within the adaptive threshold date information, wherein the adantive threshold date information is based on a running average of log time gaps between media objects,

wherein the step of inferring includes a step of determining into which group of stored media objects to store the captured media object.

41. The method according to claim 40, wherein the comparing step comprises:

comparing the date on which the captured media object was captured with entries in a global date book.

42. The method of claim 40, wherein the adaptive threshold date information is computed based on the formula:

$\log <br><br>\left(X_n\right)=\log <br><br>\left(17\right)+\frac{1}{20}<br><br>\underset{i=n-10}{\overset{n+9}{\sum <br><br>}}<br><br><br><br>\log <br><br>\left(t_i+1-t_i\right),$

wherein t_i is a creation time of a media object i and X_n is a threshold that is compared to a difference between a creation time of media object n and media object n+1.

43. A method of organizing media objects in a database, comprising:

capturing a media object;

determining attributes of the captured media object indicated in metadata for the captured media object;

determining a date on which the captured media object was captured, wherein the date comprises one of the attributes of the captured media object;

comparing the date on which the captured media object was captured with entries in a date book;

inferring organization information for the captured media object based upon the comparison; and

organizing the captured media object in the database based upon the inferred organization information,

wherein the step of inferring includes a step of determining whether the date on which the captured media object was captured is within an adaptive threshold range, wherein the adaptive threshold range is based on a running average of log time gaps between media objects,

wherein the step of inferring includes a step of determining into which group of stored media objects to store the captured media object.

44. The method of claim 43, wherein the adaptive threshold range is computed based on the formula:

$\log <br><br>\left(X_n\right)=\log <br><br>\left(17\right)+\frac{1}{20}<br><br>\underset{i=n-10}{\overset{n+9}{\sum <br><br>}}<br><br><br><br>\log <br><br>\left(t_i+1-t_i\right),$

wherein t_i is a creation time of a media object i and X_n is a threshold that is compared to a difference between a creation time of media object n and media object n+1.

45. A computer-readable medium having computer-executable instructions for performing the steps of:

capturing a media object;

determining attributes of the captured media object indicated in metadata for the captured media object;

determining a date on which the captured media object was captured, wherein the date comprises one of the attributes of the captured media object;

comparing the date with adaptive threshold date information, wherein the adaptive threshold date information is based on a running average of log time gaps between media objects;

identifying stored media objects stored in the database that are related to the captured media object based upon the comparison;

inferring organization information for the captured media object based upon information obtained from each of the stored media objects related to the captured media object; and

organizing the captured media object in the database based upon the inferred organization information,

wherein the step of inferring includes a step of determining whether the date is within the adaptive threshold date information,

wherein the step of inferring includes a step of determining into which group of stored media objects to store the captured media object.

46. The computer-readable medium having computer-executable instructions of claim 45, wherein the adaptive threshold date information is computed based on the formula:

$\log <br><br>\left(X_n\right)=\log <br><br>\left(17\right)+\frac{1}{20}<br><br>\underset{i=n-10}{\overset{n+9}{\sum <br><br>}}<br><br><br><br>\log <br><br>\left(t_i+1-t_i\right),$

wherein t_i is a creation time of a media object i and X_n is a threshold that is compared to a difference between a creation time of media object n and media object n+1.

47. A computer-readable medium having computer-executable instructions for performing the steps of:

capturing a media object;

determining attributes of the captured media object indicated in metadata for the captured media object;

determining a date on which the captured media object was captured, wherein the date comprises one of the attributes of the captured media object;

comparing the date on which the captured media object was captured with entries in a date book;

inferring organization information for the captured media object based upon the comparison; and

organizing the captured media object in the database based upon the inferred organization information,

wherein the step of inferring includes a step of determining whether the date on which the captured media object was captured is within an adaptive threshold range, wherein the adaptive threshold range is based on a running average of log time gaps between media objects,

wherein the step of inferring includes a step of determining into which group of stored media objects to store the captured media object.

48. The computer-readable medium according to claim 47, wherein the comparing step comprises:

comparing the date on which the captured media object was captured with entries in a global date book.

49. The computer-readable medium having computer-executable instructions of claim 47, wherein the adaptive threshold range is computed based on the formula:

$\log <br><br>\left(X_n\right)=\log <br><br>\left(17\right)+\frac{1}{20}<br><br>\underset{i=n-10}{\overset{n+9}{\sum <br><br>}}<br><br><br><br>\log <br><br>\left(t_i+1-t_i\right),$

wherein t_i is a creation time of a media object i and X_n is a threshold that is compared to a difference between a creation time of media object n and media object n+1.