Categorization of digital media based on media characteristics

• US 8,300,953 B2
• Filed: 06/05/2008
• Issued: 10/30/2012
• Est. Priority Date: 06/05/2008
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Lindley, Gregory Charles, Martin, Timothy B.
• Primary Examiner(s): CHANNAVAJJALA, SRIRAMA T
• Application Number:
  US12/134,074
• Publication Number:
  US 20090307261A1
• Time in Patent Office: 1,608 Days
• Field of Search: 707/609, 707705-707, 707/730, 707797-798, 707821-822, 707/828, 707/749, 707/758, 707/803, 707/805, 707913-916, 382/100, 382/118, 382/157, 382/190, 382/305, 382/307, 382/219, 3482312-2316, 345418-419, 345/594, 345/619, 345/634, 345/650, 345/676, 715/721, 715/838
• US Class Current: 382/219
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion

Abstract

Digital media categorization can include receiving information including a plurality of media objects and a metadata tag descriptive of at least a first media object; comparing the first media object with a second media object; and selectively associating the first media object'"'"'s metadata tag with the second media object based on a result of the comparison. Each media object can include a digital image.

Claims

1. A computer-implemented method comprising:

receiving, by a computer system, information comprising a plurality of media objects, including at least a first media object comprising a first digital image and a first set of metadata, and a second media object comprising a second digital image and a second set of metadata;

detecting a first feature of the first digital image, and detecting a second feature of the second digital image;

identifying similarities between the detected first feature of the first digital image and the detected second feature of the second digital image;

in response to the identifying the similarities between the detected first feature of the first digital image and the detected second feature of the second digital image, modifying, based on the identified similarities, a metadata tag within the second set of metadata of the second media object to include at least a portion of a metadata tag within the first set of metadata of the first media object that refers to the identified similarities; and

processing the second media object of the plurality of media objects based at least in part on the modified metadata tag.

2. The method of claim 1, wherein

the detected first feature of the first digital image comprises at least a first portion of the first digital image and

the detected second feature of the second digital image comprises at least a second portion of the second digital image.

3. The method of claim 2, wherein the detected first feature is a depiction of at least a portion of a person.

4. The method of claim 3, wherein the metadata tag within the first set of metadata of the first media object comprises a name of the person.

5. The method of claim 1, wherein the detecting the first feature, the detecting the second feature, and the identifying the similarities comprises:

accessing a first timestamp of the first media object;

accessing a second timestamp of the second media object; and

comparing the first timestamp with the second timestamp.

6. The method of claim 1, wherein the detecting the first feature, the detecting the second feature, and the identifying the similarities comprises:

accessing first geographical coordinates of the first media object;

accessing second geographical coordinates of the second media object; and

comparing the first geographical coordinates with the second geographical coordinates.

7. The method of claim 1, further comprising:

providing a user interface to present the first media object to a user; and

displaying in the user interface a prompt that requests a metadata tag for the first media object.

8. The method of claim 1, wherein the first digital image of the first media object comprises a frame of a first digital video.

9. The method of claim 8, wherein the second digital image of the second media object comprises a frame of either the first digital video or a second digital video.

10. The method of claim 1, wherein the first and second media object represent different video frames in a digital video sequence.

11. The method of claim 1, wherein the processing comprises generating a digital media album responsive to results of the identifying, and organizing the plurality of media objects in the digital media album.

12. The method of claim 11, wherein the processing further comprises updating a corresponding metadata tag of each media object in the digital media album with the digital media album information, wherein the metadata tag was previously associated with fewer than all of the media objects in the digital media album.

47. The method of claim 10, further comprising identifying one or more video frames in the digital video sequence in which an event occurs, the event comprising a feature appearing in each of the identified frames.

13. A computer-readable storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:

receiving information comprising a plurality of media objects, including at least a first media object comprising a first digital image and a first set of metadata, and a second media object comprising a second digital image and a second set of metadata;

detecting a first feature of the first digital image, and detecting a second feature of the second digital image;

identifying similarities between the detected first feature of the first digital image and the detected second feature of the second digital image;

in response to the identifying the similarities between the detected first feature of the first digital image and the detected second feature of the second digital image, modifying, based on the identified similarities, a metadata tag within the second set of metadata of the second media object to include at least a portion of a metadata tag within the first set of metadata of the first media object that refers to the identified similarities; and

processing the second media object of the plurality of media objects based at least in cart on the modified metadata tag.

14. The computer-readable storage medium of claim 13, wherein

the detected first feature of the first digital image comprises at least a first portion of the first digital and

the detected second feature of the second digital image comprises at least a second portion of the second digital image.

15. The computer-readable storage medium of claim 14, wherein the detected first feature is a depiction of at least a portion of a person.

16. The computer-readable storage medium of claim 15, wherein the metadata tag within the first set of metadata of the first media object comprises a name of the person.

17. The computer-readable storage medium of claim 13, wherein the detecting the first, the detecting the second feature, and the identifying the similarities comprises:

accessing a first timestamp of the first media object;

accessing a second timestamp of the second media object; and

comparing the first timestamp with the second timestamp.

18. The computer-readable storage medium of claim 13, wherein the detecting the first feature, the detecting the second feature, and the identifying the similarities comprises:

accessing first geographical coordinates of the first media object;

accessing second geographical coordinates of the second media object; and

comparing the first geographical coordinates with the second geographical coordinates.

19. The computer-readable storage medium of claim 13, the operations further comprising:

providing a user interface to present the first media object to a user; and

displaying in the user interface a prompt that requests a metadata tag for the first media object.

20. The computer-readable storage medium of claim 13, wherein the first digital image of the first media object comprises a frame of a first digital video.

21. The computer-readable storage medium of claim 20, wherein the second digital image of the second media object comprises a frame of either the first digital video or a second digital video.

22. The computer-readable storage medium of claim 13, wherein the first and second media object represent different video frames in a digital video sequence.

23. The computer-readable storage medium of claim 22, further comprising identifying one or more video frames in the digital video sequence in which an event occurs, the event comprising a feature appearing in each of the identified frames.

24. The computer-readable storage medium of claim 13, wherein the processing comprises generating a digital media album responsive to results of the identifying, and organizing the plurality of media objects in the digital media album.

25. The computer-readable storage medium of claim 24, wherein the processing further comprises updating a corresponding metadata tag of each media object in the digital media album with the digital media album information, wherein the metadata tag was previously associated with fewer than all of the media objects in the digital media album.

26. A system comprising:

a communication interface; and

a processor coupled with the communication interface, wherein the processor is configured to perform operations comprising;

receiving information from the communication interface, the information comprising a plurality of media objects, including at least a first media object comprising a first digital image and a first set of metadata, and a second media object comprising a second digital image and a second set of metadata;

detecting a first feature of the first digital image, and detecting a second feature of the second digital image;

identifying similarities between the detected first feature of the first digital image and the detected second feature of the second digital image;

in response to the identifying the similarities between the detected first feature of the first digital image and the detected second feature of the second digital image, modifying, based on the identified similarities, a metadata tag within the second set of metadata of the second media object to include at least a portion of a metadata tag within the first set of metadata of the first media object that refers to the identified similarities; and

processing the second media object of the plurality of media objects based at least in part on the modified metadata tag.

27. The system of claim 26, wherein

the detected first feature of the first digital image comprises at least a first portion of the first digital image and

the detected second feature of the second digital image comprises at least a second portion of the second digital image.

28. The system of claim 27, wherein the detected first feature is a depiction of at least a portion of a person.

29. The system of claim 28, wherein the metadata tag within the first set of metadata of the first media object comprises a name of the person.

30. The system of claim 26, wherein the detecting the first feature, the detecting the second feature, and the identifying the similarities comprises:

accessing a first timestamp of the first media object;

accessing a second timestamp of the second media object; and

comparing the first timestamp with the second timestamp.

31. The system of claim 26, wherein the detecting the first feature, the detecting the second feature, and the identifying the similarities comprises:

accessing first geographical coordinates of the first media object;

accessing second geographical coordinates of the second media object; and

comparing the first geographical coordinates with the second geographical coordinates.

32. The system of claim 26, the operations further comprising:

providing a user interface to present the first media object to a user; and

displaying in the user interface a prompt that requests a metadata tag for the first media object.

33. The system of claim 26, wherein the first digital image of the first media object comprises a frame of a first digital video.

34. The system of claim 33, wherein the second digital image of the second media object comprises a frame of either the first digital video or a second digital video.

35. The system of claim 26, wherein the communication interface is coupled with a communication network.

36. The system of claim 26, further comprising a storage medium, wherein the storage medium is coupled with the communication interface.

37. A computer-implemented method comprising:

receiving, by a computer system, a digital video clip comprising a plurality of video frames and an associated set of metadata;

detecting a first feature of a first video frame of the plurality of video frames and detecting a second feature of a second video frame of the plurality of video frames;

identifying similarities between the detected first feature of the first video frame and the detected second feature of the second video frame; and

in response to the identifying the similarities between the detected first feature of the first video frame and the detected second feature of the second video frame, modifying, based on the identified similarities, at least a portion of the associated set of metadata to associate the identified similarities with the first video frame and the second video frame.

38. The method of claim 37, wherein the detected first feature comprises a depiction of at least a portion of a person.

39. The method of claim 38, wherein the at least a portion of a person comprises a facial feature.

40. The method of claim 38, wherein the modifying of at least a portion of the associated set of metadata comprises a name of the depicted person.

41. The method of claim 37, further comprising identifying one or more video frames of the plurality of video frames in which an event occurs based, at least in part, on the identified similarities.

42. A computer-readable storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:

receiving, by a computer system, a digital video clip comprising a plurality of video frames and an associated set of metadata;

detecting a first feature of a first video frame of the plurality of video frames and detecting a second feature of a second video frame of the plurality of video frames;

identifying similarities between the detected first feature of the first video frame and the detected second feature of the second video frame; and

in response to the identifying the similarities between the detected first feature of the first video frame and the detected second feature of the second video frame, modifying, based on the identified similarities, at least a portion of the associated set of metadata to associate the identified similarities with the first video frame and the second video frame.

43. The computer-readable storage medium of claim 42, wherein the detected first feature comprises a depiction of at least a portion of a person.

44. The computer-readable storage medium of claim 43, wherein the at least a portion of a person comprises a facial feature.

45. The computer-readable storage medium of claim 43, wherein the modifying of at least a portion of the associated set of metadata comprises a name of the depicted person.

46. The computer-readable storage medium of claim 42, further comprising identifying one or more video frames of the plurality of video frames in which an event occurs based, at least in part, on the identified similarities.