HIERARCHICAL SEGMENTATION AND QUALITY MEASUREMENT FOR VIDEO EDITING

• US 20160358628A1
• Filed: 06/03/2016
• Published: 12/08/2016
• Est. Priority Date: 06/05/2015
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Liu, Shujie, Zheng, Yunfei, Zhou, Xiaosong, Wu, Hsi-Jung, Zhang, Ke, Keinath, Albert E., Chung, Chris Y.
• Granted Patent: US 10,062,412 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/47 Detecting features for summarising video content
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

Methods for organizing media data by automatically segmenting media data into hierarchical layers of scenes are described. The media data may include metadata and content having still image, video or audio data. The metadata may be content-based (e.g., differences between neighboring frames, exposure data, key frame identification data, motion data, or face detection data) or non-content-based (e.g., exposure, focus, location, time) and used to prioritize and/or classify portions of video. The metadata may be generated at the time of image capture or during post-processing. Prioritization information, such as a score for various portions of the image data may be based on the metadata and/or image data. Classification information such as the type or quality of a scene may be determined based on the metadata and/or image data. The classification and prioritization information may be metadata and may be used to organize the media data.

Claims

1. A method for organizing media data, comprising:

parsing video data into media items according to scene detection;

clustering the media items based on content analysis of the media items;

opening a media editing application; and

displaying a workspace of the media editing application in which the media items are grouped according to the clustering.

2. The method of claim 1, further comprising:

recommending the media items based on content analysis of the media items.

3. The method of claim 1, further comprising:

clustering scenes hierarchically into layers where a plurality of clusters have one or more similar characteristics.

4. The method of claim 1, further comprising:

assigning quality scores to the media items; and

providing an indicator of the assigned quality scores in the media editing application.

5. The method of claim 1, wherein a quality score is based on operational parameters of a camera at a time a respective media item was captured.

6. The method of claim 1, wherein a quality score is based on operational parameters of the media editing application, wherein the media editing application was used to edit the media data.

7. The method of claim 1, wherein a quality score is based on a location at which a respective media item was captured.

8. The method of claim 1, wherein a quality score is based on a similarity of content of a respective media item to content of a reference media item.

9. The method of claim 1, wherein a quality score is based on a level of activity in content of a respective media item.

10. A non-transitory computer readable medium comprising instructions that, when executed on the computer, cause the computer to at least:

parse video data into media items according to scene detection;

cluster the media items based on content analysis of the media items;

open a media editing application; and

display a workspace of the media editing application in which the media items are grouped according to the clustering.

11. The non-transitory computer readable medium of claim 10, wherein the instructions further cause the computer to:

recommend the media items based on content analysis of the media items.

12. The non-transitory computer readable medium of claim 10, wherein the instructions further cause the computer to:

cluster scenes hierarchically into layers where a plurality of clusters have one or more similar characteristics.

13. The non-transitory computer readable medium of claim 10, wherein the instructions further cause the computer to:

assign quality scores to the media items; and

provide an indicator of the assigned quality scores in the media editing application.

14. The non-transitory computer readable medium of claim 10, wherein a quality score is based on operational parameters of a camera at a time a respective media item was captured.

15. The non-transitory computer readable medium of claim 10, wherein a quality score is based on operational parameters of the media editing application, wherein the media editing application was used to edit the media data.

16. The non-transitory computer readable medium of claim 10, wherein a quality score is based on a location at which a respective media item was captured.

17. The non-transitory computer readable medium of claim 10, wherein a quality score is based on a similarity of content of a respective media item to content of a reference media item.

18. The non-transitory computer readable medium of claim 10, wherein a quality score is based on a level of activity in content of a respective media item.

19. A system for organizing media data, comprising a processor and a memory, the memory including instructions that, when executed on the processor, caused the system to at least:

parse video data into media items according to scene detection;

cluster the media items based on content analysis of the media items;

open a media editing application; and

display a workspace of the media editing application in which the media items are grouped according to the clustering.

20. The system of claim 19, wherein the memory further comprises instructions to:

recommend the media items based on content analysis of the media items.

21. The system of claim 19, wherein the memory further comprises instructions to:

cluster scenes hierarchically into layers where a plurality of clusters have one or more similar characteristics.

22. The system of claim 19, wherein the memory further comprises instructions to:

assign quality scores to the media items; and

provide an indicator of the assigned quality scores in the media editing application.

23. The system of claim 19, wherein a quality score is based on operational parameters of a camera at a time a respective media item was captured.

24. The system of claim 19, wherein a quality score is based on operational parameters of the media editing application, wherein the media editing application was used to edit the media data.

25. The system of claim 19, wherein a quality score is based on a location at which a respective media item was captured.

26. The system of claim 19, wherein a quality score is based on a similarity of content of a respective media item to content of a reference media item.

27. The system of claim 19, wherein a quality score is based on a level of activity in content of a respective media item.