Robust shot detection in a video

• US 7,639,873 B2
• Filed: 07/28/2005
• Issued: 12/29/2009
• Est. Priority Date: 07/28/2005
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Qian, Richard J.
• Primary Examiner(s): Bali, Vikkram
• Assistant Examiner(s): Park, Soo Jin
• Application Number:
  US11/192,216
• Publication Number:
  US 20070025614A1
• Time in Patent Office: 1,615 Days
• Field of Search: 348/700
• US Class Current: 382/168
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 5/147 Scene change detection

Abstract

Automatic detection of one or more segment boundaries in a video. An analysis window identifies a selected series of video frames of the video to analyze for segment boundaries. A property value is determined for each of the video frames in the series of video frames. A maximum difference of the differences of property values between adjacent frames is compared to a threshold median value and a ratio of the maximum difference to a determined mean of the differences of property values between adjacent frames is compared to an adaptive threshold value. A segment boundary is defined as a function of the comparisons.

Claims

1. A computerized method of automatically detecting one or more segment boundaries in a video, said video including a plurality of video frames, said method comprising:

selecting a series of video frames in the video for detection;

determining a property value of each of the video frames in the selected series of video frames, wherein the property value of each of the video frames is determined independent of the property value of the other video frames;

determining differences of the property values between adjacent video frames of the selected series of video frames;

determining a maximum difference of the differences of the property values between adjacent video frames of the selected series of video frames;

determining a mean of the differences of the property values between adjacent video frames of the selected series of video frames;

determining a ratio of the maximum difference to the determined mean;

comparing the maximum difference to a threshold median value and comparing the determined ratio to an adaptive threshold value; and

defining a segment boundary as a function of the comparisons.

2. The method of claim 1, wherein selecting a series of video frames includes defining a number of consecutive video frames based on input received from a user.

3. The method of claim 1, wherein selecting a series of video frames includes defining a number of consecutive video frames to analyze for a segment boundary based on a default window size.

4. The method of claim 1, wherein determining a property value of each of the video frames in the selected series of video frames includes computing a color histogram for each video frame.

5. The method of claim 1, wherein the threshold median value and the adaptive threshold value are predefined and stored in a memory.

6. The method of claim 1, wherein defining a segment boundary as a function of the comparisons includes defining a segment boundary between the adjacent frames having the maximum difference when the maximum difference is greater than the threshold median value and the determined ratio is greater than the adaptive threshold value.

7. The method of claim 6 further comprising segmenting the video at the defined segment boundary into video shots.

8. The method of claim 1 further comprising selecting a next series of video frames of the video to analyze for one or more segment boundaries when the maximum difference is less than or equal to the threshold median value, or the determined ratio is less than or equal to the adaptive threshold value.

9. The method of claim 1 further including:

determining a next maximum difference of the differences of the property values;

determining another ratio of the maximum difference to the next maximum difference;

comparing the determined other ratio to another adaptive threshold value and comparing the maximum difference to a threshold minimum value; and

wherein defining a segment boundary as a function of the comparisons includes defining a segment boundary between the adjacent frames having the maximum difference when the maximum difference is greater than the threshold minimum value, and the determined ratio is greater than the other adaptive threshold value.

10. A computer-readable storage medium having computer-executable instructions for defining one or more segment boundaries in a video, said video including a series of video frames, said computer-readable storage medium comprising:

a user interface window component for selecting a series of video frames in the video;

an analysis component for;

determining a property value of each of the video frames in the selected series of video frames, wherein the property value of each of the video frames is determined independent of the property value of the other video frames;

determining differences of the property values between adjacent video frames of the selected series of video frames;

determining a maximum difference of the differences of the property values between adjacent video frames of the selected series of video frames;

determining a mean of the differences of the property values between adjacent video frames of the selected series of video frames; and

determining a ratio of the maximum difference to the determined mean;

a comparison component for comparing the maximum difference to a threshold median value and comparing the determined ratio to an adaptive threshold value; and

a segmentation component for defining a segment boundary as a function of the comparisons.

11. The computer-readable storage medium of claim 10, wherein the window component is responsive to user input to selectively identify a number of consecutive video frames in the video to analyze for a segment boundary.

12. The computer-readable storage medium of claim 10, wherein the analysis component determines a property value of each of the video frames in the selected series of video frames by computing a color histogram for each video frame.

13. The computer-readable storage medium of claim 10, wherein the threshold median value and the adaptive threshold value are predefined and stored in a memory.

14. The computer-readable storage medium of claim 10, wherein the segmentation component defines a segment boundary between the adjacent frames having the maximum difference when the maximum difference is greater than the threshold median value and the determined ratio is greater than the adaptive threshold value.

15. The computer-readable storage medium of claim 10, wherein the segmentation component segments the video at the defined segment boundary.

16. The computer-readable storage medium of claim 10, wherein the window component selectively identifies a next series of video frames of the video to analyze for one or more segment boundaries when the maximum difference is less than or equal to the threshold median value, or the determined ratio is less than or equal to the adaptive threshold value.

17. The computer-readable storage medium of claim 10, wherein the analysis component further determines a next maximum difference of the differences of the property values and determines another ratio of the maximum difference to the next maximum difference, and wherein the comparison component compares the maximum difference to a threshold minimum value and compares the determined other ratio to another adaptive threshold value, and wherein the segmentation component defines a segment boundary between the adjacent frames having the maximum difference when the maximum difference is greater than the threshold minimum value, and the determined ratio is greater than the other adaptive threshold value.

18. A system for processing a video, said system comprising:

a computer-readable storage medium storing a video captured from a video source, said video including a plurality of video frames;

a processor executing computer executable instructions for;

determining a property value of each of the video frames in a selected series of video frames, wherein determining a property value of each of the video frames in the selected series of video frames includes computing a color histogram for each video frame independent of the color histogram of the other video frames;

determining differences of the property values between adjacent video frames of the selected series of video frames;

determining a maximum difference of the differences of the property values between adjacent video frames of the selected series of video frames;

determining a mean of the differences of the property values between adjacent video frames of the selected series of video frames;

determining a ratio of the maximum difference to the determined mean;

comparing the maximum difference to a threshold median value and comparing the determined ratio to an adaptive threshold value;

defining a segment boundary as a function of the comparisons;

segmenting the video at the defined segment boundary into video shots; and

storing the segmented video shots on the computer-readable storage medium;

wherein the processer is responsive to the received selection to analyze the series of video frames to detect one or more segment boundaries; and

a user interface for receiving a selection of a series of video frames for analysis from a user.