Apparatus for detecting a cut in a video

• US 5,835,163 A
• Filed: 12/21/1995
• Issued: 11/10/1998
• Est. Priority Date: 12/21/1995
• Status: Expired due to Term

• Current Assignee: Siemens Corp. (Siemens AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Loghing, David L., Arman, Farshid, Liou, Shih-Ping
• Primary Examiner(s): Chin, Tommy P.
• Assistant Examiner(s): Le, Vu
• Application Number:
  US08/576,272
• Time in Patent Office: 1,055 Days
• Field of Search: 348/700, 348/699, 348/416, 348/400, 348/409, 348/701, 348/135, 348/222, 345/328
• US Class Current: 348/700
• CPC Class Codes: G06F 16/745 the internal structure of a single video sequence
• CPC Class Codes: G06F 16/7847 using low-level visual features of the video content
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

Apparatus for detecting a cut in a video comprises arrangements for acquiring video images from a source, for deriving from the video images a pixel-based difference metric, for deriving from the video images a distribution-based difference metric, and for measuring video content of the video images to provide up-to-date test criteria. Arrangements are included for combining the pixel-based difference metric and the distribution-based difference metric, taking into account the up-to-date test criteria provided so as to derive a scene change candidate signal and for filtering the scene change candidate signal so as to generate a scene change frame list.

Claims

1. A method for detecting a cut in a video, comprising the steps of:

(a) acquiring video images from a source;

(b) deriving from said video images a pixel-based difference metric;

(c) deriving from said video images a distribution-based difference metric;

(d) measuring video content of said video images to provide up-to-date test criteria by collecting statistics from each DC image and each pair of DC images to represent current video content, being an image contrast and a motion estimate;

(e) combining said pixel-based difference metric and said distribution-based difference metric, taking into account said up-to-date test criteria provided in step (d) to derive a scene change candidate signal; and

(f) filtering said scene change candidate signal to generate a scene change frame list.

2. A method for detecting a cut in a video in accordance with claim 1, wherein said pixel-based difference metric for each frame is the summation of an absolute frame difference representative of image intensity value at selected pixel locations in a frame.

3. A method for detecting a cut in a video in accordance with claim 1, wherein said pixel-based difference represented by a metric for each frame t is the sum of an absolute frame difference, ##EQU23## where f^t_ij represents the intensity value at pixel location (i,j) in frame t.

4. A method for detecting a cut in a video in accordance with claim 1, wherein each image is divided into a number of sub-regions and wherein said distribution-based difference metric is a Kolmogorov-Smirnov test metric, said test is carried out for the entire image as well as its sub-regions.

5. A method for detecting a cut in a video in accordance with claim 1, wherein each image is equally divided into four sub-regions and wherein said distribution-based difference metric is a Kolmogorov-Smirnov test metric, except that one each is computed herein for the entire image as well as said four equally divided sub-regions.

6. A method for detecting a cut in a video in accordance with claim 1, wherein said step of measuring video content of said video images to provide said up-to-date test criteria provides said step (e) with the ability to automatically adjust to different video contents.

7. A method for detecting a cut in a video in accordance with claim 1, wherein said video images are DC images represented by the base frequency in the Discrete Cosine Transform coefficients representing the underlying image frame.

8. A method for detecting a cut in a video in accordance with claim 1, wherein said image contrast estimate is computed based on a recursive scheme to suppress influences of sudden lighting changes.

9. A method for detecting a cut in a video in accordance with claim 8, wherein said image contrast estimate is computed based on a recursive scheme to suppress the influences of sudden lighting changes.

10. A method for detecting a cut in a video in accordance with claim 8, wherein said image contrast estimate equals 0.6.

11. A method for detecting a cut in a video in accordance with claim 8, wherein said motion estimate is computed as follows:

##EQU24## where f^t-1_ij is the intensity value at pixel location (i, j) of the DC image at time t-1 , N is the size of the image.

12. A method for detecting a cut in a video in accordance with claim 8, wherein said image contrast and said motion estimates are applied to a fuzzy engine to compute a new significance level for the hierarchical Kolmogorov-Smirnov test, said fuzzy engine using a quadratic membership function, where each contrast measurement is divided into classes, from low to high, and each motion estimate is divided into classes, from slow to fast, and each significance level is divided into classes from high to low.

13. A method for detecting a cut in a video in accordance with claim 12, wherein each contrast measurement is divided into four classes, low, middle, high, and extremely high, each motion estimate into three classes, slow, middle, and fast, and each significance level into five classes, high, middle high, middle, middle low, and low, and wherein the fuzzy rules are stated in a simple IF/THEN format, where values are combined using AND (minimum) or OR (maximum) operations.

14. A method for detecting a cut in a video in accordance with claim 13, including *a step of defuzzifying said fuzzy rules to yield a crisp final output value, by finding the center of gravity of the combined output shape, whereby all rules are ensured of contributing to the final crisp result.

15. A method for detecting a cut in a video in accordance with claim 1, wherein said collecting statistics from each DC image and each pair of DC images to represent current video content represent an image contrast estimate and a motion estimate.

16. A method for detecting a cut in a video in accordance with claim 1, wherein said image contrast estimate is derived in accordance with the following:

space="preserve" listing-type="equation">contrast.sub.t =(1-τ

)contrast.sub.t-1 +τ

σ

.sub.t-1

where σ

_t-1 is the intensity variance estimate of the DC image at time t-1.

17. A method for detecting a cut in a video in accordance with claim 16, wherein τ

equals 0.6.

18. A method for detecting a cut in a video in accordance with claim 1, wherein in said step (e) of combining said pixel-based difference metric and said distribution-based difference metric, taking into account said up-to-date test criteria provided in step (d) so as to derive a scene change candidate signal, said pixel-based difference metrics are treated as time series signals, where both visually abrupt cuts and duplication of frames create observation outliers.

19. A method for detecting a cut in a video in accordance with claim 18, wherein said pixel-based difference metric is treated as a time series signal, where both visually abrupt cuts and the duplication of frames create observation outliers obeying the equation

20. Apparatus for detecting a cut in a video, comprising:

(a) means for acquiring video images from a source;

(b) means for deriving from said video images a pixel-based difference metric;

(c) means for deriving from said video images a distribution-based difference metric;

(d) means for measuring video content of said video images to provide up-to-date test criteria;

(e) means for combining said pixel-based difference metric and said distribution-based difference metric, taking into account said up-to-date test criteria provided in step (d) so as to derive a scene change candidate signal;

(f) means for filtering said scene change candidate signal so as to generate a scene change frame list; and

(g) means for presenting two cross-section images of said video images, wherein one is a horizontal cross section image in a horizontal direction and the other is a vertical cross section image in a vertical direction of the video volume.

21. Apparatus for detecting a cut in a video in accordance with claim 20, wherein each cross-section image is constructed by sampling one row (or column) from every image, and reducing the amount of information from a two-dimensional image to two one-dimensional image strips.

22. Apparatus for detecting a cut in a video in accordance with claim 21, wherein said horizontal and vertical cross-section images are combined into one image segmented into two bands according to a list of detected shots, wherein a level of abstraction is presented that is enough to reveal whether there is a missed or misdetected shot.

23. Apparatus for detecting a cut in a video in accordance with claim 20, including means for presenting at least two cross-section images of said video images, whereof one is a horizontal cross section image in a horizontal direction and the other is a vertical cross section image in a vertical direction of the video volume.

24. Apparatus for detecting a cut in a video in accordance with claim 23, wherein each cross-section image is constructed by sampling one row (or column) from every image, and reducing the amount of information from a two-dimensional image to a plurality of one-dimensional image strips.

25. Apparatus for detecting a cut in a video in accordance with claim 24, wherein said at least two cross-section images, including said horizontal and vertical cross-section images, are combined into one image segmented into a plurality of bands according to a list of detected shots, whereby a level of abstraction is presented that is enough to reveal whether there is a missed or misdetected shot.