Method and apparatus for estimating shot boundaries in a digital video sequence

• US 20070025615A1
• Filed: 07/28/2005
• Published: 02/01/2007
• Est. Priority Date: 07/28/2005
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Zhou, Hui, Wong, Alexander
• Granted Patent: US 7,551,234 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 5/147 Scene change detection

Abstract

A method of estimating a shot boundary in a digital video sequence comprises calculating a measurement of dissimilarity between each frame in the video sequence and its previous frame based on at least one of de-emphasized luminance differences and absolute color differences. Each frame is tested against shot boundary dissimilarity thresholds using the dissimilarity measurement for the frame thereby to detect the shot boundary.

Claims

1. A method of measuring dissimilarity between adjacent frames in a digital video sequence, comprising:

determining pixel accumulations at respective color component levels for each of said frames; and

calculating a dissimilarity measurement by comparing corresponding ones of said accumulations while de-emphasizing dissimilarities in luminance between said frames.

2. The method of claim 1, wherein said color components are luminance and chrominance.

3. The method of claim 2, wherein said corresponding ones of said accumulations are common ranges of respective ones of luminance and chrominance.

4. The method of claim 3 wherein during said determining, LUV color-space histograms for said frames are generated and wherein during said calculating, said LUV color-space histograms are compared.

5. The method of claim 4, wherein said dissimilarity measurement is calculated according to the formula:

6. A method of measuring dissimilarity between adjacent frames in a digital video sequence, comprising:

determining pixel accumulations at respective color component levels for each of said frames; and

calculating a dissimilarity measurement by comparing corresponding ones of said accumulations in a manner such that dominant color changes between said frames are emphasized.

7. The method of claim 6, wherein said color components are luminance and chrominance.

8. The method of claim 7, wherein said levels are ranges.

9. The method of claim 8, wherein each of said color components has a plurality of ranges.

10. The method of claim 9 wherein during said determining, LUV color-space histograms for said frames are generated and wherein during said calculating, said LUV color-space histograms are compared.

11. The method of claim 10, wherein said dissimilarity measurement is calculated according to the formula:

12. A method of verifying a candidate shot boundary in a digital video sequence, comprising:

determining a first luminance change between said candidate shot boundary and its previous adjacent frame in the video sequence;

examining neighbourhood frames to determine if the luminance change between a neighbourhood frame and its previous adjacent frame in the video sequence is comparable to and in a direction opposite of said first luminance change; and

declaring the candidate shot boundary as a false positive if the result of said examining is positive.

13. The method of claim 12, wherein the luminance change between a neighbourhood frame and its previous adjacent frame is comparable to the first luminance change when the value thereof is within a threshold percentage of the first luminance change.

14. The method of claim 13, wherein said threshold percentage is 30%.

15. The method of claim 12, wherein during said examining, neighborhood frames before and after said candidate shot boundary are examined.

16. The method of claim 12, wherein the luminance changes are average luminance changes.

17. A method of determining a candidate gradual shot transition in a digital video sequence, comprising:

detecting a gradual shot transition beginning when a dissimilarity measurement between adjacent frames in said video sequence is between predetermined minimum and maximum thresholds;

forming an accumulated dissimilarity by accumulating dissimilarity measurements between adjacent frames subsequent to said gradual shot transition beginning;

during said accumulating, in the event that there is a stall in accumulation, declaring said candidate gradual shot transition to be false; and

otherwise

detecting a gradual shot transition ending when said accumulated dissimilarity exceeds said maximum threshold.

18. The method of claim 17, wherein a stall occurs when dissimilarity measurements are less than a predetermined amount for a predetermined number of pairs of adjacent frames and said accumulated dissimilarity is less than a predetermined percentage of said maximum threshold.

19. The method of claim 18, wherein said predetermined percentage is 70%.

20. A method of establishing a maximum shot boundary threshold for a candidate frame in a digital video sequence, comprising:

calculating a sequence average of inter-frame dissimilarity measurements;

calculating a window average of inter-frame dissimilarity measurements, said window comprising a subset of sequential frames in said sequence prior to said candidate frame;

in the event that said window average is inconsistent with said sequence average, calculating said maximum shot boundary as a factor of said sequence average; and

otherwise

calculating said maximum shot boundary as a factor of said window average.