Video processing system including advanced scene break detection methods for fades, dissolves and flashes

• US 6,721,361 B1
• Filed: 02/23/2001
• Issued: 04/13/2004
• Est. Priority Date: 02/23/2001
• Status: Active Grant

• Current Assignee: Yesvideo.com
• Original Assignee: Yesvideo.com
• Inventors: Edwards, Jeffrey Layne, Covell, Michele May, Ahmad, Subutai
• Primary Examiner(s): An, Shawn S.
• Application Number:
  US09/792,280
• Time in Patent Office: 1,145 Days
• Field of Search: 375/240.14, 375/240, 375/240.26, 375/240.16, 348/700, 348/699, 348/594, 348/595, 386/52, 386/53
• US Class Current: 375/240.14
• CPC Class Codes: G11B 2220/2545 CDs
• CPC Class Codes: G11B 2220/2562 DVDs [digital versatile discs]; Digital video discs; MMCDs; HDCDs
• CPC Class Codes: G11B 2220/90 Tape-like record carriers
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

The invention provides a method and apparatus for detecting a scene transition in digitized video data. The method includes determining if a gradual transition is detected in the video data that ends with a current frame based at least on the current frame and one or more previous frames in the digitized video data. The gradual transition has a length. The method includes comparing the gradual transition to a synthetically produced gradual transition spanning a same length and generated from starting and ending frames associated with the detected gradual transition and marking the detected gradual transition only if the detected gradual transition is sufficiently similar to the synthetically produced gradual transition.

Claims

1. A method for detecting a scene transition in digitized video data comprising:

determining if a hypothesized dissolve is detected in the video data that ends with a current frame based at least on the current frame and one or more previous frames in the digitized video data, the hypothesized dissolve having a length, a starting and an ending frame;

generating a synthetically produced dissolve spanning the length of the hypothesized dissolve and based on input including the starting and ending frames of the hypothesized dissolve;

comparing the hypothesized dissolve to the synthetically produced dissolve; and

marking the hypothesized dissolve as a dissolve only if the hypothesized dissolve is sufficiently similar to the synthetically produced dissolve.

2. The method of claim 1 wherein the step of comparing includes comparing a subset of all of the frames in the hypothesized dissolve to the frames in the synthetically produced dissolve.

3. The method of claim 1 wherein the step of determining further includes evaluating a previously processed frame at a distance a half dissolve from the current frame and a full dissolve from the current frame including determining if the current frame is sufficiently different from both the flame located a half dissolve length ago and a fill dissolve length ago.

4. The method of claim 1 wherein the step of determining includes

evaluating previously processed frames, a first frame at a distance a half dissolve from the current frame and a second frame at a distance a full dissolve from the current frame, including determining if the current frame is sufficiently different from both the first and second frames; and

marking the hypothesized dissolve only if the current frame is sufficiently different from the first and second frames.

5. The method of claim 4 wherein prior to marking, the method further including

a. determining if the hypothesized dissolve overlaps a previously detected dissolve;

b. if so, combining the overlapping dissolves.

6. The method of claim 1 wherein the determining step includes

identifying starting, middle and ending frames of the hypothesized dissolve;

and

comparing the starting, middle and ending frames to determine if each is sufficiently different from the others.

7. The method of claim 1 wherein comparing the hypothesized dissolve to the synthetically produced dissolve includes comparing a color difference between frames of the hypothesized dissolve to frames of the synthetically produced dissolve.

8. The method of claim 1 wherein comparing the hypothesized dissolve to the synthetically produced dissolve includes comparing a motion difference.

9. The method of claim 1 wherein comparing the hypothesized dissolve to the synthetically produced dissolve includes deriving a score representative of similarity of frames of the hypothesized dissolve as compared to frames of the synthetically produced dissolve, the score derived from a mathematical comparison of the frames considering at least one parameter.

10. The method of claim 9 wherein the score is a product of color and motion differences between the respective frames.

11. The method of claim 9 wherein the score is a weighted mean of color and motion differences between the respective frames.

12. A method for detecting a scene transition in digitized video data comprising:

determining if a hypothesized gradual transition is detected in the video data that ends with a current frame based at least on the current frame and one or more previous frames in the digitized video data, the hypothesized gradual transition having a length, a starting frame and an ending frame;

generating a synthetically produced gradual transition spanning the length of the hypothesized gradual transition and based on input including the starting frame and ending frame of the hypothesized gradual transition;

comparing the hypothesized gradual transition to the synthetically produced gradual transition; and

marking the hypothesized gradual transition as a gradual transition only if the hypothesized gradual transition is sufficiently similar to the synthetically produced gradual transition.

13. The method of claim 12 wherein the gradual transition is selected from the group of a dissolve, a wipe, a page flip, a tear, an iris transition and a pixelation transition.

14. A computer program product, tangibly stored on a computer-readable medium for detecting a scene transition in digitized video data, comprising instructions operable to cause a programmable processor to:

determine if a hypothesized dissolve is detected in the video data that ends with a current frame based at least on the current frame and one or more previous frames in the digitized video data, the hypothesized dissolve having a length, a starting frame and an ending frame;

generate a synthetic dissolve spanning the length of the hypothesized dissolve and based on input including the starting and ending frames of the hypothesized dissolve;

compare the hypothesized dissolve to the synthetic dissolve; and

mark the hypothesized dissolve as a dissolve only if the hypothesized dissolve is sufficiently similar to the synthetic dissolve.

15. The computer program product of claim 14, wherein instructions operable to compare the hypothesized dissolve to the synthetic dissolve include instructions operable to compare a subset of the frames in the hypothesized dissolve to frames in the synthetic dissolve.

16. The computer program product of claim 14, wherein instructions operable to determine further include instructions operable to:

evaluate a previously processed frame at a distance a half dissolve from the current frame and a full dissolve from the current frame to determine if the current frame is sufficiently different from both the frame located a half dissolve length ago and a full dissolve length ago.

17. The computer program product of claim 14, wherein instructions operable to determine include instructions operable to:

evaluate previously processed frames, a first frame at a distance a half dissolve from the current frame and a second frame at a distance a full dissolve from the current frame, to determine if the current frame is sufficiently different from both the first and second frames; and

mark as a hypothesized dissolve only if the current frame is sufficiently different from the first and second frames.

18. The computer program product of claim 17, wherein prior to marking, the instructions are further operable to:

a. determine if the hypothesized dissolve overlaps a previously detected dissolve;

b. if so, combine the overlapping dissolves.

19. The computer program product of claim 14, wherein instructions operable to determine if a hypothesized dissolve is detected include instructions operable to:

identify starting, middle and ending frames of the hypothesized dissolve; and

compare the starting, middle and ending frames to determine if each is sufficiently different from the others.

20. The computer program product of claim 14, wherein instructions operable to compare the hypothesized dissolve to the synthetic dissolve include instructions operable to compare a color difference between frames of the hypothesized dissolve to frames of the synthetic dissolve.

21. The computer program product of claim 14, wherein instructions operable to compare the hypothesized dissolve to the synthetic dissolve include instructions operable to compare a motion difference.

22. The computer program product of claim 14, wherein instructions operable to compare the hypothesized dissolve to the synthetic dissolve include instructions operable to derive a score representative of a similarity of frames of the hypothesized dissolve as compared to frames of the synthetic dissolve, the score derived from a mathematical comparison of the frames considering at least one parameter.

23. The computer program product of claim 22, wherein the score is a product of color and motion differences between the respective frames.

24. The computer program product of claim 22, wherein the score is a weighted mean of color and motion differences between the respective frames.

25. A computer program product, tangibly stored on a computer-readable medium, for detecting a scene transition in digitized video data, comprising instructions operable to cause a programmable processor to:

determine if a hypothesized gradual transition is detected in the video data that ends with a current frame based at least on the current frame and one or more previous frames in the digitized video data, the hypothesized gradual transition having a length, a starting frame and an ending frame;

generate a synthetic gradual transition spanning the length of the hypothesized gradual transition and based on input including the starting frame and ending frame of the hypothesized gradual transition;

compare the hypothesized gradual transition to the synthetic gradual transition; and

mark the hypothesized gradual transition as a gradual transition only if the hypothesized gradual transition is sufficiently similar to the synthetic gradual transition.

26. The computer program product of claim 25, wherein the gradual transition is selected from the group of a dissolve, a wipe, a page flip, a tear, an iris transition and a pixelation transition.

27. A method for detecting a scene transition in digitized video data, comprising:

detecting that a hypothesized dissolve is present;

identifying where the hypothesized dissolve starts and ends, including identifying starting and ending frames and a length of the hypothesized dissolve;

generating a synthetic dissolve using the starting frame, ending frame and length of the hypothesized dissolve;

comparing one or more frames included in the hypothesized dissolve to frames included in the synthetic dissolve; and

marking the hypothesized dissolve as a dissolve only if the hypothesized dissolve is sufficiently similar to the synthetic dissolve.

28. The method of claim 27, wherein generating the synthetic dissolve comprises performing a cross-fade on the starting and ending frames of the hypothesized dissolve.

29. The method of claim 27, wherein comparing frames in the hypothesized dissolve to frames in the synthetically produced dissolve includes deriving a score representative of a similarity of the compared frames, the score derived from a mathematical comparison of the frames considering at least one parameter.

30. The method of claim 29, wherein the synthetic dissolve includes synthetic motion, the method further comprising determining a type and amount of synthetic motion based on a motion estimation of video data on either side of the hypothesized dissolve.

31. The method of claim 30, wherein the score is a product of color and motion differences between the compared frames.

32. The method of claim 30, wherein the score is a weighted mean of color and motion differences between the compared frames.

33. The method of claim 27, wherein if the hypothesized dissolve sufficiently overlaps a previously hypothesized dissolve, the method further comprising:

determining if the previously hypothesized dissolve is short enough to allow extension; and

if so, combining the hypothesized dissolved with the previously hypothesized dissolve; and

marking the combined dissolve as a dissolve.

34. The method of claim 27, wherein the step of detecting that a hypothesized dissolve is present includes comparing a current frame to one or more previous frames.

35. A computer program product, tangibly stored on a computer-readable medium, for detecting a scene transition in digitized video data, comprising instructions operable to cause a programmable processor to:

detect that a hypothesized dissolve is present;

identify where the hypothesized dissolve starts and ends, including identifying starting and ending frames and a length of the hypothesized dissolve;

generate a synthetic dissolve using the starting frame, ending frame and length of the hypothesized dissolve;

compare one or more frames included in the hypothesized dissolve to frames included in the synthetic dissolve; and

mark the hypothesized dissolve as a dissolve only if the hypothesized dissolve is sufficiently similar to the synthetic dissolve.

36. The computer program product of claim 35, wherein instructions operable to generate the synthetic dissolve comprise instructions operable to perform a cross-fade on the starting and ending frames of the hypothesized dissolve.

37. The computer program product of claim 35, wherein instructions operable to compare frames in the hypothesized dissolve to frames in the synthetically produced dissolve include instructions operable to derive a score representative of a similarity of the compared frames, the score derived from a mathematical comparison of the frames considering at least one parameter.

38. The computer program product of claim 37, wherein the synthetic dissolve includes synthetic motion, the instructions further operable to determine a type and amount of synthetic motion based on a motion estimation of video data on either side of the hypothesized dissolve.

39. The computer program product of claim 38, wherein the score is a product of color and motion differences between the compared frames.

40. The computer program product of claim 39, wherein the score is a weighted mean of color and motion differences between the compared frames.

41. The computer program product of claim 35, the instructions further operable to, if the hypothesized dissolve sufficiently overlaps a previously hypothesized dissolve:

determine if the previously hypothesized dissolve is short enough to allow extension;

if so, combine the hypothesized dissolved with the previously hypothesized dissolve; and

mark the combined dissolve as a dissolve.

42. The computer program product of claim 35, wherein instructions operable to detect that a hypothesized dissolve is present include instructions operable to compare a current frame to one or more previous frames.