Systems and methods for detecting scene changes in a video data stream

• US 7,110,452 B2
• Filed: 03/05/2002
• Issued: 09/19/2006
• Est. Priority Date: 03/05/2001
• Status: Active Grant

• Current Assignee: Corel Corporation (Corel Holdings LP)
• Original Assignee: Intervideo Incorporated (Corel Holdings LP)
• Inventors: Katsavounidis, Ioannis, Kim, Chang-Su, Zhao, Lifeng
• Primary Examiner(s): An, Shawn S.
• Application Number:
  US10/092,394
• Publication Number:
  US 20030053538A1
• Time in Patent Office: 1,659 Days
• Field of Search: 375/240.08, 375/240.13, 375/240.24, 375/240.26, 375/240.01, 375/240.12, 348/699, 382/235, 382/236, 382/238
• US Class Current: 375/240.08
• CPC Class Codes: H03M 13/00 Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes error detection or error correction for analogue/digital, digital/analogue or code conversion H03M1/00 – H03M11/00; specially adapted for digital computers G06F11/08; for information storage based on relative movement between record carrier and transducer G11B, e.g. G11B20/18; for static stores G11C
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
• CPC Class Codes: H03M 7/40 Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code
• CPC Class Codes: H04N 19/29 involving scalability at the object level, e.g. video object layer [VOL]
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/65 using error resilience
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/236 Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 5/147 Scene change detection
• CPC Class Codes: H04W 84/14 WLL [Wireless Local Loop]; RLL [Radio Local Loop]

Abstract

The present invention is related to video encoding. In an embodiment, a first root mean squared (RMS) value is calculated for a first frame relative to a second frame, and a second frame relative to a third frame. A second temporal derivative RMS value is calculated. Based at least in part on the second derivative value, the second frame is designated as a scene change frame.

Claims

1. A method of detecting a scene change in a digital video sequence having a plurality of frames, the method comprising:

calculating a first color weighted root mean squared (RMS) value for a first frame relative to a second frame and the second frame relative to a third frame;

calculating a first mean absolute difference (MAD) value for the first frame relative to the second frame;

determining if the first color weighted RMS value meets a first criterion;

determining if the first MAD value meets a second criterion; and

designating the second frame as a scene change frame at least partly in response to determining that both the first color weighted RMS value meets the first criterion and the first MAD value meets the second criterion.

2. The method as defined in claim 1, further comprising storing an I-frame designation in a file for the second frame and storing a P-frame designation for a third frame in the file.

3. The method as defined in claim 1, wherein the first color weighted RMS value is based at least in part on pixel luminance values and chromaticity-blue and chromaticity-red component values of the first and second frames.

4. A method of detecting a scene change in a digital video sequence having a plurality of frames, the method comprising:

calculating a first root mean squared (RMS) value for a first frame relative to a second frame and the second frame relative to a third frame, the first RMS value being defined as 30 RMS (Fi, Fk)=++1 w .times. hx=1 wy=1 hYi (x, y)−

Yk (x, y)2+2++1 w .times. hx=1 w/2 y=1 h/2 Ui(x, y)−

Uk (x, y)2+2++1 w .times. hx=1 w/2 y=1 h/2 Vi (x, y)−

Vk(x, y)2

where F.sub.i is the first frame, F.sub.k is the second frame, F(x, y) denotes the (x, y).sup.th pixel in frame F, w is a frame width and h is a frame height Y(x, y) corresponds to a pixel luminance value, and U(x, y) and V(x, y) corresponds to chromaticity components, and, and are weighting coefficients for luminosity, chromaticity-blue and chromaticity-red components correspondingly;

calculating a first mean absolute difference (MAD) value for the first frame relative to the second frame;

determining if the first RMS value meets a first criterion;

determining if the first MAD value meets a second criterion; and

designating the second frame as a scene change frame at least partly in response to determining that both the first RMS value meets the first criterion and the first MAD value meets the second criterion.

5. The method as defined in claim 4, wherein ===1.

6. The method as defined in claim 4, wherein the first MAD value is calculated using luminance value and excluding chromaticity components.

7. The method as defined in claim 4, wherein the first criterion is a first threshold and the second criterion is a second threshold.

8. A method of detecting a scene change in a digital video sequence, the method comprising:

calculating a second temporal derivative RMS value for a first frame relative to a second frame and the second frame relative to a third frame; and

based at least in part on the second derivative value, determining that the second frame is a scene change frame.

9. The method as defined in claim 8, wherein the determination that the second frame is a scene change frame is further based upon a mean absolute difference value calculated using at least luminosity information for the first and the second frames.

10. The method as defined in claim 8, wherein the determination that the second frame is a scene change frame is further based upon both an RMS value meeting a first criterion and the second temporal derivative RMS value meeting a second criterion.

11. The method as defined in claim 8, wherein the second temporal derivative RMS value is greater than or equal to a first threshold.

12. A method of detecting a scene change in a digital video sequence, the method comprising:

calculating a second temporal derivative RMS value for a first frame relative to a second frame and the second frame relative to a third frame; and

based at least in part on the second derivative value, determining that the second frame is designated as a scene change frame when the second temporal derivative RMS value is negative and has a greater absolute value than a first value.

13. A method of detecting a scene change in a digital video sequence, the method comprising:

calculating a first RMS value, wherein the first RMS value is color weighted;

calculating a second temporal derivative RMS value based only on temporal components for a first frame relative to a second frame and the second frame relative to a third frame; and

based at least in part on the second derivative value, determining that the second frame is a scene change frame.

14. A method of detecting a scene change in a digital video sequence, the method comprising:

calculating a second temporal derivative RMS value for a first frame relative to a second frame and the second frame relative to a third frame, the second temporal derivative RMS value being equal to (RMS(F.sub.i−

1, F.sub.i)−

2RMS(F.sub.i, F.sub.i+1)+RMS(F.sub.i+1, F.sub.I+2)), where F.sub.i−

1 is the first frame, F.sub.i is the second frame, F.sub.i+1 is a third frame, and F.sub.i+2 is a fourth frame; and

based at least in part on the second derivative value, determining that the second frame is a scene change frame.