Efficient shot change detection on compressed video data

• US 5,900,919 A
• Filed: 12/02/1996
• Issued: 05/04/1999
• Est. Priority Date: 08/08/1996
• Status: Expired due to Fees

• Current Assignee: Transpacific IP Group Limited
• Original Assignee: Industrial Technology Research Institute
• Inventors: Chen, Arbee L. P., Ni, Cheng-Yao, Lin, Y. B., Kuo, Tony C. T., Chen, Shu-Chin Su
• Primary Examiner(s): LEE, RICHARD J
• Application Number:
  US08/758,784
• Time in Patent Office: 883 Days
• Field of Search: 348/384, 348/390, 348/222, 348/402, 348/407, 348/409-413, 348/415, 348/416, 348/564, 348/699, 348/700, 345/302, 382/232, 382/236, 382/238
• US Class Current: 348/700
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109
• CPC Class Codes: H04N 5/147 Scene change detection

Abstract

A mask matching approach to detect shot changes in MPEG coded video uses reference ratio variances of macroblocks between MPEG coded frames. A function is designed to quantize the results into shot change probability values. Moreover, a conversion function modifies this probability computation to minimize misdetection and loss of detection under unusual image pattern situations. A shot change probability threshold is defined for the video to be examined. When a frame'"'"'s modified shot change probability exceeds this threshold, the frame is considered to contain a shot change. With this approach, processing time is reduced by evaluating MPEG coded data directly, rather than the raw video.

Claims

1. A method for detecting shot changes in compressed MPEG coded video data having I, P, and B frames, with an IPB ratio corresponding to the numbers of I, P, and B frames, respectively, comprising the steps of:

(a) defining a qualifying set of shot change masks in accordance with said IPB ratio of said compressed MPEG coded video data, said qualifying set of shot change masks comprising a sequence of I, P, and B types of frame masks, each of said I, P, and B types of frame masks having low reference ratio characteristics between adjacent frames within each I, P, and B type of frame mask,

(b) calculating a shot change probability function P for each I, P, and B frame of said video data, based on said reference ratio of characteristics shot change frame masks, wherein each said P and B frame is encoded with references to adjacent frames, and wherein said I frames are encoded independently,

(c) determining a threshold value T for said shot change probability function P,

(d) comparing said shot change probability function P for each said I, P, and B frame of said video data with said threshold value T, and

(e) detecting a shot change at any of said I, P, and B frames of said video data when said shot change probability function P of said I, P, or B frame exceeds said threshold value T.

2. The method of claim 1 wherein step (b) calculates said shot change probability function P for each said I, P, and B frame in accordance with the following equation:

##EQU3## where f₁, f₂, . . . F_n, .di-elect cons. said frame masks, and RR_fi is the corresponding reference ratio of a frame mask f_i.

3. The method of claim 1 wherein step (c) determines said threshold value T for said shot change probability function P in accordance with the following equation:

##EQU4## where F is the average probability of 97.5% of all said frames having lowest probabilities, and F'"'"' is the average probability of 2.5% of all said frames having highest probabilities.

4. The method of claim 2 wherein step (b) further comprises an adjustment to said shot change probability function P for each said I, P, and B frame in accordance with the following conversion function:

##EQU5## where i is the frame number of said I, P, and B frame whose shot change probability is currently being computed,

where j is a predetermined value which represents a minimum distance between two successive shot changes, and

where k is determined by said values of i and j.

5. A method for detecting shot changes in compressed MPEG coded video data having I, P, and B frames, with an IPB ratio corresponding to the numbers of I, P, and B frames, respectively, comprising the steps of:

(a) defining a shot change mask cluster in accordance with said IPB ratio of said compressed MPEG coded video data, wherein said shot change mask cluster comprises a sequence of I, P, and B types of frame masks, each of said I, P, and B types of frame masks having low reference ratio characteristics between adjacent frames within each I, P, and B type of frame mask,

(b) comparing said shot change mask cluster with each said I, P, and B frame of video data to find a match between said I, P, and B frame and its corresponding type of frame mask within said shot change mask cluster, and

(c) detecting a shot change at any of said I, P, and B frames of said video data when one of said shot change mask cluster frame masks matches one of a corresponding type of said I, P, and B frames.

6. A system for detecting shot changes in compressed MPEG coded video data having I, P, and B frames, with an IPB ratio corresponding to the numbers of I, P, and B frames, respectively, comprising:

(a) an input device for receiving MPEG coded data,

(b) a processor for operating on said received data,

(c) a memory for storing data operated on by said processor,

(d) an output device for displaying results from said processor,

(e) said processor for defining a shot change mask cluster based on said IPB ratio of said compressed MPEG coded video data,

(f) said processor for calculating a shot change probability function P for each frame of said video data, wherein each said P and B frame is encoded with references to adjacent frames, and wherein said I frames are encoded independently,

(g) said processor for determining a threshold value T for said shot change probability function P,

(h) said processor for comparing said shot change probability function P for each said frame with said threshold value T, and

(i) said processor for detecting a shot change at any of said frames when said shot change probability function P of said frame exceeds said threshold value T,

wherein said shot change mask cluster comprises a sequence of I, P, and B types of frame masks, each of said I, P, and B types of frame masks having low reference ratio characteristics between adjacent frames within said I, P, and B types of frame masks.

7. The system of claim 6 wherein step (b) calculates said shot change probability function P for each I, P, and B frame in accordance with the following equation:

##EQU6## where f₁, f₂, . . . f_n, .di-elect cons. said frame masks, and RR_fi is the corresponding reference ratio of a frame mask f_i.

8. The system of claim 6 wherein step (c) determines said threshold value T for said shot change probability function P in accordance with the following equation:

##EQU7## where F is the average probability of 97.5% of all said frames having lowest probabilities, and F'"'"' is the average probability of 2.5% of all said frames having highest probabilities.

9. The system of claim 7 wherein step (b) further comprises an adjustment to said shot change probability function P for each said I, P, and B frame in accordance with the following conversion function:

##EQU8## where i is the frame number of said I, P, and B frame whose shot change probability is currently being computed,

where j is a predetermined value which represents a minimum distance between two successive shot changes, and

where k is determined by said values of i and j.

10. A system for detecting shot changes in compressed MPEG coded video data having I, P, and B frames, with an IPB ratio corresponding to the numbers of I, P, and B frames, respectively, comprising:

(a) a processor for defining a shot change mask cluster based on said IPB ratio of said compressed MPEG coded video data, wherein said shot change mask cluster comprises a sequence of I, P, and B types of frame masks, each of said I, P, and B types of frame masks having low reference ratio characteristics between adjacent frames within said types of I, P, and B frame masks,

(b) said processor for comparing said shot change mask cluster with each said I, P, and B frame to find a match between said I, P, and B frame and its corresponding type of frame mask within said shot change mask cluster, and

(c) said processor for detecting a shot change at any of said I, P, and B frames when one of said shot change mask cluster frame masks matches one of a corresponding type of said I, P, and B frames.