Method and apparatus for shot detection

• US 7,444,018 B2
• Filed: 08/31/2004
• Issued: 10/28/2008
• Est. Priority Date: 06/14/2001
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Dong, Qi, Wei
• Primary Examiner(s): Mariam; Daniel G
• Application Number:
  US10/930,170
• Publication Number:
  US 20050025358A1
• Time in Patent Office: 1,519 Days
• Field of Search: 382/168, 382/170, 382/236, 382/239, 382/268, 382/270, 382/272, 382/274, 382/275, 382/305, 348/700
• US Class Current: 382/170
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 5/147 Scene change detection

Abstract

A system identifies an abrupt transition in content between two frames, and determines whether the abrupt transition was caused by a shot boundary between the two frames or by a flashlight event. Identification of the abrupt transition in content includes calculating a difference in light intensity histograms between the current frame and a preceding frame, and comparing the histogram difference to a dynamically determined threshold. Further, an average light intensity based method is used to determine whether the abrupt transition was caused by a shot boundary or by a flashlight event.

Claims

1. A method comprising:

identifying an abrupt transition in average light intensity between two frames; and

determining whether the abrupt transition was caused by a shot boundary between the two frames or by a flashlight event, wherein determining a cause of the abrupt transition comprises;

calculating a difference in average minimal intensity values between a window of frames preceding the current frame and a window of frames subsequent to the current frame;

generating a ratio of the average intensity change of the current frame to the calculated difference in average minimal intensity values; and

concluding that the abrupt transition is caused by a flashlight event if the ratio does not exceed a threshold.

2. A method according to claim 1, wherein identifying an abrupt transition comprises:

calculating a difference in light intensity histograms between the current frame and a preceding frame; and

comparing the histogram difference to a dynamically determined threshold, wherein an abrupt transition is indicated if the histogram difference exceeds the threshold.

3. A method according to claim 1, wherein calculating a difference in average minimal intensity values comprises:

identifying a window of frames on either side of the current frame;

generating an average minimal intensity value for each window from a first and second minimal intensity value of each frame, respectively; and

calculating the difference between the generated average minimal intensity values.

4. A method according to claim 1, wherein the threshold is equal to one (1), wherein a ratio that deviates from one indicates that the abrupt transition is due to a shot cut, while a ratio close to one indicates that the abrupt transition is due to a flashlight event.

5. A computer-readable medium encoded with a plurality of executable instructions which, when executed, implement a method according to claim 1.

6. A media analysis agent comprising:

a shot boundary detector to statistically analyze one or more attributes associated with content in video frames to detect abrupt and gradual transitions in the video content indicative of a shot boundary;

a flashlight detector, responsive to the shot boundary detector, to distinguish abrupt transitions in the video content caused by flashlight events from those caused by actual shot boundaries, to reduce false-positive identification of flashlight events as shot boundaries; and

an adaptive threshold selection module, responsive to the shot boundary detector, to dynamically set one or more thresholds for use in shot boundary identification based, at least in part, on one or more attributes of video content, wherein the adaptive threshold selection module calculates an average and standard deviation of a histogram difference in sliding windows of frames proceeding and subsequent to a current frame, and generates a proposed low threshold value (Ts) and a proposed high-threshold value (Tb) from the average histogram difference.

7. A media analysis agent according to claim 6, wherein the adaptive threshold selection module certifies the proposed thresholds if a calculated standard deviation does not reach a standard deviation threshold.

8. A media analysis agent according to claim 7, wherein if the adaptive threshold selection module cannot certify the proposed thresholds, prior thresholds are used by shot boundary detector.

9. A media analysis agent according to claim 6, wherein the high threshold is used to identify an abrupt transition, while the low threshold is used to indicate a potential gradual transition.

10. A media analysis agent according to claim 9, wherein shot boundary detector compares the histogram difference to the low threshold if the difference does not exceed the high threshold, and accumulates histogram differences of frames wherein the differences exceeds the low threshold.

11. A media analysis agent according to claim 10, wherein shot boundary detector identifies a gradual transition when the accumulated histogram differences exceed the high threshold.

12. A media analysis agent according to claim 6, wherein the shot boundary detector calculates a difference in light intensity histograms between a current frame and a preceding frame, and compares the histogram difference to a dynamically determined threshold, wherein an abrupt transition is identified by a histogram difference that exceeds the threshold.

13. A media analysis agent according to claim 12, wherein shot boundary detector invokes an instance of flashlight detector to determine whether the abrupt transition is the result of a flashlight event.

14. A media analysis agent according to claim 6, wherein flashlight detector calculates a difference in average minimal intensity values between a first sliding window preceding the current frame, and a second sliding window subsequent to the current frame.

15. A media analysis agent according to claim 14, wherein flashlight detector calculates the average intensity change of the current frame, and generates a ratio of the difference in average minimal intensity values to the average intensity change of the current frame.

16. A media analysis agent according to claim 15, wherein flashlight detector determines that the abrupt change was due to a flashlight event if the ratio does not exceed a threshold value.

17. A media analysis agent according to claim 16, wherein the threshold value is one (1).

18. A media analysis agent according to claim 16, wherein the flashlight detector concludes that the abrupt transition was caused by a shot boundary if the ratio exceeds the threshold value.

19. A computing system comprising:

a memory, to receive media content; and

a media processing system, coupled to the memory device, including a media analysis agent according to claim 6 to segment the received media content into uniquely identifiable shots.

20. A computer readable medium storing a plurality of executable instructions which, when executed, implement a media analysis agent to detect shot boundaries resulting from an abrupt or gradual transition in content between frames, and further to distinguish abrupt transitions caused by shot cuts from those caused by flashlight events, wherein the media analysis agent calculates a difference in average minimal intensity values between a first sliding window preceding the current frame, and a second sliding window subsequent to the current frame, and calculates the average intensity change of the current frame, and generates a ratio of the difference in average minimal intensity values to the average intensity change of the current frame.

21. A storage medium according to claim 20, wherein media analysis agent determines that the abrupt change was due to a flashlight event if the ratio does not exceed a threshold value.

22. A storage medium according to claim 21, wherein the threshold value is one (1).

23. One or more computer-readable media having stored thereon a computer program that, when executed by one or more processors, causes the one or more processors to:

calculate a difference in light intensity histograms between two frames;

compare the histogram difference to a dynamically determined threshold, wherein an abrupt transition is indicated if the histogram difference exceeds the threshold; and

determine whether the abrupt transition was caused by a shot boundary between the two frames or by a flashlight event, wherein to determine a cause of the abrupt transition, the one or more processors;

calculate a difference in average minimal intensity values between a window of frames preceding the current frame and a window of frames subsequent to the current frame;

generate a ratio of the average intensity change of the current frame to the calculated difference in average minimal intensity values; and

conclude that the abrupt transition is caused by a flashlight event if the ratio does not exceed a threshold.

24. One or more computer-readable media as recited in claim 23 wherein the threshold is equal to one, wherein a ratio that deviates from one indicates that the abrupt transition is due to a shot cut, while a ratio close to one indicates that the abrupt transition is due to a flashlight event.

25. An apparatus comprising:

means for identifying an abrupt transition in average light intensity between two frames, wherein the means for identifying;

calculates a difference in light intensity histograms between the current frame and a preceding frame;

compares the histogram difference to a threshold value; and

means for determining whether the abrupt transition was caused by a shot boundary between the two frames or by a flashlight event, wherein the means for determining;

calculates a difference in average minimal intensity values between a window of frames preceding the current frame and a window of frames subsequent to the current frame;

generates a ratio of the average intensity change of the current frame to the calculated difference in average minimal intensity values; and

concludes that the abrupt transition is caused by a flashlight event if the ratio does not exceed the threshold value.

26. An apparatus according to claim 25, wherein the threshold value is dynamically determined.

27. An apparatus according to claim 25, wherein an abrupt transition is indicated if the histogram difference exceeds the threshold value.