Scene change detector for video data

• US 6,539,055 B1
• Filed: 12/03/1999
• Issued: 03/25/2003
• Est. Priority Date: 12/03/1999
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Hazra, Rajeeb
• Primary Examiner(s): Le, Vu
• Application Number:
  US09/453,715
• Time in Patent Office: 1,208 Days
• Field of Search: 375/240.16, 375/240.12, 375/240.26, 375/240.01, 375/240.02, 348/699-701, 348/416.1, 382/236, 382/238
• US Class Current: 375/240.16
• CPC Class Codes: G11B 2220/65 wherein solid state memory is used for storing indexing information or metadata
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 5/147 Scene change detection

Abstract

A viewer-intuitive index may be built from a stream of video data by detecting scene changes from the stream of video data, capturing reference markers associated with the video data for those portions of the video data for which the scene changes were detected and the storing the reference markers.

Claims

1. A method of building an index of a stream of video data, comprising:

detecting scene changes from the stream of video data, wherein a scene chance for a present frame is detected from;

a first comparison of video content of the present frame and of at least one immediately prior frame, the first comparison indicating a dynamic change in video content, and

a second comparison of video content of the present frame and of at least one immediately future frame, the second comparison indicating a low chance in video content; and

capturing reference markers associated with the video data for which the scene changes were detected and storing the reference markers.

2. The method of claim 1, wherein the capturing comprises receiving a reference marker from the stream of video data.

3. The method of claim 1, wherein the capturing comprises receiving a reference marker from a reference counter.

4. The method of claim 1, wherein the reference marker is a temporal reference.

5. The method of claim 1, wherein the reference marker is a count of frames of video data.

6. A method of detecting a scene change from display data of a plurality of temporally contiguous frames, comprising:

when there exists a dynamic change in display data among a present frame and an immediately past frame,

when there exists a low change in display data among the present frames and at least one future frame,

generating an output that identifies the occurrence of a scene change at the present frame.

7. The method of claim 6, further comprising determining that there exists a dynamic change in display data among the past and present frames, by:

counting a number of covered pixels in the present frame

counting a number of uncovered pixels in the present frame, and

comparing a ratio of the covered pixels to the uncovered pixels to a predetermined threshold.

8. The method of claim 7, further comprising comparing a number of uncovered pixels to a second predetermined threshold.

9. The method of claim 6, further comprising determining that there exists a low change in display data among the present frame and the future frames by:

counting a number of covered pixels in the future frames,

counting a number of uncovered pixels in the future frame, and

comparing a ratio of the covered pixels to the uncovered pixels in the future frame against a third predetermined threshold.

10. The method of claim 9, further comprising comparing the number of uncovered pixels against a fourth predetermined threshold.

11. A method of building an index of a stream of video data, comprising:

detecting scene changes according to the method of claim 6, and

when a scene change is detected, capturing reference markers associated with the video data for which the scene changes were detected and storing the reference markers.

12. The method of claim 11, wherein the capturing comprises receiving a reference marker from the stream of video data.

13. The method of claim 12, wherein the reference marker is a temporal reference.

14. The method of claim 12, wherein the reference marker is a count of frames of video data.

15. A video processing system, comprising:

a scene change detector having an input for a plurality of frames of video data, the detector comprising;

means to compare video content of a present frame and of at least one immediately prior frame and to indicate a dynamic change in said comparison, and

means to compare video content of the present frame and of at least one immediately future frame and to indicate a low change in said comparison;

a controller coupled to the scene change detector and having an input for reference marker data associated with the video data; and

a memory coupled to the memory controller.

16. The video processing system of claim 15, further comprising a reference counter coupled to the controller.

17. The video processor of claim 15, wherein the controller receives reference marker data from a data stream that also includes the video data.

18. A scene change detector comprising:

a first pixel classifier that identifies covered and uncovered pixels in a first video frame,

a second pixel classifier that identifies covered and uncovered pixels in a second video frame adjacent to the first frame, and

a scene change detector that generates a scene change signal responsive to the identifications made by the first and second pixel classifiers.

19. The scene change detector of claim 18, wherein one of the pixel classifiers comprises:

a pair of comparators, the first comparator receiving pixel data of first pair of adjacent video frames, the second comparator receiving pixel data of a second pair of adjacent video frames, wherein a video frame from the first and second pair are the same,

a pair of slicers, one coupled to each comparator, and

a classifier coupled to each of the slicers.

20. A video processing system, comprising:

the scene change detector of claim 18 having an input for video data,

a controller coupled to the scene change detector and having an input for reference marker data associated with video data, and

a memory coupled to the memory controller.

21. The video processing system of claim 20, further comprising a reference counter coupled to the controller.

22. A scene change detector, comprising:

first through third comparators, each receiving video data of a pair of video frames,

first through third slicers, each coupled to a respective comparator,

a pair of classifiers, one coupled to the first and second slicer, the other coupled to the second and third slicer, and

a scene change identifier coupled to the pair of classifiers.

23. A method of identifying a scene change based on a stream of video data, comprising:

counting a number of covered pixels and a number of uncovered pixels for a first frame of video data,

counting a number of covered pixels and a number of uncovered pixels for a second frame of video data prior and adjacent to the first frame

detecting a scene change that occurred at the first frame if;

a ratio of covered to uncovered pixels for the first frame exceeds a first threshold,

a number of uncovered pixels for the first frame exceeds a second threshold,

a ratio of covered to uncovered pixels for the second frame exceeds a third threshold, and

a number of uncovered pixels for the second frame exceeds a fourth threshold.

24. The method of claim 23, wherein the first and third thresholds are the same.

25. The method of claim 23, wherein the second and fourth thresholds are the same.

26. The method of claim 23, wherein the thresholds are user-programmable.

27. A method of detecting a scene change from a stream of video data, comprising:

comparing pixel data of a first and second video frame and obtaining first a differential signal therefrom,

generating a first binary signal based on a magnitude of the first differential signal,

comprising pixel data of the second and a third video frames and obtaining a second differential signal therefrom,

generating a second binary signal based on a magnitude of the second differential signal, comparing pixel data of the third and a forth video frames and obtaining a third differential signal therefrom,

generating a third binary signal based on a magnitude of the third differential signal, identifying covered and uncovered pixels for the first frame based on the first and second binary signals,

identifying covered and uncovered pixels for the second frame based on the second and third binary signals, and

determining based on the covered and uncovered pixels identified for the first and second frames, whether a scene change occurred at the first frame.

28. The method of claim 27, wherein the determining comprises:

counting a number of covered pixels and a number of uncovered pixels for a first frame of video data,

counting a number of covered pixels and a number of uncovered pixels for a second frame of video data prior and adjacent to the first frame detecting a scene change that occurred at the first frame if;

a ratio of covered to uncovered pixels for the first frame exceeds a first threshold,

a number of uncovered pixels for the first frame exceeds a second threshold,

a ratio of covered to uncovered pixels for the second frame exceeds a third threshold, and

a number of uncovered pixels for the second frame exceeds a fourth threshold.

29. A computer readable medium having stored thereon instructions that, when executed by a processor, cause the processor to:

detect scene changes from a stream of video data, wherein a scene change for a present frame is detected from;

a first comparison of video content of the present frame and of at least one immediately prior frame, the first comparison indicating a dynamic change in video content, and

a second comparison of video content of the present frame and of at least one immediately future frame, the second comparison indicating a low change in video content; and

capture reference markers associated with the video data for which scene changes were detected, and

store the reference markers.

30. A computer readable medium having stored thereon instructions that, when executed by a processor, cause the processor to:

count a number of covered pixels and a number of uncovered pixels for a first frame of video data,

count a number of covered pixels and a number of uncovered pixels for a second frame of video data, the second frame being adjacent to the first frame in a video stream, and

detect a scene change for the first frame if;

a ratio of covered to uncovered pixels for the first frame exceeds a first threshold,

a number of uncovered pixels for the first frame exceeds a second threshold,

a ratio of covered to uncovered pixels for the second frame exceeds a third threshold, and

a number of uncovered pixels for the second frame exceeds a fourth threshold.

31. A computer readable medium having stored thereon instructions that, when executed by a processor, cause the processor to:

compare pixel data of a first and second video frame and obtaining first a differential signal therefrom,

generate a first binary signal based on a magnitude of the first differential signal,

compare pixel data of the second and a third video frames and obtaining a second differential signal therefrom,

generate a second binary signal based on a magnitude of the second differential signal, comparing pixel data of the third and a fourth video frames and obtaining a third differential signal therefrom,

generate a third binary signal based on a magnitude of the third differential signal, identifying covered and uncovered pixels for the first frame based on the first and second binary signals,

identify covered and uncovered pixels for the second frame based on the second and third binary signals, and

determine, based on the covered and uncovered pixels identified for the first and second frames, whether a scene change occurred at the first frame.

32. The computer readable medium of claim 31, wherein the processor determines that a scene change occurred at the first frame if:

a ratio of covered to uncovered pixels for the first frame exceeds a first threshold,

a number of uncovered pixels for the first frame exceeds a second threshold,

a ratio of covered to uncovered pixels for the second frame exceeds a third threshold, and

a number of uncovered pixels for the second frame exceeds a fourth threshold.

33. A computer readable medium having stored thereon indices of display data, the indices stored on the medium according to the method of, for a frame of display data:

counting a number of covered pixels in the frame,

counting a number of uncovered pixels in the frame,

counting a number of covered pixels in a second frame temporally adjacent to the first frame,

counting a number of uncovered pixels in the second frame, and

storing an identifier of the first frame on the computer readable medium when;

a ratio of covered to uncovered pixels for the first frame exceeds a first threshold,

a number of uncovered pixels for the first frame exceeds a second threshold,

a ratio of covered to uncovered pixels for the second frame exceeds a third threshold, and

a number of uncovered pixels for the second frame exceeds a fourth threshold.