Method and apparatus for detecting a point of change in a moving image

• US 6,157,744 A
• Filed: 02/21/1996
• Issued: 12/05/2000
• Est. Priority Date: 02/21/1995
• Status: Expired due to Term

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Nagaya, Shigeki, Miyatake, Takafumi, Nagasaka, Akio, Tanaka, Kazuaki, Fujita, Takehiro
• Primary Examiner(s): Chang, Jon
• Assistant Examiner(s): Patel, Jayanti K.
• Application Number:
  US08/604,606
• Time in Patent Office: 1,749 Days
• Field of Search: 348/97, 348/154, 348/155, 348/700, 353/21, 360/13, 364/514 A, 364/514 R, 382/103, 382/107, 382/236, 382/278, 369/83, 386/4, 386/52, 386/64, 386/51, 395/173, 395/328, 395/806
• US Class Current: 382/236
• CPC Class Codes: G06F 16/745 the internal structure of a single video sequence
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

A system for detecting a point of change between video shots from a video having a plurality of succeeding frames. The system includes video playback apparatus for playing a video chronologically one frame at a time, and a display for displaying the video. The a processing device for calculating a feature quantity of video image data for each frame, determining a first correlation coefficient between a feature quantity of a current frame and a feature quantity calculated from an immediately preceding frame and determining a second correlation coefficient between the feature quantity of the current frame and a feature quantity of at least two frames preceding the current frame, and indicating on the display a point of change between video shots when the first correlation coefficient and the second correlation coefficient are out of predetermined allowable ranges. The correlation coefficients of each frame is stored and can be used by the processing device to dynamically change a reference used for detecting a point of change between video shots. The change in the reference is performed based on the stored correlation coefficients or feature quantities of past frames.

Claims

1. A method of detecting a change between video shots from a video including a plurality of succeeding frame images, said method comprising the steps of:

inputting said video into a processing device;

obtaining a feature quantity of each frame of the inputted video;

determining a first correlation coefficient which is a difference between a feature quantity of a current frame image and a feature quantity of a first preceding frame image immediately preceding said current frame image;

determining a second correlation coefficient which is a difference between the feature quantity of said current frame image and a feature quantity of a second preceding frame at least two frame image preceding said current frame image; and

indicating a change between video shots a time when the first correlation coefficient and the second correlation coefficient are out of predetermined allowable ranges;

wherein the number of frame images preceding said current frame image for said second preceding frame image is successively increased so as to distinguish a disturbance lasting an arbitrary number of frame images from a change between video shots.

2. A method according to claim 1, wherein said feature quantity is a color histogram.

3. A method according to claim 1, further comprising the steps of:

extracting a frame located at said change between video shots or a frame located at a position a predetermined offset from said change between video shots;

resizing the frame image to a preset size; and

storing the resized frame in a storage device or a medium.

4. A method according to claim 3, further comprising the step of:

storing the resized frame image in a storage device or medium by relating it to at least one of four sets of associated information including first information representing a position of said change in video, second information representing a time when said change occurred, third information representing a distance or time between said change and an immediately preceding change in video, and fourth information describing attributes of said change or of a shot beginning with said change.

5. A method according to claim 4, wherein the video frame images and their related information are managed in a single file.

6. A method according to claim 3, wherein a buffer is used to store two or more latest frame images or resized frame images at all times so that when a change is detected in a preceding frame, the frame image containing said change is extracted from the buffer.

7. A method according to claim 1, wherein in indicating whether the first correlation coefficient and the second correlation coefficient are out of the predetermined allowable ranges, the predetermined allowable ranges are changed according to the feature quantity of an immediately preceding frame or an immediately following frame image.

8. A method according to claim 7, wherein the feature quantity of an immediately preceding frame image or an immediately following frame image is an overall brightness level of at least an immediately preceding frame image or at least an immediately following frame image.

9. A method according to claim 7, wherein the feature quantity of an immediately preceding frame image or an immediately following frame image is a correlation coefficient between the current frame image and an immediately preceding or following frame image.

10. A method according to claim 7, wherein the feature quantity of an immediately preceding frame image or an immediately following frame image is the number of frame images to be processed per unit time for the immediately preceding or following video image.

11. A method according to claim 1, wherein the number of frame images preceding said current frame image is successively increased for determining said second correlation coefficient in order to distinguish a disturbance lasting an arbitrary number of frame images from a change between video shots.

12. A system for detecting a change between video shots from a video including a plurality of succeeding frame images, said system comprising:

video playback apparatus for playing said video;

a display for displaying said video; and

a processing device for obtaining a feature quantity of each frame of the inputted video, determining a first correlation coefficient which is a difference between a feature quantity of a current frame image and a feature quantity of a first preceding frame image immediately preceding said current frame image and determining a second correlation coefficient which is a difference between the feature quantity of the current frame image and a feature quantity of a second preceding frame image at least two frame images preceding said current frame image, and indicating on said display a change between video shots when the first correlation coefficient and the second correlation coefficient are out of predetermined allowable ranges;

wherein the number of images preceding said current frame images for said second preceding frame images is successively increased in order to distinguish a disturbance lasting an arbitrary number of frames from a change between video shots.

13. A system according to claim 12, wherein said feature quantity is a color histogram.

14. A system according to claim 12, further comprising:

means for extracting a frame located at the change in shot or a frame located at a position a predetermined offset from the change;

means for resizing the frame to a preset size; and

means for storing the resized frame in a storage device or a medium.

15. A system according to claim 14, further comprising:

means for storing the resized frame image in a storage device or medium by relating it to at least one of four sets of associated information, including first information representing a position of the change in video, second information representing a time when the change occurred, third information representing a distance or time between the change and an immediately preceding change in video, and fourth information describing attributes of the change or of a shot beginning with the change.

16. A system according to claim 15, wherein the video frame images and their related information are managed in a single file.

17. A system according to claim 14, wherein a buffer is used to store two or more latest frame images or resized frame images at all times so that when of change is detected in a preceding frame image, the frame image containing the change is extracted from the buffer.

18. A system according to claim 12, wherein said processing device in indicating whether the first correlation coefficient and the second correlation coefficient are out of the predetermined allowable ranges, the predetermined allowable ranges are changed according to the feature quantity of an immediately preceding frame or an immediately following frame image.

19. A system according to claim 18, wherein the feature quantity of an immediately preceding frame or an immediately following frame image is an overall brightness level of at least an immediately preceding frame image or at least an immediately following frame image.

20. A system according to claim 18, wherein the feature quantity of an immediately preceding frame image or an immediately following frame image is a correlation coefficient between the current frame image and an immediately preceding or following frame image.

21. A system according to claim 18, wherein the feature quantity of an immediately preceding frame image or an immediately following frame image is the number of frames to be processed per unit time for the immediately preceding or following video image.

22. A system according to claim 12, wherein the number of frame images preceding said current frame image is successively increased for determining said second correlation coefficient in order to distinguish a disturbance lasting an arbitrary number of frame images from a change between video shots.

23. A computer program product for use with a computer having a display, said computer being used to detect a change between video shots from a video including a plurality of succeeding frame images, said computer program comprising:

a computer readable medium with the computer program recorded thereon, the computer program comprises;

a first code section for inputting video into a processing device,

a second code section for obtaining a feature quantity of each frame of the inputted video,

a third code section for determining a first correlation coefficient which is a difference between a feature quantity of a current frame image and a feature quantity of a first preceding frame image immediately preceding said current frame image and determining a second correlation coefficient which is a difference between the feature quantity of the current frame image and a feature quantity of a second preceding frame image at least two frame images preceding said current frame image, and

a fourth code section for indicating a change between video shots when the first correlation coefficient and the second correlation coefficient are out of predetermined allowable ranges;

wherein the number of frame images preceding said current frame image for said second preceding frame image is successively increased in order to distinguish a disturbance lasting an arbitrary number of frame images from a change between video shots.

24. A computer program product according to claim 23, wherein said feature quantity is a color histogram.

25. A computer program product according to claim 23, further comprising:

a fifth code section for extracting a frame image located at the change in shot or a frame image located at a position a predetermined offset from the change;

a sixth code section for resizing the frame image to a preset size; and

a seventh code section for storing the resized frame image in a storage device or a medium.

26. A computer program product according to claim 25, further comprising:

an eighth code section for storing the resized frame image in a storage device or medium by relating it to at least one of four sets of associated information including first information representing a position of the change in video, second information representing a time when the change occurred, third information representing a distance or time between the change in video and an immediately preceding point of change, and fourth information describing attributes of the change or of a shot beginning with the change.

27. A computer program product according to claim 26, wherein the video frame images and their related information are managed in a single file.

28. A computer program product according to claim 25, wherein a buffer is used to store two or more latest frame images or resized frame images at all times so that when a change in video is detected in a preceding frame image, the frame containing the change is extracted from the buffer.

29. A computer program product according to claim 23, wherein in indicating whether the first correlation coefficient and the second correlation coefficient are out of the predetermined allowable ranges, the predetermined allowable ranges are changed according to the feature quantity of an immediately preceding frame image or an immediately following frame image.

30. A computer program product according to claim 29, wherein the feature quantity of an immediately preceding frame image or an immediately following frame image is an overall brightness level of at least an immediately preceding frame image or at least an immediately following frame image.

31. A computer program product according to claim 29, wherein the feature quantity of an immediately preceding frame image or an immediately following frame is a correlation coefficient between the current frame image and an immediately preceding or following frame image.

32. A computer program product according to claim 29, wherein the feature quantity of an immediately preceding frame image or an immediately following frame image is the number of frame images to be processed per unit time for the immediately preceding or following video image.

33. A computer program product according to claim 23, wherein the number of frame images preceding said current frame image is successively increased for determining said second correlation coefficient in order to distinguish a disturbance lasting an arbitrary number of frame images from a point of change between video shots.

34. A computer program product for use with a computer having a display, said computer being used to detect a change between video shots from a video including a plurality of succeeding frame images, said computer program product comprising:

a logic circuit with the computer program product recorded thereon, the computer program comprises;

a first code second for inputting a video into a processing device,

a second code section for obtaining a feature quantity of each frame of the inputted video,

a third code section for determining a first correlation coefficient which is a difference between a feature quantity of current frame image and a feature quantity of a first preceding frame image immediately preceding said current frame image and determining a second correlation coefficient which is a difference between the feature quantity of the current frame image and a feature quantity of a second preceding frame image at least two frame images preceding said current frame image, and

a fourth code section for indicating a change between video shots when the first correlation coefficient and the second correlation coefficient are out of predetermined allowable ranges;

wherein the number of frame images preceding said current frame image for said second preceding frame image is successively increased in order to distinguish a disturbance lasting an arbitrary number of frame images from a change between video shots.

35. A computer program product according to claim 34, wherein said feature quantity is a color histogram.

36. A computer program product according to claim 34, further comprising:

a fifth code section for extracting a frame image located at the change in shot or a frame located at a position a predetermined offset from the change in video;

a sixth code section for resizing the frame image to a preset size; and

a seventh code section for storing the resized frame image in a storage device or a medium.

37. A computer program product according to claim 36, further comprising:

an eighth code section for storing the resized frame image in a storage device or medium by relating it to at least one of four sets of associated information including first information representing a position of the change in video, second information representing a time when the change in video occurred, third information representing a distance or time between the change in video and an immediately preceding change, and fourth information describing attributes of the change in video or of a shot beginning with the change in video.

38. A computer program product according to claim 37, wherein the video frame images and their related information are managed in a single file.

39. A computer program product according to claim 36, wherein a buffer is used to store two or more latest frame images or resized frame images at all times so that when a change in video is detected in a preceding frame image, the frame containing the change is extracted from the buffer.

40. A computer program product according to claim 34, wherein in indicating whether the first correlation coefficient and the second correlation coefficient are out of the predetermined allowable ranges, the predetermined allowable ranges are changed according to the feature quantity of an immediately preceding frame image or an immediately following frame image.

41. A computer program product according to claim 40, wherein the feature quantity of an immediately preceding frame image or an immediately following frame image is an overall brightness level of at least an immediately preceding frame image or at least an immediately following frame image.

42. A computer program product according to claim 40, wherein the feature quantity of an immediately preceding frame image or an immediately following frame image is a correlation coefficient between the current frame image and an immediately preceding or following frame image.

43. A computer program product according to claim 40, wherein the feature quantity of an immediately preceding frame image or an immediately following frame image is the number of frame image to be processed per unit time for the immediately preceding or following video image.

44. A computer program product according to claim 34, wherein the number of frame images preceding said current frame image is successively increased for determining said second correlation coefficient in order to distinguish a disturbance lasting an arbitrary number of frame images from a change between video shots.