Video processing method and apparatus

• US 5,701,163 A
• Filed: 12/18/1995
• Issued: 12/23/1997
• Est. Priority Date: 01/18/1995
• Status: Expired due to Term

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony United Kingdom Limited (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony United Kingdom Limited (Sony Group Corp.)
• Inventors: Dorricott, Martin Rex, Richards, John William
• Primary Examiner(s): Chin, Tommy P.
• Assistant Examiner(s): Snow, Frank
• Application Number:
  US08/573,992
• Time in Patent Office: 736 Days
• Field of Search: 348/578, 348/579, 348/598, 348/618, 348/619, 348/620, 348/626, 348/597, 382/107, 382/262, 382/264
• US Class Current: 348/578
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
• CPC Class Codes: H04N 5/2625 for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect sequence generated by event triggered capturing H04N7/188

Abstract

A video processing method and apparatus for processing video frames comprising animated images so as to blur movement of the animated images between successive frames, the method or apparatus comparing a first input frame with the following input frame; identifying moving areas in the first input frame by the comparison; calculating the velocity of the movement of the moving areas between the first and following input frame; summing, only for each input pixel site within the identified moving areas of the input frame, the pixel value of each input pixel site and adjacent pixel values determined according to the calculated velocities and a predetermined aperture type for the input frame; and providing an output frame from at least the summed pixel values and pixel values of areas of the input frame other than the moving area, the adjacent pixel values either being the pixel values of pixel sites in the input frame which are spatially adjacent in the direction of the velocity or predicted temporally adjacent pixel values for each input pixel site for times intermediate the predetermined aperture time.

Claims

1. A method for processing video frames comprising animated images so as to blur movement of the animated images between successive frames, the method comprising the steps of:

comparing a first input frame with the following input frame;

identifying moving areas in the first input frame by means of the comparison;

calculating the velocity of the movement of the moving areas between the first and following input frame;

summing, only for each input pixel site within the identified moving areas of the input frame, the pixel value of said each input pixel site and adjacent pixel values determined according to the calculated velocities and a predetermined aperture time for the input frame; and

providing an output frame from at least the summed pixel values and pixel values of areas of the input frame other than said moving areas.

2. A method according to claim 1 further comprising the step of:

producing a vector field indicating the velocity of each pixel in the input frame.

3. A method according to claim 2 further comprising the step of low-pass filtering the vector field so as to further blur motion at transitions between moving areas and areas other than said moving areas.

4. A method according to claim 3 wherein the bandwidths used for low-pass filtering the vector field are adaptively determined and are determined according to the magnitude of velocities in the local vicinity of the pixel being filtered.

5. A method according to claim 4 comprising the step of using an FIR filter to process pixel values on the basis of the calculated velocities such that, for each pixel site of a moving area, the sum of adjacent pixels is output, whereas, for each pixel site where the velocity is zero, the respective pixel value of the input frame is output.

6. A method according to claim 5 wherein the FIR filter operates according to ##EQU2## where:

(x,y)=pixel coordinates, lsharp=input pixel value, lblur=output pixel value, v=velocity of object measured in number of pixels moved between the first input frame and the following input frame, d=predetermined aperture time as a fraction of the time between the first input frame and the following input frame, w=a shutter transmission characteristic.

7. A method according to claim 6 when the shutter transmission characteristic is non-linear through the predetermined aperture time.

8. A method according to claim 1 wherein the adjacent pixel values are predicted temporally adjacent pixel values for said each input pixel site for times intermediate said predetermined aperture time.

9. A method according to claim 8 further comprising the steps of:

predicting the pixel sites and the pixel values for moving areas of the image at each of said intermediate times; and

providing, for each of said each input pixel site and any additional predicted sites, a weighted sum of the respective pixel values for the predetermined aperture time.

10. A method according to claim 9 further comprising the steps of:

storing, in an array having addresses corresponding to pixel sites, first flags indicating pixel sites of the moving areas in the input frame and second flags indicating all other pixel sites in the input frame; and

for predicted pixel sites of the moving areas at the last intermediate time of the predetermined aperture time, replacing the first and second flags with third flags.

11. A method according to claim 10 further comprising, for pixel sites having second or third flags, the step of summing, for each pixel site, the respective pixel values of the input temporally and intermediate times.

12. A method according to claim 11 further comprising the steps of:

adding, for each of the respective pixel sites having second flags, the respective pixel value sum and an appropriate proportion of the respective pixel value for the following input frame; and

adding, for each of the pixel sites having third flags, the respective pixel value sum and an appropriate proportion of the respective pixel value for the input frame.

13. A method according to claim 12 further comprising, prior to summing, the step of:

multiplying the pixel values of respective input and intermediate times by a weighting factor determined in accordance with a shutter transmission characteristic.

14. A method according to claim 13 wherein the shutter transmission characteristic is non-linear through the predetermined aperture time.

15. A method according to claim 13 further comprising the steps of:

recording, for each pixel site, a composite sum of the weighting factors applied to respective pixel values;

determining, for each pixel site, the fraction of the maximum possible composite sum recorded for that pixel site; and

calculating said appropriate proportion by deducting said fraction from unity.

16. A method according to claim 15 wherein the shutter transmission characteristic is non-linear through the predetermined aperture time.

17. A method according to claim 1 wherein the adjacent pixel values are the pixel values of pixel sites in the input frame which are spatially adjacent in the direction of the velocity.

18. A method according to claim 17 wherein the step of summing comprises summing a number of pixel values, the number being generally proportional to a predicted number of pixel sites moved by the moving area during the predetermined aperture time.

19. An apparatus for processing video frames comprising animated images so as to blur movement of the animated images between successive frames, the apparatus comprising the steps of:

means for comparing a first input frame with the following input frame;

means for identifying moving areas in the first input frame by means of the comparison;

means for calculating the velocity of the movement of the moving areas between the first and following input frame;

means for summing, only for each input pixel site within the identified moving areas of the input frame, the pixel value of said each input pixel site and adjacent pixel values determined according to the calculated velocities and a predetermined aperture time for the input frame; and

means for providing an output frame from at least the summed pixel values and pixel values of areas of the input frame other than said moving areas.