Frame rate converting apparatus, pan/tilt determining apparatus, and video apparatus

• US 7,929,611 B2
• Filed: 03/23/2006
• Issued: 04/19/2011
• Est. Priority Date: 03/25/2005
• Status: Expired due to Fees

• Current Assignee: Xacti Corporation (Office Support KK)
• Original Assignee: Sanyo Electric Company Limited (Panasonic Holdings Corporation)
• Inventors: Maenaka, Akihiro, Tsunekawa, Norikazu, Mori, Yukio
• Primary Examiner(s): Payne; David C
• Assistant Examiner(s): File; Erin M
• Application Number:
  US11/386,682
• Publication Number:
  US 20060215037A1
• Time in Patent Office: 1,853 Days
• Field of Search: 375/229, 375/230
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/6815 by distinguishing pan or tilt from motion
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51

Abstract

A frame rate converting apparatus includes a pan/tilt determining unit which determines whether an image is in a pan/tilt state on the basis of information related to motion of the image detected by a motion detecting unit for each frame of a moving image, a moving distance setting unit which sets 0 as a moving distance of an image with respect to a frame the image of which is not determined as an image in a pan/tilt state by the pan/tilt determining unit, which calculates a moving distance of an image from the information related to the motion of the image detected by the motion detecting unit with respect to a frame the image of which is determined as an image in a pan/tilt state, and which sets the obtained moving distance as a moving distance of the image, and a prediction image generating unit which generates a prediction image necessary for frame rate conversion on the basis of the moving distance of the image set for each frame by the moving distance setting unit.

Claims

1. A frame rate converting apparatus which converts a frame rate of a moving image, comprising:

a motion detecting module which detects information related to motion of an image for each frame of a moving image to be converted;

a pan/tilt determining module which determines whether the image is in a pan/tilt state on the basis of the information related to the motion of the image detected by the motion detecting module for each frame of the moving image to be converted;

a moving distance setting module which sets 0 as a moving distance of an image with respect to a frame the image of which is not determined as an image in a pan/tilt state by the pan/tilt determining module, which calculates a moving distance of an image from the information related to the motion of the image detected by the motion detecting module with respect to a frame the image of which is determined as an image in a pan/tilt state, and which sets the obtained moving distance as a moving distance of the image; and

a prediction image generating module which generates a prediction image necessary for frame rate conversion on the basis of the moving distance of the image set for each frame by the moving distance setting module.

3. The frame rate converting apparatus according to claim 1, wherein the motion detecting module detects a motion vector, a minimum value of correlative accumulated values, and an average value of the correlative accumulated values as information related to the motion of the image for each of a plurality of motion vector detecting regions set in a video area of each frame by a point matching method, and

the pan/tilt determining module includes

a region specifying module which specifies a region in which the detected motion vector has reliability of the motion vector detecting regions on the basis of the average value of the correlative accumulated values detected for each of the motion vector detecting regions and which calculates the number of detection regions in which the detected motion vectors have a predetermined reliability;

a first determining module which determines whether the number of detection regions in which the detected motion vectors have the predetermined reliability is not less than a first threshold value;

a second determining module which, when the number of detection regions in which the detected moving vectors have the predetermined reliability is less than the first threshold value, determines that the frame is not panned/tilted;

a specifying module which, when the number of detection regions in which the detected motion vectors have the predetermined reliability is not less than the first threshold value, specifies a detection region in which motions of a plurality of types are present and a detection region in which motions of a plurality of types are not present in the detection regions in which the motion vectors have the predetermined reliability on the basis of an average value of correlative accumulated values detected for the detection regions in which the motion vectors have the predetermined reliability and a minimum value of the correlative accumulated values, and which calculates the number of detection regions in which the motions of the plurality of types are present;

a third determining module which determines whether the number of detection regions in which the motions of the plurality of types are present is less than a second threshold value;

a fourth determining module which, when the number of detection regions in which the motions of the plurality of types are present is not less than the second threshold value, determines that the frame is not panned/tilted; and

a fifth determining module which, when the number of detection regions in which the motions of the plurality of types are present is less than the second threshold value, determines that the frame is panned/tilted.

5. The frame rate converting apparatus according to claim 1, wherein the motion detecting module detects a motion vector, a minimum value of correlative accumulated values, and an average value of the correlative accumulated values as information related to the motion of the image for each of a plurality of motion vector detecting regions set in a video area of each frame by a point matching method, and the pan/tilt determining module includes:

a region specifying module which specifies a region in which the detected motion vector has reliability of the motion vector detecting regions on the basis of the average value of the correlative accumulated values detected for each of the motion vector detecting regions and which calculates the number of detection regions in which the detected motion vectors have a predetermined reliability;

a first determining module which determines whether the number of detection regions in which the detected motion vectors have the predetermined reliability is not less than a first threshold value;

a second determining module which, when the number of detection regions in which the detected moving vectors have the predetermined reliability is less than the first threshold value, determines that the frame is not panned/tilted;

a first specifying module which, when the number of detection regions in which the detected motion vectors have the predetermined reliability is not less than the first threshold value, specifies a detection region in which motions of a plurality of types are present and a detection region in which motions of a plurality of types are not present in the detection regions in which the motion vectors have the predetermined reliability on the basis of an average value of correlative accumulated values detected for the detection regions in which the motion vectors have the predetermined reliability and a minimum value of the correlative accumulated values, and which calculates the number of detection regions in which the motions of the plurality of types are present;

a third determining module which determines whether the number of detection regions in which the motions of the plurality of types are present is less than a second threshold value;

a fourth determining module which, when the number of detection regions in which the motions of the plurality of types are present is not less than the second threshold value, determines that the frame is not panned/tilted;

a second specifying module which, when the number of detection regions in which the motions of the plurality of types are present is less than the second threshold value, specifies detection regions in which motion vectors have similarity in the detection regions in which the motion vectors have the predetermined reliability and the motions of the plurality of types are not present on the basis of a motion vector detected for the detection region in which the motion vector has the predetermined reliability and the motions of the plurality of types are not present, and which calculates the number of detection regions in which motion vectors have similarity;

a fifth determining module which determines whether the number of detection regions in which motion vectors have similarity is not less than a third threshold value;

a sixth determining module which, when the number of detection regions which are determined as detection regions in which the motion vectors have similarity is less than the third threshold value, determines that the frame is not panned/tilted; and

a seventh determining module which, when the number of detection regions which are determined as detection regions in which the motion vectors have similarity is not less than the third threshold value, determines that the frame is panned/tilted.

2. A frame rate converting apparatus which converts a frame rate of a moving image, comprising:

a motion detecting module which detects information related to motion of an image for each frame of a moving image to be converted;

a pan/tilt determining module which determines whether the image is in a pan/tilt state on the basis of the information related to the motion of the image detected by the motion detecting module for each frame of the moving image to be converted;

a moving distance setting module which sets 0 as a moving distance of an image with respect to a frame the image of which is not determined as an image in a pan/tilt state by the pan/tilt determining module, which calculates a moving distance of an image from the information related to the motion of the image detected by the motion detecting module with respect to a frame the image of which is determined as an image in a pan/tilt state, and which sets the obtained moving distance as a moving distance of the image;

a time-direction smoothing module which smoothes the moving distance of the image set for each frame by the moving distance setting module in a direction of time; and

a prediction image generating module which generates a prediction image necessary for frame rate conversion on the basis of the moving distance of the image obtained by the time-direction smoothing module.

4. The frame rate converting apparatus according to claim 2, wherein the motion detecting module detects a motion vector, a minimum value of correlative accumulated values, and an average value of the correlative accumulated values as information related to the motion of the image for each of a plurality of motion vector detecting regions set in a video area of each frame by a point matching method, and

the pan/tilt determining module includes

a region specifying module which specifies a region in which the detected motion vector has reliability of the motion vector detecting regions on the basis of the average value of the correlative accumulated values detected for each of the motion vector detecting regions and which calculates the number of detection regions in which the detected motion vectors have a predetermined reliability;

a first determining module which determines whether the number of detection regions in which the detected motion vectors have the predetermined reliability is not less than a first threshold value;

a second determining module which, when the number of detection regions in which the detected moving vectors have the predetermined reliability is less than the first threshold value, determines that the frame is not panned/tilted;

a specifying module which, when the number of detection regions in which the detected motion vectors have the predetermined reliability is not less than the first threshold value, specifies a detection region in which motions of a plurality of types are present and a detection region in which motions of a plurality of types are not present in the detection regions in which the motion vectors have the predetermined reliability on the basis of an average value of correlative accumulated values detected for the detection regions in which the motion vectors have the predetermined reliability and a minimum value of the correlative accumulated values, and which calculates the number of detection regions in which the motions of the plurality of types are present;

a third determining module which determines whether the number of detection regions in which the motions of the plurality of types are present is less than a second threshold value;

a fourth determining module which, when the number of detection regions in which the motions of the plurality of types are present is not less than the second threshold value, determines that the frame is not panned/tilted; and

a fifth determining module which, when the number of detection regions in which the motions of the plurality of types are present is less than the second threshold value, determines that the frame is panned/tilted.

6. The frame rate converting apparatus according to claim 2, wherein the motion detecting module detects a motion vector, a minimum value of correlative accumulated values, and an average value of the correlative accumulated values as information related to the motion of the image for each of a plurality of motion vector detecting regions set in a video area of each frame by a point matching method, and

the pan/tilt determining module includes:

a region specifying module which specifies a region in which the detected motion vector has reliability of the motion vector detecting regions on the basis of the average value of the correlative accumulated values detected for each of the motion vector detecting regions and which calculates the number of detection regions in which the detected motion vectors have a predetermined reliability;

a first determining module which determines whether the number of detection regions in which the detected motion vectors have the predetermined reliability is not less than a first threshold value;

a second determining module which, when the number of detection regions in which the detected moving vectors have the predetermined reliability is less than the first threshold value, determines that the frame is not panned/tilted;

a first specifying module which, when the number of detection regions in which the detected motion vectors have the predetermined reliability is not less than the first threshold value, specifies a detection region in which motions of a plurality of types are present and a detection region in which motions of a plurality of types are not present in the detection regions in which the motion vectors have the predetermined reliability on the basis of an average value of correlative accumulated values detected for the detection regions in which the motion vectors have the predetermined reliability and a minimum value of the correlative accumulated values, and which calculates the number of detection regions in which the motions of the plurality of types are present;

a third determining module which determines whether the number of detection regions in which the motions of the plurality of types are present is less than a second threshold value;

a fourth determining module which, when the number of detection regions in which the motions of the plurality of types are present is not less than the second threshold value, determines that the frame is not panned/tilted;

a second specifying module which, when the number of detection regions in which the motions of the plurality of types are present is less than the second threshold value, specifies detection regions in which motion vectors have similarity in the detection regions in which the motion vectors have the predetermined reliability and the motions of the plurality of types are not present on the basis of a motion vector detected for the detection region in which the motion vector has the predetermined reliability and the motions of the plurality of types are not present, and which calculates the number of detection regions in which motion vectors have similarity;

a fifth determining module which determines whether the number of detection regions in which motion vectors have similarity is not less than a third threshold value;

a sixth determining module which, when the number of detection regions which are determined as detection regions in which the motion vectors have similarity is less than the third threshold value, determines that the frame is not panned/tilted; and

a seventh determining module which, when the number of detection regions which are determined as detection regions in which the motion vectors have similarity is not less than the third threshold value, determines that the frame is panned/tilted.

7. A video apparatus having a frame rate converting apparatus which converts a frame rate of a moving image, wherein the frame rate converting apparatus includes:

a motion detecting module which detects information related to motion of an image for each frame of a moving image to be converted;

a pan/tilt determining module which determines whether the image is in a pan/tilt state on the basis of the information related to the motion of the image detected by the motion detecting module for each frame of the moving image to be converted;

a moving distance setting module which sets 0 as a moving distance of an image with respect to a frame the image of which is not determined as an image in a pan/tilt state by the pan/tilt determining module, which calculates a moving distance of an image from the information related to the motion of the image detected by the motion detecting module with respect to a frame the image of which is determined as an image in a pan/tilt state, and which sets the obtained moving distance as a moving distance of the image; and

a prediction image generating module which generates a prediction image necessary for frame rate conversion on the basis of the moving distance of the image set for each frame by the moving distance setting module.

8. A video apparatus having a frame rate converting apparatus which converts a frame rate of a moving image, wherein

the frame rate converting apparatus includes:

a motion detecting module which detects information related to motion of an image for each frame of a moving image to be converted;

a pan/tilt determining module which determines whether the image is in a pan/tilt state on the basis of the information related to the motion of the image detected by the motion detecting module for each frame of the moving image to be converted;

a moving distance setting module which sets 0 as a moving distance of an image with respect to a frame the image of which is not determined as an image in a pan/tilt state by the pan/tilt determining module, which calculates a moving distance of an image from the information related to the motion of the image detected by the motion detecting module with respect to a frame the image of which is determined as an image in a pan/tilt state, and which sets the obtained moving distance as a moving distance of the image;

a time-direction smoothing module which smooths the moving distance of the image set for each frame by the moving distance setting module in a direction of time; and

a prediction image generating module which generates a prediction image necessary for frame rate conversion on the basis of the moving distance of the image obtained by the time-direction smoothing module.

9. A pan/tilt determining apparatus which determines whether an image is in a pan/tilt state for each frame of a moving image, comprising:

a calculating module which calculates a motion vector, a minimum value of correlative accumulated values, and an average value of the correlative accumulated values for each of a plurality of motion vector detecting regions set in a video area of each frame of a moving image by a point matching method;

a first specifying module which specifies a region in which the detected motion vector has a predetermined reliability of the motion vector detecting regions on the basis of the average value of the correlative accumulated values detected for each of the motion vector detecting regions and which calculates the number of detection regions in which the detected motion vectors have the predetermined reliability;

a first determining module which determines whether the number of detection regions in which the detected motion vectors have the predetermined reliability is not less than a first threshold value;

a second determining module which, when the number of detection regions in which the detected moving vectors have the predetermined reliability is less than the first threshold value, determines that the frame is not panned/tilted;

a second specifying module which, when the number of detection regions in which the detected motion vectors have the predetermined reliability is not less than the first threshold value, specifies a detection region in which motions of a plurality of types are present and a detection region in which motions of a plurality of types are not present in the detection regions in which the motion vectors have the predetermined reliability on the basis of an average value of correlative accumulated values detected for the detection regions in which the motion vectors have the predetermined reliability and a minimum value of the correlative accumulated values, and which calculates the number of detection regions in which the motions of the plurality of types are present;

a third determining module which determines whether the number of detection regions in which the motions of the plurality of types are present is less than a second threshold value;

a fourth determining module which, when the number of detection regions in which the motions of the plurality of types are present is not less than the second threshold value, determines that the frame is not panned/tilted; and

a fifth determining module which, when the number of detection regions in which the motions of the plurality of types are present is less than the second threshold value, determines that the frame is panned/tilted.

10. A pan/tilt determining apparatus which determines whether an image is in a pan/tilt state for each frame of a moving image, comprising:

a calculating module which calculates a motion vector, a minimum value of correlative accumulated values, and an average value of the correlative accumulated values for each of a plurality of motion vector detecting regions set in a video area of each frame of a moving image by a point matching method;

a first specifying module which specifies a region in which the detected motion vector has a predetermined reliability of the motion vector detecting regions on the basis of the average value of the correlative accumulated values detected for each of the motion vector detecting regions and which calculates the number of detection regions in which the detected motion vectors have the predetermined reliability;

a first determining module which determines whether the number of detection regions in which the detected motion vectors have the predetermined reliability is not less than a first threshold value;

a second determining module which, when the number of detection regions in which the detected moving vectors have the predetermined reliability is less than the first threshold value, determines that the frame is not panned/tilted;

a second specifying module which, when the number of detection regions in which the detected motion vectors have the predetermined reliability is not less than the first threshold value, specifies a detection region in which motions of a plurality of types are present and a detection region in which motions of a plurality of types are not present in the detection regions in which the motion vectors have the predetermined reliability on the basis of an average value of correlative accumulated values detected for the detection regions in which the motion vectors have the predetermined reliability and a minimum value of the correlative accumulated values, and which calculates the number of detection regions in which the motions of the plurality of types are present;

a third determining module which determines whether the number of detection regions in which the motions of the plurality of types are present is less than a second threshold value;

a fourth determining module which, when the number of detection regions in which the motions of the plurality of types are present is not less than the second threshold value, determines that the frame is not panned/tilted;

a third specifying module which, when the number of detection regions in which the motions of the plurality of types are present is less than the second threshold value, specifies detection regions in which motion vectors have similarity in the detection regions in which the motion vectors have the predetermined reliability and the motions of the plurality of types are not present on the basis of a motion vector detected for the detection region in which the motion vector has the predetermined reliability and the motions of the plurality of types are not present, and which calculates the number of detection regions in which motion vectors have similarity;

a fifth determining module which determines whether the number of detection regions in which motion vectors have similarity is not less than a third threshold value;

a sixth determining module which, when the number of detection regions which are determined as detection regions in which the motion vectors have similarity is less than the third threshold value, determines that the frame is not panned/tilted; and

a seventh determining module which, when the number of detection regions which are determined as detection regions in which the motion vectors have similarity is not less than the third threshold value, determines that the frame is panned/tilted.

11. A video apparatus having a pan/tilt determining apparatus which determines whether an image is in a pan/tilt state for each frame of a moving image, wherein

the pan/tilt determining apparatus includes:

a calculating module which calculates a motion vector, a minimum value of correlative accumulated values, and an average value of the correlative accumulated values for each of a plurality of motion vector detecting regions set in a video area of each frame of a moving image by a point matching method;

a region specifying module which specifies a region in which the detected motion vector has a predetermined reliability of the motion vector detecting regions on the basis of the average value of the correlative accumulated values detected for each of the motion vector detecting regions and which calculates the number of detection regions in which the detected motion vectors have the predetermined reliability;

a first determining module which determines whether the number of detection regions in which the detected motion vectors have the predetermined reliability is not less than a first threshold value;

a second determining module which, when the number of detection regions in which the detected moving vectors have the predetermined reliability is less than the first threshold value, determines that the frame is not panned/tilted;

a specifying module which, when the number of detection regions in which the detected motion vectors have the predetermined reliability is not less than the first threshold value, specifies a detection region in which motions of a plurality of types are present and a detection region in which motions of a plurality of types are not present in the detection regions in which the motion vectors have the predetermined reliability on the basis of an average value of correlative accumulated values detected for the detection regions in which the motion vectors have the predetermined reliability and a minimum value of the correlative accumulated values, and which calculates the number of detection regions in which the motions of the plurality of types are present;

a third determining module which determines whether the number of detection regions in which the motions of the plurality of types are present is less than a second threshold value;

a fourth determining module which, when the number of detection regions in which the motions of the plurality of types are present is not less than the second threshold value, determines that the frame is not panned/tilted; and

a fifth determining module which, when the number of detection regions in which the motions of the plurality of types are present is less than the second threshold value, determines that the frame is panned/tilted.

12. A video apparatus having a pan/tilt determining apparatus which determines whether an image is in a pan/tilt state for each frame of a moving image, wherein

the pan/tilt determining apparatus includes:

a calculating module which calculates a motion vector, a minimum value of correlative accumulated values, and an average value of the correlative accumulated values for each of a plurality of motion vector detecting regions set in a video area of each frame of a moving image by a point matching method;

a first specifying module which specifies a region in which the detected motion vector has a predetermined reliability of the motion vector detecting regions on the basis of the average value of the correlative accumulated values detected for each of the motion vector detecting regions and which calculates the number of detection regions in which the detected motion vectors have the predetermined reliability;

a first determining module which determines whether the number of detection regions in which the detected motion vectors have the predetermined reliability is not less than a first threshold value;

a second determining module which, when the number of detection regions in which the detected moving vectors have the predetermined reliability is less than the first threshold value, determines that the frame is not panned/tilted;

a second specifying module which, when the number of detection regions in which the detected motion vectors have the predetermined reliability is not less than the first threshold value, specifies a detection region in which motions of a plurality of types are present and a detection region in which motions of a plurality of types are not present in the detection regions in which the motion vectors have the predetermined reliability on the basis of an average value of correlative accumulated values detected for the detection regions in which the motion vectors have the predetermined reliability and a minimum value of the correlative accumulated values, and which calculates the number of detection regions in which the motions of the plurality of types are present;

a third determining module which determines whether the number of detection regions in which the motions of the plurality of types are present is less than a second threshold value;

a fourth determining module which, when the number of detection regions in which the motions of the plurality of types are present is not less than the second threshold value, determines that the frame is not panned/tilted;

a third specifying module which, when the number of detection regions in which the motions of the plurality of types are present is less than the second threshold value, specifies detection regions in which motion vectors have similarity in the detection regions in which the motion vectors have the predetermined reliability and the motions of the plurality of types are not present on the basis of a motion vector detected for the detection region in which the motion vector has the predetermined reliability and the motions of the plurality of types are not present, and which calculates the number of detection regions in which motion vectors have similarity;

a fifth determining module which determines whether the number of detection regions in which motion vectors have similarity is not less than a third threshold value;

a sixth determining module which, when the number of detection regions which are determined as detection regions in which the motion vectors have similarity is less than the third threshold value, determines that the frame is not panned/tilted; and

a seventh determining module which, when the number of detection regions which are determined as detection regions in which the motion vectors have similarity is not less than the third threshold value, determines that the frame is panned/tilted.

13. A frame rate converting apparatus which converts a frame rate of a moving image, comprising:

a defining module which defines one of two continuous frames in a moving image as a current frame, defines the other as a previous frame, and defines a frame generated between the current frame and the previous frame as a prediction frame, which calculates a shift distance of the prediction image with respect to a previous frame image and a shift distance of the prediction image with respect to a current frame image, on the basis of a moving distance of an image between a previous frame image and a current frame image and a frame rate;

a first shifting module which shifts the previous frame image by using the shift distance of the prediction image with respect to the previous frame image to generate a first virtual prediction image constituted by a first blank portion in which the previous frame image is not present and a first original image portion in which the previous frame image is present;

a second shifting module which shifts the current frame image by using the shift distance of the prediction image with respect to the current frame image to generate a second virtual prediction image constituted by a second blank portion in which the current frame image is not present and a second original image portion in which the current frame image is present; and

a prediction image generating module which generates the prediction image on the basis of the previous frame image, the current frame image, the first virtual prediction image, and the second virtual prediction image, wherein the prediction image generating module, if the first virtual prediction image and the second virtual prediction image are overlapped, generates a prediction image from the second virtual prediction image in a portion where the first blank portion in the first virtual prediction image is overlapped on the second original image portion in the second virtual prediction image, generates a prediction image from the first virtual prediction image in a portion where the second blank portion in the second virtual prediction image is overlapped on the first original image portion in the first virtual prediction image, generates a prediction image by an image obtained by weighted-summing the current frame image and the previous frame image in a portion where the first blank portion and the second blank portion are overlapped, and generates a prediction image from one of the first virtual prediction image and the second virtual prediction image in a portion where the first original image portion in the first virtual prediction image is overlapped on the second original image portion in the second virtual prediction image.

14. A video apparatus having a frame rate converting apparatus which converts a frame rate of a moving image, wherein

the frame rate converting apparatus includes:

a defining module which defines one of two continuous frames in a moving image as a current frame, defines the other as a previous frame, and defines a frame generated between the current frame and the previous frames as a prediction frame, which calculates a shift distance of the prediction image with respect to a previous frame image and a shift distance of the prediction image with respect to a current frame image, on the basis of a moving distance of an image between a previous frame image and a current frame image and a frame rate;

a first shifting module which shifts the previous frame image by using the shift distance of the prediction image with respect to the previous frame image to generate a first virtual prediction image constituted by a first blank portion in which the previous frame image is not present and a first original image portion in which the previous frame image is present;

a second shifting module which shifts the current frame image by using the shift distance of the prediction image with respect to the current frame image to generate a second virtual prediction image constituted by a second blank portion in which the current frame image is not present and a second original image portion in which the current frame image is present; and

a prediction image generating module which generates the prediction image on the basis of the previous frame image, the current frame image, the first virtual prediction image, and the second virtual prediction image,

wherein the prediction image generating module, if the first virtual prediction image and the second virtual prediction image are overlapped, generates a prediction image from the second virtual prediction image in a portion where the first blank portion in the first virtual prediction image is overlapped on the second original image portion in the second virtual prediction image, generates a prediction image from the first virtual prediction image in a portion where the second blank portion in the second virtual prediction image is overlapped on the first original image portion in the first virtual prediction image, generates a prediction image by an image obtained by weighted-summing the current frame image and the previous frame image in a portion where the first blank portion and the second blank portion are overlapped, and generates a prediction image from one of the first virtual prediction image and the second virtual prediction image in a portion where the first original image portion in the first virtual prediction image is overlapped on the second original image portion in the second virtual prediction image.