Corrected image generating apparatus and corrected image generating program storage medium

• US 7,333,132 B2
• Filed: 02/21/2002
• Issued: 02/19/2008
• Est. Priority Date: 10/26/2001
• Status: Expired due to Fees

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Shimizu, Katsutoshi, Shimomae, Hiroki
• Primary Examiner(s): Ho; Tuan
• Application Number:
  US10/078,433
• Publication Number:
  US 20040201706A1
• Time in Patent Office: 2,189 Days
• Field of Search: 348/208.1, 348/699, 348/219.1, 382/284
• US Class Current: 348/208.1
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

This invention provides a corrected image generating apparatus and the like for generating a corrected image by correcting a deflection due to the vibration of a camera in a raw image obtained by the camera which obtains the raw images of plural frames successively by scanning an object field of vision repeatedly in a scanning direction intersecting the arrangement direction of plural light receiving sensors composing the one-dimensional image pickup device in which the plural light receiving sensors are arranged one-dimensionally, in order to generate an image whose deflection is corrected. By obtaining a motion vector of each of short strips extending in the arrangement direction of the light receiving sensors of the one-dimensional image pickup device, the respective short strips are moved to obtain a corrected image.

Claims

1. A corrected image generating apparatus for generating a corrected image by correcting a deflection due to vibration of a camera in a raw image obtained by said camera which obtains raw images of a plurality of frames successively by scanning an object field of vision repeatedly, said corrected image generating apparatus comprising:

a one-dimensional image pickup device to scan a raw image via a plurality of light receiving sensors arranged one-dimensionally in a scanning direction intersecting an arrangement direction of the plurality of light receiving sensors including the one-dimensional image pickup device;

a pattern matching portion to divide the raw image into grids and to perform pattern matching processing about adjacent frames in blocks of the grids, the blocks of the grids being of a same size, to thereby provide pattern matching data;

a feature point extracting portion to extract a block as a feature point on the pattern matching of the raw image, after the pattern matching processing has been performed and in accordance with the pattern matching data obtained by said pattern matching portion;

a motion vector computing portion to obtain a motion vector of said feature point based on the pattern matching data obtained by said pattern matching portion about the feature point extracted by said feature point extracting portion; and

an image correcting portion to obtain a corrected image in which deflection of the raw image is corrected based on the motion vector obtained by said motion vector computing portion about the feature point extracted by said feature point extracting portion.

2. A corrected image generating apparatus according to claim 1, further comprising an image culling portion to generate a culling image by culling pixels of the raw image, wherein

said feature point extracting portion controls said pattern matching portion to carry out the pattern matching processing about a culling image created by said image culling portion so as to extract a feature point based on the result of the pattern matching processing about the culling image;

and

said motion vector computing portion controls said pattern matching portion to carry out the pattern matching processing about a block of the feature point extracted by said feature point extracting portion in a raw image prior to culling so as to obtain a motion vector of the feature point based on the result of the pattern matching processing about the raw image.

3. A corrected image generating apparatus according to claim 1, wherein

said motion vector computing portion obtains a motion vector of each short strip of a plurality of short strips based on the motion vector of the feature point in the short strip including a plurality of blocks arranged in said arrangement direction of the plurality of blocks produced by dividing the raw image into grids;

and

said image correcting portion obtains a corrected image by correcting the relation of positions of the short strips based on the motion vector of each short strip obtained by said motion vector computing portion.

4. A corrected image generating apparatus according to claim 3, wherein said motion vector computing portion obtains a most frequent value vector in a motion vector of the feature point in said short strip as a motion vector of said short strip.

5. A corrected image generating apparatus according to claim 1, wherein

said motion vector computing portion obtains a new motion vector originated from vibration of said camera by carrying out processing including filtering processing aiming at the plurality of frames upon an obtained motion vector;

and

said image correcting portion obtains said corrected image based on said new motion vector obtained by said motion vector computing portion.

6. A corrected image generating apparatus according to claim 3, wherein

said motion vector computing portion obtains a new motion vector originated from vibration of said camera by carrying out processing including filtering processing aiming at the plurality of frames upon an obtained motion vector;

and

said image correcting portion obtains said corrected image based on said new motion vector obtained by said motion vector computing portion.

7. A corrected image generating apparatus according to claim 3, wherein said image correcting portion obtains a corrected image in which deflection between adjacent short strips in said arrangement direction is eliminated by shifting pixels arranged in said scanning direction in a short strip in the said arrangement direction.

8. A corrected image generating apparatus according to claim 1, further comprising an on-zooming discriminating portion to discriminate whether zooming is being carried out based on a motion vector obtained by said motion vector computing portion,

wherein when said on-zooming discriminating portion discriminates that zooming is being carried out, said image correcting portion interrupts generation of the corrected image.

9. A computer readable storage medium comprising a corrected image generating program which is executed in a computer for actuating said computer as a corrected image generating apparatus for generating a corrected image by correcting a deflection due to vibration of a camera in a raw image obtained by said camera which obtains the raw images of a plurality of frames successively by scanning an object field of vision repeatedly, said corrected image generating program comprising:

a one-dimensional image pickup portion to scan the raw image repeatedly using a one-dimensional image pickup device in which a plurality of light receiving sensors are arranged one-dimensionally in a scanning direction intersecting an arrangement direction of the plurality of light receiving sensors including the one-dimensional image pickup device;

a pattern matching portion to divide the raw image into grids and to perform pattern matching processing about adjacent frames in blocks of grids, the blocks of the grids being of a same size, to thereby obtain pattern matching data;

a feature point extracting portion to extract a block as a feature point on the pattern matching of the raw image after the pattern matching processing has been performed and in accordance with the pattern matching data obtained by said pattern matching portion;

a motion vector computing portion to obtain the motion vector of said feature point based on the pattern matching data obtained by said pattern matching portion about the feature point extracted by said feature point extracting portion; and

an image correcting portion to obtain a corrected image in which deflection of the raw image is corrected based on the motion vector obtained by said motion vector computing portion about the feature point extracted by said feature point extracting portion.

10. A corrected image generating program storage medium according to claim 9, wherein the corrected image generating program further comprising an image culling portion for generating a culling image by culling pixels of the raw image, wherein

said feature point extracting portion controls said pattern matching portion to carry out the pattern matching processing about a culling image created by said image culling portion so as to extract a feature point based on the result of the pattern matching processing about the culling image;

and

said motion vector computing portion controls said pattern matching portion to carry out the pattern matching processing about a block of the feature point extracted by said feature point extracting portion in a raw image prior to culling so as to obtain a motion vector of the feature point based on the result of the pattern matching processing about the raw image.

11. A corrected image generating program storage medium according to claim 9, wherein said motion vector computing portion obtains a motion vector of each short strip of a plurality of short strips based on the motion vector of the feature point in the short strip composed of plural blocks arranged in said arrangement direction of plural blocks produced by dividing the raw image into grids and

said image correcting portion obtains a corrected image by correcting the relation of the positions of the short strips based on the motion vector of each short strip obtained by said motion vector computing portion.

12. A corrected image generating program storage medium according to claim 11, wherein said motion vector computing portion obtains a most frequent value vector in a motion vector of the feature point in said short strip as a motion vector of said short strip.

13. A corrected image generating program storage medium according to claim 9, wherein

said motion vector computing portion obtains a new motion vector originated from vibration of said camera by carrying out processing including the filtering processing aiming at the plurality of frames upon an obtained motion vector and

said image correcting portion obtains said corrected image based on said new motion vector obtained by said motion vector computing portion.

14. A corrected image generating program storage medium according to claim 11, wherein

said motion vector computing portion obtains a new motion vector originated from vibration of said camera by carrying out processing including the filtering processing aiming at plural frames upon an obtained motion vector and

said image correcting portion obtains said corrected image based on said new motion vector obtained by said motion vector computing portion.

15. A corrected image generating program storage medium according to claim 11, wherein said image correcting portion obtains a corrected image in which deflection between adjacent short strips in said arrangement direction is eliminated by shifting pixels arranged in said scanning direction in a short strip in said arrangement direction.

16. A corrected image generating program storage medium according to claim 9, wherein the corrected image generating program comprising an on-zooming discriminating portion to discriminate whether zooming is being carried out based on a motion vector obtained by said motion vector computing portion, wherein when said on-zooming discriminating portion discriminates that zooming is being carried out, said image correcting portion interrupts generation of the corrected image.

17. An apparatus having a camera, for generating a corrected image based upon a raw image obtained by the camera, the apparatus comprising:

a one-dimensional image pickup device to receive the raw image via a piurality of light receiving sensors arranged one-dimensionally in a scanning direction intersecting an arrangement direction of the plurality of light receiving sensors including the one-dimensional image pickup device;

a pattern matching portion to divide the raw image into grids and to perform pattern matching of adjacent frames of blocks within the grids, the blocks of the grids being of a same size, to thereby provide pattern matched data;

a feature point extracting portion to extract a block as a feature point of the pattern matched data after the pattern matching has been performed;

a motion vector computing portion to obtain a motion vector of the extracted feature point; and

a correcting portion to obtain the corrected image in which deflection of the raw image is corrected based on the motion vector obtained and the extracted feature point.