Corrected image generating apparatus and corrected image generating program storage medium

• US 20040201706A1
• Filed: 02/21/2002
• Published: 10/14/2004
• Est. Priority Date: 10/26/2001
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Shimizu, Katsutoshi, Shimomae, Hiroki
• Granted Patent: US 7,333,132 B2
• Time in Patent Office: Days
• US Class Current: 348/208.4
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

This invention provides a corrected image generating apparatus and the like for generating a corrected image by correcting a deflection due to the vibration of a camera in a raw image obtained by the camera which obtains the raw images of plural frames successively by scanning an object field of vision repeatedly in a scanning direction intersecting the arrangement direction of plural light receiving sensors composing the one-dimensional image pickup device in which the plural light receiving sensors are arranged one-dimensionally, in order to generate an image whose deflection is corrected. By obtaining a motion vector of each of short strips extending in the arrangement direction of the light receiving sensors of the one-dimensional image pickup device, the respective short strips are moved to obtain a corrected image.

Claims

1. A corrected image generating apparatus for generating a corrected image by correcting a deflection due to vibration of a camera in a raw image obtained by said camera which obtains the raw images of plural frames successively by scanning an object field of vision repeatedly using a one-dimensional image pickup device in which the plural light receiving sensors are arranged one-dimensionally in a scanning direction intersecting the arrangement direction of plural light receiving sensors composing the one-dimensional image pickup device, said corrected image generating apparatus comprising:

a pattern matching portion for carrying out pattern matching processing about adjacent frames in each block obtained by dividing the raw image into grids;

a feature point extracting portion for extracting the feature point on the pattern matching of the raw image according to a result of the pattern matching processing obtained by said pattern matching portion;

a motion vector computing portion for obtaining the motion vector of said feature point based on a result of the pattern matching processing obtained by said pattern matching portion about the feature point extracted by said feature point extracting portion; and

an image correcting portion for obtaining a corrected image in which deflection of the raw image is corrected based on the motion vector obtained by said motion vector computing portion about the feature point extracted by said feature point extracting portion.

2. A corrected image generating apparatus according to claim 1, further comprising an image culling portion for generating a culling image by culling pixels of the raw image, wherein

said feature point extracting portion makes said pattern matching portion carry out the pattern matching processing about a culling image created by said image culling portion so as to extract a feature point based on the result of the pattern matching processing about the culling image and

said motion vector computing portion makes said pattern matching portion carry out the pattern matching processing about a block of the feature point extracted by said feature point extracting portion in a raw image prior to culling so as to obtain a motion vector of the feature point based on the result of the pattern matching processing about the raw image.

3. A corrected image generating apparatus according to claim 1, wherein

said motion vector computing portion obtains a motion vector of each short strip based on the motion vector of the feature point in the short strip composed of plural blocks arranged in said arrangement direction of plural blocks produced by dividing the raw image into grids and

said image correcting portion obtains a corrected image by correcting the relation of the positions of the short strips based on the motion vector of each short strip obtained by said motion vector computing portion.

4. A corrected image generating apparatus according to claim 3, wherein said motion vector computing portion obtains the most frequent value vector in a motion vector of the feature point in said short strip as a motion vector of said short strip.

5. A corrected image generating apparatus according to claim 1, wherein

said motion vector computing portion obtains a new motion vector originated from vibration of said camera by carrying out processing including filtering processing aiming at plural frames upon a just obtained motion vector and

said image correcting portion obtains said corrected image based on said new motion vector obtained by said motion vector computing portion.

6. A corrected image generating apparatus according to claim 3, wherein

said motion vector computing portion obtains a new motion vector originated from vibration of said camera by carrying out processing including filtering processing aiming at plural frames upon a just obtained motion vector and

said image correcting portion obtains said corrected image based on said new motion vector obtained by said motion vector computing portion.

7. A corrected image generating apparatus according to claim 3, wherein said image correcting portion obtains a corrected image in which deflection between adjacent short strips in said arrangement direction is eliminated by shifting pixels arranged in said scanning direction in a short strip in the said arrangement direction.

8. A corrected image generating apparatus according to claim 1, further comprising an on-zooming discriminating portion for discriminating whether or not zooming is being carried out based on a motion vector obtained by said motion vector computing portion,

wherein when said on-zooming discriminating portion discriminates that zooming is being carried out, said image correcting portion interrupts generation of the corrected image.

9. A corrected image generating program storage medium storing a corrected image generating program which is executed in a computer and employed for actuating said computer as a corrected image generating apparatus for generating a corrected image by correcting a deflection due to vibration of a camera in a raw image obtained by said camera which obtains the raw images of plural frames successively by scanning an object field of vision repeatedly using a one-dimensional image pickup device in which the plural light receiving sensors are arranged one-dimensionally in a scanning direction intersecting the arrangement direction of plural light receiving sensors composing the one-dimensional image pickup device, said corrected image generating program comprising:

a pattern matching portion for carrying out pattern matching processing about adjacent frames in each block obtained by dividing the raw image into grids;

a feature point extracting portion for extracting the feature point on the pattern matching of the raw image according to a result of the pattern matching processing obtained by said pattern matching portion;

a motion vector computing portion for obtaining the motion vector of said feature point based on a result of the pattern matching processing obtained by said pattern matching portion about the feature point extracted by said feature point extracting portion; and

an image correcting portion for obtaining a corrected image in which deflection of the raw image is corrected based on the motion vector obtained by said motion vector computing portion about the feature point extracted by said feature point extracting portion.

10. A corrected image generating program storage medium according to claim 9, wherein the corrected image generating program further comprising an image culling portion for generating a culling image by culling pixels of the raw image, wherein

said feature point extracting portion makes said pattern matching portion carry out the pattern matching processing about a culling image created by said image culling portion so as to extract a feature point based on the result of the pattern matching processing about the culling image and

said motion vector computing portion makes said pattern matching portion carry out the pattern matching processing about a block of the feature point extracted by said feature point extracting portion in a raw image prior to culling so as to obtain a motion vector of the feature point based on the result of the pattern matching processing about the raw image.

11. A corrected image generating program storage medium according to claim 9, wherein said motion vector computing portion obtains a motion vector of each short strip based on the motion vector of the feature point in the short strip composed of plural blocks arranged in said arrangement direction of plural blocks produced by dividing the raw image into grids and

said image correcting portion obtains a corrected image by correcting the relation of the positions of the short strips based on the motion vector of each short strip obtained by said motion vector computing portion.

12. A corrected image generating program storage medium according to claim 11, wherein said motion vector computing portion obtains the most frequent value vector in a motion vector of the feature point in said short strip as a motion vector of said short strip.

13. A corrected image generating program storage medium according to claim 9, wherein

said motion vector computing portion obtains a new motion vector originated from vibration of said camera by carrying out processing including the filtering processing aiming at plural frames upon a just obtained motion vector and

said image correcting portion obtains said corrected image based on said new motion vector obtained by said motion vector computing portion.

14. A corrected image generating program storage medium according to claim 11, wherein

said motion vector computing portion obtains a new motion vector originated from vibration of said camera by carrying out processing including the filtering processing aiming at plural frames upon a just obtained motion vector and

said image correcting portion obtains said corrected image based on said new motion vector obtained by said motion vector computing portion.

15. A corrected image generating program storage medium according to claim 11, wherein said image correcting portion obtains a corrected image in which deflection between adjacent short strips in said arrangement direction is eliminated by shifting pixels arranged in said scanning direction in a short strip in said arrangement direction.

16. A corrected image generating program storage medium according to claim 9, wherein the corrected image generating program comprising an on-zooming discriminating portion for discriminating whether or not zooming is being carried out based on a motion vector obtained by said motion vector computing portion, wherein when said on-zooming discriminating portion discriminates that zooming is being carried out, said image correcting portion interrupts generation of the corrected image.