Image processing device and method, recording medium, and program

• US 20060072018A1
• Filed: 01/21/2004
• Published: 04/06/2006
• Est. Priority Date: 01/22/2003
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Image Processing Apparatus and Method Recording Medium and Program
• Inventors: Kinoshita, Masaya, Tanaka, Kenji, Inaba, Seijiro
• Granted Patent: US 7,750,942 B2
• Time in Patent Office: Days
• US Class Current: 348/208.990
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/689 Motion occurring during a rolling shutter mode

Abstract

The present invention relates to an image processing apparatus and method enabling to correct for camera shake given in image capturing, a recording medium, and a program. Discrete data pieces indicating shifts are obtained. The discrete data pieces are interpolated to obtain data indicating a shift every line. By the interpolation, a correction is calculated every line on the basis of the corresponding shift. On the basis of the corrections, pixel data read positions are determined. Pixel data is read based on the determined read positions. When a pixel to be read overlaps two pixels, pixel data of the pixel to be read is produced (interpolated) from pixel data pieces. In this manner, the effects of camera shake are corrected. The present invention is applicable to video cameras.

Claims

1. An image processing apparatus comprising:

imaging means for capturing an image through an X-Y addressing solid-state imaging device;

output means for detecting given vibration to generate vibration detection information;

converting means for converting the vibration detection information generated from the output means into shifts indicating the magnitudes of the given vibration;

calculating means for calculating corrections from the shifts obtained by the converting means, the corrections being used to reduce the effects of the given vibration on the image;

interpolating means for interpolating the shifts or the corrections; and

correcting means for correcting the image captured by the imaging means on the basis of the corrections to output the corrected image,

wherein

the converting means converts the vibration detection information into a first shift and a second shift, the first shift being obtained when a first line is imaged, the second shift being obtained when a second line is imaged,

the interpolating means interpolates the first and second shifts to obtain a shift of each line between the first and second lines, and

the calculating means calculates a correction every line on the basis of the corresponding shift obtained by the interpolating means,

or wherein

the converting means converts the vibration detection information into a first shift and a second shift, the first shift being obtained when a first line is imaged, the second shift being obtained when a second line is imaged,

the calculating means calculates a first correction and a second correction from the first shift and the second shift, respectively, and

the interpolating means interpolates the first and second corrections to obtain a correction of each line between the first and second lines.

2. The image processing apparatus according to claim 1, wherein on the basis of the corrections, the correcting means generates corrected pixels using pixel data of pixels in the image captured by the imaging means, the image corresponding to pixels located in corrected positions.

3. The image processing apparatus according to claim 1, wherein the converting means generates each shift on the basis of the middle information piece and the previous and subsequent information pieces of the vibration detection information pieces generated from the output means for a predetermined time.

4. The image processing apparatus according to claim 1, wherein the converting means is capable of predicting future vibration information based on past vibration information.

5. The image processing apparatus according to claim 1, wherein while one image is captured, the calculating means calculates a shift once or more times, or calculates a correction once or more times.

6. The image processing apparatus according to claim 1, wherein each of the shift obtained by the converting means and the correction obtained by the calculating means includes a value corresponding to one pixel or smaller.

7. The image processing apparatus according to claim 1, wherein the correcting means interpolates pixels with an accuracy of one pixel or smaller.

8. The image processing apparatus according to claim 1, further comprising:

storage means for storing only data of the image captured by the imaging means, the data being needed to correct the image in the vertical direction through the correcting means.

9. The image processing apparatus according to claim 1, wherein an area needed to correct shifts of the image is stored in said storage means.

10. An image processing method for an image processing apparatus for correcting an image captured through an X-Y addressing solid-state imaging device to reduce the effects of given vibration on the image, the method comprising:

an imaging control step of controlling the imaging device to capture an image;

a detection step of detecting given vibration;

a conversion step of converting vibration detection information regarding the vibration detected in the detection step into shifts indicating the magnitudes of the given vibration;

a calculation step of calculating corrections to reduce the effects of the given vibration on the image on the basis of the shifts obtained in the conversion step;

an interpolation step of interpolating the shifts or the corrections; and

a correction step of correcting the image, captured under control in the imaging control step, on the basis of the corrections to generate the corrected image,

wherein

in the conversion step, the vibration detection information is converted into a first shift and a second shift, the first shift being obtained when a first line is imaged, the second shift being obtained when a second line is imaged,

in the interpolation step, the first and second shifts are interpolated to obtain a shift of each line between the first and second lines, and

in the calculation step, a correction is calculated every line on the basis of the corresponding shift obtained in the interpolation step,

or wherein

in the conversion step, the vibration detection information is converted into a first shift and a second shift, the first shift being obtained when a first line is imaged, the second shift being obtained when a second line is imaged,

in the calculation step, a first correction and a second correction are calculated from the first shift and the second shift, respectively, and

in the interpolation step, the first and second corrections are interpolated to obtain a correction of each line between the first and second lines.

11. A recording medium storing a computer-readable program for an image processing apparatus which corrects an image captured through an X-Y addressing solid-state imaging device to reduce the effects of given vibration on the image, the program comprising:

an imaging control step of controlling the imaging device to capture an image;

a detection step of detecting given vibration;

a conversion step of converting vibration detection information regarding the vibration detected in the detection step into shifts indicating the magnitudes of the given vibration;

a calculation step of calculating corrections in units of pixels on the basis of the shifts obtained in the conversion step, the corrections being used to reduce the effects of the given vibration on the image;

an interpolation step of interpolating the shifts or the corrections; and

a correction step of correcting the image, captured under control in the imaging control step, on the basis of the corrections to generate the corrected image,

wherein

in the conversion step, the vibration detection information is converted into a first shift and a second shift, the first shift being obtained when a first line is imaged, the second shift being obtained when a second line is imaged,

in the interpolation step, the first and second lines are interpolated to obtain a shift of each line between the first and second lines, and

in the calculation step, a correction is calculated every line on the basis of the corresponding shift obtained in the interpolation step,

or wherein

in the conversion step, the vibration detection information is converted into a first shift and a second shift, the first shift being obtained when a first line is imaged, the second shift being obtained when a second line is imaged,

in the calculation step, a first correction and a second correction are calculated from the first shift and the second shift, respectively, and

in the interpolation step, the first and second corrections are interpolated to obtain a correction of each line between the first and second lines.

12. A program which is executed on a computer to control an image processing apparatus for correcting an image captured through an imaging device in order to reduce the effects of given vibration on the image, the program comprising:

an imaging control step of controlling the imaging device to capture an image, the imaging device capturing the image every line;

a detection step of detecting given vibration;

a conversion step of converting vibration detection information regarding the vibration detected in the detection step into shifts indicating the magnitudes of the given vibration;

a calculation step of calculating corrections in units of pixels on the basis of the shifts obtained in the conversion step, the corrections being used to reduce the effects of the given vibration on the image;

an interpolation step of interpolating the shifts or the corrections; and

a correction step of correcting pixels constituting the image, captured under control in the imaging control step, on the basis of the corrections to generate an image composed of the corrected pixels,

wherein

in the conversion step, the vibration detection information is converted into a first shift and a second shift, the first shift being obtained when a first line is imaged, the second shift being obtained when a second line is imaged,

in the interpolation step, the first and second lines are interpolated to obtain a shift of each line between the first and second lines, and

in the calculation step, a correction of each line is calculated on the basis of the corresponding shift obtained in the interpolation step,

or wherein

in the conversion step, the vibration detection information is converted into a first shift and a second shift, the first shift being obtained when a first line is imaged, the second shift being obtained when a second line is imaged,

in the calculation step, a first correction and a second correction are calculated from the first shift and the second shift, respectively, and

in the interpolation step, the first and second corrections are interpolated to obtain a correction of each line between the first and second lines.