IMAGE-CAPTURING APPARATUS, IMAGE-CAPTURING METHOD AND PROGRAM

• US 20100231731A1
• Filed: 08/01/2008
• Published: 09/16/2010
• Est. Priority Date: 08/03/2007
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Motomura, Hideto, Azuma, Takeo
• Granted Patent: US 7,916,177 B2
• Time in Patent Office: Days
• US Class Current: 348/208.400
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction

Abstract

In an image-capturing apparatus and image-capturing method which can, in electronic image blur correction, correct camera motion-caused blurring occurring in one frame, an image-capturing apparatus (100) includes: a first picture feature amount transforming unit (104) transforming picture feature amount of input picture (117); a blur detecting unit 105 detecting blurring by evaluating input picture feature amount (118) chronologically; a blur synthesizing unit (107) artificially causing blurring in blur-free image-captured picture (129) by using blur model (124), and generating blur-synthesized picture (130); a blur removal picture feature amount replacement table unit (110) for replacing picture feature amount of a blurred picture with picture feature amount of a blur-free picture; and a closest-picture feature amount detecting unit (113) outputting, as blur-removed picture feature vector picture (133), picture feature amount of a blur-free image-captured picture that pairs with picture feature amount closest to picture feature amount of input picture feature amount (118).

Claims

1. An image-capturing apparatus which corrects blurring that occurs in an image-captured picture, said image-capturing apparatus comprising:

an image-capturing unit configured to image-capture pictures in video form;

a blur detecting unit configured to detect occurrence of blurring in an image-captured picture that is image-captured by said image-capturing unit, and a temporal position at which the blurring has occurred;

a blur synthesizing unit configured to generate blur-synthesized pictures by using blur-free image-captured pictures and a blurred image-captured picture, and synthesizing artificial blurring onto the blur-free image-captured pictures, the blur-free image-captured pictures being image-captured pictures in which the blurring is not detected by said blur detecting unit, and the blurred image-captured picture being a picture in which the blurring is detected;

a picture feature amount transforming unit configured to transform each of the blur-free image-captured pictures, the blur-synthesized pictures, and the blurred image-captured picture into a picture feature amount; and

a blur correcting unit configured to correct the blurring in the blurred image-captured picture by identifying a blur-synthesized picture having a picture feature amount that is similar to the picture feature amount of the blurred image-captured picture and replacing the picture feature amount of the blurred image-captured picture with the picture feature amount of a blur-free image-captured picture corresponding temporally to the identified blur-synthesized picture, the identifying and replacing being performed for each of pixels or blocks in the temporal position at which the blurring is detected by said blur detecting unit.

2. The image-capturing apparatus according to claim 1,

wherein said blur correcting unit includes:

a replacement table unit configured to generate a replacement table in which the picture feature amounts of the blur-synthesized pictures and the picture feature amounts of the blur-free image-captured pictures are associated with each other, each of the blur-synthesized pictures corresponding temporally to a corresponding one of the blur-free image-captured pictures; and

a correcting unit configured to identify a blur-synthesized picture with a picture feature amount that is closest to the picture feature amount of the blurred image-captured picture by using the replacement table, and to replace the picture feature amount of the blurred image-captured picture with the picture feature amount of the blur-free image-captured picture corresponding temporally to the identified blur-synthesized picture.

3. The image-capturing apparatus according to claim 1,

wherein said blur synthesizing unit is configured to generate each of the blur-synthesized pictures by detecting a motion of said image-capturing apparatus, creating a blur model which temporally interpolates between the blur-free image-captured picture and the blurred image-captured picture according to the motion of said image-capturing apparatus during a period between the blur-free image-captured picture and the blurred image-captured picture, and synthesizing the artificial blurring onto the blur-free image-captured picture using the created blur model.

4. The image-capturing apparatus according to claim 1,

wherein said blur correcting unit is configured to separately execute a table generating mode in which a replacement table unit generates a replacement table, and a correcting mode in which said correcting unit corrects the blurring in the blurred image-captured picture using the replacement table.

5. The image-capturing apparatus according to claim 1, further comprising

a storage unit configured to store the image-captured pictures that are image-captured by said image-capturing unit, together with blur detection information indicating the occurrence of the blurring detected by said blur detecting unit and a position at which the blurring has occurred,

wherein said blur detecting unit, said blur synthesizing unit, and said picture feature amount transforming unit are each configured to read the pictures and the blur detecting information stored in the storage unit, and to perform processing on the read pictures.

6. The image-capturing apparatus according to claim 1,

wherein said picture feature amount transforming unit is configured to transform each of the image-captured pictures that are image-captured by said image-capturing unit into a picture feature amount on a per pixel basis or a per block basis, and

said blur detecting unit is configured to track similarity of the picture feature amounts of the image-captured pictures in chronological order, and to judge, when an amount of change in a picture feature amount exceeds a predetermined threshold value, that the blurring has occurred in a position of a pixel or a block of the picture feature amount that exceeded the threshold value.

7. The image-capturing apparatus according to claim 1, further comprising

a temporary storage unit configured to temporarily store the blur-free image-captured pictures,

wherein, when said blur detection unit detects the blurring, said blur synthesizing unit is configured to generate the blur-synthesized pictures by synthesizing the artificial blurring onto the blur-free image-captured pictures stored in said temporary storage unit.

8. The image-capturing apparatus according to claim 1, further comprising

an instruction accepting unit configured to accept a selection for whether or not blur correction is to be performed,

wherein said blur correcting unit is configured to correct the blurring in the blurred image-captured picture only when the selection to perform the blur correction is accepted by said instruction accepting unit.

9. The image-capturing apparatus according to claim 8,

wherein said instruction accepting unit is further configured to accept a region on which the blur correction is to be performed, and

said blur correcting unit is configured to correct the blurring in the burred image-captured picture only on the region on which the blur correction is to be performed accepted by said instruction accepting unit.

10. The image-capturing apparatus according to claim 8,

wherein said instruction accepting unit is further configured to accept a region on which the blur correction is not to be performed, and

said blur correcting unit is configured not to correct the blurring in the burred image-captured picture on the region on which the blur correction is not to be performed accepted by said instruction accepting unit.

11. The image-capturing apparatus according to claim 1,

wherein said blur detection unit includes:

a blur detection frequency calculating unit configured to calculate a blur detection frequency within the blurred image-captured picture;

a threshold value comparing unit configured to judge whether to execute the replacement of the picture feature amounts for correcting the blurring, on a whole or part of the blurred image-captured picture, by comparing the blur detection frequency with a threshold value; and

a blur detection information generating unit configured to add a unit of picture-feature-amount replacement to the blur detection information, based on a result of the judgment by said threshold value comparing unit, and

when the blur detection information specifies replacement of the picture feature amounts on the whole of the blurred image-captured picture, the blurring is corrected by replacing the whole of the blurred image-captured picture with the blur-free image-captured picture instead of executing the replacement of the picture feature amounts.

12. The image-capturing apparatus according to claim 11, further comprising:

a zoom setting unit configured to set a zoom factor defining an angle of view for the image-capturing,

wherein said threshold value comparing unit is configured to perform the judgment by comparing the blur detection frequency with the threshold value that is adjusted according to the zoom factor set by said zoom setting unit.

13. An image-capturing method of correcting blurring that occurs in an image-captured picture, said image-capturing method comprising:

image-capturing pictures in video form;

detecting occurrence of blurring in an image-captured picture that is image-captured in said image-capturing, and a temporal position at which the blurring has occurred;

generating blur-synthesized pictures by using blur-free image-captured pictures and a blurred image-captured picture, and synthesizing artificial blurring onto the blur-free image-captured pictures, the blur-free image-captured pictures being image-captured pictures in which the blurring is not detected in said detecting, and the blurred image-captured picture being a picture in which the blurring is detected;

transforming each of the blur-free image-captured pictures, the blur-synthesized pictures, and the blurred image-captured picture into a picture feature amount; and

correcting the blurring in the blurred image-captured picture by identifying a blur-synthesized picture having a picture feature amount that is similar to the picture feature amount of the blurred image-captured picture and replacing the picture feature amount of the blurred image-captured picture with the picture feature amount of a blur-free image-captured picture corresponding temporally to the identified blur-synthesized picture, the identifying and replacing being performed for each of pixels or blocks in the temporal position at which the blurring is detected in said detecting.

14. A program for correcting blurring that occurs in an image-captured picture, said program causing a computer to execute:

image-capturing pictures in video form;

detecting occurrence of blurring in an image-captured picture that is image-captured in said image-capturing, and a temporal position at which the blurring has occurred;

generating blur-synthesized pictures by using blur-free image-captured pictures and a blurred image-captured picture, and synthesizing artificial blurring onto the blur-free image-captured pictures, the blur-free image-captured pictures being image-captured pictures in which the blurring is not detected in said detecting, and the blurred image-captured picture being a picture in which the blurring is detected;

transforming each of the blur-free image-captured pictures, the blur-synthesized pictures, and the blurred image-captured picture into a picture feature amount; and

correcting the blurring in the blurred image-captured picture by identifying a blur-synthesized picture having a picture feature amount that is similar to the picture feature amount of the blurred image-captured picture and replacing the picture feature amount of the blurred image-captured picture with the picture feature amount of a blur-free image-captured picture corresponding temporally to the identified blur-synthesized picture, the identifying and replacing being performed for each of pixels or blocks in the temporal position at which the blurring is detected in said detecting.