HETEROGENEOUS VIDEO CAPTURING SYSTEM

• US 20100182440A1
• Filed: 12/31/2009
• Published: 07/22/2010
• Est. Priority Date: 05/09/2008
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: McCloskey, Scott
• Granted Patent: US 8,436,907 B2
• Time in Patent Office: Days
• US Class Current: 348/208.400
• CPC Class Codes: G06K 7/10 by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
• CPC Class Codes: G06K 7/10722 Photodetector array or CCD scanning
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/003 Deblurring; Sharpening vibration or motion blur correction for cameras comprising an electronic image sensor H04N23/682
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

A heterogeneous video capturing system having motion and blur estimation. An item of motion may be captured with a continuous shutter and a flutter shutter. The flutter shutter image may be preceded and following by a continuous shutter image. Motion of a subject of the images may be estimated from the shutter images. The estimated motion may be a basis for blur estimation and blur removal from the flutter shutter image.

Claims

1. A method for capturing video, comprising:

capturing a first image with a continuous shutter;

capturing a second image with a flutter shutter;

capturing a third image with a continuous shutter;

determining the motion of an object relative to the continuous and flutter shutters from the first and third images;

estimating blur from the motion of the object and the second image;

developing one or more de-blur parameters from the estimated blur; and

applying the one or more de-blur parameters to the second image.

2. The method of claim 1, further comprising using the motion of the object to select a fluttering pattern for the flutter shutter.

3. The method of claim 1, further comprising performing auto-exposure while capturing the first, second and third images.

4. The method of claim 1, wherein the object is an iris.

5. A flutter shutter image capturing system comprising:

a flutter shutter;

a continuous shutter; and

a processor controlling the flutter shutter and the continuous shutter; and

wherein;

an image captured by the flutter shutter is preceded by an image captured by the continuous shutter and followed by an image captured by the continuous shutter;

parameters of motion of an object in the images relative to the continuous and flutter shutters are approximated from the images preceding and following the image captured by the flutter shutter; and

the parameters of motion are input to the processor for obtaining a blur estimate of the image captured by the flutter shutter.

6. The system of claim 5, wherein the processor derives de-blurring parameters from the blur estimate and applies the de-blurring parameters to the image captured by the flutter shutter.

7. The system of claim 6, wherein:

the flutter shutter has a flutter pattern of a group consisting of various fluttering patterns; and

the processor uses the parameters of motion to select an appropriate fluttering pattern from the group consisting of various fluttering patterns for the flutter shutter; and

wherein;

a fluttering pattern has a number of chops;

each chop has an active duration of time;

each chop is separated from another chop by a inactive duration of time; and

various fluttering patterns may differ from one another in terms of the number of chops, the active durations of time, and/or the inactive durations of time.

8. The system of claim 5, wherein:

the continuous shutter and the flutter shutter alternatively capture a series of images; and

the object in the images is of a group consisting of irises, vehicular license plates, and items moveable relative to the continuous and flutter shutters.

9. The system of claim 5, wherein:

the continuous shutter has a profile of a first time exposure and a first light gain;

the flutter shutter has a profile of a second time exposure and a second light gain;

the second time exposure is greater than the first time exposure; and

the first light gain is greater than the second light gain.

10. The system of claim 9, wherein the total amounts of light captured by the continuous shutter and the flutter shutter for an image are the same.

11. A video capture system comprising:

a camera;

a processor connected to the camera; and

wherein the camera comprises;

a continuous shutter; and

a flutter shutter; and

wherein;

a first image of a target is captured with the continuous shutter;

a second image is captured with a flutter shutter; and

a third image is captured with the continuous shutter.

12. The system of claim 11, wherein the first, second and third images are captured in that order, respectively.

13. The system of claim 12, wherein a motion of the target is determined from the first to third images.

14. The system of claim 13, wherein:

a fourth image of the target is captured with a continuous shutter;

a fifth image of the target is captured with a flutter shutter; and

a pattern of the flutter shutter is determined according to the motion of the target in the first to third images.

15. The system of claim 14, wherein

the pattern of the flutter shutter has a number of chops;

each chop has an active duration of time;

each chop is separated from another chop by a inactive duration of time; and

various patterns of the flutter shutter may differ from one another in terms of the number of chops, the active durations of time, and/or the inactive durations of time.

16. The system of claim 14, wherein the fifth image is de-blurred with information from the motion of the target.

17. The system of claim 12, wherein:

the continuous shutter has a profile of a first time exposure and a first light amplification;

the flutter shutter has a profile of a second time exposure and a second light amplification;

the second time exposure is greater than the first time exposure; and

the first light amplification is greater than the second light amplification.

18. The system of claim 17, wherein the total amounts of light captured by the continuous shutter and he flutter shutter for an image are the same.

19. The system of claim 12, wherein:

the images taken with a continuous shutter before and after each image with a flutter shutter are analyzed to provide an approximation of motion of the target to obtain a blur estimation of the image taken with the flutter shutter; and

the blur estimation is used for de-blurring the image taken with the flutter shutter.

20. The system of claim 19, wherein the blur estimation is provided to the processor to obtain an appropriate fluttering pattern for the flutter shutter.