Camera and method for optimizing the exposure of an image frame in a sequence of image frames capturing a scene based on level of motion in the scene

• US 9,235,880 B2
• Filed: 07/27/2012
• Issued: 01/12/2016
• Est. Priority Date: 12/22/2011
• Status: Active Grant

• Current Assignee: Axis AB (Canon Inc.)
• Original Assignee: Axis AB (Canon Inc.)
• Inventors: Kannermark, Per, Nilsson, Andreas
• Primary Examiner(s): Haskins, Twyler
• Assistant Examiner(s): TEJANO, DWIGHT ALEX C
• Application Number:
  US13/560,592
• Publication Number:
  US 20130162855A1
• Time in Patent Office: 1,264 Days
• Field of Search: 34820899-20811
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/20216 Image averaging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 25/587 acquired sequentially, e.g. using the combination of odd and even image fields

Abstract

A method and camera are used for optimizing the exposure of an image frame in a sequence of image frames capturing a scene based on level of motion in the scene. Based on image data from a plurality of image sensor frames, regions of the scene are determined including different level of motion. Image frame regions for the image frame are determined, wherein an image frame region corresponds to at least one region of the scene. The exposure of the image frame is optimized by emulating a region specific exposure time for each image frame region by producing each image frame region using image data from a number of image sensor frames. The number of image sensor frames used to produce a specific image frame region is based on the level of motion in the at least one corresponding region of the scene.

Claims

1. A method for producing an image frame in a sequence of image frames capturing a scene based on level of motion in the scene, the image frame in a sequence of image frames having an optimized exposure, the method comprising:

accessing image sensor frames representing the scene;

detecting motion in image data from a plurality of the image sensor frames;

determining regions of the scene comprising different levels of motion based on the motion detected in image data from the plurality of the image sensor frames;

determining image frame regions for the image frame, wherein an image frame region corresponds to at least one of the regions of the scene;

optimizing the exposure of the image frame by emulating a region specific exposure time for each image frame region by producing different image frame regions using image data from different numbers of the image sensor frames, wherein a number of the image sensor frames used to produce a specific image frame region is based on the level of motion in the at least one corresponding region of the scene.

2. The method according to claim 1, wherein the image data used to produce each image frame region is image data originating from the corresponding at least one region of the scene.

3. The method according to claim 1, wherein at least one image frame region is produced by combining image data from two or more image sensor frames.

4. The method according claim 3, wherein combining image data from two or more image sensor frames is carried out through adding and/or averaging image data from two or more image sensor frames.

5. The method according to claim 1, wherein the level of motion in a region of the scene corresponds to the speed of events of motion in the region of the scene and/or to the number of events of motion in the region of the scene.

6. The method according to claim 1, wherein each image sensor frame is captured using a unit exposure time.

7. The method according to claim 1, wherein the region specific exposure time for each image frame region is equal to or longer than the unit exposure time for each image sensor frame.

8. The method according to claim 1, wherein the plurality of image sensor frames are used to construct a statistical map of level of motion for different regions of the scene, wherein the determining regions of the scene comprising different level of motion is made by analyzing the statistical map.

16. The method according to claim 8, wherein the statistical map indicates frequency of motion events occurring at specific location.

17. The method according to claim 8, wherein the statistical map is updated over time based on data relating to the motion events acquired from the image sensor frames.

20. The method according to claim 1, wherein the level of motion in a region of the scene corresponds to the speed of events of motion in the region of the scene and to the number of events of motion in the region of the scene.

9. A non-transitory computer-readable recording medium having recorded thereon a program for implementing the method that when executed on a device having processing capabilities, performs the following steps:

accessing image sensor frames representing the scene;

detecting motion in image data from a plurality of the image sensor frames;

determining regions of the scene comprising different levels of motion based on the motion detected in image data from the plurality of the image sensor frames;

determining image frame regions for the image frame, wherein an image frame region corresponds to at least one of the regions of the scene;

optimizing the exposure of the image frame by emulating a region specific exposure time for each image frame region by producing different image frame regions using image data from different numbers of the image sensor frames, wherein a number of the image sensor frames used to produce a specific image frame region is based on the level of motion in the at least one corresponding region of the scene.

10. A device having processing capabilities executing the computer-readable recording medium according to claim 9.

11. A camera arranged to produce an image frame in a sequence of image frames capturing a scene based on level of motion in the scene, the image frame in a sequence of image frames having an optimized exposure, the camera comprising:

an image sensor arranged to capture and output image data as image sensor frames;

an image frame production module arranged to produce the image frame;

wherein the image frame production module is arranged to divide the image frame into image frame regions, and

wherein the image frame production module is arranged to produce each image frame region by using image data from a number of the image sensor frames; and

a motion detection module arranged to detect motion in image data from a plurality of the image sensor frames, and to determine regions of the scene comprising different levels of motion based on the motion detected in image data from the plurality of the image sensor frames, wherein an image frame region corresponds to at least one of the regions of the scene, and

wherein a number of image sensor frames used to produce a specific image frame region is based on the level of motion in the at least one corresponding region of the scene,

whereby the image frame production module is arranged to emulate, based on the level of motion in at least one corresponding region of the scene, different exposure times for different image frame regions and thereby the exposure of the image frame is optimized.

12. The camera according to claim 11, wherein the image frame production module is arranged to produce each image frame region by using image data originating from the corresponding at least one region of the scene.

13. The camera according to claim 11, wherein the image frame production module is arranged to produce at least one image frame region by combining image data from two or more image sensor frames, wherein the image frame production module is arranged to combine image data from two or more image sensor frames by adding and/or averaging image data from two or more image sensor frames.

14. The camera according to claim 11, wherein the level of motion in a region of the scene corresponds to the speed of events of motion in the region of the scene and/or to the number of events of motion in the region of the scene.

15. The camera according to claim 11, wherein the motion detection module is arranged to construct a statistical map of level of motion for different regions of the scene by using the plurality of image sensor frames, wherein the motion detection module is arranged to determine the regions of the scene comprising different level of motion by analyzing the statistical map.

18. The camera according to claim 15, wherein the statistical map indicates frequency of motion events occurring at specific location.

19. The camera according to claim 15, wherein the statistical map is updated over time based on data relating to the motion events acquired from the image sensor frames.

21. The method according to claim 11, wherein the level of motion in a region of the scene corresponds to the speed of events of motion in the region of the scene and to the number of events of motion in the region of the scene.