Method for correcting an acquired image

• US 10,515,439 B2
• Filed: 04/02/2018
• Issued: 12/24/2019
• Est. Priority Date: 02/19/2016
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: Stec, Piotr
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US15/942,789
• Publication Number:
  US 20180330478A1
• Time in Patent Office: 631 Days
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 23/681 Motion detection
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/689 Motion occurring during a rolling shutter mode

Abstract

A method of correcting an image obtained by an image acquisition device includes obtaining successive measurements, G_n, of device movement during exposure of each row of an image. An integration range, idx, is selected in proportion to an exposure time, t_e, for each row of the image. Accumulated measurements, C_n, of device movement for each row of an image are averaged across the integration range to provide successive filtered measurements, G, of device movement during exposure of each row of an image. The image is corrected for device movement using the filtered measurements G.

Claims

1. A method of correcting an image obtained by an image acquisition device comprising:

obtaining a first measurement of device movement during exposure of a first row of an image and a second measurement of device movement during exposure of a second row of the image, G_n;

selecting an integration range, idx, based, at least in part, on an exposure time, t_e, for the first row of the image and the second row of the image,

averaging accumulated measurements, C_n, of device movement for the first row of the image and the second row of the image to provide a filtered measurement,G, of device movement during exposure of the first row of the image and the second row of the image, and

correcting the image for device movement using the filtered measurement G.

2. A method according to claim 1, wherein the integration range, idx, is calculated based, at least in part, on a sampling frequency F for the first measurement of device movement and the second measurement of device movement and the exposure time t_e as follows:

idx=F*t_e.

3. A method according to claim 1, wherein the averaging comprises calculating the filtered measurement G as follows:

G=(C_n−

C_n-idx)/idx where C_n is an accumulated value corresponding to measurement Gn and Cn-idx is an accumulated value corresponding to measurement G_n-idx at the start of the integration range.

4. A method according to claim 1, wherein the first measurement of device movement or the second measurement of device movement comprise one or more of:

gyroscope, accelerometer or magnetometer measurements.

5. A method according to claim 1, wherein the image comprises a still image or an image within a sequence of images.

6. A method according to claim 1, wherein the first row of the image is acquired during a first exposure time and the second row of the image is acquired during a second exposure time.

7. A method according to claim 6, wherein the first measurement of device movement corresponds to the first exposure time and the second measurement of device movement corresponds to the second exposure time.

8. A method according to claim 1, wherein the image is acquired using a rolling shutter technique.

9. A method according to claim 1, wherein the image is part of a video sequence and wherein the correcting the image for device movement using the filtered measurement comprises stabilizing multiple images of the video sequence.

10. A method according to claim 1, wherein the correcting the image for device movement using the filtered measurement comprises correcting blur in at least a portion of the image to account for device motion during image capture.

11. A system comprising:

a processor; and

memory in communication with the processor;

an image processing pipeline configured to provide an image for correction, and in communication with the processor;

instructions, stored in the memory, which when executed by the processor configure the system to perform actions comprising;

obtaining a first measurement of device movement during exposure of a first row of an image and a second measurement of device movement during exposure of a second row of the image, G_n;

selecting an integration range, idx, based, at least in part, on an exposure time, t_e, for the first row of the image and the second row of the image,

averaging accumulated measurements, C_n, of device movement for the first row of the image and the second row of the image to provide a filtered measurement,G, of device movement during exposure of the first row of the image and the second row of the image, and

correcting the image for device movement using the filtered measurement G.

12. A system according to claim 11, wherein the integration range, idx, is calculated based, at least in part, on a sampling frequency F for the first measurement of device movement and the second measurement of device movement and the exposure time t_e as follows:

idx=F*t_e.

13. A system according to claim 11, wherein the averaging comprises calculating the filtered measurement G as follows:

G=(C_n−

C_n-idx)/where C_n is an accumulated value corresponding to measurement G_n-idx is an accumulated value corresponding to measurement G_n-idx at the start of the integration range.

14. A system according to claim 11, wherein the first measurement of device movement or the second measurement of device movement comprise one or more of:

gyroscope, accelerometer or magnetometer measurements.

15. A system according to claim 11, wherein the image comprises a still image or an image within a sequence of images.

16. A system according to claim 11, wherein the first row of the image is acquired during a first exposure time and the second row of the image is acquired during a second exposure time.

17. A system according to claim 16, wherein the first measurement of device movement corresponds to the first exposure time and the second measurement of device movement corresponds to the second exposure time.

18. A system according to claim 11, wherein the image is acquired using a rolling shutter technique.

19. A system according to claim 11, further comprising an electronic image stabilisation component and wherein the instructions are further configured to perform actions comprising stabilizing the image based, at least in part, on the filtered measurement.

20. A system according to claim 11, wherein the device movement corresponds to a vibration of a camera during image capture.