Method and system for deblurring an image based on motion tracking

• US 7,643,062 B2
• Filed: 06/08/2005
• Issued: 01/05/2010
• Est. Priority Date: 06/08/2005
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Silverstein, D. Amnon, Lim, Suk Hwan
• Primary Examiner(s): Tran, Sinh
• Assistant Examiner(s): BERARDESCA, PAUL M
• Application Number:
  US11/148,985
• Publication Number:
  US 20060279639A1
• Time in Patent Office: 1,672 Days
• Field of Search: 348/208.7, 348/208.5, 348/208.99, 348/208.2, 348/208.4, 396/53
• US Class Current: 348/208.4
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position

Abstract

A method and system for deblurring an image based on motion tracking. light is recorded at an image sensor of an imaging device to capture an image. motion of said image sensor is recorded at a motion sensor of the imaging device during the recording of the light at the image sensor. A blur kernel is generated based on the motion.

Claims

1. A method for deblurring an image based on motion tracking, said method comprising:

recording light at an image sensor of an imaging device to capture an image;

recording motion of said imaging device at a motion sensor of said imaging device during said recording said light at said image sensor; and

generating a blur kernel based on said motion

further comprising deblurring said image based on said blur kernel;

wherein said deblurring said image based on said blur kernel comprises deconvolving said image based on said blur kernel;

recording motion of said image sensor during said recording said light at said image sensor; and

determining an imperfection between said motion of said image sensor and said motion of said imaging device.

2. The method as recited in claim 1 further comprising estimating noise of said image.

3. The method as recited in claim 2 wherein said deconvolving said image comprises performing a Wiener deconvolution on said image based on said blur kernel and said noise.

4. The method as recited in claim 1 wherein said recording said motion is performed periodically during said recording said light at said image sensor.

5. The method as recited in claim 1 wherein said blur kernel is a point spread function.

6. The method as recited in claim 1 further comprising:

determining whether a measure of said motion of said image sensor is greater than a displacement threshold; and

if a measure of said motion is greater than a displacement threshold, repositioning said image sensor such that a subsequent measure of said motion is not greater than said displacement threshold.

7. The method as recited in claim 6 wherein said displacement threshold is associated with a maximum displacement of said image sensor.

8. The method as recited in claim 6 wherein said image is a double-exposed image and wherein said blur kernel comprises at least two points.

9. The method as recited in claim 1 wherein said recording motion of said image sensor is performed at a second motion sensor of said imaging device.

10. The method as recited in claim 1 wherein said recording motion of said image sensor is performed using a prediction module of said motion of said image sensor based on motion of said imaging device.

11. An imaging system comprising:

an image sensing means for recording light to capture an image;

a motion sensing means for recording motion of said imaging system during said capture of said image;

a memory means for storing said image and said motion; and

a processing means for generating a blur kernel based on said motion

wherein said processing means is also for deblurring said image based on said blur kernel;

wherein said processing means is operable to deblur said image by deconvolving said image based on said blur kernel;

a second motion sensing means for recording motion of said image sensing means during said capture of said image; and

imperfection determination means for determining an imperfection between said motion of said imaging system and said motion of said image sensing means.

12. The imaging system as recited in claim 11 wherein said processing means is also for estimating noise of said image.

13. The imaging system as recited in claim 12 wherein said deconvolving is performed using a Wiener deconvolution based on said blur kernel and said noise.

14. The imaging system as recited in claim 11 wherein said motion sensing means is operable to periodically record said motion during said capture of said image.

15. The imaging system as recited in claim 11 wherein said blur kernel is a point spread function.

16. The imaging system as recited in claim 11 further comprising:

means for determining whether a measure of said motion of said image sensing means is greater than a displacement threshold; and

means for repositioning said image sensor if a measure of said motion is greater than a displacement threshold such that a subsequent measure of said motion is not greater than said displacement threshold.

17. The imaging system as recited in claim 16 wherein said displacement threshold is associated with a maximum displacement of said image sensing means.

18. The imaging system as recited in claim 16 wherein said image is a double-exposed image and wherein said blur kernel comprises at least two points.

19. The imaging system as recited in claim 11 wherein said second motion sensing means comprises a motion sensor.

20. The imaging system as recited in claim 11 wherein said second motion sensing means comprises a prediction module of said motion of said image sensing means based on motion of said imaging system.

21. A computer-usable medium having computer-readable program code embodied therein for causing a computer system to perform a method for deblurring an image based on motion tracking, said method comprising:

recording light at an image sensor of an imaging device to capture an image;

recording motion of said image sensor at a motion sensor of said imaging device during said recording said light at said image sensor; and

generating a blur kernel based on said motion

further comprising deblurring said image based on said blur kernel;

wherein said deblurring said image based on said blur kernel comprises deconvolving said image based on said blur kernel;

recording motion of said image sensor during said recording said light at said image sensor; and

determining an imperfection between said motion of said image sensor and said motion of said imaging device.

22. The computer-usable medium as recited in claim 21 wherein said method further comprises:

determining whether a measure of said motion of said image sensor is greater than a displacement threshold; and

if a measure of said motion is greater than a displacement threshold, repositioning said image sensor such that a subsequent measure of said motion is not greater than said displacement threshold.