Method and apparatus for improving photo image quality

• US 8,040,382 B2
• Filed: 01/07/2008
• Issued: 10/18/2011
• Est. Priority Date: 01/07/2008
• Status: Active Grant

• Current Assignee: DP Technologies, Inc.
• Original Assignee: DP Technologies, Inc.
• Inventors: Kahn, Philippe, Easwar, Venkat, Kinsolving, Arthur
• Primary Examiner(s): Daniels, Anthony J
• Application Number:
  US11/970,499
• Publication Number:
  US 20090174782A1
• Time in Patent Office: 1,380 Days
• Field of Search: None
• US Class Current: 348/208.99
• CPC Class Codes: G02B 27/646 compensating for small deviations, e.g. due to vibration or shake movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

A method and apparatus for receiving motion data and determining acceleration vectors based on biomechanical data and the received motion data. The method further comprises creating an improved image based on the acceleration vectors.

Claims

1. A method comprising:

receiving motion data of an image capture device;

applying the received motion data to a model that models tremor motion about a users joints and limbs as an approximation of a simple harmonic motion to determine one or more sinusoids that describe the motion of the image capture device, wherein the model is based on biomechanical data that defines a range of human arm motion;

selecting a time at which to capture image data by the image capture device, the time corresponding to a predicted point of minimum adverse motion of the image capture device based on the determined one or more sinusoids; and

creating an improved image based on the one or more sinusoids by capturing image data at the selected time.

2. The method of claim 1, wherein the sinusoids are derived from acceleration vectors.

3. The method of claim 1, wherein the sinusoids are derived from one or more of the following:

velocity vectors, angular velocity vectors, acceleration vectors.

4. The method of claim 1, wherein the received motion data includes acceleration vectors and determining the one or more sinusoids includes predicting an acceleration waveform based on the acceleration vectors, the method further comprising:

estimating motion corresponding to biomechanical tremor of a user interacting with the camera based on the acceleration waveform and the biomechanical data; and

identifying the predicted points of minimum adverse motion based on the motion estimation.

5. The method of claim 4, further comprising:

overlaying a point of minimum adverse motion with a frame sync signal; and

activating an image capture mechanism at a frame sync point that is close to the point of minimum adverse motion.

6. The method of claim 4, further comprising:

utilizing spectral estimation to predict the acceleration wave form.

7. The method of claim 6, further comprising:

utilizing a bandpass filter, and a narrow band filter to isolate a frequency; and

detecting phases and amplitudes of the frequencies, the phases and amplitudes utilized for the prediction of the acceleration waveform.

8. The method of claim 4, further comprising:

utilizing a quadrature filter to calculate a phase shifted signal;

calculating phase based on the phase shifted signal and the original signal; and

predicting the waveform based on an amplitude and the phase.

9. The method of claim 1, wherein creating the improved image comprises removing blur from an image in post processing on the basis of the biomechanical data and the received motion data.

10. The method of claim 9, further comprising:

calculating a global motion vector based on the motion data; and

adjusting a captured image based on the global motion vector.

11. The method of claim 10, wherein the global motion vector is a constant for a sub-portion of the captured image, and the adjusting is for the sub-portion.

12. The method of claim 1, further comprising, in a video:

identifying a frame having minimal global motion;

designating the frame as an I frame for encoding purposes.

13. The method of claim 12, further comprising:

identifying a blurry frame based on projected motion calculated from the received motion data and the biomechanical information;

removing the blurry frame; and

replacing the blurry frame with a frame interpolated from adjacent frames.

14. The method of claim 1, further comprising:

receiving data from a cardiac sensor that indicates cardio ballistic impulses of a user of the media capture device; and

utilizing the data from the cardiac sensor to delay the prediction of the sinusoids for a period of time after the cardio ballistic impulses were received.

15. The method of claim 1, further comprising:

a proximity sensor input to initiate prediction and capture process prior to actual shutter press.

16. An image improvement system for an image capture device, the system comprising:

a receiving logic to receive motion data of the image capture device;

a sinusoidal signal analysis logic to apply the received motion data to a model that models tremor motion about a users joints and limbs as an approximation of a simple harmonic motion to calculate one or more sinusoids that describe the motion of the image capture device, wherein the model is based on biomechanical data that defines a range of human arm motion; and

a pre-processing system to select a time at which to capture image data, the time corresponding to a predicted a point of minimum motion of the image capture device based on the determined one or more sinusoids;

wherein the image improvement system creates an improved image based on the one or more sinusoids by capturing the image at the selected time.

17. The system of claim 16, wherein the sinusoids are derived from one or more of the following:

velocity vectors, angular velocity vectors, acceleration vectors.

18. The system of claim 16, further comprising:

a minimum motion identification logic to estimate motion corresponding to biomechanical tremor of a user interacting with the image capture device based on the sinusoids and to identify the points of minimum motion; and

an overlay logic to overlay the points of minimum motion with a frame sync signal; and

a shutter logic to activate an image capture mechanism at a frame sync point that is close to the point of minimum motion.

19. The system of claim 18, further comprising:

a proximity sensor input to initiate prediction and capture process prior to actual shutter press.

20. The system of claim 16, further comprising:

a bandpass filter to isolate a frequency;

a spectral estimator to predict the acceleration wave form; and

a phase detector and an amplitude detector to identify phases and amplitudes of the frequency, the phases and amplitudes utilized for the calculation of the one or more sinusoids.

21. The system of claim 16, further comprising:

a quadrature filter to calculate a phase shifted signal;

an ArcTan calculator to calculate phase based on the phase shifted signal and the original signal; and

a prediction waveform generator to predict a waveform based on an amplitude and the phase.

22. The system of claim 16, further comprising a post-processing system to create the improved image by removing blur from an image on the basis of the biomechanical data and the received motion data.

23. The system of claim 22, further comprising:

a global motion vector calculator to calculate global motion vectors based on the motion data; and

image restoration logic to adjust a captured image based on the global motion vector.

24. The system of claim 23, wherein the global motion vector is a constant for a sub-portion of the captured image, and the adjusting is for the sub-portion of the captured image.

25. The system of claim 23, further comprising, in a video:

the global motion vector calculator to identify a frame having minimal global motion; and

the image restoration logic to designate the frame as an I frame for encoding purposes.

26. The system of claim 23, further comprising:

the image restoration logic to identify a blurry frame, remove the blurry frame, and replace the blurry frame with a frame interpolated from adjacent frames.

27. The system of claim 16, further comprising:

a cardiac sensor to detect cardio ballistic impulses of a user of the media capture device; and

a minimum motion identification logic utilizing data from the cardiac sensor to delay the calculation of the sinusoids for a period of time after the cardio ballistic impulses are detected.