Camera exposure optimization techniques that take camera and scene motion into account

• US 20070092244A1
• Filed: 10/25/2005
• Published: 04/26/2007
• Est. Priority Date: 10/25/2005
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Zoran Corporation (Qualcomm, Inc.)
• Inventors: Pertsel, Shimon, Meitav, Ohad, Pozniansky, Eli
• Granted Patent: US 7,546,026 B2
• Time in Patent Office: Days
• US Class Current: 396/153
• CPC Class Codes: G03B 2217/005 Blur detection
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/684 performed by controlling the image sensor readout, e.g. by controlling the integration time

Abstract

Quantities of motion between images of an electronic camera are calculated and used to adjust the exposure time and one or more other exposure parameters used to capture an image in order to improve the quality of the image. Motion blur caused by movement of the camera or by movement of an object within the scene being photographed is reduced by selecting appropriate exposure parameters. Further, when there is little or no motion detected, the exposure parameters may be selected to improve the depth of field and reduce the noise in the captured image.

Claims

1. A method of controlling operation of an electronic imaging device to capture data of at least one image, comprising:

monitoring motion vectors within the image prior to its capture, utilizing information from monitoring the motion vectors within the image to calculate one or more exposure parameters including duration for use to capture data of the image, and thereafter capturing data of the image by use of the calculated one or more exposure parameters.

2. The method of claim 1, wherein at least one other parameter in addition to duration is calculated and used to capture data of the image.

3. The method of claim 2, wherein said at least one other exposure parameter that is calculated includes a size of an aperture through which light of the image passes.

4. The method of claim 2, wherein said at least one other exposure parameter that is calculated includes a gain quantity.

5. The method of claim 4, wherein the calculated gain is the gain of an analog signal.

6. The method of claim 4, wherein the calculated gain is a gain applied to digitized image data.

7. The method of claim 1, wherein utilizing information from monitoring motion vectors within the image includes utilizing individual motion vectors of at least some of distinct areas of the image in a manner that recognizes motion of one portion of the image with respect to another portion when calculating the one or more exposure parameters used to capture the image data.

8. The method of claim 1, wherein utilizing information from monitoring motion vectors within the image includes utilizing at least one overall motion quantity calculated from individual motion vectors of at least some of distinct areas of the image when calculating the one or more exposure parameters used to capture the image data.

9. The method of claim 2, wherein utilizing information to calculate the duration and at least one other exposure parameter includes first calculating a preliminary duration and at least one other preliminary exposure parameter without use of the information from monitoring the motion vectors, and thereafter adjusting the preliminary duration and at least one other preliminary exposure parameter by use of the information from monitoring the motion vectors.

10. The method of claim 9, wherein adjusting the preliminary duration includes increasing the preliminary duration in response to the information from monitoring the motion vectors.

11. The method of claim 10, wherein the information from monitoring the motion vectors indicates a level of image motion that is below a predetermined threshold.

12. The method of claim 9, wherein adjusting the preliminary duration includes decreasing the preliminary duration in response to the information from monitoring the motion vectors.

13. The method of claim 10, wherein the information from monitoring the motion indicates a level of image motion that is above a predetermined threshold.

14. The method of claim 1, which additionally comprises utilizing information from monitoring the motion vectors within the image to estimate a time for capturing data of the image, and thereafter capturing the data of the image at the estimated time.

15. The method of claim 14, wherein the time for capturing data of the image is determined as an estimate of a time during a predefined interval when the motion is expected to be minimal or less than a predetermined threshold.

16. The method of claim 14, wherein information from monitoring the motion vectors within the image from which the one or more exposure parameters are calculated includes an estimate of said motion at the estimated time for capturing data of the image.

17. The method of claim 1, wherein monitoring the motion vectors within the image prior to its capture includes comparing data of at least two pre-capture images.

18. A method of operating an electronic imaging device, comprising:

repetitively acquiring data of two or more images of a scene,

calculating at least one quantity of motion of at least a portion of the scene relative to the camera from the acquired data of said two or more images,

preliminarily setting, from luminescence information within the data of said two or more images, parameters including a duration for capturing data of a final image of the scene without regard to the relative motion,

in response to said at least one quantity of relative motion being below a first threshold, adjusting the preliminarily set parameters by at least increasing the duration of exposure,

in response to said at least one quantity of relative motion being above a second threshold, adjusting the preliminarily set parameters by decreasing at least the duration of exposure, and

thereafter capturing data of the final image of the scene by use of the adjusted duration and at least one other adjusted exposure parameter selected from a group comprising aperture size and gain.

19. The method of claim 18, wherein the first and second thresholds are the same.

20. The method of claim 18, wherein preliminarily setting parameters includes setting at least one other exposure parameter, and adjusting the preliminarily set parameters in response to said at least one quantity of relative motion being below a first threshold by additionally decreasing said at least one other exposure parameter, and adjusting the preliminarily set parameters in response to said at least one quantity of relative motion being above a second threshold by additionally increasing said at least one other exposure parameter.

21. The method of claim 20, wherein the first and second thresholds are the same.

22. The method of claim 20, wherein said at least one other exposure parameter includes a size of an aperture through which light of the image passes.

23. The method of claim 20, wherein said at least one other exposure parameter includes image gain.

24. The method of claim 23, wherein the image gain includes a level of gain of an analog signal carrying image information prior to capturing the data.

25. The method of claim 23, wherein capturing the data includes digitizing the captured data and adjusting the image gain includes adjusting a level of gain of the digitized image data.

26. The method of claim 18, wherein said at least one quantity of relative motion includes motion of the camera with respect to the scene.

27. The method of claim 18, wherein said at least one quantity of relative motion includes motion of one portion of the scene.

28. The method of claim 18, which additionally comprises using said at least one quantity of relative motion to determine a time to capture data of the final image, and therafter capturing said data of the final image.

29. The method of claim 28, wherein the time to capture data of the final image is determined to include an estimate of a time during a predefined interval when the relative motion is expected to be minimal or less than a predetermined threshold.

30. The method of claim 28, wherein calculating at least one quantity of relative motion includes estimating said at least one quantity at the time determined to capture data of the final image.

31. The method of claim 18, which additionally comprises, after preliminarily setting exposure parameters, determining whether the preliminarily set exposure parameters fall within predefined ranges, and if so, adjusting of the preliminarily set exposure parameters is omitted.

32. The method of claim 18, wherein acquiring data of two or more images of a scene includes acquiring data of two or more successive images having a resolution that is lower than a resolution of the final image.

57. The method of claim 18, wherein calculating at least one quantity of motion includes utilizing motion vectors within said at least a portion of the scene.

33. A method of operating an electronic imaging device to capture data of at least one image of a scene, comprising:

acquiring information of relative motion between at least a portion of the scene and the imaging device prior to the capture of image data,

calculating from the acquired information at least one time varying quantity of relative motion between at least the portion of the scene and the imaging device,

estimating from said at least one quantity a future time when the relative motion will be minimal or less than a predetermined threshold,

calculating one or more exposure parameters including an exposure duration that enhance the captured image based upon the relative motion at the estimated future time, and

capturing the data of said at least one image of the scene at the estimated future time with the calculated one or more exposure parameters.

34. The method of claim 33, wherein acquiring information of relative motion between the scene and the imaging device includes repetitively acquiring data of a plurality of successive images of the scene, and wherein calculating at least one time varying quantity of relative motion use data from at least some of the plurality of successive images.

35. The method of claim 33, wherein calculating one or more exposure parameters includes calculating the exposure duration followed by calculating at least one other exposure parameter.

36. The method of claim 35, wherein the at least one other calculated exposure parameter includes at least one of a size of an aperture opening or image gain.

37. The method of claim 36, wherein the image gain includes a level of gain of an analog signal carrying image information prior to capturing the data.

38. The method of claim 36, wherein capturing the data includes digitizing the captured data and the calculated exposure parameter of image gain includes adjusting a level of gain of the digitized image data.

39. The method of claim 33, wherein acquiring information of relative motion includes acquiring information of movement of one portion of the scene with respect to another portion of the scene.

40. The method of claim 33, wherein acquiring information of relative motion includes acquiring information of movement of the entire scene image with respect to the imaging device.

58. The method of claim 33, wherein calculating at least one time varying quantity of relative motion includes utilizing motion vectors within said at least the portion of the scene.

41. A method of operating an electronic imaging device, comprising:

acquiring data of two or more preliminary images of a scene,

displaying the preliminary images, as their data are acquired, on a visual display that is part of the imaging device,

calculating at least one quantity of motion of at least a portion of the scene from the data of the acquired two or more preliminary images,

preliminarily setting, from luminescence information within at least one of the acquired two or more preliminary images, parameters including a duration for capturing data of a final image of the scene without regard to the calculated quantity of motion,

utilizing information from monitoring motion within the two or more preliminary images to calculate one or more exposure parameters including duration for use to capture data of the final image, and

thereafter capturing data of the final image by use of the calculated one or more exposure parameters, wherein the final image has a higher resolution than the preliminary images.

42. The method of claim 41, wherein calculating at least one quantity of motion includes utilizing motion vectors within said at least a portion of the scene.

43. An electronic imaging device, comprising:

a photodetector,

an optical system that projects light from a scene onto the photodetector,

an electronic processing unit that receives data of the scene projected onto the photodetector and processes the scene data to provide data of a captured image, and

wherein the processing unit additionally quantifies, from the data of the scene projected onto the photodetector, an amount of motion of the scene and uses this motion amount to set one or more exposure parameters including gain that are used to provide the captured image data.

44. The device of claim 43, wherein the optical system includes an adjustable aperture and the processing unit uses the quantified motion amount to set a parameter of the size of the aperture.

45. The device of claim 43, wherein the processing unit uses the quantified motion amount to set a parameter of analog gain within the device.

46. The device of claim 43, wherein the processing unit uses the quantified motion amount to set a parameter of digital gain of the captured image data.

47. The device of claim 43, wherein the processing unit quantifies an amount of motion in the scene by comparing motion vectors of at least some of distinct areas of the scene between two or more images of the scene projected onto the photodetector prior to providing the captured image data.

48. The device of claim 43, wherein the processing unit quantifies an amount of motion in the scene by m determining an overall motion of two or more images the scene projected onto the photodetector from motion vectors of at least some of distinct areas of the scene between the two or more images prior to providing the captured image data.

49. The device of claim 43, wherein the processing unit additionally calculates at least a preliminary exposure duration in response to a luminescence level of the scene but without use of the quantified amount of motion of the scene, and uses the quantified amount of motion of the scene to adjust the calculated preliminary exposure duration.

50. The device of claim 43, wherein the processing unit additionally uses the quantified amount of motion of the scene to determine a time to capture data of the scene and causes the data of the scene to be captured at the determined time.

51. The device of claim 50, wherein the processing unit additionally determines the time to capture data of the scene by estimating a time during a predefined interval when the amount of motion of the scene is expected to be minimal or less than a predetermined threshold.

52. An electronic imaging device, comprising:

a photodetector,

an optical system that projects light from a scene onto the photodetector,

an electronic processing unit that receives data of the scene projected onto the photodetector and processes the scene data to provide data of a captured image, and

wherein the processing unit additionally calculates, from the data of the scene projected onto the photodetector, individual motion vectors of at least some of distinct areas of the scene and utilizes the calculated motion vectors to determine a motion of the scene that is used to set one or more exposure parameters including an exposure duration that are used to provide the captured image data.

53. The device of claim 52, which additionally comprises an electronic display of data of a succession of preview images projected onto the photodetector, and wherein the processing unit additionally calculates the motion vectors from the data of two or more such preview images occurring prior to capturing data of the image with a resolution that is greater than a resolution of the preview images.

54. The device of claim 52, wherein the processing unit additionally utilizes the determined motion of the scene to set an exposure parameter of an electronic analog gain prior to processing the scene data to provide data of a captured image.

55. The device of claim 52, wherein the processing unit additionally utilizes the determined motion of the scene to set an exposure parameter of a digital gain subsequent to providing data of a captured image by processing the scene data.

56. The device of claim 52, wherein the processing unit additionally utilizes the determined motion of the scene to set an exposure parameter of a size of an aperture within the optical system through which light of the scene passes.