Image Acquisition Method and Apparatus

• US 20110115928A1
• Filed: 10/19/2010
• Published: 05/19/2011
• Est. Priority Date: 06/05/2006
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia, Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia, Inc.)
• Inventors: Steinberg, Eran, Zamfir, Adrian, Florea, Corneliu, Drimbarean, Alexandru, Bigioi, Petronel, Corcoran, Peter
• Granted Patent: US 8,520,082 B2
• Time in Patent Office: Days
• US Class Current: 348/208.2
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/681 Motion detection
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken

Abstract

An image acquisition sensor of a digital image acquisition apparatus is coupled to imaging optics for acquiring a sequence of images. Images acquired by the sensor are stored. A motion detector causes the sensor to cease capture of an image when the degree of movement in acquiring the image exceeds a threshold. A controller selectively transfers acquired images for storage. A motion extractor determines motion parameters of a selected, stored image. An image re-constructor corrects the selected image with associated motion parameters. A selected plurality of images nominally of the same scene are merged and corrected by the image re-constructor to produce a high quality image of the scene.

Claims

1. A digital image acquisition apparatus, comprising:

an image acquisition sensor coupled to imaging optics for acquiring a sequence of images;

an image store for storing one or more of said sequence of images acquired by said sensor;

a dedicated motion detector hardware unit for providing hardware-based sensor control including causing said sensor to cease capture of an image when the degree of movement of the apparatus in acquiring said image exceeds a threshold, and for selectively transferring said one or more of said images acquired by said sensor to said image store; and

a dedicated motion extractor hardware unit for providing hardware-based determination of motion parameters of at least one a selected image stored in said image store.

2. The apparatus of claim 1, wherein exposure times of two or more of said sequence of images differ based on different degrees of movement of the apparatus.

3. The apparatus of claim 1, further comprising a dedicated image re-constructor hardware unit for providing hardware-based correction of at least one selected image with associated motion parameters.

4. The apparatus of claim 3, further comprising a dedicated image merger hardware unit for providing hardware-based merging of a selected plurality of images including said at least one selected image corrected by said dedicated image re-constructor hardware unit, to produce a high quality image of said scene.

5. An apparatus as claimed in claim 1, wherein the motion extractor is configured to estimate a point spread function (PSF) for said selected image.

6. An apparatus as claimed in claim 5, wherein said dedicated motion extractor hardware unit is configured to:

calculate a Cepstrum for said selected image;

identify one or more spikes in said Cepstrum; and

select one of said spikes in said Cepstrum as an end point for said PSF.

7. An apparatus as claimed in claim 6, wherein said dedicated motion extractor hardware unit is configured to calculate a negative Cepstrum, and to set points in said negative Cepstrum having a value less than a threshold to zero.

8. An apparatus as claimed in claim 1, wherein said image store comprises a temporary image store, and wherein said apparatus further comprises a non-volatile memory, said dedicated motion detector hardware unit being configured to store said one or more of said images in said non-volatile memory.

9. An apparatus as claimed in claim 1, wherein said dedicated motion detector hardware unit comprises a gyro-sensor or an accelerometer, or both.

20. The apparatus of claim 1, wherein said different degrees of movement comprise different degrees of non linear movement.

22. The apparatus of claim 1, wherein said image re-constructor further for enhancing an exposure of the at least one selected image.

10. A digital image acquisition apparatus, comprising:

an image acquisition sensor coupled to imaging optics for acquiring a sequence of images;

an image store for storing one or more of said sequence of images acquired by said sensor;

a dedicated motion detector hardware unit for providing hardware-based control of said sensor to cease capture of an image when the degree of movement of the apparatus in acquiring said image exceeds a first threshold, and for controlling the sensor to restart capture when the degree of movement of the apparatus is less than a given second threshold and for selectively transferring said one or more of said sequence of images acquired by said sensor to said image store; and

a dedicated motion extractor hardware unit for determining motion parameters of at least one selected image stored in said image store.

11. The apparatus of claim 10, wherein exposure times of two or more of said sequence of images differ based on different degrees of movement of the apparatus.

12. The apparatus of claim 10, further comprising a dedicated image re-constructor hardware unit for providing hard-based correction of at least one selected image with associated motion parameters.

13. The apparatus of claim 12, further comprising a dedicated image merger hardware unit for providing hardware-based merging of a selected plurality of images including said at least one selected image corrected by said image re-constructor, to produce a high quality image of said scene.

14. The apparatus as claimed in claim 10, further comprising a first exposure timer for storing an aggregate exposure time of said sequence of images, and wherein said apparatus is configured to acquire said sequence of images until the aggregate exposure time of at least a stored number of said sequence of images exceeds a predetermined exposure time for said high quality image.

15. An apparatus as claimed in claim 14, further comprising a second timer for storing an exposure time for a single image, and wherein said apparatus is configured to dispose of an image having an exposure time less than a threshold time.

16. An apparatus as claimed in claim 10, further comprising an image quality analyzer for a single image, and wherein said apparatus is configured to dispose of an image having a quality less than a given threshold quality.

17. An apparatus as claimed in claim 10, wherein said image merger is configured to align said selected plurality of images prior to merging said images.

18. An apparatus as claimed in claim 10, wherein said first and second thresholds comprise threshold amounts of motion energy.

19. The apparatus of claim 10, wherein said different degrees of movement comprise different degrees of non linear movement.

21. The apparatus of claim 10, wherein said image re-constructor further for enhancing an exposure of the at least one selected image.

23. A digital image acquisition apparatus, comprising:

an image acquisition sensor coupled to imaging optics for acquiring a sequence of images;

an image store for storing one or more of said images acquired by said sensor;

a dedicated motion detector hardware unit for providing hardware-based control of said sensor to cease capture of an image when the degree of movement of the apparatus in acquiring said image exceeds a threshold;

a controller for selectively transferring said one or more of said images acquired by said sensor to said image store; and

a dedicated motion extractor hardware unit for providing hardware-based determination of motion parameters of a selected image stored in said image store;

24. The apparatus of claim 23, wherein the motion extractor is configured to:

calculate a Cepstrum for said selected image;

identify one or more spikes in said Cepstrum; and

select one of said spikes in said Cepstrum as an end point for said PSF.

25. The apparatus of claim 23, further comprising a dedicated image re-constructor hardware unit for providing hardware-based correction of a selected image with associated motion parameters.

26. The apparatus of claim 25, further comprising a dedicated image merger hardware unit for providing hardware-based merging of a selected plurality of images nominally of the same scene and corrected by said image re-constructor to produce a high quality image of said scene.

27. An apparatus as claimed in claim 23, wherein the dedicated motion extractor hardware unit is configured to estimate a point spread function (PSF) for said selected image.

28. An apparatus as claimed in claim 23, wherein said dedicated motion extractor hardware unit is configured to calculate a negative Cepstrum, and to set points in said negative Cepstrum having a value less than a threshold to zero.

29. An apparatus as claimed in claim 23, wherein said image stored comprises a temporary image store, and wherein said apparatus further comprises a non-volatile memory, said dedicated image merger hardware unit being configured to store said high quality image in said non-volatile memory.

30. An apparatus as claimed in claim 23, wherein said dedicated motion detector hardware unit comprises a gyro-sensor or an accelerometer, or both.

31. A digital image acquisition apparatus, comprising:

an image acquisition sensor coupled to imaging optics for acquiring a sequence of images;

an image store for storing one or more of said images acquired by said sensor;

a dedicated motion detector hardware unit for providing hardware-based control of said sensor to cease capture of an image when the degree of movement of the apparatus in acquiring said image exceeds a first threshold;

one or more controllers that causes the sensor to restart capture when the degree of movement of the apparatus is less than a given second threshold and that selectively transfers said one or more of said images acquired by said sensor to said image store;

a dedicated motion extractor hardware unit for providing hardware-based determination of motion parameters of a selected image stored in said image store;

32. The apparatus of claim 31, wherein the dedicated motion extractor hardware unit is configured to:

calculate a Cepstrum for said selected image;

identify one or more spikes in said Cepstrum; and

select one of said spikes in said Cepstrum as an end point for said PSF.

33. The apparatus of claim 31, further comprising a dedicated image re-constructor hardware unit for providing hardware-based correction of a selected image with associated motion parameters.

34. The apparatus of claim 33, further comprising a dedicated image merger hardware unit for providing hardware-based merging of a selected plurality of images nominally of the same scene and corrected by said dedicated image re-constructor hardware unit to produce a high quality image of said scene.

35. An apparatus as claimed in claim 31, further comprising a first exposure timer for storing an aggregate exposure time of said sequence of images, and wherein said apparatus is configured to acquire said sequence of images until the aggregate exposure time of at least a stored number of said sequence of images exceeds a predetermined exposure time for said high quality image.

36. An apparatus as claimed in claim 35, further comprising a second timer for storing an exposure time for a single image, and wherein said apparatus is configured to dispose of an image having an exposure time less than a threshold time.

37. An apparatus as claimed in claim 35, further comprising an image quality analyzer for a single image, and wherein said apparatus is configured to dispose of an image having a quality less than a given threshold quality.

38. An apparatus as claimed in claim 31, wherein said dedicated image merger hardware unit is configured to align said selected plurality of images prior to merging said images.

39. An apparatus as claimed in claim 31, wherein said first and second thresholds comprise threshold amounts of motion energy.

40. One or more dedicated hardware units for installation within a digital image acquisition apparatus having an image acquisition sensor coupled to imaging optics for acquiring a sequence of images, and an image store for storing one or more of said sequence of images acquired by said sensor, the one or more dedicated hardware units comprising:

a dedicated motion detector hardware unit for providing hardware-based sensor control including causing said sensor to cease capture of an image when the degree of movement of the apparatus in acquiring said image exceeds a threshold, and for selectively transferring said one or more of said images acquired by said sensor to said image store; and

a dedicated motion extractor hardware unit for providing hardware-based determination of motion parameters of at least one a selected image stored in said image store.

41. The one or more dedicated hardware units of claim 40, wherein exposure times of two or more of said sequence of images differ based on different degrees of movement of the apparatus.

42. The one or more dedicated hardware units of claim 40, further comprising a dedicated image re-constructor hardware unit for providing hardware-based correction of at least one selected image with associated motion parameters.

43. The one or more dedicated hardware units of claim 42, further comprising a dedicated image merger hardware unit for providing hardware-based merging of a selected plurality of images including said at least one selected image corrected by said dedicated image re-constructor hardware unit, to produce a high quality image of said scene.

44. The one or more dedicated hardware units as claimed in claim 40, wherein the motion extractor is configured to estimate a point spread function (PSF) for said selected image.

45. The one or more dedicated hardware units as claimed in claim 44, wherein said dedicated motion extractor hardware unit is configured to:

calculate a Cepstrum for said selected image;

identify one or more spikes in said Cepstrum; and

select one of said spikes in said Cepstrum as an end point for said PSF.

46. The one or more dedicated hardware units as claimed in claim 45, wherein said dedicated motion extractor hardware unit is configured to calculate a negative Cepstrum, and to set points in said negative Cepstrum having a value less than a threshold to zero.

47. The one or more dedicated hardware units as claimed in claim 40, wherein said image store comprises a temporary image store, and wherein said apparatus further comprises a non-volatile memory, said dedicated motion detector hardware unit being configured to store said one or more of said images in said non-volatile memory.

48. The one or more dedicated hardware units as claimed in claim 40, wherein said dedicated motion detector hardware unit comprises a gyro-sensor or an accelerometer, or both.

59. The one or more dedicated hardware units of claim 40, wherein said different degrees of movement comprise different degrees of non linear movement.

61. The one or more dedicated hardware units of claim 40, wherein said dedicated image re-constructor hardware unit further for enhancing an exposure of the at least one selected image.

49. One or more dedicated hardware units for installation within a digital image acquisition apparatus having an image acquisition sensor coupled to imaging optics for acquiring a sequence of images, and an image store for storing one or more of said sequence of images acquired by said sensor, the one or more dedicated hardware units comprising:

a dedicated motion detector hardware unit for providing hardware-based control of said sensor to cease capture of an image when the degree of movement of the apparatus in acquiring said image exceeds a first threshold, and for controlling the sensor to restart capture when the degree of movement of the apparatus is less than a given second threshold and for selectively transferring said one or more of said sequence of images acquired by said sensor to said image store; and

a dedicated motion extractor hardware unit for determining motion parameters of at least one a selected image stored in said image store.

50. The one or more dedicated hardware units of claim 49, wherein exposure times of two or more of said sequence of images differ based on different degrees of movement of the apparatus.

51. The one or more dedicated hardware units of claim 49, further comprising a dedicated image re-constructor hardware unit for providing hard-based correction of at least one selected image with associated motion parameters.

52. The one or more dedicated hardware units of claim 51, further comprising a dedicated image merger hardware unit for providing hardware-based merging of a selected plurality of images including said at least one selected image corrected by said image re-constructor, to produce a high quality image of said scene.

53. The one or more dedicated hardware units as claimed in claim 49, further comprising a first exposure timer for storing an aggregate exposure time of said sequence of images, and wherein said apparatus is configured to acquire said sequence of images until the aggregate exposure time of at least a stored number of said sequence of images exceeds a predetermined exposure time for said high quality image.

54. The one or more dedicated hardware units as claimed in claim 53, further comprising a second timer for storing an exposure time for a single image, and wherein said apparatus is configured to dispose of an image having an exposure time less than a threshold time.

55. The one or more dedicated hardware units as claimed in claim 49, further comprising an image quality analyzer for a single image, and wherein said apparatus is configured to dispose of an image having a quality less than a given threshold quality.

56. The one or more dedicated hardware units as claimed in claim 49, wherein said dedicated image merger hardware unit is configured to align said selected plurality of images prior to merging said images.

57. The one or more dedicated hardware units as claimed in claim 49, wherein said first and second thresholds comprise threshold amounts of motion energy.

58. The one or more dedicated hardware units of claim 49, wherein said different degrees of movement comprise different degrees of non linear movement.

60. The one or more dedicated hardware units of claim 49, wherein said dedicated image re-constructor hardware unit further for enhancing an exposure of the at least one selected image.

62. One or more dedicated hardware units for installation within a digital image acquisition apparatus having an image acquisition sensor coupled to imaging optics for acquiring a sequence of images, and an image store for storing one or more of said sequence of images acquired by said sensor, the one or more dedicated hardware units comprising:

a dedicated motion detector hardware unit for providing hardware-based control of said sensor to cease capture of an image when the degree of movement of the apparatus in acquiring said image exceeds a threshold;

a controller for selectively transferring said one or more of said images acquired by said sensor to said image store; and

a dedicated motion extractor hardware unit for providing hardware-based determination of motion parameters of a selected image stored in said image store;

63. The one or more dedicated hardware units of claim 62, wherein the dedicated motion extractor hardware unit is configured to:

calculate a Cepstrum for said selected image;

identify one or more spikes in said Cepstrum; and

select one of said spikes in said Cepstrum as an end point for said PSF.

64. The one or more dedicated hardware units of claim 62, further comprising a dedicated image re-constructor hardware unit for providing hardware-based correction of a selected image with associated motion parameters.

65. The one or more dedicated hardware units of claim 64, further comprising a dedicated image merger hardware unit for providing hardware-based merging of a selected plurality of images nominally of the same scene and corrected by said image re-constructor to produce a high quality image of said scene.

66. The one or more dedicated hardware units as claimed in claim 62, wherein the dedicated motion extractor hardware unit is configured to estimate a point spread function (PSF) for said selected image.

67. The one or more dedicated hardware units as claimed in claim 62, wherein said dedicated motion extractor hardware unit is configured to calculate a negative Cepstrum, and to set points in said negative Cepstrum having a value less than a threshold to zero.

68. The one or more dedicated hardware units as claimed in claim 62, wherein said image stored comprises a temporary image store, and wherein said apparatus further comprises a non-volatile memory, said dedicated image merger hardware unit being configured to store said high quality image in said non-volatile memory.

69. The one or more dedicated hardware units as claimed in claim 62, wherein said dedicated motion detector hardware unit comprises a gyro-sensor or an accelerometer, or both.

70. One or more dedicated hardware units for installation within a digital image acquisition apparatus having an image acquisition sensor coupled to imaging optics for acquiring a sequence of images, and an image store for storing one or more of said sequence of images acquired by said sensor, the one or more dedicated hardware units comprising:

a dedicated motion detector hardware unit for providing hardware-based control of said sensor to cease capture of an image when the degree of movement of the apparatus in acquiring said image exceeds a first threshold;

one or more controllers that causes the sensor to restart capture when the degree of movement of the apparatus is less than a given second threshold and that selectively transfers said one or more of said images acquired by said sensor to said image store;

a dedicated motion extractor hardware unit for providing hardware-based determination of motion parameters of a selected image stored in said image store;

71. The one or more dedicated hardware units of claim 70, wherein the dedicated motion extractor hardware unit is configured to:

calculate a Cepstrum for said selected image;

identify one or more spikes in said Cepstrum; and

select one of said spikes in said Cepstrum as an end point for said PSF.

72. The one or more dedicated hardware units of claim 70, further comprising a dedicated image re-constructor hardware unit for providing hardware-based correction of a selected image with associated motion parameters.

73. The one or more dedicated hardware units of claim 72, further comprising a dedicated image merger hardware unit for providing hardware-based merging of a selected plurality of images nominally of the same scene and corrected by said dedicated image re-constructor hardware unit to produce a high quality image of said scene.

74. The one or more dedicated hardware units as claimed in claim 70, further comprising a first exposure timer for storing an aggregate exposure time of said sequence of images, and wherein said apparatus is configured to acquire said sequence of images until the aggregate exposure time of at least a stored number of said sequence of images exceeds a predetermined exposure time for said high quality image.

75. The one or more dedicated hardware units as claimed in claim 74, further comprising a second timer for storing an exposure time for a single image, and wherein said apparatus is configured to dispose of an image having an exposure time less than a threshold time.

76. The one or more dedicated hardware units as claimed in claim 74, further comprising an image quality analyzer for a single image, and wherein said apparatus is configured to dispose of an image having a quality less than a given threshold quality.

77. The one or more dedicated hardware units as claimed in claim 70, wherein said dedicated image merger hardware unit is configured to align said selected plurality of images prior to merging said images.

78. The one or more dedicated hardware units as claimed in claim 70, wherein said first and second thresholds comprise threshold amounts of motion energy.