Image acquisition method and apparatus

• US 8,520,082 B2
• Filed: 10/19/2010
• Issued: 08/27/2013
• Est. Priority Date: 06/05/2006
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Corcoran, Peter, Steinberg, Eran, Bigioi, Petronel, Drimbarean, Alexandru, Zamfir, Adrian, Florea, Corneliu
• Primary Examiner(s): CUTLER, ALBERT H
• Application Number:
  US12/907,921
• Publication Number:
  US 20110115928A1
• Time in Patent Office: 1,043 Days
• Field of Search: 348/208.2, 348/208.4, 348/208.5, 348/208.13, 348/208.16, 348/362, 348/364, 348/367
• US Class Current: 348/208.13
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/681 Motion detection
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken

Abstract

An image acquisition sensor of a digital image acquisition apparatus is coupled to imaging optics for acquiring a sequence of images. Images acquired by the sensor are stored. A motion detector causes the sensor to cease capture of an image when the degree of movement in acquiring the image exceeds a threshold. A controller selectively transfers acquired images for storage. A motion extractor determines motion parameters of a selected, stored image. An image re-constructor corrects the selected image with associated motion parameters. A selected plurality of images nominally of the same scene are merged and corrected by the image re-constructor to produce a high quality image of the scene.

Claims

1. A digital image acquisition apparatus, comprising:

an image acquisition sensor coupled to imaging optics for acquiring a sequence of images;

an image store for storing one or more of said sequence of images acquired by said sensor;

a dedicated motion detector hardware unit for providing hardware-based sensor control including causing said sensor to cease capture of an image when the degree of movement of the apparatus in acquiring said image exceeds a threshold, and for selectively transferring said one or more of said images acquired by said sensor to said image store;

a temporary image cache for receiving image data directly from said image acquisition sensor; and

a dedicated motion extractor hardware unit configured for directly accessing said image data received at said temporary image cache and for providing hardware-based determination of motion parameters of at least one selected image stored in said temporary cache based on analyzing said image data received at said temporary image cache,

wherein the dedicated motion extractor hardware unit is configured to

estimate a point spread function (PSF) for said selected image; and

to

calculate a Cepstrum for said selected image;

identify one or more spikes in said Cepstrum; and

select one of said spikes in said Cepstrum as an end point for said PSF.

2. The apparatus of claim 1, wherein exposure times of two or more of said sequence of images differ based on different degrees of movement of the apparatus.

3. The apparatus of claim 1, further comprising a dedicated image re-constructor hardware unit for providing hardware-based correction of at least one selected image with associated motion parameters.

4. The apparatus of claim 3, further comprising a dedicated image merger hardware unit for providing hardware-based merging of a selected plurality of images including said at least one selected image corrected by said dedicated image re-constructor hardware unit, to produce a high quality image of said scene.

5. The apparatus of claim 3, wherein said dedicated image re-constructor hardware unit is configured for enhancing an exposure of the at least one selected image.

6. An apparatus as claimed in claim 1, wherein said dedicated motion extractor hardware unit is configured to calculate a negative Cepstrum, and to set points in said negative Cepstrum having a value less than a threshold to zero.

7. An apparatus as claimed in claim 1, wherein said image store comprises a temporary image store, and wherein said apparatus further comprises a non-volatile memory, said dedicated motion detector hardware unit being configured to store said one or more of said images in said non-volatile memory.

8. An apparatus as claimed in claim 1, wherein said dedicated motion detector hardware unit comprises a gyro-sensor or an accelerometer, or both.

9. The apparatus of claim 1, wherein said different degrees of movement comprise different degrees of non linear movement.

10. A digital image acquisition apparatus, comprising:

an image acquisition sensor coupled to imaging optics for acquiring a sequence of images;

an image store for storing one or more of said sequence of images acquired by said sensor;

a dedicated motion detector hardware unit for providing hardware-based control of said sensor to cease capture of an image when the degree of movement of the apparatus in acquiring said image exceeds a first threshold, and for controlling the sensor to restart capture when the degree of movement of the apparatus is less than a given second threshold and for selectively transferring said one or more of said sequence of images acquired by said sensor to said image store;

a temporary image cache for receiving image data directly from said image acquisition sensor; and

a dedicated motion extractor hardware unit configured for directly accessing said image data received at said temporary image cache and for providing hardware-based determination of motion parameters of at least one selected image stored in said temporary cache based on analyzing said image data received at said temporary image cache,

wherein the dedicated motion extractor hardware unit is configured to

estimate a point spread function (PSF) for said selected image; and

to

calculate a Cepstrum for said selected image;

identify one or more spikes in said Cepstrum; and

select one of said spikes in said Cepstrum as an end point for said PSF.

11. The apparatus of claim 10, wherein exposure times of two or more of said sequence of images differ based on different degrees of movement of the apparatus.

12. The apparatus of claim 10, further comprising a dedicated image re-constructor hardware unit for providing hardware-based correction of at least one selected image with associated motion parameters.

13. The apparatus of claim 12, further comprising a dedicated image merger hardware unit for providing hardware-based merging of a selected plurality of images including said at least one selected image corrected by said image re-constructor, to produce a high quality image of said scene.

14. The apparatus as claimed in claim 13, further comprising a first exposure timer for storing an aggregate exposure time of said sequence of images, and wherein said apparatus is configured to acquire said sequence of images until the aggregate exposure time of at least a stored number of said sequence of images exceeds a predetermined exposure time for said high quality image.

15. An apparatus as claimed in claim 14, further comprising a second timer for storing an exposure time for a single image, and wherein said apparatus is configured to dispose of an image having an exposure time less than a threshold time.

16. An apparatus as claimed in claim 13, wherein said dedicated image merger hardware unit is configured to align said selected plurality of images prior to merging said images.

17. The apparatus of claim 12, wherein said dedicated image re-constructor hardware unit is configured for enhancing an exposure of the at least one selected image.

18. An apparatus as claimed in claim 10, further comprising an image quality analyzer for a single image, and wherein said apparatus is configured to dispose of an image having a quality less than a given threshold quality.

19. An apparatus as claimed in claim 10, wherein said first and second thresholds comprise threshold amounts of motion energy.

20. The apparatus of claim 10, wherein said different degrees of movement comprise different degrees of non linear movement.

21. A digital image acquisition apparatus, comprising:

an image acquisition sensor coupled to imaging optics for acquiring a sequence of images;

an image store for storing one or more of said images acquired by said sensor;

a dedicated motion detector hardware unit for providing hardware-based control of said sensor to cease capture of an image when the degree of movement of the apparatus in acquiring said image exceeds a threshold;

a controller for selectively transferring said one or more of said images acquired by said sensor to said image store;

a temporary image cache for receiving image data directly from said image acquisition sensor; and

a dedicated motion extractor hardware unit configured for directly accessing said image data received at said temporary image cache and for providing hardware-based determination of motion parameters of at least one selected image stored in said temporary cache based on analyzing said image data received at said temporary image cache,

wherein the dedicated motion extractor hardware unit is configured to;

calculate a Cepstrum for said selected image;

identify one or more spikes in said Cepstrum; and

select one of said spikes in said Cepstrum as an end point for said hardware-based determination of motion parameters.

22. The apparatus of claim 21, further comprising a dedicated image re-constructor hardware unit for providing hardware-based correction of a selected image with associated motion parameters.

23. The apparatus of claim 22, further comprising a dedicated image merger hardware unit for providing hardware-based merging of a selected plurality of images nominally of the same scene and corrected by said image re-constructor to produce a high quality image of said scene.

24. An apparatus as claimed in claim 23, wherein said image store comprises a temporary image store, and wherein said apparatus further comprises a non-volatile memory, said dedicated image merger hardware unit being configured to store said high quality image in said nonvolatile memory.

25. An apparatus as claimed in claim 21, wherein the dedicated motion extractor hardware unit is configured to estimate a point spread function (PSF) for said selected image.

26. An apparatus as claimed in claim 21, wherein said dedicated motion extractor hardware unit is configured to calculate a negative Cepstrum, and to set points in said negative Cepstrum having a value less than a threshold to zero.

27. An apparatus as claimed in claim 21, wherein said dedicated motion detector hardware unit comprises a gyro-sensor or an accelerometer, or both.

28. A digital image acquisition apparatus, comprising:

an image acquisition sensor coupled to imaging optics for acquiring a sequence of images;

an image store for storing one or more of said images acquired by said sensor;

a dedicated motion detector hardware unit for providing hardware-based control of said sensor to cease capture of an image when the degree of movement of the apparatus in acquiring said image exceeds a first threshold;

one or more controllers that causes the sensor to restart capture when the degree of movement of the apparatus is less than a given second threshold and that selectively transfers said one or more of said images acquired by said sensor to said image store;

a temporary image cache for receiving image data directly from said image acquisition sensor; and

a dedicated motion extractor hardware unit configured for directly accessing said image data received at said temporary image cache and for providing hardware-based determination of motion parameters of at least one selected image stored in said temporary cache based on analyzing said image data received at said temporary image cache,

wherein the dedicated motion extractor hardware unit is configured to;

calculate a Cepstrum for said selected image;

identify one or more spikes in said Cepstrum; and

select one of said spikes in said Cepstrum as an end point for said hardware-based determination of motion parameters.

29. The apparatus of claim 28, further comprising a dedicated image re-constructor hardware unit for providing hardware-based correction of a selected image with associated motion parameters.

30. The apparatus of claim 29, further comprising a dedicated image merger hardware unit for providing hardware-based merging of a selected plurality of images nominally of the same scene and corrected by said dedicated image re-constructor hardware unit to produce a high quality image of said scene.

31. An apparatus as claimed in claim 30, further comprising a first exposure timer for storing an aggregate exposure time of said sequence of images, and wherein said apparatus is configured to acquire said sequence of images until the aggregate exposure time of at least a stored number of said sequence of images exceeds a predetermined exposure time for said high quality image.

32. An apparatus as claimed in claim 31, further comprising a second timer for storing an exposure time for a single image, and wherein said apparatus is configured to dispose of an image having an exposure time less than a threshold time.

33. An apparatus as claimed in claim 31, further comprising an image quality analyzer for a single image, and wherein said apparatus is configured to dispose of an image having a quality less than a given threshold quality.

34. An apparatus as claimed in claim 30, wherein said dedicated image merger hardware unit is configured to align said selected plurality of images prior to merging said images.

35. An apparatus as claimed in claim 28, wherein said first and second thresholds comprise threshold amounts of motion energy.

36. One or more dedicated hardware units for installation within a digital image acquisition apparatus having an image acquisition sensor coupled to imaging optics for acquiring a sequence of images, a temporary image cache for receiving image data directly from said image acquisition sensor, and an image store for storing one or more of said sequence of images acquired by said sensor, the one or more dedicated hardware units comprising:

a dedicated motion detector hardware unit for providing hardware-based sensor control including causing said sensor to cease capture of an image when the degree of movement of the apparatus in acquiring said image exceeds a threshold, and for selectively transferring said one or more of said images acquired by said sensor to said image store; and

a dedicated motion extractor hardware unit configured for directly accessing said image data received at said temporary image cache and for providing hardware-based determination of motion parameters of at least one selected image stored in said temporary cache based on analyzing said image data received at said temporary image cache, wherein said dedicated motion extractor hardware unit is configured to;

estimate a point spread function (PSF) for said selected image; and

to

calculate a Cepstrum for said selected image;

identify one or more spikes in said Cepstrum; and

select one of said spikes in said Cepstrum as an end point for said PSF.

37. The one or more dedicated hardware units of claim 36, wherein exposure times of two or more of said sequence of images differ based on different degrees of movement of the apparatus.

38. The one or more dedicated hardware units of claim 36, further comprising a dedicated image re-constructor hardware unit for providing hardware-based correction of at least one selected image with associated motion parameters.

39. The one or more dedicated hardware units of claim 38, further comprising a dedicated image merger hardware unit for providing hardware-based merging of a selected plurality of images including said at least one selected image corrected by said dedicated image reconstructor hardware unit, to produce a high quality image of said scene.

40. The one or more dedicated hardware units of claim 38, wherein said dedicated image reconstructor hardware unit is configured for enhancing an exposure of the at least one selected image.

41. The one or more dedicated hardware units as claimed in claim 36, wherein said dedicated motion extractor hardware unit is configured to calculate a negative Cepstrum, and to set points in said negative Cepstrum having a value less than a threshold to zero.

42. The one or more dedicated hardware units as claimed in claim 36, wherein said image store comprises a temporary image store, and wherein said apparatus further comprises a nonvolatile memory, said dedicated motion detector hardware unit being configured to store said one or more of said images in said non-volatile memory.

43. The one or more dedicated hardware units as claimed in claim 36, wherein said dedicated motion detector hardware unit comprises a gyro-sensor or an accelerometer, or both.

44. The one or more dedicated hardware units of claim 36, wherein said different degrees of movement comprise different degrees of non linear movement.

45. One or more dedicated hardware units for installation within a digital image acquisition apparatus having an image acquisition sensor coupled to imaging optics for acquiring a sequence of images, a temporary image cache for receiving image data directly from an image acquisition sensor, and an image store for storing one or more of said sequence of images acquired by said sensor, the one or more dedicated hardware units comprising:

a dedicated motion detector hardware unit for providing hardware-based control of said sensor to cease capture of an image when the degree of movement of the apparatus in acquiring said image exceeds a first threshold, and for controlling the sensor to restart capture when the degree of movement of the apparatus is less than a given second threshold and for selectively transferring said one or more of said sequence of images acquired by said sensor to said image store; and

a dedicated motion extractor hardware unit configured for directly accessing said image data received at said temporary image cache and for providing hardware-based determination of motion parameters of at least one selected image stored in said temporary cache based on analyzing said image data received at said temporary image cache,

wherein the dedicated motion extractor hardware unit is configured to;

calculate a Cepstrum for said selected image;

identify one or more spikes in said Cepstrum; and

select one of said spikes in said Cepstrum as an end point for said hardware-based determination of motion parameters.

46. The one or more dedicated hardware units of claim 45, wherein exposure times of two or more of said sequence of images differ based on different degrees of movement of the apparatus.

47. The one or more dedicated hardware units of claim 45, further comprising a dedicated image re-constructor hardware unit is configured for providing hardware-based correction of at least one selected image with associated motion parameters.

48. The one or more dedicated hardware units of claim 47, further comprising a dedicated image merger hardware unit for providing hardware-based merging of a selected plurality of images including said at least one selected image corrected by said image re-constructor, to produce a high quality image of said scene.

49. The one or more dedicated hardware units as claimed in claim 48, further comprising a first exposure timer for storing an aggregate exposure time of said sequence of images, and wherein said apparatus is configured to acquire said sequence of images until the aggregate exposure time of at least a stored number of said sequence of images exceeds a predetermined exposure time for said high quality image.

50. The one or more dedicated hardware units as claimed in claim 49, further comprising a second timer for storing an exposure time for a single image, and wherein said apparatus is configured to dispose of an image having an exposure time less than a threshold time.

51. The one or more dedicated hardware units as claimed in claim 48, wherein said dedicated image merger hardware unit is configured to align said selected plurality of images prior to merging said images.

52. The one or more dedicated hardware units of claim 47, wherein said dedicated image reconstructor hardware unit is configured for enhancing an exposure of the at least one selected image.

53. The one or more dedicated hardware units as claimed in claim 45, further comprising an image quality analyzer for a single image, and wherein said apparatus is configured to dispose of an image having a quality less than a given threshold quality.

54. The one or more dedicated hardware units as claimed in claim 45, wherein said first and second thresholds comprise threshold amounts of motion energy.

55. The one or more dedicated hardware units of claim 45, wherein said different degrees of movement comprise different degrees of non linear movement.

56. One or more dedicated hardware units for installation within a digital image acquisition apparatus having an image acquisition sensor coupled to imaging optics for acquiring a sequence of images, a temporary image cache for receiving image data directly from an image acquisition sensor, and an image store for storing one or more of said sequence of images acquired by said sensor, the one or more dedicated hardware units comprising:

a dedicated motion detector hardware unit for providing hardware-based control of said sensor to cease capture of an image when the degree of movement of the apparatus in acquiring said image exceeds a threshold;

a controller for selectively transferring said one or more of said images acquired by said sensor to said image store; and

a dedicated motion extractor hardware unit configured for directly accessing said image data received at said temporary image cache and for providing hardware-based determination of motion parameters of at least one selected image stored in said temporary cache based on analyzing said image data received at said temporary image cache,

wherein the dedicated motion extractor hardware unit is configured to;

calculate a Cepstrum for said selected image;

identify one or more spikes in said Cepstrum; and

select one of said spikes in said Cepstrum as an end point for said hardware-based determination of motion parameters.

57. The one or more dedicated hardware units of claim 56, further comprising a dedicated image re-constructor hardware unit for providing hardware-based correction of a selected image with associated motion parameters.

58. The one or more dedicated hardware units of claim 57, further comprising a dedicated image merger hardware unit for providing hardware-based merging of a selected plurality of images nominally of the same scene and corrected by said image re-constructor to produce a high quality image of said scene.

59. The one or more dedicated hardware units as claimed in claim 58, wherein said image store comprises a temporary image store, and wherein said apparatus further comprises a nonvolatile memory, said dedicated image merger hardware unit being configured to store said high quality image in said non-volatile memory.

60. The one or more dedicated hardware units as claimed in claim 56, wherein the dedicated motion extractor hardware unit is configured to estimate a point spread function (PSF) for said selected image.

61. The one or more dedicated hardware units as claimed in claim 56, wherein said dedicated motion extractor hardware unit is configured to calculate a negative Cepstrum, and to set points in said negative Cepstrum having a value less than a threshold to zero.

62. The one or more dedicated hardware units as claimed in claim 56, wherein said dedicated motion detector hardware unit comprises a gyro-sensor or an accelerometer, or both.

63. One or more dedicated hardware units for installation within a digital image acquisition apparatus having an image acquisition sensor coupled to imaging optics for acquiring a sequence of images, a temporary image cache for receiving image data directly from an image acquisition sensor, and an image store for storing one or more of said sequence of images acquired by said sensor, the one or more dedicated hardware units comprising:

a dedicated motion detector hardware unit for providing hardware-based control of said sensor to cease capture of an image when the degree of movement of the apparatus in acquiring said image exceeds a first threshold;

one or more controllers that causes the sensor to restart capture when the degree of movement of the apparatus is less than a given second threshold and that selectively transfers said one or more of said images acquired by said sensor to said image store; and

a dedicated motion extractor hardware unit configured for directly accessing said image data received at said temporary image cache and for providing hardware-based determination of motion parameters of at least one selected image stored in said temporary cache based on analyzing said image data received at said temporary image cache,

wherein the dedicated motion extractor hardware unit is configured to;

calculate a Cepstrum for said selected image;

identify one or more spikes in said Cepstrum; and

select one of said spikes in said Cepstrum as an end point for said hardware-based determination of motion parameters.

64. The one or more dedicated hardware units of claim 63, further comprising a dedicated image re-constructor hardware unit for providing hardware-based correction of a selected image with associated motion parameters.

65. The one or more dedicated hardware units of claim 64, further comprising a dedicated image merger hardware unit for providing hardware-based merging of a selected plurality of images nominally of the same scene and corrected by said dedicated image re-constructor hardware unit to produce a high quality image of said scene.

66. The one or more dedicated hardware units as claimed in claim 65, further comprising a first exposure timer for storing an aggregate exposure time of said sequence of images, and wherein said apparatus is configured to acquire said sequence of images until the aggregate exposure time of at least a stored number of said sequence of images exceeds a predetermined exposure time for said high quality image.

67. The one or more dedicated hardware units as claimed in claim 66, further comprising a second timer for storing an exposure time for a single image, and wherein said apparatus is configured to dispose of an image having an exposure time less than a threshold time.

68. The one or more dedicated hardware units as claimed in claim 66, further comprising an image quality analyzer for a single image, and wherein said apparatus is configured to dispose of an image having a quality less than a given threshold quality.

69. The one or more dedicated hardware units as claimed in claim 65, wherein said dedicated image merger hardware unit is configured to align said selected plurality of images prior to merging said images.

70. The one or more dedicated hardware units as claimed in claim 63, wherein said first and second thresholds comprise threshold amounts of motion energy.