Method and system for image construction using multiple exposures

• US RE46,239 E1
• Filed: 05/29/2013
• Issued: 12/13/2016
• Est. Priority Date: 06/22/2006
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Conversant Wireless Licensing S.à.r.l. (f/k/a Core Wireless Licensing S.a.r.l.) (MOSAID Technologies Inc. (f/k/a Conversant Intellectual Property Management))
• Inventors: Kakkori, Hannu
• Primary Examiner(s): KHAN, USMAN A
• Application Number:
  US13/904,351
• Time in Patent Office: 1,294 Days
• Field of Search: 348/208.6
• US Class Current: 1/1
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 5/23232 by using more than one image in order to influence resolution, frame rate or aspect ratio providing high dynamic range image H04N5/2355
• CPC Class Codes: H04N 5/23248 for stable pick-up of the scene in spite of camera body vibration image-sensor selective scanning per se H04N3/1562
• CPC Class Codes: H04N 5/23258 based on additional sensors cameras when not peculiar to the use or presence of the EIS G03B2217/005
• CPC Class Codes: H04N 5/23277 by combination of a plurality of images sequentially taken

Abstract

A motion sensor is used to sense the movement of the camera during an exposure period. The camera has an image sensor to form one or more exposures. When the movement is within a certain range, the exposures are used to provide one or more frames so that an image can be constructed based on the frames. In one embodiment, the exposure period is divided into several short intervals in order to capture several image frames and only the image frames captured when the position of the camera is within a predetermined range are used to form the final image. The exposure time for each frame is small in order to reduce the motion blur degradation of the individual frames. If the camera is stable and substantially stationary relative to the scene, then all or many of the shorter frames are used to form the final image.

Claims

1. A method comprising:

exposing a projected image on an image sensor of an imaging system during at least part of an exposure period for attaining at least two exposures;

sensing movement of the imaging system, during the exposure period, in order to determine an amount of movement of the imaging system relative to an initial position of the imaging system in the exposure period; and

constructing an image based on at least a first exposure and a second exposure, wherein the first exposure is attained during a first time period, within the exposure period, in which the determined amount of movement of the imaging system is within a predetermined movement range relative to the initial position of the imaging system, the second exposure is attained during a second time period, within the exposure period, in which the determined amount of movement of the imaging system is within the predetermined movement range relative to the initial position of the imaging system, and between the first time period and the second time period there is a third time period in which the determined amount of movement of the imaging system is outside the predetermined movement range relative to the initial position of the imaging system.

2. A method according to claim 1, wherein the first exposure and the second exposure, attained when the determined amount of movement of the imaging system is within the predetermined movement range relative to the initial position of the imaging system, form a single image frame during the exposure period, said method further comprising:

capturing the single image frame after the exposure period for constructing the acquired image.

3. A method according to claim 1, wherein the first exposure and the second exposure, attained when the determined amount of movement of the imaging system is within the predetermined movement range relative to the initial position of the imaging system, separately form at least two image frames during the exposure period, said method further comprising:

capturing the at least two image frames at least during the exposure period for constructing the acquired image.

4. A method according to claim 1, wherein the exposure period is divided into a plurality of shorter time periods and wherein the first exposure, attained during the first time period, forms a first image frame;

and the second exposure, attained during the second time period, forms a second image frame, said method further comprising;

capturing said first and second image frames during the exposure period; and

selecting at least the first image frame and the second image frame for use in constructing the image.

5. A method according to claim 1, further comprising

preventing the projected image from reaching the image sensor, during the third time period, when the determined amount of movement of the imaging system is outside the predetermined movement range relative to the initial position of the imaging system.

6. A method according to claim 1, wherein the image sensor comprises an array of pixels, each pixel having a pixel area, and the predetermined movement range is determined based on one or more pixel areas.

7. A method according to claim 1, wherein the imaging system comprises an optical system for providing the projected image on the image sensor, and the predetermined movement range is determined based on a focal distance of the optical system.

8. A method according to claim 1, wherein the predetermined movement range is determined based on brightness of at least part of light forming the projected image.

9. An imaging system comprising:

an image sensor configured to attain at least a first exposure and a second exposure during an exposure period;

a movement sensor configured to sense movement of the imaging system, during the exposure period, in order to enable an amount of movement of the imaging system to be determined, relative to an initial position of the imaging system in the exposure period; and

a processor configured to construct an image based on at least the first exposure and the second exposure, wherein the first exposure is attained during a first time period, within the exposure period, in which the determined amount of movement of the imaging system is within a predetermined movement range relative to the initial position of the imaging system, the second exposure is attained during a second time period, within the exposure period, in which the determined amount of movement of the imaging system is within the predetermined movement range relative to the initial position of the imaging system, and between the first time period and the second time period there is a third time period in which the determined amount of movement of the imaging system is outside the predetermined movement range relative to the initial position of the imaging system.

10. An imaging system according to claim 9, wherein the first exposure and the second exposure, attained when the determined amount of movement of the imaging system is within the predetermined movement range relative to the initial position of the imaging system, form a single image frame during the exposure period, and wherein the image is constructed based on the single image frame captured after the exposure period.

11. An imaging system according to claim 9, wherein the first exposure and the second exposure, attained when the determined amount of movement of the imaging system is within the predetermined movement range relative to the initial position of the imaging system, separately form at least two image frames during the exposure period, and wherein the image is constructed based on the at least two image frames captured at least during the exposure period.

12. An imaging system according to claim 9, wherein the image sensor is configured to divide the exposure period into a plurality of shorter time periods and the first exposure, attained during the first time period, forms a first image frame;

and the second exposure, attained during the second time period, forms a second image frame, and wherein the processor is configured to capture the first and second and third image frames during the exposure period, configured to select the first image frame and the second image frame for use in constructing the image.

13. An imaging system according to claim 9, further comprising

an optical system for providing a projected image on the image sensor so as to allow the image sensor to attain the first exposure and the second exposure during the exposure period.

14. An imaging system according to claim 13, further comprising

a shutter, positioned in relationship to the optical system, configured to prevent the projected image from reaching the image sensor during the third time period, when the determined amount of movement of the imaging system is outside the predetermined movement range relative to the initial position of the imaging system.

15. An imaging system according to claim 9, further comprising

an electronic circuit configured to prevent the image sensor from attaining the third exposure during the third time period, when the determined amount of movement of the imaging system is outside the predetermined movement range relative to the initial position of the imaging system.

16. An imaging system according to claim 9, wherein the movement sensor provides a signal to indicate that the determined amount of movement of the imaging system is within the predetermined movement range so as to cause the image sensor to attain said first exposure during the first time period and so as to cause the image sensor to attain said second exposure during the second time period.

17. An imaging system, comprising:

means for sensing an image;

means for exposing a projected image on the image sensing means during an exposure period so as to allow the image sensing means to attain at least two exposures;

means for sensing movement of the imaging system, during the exposure period, in order to determine an amount of movement of the imaging system relative to an initial position of the imaging system in the exposure period; and

means for constructing an image based on at least a first exposure and a second exposure, wherein the first exposure is attained during a first time period, within the exposure period, in which the determined amount of movement of the imaging system is within a predetermined movement range relative to the initial position of the imaging system, the second exposure is attained during a second time period, within the exposure period, in which the determined amount of movement of the imaging system is within the predetermined movement range relative to the initial position of the imaging system, and between the first time period and the second time period there is a third time period in which the determined amount of movement of the imaging system is outside the predetermined movement range relative to the initial position of the imaging system.

18. An imaging system according to claim 17, further comprising means for capturing at least two image frames for constructing the acquired image.

19. An imaging system according to claim 18, further comprising:

means for selecting the captured image frames for constructing the acquired image.

20. An imaging system according to claim 17, further comprising:

means for preventing the projected image from forming an exposure, during the third time period, when the determined amount of movement of the imaging system is outside the predetermined movement range relative to the initial position of the imaging system.

21. A method for constructing an image, comprising:

beginning an exposure cycle when a shutter button activation of an imaging system is detected;

sensing movement of the imaging system during the exposure cycle for obtaining an amount of movement of the imaging system relative to an initial position of the imaging system;

capturing a plurality of frames of a same scene during the exposure cycle;

for each captured frame of the plurality of captured frames;

determining whether a corresponding amount of movement of the imaging system during the capturing is within a predetermined movement range;

responsive to a determination that the corresponding amount of movement is within the predetermined movement range, labeling the captured frame of the same scene with a first identifier, the first identifier indicating that the captured frame is not to be discarded; and

responsive to a determination that the corresponding amount of movement is not within the predetermined movement range, labeling the captured frame of the same scene with a second identifier, the second identifier indicating that the captured frame is to be discarded; and

constructing an image of the same scene based on at least one captured frame of the same scene that is labeled with the first identifier after discarding at least one captured frame of the same scene that is labeled with the second identifier.

22. The method of claim 21, wherein a length of the exposure cycle varies with illumination.

23. The method of claim 21, wherein the amount of movement of the imaging system is determined using a subset of pixels on the image sensor.

24. The method of claim 21, wherein the amount of movement of the imaging system is determined by a motion sensing component.

25. A method for constructing an image, comprising:

beginning an exposure cycle when a shutter button activation of an imaging system is detected;

sensing movement of the imaging system during the exposure cycle for obtaining an amount of movement of the imaging system relative to a fixed reference point;

controlling the imaging system such that an image sensor is effectively exposed when the amount of movement of the imaging system is within a predetermined movement range relative to the fixed reference point, the image sensor is not effectively exposed when the amount of movement of the imaging system is not within the predetermined movement range relative to the fixed reference point, and the image sensor is effectively re-exposed when the amount of movement of the imaging system is back within the predetermined movement range relative to the fixed reference point; and

constructing an image based at least one exposure, each exposure obtained when the image sensor is one of exposed and re-exposed during the exposure cycle.

26. The method of claim 25, wherein a frame is captured for each exposure, and the image is constructed based on the captured frames.

27. The method of claim 25, wherein the image is constructed by accumulating pixel intensities that were accumulated during each exposure.

28. The method of claim 25, wherein the image sensor is not effectively exposed by controlling at least one of a mechanical shutter, an optical shutter, and at least one electronic element within the image sensor.

29. The method of claim 25, wherein a length of the exposure cycle varies with illumination.

30. The method of claim 25, wherein the amount of movement of the imaging system is determined using a subset of pixels on the image sensor.

31. The method of claim 25, wherein the amount of movement of the imaging system is determined by a motion sensing component.

32. An imaging system, comprising:

a movement sensor for sensing movement of the imaging system during an exposure cycle to obtain an amount of movement of the imaging system relative to an initial position of the imaging system, the initial position determined a beginning of the exposure cycle, and the exposure cycle beginning when a shutter button activation of the imaging system is detected;

an image sensor for capturing a plurality of frames of a same scene during the exposure cycle;

a processor is configured, for each captured frame of the plurality of captured frames, to;

determine whether a corresponding amount of movement of the imaging system during the capturing is within a predetermined movement range;

responsive to a determination that the corresponding amount of movement is within the predetermined movement range, label the captured frame of the same scene with a first identifier, the first identifier indicating that the captured frame is not to be discarded; and

responsive to a determination that the corresponding amount of movement is not within the predetermined movement range, label the captured frame of the same scene with a second identifier, the second identifier indicating that the captured frame is to be discarded; and

control an overall operation of the imaging system and construct an image of the same scene based on at least one captured frame of the same scene that is labeled with the first identifier after discarding at least one captured frame of the same scene that is labeled with the second identifier.

33. The imaging system of claim 32, wherein a length of the exposure cycle varies with illumination.

34. The imaging system of claim 32, wherein the amount of movement of the imaging system is determined using a subset of pixels on the image sensor.

35. The imaging system of claim 32, wherein the amount of movement of the imaging system is determined by a motion sensing component.

36. An imaging system, comprising:

a movement sensor for sensing movement of the imaging system during an exposure cycle to obtain an amount of movement of the imaging system relative to a fixed reference point, the exposure cycle beginning when a shutter button activation of the imaging system is detected;

a processor configured to;

control the imaging system such that that an image sensor is effectively exposed when the amount of movement of the imaging system relative to the fixed reference point is within a predetermined movement range, the image sensor is not effectively exposed when the amount of movement of the imaging system relative to the fixed reference point is not within the predetermined movement range, and the image sensor is effectively re-exposed when the amount of movement of the imaging system relative to the fixed reference point is back within the predetermined movement range, and

construct an image based at least one exposure, each exposure obtained when the image sensor is one of exposed and re-exposed during the exposure cycle.

37. The imaging system of claim 36, wherein a frame is captured for each exposure, and the processor constructs the image based on the captured frames.

38. The imaging system of claim 36, wherein the process constructs the image by accumulating pixel intensities that were accumulated during each exposure.

39. The imaging system of claim 36, wherein the image sensor is not effectively exposed by controlling at least one of a mechanical shutter, an optical shutter, and at least one electronic element within the image sensor.

40. The imaging system of claim 36, wherein a length of the exposure cycle varies with illumination.

41. The imaging system of claim 36, wherein the movement sensor determines the amount of movement of the imaging system based on a subset of pixels on the image sensor.

42. The imaging system of claim 36, wherein the amount of movement of the imaging system is determined by a motion sensing component.