Image stabilization techniques

• US 9,232,138 B1
• Filed: 06/03/2013
• Issued: 01/05/2016
• Est. Priority Date: 06/03/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Baldwin, Leo Benedict
• Primary Examiner(s): Tran, Sinh
• Assistant Examiner(s): Wu, Zhenzhen
• Application Number:
  US13/908,794
• Time in Patent Office: 946 Days
• Field of Search: 348/208.99, 348/208.2, 348/208.4, 348/208.5, 348/208.6, 348/208.7, 348/208.11, 396/55
• US Class Current: 1/1
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 23/685 performed by mechanical compensation

Abstract

Approaches to enable a computing device, such as a phone or tablet computer, to apply electronic image stabilization (EIS) and optical image stabilization (OIS) across their respective ranges of effectiveness. The computing device may include a gyroscope that produces data about the orientation or movement of the device and a filter that filters the orientation data and conveys the filtered data to either the OIS system or the EIS system, depending on the amplitude of the excursion. The filter conveys excursions of a first range of amplitude to be corrected by the OIS system and conveys excursions of a second range of amplitude (the portion that is uncorrected by the OIS) to be corrected by the EIS system.

Claims

1. A computing device, comprising:

a gyroscope configured to produce orientation data;

a camera including a lens and an image sensor, the camera operable to capture an image;

an electronic image stabilization (EIS) system operable to shift a frame of a video by selecting a subset of pixels of the image sensor to compensate for jitter between sequential frames of the video;

an optical image stabilization (OIS) system operable to align an optical axis of the camera by physically moving one or more of;

the camera, the image sensor, or the lens; and

a filter having connections to the gyroscope, the EIS system and the OIS system, the filter configured to;

receive the orientation data from the gyroscope,

determine that the orientation data corresponds to an excursion with an amplitude;

determine that the amplitude is between a first threshold and a second threshold;

determine a proximity of the amplitude to the second threshold relative to the first threshold; and

apply a transfer function to convey the orientation data to the OIS system to align the optical axis of the camera to compensate for a first portion of the excursion, and convey the orientation data to the EIS system to cause the EIS system to shift the frame of the video within the image sensor to compensate for a second portion of the excursion, the transfer function conveying a progressively larger portion of the excursion to be compensated to the EIS system relative to the OIS system based on the proximity of the amplitude to the second threshold relative to the first threshold.

2. The computing device of claim 1, wherein the filter is a digital signal processor (DSP) that is coupled to a driver of the OIS system and a driver of the EIS system.

3. The computing device of claim 1, wherein the first portion and the second portion are further determined based at least in part on a frequency of the excursion.

4. A computer implemented method, comprising:

under control of one or more computer systems configured with executable instructions,

receiving orientation data;

determining, based on the orientation data, information about an excursion associated with a camera of a computing device, the excursion having an amplitude;

determining that the information about the excursion corresponds to a transition range including a first threshold and a second threshold;

applying a transfer function to compensate for a first portion of the excursion by applying an optical image stabilization (OIS) function to the camera and a second portion of the excursion by applying an electronic image stabilization (EIS) function to the camera, the transfer function conveying a progressively larger portion of the excursion to be compensated to the EIS function relative to the OIS function based on a proximity of the amplitude to the second threshold relative to the first threshold; and

capturing an image using the camera.

5. The computer implemented method of claim 4, wherein the information about the excursion further comprises a frequency of the excursion.

6. The computer implemented method of claim 4, wherein the first threshold is substantially within an effective range of the OIS function and the second threshold is outside the effective range of the OIS function.

7. The computer implemented method of claim 4, wherein the EIS function shifts a frame of a video within an image sensor to compensate for jitter between a plurality of sequential frames of the video.

8. The computer implemented method of claim 4, wherein the camera further includes a lens and an image sensor, and wherein the OIS function adjusts an optical axis of the camera by physically moving one or more of:

the camera, the image sensor, or the lens.

9. The computer implemented method of claim 4, wherein the first portion of the excursion is blocked in firmware associated with the EIS function such that the EIS function is not applied to the first portion of the excursion.

10. The computer implemented method of claim 4, wherein the orientation data is produced by a gyroscope embedded in the computing device.

11. The computer implemented method of claim 4, wherein the orientation data is filtered by a digital signal processor (DSP) embedded in the computing device and coupled to a gyroscope, wherein the DSP is configured to:

convey the orientation data to the OIS function; and

convey the orientation data to the EIS function.

12. The computer implemented method of claim 4, wherein applying the OIS function further comprises:

determining at least one of;

an amount of ambient light present in an environment around the camera or a shutter speed of the camera; and

controlling the OIS function based at least in part on the amount of the ambient light or the shutter speed of the camera.

13. The computer implemented method of claim 12, wherein controlling the OIS function further comprises:

enabling the OIS function if the amount of the ambient light is less than a defined threshold or disabling the OIS function if the amount of the ambient light is more than the defined threshold.

14. The computer implemented method of claim 12, wherein controlling the OIS function further comprises:

adjusting gain of the OIS function based at least in part on the amount of the ambient light or the shutter speed of the camera.

15. A non-transitory computer readable storage medium storing one or more sequences of instructions that, when executed by at least one processor, cause the at least one processor to:

receive orientation data;

determine, based on the orientation data, information about an excursion associated with a camera of a computing device, the excursion having an amplitude;

determine that the information about the excursion corresponds to a transition range including a first threshold and a second threshold;

apply a transfer function to compensate for a first portion of the excursion by applying an optical image stabilization (OIS) function to the camera and a second portion of the excursion by applying an electronic image stabilization (EIS) function to the camera, the transfer function conveying a progressively larger portion of the excursion to be compensated to the EIS function relative to the OIS function based on a proximity of the amplitude to the second threshold relative to the first threshold; and

capture an image using the camera.

16. The non-transitory computer readable storage medium of claim 15, wherein the information about the excursion further comprises a frequency of the excursion.

17. The non-transitory computer readable storage medium of claim 15, wherein the first threshold is substantially within an effective range of the OIS function and the second threshold is outside the effective range of the OIS function.

18. The non-transitory computer readable storage medium of claim 15, wherein the EIS function shifts a frame of a video within an image sensor to compensate for jitter between a plurality of sequential frames of the video.

19. The non-transitory computer readable storage medium of claim 15, wherein the camera further includes a lens and an image sensor, and wherein the OIS function adjusts an optical axis of the camera by physically moving one or more of:

the camera, the image sensor, or the lens.

20. The non-transitory computer readable storage medium of claim 15, wherein the first portion of the excursion is blocked in firmware associated with the EIS function such that the EIS function is not applied to the first portion of the excursion.

21. The non-transitory computer readable storage medium of claim 15, wherein the orientation data is produced by a gyroscope embedded in the computing device.