Automated tripod detection and handling in video stabilization

• US 9,300,873 B2
• Filed: 06/22/2012
• Issued: 03/29/2016
• Est. Priority Date: 06/22/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Zhou, Jianping, Beysserie, Sebastien X.
• Primary Examiner(s): Tran, Sinh
• Assistant Examiner(s): Wang, Xi
• Application Number:
  US13/530,500
• Publication Number:
  US 20130342714A1
• Time in Patent Office: 1,376 Days
• Field of Search: 3482081-20813, 348/208.99, 348/219.1, 348/220.1, 396/13, 396 52- 55, 359554-557
• US Class Current: 1/1
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory

Abstract

An apparatus, method, and computer-readable medium for motion sensor-based video stabilization. A motion sensor may capture motion data of a video sequence. A controller may compute instantaneous motion of the camera for a current frame of the video sequence. The controller may compare the instantaneous motion to a threshold value representing a still condition and reduce a video stabilization strength parameter for the current frame if the instantaneous motion is less than the threshold value. A video stabilization unit may perform video stabilization on the current frame according to the frame'"'"'s strength parameter.

Claims

1. A video stabilization method, comprising:

obtaining motion data from a motion sensor different from an imaging sensor corresponding to motion of a camera;

estimating from the motion data and previous motion data from the motion sensor, instantaneous motion of a camera for a current frame of a video sequence;

comparing the instantaneous motion to a threshold value representing a relatively still condition;

reducing a video stabilization strength parameter for the current frame if the instantaneous motion is less than the threshold value, the video stabilization strength parameter defining an overall degree of video stabilization to be performed for the current frame,

wherein the video stabilization strength parameter is decreased by a first value if the instantaneous motion of the current frame is below the threshold value and the instantaneous motion of a preceding frame is equal to or above the threshold value,

the video stabilization strength parameter is decreased by a second value if the instantaneous motion of the current frame and the previous frame are below the threshold value, and wherein the first value is smaller than the second value;

and performing video stabilization on the current frame according to the frame'"'"'s video stabilization strength parameter.

2. The method of claim 1, wherein the instantaneous motion of the camera corresponds to a motion difference between the current frame and a previous frame.

3. The method of claim 1, wherein estimating the instantaneous motion of the camera includes determining a difference between instantaneous motion for the current frame and instantaneous motion of a frame preceding the current frame.

4. The method of claim 1, wherein if the instantaneous motion is less than the threshold value, pixels in one or more frames of the video sequence are processed to determine motion in the current frame.

5. The method of claim 1, wherein the motion data is derived from gyroscopic sensor data.

6. The method of claim 1, wherein computing the instantaneous motion includes filtering the motion data.

7. The method of claim 1, wherein the motion data is derived from accelerometer sensor data.

8. The method of claim 1, wherein the motion data is rotational data.

9. The method of claim 1, wherein the camera is a portable electronic device.

10. A non-transitory computer-readable medium embodied with computer-executable instructions for causing a computer to execute instructions, the computer instructions comprising:

obtaining motion data from a motion sensor different from an imaging sensor corresponding to motion of a camera;

estimating from the motion data and previous motion data from the motion sensor, instantaneous motion of a camera for a current frame of a video sequence;

comparing the instantaneous motion to a threshold value representing a relatively still condition;

reducing a video stabilization strength parameter for the current frame if the instantaneous motion is less than the threshold value, the video stabilization strength parameter defining an overall degree of video stabilization to be performed for the current frame,

wherein the video stabilization strength parameter is decreased by a first value if the instantaneous motion of the current frame is below the threshold value and the instantaneous motion of a preceding frame is equal to or above the threshold value,

the video stabilization strength parameter is decreased by a second value if the instantaneous motion of the current frame and the previous frame are below the threshold value, and

the first value is smaller than the second value;

and performing video stabilization on the current frame according to the frame'"'"'s video stabilization strength parameter.

11. The non-transitory computer-readable medium of claim 10, wherein the instantaneous motion of the camera corresponds to a motion difference between the current frame and a previous frame.

12. The non-transitory computer-readable medium of claim 10, wherein estimating the instantaneous motion of the camera includes determining a difference between instantaneous motion for the current frame and instantaneous motion of a frame preceding the current frame.

13. The non-transitory computer-readable medium of claim 10, wherein the motion data is derived from gyroscopic sensor data.

14. A method comprising:

determining whether a current frame of a video sequence is in a stationary mode based on motion data associated with the current frame, wherein the motion data are obtained from a motion sensor different from an image sensor;

determining whether a frame preceding the current frame was determined to be in the stationary mode based on motion data associated with the frame preceding the current frame;

decreasing the video stabilization strength parameter by a first value if it is determined that the current frame is in the stationary mode and the frame preceding the current frame is not in the stationary mode;

decreasing the video stabilization strength parameter by a second value if it is determined that the current frame is in the stationary mode and the frame preceding the current frame is in the stationary mode, wherein the first value is smaller than the second value; and

performing video stabilization on the current frame according to the frame'"'"'s video stabilization strength parameter.

15. The method of claim 14, wherein the video stabilization strength parameter is decreased by a greater value with each consecutive frame that is determined to be in the stationary mode.

16. An apparatus comprising:

a camera to capture a video sequence;

a motion sensor different from an image sensor to capture motion data associated with the video sequence;

a controller to;

compute, from the motion data and previous motion data from the motion sensor, instantaneous motion of the camera for a current frame of the video sequence,

compare the instantaneous motion to a threshold value representing a relatively still condition, and

reduce a video stabilization strength parameter for the current frame if the instantaneous motion is less than the threshold value, the video stabilization strength parameter defining an overall degree of video stabilization to be performed for the current frame,

wherein the video stabilization strength parameter is decreased by a first value if the instantaneous motion of the current frame is below the threshold value and the instantaneous motion of a preceding frame is equal to or above the threshold value,

the video stabilization strength parameter is decreased by a second value if the instantaneous motion of the current frame and the previous frame are below the threshold value, and the first value is smaller than the second value;

and a video stabilization unit to perform video stabilization on the current frame according to the frame'"'"'s strength parameter.