AUTOMATED TRIPOD DETECTION AND HANDLING IN VIDEO STABILIZATION

• US 20130342714A1
• Filed: 06/22/2012
• Published: 12/26/2013
• Est. Priority Date: 06/22/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Zhou, Jianping, Beysserie, Sebastien X.
• Granted Patent: US 9,300,873 B2
• Time in Patent Office: Days
• US Class Current: 348/208.2
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory

Abstract

An apparatus, method, and computer-readable medium for motion sensor-based video stabilization. A motion sensor may capture motion data of a video sequence. A controller may compute instantaneous motion of the camera for a current frame of the video sequence. The controller may compare the instantaneous motion to a threshold value representing a still condition and reduce a video stabilization strength parameter for the current frame if the instantaneous motion is less than the threshold value. A video stabilization unit may perform video stabilization on the current frame according to the frame'"'"'s strength parameter.

Claims

1. A video stabilization method, comprising:

obtaining motion data corresponding to motion of a camera;

estimating from the motion data instantaneous motion of a camera for a current frame of a video sequence;

comparing the instantaneous motion to a threshold value representing a still condition;

reducing a video stabilization strength parameter for the current frame if the instantaneous motion is less than the threshold value; and

performing video stabilization on the current frame according to the frame'"'"'s video stabilization strength parameter.

2. The method of claim 1, wherein the instantaneous motion of the camera corresponds to a motion difference between the current frame and a previous frame.

3. The method of claim 1, wherein estimating the instantaneous motion of the camera includes determining a difference between instantaneous motion for the current frame and instantaneous motion of a frame preceding the current frame.

4. The method of claim 1, wherein:

the video stabilization strength parameter is decreased by a first value if the instantaneous motion of the current frame is below the threshold value and the instantaneous motion of a preceding frame is equal to or above the threshold value;

the video stabilization strength parameter is decreased by a second value if the instantaneous motion of the current frame and the previous frame are below the threshold value; and

wherein the first value is smaller than the second value.

5. The method of claim 1, wherein if the instantaneous motion is less than the threshold value, pixels in one or more frames of the video sequence are processed to determine motion in the current frame.

6. The method of claim 1, wherein the motion data is derived from gyroscopic sensor data.

7. The method of claim 1, wherein computing the instantaneous motion includes filtering the motion data.

8. The method of claim 1, wherein the motion data is derived from accelerometer sensor data.

9. The method of claim 1, wherein the motion data is rotational data.

10. The method of claim 1, wherein the camera is a portable electronic device.

11. A non-transitory computer-readable medium embodied with computer-executable instructions for causing a computer to execute instructions, the computer instructions comprising:

obtaining motion data corresponding to motion of a camera;

estimating from the motion data instantaneous motion of a camera for a current frame of a video sequence;

comparing the instantaneous motion to a threshold value representing a still condition;

reducing a video stabilization strength parameter for the current frame if the instantaneous motion is less than the threshold value; and

performing video stabilization on the current frame according to the frame'"'"'s video stabilization strength parameter.

12. The non-transitory computer-readable medium of claim 11, wherein the instantaneous motion of the camera corresponds to a motion difference between the current frame and a previous frame.

13. The non-transitory computer-readable medium of claim 11, wherein estimating the instantaneous motion of the camera includes determining a difference between instantaneous motion for the current frame and instantaneous motion of a frame preceding the current frame.

14. The non-transitory computer-readable medium of claim 11, wherein:

the video stabilization strength parameter is decreased by a first value if the instantaneous motion of the current frame is below the threshold value and the instantaneous motion of a preceding frame is equal to or above the threshold value;

the video stabilization strength parameter is decreased by a second value if the instantaneous motion of the current frame and the previous frame are below the threshold value; and

the first value is smaller than the second value.

15. The non-transitory computer-readable medium of claim 11, wherein the motion data is derived from gyroscopic sensor data.

16. A method comprising:

determining whether a current frame of a video sequence is in a stationary mode based on motion data associated with the current frame;

determining whether a frame preceding the current frame was determined to be in the stationary mode based on motion data associated with the frame preceding the current frame;

decreasing the video stabilization strength parameter by a first value if it is determined that the current frame is in the stationary mode and the frame preceding the current frame is not in the stationary mode;

decreasing the video stabilization strength parameter by a second value if it is determined that the current frame is in the stationary mode and the frame preceding the current frame is in the stationary mode, wherein the first value is smaller than the second value; and

performing video stabilization on the current frame according to the frame'"'"'s video stabilization strength parameter.

17. The method of claim 16, wherein the video stabilization strength parameter is decreased by a greater value with each consecutive frame that is determined to be in the stationary mode.

18. An apparatus comprising:

a camera to capture a video sequence;

a motion sensor to capture motion data associated with the video sequence;

a controller to;

compute, from the motion data, instantaneous motion of the camera for a current frame of the video sequence,

compare the instantaneous motion to a threshold value representing a still condition, and

reduce a video stabilization strength parameter for the current frame if the instantaneous motion is less than the threshold value; and

a video stabilization unit to perform video stabilization on the current frame according to the frame'"'"'s strength parameter.

19. The apparatus of claim 18, wherein:

the video stabilization strength parameter is decreased by a first value if the instantaneous motion of the current frame is below the threshold value and the instantaneous motion of a preceding frame is equal to or above the threshold value;

the video stabilization strength parameter is decreased by a second value if the instantaneous motion of the current frame and the previous frame are below the threshold value; and

the first value is smaller than the second value.

20. A physical storage medium storing video data generated according to a process comprising:

capturing video data with a camera,

capturing motion data with a motion sensor,

processing the video data with a processor by;

computing, from the motion data, instantaneous motion of the camera for a current frame of a video sequence,

comparing the instantaneous motion to a threshold value representing a still condition,

reducing a video stabilization strength parameter for the current frame if the instantaneous motion is less than the threshold value, and

performing video stabilization on the current frame according to the frame'"'"'s video stabilization strength parameter; and

storing the stabilized video data on the storage medium.