MOTION SENSOR BASED VIRTUAL TRIPOD METHOD FOR VIDEO STABILIZATION

• US 20130155264A1
• Filed: 12/15/2011
• Published: 06/20/2013
• Est. Priority Date: 12/15/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Zhou, Jianping
• Granted Patent: US 9,628,711 B2
• Time in Patent Office: Days
• US Class Current: 348/208.6
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory

Abstract

An apparatus, method, and computer-readable medium for motion sensor-based video stabilization. A motion sensor may capture motion data of a video sequence. A controller may compute average motion data of the camera used to capture the video sequence based on motion data from the motion sensor. The controller may then determine the difference between the actual camera motion and the average camera motion to set a video stabilization strength parameter for the frames in the video sequence. A video stabilization unit may utilize the strength parameter to stabilize the frames in the video sequence.

Claims

1. A video stabilization method, comprising, for a frame within a video sequence:

responsive to motion sensor data associated with the video sequence;

computing average motion of a camera, during a capture period about the frame;

determining a difference between actual camera motion for the frame and the average camera motion;

setting a video stabilization strength parameter for the frame based on the difference value of the respective frame; and

performing video stabilization on the frame according to the frame'"'"'s strength parameter.

2. The method of claim 1, wherein the video stabilization strength parameter is set to:

a high strength value when the difference value of the respective frame is below a first threshold,

a low strength value when the difference value of the respective frame is above a second threshold, and

an intermediate value between the high and low strength value when the difference value of the respective frame is above the first threshold and below the second threshold.

3. The method of claim 1, wherein the motion sensor data is derived from gyroscopic sensor data.

4. The method of claim 1, wherein the motion sensor data is derived from accelerometer sensor data.

5. The method of claim 1, wherein the motion sensor data is rotational data.

6. The method of claim 1, wherein computing the average motion includes filtering the motion sensor data.

7. The method of claim 1, wherein the camera is a portable electronic device.

8. The method of claim 2, wherein the intermediate value is based on a function of the difference value for the frame.

9. A physical computer-readable medium embodied with computer-executable instructions for causing a computer to execute instructions, the computer instructions comprising:

computing average motion of a camera from motion sensor data associated with a video sequence, wherein the average motion is computed during a capture period about a frame in the video sequence;

determining a difference between actual camera motion for the frame and the average camera motion;

setting a video stabilization strength parameter for the frame based on the difference value of the respective frame; and

performing video stabilization on the frame according to the frame'"'"'s strength parameter.

10. The physical computer-readable medium of claim 9, wherein the video stabilization strength parameter is set to:

a high strength value when the difference value of the respective frame is below a first threshold,

a low strength value when the difference value of the respective frame is above a second threshold, and

an intermediate value between the high and low strength value when the difference value of the respective frame is above the first threshold and below the second threshold.

11. The physical computer-readable medium of claim 9, wherein the motion sensor data is derived from gyroscopic sensor data.

12. The physical computer-readable medium of claim 9, wherein the motion sensor data is derived from accelerometer sensor data.

13. The physical computer-readable medium of claim 9, wherein the motion sensor data is rotational data.

14. The physical computer-readable medium of claim 9, wherein computing the average motion includes filtering the motion sensor data.

15. The physical computer-readable medium of claim 9, wherein the camera is a portable electronic device.

16. The physical computer-readable medium of claim 10, wherein the intermediate value is based on a function of the difference value for the frame.

17. An apparatus comprising:

a processor for executing computer instructions, the computer instructions causing the processor to;

responsive to motion sensor data associated with a video sequence;

compute average motion of a camera, during a capture period about a frame within the video sequence;

determine a difference between actual camera motion for the frame and the average camera motion;

set a video stabilization strength parameter for the frame based on the difference value of the respective frame; and

perform video stabilization on the frame according to the frame'"'"'s strength parameter.

18. The apparatus of claim 17, wherein the video stabilization strength parameter is set to:

a high strength value when the difference value of the respective frame is below a first threshold,

a low strength value when the difference value of the respective frame is above a second threshold, and

an intermediate value between the high and low strength value when the difference value of the respective frame is above the first threshold and below the second threshold.

19. The apparatus of claim 17, wherein the motion sensor data is derived from gyroscopic sensor data.

20. The apparatus of claim 17, wherein the motion sensor data is derived from accelerometer sensor data.

21. The apparatus of claim 17, wherein the motion sensor data is rotational data.

22. The apparatus of claim 17, wherein computing the average motion includes filtering the motion sensor data.

23. The apparatus of claim 17, wherein the camera is a portable electronic device.

24. The apparatus of claim 18, wherein the intermediate value is based on a function of the difference value for the frame.

25. An apparatus comprising:

a camera to capture a video sequence;

a motion sensor to capture motion data associated with the video sequence;

a controller to;

compute average motion of the camera, during a capture period about a frame within the video sequence,

determine a difference between actual camera motion for the frame and the average camera motion, and

set a video stabilization strength parameter for the frame based on the difference value of the respective frame; and

a video stabilization unit to perform video stabilization on the frame according to the frame'"'"'s strength parameter.

26. The apparatus of claim 25, wherein the video stabilization strength parameter is set to:

a high strength value when the difference value of the respective frame is below a first threshold,

a low strength value when the difference value of the respective frame is above a second threshold, and

an intermediate value between the high and low strength value when the difference value of the respective frame is above the first threshold and below the second threshold.

27. The apparatus of claim 26, wherein the intermediate value is based on a function of the difference value for the frame.

28. A physical storage medium storing video data generated according to a process comprising:

capturing video data with a camera,

capturing motion sensor data with a motion sensor,

processing the video data with a processor by;

computing, from the motion sensor data, average motion of a camera during a capture period about a frame from the video data;

determining a difference between actual camera motion for the frame and the average camera motion;

setting a video stabilization strength parameter for the frame based on the difference value of the respective frame;

performing video stabilization on the frame according to the frame'"'"'s strength parameter; and

storing the stabilized video data on the storage medium.