Method to stabilize digital video motion
First Claim
1. A method for stabilizing digital video motion comprising:
- selecting a current video frame and a previous video frame from a video stream;
dividing the current video frame into a plurality of macroblocks;
selecting a subset of macroblocks from the current video frame;
comparing the subset of macroblocks to the previous video frame within a two-dimensional comparison range to generate a set of motion vectors;
calculating a frame motion vector for the current video frame;
estimating a steady motion vector for the current video frame by a first-order autoregression, wherein the first-order autoregression comprises;
multiplying the frame motion vector of the current frame by a first constant;
multiplying the steady motion vector of the previous video frame by a second constant, wherein the first constant is a floating-point value in a range of 0-1 inclusive, the second constant is calculated by subtracting the first constant from 1, and the first constant is chosen to be a lesser of a third constant and a correlation value;
computing an estimated hand motion vector of the current video frame by subtracting the steady motion vector of the previous video frame from the frame motion vector of the current video frame; and
calculating the correlation value between the estimated hand motion vector and the steady motion vector by taking an absolute value of an inner product of the estimated hand motion vector of the current video frame and steady motion vector of the previous video frame, and dividing the absolute value by a product of a square root of an inner product of the estimated hand motion vector of the current video frame and the estimated hand motion vector of the current video frame and a square root of an inner product of the steady motion vector of the previous video frame and the steady motion vector of the previous video frame;
calculating a hand motion vector for the current video frame by taking a difference between the steady motion vector and the frame motion vector of the current video frame;
cropping a selected video frame from the current video frame, wherein a center of the selected video frame is offset from a center of the current video frame by the hand motion vector; and
recording the selected video frame to a storage device.
1 Assignment
0 Petitions
Accused Products
Abstract
A digital video image stabilization method features scene change detection, adaptive stabilization in two dimensions, prevention of accumulated-compensation saturation, multi-resolution motion estimation, and compatibility with MPEG video encoding standards. The method uses block-based motion estimation on a selected set of macroblocks, which generates and validates a set of motion vectors, to determine how a digital camcorder or other video recording device is moving; accumulates motion data to estimate and preserve overall motion due to panning while eliminating small variations due to shaking; and resets its accumulated motion data when it detects a scene change. The method uses this information to select a sub-image from a larger image for the selected video frame. When one or more edges of the sub-image approach the edges of the larger image, the method reduces its correction factor to transition smoothly to a faster pan and prevent saturation.
47 Citations
14 Claims
-
1. A method for stabilizing digital video motion comprising:
-
selecting a current video frame and a previous video frame from a video stream; dividing the current video frame into a plurality of macroblocks; selecting a subset of macroblocks from the current video frame; comparing the subset of macroblocks to the previous video frame within a two-dimensional comparison range to generate a set of motion vectors; calculating a frame motion vector for the current video frame;
estimating a steady motion vector for the current video frame by a first-order autoregression, wherein the first-order autoregression comprises;multiplying the frame motion vector of the current frame by a first constant; multiplying the steady motion vector of the previous video frame by a second constant, wherein the first constant is a floating-point value in a range of 0-1 inclusive, the second constant is calculated by subtracting the first constant from 1, and the first constant is chosen to be a lesser of a third constant and a correlation value; computing an estimated hand motion vector of the current video frame by subtracting the steady motion vector of the previous video frame from the frame motion vector of the current video frame; and calculating the correlation value between the estimated hand motion vector and the steady motion vector by taking an absolute value of an inner product of the estimated hand motion vector of the current video frame and steady motion vector of the previous video frame, and dividing the absolute value by a product of a square root of an inner product of the estimated hand motion vector of the current video frame and the estimated hand motion vector of the current video frame and a square root of an inner product of the steady motion vector of the previous video frame and the steady motion vector of the previous video frame; calculating a hand motion vector for the current video frame by taking a difference between the steady motion vector and the frame motion vector of the current video frame; cropping a selected video frame from the current video frame, wherein a center of the selected video frame is offset from a center of the current video frame by the hand motion vector; and recording the selected video frame to a storage device. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A method for stabilizing digital video motion comprising:
-
selecting a current video frame and a previous video frame from a video stream; dividing the current video frame into a plurality of macroblocks, such that the macroblock size is suitable for either autonomously encoded or predictively encoded pictures; selecting a subset of macroblocks from the current video frame; comparing the subset of macroblocks to the previous video frame within a two-dimensional comparison range to generate a set of motion vectors; calculating a frame motion vector for the current video frame by selecting a median vertical component and a median horizontal component from the set of motion vectors and the frame motion vector for the previous video frame; estimating a steady motion vector for the current video frame by a first-order autoregression, wherein the first-order autoregression comprises; multiplying the frame motion vector of the current frame by a first constant; multiplying the steady motion vector of the previous video frame by a second constant, wherein the first constant is a floating-point value in a range of 0-1 inclusive, the second constant is calculated by subtracting the first constant from 1, and the first constant is chosen to be a lesser of a third constant and a correlation value; computing an estimated hand motion vector of the current video frame by subtracting the steady motion vector of the previous video frame from the frame motion vector of the current video frame; and calculating the correlation value between the estimated hand motion vector and the steady motion vector by taking an absolute value of an inner product of the estimated hand motion vector of the current video frame and steady motion vector of the previous video frame, and dividing the absolute value by a product of a square root of an inner product of the estimated hand motion vector of the current video frame and the estimated hand motion vector of the current video frame and a square root of an inner product of the steady motion vector of the previous video frame and the steady motion vector of the previous video frame; calculating a hand motion vector for the current video frame by taking a difference between the steady motion vector and the frame motion vector of the current video frame; cropping a selected video frame from the current video frame, wherein a center of the selected video frame is offset from a center of the current video frame by the hand motion vector; and recording the selected video frame to a storage device. - View Dependent Claims (10, 11, 12, 13, 14)
-
Specification