Apparatus and method for removing blank areas from real-time stabilized images by inserting background information
First Claim
1. An image stabilizer system comprising:
- a terminal at which a video signal representing a current video frame is received;
a memory which holds a frame of a background video signal;
an image analyzer, responsive to the current video frame and the background video frame which determines a transformation to be applied to the current video frame to align the current video frame to the background video frame;
an image warping processor which applies the transformation determined by the image analyzer to produce a transformed current video frame which is aligned with the background video frame, the transformed current video frame having blank picture elements (pixels) along at least one edge; and
an image merging processor which substitutes pixels from the background frame for the blank pixels in the transformed current video frame to produce an output video frame.
2 Assignments
0 Petitions
Accused Products
Abstract
An image stabilizer selectively adds image data from a background image to the current image to compensate for data in the current image that is missing due to a sudden shift in the current image relative to the previous images. The current image is warped into the coordinate system of the background image and then the warped current image is merged with the background image to replace any blank areas in the current image with corresponding pixel values from the background image. The image data from the background image which is to be substituted into the warped current image is subject to a low-pass filtering operation before it is merged with the warped current image. The warped current image is merged with the background image to form a modified background image which is then merged with the warped current image. The background image is, itself, warped to track camera motion in obtaining the current image before the background image is merged with the warped current image.
94 Citations
12 Claims
-
1. An image stabilizer system comprising:
-
a terminal at which a video signal representing a current video frame is received;
a memory which holds a frame of a background video signal;
an image analyzer, responsive to the current video frame and the background video frame which determines a transformation to be applied to the current video frame to align the current video frame to the background video frame;
an image warping processor which applies the transformation determined by the image analyzer to produce a transformed current video frame which is aligned with the background video frame, the transformed current video frame having blank picture elements (pixels) along at least one edge; and
an image merging processor which substitutes pixels from the background frame for the blank pixels in the transformed current video frame to produce an output video frame. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. An image stabilizer system comprising:
-
a terminal at which a video signal representing a current video frame is received;
a memory which holds a frame of a background video signal;
an image analyzer, responsive to the current video frame and the background video frame which determines a transformation to be applied to the current video frame to align the current video frame to the background video frame;
an image warping processor which applies the transformation determined by the image analyzer to produce a transformed current video frame which is aligned with the background video frame, the transformed current video frame having blank picture elements (pixels) along at least one edge; and
a two-input image filtering processor which provides pixels of the current video frame when the pixels of the current video frame are not blank pixels and provides filtered pixel values when the pixels of the current video frame are blank pixels to produce an output video frame, wherein the filtered pixel values are formed by processing both non-blank pixels from the current video frame and pixels from the background video frame.
-
-
9. A method for stabilizing a current video frame comprising the steps of:
-
storing a frame of a background video signal;
comparing the background video frame to the current video frame to determine a transformation to be applied to the current video frame which aligns the current video frame to the background video frame;
warping the current video frame according to the determined transformation to produce a transformed current video frame, wherein the transformed current video frame has blank picture elements (pixels) along at least one edge; and
merging the background video frame with the current video frame to replace the blank pixels in the transformed current video frame with corresponding pixels from the background frame. - View Dependent Claims (10, 11, 12)
determining a frame-to-frame displacement of the current video frame with respect to the background video frame over several frame intervals to define an average frame-to-frame displacement over the several frame intervals; and
warping the background frame by the average frame-to-frame displacement to produce a warped background frame which is merged with the current video frame.
-
Specification