Processing of video data to compensate for unintended camera motion between acquired image frames
First Claim
1. A method of operating a video device, comprising:
- acquiring video data from a two-dimensional sensor of a succession of image frames, making a calculation based upon the acquired video data of quantities estimating motion of components of a current acquired image frame with respect to components of a another acquired image frame, said calculation being made at least in part by passing the acquired video data through an electronic circuit that executes a given motion estimation algorithm, calculating an estimate of an overall motion of the current image frame with respect to said another image frame from the quantities estimating motion of components of the current frame with respect to components of said another frame, modifying the acquired video data of the current image frame by the estimate of the overall motion of the current image frame with respect to said another image frame in a manner to provide data of a stabilized image frame, compressing the data of the stabilized image frame, including passing the data of the stabilized image frame through said electronic circuit to calculate quantities estimating motion of components of the current stabilized image frames with respect to another stabilized image frame by executing said given motion estimation algorithm, and storing the compressed data of the stabilized image frame.
8 Assignments
0 Petitions
Accused Products
Abstract
Video data of successive image frames are processed in a digital camera or other video image data acquisition device immediately after acquiring the video data, or during post-processing of the video data at some time after acquiring the video data, to reduce the effects of unintended motion (jitter) of the hand-held devices by stabilizing the images. A processing circuit used to calculate an estimate of motion between components of successive image frames as part of a MPEG-4 or other compression algorithm is also used to estimate motion upon which the video data are altered to stabilize the images. The individual images may be pre-stabilized by using the results of stabilizing a prior image in order to reduce the amount of processing necessary to stabilize a current image.
-
Citations
32 Claims
-
1. A method of operating a video device, comprising:
-
acquiring video data from a two-dimensional sensor of a succession of image frames, making a calculation based upon the acquired video data of quantities estimating motion of components of a current acquired image frame with respect to components of a another acquired image frame, said calculation being made at least in part by passing the acquired video data through an electronic circuit that executes a given motion estimation algorithm, calculating an estimate of an overall motion of the current image frame with respect to said another image frame from the quantities estimating motion of components of the current frame with respect to components of said another frame, modifying the acquired video data of the current image frame by the estimate of the overall motion of the current image frame with respect to said another image frame in a manner to provide data of a stabilized image frame, compressing the data of the stabilized image frame, including passing the data of the stabilized image frame through said electronic circuit to calculate quantities estimating motion of components of the current stabilized image frames with respect to another stabilized image frame by executing said given motion estimation algorithm, and storing the compressed data of the stabilized image frame. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
-
-
15. A method of processing video data of image frames acquired in succession of moving images, comprising:
-
acquiring video data of larger image frames than required for resulting processed image frames, calculating a first set of global motion vectors representing estimates of global motion between individual pairs of image frames, pre-stabilizing the individual image frames in sequence by individually cropping the image frames of the acquired video data in accordance with the motion vectors between images of two preceding image frames, thereafter further stabilizing the individual pre-stabilized image frames in sequence by individually cropping the pre-stabilized image frames in accordance with the motion vectors between their images and images of immediately preceding frames, and thereafter compressing data of the stabilized image frames. - View Dependent Claims (16, 20, 21)
-
-
17. A method of processing video data of image frames acquired in succession of moving images, comprising:
-
acquiring video data of larger image frames than required for resulting processed image frames, calculating a first set of global motion vectors representing estimates of global motion between individual pairs of image frames, stabilizing the individual image frames in sequence by individually cropping the image frames of the acquired video data in accordance with the motion vectors between images of two others of the acquired image frames, and thereafter compressing data of the stabilized image frames. - View Dependent Claims (18, 19)
-
-
22. A method of operating a video device, comprising:
-
acquiring video data from a two-dimensional sensor of a succession of image frames, making calculations based upon the acquired video data of quantities that estimate motion of components of individual ones of the acquired image frames in process with respect to components of one or more other image frames, utilizing the quantities that estimate motion of components of the individual image frames to compress data of the image frames, calculating estimates of overall motion of the image frames from the quantities that estimate motion of components of the individual image frames that are used to compress the data, modifying the acquired video data by the estimates of overall motion of the image frames in a manner to provide data of stabilized image frames that are compressed, and storing the compressed data of the stabilized image frame. - View Dependent Claims (23, 24, 25, 26, 27, 28, 29)
-
-
30. A method of processing video data of frames of moving images that occur in sequence, which comprises:
-
calculate a first set of motion vectors representing estimates of global motion of a first image frame by use of data of the first image frame and data of another image frame, compress the data of the first image frame by use of the first set of motion vectors, and stabilize the data of a second image frame following the first image frame by use of the first set of motion vectors.
-
-
31. An integrated circuit chip, comprising:
-
a first set of conductors on the chip that carry digital data of a succession of video image frames, an electronic circuit that calculates motion vectors of image frames whose digital data are passed therethrough, a data processor programmed to cause digital data of image frames received through the first set of conductors to pass through the electronic circuit, thereby to calculate motion vectors on the received image frame data, to thereafter cause the image frames to be stabilized by use of the motion vectors calculated from the received data motion vectors, to thereafter cause data of the stabilized image frames to be passed through the electronic circuit, thereby to calculate motion vectors on the stabilized image frames, and to thereafter compress the stabilized image frames by use of the calculated motion vectors on the stabilized image frames, and a second set of conductors on the chip that carry data of the compressed stabilized image frames.
-
-
32. A video system, comprising:
-
an input of data of successive frames of a moving image, a processor that causes motion vectors to be calculated from the data of two of the image frames and used both to compress data of individual image frames and to stabilize subsequent image frames prior to their compression, and an output of data of successive stabilized and compressed versions of the input data of successive frames.
-
Specification