Memory-to-memory low resolution motion estimation systems and methods
First Claim
1. A video encoding pipeline comprising:
- a low resolution pipeline coupled to memory that stores source image data to be encoded by the video encoding system, wherein the low resolution pipeline comprises a low resolution motion estimation block configured to;
determine first downscaled source image data generated by reducing a resolution of first source image data corresponding with a first image frame;
generate second downscaled source image data by reducing a resolution of second source image data corresponding with a second image frame when the second source image data is retrieved from the memory;
perform a first motion estimation search in the first downscaled source image data based on the second downscaled source image data to determine a first downscaled reference sample; and
determine a low resolution inter-frame prediction mode that indicates location of the second source image data relative to a first reference sample corresponding with the downscaled reference sample; and
a main pipeline coupled to the memory that stores the source image data to be encoded by the video encoding system, wherein the main pipeline comprises;
a reconstruction block configured to determine first reconstructed image data by decoding encoded image data corresponding with the first source image data;
a motion estimation block configured to determine a candidate inter-frame prediction mode by performing a second motion estimation search in the first reconstructed image data based at least in part on the low resolution inter-frame prediction mode; and
a mode decision block configured to determine encoding parameters to be used to encode the second source image data based at least in part on a first rate-distortion cost associated with the candidate inter-frame prediction mode.
1 Assignment
0 Petitions
Accused Products
Abstract
System and method for improving operational efficiency of a video encoding pipeline used to encode image data. In embodiments, the video encoding pipeline includes a low resolution pipeline that includes a low resolution motion estimation block, which generates downscaled image data by reducing resolution of the image data and determines a low resolution inter-frame prediction mode by performing a motion estimation search using the downscaled image data and previously downscaled image data. The video encoding pipeline also includes a main pipeline in parallel with the low resolution pipeline that includes a motion estimation block, which determines a candidate inter-frame prediction mode based at least in part on the low resolution inter-frame prediction mode, and a mode decision block, which determines a first rate-distortion cost associated with the candidate inter-frame prediction mode and determines prediction mode used to prediction encode the image data based at least in part on the first rate-distortion cost.
-
Citations
28 Claims
-
1. A video encoding pipeline comprising:
-
a low resolution pipeline coupled to memory that stores source image data to be encoded by the video encoding system, wherein the low resolution pipeline comprises a low resolution motion estimation block configured to; determine first downscaled source image data generated by reducing a resolution of first source image data corresponding with a first image frame; generate second downscaled source image data by reducing a resolution of second source image data corresponding with a second image frame when the second source image data is retrieved from the memory; perform a first motion estimation search in the first downscaled source image data based on the second downscaled source image data to determine a first downscaled reference sample; and determine a low resolution inter-frame prediction mode that indicates location of the second source image data relative to a first reference sample corresponding with the downscaled reference sample; and a main pipeline coupled to the memory that stores the source image data to be encoded by the video encoding system, wherein the main pipeline comprises; a reconstruction block configured to determine first reconstructed image data by decoding encoded image data corresponding with the first source image data; a motion estimation block configured to determine a candidate inter-frame prediction mode by performing a second motion estimation search in the first reconstructed image data based at least in part on the low resolution inter-frame prediction mode; and a mode decision block configured to determine encoding parameters to be used to encode the second source image data based at least in part on a first rate-distortion cost associated with the candidate inter-frame prediction mode. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 26, 27, 28)
-
-
9. A tangible, non-transitory, computer-readable medium that stores instructions executable by one or more processors in a computing device, wherein the instructions comprise instructions to:
-
instruct, using the one or more processors, a video encoding pipeline implemented in the computing device to generate first downscaled source image data by downscaling first source image data corresponding with a first image frame; instruct, using the one or more processors, the video encoding pipeline to generate second downscaled source image data by downscaling the second source image data corresponding with a second image frame; instruct, using the one or more processors, the video encoding pipeline to determine a low resolution inter-frame prediction mode by performing a first motion estimation search in the first downscaled source image data based on the second downscaled source image data to determine a downscaled reference sample; instruct, using the one or more processors, the video encoding pipeline to determine reconstructed image data by decoding first encoded image data corresponding with the first source image data; instruct, using the one or more processors, the video encoding pipeline to determine a candidate inter-frame prediction mode by performing a second motion estimation search in the reconstructed image data corresponding with the first source image data based on the low resolution inter-frame prediction mode; instruct, using the one or more processors, the video encoding pipeline to determine encoding parameters to be used to encode the second source image data based at least in part on a rate-distortion cost associated with the candidate inter-frame prediction mode; and instruct, using the one or more processors, the video encoding pipeline to generate second encoded image data corresponding with the second source image data by applying the encoding operational parameters. - View Dependent Claims (10, 11, 12, 13, 14)
-
-
15. A video encoding system comprising:
-
memory configured to store source image data corresponding with image frames to be encoded by the video encoding system, wherein each image frame is divided into a plurality of coding units and each of the plurality of coding units comprises one or more prediction units; a low resolution motion estimation circuitry coupled to the memory, wherein the low resolution motion estimation circuitry is configured to; retrieve first source image data corresponding with a first coding unit in a first image frame and downscaled source image data corresponding with a second image frame from the memory using direct memory access; downscale the first source image data to generate a first downscaled coding unit, wherein the first downscaled coding unit comprises a first downscaled prediction unit corresponding with a first prediction unit in the first coding unit; and determine a first low resolution inter-frame prediction mode by performing a first motion estimation search in the downscaled source image data corresponding with the second image frame based on the first downscaled prediction unit in the first downscaled coding unit to determine a first downscaled reference sample; and a main pipeline coupled to the memory in parallel with the low resolution motion estimation circuitry, wherein the main pipeline is configured to; retrieve the first source image data corresponding with the first coding unit and the first low resolution inter-frame prediction mode from the memory using direct memory access; and determine a first prediction mode to be used to encode the first source image data corresponding with the first prediction unit in the first coding unit based at least in part on the first low resolution inter-frame prediction mode. - View Dependent Claims (16, 17)
-
-
18. The video encoding pipeline of system 15, wherein the first coding unit is a 64×
- 64 sample.
-
19. The video encoding pipeline of system 15, wherein the second image frame is configured to be display directly before or directly after the first image frame.
-
20. A computing device comprising:
-
an image data source configured to generate first image data corresponding with a first image and second image data corresponding with a second image; a low resolution motion estimation pipeline comprising first image data processing circuitry programmed to; generate first downscaled image data by downscaling the first image data generated by the image data source when the low resolution motion estimation pipeline receives the first image data; when the low resolution motion estimation pipeline receives the second image data; generate second downscaled image data by downscaling the second image data generated by the image data source; perform a first motion estimation search in the first downscaled image data based on the second downscaled image data to determine a low resolution inter-frame prediction mode; and determine statistics based at least in part on luma of the second downscaled image data; and a main video encoding pipeline configured to generate first encoded image data by encoding the first image data and second encoded image data by encoding the second image data, wherein the video encoding pipeline comprises second processing circuitry programmed to; determine first reconstructed image data by decoding the first encoded image data; and when the main video encoding pipeline receives the second image data; determine a candidate inter-frame prediction mode by performing a second motion estimation search in the first reconstructed image data based at least in part on the low resolution inter-frame prediction mode; and determine encoding operational parameters to be used to encode the second image data based at least in part on the candidate inter-frame prediction mode and the statistics determined by the low resolution motion estimation pipeline. - View Dependent Claims (21, 22, 23, 24, 25)
-
Specification