Methods and systems for digitally re-mastering of 2D and 3D motion pictures for exhibition with enhanced visual quality
First Claim
1. A method for enhancing the quality of a motion picture image sequence, the method comprising:
- receiving an original motion picture image sequence comprising digital data of a plurality of image frames;
creating additional image details at multiple levels of image details and generating a first enhanced image sequence by applying a spatial resolution enhancement process to the original image sequence; and
generating a second enhanced image sequence by applying a temporal resolution enhancement process to the first enhanced image sequence using frame interpolation by adding at least one synthesized image frame to the first enhanced image sequence, wherein the at least one synthesized image frame is created based on computed local motion vectors determined by a voting-based method applied to multiple initial local motion estimates for every pixel at each level of a multi-level representation of each image frame of the first enhanced image sequence, wherein the temporal resolution enhancement process includes creating synthesized image frames based on motion estimates calculated using a local motion estimation process that comprises;
calculating an edge mask map and a color segmentation map for each image frame;
warping the edge mask map and the color segmentation map using a global motion estimate for every pixel of each image frame;
generating multiple motion vectors for each pixel of each image frame using multiple local motion estimation methods;
computing forward and backward motion vectors for each pixel; and
applying a voting process to select a motion vector for each pixel,wherein the second enhanced image sequence has a greater frame rate than the original image sequence and the second enhanced image sequence has greater image detail than the original image sequence.
2 Assignments
0 Petitions
Accused Products
Abstract
The present invention relates to methods and systems for the exhibition of a motion picture with enhanced perceived resolution and visual quality. The enhancement of perceived resolution is achieved both spatially and temporally. Spatial resolution enhancement creates image details using both temporal-based methods and learning-based methods. Temporal resolution enhancement creates synthesized new image frames that enable a motion picture to be displayed at a higher frame rate. The digitally enhanced motion picture is to be exhibited using a projection system or a display device that supports a higher frame rate and/or a higher display resolution than what is required for the original motion picture.
170 Citations
42 Claims
-
1. A method for enhancing the quality of a motion picture image sequence, the method comprising:
-
receiving an original motion picture image sequence comprising digital data of a plurality of image frames; creating additional image details at multiple levels of image details and generating a first enhanced image sequence by applying a spatial resolution enhancement process to the original image sequence; and generating a second enhanced image sequence by applying a temporal resolution enhancement process to the first enhanced image sequence using frame interpolation by adding at least one synthesized image frame to the first enhanced image sequence, wherein the at least one synthesized image frame is created based on computed local motion vectors determined by a voting-based method applied to multiple initial local motion estimates for every pixel at each level of a multi-level representation of each image frame of the first enhanced image sequence, wherein the temporal resolution enhancement process includes creating synthesized image frames based on motion estimates calculated using a local motion estimation process that comprises; calculating an edge mask map and a color segmentation map for each image frame; warping the edge mask map and the color segmentation map using a global motion estimate for every pixel of each image frame; generating multiple motion vectors for each pixel of each image frame using multiple local motion estimation methods; computing forward and backward motion vectors for each pixel; and applying a voting process to select a motion vector for each pixel, wherein the second enhanced image sequence has a greater frame rate than the original image sequence and the second enhanced image sequence has greater image detail than the original image sequence. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27)
-
-
28. A system for enhancing the quality of a motion picture image sequence, the system comprising:
a back-end subsystem comprising; a central data storage for storing an original motion picture image sequence comprising digital data of a plurality of image frames; a render client configured to; create additional image details at multiple levels of image details and generate a first enhanced image sequence by applying a spatial resolution process to the original motion picture image sequence; and generate a second enhance image sequence by applying a temporal resolution enhancement process to the first enhanced image sequence using frame interpolation by adding at least one synthesized image frame to the first enhanced image sequence, wherein the render client is configured to create the least one synthesized image frame based on computed local motion vectors determined by a voting-based method applied to multiple initial local motion estimates for every pixel at each level of a multi-level representation of each image frame of the first enhanced image sequence, wherein the temporal resolution enhancement process includes creating synthesized image frames based on motion estimates calculated using a local motion estimation process that comprises; calculating an edge mask map and a color segmentation map for each image frame; warping the edge mask map and the color segmentation map using a global motion estimate for every pixel of each image frame; generating multiple motion vectors for each pixel of each image frame using multiple local motion estimation methods; computing forward and backward motion vectors for each pixel; and applying a voting process to select a motion vector for each pixel; and an intelligent controller for controlling the render client and accessing the central data storage, wherein the second enhanced image sequence is configured to have greater frame rate and greater detail than the original motion picture image sequence. - View Dependent Claims (29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39)
-
40. A method for enhancing the quality of an original motion picture image sequence, the method comprising:
-
receiving a three-dimensional (3D) original motion picture image sequence; applying a spatial resolution enhancement process to the 3D original image sequence to create an enhanced image sequence, the spatial resolution enhancement process comprising; a motion-based spatial resolution enhancement process; and a learning-based resolution enhancement process comprising; generating a codebook comprising codewords, each codeword being associated with a pattern having a higher resolution than the original image sequence; applying a clustering analysis to reduce the size of the codebook; up sizing an original image of the original image sequence to the higher resolution, the original image comprising a plurality of pixels; matching each pixel of the upsized image to a codeword; and replacing each pixel by a central pixel of the pattern associated with the matched codeword, wherein the enhanced image sequence has greater image detail than the original image sequence. - View Dependent Claims (41)
-
-
42. A method for enhancing the quality of an original motion picture image sequence, the method comprising:
-
receiving a three-dimensional (3D) original motion picture image sequence; applying a spatial resolution enhancement process to the 3D original image sequence to create an enhanced image sequence, the spatial resolution enhancement process comprising; a motion-based spatial resolution enhancement process; and a learning-based resolution enhancement process comprising; generating a codebook comprising codewords, each codeword being associated with a pattern having a higher resolution than the original image sequence; applying a clustering analysis to reduce the size of the codebook; upsizing an original image of the original image sequence to a higher resolution, the original image comprising at least one block of pixels; matching each block of pixels of the upsized image to a codeword; replacing the block of pixels by the pattern associated with the matched codeword using a transformation process to create an enhanced block of pixels; applying a blending process to the enhanced block of pixels; and applying a temporal filtering process, wherein the enhanced image sequence has greater image detail than the original image sequence.
-
Specification