METHODS AND SYSTEMS FOR DIFFERENTIATING SYNTHETIC AND NON-SYNTHETIC IMAGES

• US 20140241629A1
• Filed: 02/28/2013
• Published: 08/28/2014
• Est. Priority Date: 02/28/2013
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Lerios, Apostolos, Stoop, Dirk, Mack, Ryan, Bourdev, Lubomir, Paluri, Balmanohar
• Granted Patent: US 8,903,186 B2
• Time in Patent Office: Days
• US Class Current: 382/166
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/27 involving both synthetic and natural picture components, e.g. synthetic natural hybrid coding [SNHC]
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

The techniques introduced here include a system and method for transcoding multimedia content based on the results of content analysis. The determination of specific transcoding parameters, used for transcoding multimedia content, can be performed by utilizing the results of content analysis of the multimedia content. One of the results of the content analysis is the determination of image type of any images included in the multimedia content. The content analysis uses one or more of several techniques, including analyzing content metadata, examining colors of contiguous pixels in the content, using histogram analysis, using compression distortion analysis, analyzing image edges, or examining user provided inputs. Transcoding the multimedia content can include adapting the content to the constraints in delivery and display, processing and storage of user computing devices.

Claims

1. A method for transcoding an image into one or more of a plurality of image formats, the method comprising:

receiving, by a transcoding module executed by a processer of a computer system, the image to be transcoded;

analyzing, by the transcoding module, content of the received image using compression distortion analysis, wherein the compression distortion analysis includes analysis of distortion between a received version of the received image and a processed version of the received image, the processed version resulting from a compression and subsequent decompression of the received image;

based on the content analysis, determining an image type associated with the received image, the image type being one of at least a synthetic image type or a natural image type;

determining a transcoding process to apply to the received image based, at least in part, on the determination of the image type associated with the received image; and

transcoding the received image into one or more of the plurality of image formats using the determined transcoding process.

2. The method of claim 1, wherein the synthetic image includes a graphically rendered image.

3. The method of claim 1, wherein the natural image includes one or more of:

a photograph;

a digital version of the photograph, wherein the digital version is generated by scanning the photograph;

or an image produced with a camera.

4. The method of claim 1, wherein the compression of the received image is performed using a natural imagery encoder.

5. The method of claim 4, wherein the decompression of the compressed version of the received image is performed using a natural imagery decoder.

6. The method of claim 5, wherein the natural imagery encoder includes a JPEG encoder and a natural imagery decoder includes a JPEG decoder.

7. The method of claim 1, wherein the measurement of distortion between the original version and the processed version of the image is performed using one or more of:

a Peak Signal-to-Noise Ratio (PSNR) index;

or

a structural similarity (SSIM) index.

8. The method of claim 7, further comprising:

determining that the image type is the synthetic image type if the measured distortion is greater than a given threshold; and

determining that the image type is the natural image type if the measured distortion is lesser than the given threshold.

9. The method of claim 1, wherein the step of determining a transcoding process to apply to the received image includes one or more of:

determining whether a compression is to be applied to the image based on the determination of the image type;

determining a type of compression to be applied to the image based on the determination of the image type;

determining one of more of a type or intensity of size reduction to be applied to the image based on the determination of the image type;

determining one or more of a type or intensity of color reduction to be applied to the image based on the determination of the image type

10. A method for transcoding an image into one or more of a plurality of image formats, the method comprising:

analyzing, by a transcoding module executed by a processor of a computer system, content of the image using a contiguous pixels analysis, wherein the contiguous pixels analysis includes analysis of similarity of colors in contiguous pixels of the received image;

based on the content analysis, determining, by the transcoding module, an image type associated with the received image, the image type being one of at least a synthetic image type or a natural image type;

determining one or more of a plurality of parameters associated with transcoding the image, the one or more of the plurality of parameters having a given value selected according to the determination of whether the image type is the synthetic image type of the natural image type; and

utilizing the determined parameters in transcoding the image into one or more of the plurality of image formats.

11. The method of claim 10, wherein the plurality of parameters includes one or more of a type of compression to be applied to the image;

one or more of a type or intensity of size reduction to be applied to the image;

one or more of a type or intensity of color reduction to be applied to the image.

12. The method of claim 10, wherein the synthetic image includes a graphically rendered image.

13. The method of claim 10, wherein the natural image includes one or more of:

a photograph;

a digital version of the photograph, wherein the digital version is generated by scanning the photograph;

or an image produced with a camera.

14. The method of claim 10, further comprising:

determining that the image type is the synthetic image type when at least a threshold number of contiguous pixels in the received image are of similar color;

determining that the image type is the natural image type when less than the threshold number of contiguous pixels in the received image are of similar color.

15. The method of claim 14, further comprising:

determining that a first pixel and a second pixel of the received image are of similar color when one or more of a hue value or a saturation value associated with the first and the second image are substantially similar.

16. The method of claim 10, wherein the step of determining a transcoding process to apply to the received image includes one or more of:

determining whether a compression is to be applied to the image based on the determination of the image type;

determining a type of compression to be applied to the image based on the determination of the image type;

determining one of more of a type or intensity of size reduction to be applied to the image based on the determination of the image type;

determining one or more of a type or intensity of color reduction to be applied to the image based on the determination of the image type

17. A system comprising:

a processor;

a memory configured to store a set of instructions, which when executed by the processor cause the system to perform a method, the method including;

receiving the image to be transcoded;

analyzing content of the received image using compression distortion analysis, wherein the compression distortion analysis includes analysis of distortion between a received version of the received image and a processed version of the received image, the processed version resulting from a compression and subsequent decompression of the received image;

based on the content analysis, determining an image type associated with the received image, the image type being one of at least a synthetic image type or a natural image type;

determining a transcoding process to apply to the received image based, at least in part, on the determination of the image type associated with the received image; and

transcoding the received image into one or more of the plurality of image formats using the determined transcoding process.

18. The system of claim 17, wherein the step of determining a transcoding process to apply to the received image includes one or more of:

determining whether a compression is to be applied to the image based on the determination of the image type;

determining a type of compression to be applied to the image based on the determination of the image type;

determining one of more of a type or intensity of size reduction to be applied to the image based on the determination of the image type;

determining one or more of a type or intensity of color reduction to be applied to the image based on the determination of the image type

19. The system of claim 17, wherein the synthetic image includes a graphically rendered image.

20. The system of claim 17, wherein the natural image includes one or more of:

a photograph;

a digital version of the photograph, wherein the digital version is generated by scanning the photograph;

or an image produced with a camera.

21. The system of claim 17, wherein the compression of the received image is performed using a natural imagery encoder.

22. The system of claim 17, wherein the decompression of the compressed version of the received image is performed using a natural imagery decoder.

23. The system of claim 21, wherein the natural imagery encoder includes a JPEG encoder and a natural imagery decoder includes a JPEG decoder.

24. The system of claim 17, wherein the measurement of distortion between the original version and the processed version of the image is performed using one or more of:

a Peak Signal-to-Noise Ratio (PSNR) index;

or

a structural similarity (SSIM) index.

25. The system of claim 24, further comprising:

determining that the image type is the synthetic image type if the measured distortion is greater than a given threshold; and

determining that the image type is the natural image type if the measured distortion is lesser than the given threshold.