Methods and systems for differentiating synthetic and non-synthetic images

• US 9,280,723 B2
• Filed: 10/28/2014
• Issued: 03/08/2016
• Est. Priority Date: 02/28/2013
• Status: Expired due to Fees

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Lerios, Apostolos, Stoop, Dirk, Mack, Ryan, Bourdev, Lubomir Dimitrov, Paluri, Balmanohar
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US14/526,190
• Publication Number:
  US 20150110394A1
• Time in Patent Office: 497 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/27 involving both synthetic and natural picture components, e.g. synthetic natural hybrid coding [SNHC]
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

The techniques introduced here include a system and method for transcoding multimedia content based on the results of content analysis. The determination of specific transcoding parameters, used for transcoding multimedia content, can be performed by utilizing the results of content analysis of the multimedia content. One of the results of the content analysis is the determination of image type of any images included in the multimedia content. The content analysis uses one or more of several techniques, including analyzing content metadata, examining colors of contiguous pixels in the content, using histogram analysis, using compression distortion analysis, analyzing image edges, or examining user provided inputs. Transcoding the multimedia content can include adapting the content to the constraints in delivery and display, processing and storage of user computing devices.

Claims

1. A method for comprising:

receiving, by an analysis module executed by a processor of a computer system, an image to be classified;

analyzing the received image to produce analyzing results, the analyzing using one or more of;

compression distortion analysis, wherein the compression distortion analysis includes analysis of distortion between the received image and a processed version of the received image, the processed version of the received image resulting from a compression and subsequent decompression of the received image;

contiguous pixels analysis, wherein the contiguous pixels analysis includes computing whether a number of pixel groups from the received image, each pixel group defined by a similarity of color in at least a selected number of contiguous pixels, is above a color continuity threshold, wherein the color continuity threshold corresponds to a distinction in image types;

edge detect analysis, wherein the edge detect analysis includes computing whether an edge analysis of the received image resulted in an edge threshold indicating edge geometry or distribution, wherein the edge threshold corresponds to a distinction in image types;

or

color histogram analysis, wherein the color histogram analysis includes identifying whether a periodicity threshold is reached in a computation of a number of pixels of the received image, discretized by color, that fall within a list of color ranges that span a color space of the received image, wherein the periodicity threshold corresponds to a distinction in image types; and

classifying the received image with an image type, wherein the image type is at least one of;

a synthetic image type or a natural image type, wherein a synthetic image type is for images generated in part by capturing reflected light and a natural image type is for images that are computer generated.

2. The method of claim 1, wherein analyzing the received image comprises the contiguous pixels analysis.

3. The method of claim 2, wherein the contiguous pixels analysis comprises identifying that a first pixel of the contiguous pixels and a second pixel of the contiguous pixels are of similar color when one or more of a hue value or a saturation value associated with the first pixel and the second pixel are within a specified threshold level of each other.

4. The method of claim 1, wherein analyzing the received image comprises the color histogram analysis.

5. The method of claim 1, wherein analyzing the received image comprises the compression distortion analysis.

6. The method of claim 5, wherein the analysis of distortion between the received image and the processed version of the received image is performed using one or more of:

a Peak Signal-to-Noise Ratio (PSNR) index;

or

a structural similarity (SSIM) index.

7. The method of claim 5, further comprising:

identifying that the image type is the synthetic image type if the distortion is greater than a specified threshold; and

identifying that the image type is the natural image type if the distortion is less than the specified threshold.

8. The method of claim 1, wherein analyzing the received image comprises the edge detect analysis.

9. The method of claim 1, further comprising identifying a plurality of parameters for transcoding the received image based on the image type classification of the received image.

10. The method of claim 1 further comprising identifying a transcoding process to apply to the received image by:

identifying whether a compression is to be applied to the received image based on the image type classification of the received image;

identifying a type of compression to be applied to the received image based on the image type classification of the received image;

identifying one or more of a type or intensity of size reduction to be applied to the received image based on the image type classification of the received image;

or

identifying one or more of a type or intensity of color reduction to be applied to the received image based on the image type classification of the received image.

11. The method of claim 1, wherein:

the image type is the natural image type; and

the received image is one or more of;

a photograph;

a digital version of the photograph, wherein the digital version is generated by scanning the photograph;

or

a camera-produced image.

12. A system comprising:

a memory;

one or more processors;

an interface configured to receive an image to be classified;

an analysis module, executed by the one or more processors, configured to analyze the received image to produce analyzing results, the analyzing using one or more of;

compression distortion analysis, wherein the compression distortion analysis includes analysis of distortion between the received image and a processed version of the received image, the processed version of the received image resulting from a compression and subsequent decompression of the received image;

contiguous pixels analysis, wherein the contiguous pixels analysis includes computing whether a number of pixel groups from the received image, each pixel group defined by a similarity of color in at least a selected number of contiguous pixels, is above a color continuity threshold, wherein the color continuity threshold corresponds to a distinction in image types;

edge detect analysis, wherein the edge detect analysis includes computing whether an edge analysis of the received image resulted in an edge threshold indicating edge geometry or distribution, wherein the edge threshold corresponds to a distinction in image types;

or

color histogram analysis, wherein the color histogram analysis includes identifying whether a periodicity threshold is reached in a computation of a number of pixels of the received image, discretized by color, that fall within a list of color ranges that span a color space of the received image, wherein the periodicity threshold corresponds to a distinction in image types; and

a classification module, executed by the one or more processors, configured to classify the received image with an image type, wherein the image type is at least one of;

a synthetic image type or a natural image type, wherein a synthetic image type is for images generated in part by capturing reflected light and a natural image type is for images that are computer generated.

13. The system of claim 12, wherein the analysis module uses the contiguous pixels analysis.

14. The system of claim 13, wherein the contiguous pixels analysis comprises identifying that a first pixel of the contiguous pixels and a second pixel of the contiguous pixels are of similar color when one or more of a hue value or a saturation value associated with the first pixel and the second pixel are within a specified threshold level of each other.

15. The system of claim 12, wherein the analysis module uses the color histogram analysis or the edge detect analysis.

16. The system of claim 12, wherein the analysis module uses the compression distortion analysis.

17. The system of claim 16, wherein the analysis of distortion between the received image and the processed version of the received image is performed using one or more of:

a Peak Signal-to-Noise Ratio (PSNR) index;

or

a structural similarity (SSIM) index.

18. The system of claim 16, wherein the classification module is further configured to:

identify that the image type is the synthetic image type if the distortion is greater than a specified threshold; and

identify that the image type is the natural image type if the distortion is less than the specified threshold.

19. The system of claim 12 further comprising a transcoding module configured to identify a transcoding process to apply to the received image by:

identifying whether a compression is to be applied to the received image based on the image type classification of the received image;

identifying a type of compression to be applied to the received image based on the image type classification of the received image;

identifying one or more of a type or intensity of size reduction to be applied to the received image based on the image type classification of the received image;

or

identifying one or more of a type or intensity of color reduction to be applied to the received image based on the image type classification of the received image.

20. A computer-readable medium storing instructions that, when executed by a computing system, cause the computing system to perform operations comprising:

receiving, by an analysis module executed by a processor of a computer system, an image to be classified;

analyzing the received image to produce analyzing results, the analyzing using one or more of;

compression distortion analysis, wherein the compression distortion analysis includes analysis of distortion between the received image and a processed version of the received image, the processed version of the received image resulting from a compression and subsequent decompression of the received image;

contiguous pixels analysis, wherein the contiguous pixels analysis includes computing whether a number of pixel groups from the received image, each pixel group defined by a similarity of color in at least a selected number of contiguous pixels, is above a color continuity threshold, wherein the color continuity threshold corresponds to a distinction in image types;

edge detect analysis, wherein the edge detect analysis includes computing whether an edge analysis of the received image resulted in an edge threshold indicating edge geometry or distribution, wherein the edge threshold corresponds to a distinction in image types;

or

color histogram analysis, wherein the color histogram analysis includes identifying whether a periodicity threshold is reached in a computation of a number of pixels of the received image, discretized by color, that fall within a list of color ranges that span a color space of the received image, wherein the periodicity threshold corresponds to a distinction in image types; and

classifying the received image with an image type, wherein the image type is at least one of;

a synthetic image type or a natural image type, wherein a synthetic image type is for images generated in part by capturing reflected light and a natural image type is for images that are computer generated.