Methods and systems for differentiating synthetic and non-synthetic images

• US 9,558,422 B2
• Filed: 01/22/2016
• Issued: 01/31/2017
• Est. Priority Date: 02/28/2013
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Lerios, Apostolos, Stoop, Dirk John, Mack, Ryan, Bourdev, Lubomir Dimitrov, Paluri, Balmanohar
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US15/004,656
• Publication Number:
  US 20160140415A1
• Time in Patent Office: 375 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/27 involving both synthetic and natural picture components, e.g. synthetic natural hybrid coding [SNHC]
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

The techniques introduced here include a system and method for transcoding multimedia content based on the results of content analysis. The determination of specific transcoding parameters, used for transcoding multimedia content, can be performed by utilizing the results of content analysis of the multimedia content. One of the results of the content analysis is the determination of image type of any images included in the multimedia content. The content analysis uses one or more of several techniques, including analyzing content metadata, examining colors of contiguous pixels in the content, using histogram analysis, using compression distortion analysis, analyzing image edges, or examining user provided inputs. Transcoding the multimedia content can include adapting the content to the constraints in delivery and display, processing and storage of user computing devices.

Claims

1. A method comprising:

receiving an image;

analyzing, using a processer of a computer system, the received image to produce analyzing results;

assigning, based on the analyzing results, an image type to the received image, wherein the image type corresponds to at least one of;

a synthetic image type or a natural image type, wherein the natural image type is for images generated in part by capturing reflected light and the synthetic image type is for images that are computer generated; and

transcoding the received image, wherein transcoding of the received image is based on the image type.

2. The method of claim 1, wherein analyzing the received image comprises compression distortion analysis, wherein the compression distortion analysis includes analysis of distortion between the received image and a processed version of the received image.

3. The method of claim 2, wherein the analysis of distortion between the received image and the processed version of the received image is performed using one or more of:

a Peak Signal-to-Noise Ratio (PSNR) index;

or

a structural similarity (SSIM) index.

4. The method of claim 2, further comprising:

identifying that the image type is the synthetic image type if the distortion is greater than a specified threshold; and

identifying that the image type is the natural image type if the distortion is less than the specified threshold.

5. The method of claim 1, wherein analyzing the received image comprises contiguous pixels analysis, wherein the contiguous pixels analysis includes computing whether a number of pixel groups from the received image is above a color continuity threshold, wherein the color continuity threshold corresponds to a distinction in image types.

6. The method of claim 5, wherein the contiguous pixels analysis comprises identifying that a first pixel of the contiguous pixels and a second pixel of the contiguous pixels are of similar color when one or more of a hue value or a saturation value associated with the first pixel and the second pixel are within a specified threshold level of each other.

7. The method of claim 1, wherein analyzing the received image comprises an analysis of edges detected in the received image.

8. A non-transitory computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform operations comprising:

receiving an image;

analyzing the received image to produce analyzing results; and

assigning, based on the analyzing results, an image type to the received image, wherein the image type corresponds to at least one of;

a synthetic image type or a natural image type, wherein the natural image type is for images generated in part by capturing reflected light and the synthetic image type is for images that are computer generated.

9. The computer-readable storage medium of claim 8, wherein the operations further comprise:

identifying a plurality of transcoding parameters, wherein at least one of the transcoding parameters is selected based on the image type assigned to the received image; and

transcoding the received image using the plurality of transcoding parameters.

10. The computer-readable storage medium of claim 9, wherein the transcoding the received image comprises converting the image from one format to another based on whether the image type is the synthetic image type.

11. The computer-readable storage medium of claim 9, wherein transcoding the received image comprises adding redundancy to protect against data loss.

12. The computer-readable storage medium of claim 9,

wherein assigning the image type comprises assigning the natural image type;

and

wherein transcoding the received image comprises converting, based on the natural image type being assigned to the received image, the received image into a GIF compression format.

13. The computer-readable storage medium of claim 9,

wherein assigning the image type comprises assigning the synthetic image type;

and

wherein transcoding the received image comprises setting, based on the synthetic image type being assigned to the received image, a parameter indicating that no resizing of the received image should be performed.

14. The computer-readable storage medium of claim 8,

wherein assigning the image type comprises assigning the natural image type;

and

wherein the operations further comprise determining, based on the natural image type being assigned to the received image, to run facial recognition on the received image.

15. The computer-readable storage medium of claim 8,

wherein assigning the image type comprises assigning the natural image type;

and

wherein the operations further comprise selecting, based on the natural image type being assigned to the received image, a set of photo editing tools to provide to a user that correspond to natural images.

16. The computer-readable storage medium of claim 15, wherein the set of photo editing tools comprises a red eye reduction tool.

17. The computer-readable storage medium of claim 8, wherein the image type further corresponds to one or more of a color image or a black-and-white image.

18. A system comprising:

a memory;

one or more processors;

an interface configured to receive an image; and

a content analyzer configured to;

analyze the received image to produce analyzing results; and

assign, based on the analyzing results, an image type to the received image, wherein the image type corresponds to at least one of;

a synthetic image type or a natural image type, wherein the natural image type is for images generated in part by capturing reflected light and the synthetic image type is for images that are computer generated.

19. The system of claim 18 further comprising a transcoder configured to identify a transcoding process to apply to the received image by one or more of:

identifying whether a compression is to be applied to the received image based on the image type assigned to the received image;

identifying a type of compression to be applied to the received image based on the image type assigned to the received image;

identifying one or more of a type or intensity of size reduction to be applied to the received image based on the image type assigned to the received image;

identifying one or more of a type or intensity of color reduction to be applied to the received image based on the image type assigned to the received image;

or

any combination thereof.

20. The system of claim 18, wherein:

the image type is the natural image type; and

the received image is one or more of;

a photograph;

a digital version of a photograph, wherein the digital version is generated by scanning;

a camera-produced image;

or

any combination thereof.