System and method for lossy image and video compression and transmission utilizing neural networks

• US 10,373,300 B1
• Filed: 04/29/2019
• Issued: 08/06/2019
• Est. Priority Date: 04/29/2019
• Status: Active Grant

• Current Assignee: Deep Render Ltd.
• Original Assignee: Deep Render Ltd.
• Inventors: Besenbruch, Christian Lars, Zafar, Arsalan Ali
• Primary Examiner(s): Nakhjavan, Shervin K
• Application Number:
  US16/397,725
• Time in Patent Office: 99 Days
• Field of Search: None
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06T 9/002 using neural networks
• CPC Class Codes: H04N 19/90 using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
• CPC Class Codes: H04N 7/035 Circuits for the digital non-picture data signal, e.g. for slicing of the data signal, for regeneration of the data-clock signal, for error detection or correction of the data signal
• CPC Class Codes: H04N 7/12 Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal H04N7/24 takes precedence

Abstract

A system and method for lossy image and video compression and transmission that utilizes a neural network as a function to map a known noise image to a desired or target image, allowing the transfer only of hyperparameters of the function instead of a compressed version of the image itself. This allows the recreation of a high-quality approximation of the desired image by any system receiving the hyperparameters, provided that the receiving system possesses the same noise image and a similar neural network. The amount of data required to transfer an image of a given quality is dramatically reduced versus existing image compression technology. Being that video is simply a series of images, the application of this image compression system and method allows the transfer of video content at rates greater than existing technologies in relation to the same image quality.

Claims

1. A system for lossy image and video compression and transmission utilizing neural networks, comprising:

an image compression engine comprising a first processor, a first memory, and a first plurality of programming instructions stored in the first memory, wherein the first plurality of programming instructions, when operating on the first processor, cause the first processor to;

receive a desired image;

retrieve a noise image;

map the noise image to the desired image using a first neural network to find hyperparameters such that the hyperparameters, when applied to the noise image using the first neural network, produce an approximation of the desired image within an error that is less than a pre-determined threshold; and

transmit the hyperparameters; and

an image decompression engine comprising a second processor, a second memory, and a second plurality of programming instructions stored in the memory, wherein the second plurality of programming instructions, when operating on the second processor, cause the second processor to;

receive the hyperparameters;

retrieve the noise image; and

apply the hyperparameters to the noise image using a second neural network to produce an approximation of the desired image within an error that is less than the pre-determined threshold.

2. The system of claim 1, wherein the image compression engine further comprises a dedicated 2D convolutional processor to accelerate the operation of the first neural network.

3. The system of claim 1, wherein the image decompression engine further comprises a dedicated 2D convolutional processor to accelerate the operation of the second neural network.

4. A method for lossy image and video compression and transmission utilizing neural networks, comprising the steps of:

receiving a desired image at a first computing device;

retrieving a noise image using the first computing device;

mapping, using the first computing device, the noise image to the desired image using a first neural network to find hyperparameters such that the hyperparameters, when applied to the noise image using the first neural network, produce an approximation of the desired image within an error that is less than a pre-determined threshold; and

transmitting the hyperparameters to a second computing device; and

receive the hyperparameters at a second computing device;

retrieving the noise image at the second computing device; and

applying, using the second computing device, the hyperparameters to the noise image using a second neural network to produce an approximation of the desired image within an error that is less than the pre-determined threshold.

5. The method of claim 4, wherein the image compression engine further comprises a dedicated 2D convolutional processor to accelerate the operation of the first neural network.

6. The system of claim 4, wherein the image decompression engine further comprises a dedicated 2D convolutional processor to accelerate the operation of the second neural network.