Visual data processing using energy networks

• US 10,685,264 B2
• Filed: 12/27/2017
• Issued: 06/16/2020
• Est. Priority Date: 04/12/2016
• Status: Active Grant

• Current Assignee: Magic Pony Technology Limited (X Holdings Corp.)
• Original Assignee: Magic Pony Technology Limited (X Holdings Corp.)
• Inventors: Huszar, Ferenc, Bishop, Robert David, Wang, Zehan
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US15/855,807
• Publication Number:
  US 20180121769A1
• Time in Patent Office: 902 Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 5/60 using machine learning, e.g. neural networks
• CPC Class Codes: G06T 5/70 Denoising; Smoothing

Abstract

The present disclosure relates to a method for processing input visual data using a generated algorithm based upon input visual data and the output of a calculated energy function. According to a first aspect of the disclosure, there is provided a method for enhancing input visual data using an algorithm, the method comprising evaluating gradients of the output of an energy function with respect to the input visual data; using the gradient output to enhance the input visual data; and outputting the enhanced visual data.

Claims

1. A method for enhancing input visual data using an algorithm comprising:

evaluating gradients of an output of an energy function with respect to the input visual data;

generating a processing algorithm using the gradients of the output and the input visual data; and

using the processing algorithm to output enhanced visual data,

wherein the output of the energy function is a scalar, and

wherein the enhanced visual data is based on the input visual data.

2. The method of claim 1, wherein the energy function is any of an algorithm, hierarchical algorithm, computation, formula, or subroutine.

3. The method of claim 1, wherein the energy function receives one or more sections of the input visual data.

4. The method of claim 1, wherein the energy function outputs a real valued scalar.

5. The method of claim 1, wherein the energy function comprises a plurality of layers.

6. The method of claim 1, wherein evaluating or calculating the gradients of the output comprises automatic differentiation.

7. The method of claim 1, wherein evaluating or calculating the gradients of the output comprises backpropagation.

8. The method of claim 1, further comprising receiving the input visual data.

9. The method of claim 1, wherein the energy function comprises a plurality of parameters.

10. The method of claim 9, wherein one or more of the plurality of parameters are trainable.

11. The method of claim 1, wherein the processing algorithm calculates a linear combination of the input visual data it receives, and the gradients of the energy function.

12. The method of claim 11, wherein a plurality of parameters are trained via a training algorithm.

13. The method of claim 1, wherein the energy function comprises a plurality of layers, and wherein the layers are any of sequential, recurrent, recursive, branching, or merging.

14. The method of claim 1, wherein the enhanced visual data is fed back into the processing algorithm and processed further in a recurrent fashion.

15. The method of claim 14, wherein the energy function is pre-trained, fine-tuned, or trained using a learned approach.

16. The method of claim 1, wherein the input visual data comprises a plurality of sections.

17. The method of claim 16, wherein the plurality of sections comprises a plurality of low-quality inputs.

18. The method of claim 16, wherein quality can be measured subjectively.

19. An apparatus for enhancing input visual data using an algorithm comprising:

at least one processor; and

at least one memory including computer program code which, when executed by the at least one processor, causes the apparatus to;

evaluate gradients of an output of an energy function with respect to the input visual data;

generate a processing algorithm using the gradients of the output and the input visual data; and

use the processing algorithm to output enhanced visual data,

wherein the output of the energy function is a scalar, and

wherein the enhanced visual data is based on the input visual data.

20. A non-transitory computer-readable medium having computer-readable code stored thereon, the computer-readable code, when executed by at least one processor, cause the at least one processor to:

evaluate gradients of an output of an energy function with respect to an input visual data;

generate a processing algorithm using the gradients of the output and the input visual data; and

use the processing algorithm to output enhanced visual data,

wherein the output of the energy function is a scalar, and

wherein the enhanced visual data is based on the input visual data.