Neural network trained with spatial errors

• US 7,409,372 B2
• Filed: 06/20/2003
• Issued: 08/05/2008
• Est. Priority Date: 06/20/2003
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Greig, Darryl, Staelin, Carl, Maurer, Ron, Flacher, Manl
• Primary Examiner(s): Holmes; Michael B
• Application Number:
  US10/600,671
• Publication Number:
  US 20040260662A1
• Time in Patent Office: 1,873 Days
• Field of Search: 706/45, 382/155, 382/254, 382/293, 382/298
• US Class Current: 706/15
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

A neural network is trained with input data. The neural network is used to rescale the input data. Errors for the rescaled values are determined, and neighborhoods of the errors are used adjust connection weights of the neural network.

Claims

1. A method of using a computer to train a computer-based neural network with input image data, the neural network including a plurality of connection weights, the method comprising:

using the neural network to rescale the input image data;

determining errors for the rescaled data; and

using neighborhoods of the errors to adjust the connection weights.

2. The method of claim 1, wherein the input data represents a set of images, wherein the data rescaling includes image upscaling, and wherein the neighborhoods are spatial error neighborhoods.

3. The method of claim 1, wherein the error neighborhoods are used with a non-gradient algorithm to adjust the connection weights.

4. The method of claim 1, wherein the error neighborhoods are used to generate derivatives of total error with respect to a neighborhood of errors;

wherein gradients are computed from the derivatives; and

wherein the gradients are used to adjust the connection weights.

5. The method of claim 4, wherein each derivative is computed as the sum of the partial derivatives of the errors in an error neighborhood.

6. The method of claim 4, wherein each derivative of total error with respect to a neighborhood of errors is proportional to a product of a penalty matrix and an error vector, the error vector describing the neighborhood of errors, the penalty matrix punishing any spatially correlated errors.

7. The method of claim 6, wherein the penalty matrix is positive definite, and includes weights that penalize undesirable patterns of errors.

8. The method of claim 6, wherein the penalty matrix is based on use of a pattern detector that detects the spatially correlated errors.

9. The method of claim 1, wherein determining the errors includes forming an error image from the rescaled data, identifying patterns in the error image, and punishing the spatially correlated errors in the error image.

10. The method of claim 1, wherein input and output data of the neural network are coded to improve the neural network accuracy.

11. The method of claim 1, wherein the errors are a combination of SSE and spatial errors.

12. The method of claim 11, wherein SSE is applied to crisp edges and spatial errors are applied to blurred edges.

13. A method of upscaling an input image, the method comprising using the neural network trained according to claim 1.

14. The method of claim 13, wherein the input and upscaled images are color images;

wherein the input image is upscaled by pixel replication;

a luminance channel of the input image is upscaled by the neural network; and

the upscaled luminance channel and the pixel-replicated image are used to generate the upscaled color image.

15. The method of claim 14, wherein the using the upscaled luminance channel and the pixel-replicated image include adding deltas to pixels in the pixel-replicated image, each delta computed as the difference between the corresponding luminance value in the upscaled luminance channel and the corresponding luminance value in the input luminance channel.

16. The method of claim 15, wherein using the upscaled luminance channel and the pixel-replicated image further includes gamut mapping the upscaled image.

17. An article comprising computer memory encoded with data upscaled by the neural network trained according to the method of claim 1.

18. Apparatus comprising a processor programmed with a neural network, the network trained according to the method of claim 1.

19. A method of using image input data and target data to train a neural network for image upscaling, the method comprising:

using the neural network to generate predicted values from the input data;

determining errors for the predicted values, the error for each predicted value a function of differences between predicted values in a spatial neighborhood and the corresponding values in the target data; and

back-propagating the errors through the neural network.

20. Apparatus for training a neural network on input data, the apparatus comprising:

means for using the neural network to rescale the input data;

means for determining errors for the rescaled data; and

means for using neighborhoods of the errors to adjust the connection weights.

21. Apparatus for training a neural network on input data to perform image rescaling, the neural network having a plurality of connection weights, the apparatus comprising a processor programmed to use the neural network to rescale the input data;

determine errors for the rescaled data; and

use neighborhoods of the errors to adjust the connection weights of the neural network.

22. The apparatus of claim 21, wherein the input data represents images, wherein the neural network upscales the image data, and wherein the neighborhoods are spatial error neighborhoods.

23. The apparatus of claim 21, wherein the processor is programmed to use the error neighborhoods and a non-gradient algorithm to adjust the connection weights.

24. The apparatus of claim 21, wherein the error neighborhoods are used to generate derivatives of total error with respect to a neighborhood of errors;

wherein gradients are computed from the derivatives; and

wherein the gradients are used to adjust the connection weights.

25. The apparatus of claim 24, wherein each derivative is computed as the sum of the partial derivatives of the errors in an error neighborhood.

26. The apparatus of claim 24, wherein each derivative of total error with respect to a neighborhood of errors is proportional to a product of a penalty matrix and an error vector, the error vector describing the neighborhood of errors, the penalty matrix punishing any spatially correlated errors.

27. The apparatus of claim 26, wherein the penalty matrix is positive definite, and includes weights that penalize undesirable patterns of errors.

28. The apparatus of claim 26, wherein the penalty matrix is based on use of a pattern detector that detects the spatially correlated errors.

29. The apparatus of claim 21, wherein determining the errors includes forming an error image from the rescaled data, identifying patterns in the error image, and punishing the spatially correlated errors in the error image.

30. The apparatus of claim 21, wherein the processor is programmed to code input and output data of the neural network to improve the neural network accuracy.

31. The apparatus of claim 21, wherein the errors are a combination of SSE and spatial errors.

32. The apparatus of claim 21, wherein the input and upscaled images are color images;

wherein the processor is programmed to upscale the input image by pixel replication, use the neural network to upscale a luminance channel of the input image; and

generate the upscaled color image from the upscaled luminance channel and the pixel-replicated image.

33. The apparatus of claim 32, wherein the processor is further programmed to perform gamut mapping of the upscaled image.

36. The apparatus of claim 32, further comprising means for gamut mapping the rescaled color image.

34. Apparatus for rescaling a color image, the apparatus comprising:

means for rescaling the input image by pixel replication;

a neural network that has been trained to rescale a luminance channel of the color image, the neural network for producing a rescaled luminance image; and

means for using the rescaled luminance image and the pixel-replicated image to generate a rescaled color image.

35. The apparatus of claim 34, wherein the use of the rescaled luminance image and the pixel-replicated image includes adding deltas to pixels in the pixel-replicated image, each delta computed as the difference between the corresponding luminance value in the rescaled luminance image and the corresponding luminance value in the input luminance channel.

37. An article for causing a processor to use input image data to adjust connection weights of a neural network so the neural network can perform image rescaling, the article comprising:

computer memory; and

data encoded in the computer memory, the data causing the processor to use the neural network to rescale the input image data;

determine errors for the rescaled data; and

use neighborhoods of the errors to adjust the connection weights of the neural network.