Method and apparatus for training a neural network to learn and use fidelity metric as a control mechanism

• US 6,075,884 A
• Filed: 03/28/1997
• Issued: 06/13/2000
• Est. Priority Date: 03/29/1996
• Status: Expired due to Term

• Current Assignee: Asev Display Labs Inc. (AmTRAN Technology Co., Ltd.)
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Lubin, Jeffrey, Peterson, Heidi A., Spence, Clay D., de Vries, Aalbert
• Primary Examiner(s): Johns, Andrew W.
• Application Number:
  US08/828,161
• Time in Patent Office: 1,173 Days
• Field of Search: 382/159, 382/260, 382/156, 382/218, 382/238, 382/239, 395/21, 706/15, 706/20, 706/22, 706/23, 348/404, 348/845.1, 358/430
• US Class Current: 382/156
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06V 30/194 References adjustable by an adaptive method, e.g. learning
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/179 the unit being a scene or a shot
• CPC Class Codes: H04N 19/192 the adaptation method, adaptation tool or adaptation type being iterative or recursive
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/198 including smoothing of a sequence of encoding parameters, e.g. by averaging, by choice of the maximum, minimum or median value
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/87 involving scene cut or scene change detection in combination with video compression

Abstract

A signal processing apparatus and concomitant method for learning and using fidelity metric as a control mechanism and to process large quantities of fidelity metrics from a visual discrimination measure (VDM) to a manageable subjective image quality ratings. The signal processing apparatus incorporates a VDM and a neural network. The VDM receives input image sequences and generates fidelity metrics, which are received by a neural network. The neural network is trained to learn and use the fidelity metrics as a control mechanism, e.g., to control a video encoder.

Claims

1. Apparatus for processing a plurality of input signals comprising:

a perceptual metric generator for determining a perceptual metric that represents the fidelity between two of said input signals; and

a first neural network, coupled to said perceptual metric generator, for evaluating said perceptual metric to produce a control signal.

2. The apparatus of claim 1, wherein said plurality of input signals are image signals.

3. The apparatus of claim 1, wherein said control signal is used for controlling an encoder having a plurality of encoder parameters.

4. The apparatus of claim 3, wherein said control signal is used for selecting encoder parameters of said encoder.

5. The apparatus of claim 1, wherein said control signal is used to generate fidelity ratings.

6. The apparatus of claim 1, further comprising:

an encoder, coupled to said first neural network, for encoding the plurality of input signals, wherein said control signal is used for selecting encoder parameters of said encoder.

7. The apparatus of claim 1, further comprising:

an encoder, coupled to said first neural network, for encoding the plurality of input signal; and

a controller, coupled to said first neural network, for controlling a plurality of encoder parameters of said encoder in response to said control signal of said first neural network.

8. The apparatus of claim 1, wherein each of said input signal is decomposed into a set of varying resolution signals, and wherein said perceptual metric generator determines a perceptual metric between two of said varying resolution signals.

9. The apparatus of claim 8, further comprising a plurality of neural networks, coupled to said first neural network in a hierarchical manner, where each of said neural network evaluates a different perceptual metric generated from different sets of varying resolution signals.

10. Apparatus for processing a plurality of input signals comprising:

a first neural network for determining a perceptual metric that represents the fidelity between two of said input signals;

a second neural network, coupled to said first neural network, for evaluating said perceptual metric to produce a control signal; and

an encoder, coupled to said second neural network, for encoding the plurality of input signals, where said control signal is used for selecting encoder parameters of said encoder.

11. Method for processing a plurality of input signals comprising the steps of:

(a) determining a perceptual metric that represents the fidelity between two of said input signals; and

(b) using a first neural network to evaluate said perceptual metric to produce a control signal.

12. The method of claim 11, wherein said plurality of input signals are image signals.

13. The method of claim 11, further comprising the step of:

(c) using said control signal to control an encoder having a plurality of encoder parameters.

14. The method of claim 13, wherein said using step (c) uses said control signal to select said encoder parameters of said encoder.

15. The method of claim 11, further comprising the step of:

(c) using said control signal to generate fidelity ratings.

16. The method of claim 11, further comprising the step of:

(c) encoding the plurality of input signals; and

(d) using said control signal to control said encoding step (c).

17. The method of claim 11, further comprising the step of:

(c) decomposing each of said input signal into a set of varying resolution signals, and wherein said determining step (a) determines a perceptual metric between two of said varying resolution signals.

18. The method of claim 17, wherein said using step (b) comprises the step of using a plurality of neural networks, where each of said neural network evaluates a different perceptual metric generated from different sets of varying resolution signals.

19. The method of claim 11, further comprising the step of:

(c) decomposing each of said input signal into a set of regions, and wherein said determining step (a) determines a perceptual metric between said regions of two input signals.