Neural network video image processor

• US 5,376,962 A
• Filed: 09/10/1993
• Issued: 12/27/1994
• Est. Priority Date: 03/31/1993
• Status: Expired due to Term

• Current Assignee: Matsushita Electric Corporation Of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Technologies, Inc. (Panasonic Holdings Corporation)
• Inventors: Zortea, Anthony
• Primary Examiner(s): Kostak, Victor R.
• Application Number:
  US08/119,873
• Time in Patent Office: 473 Days
• Field of Search: 358/21 R, 358/32, 358/37, 395/20, 395/21, 395/22, 395/23, 348/222, 348/223, 348/708
• US Class Current: 348/222.1
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 5/60 using machine learning, e.g. neural networks
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof

Abstract

A signal processing system for a video camera uses a single neural network to implement multiple nonlinear signal processing functions. In one example, the neural network implements gamma correction, contrast compression, color correction, high pass filtering and aperture correction as a combined function which is emulated by the network. The network is trained off-line using back propagation to emulate the entire composite function for a set of parameters which results in multiple sets of weighting factors. Then, using the stored multiple sets of weighting factors as initial values, the neural network is "re-trained" on-line for each new parameter setting. The use of a single neural network in place of the multiple dedicated processing functions reduces engineering effort to develop the product and may reduce the cost of the total system.

Claims

1. A signal processing system having a plurality of parameter inputs comprising:

a neural network for emulating a plurality of non-linear signal processing functions, the neural network having a plurality of input terminals for receiving a plurality of input signals;

a plurality of output terminals for providing a plurality of output signals; and

receiving means for receiving a plurality of weighting factors, wherein the weighting factors are programmable and are updated on-line each time one of the plurality of parameter inputs changes; and

means for storing a plurality of sets of predetermined weighting factors which were calculated off-line, each set corresponding to a respectively different set of values for the plurality of parameter inputs;

on-line training means for determining on-line, a set of weighting factors for each set of parameter settings for which no set off-line predetermined weighting factors exists, said determined weighting factors being determining by using, as initial values for on-line training, a set of off-line predetermined weighting factors having a corresponding plurality of parameter inputs which approximate the parameter settings; and

programmable means for applying the determined weighting factors to the receiving means to enable the neural network to emulate the plurality of nonlinear signal processing functions.

2. A signal processing system according to claim 1, wherein:

the neural network emulates a plurality of video signal processing functions; and

the input terminals of the neural network are coupled to receive signals representing red, green and blue color signals.

3. A signal processing system according to claim 1, wherein:

the signal processing system is in a color video camera;

the input terminals of the neural network are coupled to receive signals representing red, green and blue color signals; and

the plurality of signal processing functions are selected from a group consisting of gamma correction, contrast compression, high pass filtering, color balance and aperture correction.

4. A signal processing system having a plurality of parameter inputs comprising:

a neural network for emulating a plurality of non linear signal processing functions, the neural network having a plurality of input terminals for receiving a plurality of input signals;

a plurality of output terminals for providing a plurality of output signals; and

receiving means for receiving a plurality of weighting factors, wherein the weighting factors are changed on-line each time a significant change is made in one of the plurality of parameter inputs; and

means for storing a plurality of sets of predetermined weighting factors which were calculated off-line, each set corresponding to a respectively different set of values for the plurality of parameter inputs;

on-line training means for determining on-line a set of weighting factors for a new set of parameter settings for which no set of off-line predetermined weighting factors exists, said determined weighting factors being determined by using a set of off-line predetermined weighting factors, having a similar corresponding plurality of parameter inputs, as initial values for training, the on-line training means comprising;

means for determining which one of the stored sets of off-line predetermined weighting factors is to be used as initial values during training;

means for training the neural network as a function of the initial values to determine weighting factors for the new set of parameter settings; and

programmable means for applying the weighting factors to the receiving means to enable the neural network to emulate the plurality of nonlinear signal processing functions.

5. A system according to claim 4, wherein the means for training comprises:

simulator means for simulating a response of the neural network to each of the input vectors of the test patterns to produce a vector of simulated output values;

means for calculating a plurality of cost function values, each cost function value representing a mathematical function of selected ones of the simulated output values and corresponding desired output values from the desired output vector;

back propagation means, responsive to the cost function values, for modifying each of the programmable weighting factors to minimize the cost function values; and

means for determining when the weighting factors have converged to values which allow the neural network to emulate the signal processing function with acceptable error.

6. A system according to claim 5 wherein each cost function value is a function of all of the simulated output values of the neural network.

7. A method of processing a plurality of input signals according to a plurality of nonlinear signal processing functions to produce a plurality of output signals, in a system which includes a neural network having a plurality of programmable weighting factors, the method comprising the steps of:

calculating off-line a plurality of sets of weighting factors corresponding to a plurality of predetermined sets of parameter input settings;

storing the plurality of sets of weighting factors for use during on-line operation of the neural network;

training the neural network on-line for each set of parameter input settings which is not equivalent to one of the predetermined sets of parameter settings wherein initial values used for on-line training are directly related to the stored sets of off-line calculated weighting factors; and

processing the input signals using the neural network to produce the output signals.

8. The method according to claim 7 wherein the step of training the neural network further includes the steps of:

setting the initial values for the neural network as a function of at least one set of off-line calculated sets of weighting factors;

generating a plurality of test patterns, representing the composite nonlinear signal processing function, each pattern in the plurality of test patterns containing an input vector and an output vector;

simulating a response of the neural network to the first plurality of test patterns and modifying the initial weighting factors of the neural network to conform the simulated response of the neural network to the output vectors contained in the first plurality of test patterns to generate a plurality of new values for the weighting factors;

repeating, as necessary, the step of simulating until the new values for the weighting factors conform the response of the neural network to within acceptable predetermined limits of the simulated response; and

applying the final new values of the weighting factors to the neural network.

9. The method according to claim 8 wherein the step of simulating is actually performed using the neural network.

10. A signal processing system having a plurality of parameter inputs comprising:

a neural network for emulating a plurality of nonlinear signal processing functions, the neural network having a plurality of input terminals for receiving a plurality of input signals;

a plurality of output terminals for providing a plurality of output signals; and

receiving means for receiving a plurality of programmable weighting factors, wherein the programmable weighting factors are changed on-line each time a significant change is made in one of the plurality of parameter inputs; and

means for storing a plurality of sets of predetermined programmable weighting factors, each set corresponding to a respectively different set of values for the plurality of parameter inputs;

on-line training means for determining on-line a set of weighting factors for a new set of parameter settings for which no set of weighting factors exists, said determined weighting factors being a function of at least one of the stored sets of weighting factors, the on-line training means comprising;

means for determining which one of the stored sets of weighting factors is to be used as initial values during training;

means for training the neural network as a function of the initial values to determine weighting factors for the new set of parameter settings, wherein the means for training comprises;

simulator means for simulating a response of the neural network to each of the input vectors of the test patterns to produce a vector of simulated output values;

means for calculating a plurality of cost function values, each cost function value representing a mathematical function of selected ones of the simulated output values and corresponding desired output values from the desired output vector;

back propagation means, responsive to the cost function values, for modifying each of the programmable weighting factors to minimize the cost function values; and

means for determining when the weighting factors have converged to values which allow the neural network to emulate the signal processing function with acceptable error; and

programmable means for applying the weighting factors to the receiving means to enable the neural network to emulate the plurality of nonlinear signal processing functions,

wherein the system is a color video signal processing system which produces at least first and second color signal values as respective first and second output values in both the simulated and desired output vectors, and the cost function is a function of a difference between a ratio of the simulated first and second output values and a ratio of the desired first and second output values.

11. A signal processing system having a plurality of parameter inputs comprising:

a neural network for emulating a plurality of nonlinear signal processing functions, the neural network having a plurality of input terminals for receiving a plurality of input signals;

a plurality of output terminals for providing a plurality of output signals; and

receiving means for receiving a plurality of programmable weighting factors, wherein the programmable weighting factors are changed on-line each time a significant change is made in one of the plurality of parameter inputs; and

means for storing a plurality of sets of predetermined programmable weighting factors, each set corresponding to a respectively different set of values for the plurality of parameter inputs;

on-line training means for determining on-line a set of weighting factors for a new set of parameter settings for which no set of weighting factors exists, said determined weighting factors being a function of at least one of the stored sets of weighting factors, the on-line training means comprising;

means for determining which one of the stored sets of weighting factors is to be used as initial values during training;

means for training the neural network as a function of the initial values to determine weighting factors for the new set of parameter settings, wherein the means for training comprises;

simulator means for simulating a response of the neural network to each of the input vectors of the test patterns to produce a vector of simulated output values;

means for calculating a plurality of cost function values, each cost function value representing a mathematical function of selected ones of the simulated output values and corresponding desired output values from the desired output vector;

back propagation means, responsive to the cost function values, for modifying each of the programmable weighting factors to minimize the cost function values; and

means for determining when the weighting factors have converged to values which allow the neural network to emulate the signal processing function with acceptable error; and

programmable means for applying the weighting factors to the receiving means to enable the neural network to emulate the plurality of nonlinear signal processing functions,

wherein the system is a color video signal processing system which produces at least first and second color signal values as respective first and second ones of both the simulated and desired output values, and the cost function is a function of a combination of the first and second simulated output values and the first and second desired output values which represents a difference between a first luminance value produced by the simulated output values and a second luminance value produced by corresponding desired output values.