NOISE-ENHANCED CONVOLUTIONAL NEURAL NETWORKS

• US 20160019459A1
• Filed: 07/20/2015
• Published: 01/21/2016
• Est. Priority Date: 07/18/2014
• Status: Active Grant

• Current Assignee: University of Southern California
• Original Assignee: University of Southern California
• Inventors: Audhkhasi, Kartik, Kosko, Bart, Osoba, Osonde
• Granted Patent: US 11,256,982 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 10/70 using pattern recognition or machine learning optical pattern recognition or electronic computations therefor G06V10/88
• CPC Class Codes: G06V 30/1914 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries, e.g. user dictionaries
• CPC Class Codes: G06V 30/194 References adjustable by an adaptive method, e.g. learning

Abstract

A learning computer system may include a data processing system and a hardware processor and may estimate parameters and states of a stochastic or uncertain system. The system may receive data from a user or other source; process the received data through layers of processing units, thereby generating processed data; apply masks or filters to the processed data using convolutional processing; process the masked or filtered data to produce one or more intermediate and output signals; compare the output signals with reference signals to generate error signals; send and process the error signals back through the layers of processing units; generate random, chaotic, fuzzy, or other numerical perturbations of the received data, the processed data, or the output signals; estimate the parameters and states of the stochastic or uncertain system using the received data, the numerical perturbations, and previous parameters and states of the stochastic or uncertain system; determine whether the generated numerical perturbations satisfy a condition; and, if the numerical perturbations satisfy the condition, inject the numerical perturbations into the estimated parameters or states, the received data, the processed data, the masked or filtered data, or the processing units.

Claims

1. A learning computer system that estimates parameters and states of a stochastic or uncertain system comprising a data processing system that includes a hardware processor that has a configuration that:

receives data from a user or other source;

processes the received data through layers of processing units, thereby generating processed data;

applies masks or filters to the processed data using convolutional processing;

processes the masked or filtered data to produce one or more intermediate and output signals;

compares the output signals with reference signals to generate error signals;

sends and processes the error signals back through the layers of processing units;

generates random, chaotic, fuzzy, or other numerical perturbations of the received data, the processed data, or the output signals;

estimates the parameters and states of the stochastic or uncertain system using the received data, the numerical perturbations, and previous parameters and states of the stochastic or uncertain system;

determines whether the generated numerical perturbations satisfy a condition; and

if the numerical perturbations satisfy the condition, injects the numerical perturbations into the estimated parameters or states, the received data, the processed data, the masked or filtered data, or the processing units.

2. The learning computer system of claim 1 wherein the learning computer system unconditionally injects noise or chaotic or other perturbations into the estimated parameters or states, the received data, the processed data, the masked or filtered data, or the processing units.

3. The learning computer system of claim 2 wherein the unconditional injection speeds up learning by the learning computer system.

4. The learning computer system of claim 2 wherein the unconditional injection improves the accuracy of the learning computer system.

5. The learning computer system of claim 1 wherein the received data represents an image.

6. The learning computer system of claim 1 wherein the injection speeds up learning by the learning computer system.

7. The learning computer system of claim 1 wherein the injection improves the accuracy of the learning computer system.

8. A learning computer system that estimates parameters and states of a stochastic or uncertain system comprising a data processing system that includes a hardware processor that has a configuration that:

receives data from a user or other source;

processes only a portion of the received data through layers of processing units, thereby generating processed data;

processes the masked or filtered data to produce one or more intermediate and output signals;

compares the output signals with reference signals to generate error signals;

sends and processes the error signals back through the layers of processing units;

generates random, chaotic, fuzzy, or other numerical perturbations of the portion of the received data, the processed data, or the output signals;

estimates the parameters and states of the stochastic or uncertain system using the portion of the received data, the numerical perturbations, and previous parameters and states of the stochastic or uncertain system;

determines whether the generated numerical perturbations satisfy a condition; and

if the numerical perturbations satisfy the condition, injects the numerical perturbations into the estimated parameters or states, the portion of the received data, the processed data, the masked or filtered data, or the processing units.

9. The learning computer system of claim 8 wherein the system applies masks or filters to the processed data using convolutional processing.

10. The learning computer system of claim 8 wherein the injection speeds up learning by the learning computer system.

11. The learning computer system of claim 8 wherein the injection improves the accuracy of the learning computer system.

12. The learning computer system of claim 8 wherein the system injects the random, chaotic, fuzzy, or other numerical perturbations into the portion of the received data.

13. The learning computer system of claim 8 wherein the learning computer system unconditionally injects noise or chaotic or other perturbations into the estimated parameters or states, the portion of the received data, the processed data, the masked or filtered data, or the processing units.

14. The learning computer system of claim 8 wherein the unconditional injection speeds up learning by the learning computer system.

15. The learning computer system of claim 8 wherein the unconditional injection improves the accuracy of the learning computer system.

16. A non-transitory, tangible, computer-readable storage medium containing a program of instructions that causes a computer learning system comprising a data processing system that includes a hardware processor running the program of instructions to estimate parameters and states of a stochastic or uncertain system by:

receiving data from a user or other source;

processing the received data through layers of processing units, thereby generating processed data;

applying masks or filters to the processed data using convolutional processing;

processing the masked or filtered data to produce one or more intermediate and output signals;

comparing the output signals with reference signals to generate error signals;

sending and processing the error signals back through the layers of processing units;

generating random, chaotic, fuzzy, or other numerical perturbations of the received data, the processed data, or the output signals;

estimating the parameters and states of the stochastic or uncertain system using the received data, the numerical perturbations, and previous parameters and states of the stochastic or uncertain system;

determining whether the generated numerical perturbations satisfy a condition; and

if the numerical perturbations satisfy the condition, injecting the numerical perturbations into the estimated parameters or states, the received data, the processed data, the masked or filtered data, or the processing units.

17. The storage medium of claim 16 wherein the program of instructions causes the computer learning system to unconditionally inject noise or chaotic or other perturbations into the estimated parameters or states, the received data, the processed data, the masked or filtered data, or the processing units.

18. The storage medium of claim 17 wherein the unconditional injection speeds up learning by the learning computer system.

19. The storage medium of claim 17 wherein the unconditional injection improves the accuracy of the learning computer system.

20. The storage medium of claim 16 wherein the received data represents an image.

21. The storage medium of claim 16 wherein the injection speeds up learning by the learning computer system.

22. The storage medium of claim 16 wherein the injection improves the accuracy of the learning computer system.

23. A non-transitory, tangible, computer-readable storage medium containing a program of instructions that causes a computer learning system comprising a data processing system that includes a hardware processor running the program of instructions to estimate parameters and states of a stochastic or uncertain system by:

receiving data from a user or other source;

processing only a portion of the received data through layers of processing units, thereby generating processed data;

processing the masked or filtered data to produce one or more intermediate and output signals;

comparing the output signals with reference signals to generate error signals;

sending and processing the error signals back through the layers of processing units;

generating random, chaotic, fuzzy, or other numerical perturbations of the portion of the received data, the processed data, or the output signals;

estimating the parameters and states of the stochastic or uncertain system using the portion of the received data, the numerical perturbations, and previous parameters and states of the stochastic or uncertain system; and

determining whether the generated numerical perturbations satisfy a condition;

if the numerical perturbations satisfy the condition, injecting the numerical perturbations into the estimated parameters or states, the portion of the received data, the processed data, the masked or filtered data, or the processing units.

24. The storage medium of claim 23 wherein the program of instructions causes the computer learning system to apply masks or filters to the processed data using convolutional processing.

25. The storage medium of claim 23 wherein the injection speeds up learning by the learning computer system.

26. The storage medium of claim 23 wherein the injection improves the accuracy of the learning computer system.

27. The storage medium of claim 23 wherein the program of instructions causes the computer learning system to inject the random, chaotic, fuzzy, or other numerical perturbations into the portion of the received data.

28. The storage medium of claim 23 wherein the program of instructions causes the computer learning system to to unconditionally inject noise or chaotic or other perturbations into the estimated parameters or states, the portion of the received data, the processed data, the masked or filtered data, or the processing units.

29. The storage medium of claim 23 wherein the unconditional injection speeds up learning by the learning computer system.

30. The storage medium of claim 23 wherein the unconditional injection improves the accuracy of the learning computer system.