Encoding and decoding machine with recurrent neural networks

• US 8,874,496 B2
• Filed: 07/23/2013
• Issued: 10/28/2014
• Est. Priority Date: 02/09/2011
• Status: Active Grant

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Lazar, Aurel A., Zhou, Yiyin
• Primary Examiner(s): CHANG, LI WU
• Application Number:
  US13/948,615
• Publication Number:
  US 20130311412A1
• Time in Patent Office: 462 Days
• Field of Search: None
• US Class Current: 706/16
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: H03M 1/82 with intermediate conversion to time interval
• CPC Class Codes: H03M 3/502 Details of the final digital/analogue conversion following the digital delta-sigma modulation

Abstract

Techniques for reconstructing a signal encoded with a time encoding machine (TEM) using a recurrent neural network including receiving a TEM-encoded signal, processing the TEM-encoded signal, and reconstructing the TEM-encoded signal with a recurrent neural network.

Claims

1. A method of reconstructing a signal encoded with a time encoding machine (TEM) using a recurrent neural network, comprising:

a. receiving a TEM-encoded signal;

b. processing the TEM-encoded signal for input into a recurrent neural network;

c. reconstructing the TEM-encoded signal with the recurrent neural network, comprising;

d. formulating the reconstruction into a variational problem having a solution equal to the series of sums of a sequence of functions multiplied by a sequence of coefficients, wherein the coefficients can be obtained by solving a quadratic problem;

e. solving the quadratic problem with a recurrent neural network, by integrating time derivatives and providing the outputs back to the recurrent neural network, thereby generating the coefficients for the solution; and

f. reconstructing the signal with the coefficients.

2. The method of claim 1, wherein the quadratic problem is an unconstrained quadratic problem with an explicit analytical solution.

3. The method of claim 2, wherein the recurrent neural network solves the unconstrained quadratic problem using a general gradient approach.

4. The method of claim 3, wherein the recurrent neural network has a plurality of outputs, each output corresponding to a coefficient.

5. The method of claim 1, wherein reconstructing further comprises

a. formulating the reconstruction into a variational problem having a solution equal to the series of sums of a sequence of functions multiplied by a sequence of coefficients, wherein the coefficients can be obtained by solving a spline interpolation problem;

b. solving the interpolation problem with a recurrent neural network, thereby generating the coefficients to the solution; and

c. reconstructing the signal with the coefficients.

6. The method of claim 5, wherein the spline interpolation problem is reformulated as a standard quadratic programming problem.

7. The method of claim 1, wherein the TEM-encoded signal is one dimensional, and wherein reconstructing further comprises:

a. formulating the reconstruction as the variational problem

8. The method of claim 7, wherein a set of differential equations,

9. The method of claim 8, wherein the recurrent neural network solves the unconstrained quadratic problem using a general gradient approach.

10. The method of claim 1, wherein the TEM-encoded signal is one dimensional, and wherein reconstructing further comprises:

a. formulating the reconstruction as a spline interpolation problem,

11. The method of claim 10, wherein the quadratic problem is reformulated as a standard quadratic programming problem, setting x=[x₊^Tx₋

^T]^T and imposing x₊≧

0, x₋

≧

0 such that c=x₊−

x₋

, giving a convex programming problem

12. The method of claim 1, wherein the TEM-encoded signal is a multidimensional signal encoded with a video time encoding machine (vTEM).

13. The method of claim 12, wherein reconstructing further comprises:

a. formulating the reconstruction as the variational problem

14. The method of claim 13, wherein a set of differential equations,

15. A system for reconstructing a signal encoded with a time encoding machine (TEM) using a recurrent neural network circuit, comprising:

a. at least one input for receiving a TEM-encoded signal, wherein the TEM-encoded signal comprises at least one spike train;

b. at least one processor electrically coupled with the at least one input for determining characteristics of the at least one spike;

c. a recurrent neural network circuit electrically coupled with the at least one processor, wherein the recurrent neural network circuit is configured with reference to the characteristics of the at least one spike train, and wherein reconstructing the encoded signal is formulated into a variational problem having a solution equal to the series of sums of a sequence of functions multiplied by a sequence of coefficients, wherein the coefficients can be obtained by solving a quadratic problem;

d. the recurrent neural network circuit comprises an arrangement of adders, integrators, multipliers, and/or piecewise linear activators capable of solving the quadratic problem by integrating time derivatives and providing the outputs back to the recurrent neural network circuit, thereby generating the coefficients for the solution; and

e. at least one output electrically coupled to the recurrent neural network circuit for providing a reconstructed signal.

16. The system of claim 15, wherein the recurrent neural network circuit is arranged into three layers, comprising:

a. a first layer, consisting of a plurality (n) of multiply/add units, wherein the vector (G+nλ

I))c−

q is computed, wherein the multiplication factors are entries of the matrix G+nλ

I and the vector q;

b. a second layer, consisting of a plurality (n) of multiply/add units, wherein Error! Bookmark not defined. ∇

E(c) is evaluated, where ∇

E(c)=G^T(G+nλ

I))c−

q), and wherein multiplication factors are provided by the entries of matrix G;

c. a third layer, consisting of a plurality (n) of multiply/add units, wherein the gradient is weighted by the learning rate μ

, and wherein the outputs of the third layer provide the time derivative of the vector c(t), the outputs of the third layer being integrated and fed back into the first layer.

17. The system of claim 15, wherein the recurrent neural network is arranged into a single layer, comprising:

a. a plurality (n) of multiply/add units, wherein a vector (q=Gc) is computed, and wherein a plurality of multiplication factors are entries of a matrix G and the vector q;

b. a plurality of outputs, wherein the outputs provide a time derivative of a vector c(t), the outputs being integrated and fed back into the first layer.

18. The system of claim 15, wherein the at least one input comprises a single input, and the at least one output comprises a plurality of outputs.

19. The system of claim 15, wherein the at least one input comprises a plurality of inputs, and the at least one output comprises a plurality of outputs.