Multi-input multi-output time encoding and decoding machines

• US 9,013,635 B2
• Filed: 08/19/2011
• Issued: 04/21/2015
• Est. Priority Date: 06/28/2007
• Status: Expired due to Fees

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Lazar, Aurel A., Pnevmatikakis, Eftychios A.
• Primary Examiner(s): Mai, Lam T
• Application Number:
  US13/214,041
• Publication Number:
  US 20120039399A1
• Time in Patent Office: 1,341 Days
• Field of Search: 341110-172, 348/723, 375/295, 375/340
• US Class Current: 348/723
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: H03M 1/1205 Multiplexed conversion systems
• CPC Class Codes: H03M 1/125 Asynchronous, i.e. free-running operation within each conversion cycle
• CPC Class Codes: H03M 1/50 with intermediate conversion to time interval H03M1/64 takes precedence
• CPC Class Codes: H03M 1/662 Multiplexed conversion systems
• CPC Class Codes: H03M 1/82 with intermediate conversion to time interval
• CPC Class Codes: H03M 3/43 the quantiser being a single bit one
• CPC Class Codes: H03M 3/466 Multiplexed conversion systems
• CPC Class Codes: H03M 3/504 the final digital/analogue converter being constituted by a finite impulse response [FIR] filter, i.e. FIRDAC

Abstract

Methods and systems for encoding and decoding signals using a Multi-input Multi-output Time Encoding Machine (TEM) and Time Decoding Machine are disclosed herein.

Claims

1. A multiple-input, multiple-output encoder comprising:

a filtering kernel comprising a plurality (N) of sets of filters, each of the N sets of filters having a plurality (M) of filter elements; and

at least one time encoding machine (TEM), each of the M filter elements of the N sets of filter elements being additively coupled to a corresponding input of the at least one TEM.

2. The encoder of claim 1 wherein the at least one TEM comprises an integrate-and-fire neuron.

3. The encoder of claim 1 wherein the at least one TEM comprises a neuron having multiplicative coupling.

4. The encoder of claim 1 wherein the at least one TEM comprises an asynchronous sigma/delta modulator.

5. The encoder of claim 1 further comprising a plurality (N) of adders, each adder coupled to a corresponding one of the N sets of filters and one of a plurality (N) of bias values.

6. The encoder of claim 1 wherein the at least one TEM comprises an irregular sampler.

7. The encoder of claim 1 wherein the at least one TEM generates a spike sequence.

8. The encoder of claim 1 wherein the at least one TEM comprises at least two ON-OFF neurons, each neuron being interconnected and having a feedback mechanism.

9. A decoder to decode a signal encoded by a multiple-input, multiple-output TEM encoder comprising:

at least one Time Decoding Machine (TDM); and

a filtering kernel comprising a plurality (N) of sets of filters, each of the N sets of filters having a plurality (M) of filter elements, a corresponding output of the at least one TDM being coupled to a corresponding one of the N sets of filters, each of the M filter elements of each set of filters being additively coupled to a corresponding M filter element of each other set of the N sets of filters.

10. The decoder of claim 9, wherein the filtering kernel has an overall response selected to at least substantially invert an encoding process of a TEM encoder used to encode the TEM-encoded signal.

11. The decoder of claim 9, wherein the TEM-encoded signal is irregularly sampled.

12. An encoder to encode a video stream signal comprising:

a filter to receive the video stream signal, the filter having a plurality (N) of filter elements; and

a Time Encoding Machine (TEM) to generate a plurality (N) of TEM encoded spatiotemporal field signals in response to a spatiotemporal field signal received from a corresponding one of the N filter elements.

13. The encoder of claim 12, wherein the spatiotemporal field signals are described by an equation:

v^j(t)=∫

<sub>−

∞</sub>

^+∞

(∫

∫

_XD^j(x,y,s)I(x,y,t−

s)dxdy)ds,

where D^j(x,y,s) is a filter function, and I(x,y,t) represents the video stream signal.

14. The encoder of claim 12 wherein the N TEM-encoded spatiotemporal field signals are represented by a sampling function:

ψ

_k^j(x,y,t)=D(x,y,−

t)*g(t−

s_k^j), for k spike times, for each (x, y) in a bounded spatial set, where j corresponds to each of the N TEM-encoded spatiotemporal field signals, and where g(t)=sin(Ω

t)/π

t.

15. A decoder to decode a TEM-encoded video stream signal comprising:

a Time Decoding Machine (TDM) to receive a plurality (N) of TEM-encoded spatiotemporal field signals, and for each TEM-encoded spatiotemporal field signal generate a TDM-decoded spatiotemporal field signal; and

an adder to combine each of said TDM-decoded spatiotemporal field signals to recover the video stream signal.

16. A system to alter a video stream signal comprising:

a plurality (N) of TEM-filters;

a plurality (N) of reconstruction filters;

a switching matrix, operatively coupled to the plurality (N) of TEM-filters and the plurality (N) of reconstruction filters, to map a plurality (N) of TEM-encoded spatiotemporal field signals received from the N TEM-filters to the plurality (N) of reconstruction filters in a video stream signal TDM.

17. The system of claim 16, adapted to rotate the video stream signal, wherein the switching matrix maps each of the N TEM-encoded spatiotemporal field signals from a corresponding TEM-filter ([x,y], α

, θ

) to a corresponding reconstruction filter ([x,y],α

,θ

+lθ

₀), where lθ

₀ represents a desired value of rotation.

18. The system of claim 16, adapted to zoom the video stream signal, wherein the switching matrix maps each of the N TEM-encoded spatiotemporal field signals from a corresponding TEM-filter ([x,y], α

, θ

) to a corresponding reconstruction filter ([x,y], α

₀^m α

, θ

), where α

₀^m represents a desired value of zoom.

19. The system of claim 16, adapted to translate the video stream signal by a value [nb₀, kb₀], wherein the switching matrix maps each of the N TEM-encoded spatiotemporal field signals from a corresponding TEM-filter ([x,y], α

, θ

) to a corresponding reconstruction filter at ([x+nb₀, y+kb₀], α

, θ

).

20. The system of claim 16, adapted to zoom the video stream signal by a value α

₀^m and translate said video stream signal by a value [nb₀, kb₀], wherein the switching matrix maps each of the N TEM-encoded spatiotemporal field signals from a corresponding TEM-filter ([x,y], α

, θ

) to a corresponding reconstruction filter at ([x+α

₀^m nb₀, y+α

₀^m kb₀], α

₀^m α

, θ

).