Lens distortion correction using a neurosynaptic system

• US 10,043,248 B2
• Filed: 04/24/2017
• Issued: 08/07/2018
• Est. Priority Date: 06/25/2014
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Andreopoulos, Alexander, Gruhl, Daniel F., Karasick, Michael, Modha, Dharmendra S.
• Primary Examiner(s): Wang, Carol
• Application Number:
  US15/495,472
• Publication Number:
  US 20170287119A1
• Time in Patent Office: 470 Days
• CPC Class Codes: G06F 17/11 for solving equations , e.g. nonlinear equations, general mathematical optimization problems optimization specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/20172 Image enhancement details
• CPC Class Codes: G06T 3/4046 using neural networks
• CPC Class Codes: G06T 5/60 using machine learning, e.g. neural networks
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 30/194 References adjustable by an adaptive method, e.g. learning

Abstract

One or more embodiments provide a system and circuit for image distortion correction. The system includes neurosynaptic core circuits that: receive a set of inputs comprising image dimensions and pixel distortion coefficients for at least one image frame via at least one input core circuit, map each distorted pixel to zero or more undistorted pixels by processing the set of inputs corresponding to each pixel of the at least one image frame by the at least one input core circuit, and route corresponding pixel intensity values of each distorted pixel to output undistorted pixels for each image frame via at least one output core circuit.

Claims

1. A neuromorphic system for image distortion correction, comprising:

a plurality of neurosynaptic core circuits that;

receive a set of inputs comprising image dimensions and pixel distortion coefficients for at least one image frame via at least one input core circuit;

map each distorted pixel to zero or more undistorted pixels by processing the set of inputs corresponding to each pixel of the at least one image frame by the at least one input core circuit, wherein mapping of each distorted pixel comprises a corelet of the at least one input core circuit estimating a closest pixel coordinate (x′

, y′

) in a distorted image plane that a pixel (x, y) maps to under a selected metric and creating a neurosynaptic network within the plurality of neurosynaptic core circuits for routing corresponding intensity values of pixel (x′

, y′

) to at least one output core circuit for output of the pixel (x, y); and

route corresponding pixel intensity values of each distorted pixel to output undistorted pixels for each image frame via the at least one output core circuit.

2. The system of claim 1, wherein:

each neurosynaptic core circuit of the plurality of neurosynaptic core circuits comprises one or more electronic neurons, one or more electronic axons, and a plurality of synapse devices for interconnecting said one or more electronic neurons with said one or more electronic axons.

3. The system of claim 2, wherein the system further includes periodically spiking neurons for synchronization, wherein periodic spikes of the periodically spiking neurons are distributed across a plurality of axon lines and comprise control signals for synchronizing a subset of the neurons by at least one of:

resetting the neurons to a pre-specified membrane potential value at regular intervals;

biasing a membrane potential of the neurons at regular intervals; and

probing the neurons to force the neurons to output a result that corresponds to a function of all respective input spikes for a current frame.

4. The system of claim 3, wherein the set of inputs are based on using a pixel distortion model for determining the pixel distortion coefficients of each image frame, where a selected pixel encoding scheme is used for encoding the pixels of each image frame, and the encoded pixels of each image frame are routed to the at least one input core circuit.

5. The system of claim 4, wherein the selected pixel encoding scheme is based on a determined number of spikes entering a particular axon type.

6. The system of claim 1, wherein a number of frames per second processed by the plurality of neurosynaptic core circuits depends on at least one of:

a selected pixel encoding scheme, distortion coefficients, image size, and a selected maximum number of neurons used.

7. A neuromorphic system for image distortion correction, comprising:

a plurality of neurosynaptic core circuits that;

receive a set of inputs comprising image dimensions and pixel distortion coefficients for at least one image frame via at least one input core circuit, wherein the set of inputs are based on using a pixel distortion model for determining the pixel distortion coefficients of each image frame, a selected pixel encoding scheme is used for encoding the pixels of each image frame, and the selected pixel encoding scheme is based on a determined number of spikes entering a particular axon type;

map each distorted pixel to zero or more undistorted pixels by processing the set of inputs corresponding to each pixel of the at least one image frame by the at least one input core circuit; and

route corresponding pixel intensity values of each distorted pixel to output undistorted pixels for each image frame via at least one output core circuit.