Lens distortion correction using a neurosynaptic circuit

• US 10,157,453 B2
• Filed: 04/30/2018
• Issued: 12/18/2018
• Est. Priority Date: 06/25/2014
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Andreopoulos, Alexander, Gruhl, Daniel F., Karasick, Michael, Modha, Dharmendra S.
• Primary Examiner(s): Wang, Carol
• Application Number:
  US15/967,373
• Publication Number:
  US 20180253833A1
• Time in Patent Office: 232 Days
• CPC Class Codes: G06F 17/11 for solving equations , e.g. nonlinear equations, general mathematical optimization problems optimization specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/20172 Image enhancement details
• CPC Class Codes: G06T 3/4046 using neural networks
• CPC Class Codes: G06T 5/60 using machine learning, e.g. neural networks
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 30/194 References adjustable by an adaptive method, e.g. learning

Abstract

One or more embodiments provide method for image distortion correction including receiving, by multiple neurosynaptic core circuits, a set of inputs comprising image dimensions and pixel distortion coefficients for at least one image frame via at least one input core circuit. Each distorted pixel is mapped to zero or more undistorted pixels by processing the set of inputs corresponding to each pixel of the at least one image frame by the at least one input core circuit. Corresponding pixel intensity values of each distorted pixel are routed to output undistorted pixels for each image frame via the at least one output core circuit.

Claims

1. A method for image distortion correction, the method comprising:

receiving, by a plurality of neurosynaptic core circuits, a set of inputs comprising image dimensions and pixel distortion coefficients for at least one image frame via at least one input core circuit;

mapping each distorted pixel to zero or more undistorted pixels by processing the set of inputs corresponding to each pixel of the at least one image frame by the at least one input core circuit, wherein mapping of each distorted pixel comprises a corelet of the at least one input core circuit estimating a closest pixel coordinate (x′

, y′

) in a distorted image plane that a pixel (x, y) maps to under a selected metric and creating a neurosynaptic network within the plurality of neurosynaptic core circuits for routing corresponding intensity values of pixel (x′

, y′

) to at least one output core circuit for output of the pixel (x, y); and

routing corresponding pixel intensity values of each distorted pixel to output undistorted pixels for each image frame via the at least one output core circuit.

2. The method of claim 1, further comprising:

synchronizing by periodically spiking neurons, where periodic spikes of the periodically spiking neurons are distributed across a plurality of axon lines and comprise control signals for synchronizing a subset of the neurons.

3. The method of claim 2, wherein synchronizing the subset of the neurons is based on at least one of:

resetting the neurons to a pre-specified membrane potential value at regular intervals;

biasing a membrane potential of the neurons at regular intervals; and

probing the neurons to force the neurons to output a result that corresponds to a function of all respective input spikes for a current frame.

4. The method of claim 3, wherein the set of inputs are based on using a pixel distortion model for determining the pixel distortion coefficients of each image frame.

5. The method of claim 4, wherein a selected pixel encoding scheme is used for encoding the pixels of each image frame, and the encoded pixels of each image frame are routed to the at least one input core circuit.

6. The method of claim 5, wherein the selected pixel encoding scheme is based on a determined number of spikes entering a particular axon type.

7. The method of claim 1, wherein a number of frames per second processed by the plurality of neurosynaptic core circuits depends on at least one of:

a selected pixel encoding scheme, distortion coefficients, image size, and a selected maximum number of neurons used.

8. A method for image distortion correction, the method comprising:

receiving, by at least one neurosynaptic core circuit of a plurality of neurosynaptic core circuits, a set of inputs comprising image dimensions and pixel distortion coefficients for at least one image frame via at least one input core circuit, wherein the set of inputs are based on using a pixel distortion model for determining the pixel distortion coefficients of each image frame, a selected pixel encoding scheme is used for encoding the pixels of each image frame, and the selected pixel encoding scheme is based on a determined number of spikes entering a particular axon type;

mapping each distorted pixel to zero or more undistorted pixels by processing the set of inputs corresponding to each pixel of the at least one image frame by the at least one input core circuit; and

routing corresponding pixel intensity values of each distorted pixel to output undistorted pixels for each image frame via at least one output core circuit.

9. The method of claim 8, further comprising:

synchronizing by periodically spiking neurons, where periodic spikes of the periodically spiking neurons are distributed across a plurality of axon lines and comprise control signals for synchronizing a subset of the neurons.

10. The method of claim 9, wherein synchronizing the subset of the neurons is based on at least one of:

resetting the neurons to a pre-specified membrane potential value at regular intervals;

biasing a membrane potential of the neurons at regular intervals; and

probing the neurons to force the neurons to output a result that corresponds to a function of all respective input spikes for a current frame.

11. The method of claim 10, wherein the selected pixel encoding scheme is based on a determined number of spikes entering a particular axon type.

12. The method of claim 11, wherein a number of frames per second processed by the plurality of neurosynaptic core circuits depends on at least one of:

a selected pixel encoding scheme, distortion coefficients, image size, and a selected maximum number of neurons used.