Lens distortion correction using a neurosynaptic circuit

• US 10,169,844 B2
• Filed: 04/30/2018
• Issued: 01/01/2019
• Est. Priority Date: 06/25/2014
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Andreopoulos, Alexander, Gruhl, Daniel F., Karasick, Michael, Modha, Dharmendra S.
• Primary Examiner(s): Wang, Carol
• Application Number:
  US15/967,362
• Publication Number:
  US 20180253825A1
• Time in Patent Office: 246 Days
• CPC Class Codes: G06F 17/11 for solving equations , e.g. nonlinear equations, general mathematical optimization problems optimization specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/20172 Image enhancement details
• CPC Class Codes: G06T 3/4046 using neural networks
• CPC Class Codes: G06T 5/60 using machine learning, e.g. neural networks
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 30/194 References adjustable by an adaptive method, e.g. learning

Abstract

One or more embodiments provide a neurosynaptic circuit that includes multiple neurosynaptic core circuits that: perform image distortion correction by converting a source image to a destination image by: taking as input a sequence of image frames of a video with one or more channels per frame, and converting dimensions and pixel distortion coefficients of each frame as one or more corresponding neuronal firing events. Each distorted pixel is mapped to zero or more undistorted pixels by processing each neuronal firing event corresponding to each pixel of each image frame. Corresponding pixel intensity values of each distorted pixel are processed to output undistorted pixels for each image frame as neuronal firing events for a spike representation of the destination image.

Claims

1. A neurosynaptic circuit comprising:

a plurality of neurosynaptic core circuits that;

perform image distortion correction by converting a source image to a destination image by;

taking as input a sequence of image frames of a video with one or more channels per frame, and converting dimensions and pixel distortion coefficients of each frame as one or more corresponding neuronal firing events;

mapping each distorted pixel to zero or more undistorted pixels by processing each neuronal firing event corresponding to each pixel of each image frame; and

processing corresponding pixel intensity values of each distorted pixel to output undistorted pixels for each image frame as neuronal firing events for a spike representation of the destination image.

2. The neurosynaptic circuit of claim 1, wherein one of nearest-neighbor or bilinear interpolation is performed during the image distortion correction, and bilinear interpolation involves taking from an intermediate representation of the source image, convex combinations of the pixel intensity values at a number of pixel coordinates.

3. The neurosynaptic circuit of claim 1, further comprising a set of synchronization neurons that gate the input.

4. The neurosynaptic circuit of claim 3, wherein the set of synchronization neurons indicate which input frames to suppress.

5. The neurosynaptic circuit of claim 3, wherein a pixel encoding scheme is selected for encoding pixels of each image frame, and encoded pixels of each image frame are routed to the plurality of neurosynaptic core circuits.

6. The neurosynaptic circuit of claim 5, wherein the selected pixel encoding scheme is based on a number of spikes entering a particular axon type.

7. The neurosynaptic circuit of claim 6, wherein the neurosynaptic circuit comprises a finite number of cores, each core includes a binary crossbar with a finite number of input axons and output neurons, each of the input axons in a core is associated with one of K possible axon types, and K is a positive integer.

8. The neurosynaptic circuit of claim 7, wherein each output neuron in a core accepts input spikes from any subset of the input axons of the core.

9. The neurosynaptic circuit of claim 8, wherein for a crossbar-connected neuron and axon pair, a synaptic weight is assigned in a preselected range.

10. The neurosynaptic circuit of claim 9, wherein for any axons with identical axon type, that are connected to a same neuron, then a same synaptic weight from the preselected range is associated with axon lines and the same neuron.

11. A method for performing image sharpening comprising:

converting, by a plurality of neurosynaptic core circuits, a source image to a destination image by taking as input a sequence of image frames of a video with one or more channels per frame and converting dimensions and pixel distortion coefficients of each frame as one or more corresponding neuronal firing events;

mapping each distorted pixel to zero or more undistorted pixels by processing each neuronal firing event corresponding to each pixel of each image frame; and

processing corresponding pixel intensity values of each distorted pixel to output undistorted pixels for each image frame as neuronal firing events for a spike representation of the destination image.

12. The method of claim 11, further comprising:

performing one of nearest-neighbor or bilinear interpolation during image sharpening,

wherein bilinear interpolation comprises taking from an intermediate representation of the source image, convex combinations of the pixel intensity values at a number of pixel coordinates.

13. The method of claim 12, further comprising:

gating, using a set of synchronization neurons, the input, wherein the set of synchronization neurons indicate which input frames to suppress.

14. The method of claim 13, wherein a pixel encoding scheme is selected for encoding pixels of each image frame, and encoded pixels of each image frame are routed to the plurality of neurosynaptic core circuits.

15. The method of claim 14, wherein the selected pixel encoding scheme is based on a number of spikes entering a particular axon type.