METHOD AND APPARATUS FOR IMAGE PROCESSING

• US 20100183217A1
• Filed: 04/24/2008
• Published: 07/22/2010
• Est. Priority Date: 04/24/2007
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology, Max-Planck-Gesellschaft Zur Foerderung Der Wissenschaften e.V. (Max-Planck-Gesellschaft zur Frderung der Wissenschaften e)
• Original Assignee: Max-Planck-Gesellschaft Zur Foerderung Der Wissenschaften e.V. (Max-Planck-Gesellschaft zur Frderung der Wissenschaften e)
• Inventors: Denk, Winfried, Murray, Joseph F., Helmstaedter, Moritz, Seung, H. Sebastian, Jain, Viren, Turaga, Srinivas C.
• Granted Patent: US 9,799,098 B2
• Time in Patent Office: Days
• US Class Current: 382/156
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/60 using machine learning, e.g. neural networks
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 10/267 by performing operations on regions, e.g. growing, shrinking or watersheds
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]

Abstract

Identifying objects in images is a difficult problem, particularly in cases an original image is noisy or has areas narrow in color or grayscale gradient. A technique employing a convolutional network has been identified to identify objects in such images in an automated and rapid manner. One example embodiment trains a convolutional network including multiple layers of filters. The filters are trained by learning and are arranged in successive layers and produce images having at least a same resolution as an original image. The filters are trained as a function of the original image or a desired image labeling; the image labels of objects identified in the original image are reported and may be used for segmentation. The technique can be applied to images of neural circuitry or electron microscopy, for example. The same technique can also be applied to correction of photographs or videos.

Claims

1. A method for image processing comprising:

training a convolutional network including multiple layers of filters to produce at least one image in succeeding layers with at least the same resolution as an original image.

2. The method of claim 1 wherein training the convolutional network includes training with back propagation learning.

3. The method of claim 1 wherein training the convolutional network includes training with stochastic gradient learning.

4. The method of claim 1 further including training the convolutional network as a function of training data.

5. The method of claim 1 further including producing an image at a resolution higher than the original image.

6. The method of claim 1 further including selectably configuring the convolutional network to apply linear or non-linear filtering.

7. An image processing system comprising:

a training module to train a convolutional network including multiple layers of filters to produce at least one image in succeeding layers with at least the same resolution as an original image.

8. The image processing system of claim 7 wherein the training module is configured to train the convolutional network with back propagation learning.

9. The image processing system of claim 7 wherein the training module is configured to train the convolutional network with stochastic gradient learning.

10. The image processing system of claim 7 wherein the training module is configured to train the convolutional network as a function of training data.

11. The image processing system of claim 7 wherein the convolutional network is arranged to produce an image at a resolution higher than the original image.

12. The image processing system of claim 7 wherein the convolutional network is configured to selectably apply linear or non-linear filtering.

13. A method for image processing comprising:

training a convolutional network including multiple layers of filters trained by learning and arranged in successive layers and producing images having at least a same resolution as an original image;

training the filters as a function of the original image or a desired image labeling; and

reporting image labels of objects identified in the original image.

14. The method of claim 13 wherein training the filters includes training with back propagation.

15. The method of claim 13 wherein training the filters includes training with stochastic gradient learning.

16. The method of claim 13 further including training the filters as a function of training data.

17. The method of claim 13 further including producing an image with a higher resolution than the original image.

18. The method of claim 13 further including enabling selection between linear and non-linear filtering.

19. The method of claim 13 further including accepting imaging labels, grouping together neighboring pixels with similar labels to produce new image labels corresponding to objects, and reporting the new image labels corresponding to the objects.

20. The method of claim 19 further including displaying the objects on a screen with different illustrations representing differences between or among the objects.

21. The method of claim 20 further including applying different colors to the objects.

22. An image processing system comprising:

a first training module to train a convolutional network including multiple layers of filters trained by learning and arranged in successive layers and producing images having at least a same resolution as an original image;

a second training module to train the filters as a function of the original image or a desired image labeling; and

a reporting module to report image labels of objects identified in the original image.

23. The image processing system of claim 22 wherein the first training module is configured to train the convolutional network by back propagation.

24. The image processing system of claim 22 wherein the first training module is configured to train the convolutional network by gradient learning.

25. The image processing system of claim 22 wherein the second training module is configured to train the filters as a function of training data.

26. The image processing system of claim 22 wherein the convolutional network is arranged to produce an image with a higher resolution than the original image.

27. The image processing system of claim 22 wherein the second training module is arranged to enable selection between linear and non-linear filtering.

28. The image processing system of claim 22 further including a connected components module to accept imaging labels, group together neighboring pixels with similar labels to produce new image labels corresponding to objects, and report the new image labels corresponding to the objects.

29. The image processing system of claim 28 further including a display module to display the objects on a screen with different indications representing differences between or among the objects.

30. The image processing system of claim 29 further including a coloring module to apply the different indications as different colors to the objects.

31-94. -94. (canceled)