DETECTION OF OBJECTS IN IMAGES USING REGION-BASED CONVOLUTIONAL NEURAL NETWORKS

• US 20170316285A1
• Filed: 04/28/2016
• Published: 11/02/2017
• Est. Priority Date: 04/28/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ahmed, Mohamed N., Toor, Andeep S.
• Granted Patent: US 9,996,772 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/23213 with fixed number of clusters, e.g. K-means clustering
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions

Abstract

A transformed image is received. The transformed image includes an other-than-visible light image that has been captured using a transformation device. A region of the transformed image is isolated, the region being less than an entirety of the transformed image. By applying to the region a convolutional Neural Network (CNN) which executes using a processor and a memory, and by processing only the region of the transformed image, an object of interest is detected in the region. Upon detecting, an indication is produced to indicate the presence of the object of interest in the region.

Claims

1. A method comprising:

receiving a transformed image, the transformed image comprising an other-than-visible light image that has been captured using a transformation device;

isolating a region of the transformed image, the region being less than an entirety of the transformed image;

detecting, by applying to the region a convolutional Neural Network (CNN) which executes using a processor and a memory, and by processing only the region of the transformed image, an object of interest in the region; and

outputting, responsive to the detecting, an indication of a presence of the object of interest in the region.

2. The method of claim 1, further comprising:

detecting a second object of interest in a second transformed image, wherein the transformed image is captured by the transformation device at a first physical location, and the second transformed image is captured by a second transformation device at a second physical location;

combining a generic representation of the object of interest with a generic representation of the second object of interest;

generating an indication, responsive to determining that the combining forms a third object of interest.

3. The method of claim 2, further comprising:

identifying a first person associated with the object of interest;

determining a first travel plan of the first person;

identifying a second person associated with the second object of interest; and

determining a second travel plan of the first person;

identifying, using the first travel plan and the second travel plan, a future time and place where the first person can bring the object of interest and the second person can bring the second object of interest together to actually form the third object of interest.

4. The method of claim 1, further comprising:

training the CNN using training imagery of the object of interest, wherein the training imagery comprises a generated transformed image of the object of interest, wherein the generated transformed image is computed from a generic visible light image of the object of interest and not actually captured by the transformation device.

5. The method of claim 4, further comprising:

computing the generated transformed image;

determining whether the generated transformed image can be classified under a classification of the object of interest within a threshold error value;

adding the generated transformed image to the training imagery responsive to the generated transformed image being classifiable under the classification of the object of interest within the threshold error value.

6. The method of claim 5, further comprising:

generating, as a part of the computing, the generated transformed image such that a variation between the generated transformed image and the transformed image is at least a second threshold error value.

7. The method of claim 1, further comprising:

generating a degree of confidence in the detecting of the object of interest in the region.

8. The method of claim 1, wherein the detecting occurs without any knowledge of any pre-determined feature of the object of interest, and by using a feed-forward network of weighted nodes in the CNN.

9. The method of claim 1, further comprising:

defining a neighborhood of a pixel in the transformed image;

biasing a classification of the pixel according to a classification of a pixel in the neighborhood;

comparing the classification of a group of pixels in the transformed image with a classification of the object of interest; and

identifying, as a part of the isolating, the region wherein the region comprises the group of pixels.

10. The method of claim 9, wherein the neighborhood is defined by a pixel distance from the pixel in the transformed image, such that any pixel within the pixel distance from the pixel in the transformed image is within the neighborhood of the pixel in the transformed image.

11. The method of claim 9, wherein the classification of the object of interest is defined by a utility of the object of interest.

12. The method of claim 9, wherein the biasing is performed using a Modified Fuzzy C-Means algorithm.

13. The method of claim 1, further comprising:

generating a likelihood of a presence of the object of interest in the region.

14. The method of claim 1, wherein the transformed image is an x-ray image, and wherein the transformation device is an x-ray scanner.

15. A computer usable program product comprising one or more computer-readable storage devices, and program instructions stored on at least one of the one or more storage devices, the stored program instructions comprising:

program instructions to receive a transformed image, the transformed image comprising an other-than-visible light image that has been captured using a transformation device;

program instructions to isolate a region of the transformed image, the region being less than an entirety of the transformed image;

program instructions to detect, by applying to the region a convolutional Neural Network (CNN) which executes using a processor and a memory, and by processing only the region of the transformed image, an object of interest in the region; and

program instructions to output, responsive to the detecting, an indication of a presence of the object of interest in the region.

16. The computer usable program product of claim 15, further comprising:

program instructions to detect a second object of interest in a second transformed image, wherein the transformed image is captured by the transformation device at a first physical location, and the second transformed image is captured by a second transformation device at a second physical location;

program instructions to combine a generic representation of the object of interest with a generic representation of the second object of interest;

program instructions to generate an indication, responsive to determining that the combining forms a third object of interest.

17. The computer usable program product of claim 16, further comprising:

program instructions to identify a first person associated with the object of interest;

program instructions to determine a first travel plan of the first person;

program instructions to identify a second person associated with the second object of interest; and

program instructions to determine a second travel plan of the first person;

program instructions to identify, using the first travel plan and the second travel plan, a future time and place where the first person can bring the object of interest and the second person can bring the second object of interest together to actually form the third object of interest.

18. The computer usable program product of claim 15, wherein the computer usable code is stored in a computer readable storage device in a data processing system, and wherein the computer usable code is transferred over a network from a remote data processing system.

19. The computer usable program product of claim 15, wherein the computer usable code is stored in a computer readable storage device in a server data processing system, and wherein the computer usable code is downloaded over a network to a remote data processing system for use in a computer readable storage device associated with the remote data processing system.

20. A computer system comprising one or more processors, one or more computer-readable memories, and one or more computer-readable storage devices, and program instructions stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, the stored program instructions comprising:

program instructions to receive a transformed image, the transformed image comprising an other-than-visible light image that has been captured using a transformation device;

program instructions to isolate a region of the transformed image, the region being less than an entirety of the transformed image;

program instructions to detect, by applying to the region a convolutional Neural Network (CNN) which executes using a processor and a memory, and by processing only the region of the transformed image, an object of interest in the region; and

program instructions to output, responsive to the detecting, an indication of a presence of the object of interest in the region.