AUTOMATED BOUNDING BOX GENERATION FOR OBJECTS IN AN IMAGE

• US 20200160084A1
• Filed: 11/19/2018
• Published: 05/21/2020
• Est. Priority Date: 11/19/2018
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bhattacharjee, Bishwaranjan, Smith, John R., Greenberg, Matthew
• Granted Patent: US 10,643,093 B1
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06T 7/181 involving edge growing; involving edge linking
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/809 of classification results, e.g. where the classifiers operate on the same input data
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

One or more embodiments described herein include a computer-implemented method of determining a bounding box for an object in an image. The method includes determining a label for an object in a first image using a first algorithm, and generating a set of images based on the first image, by cropping the first image from a selected direction. The method further includes determining labels for each image in the set using the first algorithm, and removing images from the set such that the remaining images have a label matching the initial label. The method further includes determining a key image for the set, which is the smallest image from the set that has a confidence score exceeding a threshold. Further, the method includes determining a bounding box for the object in the first image based on a perimeter of a portion of the first image that overlaps the key image.

Claims

1. A computer-implemented method of determining a bounding box for an object in an image, the method comprising:

determining a label for an object in a first image using a first algorithm;

determining a selected direction;

generating a set of images based on the first image, wherein each image of the set of images is generated by cropping the first image from the selected direction;

determining labels for each image in the set of images using the first algorithm;

removing images from the set of images such that each image remaining in the set of images has a label matching the label for the object in the first image; and

determining a key image for the set of images, wherein the key image is the smallest image from the set of images that has a confidence score exceeding a threshold, the smallest image being an image with the smallest dimension in the selected direction; and

determining a bounding box for the object in the first image based on a perimeter of a portion of the first image that overlaps the key image.

2. The computer-implemented method of claim 1, wherein the selected direction is one from a group of directions comprising right direction, left direction, top direction, and bottom direction.

3. The computer-implemented method of claim 2, wherein the bounding box for the object in the first image is determined based on the perimeter of the portion of the first image that overlaps each key image for each direction.

4. The computer-implemented method of claim 1, wherein the bounding box is a first bounding box along the selected direction, the method further comprising, determining a second bounding box along a second selected direction.

5. The computer-implemented method of claim 4, wherein the first bounding box is for the object and the second bounding box is for another object in the first image.

6. The computer-implemented method of claim 1, wherein the first image is a two-dimensional (2D) image.

7. The computer-implemented method of claim 1, wherein the first image is an n-dimensional image.

8. A system comprising:

an image database comprising a plurality of images that contain one or more objects, and a plurality of labels identifying the one or more objects in the images;

a plurality of processors configured to determine a bounding box for an object in a first image from the image database, wherein determining the bounding box comprises a method, which comprises;

determining a label for the object in the first image using a first algorithm;

for a selected direction;

generating a set of images based on the first image, wherein each image of the set of images is generated by cropping the first image from a given direction;

determining labels for each image in the set of images using the first algorithm;

removing images from the set of images such that each image remaining in the set of images has a label matching the label for the object in the first image; and

determining a key image for the set of images, wherein the key image is the smallest image of the set of images that has a confidence score exceeding a threshold; and

determining a bounding box for the object in the first image based on a perimeter of a portion of the first image that overlaps the key image.

9. The system of claim 8, wherein the selected direction is one from a group of directions comprising right direction, left direction, top direction, and bottom direction.

10. The system of claim 9, wherein the bounding box for the object in the first image is determined based on the perimeter of the portion of the first image that overlaps each key image for each direction.

11. The system of claim 8, wherein the bounding box is a first bounding box along the selected direction, the method further comprising, determining a second bounding box along a second selected direction.

12. The system of claim 11, wherein the first bounding box is for the object and the second bounding box is for another object in the first image.

13. The system of claim 8, wherein the first image is a two-dimensional (2D) image.

14. The system of claim 8, wherein the first image is a three-dimensional (3D) image.

15. A computer program product comprising a computer readable storage medium having stored thereon program instructions executable by one or more processing devices to perform a method comprising:

determining a label for an object in a first image using a first algorithm;

for a selected direction;

generating a set of images based on the first image, wherein each image of the set of images is generated by cropping the first image from a given direction;

determining labels for each image in the set of images using the first algorithm;

removing images from the set of images such that each image remaining in the set of images has a label matching the label for the object in the first image; and

determining a key image for the set of images, wherein the key image is the smallest image of the set of images that has a confidence score exceeding a threshold; and

determining a bounding box for the object in the first image based on a perimeter of a portion of the first image that overlaps the key image.

16. The computer program product of claim 15, wherein the selected direction is one from a group of directions comprising right direction, left direction, top direction, and bottom direction.

17. The computer program product of claim 16, wherein the bounding box for the object in the first image is determined based on the perimeter of the portion of the first image that overlaps each key image for each direction.

18. The computer program product of claim 15, wherein the bounding box is a first bounding box along the selected direction, the method further comprising, determining a second bounding box along a second selected direction.

19. The computer program product of claim 15, wherein the first image is a two-dimensional (2D) image.

20. The computer program product of claim 15, wherein the first image is a three-dimensional (3D) image.