Method to identify an object and a system for doing the same

• US 9,672,226 B2
• Filed: 10/25/2012
• Issued: 06/06/2017
• Est. Priority Date: 10/27/2011
• Status: Expired due to Fees

• Current Assignee: Agency For Science Technology and Research (Government of Singapore), Illinois Advanced Research Center At Singapore Limited
• Original Assignee: Agency For Science Technology and Research (Government of Singapore)
• Inventors: Lu, Jiangbo, Wang, Gang
• Primary Examiner(s): Allison, Andrae S
• Application Number:
  US14/354,274
• Publication Number:
  US 20140304271A1
• Time in Patent Office: 1,685 Days
• Field of Search: None
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30088 Skin; Dermal
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/26 Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
• CPC Class Codes: G06V 10/421 by analysing segments intersecting the pattern
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 10/758 Involving statistics of pixels or of feature values, e.g. histogram matching
• CPC Class Codes: G06V 20/66 Trinkets, e.g. shirt buttons or jewellery items recognising microscopic objects G06V20/69

Abstract

Various embodiments relate to a method to identify an object comprising: receiving an image, the image having an object in front of a background; segmenting the image into a segmented image using a segmentation technique, the segmented image having a foreground component showing at least a part of the object and a background component showing at least a part of the background; determining at least one property of the foreground component of the segmented image; and matching the at least one property of the foreground component with a database of identified objects having the corresponding at least one property to identify the object.

Claims

1. A method to identify an object comprising:

receiving an image, the image having an object in front of a background;

segmenting the image into a segmented image using a segmentation technique, the segmented image having a foreground component showing at least a part of the object and a background component showing at least a part of the background;

determining at least one property of the foreground component of the segmented image; and

matching the at least one property of the foreground component with a database of identified objects having the corresponding at least one property to identify the object;

wherein determining the at least one property of the foreground component comprises;

identifying a contour of the foreground component;

identifying a plurality of points along the contour of the foreground component;

selecting a shape out a plurality of predefined shapes based on the plurality of points, wherein the at least one property of the foreground component comprises the shape selected based on the plurality of points; and

reducing the plurality of points identified comprising;

selecting a point;

identifying a first neighboring point nearest the selected point;

identifying a second neighboring point the second next nearest the selected point;

determining an angle between a first line drawn between the selected point and the first neighboring point and a second line drawn between the selected point and the second neighbouring point; and

removing the selected point if the angle exceeds a predetermined value.

2. The method to identify an object according to claim 1 further comprising:

segmenting the image using further segmentation techniques into further segmented images to produce a plurality of segmented images; and

choosing one or more segmented images from the plurality of segmented images for determining the at least one property of the foreground component.

3. The method to identify an object according to claim 2 wherein choosing the one or more segmented image from the plurality of segmented images comprises:

determining a center point of the foreground component in each segmented image;

calculating a coefficient of variation for each segmented image based on the distance of pixels belonging to the foreground component from the center point of the foreground component in the segmented image; and

determining whether the coefficient of variation of each segmented image falls below a predetermined value.

4. The method to identify an object according to claim 2 wherein choosing the one or more segmented image from the plurality of segmented images comprises:

determining number of pixels belonging to the foreground component in each segmented image; and

determining whether the number of pixels belonging to the foreground component in each segmented image falls within a predetermined range of values.

5. The method to identify an object according to claim 1 wherein segmenting the image using the segmentation technique or techniques comprises:

determining a number of color categories in the image;

grouping each pixel of the image under one of the number of color categories based on similarities of color of each pixel to color of the color categories; and

determining the foreground component and the background component of the image based on the grouped pixels.

6. The method to identify an object according to claim 5 wherein segmenting the image using the segmentation technique or techniques further comprises:

determining a center point of the image;

calculating distance of each pixel from the center points of the image; and

grouping each pixel in the image under one of the number of color categories further based on the distance of the pixel from the center points of the image.

7. The method to identify an object according to claim 5 wherein segmenting the image using the segmentation technique or techniques further comprises:

performing color conversion on each pixel to increase algorithm efficacy before determining the number of color categories in the image comprising;

assigning hue, saturation, and lightness for each pixel in hue, saturation, and lightness (HSL) color space;

equating value to the lightness for each pixel such that hue, saturation, and value (HSV) color space is defined for each pixel; and

transforming hue, saturation, and value (HSV) color space to red, green, blue (RGB) color space for each pixel.

8. The method to identify an object according to claim 1 wherein segmenting the image using the segmentation technique or techniques comprises:

identifying a range of colors such that pixels having the range of colors are grouped as the background component of the segmented image; and

grouping pixels not having the range of colors as the foreground component of the segmented image.

9. The method to identify an object according to claim 1 wherein matching the at least one property of the foreground component comprises:

rating similarity between the shape of the foreground component based on the plurality of points and an identified object in the database comprising;

forming a first polygon having vertices formed from the shape a first of the number of foreground component and a second polygon having a second number of vertices derived from the identified object in the database;

determining whether the first number of vertices is equal to the second number of vertices;

removing a vertice from the polygon having a greater number of vertices and changing an index if the first number of vertices is not equal to the second number of vertices;

reiterating comparison between the polygonal with reduced vertice and the remaining polygonal and changing the index until the first number of vertices is equal to the second number of vertices;

determining whether the first polygon is similar to the second polygon if the first number of vertices is equal to the second number of vertices;

removing a vertice from both the first polygon and the second polygon and changing the index if the first polygon is not similar to the second polygon but the first number of vertices is equal to the second number of vertices;

assigning the index to the identified object if the first polygon is similar to the second polygon and the first number of vertices is equal to the second number of vertices; and

reiterating comparison between polygonals with removed vertices and changing the index until the index is assigned;

wherein the index indicates similarity between the shape of the foreground component and the identified objects;

repeating rating similarity between the shape of the foreground component and subsequent identified objects in the database; and

matching the foreground component to the identified object in the database having the highest index.

10. The method to identify an object according to claim 1 wherein the segmented image is a binary image having a first color and a second color, the first color for identifying the foreground component and the second color for identifying the background component.

11. The method to identify an object according to claim 1 wherein the object is a pill.

12. A method to identify an object comprising

receiving an image, the image having an object in front of a background;

segmenting the image into a segmented image using a segmentation technique, the segmented image having a foreground component showing at least a part of the object and a background component showing at least a part of the background;

determining at least one property of the foreground component of the segmented image; and

matching the at least one property of the foreground component with a database of identified objects having the corresponding at least one property to identify the object;

wherein determining the at least one property of the foreground component comprises;

identifying a contour of the foreground component;

identifying a plurality of points along the contour of the foreground component;

selecting a shape out a plurality of predefined shapes based on the plurality of points, wherein the at least one property of the foreground component comprises the shape selected based on the plurality of points; and

reducing the plurality of points identified comprising;

selecting a first point;

selecting a second point;

identifying a bounding box such that the first point falls on a first corner of the bounding box and the second point falls on a second corner of the bounding box diagonally opposite the bounding box from the first corner, the bounding box having a width and a height;

determining a threshold of the bounding box based on the width and the height; and

removing the second point if the threshold falls below a predetermined value.

13. A system for identifying an object comprising:

an image receiver to receive an image, the image having the object in front of a background;

a segmentor to segment the image into a segmented image using a segmentation technique, the segmented image having a foreground component showing at least a part of the object and a background component showing at least a part of the background;

a determinor to determine at least one property of the foreground component of the segmented image; and

a matcher to match the at least one property of the foreground component with a database of identified objects having the corresponding at least one property to identify the object;

wherein the determinor to determine the at least one property of the foreground component comprises;

the determinor to identify a contour of the foreground component;

the determinor to identify a plurality of points along the contour of the foreground component;

the determinor to select a shape out a plurality of predefined shapes based on the plurality of points, wherein the at least one property of the foreground component comprises the shape selected based on the plurality of points; and

the determinor to reduce the plurality of points identified by;

selecting a point;

identifying a first neighboring point nearest the selected point;

identifying a second neighboring point the second next nearest the selected point;

determining an angle between a first line drawn between the selected point and the first neighboring point and a second line drawn between the selected point and the second neighboring point; and

removing the selected point if the angle exceeds a predetermined value.