Object characterization

• US 10,186,029 B2
• Filed: 09/25/2015
• Issued: 01/22/2019
• Est. Priority Date: 09/26/2014
• Status: Active Grant

• Current Assignee: Wisconsin Alumni Research Foundation (University of Wisconsin System)
• Original Assignee: Wisconsin Alumni Research Foundation (University of Wisconsin System)
• Inventors: Spalding, Edgar P., Miller, Nathan D.
• Primary Examiner(s): Bitar, Nancy
• Application Number:
  US14/865,828
• Publication Number:
  US 20160093037A1
• Time in Patent Office: 1,215 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 2207/30128 Food products
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume

Abstract

Various aspects of the present disclosure are directed to characterizing objects or other items, which as may be implemented in a machine-vision type approach. As may be implemented with one or more embodiments, some of which may involve a Fourier transform analysis, object regions (one or more objects) are identified in image data depicting several objects, and the size of each object-region is identified. Respective ratios of the size of each object-region to the size of each of the other object regions are determined, and object regions having a higher total number of ratios that round to one (relative to the other object regions) are identified. An expected (e.g., average) individual object size is determined based on the sizes of the identified object regions, and a total number of the objects in the image is determined based on the expected individual object size and the sizes of the object regions.

Claims

1. A method comprising:

processing first image data for an image of a plurality of objects, and identifying therefrom object regions in the first image data, at least one of the object regions including a cluster of at least two of the plurality of objects, based on value differences between data characterizing the object-regions and data characterizing a background region of the image;

determining a size, including an area or perimeter-based dimension, of each object-region;

determining an individual object size for the plurality of objects by

for each object-region, determining respective ratios of the size of the object-region to the size of each of the other object regions,

identifying ones of the object regions having a higher total number of the ratios that round to one, relative to the other object regions, and

determining an expected individual object size based on the sizes of the identified ones of the object regions; and

determining a total number of the objects in the image based on the determined expected individual object size and the sizes of the object regions, by using the determined expected individual object size and the size of each object region to detect a number of objects within each object region.

2. The method of claim 1, wherein

identifying the ones of the object regions having a higher total number of the ratios that round to one, relative to the other object regions, includes identifying said ones of the object regions as individual objects and identifying said other object regions as including clusters of the objects,

determining the expected individual object size includes determining an average individual object size, by averaging the sizes of the identified ones of the object regions, and

determining the total number of the objects in the image includes determining a total area of all object regions in the image and dividing the total area by the average individual object size, based on the determined average individual object size.

3. The method of claim 1, wherein determining respective ratios of the size of the object-region to the size of each of the other object regions includes, for each object-region, respectively dividing the determined size of the object-region by the size of each of the other object regions to provide a plurality of ratios of the object-region to each of the other object regions.

4. The method of claim 1, wherein

identifying the object regions includes identifying object regions in a sample portion of the image that is less than half of the image, the expected individual object size being an average object size in the sample portion, and

determining the total number of the objects in the image includes using the average object size from the sample portion as an estimate of a total expected object size for the image.

5. The method of claim 1, wherein

determining a size of each object-region includes determining the area of each object region, and determining the expected individual object size for the plurality of objects includes determining the average individual object area by

identifying the object regions in a sample portion of the image that is less than half of the image, the average individual object area being an average object area in the sample portion, and

determining the total number of the objects in the image includes, using the average object area from the sample portion as an estimate of a total average object area for the image, determining a total area of all object regions in the image and dividing the total area by the average individual object area.

6. The method of claim 1, wherein identifying the object regions includes filtering the first image data by identifying the object regions as object regions having a size that is greater than a minimum threshold.

7. The method of claim 1, wherein identifying the object regions includes filtering the first image data by identifying the object regions as object regions having a shape within a cross-sectional threshold.

15. The method of claim 1, wherein the plurality of objects include corn kernels, and wherein determining the expected individual object size for the plurality of objects includes:

determining a point of maximum contour curvature of each kernel;

determine a centroid of each kernel;

computing a plurality of features characterizing each kernel;

creating an n-dimensional distribution of the plurality of features;

for each kernel,

calculating the plurality of features for respective contour points on the kernel, using each contour point as a putative tip point,

comparing the calculated features at each contour point with the n-dimensional distribution, and

assessing the probability of each contour point being a true tip of the kernel based on the calculated features for the contour point and the n-dimensional distribution,

for each contour point exhibiting a probability that matches a peak of the n-dimensional distribution better than the determined point of maximum contour curvature for the kernel, updating the n-dimensional distribution of the plurality of features using the contour point; and

repeating the steps of calculating, comparing and assessing for each kernel.

16. The method of claim 15, wherein computing the plurality of features includes computing a plurality of features selected from the group consisting of putative tip point-centroid axis, depth, width, depth:

width ratio, curvature for a portion of a base of the kernel, curvature for a portion of the apex of the kernel, and overall curvature of the kernel.

8. A method comprising:

processing first image data for an image of a plurality of objects that include at least one of corn ovules and corn kernels, and identifying therefrom object regions in the first image data that represent objects or groups of objects in the image based on value differences between data characterizing the objects and data characterizing a background region of the image;

determining a size, including an area or perimeter-based dimension, of each object-region;

determining an individual object size for the plurality of objects by

for each object-region, determining respective ratios of the size of the object-region to the size of each of the other object regions,

identifying ones of the object regions having a higher total number of the ratios that round to one, relative to the other object regions, and

determining an expected individual object size based on the sizes of the identified ones of the object regions;

determining a total number of the objects in the image based on the determined expected individual object size and the sizes of the object regions;

processing second image data from an image of a corn ear, and identifying therefrom a portion of the second image data including the corn ear relative to a portion of the second image data corresponding to a background of the image;

applying a Fourier transform to an image gradient of the portion of the second image data including the corn ear, and providing transformed image data therefrom;

determining a frequency of peaks in the transformed image data; and

estimating a length of each ovule or kernel based on the determined peak frequency.

9. The method of claim 8, wherein the image of the plurality of objects includes images of kernels of corn while they are attached to the corn ear.

10. The method of claim 8, wherein the image gradient is a gradient along an axial direction of the corn ear.

11. The method of claim 8, further including applying a Fourier transform to the image of the corn ear, wherein the determining the frequency of peaks in the transformed image data includes determining a frequency of peaks in the transformed image data for the corn ear and for the image gradient.

12. The method of claim 8, wherein estimating the length of each ovule or kernel based on the determined peak frequency includes using a corresponding period of the frequency peaks to estimate the length of each ovule or kernel.

13. The method of claim 8, wherein

processing the second image data from the image of the corn ear includes thresholding the image data, and wherein identifying the portion of the second image data including the corn ear includes using the thresholded image data,

applying the Fourier transform includes separating the second image data into color channels and calculating the Fourier transform of a gradient channel within the color channels along an axial length of the ear,

determining the frequency of the peaks includes separately decomposing positive and negative values of the gradient of the one of the color channels with a sliding window Fourier transform, determining a frequency of a peak amplitude for each sliding window, and determining an average peak frequency across sliding windows for both the positive and negative parts of the gradient, and

estimating the length of each ovule or kernel includes plotting a cosine wave having the determined average peak frequency along the length of the ear, and estimating an average length of the kernels based upon a period T of the cosine wave.

14. The method of claim 8, wherein the image of the plurality of objects includes images of corn ovules on the corn ear.

17. A non-transitory computer-readable medium having instructions stored thereupon that, when executed by a computer processor, cause the computer processor to:

process first image data from an image of a plurality of objects, and identify therefrom object regions in the first image data that represent objects or groups of objects in the image, at least one of the object regions including a cluster of at least two of the plurality of objects, based on value differences between data characterizing the object regions of the image and data characterizing a background of the image;

determine a size, including an area or perimeter-based dimension, of each object-region;

determine an average individual object size for the plurality of objects by for each object-region, determine respective ratios of the size of the object-region to the size of each of the other object regions, identify a subset of the object regions, in which the object regions have a higher total number of the ratios that round to one relative to the other object regions, and determine an expected individual object size based on the sizes of the object regions in the subset; and

determine a total number of the objects in the image based on the determined expected individual object size and the sizes of the object regions, by using the determined expected individual object size and the size of each object region to detect a number of objects within each object region.

18. The non-transitory computer-readable medium of claim 17, wherein the instructions, when executed by the computer processor, cause the computer processor to determine a total number of the objects in the image by determining a total area of all object regions in the image and dividing the total area by the expected individual object size, based on the determined expected individual object size.

19. The non-transitory computer-readable medium of claim 17, wherein the size of the object regions is one of an area of the object regions or a perimeter of the object regions, and wherein the subset of the object regions are identified as individual objects, and said other object regions are identified as including clusters of the objects.

20. The non-transitory computer-readable medium of claim 17, wherein the instructions, when executed by the computer processor, cause the computer processor to

identify the object regions by identifying object regions in a sample portion of the image that is less than the entire image, the expected individual object size being an average object size in the sample portion, and

determine the total number of the objects in the image by using the average object size from the sample portion as an estimate of a total average object size for the image.

21. The non-transitory computer-readable medium of claim 17, wherein the objects are kernels from a corn ear and the instructions, when executed by the computer processor, cause the computer processor to

process second image data from an image of the corn ear, and identify therefrom a portion of the second image data including the corn ear relative to a portion of the second image data corresponding to a background of the image;

apply a Fourier transform to an image gradient of the portion of the second image data including the corn ear, and provide transformed image data therefrom;

determine a frequency of peaks in the transformed image data; and

estimate a length of each kernel based on the determined peak frequency.

22. The non-transitory computer-readable medium of claim 21, wherein the instructions, when executed by the computer processor, cause the computer processor to

process the second image data from the image of the corn ear by thresholding the image data, and to identify the portion of the second image data including the corn ear by using the thresholded image data,

apply the Fourier transform by separating the second image data into color channels and calculating the Fourier transform of the gradient of one of the color channels along an axial length of the ear,

determine the frequency of the peaks by separately decomposing positive and negative values of the gradient of the one of the color channels with a sliding window Fourier transform, determining a frequency of a peak amplitude for each sliding window, and determining an average peak frequency across sliding windows for both the positive and negative parts of the gradient, and

estimate the length of each kernel by plotting a cosine wave having the determined average peak frequency along the length of the ear, and estimating the average kernel length based upon a period T of the cosine wave.

23. An apparatus comprising:

an imaging device configured and arranged to capture an image of a plurality of objects; and

a computer processor circuit configured and arranged with stored instructions to;

process first image data from an image of a plurality of objects, and identify therefrom object regions in the first image data that represent objects or groups of objects in the image, at least one of the object regions including a cluster of at least two of the plurality of objects, based on differences between the object regions of the image and a background of the image;

determine a size, including an area or perimeter-based dimension, of each object-region;

determine an expected individual object size for the plurality of objects by for each object-region, determining respective ratios of the size of the object-region to the size of each of the other object regions, identifying ones of the object regions having a higher total number of the ratios that round to one, relative to the other object regions, and determining the average individual object size by averaging the sizes of the ones of the object regions; and

determine a total number of the objects in the image based on the determined average individual object size and the sizes of the object regions, by using the determined expected individual object size and the size of each object region to detect a number of objects within each object region.

24. The apparatus of claim 23, wherein identifying said ones of the object regions includes differentiating clusters of the objects from individual ones of the objects by identifying the individual objects as ones of the object regions having a higher total number of the ratios that round to one, relative to the other object regions, and identifying the other regions as including clusters of the objects.