VISUAL SEGMENTATION OF LAWN GRASS

• US 20120212638A1
• Filed: 07/01/2010
• Published: 08/23/2012
• Est. Priority Date: 07/01/2009
• Status: Active Grant

• Current Assignee: Case Western Reserve University, Mtd Products Incorporated (Stanley Black & Decker, Inc.)
• Original Assignee: Case Western Reserve University, Mtd Products Incorporated (Stanley Black & Decker, Inc.)
• Inventors: Quinn, Roger D., Merat, Francis L., Schepelmann, Alexander, Daltorio, Kathryn A., Rolin, Amaury D., Beno, Jonathan, Hughes, Bradley E., Green, James M., Branicky, Michael S., Snow, Henry H., Hudson, Richard E.
• Granted Patent: US 8,731,295 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: A01D 34/008 for automated or remotely controlled operation
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30188 Vegetation; Agriculture
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/44 using image operators, e.g. filters, edge density metrics or local histograms
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/449 Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/809 of classification results, e.g. where the classifiers operate on the same input data

Abstract

This invention provides a method for identifying lawn grass comprising capturing an image of the terrain in front of a mower, segmenting the image into neighborhoods, calculating at least two image statistics for each of the neighborhoods, generating a binary representation of each image statistic. The binary representation of each image statistic is generated by comparing the calculated image statistic values to predetermined image statistic values for grass. The method further comprises weighting each of the binary representations of each image statistic, and summing corresponding neighborhoods for all image statistics. A binary threshold is applied to each of the summed neighborhoods to generate a binary map representing grass containing areas and non-grass containing areas.

Claims

1. A method for identifying lawn grass comprising:

capturing an image of terrain in front of a mower;

segmenting said image into neighborhoods;

calculating values for at least two image statistics for each of said neighborhoods;

comparing said image statistic values of said neighborhoods to predetermined image statistic values for grass;

generating a binary representation of each image statistic value, wherein a neighborhood for an image statistic value is marked as grass containing when said image statistic value of said neighborhood compares favorably to a predetermined image statistic value for grass;

weighting each of said binary representations of each image statistic;

summing corresponding neighborhoods for all image statistics; and

applying a binary threshold to each of said summed neighborhoods, wherein said binary threshold produces a binary map representing grass containing areas and non-grass containing areas.

2. The method of claim 1, wherein one of said image statistics is mean hue.

3. The method of claim 1, wherein said image statistic value of said neighborhood compares favorably to a predetermined image statistic for grass when said image statistic value of said neighborhood is within a predetermined number of standard deviations of said predetermined image statistic value for grass.

4. The method of claim 1, wherein at least one of said image statistics is calculated using an intensity plane extracted from said captured image.

5. The method of claim 4, wherein one of said image statistics is:

mean grayscale horizontal intensity, mean grayscale vertical intensity, vertical binary variance, mean vertical binary edge response area, horizontal and vertical neighborhood centroid location, horizontal binary variance, or mean horizontal binary edge response area.

6. The method of claim 1, wherein said binary representations of each image statistic are weighted by multiplying said neighborhoods in said binary representations of each image statistic by predetermined normalizing weighting coefficients.

7. The method of claim 1, wherein said binary map representing grass containing areas and non-grass containing areas is abstracted to a range image.

8. A vision processing system for an autonomous lawnmower comprising:

an image capturing camera mounted on said lawnmower for capturing an image of terrain in front of said lawnmower;

a processing arrangement connected to said camera for segmenting said image into neighborhoods;

said processing arrangement further providing structure for calculating values for at least two image statistics for each neighborhood;

said processing arrangement further providing structure for generating a binary representation of each image statistic value, wherein a neighborhood for an image statistic value is marked as grass containing when said image statistic value of said neighborhood compares favorably to a predetermined image statistic for grass;

said processing arrangement further providing structure for applying a weight to each of said binary representations of each image statistic.

said processing arrangement further providing structure for summing corresponding neighborhoods for all image statistics; and

said processing arrangement further providing structure for applying a binary threshold to each of said summed neighborhoods, wherein a binary map is produced representing grass containing areas and non-grass containing areas.

9. The vision system of claim 8, wherein said image statistic of said neighborhood compares favorably to a predetermined image statistic for grass when said image statistic of said neighborhood is within a predetermined number of standard deviations of the predetermined image statistic for grass.

10. The vision system of claim 8, wherein one of said image statistics is mean hue.

11. The vision system of claim 8, wherein at least one of said image statistics is calculated using an intensity plane extracted from said captured image.

12. The vision system of claim 11, wherein one of said image statistics is:

mean grayscale horizontal intensity, mean grayscale vertical intensity, vertical binary variance, mean vertical binary edge response area, horizontal and vertical neighborhood centroid location, horizontal binary variance, or mean horizontal binary edge response area.

13. The vision system of claim 8, wherein said binary representations of each image statistic are weighted by multiplying said neighborhoods in said binary representations of each image statistic by predetermined normalizing weighting coefficients.

14. The vision system of claim 8, wherein said binary map representing grass containing areas and non-grass containing areas is abstracted to a range image.

15. A method for identifying lawn grass by an autonomous lawnmower comprising:

capturing an image of terrain in front of said mower;

segmenting said image into neighborhoods;

calculating values for at least two image statistics for each of said neighborhoods;

generating a binary representation of each image statistic, wherein said binary representation is generated by comparing said calculated image statistic values to predetermined image statistic values for grass;

weighting each of said binary representations of each image statistic;

summing corresponding neighborhoods for all image statistics; and

applying a binary threshold to each of said summed neighborhoods to generate a binary map representing grass containing areas and non-grass containing areas.

16. The method of claim 15, wherein one of said image statistics is mean hue.

17. The method of claim 15, wherein at least one of said image statistics is calculated using an intensity plane extracted from said captured image.

18. The method of claim 17, wherein one of said image statistics is:

mean grayscale horizontal intensity, mean grayscale vertical intensity, vertical binary variance, mean vertical binary edge response area, horizontal and vertical neighborhood centroid location, horizontal binary variance, or mean horizontal binary edge response area.

19. The method of claim 15, wherein said binary representations of each image statistic are weighted before summing said corresponding neighborhoods of all image statistics.

20. The method of claim 15, wherein said image statistic of said neighborhood compares favorably to a predetermined image statistic for grass when said image statistic of said neighborhood is within a predetermined number of standard deviations of the predetermined image statistic for grass.