Method and arrangement for automatic optical classification of plants

• US 5,253,302 A
• Filed: 12/30/1992
• Issued: 10/12/1993
• Est. Priority Date: 02/28/1989
• Status: Expired due to Fees

• Current Assignee: Robert Massen
• Original Assignee: Robert Massen
• Inventors: Massen, Robert
• Primary Examiner(s): Moore, David K.
• Assistant Examiner(s): Johns, Andrew W.
• Application Number:
  US07/998,760
• Time in Patent Office: 286 Days
• Field of Search: 382/1, 382/9, 382/17, 382/25, 209/580, 209/939, 356/12, 356/402, 358/88, 358/107, 358/108, 358/109
• US Class Current: 382/110
• CPC Class Codes: A01G 7/00 Botany in general
• CPC Class Codes: B07C 5/10 measured by light-responsive means sorting according to colour B07C5/342
• CPC Class Codes: B07C 5/3422 using video scanning devices, e.g. TV-cameras
• CPC Class Codes: G01N 2021/8466 Investigation of vegetal material, e.g. leaves, plants, fruits
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 7/0006 using a design-rule based approach
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: H04N 13/139 Format conversion, e.g. of frame-rate or size
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 2013/0077 Colour aspects
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

For automatic optical classification of plants an image of in each case one plant is captured by a color video camera. The color video signals furnished by the color video camera are digitized and classified pixelwise in predetermined color classes which are associated on the one hand with the image of the plant and on the other hand with the image of the background. A digitized image of the plant designated by the color classes is stored in an image memory and evaluated by an image computer for classification of the plant. The segmentation of the image of the plant from the image of the background is carried out on the basis of the allocation of the pixels to the predetermined color classes and from the segmented image of the plant geometrical form features and color features are determined. Furthermore, the height of the plant is determined, for which purpose possibly two video cameras and a light pattern projector are employed. The color classifier used for color classification is configured so that it can learn the predetermined color classes by presentation of image fragments.

Claims

1. A method for automatic optical classification of plants comprising the steps of:

(a) obtaining an image of a plant with a color video camera;

(b) digitizing pixelwise the color video signals obtained by the color video camera;

(c) classifying pixelwise the digitized color video signals in accordance with predetermined color classes, said color classes comprising sets of arbitrarily arranged color vectors to which a respective class code is attributed to under supervised learning on the basis of their belonging to the same meaningful regions;

(d) segmenting the image of the plant from at least one of the image of the background and the images of different parts of the plant from one another on the basis of the allocation of the pixels of the stored image to the predetermined color classes;

(e) determining at least one of geometrical form features and color features from at least one of the segmented image of the plant and the segmented images of the plant parts; and

(f) evaluating at least one of the determined form features and color features for deriving a quality measure.

2. Method according to claim 1, characterized in that the form and/or color features determined are compared with predetermined threshold values and that the quality measure is derived from the comparison result.

3. Method according to claim 1, characterized in that the color classification takes place in a transformed color space.

4. Method according to claim 3, characterized in that the color video camera furnishes RGB signals and that the digitized RGB signals are converted to digital IHS signals.

5. Method according to claim 1, characterized in that from the image of the plant captured by the color video camera the height of the plant is determined by a light section method.

6. Method according to claim 5, characterized in that a light pattern evaluatable by the light section method is projected onto the plant.

7. Method for automatic optical classification of plants comprising:

acquiring an image of in each case one plant by a color video camera, digitizing pixelwise video signals of the acquired image, storing a digitized image of the plant in an image memory and evaluating the stored digitized image by an image computer for classification of the plant, classifying pixelwise the digitized color video signals prior to storing of the digitized image in accordance with predetermined color classes which are assigned on the one hand to the image of the plant and on the other to the image of the background, effecting the segmentation of the image of the plant from the image of the background on the basis of the allocation of the pixels to the predetermined color classes, determining the segmented image of the plant geometrical form features and evaluating the form features for derivation of a quality measure, the color classification is effected by a color classifier comprising a memory having a plurality of memory cells each of which is allocated to a different color value in the color space, said color classifier being interactively trainable by presentation and said training includes;

a. erasing the contents of all memory cells of the memory of the color classifier;

b. displaying the image of a plant captured by the color video camera on a monitor;

c. marking a fragment containing as many pixels as possible of a color class to be identified by the user in the image displayed on the monitor;

d. incrementing in the memory of the color classifier, for each pixel of the marked image fragment the content of the memory cell allocated to the color value of the pixel;

e. designating all the memory cells having contents which due to the incrementations are greater than a predetermined threshold value as belonging to the color class to be identified after the evaluation of all the pixels of the marked image fragment; and

f. repeating steps c to e for each further color class to be identified.

8. Method according to claim 7, characterized in that the clusters formed by the designated memory cells in the color space used are subjected to a three-dimensional dilatation with subsequent erosion to close gaps.

9. Method according to claim 7, characterized in that in the incrementation of the content of the memory cell allocated to the color value of a pixel the contents of memory cells allocated to neighbouring color values in the color space used are incremented by amounts which are the smaller the greater the distances of the color values associated with said memory cells from the color value of the pixel.

10. Method according to claim 7, characterized in that a class code is allocated to each color class to be identified and that for designating the allocation of the memory cells to a color class the respective class code is written into each memory cell.

11. Method according to claim 7, characterized in that after the training of the color classifier for each pixel of the segmented image of a plant the memory cell of the image memory allocated to the pixel is designated as belonging to the same color class as the memory cell of the color classifier allocated to the color value of the pixel.

12. A method for automatic optical classification of plants characterized in that:

(a) a stereo image of each plant is captured by means of two color video cameras;

(b) the video signals furnished by each of said two color video cameras and corresponding to the image captured by said color video cameras are digitized pixelwise;

(c) the digitized color video signals corresponding to the image captured by one of said two color video cameras are classified pixelwise in accordance with predetermined color classes which are assigned on the one hand to the image of the plant and on the other to the image of the background;

(d) the segmentation of the image of the plant from the image of the background is effected on the basis of the allocation of the pixels to the predetermined color classes;

(e) from the segmented image of the plant geometrical form features are determined;

(f) each of the digitized stereo images represented by said digitized color video signals are stored in an image memory;

(g) a search is performed in said stored digitized stereo image for associated pixel pairs in search regions which correspond to plant parts which have been segmented by means of said color classification;

(h) the height of said plant parts is determined from the pixels of each pair ascertained by the search; and

(i) a quality measure is derived from both said geometrical form features and the heights of said plant parts.

13. Method according to claim 12 characterized in that the stored digitized stereo image is evaluated by the image computer by the method of stereo triangulation.

14. Method according to claim 12 characterized in that a light pattern is projected onto the plant during the taking of the stereo image.

15. Arrangement for automatic optical classification of plants comprising:

(a) two color video cameras arranged at a distance apart and directed onto the plant in such a manner that they capture a stereo image of the plant;

(b) an analog-digital converter arrangement for separately digitizing pixelwise the analog image signals furnished by the one and the other color video camera;

(c) a color classifier for classifying the digitized color video signals corresponding to the image captured by one of said two color video cameras pixelwise in accordance with predetermined color classes which are assigned on the one hand to the image of the plant and on the other hand to the image of the background;

(d) means for effecting the segmentation of the image of the plant from the image of the background on the basis of the allocation of the pixels to the predetermined color classes;

(e) means for determining geometrical form features from the segmented image of the plant;

(f) an image memory for storing the digitized stereo image represented by said digitized color video signals;

(g) a height computer performing a search for associated pixel pairs in the stored digitized stereo image in search regions corresponding to plant parts which have been segmented by means of said color classification and determining the height of said plant parts from the pixels of each pair ascertained by the search; and

(h) means for deriving a quality measure from both said geometrical form features and the heights of said plant parts.

16. Arrangement according to claim 15, characterized in that the height computer calculates the height of the plant by the method of stereo triangulation.

17. Arrangement according to claim 15 characterized in that the form identifying unit is connected to the height computer for definition of the search regions.

18. Arrangement according to claim 15 characterized by a light pattern projector which during the acquisition of the stereo image projects a light pattern onto the plant.

19. Arrangement according to claim 18 characterized in that the light pattern projector contains a light pattern generator controllable for generating different light patterns.

20. Arrangement according to claim 19, characterized in that the controllable light pattern generator is a three-dimensional light modulator.

21. Arrangement according to claim 20, characterized in that the three dimensional light modulator consists of a liquid crystal matrix.

22. Arrangement according to claim 19, characterized in that the controllable light pattern generator consists of a power-driven diapositive means.