Color classification apparatus

• US 5,717,605 A
• Filed: 10/12/1994
• Issued: 02/10/1998
• Est. Priority Date: 10/14/1993
• Status: Expired due to Term

• Current Assignee: Olympus Optical Corporation Limited (Olympus Corporation)
• Original Assignee: Olympus Optical Corporation Limited (Olympus Corporation)
• Inventors: Hibi, Yasushi, Wada, Tohru, Komiya, Yasuhiro, Ishii, Kensuke, Kikuchi, Susumu
• Primary Examiner(s): Trammell, James P.
• Assistant Examiner(s): PIPALA, EDWARD J
• Application Number:
  US08/321,667
• Time in Patent Office: 1,217 Days
• Field of Search: 364/525, 364/526, 364/497, 364/498, 364/550, 250/252.1, 250/227.11, 250/227.14, 250/226, 250/578.1, 73/53.01, 73/61.48, 73/61.49, 356/300, 356/303, 356/304, 356/306, 356/307, 356/309, 356/310, 356/319, 356/320, 356/332, 356/402, 356/407, 356/406, 356/408, 356/411, 356/414, 356/416, 356/418, 356/419, 356/420, 356/423, 356/425
• US Class Current: 356/406
• CPC Class Codes: G01J 3/02 Details
• CPC Class Codes: G01J 3/0218 using optical fibers
• CPC Class Codes: G01J 3/0229 using masks, aperture plates, spatial light modulators or spatial filters, e.g. reflective filters
• CPC Class Codes: G01J 3/12 Generating the spectrum; Monochromators
• CPC Class Codes: G01J 3/462 Computing operations in or between colour spaces; Colour management systems
• CPC Class Codes: G01J 3/501 Colorimeters using spectrally-selective light sources, e.g. LEDs
• CPC Class Codes: G01J 3/51 using colour filters
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/10144 Varying exposure
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 2207/30088 Skin; Dermal
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction

Abstract

The present invention provides a simple, low-cost color classification apparatus having a high resistance to mechanical vibrations and capable of properly performing color classification even if a light source is not specified and its spectrum changes. A color classification apparatus of the present invention is further including an image pickup means for picking up a spectrum of light reflected by a target object, a plurality of bandpass filters having different bands and arranged between the target object and the image pickup means, and a classifying means for calculating a classification spectrum for classification using a statistic technique from a spectrum of light reflected by the target object and picked up by the image pickup means, and classifying the target object using the classification spectrum.

Claims

1. A color classification apparatus comprising:

image pickup means for picking up a target object;

filter means for changing band characteristics, said filter means being arranged between the target object and said image pickup means; and

classification means for calculating a classification spectrum for classifying a spectrum of light reflected by the target object and picked up by said image pickup means, said classification means classifying the target object using the classification spectrum,

wherein said classification means comprises normalization means for normalizing both a spectral characteristic and a luminance of a light source which illuminates the target object so as to account for different light sources.

2. A color classification apparatus according to claim 1, wherein said classification means comprises reference plate data storage means for storing data obtained when a predetermined reference plate is picked up by said image pickup means, and

wherein said normalization means normalizes the spectral characteristic and the luminance of the light source based on data stored in the reference plate storage means.

3. A color classification apparatus according to claim 2, wherein:

said image pickup means picks up light reflected by the target object;

said filter means comprises a plurality of bandpass filters having different bands and arranged between the target object and said image pickup means; and

said classification means calculates a classification spectrum for classification using a statistic technique from a spectrum of the light reflected by the target object and picked up by said image pickup means.

4. A color classification apparatus according to claim 3, wherein said classification means comprises:

storage means for storing a spectrum of light reflected by a known-class target object picked up by said image pickup means;

classification spectrum calculation means for calculating the classification spectrum from the spectrum of light reflected by the known-class target object stored in said storage means; and

means for classifying an unknown-class target object picked up by said image pickup means using the classification spectrum calculated by said calculation means.

5. A color classification apparatus according to claim 4, wherein said classification spectrum calculation means comprises Foley Sammon transform operation means.

6. A color classification apparatus according to claim 4, wherein said classification spectrum calculation means comprises Hotelling Trace Criterion operation means.

7. A color classification apparatus according to claim 4, wherein said classification spectrum calculation means comprises difference spectrum calculation means.

8. A color classification apparatus according to claim 4, wherein said classification spectrum calculation means comprises orthogonal complement calculation means.

9. A color classification apparatus according to claim 3, wherein said classification means comprises:

a plurality of storage means which store, in advance, spectra of light components reflected by known-class target objects having different classes;

selective output means for outputting the stored spectrum of reflected light by properly selecting said storage means;

calculation means for calculating the classification spectrum from the spectrum of reflected light output by said selective output means; and

means for classifying an unknown-class target object picked up by said image pickup means using the classification spectrum calculated by said calculation means.

10. A color classification apparatus according to claim 3, wherein said plurality of bandpass filters have bands and amplitudes calculated from data obtained by picking up a spectrum of light reflected by a target object at a predetermined wavelength interval.

11. A color classification apparatus according to claim 3, further comprising means for controlling a transmittance or an exposure time of each of said plurality of bandpass filters.

12. A color classification apparatus according to claim 3, wherein said plurality of bandpass filters are arranged such that a spectrum of light reflected by a known-class target object is picked up in advance, statistic processing is performed using Foley Sammon transform to calculate a classification spectrum, and the classification spectrum is approximated when a light source for illuminating the target object is specified.

13. A color classification apparatus according to claim 3, wherein said image pickup means comprises image pickup elements having the number corresponding to said plurality of bandpass filters.

14. A color classification apparatus according to claim 3, wherein said plurality of bandpass filters comprise filter means having a predetermined number of bandpass filters and bandpass filter switching means for sequentially inserting and arranging each bandpass filter between the target object and said image pickup means.

15. A color classification apparatus according to claim 3, wherein said plurality of bandpass filters are constituted by a plurality of pairs of filter means each having a predetermined number of bandpass filters, and

said color classification apparatus comprises:

a light source for illuminating the target object;

light source detection means for detecting a type of said light source; and

second switching means for selectively and properly inserting and arranging one of said plurality of pairs of filter means between the target object and said image pickup means in accordance with the type of said light source detected by said light source detection means.

16. A color classification apparatus according to claims 14 or 15, wherein said filter means is constituted by filter means having a rotary color filter obtained by arranging a predetermined number of bandpass filters on a circumference.

17. A color classification apparatus according to claim 3, wherein said classification means comprises:

classification operation means for calculating the classification spectrum for classification using the statistic technique from the spectrum of light reflected by the target object picked up by said image pickup means, and calculating an inner product value between the classification spectrum and the spectrum of light reflected by the target object; and

classification determination means, including a neural network, for classifying the target object using the inner product value from said classification operation means.

18. A color classification apparatus according to claim 17, wherein said classification determination means comprises:

table forming means for forming a classification table using the neural network obtained by learning upon picking up a known-class target object by said image pickup means; and

means for classifying an unknown-class target object picked up by said image pickup means using the classification table formed by said table forming means.

19. A color classification apparatus according to claim 3, further comprising a mask filter or an image shift correction circuit for correcting a shift between images picked up by said plurality of bandpass filters.

20. A color classification apparatus according to claim 3, wherein, in classification of a plurality of classes, said classification means comprises means for obtaining a classification spectrum utilizing Foley Sammon transform, Hotelling Trace Criterion, a difference spectrum, or an orthogonal complement using arbitrary two of the plurality of classes.

21. A color classification apparatus according to claim 2, wherein:

said image pickup means picks up a light component reflected by the target object;

optical means is provided for focusing the light component reflected by the target object on said image pickup means;

said filter means comprises a plurality of bandpass filters having different bands and arranged between the target object and said image pickup means; and

said classification means calculates a classification spectrum for classification using a statistic technique from a spectrum of the light component reflected by the target object and picked up by said image pickup means, and performs said classification for a plurality of classes on the target object using the classification spectrum.

22. A color classification apparatus according to claim 2, wherein:

said image pickup means picks up a light component reflected by the target object;

optical means is provided for focusing the light component reflected by the target object on said image pickup means;

said filter means comprises a band variable bandpass filter capable of changing band characteristics and arranged between the target object and said image pickup means; and

said classification means calculates a classification spectrum for classification using a statistic technique from a spectrum of the light component reflected by the target object and picked up by said image pickup means, and performs said classification for a plurality of classes on the target object using the classification spectrum.

23. A color classification apparatus according to claim 21, wherein said optical means comprises optical path division means for splitting the light component reflected by the target object into a plurality of components.

24. A color classification apparatus according to claim 23, wherein said optical path division means is constituted by an optical fiber.

25. A color classification apparatus according to claim 23, wherein said optical path division means is constituted by a prism and a mirror.

26. A color classification apparatus according to claim 21 or 22, wherein said classification means comprises, in the classification for the plurality of classes;

means for obtaining a classification spectrum using arbitrary two of the plurality of classes;

inner product operation means for obtaining an inner product value between data of each class and the classification spectrum;

statistic amount calculation means for obtaining an average value of the inner product values and a variance value for the each class; and

boundary decision means for deciding a boundary of each class,

wherein the boundary is decided by said boundary decision means from an output from said statistic amount calculation means.

27. A color classification apparatus according to claim 21 or 22, wherein said classification means is constituted by a plurality of classification determination sections.

28. A color classification apparatus according to claim 27, wherein a first classification determination section of said plurality of classification determination sections receives spectrum data of a picked target object, and is constituted by:

first classification spectrum storage means for storing a classification spectrum obtained in advance; and

first operation means for performing an operation for classification from a spectrum of the target object and the spectrum stored in said first classification spectrum storage means, and

second and subsequent classification determination sections of said plurality of classification determination sections are connected in series, subsequently receive calculation results of previous stages and the spectrum data of the target object, and are constituted by;

a plurality of classification spectrum storage means for storing a plurality of classification spectra obtained in advance;

selection means for selecting said plurality of classification spectrum storage means in which classification spectra to be used are stored in accordance with the operation results of the previous stages, and outputting the classification spectra; and

operation means for outputting operation results from a spectrum of an unknown target object and the classification spectra output from said selection means.

29. A color classification apparatus according to claim 21 or 22, wherein said classification means comprises:

a plurality of spectrum storage means for storing known class data of the plurality of classes;

class selection means for selecting two spectrum storage means as a pair from said plurality of spectrum storage means;

classification spectrum operation means for operating a classification spectrum from spectrum data stored in said two spectrum storage means;

a plurality of classification spectrum storage means for storing a plurality of classification spectra calculated and output by said classification spectra calculation means from a plurality of pairs of spectrum data sequentially selected by said class selection means; and

classification determination means for performing classification determination from the plurality of classification spectra stored in said plurality of classification spectrum storage means and spectrum data of an unknown target object.

30. A color classification apparatus according to claim 21 or 22, wherein said classification means comprises a measurement region setting section,

said measurement region setting section comprising a specific region detection section for detecting a specific portion of the target object, and

a non-measurement region setting section for setting a non-measurement region including a region detected by said specific region detection section.

31. A color classification apparatus according to claim 21 or 22, wherein said classification means comprises:

a Fisher ratio calculation section for calculating a Fisher ratio of a formed classification spectrum;

dimension reducing means for reducing dimensionality of spectrum data used for an operation; and

means for forming a classification spectrum having a minimum dimensionality when the Fisher ratio of the formed classification spectrum is not less than a predetermined value.

32. A color classification apparatus according to claim 31, wherein said dimension reducing means eliminates a dimension having a small absolute value in the classification spectrum.

33. A color classification apparatus according to claim 31, wherein said dimension reducing means comprises difference spectrum calculation means for obtaining a difference spectrum between spectra of two classes used when the classification spectra are obtained;

and

means for eliminating a dimension having a small absolute value in the difference spectrum.

34. A color classification apparatus according to claim 31, wherein said dimension reducing means eliminates a dimension having a small value in a spectrum of a target object picked up when the classification spectra are obtained.

35. A color classification apparatus according to claim 21 or 22, wherein said classification means comprises:

image processing means for obtaining a feature amount of image data; and

spectrum expansion means for synthesizing a multi-dimensional spectrum of the target object and the feature amount obtained by said image processing means, and generating an expanded spectrum.

36. A color classification apparatus according to claim 21 or 22, further comprising storage means for storing image data picked up by said image pickup means.

37. A color classification apparatus according to claim 36, wherein said storage means comprises:

a first frame memory for temporarily storing the image data obtained by said image pickup means corresponding to an entire frame;

data conversion means for converting the data stored in said first frame memory into image data to be output to a TV monitor; and

a second frame memory for storing a TV monitor output image.

38. A color classification apparatus according to claim 37, wherein said first frame memory has a plurality of planes corresponding to said plurality of bandpass filters, and

said second frame memory has a plurality of planes corresponding to R, G, and B colors of a color monitor.

39. A color classification apparatus according to claim 38, wherein said data conversion means comprises:

a coefficient memory for storing a plurality of coefficients to be multiplied with outputs from the plurality of planes of said first frame memory;

a plurality of multipliers for multiplying the plurality of coefficients with the outputs from the plurality of planes of said first frame memory; and

an adder for adding outputs from said plurality of multipliers,

wherein an output from said adder can be separately written in the plurality of planes of said second frame memory.

40. A color classification apparatus according to claim 39, wherein the plurality of coefficients are sequentially set so as to approximate R, G, and B emission spectra of said color monitor, wherein data multiplied with R, G, and B coefficients are respectively written in R, G, and B planes of said second frame memory.

41. A color classification apparatus according to claim 39, wherein the plurality of coefficients are set so as to approximate the classification spectra obtained by said classification means.

42. A color classification apparatus according to claim 41, wherein, in said second frame memory, the output from said adder is directly written in one plane and data obtained by inverting a magnitude of the output from said adder is written in another plane.

43. A color classification apparatus according to claim 2, wherein:

said image pickup means picks up light reflected by the target object;

optical means is provided for focusing the light reflected by the target object on said image pickup means;

said filter means comprises a plurality of bandpass filters having different bands and arranged between the target object and said image pickup means;

said classification means calculates a classification spectrum for classification using a statistic technique from a spectrum of the light reflected by the target object picked up by said image pickup means; and

exposure condition control means is provided for controlling exposure conditions for image pickup operations of said plurality of bandpass filters,

said exposure condition control means having exposure limiting means for limiting exposure amounts corresponding to said plurality of bandpass filters.

44. A color classification apparatus according to claim 43, wherein said exposure control means is arranged to be detachable and exchangeable.

45. A color classification apparatus according to claim 43, wherein said exposure condition control means comprises exposure condition storage means for storing the exposure conditions for the image pickup operations of said plurality of bandpass filters.

46. A color classification apparatus according to claim 45, wherein said exposure condition control means comprises exposure condition correction means for correcting data picked up on the basis of the exposure conditions.

47. A color classification apparatus according to any one of claims 43 to 46, wherein said exposure control means comprises a plurality of openings corresponding to said plurality of bandpass filters, and a stop plate which can be switched in synchronism with said plurality of bandpass filters.

48. A color classification apparatus according to any one of claims 43 to 46, wherein said exposure control means comprises a plurality of filters having different transmittances corresponding to said plurality of bandpass filters, and filter means which can be switched in synchronism with said plurality of bandpass filters.

49. A color classification apparatus according to any one of claims 43 to 46, wherein said exposure control means comprises a light-shielding plate with a plurality of non-light-shielding portions having different sizes corresponding to said plurality of bandpass filters, and light-shielding means which can be switched in synchronism with said plurality of bandpass filters.

50. A color classification apparatus according to claim 2, wherein:

said image pickup means picks up light reflected by the target object;

optical means is provided for focusing the light reflected by the target object on said image pickup means;

said filter means comprises a plurality of bandpass filters having different bands and arranged between the target object and said image pickup means;

said classification means calculates a classification spectrum for classification using a statistic technique from a spectrum of the light reflected by the target object and picked up by said image pickup means; and

exposure condition control means is provided for controlling exposure conditions for image pickup operations of said plurality of bandpass filters,

said exposure condition control means having exposure condition storage means for storing the exposure conditions for the image pickup operations of said plurality of bandpass filters.

51. A color classification apparatus according to claim 50, wherein said exposure condition control means comprises exposure condition correction means for correcting data picked up on the basis of the exposure conditions.

52. A color classification apparatus according to claim 2, wherein:

said image pickup means picks up light reflected by the target object;

optical means is provided for focusing the light reflected by the target object on said image pickup means;

said filter means comprises a band variable bandpass filter capable of changing band characteristics and arranged between the target object and said image pickup means;

said classification means calculates a classification spectrum for classification using a statistic technique from a spectrum of the light reflected by the target object and picked up by said image pickup means;

exposure condition control means is provided for controlling exposure conditions when said plurality of bandpass filters are to be picked up; and

exposure condition control means is provided for separately controlling exposure conditions for image pickup operations for the different band characteristics set in said band variable bandpass filter.

53. A color classification apparatus according to claim 52, wherein said exposure condition control means comprises exposure condition storage means for storing the exposure conditions for the image pickup operations for the band characteristics of said band variable bandpass filter.

54. A color classification apparatus according to claim 51, wherein said exposure condition control means comprises exposure condition correction means for correcting data picked up on the basis of the exposure conditions.

55. A color classification apparatus according to any one of claims 50, 51, 53, and 54, wherein said exposure condition control means comprises stop control means for controlling stop adjustment means of said image pickup means on the basis of data from said exposure condition storage means.

56. A color classification apparatus according to any one of claims 50, 51, 53, and 54, wherein said exposure condition control means comprises:

a transmittance variable filter capable of changing a transmittance and arranged between the target object and said image pickup means; and

transmittance control means for controlling the transmittance of said transmittance variable filter on the basis of data from said exposure condition storage means.

57. A color classification apparatus according to any one of claims 50, 51, 53, and 54, wherein an exposure time of said image pickup means can be controlled in accordance with a drive pulse, and

said exposure condition control means has means for controlling the exposure time of said image pickup means on the basis of data stored in said exposure condition storage means in image pickup operations of said plurality of bandpass filters.

58. A color classification apparatus according to claim 2, wherein:

said image pickup means picks up light reflected by the target object;

optical means is provided for focusing the light reflected by the target object on said image pickup means;

said filter means comprises a plurality of bandpass filters having different bands and arranged between the target object and said image pickup means;

said classification means calculates a classification spectrum for classification using a statistic technique from a spectrum of the light reflected by the target object and picked up by said image pickup means; and

normalization means is provided for normalizing at least one or a spectrum characteristic and a luminance of a light source which illuminates the target object,

said normalization means having light source observation means for obtaining data of said light source.

59. A color classification apparatus according to claim 2, wherein:

said image pickup means picks up light reflected by the target object;

optical means is provided for focusing the light reflected by the target object on said image pickup means;

said filter means comprises a band variable bandpass filter capable of changing band characteristics and arranged between the target object and said image pickup means;

said classification means calculates a classification spectrum for classification using a statistic technique from a spectrum of the light reflected by the target object and picked up by said image pickup means; and

normalization means is provided for normalizing at least one or a spectrum characteristic and a luminance of a light source which illuminates the target object,

said normalization means having light source observation means for obtaining data of said light source.

60. A color classification apparatus according to claim 58 or 59, wherein said light source observation means comprises:

a reference plate for reflecting or transmitting light from said light source, and scattering the light; and

second optical means for guiding light reflected by said reference plate to said image pickup means.

61. A color classification apparatus according to claim 59, wherein said reference plate is arranged on an upper portion of a housing, and

said second optical means comprises an optical system for focusing said reference plate on said image pickup means, and a mirror for guiding an optical axis of said optical system to said image pickup means, and

a position of said mirror is displaceable.

62. A color classification apparatus according to claim 60, wherein said reference plate is arranged on an upper portion of a housing, and

said second optical means has a mirror for guiding an optical image of said reference plate to a visual field of said image pickup means.

63. A color classification apparatus according to claim 58, wherein said light source observation means comprises:

a reference plate, arranged at a front portion of a housing, for causing irregular reflection of light from said light source;

an optical sensor for detecting an intensity of light reflected by said reference plate;

a plurality of bandpass filters arranged between said reference plate and said optical sensor; and

optical means for focusing the light reflected by said reference plate on said optical sensor.

64. A color classification apparatus according to claim 63, wherein said plurality of bandpass filters arranged between said reference plate and said optical sensor also serve as said plurality of bandpass filters arranged between the target object and said image pickup means.

65. A color classification apparatus according to claim 63, wherein said optical sensor uses an image pickup element.

66. A color classification apparatus according to any one of claims 61, 62, and 65, wherein said light source observation means comprises light source direction detection means for detecting a direction of said light source.

67. A color classification apparatus according to claim 66, wherein said light source direction detection means comprises:

a rod- or plate-like member arranged on said reference plate such that a shadow shape or position of said member is changed in accordance with a change in the direction of said light source; and

illumination condition detection means for processing image data of said reference plate obtained by said image pickup means, and obtaining the direction, an intensity, and a spectrum of said light source.

68. A color classification apparatus according to claim 60, wherein said second optical means is constituted by an optical fiber.

69. A color classification apparatus according to claim 68, wherein said reference plate can be used detachably from said housing.

70. A color classification apparatus according to claim 57 or 58, wherein said light source observation means comprises light source spectral sensor means for detecting a spectrum of light from said light source.

71. A color classification apparatus according to claim 70, wherein said light source spectral sensor means comprises:

a transmittance variable filter capable of changing a transmittance;

a plurality of bandpass filters having same characteristics as those of said plurality of bandpass filters;

optical sensors, having the same number as that of said bandpass filters, for detecting an intensity of light transmitting through each bandpass filter;

a transmittance control circuit for monitoring outputs from said optical sensors, and controlling the transmittance of said transmittance variable filter; and

an illumination condition detection circuit for receiving signals from said plurality of optical sensors and said transmittance control circuit, and outputting a brightness and spectrum of said light source.

72. A color classification apparatus according to claim 70, wherein said light source spectral sensor means can be used detachably from said housing.