Image processing apparatus, endoscope image sensing and processing apparatus, and image processing method for performing different displays depending upon subject quantity

• US 5,512,940 A
• Filed: 03/17/1994
• Issued: 04/30/1996
• Est. Priority Date: 03/19/1993
• Status: Expired due to Term

• Current Assignee: Olympus Optical Company Ltd. (Olympus Corporation)
• Original Assignee: Olympus Optical Company Ltd. (Olympus Corporation)
• Inventors: Nakamura, Kazunari, Takasugi, Kei
• Primary Examiner(s): Britton, Howard W.
• Application Number:
  US08/214,202
• Time in Patent Office: 775 Days
• Field of Search: 348/45, 348/65, 348/68-71, 348/30
• US Class Current: 348/71
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/30028 Colon; Small intestine
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: H04N 17/02 for colour television signals
• CPC Class Codes: H04N 9/64 Circuits for processing colour signals H04N9/77 takes precedence; camera processing pipelines for processing colour signals H04N23/84

Abstract

An endoscope image sensing and processing apparatus uses the level of brightness of G and B signals at position (i, j) generated by subjecting an image of a subject picked up by a CCD to a signal process to detect the hemoglobin quantity IHb (i, j), discriminates whether or not the hemoglobin quantity IHb (i, j) in the image portion, which is the subject, is included in a predetermined range of a set subject quantity or outside the same to generate pseudo color data for displaying an image portion within the predetermined range and an image portion outside the predetermined range in different display colors so as to display the image portions on a monitor.

Claims

1. An image processing apparatus comprising:

calculating means that subjects, to a predetermined calculation process, a plurality of image signals obtained by image sensing means by taking an image of a subject having light beams in a plurality of different wavelength regions to calculate a subject quantity of said subject or a subject quantity caused from a medication dosed into said subject;

range setting means for setting a predetermined range of said subject quantity; and

display processing means for subjecting said image signals to a predetermined process in order to display, by different display methods, a portion of said image of said subject in which said subject quantity calculated by said calculating means is included in said predetermined range of said subject quantity and a portion of said image of said subject in which said subject quantity calculated by said calculating means is outside said predetermined range of said subject quantity.

2. An image processing apparatus according to claim 1, wherein

said range setting means sets a peculiar value displayed by said subject quantity in a physically changed portion caused by a disease of said subject as a value within said predetermined range or a value outside said predetermined range,

said display processing means discriminates whether or not said subject quantity calculated by said calculating means is included in said predetermined range set by said range setting means and subjects said image signals to a predetermined conversion process in order to display said physically changed portion caused by a disease and a normal portion of said subject by different displaying methods in accordance with results of said discrimination.

3. An image processing apparatus according to claim 1, wherein

said display processing means includes

said calculating means for calculating said subject quantity for each pixel unit,

discrimination means for detecting, for each pixel unit, a fact whether or not said subject quantity calculated by said calculating means is a portion of said subject image included in said predetermined range of said subject quantity set by range setting means or a fact whether or not the same is outside said predetermined range of said subject quantity, and

processing means for subjecting said image signals to a predetermined process in order to differently display pixels included in said predetermined range of said subject quantity detected by said discrimination means and pixels outside said predetermined range of said subject quantity.

4. An image processing apparatus according to claim 1 further comprising

instruction means for selecting said predetermined range set by said range setting means, wherein

said range setting means has, set therein, a plurality of different predetermined ranges as said predetermined range of said subject, to select any one of said plurality of predetermined ranges in accordance with said instruction means so as to set said predetermined range.

5. An image processing apparatus according to claim 1, wherein said range setting means sets a rated value of said subject quantity and sets values larger than said rated value to be within said predetermined range or outside said predetermined range.

6. An image processing apparatus according to claim 1, wherein said range setting means sets a rated value of said subject quantity and sets values smaller than said rated value to be within said predetermined range or outside said predetermined range.

7. An image processing apparatus according to claim 1, wherein said range setting means sets a reference value of said subject quantity and sets a rated width from said reference value to be within said predetermined range or outside said predetermined range.

8. An image processing apparatus according to claim 1, wherein

said range setting means further sets a reference value in said predetermined range of said subject quantity, and

said display processing means subjects said image signals to a predetermined process in order to differently display a portion larger than said reference value and a portion smaller than said reference value in a portion of said subject image in which said subject quantity is included in said predetermined range.

9. An image processing apparatus according to claim 1, wherein

said range setting means includes

average value calculating means for obtaining the average value of said subject quantities calculated by said calculating means, and

means that uses said average value obtained by said average value calculating means to set a predetermined range of said subject quantity.

10. An image processing apparatus according to claim 1, wherein

said range setting means sets a minimum value and a maximum value of said predetermined range of said subject quantity respectively and sets a range between said minimum value and said maximum value to be within said predetermined range or outside said predetermined range, and

said display processing means subjects said image signals to a predetermined process to differently display a portion of said subject image in which said subject quantity calculated by said calculating means is included in a range between said minimum value and said maximum value set by said range setting means, a portion of said subject image in which said subject quantity is smaller than said minimum value and a portion of said subject image in which said subject quantity is larger than said maximum value.

11. An image processing apparatus according to claim 10, wherein

said display processing means subjects, to a predetermined conversion process, a portion of said subject image in which said subject quantity calculated by said calculating means is between said minimum value and said maximum value to differently display said portion of said subject image in accordance with the value of said subject quantity calculated by said calculating means.

12. An image processing apparatus according to claim 10, wherein

said display processing means performs a predetermined conversion process in order to differently display a case where said subject quantity obtained by said calculating means is smaller than said minimum value of said predetermined range and a case where said subject quantity is larger than said maximum value of said predetermined range.

13. An image processing apparatus according to claim 1 further comprising detection means for detecting a portion of said subject image in said image signals picked up by said image sensing means which is effective to perform observation and a portion of said subject image which is ineffective to perform said observation, and

said display processing means subjects said image signals to different conversion processes corresponding to at least two portions of said subject image in order to differently display a portion of said subject image which is ineffective to perform said observation, a portion of said subject image which is effective to perform said observation and which is outside said predetermined range of said subject quantity and a portion of said subject image which is effective to perform said observation and which is included in said predetermined range of said subject quantity.

14. An image processing apparatus according to claim 13, wherein

said range setting means includes

average value calculating means for obtaining the average value of said subject quantity calculated by said calculating means for only an effective portion of said subject image detected by said detection means, and

means for setting a predetermined range of said subject quantity by using said average value obtained by said average value calculating means.

15. An image processing apparatus according to claim 13, wherein said detection means detects a portion encountered halation or a dark portion as said portion effective to perform said observation.

16. An image processing apparatus according to claim 1, wherein said subject quantity is a value correlated to information about the function of an organism.

17. An image processing apparatus according to claim 1, wherein said subject quantity is a value correlated to information about the function of an organism on which the distribution of the concentrations of said medication dosed into said organism, which is said subject, is reflected.

18. An image processing apparatus according to claim 16 or 17, wherein said information about said function of said organism is a quantity of a specific texture of said organism or a quantity denoting the state of a specific texture.

19. An image processing apparatus according to claim 18, wherein

said quantity of said specific texture of said organism is a quantity of hemoglobin, and

said state of said specific texture of said organism is the degree of oxygen saturation of said hemoglobin.

20. An image processing apparatus according to claim 1, wherein said calculating means subjects, to a predetermined calculating process, a plurality of image signals obtained by picking up, by said image sensing means, an image of a subject or a subject dosed with said medication obtained due to irradiation performed with light beams having wavelength regions in which the degree of absorbance or reflection are different from one another so that said subject quantity is calculated.

21. An image processing apparatus according to claim 20, wherein said calculating means subjects, to a predetermined calculation process, a plurality of image signals obtained by picking up, by said image sensing means, said image of said subject having light beams in a plurality of different wavelength regions so that a quantity of hemoglobin in said subject is calculated,

said range setting means sets a quantity of hemoglobin within said predetermined range among said quantities of hemoglobin serving as said subject quantity as said quantity within said predetermined range of said subject quantity;

and

said display processing means subjects said image signals to a predetermined process in order to display, by different displaying methods, a portion of said subject image in which said quantity of hemoglobin calculated by said calculating means is included in said predetermined range of said quantity of hemoglobin set by range setting means and a portion of said subject image in which said quantity of hemoglobin calculated by said calculating means is outside said predetermined range.

22. An image processing apparatus according to claim 20, wherein

said calculating means subjects, to a predetermined calculating process, said subject images having a plurality of different wavelength regions with respect to a plurality of image signals picked up by said image sensing means so as to calculate the degree of oxygen saturation of hemoglobin of said subject,

said range setting means sets said degree of oxygen saturation of hemoglobin among degrees of oxygen saturation of hemoglobin serving as said subject quantity within said predetermined range to be a quantity within said predetermined range, and

said display processing means subjects, to a predetermined calculating process, said image signals in order to display, by different displaying methods, a portion of said subject image in which said degree of oxygen saturation of hemoglobin calculated by said calculating means is included in said predetermined range of said degree of oxygen saturation of hemoglobin and a portion of said subject image in which said degree of oxygen saturation of hemoglobin calculated by said calculating means is outside said predetermined range.

23. An image processing apparatus according to claim 20, wherein

said calculating means includes means which subjects, to a predetermined calculating process, said subject image in a state where fluorochrome has been dosed into said subject and having light beams in a plurality of said wavelength regions with respect to a plurality of said image signals picked up by said image sensing means to calculate the concentration of said fluorochrome in said subject,

said range setting means sets said concentration of said fluorochrome serving as said subject quantity within said predetermined range of said concentration of said fluorochrome as a quantity within said predetermined range of said subject quantity, and

said display processing means subjects, to a predetermined calculating process, said image signals in order to display, by different displaying methods, a portion of said subject image in which said concentration of said fluorochrome calculated by said calculating means is included in said predetermined range of said concentration of said fluorochrome and a portion of said subject image in which said concentration of said fluorochrome calculated by said calculating means is outside said predetermined range.

24. An image processing apparatus according to claim 20, wherein

said calculating means includes means which subjects, to a predetermined calculating process, said subject image in a state where pigment has been dosed into said subject and having light beams in a plurality of said wavelength regions with respect to a plurality of said image signals picked up by said image sensing means to calculate the concentration of said pigment in said subject,

said range setting means sets said concentration of said pigment serving as said subject quantity within said predetermined range of said concentration of said pigment as a quantity within said predetermined range of said subject quantity, and

said display processing means subjects, to a predetermined calculating process, said image signals in order to display, by different displaying methods, a portion of said subject image in which said concentration of said pigment calculated by said calculating means is included in said predetermined range of said concentration of said pigment and a portion of said subject image in which said concentration of said pigment calculated by said calculating means is outside said predetermined range.

25. An image processing apparatus according to claim 24, wherein said pigment is methylene blue.

26. An image processing apparatus according to claim 24, wherein said pigment is ICG (Indocyanine Green).

27. An image processing apparatus according to claim 1, wherein said display processing means subjects said image signals to a predetermined process to make the correlation between information about an original image contained in said image signal picked up by said image sensing means and information of said image to be displayed to be different in at least either of a portion of said subject image in which said subject quantity is included in said predetermined range or a portion of said subject image in which said subject quantity is outside said predetermined range in order to differently display a portion of said subject image included in said predetermined range and a portion of said subject image outside said range.

28. An image processing apparatus according to claim 27, wherein

said display processing means makes said correlation to be different by subjecting said image signals to a conversion process to make information, which is included in said image signal obtained due to image sensing performed by said image sensing means and which cannot be taken by said original image to be information of said image to be displayed.

29. An image processing apparatus according to claim 27, wherein

said display processing means makes said correlation to be different by performing a chromaticity conversion process to make chromaticity, which is information included in said image to be displayed, to be different from chromaticity, which is information of said original image included in said image signal.

30. An image processing apparatus according to claim 27, wherein

said display processing means makes said correlation to be different by performing a hue conversion process in accordance with a table set in such a manner that the hue, which is information of said image to be displayed, is different from the hue which is information about said original image included in said image signal.

31. An image processing apparatus according to claim 27, wherein

said display processing means makes said correlation to be different by performing a color saturation conversion process in accordance with a table set in such a manner that the color saturation, which is information of said image to be displayed, is different from the color saturation which is information about said original image included in said image signal.

32. An image processing apparatus according to claim 27, wherein

said display processing means makes said correlation to be different by performing a conversion process as said predetermined process in such a manner that said image to be displayed is made in achromatic color with respect to information of said original image included in said image signal.

33. An image processing apparatus according to claim 27, wherein

said display processing means makes said correlation to be different by performing a conversion process as said predetermined process in such a manner that said image to be displayed is made to have a specific pattern with respect to information of said original image included in said image signal.

34. An endoscope image sensing and processing apparatus comprising:

an endoscope having at least an imaging optical system for imaging an image of a subject;

wavelength separating means for separating light into light beams in a plurality of wavelength regions in which an absorbance or a quantity of change in the absorbance is made different due to a subject quantity of a subject or a subject quantity realized due to a medication dosed in said subject;

image sensing means for picking up an endoscope image of said subject having light beams in a plurality of wavelength regions separated by said wavelength separating means to transmit a plurality of image signals,

calculating means that subjects, to a predetermined calculation process, a plurality of said image signals obtained by said image sensing means by picking up said image of said subject formed by light beams in a plurality of wavelength regions in which said absorbance or said quantity of change in said absorbance is different to calculate said subject quantity;

range setting means for setting a predetermined range of said subject quantity; and

display processing means for subjecting said image signals to a predetermined process in order to differently display a portion of said subject image in which said subject quantity calculated by said calculating means is included in said predetermined range of said subject quantity set by said range setting means and a portion of said subject image in which the same is outside said predetermined range of said subject quantity.

35. An endoscope image sensing and processing apparatus according to claim 34, wherein

said calculating means includes

means for calculating the ratio of an image signal obtained from a light beam in a wavelength region in which the quantity of change in said absorbance is large and an image signal obtained from a light beam in a wavelength region in which the quantity of change in said absorbance is small, said image signals being included a plurality of image signals obtained from light beams in a plurality of said wavelength regions which have been separated by said wavelength separating means and in which said quantities of change in said absorbance are different.

36. An endoscope image sensing and processing apparatus according to claim 34, wherein

said calculating further includes means for calculating the logarithm of said ratio obtained by said means for calculating said ratio.

37. An endoscope image sensing and processing apparatus according to claim 34, wherein

said calculating means includes means for calculating the ratio of an image signal obtained from a light beam in a wavelength region in which the quantity of change in said absorbance is large and an image signal obtained from a light beam in a wavelength region in which the quantity of change in said absorbance is small, said image signals being included a plurality of image signals obtained from light beams in a plurality of said wavelength regions which have been separated by said wavelength separating means and in which said quantities of change in said absorbance are different.

38. An endoscope image sensing and processing apparatus according to claim 34, wherein

said calculating means includes selection means for selectively transmitting an image signal obtained from a light beam in a wavelength region in which the quantity of change in said absorbance is large or an image signal obtained from a light beam in a wavelength region in which the quantity of change in said absorbance is small, said image signals being included a plurality of image signals obtained from light beams in a plurality of said wavelength regions which have been separated by said wavelength separating means and in which said quantities of change in said absorbance are different.

39. An endoscope image sensing and processing apparatus according to claim 34, wherein

said wavelength separating means for separating light into light beams in a plurality of wavelength regions in which absorbances are different due to hemoglobin in said subject,

said calculating means includes hemoglobin quantity calculating means for calculating the ratio of an image signal obtained from a light beam in a wavelength region in which the absorbance due to hemoglobin is large and an image signal obtained from a light beam in a wavelength region in which the absorbance due to hemoglobin is small and calculating the logarithm of said obtained ratio, said light beams being in said wavelength regions separated by said wavelength separating means,

said range setting means set a predetermined range of said quantity of hemoglobin, and

said display processing means discriminates that said quantity of hemoglobin obtained by said hemoglobin quantity calculating means is included in said predetermined range set by said range setting means or outside said predetermined range and performs said predetermined process in order to differently display an image portion, which is the subject of said discrimination, between a case where the same is included in said predetermined range and a case where the same is included in the outside of said predetermined range.

40. An endoscope image sensing and processing apparatus according to claim 34, wherein

said wavelength separating means separates light into light beams in a plurality of wavelength regions in which the quantity of change in said absorbance is different due to the degree of oxygen saturation of hemoglobin in said subject,

said calculating means includes means for calculating degree of oxygen saturation of hemoglobin for calculating the ratio of an image signal obtained from a light beam in a wavelength region in which the change in the quantity of said absorbance due to said degree of oxygen saturation of hemoglobin is large and an image signal obtained from a light beam in a wavelength region in which the change in the quantity of said absorbance due to said degree of oxygen saturation of hemoglobin is small and calculating the logarithm of said obtained ratio, said light beams being in said wavelength regions separated by said wavelength separating means,

said range setting means sets a predetermined range of said degree of oxygen saturation of hemoglobin, and

said display processing means discriminates that said degree of oxygen saturation of hemoglobin obtained by said means for calculating degree of oxygen saturation of hemoglobin is included in said predetermined range set by said range setting means or outside said predetermined range and performs said predetermined process in order to differently display an image portion, which is the subject of said discrimination, between a case where the same is included in said predetermined range and a case where the same is included in the outside of said predetermined range.

41. An endoscope image sensing and processing apparatus according to claim 34, wherein

said wavelength separating means includes a filter having the characteristics which permit a wavelength region including 805 nm to pass through and a filter having the characteristics which permit a predetermined region except said wavelength region to pass through.

42. An endoscope image sensing and processing apparatus according to claim 34, wherein

said wavelength separating means includes a filter having the characteristics which permit a wavelength region including 650 nm to pass through and a filter having the characteristics which permit a predetermined region except said wavelength region to pass through.

43. An endoscope image sensing and processing apparatus according to claim 34, wherein

said wavelength separating means includes

first wavelength separating means for separating the subject image into images in a plurality of wavelength regions in order to obtain an image in a visible region,

second wavelength separating means for separating an image in a wavelength region in which the quantity of light made incident upon a light receiving portion of said image sensing means is changed due to the quantity of change in said degree of oxygen saturation of hemoglobin, said second wavelength separating means being disposed integrally or individually from said first wavelength separating means, and

said endoscope image sensing and processing apparatus is arranged in such a manner that at least a portion of said first wavelength separating means and said second wavelength separating means are used to cause an optical image in a visible region including image information, with which the quantity of the change in the degree of oxygen saturation of hemoglobin is obtained can be made larger, to be picked up by said image sensing means.

44. An image processing method for performing different displays depending upon a subject quantity, said image processing method comprising:

a calculating step that subjects, to a predetermined calculation, a plurality of image signals obtained by image sensing means due to an operation of picking up an image of a subject obtained due to light beams in wavelength regions, which are absorbed or reflected by said subject at different degrees in accordance with a subject quantity of said subject or a subject quantity caused from a dosed medication so as to obtain said subject quantity as data;

a step for discriminating whether or not said data obtained by said calculating means is included in said predetermined range;

a step for formalizing said data in a case where a discrimination has been made in said discrimination step that said data is included in said predetermined range;

a step for converting data in such a manner that information included in an image to be displayed is made different from information of an original image included in said image signal in accordance with the value of said data normalized in said normalizing step; and

a step for converting said data by data conversion performed in such a manner that the correlation between said information of said original image included in said image signal and said information included in said image to be displayed is different from that in said step for converting data in a case where a discrimination has been made in said discrimination step that said data is included in said outside portion of said predetermined range.

45. An image processing method for performing different displays depending upon a subject quantity according to claim 44, wherein

said range setting means sets the minimum value and the maximum value of said subject quantity and sets a range between said minimum value and said maximum value as a predetermined range, and

said discriminating step discriminates whether or not said data obtained by said calculating means is included in said predetermined range in accordance with a fact whether or not said data is present between said minimum value and said maximum value.