Image pickup system, image processor, and camera

• US 20050117045A1
• Filed: 09/04/2001
• Published: 06/02/2005
• Est. Priority Date: 09/17/1999
• Status: Abandoned Application

• Current Assignee: Nature Technologies
• Original Assignee: Nature Technologies
• Inventors: Kitamura, Koji, Abdellatif, Mohamed Abolella
• Application Number:
  US10/088,263
• Publication Number:
  US 20050117045A1
• Time in Patent Office: Days
• US Class Current: 348/335
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control
• CPC Class Codes: H04N 25/611 Correction of chromatic aberration

Abstract

An image capturing system, image processing system, and camera in which color can be precisely corrected to realize invariability of color through simple calibration while decreasing the area of a reference image part. The system comprises a camera having a lens, an image capturing system device, and a reflecting surface. Reference signal values (rn, gn, bn) are determined by averaging the intensities of the reflected lights of a reference scene received by the image capturing system device from the respective color channels of a plurality of pixel parts. The reference signal values represent the light source color, and a main image is corrected using the reference signal values.

Claims

1. An image capturing system for correction of colors in an image, comprising:

a camera (2) including a lens (41), image capturing devices (31, 37), light detecting elements (31, 33, 37, 38) and a reflection surface (61) for capture of a main scene (110) in the image capturing devices (31, 37), each of the image capturing devices (31, 37) and the light detecting elements (31, 33, 37, 38) having a plurality of color channels, the reflection surface (61) being disposed within a visual field of the camera (2) for reflection of light from the main scene (110) or a reference scene (121, 121a˜

e) disposed near the main scene (110) for reception by the light detecting elements (31, 33, 37, 38) via the lens (41), a light-color measuring portion (72, 153) obtaining a value from one pixel (136d) or an average value from a plurality of pixels (131, 131a˜

e, 136a·

c), for each of the color channels as reference signal values (rn, gn, bn), out of reflected light from the reference scene (121, 121a˜

e) received by the light detecting elements (31, 33, 37, 38); and

a correction unit (72) for correction of colors in the image by the reference signal values (rn, gn, bn).

2. The image capturing system according to claim 1, wherein the correction unit is a correcting portion (72) for practical division by the reference signal values (rn, gn, bn) obtained for each of the color channels, of respective main signal values (r[x] [y], g[x] [y], b[x] [y]) at each of corresponding locations on coordinates in the main scene (110) captured by the image capturing devices (31, 37), whereby obtaining corrected signal values (rc[x] [y], gc[x] [y], bc[x][y]) as corrected values of the main signal value.

3. The image processing unit used in the image capturing system according to claim 2, wherein coefficients (sr, sg, sb) having the reference signal values (rn, gn, bn) as respective denominators are obtained in advance for respective multiplication of these coefficients (sr, sg, sb) with the main signal values (r[x][y], g[x][y], b[x][y]), whereby performing correction of the main signal.

4. The image processing unit according to claim 3, wherein the coefficients (sr, sg, sb) have denominators respectively represented by the corresponding reference signal values (rn, gn, bn), and a numerator represented by another coefficient (s) common to all of the color channels.

5. The image processing unit according to claim 4, wherein the coefficients (sr, sg, sb) are obtained from one of frame signals sequentially sent from the image capturing devices (31, 37) or the light detecting elements (31, 33, 37, 38), said coefficients (sr, sg, sb) being multiplied respectively with the main signal values (r[x][y], g[x][y], b[x][y]) obtained from another frame signal received at a later time, whereby performing correction of the main signal.

6. The image processing unit according to claim 5, wherein the coefficients (sr, sg, sb) are multiplied respectively with a plurality of sets of the main signal values (r[x][y], g[x] [y], b[x] [y]) obtained from the plurality of other frames, whereby performing correction of the main signal.

7. The image processing unit according to claim 5, further including a video amplifier (79) for multiplication of the coefficients (sr, sg, sb) with the signals from the other frames.

8. The image processing unit according to claim 4, wherein if one of the main signal values (r[x] [y], g[x] [y], b[x] [y]) takes a presumably maximum value (rm, gm, bm) within a set of this signal, then said another coefficient (s) is set to a value which brings the presumably maximum value (rm, gm, bm) close to a maximum scale value (D) of the main signal values.

9. The image processing unit according to claim 4, wherein a pixel is defined as a corrupted pixel if the main signal values in the pixel have reached the maximum scale value (D) in two of the channels an if the main signal value in the remaining channel has not reached the maximum value (D), said another coefficient (s) having a value which brings presumably minimum values (rcm, bcm) of the main signal values in said remaining channel within a set of the corrupted pixels at least to the maximum scale value (D).

10. The image processing unit used in the image capturing system according to claim 2, wherein a corrected value (bc) of the main signal in a blue channel is calculated based on a ratio between corrected values (rc, gc) in red and green channels if the main signal value only in the blue channel has reached the maximum scale value (D) and if the main signal values in the red and green channels have not reached the maximum scale value (D).

11. The image processing unit used in the image capturing system according to claim 2, further including a compressing unit (81) of the main signal for compression of the main signal after the correction.

12. The camera used in the image capturing system according to claim 1, further including a reflection surface moving mechanism (65) capable of disposing the reflection surface (61) out of the visual field of the camera (2).

13. The image capturing system according to claim 1, further comprising a reflection surface moving mechanism (65) capable of disposing the reflection surface (61) out of the visual field of the camera (2) for disposition of the reflection surface (61) out of the visual field of the camera (2) by the reflection surface (61) after obtaining the reference signal values (rn, gn, bn) for capture of the main image, the main signal values (r[x] [y], g[x] [y], b[x] [y]) being corrected based on the reference signal values (rn, gn, bn).

14. The image capturing system according to claim 1, wherein each of the image capturing device (31) and the light detecting element (38) is constituted by an individual element of a same characteristic, the lens (41, 41) being provided individually for each of the image capturing device (31) and the light detecting element (38), the lenses (41, 41) being synchronized in zooming and iris controls, the angle and coordinate positions of a starting point of the reflection surface (61) being changed continuously in accordance with the focal length of the lens (41), the reflection surface (61) being fixed within a maximum visual field of the lens (41) for selection from a reference image portion (130), of selected reference portions (137a, 137b) corresponding to the reflection surfaces (61a, 61b) in accordance with the focal length.

15. The image capturing system according to claim 14, further comprising a coordinate table for elimination of the corrupted pixels of the light detecting element (38) when selecting the selected reference portions (137a, 137b).

16. The image capturing system according to claim 1, wherein the reference scene is limited mainly to a center portion or an adjacent portion of the main scene, by disposition of the reflection surface or selection of the plurality of pixels for the reference signals.

17. The image capturing system according to claim 2, further comprising at least another of the camera, the corrected signal values (rc[x] [y], gc[x][y], bc[x] [y]) being provided from one of the cameras for virtual multiplication in each of the color channels with the reference signal values provided from the other camera for obtaining a secondary corrected image, the secondary corrected image being merged with an image from said other camera into a synthesized image.

18. The image capturing system according to claim 2, further comprising a CG image generating portion (86) for generation of a computer image and a CG light source determining portion (87) for determining a light source color for the computer image for virtual multiplication of the corrected signal values (rc[x] [y], gc[x] [y], bc[x] [y]) in each of the color channels with a light source color value obtained by the CG light source determining portion (87) for obtaining a secondary corrected image, the secondary corrected image being merged with the computer image generated by the CG image generating portion (86) into a synthesized image.

19. The camera used in the image capturing system according to claim 1, wherein each of the image capturing devices (31, 37) and the light detecting elements (31, 33, 37, 38) is constituted by an individual element of a same characteristic.

20. The camera according to claim 19, wherein the light detecting elements (31, 37) are part of the image capturing devices (31, 37) respectively.

21. The camera used in the image capturing system according to claim 1, further including a storing portion (77) for storage of an image file containing images captured in the image capturing devices (31, 37) or a holding portion (36) for storage of a film (37) recorded with said images, said images containing the main scene (110) and the reference image portion (130) located at an end portion of an overall image region (100).

22. The camera used in the image capturing system according to claim 1, wherein the overall image region (100) is rectangular, having a corner portion disposed with the reference image portion (130).

23. The camera according to claim 22, wherein the reflection surface (61) is rotatable about a center axis of the lens (41), a position of the reflection surface (61) selectively determining one of the corners at which the reference image portion (130) being placed or the reference image portion not being placed within the overall image region (100).

24. The camera used in the image capturing system according to claim 1, wherein the main image is laterally elongated rectangular, the reference image portion being placed at an upper portion or a lower portion of the overall image region (100).

25. The camera used in the image capturing system according to claim 1, wherein the lens (41) is a zoom lens, the angle and coordinate positions of a starting point of the reflection surface (61) being changed in accordance with a focal length of the lens (41).

26. The camera used in the image capturing system according to claim 1, wherein the angle and coordinate positions of a starting point of the reflection surface (61) being changed continuously in accordance with the focal length of the lens (41), relative position between the reflection surface and the lens being changed in accordance with the focal length of the lens (41) by a reflection surface moving mechanism (65).

27. An IC chip or an electric circuit provided with function realized by the image processing unit according to any one of claims 3-11, or the image capturing system according to any one of claims 13-18.

28. A recording medium recorded with software to be loaded into a computer for execution of the function realized by the image processing unit according to any one of claims 3˜

11, or the image capturing system according to any one of claims 13˜

18.

29. The image processing unit according to any one of claims 3˜

11, or the image capturing system according to any one of claims 13˜

18, wherein the image correction is performed between two computers connected with each other via a communication link such as a telephone line or Internet.

30. The camera according to any one of claims 13, or 19˜

25, provided with a cover for prevention of light from entering into the reflection surface from outside of the main scene or the reference scene.

32. The camera used in the image capturing system according to claim 1, wherein the camera has an image capturing device sensitive to visible or invisible light.

33. The image capturing system according to claim 2, wherein the correction unit includes means for measuring a complimentary color of a color determined by the reference signal values (rn, gn, bn), and optical filter means including an optical filter for reproducing the complementary color and altering a color of an image which reaches the image capturing devices.

34. The image capturing system according to claim 33, wherein the optical filter is disposed so as to alter a color of the image which reaches the light detecting elements, the means for obtaining the complementary color controlling the optical filter so as to bring the color balance of the reference signal values (rn, gn, bn) as close as possible to a required color balance.

35. The image capturing system according to claim 33, wherein the optical filter means includes a plurality of preset filters each having a color balance different from the others, one of the present filters closest to the complementary color being selected.

36. The image capturing system according to claim 35, wherein a plurality of the preset filters can be used in combination.

37. The image capturing system according to claim 33, wherein the optical filter means includes a pump for pumping a medium, an ink injector capable of injecting a plurality of color inks individually, a mixer for making a mixture of the medium and the color inks, and a transparent passage serving as the optical filter for allowing the mixture to pass through.

38. The image capturing system according to claim 33, wherein the optical filter means includes a pump for pumping a medium, an ink injector capable of injecting a plurality of color inks individually, a plurality of mixers each for making a mixture of the medium and one of the color inks individually, and a plurality of transparent passages each serving as the optical filter for allowing one of the mixtures to pass through.

39. The image capturing system according to claim 33, wherein the optical filter means includes a pump for pumping a medium, an ink injector capable of injecting a plurality of color inks individually, a plurality of mixers each for making a mixture of the medium and one of the color inks individually, and a plurality of transparent cells each serving as the optical filter for allowing one of the mixtures to pass through, each cell being provided on a front surface of a black-and-white image capturing device, to correspond to one of RGB in one pixel, the cells assigned to a same color being interconnected via bridge path.

40. The image capturing system according to claim 33, wherein a filter characteristic of the optical filter is changeable, the optical filter means including a transmittance level changing means capable of changing a transmittance in accordance with the filter characteristic change.

41. The image capturing system according to claim 33, wherein the camera includes a optical block for separating light into RGB and, three image capturing elements respectively corresponding to RGB, the optical filter being provided by the optical block, the optical filter means including for each of the image capturing devices a transmittance level changing means capable of changing a darkness level of the image.

42. The image capturing system according to claim 40 or 41, wherein each of the transmittance level changing means includes two polar filters each capable of changing its angle.

43. The image capturing system according to claim 41, wherein each of the transmittance level changing means includes two polar filters each capable of changing its angle, one of the two polar filters being provided as a common filter in front of the optical block, the other of the two being provided individually per color channel behind the optical block.

44. The image capturing system according to claim 33, wherein the image capturing device is provided by a film (37), the means for measuring a complementary color including a lamp, a color-of-light detector for detecting a color of light having passed the light detecting elements, a light-source-color measuring portion, and a complementary color measuring portion based on the light-source-color measuring portion, the optical filter means including a filter for further allowing the light from the lamp through the film to a printing paper, and a filter changing unit for giving this filter the complementary color.

45. The image capturing system according to one of claims 33 through 41, claims 43 and 44, wherein the correction unit further includes an electrical correcting portion (72) for practical division by the reference signal values (rn, gn, bn) obtained for each of the color channels, of respective main signal values (r[x][y], g[x] [y], b[x] [y]) at each of corresponding locations on coordinates in the main scene (110) captured by the image capturing devices (31, 37), whereby obtaining corrected signal values (rc[x] [y] gc[x] [y], bc[x] [y]) as corrected values of the main signal value, the electrical correcting portion providing a color correction transitionally before completion of a color correction by the optical filter means.

31. An image capturing system for stabilization of intensity in an image, comprising:

a camera (2) including a lens (41), image capturing devices (31, 37), light detecting elements (31, 33, 37, 38) and a reflection surface (61) for capture of a main scene (110) in the image capturing devices (31, 37), the reflection surface (61) being disposed within a visual field of the camera (2) for reflection of light from the main scene (110) or a reference scene (121, 121a˜

e) disposed near the main scene (110) for reception by the light detecting elements (31, 33, 37, 38) via the lens (41); and

an image processing unit (7) obtaining a value from one pixel (136d) or an average value from a plurality of pixels (131, 131a˜

e), for each of the color channels as reference signal values (rn, gn, bn), out of reflected light from the reference scene (121, 121a˜

e) received by the light detecting elements (31, 33, 37, 38), for practical division by the reference signal values (rn, gn, bn) of respective main signal values (r[x] [y], g[x] [y], b[x] [y]) at each of corresponding locations on coordinates in the main scene (110) captured by the image capturing devices (31, 37), whereby obtaining corrected signal values (rc[x] [y], gc[x] [y], bc[x] [y]) as corrected values of the main signal value.