Image recording apparatus employing a single CCD chip to record two digital optical images

• US 7,046,288 B1
• Filed: 06/25/1999
• Issued: 05/16/2006
• Est. Priority Date: 06/27/1998
• Status: Expired due to Term

• Current Assignee: Apple Inc.
• Original Assignee: University of East Anglia
• Inventors: Finlayson, Graham
• Primary Examiner(s): Groody, James J.
• Assistant Examiner(s): JONES, HEATHER RAE
• Application Number:
  US09/720,572
• Time in Patent Office: 2,517 Days
• Field of Search: 348/272, 348/218.1, 348/342, 348/239, 348/178, 250/208.1
• US Class Current: 348/342
• CPC Class Codes: H01L 27/1462 Coatings
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control

Abstract

An image recording apparatus for processing an image includes a camera (10), or other optical sensor means, for recording a first digital optical image of at least part of a scene illuminated by an illuminant light and for recording a second digital optical image of at least a part of substantially the same scene under substantially the same illuminant light. The images may be recorded on respectively different sensor arrays (12a and 12b). The light producing the first and second images undergoes different optical processing in order to produce those images, and processing means (18) in communication with the optical sensor means is provided for processing information relating to the first and second images. By relating the two images, the processing means (18) is able to provide information regarding the color of the illuminate light or other physical properties of the illuminated scene.

Claims

1. Image recording apparatus (10) for processing an image, the apparatus including:

optical sensor means (12) for recording a first digital optical image of at least a part of a scene (0) illuminated by an illuminant light and for recording a second digital optical image of at least a part of substantially the same scene under substantially the same illuminant light, the light producing the first and second images undergoing different optical processing; and

means (18) in communication with the optical sensor means for processing information relating to the first and second images, wherein the processing means relates one of the first and second images to the other of the first and second images,

and wherein the optical sensor means are provided by a single CCD chip (12) which records the first and second digital optical images, and a filter (14) is provided in front of or on a part of the CCD chip (12) such that the first or second digital optical image is recorded by that part of the chip, and the other of the digital optical images is recorded by the remainder of the chip.

2. Image recording apparatus (10) according to claim 1 wherein the processing means (18) correlates the first and second images.

3. Image recording apparatus according to claim 1 wherein the CCD chip comprises photoelectric detector pixels having a broad response centering on a particular wavelength of light.

4. Image recording apparatus according to claim 1 wherein the CCD chip is coated with a filter (14).

5. Image recording apparatus according to claim 1 wherein the optical processing means comprises an optical filter (14).

6. Image recording apparatus according to claim 5 wherein the filter (14) has characteristics such that its output is linearly related to its input.

7. Image recording apparatus according to claim 5 wherein the response of the filter (14) is a smooth function with respect to wavelength and the filter (14) has an average transmittance of more than 30%.

8. Image recording apparatus according to claim 5 wherein the filter (14) produces an output which includes relatively more light of one wavelength than of another wavelength as compared with the input.

9. Image recording apparatus according to claim 5 wherein the filter (14) is located in the image light path before the optical sensor means.

10. Image recording apparatus according to claim 1 wherein the first and second sensor means may comprise respectively different parts of the chip (12).

11. Image recording apparatus according to claim 1 wherein the first and second images comprise different parts of the image recorded by the CCD chip (12) in spatial terms or in terms of the frequencies of light recorded.

12. Image recording apparatus according to claim 1 wherein the processing means is microprocessor based, having electrical memory means.

13. Image recording apparatus according to claim 1 wherein the processing means includes means for providing information relating to the spectral characteristics of the illuminant light.

14. Image recording apparatus according to claim 13 wherein information relating to the spectral characteristics of the illuminant light is used to facilitate removal of at least some of any illuminant colour bias present in the recorded image.

15. Image recording apparatus according to claim 13 wherein the processing means includes means for facilitating the removal of at least some of any demosaicing errors and/or interreflection errors and/or shadows present in the recorded image.

16. Image recording apparatus according to claim 13 wherein the processing means includes means for providing information relating to the physics of the scene, such as the physical characteristics of the scene.

17. A method for calibrating image recording apparatus, the method including the steps of:

(a) recording a first digital optical image of at least a part of a scene illuminated by an illuminant light and recording a second digital optical image of at least a part of substantially the same scene illuminated by substantially the same illuminant light, the light producing the first and second images undergoing different optical processing; and

(b) processing information relating to the first and second images, wherein the processing step includes relating one of the first and second images to the other of the first and second images,

and wherein the method includes the carrying out of steps (a) and (b) for each of a plurality of different known illuminant lights, step (b) includes the step of processing the information relating to the first and second images to provide a transform function indicative of the relationship between the first and second images, and the method provides a set of reference transform functions, each reference transform function relating to a different known illuminant light.

18. A method for recording an image according to claim 17 wherein the first and second images are correlated.

19. A method according to claim 17 wherein different optical processing results at least partly from the filtering of light producing the first or second image.

20. A method according to claim 17 wherein the different optical processing is provided by the use of sensors responsive to respectively different frequencies of light in recording the first and second images.

21. A method according to claim 17 wherein the first and second images comprise respectively different parts of a global image of a scene.

22. A method according to claim 17 wherein the processing of the information relating to the first and second images provides an estimate of the spectral characteristics of the illuminant light.

23. A method according to claim 17 wherein the transform function is a transform matrix.

24. A method of operating image recording apparatus that has been calibrated by a method according to claim 17, the method including the steps of:

(a) recording a first digital optical image of at least a part of a scene illuminated by an unknown illuminant and recording a second digital optical image of at least a part of substantially the same scene illuminated by substantially the same illuminant, the light producing the first and second images undergoing different optical processing,

b) applying at least two reference transform functions to the first or second image recorded in step (a), and

c) determining which reference transform function better relates the first and second images recorded in step (a).

25. A method according to claim 24 wherein each reference transform function is applied to the first image to produce a transformed first image, which is subsequently compared to the second image and the reference transform function which produces a transformed first image most closely resembling the second image is selected as the best reference transform function.

26. A method according to claim 25 wherein the known illuminant light to which the best reference transform function relates is determined, to provide information relating to the spectral characteristics of the light illuminating the scene to be recorded.

27. A method according to claim 26 wherein at least some of the colour bias due to the illuminating light is removed from the image of the scene to be recorded and/or at least some demosaicing errors and/or interreflection errors and/or shadows present in the recorded image are removed.

28. A method for processing an image recorded using image recording apparatus, the method including the steps of:

(a) recording a first digital optical image of at least a part of a scene illuminated by an illuminant light and recording a second digital optical image of at least a part of substantially the same scene illuminated by substantially the same illuminant light, the light producing the first and second images undergoing different optical processing; and

(b) applying at least two reference transform functions to the first or second image, each reference transform function being indicative of the relationship between two digital optical images recorded under different respective known illuminant lights, and

(c) determining which reference transform function better relates the first and second images.

29. A method according to claim 28 wherein each reference transform function is applied to the first image to produce a transformed first image, which is subsequently compared to the second image and the reference transform function which produces a transformed first image most closely resembling the second image is selected as the best reference transform function.

30. A method according to claim 29 wherein the known illuminant light to which the best reference transform function relates is determined, to provide information relating to the spectral characteristics of the light illuminating the scene to be recorded.

31. A method according to claim 30 wherein at least some of the colour bias due to the illuminating light is removed from the image of the scene to be recorded and/or at least some demosaicing errors and/or interreflection errors and/or shadows present in the recorded image are removed.