QUANTUM DOT IMAGE SENSOR WITH DUMMY PIXELS USED FOR INTENSITY CALCULATIONS

• US 20120262601A1
• Filed: 03/06/2012
• Published: 10/18/2012
• Est. Priority Date: 03/08/2011
• Status: Abandoned Application

• Current Assignee: Malikie Innovations Limited (Key Patent Innovations Limited)
• Original Assignee: Blackberry Limited
• Inventors: CHOI, Yun Seok, TOWNSEND, Graham Charles
• Application Number:
  US13/413,454
• Publication Number:
  US 20120262601A1
• Time in Patent Office: Days
• US Class Current: 348/223.1
• CPC Class Codes: H04N 2209/047 using multispectral pick-up elements
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/633 by using optical black pixels

Abstract

An image sensor of a camera unit comprises raw image pixels for generating raw image color representing the exposed scene image. The sensor also comprises dummy pixels for supplemental data pertaining to characteristics of light exposing the scene image. A image sensor processor generates a processed digital image by adjusting component color values, obtained from raw image color data, by one or more adjustment factors calculated from supplemental image data. The adjustment factors are calculated according a selected mode of operation.

Claims

1. A camera unit for generating a processed digital image represented by a plurality of image pixels, the camera unit comprising:

an image sensor comprising a plurality of raw image pixels and a plurality of dummy pixels, the plurality of raw image pixels configured to generate raw color image data representing an image exposed by the image sensor, and the plurality of dummy pixels configured to generate supplemental image data representing at least one characteristic of a light source used to expose the scene image; and

an image sensor processor coupled to the image sensor to receive the raw color image data and the supplemental image data, the image sensor processor configured to generate the processed digital image by processing the raw color image data using the supplemental image data to adjust at least one image attribute of the processed digital image based on the at least one characteristic of the light source.

2. The camera unit of claim 1, wherein the image sensor processor is configured to determine a plurality of processed color component values associated with each pixel of the processed digital image by

for each image pixel of the processed digital image, determining an associated plurality of pre-processed color component values based on the raw color image data;

and

for at least one image pixel of the processed digital image, adjusting the associated plurality of pre-processed color component values based on the supplemental image data to determine the associated plurality of processed color component values.

3. The camera unit of claim 2, wherein for the at least one image pixel of the processed digital image, the image sensor processor is configured to scale each of the plurality of pre-processed color component values proportionately by a common factor determined based on the supplemental image data to adjust an effective exposure value of the processed digital image.

4. The camera unit of claim 3, wherein the supplemental image data comprises intensity values of the ambient light in each of an ultraviolet, a visible and an infrared range of light.

5. The camera unit of claim 2, wherein the image sensor processor, for the at least one image pixel of the processed digital image, is configured to scale each of the plurality of pre-processed color component values by corresponding factors determined based on the supplemental image data to adjust an effective white balance of the processed digital image.

6. The camera unit of claim 5, wherein the supplemental image data comprises intensity values of the ambient light in each of a red, a green and a blue range of visible light.

7. The camera unit of claim 6, wherein the supplemental image data further comprises intensity values of the ambient light in each of an ultraviolet and an infrared range of light.

8. The camera unit of claim 1, wherein

the image sensor is configured to produce a stream of raw color image data representing a plurality of images comprising at least a first image and a second image successively exposed by the image sensor, and a stream of supplemental image data representing the at least one characteristic of the light source for each corresponding one of the plurality of images;

and

the image sensor processor is configured to process the stream of supplemental image data to determine at least one image attribute of the first image, and to process the stream of raw color image data based on the at least one image attribute of the first image to adjust at least one image attribute of the second image.

9. The camera unit of claim 1, wherein

the image sensor is configured to produce a stream of raw color image data representing a plurality of images comprising at least a first image and a second image successively exposed by the image sensor, and a stream of supplemental image data representing the at least one characteristic of the light source for each corresponding one of the plurality of images;

and

the image sensor processor is configured to process the stream of supplemental image data to determine at least one image attribute of the first image, and to control a camera sub-unit based on the at least one image attribute of the first image to generate the raw color image data representing the second image with at least one image attribute adjusted.

10. The camera unit of claim 9, wherein the camera sub-unit is a camera sensor sub-unit.

11. A method for controlling a camera unit to generate a processed digital image represented by a plurality of image pixels, the method comprising:

receiving raw color image data representing an image exposed by an image sensor;

receiving supplemental image data representing at least one characteristic of a light source used to expose the scene image; and

processing the raw color image data in an image sensor processor of the camera unit to generate the processed digital image using the supplemental image data to adjust at least one image attribute of the processed digital image based on the at least one characteristic of the light source.

12. The method of claim 11, wherein the processing the raw color image data to generate the processed digital image comprises determining a plurality of processed color component values associated with each pixel of the processed digital image by

for each image pixel of the processed digital image, determining an associated plurality of pre-processed color component values based on the raw color image data;

and

for at least one image pixel of the processed digital image, adjusting the associated plurality of pre-processed color component values based on the supplemental image data to determine the associated plurality of processed color component values.

13. The method of claim 12, wherein for the at least one image pixel of the processed digital image, the adjusting the associated plurality of pre-processed color component values comprises scaling each of the plurality of pre-processed color component values proportionately by a common factor determined based on the supplemental image data to adjust an effective exposure value of the processed digital image.

14. The method of claim 13, wherein the supplemental image data comprises intensity values of the ambient light in each of an ultraviolet, a visible and an infrared range of light.

15. The method of claim 12, wherein for the at least one image pixel of the processed digital image, the adjusting the associated plurality of pre-processed color component values comprises scaling each of the plurality of pre-processed color component values by corresponding factors determined based on the supplemental image data to adjust an effective white balance of the processed digital image.

16. The method of claim 15, wherein the supplemental image data comprises intensity values of the ambient light in each of a red, a green and a blue range of visible light.

17. The method of claim 16, wherein the supplemental image data further comprises intensity values of the ambient light in each of an ultraviolet and an infrared range of light.

18. The method of claim 11, further comprising

receiving a stream of raw color image data representing a plurality of images comprising at least a first image and a second image successively exposed by the image sensor;

receiving a stream of supplemental image data representing the at least one characteristic of the light source for each corresponding one of the plurality of images;

processing the stream of supplemental image data to determine at least one image attribute of the first image; and

processing the stream of raw color image data based on the at least one image attribute of the first image to adjust at least one image attribute of the second image.

19. The method of claim 11, further comprising

receiving a stream of raw color image data representing a plurality of images comprising at least a first image and a second image successively exposed by the image sensor;

receiving a stream of supplemental image data representing the at least one characteristic of the light source for each corresponding one of the plurality of images;

processing the stream of supplemental image data to determine at least one image attribute of the first image; and

controlling a camera sub-unit based on the at least one image attribute of the first image to generate the raw color image data representing the second image with at least one image attribute adjusted.

20. The method of claim 19, wherein the camera sub-unit is a camera sensor sub-unit.

21. An image sensor for a camera unit comprising an image sensor processor for generating a processed digital image represented by a plurality of image pixels, the image sensor comprising:

a plurality of raw image pixels, each of the raw image pixels sensitive to light in a corresponding one of a plurality of visible light ranges to generate raw color image data representing an image exposed by the image sensor; and

a plurality of dummy pixels comprising at least one dummy pixel sensitive to light in a different light range from each of the plurality of visible light ranges to generate supplemental image data representing at least one characteristic of a light source used to expose the scene image, the supplemental image data for processing the raw color image data in the image sensor processor to adjust at least one image attribute of the processed digital image based on the at least one characteristic of the light source.

22. The image sensor of claim 21, wherein the plurality of raw image pixels and the plurality of dummy pixels are proximately situated on a substrate of the image sensor.

23. The image sensor of claim 22, wherein the plurality of raw image pixels are arranged into a pixel array on the substrate and the plurality of dummy pixels are interspersed among the plurality of the sensor image pixels in the pixel array.

24. The image sensor of claim 23, wherein the plurality of raw image pixels are supported on the substrate in a first layer and the plurality of dummy pixels are supported on the substrate in one or more additional layers.

25. The image sensor of claim 24, wherein the plurality of raw image pixels are arranged in a plurality of pixel blocks, each pixel block comprising a number of raw image pixels and spaced from adjacent pixel blocks in the pixel array forming a repeating pixel pattern, and wherein each of the plurality of dummy pixels overlaps a corresponding one of the plurality of pixel blocks.

26. The image sensor of claim 25, wherein each of the plurality of dummy pixels overlaps the corresponding one of the plurality of pixel blocks at a center of the corresponding one of the plurality of pixel blocks.

27. The image sensor of claim 26, wherein each pixel block comprises four raw image pixels arranged into a 2×

2 grid, and each of the plurality of dummy pixels partially overlaps each of the four raw image pixels at a common vertex of the four raw image pixels.

28. The image sensor of claim 23, wherein the plurality of raw image pixels and the plurality of dummy pixels are supported on the substrate in a common layer.

29. The image sensor of claim 28, wherein the plurality of raw image pixels and the plurality of dummy pixels are jointly arranged into a plurality of pixel blocks, each pixel block comprising one dummy pixel and a plurality of raw image pixels and spaced from adjacent pixel blocks in the pixel array forming a repeating pixel pattern.

30. The image sensor of claim 21, wherein each of the plurality of dummy pixels comprises a photosensitive quantum dot layer.

31. The image sensor of claim 30, wherein each of the plurality of raw image pixels comprises a photosensitive quantum dot layer.

32. The image sensor of claim 21, wherein the plurality of visible light ranges corresponds to a plurality of primary color components used to represent colors in the processed digital image.

33. The image sensor of claim 32, wherein the plurality of primary color components comprises a red component, a blue component and a green component.

34. The image sensor of claim 31, wherein the plurality of dummy pixels comprises at least one dummy pixel sensitive to light in substantially all of a visible light spectrum.

35. The image sensor of claim 31, wherein the plurality of dummy pixels comprises at least one dummy pixel sensitive to light in one of a plurality of visible light ranges.

36. The image sensor of claim 34, wherein the plurality of dummy pixels further comprises at least one dummy pixel sensitive to light in an ultraviolet light range.

37. The image sensor of claim 35, wherein the plurality of dummy pixels further comprises at least one dummy pixel sensitive to light in an ultraviolet light range.