Image stabilization using multi-exposure pattern

• US 7,825,969 B2
• Filed: 12/15/2006
• Issued: 11/02/2010
• Est. Priority Date: 12/15/2006
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Tico, Marius, Trimeche, Mejdi
• Primary Examiner(s): Ometz; David L
• Assistant Examiner(s): Trehan; Akshay
• Application Number:
  US11/640,529
• Publication Number:
  US 20080143841A1
• Time in Patent Office: 1,418 Days
• Field of Search: 348/221.1, 348/229.1, 348/262, 348272-283, 358/512
• US Class Current: 348/273
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/135 based on four or more different wavelength filter elements
• CPC Class Codes: H04N 25/44 by partially reading an SSIS array
• CPC Class Codes: H04N 25/447 by preserving the colour pattern with or without loss of information
• CPC Class Codes: H04N 25/583 with different integration times
• CPC Class Codes: H04N 25/76 Addressed sensors, e.g. MOS or CMOS sensors

Abstract

The specification and drawings present a new method, apparatus and software product for image stabilization of an image taken with a fixed (i.e., pre-selected) multi-exposure pattern for at least one color channel by an image sensor of a camera, wherein a plurality of groups of pixels of the image sensor have different pre-selected exposure times for said at least one color channel. The camera can be a part of, e.g., an electronic device such as mobile phone or a portable electronic device.

Claims

1. A method comprising:

capturing an image with a pre-selected multi-exposure pixel pattern by an image sensor of a camera for at least one color channel, wherein a plurality of groups of pixels of the image sensor have different pre-selected exposure times for said at least one color channel; and

determining a value of each pixel of said image for said at least one color channel using a weighted combination of a value of said each pixel and values of one or more of neighboring pixels of said each pixel, wherein at least one of said one or more neighboring pixels has a different exposure time than said each pixel using a predetermined algorithm;

wherein said determining comprises;

calculating an interpolating value of said each pixel of said image using said value of the pixel and said values of one or more neighboring pixels;

calculating a weight coefficient for said each pixel using said value, said interpolating value and different exposure times for said each pixel and for said at least one of said one or more neighboring pixels; and

calculating a final value for said each pixel using said interpolating value and said weight coefficient.

2. The method of claim 1, wherein said plurality of the groups of said pixels comprises two groups of said pixels with two different exposure times, wherein each pixel of one of the two groups is surrounded by four neighboring pixels of another of said two groups.

3. The method of claim 1, wherein said plurality of the groups of said pixels comprises two groups of said pixels.

4. The method of claim 3, wherein said two groups of said pixels are one of:

a) alternating rows of said pixels with two different exposure times, and b) alternating columns of said pixels with two different exposure times.

5. The method of claim 1, wherein said plurality of the groups comprises three groups of the pixels with three different exposure times.

6. The method of claim 1, wherein said multi-exposure pixel pattern is defined separately for each color used for an image formation in said electronic device.

7. The method of claim 1, wherein said camera is a part of the electronic device.

8. The method of claim 1, wherein said image sensor is a complimentary metal oxide semiconductor image sensor.

9. A computer program product comprising:

a computer readable storage structure embodying computer program code thereon for execution by a computer processor with said computer program code, wherein said computer program code comprises instructions for performing the method of claim 1, indicated as being performed by any component or a combination of components of said camera or of an electronic device comprising said camera.

10. An apparatus, comprising:

an image sensor, for capturing an image with a pre-selected multi-exposure pixel pattern for at least one color channel, wherein a plurality of groups of pixels of the image sensor have different pre-selected exposure times for said at least one color channel; and

a multiple exposure combining module, for determining a value of each pixel of said image for said at least one color channel using a weighted combination of a value of said each pixel and values of one or more of neighboring pixels of said each pixel, wherein at least one of said one or more neighboring pixels has a different exposure time than said each pixel using a predetermined algorithm;

wherein said multiple exposure combining module is configured to perform said determining which comprises of;

calculating an interpolating value of said each pixel of said image using said value of the pixel and said values of one or more neighboring pixels;

calculating a weight coefficient for said each pixel using said value, said interpolating value and different exposure times for said each pixel and for said at least one of said one or more neighboring pixels; and

calculating a final value for said each pixel using said interpolating value and said weight coefficient.

11. The apparatus of claim 10, wherein said plurality of the groups of said pixels comprises two groups of said pixels with two different exposure times, wherein each pixel of one of the two groups is surrounded by four neighboring pixels of another of said two groups.

12. The apparatus of claim 10, wherein said plurality of the groups of said pixels comprises two groups of said pixels.

13. The apparatus of claim 12, wherein said two groups of said pixels are one of:

a) alternating rows of said pixels with two different exposure times, and b) alternating columns of said pixels with two different exposure times.

14. The apparatus of claim 10, wherein said plurality of the groups comprises three groups of the pixels with three different exposure times.

15. The apparatus of claim 10, wherein said multi-exposure pixel pattern is defined separately for each color used for an image formation in said electronic device.

16. The apparatus of claim 10, wherein said image sensor is a complimentary metal oxide semiconductor image sensor.

17. The apparatus of claim 10, wherein an integrated circuit comprises all or selected modules of said apparatus.

18. An electronic device, comprising:

an image sensor for capturing an image with a pre-selected multi-exposure pixel pattern for at least one color channel, wherein a plurality of groups of pixels of the image sensor have different pre-selected exposure times for said at least one color channel; and

a multiple exposure combiner, for determining a value of each pixel of said image for said at least one color channel using a weighted combination of a value of said each pixel and values of one or more of neighboring pixels of said each pixel, wherein at least one of said one or more neighboring pixels has a different exposure time than said each pixel using a predetermined algorithm;

wherein said determining comprises;

calculating an interpolating value of said each pixel of said image using said value of the pixel and said values of one or more neighboring pixels;

calculating a weight coefficient for said each pixel using said value, said interpolating value and different exposure times for said each pixel and for said at least one of said one or more neighboring pixels;

and calculating a final value for said each pixel using said interpolating value and said weight coefficient.

19. The electronic device of claim 18, wherein said electronic device comprises a camera and said image sensor, said multiple exposure combiner are parts of said camera.

20. An apparatus, comprising:

sensing means, for capturing an image with a pre-selected multi-exposure pixel pattern for at least one color channel, wherein a plurality of groups of pixels of the sensing means have different pre-selected exposure times for said at least one color channel; and

combining means, for determining a value of each pixel of said image for said at least one color channel using a weighted combination of a value of said each pixel and values of one or more of neighboring pixels of said each pixel, wherein at least one of said one or more neighboring pixels has a different exposure time than said each pixel using a predetermined algorithm, wherein said determining comprises;

calculating an interpolating value of said each pixel of said image using said value of the pixel and said values of one or more neighboring pixels;

calculating a weight coefficient for said each pixel using said value, said interpolating value and different exposure times for said each pixel and for said at least one of said one or more neighboring pixels;

and calculating a final value for said each pixel using said interpolating value and said weight coefficient.

21. The apparatus of claim 20, wherein said sensing means is an image sensor and said combining means is a multiple exposure combining module.