Multi-color image processing apparatus and signal processing apparatus

• US 7,825,968 B2
• Filed: 08/29/2008
• Issued: 11/02/2010
• Est. Priority Date: 09/07/2007
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Nobori, Kunio, Kanamori, Katsuhiro, Imagawa, Taro, Azuma, Takeo
• Primary Examiner(s): Misleh; Justin P
• Application Number:
  US12/517,391
• Publication Number:
  US 20100013948A1
• Time in Patent Office: 795 Days
• Field of Search: None
• US Class Current: 348/262
• CPC Class Codes: H04N 23/13 with multiple sensors
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/84 for processing colour signals
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

A multi-color image processor according to the present invention includes an image capturing section 101 and a signal processing section 104. The image capturing section 101 includes a color separating section 10 for separating visible radiation into at least two light rays with first- and second-color components, respectively, and first and second imagers 12 and 14 that receive the light rays with the first- and second-color components. The image capturing section 101 gets images with the first- and second-color components by making the first imager 12 decimate pixels to read, but making the second imager 14 read every pixel, on a field-by-field basis on respective arrangements of pixels of the first and second imagers 12 and 14. The signal processing section 104 includes: a motion estimating section 107 for estimating, based on the images with the second-color components, a motion in a moving picture made up of images with the second-color components and outputting motion information; an image generating section 108 for generating a synthetic image of the first-color components of respective fields based on the images with the first-color components and the motion information and outputting it as a first synthetic image with a higher spatial resolution than the images with the first-color components; and a multi-color image synthesizing section 109 for generating and outputting a multi-color moving picture with the first- and second-color components based on the first synthetic image and the images with the second-color components.

Claims

1. A multi-color image processor comprising:

an image capturing section including a color separating section for separating visible radiation into a light ray with a first-color component and a light ray with a second-color component, which is different from the first-color component, and first and second imagers that receive the light ray with the first-color component and the light ray with the second-color component, respectively, wherein the image capturing section gets multiple images with the first-color components by making the first imager decimate pixels to read on a field-by-field basis on an arrangement of pixels of the first imager and also gets multiple images with the second-color components using the second imager on a field-by-field basis on an arrangement of pixels of the second imager;

a motion estimating section for estimating, based on the images with the second-color components, a motion in a moving picture made up of those images with the second-color components and outputting motion information;

an image generating section for generating a synthetic image of the first-color components of respective fields based on the images with the first-color components and the motion information and outputting the image thus generated as a first synthetic image that has a higher spatial resolution than the images with the first-color components; and

a multi-color image synthesizing section for generating and outputting a picture with the first- and second-color components based on the first synthetic image and the images with the second-color components.

2. The multi-color image processor of claim 1, wherein the image capturing section gets the images with the first-color components, of which one frame is comprised of n fields (where n is an integer that is equal to or greater than two), by changing the positions of pixels to read on the first imager one field after another.

3. The multi-color image processor of claim 2, wherein in accordance with the motion information, the image generating section defines the spatial resolution of the first synthetic image to be n times as high as that of the images with the first-color components.

4. The multi-color image processor of claim 3, wherein the spatial resolution of the first synthetic image is higher than that of the images with the second-color components.

5. The multi-color image processor of claim 4, wherein the image capturing section gets each said image with the second-color component by using at least one pixel, of which the size is bigger than each pixel of the first imager, as a minimum unit of pixels to read on the arrangement of pixels of the second imager.

6. The multi-color image processor of claim 3, wherein the images with the second-color components have a different spatial arrangement of pixels from the images with the third-color components.

7. The multi-color image processor of claim 1, comprising a resolution increasing section for generating and outputting a second synthetic image based on the first synthetic image and the images with the second-color components by increasing the resolution of the images with the second-color components by reference to information about the first synthetic image,

wherein the multi-color image synthesizing section generates and outputs a picture, including the first- and second-color components, based on the first and second synthetic images.

8. The multi-color image processor of claim 7, wherein the spatial resolution of the second synthetic image is as high as that of the first synthetic image.

9. The multi-color image processor of claim 1, wherein the pixels to read selected by the image capturing section on a field-by-field basis on the arrangement of pixels of the first imager have a first pattern in which multiple pixels, selected from the arrangement of pixels, are arranged in line.

10. The multi-color image processor of claim 1, wherein the pixels to read selected by the image capturing section on a field-by-field basis on the arrangement of pixels of the first imager have a second pattern in which multiple pixels, selected from the arrangement of pixels, are distributed discretely.

11. The multi-color image processor of claim 1, wherein the pixels to read selected by the image capturing section on a field-by-field basis on the arrangement of pixels of the first imager have their arrangement patterns changed between a first pattern in which multiple pixels, selected from the arrangement of pixels, are arranged in line and a second pattern in which multiple pixels, selected from the arrangement of pixels, are distributed discretely.

12. The multi-color image processor of claim 1, wherein the color separating section is designed to separate the visible radiation into not only the respective light rays with the first- and second-color components but also a light ray with a third-color component that is different from the first- and second-color components, and

wherein the image capturing section further includes a third imager that receives the light ray with the third-color component and gets multiple images with the third-color components on a field-by-field basis on the arrangement of pixels of the third imager, and

wherein the motion estimating section estimates, based on the images with the second-color components and third-color components, a motion in a moving picture made up of the images with the second-color components and third-color components and outputs the motion information, and

wherein the multi-color image synthesizing section generates and outputs a picture with the first-, second- and third-color components based on not only the first synthetic image and the images with the second-color components but also the images with the third-color components.

13. The multi-color image processor of claim 1, wherein the first-color component is green.

14. The multi-color image processor of claim 1, wherein the image capturing section includes a spatial light modulator that is arranged in front of the first imager, and

wherein the spatial light modulator has multiple pixels, which face the respective pixels of the first imager and which selectively transmit or cut off incoming light, thereby making the incoming light incident on selected ones of the pixels of the first imager.

15. A signal processor for outputting a picture based on multiple images with first-color components, of which one frame is comprised of n fields (where n is an integer that is equal to or greater than two), and multiple images, which have second-color components different from the first-color components and of which one frame consists of one field, the processor comprising:

a motion estimating section for outputting, based on the images with the second-color components, motion information about a moving picture made up of those images with the second-color components;

an image generating section for generating a synthetic image of the first-color components of respective fields based on the images with the first-color components and the motion information and outputting the image thus generated as a first synthetic image that has a higher spatial resolution than the images with the first-color components; and

a multi-color image synthesizing section for generating and outputting a picture with the first- and second-color components based on the first synthetic image and the images with the second-color components.

16. A multi-color image processing method for outputting a picture based on multiple images with first-color components, of which one frame is comprised of n fields (where n is an integer that is equal to or greater than two), and multiple images, which have second-color components different from the first-color components and of which one frame consists of one field, the method comprising the steps of:

obtaining images with the first-color components;

obtaining images with the second-color components;

estimating, based on the images with the second-color components, a motion in a moving picture made up of those images with the second-color components and outputting motion information;

getting a motion compensation done by outputting a synthetic image of the first-color components based on the images with the first-color components and the motion information at a higher frame rate than a frame rate at which the images with the first-color components are obtained; and

generating and outputting a picture with the first- and second-color components based on the synthetic image of the first-color components and the images with the second-color components.

17. A program embodied on a non-transitory machine readable storage medium and defined to operate a signal processor that outputs a picture based on multiple images with first-color components, of which one frame is comprised of n fields (where n is an integer that is equal to or greater than two), and multiple images, which have second-color components and of which one frame consists of one field, the program, when executed by the signal processor, causing the signal processor to perform the steps of:

outputting, based on the images with the second-color components, motion information about a moving picture made up of those images with the second-color components;

generating a synthetic image of the first-color components of respective fields based on the images with the first-color components and the motion information and outputting the image thus generated as a first synthetic image that has a higher spatial resolution than the images with the first-color components; and

generating and outputting a picture with the first- and second-color components based on the first synthetic image and the images with the second-color components.