Image processing circuit of image input device

• US 7,116,358 B1
• Filed: 12/13/1999
• Issued: 10/03/2006
• Est. Priority Date: 02/12/1999
• Status: Expired due to Term

• Current Assignee: Megachips Corporation
• Original Assignee: Megachips Corporation
• Inventors: Sasaki, Gen
• Primary Examiner(s): Ometz, David
• Assistant Examiner(s): TRAN, NHAN T
• Application Number:
  US09/459,574
• Time in Patent Office: 2,486 Days
• Field of Search: 348/222.1, 348/231.6, 348/241, 348/251, 348/333, 358/530, 358/443, 358/448, 382/162, 382/276
• US Class Current: 348/222.1
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: H04N 1/40 Picture signal circuits H04N1/387 takes precedence
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 23/651 for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
• CPC Class Codes: H04N 25/68 applied to defects

Abstract

In image input devices such as digital still cameras, processing is speeded up and power consumption is reduced by arranging in a RPU (23) performing real time processing of a pixel data from a CCD (21), such that only special exceptional image processing not being prepared previously is subjected to a software program processing in a CPU (24) and, in post processing in which a general image processing is carried out, a pixel data temporarily stored in a main memory (29) is inputted again to the RPU (23) and then processed. This enables to sharply speed up processing, and minimize a prolonged processing in the CPU (24) to reduce power consumption, when compared to the case of executing by software problem processing.

Claims

1. An image processing circuit configured to perform predetermined image processing of pixel data included in an image photographed by an image pickup device, said circuit comprising:

a selector configured to receive said pixel data from the image pickup device and stored pixel data from a main memory;

a real time processing unit connected to an output of said selector and configured to receive said stored pixel data from the main memory, perform a general image processing of the stored pixel data by real time processing to produce processed pixel data, and output said processed pixel data to the main memory, the main memory being configured to store the processed pixel data in image frame units as said stored pixel data; and

a central control unit including an input connected to the main memory and configured to execute exceptional image processing as software program processing with respect to the stored pixel data received from said main memory,

wherein said selector is configured to select at least one of said pixel data from said image pickup device and said stored pixel data from said main memory to be provided to the output connected to the real time processing unit,

said real time processing unit includes a plurality of image processing blocks connected sequentially,

a foremost stage image processing block is configured to selectively receive the pixel data of image temporarily stored in said main memory through said selector,

at least one of a second and later image processing blocks is configured to selectively receive at least one of pixel data from said foremost stage image processing block and the pixel data of image temporarily stored in said main memory through the selector,

a rearmost output stage image processing block configured to output a rearmost processed pixel data to said main memory, and

at least one other output stage image processing block preceding said the rearmost output stage image processing block, said at least one other output stage image processing block being configured to output preceding processed pixel data to both a succeeding image processing block and said main memory.

5. The image processing circuit according to claim 1 wherein said real time processing unit further comprising:

a pixel reference block having a 3×

3 pixel registers and two line memories, in which, to an objective pixel residing at the center of said 3×

3 pixel registers, a reference is made from pixels surrounding said objective pixel; and

a color selection block performing pixel interpolation about a color component, to said objective pixel in said pixel reference block, characterized in that;

said color selection block having a function of performing pixel interpolation, (i) when each pixel residing in said pixel reference block is of a 2×

2 pixel array of four-color system, by using a first arithmetic processing in which a similar interpolation is conducted pixel by pixel, and (ii) when each pixel residing in said pixel reference block is of a pixel array of three-color system, by using a second arithmetic processing in which one component of a pixel in said three-color system is used as a fourth color pixel of a 2×

2 pixel array similar to said four-color system, so that interpolation differs between one said component and other component is conducted.

6. The image processing circuit according to claim 5 wherein,

said pixel array of three-color system is a RGB-Bayer produced by red, green and blue components;

and

said color selection block in said real time processing unit has a function of performing pixel interpolation in which, when each pixel residing in said pixel reference block is of a RGB-Bayer, said green component is used as a fourth color pixel of a 2×

2 pixel array similar to said four-color system, so that said green component is disposed diagonally, and a pixel interpolation processing of interpolating said green component to other component is performed by finding a mean value of said green components in four pixels residing in both longitudinal and transverse directions of an objective pixel being other component, alternatively, by finding a mean value of two pixels that are obtained by eliminating the minimum and maximum values of said green components of four pixels residing in both longitudinal and transverse directions of an objective pixel being other component.

7. The image processing circuit according to claim 5 wherein said real time processing unit comprises:

a signal pixel selector that selects a high-frequency component signal of green component obtained on said pixel reference block in said color selection block, and the fourth color pixel stored in said main memory;

a feature data look-up table in which data selected by said signal pixel selector is inputted as a feature data, for numerical transformation with a predetermined function, and the result is outputted; and

a multiplier in which each component of a pixel data of a predetermined pixel array is multiplied by a value outputted from said feature data look-up table.

8. The image processing circuit according to claim 5 wherein said real time processing unit comprises:

a pixel selector that selects the fourth color pixel of each pixel that is stored in said pixel reference block in said color selection block, alternatively, provided from said image pickup device, and a pixel of one component in said three-color system pixel employed as a fourth color pixel of a 2×

2 pixel array when handling said three-color system pixel array;

a feature data look-up table in which a pixel selected by said pixel selector is inputted as a feature data, for numerical transformation by a predetermined function, and the result is outputted; and

a multiplier in which each component of a pixel data of a predetermined pixel array is multiplied by a value outputted from said feature data look-up table.

9. The image processing circuit according to claim 5 wherein said real time processing unit comprises:

a color space transformation circuit on which a first array system pixel data having a predetermined color component array is transformed to a second array system pixel data having a predetermined special color component; and

a switching selector performing switching as to whether the fourth color pixel data outputted from said color selection block, or data of one component in said first array system pixel data is allowed to be inputted to said color space transformation circuit, characterized in that;

said color space transformation circuit having a function of, when said switching selector allows data to be inputted to said color space transformation circuit, transforming said data thus allowed to said second array system pixel data, as a fourth color pixel data.

11. The image processing circuit according to claim 1 wherein said real time processing unit comprises:

a color space transformation circuit on which a pixel data of a first array system having a predetermined color component array is transformed to a second array system pixel data having a predetermined special color component;

a special color look-up table in which only said predetermined special color component in said second array system pixel data transformed on said color space transformation circuit is inputted for numerical transformation with a predetermined function, and the result is outputted; and

a multiplier in which components other than said predetermined special color component in said second array system pixel data are respectively multiplied by a value outputted from said special color look-up table.

12. The image processing circuit according to claim 1 wherein,

said real time processing unit further comprises an exposure determination evaluating unit in which an image frame is divided into multiple blocks of at least 3×

3, and the integral value of a special color component of the preceding pixel in each block is obtained to output an evaluation value used in exposure determination when photographing with said image pickup device, characterized in that;

said exposure determination evaluating unit can arbitrarily change the border line between blocks in both longitudinal and transverse directions.

13. The image processing circuit according to claim 1 wherein said real time processing unit further comprises:

a spatial filter that processes a data constellation in a predetermined color space; and

a contour correction block on which only a component after frequency conversion in said data constellation in said spatial filter, is subjected to gain adjustment for contour correction processing, characterized in that;

said contour correction block has a function of outputting, as an individual data, data obtained by performing gain adjustment only to a component after frequency conversion in said data constellation in said spatial filter, and original data in said spatial filter.

14. The image processing circuit according to claim 13 wherein said real time processing unit further comprises:

a coring function block that removes noise component in image during contour correction processing in said contour correction block, by outputting, only when said high-frequency special color component has an amplitude smaller than a predetermined threshold width, said amplitude regarding as being zero, by using a predetermined linear operation function; and

a reverse gamma effect block that relatively changes an amplitude ratio of said high-frequency special color component, with respect to said predetermined threshold width in said coring function block, characterized in that;

said revere gamma effect block determines said ratio by a predetermined ratio transform function, based on a special color component of an original data provided from said spatial filter; and

said predetermined ratio transform function is set so that said amplitude ratio of said high-frequency special color component with respect to said predetermined threshold width, is gradually increased as the amount of said special color component of said original data is larger.

15. The image processing circuit according to claim 13 wherein,

said data constellation of said spatial filter in said real time processing unit is configured as a three-component data integration;

and

said real time processing unit can selectively perform a three-component output or four-color component output to said main memory and, in either case, can selectively output three components in said spatial filter and one component of a special color component after frequency conversion, with respect to at least output of one component.

16. The image processing circuit according to claim 1 wherein,

when employing, as said image pickup device, an interlace type one in which even lines and odd lines are read out as two fields at a different timing, respectively, a first field pixel data related to either said even lines or said odd lines is stored in said main memory;

and

said real time processing unit performs, when a second field pixel data related to either said even lines or said odd lines is inputted sequentially from said image pickup device, a predetermined image processing including pixel interpolation, color space transformation and contour correction processing, by reading and referring to said first field pixel data of said main memory that corresponds to said second field pixel data, in synchronization with input of said second field pixel data.

17. The image processing circuit according to claim 1 wherein,

said real time processing unit has a line memory storing a predetermined number of pixel data per line;

said main memory stores image in frame units to be provided from said image pickup device;

said real time processing unit, when the number of pixels per line, photographed by said image pickup device, is greater than the number of pixel data in said line memory, divides horizontally said image in frame units to be outputted from said main memory into multiple blocks; and

image data related to said image divided into said multiple blocks in said main memory is inputted sequentially to said line memory in said real time processing unit.

2. An image processing circuit configured to perform predetermined image processing of pixel data included in an image photographed by an image pickup device, said circuit comprising:

a selector configured to receive said pixel data from the image pickup device and stored pixel data from a main memory;

a real time processing unit connected to an output of said selector and configured to receive said stored pixel data from the main memory, perform a general image processing of the stored pixel data by real time processing to produce processed pixel data, and output said processed pixel data to the main memory, the main memory being configured to store the processed pixel data in image frame units as said stored pixel data;

a central control unit including an input connected to the main memory and configured to execute exceptional image processing as software program processing with respect to the stored pixel data received from said main memory;

a timing generator configured to regulate operation timing of said real time processing unit and said image pickup device, said timing generator comprising;

a synchronous controller configured to synchronously regulate operation timing of said real time processing unit and operation timing of said image pickup device when said selector selects the pixel data from said image pickup device; and

an asynchronous controller configured to asynchronously regulate operation timing of said real time processing unit and operation timing of said image pickup device when said selector selects the pixel data of image temporarily stored in said main memory,

wherein said selector is configured to select at least one of said pixel data from said image pickup device and said stored pixel data from said main memory to be provided to the output connected to the real time processing unit.

3. An image processing circuit of an image input device configured to perform a predetermined image processing of an image photographed by an image pickup device in said image input device, said circuit comprising:

a real time processing unit configured to sequentially input a pixel data in the image photographed by said image pickup device and configured to perform a general image processing of the pixel data by real time processing;

a main memory configured to store a pixel data output from at least said real time processing unit in image frame units;

a central control unit configured to execute exceptional image processing as a software program processing with respect to the pixel data stored in said main memory,

wherein said real time processing unit includes a selector configured to select one of said pixel data in the image photographed by said image pickup device and said pixel data stored in said main memory;

said real time processing unit including a plurality of image processing blocks connected sequentially;

a foremost stage image processing block configured to selectively receive said pixel data stored in said main memory through said selector;

at least one of a second and later image processing blocks configured to selectively receive at least one of a pixel data from said foremost stage image processing block and the pixel data stored in said main memory through a predetermined other selector;

a rearmost stage image processing block configured to send a first processed pixel data to said main memory; and

at least one of an image processing block that precedes said the rearmost stage image processing block configured to send a second processed pixel data to both the succeeding image processing block and said main memory.

4. The image processing circuit according to claim 3, wherein said pixel data is input repetitively from said main memory to said real time processing unit to circulate over and over again when said selector selects the stored pixel data from said main memory.

10. The image processing circuit according to claim 3 wherein,

said real time processing unit further comprises an auto focus evaluation block from which a high-frequency component evaluation value is outputted as an evaluation value used in an appropriate evaluation for auto focusing, said evaluation value being obtained by integration of the absolute value of a difference in a plurality of adjacent pixels having a predetermined identical component, with respect to a given region in a pixel array of image, said auto focus evaluation block comprising:

a timing selector capable of selectively changing a clearance timing between a pair of pixels which are identical in component and objects for obtaining a difference;

an arithmetic circuit for calculating the absolute value of a difference between a pair of pixels which are identical in component and spaced at a clearance timing selected by said timing selector; and

a cumulative adder in which cumulative addition of absolute values sequentially outputted from said arithmetic circuit is performed a given number of times.

18. An image processing circuit of an image input device configured to perform a predetermined image processing of an image photographed by an image pickup device in said image input device, said circuit comprising:

a real time processing unit configured to sequentially input a pixel data in the image photographed by said image pickup device and configured to perform a general image processing of the pixel data by real time processing;

a main memory configured to store a pixel data output from at least said real time processing unit in image frame units;

a central control unit configured to execute exceptional image processing as a software program processing with respect to the pixel data stored in said main memory,

wherein said real time processing unit includes a selector configured to select one of said pixel data in the image photographed by said image pickup device and said pixel data stored in said main memory;

a timing generator configured to regulate operation timing of said real time processing unit and said image pickup device, said timing generator comprising;

a synchronous controller configured to synchronously regulate operation timing of said real time processing unit and operation timing of said image pickup device when said selector selects the pixel data in the image photographed by said image pickup device; and

an asynchronous controller configured to asynchronously regulate operation timing of said real time processing unit and operation timing of said image pickup device when said selector selects the pixel data stored in said main memory.