Image pickup device, integrated circuit of image pickup element, and image pickup result processing method

• US 7,936,376 B2
• Filed: 06/22/2005
• Issued: 05/03/2011
• Est. Priority Date: 07/13/2004
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Fukuhara, Takahiro
• Primary Examiner(s): Tran; Sinh
• Assistant Examiner(s): Peterson; Christopher K
• Application Number:
  US11/632,317
• Publication Number:
  US 20090002507A1
• Time in Patent Office: 2,141 Days
• Field of Search: 348/222.1
• US Class Current: 348/222.1
• CPC Class Codes: H01L 27/14609 Pixel-elements with integrated switching, control, storage or amplification elements scanning details of imagers (circuitry of solid-state image sensors H04N25/00); circuitry of imagers H04N25/70
• CPC Class Codes: H01L 27/1464 Back illuminated imager structures
• CPC Class Codes: H01L 27/14643 Photodiode arrays; MOS imagers
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules

Abstract

The present invention is applied to for example video cameras recording an image pickup result by moving picture, electronic still cameras, monitoring devices and the like. Image pickup means 3 and image compressing means 6 are connected to each other by a wiring layer formed on a surface on an opposite side from an image pickup surface of the image pickup means 3, and are thereby integrated with each other. A data compression ratio used for data compression of at least a next block is changed on the basis of an amount of code generated by data compression of an image pickup result S1 in a block unit. Also, the image pickup result S1 of each photoelectric conversion unit is output from the image pickup means 3 in such a manner as to correspond to the processing in the block unit.

Claims

1. An image pickup device characterized by comprising:

image pickup means for outputting image pickup results by XY address control, said image pickup means including a plurality of photoelectric conversion units arranged in a form of a matrix; and

a peripheral circuit retained integrally with said image pickup means, said peripheral circuit being connected to said photoelectric conversion units by a wiring layer formed on a surface on an opposite side from a light receiving surface of said image pickup means,

the image pickup results of said image pickup means being processed and output by said peripheral circuit,

wherein said peripheral circuit includes

image compressing means for data-compressing and outputting at least said image pickup results in each predetermined processing unit, and

controlling means for controlling a data compression ratio of said image compressing means,

said peripheral circuit is supplied with said image pickup results from said image pickup means in units of blocks formed by dividing an effective image area in a horizontal direction and a vertical direction, and sequentially data-compresses said image pickup results, and

within one frame, on a basis of an amount of code generated by data compression of one block by said image compressing means, said peripheral circuit changes the data compression ratio when at least a next block is data-compressed by said image compressing means, and

said image pickup means outputs the image pickup results of said photoelectric conversion units with said blocks as units in a sequence corresponding to the data compression by said image compressing means.

2. The image pickup device as claimed in claim 1, characterized by further comprising data amount correcting means for accumulating one frame of data output from said image compressing means and correcting data size.

3. The image pickup device as claimed in claim 1, characterized in that:

a data compressing process by said image compressing means is a data compressing process using a line-based wavelet transform process.

4. The image pickup device as claimed in claim 1, characterized in that:

a data compressing process by said image compressing means is a data compressing process using a tile-based wavelet transform process.

5. The image pickup device as claimed in claim 1, characterized in that:

a data compressing process by said image compressing means is a data compressing process using a discrete cosine transform process.

6. The image pickup device as claimed in claim 1, characterized in that:

said image compressing means converts said image data into coefficient data, subjects the coefficient data to a quantizing process and a coding process, and outputs a result, and

the changing of the data compression ratio by said controlling means is changing of a quantization scale in said quantizing process.

7. The image pickup device as claimed in claim 1, characterized in that:

said peripheral circuit is disposed in a layer under said wiring layer.

8. The image pickup device as claimed in claim 7, characterized in that:

said peripheral circuit is formed by a wafer producing process different from a wafer producing process of said image pickup means.

14. The image pickup device as claimed in claim 1, wherein said blocks are assigned a period for outputting the image pickup results based on the order in which the blocks were formed, and wherein said image pickup means outputs the image pickup results based on the period.

15. The image pickup device as claimed in claim 1, wherein said effective image area is divided into a predetermined number of said blocks.

16. The image pickup device as claimed in claim 1, wherein said peripheral circuit changes the data compression ratio based on a difference between a target code amount and the amount of code generated, the target code amount being obtained by an amount of code assigned to the one frame by a total number of blocks per frame.

17. The image pickup device as claimed in claim 1, wherein said blocks are rectangular.

9. An integrated circuit of an image pickup element characterized by comprising:

image pickup means for outputting image pickup results by XY address control, said image pickup means including a plurality of photoelectric conversion units arranged in a form of a matrix; and

a peripheral circuit retained integrally with said image pickup means, said peripheral circuit being connected to said photoelectric conversion units by a wiring layer formed on a surface on an opposite side from a light receiving surface of said image pickup means,

the image pickup results of said image pickup means being processed and output by said peripheral circuit,

wherein said peripheral circuit includes

image compressing means for data-compressing and outputting at least said image pickup results in each predetermined processing unit, and

controlling means for controlling a data compression ratio of said image compressing means,

said peripheral circuit is supplied with said image pickup results from said image pickup means in units of blocks formed by dividing an effective image area in a horizontal direction and a vertical direction, and sequentially data-compresses said image pickup results, and

within one frame, on a basis of an amount of code generated by data compression of one block by said image compressing means, said peripheral circuit changes the data compression ratio when at least a next block is data-compressed by said image compressing means, and

said image pickup means outputs the image pickup results of said photoelectric conversion units with said blocks as units in a sequence corresponding to the data compression by said image compressing means.

10. An image pickup device characterized by comprising:

an image pickup section configured to output image pickup results by XY address control, said image pickup section including a plurality of photoelectric conversion units arranged in a form of a matrix; and

a peripheral circuit retained integrally with said image pickup section, said peripheral circuit being connected to said photoelectric conversion units by a wiring layer formed on a surface on an opposite side from a light receiving surface of said image pickup section;

the image pickup results of said image pickup section being processed and output by said peripheral circuit,

wherein said peripheral circuit includes

an image compressing section configured to data-compress and output at least said image pickup results in each predetermined processing unit, and

a controlling section configured to control a data compression ratio of said image compressing section,

said peripheral circuit is supplied with said image pickup results from said image pickup section in units of blocks formed by dividing an effective image area in a horizontal direction and a vertical direction, and sequentially data-compresses said image pickup results, and

within one frame, on a basis of an amount of code generated by data compression of one block by said image compressing section, said peripheral circuit changes the data compression ratio when at least a next block is data-compressed by said image compressing section, and

said image pickup section outputs the image pickup results of said photoelectric conversion units with said blocks as units in a sequence corresponding to the data compression by said image compressing section.

11. An integrated circuit of an image pickup element characterized by comprising:

an image pickup section configured to output image pickup results by XY address control, said image pickup section including a plurality of photoelectric conversion units arranged in a form of a matrix; and

a peripheral circuit retained integrally with said image pickup section, said peripheral circuit being connected to said photoelectric conversion units by a wiring layer formed on a surface on an opposite side from a light receiving surface of said image pickup section;

the image pickup results of said image pickup section being processed and output by said peripheral circuit,

wherein said peripheral circuit includes

an image compressing section configured to data-compress and output at least said image pickup results in each predetermined processing unit, and

a controlling section configured to control a data compression ratio of said image compressing section,

said peripheral circuit is supplied with said image pickup results from said image pickup section in units of blocks formed by dividing an effective image area in a horizontal direction and a vertical direction, and sequentially data-compresses said image pickup results, and

within one frame, on a basis of an amount of code generated by data compression of one block by said image compressing section, said peripheral circuit changes the data compression ratio when at least a next block is data-compressed by said image compressing section, and

said image pickup section outputs the image pickup results of said photoelectric conversion units with said blocks as units in a sequence corresponding to the data compression by said image compressing section.

12. A method of processing image pickup results in an image pickup device, said image pickup device including image pickup means for outputting image pickup results by XY address control, said image pickup means including a plurality of photoelectric conversion units arranged in a form of a matrix, and a peripheral circuit retained integrally with said image pickup means, said peripheral circuit being connected to said photoelectric conversion units by a wiring layer formed on a surface on an opposite side from a light receiving surface of said image pickup means, the image pickup results of said image pickup means being processed and output by said peripheral circuit, said method characterized by comprising:

an image pickup result outputting step of outputting the image pickup results of said photoelectric conversion units from said image pickup means to said peripheral circuit;

an image compressing step of data-compressing and outputting at least said image pickup results in each predetermined processing unit by said peripheral circuit; and

a controlling step of controlling a data compression ratio of said image compressing step,

wherein in said image compressing step, said image pickup results are input from said image pickup means in units of blocks formed by dividing an effective image area in a horizontal direction and a vertical direction, and sequentially data-compressed,

in said controlling step, within one frame, on a basis of an amount of code generated by data compression of one block by said image compressing step, the data compression ratio when at least a next block is data-compressed by said image compressing step is changed, and

in said image pickup result outputting step, the image pickup results of said photoelectric conversion units are output with said blocks as units in a sequence corresponding to the data compression by said image compressing step.

13. A method of processing image pickup results in an integrated circuit of an image pickup element, said integrated circuit of the image pickup element including image pickup means for outputting image pickup results by XY address control, said image pickup means including a plurality of photoelectric conversion units arranged in a form of a matrix, and a peripheral circuit retained integrally with said image pickup means, said peripheral circuit being connected to said photoelectric conversion units by a wiring layer formed on a surface on an opposite side from a light receiving surface of said image pickup means, the image pickup results of said image pickup means being processed and output by said peripheral circuit, said method characterized by comprising:

an image pickup result outputting step of outputting the image pickup results of said photoelectric conversion units from said image pickup means to said peripheral circuit;

an image compressing step of data-compressing and outputting at least said image pickup results in each predetermined processing unit by said peripheral circuit; and

a controlling step of controlling a data compression ratio of said image compressing step,

wherein in said image compressing step, said image pickup results are input from said image pickup means in units of blocks formed by dividing an effective image area in a horizontal direction and a vertical direction, and sequentially data-compressed,

in said controlling step, within one frame, on a basis of an amount of code generated by data compression of one block by said image compressing step, the data compression ratio when at least a next block is data-compressed by said image compressing step is changed, and

in said image pickup result outputting step, the image pickup results of said photoelectric conversion units are output with said blocks as units in a sequence corresponding to the data compression by said image compressing step.