CMOS image sensor apparatus with on-chip real-time pipelined JPEG compression module

• US 7,532,760 B2
• Filed: 08/27/2002
• Issued: 05/12/2009
• Est. Priority Date: 08/27/2002
• Status: Active Grant

• Current Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Original Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Inventors: Lin, Xiuhong, Subbotin, Igor, Kaplinsky, Michael
• Primary Examiner(s): Hung; Yubin
• Application Number:
  US10/227,809
• Publication Number:
  US 20040042668A1
• Time in Patent Office: 2,450 Days
• Field of Search: 382/232, 382/233, 382/321, 382/324, 348/294, 365/189.01, 710/36
• US Class Current: 382/232
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof

Abstract

A CMOS imager in which a CMOS image sensor, a color image processing module and an image compression module are all provided on a single die. Both the color image processing module and the image compression module incorporate pipelined architectures to process the image data at a video rate in a massively parallel fashion.

Claims

1. A CMOS imaging device comprising:

a single integrated circuit chip comprising;

a CMOS image sensor;

a color image processing module;

an image storage module coupled to said color image processing module; and

an image compression module coupled to said image storage module, wherein;

said image storage module is configured to store color video image data from said color image processing module,

said image storage module is further configured to transfer a plurality of color components of said color video image data in parallel to said image compression module,

said color video image data is stored in locations in which color video image data was recently transferred out,

said image storage module is further configured to use a memory addressing scheme other than a ping-pong memory addressing scheme, and

wherein said image compression module has a pipelined architecture for processing image data in a parallel fashion at video rates without requiring a full frame memory buffer.

2. The CMOS imaging device of claim 1, wherein said image compression module implements JPEG compression at video rates.

3. The CMOS imaging device of claim 2, wherein said image storage module comprises a plurality of line buffers at a front end of the compression module to allow two-dimensional processing.

4. The CMOS imaging device of claim 2, wherein said JPEG image compression module implements baseline JPEG and contains one or more of a DCT module, a Zig-Zag scan module, a quantization module and an entropy encoder module.

5. The CMOS imaging device of claim 2, further comprising a buffer at the output of the image compression module to allow for periods of fixed-rate data output interspersed with periods of inactivity.

6. The CMOS imager device of claim 5, wherein the output buffer is a dual-ported memory provided with a pointer control block, such that the buffer functions as a FIFO memory.

7. The CMOS imager device of claim 6, wherein the pointer control block allows for the storage of the output of the image compression module as soon as it becomes available, while allowing independent retrieval of data based on external requests.

8. The CMOS imager device of claim 6, further comprising a FIFO &

Register &

CPU control block to enable register access and FIFO access via a single system data/memory bus.

9. The CMOS imager device of claim 8, wherein the FIFO &

Register &

CPU control block generates an INT signal reflecting the current state of the FIFO.

10. The CMOS imager device of claim 8, wherein control signals are provided to distinguish between image access, register write, and register read traffic over the output pins.

11. The CMOS imager device of claim 6, wherein the FIFO pointer control block generates fill level flags reflecting the fill level of the FIFO memory.

12. The CMOS imager device of claim 6, wherein the FIFO outputs are connected directly to a data/memory bus of a system, allowing for access to the system to be performed over standard system communication channels.

13. The CMOS imager device of claim 12, wherein access to registers and control functions of the imager system is also performed over said data/memory bus.

14. The CMOS imager device of claim 6, further comprising circuitry for multiplexing compressed and uncompressed video to an input port of the FIFO memory.

15. The CMOS imaging device of claim 1, wherein said color imaging processing module converts digitized imager data from first format which is native to a CMOS imager to a second format suitable for image compression.

16. The CMOS imaging device of claim 15 wherein said first format is RGB and said second format is YCrCb.

17. A CMOS imaging device comprising:

a single integrated circuit chip comprising;

a CMOS image sensor;

a color image processing module;

an image storage module coupled to said color image processing module; and

an image compression module coupled to said image storage module,

wherein;

said image storage module is configured to store color video image data from said color image processing module,

said image storage module is further configured to transfer a plurality of color components of said color video image data in parallel to said image compression module,

said color video image data is stored in locations in which color video image data was recently transferred out,

said image storage module is further configured to use a memory addressing scheme other than a ping-pong memory addressing scheme,

said image storage module uses an SRAM addressing scheme to allow maximum utilization rate of memory, comprising reading and writing to one SRAM at the same rate without overwriting, and

an address calculation formula is used in SRAM addressing, the address calculation formula comprising;

Address—

next=Clip (Address_current +Delta_current);

Delta_next =Delta_current * (LineSize /8)−

(LineSize−

1) *

[Delta_current * (LineSize/8)/ (LineSize−

1)];

$\begin{array}{c}\mathrm{Clip}<br><br>\left(A\right)=<br><br>A-\mathrm{LineSize}-1<br><br><br><br>\mathrm{when}<br><br><br><br>A><br><br>\left(\mathrm{LineSize}-1\right)\\ =<br><br>2*\left(\mathrm{LineSize}-1\right)<br><br><br><br>\mathrm{when}<br><br><br><br>A><br><br>2*\left(\mathrm{LineSize}-1\right),\end{array}$

where LineSize is a number of columns in the video frame, Address_next is a next address to be calculated, Address_current is a current address being calculated, Delta_next is a change in the next address calculation, and Delta_current is a change in the current address calculation.

18. A process for processing image data from a CMOS imager device, comprising the steps of:

converting an image to video image data using a CMOS image sensor;

color image processing said video image data from a first format which is native to a CMOS imager to a second format more suitable for image compression;

storing said processed video image data in a memory; and

compressing said stored video image data using a processing module having a pipelined architecture for processing image data in a parallel fashion at video rates without requiring a full frame memory buffer,

wherein;

said converting, color image processing, storing, and compressing steps are performed on a single chip,

said memory is further configured to transfer a plurality of components of said stored processed video image data in parallel to said processing module,

said processed video image data is stored in locations in which said stored processed video image data was recently transferred out,

said memory is further configured to use a memory addressing scheme other than a ping-pang memory addressing scheme.

19. The process of claim 18, wherein said step of compressing the image comprises JPEG compression at video rates.

20. The process of claim 19, wherein said step of storing said processed image data in a memory comprises using an SRAM addressing scheme to allow maximum utilization rate of memory, said SRAM addressing scheme comprising reading and writing to one SRAM at the same rate without overwriting.

21. The process of claim 18, wherein said step of color image processing comprises converting digitized imager data from first format which is native to a CMOS imager to a second format more suitable for image compression.

22. The process of claim 18, wherein said first format is RGB and said second format is YCrCb.

23. A processing system, comprising:

a processor for receiving and processing video image data; and

an image data generator for supplying video image data to the processor, the image data generator comprising;

a single chip comprising;

a CMOS image sensor;

a color image processing module;

an image storage module coupled to said color image processing module; and

an image compression module coupled to said image storage module,

wherein;

said image storage module is configured to store color video image data from said color image processing module,

said image storage module is further configured to transfer a plurality of color components of said color video image data in parallel to said image compression module,

said color video image data is stored in locations in which color image data was recently transferred out,

said image storage module is further configured to use a memory addressing scheme other than a ping-pong memory addressing scheme, and

wherein said image compression module has a pipelined architecture for processing image data in a parallel fashion at video rates without requiring a full frame memory buffer.