On-chip image sensor data compression

• US 10,701,291 B2
• Filed: 05/02/2019
• Issued: 06/30/2020
• Est. Priority Date: 03/30/2012
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Campbell, Scott Patrick, Newman, David A., Schunck, Brian G.
• Primary Examiner(s): Nazrul, Shahbaz
• Application Number:
  US16/401,529
• Publication Number:
  US 20190260901A1
• Time in Patent Office: 425 Days
• CPC Class Codes: H04N 1/00209 Transmitting or receiving image data, e.g. facsimile data, via a computer, e.g. using e-mail, a computer network, the internet, I-fax
• CPC Class Codes: H04N 1/2137 with temporary storage before final recording, e.g. in a frame buffer
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/426 using memory downsizing methods
• CPC Class Codes: H04N 19/625 using discrete cosine transform [DCT]
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/91 Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/40 Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled

Abstract

Systems and methods are disclosed that capture and compress frames of pixel data. In an implementation, an image sensor chip is configured to convert light into pixel data and generate compressed pixel data at a variable compression rate including applying a transform to pixel data associated with a pixel category from a plurality of pixel categories. The variable compression rate is within an available bandwidth of an output bus configured to output the compressed pixel data.

Claims

1. A method for compressing frames of pixel data, comprising:

organizing pixel data into groups of pixel data;

outputting compressed pixel data by an image sensor system using image compression on the groups of pixel data, the compressed pixel data outputted at a rate serviceable by an output bus of the image sensor system; and

transmitting, using the output bus, the compressed pixel data to a digital signal processor for processing into a video data format.

2. The method of claim 1, wherein the image compression includes using a combination of lossless image compression and lossy image compression.

3. The method of claim 2, wherein the image compression includes at least one of entropy encoding, predictive coding, and run-length encoding.

4. The method of claim 3, wherein the outputting of compressed pixel data includes parallel processing the groups of pixel data according to one or more common pixel properties.

5. The method of claim 1, wherein the rate is adjusted to accommodate a bandwidth limited output bus.

6. The method of claim 5, further comprising:

buffering the pixel data to permit compression variability.

7. The method of claim 6, wherein the rate of generating compressed pixel data is variable based on power availability.

8. An image sensor system on a chip comprising:

a compression engine configured to;

transform code commonly categorized groups of pixel data; and

output compressed pixel data based on the transform coded commonly categorized groups of pixel data within an available bandwidth of an output bus configured to output the compressed pixel data.

9. The image sensor system on a chip of claim 8, further comprising:

a logic configured to perform a compression operation on rows or columns of pixels of the pixel data before the pixel data is transform coded.

10. The image sensor system on a chip of claim 9, wherein the compression operation includes a first wavelet transform.

11. The image sensor system on a chip of claim 10, wherein the compression engine is configured to:

receive output of the first wavelet transform; and

transform code the commonly categorized groups of pixel data using a second wavelet transform.

12. The image sensor system on a chip of claim 11, wherein the compression engine is configured to:

receive the commonly categorized groups of pixel data using distinct pixel data feeds; and

compress the commonly categorized groups of pixel data using a parallel processing technique.

13. The image sensor system on a chip of claim 8, wherein pins of the output bus of the image sensor system are coupled to pins of a digital signal processor, the digital signal processor configured to:

receive the compressed pixel data; and

process the compressed pixel data into a video format using motion compensation encoding.

14. The image sensor system on a chip of claim 8, further comprising:

a buffer operable to buffer the pixel data to compensate for performance variances of the compression engine.

15. A method for compressing frames of pixel data, comprising:

categorizing pixel data into groups of pixel data;

providing compressed pixel data by the image sensor system using a transform coding of the groups of pixel data, the compressed pixel data generated at a rate accommodating an available bandwidth of an output bus of the image sensor system; and

transmitting, using the output bus, the compressed pixel data to a digital signal processor for video data format processing.

16. The method of claim 15, wherein the transform coding includes a wavelet transform and the providing of compressed pixel data includes entropy encoding the transformed groups of pixel data.

17. The method of claim 15, wherein the transform coding includes a plurality of wavelet transforms and the providing of compressed pixel data includes lossless image compression of the transformed groups of pixel data.

18. The method of claim 15, wherein the transform coding of the groups of pixel data includes a wavelet transform performed by compression logic associated with respective rows and columns of a pixel array of the image sensor system.

19. The method of claim 18, wherein the providing of compressed pixel data includes parallel processing the groups of pixel data according to pixel categories.

20. The method of claim 19, further comprising:

storing the pixel data in a buffer to permit compression variability.