On-chip image sensor data compression

• US 10,326,904 B2
• Filed: 07/31/2018
• Issued: 06/18/2019
• Est. Priority Date: 03/30/2012
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Campbell, Scott Patrick, Newman, David A., Schunck, Brian G.
• Primary Examiner(s): Nazrul, Shahbaz
• Application Number:
  US16/050,305
• Publication Number:
  US 20180338060A1
• Time in Patent Office: 322 Days
• CPC Class Codes: H04N 1/00209 Transmitting or receiving image data, e.g. facsimile data, via a computer, e.g. using e-mail, a computer network, the internet, I-fax
• CPC Class Codes: H04N 1/2137 with temporary storage before final recording, e.g. in a frame buffer
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/426 using memory downsizing methods
• CPC Class Codes: H04N 19/625 using discrete cosine transform [DCT]
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/91 Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/40 Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled

Abstract

Systems and methods are disclosed that capture and compress frames of pixel data. In an implementation, an image sensor chip is configured to convert light into pixel data and generate compressed pixel data at a variable compression rate including applying a transform to pixel data associated with a pixel category from a plurality of pixel categories. The variable compression rate is within an available bandwidth of an output bus configured to output the compressed pixel data.

Claims

1. An image capture system, comprising:

an image sensor chip configured to;

convert light into pixel data; and

generate compressed pixel data at a variable compression rate including applying a transform to pixel data associated with a pixel category from a plurality of pixel categories, wherein the variable compression rate is within an available bandwidth of an output bus configured to output the compressed pixel data.

2. The system of claim 1, wherein the image sensor chip comprises transform logic that transforms at least some of the pixel data on a row and column basis and outputs the transformed pixel data to a compression engine configured to generate the compressed pixel data, wherein the transform logic is configured to perform a first transformation and the compression engine is operable to perform a second transformation.

3. The system of claim 1, wherein the image sensor chip is configured to convert light into pixel data by converting electrical signals representative of the light into pixel data.

4. The system of claim 1, wherein each pixel category is associated with a group of pixel data and the transform is a wavelet transform.

5. The system of claim 4, wherein the image sensor chip is configured to generate the compressed pixel data using at least one of quantization and entropy encoding.

6. The system of claim 5, wherein the image sensor chip includes a Bayer color filter and the plurality of pixel categories include red, green1, blue, and green2 pixel categories.

7. The system of claim 5, wherein the pixel data is chroma channel differenced before the transform data is generated.

8. The system of claim 5, wherein the image sensor chip includes a plurality of pixel data feeds configured to communicate respective groups of pixel data associated with respective pixel categories of the plurality of pixel categories.

9. The system of claim 8, wherein the compressed pixel data is generated according to the plurality of pixel categories using a parallel processing technique.

10. The system of claim 1, wherein the compressed pixel data is generated using predictive encoding.

11. The system of claim 10, wherein the image sensor chip includes a color filter and the pixel categories correspond to color planes.

12. The system of claim 1, wherein the pixel data is a digital image.

13. The system of claim 1, wherein the image sensor chip is configured to adjust the variable compression rate in response to at least one of a frame capture rate and an amount of the pixel data.

14. The system of claim 1, wherein the image sensor chip is configured to receive settings for at least one of a frame capture rate and a frame resolution.

15. The system of claim 1, wherein the image sensor chip is configured to adjust the variable compression rate based on a battery power consumption.

16. The system of claim 1, wherein the image sensor chip is configured to generate compressed pixel data without using motion compensation.

17. The system of claim 1, further comprising:

a digital signal processor chip comprising;

a bus coupled to an output bus of the image sensor chip and configured to receive the compressed pixel data from the image sensor chip;

hardware configured to decompress the compressed pixel data to obtain decompressed pixel data and to compress the decompressed pixel data using motion compensation.

18. The system of claim 17, wherein the digital signal processor chip is configured to convert a color mapping scheme of the decompressed pixel data before compression of the decompressed pixel data.

19. The system of claim 17, wherein the digital signal processor chip is configured to convert the decompressed pixel data from a Bayer scheme to a YUV scheme.

20. The system of claim 17, wherein the digital signal processor chip is configured to downscale the decompressed pixel data.