IMAGE SENSOR DATA COMPRESSION AND DSP DECOMPRESSION

• US 20160227068A1
• Filed: 04/12/2016
• Published: 08/04/2016
• Est. Priority Date: 03/30/2012
• Status: Abandoned Application

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Campbell, Scott Patrick, Newman, David A., Schunck, Brian G.
• Application Number:
  US15/096,479
• Publication Number:
  US 20160227068A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04N 1/00209 Transmitting or receiving image data, e.g. facsimile data, via a computer, e.g. using e-mail, a computer network, the internet, I-fax
• CPC Class Codes: H04N 1/2137 with temporary storage before final recording, e.g. in a frame buffer
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/426 using memory downsizing methods
• CPC Class Codes: H04N 19/625 using discrete cosine transform [DCT]
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/91 Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/40 Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled

Abstract

An image sensor compresses image data prior to transmitting the image data to a DSP. The image sensor captures light representing an image, for instance via a camera'"'"'s aperture. A focal plane array converts the captured light into pixel data. The pixel data is sorted into categories, and is compressed in parallel by a compression engine. The compressed pixel data is then sent to a DSP, which may be located off-chip. The DSP then decompresses the compressed pixel data, performs image signal processing operations on the compressed pixel data, and then compresses the processed pixel data into a digital image format. The image sensor may buffer the pixel data for one or more images to accommodate for slowdown by the compression engine. The pixel data may be sorted by row and column of a pixel array. Alternatively, the pixel data may be sorted by color from a Bayer color filter.

Claims

1. A method for capturing image data by a digital camera, the method comprising:

capturing, by an image sensor chip, image data representative of light incident upon the image sensor chip over a capture interval;

categorizing, by the image sensor chip, the captured image data into a plurality of categories;

compressing, by the image sensor chip, each category of image data to produce compressed image data; and

transmitting, by an output of the digital camera, the compressed image data to an external device communicatively coupled to the digital camera, the external device configured to;

decompress the compressed image data to produce categories of decompressed image data;

combine the categories of decompressed image data to produce combined pixel data;

compress the combined pixel data to produce a digital image; and

display the digital image.

2. The method of claim 1, wherein the captured image data is categorized based on one or more of:

pixel luminosity, pixel chrominance, and pixel entropy.

3. The method of claim 1, wherein the captured image data is categorized based on one or more of:

pixel row and pixel column.

4. The method of claim 1, wherein combining the categories of decompressed image data comprises interleaving pixel data comprising odd column pixel data and pixel data comprising even column pixel data, and interleaving pixel data comprising odd row pixel data and pixel data comprising even row pixel data.

5. The method of claim 1, wherein the decompressed image data includes a plurality of image data categories comprising red pixel data, blue pixel data, a first portion of green pixel data, and a second portion of green pixel data.

6. The method of claim 5, wherein combining the categories of decompressed image data comprises forming a combined pixel, the combined pixel comprising a pixel of the red pixel data, a pixel of the blue pixel data, a pixel of the first portion of green pixel data, and a pixel of the second portion of the green pixel data.

7. The method of claim 1, wherein the external device comprises one of a computer and a mobile device.

8. The method of claim 1, wherein the digital camera and the external device are wirelessly coupled.

9. The method of claim 1, wherein the digital camera and the external device are coupled via the Internet.

10. The method of claim 1, wherein the each category of image data is compressed based on one or more of:

an amount of output bandwidth available to the image sensor chip, an amount of power available to the image sensor chip to output image data, a quantity of image data captured by the image sensor chip, and a frame rate of image data captured by the image sensor chip.

11. A digital camera comprising:

an image sensor chip configured to;

capture image data representative of light incident upon the image sensor chip over a capture interval;

categorize the captured image data into a plurality of categories; and

compress each category of image data to produce compressed image data; and

an output configured to transmit the compressed image data to an external device communicatively coupled to the digital camera, the external device configured to;

decompress the compressed image data to produce categories of decompressed image data;

combine the categories of decompressed image data to produce combined pixel data;

compress the combined pixel data to produce a digital image; and

display the digital image.

12. The digital camera of claim 11, wherein the captured image data is categorized based on one or more of:

pixel luminosity, pixel chrominance, and pixel entropy.

13. The digital camera of claim 11, wherein the captured image data is categorized based on one or more of:

pixel row and pixel column.

14. The digital camera of claim 11, wherein combining the categories of decompressed image data comprises interleaving pixel data comprising odd column pixel data and pixel data comprising even column pixel data, and interleaving pixel data comprising odd row pixel data and pixel data comprising even row pixel data.

15. The digital camera of claim 11, wherein the decompressed image data includes a plurality of image data categories comprising red pixel data, blue pixel data, a first portion of green pixel data, and a second portion of green pixel data.

16. The digital camera of claim 15, wherein combining the categories of decompressed image data comprises forming a combined pixel, the combined pixel comprising a pixel of the red pixel data, a pixel of the blue pixel data, a pixel of the first portion of green pixel data, and a pixel of the second portion of the green pixel data.

17. The digital camera of claim 11, wherein the external device comprises one of a computer and a mobile device.

18. The digital camera of claim 11, wherein the digital camera and the external device are wirelessly coupled.

19. The digital camera of claim 11, wherein the digital camera and the external device are coupled via the Internet.

20. The digital camera of claim 11, wherein the each category of image data is compressed based on one or more of:

an amount of output bandwidth available to the image sensor chip, an amount of power available to the image sensor chip to output image data, a quantity of image data captured by the image sensor chip, and a frame rate of image data captured by the image sensor chip.