Method and apparatus for optimizing memory usage in image processing

• US 8,031,952 B2
• Filed: 04/21/2008
• Issued: 10/04/2011
• Est. Priority Date: 04/21/2008
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Broadcom Corporation (Broadcom, Inc.)
• Inventors: Vitsnudel, Ilia
• Primary Examiner(s): BALI, VIKKRAM
• Application Number:
  US12/106,729
• Publication Number:
  US 20090262208A1
• Time in Patent Office: 1,261 Days
• Field of Search: 382/164, 382/167, 382/173, 382/232, 382/233, 382/240, 382/250, 382/251, 382274-276, 382/294, 382/304, 382/305, 382/307, 348/143, 348/222.1, 348/231.99, 348/340, 348/E5.032, 348/E5.051, 348/E7.085, 348/E17.002, 375/E7.088, 375/E7.103, 375/E7.222, 345/629, 341/51, 341/107, 358/1.9, 358/504, 358/518
• US Class Current: 382/232
• CPC Class Codes: H04N 19/428 Recompression, e.g. by spatial or temporal decimation
• CPC Class Codes: H04N 19/547 Motion estimation performed in a transform domain
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/647 using significance based coding, e.g. Embedded Zerotrees of Wavelets [EZW] or Set Partitioning in Hierarchical Trees [SPIHT]
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

A method and digital camera avoiding the use of delay lines for block processing within the camera, such as compressing images, demosaicing, or the like. Processing is done for a block comprising a single newly received line, together with previously acquired information. In one preferred embodiment, the block processing is performed for the single line using previous lines as retrieved from the results of previous processing. In another embodiment, the processing results for the current block are determined by the processing results of the previous block together with the newly received line. Saving delay lines provides for lower memory requirements, and enables the production of smaller and lower cost cameras.

Claims

1. A method for processing a pixel block, comprising:

using a processor to perform the following;

receiving data related to at least one previously captured image line, the data differing from the at least one previously captured image line;

receiving at least one captured pixel, the at least one captured pixel belonging to at least one captured image line; and

processing the at least one captured pixel and the data related to the at least one previously captured image line to obtain a processed block result, wherein the processing comprises performing JPEG compression, and wherein JPEG coefficients are determined by rotating previous JPEG coefficients, subtracting a normalized value of an outdated pixel, and adding a normalized value of the at least one captured pixel.

2. The method of claim 1 wherein the pixel block comprises a rectangular pixel block.

3. The method of claim 1 further comprising deprocessing the processed block results to obtain at least one image line.

4. The method of claim 3 wherein the deprocessing comprises performing one or more of the following:

JPEG decompression;

JPEG 2000 decompression;

wavelet decompression;

MPEG decompression; and

mosaicing.

5. The method of claim 1 wherein the data related to the at least one previously captured image line is a reconstruction of the at least one previously captured image line.

6. The method of claim 1 wherein the data related to the at least one previously captured image line is an estimated reconstruction of the at least one previously captured image line.

7. The method of claim 1 wherein the data related to the at least one previously captured image line comprises a block processing result of an estimation of the at least one previously captured image line.

8. The method of claim 1 further comprising storing the block processing results.

9. The method of claim 1 further comprising correcting an error in the processed block result.

10. The method of claim 1 wherein the processing comprises performing one or more of the following:

JPEG compression;

JPEG 2000 compression;

wavelet compression;

MPEG compression; and

demosaicing.

11. The method of claim 1 wherein the method is performed in an image processing device.

12. The method of claim 1 wherein the method is performed in a digital camera.

13. A method for processing a pixel block using a reduced number of delay lines, comprising:

using a processor to perform the following;

receiving data related to previously captured at least one image line;

receiving at least one captured pixel, the at least one captured pixel belonging to a single captured image line; and

processing the at least one captured pixel and the data related to the previously captured at least one image line to obtain a processed block result, wherein the processing comprises performing JPEG compression, and wherein JPEG coefficients are determined by rotating previous JPEG coefficients, subtracting a normalized value of an outdated pixel, and adding a normalized value of the at least one captured pixel.

14. An image processing device for processing a pixel block, the device comprising:

a memory unit comprising a delay line that stores at least one image line;

a processor that executes a processing engine that processes a pixel block comprising the at least one image line, from the delay line and from data related to at least one previously captured image line, to obtain a processed block result, wherein the processing engine performs JPEG compression, and wherein JPEG coefficients are determined by rotating previous JPEG coefficients, subtracting a normalized value of an outdated pixel and adding a normalized value of the at least one captured pixel.

15. The image processing device of claim 14 wherein the pixel block comprises a rectangular pixel block.

16. The image processing device of claim 14 wherein the processor further executes a deprocessing engine that deprocesses the processed block results to obtain an estimation to the at least one previously captured image line.

17. The image processing device of claim 16 wherein the deprocessing engine performs one or more of the following:

JPEG decompression;

JPEG 2000 decompression;

wavelet decompression;

MPEG decompression; and

mosaicing.

18. The image processing device of claim 14 wherein the data related to previously captured at least one image line is a reconstruction of the at least one image line.

19. The image processing device of claim 14 wherein the data related to previously captured at least one image line is an estimated reconstruction of the at least one image line.

20. The image processing device of claim 14 wherein the data related to the previously captured at least one image line comprises a block processing result of an estimation of the at least one previously captured image line.

21. The image processing device of claim 14 further comprising a storage device that stores the block processing results.

22. The image processing device of claim 14 wherein the processor further executes error detection and correction in the at least one image line.

23. The image processing device of claim 14 wherein the processing engine performs one or more of the following:

JPEG compression;

JPEG 2000 compression;

wavelet compression;

MPEG compression; and

demosaicing.

24. A non-transitory computer readable storage medium containing a set of computer instructions the set of computer instructions comprising:

receiving data related to at least one previously captured image line, the data differing from the previously captured at least one image line;

receiving at least one captured pixel, the at least one captured pixel belonging to at least one captured image lines; and

processing the at least one captured pixel and the data related to the at least one previously captured image line to obtain a processed block result, wherein the processing comprises performing JPEG compression, and wherein JPEG coefficients are determined by rotating previous JPEG coefficients, subtracting a normalized value of an outdated pixel, and adding a normalized value of the at least one captured pixel.