Image processing system for high performance digital imaging devices

• US 7,259,783 B2
• Filed: 02/03/2003
• Issued: 08/21/2007
• Est. Priority Date: 05/19/1998
• Status: Expired due to Fees

• Current Assignee: FlashPoint Technology, Inc.
• Original Assignee: FlashPoint Technology, Inc.
• Inventors: Anderson, Eric C.
• Primary Examiner(s): Ometz, David
• Assistant Examiner(s): Nguyen, Luong T.
• Application Number:
  US10/357,719
• Publication Number:
  US 20030117513A1
• Time in Patent Office: 1,660 Days
• Field of Search: 348/333.11, 348/333.12, 348/333.05, 348/333.06, 348/231.6, 348/231.9
• US Class Current: 348/222.1
• CPC Class Codes: H04N 1/0044 for image preview or review, e.g. to help the user position a sheet
• CPC Class Codes: H04N 1/2112 using still video cameras
• CPC Class Codes: H04N 1/2141 in a multi-frame buffer
• CPC Class Codes: H04N 2201/325 Modified version of the image, e.g. part of the image, image reduced in size or resolution, thumbnail or screennail

Abstract

An image processing system for high performance digital imaging in a digital camera. The reflected light from an image is focused through a lens and optically filtered. A CCD array converts this image into an electrical signal. This electrical signal is processed and then converted into an equivalent digital signal. A digital signal processor is then used to process the raw digital signal. The DSP includes a capture data path, a data flow control, an image processing data path, a compression/decompression engine, a resize circuit, a display processing circuit, and a rotation circuit. Data is routed between the DSP and memory via a bus. By selectively activating and reusing certain parts of the hardware architecture and various data paths, at least four modes of operation can be supported: live view, instant review, and play mode. Furthermore, the correct image is automatically displayed in all four modes, regardless of the orientation of the image or the physical orientation of the camera (both at the time the picture was taken and at the time the picture is being rendered for display).

Claims

1. A method for processing an image in an image capture unit including a display, the method comprising the steps of:

capturing first raw image data;

processing the first raw image data through a capture data path;

storing the first raw image data in a first buffer, wherein the first buffer contains second raw image data for one or more images;

accessing a particular raw image data from the first buffer and performing image processing on the particular raw image data to produce a rotated processed image;

loading the rotated processed image in a second buffer from an image processing data path;

compressing the rotated processed image to generate a compressed image in a third buffer;

unrotating the compressed image;

performing display processing on the unrotated compressed image to produce graphics for display;

storing the graphics and the unrotated compressed image in a fourth buffer; and

rendering the graphics and the unrotated compressed image from the fourth buffer for display on the image capture unit.

2. The method of claim 1 further comprising the steps of:

transferring the first raw image data from an image sensor to the first buffer via a bus;

transferring the first raw image data from the first buffer to a data flow control via the bus;

transferring the rotated processed image from the image processing data path to the second buffer through a first dedicated path;

establishing a data path between the second buffer and a compression/decompression engine;

transferring the compressed image between the compression/decompression engine and the third buffer via the bus;

transferring the rotated processed image from the second buffer to a resize circuit through a second dedicated path;

transferring the graphics and the unrotated compressed image to the fourth buffer via the bus;

loading the graphics and the unrotated compressed image from the fourth buffer to a rotate circuit through the bus; and

establishing a third dedicated path from the rotate circuit to the second buffer.

3. The method of claim 1 further comprising the step of performing a second pass when the image capture unit is rotated, the second pass comprising the steps of:

accessing the unrotated compressed image from the fourth buffer;

rotating the unrotated compressed image;

storing the unrotated compressed image in the second buffer;

passing the unrotated compressed image through a resize circuit;

adding the graphics to the unrotated compressed image during the second pass;

storing the unrotated compressed image in a fifth buffer.

4. The method of claim 1, wherein the capture data path comprises statistic generation, pixel averaging, and compression.

5. The method of claim 1, wherein the image processing data path comprises edge enhancement, interpolation, ring insertion, pixel shuffle, line averaging, and decompression.

6. The method of claim 1 further comprising the steps of:

resizing the image from an NTSC to a PAL format; and

generating a video signal for output from the image capture unit.

7. The method of claim 1, wherein the graphics include padding and overlay bars.

8. In a digital camera, a method for digital image processing, comprising the steps of:

generating a live view image for display with the digital camera in a horizontal, vertical, or upside-down orientation;

capturing an image for subsequent display, wherein the image is captured with the digital camera in either the horizontal or vertical orientation within a first buffer;

rendering a captured image for instant review with the digital camera in either the horizontal or vertical orientation and with the captured image rotated or not rotated within a second buffer;

rendering the captured image for a play mode of operation with the digital camera in either the horizontal or vertical orientation and with the captured image rotated or not rotated within a third buffer.

9. The method of claim 8 further comprising the steps of:

rotating the image when the digital camera is in the vertical orientation to produce a horizontal image;

storing the horizontal image within a fourth buffer; and

rotating the horizontal image from the fourth buffer for displaying the live view image.

10. The method of claim 9 further comprising the steps of:

resizing the horizontal image for instant review or play mode if the digital camera is in the horizontal orientation;

displaying the resized horizontal image with a pre-determined background on either side of the resized horizontal image.

11. The method of claim 10 further comprising the steps of:

displaying an indication that the horizontal image was taken when the digital camera was in the vertical orientation;

rotating the horizontal image in a second pass if the digital camera is physically rotated to the vertical orientation.

12. A digital signal processor for use in an image capture apparatus, comprising:

a bus;

a capture data path coupled to the bus for accepting CCD data and outputting raw lines to the bus;

a first buffer coupled to the bus for storing the raw lines;

a data flow controller coupled to the bus for generating raw minimum computational units (MCUs) based on the raw lines stored in the first buffer;

an image processing data path for processing the raw MCUs from the data flow controller to create processed MCUs;

a second buffer for storing the processed MCUs from the image processing data path;

a JPEG engine for compressing the processed MCUs before storage in a third buffer and for decompressing the processed MCUs read from the third buffer;

a resize circuit coupled to a fourth buffer for resizing image data;

a display processor coupled to the resize circuit for generating graphical information for display with the image data; and

a rotation circuit coupled to the bus for rotating the image data and storing a rotated image data in a fifth buffer.

13. The digital signal processor of claim 12 further comprising a bus interface unit for coupling the capture data path, the data flow controller, the JPEG engine, the image processing data path, and the rotation circuit to the first, second, third, fourth, and fifth buffers through the bus.

14. The digital signal processor of claim 12 further comprising a direct path between an output from the rotation circuit and an input to the resize circuit.

15. The digital signal processor of claim 12 further comprising a path between the resize circuit and the display processor.

16. The digital signal processor of claim 12, wherein the capture data path includes a statistics circuit, a pixel average circuit, and a compression circuit.

17. The digital signal processor of claim 12, wherein the image processing data path includes an edge enhancement circuit, an interpolation circuit, a ring insertion circuit, a pixel shuffle circuit, a line averaging circuit, a decompression circuit, a gamma circuit, and a color space converter.

18. The digital signal processor of claim 12, wherein at least four modes of operation are supported:

live view, image capture, instant review, and play mode.

19. The digital signal processor of claim 18, wherein the an image is automatically rotated in a second pass in accordance with a current physical orientation of the image capture apparatus and with physical orientation of the image capture apparatus when the image was captured.

20. The digital signal processor of claim 18, wherein in the live view mode of operation, the image from the capture data path is stored in the first buffer, retrieved and processed by the image processing data path, buffered within the second buffer, resized if necessary, and displayed along with any appropriate graphics.

21. The digital signal processor of claim 20, wherein if the image capture apparatus is in a vertical orientation, an image is rotated by the rotation circuit before being stored in the fifth buffer and the image is rotated again before being rendered for display.

22. The digital signal processor of claim 18, wherein in the image capture mode of operation, an image from the capture data path is stored in the first buffer of a memory, retrieved and processed by the image processing data path, buffered within the second buffer, compressed by the JPEG engine, and stored in a file within the memory.

23. The digital signal processor of claim 22, wherein if the image capture apparatus is currently in a horizontal orientation and the image was taken while the image capture apparatus was in a vertical orientation, the image is reduced in size before being rendered for display.

24. The digital signal processor of claim 23, wherein the display processor generates an indication to a user to change an orientation of the image capture apparatus and when the image capture apparatus is rotated to be in the vertical orientation, the image is rendered for display without resizing.

25. The image capture apparatus of claim 18, wherein in the instant review mode of operation, an image from the capture data path is stored in the first buffer, retrieved and processed by the image processing data path, buffered within the second buffer, compressed by the JPEG engine, stored in a file within a memory, read back from the file, decompressed, resized if necessary, and displayed along with any appropriate graphics.

26. The image capture apparatus of claim 18 further comprising an orientation detector for detecting a current physical orientation of the image capture apparatus, wherein image from the fourth buffer is rotated by the rotation circuit, depending on the current physical orientation, and stored in the fifth buffer.

27. The image capture apparatus of claim 18, wherein in the playback mode of operation, one or more requested files are read, decompressed, resized if necessary, and displayed along with any appropriate graphics.

28. The image capture apparatus of claim 18 further comprising an orientation detector for detecting a current physical orientation of the image capture apparatus, wherein an image from the fourth buffer is rotated by the rotation circuit, depending on the current physical orientation, stored in the fifth buffer, and displayed in a correct orientation.

29. The image capture apparatus of claim 18 further comprising a double buffered register, wherein transfers of data between the first, second, third, fourth, and fifth buffers is activated at frame boundaries.

30. The image capture apparatus of claim 18 further comprising a YCC shuffle block having a YCC shuffle data path coupled directly to the fourth buffer.