Image processing system for high performance digital imaging devices

• US 6,563,535 B1
• Filed: 05/19/1998
• Issued: 05/13/2003
• Est. Priority Date: 05/19/1998
• Status: Expired due to Term

• Current Assignee: FlashPoint Technology, Inc.
• Original Assignee: FlashPoint Technology, Inc.
• Inventors: Anderson, Eric C.
• Primary Examiner(s): Garber, Wendy R.
• Assistant Examiner(s): Tillery, Rashawn N
• Application Number:
  US09/081,694
• Time in Patent Office: 1,820 Days
• Field of Search: 348/232, 348/333.06, 348/333.01, 348/231, 348/222, 348/376, 348/207, 348/231.2, 348/231.3, 348/231.99, 348/222.1, 348/207.99, 358/909.1, 358/906
• US Class Current: 348/231.2
• CPC Class Codes: H04N 1/0044 for image preview or review, e.g. to help the user position a sheet
• CPC Class Codes: H04N 1/2112 using still video cameras
• CPC Class Codes: H04N 1/2141 in a multi-frame buffer
• CPC Class Codes: H04N 2201/325 Modified version of the image, e.g. part of the image, image reduced in size or resolution, thumbnail or screennail

Abstract

An image processing system for high performance digital imaging in a digital camera. The reflected light from an image is focused through a lens and optically filtered. A CCD array converts this image into an electrical signal. This electrical signal is processed and then converted into an equivalent digital signal. A digital signal processor is then used to process the raw digital signal. The DSP includes a capture data path, a data flow control, an image processing data path, a compression/decompression engine, a resize circuit, a display processing circuit, and a rotation circuit. Data is routed between the DSP and memory via a bus. By selectively activating and reusing certain parts of the hardware architecture and various data paths, at least four modes of operation can be supported: live view, instant review, and play mode. Furthermore, the correct image is automatically displayed in all four modes, regardless of the orientation of the image or the physical orientation of the camera (both at the time the picture was taken and at the time the picture is being rendered for display).

Claims

1. A method for processing an image in an image capture unit including a display, the method comprising the steps of:

capturing raw image data;

processing the raw image data through a capture data path, wherein the capture data path comprises statistic generation, pixel averaging, and compression;

storing the raw image data in a first memory, wherein the first memory contains raw image data for one or more images;

accessing a particular raw image data from the first memory and performing image processing on the particular raw image data to produce a rotated processed image;

loading the processed image in a second buffer;

compressing the processed image to generate a compressed image;

unrotating the image;

performing display processing to produce graphics for display;

storing the graphics and image in a fourth buffer; and

rendering the graphics and image from the fourth buffer for display on the image capture unit.

2. A method for processing an image in an image capture unit including a display, the method comprising the steps of:

capturing raw image data;

processing the raw image data through a capture data path;

storing the raw image data in a first memory, wherein the first memory contains raw image data for one or more images;

accessing a particular raw image data from the first memory and performing image processing on the particular raw image data through an image processing data path to produce a rotated processed image, wherein the image processing data path comprises edge enhancement, interpolation, ring insertion, pixel shuffle, line averaging, and decompression;

loading the processed image in a second buffer;

compressing the processed image to generate a compressed image;

unrotating the image;

performing display processing to produce graphics for display;

storing the graphics and image in a fourth buffer; and

rendering the graphics and image from the fourth buffer for display on the image capture unit.

3. In a digital camera, a method for digital image processing, comprising the steps of:

generating a live view image for display with the digital camera in a horizontal, vertical, or upside-down orientation;

capturing an image for subsequent display, wherein the image is captured with the digital camera in either a horizontal or vertical orientation;

rendering a captured image for instant review with the digital camera in either the horizontal or vertical orientation and with the captured image rotated or not rotated;

rendering the captured image for a play mode of operation with the digital camera in either the horizontal or vertical orientation and with the captured image rotated or not rotated;

rotating the image when the digital camera is in the vertical orientation to produce a horizontal image;

storing the horizontal image in memory;

rotating the horizontal image from memory for displaying the live view;

resizing the horizontal image for instant review or play mode if the digital camera is in a horizontal orientation;

displaying a resized horizontal image with a pre-determined background on either side of the resized horizontal image;

displaying an indication that the horizontal image was taken when the digital camera was in the vertical orientation;

rotating the horizontal image in a second pass if the digital camera is physically rotated to the vertical orientation.

4. A digital signal processor for use in an image capture apparatus, comprising:

a bus

a capture data path coupled to the bus for accepting CCD data and outputting raw lines to the bus, wherein the capture data path includes a statistics circuit, a pixel average circuit, and a compression circuit;

a memory coupled to the bus for storing the raw lines;

a data flow controller coupled to the bus for generating raw minimum computational units (MCUs) based on the raw lines stored in the memory;

an image processing data path for processing the raw MCUs from the data flow controller;

a buffer for storing the MCU'"'"'s from the image processing data path;

a JPEG engine for compressing the MCU'"'"'s before storage in the memory and for decompressing MCU'"'"'s read from the memory;

a resize circuit coupled to the buffer for resizing image data;

a display processor coupled to the resize circuit for generating graphical information for display with the image data;

a rotation circuit coupled to the bus for rotating the image data and storing a rotated image data in the buffer.

5. A digital signal processor for use in an image capture apparatus, comprising:

a bus

a capture data path coupled to the bus for accepting CCD data and outputting raw lines to the bus;

a memory coupled to the bus for storing the raw lines;

a data flow controller coupled to the bus for generating raw minimum computational units (MCUs) based on the raw lines stored in the memory;

an image processing data path for processing the raw MCUs from the data flow controller;

a buffer for storing the MCU'"'"'s from the image processing data path;

a JPEG engine for compressing the MCU'"'"'s before storage in the memory and for decompressing MCU'"'"'s read from the memory;

a resize circuit coupled to the buffer for resizing image data;

a display processor coupled to the resize circuit for generating graphical information for display with the image data;

a rotation circuit coupled to the bus for rotating the image data and storing a rotated image data in the buffer,

wherein at least four modes of operation are supported;

live view, image capture, instant review, and play mode, and wherein the image is automatically rotated in a second pass in accordance with a current physical orientation of the image capture apparatus and with the physical orientation of the image capture apparatus when the image was captured.

6. The digital signal processor of claim 5, wherein in the live view mode of operation, the image from the capture data path is stored in an input buffer of the memory, retrieved and processed by the image processing path, buffered, resized if necessary, and displayed along with any appropriate graphics.

7. The digital signal processor of claim 6, wherein if the image capture apparatus is in a vertical orientation, the image is rotated by the rotation circuit before being stored in the memory and the image is rotated again before being rendered for display.

8. The digital signal processor of claim 5, wherein in a capture mode of operation, the image from the capture data path is stored in an input buffer of the memory, retrieved and processed by the image processing path, buffered, compressed by the compression/decompression engine, and stored in a file of the memory.

9. The digital signal processor of claim 8, wherein if the image capture apparatus is currently in a horizontal orientation and the image was taken while the image capture apparatus was in a vertical orientation, the image is reduced in size before being rendered for display.

10. The digital signal processor of claim 9, wherein the display processor generates an indication to the user to change the orientation of the image capture apparatus and when the image capture apparatus is rotated to be in the vertical orientation, the image is rendered for display without resizing.

11. The image capture apparatus of claim 5, wherein in an instant playback mode of operation, the image from the capture data path is stored in an input buffer of the memory, retrieved and processed by the image processing path, buffered, compressed by the compression/decompression engine, stored in a file of the memory, read back from the file, decompressed, resized if necessary, and displayed along with any appropriate graphics.

12. The image capture apparatus of claim 5 further comprising an orientation detector for detecting a current physical orientation of the image capture apparatus, wherein the image from a first draw buffer is rotated by the rotation circuit, depending on the current orientation, and stored in a second draw buffer.

13. The image capture apparatus of claim 5, wherein in a playback mode of operation, one or more requested files are read, decompressed, resized if necessary, and displayed along with any appropriate graphics.

14. The image capture apparatus of claim 5 further comprising an orientation detector for detecting a current physical orientation of the image capture apparatus, wherein the image from a first draw buffer is rotated by the rotation circuit, depending on the current orientation, stored in a second draw buffer, and displayed in a correct orientation.

16. The image capture apparatus of claim 5 further comprises a YCC shuffle block having a data path coupled directly to the buffer.

15. A digital signal processor for use in an image capture apparatus, comprising:

a bus

a capture data path coupled to the bus for accepting CCD data and outputting raw lines to the bus;

a memory coupled to the bus for storing the raw lines;

a data flow controller coupled to the bus for generating raw minimum computational units (MCUs) based on the raw lines stored in the memory;

an image processing data path for processing the raw MCUs from the data flow controller;

a buffer for storing the MCU'"'"'s from the image processing data path;

a JPEG engine for compressing the MCU'"'"'s before storage in the memory and for decompressing MCU'"'"'s read from the memory;

a resize circuit coupled to the buffer for resizing image data;

a display processor coupled to the resize circuit for generating graphical information for display with the image data;

a rotation circuit coupled to the bus for rotating the image data and storing a rotated image data in the buffer;

a double buffered register, wherein transfers of data between buffers is activated at frame boundaries;

wherein at least four modes of operation are supported;

live view, image capture, instant review, and play mode.

17. A method for processing an image in an image capture unit including a display, the method comprising the steps of:

capturing raw image data with an image sensor;

processing the raw image data through a capture data path;

transferring the raw image data from the image sensor to a first memory via a bus for storage, wherein the first memory contains raw image data for one or more images;

transferring a particular raw image data from the first memory to a data flow control via the bus;

passing the raw image data from the data flow control to an image processing data path, the image processing data path for performing image processing on the particular raw image data to produce a rotated processed image;

transferring the processed image from the image processing data path to a second buffer through a first dedicated path;

establishing a data path between the second buffer an a compression/decompression engine;

compressing the processed image to generate a compressed image;

transferring the compressed image from the compression/decompression engine to a third buffer via the bus to store a compressed displayable image;

retrieving the compressed displayable image from the third buffer for decompression by the compression/decompression engine to produce displayable image data;

transferring the displayable image data to the second buffer;

transferring the displayable image data from the second buffer to a resize circuit through a second dedicated path;

performing display processing to produce graphics for display;

transferring the graphics and displayable image to a fourth buffer via the bus;

loading the graphics and displayable image from the fourth buffer to a rotate circuit through the bus;

unrotating the displayable image;

rendering the graphics and displayable image from the fourth buffer for display on the image capture unit; and

establishing a third dedicated path from the rotate circuit to the second buffer.

18. A method for processing an image in an image capture unit including a display, the method comprising the steps of:

capturing raw image data;

processing the raw image data through a capture data path;

storing the raw image data in a first memory, wherein the first memory contains raw image data for one or more images;

accessing a particular raw image data from the first memory and performing image processing on the particular raw image data to produce a rotated processed image;

loading the processed image in a second buffer;

compressing the processed image to generate a compressed image;

unrotating the image;

performing display processing to produce graphics for display;

storing the graphics and image in a fourth buffer;

rendering the graphics and image from the fourth buffer for display on the image capture unit; and

performing a second pass when the image capture unit is rotated, the second pass comprising the steps of;

accessing the image from the fourth buffer;

rotating the image;

storing the image in the second buffer;

passing the image through a resize circuit;

adding the graphics during the second pass; and

storing the image in a fifth buffer.