Digital camera with reduced image buffer memory and minimal processing for recycling through a service center

• US 20040252201A1
• Filed: 06/10/2003
• Published: 12/16/2004
• Est. Priority Date: 06/10/2003
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Zoran Corporation (Qualcomm, Inc.)
• Inventors: Meitav, Ohad, Seltz, Daniel, Shenberg, Itzhak
• Granted Patent: US 7,612,803 B2
• Time in Patent Office: Days
• US Class Current: 348/211.3
• CPC Class Codes: H04N 1/32128 attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 2201/3212 of data relating to a job, e.g. communication, capture or filing of an image
• CPC Class Codes: H04N 2201/3242 of processing required or performed, e.g. for reproduction or before recording H04N2201/3256 and subgroups take precedence
• CPC Class Codes: H04N 2201/3277 The additional information being stored in the same storage device as the image data

Abstract

A digital camera system in which capture, processing and storage functions are partitioned differently than in existing systems. A hand-held digital camera is used with a workstation that may exist in a commercial image processing service center. A minimal amount of image data processing is performed in the digital camera, thereby allowing significant digital camera cost reductions due to lower memory requirements, lower processing requirements, and lower power requirements. Real-time single pass image compression techniques are employed within this digital camera to permit rapid gathering and storage of raw or minimally processed image data. The workstation to which the image data are transferred performs the image processing normally done within existing cameras. This processing takes advantage of the increased computational power that is possible to have in such a workstation, compared to that of a small camera, and the increased time over which such processing may be performed.

Claims

1. A method of capturing and processing optical radiation data, comprising:

within a hand-held camera, generating digital data of at least one frame of a pattern of optical radiation from a sensor having an array of photosensitive pixels upon which the pattern of optical radiation is incident,

storing, in a non-volatile memory within the camera, the generated data and support data necessary to perform at least a first processing of the generated data to improve the quality of a resulting optical radiation pattern display frame formed therefrom,

subsequently transferring the generated data and support data from the camera'"'"'s non-volatile memory to a workstation, and

performing the first processing of the generated data within the workstation with use of the support data to produce processed data from which the resulting optical radiation pattern display frame has the improved quality.

2. The method of claim 1, wherein

the support data include data of responses of at least some of the photosensitive pixels without optical radiation incident thereon, and

said first processing includes combining said data of responses with the generated data to correct for noise generated in the array of photosensitive pixels.

3. The method of claim 1, wherein

the support data include data of characteristics of an optical system within the camera that directs the pattern of optical radiation onto the sensor, and

said first processing includes combining said data of optical system characteristics with the generated data to correct for at least imperfect optical characteristics of the optical system.

4. The method of claim 3, wherein the imperfect optical characteristics that are corrected are distortions created by the optical system in the pattern of optical radiation incident on the sensor.

5. The method of claim 1, wherein

the support data include data of defective ones of the photosensitive pixels of the array, and

said first processing includes combining said data of defective photosensitive pixels with the generated data to correct for the effect of the defective photosensitive pixels.

6. The method of claim 1, wherein

the support data include digital data of a second frame of a pattern of optical radiation generated by the sensor within one-half second of said at least one frame, and

said first processing includes combining said at least one frame and the second frame of digital data.

7. The method of claim 6, wherein combining said at least one frame and the second frame of digital data corrects for defective ones of the photosensitive pixels of the array.

8. The method of claim 6, wherein combining said at least one frame and the second frame of digital data corrects for effects of motion of at least a portion of the incident pattern of optical radiation of one of the first or second frames relative to the sensor.

9. The method of claim 6, wherein combining said at least one frame and the second frame of digital data expands an effective dynamic range of the photosensitive pixels.

10. The method of claim 1, additionally comprising compressing the generated data and the support data prior to storing them within the camera.

11. A method of capturing and processing data of image frames, comprising:

generating digital data representative of pixels of a plurality of image frames in sequence in a plurality of primary colors from an image sensor within a hand held camera that includes an array of photosensitive elements,

storing data representative of the generated digital data in a non-volatile memory internal to the camera without affecting the stored data by any of the following within the camera;

(a) processing the generated digital data sufficient to be able to display images of said plurality of image frames,

(b) processing the generated digital data into a luminance/chrominance format, and

(c) storing at least one full frame of data representative of the generated digital data in a frame buffer at one time,

subsequently transferring the stored digital data from the non-volatile memory of the camera to a workstation, and

processing the stored digital data within the workstation in a manner sufficient to be able to display images of said plurality of image frames including processing the transferred digital data into a YUV format.

12. The method of claim 11, which additionally comprises transferring support data from the camera to the workstation, and use of the support data when processing the acquired image frame data within the workstation.

13. The method of claim 11 wherein processing within the workstation includes de-mosaicing of the image frame data rather than performing de-mosaicing processing within the camera.

14. The method of claim 13 wherein processing within the workstation additionally includes edge enhancement of the image frame data rather than performing edge enhancement processing within the camera.

15. The method of claim 11 wherein processing within the workstation includes white correction of the image frame data rather than performing white correction processing within the camera.

16. The method of claim 11 wherein processing within the workstation includes gamma correction of the image frame data rather than performing gamma correction processing within the camera.

17. The method of claim 12 wherein the support data transferred from the camera to the workstation includes data of an image frame acquired without light on the sensor, and processing within the workstation includes correction of the image frame data for black level noise with use of the support data rather than performing correction processing for black level noise within the camera.

18. The method of claim 12 wherein the support data transferred from the camera to the workstation includes data of unevenness of imaging by an optical system within the camera onto the image sensor, and processing within the workstation includes correction of the image frame data for unevenness of imaging by the camera optical system with use of the support data rather than performing correction processing for processing for unevenness of imaging by the camera optical system within the camera.

19. The method of claim 12, wherein the support data transferred from the camera to the workstation includes data of defective pixels of the image sensor, and processing within the workstation includes correction of the image frame data for defective pixels with use of the support data rather than performing correction processing for the defective pixels within the camera.

20. The method of claim 11, additionally comprising compressing the acquired image data and then storing the compressed data in the camera.

21. The method of claim 20, wherein the acquired image frame data is stored prior to compression in a buffer memory having a capacity of less than fifteen percent of that necessary to store an image frame.

22. The method of claim 11, wherein acquiring digital data of at least one image frame includes acquiring digital data of first and second image frames from the image sensor within one-half second of each other, and wherein processing of the acquired image frame data within the workstation includes use of the second image data to correct, enhance or modify the data of the first image.

23. The method of claim 22 wherein the processing includes use of the second image data to supply missing data of the first image due to defective pixels.

24. The method of claim 22 wherein the processing includes use of the second image data to correct for fixed pattern noise of the first image.

25. The method of claim 22 wherein the processing includes use of the second image to correct for motion of first image.

26. The method of claim 22 where the second image is used to extend the dynamic range of the first image beyond that provided by the sensor.

27. The method of claim 11 wherein storing the acquired digital data includes storing the acquired digital data in a non-removable non-volatile memory within the camera, and wherein subsequently transferring the acquired image data from the camera includes transferring data from the non-volatile memory of the camera into the workstation.

28. The method of claim 27, wherein transferring data from the non-volatile camera memory into the workstation is performed wirelessly.

29. A method of processing a frame of data from a sensor of an image thereon within a hand-held camera having a two-dimensional array of photodetecting elements that generate data in a plurality of primary colors, comprising within the camera:

reading the image data from the photodetecting elements in a time sequence in different ones of the primary colors,

writing the image data of different ones of the primary colors into a first memory in the order read from the photodetector in sequential strips across the image,

reading the image data of individual image strips from the first memory in groups of image data of different ones of the plurality of primary colors,

wherein the writing and reading occur at rates wherein less than fifteen per-cent of the frame of image data are stored in the first memory at one time,

compressing the groups of image data of different ones of the plurality of primary colors within individual strips across the image read from of the first memory, and

storing the compressed groups of image data in a second memory within the camera that is non-volatile.

30. The method of claim 29, wherein storing the compressed groups of image data in the second memory includes utilizing a memory permanently connected within the camera as the second memory.

31. The method of claim 29, wherein no processing occurs in the camera for displaying the image data.

32. The method of claim 29, wherein no processing occurs in the camera for any of de-mosaicing, white balance correction, gamma correction and lens compensation.

33. The method of claim 29, additionally comprises within the camera of reading and storing in the second memory compressed groups of a plurality of frames of image data, and thereafter further comprises transferring the stored compressed groups of a plurality of frames of image data from the second memory to a workstation outside of the camera.

34. The method of claim 33, additionally comprising within the workstation:

decompressing the compressed groups of image data, and

processing the decompressed groups of image data to form data files for the plurality of frames in a form from which images may be reproduced.

35. The method of claim 34, wherein the processing within the workstation includes all of de-mosaicing, edge enhancement, white balance correction and gamma correction.

36. The method of claim 35, wherein no processing occurs in the camera for any of de-mosaicing, edge enhancement, white balance correction and gamma correction.

37. The method of claim 34, which additionally comprises within the camera of storing in the second memory characteristics of camera optics used to form the image on the sensor.

38. The method of claim 37, wherein the characteristics of the camera optics are transferred from the second memory of the camera to the workstation, and wherein the processing within the workstation includes compensating the image data for the characteristics of the camera optics.

39. The method of claim 38, wherein no processing to compensate for characteristics of the camera optics is performed within the camera.

40. A hand held camera, comprising within an enclosure:

a two-dimensional sensor having an array of individual photodetectors of optical radiation that generate signals related to optical radiation directed onto the sensor in a plurality of primary colors,

an optical system positioned through the enclosure to direct a field of optical radiation from outside the camera onto the sensor,

a first memory receiving and temporarily storing data of values of the optical radiation directed onto the individual detectors of the sensor in a time sequence, said first memory having a storage capacity of an amount of data less than fifteen percent of an amount of data generated by the individual detectors of the sensor to represent one field of optical radiation directed onto the sensor,

a processor that reads out the data in the first memory and compresses said data for each of the plurality of primary colors,

a second memory that stores the compressed data of a plurality of fields of optical radiation in a non-volatile manner, and

a communications device connected to the second memory for communicating the compressed data files outside of the enclosure.

41. The camera of claim 40, wherein at least the sensor, the first memory and the processor are formed on a single integrated circuit chip.

42. The camera of claim 41, additionally comprising an analog-to-digital converter connected to receive the signals from the sensor and provide digital signals for the processor and the first memory, and wherein said analog-to-digital converter is also formed on said single integrated circuit chip.