Digital camera device and methodology for distributed processing and wireless transmission of digital images

• US 7,372,485 B1
• Filed: 11/05/1999
• Issued: 05/13/2008
• Est. Priority Date: 06/08/1999
• Status: Expired due to Fees

• Current Assignee: RPX Corporation
• Original Assignee: LightSurf Technologies, Inc. (VeriSign, Inc.)
• Inventors: Kahn, Philippe R., Easwar, Venkat V., Kirani, Shekhar, Bodnar, Eric O., Kahn, Sonia Lee
• Primary Examiner(s): Ometz, David
• Assistant Examiner(s): NGUYEN, LUONG TRUNG
• Application Number:
  US09/434,703
• Time in Patent Office: 3,112 Days
• Field of Search: 348/234, 348/394.1, 348/396.1, 348/552, 348/273, 375/240.18, 375/240.19, 375/240.2, 382/248, 382/232
• US Class Current: 348/234
• CPC Class Codes: G06T 9/007 Transform coding, e.g. discrete cosine transform

Abstract

A digital imaging system is described that provides techniques for reducing the amount of processing power required by a given digital camera device and for reducing the bandwidth required for transmitting image information to a target platform. The system defers and/or distributes the processing between the digital imager (i.e., digital camera itself) and the target platform that the digital imager will ultimately be connected to. In this manner, the system is able to decrease the actual computation that occurs at the digital imager. Instead, the system only performs a partial computation at the digital imager device and completes the computation somewhere else, such as at a target computing device (e.g., desktop computer) where time and size are not an issue (relative to the imager). By deferring resource-intensive computations, the present invention substantially reduces the processor requirements and concomitant battery requirements for digital cameras. Further, by adopting an image strategy optimized for compression (compressed luminosity record), the present invention decreases the bandwidth requirements for transmitting images, thereby facilitating the wireless transmission of digital camera images.

Claims

1. In a digital imaging system, a method for distributed digital image processing, the method comprising:

recording luminosity information at a first device, for representing an image that has been digitally captured at the first device;

without performing color interpolation at the first device, generating compressed luminosity information at the first device by applying a wavelet transform compression to individual color planes that comprise the luminosity information, followed by applying quantization and compression to the luminosity information;

packaging said compressed luminosity information, in a plurality of data packets suitable for progressive transmission of image data corresponding to varying levels of photographic significance, with header information identifying the individual color planes that comprise the luminosity information;

progressively transmitting a first set of data packets from said plurality of data packets of said compressed luminosity information to a second device, while remaining data packets from said plurality of data packets are maintained at the first device;

restoring said luminosity information from said first set of data packets of said compressed luminosity information at the second device;

converting said luminosity information at the second device into a color image, including performing color interpolation at the second device, corresponding to the progressively transmitted data packets received by the second device; and

in response to receipt of a second set of data packets from the remaining data packets, said second set of data packets corresponding to a higher level of photographic significance, converting a lower-quality representation of the image into a higher-quality representation by synchronizing said lower-quality representation with said higher-quality representation at the second device.

2. The method of claim 1, wherein said luminosity information comprises light-level information for representing an image that has been digitally captured at the first device.

3. The method of claim 1, wherein said generating step includes:

applying generic binary compression to said compressed luminosity information at the first device.

4. The method of claim 3, wherein said step of applying generic binary compression includes applying run-length encoding.

5. The method of claim 3, wherein said step of applying generic binary compression includes applying Huffman coding.

6. The method of claim 1, wherein said restoring step includes:

reversing said compression that occurred at the first device.

7. The method of claim 1, wherein said step of converting said luminosity information into a color image includes:

interpolating color information for the image from said luminosity information.

8. The method of claim 7, wherein said interpolating step includes:

apply a YUV transformation to said luminosity information at the second device for converting said luminosity information into a color image in YUV color space.

9. The method of claim 7, wherein said step of converting said luminosity information into a color image further includes:

converting the color image into a standard file format at the second device.

10. The method of claim 9, wherein said standard file format comprises a JPEG file format.

11. The method of claim 9, wherein said step of converting said luminosity information into a color image further includes:

applying JPEG compression to the color image at the second device.

12. The method of claim 1, wherein said step of transmitting said compressed luminosity information to a second device includes:

transmitting said compressed luminosity information from a digital camera to a computer using a packet-based communication protocol.

13. The method of claim 12, wherein said step of transmitting said compressed luminosity information from a digital camera to a computer using packet-based communication protocol includes:

selectively connecting the digital camera to a cellular phone for establishing a wireless communication session with the computer.

14. The method of claim 1, wherein said second device comprises a computer with connectivity to the Internet and wherein said method further includes making the color image available to multiple users.

15. In a digital imaging system, a method for deferring digital image processing, the method comprising:

recording sensor information from an image sensor at a first device, for representing an image that has been recorded at the image sensor of the first device;

compressing said sensor information prior to color processing by applying a transformation compression to individual color planes that comprise the sensor information, for generating compressed sensor information at the first device;

packaging said compressed sensor information, in a plurality of data packets suitable for progressive transmission of image data corresponding to varying levels of photographic significance, with header information identifying the individual color planes that comprise the sensor information;

without having performed color processing at the first device, progressively transmitting a first set of data packets from said plurality of data packets of said compressed sensor information to a second device, while remaining data packets from said plurality of data packets are maintained at the first device;

decompressing said compressed sensor information at the second device, whereupon said sensor information may thereafter be processed into a color image corresponding to the progressively transmitted data packets received by the second device; and

in response to receipt of a second set of data packets from the remaining data packets, said second set of data packets corresponding to a higher level of photographic significance, decompressing said second set of data packets to convert a lower-quality representation of the image into a higher-quality representation by synchronizing said lower-quality representation with said higher-quality representation at the second device.

16. The method of claim 15, wherein said sensor information comprises light-level information for representing an image that has been digitally recorded at the first device.

17. The method of claim 15, wherein said compression step includes:

applying a wavelet transform to individual bit planes that comprise the sensor image; and

applying compression to the transformed sensor image, to create said compressed sensor information at the first device.

18. The method of claim 17, wherein said step of applying compression to the transformed sensor image includes:

applying compression using run-length encoding.

19. The method of claim 17, wherein said step of applying compression to the transformed sensor image includes:

applying compression using Huffman coding.

20. The method of claim 17, wherein said decompression step includes:

reversing said wavelet transform that occurred at the first device.

21. The method of claim 15, further comprising:

converting said sensor information into a color image by interpolating color information for the image from said sensor information.

22. The method of claim 21, wherein said converting step includes:

apply a YUV transformation to said sensor information at the second device for converting said sensor information into a color image in YUV color space.

23. The method of claim 21, wherein said converting step includes:

converting the color image into a standard file format at the second device.

24. The method of claim 23, wherein said standard file format comprises a JPEG file format.

25. The method of claim 23, wherein said converting step includes:

applying JPEG compression to the color image at the second device.

26. The method of claim 15, wherein said step of transmitting said compressed sensor information to a second device includes:

transmitting said compressed sensor information from a digital camera to a computer in a wireless manner using a communication protocol.

27. The method of claim 26, wherein said step of transmitting said compressed sensor information from a digital camera to a computer includes:

selectively connecting the digital camera to a cellular phone for establishing a wireless communication session with the computer.

28. The method of claim 15, wherein said second device comprises a computer with connectivity to the Internet and wherein said method further includes making the color image available to multiple users.

29. An imaging system providing deferred image processing, the system comprising:

an imager having a sensor for recording luminosity information for a visual image captured by the imager, said luminosity information comprising luminosity values recorded by the sensor;

a compressor module for compressing said luminosity information by applying a transformation compression to each individual color planes that comprise the luminosity information, for generating compressed luminosity information at the imager without having performed color processing, wherein the compressed luminosity information is packaged into a plurality of data packets suitable for progressive transmission of image data, corresponding to varying levels of photographic significance, in a bit stream with header information identifying the individual color planes that comprise the luminosity information;

a wireless communication link for progressively transmitting a first set of data packets from said plurality of data packets of said compressed luminosity information to a target device, while remaining data packets from said plurality of data packets are maintained in a storage coupled with a first device;

a decompression module for decompressing said compressed luminosity information at the target device, whereupon said sensor information may thereafter be processed into a color image corresponding to the progressively transmitted data packets received by the target device; and

in response to receipt of a second set of data packets from the remaining data packets, said second set of data packets corresponding to a higher level of photographic significance, the decompression module for decompressing said second set of data packets to convert a lower-quality representation of the image into a higher-quality representation by synchronizing said lower-quality representation with said higher-quality representation at the target device.

30. The system of claim 29, wherein said luminosity information comprises brightness information for representing an image that has been digitally captured at the imager.

31. The system of claim 29, wherein said compression module includes:

a generic binary compression module for compressing said luminosity information at the first device.

32. The system of claim 31, wherein said generic binary compression module applies run-length encoding.

33. The system of claim 31, wherein said generic binary compression module applies Huffman coding.

34. The system of claim 31, further comprising a generic binary decompression module for reversing generic binary compression that has been applied at the imager.

35. The system of claim 29, wherein said target device includes:

an interpolation module for interpolating color information for the image from said luminosity information.

36. The system of claim 35, wherein said interpolation module applies a YUV transformation to said luminosity information at the target device for converting said luminosity information into a color image in YUV color space.

37. The system of claim 29, wherein said target device further includes:

a compression module for converting the color image into a standard compressed file format at the target device.

38. The system of claim 37, wherein said standard compressed file format comprises a JPEG file format.

39. The system of claim 37, wherein said compression module of said target device includes a JPEG module for applying JPEG compression to the color image at the target device.

40. The system of claim 29, wherein said imager comprises a digital camera, wherein said target device comprises a computer, and wherein said communication link is coupled to a cellular phone device for transmitting said compressed luminosity information from said digital camera to said computer in a wireless manner using a communication protocol.

41. The system of claim 40, wherein said communication link is selectively coupled to the cellular phone for establishing a wireless communication session between the digital camera and the computer.

42. The system of claim 29, wherein said target device comprises a computer with connectivity to the Internet, which provides access to the color image to multiple users.

43. The system of claim 29, wherein said imager comprises a selected one of a digital camera, a digital camcorder, and a closed circuit surveillance camera.

44. The system of claim 29, wherein said target device comprises a desktop computer.

45. The system of claim 29, wherein said target device comprises a server computer.

46. The system of claim 29, wherein said sensor comprises a complementary metal-oxide semiconductor (CMOS) image sensor.

47. The system of claim 29, wherein said sensor comprises a charge-coupled device (CCD) image sensor.

48. The system of claim 29, wherein said luminosity information comprises gray-scale luminosity information, prior to being processed into a color image.

49. The system of claim 29, wherein said compressed luminosity information comprises a wavelet transformed and compressed luminosity record of the image recorded at the sensor.

50. In a digital imaging system, a method for distributed digital image processing, the method comprising:

recording luminosity information at a first device, for representing an image that has been digitally captured at the first device;

while deferring color interpolation to a second device, generating compressed luminosity information at the first device by applying a wavelet transform compression to individual color planes that comprise the luminosity information, followed by applying quantization and compression to the luminosity information;

packaging said compressed luminosity information, in a plurality of data packets suitable for progressive transmission of image data corresponding to varying levels of photographic significance, with header information identifying the individual color planes;

progressively transmitting a first set of data packets from said plurality of data packets of said compressed luminosity information to the second device, while remaining data packets from said plurality of data packets are maintained at the first device;

restoring said luminosity information from said first set of data packets of said compressed luminosity information at the second device;

converting said luminosity information at the second device into a color image, including performing color interpolation at the second device, corresponding to the progressively transmitted data packets received by the second device; and

in response to receipt of a second set of data packets from the remaining data packets, said second set of data packets corresponding to a higher level of photographic significance, converting a lower-quality representation of the image into a higher-quality representation by synchronizing said lower-quality representation with said higher-quality representation at the second device.