Configurable hardware based digital imaging apparatus

• US 7,609,297 B2
• Filed: 06/25/2003
• Issued: 10/27/2009
• Est. Priority Date: 06/25/2003
• Status: Expired due to Fees

• Current Assignee: Altera Corporation (Intel Corporation)
• Original Assignee: QST Holdings LLC
• Inventors: Master, Paul L., Watson, John
• Primary Examiner(s): Ye, Lin
• Assistant Examiner(s): HERNANDEZ, NELSON D
• Application Number:
  US10/606,031
• Publication Number:
  US 20040268096A1
• Time in Patent Office: 2,316 Days
• Field of Search: 712/29, 712/28, 712/11, 712/15, 348/222.1
• US Class Current: 348/222.1
• CPC Class Codes: G06F 2207/4814 Non-logic devices, e.g. operational amplifiers
• CPC Class Codes: G06F 7/4806 Computations with complex numbers
• CPC Class Codes: G06F 7/5443 Sum of products for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00
• CPC Class Codes: G06F 9/3001 Arithmetic instructions
• CPC Class Codes: G06F 9/3885 using a plurality of independent parallel functional units
• CPC Class Codes: G06F 9/3897 with adaptable data path
• CPC Class Codes: H03H 17/0294 Variable filters; Programmable filters

Abstract

The present invention provides a digital imaging apparatus having an optical sensor, an analog-to-digital converter, a plurality of computational elements, and an interconnection network. The optical sensor converts an object image into a detected image, which is then converted to digital image information by the analog-to-digital converter. The plurality of computational elements includes a first computational element having a first fixed architecture and a second computational element having a second, different fixed architecture. The interconnection network is capable of providing a processed digital image from the digital image information by configuring and reconfiguring the plurality of computational elements for performance of a plurality of different imaging functions. The invention may be embodied, for example, as a digital camera, a scanner, a printer, or a dry copier.

Claims

1. A digital imaging apparatus, comprising:

an optical sensor converting an object image into a detected image;

an analog-to-digital converter coupled to the optical sensor, the analog-to-digital converter converting the detected image to digital image information;

a plurality of heterogeneous computational units processing the digital image information to produce a processed digital image, the plurality of heterogeneous computational units including a first computational unit having a first architecture of a first plurality of computational elements each including algorithmic logic for simple computations, a data input and a data output, and a second computational unit having a second digital signal processing architecture of a second plurality of computational elements each including algorithmic logic for digital signal processing, a data input and a data output; and

an interconnection network coupled to the plurality of heterogeneous computational units and to the analog-to-digital converter, the interconnection network providing the digital image information to the plurality of heterogeneous computational units, configuring the plurality of heterogeneous computational units for performance of a first imaging function on the digital imaging information in response to configuration information that configures interconnections between at least the first plurality of computational elements and simultaneously in response to different configuration information to configure interconnections between the second plurality of computational elements, and reconfiguring the plurality of heterogeneous computational units for performance of a second imaging function on the digital image information in response to configuration information that reconfigures the interconnections between at least the first plurality of computational elements, the first imaging function being different than the second imaging function.

2. The digital imaging apparatus of claim 1, further comprising:

focusing means providing the object image to the optical sensor.

3. The digital imaging apparatus of claim 2, wherein the focusing means comprises a focusing assembly, the focusing assembly further comprising:

a lens;

a shutter;

an aperture; and

a focusing motor.

4. The digital imaging apparatus of claim 1, wherein the optical sensor is at least one of a charge coupled device (CCD), a charge injection device (CID), an optical complementary metal oxide silicon (CMOS) array, an optical bipolar junction transistor (BJT) array, a photogate array, or a photodiode array.

5. The digital imaging apparatus of claim 1, further comprising:

an analog output interface coupled to the interconnection network, the analog output interface converting the processed digital image to an analog form; and

a viewfinder screen coupled to the analog output interface, the viewfinder screen visually displaying the analog form of the processed digital image.

6. The digital imaging apparatus of claim 1, further comprising:

an analog output interface coupled to the interconnection network, the analog output interface converting the processed digital image to an analog form; and

an analog output port coupled to the analog output interface, the analog output port outputting the analog form of the processed digital image.

7. The digital imaging apparatus of claim 1, further comprising:

a first memory couplable to the interconnection network, the first memory storing the processed digital image.

8. The digital imaging apparatus of claim 7, wherein the first memory is selectively removable flash memory.

9. The digital imaging apparatus of claim 7, further comprising:

a second memory coupled to the interconnection network, the second memory storing the first configuration information and the second configuration information.

10. The digital imaging apparatus of claim 9, wherein the second memory is synchronous dynamic random access memory (SDRAM).

11. The digital imaging apparatus of claim 9, wherein the first memory and the second memory are at least one of flash memory, DRAM, SRAM, SDRAM, FeRAM, MRAM, ROM, EPROM and E²PROM.

12. The digital imaging apparatus of claim 1, further comprising:

a digital output port coupled to the interconnection network, the digital output port outputting the processed digital image.

13. The digital imaging apparatus of claim 1, wherein the processed digital image is provided as a plurality of processed digital image data packets.

14. The digital imaging apparatus of claim 13, further comprising:

a digital output interface coupled to the interconnection network, the digital output interface selecting a plurality of processed digital image data words from the plurality of processed digital image data packets and assembling the plurality of processed digital image data words to form the processed digital image.

15. The digital imaging apparatus of claim 1, wherein the digital image information is provided as a plurality of digital image information data packets.

16. The digital imaging apparatus of claim 1, further comprising:

a light source providing light for reflection from an object to form the object image.

17. The digital imaging apparatus of claim 1, further comprising:

a printer coupled to the interconnection network, the printer printing the processed digital image on a tangible medium.

18. The digital imaging apparatus of claim 1, further comprising:

a dry copier coupled to the interconnection network, the dry copier transferring the processed digital image to a tangible medium.

19. The digital imaging apparatus of claim 1, further comprising:

a data transmitter coupled to the interconnection network, the data transmitter transmitting the processed digital image to a remote location.

20. The digital imaging apparatus of claim 19, wherein the data transmitter is at least one of an analog (voice band) modem;

a digital modem;

a digital subscriber line modem; and

a cable modem.

21. The digital imaging apparatus of claim 1, wherein the plurality of imaging functions comprises at least two of linear filtering;

nonlinear filtering;

morphological filtering;

median filtering;

local weighted median filtering;

center weighted median filtering;

vector weighted median filtering;

multichannel image recovery;

multiframe image restoration;

iterative image restoration;

motion detection;

motion estimation;

low pass filtering;

multirate filtering;

wavelet-based multirate filtering;

autofocusing;

contrast enhancement; and

blur removal.

22. The digital imaging apparatus of claim 1, wherein the algorithmic logic of the first and second plurality of computational elements each perform different functions including at least two of linear filtering, non-linear filtering, memory, addition, multiplication, complex multiplication, subtraction, synchronization, queuing, over sampling, under sampling, adaptation, configuration, reconfiguration, control, input, output, and field programmability.

23. The digital imaging apparatus of claim 1, wherein the detected image comprises an electrical signal corresponding to brightness and color variations of the object image.

24. The digital imaging apparatus of claim 1, wherein the digital imaging apparatus is embodied as at least one integrated circuit.

25. The digital imaging apparatus of claim 1, wherein the digital imaging apparatus is embodied as a digital camera.

26. The digital imaging apparatus of claim 1, wherein the digital imaging apparatus is embodied as one or more of a scanner, a printer, or a dry copier.

27. A digital imaging apparatus, comprising:

an optical sensor converting an object image into a detected image;

an analog-to-digital converter coupled to the optical sensor, the analog-to-digital converter convening the detected image to a plurality of digital image information data packets;

a plurality of heterogeneous computational units including a first computational unit having a first architecture of a first plurality of computational elements each including algorithmic logic for simple computations, a data input and a data output, and a second computational unit having a second digital signal processing architecture of a second plurality of computational elements each including algorithmic logic for digital signal processing, a data input and a data output;

an interconnection network coupled to the plurality of heterogeneous computational units and to the analog-to-digital converter, the interconnection network providing the plurality of digital image information data packets, configuring the plurality of heterogeneous computational units for performance of a first imaging function in response to configuration information that configures the interconnections between at least the first plurality of computational elements and simultaneously in response to different configuration information to configure interconnections between the second plurality of computational elements, and by reconfiguring the plurality of heterogeneous computational units for performance of a second imaging function in response to configuration information that reconfigures interconnections between at least the first plurality of computational elements, the first imaging function being different than the second imaging function thereby providing a plurality of processed digital image data packets based on at least the first or second imaging function; and

a digital output interface coupled to the interconnection network, the digital output interface selecting a plurality of processed digital image data words from the plurality of processed digital image data packets and assembling the plurality of processed digital image data words to form a processed digital image.

28. The digital imaging apparatus of claim 27, wherein the optical sensor is at least one of a charge coupled device (CCD), a charge injection device (CID), an optical complementary metal oxide silicon (CMOS) array, an optical bipolar junction transistor (BJT) array, a photogate array, or a photodiode array.

29. The digital imaging apparatus of claim 27, further comprising:

an analog output interface coupled to the interconnection network, the analog output interface convening the processed digital image to an analog form;

a viewfinder screen coupled to the analog output interface, the viewfinder screen visually displaying the analog form of the processed digital image;

an analog output port coupled to the analog output interface, the analog output port outputting the analog form of the processed digital image; and

a digital output port coupled to the interconnection network, the digital output port outputting the processed digital image.

30. The digital imaging apparatus of claim 27, wherein the plurality of imaging functions comprises linear filtering;

nonlinear filtering;

morphological filtering;

median filtering;

local weighted median filtering;

center weighted median filtering;

vector weighted median filtering;

multichannel image recovery;

multiframe image restoration;

iterative image restoration;

motion detection;

motion estimation;

low pass filtering;

multirate filtering;

wavelet-based multirate filtering;

autofocusing;

contrast enhancement; and

blur removal.

31. The digital imaging apparatus of claim 27, wherein the algorithmic logic of the first and second plurality of computational elements each perform different functions including at least two of:

linear filtering, non-linear filtering, memory, addition, multiplication, complex multiplication, subtraction, synchronization, queuing, over sampling, under sampling, adaptation, configuration, reconfiguration, control, input, output, and field programmability.

32. The digital imaging apparatus of claim 27, wherein the digital imaging apparatus is embodied as a digital camera.

33. A method of processing a digital image comprising:

convening an object image into a detected image via an analog-to-digital converter;

convening the detected image to digital image information;

providing a plurality of heterogeneous computational units including a first simple computational unit having a first architecture of a first plurality of computational elements each including algorithmic logic for simple computations, a data input and a data output, and a second digital signal processing computational unit having a second architecture of a second plurality of computational elements each including algorithmic logic for digital signal processing, a data input and a data output;

providing an interconnection network coupled to each of the plurality of computational elements and to the analog-to-digital converter;

configuring the plurality of computational units via the interconnections between at least the first plurality of computational elements and simultaneously in response to different configuration information to configure interconnections between the second plurality of computational elements for performance of a first imaging function; and

reconfiguring the plurality of computational units for performance of a second imaging function in response to second configuration information by changing the interconnections between at least the first plurality of computational elements, the first imaging function being different than the second imaging function.

34. A digital imaging apparatus, comprising:

a digital image information input;

a plurality of heterogeneous computational units to process digital image information to produce a processed digital image, the plurality of heterogeneous computational units including a first simple computational unit having a first architecture of a first plurality of computational elements each including algorithmic logic for simple computations, a data input and a data output, and a second computational unit having a second digital signal processing architecture of a second plurality of computational elements each including algorithmic logic for digital signal processing, a data input and a data output; and

an interconnection network coupled to the plurality of heterogeneous computational units, the interconnection network providing the digital image information to the plurality of heterogeneous computational elements and configuring the plurality of heterogeneous computational units to perform a first imaging function by simultaneous providing different configuration information to interconnections between the first plurality of computational elements and the interconnections between the second plurality of computational elements, and reconfiguring the plurality of heterogeneous computational units for performance of a second imaging function in response to providing configuration information that reconfigures the interconnections between at least some of the first plurality of heterogeneous computational elements.

35. The image apparatus of claim 34, further comprising:

an optical sensor converting an object image into a detected image; and

an analog-to-digital converter coupled to the optical sensor, the analog-to-digital converter converting the detected image to the digital image information, the converter coupled to the digital image input.