Using a graphics processing unit to correct video and audio data

• US 20070103567A1
• Filed: 11/09/2005
• Published: 05/10/2007
• Est. Priority Date: 11/09/2005
• Status: Active Grant

• Current Assignee: Matthias Wloka
• Original Assignee: Matthias Wloka
• Inventors: Wloka, Matthias
• Granted Patent: US 7,750,956 B2
• Time in Patent Office: Days
• US Class Current: 348/231.990
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 25/68 applied to defects
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 5/907 using static stores, e.g. storage tubes or semiconductor memories H04N5/91 takes precedence

Abstract

Described is a device (e.g., a cell phone incorporating a digital camera) that incorporates a graphics processing unit (GPU) to process image data in order to increase the quality of a rendered image. The processing power provided by a GPU means that, for example, an unacceptable pixel value (e.g., a pixel value associated with a malfunctioning or dead detector element) can be identified and replaced with a new value that is determined by averaging other pixel values. Also, for example, the device can be calibrated against benchmark data to generate correction factors for each detector element. The correction factors can be applied to the image data on a per-pixel basis. If the device is also adapted to record and/or play digital audio files, the audio performance of the device can be calibrated to determine correction factors for a range of audio frequencies.

Claims

1. A device comprising:

an image capture device comprising an array of detector elements that generates a corresponding first plurality of pixel values comprising a first frame of image data;

a memory operable to store said first plurality of pixel values;

a display device operable to display an image based on said first plurality of pixel values; and

a graphics processing unit (GPU) operable to execute a method of processing said first plurality of pixel values, said method comprising;

identifying an unacceptable pixel value, wherein said unacceptable pixel value is considered unacceptable if a criterion is satisfied; and

determining a new value for said unacceptable pixel value.

2. The device of claim 1 wherein said determining further comprises averaging a subset of said first plurality of pixel values, wherein pixel values in said subset correspond to detector elements that neighbor a detector element that corresponds to said unacceptable pixel value.

3. The device of claim 1 wherein said identifying further comprises:

accessing a second plurality of pixel values comprising a second frame of image data that is next after said first frame in chronological order; and

comparing a first pixel value in said first plurality of pixel values to a second pixel value in said second plurality of pixel values, wherein said first pixel value and said second pixel value correspond to a same detector element.

4. The device of claim 3 wherein said criterion is satisfied if said first and second pixel values are considered equal.

5. The device of claim 3 wherein said method further comprises:

incrementing a counter value each time pixel values corresponding to said same detector element are considered equal over two successive chronologically-ordered frames, wherein said criterion is satisfied if said counter value satisfies a threshold; and

resetting said counter value each time pixel values corresponding to said same detector element are considered not equal over two successive chronologically-ordered frames.

6. The device of claim 1 wherein said method further comprises:

identifying a plurality of unacceptable pixel values that correspond to adjacent detector elements, wherein said plurality of unacceptable pixel values are considered unacceptable according to said criterion;

selecting a first one of said unacceptable pixel values;

averaging a first subset of said first plurality of pixel values to determine a first new pixel value for a first one of said unacceptable pixel values, wherein pixel values in said first subset correspond to detector elements that neighbor a detector element that corresponds to said first one of said unacceptable pixel values; and

averaging a second subset of pixel values including said first new pixel value to determine a second new pixel value for a second one of said unacceptable pixel values, wherein pixel values in said second subset correspond to detector elements that neighbor a detector element that corresponds to said second one of said unacceptable pixel values.

7. The device of claim 1 further comprising a wireless cell phone comprising:

a microphone for receiving a first audio input;

a wireless transmitter for transmitting a signal comprising said first audio input;

a receiver for receiving a wireless signal comprising a second audio input; and

a speaker for audibly rendering said second audio input.

8. The device of claim 1 wherein said method further comprises:

accessing a pixel value corresponding to a detector element;

accessing a correction factor corresponding to said detector element; and

applying said correction factor to said pixel value.

9. A device comprising:

an image capture device comprising an array of detector elements that generates a corresponding plurality of pixel values comprising a frame of image data;

a memory operable to store said pixel values;

a display device operable to display an image produced using said pixel values; and

a graphics processing unit (GPU) operable to execute a method of processing said pixel values, said method comprising;

accessing a pixel value corresponding to a detector element;

accessing a correction factor corresponding to said detector element; and

applying said correction factor to said pixel value.

10. The device of claim 9 wherein said correction factor is read from a lookup table comprising a plurality of correction factors indexed by detector element.

11. The device of claim 9 wherein said correction factor compensates for an effect introduced by a lens that transmits light to said image capture device.

12. The device of claim 11 wherein said effect is a displacement effect, wherein a distortion in said lens causes light passing through said distortion to change direction.

13. The device of claim 11 wherein said effect is a color shift effect introduced by different color wavelengths of light passing through said lens, wherein further a compensation factor is applied to each color component of said pixel value.

14. The device of claim 9 wherein said correction factor compensates for a difference in color between a color represented by said pixel value and a benchmark color value.

15. The device of claim 9 wherein said correction factor compensates for loss of sharpness relative to a benchmark.

16. The device of claim 9 further comprising a wireless cell phone comprising:

a microphone for receiving a first audio input;

a wireless transmitter for transmitting a signal comprising said first audio input;

a receiver for receiving a wireless signal comprising a second audio input; and

a speaker for audibly rendering said second audio input.

17. A device comprising:

a memory operable to store an audio file comprising digital audio data; and

a graphics processing unit (GPU) operable to execute a method of processing said audio data, said method comprising;

determining a frequency associated with an instance of said audio data;

accessing a correction factor corresponding to said frequency, said correction factor for adjusting amplitude; and

applying said correction factor to said instance of audio data.

18. The device of claim 17 further comprising a digital camera comprising an array of detector elements operable to generate a corresponding plurality of pixel values comprising a frame of image data.

19. The device of claim 18 wherein said method further comprises adjusting a pixel value.

20. The device of claim 17 further comprising a wireless cell phone comprising:

a microphone operable to receive a first audio input;

a wireless transmitter operable to transmit a signal comprising said first audio input;

a receiver operable to receive a wireless signal comprising a second audio input; and

a speaker operable to audibly render said second audio input.