Using a graphics processing unit to correct video and audio data

• US 8,456,547 B2
• Filed: 12/31/2009
• Issued: 06/04/2013
• Est. Priority Date: 11/09/2005
• Status: Active Grant

• Current Assignee: NVIDIA Corporation
• Original Assignee: NVIDIA Corporation
• Inventors: Wloka, Matthias M.
• Primary Examiner(s): Velez, Roberto
• Assistant Examiner(s): FOSSELMAN, JOEL W
• Application Number:
  US12/651,364
• Publication Number:
  US 20100173669A1
• Time in Patent Office: 1,251 Days
• Field of Search: 348/246, 348/231.99, 455/556.1
• US Class Current: 348/246
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 25/68 applied to defects
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 5/907 using static stores, e.g. storage tubes or semiconductor memories H04N5/91 takes precedence

Abstract

Described is a device (e.g., a cell phone incorporating a digital camera) that incorporates a graphics processing unit (GPU) to process image data in order to increase the quality of a rendered image. The processing power provided by a GPU means that, for example, an unacceptable pixel value (e.g., a pixel value associated with a malfunctioning or dead detector element) can be identified and replaced with a new value that is determined by averaging other pixel values. Also, for example, the device can be calibrated against benchmark data to generate correction factors for each detector element. The correction factors can be applied to the image data on a per-pixel basis. If the device is also adapted to record and/or play digital audio files, the audio performance of the device can be calibrated to determine correction factors for a range of audio frequencies.

Claims

1. A device comprising:

an image capture device comprising an array of detector elements that generates a corresponding plurality of pixel values comprising a frame of image data;

a memory operable to store said pixel values;

a display device operable to display an image produced using said pixel values; and

a graphics processing unit (GPU) operable to execute a method of processing said pixel values, said method comprising;

accessing a first pixel value corresponding to a detector element;

comparing said first pixel value to a second pixel value, wherein said second pixel value is generated by said detector element;

incrementing a counter value if said first pixel value and said second pixel value are equal within a tolerance;

if said counter value exceeds a threshold value, then accessing a correction factor corresponding to said detector element and applying said correction factor to said first pixel value, and otherwise continuing said comparing and incrementing for other pixel values generated by said detector element.

2. The device of claim 1 wherein said correction factor is read from a lookup table comprising a plurality of correction factors indexed by detector element.

3. The device of claim 1 wherein said correction factor compensates for an effect introduced by a lens that transmits light to said image capture device.

4. The device of claim 3 wherein said effect is a displacement effect, wherein a distortion in said lens causes light passing through said distortion to change direction.

5. The device of claim 3 wherein said effect is a color shift effect introduced by different color wavelengths of light passing through said lens, wherein further a compensation factor is applied to each color component of said first pixel value.

6. The device of claim 1 wherein said correction factor compensates for a difference in color between a color represented by said first pixel value and a benchmark color value.

7. The device of claim 1 wherein said correction factor compensates for loss of sharpness relative to a benchmark.

8. The device of claim 1 further comprising a wireless cell phone comprising:

a microphone for receiving a first audio input;

a wireless transmitter for transmitting a signal comprising said first audio input;

a receiver for receiving a wireless signal comprising a second audio input; and

a speaker for audibly rendering said second audio input.