Apparatus and method for performing image transforms in a digital display system

• US 6,456,340 B1
• Filed: 04/19/1999
• Issued: 09/24/2002
• Est. Priority Date: 08/12/1998
• Status: Expired due to Term

• Current Assignee: RPX Corporation
• Original Assignee: Pixonics, LLC
• Inventors: Margulis, Neal
• Primary Examiner(s): Miller, John
• Assistant Examiner(s): Tran, Trang U.
• Application Number:
  US09/294,259
• Time in Patent Office: 1,254 Days
• Field of Search: 348/745, 348/746, 348/747, 348/806, 348/807, 348/441, 348/448, 348/452, 348/50, 345/501, 345/507, 345/509, 345/196, 345/419, 345/420, 345/427, 345/429-431, 345/433, 345/437, 345/147, 345/150, 345/136, 386/52, 172/32, 172/36, 172/35, 172/34
• US Class Current: 348/745
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 3/007 Use of pixel shift techniques, e.g. by mechanical shift of the physical pixels or by optical shift of the perceived pixels
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 5/14 Picture signal circuitry for video frequency region cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

An image processing apparatus for use in a display system comprises a display device for viewing image, and a geometric transformation module that is configured to precondition said image data with geometric transformations to thereby compensate for characteristics of the display system. The geometric transformation module may include a spatial transformation module for redefining spatial relationships between image pixels, an alignment and rotation correction module for repositioning image pixels, a focus correction module for correcting image defocus, a distortion correction module for correcting image distortions, and a multi-frame correlation module for performing motion-compensated frame rate conversion.

Claims

1. An image processing apparatus for receiving bitstream data and processing said bitstream data to provide video stream image data to a display device, comprising:

a display input processor (DIP) coupled to a databus, said DIP comprising an input data connector and a first plurality of processing modules configured to receive bitstream data input and reconstruct said input to generate DIP outputs;

a display output processor (DOP) coupled to said databus, said DOP comprising a second plurality of processing modules configured to process said DIP outputs for generating DOP outputs, said second plurality comprising a geometric transformation (GT) module and a post GT filtering module; and

a buffer memory, coupled to said databus, configured to store said DIP outputs and said DOP outputs, and to provide said video stream image data to said display device.

2. The apparatus of claim 1 wherein said DOP comprises a display map memory (DMM).

3. The apparatus of claim 2 wherein said DMM is configured to store system configuration information which includes intensity values for setup of said display device.

4. The apparatus of claim 1 wherein said geometric transformation (GT) module is configured to geometrically transform said DIP inputs.

5. The apparatus of claim 4 wherein said GT module comprises:

a spatial transformation module configured to redefine spatial relationships between image pixels;

an alignment and rotation correction module configured to reposition image pixels;

a focus correction module configured to correct image defocus; and

a distortion correction module configured to correct image distortions.

6. The apparatus of claim 5 wherein said alignment and rotation correction module is configured to rotate images.

7. The apparatus of claim 5 wherein said focus correction module is configured to correct said image for defocus resulting from image deformation and from display optics.

8. The apparatus of claim 5 wherein said spatial transformation module is configured to use frame information and motion tracking information from multiple input images to increase image resolution.

9. The apparatus of claim 8 wherein said spatial transformation module is configured to select motion tracking information from either a compressed bitstream or a motion estimator output.

10. The apparatus of claim 4 wherein said GT module is configured to improve skew, tangential symmetry, aspect angle, and scale-related distortions of said display images.

11. The apparatus of claim 4 wherein said GT module is configured to correct environment-introduced image artifacts.

12. The apparatus of claim 4 wherein said GT module is configured to correct artifacts resulting from non-uniformity of the display device.

13. The apparatus of claim 4 wherein said GT module comprises a texture mapping module.

14. The apparatus of claim 4 wherein said DOP is configured to use a mathematical formula for providing DOP outputs suitable for a panoramic projection.

15. The texture mapping module of claim 14, where said module is configured to use texture mapping to perform transitions for multi-picture displays.

16. The apparatus of claim 4 wherein said GT module comprises a multi-frame correlation module.

17. The apparatus of claim 16 wherein said multi-frame correlation module is configured to select motion compensation information from either a selected display image or a motion estimator output.

18. The apparatus of claim 1 wherein said DIP is configured to receive data as a coded bitstream, said bitstream comprising image object information, image object depths, and image motion tracking information.

19. The apparatus of claim 18 configured to provide image data for a 3D and/or a panoramic display device.

20. The apparatus of claim 19 configured to use said image object information to reposition objects in output coordinates of said panoramic display device.

21. The apparatus of claim 19 configured to output image data to film.

22. The apparatus of claim 19 configured to receive a coded input that represents two images and use said coded input to present a 3D stereoscopic image on said display device.

23. The apparatus of claim 1 configured to simultaneously receive multiple video streams and process such streams to provide an image from each video stream in a single display using Picture-in-Picture and windowing controls.

24. The apparatus of claim 23 wherein said GT module is configured to perform transition effects between the different video streams, such transition effect including fades, blends, wipes and warps.

25. The apparatus of claim 1 wherein said DIP comprises an image reconstruction module configured for performing multi-frame reconstruction to increase image resolutions.

26. The apparatus of claim 25, wherein said image reconstruction module is configured to use motion estimation vectors from an input bitstream to correlate multiple images.

27. An apparatus configured for processing bitstream data to form video stream image data for use in a display system, comprising:

a display device, coupled to said display system, for viewing image data;

a geometric transformation GT module coupled to said display device, said GT module configured to precondition said bitstream data using geometric transformations to compensate for characteristics of said display device; and

a temporal gamma processing TGP module coupled to said display device, said TGP module configured to independently determine an output intensity value for each color component output to said display device.

28. The TGP module of claim 27 comprising a plurality of look-up tables, wherein said TGP is configured to use at least one of said plurality of tables for determining color correction.

29. The apparatus of claim 27 wherein said geometric transformation module comprises a spatial transformation module configured for redefining spatial relationships between image pixels derived from said bitstream information.

30. The apparatus of claim 27 wherein said geometric transformation module comprises an alignment and rotation correction module configured for repositioning said image pixels.

31. The apparatus of claim 27 wherein said geometric transformation module comprises a focus correction module configured for correcting image defocus.

32. The apparatus of claim 27 wherein said geometric transformation module comprises a distortion correction module configured for correcting image distortions.

33. The apparatus of claim 27 wherein said geometric transformation module comprises a multi-frame correlation module configured for performing motion-compensated frame rate conversion.

34. The apparatus of claim 27 wherein said geometric transformation module is configured to improve skew, tangential symmetry, aspect angle, and scale-related distortions of said image data.