Image construction based video display system

• US 8,970,646 B2
• Filed: 07/08/2009
• Issued: 03/03/2015
• Est. Priority Date: 07/09/2008
• Status: Active Grant

• Current Assignee: Ostendo Technologies Inc.
• Original Assignee: Ostendo Technologies Inc.
• Inventors: Guncer, Selim E.
• Primary Examiner(s): Boddie, William
• Assistant Examiner(s): ENGLISH, ALECIA DIANE
• Application Number:
  US12/499,560
• Publication Number:
  US 20100007804A1
• Time in Patent Office: 2,064 Days
• Field of Search: None
• US Class Current: 345/694
• CPC Class Codes: G09G 2340/02 Handling of images in compressed format, e.g. JPEG, MPEG
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 3/2018 by time modulation using two or more time intervals
• CPC Class Codes: G09G 3/2085 Special arrangements for addressing the individual elements of the matrix, other than by driving respective rows and columns in combination
• CPC Class Codes: G09G 3/3426 the different display panel areas being distributed in two dimensions, e.g. matrix

Abstract

A video display system based on constructing images through displaying orthogonal basis function components of the image is disclosed. The system is comprised of two display components aligned and driven concurrently. The first display component is a coarse pixel array. The second display component is a spatial light modulator whose geometric details are finer than the first pixel array. The overall system reconstructs the intended video to be displayed at the finer geometric details of the second display component at a minimal image quality loss through the use of time-domain display of orthogonal image basis function components. The resultant system has a considerably reduced interconnection complexity and number of active circuit elements, and also requires a considerably smaller video data rate if a lossy image reconstruction scheme is used. An embodiment with a LED based display and an LCD based spatial light modulator utilizing the concepts, and methods to drive the displays are described herein.

Claims

1. A video system comprised of:

a video display having an array of M×

N coarse pixels in which each coarse pixel is comprised of a set of primary color light sources for color operation, or a white light source for gray-scale operation, wherein the intensity of each light source is controllable;

a spatial light modulator aligned with the array of M×

N coarse pixels to generate spatial masking patterns for blocking or passing light, the spatial masking patterns having a resolution finer than the coarse pixel sizes by a factor of p;

an image processor coupled to receive video image information to be displayed, the image processor being configured so that, for each video image, the following is carried out;

generating, for each coarse pixel and for each color to be displayed, a sequence of Walsh orthogonal function image components (D^cuv), each Walsh orthogonal function only having a value of −

1 or +1, each image component being determined from the video image information (f^c(x,y)) and a corresponding masking pattern of the sequence of masking patterns corresponding to the Walsh orthogonal function image components (D^cuv), where u and v are indices for the basis functions and x and y are the coordinates of the video image pixels,

for any image components other than D^c₀₀ that are negative, using the absolute value of the image component and using the inverse of the corresponding masking pattern;

correcting the D^c₀₀ image component by subtracting one half the summation of D^c_uv over all D^c_uv,

controlling the spatial light modulator to generate a sequence of spatial masking patterns for each coarse pixel,

and providing driving information for the light source or light sources for each color to be displayed in each of the M×

N coarse pixels corresponding to the sequence of image components (D^c_uv) for the respective color, so that the light source or light sources is/are driven with the light strength proportional to an image component (D^c_uv)while the corresponding masking pattern is illuminated;

whereby the video system can display video images at a resolution up to p times finer than the M×

N coarse pixels.

2. The video system of claim 1 wherein the light sources are primary color solid state light sources.

3. The video system of claim 2 wherein the primary color solid state light sources are red, green and blue LED light sources.

4. The video system of claim 1 wherein the spatial light modulator is an active or passive matrix liquid crystal spatial light modulator.

5. The video system of claim 1 wherein the spatial light modulator is configured to simultaneously generate the same spatial masking patterns for all coarse pixels.

6. The video system of claim 1 wherein the spatial light modulator is configured to simultaneously generate the same spatial masking patterns for an array of multiple coarse pixels, the array of multiple coarse pixels being a sub-array of the array of M×

N coarse pixels, whereby timing of the spatial masking patterns will be simultaneous for each coarse pixel within any one sub-array, but the timing of patterns within different sub-arrays is different.

7. The video system of claim 6 in which the number of image components to be used to reproduce an image for any given coarse pixel on the display is dynamically determined in the image processor through the use of certain thresholds below which the component is discarded when displaying the coarse pixel.

8. The video system of claim 1 wherein the spatial light modulator is configured to separately generate spatial masking patterns for each coarse pixel, the timing of patterns for different coarse pixels being different.

9. The video system of claim 1 wherein the spatial masking patterns have lower order and higher order spatial frequency components, and wherein the image processor allocates more time to the spatial masking patterns having lower order spatial frequency components and less time to the spatial masking patterns having higher order spatial frequency components.

10. The video system of claim 1 wherein the spatial masking patterns have lower order and higher order spatial frequency components, and wherein the image processor is configured to ignore at least one higher order spatial masking pattern at least once.

11. The video system of claim 10 wherein the image processor allocates more time to at least one of the non-ignored spatial masking patterns when ignoring at least one higher order spatial masking pattern.

12. The video system of claim 10 wherein the at least one higher order spatial masking pattern to be ignored is chosen by the image processor responsive to the image component for that spatial masking pattern.

13. The video system of claim 1 wherein the spatial masking patterns have lower order and higher order spatial frequency components, and wherein the image processor is configured to reduce a video data rate applied to the video system by using a subset of available image components corresponding to the lower order spatial frequency components.

14. The video system of claim 1 wherein the spatial masking patterns have lower order and higher order spatial frequency components, and in which the image components for each coarse pixel are described with bit precision determined by a quantization matrix that allocates more bits to image components associated with lower order masking patterns, and less bits to image components associated with higher order masking patterns, thereby reducing a total video data rate.

15. A method of displaying a video image, the video image being a frame of a video or a still image, the method comprising:

providing a video display having an array of M×

N coarse pixels in which each coarse pixel is comprised of a set of primary color light sources for color operation, or a white light source for gray-scale operation;

providing a spatial light modulator aligned with the array of M×

N coarse pixels to generate spatial masking patterns for blocking or passing light of the light sources, the spatial masking patterns having a resolution finer than the coarse pixel sizes by a factor of p;

generating, for each coarse pixel and for each color to be displayed, a sequence of Walsh function orthogonal image components (D^c_uv) where u and v are indices for the basis function, each Walsh orthogonal function only having a value of −

1 or +1, each image component being determined from the video image information (f^c(x,y)) and a corresponding masking pattern of the sequence of masking patterns corresponding to the Walsh orthogonal function image components (D^c_uv), where u and v are indices for the basis functions and x and y are the coordinates of the video image pixels;

for any image components other than D^c₀₀that are negative, using the absolute value of the image component and using the inverse of the corresponding masking pattern;

correcting for each color to be displayed, the D^c₀₀ image component by subtracting one half the summation of all D^c_uv for the respective color,

controlling the spatial light modulator to generate a sequence of spatial masking patterns for each coarse pixel,

and providing driving information for the light source or light sources for each color to be displayed in each of the M×

N coarse pixels corresponding to the sequence of image components (D^c_uv) for the respective color, so that the light source or light sources is/are driven with the light strength proportional to an image component (D^c_uv)while the corresponding masking pattern is illuminated;

whereby the video image is displayed at a resolution up to p times finer than the M×

N coarse pixels.

16. The method of claim 15 wherein the light sources are primary color solid state light sources.

17. The method of claim 16 wherein the primary color solid state light sources are red, green and blue LED light sources.

18. The method of claim 15 wherein an active or passive matrix liquid crystal spatial light modulator is used.

19. The method of claim 15 wherein the same spatial masking patterns for all coarse pixels are simultaneously generated.

20. The method of claim 15 wherein the same spatial masking patterns are simultaneously generated for an array of multiple coarse pixels, the array of multiple coarse pixels being a sub-array of the array of M×

N coarse pixels, whereby timing of the spatial masking patterns will be simultaneous for each coarse pixel within any one sub-array, but the timing of each pattern within different sub-arrays is different.

21. The method of claim 20 in which the number of image components to be used to reproduce an image for any given coarse pixel is dynamically determined through the use of certain thresholds below which the component is discarded when displaying the subarray.

22. The method of claim 15 wherein spatial masking patterns for each coarse pixel are separately generated, the timing of each pattern for different coarse pixels being different.

23. The method of claim 15 wherein the spatial masking patterns have lower order and higher order spatial frequency components, and wherein more time is allocated to the spatial masking patterns having lower order spatial frequency components and less time to the spatial masking patterns having higher order spatial frequency component.

24. The method of claim 15 wherein the spatial masking patterns have lower order and higher order spatial frequency components, and wherein at least one higher order spatial masking pattern is ignored at least once.

25. The method of claim 24 wherein more time is allocated to at least one of the non-ignored spatial masking patterns when ignoring at least one higher order spatial masking pattern.

26. The method of claim 24 wherein the at least one higher order spatial masking pattern to be ignored is chosen responsive to the image component for that spatial masking pattern.

27. The method of claim 15 wherein the spatial masking patterns have lower order and higher order spatial frequency components, and wherein the video data rate is reduced by using a subset of available image components corresponding to the lower order spatial frequency components.

28. The method of claim 15 wherein the spatial masking patterns have lower order and higher order spatial frequency components, and in which the image components for each coarse pixel are described with bit precision determined by a quantization matrix that allocates more bits to image components associated with lower order masking patterns, and less bits to image components associated with higher order masking patterns, thereby reducing a total video data rate.