Image Construction Based Video Display System

• US 20100007804A1
• Filed: 07/08/2009
• Published: 01/14/2010
• Est. Priority Date: 07/09/2008
• Status: Active Grant

• Current Assignee: Ostendo Technologies Inc.
• Original Assignee: Ostendo Technologies Inc.
• Inventors: Guncer, Selim E.
• Granted Patent: US 8,970,646 B2
• Time in Patent Office: Days
• US Class Current: 348/790
• CPC Class Codes: G09G 2340/02 Handling of images in compressed format, e.g. JPEG, MPEG
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 3/2018 by time modulation using two or more time intervals
• CPC Class Codes: G09G 3/2085 Special arrangements for addressing the individual elements of the matrix, other than by driving respective rows and columns in combination
• CPC Class Codes: G09G 3/3426 the different display panel areas being distributed in two dimensions, e.g. matrix

Abstract

A video display system based on constructing images through displaying orthogonal basis function components of the image is disclosed. The system is comprised of two display components aligned and driven concurrently. The first display component is a coarse pixel array. The second display component is a spatial light modulator whose geometric details are finer than the first pixel array. The overall system reconstructs the intended video to be displayed at the finer geometric details of the second display component at a minimal image quality loss through the use of time-domain display of orthogonal image basis function components. The resultant system has a considerably reduced interconnection complexity and number of active circuit elements, and also requires a considerably smaller video data rate if a lossy image reconstruction scheme is used. An embodiment with a LED based display and an LCD based spatial light modulator utilizing the concepts, and methods to drive the displays are described herein.

Claims

1. A video system comprised of:

a video display having an array of M×

N coarse pixels in which each coarse pixel is comprised of a set of primary color light sources for color operation, or a white light source for gray-scale operation, wherein the intensity of each light source is controllable;

a spatial light modulator aligned with the array of M×

N coarse pixels first video display to generate spatial masking patterns for blocking or passing light, the spatial masking patterns having a resolution finer than the coarse pixel sizes by a factor of p;

an image processor coupled to receive video information to be displayed, to control the spatial light modulator to generate the spatial masking patterns and to generate image components to provide driving information for the light source or light sources in each of the M×

N coarse pixels corresponding to the spatial masking patterns generated by the spatial light modulator;

whereby the resultant video system can display images at a resolution up to p times finer than the M×

N coarse pixels.

2. The video system of claim 1 wherein the video system can display video and still images.

3. The video system of claim 1 wherein the image processor decomposes the video information into components proportional to a series of orthogonal image basis functions for each coarse pixel, and the spatial light modulator generates spatial masking patterns corresponding to orthogonal basis functions for image reconstruction.

4. The video system of claim 3 wherein, for gray-scale images, the image processor calculates the image components D_uv related to the image f^c (x,y) for each course pixel, where u and v are indices for the basis function and x and y are the coordinates of the coarse pixel, applies a light intensity mask through the use of a spatial light modulator corresponding to w*_uv(x,y), where w*_uv(x,y)=(w_uv*x,y)+1)/2 and w is an orthogonal basis function, and provides driving information proportional to D_uv for the light source for each coarse pixel.

5. The video system of claim 4 wherein for image components D_uv that are negative, the inverse of the respective masking pattern is used.

6. The video system of claim 3 wherein, for color images, the image processor calculates the image components D^C_uv related to the image f^c (x,y) for each color of each coarse pixel, where u and v are indices for the basis function and x and y are the coordinates of the coarse pixel, applies a light intensity mask through the use of a spatial light modulator corresponding to w*_uv(x,y), where w*_uv(x,y)=(w_uv(x,y)+1)/2 and w is an orthogonal basis function, and provides driving information proportional to D_uv for the respective color light source for each coarse pixel.

7. The video system of claim 6 wherein for image components D^C_uv that are negative, the inverse of the respective masking pattern is used.

8. The video system of claim 1 wherein the light sources are primary color solid state light sources.

9. The video system of claim 8 wherein the primary color solid state light sources are red, green and blue LED light sources.

10. The video system of claim 1 wherein the spatial light modulator is an active or passive matrix liquid crystal spatial light modulator.

11. The video system of claim 3 wherein the spatial light modulator is configured to simultaneously generate the same spatial masking patterns for all coarse pixels.

12. The video system of claim 3 wherein the spatial light modulator is configured to simultaneously generate the same spatial masking patterns for an array of multiple coarse pixels, the array of multiple coarse pixels being a sub-array of the array of M×

N coarse pixels, whereby timing of the spatial masking patterns will be simultaneous for each coarse pixel within any one sub-array, but the timing of each pattern within different sub-arrays may be different.

13. The video system of claim 3 wherein the spatial light modulator is configured to separately generate spatial masking patterns for each coarse pixel, whereby the timing of each pattern for different coarse pixels may be different.

14. The video system of claim 3 wherein the spatial masking patterns have lower order and higher order spatial frequency components, and wherein the image processor allocates more time to the spatial masking patterns having lower order spatial frequency components and less time to the spatial masking patterns having higher order spatial frequency components.

15. The video system of claim 3 wherein the spatial masking patterns have lower order and higher order spatial frequency components, and wherein the image processor is configured to ignore at least one higher order spatial masking pattern at least once.

16. The video system of claim 15 wherein the image processor allocates more time to at least one of the non-ignored spatial masking patterns when ignoring at least one higher order spatial masking pattern.

17. The video system of claim 15 wherein the at least one higher order spatial masking pattern to be ignored is chosen by the image processor responsive to the image component for that spatial masking pattern.

18. The video system of claim 1 wherein the spatial masking patterns have lower order and higher order spatial frequency components, and wherein the image processor is configured to reduce a video data rate applied to the video system by using a subset of available image components corresponding to the lower order spatial frequency components.

19. The video system of claim 12 in which the number of image components to be used to reproduce an image for any given coarse pixel on the display is dynamically determined in the image processor through the use of certain thresholds below which the component is discarded when displaying the subarray.

20. The video system of claim 3 wherein the spatial masking patterns have lower order and higher order spatial frequency components, and in which the image components for each coarse pixel are described with bit precision determined by a quantization matrix that allocates more bits to image components associated with lower order masking patterns, and less bits to image components associated with higher order masking patterns, thereby reducing a total video data rate.

21. A method of displaying a video image comprising:

separating a video image into an array of M×

N coarse pixels in which each coarse pixel is comprised of a set of primary color light sources for color operation, or a white light source for gray-scale operation;

providing a spatial light modulator aligned with the array of M×

N coarse pixels first video display to generate spatial masking patterns for blocking or passing light, the spatial masking patterns having a resolution finer than the coarse pixel sizes by a factor of p;

controlling the spatial light modulator to generate the spatial masking patterns and to generate image components to provide driving information for the light source or light sources in each of the M×

N coarse pixels corresponding to the spatial masking patterns generated by the spatial light modulator;

whereby the resultant video images are displayed at a resolution up to p times finer than the M×

N coarse pixels.

22. The method of claim 21 wherein the video information is decomposed into components proportional to a series of orthogonal image basis functions for each coarse pixel, and the spatial light modulator is controlled to generate spatial masking patterns corresponding to orthogonal basis functions for image reconstruction.

23. The method of claim 22 wherein, for gray-scale images, image components D_uv related to the image f^c (x,y) are calculated for each coarse pixel, where u and v are indices for the basis function and x and y are the coordinates of the coarse pixel, a light intensity mask is applied through the use of a spatial light modulator corresponding to w*_uv(x,y), where w*_uv(x,y)=(w+1)/2 and w is an orthogonal basis function, and driving information proportional to D_uv is applied to the light source for each coarse pixel is provided to the light sources.

24. The method of claim 23 wherein for image components D_uv that are negative, the inverse of the respective masking pattern is used.

25. The method of claim 22 wherein, for color images, the image components D^C_uv related to the image f^c (x,y) for each color of each coarse pixel are calculated, where u and v are indices for the basis function and x and y are the coordinates of the coarse pixel, a light intensity mask through the use of a spatial light modulator corresponding to w*_uv(x,y) is applied, where w*_uv(x,y)=(w_uv(x,y)+1)/2 and w is an orthogonal basis function, and driving information proportional to D_uv for the respective color light source for each coarse pixel is provided to the light sources.

26. The method of claim 25 wherein for image components D_uv that are negative, the inverse of the respective masking pattern is used.

27. The method of claim 21 wherein the light sources are primary color solid state light sources.

28. The method of claim 27 wherein the primary color solid state light sources are red, green and blue LED light sources.

29. The method of claim 21 wherein an active or passive matrix liquid crystal spatial light modulator is used.

30. The method of claim 22 wherein the same spatial masking patterns for all coarse pixels are simultaneously generated.

31. The method of claim 22 wherein the same spatial masking patterns are simultaneously generated for an array of multiple coarse pixels, the array of multiple coarse pixels being a sub-array of the array of M×

N coarse pixels, whereby timing of the spatial masking patterns will be simultaneous for each coarse pixel within any one sub-array, but the timing of each pattern within different sub-arrays may be different.

32. The method of claim 22 wherein spatial masking patterns for each coarse pixel are separately generated, whereby the timing of each pattern for different coarse pixels may be different.

33. The method of claim 22 wherein the spatial masking patterns have lower order and higher order spatial frequency components, and wherein more time is allocated to the spatial masking patterns having lower order spatial frequency components and less time to the spatial masking patterns having higher order spatial frequency component.

34. The method of claim 22 wherein the spatial masking patterns have lower order and higher order spatial frequency components, and wherein at least one higher order spatial masking pattern is ignored at least once.

35. The method of claim 34 wherein more time is allocated to at least one of the non-ignored spatial masking patterns when ignoring at least one higher order spatial masking pattern.

36. The method of claim 34 wherein the at least one higher order spatial masking pattern to be ignored is chosen responsive to the image component for that spatial masking pattern.

37. The method of claim 22 wherein the spatial masking patterns have lower order and higher order spatial frequency components, and wherein the video data rate is reduced by using a subset of available image components corresponding to the lower order spatial frequency components.

38. The method of claim 31 in which the number of image components to be used to reproduce an image for any given coarse pixel is dynamically determined through the use of certain thresholds below which the component is discarded when displaying the subarray.

39. The method of claim 21 wherein the spatial masking patterns have lower order and higher order spatial frequency components, and in which the image components for each coarse pixel are described with bit precision determined by a quantization matrix that allocates more bits to image components associated with lower order masking patterns, and less bits to image components associated with higher order masking patterns, thereby reducing a total video data rate.