Methods, systems, and programming for producing and displaying subpixel-optimized images and digital content including such images

• US 7,737,993 B2
• Filed: 11/03/2003
• Issued: 06/15/2010
• Est. Priority Date: 05/02/2001
• Status: Expired due to Fees

• Current Assignee: Marlborough Software Development Holdings Inc. (Pageflex, Inc.)
• Original Assignee: Edward W. Porter, John S. Collins, Sampo J. Kaasila
• Inventors: Collins, John S., Kaasila, Sampo J., Porter, Edward W.
• Primary Examiner(s): Caschera; Antonio A
• Application Number:
  US10/700,376
• Publication Number:
  US 20050062758A1
• Time in Patent Office: 2,416 Days
• Field of Search: 345/472.1, 345/612, 345/613, 345/670
• US Class Current: 345/613
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06T 11/203 Drawing of straight lines or curves
• CPC Class Codes: G06T 11/40 Filling a planar surface by adding surface attributes, e.g. colour or texture
• CPC Class Codes: G06T 2200/12 involving antialiasing
• CPC Class Codes: G09G 2320/0606 Manual adjustment
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/0414 Vertical resolution change
• CPC Class Codes: G09G 2340/0421 Horizontal resolution change
• CPC Class Codes: G09G 2340/045 Zooming at least part of an image, i.e. enlarging it or shrinking it
• CPC Class Codes: G09G 2340/0457 Improvement of perceived resolution by subpixel rendering
• CPC Class Codes: G09G 2340/0492 Change of orientation of the displayed image, e.g. upside-down, mirrored
• CPC Class Codes: G09G 2340/145 related to small screens
• CPC Class Codes: G09G 2370/027 Arrangements and methods specific for the display of internet documents
• CPC Class Codes: G09G 3/2003 Display of colours specific for liquid crystal displays G09G3/3607
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: G09G 5/026 Control of mixing and/or overlay of colours in general G09G5/022 and G09G5/024 take precedence
• CPC Class Codes: G09G 5/24 Generation of individual character patterns
• CPC Class Codes: G09G 5/28 : for enhancement of character form, e.g. smoothing
• CPC Class Codes: G09G 5/34 : for rolling or scrolling
• CPC Class Codes: G09G 5/42 : characterised by the display of patterns using a display memory without fixed position correspondence between the display memory contents and the display position on the screen
• CPC Class Codes: H04M 1/72445 : for supporting Internet browser applications

Abstract

Methods, systems, and programming for producing and displaying a scaled-down representation of subpixel-optimized images and digital content including such images that is represented by a mark-up language. Some embodiments display the text in such digital content using a different subpixel optimization than that used for the images. Others produce the subpixel-optimized images by calculating the luminosity of a subpixel as a function of the length of a plurality of coverage lines within a window in a source image corresponding to the subpixel that is covered by source image pixels having the subpixel'"'"'s color. Some embodiments calculate the luminosity of a subpixel in such a subpixel-optimized image as a function both of the average luminosity of pixels in the subpixel'"'"'s source image window and as a function of any color balancing distribution between resulting subpixel luminosities necessary to reduce color imbalance.

Claims

1. A method of displaying, on a subpixel addressable screen of an image displaying device having pixels comprised of separately-addressable, differently-colored subpixels, digital content including text and/or images represented by a mark-up language including tags that dictate the format in which such content is to be displayed and tags that identify images to be displayed as part of said content, said method comprising:

accessing said digital content, including accessing one or more images, from one or more devices in which said content is stored or is generated dynamically;

performing a down-scaling and subpixel-optimization process in which;

each of said accessed images is a source image for the process;

said process produces from each such source image a corresponding scaled-down, subpixel optimized image;

each such scaled image represents the source image from which it has been produced at a lower pixel resolution than the pixel resolution of said source image;

each such scaled image also represents the corresponding source image in a subpixel-optimized manner by causing the luminosity associated with each subpixel within a given pixel of the scaled image to represent the luminosity of the subpixel'"'"'s color in a portion of the corresponding source image that differs for each subpixel as a function of the subpixel'"'"'s different position in the given pixel; and

displaying said accessed digital content on said subpixel-addressable screen in a format determined at least in part by one or more tags in said content, including, as part of said formatted display, displaying said scaled images in a subpixel-optimized manner on said screen;

wherein;

said accessed digital content includes one or more strings of displayable text characters; and

said display of said digital content also includes drawing a string bitmap on said screen to represent said string, the string bitmap being composed from a succession of separate font bitmaps each selected to correspond to an individual character in the text of said string;

said font bitmaps are anti-aliased subpixel-optimized images of character-font shapes; and

said subpixel-optimized character-font shape images are subpixel-optimized in a different manner than the scaled images in that;

the luminosity assigned to each given subpixel of a pixel in one of said scaled images represents the intensity of the given subpixel'"'"'s corresponding color in a portion of the source image having a position corresponding to the position of the given subpixel in the scaled image; and

the luminosity assigned to each given subpixel in one of said font bitmaps is a function of;

a coverage value representing the percent of the given subpixel'"'"'s area in the font bitmap that is covered by the character-font shape represented by the bitmap; and

for at least some subpixels of said font bitmaps, a color balancing distribution between the given subpixel'"'"'s coverage value and coverage values of other nearby subpixels that reduces perceptible color imbalances that would result from differences between coverage values of nearby subpixels of different colors in the absence of such color balancing distributions.

2. A method as in claim 1 wherein one or more of said strings are represented by font bitmaps that:

have a font size of 10 pixels per em or less;

are anti-aliased font bitmaps that assign a color value to a given screen pixel as a graded function of a coverage value representing the percent of the given pixel that is covered by a character shape being represented by the font bitmap; and

have the shape and pixel alignment of the character represented by such a font bitmap selected to increase the degree of alignment of edges of the character shape with pixel boundaries of said bitmap.

3. A method as in claim 2 wherein:

said anti-aliased font bitmaps include small font bitmaps having a small font size of eight pixels per em or less; and

the shape and pixel alignment of the character represented by such a small font bitmap have been selected to increase the degree of alignment of edges of the character shape with pixel boundaries of said small bitmap.

4. A method as in claim 3 wherein the font bitmaps of said small font size represent a majority of characters of the Roman alphabet within an advance width of 4 pixel columns or less.

5. A method as in claim 4 wherein the font bitmaps of said small font size represent a majority of lowercase letters with an x-height greater than 4 pixels.

6. A method as in claim 1 wherein said accessing of said digital content is performed over an computer network.

7. A method as in claim 6 wherein said accessing of said digital content is performed over the Internet.

8. A method as in claim 1 wherein said digital content includes web pages.

9. A method as in claim 1 wherein:

said screen is part of a browser computer capable of browsing digital content;

the browser computer includes browser programming that responds to user input requesting a given portion of digital content by requesting that content from another entity, either a storage device, another computer, or other programming running on the browser computer;

said accessing of said digital content is performed in response to the request from the browser programming; and

said display of said accessed digital content, including said scaled images, is performed on the screen of said browser computer.

10. A method as in claim 9 wherein:

said user request is communicated over a computer network from said browser computer to one or more servers;

the digital content is accessed by being read from memory or dynamically generated by one or more of said servers;

said down-scaling and subpixel-optimization of said source images is performed by one or more of said servers; and

said digital content, including said scaled images, is downloaded over said computer network to said browser, which then performs said display of the digital content.

11. A method as in claim 9 wherein:

said browser computer communicates said user request over a computer network to a proxy server;

said proxy server communicates said user request over a computer network to one or more servers that store or dynamically generate said digital content, including said source images;

said one or more servers sends said source image to said proxy server;

said scaling down of said source images is performed by said proxy server; and

the proxy server downloads the digital content, including said scaled images to said browser, which then performs said display of the digital content.

12. A method as in claim 9 wherein said down-scaling and subpixel-optimization of said source images is performed by said browser computer.

13. A method as in claim 9 further including:

allowing a user of the browser computer to select one of a plurality of scale factors; and

said down-scaling and subpixel-optimizing produces one or more scaled, subpixel-optimized images that each have a pixel size relative to the respective source image that is selected as a function of said user selected scale factor.

14. A method as in claim 13 wherein

the size of said font bitmaps varies as a function of said user selected scale factor.

15. A method as in claim 13 wherein:

said other entity from which said browser programming requests said digital content is one or more remote computers that said browser programming communicates said request to over a computer network;

said user selected scale factor is also communicated from said browser programming over said computer network to one or more of said remote computers; and

said scaling down and subpixel optimizing of said source images is performed on one or more of said remote computers at a scale factor that varies as a function of said user selected scale factor communicated over said network.

16. A method as in claim 1 wherein the percent of a given font bitmap subpixel'"'"'s coverage value that is distributed to achieve color balance is a function of the percent of the given subpixel'"'"'s coverage value that causes color imbalance within pixel of which the given subpixel is part.

17. A method of displaying, on a subpixel addressable screen of an image displaying device having pixels comprised of separately-addressable, differently-colored subpixels, digital content including text and/or images represented by a mark-up language including tags that dictate the format in which such content is to be displayed and tags that identify images to be displayed as part of said content, said method comprising:

accessing said digital content, including accessing one or more images, from one or more devices in which said content is stored or is generated dynamically;

performing a down-scaling and subpixel-optimization process in which;

each of said accessed images is a source image for the process;

said process produces from each such source image a corresponding scaled-down, subpixel optimized image;

each such scaled image represents the source image from which it has been produced at a lower pixel resolution than the pixel resolution of said source image;

each such scaled image also represents the corresponding source image in a subpixel-optimized manner by causing the luminosity associated with each subpixel within a given pixel of the scaled image to represent the luminosity of the subpixel'"'"'s color in a portion of the corresponding source image that differs for each subpixel as a function of the subpixel'"'"'s different position in the given pixel; and

displaying said accessed digital content on said subpixel-addressable screen in a format determined at least in part by one or more tags in said content, including, as part of said formatted display, displaying said scaled images in a subpixel-optimized manner on said screen;

wherein;

said accessed digital content includes one or more strings of displayable text characters; and

said display of said digital content also includes drawing a string bitmap on said screen to represent said string, the string bitmap being composed from a succession of separate font bitmaps each selected to correspond to an individual character in the text of said string;

one or more of said strings are represented by font bitmaps that;

have a font size of 10 pixels per em or less;

are anti-aliased font bitmaps that assign a color value to a given screen pixel as a graded function of a coverage value representing the percent of the given pixel that is covered by a character shape being represented by the font bitmap; and

have the shape and pixel alignment of the character represented by such a font bitmap selected to increase the degree of alignment of edges of the character shape with pixel boundaries of said bitmap;

said font bitmaps are anti-aliased subpixel-optimized images of character-font shapes; and

said subpixel-optimized font character-images shape are subpixel-optimized in a different manner than the scaled images in that;

the luminosity assigned to each given subpixel of a pixel in one of said scaled images represents the intensity of the given subpixel'"'"'s corresponding color in a portion of the source image having a position corresponding to the position of the given subpixel in the scaled image; and

the luminosity assigned to each given subpixel in one of said font bitmaps is a function of;

a coverage value representing the percent of the given subpixel'"'"'s area in the font bitmap that is covered by the character-font shape represented by the bitmap; and

for at least some subpixels of said font bitmaps, a color balancing distribution between the given subpixel'"'"'s coverage value and coverage values of other nearby subpixels that reduces perceptible color imbalances that would result from differences between coverage values of nearby subpixels of different colors in the absence of such color balancing distributions.

18. A method as in claim 17 wherein the percent of a given font bitmap subpixel'"'"'s coverage value that is distributed to achieve color balance is a function of the percent of the given subpixel'"'"'s coverage value that causes color imbalance within pixel of which the given subpixel is part.

19. A computer system for displaying, on a subpixel addressable screen having pixels comprised of separately-addressable, differently-colored subpixels, digital content including text and/or images represented by a mark-up language including tags that dictate the format in which such content is to be displayed and tags that identify images to be displayed as part of said content, said system comprising:

one or more processors for executing program instructions stored in computer readable memory; and

one or more computer readable memories storing program instructions for causing said one or more processors to;

access said digital content, including accessing one or more images, from one or more devices in which said content is stored or is generated dynamically;

perform a down-scaling and subpixel-optimization process in which;

each of said accessed images is a source image for the process;

said process produces from each such source image a corresponding scaled-down, subpixel optimized image;

each such scaled image represents the source image from which it has been produced at a lower pixel resolution than the pixel resolution of said source image;

each such scaled image also represents the corresponding source image in a subpixel-optimized manner by causing the luminosity associated with each subpixel within a given pixel of the scaled image to represent the luminosity of the subpixel'"'"'s color in a portion of the corresponding source image that differs for each subpixel as a function of the subpixel'"'"'s different position in the given pixel; and

display said accessed digital content on said subpixel-addressable screen in a format determined at least in part by one or more tags in said content, including, as part of said formatted display, displaying said scaled images in a subpixel-optimized manner on said screen

wherein;

said accessed digital content includes one or more strings of displayable text characters; and

said display of said digital content also includes drawing a string bitmap on said screen to represent said string, the string bitmap being composed from a succession of separate font bitmaps each selected to correspond to an individual character in the text of said string;

said font bitmaps are anti-aliased subpixel-optimized images of character-font shapes; and

said subpixel-optimized character-font shape images are subpixel-optimized in a different manner than the scaled images in that;

the luminosity assigned to each given subpixel of a pixel in one of said scaled images represents the intensity of the given subpixel'"'"'s corresponding color in a portion of the source image having a position corresponding to the position of the given subpixel in the scaled image; and

the luminosity assigned to each given subpixel in one of said font bitmaps is a function of;

a coverage value representing the percent of the given subpixel'"'"'s area in the font bitmap that is covered by the character-font shape represented by the bitmap; and

for at least some subpixels of said font bitmaps, a color balancing distribution between the given subpixel'"'"'s coverage value and coverage values of other nearby subpixels that reduces perceptible color imbalances that would result from differences between coverage values of nearby subpixels of different colors in the absence of such color balancing distributions.

20. A computer system as in claim 19 wherein the percent of a given font bitmap subpixel'"'"'s coverage value that is distributed to achieve color balance is a function of the percent of the given subpixel'"'"'s coverage value that causes color imbalance within pixel of which the given subpixel is part.

21. A method of displaying, on a subpixel addressable screen of an image displaying device having pixels comprised of separately-addressable, differently-colored subpixels, digital content including text and/or images represented by a mark-up language including tags that dictate the format in which such content is to be displayed and tags that identify images to be displayed as part of said content, said method comprising:

accessing said digital content, including accessing one or more images, from one or more devices in which said content is stored or is generated dynamically;

performing a down-scaling and subpixel-optimization process in which;

each of said accessed images is a source image for the process;

said process produces from each such source image a corresponding scaled-down, subpixel optimized image;

each such scaled image represents the source image from which it has been produced at a lower pixel resolution than the pixel resolution of said source image;

each such scaled image also represents the corresponding source image in a subpixel-optimized manner by causing the luminosity associated with each subpixel within a given pixel of the scaled image to represent the luminosity of the subpixel'"'"'s color in a portion of the corresponding source image that differs for each subpixel as a function of the subpixel'"'"'s different position in the given pixel; and

displaying said accessed digital content on said subpixel-addressable screen in a format determined at least in part by one or more tags in said content, including, as part of said formatted display, displaying said scaled images in a subpixel-optimized manner on said screen;

wherein said down-scaling and subpixel-optimization process causes said luminosity associated with each subpixel of the scaled image to differ as a function of the subpixel'"'"'s different position by determining the luminosity of each given subpixel in a given pixel of said display image by;

defining a plurality of coverage lines within a window in said source image having a position relative to the source image corresponding to the given subpixel'"'"'s position relative to the display image, with the position of different source image windows associated with different subpixels of a given display image pixel differing as a function of the different positions of said subpixels within the given display image pixel;

determining which source image pixels overlaps each of said coverage lines within the given supixel'"'"'s source image window;

determining what length of each of said coverage lines is overlapped by each such overlapping source image pixel; and

determining the luminosity of the given subpixel as a function of the length of each coverage line overlapped by each such overlapping source image pixel and the respective luminosity, in the given subpixel'"'"'s color, of each such overlapping source image pixel.

22. A method as in claim 21 wherein the coverage lines associated with a given subpixel include at least two coverage lines that run in nonparallel directions on said subpixel'"'"'s source image window.

23. A method as in claim 21 wherein the source image window associated with each given subpixel has a size relative to the source image corresponding to the size of a whole pixel relative to said display image.

24. A method of displaying, on a subpixel addressable screen of an image displaying device having pixels comprised of separately-addressable, differently-colored subpixels, digital content including text and/or images represented by a mark-up language including tags that dictate the format in which such content is to be displayed and tags that identify images to be displayed as part of said content, said method comprising:

accessing said digital content, including accessing one or more images, from one or more devices in which said content is stored or is generated dynamically;

performing a down-scaling and subpixel-optimization process in which;

each of said accessed images is a source image for the process;

said process produces from each such source image a corresponding scaled-down, subpixel optimized image;

each such scaled image represents the source image from which it has been produced at a lower pixel resolution than the pixel resolution of said source image;

each such scaled image also represents the corresponding source image in a subpixel-optimized manner by causing the luminosity associated with each subpixel within a given pixel of the scaled image to represent the luminosity of the subpixel'"'"'s color in a portion of the corresponding source image that differs for each subpixel as a function of the subpixel'"'"'s different position in the given pixel; and

displaying said accessed digital content on said subpixel-addressable screen in a format determined at least in part by one or more tags in said content, including, as part of said formatted display, displaying said scaled images in a subpixel-optimized manner on said screen;

wherein said down-scaling and subpixel-optimization process causes said luminosity associated with each subpixel of the scaled image to differ as a function of the subpixel'"'"'s different position by;

defining a window in the corresponding source image having a position relative to the source image corresponding to the given subpixel'"'"'s position relative to the display image, with the position of different source image windows associated with different subpixels of a given display image pixel differing as a function of the different positions of said subpixels within the given display image pixel; and

determining the luminosity of each given subpixel in a pixel of said display image as a function of;

the whole pixel luminosity of each of one or more source image pixels that overlaps the given subpixel'"'"'s source image window;

the percent of the given subpixel'"'"'s source image window overlapped by each such overlapping source image pixel; and

a color balancing function that distributes subpixel luminosity values between nearby subpixels in the display image to reduce color imbalance.

25. A method as in claim 24 wherein the percent, if any, of a given subpixel'"'"'s luminosity value that is distributed by said color balancing varies as a function of the percent of said luminosity value that causes color imbalance within the pixel of which the given subpixel is part.

26. A method as in claim 24 wherein a source image window has a size relative to the source image corresponding to the size of a subpixel relative to the display image.

27. A method as in claim 26 wherein:

a plurality of coverage lines are defined within the given subpixel'"'"'s source image window;

the luminosity of the given subpixel is determined by;

determining which source image pixels overlaps each of said coverage lines within the given supixel'"'"'s source image window;

determining what length of each of said coverage lines is overlapped by each such overlapping source image pixel; and

determining the luminosity of the given subpixel as a function of the length of each coverage line overlapped by each such overlapping source image pixel and the respective whole pixel luminosity of each such overlapping source image pixel.

28. A method as in claim 26 wherein the luminosity of the given subpixel is determined by:

determining which source image pixels overlaps the given supixel'"'"'s source image window;

determining what area of the given subpixel'"'"'s source image window is overlapped by each such overlapping source image pixel; and

determining the luminosity of the given subpixel as a function of the area of the source image window overlapped by each such overlapping source image pixel and the respective whole pixel luminosity of each such overlapping source image pixel.