Computer screen image magnification system and method

• US 6,407,747 B1
• Filed: 05/07/1999
• Issued: 06/18/2002
• Est. Priority Date: 05/07/1999
• Status: Expired due to Term

• Current Assignee: CSR Technology Incorporated (Qualcomm, Inc.)
• Original Assignee: Picsurf, Inc.
• Inventors: Wang, Haishan, Chui, Charles K.
• Primary Examiner(s): Luu, Matthew
• Assistant Examiner(s): Chung, Daniel J
• Application Number:
  US09/307,547
• Time in Patent Office: 1,138 Days
• Field of Search: 345/660, 345/671, 345/668, 345/669, 345/667, 345/661
• US Class Current: 345/660
• CPC Class Codes: G06T 3/053 Detail-in-context presentations fisheye or wide-angle transformations G06T3/047
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/045 Zooming at least part of an image, i.e. enlarging it or shrinking it
• CPC Class Codes: G09G 5/391 Resolution modifying circuits, e.g. variable screen formats

Abstract

An image magnifying method and apparatus magnifies a portion of an image displayed on a computer display device. The user of the computer selects a first region (called a magnification window) of the displayed image, for instance using a mouse or trackball pointer device. Image data from the screen buffer for the user selected magnification window is copied to a first buffer. A magnified image is generated from the copied image data and the magnified image is stored in the screen buffer so as to replace the copied image data. When the user moves the screen cursor, or otherwise selects a second magnification window that overlaps with the first, the magnification application copies image data for a combined window, covering both the first and second magnification windows, from the screen buffer to a second buffer. The image data stored in the first buffer is copied into the portion of the second buffer corresponding to the first magnification window, at which point the second buffer contains unmagnified data for the entire combined window. The application generates a magnified image for the second magnification window and stores it in a portion of the second buffer corresponding to the second magnification window. Finally, the portion of the second image buffer corresponding to the combined window is copied into the corresponding portion of the screen buffer. As a result, the magnification window moves smoothly from the first to the second window, without creating any flashing artifacts.

Claims

1. A method of magnifying a portion of an image displayed on a computer display device, comprising:

displaying an image corresponding to image data stored in a screen buffer;

selecting a first region of the displayed image in accordance with user input;

copying image data from the screen buffer to a first buffer, the copied image data corresponding to the first selected region;

generating first magnified image data corresponding to a portion of the first selected region and storing the first magnified image data in the screen buffer so as to replace the copied image data;

receiving user input that selects a second region of the displayed image, the second selected region overlapping the first selected region;

copying image data from the screen buffer to a second buffer, the copied image data corresponding to a combined region that includes both the first and second selected regions;

copying the image data stored in the first buffer into a portion of the second buffer corresponding to the first selected region;

generating second magnified image data corresponding to a portion of the second selected region and storing the second magnified image data in a portion of the second buffer corresponding to the second selected region; and

copying into the screen buffer the image data and second magnified image data in the second buffer corresponding to the combined region.

2. The method of claim 1, wherein

the first and second magnified image data are each generated using a magnification filter that includes a lowpass filter, a highpass filter, and an adder that generates a weighted sum of image data generated by the lowpass and highpass filters.

3. The method of claim 2, wherein the first and second image data generated by the highpass filter are each weighted by a sharpness parameter S prior to summing by the adder with the image data generated by the lowpass filter, where S is a value between 0 and 1.

4. The method of claim 2, wherein the lowpass and highpass filters are both wavelet like convolution filters, wherein the lowpass filter has coefficients whose sum is equal to 1, and the highpass filter has coefficients whose sum is equal to zero.

24. The method of claim 1, wherein the first and second magnified image data are each generated using a lowpass filter, a first highpass filter having filter coefficients weighted by a first sharpness parameter S, a second highpass filter having filter coefficients weighted by a second sharpness parameter T, and an adder that generates a weighted sum of filtered image data generated by the lowpass, first highpass and second highpass filters.

5. A method of magnifying a portion of an image displayed on a computer display device, comprising:

displaying an image corresponding to image data stored in a screen buffer;

selecting a first region of the displayed image in accordance with user input;

copying image data from the screen buffer to a first buffer, the copied image data corresponding to the first selected region;

generating first magnified image data corresponding to a portion of the first selected region and storing the first magnified image data in the screen buffer so as to replace the copied image data;

receiving user input that selects a second region of the displayed image, the second selected region overlapping the first selected region;

copying image data from the screen buffer to a second buffer, the copied image data corresponding to a combined region that includes both the first and second selected regions;

copying the image data stored in the first buffer into a portion of the second buffer corresponding to the first selected region;

generating second magnified image data corresponding to a portion of the second selected region and storing the second magnified image data in a portion of the second buffer corresponding to the second selected region; and

copying into the screen buffer the image data and second magnified image data in the second buffer corresponding to the combined region;

wherein

the first and second magnified image data are each generated using a magnification filter that includes a lowpass filter, a highpass filter, and an adder that generates a weighted sum of image data generated by the lowpass and highpass filters ; and

the lowpass filter generates an interpolated value for each of a plurality of initially undefined f_k,l values in accordance with the filter equation;

$f_{k,l}=\underset{i,j}{\sum <br><br>}<br><br>P_{k-2<br><br>\mathrm{ni},l-2<br><br>\mathrm{nj}}<br><br>d_{i,j}$

wherein

n is a positive integer;

d_i,j are the subset of f_k,l values which have defined values prior to the generation of the interpolated f_k,l values, such that f_2ni,2nj=d_i,j for integer values of i and j that fall within the initial set of image data;

P_k,l are coefficients, no more than eighteen of which have non-zero values; and

each non-zero P_k,l coefficient is an integer divided by an integer power of two.

6. The method of claim 5, wherein the set of interpolated f_k,l values in conjunction with the d_i,j values represent a smooth surface that is continuous, and that has a two-dimensional spatial first derivative that is continuous.

7. The method of claim 6, wherein the smooth surface represented by the set of interpolated f_k,l values in conjunction with the d_i,j values has at each of a plurality of locations (k,l) for which an interpolated value is generated a tangent plane that is substantially parallel to a plane formed by adjacent ones of the data points (k,l) corresponding to data values d_i,j.

8. The method of claim 6, wherein

whenever a local contiguous set of the initial data values d_i,j all fall on a polynomial surface, the interpolated f_k,l values that are located between the initial data points also fall on that same polynomial surface.

9. The method of claim 6, wherein the non-zero P_k,l coefficients, when summed, have an associated sum equal to 1.

25. The method of claim 5, wherein the first and second magnified image data are each generated using a lowpass filter, a first highpass filter having filter coefficients weighted by a first sharpness parameter S, a second highpass filter having filter coefficients weighted by a second sharpness parameter T, and an adder that generates a weighted sum of filtered image data generated by the lowpass, first highpass and second highpass filters.

26. The method of claim 9, wherein the P_k,l coefficients used by the lowpass filter have coefficient values corresponding to the filter coefficients shown in FIG. 10A.

27. The method of claim 26, wherein the highpass filter utilizes highpass filter coefficients corresponding to the filter coefficients in FIG. 10B.

10. A method of magnifying a portion of an image displayed on a computer display device, comprising:

displaying an image corresponding to image data stored in a screen buffer;

selecting a first region of the displayed image in accordance with user input;

copying first image data from the screen buffer to an image buffer, the first copied image data corresponding to the selected region;

generating first magnified image data corresponding to a portion of the first selected region and storing the first magnified image data in the screen buffer so as to replace the first copied image data;

receiving user input that selects a second region of the displayed image;

restoring the first copied image data stored in the image buffer to the screen buffer;

copying second image data from the screen buffer to the image buffer, the second copied image data corresponding to the second selected region; and

generating second magnified image data corresponding to a portion of the second selected region and storing the second magnified image data in the screen buffer so as to replace the copied image data.

28. The method of claim 10, wherein the first and second magnified image data are each generated using a lowpass filter, a first highpass filter having filter coefficients weighted by a first sharpness parameter S, a second highpass filter having filter coefficients weighted by a second sharpness parameter T, and an adder that generates a weighted sum of filtered image data generated by the lowpass, first highpass and second highpass filters.

11. A computer program product for use in conjunction with a computer system having a screen buffer for storing image data and a display device for displaying an image corresponding to the image data stored in the screen buffer, the computer program product comprising a computer readable storage medium and a computer program mechanism embedded therein, the computer program mechanism comprising:

a screen image magnification module that has instructions for;

selecting a first region of the displayed image in accordance with user input;

copying image data from the screen buffer to a first buffer, the copied image data corresponding to the first selected region;

generating first magnified image data corresponding to a portion of the first selected region and storing the first magnified image data in the screen buffer so as to replace the copied image data;

receiving user input that selects a second region of the displayed image, the second selected region overlapping the first selected region;

copying image data from the screen buffer to a second buffer, the copied image data corresponding to a combined region that includes both the first and second selected regions;

copying the image data stored in the first buffer into a portion of the second buffer corresponding to the first selected region;

generating second magnified image data corresponding to a portion of the second selected region and storing the second magnified image data in a portion of the second buffer corresponding to the second selected region; and

copying into the screen buffer the image data and second magnified image data in the second buffer corresponding to the combined region.

12. The computer program product of claim 11, wherein the instructions in the screen image magnification module for generating the first and second magnified image data include lowpass filter instructions for generating lowpass filtered image data, highpass filter instructions for generating highpass filtered image data, and summing instructions for generating the first magnified image data and the second magnified image data by summing the lowpass filtered image data and the highpass filtered image data.

13. The computer program product of claim 12, wherein the highpass filtered image data generated by the highpass filter instructions is weighted by a sharpness parameter S prior to summing by the summing instructions with the lowpass filtered image data generated by the lowpass filter instructions, where S is a value between 0 and 1.

14. The computer program product of claim 12, wherein the lowpass filter instructions and highpass filter instructions implement lowpass and highpass wavelet like convolution filters, respectively, wherein the lowpass wavelet like convolution filter has coefficients whose sum is equal to 1, and the highpass wavelet like convolution filter has coefficients whose sum is equal to zero.

23. The computer program product of claim 11, wherein the instructions in the screen image magnification module for generating the first and second magnified image data include lowpass filter instructions for generating lowpass filtered image data, highpass filter instructions for generating first highpass image data using first highpass filter coefficients weighted by a first sharpness parameter S and for generating second highpass image data using second highpass filter coefficients weighted by a second sharpness parameter T, and summing instructions for generating the first magnified image data and the second magnified image data by summing the lowpass filtered image data, first highpass image data and second highpass image data.

15. A computer program product for use in conjunction with a computer system having a screen buffer for storing image data and a display device for displaying an image corresponding to the image data stored in the screen buffer, the computer program product comprising a computer readable storage medium and a computer program mechanism embedded therein, the computer program mechanism comprising:

a screen image magnification module that has instructions for;

selecting a first region of the displayed image in accordance with user input;

copying image data from the screen buffer to a first buffer, the copied image data corresponding to the first selected region;

generating first magnified image data corresponding to a portion of the first selected region and storing the first magnified image data in the screen buffer so as to replace the copied image data;

receiving user input that selects a second region of the displayed image, the second selected region overlapping the first selected region;

copying image data from the screen buffer to a second buffer, the copied image data corresponding to a combined region that includes both the first and second selected regions;

copying the image data stored in the first buffer into a portion of the second buffer corresponding to the first selected region;

generating second magnified image data corresponding to a portion of the second selected region and storing the second magnified image data in a portion of the second buffer corresponding to the second selected region; and

copying into the screen buffer the image data and second magnified image data in the second buffer corresponding to the combined region;

wherein

the instructions in the screen image magnification module for generating the first and second magnified image data include lowpass filter instructions for generating lowpass filtered image data, highpass filter instructions for generating highpass filtered image data, and summing instructions for generating the first magnified image data and the second magnified image data by summing the lowpass filtered image data and the highpass filtered image data; and

the lowpass filter instructions include instructions for generating an interpolated value for each of a plurality of initially undefined f_k,l values in accordance with the filter equation;

$f_{k,l}=\underset{i,j}{\sum <br><br>}<br><br>P_{k-2<br><br>\mathrm{ni},l-2<br><br>\mathrm{nj}}<br><br>d_{i,j}$

wherein

n is a positive integer;

d_i,j are the subset of f_k,l values which have defined values prior to the generation of the interpolated f_k,l values, such that f_2ni,2nj=d_i,j for integer values of i and j that fall within the initial set of image data;

P_k,l are coefficients, no more than eighteen of which have non-zero values; and

each non-zero P_k,l coefficient is an integer divided by an integer power of two.

16. The computer program product of claim 15, wherein the set of interpolated f_k,l values in conjunction with the d_i,j values represent a smooth surface that is continuous, and that has a two-dimensional spatial first derivative that is continuous.

17. The computer program product of claim 16, wherein the smooth surface represented by the set of interpolated f_k,l values in conjunction with the d_i,j values has at each of a plurality of locations (k,l) for which an interpolated value is generated a tangent plane that is substantially parallel to a plane formed by adjacent ones of the data points (k,l) corresponding to data values d_i,j.

18. The computer program product of claim 16, wherein

whenever a local contiguous set of the initial data values d_i,j all fall on a polynomial surface, the interpolated f_k,l values that are located between the initial data points also fall on that same polynomial surface.

19. The computer program product of claim 16, wherein the non-zero P_k,l coefficients, when summed, have an associated sum equal to 1.

20. The computer program product of claim 19, wherein the P_k,l coefficients used by the lowpass filter instructions have coefficient values corresponding to the filter coefficients shown in FIG. 10A.

21. The computer program product of claim 20, wherein the highpass filter instructions utilize highpass filter coefficients corresponding to the filter coefficients in FIG. 10B.

22. The computer program product of claim 15, wherein the highpass filter instructions include instructions for generating first highpass image data using first highpass filter coefficients weighted by a first sharpness parameter S and for generating second highpass image data using second highpass filter coefficients weighted by a second sharpness parameter T, and the summing instructions including instructions for generating the first magnified image data and the second magnified image data by summing the lowpass filtered image data, first highpass image data and second highpass image data.

29. A computer system, comprising:

a screen buffer for storing image data;

a display device for displaying an image corresponding to the image data stored in the screen buffer;

a processing unit for executing instructions; and

a screen image magnification module that has instructions, executable by the processing unit, for;

selecting a first region of the displayed image in accordance with user input;

copying image data from the screen buffer to a first buffer, the copied image data corresponding to the first selected region;

generating first magnified image data corresponding to a portion of the first selected region and storing the first magnified image data in the screen buffer so as to replace the copied image data;

receiving user input that selects a second region of the displayed image, the second selected region overlapping the first selected region;

copying image data from the screen buffer to a second buffer, the copied image data corresponding to a combined region that includes both the first and second selected regions;

copying the image data stored in the first buffer into a portion of the second buffer corresponding to the first selected region;

generating second magnified image data corresponding to a portion of the second selected region and storing the second magnified image data in a portion of the second buffer corresponding to the second selected region; and

copying into the screen buffer the image data and second magnified image data in the second buffer corresponding to the combined region.

30. The computer system of claim 29, wherein the instructions in the screen image magnification module for generating the first and second magnified image data include lowpass filter instructions for generating lowpass filtered image data, highpass filter instructions for generating highpass filtered image data, and summing instructions for generating the first magnified image data and the second magnified image data by summing the lowpass filtered image data and the highpass filtered image data.

31. The computer system of claim 30, wherein the highpass filtered image data generated by the highpass filter instructions is weighted by a sharpness parameter S prior to summing by the summing instructions with the lowpass filtered image data generated by the lowpass filter instructions, where S is a value between 0 and 1.

32. The computer system of claim 30, wherein the lowpass filter instructions and highpass filter instructions implement lowpass and highpass wavelet like convolution filters, respectively, wherein the lowpass wavelet like convolution filter has coefficients whose sum is equal to 1, and the highpass wavelet like convolution filter has coefficients whose sum is equal to zero.

33. The computer program product of claim 30, wherein the lowpass filter instructions utilize lowpass filter coefficients corresponding to the filter coefficients in FIG. 10A, and the highpass filter instructions utilize highpass filter coefficients corresponding to the filter coefficients in FIG. 10B.

34. The computer system of claim 30, wherein the highpass filter instructions include instructions for generating first highpass image data using first highpass filter coefficients weighted by a first sharpness parameter S and for generating second highpass image data using second highpass filter coefficients weighted by a second sharpness parameter T, and the summing instructions including instructions for generating the first magnified image data and the second magnified image data by summing the lowpass filtered image data, first highpass image data and second highpass image data.

35. A computer system comprising:

a screen buffer for storing image data;

a display device for displaying an image corresponding to the image data stored in the screen buffer;

a processing unit for executing instructions; and

a screen image magnification module that has instructions, executable by the processing unit, for;

selecting a first region of the displayed image in accordance with user input;

copying image data from the screen buffer to a first buffer, the copied image data corresponding to the first selected region;

generating first magnified image data corresponding to a portion of the first selected region and storing the first magnified image data in the screen buffer so as to replace the copied image data;

receiving user input that selects a second region of the displayed image, the second selected region overlapping the first selected region;

copying image data from the screen buffer to a second buffer, the copied image data corresponding to a combined region that includes both the first and second selected regions;

copying the image data stored in the first buffer into a portion of the second buffer corresponding to the first selected region;

generating second magnified image data corresponding to a portion of the second selected region and storing the second magnified image data in a portion of the second buffer corresponding to the second selected region; and

copying into the screen buffer the image data and second magnified image data in the second buffer corresponding to the combined region;

wherein

the instructions in the screen image magnification module for generating the first and second magnified image data include lowpass filter instructions for generating lowpass filtered image data, highpass filter instructions for generating highpass filtered image data, and summing instructions for generating the first magnified image data and the second magnified image data by summing the lowpass filtered image data and the highpass filtered image data; and

the lowpass filter instructions include instructions for generating an interpolated value for each of a plurality of initially undefined f_k,l values in accordance with the filter equation;

$f_{k,l}=\underset{i,j}{\sum <br><br>}<br><br>P_{k-2<br><br>\mathrm{ni},l-2<br><br>\mathrm{nj}}<br><br>d_{i,j}$

wherein

n is a positive integer;

d_i,j are the subset of f_k,l values which have defined values prior to the generation of the interpolated f_k,l values, such that f_2ni,2nj=d_i,j for integer values of i and j that fall within the initial set of image data;

P_k,l are coefficients, no more than eighteen of which have non-zero values; and

each non-zero P_k,l coefficient is an integer divided by an integer power of two.

36. The computer system of claim 35, wherein the set of interpolated f_k,l values in conjunction with the d_i,j values represent a smooth surface that is continuous, and that has a two-dimensional spatial first derivative that is continuous.

37. The computer system of claim 36, wherein the smooth surface represented by the set of interpolated f_k,1 values in conjunction with the d_i,j values has at each of a plurality of locations (k,l) for which an interpolated value is generated a tangent plane that is substantially parallel to a plane formed by adjacent ones of the data points (k,l) corresponding to data values d_i,j.

38. The computer system of claim 36, wherein whenever a local contiguous set of the initial data values d_i,j all fall on a polynomial surface, the interpolated f_k,l values that are located between the initial data points also fall on that same polynomial surface.

39. The computer system of claim 36, wherein the non-zero P_k,l coefficients, when summed, have an associated sum equal to 1.

40. The computer system of claim 39, wherein the P_k,l coefficients used by the lowpass filter instructions have coefficient values corresponding to the filter coefficients shown in FIG. 10A.

41. The computer program product of claim 40, wherein the highpass filter instructions utilize highpass filter coefficients corresponding to the filter coefficients in FIG. 10B.

42. The computer system of claim 35, wherein the highpass filter instructions including instructions for generating first highpass image data using first highpass filter coefficients weighted by a first sharpness parameter S and for generating second highpass image data using second highpass filter coefficients weighted by a second sharpness parameter T, and the summing instructions including instructions for generating the first magnified image data and the second magnified image data by summing the lowpass filtered image data, first highpass image data and second highpass image data.

43. A computer system comprising:

a screen buffer for storing image data;

a display device for displaying an image corresponding to the image data stored in the screen buffer;

a processing unit for executing instructions; and

a screen image magnification module that has instructions, executable by the processing unit, for;

selecting a first region of the displayed image in accordance with user input;

copying image data from the screen buffer to a first buffer, the copied image data corresponding to the first selected region;

generating first magnified image data corresponding to a portion of the first selected region and storing the first magnified image data in the screen buffer so as to replace the copied image data;

receiving user input that selects a second region of the displayed image, the second selected region overlapping the first selected region;

copying image data from the screen buffer to a second buffer, the copied image data corresponding to a combined region that includes both the first and second selected regions;

copying the image data stored in the first buffer into a portion of the second buffer corresponding to the first selected region;

generating second magnified image data corresponding to a portion of the second selected region and storing the second magnified image data in a portion of the second buffer corresponding to the second selected region; and

copying into the screen buffer the image data and second magnified image data in the second buffer corresponding to the combined region;

wherein

wherein the instructions in the screen image magnification module for generating the first and second magnified image data include lowpass filter instructions for generating lowpass filtered image data, highpass filter instructions for generating first highpass image data using first highpass filter coefficients weighted by a first sharpness parameter S and for generating second highpass image data using second highpass filter coefficients weighted by a second sharpness parameter T, and summing instructions for generating the first magnified image data and the second magnified image data by summing the lowpass filtered image data, first highpass image data and second highpass image data.

44. The computer system of claim 43, wherein the lowpass filter instructions include instructions for generating an interpolated value for each of a plurality of initially undefined f_k,l values in accordance with the filter equation:

$f_{k,l}=\underset{i,j}{\sum <br><br>}<br><br>P_{k-2<br><br>\mathrm{ni},l-2<br><br>\mathrm{nj}}<br><br>d_{i,j}$

wherein

n is a positive integer;

d_i,j are the subset of f_k,i values which have defined values prior to the generation of the interpolated f_k,l values, such that f_2ni,2nj=d_i,j for integer values of i and j that fall within the initial set of image data;

P_k,l are coefficients, no more than eighteen of which have non-zero values; and

each non-zero P_k,l coefficient is an integer divided by an integer power of two.

45. The computer system of claim 44, wherein the set of interpolated f_k,l values in conjunction with the d_i,j values represent a smooth surface that is continuous, and that has a two-dimensional spatial first derivative that is continuous.

46. The computer system of claim 45, wherein the smooth surface represented by the set of interpolated f_k,l values in conjunction with the d_i,j values has at each of a plurality of locations (k,l) for which an interpolated value is generated a tangent plane that is substantially parallel to a plane formed by adjacent ones of the data points (k,l) corresponding to data values d_i,j.

47. The computer system of claim 45, wherein whenever a local contiguous set of the initial data values d_i,j all fall on a polynomial surface, the interpolated f_k,l values that are located between the initial data points also fall on that same polynomial surface.

48. The computer system of claim 45, wherein the non-zero P_k,l coefficients, when summed, have an associated sum equal to 1.

49. The computer system of claim 48, wherein the P_k,l coefficients used by the lowpass filter instructions have coefficient values corresponding to the filter coefficients shown in FIG. 10A.

50. The computer program product of claim 49, wherein the first highpass filter coefficients utilized by the highpass filter instructions corresponding to the filter coefficients in FIG. 10B.