Systems and methods for providing image rendering using variable rate source sampling

• US 20040001069A1
• Filed: 06/28/2002
• Published: 01/01/2004
• Est. Priority Date: 06/28/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Olynyk, Kirk, Snyder, John Michael, Whitted, John Turner, Blank, William Thomas
• Granted Patent: US 6,943,805 B2
• Time in Patent Office: Days
• US Class Current: 345/582
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/80 Shading

Abstract

Systems and methods are provided for variable source rate sampling in connection with image rendering, which accumulate and resolve over all samples forward mapped to each pixel bin. In accordance with the invention, the textured surface to be rendered is sampled, or oversampled, at a variable rate that reflects variations in frequency among different regions, taking into account any transformation that will be applied to the surface prior to rendering and the view parameters of the display device, thus ensuring that each bin of the rendering process receives at least a predetermined minimum number of samples. A variety of image processing applications are contemplated wherein variable rate source sampling, and accumulation and resolution of forward mapped point samples can be applied, ranging from 3-D graphics applications to applications wherein images recorded in a recording/storage environment are mapped to the arbitrary requirements of a display environment.

Claims

1. A method of mapping visual content to a destination device, comprising:

accumulating at least one point sample forward mapped to at least one bin of the destination device wherein a first bin receives a different number of samples from a second bin; and

resolving over all of the at least one point sample forward mapped to the at least one bin to determine a setting.

2. A method according to claim 1, wherein said resolving includes applying at least one of averaging, weighted averaging, taking the median, taking the mode, taking the minimum, taking the maximum and applying a least squares fit based upon said at least one point sample.

3. A method according to claim 1, wherein each point sample maps to only one bin.

4. A method according to claim 1, wherein said accumulating and resolving is applied to each bin.

5. A method according to claim 1, wherein said accumulating and resolving is applied to the at least one bin of each sample square of the destination device.

6. A method according to claim 4, wherein for each setting, the at least one bin receives at least a predetermined minimum number of samples from said forward mapping.

7. A method according to claim 6, wherein said predetermined minimum number is one.

8. A method according to claim 1, wherein said resolving includes resolving the at least one point sample based on at least one of (A) the location of the at least one point sample in a sample square of the destination device, (B) at least one value associated with the at least one point sample, (C) the location of at least one neighbor of the at least one point sample and (D) at least one value associated with the at least one neighbor of the at least one point sample.

9. A method according to claim 8, wherein a weight is determined for each sample, wherein the weight for each sample is determined by computing a local projected area in the neighborhood of the sample.

10. A method according to claim 1, further comprising:

in order to generate said at least one point sample, sampling at least one portion of a source signal at a rate that varies based on at least one of (A) the frequency content of the at least one portion of the source signal being sampled, (B) at least one transformation that the at least one portion of the source signal undergoes prior to rendering and (C) a parameter of a rendering device which renders the visual content.

11. A method according to claim 10, wherein the source signal is one of a surface and an image recorded on a charge coupled device (CCD).

12. At least one of an operating system, driver code, an application programming interface, a tool kit and a coprocessing device for rendering visual content on a display device according to the method of claim 1.

13. A modulated data signal carrying computer executable instructions for performing the method of claim 1.

14. A computing device comprising means for performing the method of claim 1.

15. A method of rendering visual content using variable rate source sampling and forward mapping of point samples, comprising:

sampling at least one portion of a source signal at a rate that varies based on at least one of (A) the frequency content of the at least one portion of a source signal being sampled, (B) at least one transformation that the at least one portion of the source signal undergoes prior to rendering and (C) a parameter of a rendering device which renders the visual content; and

based on samples generated by said sampling, displaying an image on a rendering device.

16. The method of claim 15, further comprising:

dividing said source signal into a plurality of regions;

assessing that a first of said plurality of regions has a first maximum frequency component; and

assessing that a second of said regions has a second maximum frequency component higher than said first frequency component;

and wherein said sampling comprises;

sampling said first region at a first rate; and

sampling said second region at a second rate higher than said first rate.

17. The method of claim 15, further comprising:

performing said at least one transformation on said samples subsequent to said sampling and prior to said displaying.

18. The method of claim 15, wherein said displaying comprises:

receiving the samples into a plurality of bins based on said samples'"'"' locations;

determining at least one value for at least one bin of the plurality of bins based on at least one sample in each of the at least one bin; and

setting one of the grey value and color values in accordance with said at least one value.

19. The method of claim 18, wherein said sampling includes ensuring that each of said plurality of bins receives at least a predetermined minimum number of samples.

20. The method of claim 19, wherein said sampling includes ensuring that each of said plurality of bins receives at least one sample.

21. The method of claim 15, further comprising:

applying at least one texture to said source signal to create a textured source signal, wherein said source signal defines a surface to which said at least one texture is applied, and wherein said displaying is based on samples from sampling of the textured source signal.

22. The method of claim 21, further comprising:

using a geometry map to correlate samples of said at least one texture with samples of said source signal.

23. The method of claim 15, wherein said sampling further comprises:

adjusting said rate such that said rate exceeds the highest frequency of the portion being sampled.

24. The method of claim 15, wherein the source signal is one of a surface and an image recorded on a charge coupled device (CCD).

25. At least one of an operating system, driver code, an application programming interface, a tool kit and a coprocessing device for rendering visual content using variable rate source sampling and forward mapping of point samples according to the method of claim 15.

26. A modulated data signal carrying computer executable instructions for performing the method of claim 15.

27. A computing device comprising means for performing the method of claim 15.

28. A computer readable medium for mapping visual content on a destination device having stored thereon at least one computer-executable module comprising computer executable instructions for performing a method, the method comprising:

accumulating at least one point sample forward mapped to at least one bin wherein a first bin receives a different number of samples from a second bin; and

resolving over all of the at least one point sample forward mapped to the at least one bin to determine a setting.

29. A computer readable medium according to claim 28, wherein said resolving includes applying at least one of averaging, weighted averaging, taking the median, taking the mode, taking the minimum, taking the maximum and applying a least squares fit based upon said at least one point sample.

30. A computer readable medium according to claim 28, wherein each point sample maps to only one bin.

31. A computer readable medium according to claim 28, wherein said accumulating and resolving is applied to each bin.

32. A computer readable medium according to claim 28, wherein said accumulating and resolving is applied to the at least one bin of each sample square of the destination device.

33. A computer readable medium according to claim 31, wherein each at least one bin receives at least a predetermined minimum number of samples from said forward mapping.

34. A computer readable medium according to claim 33, wherein said predetermined minimum number is one.

35. A computer readable medium according to claim 28, wherein said resolving includes resolving the at least one point sample based on at least one of (A) the location of the at least one point sample in a sample square of the destination device, (B) at least one value associated with the at least one point sample, (C) the location of at least one neighbor of the at least one point sample and (D) at least one value associated with the at least one neighbor of the at least one point sample.

36. A computer readable medium according to claim 35, wherein a weight is determined for each sample, wherein the weight for each sample is determined by computing a local projected area in the neighborhood of the sample.

37. A computer readable medium according to claim 28, further comprising:

in order to generate said at least one point sample, sampling at least one portion of a source signal at a rate that varies based on at least one of (A) the frequency content of the at least one portion of the source signal being sampled, (B) at least one transformation that the at least one portion of the source signal undergoes prior to rendering and (C) a parameter of a rendering device which renders the visual content.

38. A computer readable medium according to claim 37, wherein the source signal is one of a surface and an image recorded on a charge coupled device (CCD).

39. A computer readable medium for rendering visual content using variable rate source sampling and forward mapping of point samples having stored thereon at least one computer-executable module comprising computer executable instructions for performing a method, the method comprising:

sampling at least one portion of a source signal at a rate that varies based on at least one of (A) the frequency content of the at least one portion of the source signal being sampled, (B) at least one transformation that the at least one portion of the source signal undergoes prior to rendering and (C) a parameter of a rendering device which renders the visual content; and

based on samples generated by said sampling, displaying an image on a rendering device.

40. A computer readable medium according to claim 39, further comprising:

dividing said source signal into a plurality of regions;

assessing that a first of said plurality of regions has a first maximum frequency component; and

assessing that a second of said regions has a second maximum frequency component higher than said first frequency component;

and wherein said sampling comprises;

sampling said first region at a first rate; and

sampling said second region at a second rate higher than said first rate.

41. A computer readable medium according to claim 39, further comprising:

performing said at least one transformation on said samples subsequent to said sampling and prior to said displaying.

42. A computer readable medium according to claim 39, wherein said displaying comprises:

receiving the samples into a plurality of bins based on said samples'"'"' locations;

determining at least one value for at least one bin of the plurality of bins based on at least one sample in each of the at least one bin; and

setting one of the grey value and color values in accordance with said at least one value.

43. A computer readable medium according to claim 42, wherein said sampling includes ensuring that each of said plurality of bins receives at least a predetermined minimum number of samples.

44. A computer readable medium according to claim 43, wherein said sampling includes ensuring that each of said plurality of bins receives at least one sample.

45. A computer readable medium according to claim 39, further comprising:

applying at least one texture to said source signal to create a textured source signal, wherein said source signal defines a surface to which said at least one texture is applied, and wherein said displaying is based on samples from sampling of the textured source signal.

46. A computer readable medium according to claim 45, further comprising:

using a geometry map to correlate samples of said at least one texture with samples of said source signal.

47. A computer readable medium according to claim 39, wherein said sampling further comprises:

adjusting said rate such that said rate exceeds the highest frequency of the portion being sampled.

48. A computer readable medium according to claim 39, wherein the source signal is one of a surface and an image recorded on a charge coupled device (CCD).

49. A computer device for mapping visual content on a destination device, comprising:

means for accumulating at least one point sample forward mapped to at least one bin wherein a first bin receives a different number of samples from a second bin; and

means for resolving over all of the at least one point sample forward mapped to the at least one bin to determine a setting.

50. A computing device according to claim 49, wherein said means for resolving includes at least one of means for averaging, means for weighted averaging, means for taking the median, means for taking the mode, means for taking the minimum, means for taking the maximum and means for applying a least squares fit based upon said at least one point sample

51. A computing device according to claim 49, wherein each point sample maps to only one bin.

52. A computing device according to claim 49, wherein said means for accumulating and means for resolving are applied to each bin.

53. A computing device according to claim 49, wherein said means for accumulating and means for resolving are applied to the at least one bin of each sample square of the destination device.

54. A computing device according to claim 52, wherein each of the at least one bin receives at least a predetermined minimum number of samples from said forward mapping.

55. A computing device according to claim 54, wherein said predetermined minimum number is one.

56. A computing device according to claim 49, wherein said means for resolving includes means for resolving the at least one point sample based on at least one of (A) the location of the at least one point sample in a sample square of the destination device, (B) at least one value associated with the at least one point sample, (C) the location of at least one neighbor of the at least one point sample and (D) at least one value associated with the at least one neighbor of the at least one point sample.

57. A computing device according to claim 56, wherein a weight is determined for each sample, wherein the weight for each sample is determined by computing a local projected area in the neighborhood of the sample.

58. A computing device according to claim 49, further comprising:

in order to generate said at least one point sample, sampling at least one portion of a source signal at a rate that varies based on at least one of (A) the frequency content of the at least one portion of the source signal being sampled, (B) at least one transformation that the at least one portion of the source signal undergoes prior to rendering and (C) a parameter of a rendering device which renders the visual content.

59. A computing device according to claim 58, wherein the source signal is one of a surface and an image recorded on a charge coupled device (CCD).

60. A computing device for rendering visual content using variable rate source sampling and forward mapping of point samples, comprising:

means for sampling at least one portion of a source signal at a rate that varies based on at least one of (A) the frequency content of the at least one portion of the source signal being sampled, (B) at least one transformation that the at least one portion of the source signal undergoes prior to rendering and (C) a parameter of a rendering device which renders the visual content; and

means for displaying an image on a rendering device based on samples generated by said sampling.

61. A computing device of claim 60, further comprising:

means for dividing said source signal into a plurality of regions;

means for assessing that a first of said plurality of regions has a first maximum frequency component; and

means for assessing that a second of said regions has a second maximum frequency component higher than said first frequency component;

and wherein said means for sampling comprises;

means for sampling said first region at a first rate; and

means for sampling said second region at a second rate higher than said first rate.

62. A computing device of claim 60, further comprising:

means for performing said at least one transformation on said samples subsequent to said sampling of said means for sampling and prior to said displaying by said means for displaying.

63. A computing device of claim 60, wherein said means for displaying comprises:

means for receiving the samples into a plurality of bins based on said samples'"'"' locations;

means for determining at least one value for at least one bin of the plurality of bins based on at least one sample in each of the at least one bin; and

means for setting one of the grey value and color values in accordance with said at least one value.

64. A computing device of claim 63, wherein said means for sampling includes means for ensuring that each of said plurality of bins receives at least a predetermined minimum number of samples.

65. A computing device of claim 64, wherein said means for sampling includes means for ensuring that each of said plurality of bins receives at least one sample.

66. A computing device of claim 60, further comprising:

means for applying at least one texture to said source signal to create a textured source signal, wherein said source signal defines a surface to which said at least one texture is applied, and wherein said means for displaying displays based on samples from sampling of the textured source signal by said means for sampling.

67. A computing device of claim 66, further comprising:

means for using a geometry map to correlate samples of said at least one texture with samples of said source signal.

68. A computing device of claim 60, wherein said means for sampling further comprises:

means for adjusting said rate such that said rate exceeds the highest frequency of the portion being sampled.

69. A computing device of claim 60, wherein the source signal is one of a surface and an image recorded on a charge coupled device (CCD).