Interactive water effects using texture coordinate shifting

• US 7,050,058 B2
• Filed: 07/22/2005
• Issued: 05/23/2006
• Est. Priority Date: 12/13/2001
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Shum, Heung-Yeung, Chen, Yanyun, Guo, Baining, Xu, Ying-Qing, Liang, Lin
• Primary Examiner(s): Chauhan, Ulka J.
• Assistant Examiner(s): Hajnik, Daniel
• Application Number:
  US11/187,507
• Publication Number:
  US 20050256721A1
• Time in Patent Office: 305 Days
• Field of Search: None
• US Class Current: 345/473
• CPC Class Codes: G06T 13/60 of natural phenomena, e.g. rain, snow, water or plants
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 2210/24 Fluid dynamics
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects

Abstract

A system and process for adding a photorealistic rendering of a body of water to a virtual 3D scene or image and creating a video therefrom having interactive water effects. A region of water is added to an image by adding an area depicting the original scene as it would appear if reflected by still body of water. Then, the appearance of the added water region is distorted over a series of image frames in such a way as to simulate how the reflected scene would look if the surface of the water were in motion. The water can have dynamic waves and the user can interact with the water in numbers of ways, including generating ripples on the water surface and creating rain. In addition, these effects can be achieved at full screen resolution with the use of the latest graphics hardware by employing a texture shifting technique.

Claims

1. A computer-implemented process for generating a video having interactive water effects, comprising:

an inputting step for inputting an image of a scene;

a generating step for generating a reflection copy of the scene depicted in the input image to create a water region image;

a merging step for merging the input image and water region image, and sizing the merged image to fit a desired frame size, to produce a first frame of the video;

a distorting step for distorting the water region portion of the merged image to simulate at least one ripple originating at a site selected by a viewer, wherein said distorting step comprises,

an establishing step for establishing a height map comprising a plurality of points each of which is assigned a height value and each of which corresponds to a different location of the water region portion of the merged image,

a displaying step for displaying the merged image to the viewer and inputting the location of each site selected by the viewer in the water region of the merged image,

an imposing step for imposing a ripple mask onto the height map established for the merged image by modifying the assigned height values of the points thereof corresponding to each viewer-selected location of the merged image, as well as height map points corresponding to locations of the merged image adjoining the viewer-selected locations, wherein said imposing step comprises, for each viewer-selected location of the merged image,

an first adding step for adding a height value representing a ripple amplitude of the ripple being simulated to the existing height value assigned to the height map point corresponding to the viewer-selected location,

a second adding step for adding a height value representing one-half of the ripple amplitude to the height map points corresponding to locations of the merged image above, below, to the right and to the left of the viewer-selected location, and

a third adding step for adding a height value representing one-quarter of the ripple amplitude to the height map points corresponding to locations of the merged image directly diagonal to the viewer-selected location,

a computing step for computing new image coordinates for each location in the water region of the merged image that correspond to a height map point using the height map values associated with each of said locations, and

a moving step for moving a pixel or pixels associated with each of said locations to the new image coordinates; and

a generating step for generating a sequence of additional frames, each of which comprises a distorted version of the water region of the immediately preceding frame in which any ripple simulated in the immediately preceding frame is shown in a new location that simulates the natural motion of a ripple across a surface of a body of water.

2. The process of claim 1, wherein the computing step comprises, for each location:

a first subtracting step for subtracting the height value assigned to the height map for the merged image location to the left of the location under consideration from the height value assigned to the height map for the merged image location to the right of the location under consideration, and dividing the resulting difference by a scaling factor that relates the scale of the height map values to the merged image, to compute a horizontal translation value for the merged image location under consideration;

a second subtracting step for subtracting the height value assigned to the height map for the merged image location directly above the location under consideration from the height value assigned to the height map for the merged image location immediately below the location under consideration, and dividing the resulting difference by the scaling factor, to compute a vertical translation value for the merged image location under consideration;

a horizontal image coordinate adding step for adding the horizontal translation value to the horizontal image coordinate of the merged image location under consideration to establish a new horizontal image coordinate for the location; and

a vertical image coordinate adding step for adding the vertical translation value to the vertical image coordinate of the merged image location under consideration to establish a new vertical image coordinate for the location.

3. The process of claim 1, wherein the generating step comprises, for each new frame:

a height map computing step for computing a new height map for the frame being generated from the height map associated with the immediately preceding frame;

an image coordinate computing step for computing new image coordinates for each location in the water region of the frame being generated that correspond to a height map point using the new height map values associated with each of said locations; and

a pixel moving step for moving a pixel or pixels associated with each of said locations to the new image coordinates.

4. A process for generating a video having interactive water effects, comprising:

an inputting step for inputting an image of a scene;

a reflection copy generating step for generating a reflection copy of the scene depicted in the input image to create a water region image;

a merging step for merging the input image and water region image, and sizing the merged image to fit a desired frame size, to produce a first frame of the video;

a distorting step for distorting the water region portion of the merged image to simulate at least one ripple originating at a site selected by a viewer;

a second distorting step for further distorting the water region portion of the merged image to simulate a periodic wave, said second distorting step comprising,

an establishing step for establishing a height map comprising a plurality of points each of which is assigned a height value and each of which corresponds to a different location of the water region portion of the merged image,

a periodic wave generating step for generating periodic wave height values for each height map point which simulate the shape and location of a wave on the surface of a body of water at a given point in time,

a computing step for computing new image coordinates for each location in the water region of the merged image that correspond to a height map point using the height map values and periodic wave height values associated with each of said locations, said computing step comprising, for each such location,

a first adding step for adding the height value assigned to the height map for the merged image location to the left of the location under consideration to the periodic wave height value generated for the same location and associated with said given point in time to produce a combined left-side neighbor height value,

a second adding step for adding the height value assigned to the height map for the merged image location to the right of the location under consideration to the periodic wave height value generated for the same location and associated with said given point in time to produce a combined right-side neighbor height value,

a first subtracting step for subtracting the combined left-side neighbor height value from the combined right-side neighbor height value, and dividing the resulting difference by a scaling factor that relates the scale of the height map values to the merged image, to compute a horizontal translation value for the merged image location under consideration,

a third adding step for adding the height value assigned to the height map for the merged image location directly above the location under consideration to the periodic wave height value generated for the same location and associated with said given point in time to produce a combined upper neighbor height value,

a fourth adding step for adding the height value assigned to the height map for the merged image location immediately below the location under consideration to the periodic wave height value generated for the same location and associated with said given point in time to produce a combined lower neighbor height value,

a second subtracting step for subtracting the combined upper neighbor height value from the combined lower neighbor height value, and dividing the resulting difference by the scaling factor that relates the scale of the height map values to the merged image, to compute a vertical translation value for the merged image location under consideration,

a fifth adding step for adding the horizontal translation value to the horizontal image coordinate of the merged image location under consideration to establish a new horizontal image coordinate for the location, and

a sixth adding step for adding the vertical translation value to the vertical image coordinate of the merged image location under consideration to establish a new vertical image coordinate for the location, and

a moving step for moving a pixel or pixels associated with each of said locations to the new image coordinates; and

an image generating step for generating a sequence of additional images, each of which comprises a distorted version of the water region of the immediately preceding frame in which each ripple and the periodic wave simulated in the immediately preceding frame is shown in a new location that simulates the natural motion of a ripple and wave across a surface of a body of water.

5. A computer-implemented process for generating a video depicting water effects, comprising:

a defining step for defining an orthogonal grid comprising a plurality of vertex points, wherein said grid being sized to match a desired size of a water region in an overall scene depicted in each frame of the video, and wherein said defining step comprises,

an establishing step for establishing the coordinates of the four corner vertices of the video frames,

a height specifying step for specifying a height of a horizontal line which divides said upper and lower portions of the video frames based on a viewer-selected height value,

a grid line specifying step for specifying the number of equally-spaced horizontal grid lines and the number of equally-spaced vertical grid lines to be employed in the grid, and

an identifying step for identifying the grid coordinates of each vertex of the grid given the specified number of horizontal and vertical grid lines;

an inputting step for inputting a texture map that is used in conjunction with a model of a scene to generate an image of the scene;

a generating step for generating a reflection copy of the texture map and sizing it to match the overall size of the grid to create a water region texture map;

a merging step for merging the input texture map and water region texture map to create a combined texture map, wherein the input texture map is sized to fit and associated with an upper portion of the video frames and the water region texture map is associated with a lower portion of the video frames, and wherein the water region texture map is inverted so that the water region of the video frames will appear to be an upside down image of the upper portion of the frames when rendered;

a superimposing step for superimposing the grid on the water region texture map and associating each point of the grid with the closest water region texture map coordinate; and

a rendering step for rendering a sequence of video frames using a combined texture map, wherein each frame is rendered using a separate combined texture map whose associated grid'"'"'s vertices, except for the first frame in the sequence, have been assigned height values which are at least initially derived from the immediately preceding frame in the sequence, and wherein the water region portion of the combined texture map associated with each frame in which water effects are to be depicted is distorted to simulate movement of the surface of the water in the water region of that frame using the height values assigned to its grid.