Method and apparatus for providing animation in a three-dimensional computer generated virtual world using a succession of textures derived from temporally related source images

• US 5,696,892 A
• Filed: 06/07/1995
• Issued: 12/09/1997
• Est. Priority Date: 07/10/1992
• Status: Expired due to Term

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: The Walt Disney Company
• Inventors: Redmann, William G., Watson, Scott F.
• Primary Examiner(s): Zimmerman, Mark K.
• Application Number:
  US08/486,870
• Time in Patent Office: 916 Days
• Field of Search: 395/125, 395/127, 395/130, 395/135, 395/152, 395/173, 395/960, 345/139
• US Class Current: 345/582
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G09B 5/02 with visual presentation of the material to be studied, e.g. using film strip
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 5/222 Studio circuitry; Studio devices; Studio equipment cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 5/2224 related to virtual studio applications
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

Methods and systems for rendering and displaying in a real time 3-D computer graphic system a sequence of images of a subject using a plurality of time-sequenced textures such that at least a portion of the subject appears animated. The time-sequenced textures are derived from sources such as digitized frames or fields captured from a video recording of a live actor who may be engaged in a scripted performance, or a digitally-recorded cartoon animation sequence, and can be mapped in different ways to different types of surface geometries to achieve animation.

Claims

1. A method for animating a rendition of an object situated in a three-dimensional virtual world generated by a computer graphic system, the object including a plurality of surfaces, and the rendition capable of being displayed from a plurality of viewpoints looking into the virtual world from specific positions and orientations, the method comprising the steps of:

storing three-dimensional data representative of the surfaces of the object;

storing data representative of a plurality of time-sequenced textures; and

rendering in real time a sequence of images based on the data representative of the surfaces and the data representative of subsequent ones of said succession of textures to display the object relative to different ones of said plurality of viewpoints with selected portions of each said time-sequenced texture mapped respectively to selected surfaces of the object, wherein each said time-sequenced texture depicting the figure in action is used to texture multiple surfaces and is changing at intervals over a period of time so that the changing, selectively-mapped textured surfaces of the object appear to animate the figure, wherein;

the plurality of time-sequenced textures comprises a time-sequenced set of recorded images of at least a portion of an entity existing in the real world; and

the plurality of surfaces model an object having a three-dimensional shape approximating the three-dimensional shape of at least that portion of the entity,

whereby different perspective views of the texture can be displayed for different ones of the plurality of viewpoints such that the object displayed appears to be an animated three-dimensional likeness of at least that portion of the entity.

2. The method of claim 1, wherein the entity is a person.

3. A method for animating a rendition of a figure in action at a position in a three-dimensional virtual world generated by a three-dimensional computer graphic system, the method comprising the steps of:

storing data representative of a first surface, said data characterizing said first surface as an uneven surface having geometrically modeled features in the form of said figure;

storing data derived from a succession of at least three temporally related source images depicting the figure in action at different moments in time in the form of a succession of at least three time-sequenced textures;

storing three-dimensional data representative of a plurality of other surfaces forming the virtual world; and

rendering in real time images of the virtual world based on the three-dimensional data representative of the plurality of other surfaces, and a sequence of images based on the data representative of the first surface and the data representative of subsequent ones of said succession of textures, to display the figure in action in the virtual world on at least a portion of the first surface as texture which changes at intervals over a period of time so that the figure appears animated, wherein each said time-sequenced texture is mapped to said first surface, said mapping providing a projection of said figure in action onto said first surface by which features of said figure in action depicted in said texture are selectively aligned with and displayed with varying perspective on corresponding geometric features of said first surface.

4. The method of claim 3 wherein, between rendering of images of the virtual world, the data representative of the first surface changes to transform geometric features of the first surface, said mapping of time-sequenced textures to said first surface providing a projection of said figure in action onto said first surface by which features of said figure in action depicted in said texture remain aligned with and displayed with varying perspective on corresponding geometric features of said first surface after said transformation.

5. The method of claim 3, further comprising the step of:

capturing sound simultaneously with a recording of said succession of at least three temporally related source images depicting said figure in action for synchronizing the animated textures of the figure in action with reproduced voice or other sound generated by the figure in action.

6. The method of claim 3, further comprising the steps of:

storing data representative of a second surface;

storing data derived from a succession of at least three temporally related source images depicting a second figure in action at different moments in time in the form of a second succession of at least three time-sequenced textures; and

rendering in real time a sequence of images based on the data representative of the second surface and the data representative of subsequent ones of said second succession of textures to display the second figure in action on at least a portion of the second surface as texture which changes at intervals over a period of time so that the second figure appears animated, wherein two texture-animated figures in action are displayed in the virtual world simultaneously.

7. The method of claim 6, wherein the two texture-animated figures in action are displayed in a common scene in the virtual world, and wherein the source images depicting the figures in action are recorded in a setting including at least one of props and furniture which is reproduced in the scene as part of the animated textures or as part of a geometric model used to render the scene.

8. The method of claim 3, wherein:

the succession of time-sequenced textures are derived from a recorded succession of at least three temporally related source images depicting an entity existing in the real world; and

the texture displayed appears to be an animated likeness of the entity.

9. The method of claim 8, wherein the entity is a person.

10. The method of claim 3, wherein:

the succession of textures derives from a succession of at least three temporally related drawings.

11. The method of claim 10, wherein the drawings are cartoon drawings and the texture displayed appears to be an animated cartoon.

12. A method for use by a computer graphic system for rendering a likeness of a real-world living being at a specific three-dimensional position in a computer-generated three-dimensional virtual world, the rendition capable of being displayed from viewpoints having specific positions and orientations within the virtual world, the method comprising the steps of:

storing three-dimensional data representative of a plurality of surfaces modeling an object positioned in the virtual world, the object having a three-dimensional form approximating a three-dimensional form of the being;

storing data representative of a plurality of textures derived from a time-sequenced set of recorded images of the being; and

rendering in real time a sequence of images based on the data representative of the plurality of surfaces and the data representative of subsequent ones of said plurality of textures to display the being relative to different ones of said viewpoints as texture conforming to the three-dimensional form of the object, wherein the texture changes so that the being appears animated, and wherein different perspective views of the texture are displayed for different ones of the plurality of viewpoints such that the object displayed appears to be an animated three-dimensional likeness of the being.

13. The method of claim 12, wherein the live being comprises a person.

14. The method of claim 13, wherein the person is an actor performing a scripted event.

15. The method of claim 12, wherein the live being comprises an animal.

16. The method of claim 12, wherein the being comprises a plant.