System and method for parameter-based image synthesis using hierarchical networks

• US 5,649,086 A
• Filed: 03/08/1995
• Issued: 07/15/1997
• Est. Priority Date: 03/08/1995
• Status: Expired due to Term

• Current Assignee: nFX Corp. (180 Degree Capital)
• Original Assignee: nFX Corp. (180 Degree Capital)
• Inventors: Cheng, Chie-Jin, Tam, Va-On, Belfer, Russell, Librande, Steve, Zhang, Bin
• Primary Examiner(s): Jankus, Almis R.
• Application Number:
  US08/401,023
• Time in Patent Office: 860 Days
• Field of Search: 395/141, 395/133, 395/135, 395/152-154, 382/293-300, 382/308
• US Class Current: 345/441
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]

Abstract

Synthesis of novel images from example images is achieved by interpolating among example images based on user selection of parameter values in a hierarchy of networks referred to as parent networks and child networks. Child networks describe distinct physical characteristics of the image, such as thickness of eyebrows. Parent networks describe more general or abstract characteristics related to the image, such as emotional states expressed by the image.

Claims

1. A method of using a computer to synthesize an object, the object including a plurality of components, each component being capable of description by a plurality of known characteristics, the method comprising:

forming a plurality of child networks from said components responsive to corresponding ones of said known characteristics;

forming a parent network superordinate to said child networks, said parent network including a parameter, the parent network being hierarchically related to said child networks such that a change in said parameter of said parent network corresponds to changes in said corresponding ones of said known characteristics;

associating with said parameter a first parameter state and a second parameter state;

associating with each of said known characteristics a first characteristic state corresponding to said first parameter state and a second characteristic state corresponding to said second parameter state;

providing a user interface allowing user selection of a desired parameter state, said desired parameter state being bounded by said first parameter state and said second parameter state; and

synthesizing said object responsive to the desired parameter state by generating a set of desired object characteristics from said first characteristic states, said second characteristic states, and said desired parameter state.

2. A method as in claim 1, wherein the object is an image of a face, wherein the known characteristics include at least one of a group of skull shape, chin shape, eye shape, nose shape, and mouth shape, and wherein said parameter represents a human emotional state.

3. A method as in claim 1, wherein the object is an image of a face, wherein the known characteristics include at least one of a group of skull shape, chin shape, eye shape, nose shape, and mouth shape, and wherein said parameter represents a direction of view.

4. A method as in claim 1, wherein the object is an image of a face, wherein the known characteristics include at least one of a group of skull shape, chin shape, eye shape, nose shape, and mouth shape, and wherein said parameter represents a gender.

5. A method as in claim 1, wherein the object is an image of a face, wherein the known characteristics include at least one of a group of skull shape, chin shape, eye shape, nose shape, and mouth shape, and wherein said parameter is one of a group including surprise, shock, anger, fearfulness, contentedness, tiredness, inquisitiveness, concentration, happiness, gender, direction of view.

6. A method as in claim 1, wherein said synthesizing includes producing an image of said object.

7. A method as in claim 1, wherein said synthesizing includes interpolating among example objects.

8. A method of using a computer to synthesize an image of a physical object, the physical object capable of being described by a plurality of known characteristics, the method comprising:

creating a computer-implemented hierarchical network to associate said characteristics with a plurality of parameters, the hierarchical network including (i) a plurality of child networks each providing correspondence between a portion of said image and a pertinent subset of said characteristics and (ii) a parent network providing correspondence between said pertinent subset for each one of said child networks and said plurality of parameters;

storing exemplars of said characteristics in the computer;

providing a user interface allowing user selection of a set of desired parameter states; and

synthesizing said image by interpolating among said exemplars responsive to said set of desired parameter states.

9. An image processing system, comprising:

a central processing unit;

an input device, operatively connected with said central processing unit;

a program storage subsystem, operatively connected with said central processing unit, for storing instructions for the control of said central processing unit to provide a user with user interface controls for specifying parameters of an object to be synthesized by said image processing system; and

an animation network subsystem, operatively connected with said central processing unit, said animation network subsystem comprising a parent network and at least one child network, the parent network being hierarchically related to said child network such that said parameters operate as inputs to said parent network and that said parent network generates as output characteristics that operate as inputs to said child network, said animation network subsystem and said central processing unit operating in conjunction to synthesize said object responsive to said parameters specified by said user through said user interface controls.

10. An image processing system as in claim 9, wherein said object is an image of a face, wherein said parent network formed of at least one of a group of a skull network, a hair network, an ear network, an eye frame network, a nose network, a mouth network, and a chin network.

11. An image processing system as in claim 9, wherein said animation network subsystem is comprised of a plurality of exemplars, each indexed with respect to a corresponding parameter.