Automatic sketch generation
First Claim
1. A computer-implemented process for training a computer using a set of training images each of which depicts a person'"'"'s face to generate a sketch of a person'"'"'s face from an image of that person'"'"'s face, comprising using a computer to perform the following process actions:
- drawing a training sketch of the face of each person depicted in a training image, each of said training sketches comprising a plurality of lines that represent and correspond in location to facial features of the person depicted in the associated training image;
defining an average face shape which represents the average shape and location of a prescribed set of facial features depicted in the training images;
defining a geometric transform function for each training image which when applied to its associated training image warps the image so that said prescribed set of facial features correspond in shape and location to the average face shape;
warping each training image and its corresponding sketch using the geometric transform function associated with the training image to create transformed versions thereof;
training an active shape model using the training images to automatically identify the shape and location of said prescribed set of facial features in an arbitrary image depicting a person'"'"'s face; and
defining a sketch model comprising a set of lines representative of the lines found in the transformed training sketches.
2 Assignments
0 Petitions
Accused Products
Abstract
An example-based facial sketch system and process that automatically generates a sketch from an input image depicting a person'"'"'s face. Sketch generation is accomplished by first training the system using example facial images and sketches of the depicted faces drawn with a particular style by a sketch artist. The trained system is then used to automatically generate a facial sketch that simulates the artist'"'"'s style from an input image depicting a person'"'"'s face. Nonparametric sampling and a flexible sketch model are employed to capture the complex statistical characteristics between an image and its sketch.
-
Citations
28 Claims
-
1. A computer-implemented process for training a computer using a set of training images each of which depicts a person'"'"'s face to generate a sketch of a person'"'"'s face from an image of that person'"'"'s face, comprising using a computer to perform the following process actions:
-
drawing a training sketch of the face of each person depicted in a training image, each of said training sketches comprising a plurality of lines that represent and correspond in location to facial features of the person depicted in the associated training image;
defining an average face shape which represents the average shape and location of a prescribed set of facial features depicted in the training images;
defining a geometric transform function for each training image which when applied to its associated training image warps the image so that said prescribed set of facial features correspond in shape and location to the average face shape;
warping each training image and its corresponding sketch using the geometric transform function associated with the training image to create transformed versions thereof;
training an active shape model using the training images to automatically identify the shape and location of said prescribed set of facial features in an arbitrary image depicting a person'"'"'s face; and
defining a sketch model comprising a set of lines representative of the lines found in the transformed training sketches. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. A system for generating a sketch of a person'"'"'s face from an image of the person'"'"'s face, comprising:
-
a general purpose computing device trained to generate a sketch of a person'"'"'s face from an image of the person'"'"'s face, said training comprising inputting a set of training images each of which depicts a person'"'"'s face, drawing a set of training sketches each representing a respective face depicted in the training images, and executing a training computer program comprising a set of program modules that direct the computing device to, identify a prescribed set of points in each training image, said points marking the contours of a prescribed set of facial features, train an active shape model module using the identified feature points to automatically locate said prescribed set of points in an arbitrary image of a person'"'"'s face, compute a meanshape vector by forming a separate shape vector for each training image from the identified feature points of that image and averaging the corresponding elements of each shape vector, wherein each shape vector has identically ordered elements, define a geometric transform function for each training image which warps that image so that the identified feature points in vectorized form equals the meanshape vector, warp each training image and its corresponding sketch using the geometric transform function associated with the training image to create transformed versions thereof, and define a sketch model comprising a set of lines representative of the lines found in the transformed training sketches; and
a sketch generating computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to, input an image of a person'"'"'s face, employ the active shape model to locate said prescribed set of facial feature points in the input image, define a geometric transform function for the input image which warps the input image so the located feature points thereof in vectorized form equals the meanshape vector, warp the input image using the geometric transform function associated with the input image to create a transformed input image, employ a non-parametric sampling module to create an expected sketch image from the transformed input image as well as the transformed training images and sketches, fit the lines defined in the sketch model to the expected sketch image to create a transformed input image sketch comprising those lines of the sketch model that correspond to facial features found in the expected sketch image, apply an inverse of the input image geometric transform function to the transformed input image sketch to unwarp the sketch such that the shape and location of the lines making up the sketch match the shape and location of corresponding facial features in the input image. - View Dependent Claims (20, 21, 22, 23, 24, 25)
-
-
26. A computer-readable medium having computer-executable instructions for training a computer using a set of training images each of which depicts a person'"'"'s face to generate a sketch of a person'"'"'s face from an image of that person'"'"'s face, said computer-executable instructions comprising:
-
drawing a training sketch of the face of each person depicted in a training image, each of said training sketches comprising a plurality of lines that represent and correspond in location to facial features of the person depicted in the associated training image;
defining an average face shape which represents the average shape and location of a prescribed set of facial features depicted in the training images;
warping each training image and its corresponding sketch so as to conform them to the average face shape, thereby creating a set of transformed training images and a corresponding set of transformed training sketches;
training an active shape model using the training images to automatically identify the shape and location of said prescribed set of facial features in an arbitrary image depicting a person'"'"'s face; and
defining a sketch model comprising a set of lines representative of the lines found in the transformed training sketches. - View Dependent Claims (27, 28)
-
Specification