Method and system for enhancing portrait images that are processed in a batch mode

• US 20060228037A1
• Filed: 06/07/2006
• Published: 10/12/2006
• Est. Priority Date: 02/28/2003
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Richard Adam Simon, Tomasz Matraszek
• Inventors: Simon, Richard A., Matraszek, Tomasz
• Granted Patent: US 7,602,949 B2
• Time in Patent Office: Days
• US Class Current: 382/254
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/005 Retouching; Inpainting; Scratch removal detecting, correction, reducing or removing defects, e.g. non-responsive pixels of solid state image sensors H04N25/68, scratch removal for cinematographic films scanned by electronic image sensor H04N5/253
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/178 estimating age from face image; using age information for improving recognition
• CPC Class Codes: H04N 1/622 with simulation on a subsidiary picture reproducer
• CPC Class Codes: H04N 1/628 Memory colours, e.g. skin or sky

Abstract

A batch processing method for enhancing an appearance of a face located in a digital image, where the image is one of a large number of images that are being processed through a batch process, comprises the steps of: (a) providing a script file that identifies one or more original digital images that have been selected for enhancement, wherein the script file includes an instruction for the location of each original digital image; (b) using the instructions in the script file, acquiring an original digital image containing one or more faces; (c) detecting a location of facial feature points in the one or more faces, said facial feature points including points identifying salient features including one or more of skin, eyes, eyebrows, nose, mouth, and hair; (d) using the location of the facial feature points to segment the face into different regions, said different regions including one or more of skin, eyes, eyebrows, nose, mouth, neck and hair regions; (e) determining one or more facially relevant characteristics of the different regions; (f) based on the facially relevant characteristics of the different regions, selecting one or more enhancement filters each customized especially for a particular region and selecting the default parameters for the enhancement filters; (g) executing the enhancement filters on the particular regions, thereby producing an enhanced digital image from the original digital image; (h) storing the enhanced digital image; and (i) generating an output script file having instructions that indicate one or more operations in one or more of the steps (c)-(f) that have been performed on the enhanced digital image.

Claims

1-63. -63. (canceled)

64. A processing method for enhancing an appearance of a face located in a digital image that is processed in a user-interactive kiosk environment, the method comprising the steps of:

acquiring an original digital image containing one or more faces;

detecting a face in the original digital image;

processing the original digital image to determine if the face is suitable for enhancement based on at least one of size and state of focus of the face in the image;

if the face is found to be acceptable for enhancement relative to a predetermined threshold related to size and state of focus, then enhancing the image according to one or more enhancement algorithms and displaying the image for review by a user of the kiosk; and

if the face is found not to be acceptable by the user, then providing an interactive retouching mode for the user to interactively adjust the one or more enhancement algorithms until the image is acceptable.

65. A user-interactive photographic kiosk for processing a digital image for enhancing an appearance of a face located in the image, the kiosk comprising:

an input interface for acquiring an original digital image containing one or more faces;

a processor for (a) detecting a face in the original digital image;

(b) processing the original digital image to determine if the face is suitable for enhancement based on at least one of size and state of focus of the face in the image; and

(c) if the face is found to be acceptable for enhancement relative to a predetermined threshold related to size and state of focus, then enhancing the image according to one or more enhancement algorithms;

a display for displaying the enhanced image for review by a user of the kiosk; and

an interactive retouching mode for enabling the user, if the face is found not to be acceptable, to interactively adjust the one or more enhancement algorithms until the image is acceptable.

66-69. -69. (canceled)

70. A processing method for enhancing an appearance of a face located in a digital image that is processed in a user-interactive kiosk environment, the method comprising the steps of:

acquiring an original digital image;

detecting one or more faces in said original digital image;

identifying among said detected faces one or more faces meeting predetermined enhancement criteria to provide one or more enhanceable faces;

enhancing said one or more enhanceable faces to provide an enhanced image;

displaying said enhanced image to the user;

concurrent with said displaying, allowing the user to select one of;

(a) continuing processing with said enhanced image, (b) continuing processing with said original image, and (c) manually retouching said original image to provide a retouched image;

wherein said detecting, identifying, and enhancing steps are non-interactive with the user.

71. The method of claim 70 wherein said enhancement criteria are based upon at least one of size and sharpness of the respective said face in said original digital image.

72. The method of claim 70 wherein said enhancement criteria are based upon both size and sharpness of the respective said face in said original digital image.

73. The method of claim 72 wherein said detecting further comprises locating facial feature points on each of said faces and segmenting each of said detected faces into two or more regions using said facial feature points;

and said enhancing further comprises applying one or more enhancement filters to one or more of said regions of each of said enhanceable faces, said enhancement filters each being customized to a respective said region.

74. The method of claim 73 wherein said regions include two or more of:

facial skin, eyes, nose, mouth, hair, facial hair, and neck regions.

75. The method of claim 73 further comprising:

determining one or more facially relevant characteristics of the different regions, and

selecting one or more parameters of one or more of said filters based on said determined facially relevant characteristics of the respective said regions.

76. The method of claim 75 wherein said facially relevant characteristics include at least one characteristic selected from the group:

a size of the face, a distance between facial features in the respective said region, a gender of the respective said face, and an age of the respective said face.

77. The method of claim 75 wherein one of said facially relevant characteristics is gender of said one or more faces and said method further comprises determining the gender of respective said faces prior to said selecting.

78. The method of claim 75 wherein one of said facially relevant characteristics is age of said one or more faces and said method further comprises determining the ages of respective said faces prior to said selecting.

79. The method of claim 75 wherein one of said facially relevant characteristics is distance between eyes of respective said faces and said method further comprises determining said distance in respective said faces prior to said selecting.

80. The method of claim 70 wherein said detecting further comprises locating facial feature points on each of said faces and segmenting each of said detected faces into two or more regions using said facial feature points;

and said enhancing further comprises applying one or more enhancement filters to one or more of said regions of each of said enhanceable faces, said enhancement filters each being customized to a respective said region.

81. The method of claim 70 further comprising completing processing without further interaction with the user, when the user selects:

(a) continuing processing with said enhanced image or (b) continuing processing with said original image.

82. The method of claim 81 wherein said completing processing further comprises printing the respective said image.

83. The method of claim 70 wherein said acquiring further comprises one of:

scanning a photo, scanning a negative, receiving a file contained on digital media, and receiving a file via a communication link to a network.

84. A processing system for enhancing an appearance of a face located in a digital image that is processed in a user-interactive kiosk environment, the system comprising:

an input interface adapted to acquire an original digital image;

a processor adapted to;

detect one or more faces in said original digital image;

identify among said detected faces one or more faces meeting predetermined enhancement criteria to provide one or more enhanceable faces; and

enhance said one or more enhanceable faces to provide an enhanced image;

a display adapted to display said enhanced image to the user;

wherein said processor is adapted to accept user input via said input interface when said enhanced image is displayed, said user input selecting one of;

(a) continuing processing with said enhanced image, (b) continuing processing with said original image, and (c) manually retouching said original image to provide a retouched image, said user interface being otherwise non-interactive.

85. The system of claim 84 wherein said processor is adapted to locating facial feature points on each of said faces and segment each of said faces into two or more regions using said facial feature points;

and apply one or more enhancement filters to one or more of said regions of each of said enhanceable faces, said enhancement filters each being customized to a respective said region.

86. The system of claim 85 wherein said processor is adapted to:

determine one or more facially relevant characteristics of the different regions, and

select one or more parameters of one or more of said filters based on said facially relevant characteristics of the respective said regions.

87. The processing system of claim 84 wherein said processor has an interactive mode wherein said processor accepts said user input and a non-interactive mode wherein said processor is non-interactive with said user input.