Interactive eyewear selection system

• US 6,095,650 A
• Filed: 09/22/1998
• Issued: 08/01/2000
• Est. Priority Date: 09/22/1998
• Status: Expired due to Fees

• Current Assignee: Virtual Visual Devices LLC
• Original Assignee: Virtual Visual Devices LLC
• Inventors: Li, Wei, Gao, Feng
• Primary Examiner(s): Manuel, George
• Application Number:
  US09/158,480
• Time in Patent Office: 679 Days
• Field of Search: 351/205, 351/178, 351/227, 351/177, 351/246, 351/204, 33/507, 33/200
• US Class Current: 351/227
• CPC Class Codes: G02C 13/003 Measuring during assembly or fitting of spectacles
• CPC Class Codes: G02C 13/005 Measuring geometric parameters required to locate ophtalmic lenses in spectacles frames apparatus for testing or instruments for examining the eyes per se A61B3/00; machines or devices and accessories for grinding the edges of lenses using spectacles as a template B24B9/144

Abstract

An interactive eyewear selection system comprising: a video camera and digitizing system for acquiring a digital image of a person'"'"'s face, including the temple areas of the face; an eyeglass frames database for storing digital eyeglass frame images; an automated interface for selecting an eyeglass frame image from the frames database, the image containing frame temple portions; a digital processing system for automatically superimposing the selected frame image on the image of the person'"'"'s face, in a composite image, such that the frame appears to be worn naturally on the person'"'"'s face and the frame temple portions are shown at the temple areas of the face; and a display device for displaying the digital image of the person'"'"'s face and for displaying the composite image.

Claims

1. An interactive eyewear selection system, comprising:

means for acquiring a digital image of a person'"'"'s face, including the temple areas of the face;

means for storing digital eyeglass frame images, said storage means containing a plurality of eyeglass frame images;

means for selecting an eyeglass frame image from said storage means, said image containing frame temple portions;

means for superimposing the selected frame image on the image of the person'"'"'s face, in a composite image, such that the frame appears to be worn naturally on the person'"'"'s face and the frame temple portions are shown at the temple areas of the face; and

means for displaying the digital image of the person'"'"'s face and for displaying the composite image.

2. A method of combining an eyeglass frame image with a facial image, comprising the steps of:

(a) obtaining a digital image of a person'"'"'s face;

(b) locating the pupil centers in said face image and determining the midpoint Pc between the pupil centers;

(c) locating left and right face edge points T_L and T_R in said face image, T_L and T_R being located at the edges of the person'"'"'s face, adjacent to the left and right temple areas, respectively, of the person'"'"'s face;

(d) selecting an eyeglass frame image, said frame image having frame points, frame edge points, background points, a frame center point f_c, left and right temple points Q_L and Q_R, and left and right temple segments L_L and L_R ;

(e) assigning a blending coefficient α

(x,y) to the frame points, to the frame edge points, and to the background points of said frame image, the value of said blending coefficient being set to zero for the frame points, to one for the background points, and to a value in the range of about 0.5 to about 0.7 for the frame edge points;

(f) scaling, if necessary, either said frame image or said face image so that the scale of said frame image is the same as that of said face image;

(g) aligning said frame image and said face image in accordance with the frame center point f_c and the pupil midpoint P_c ;

(h) mapping the left and right temple points Q_L and Q_R of said frame image on said face image, to define left and right temple points q_L and q_R, respectively, on said face image, and to determine left and right temple lengths, q_L -T_L and T_R -q_R, on said face image; and

(i) combining said frame image with said face image to produce a composite image, in accordance with the equation

space="preserve" listing-type="equation">C(x,y)=α

(x,y)·

face(x,y)+[1-α

(x,y)]·

frame(x,y),

where face(x,y) is said face image and frame(x,y) is said frame image.

3. The method of claim 2, wherein the scaling step (f) includes:

scaling the left and right temple segments L_L and L_R of said frame image to the temple lengths q_L -T_L and T_R -q_R of said face image, respectively, the scaled temple segments appearing in the composite image between points q_L and T_L and points T_R and q_R, respectively.

4. The method of claim 3, further comprising the steps of:

(j) assigning a fading coefficient α

₁ (x,y) to the scaled temple segments, the fading coefficient being defined by the expressions

space="preserve" listing-type="equation">α

.sub.1 (x,y)=k·

[x-T.sub.L (x)]/[q.sub.L -T.sub.L (x)]

(for left segment), and

space="preserve" listing-type="equation">α

.sub.1 (x,y)=k·

[x-T.sub.R (x)]/[q.sub.R -T.sub.R (x)]

(for right segment),

where k is a constant with a value in the range of about 0.5 to about 0.7; and

(k) combining said frame image with said face image along the scaled temple segments in accordance with the equation

space="preserve" listing-type="equation">C(x,y)=α

.sub.1 (x,y)·

face(x,y)+[1-α

.sub.1 (x,y)]·

frame(x,y),

whereby the temples gradually fade out in the composite image from the points q_L and q_R to the points T_L and T_R, respectively.

5. A method of combining an eyeglass frame image with a facial image, comprising the steps of:

(a) obtaining a digital image of a person'"'"'s face;

(b) selecting an eyeglass frame image, said frame image having frame points, frame edge points, background points;

(c) assigning a blending coefficient α

(x,y) to the frame points, to the frame edge points, and to the background points of said frame image, the value of said blending coefficient being set to zero for the frame points, to one for the background points, and to a value in the range of 0<

α

(x,y)<

1 for the frame edge points;

(d) scaling, if necessary, either said frame image or said face image so that the scale of said frame image is the same as that of said face image; and

(e) combining said frame image with said face image to produce a composite image, substantially in accordance with the equation

space="preserve" listing-type="equation">C(x,y)=α

(x,y)·

face(x,y)+[1-α

(x,y)]·

frame(x,y),

where face(x,y) is said face image and frame(x,y) is said frame image.

6. A method of combining an eyeglass frame image with a facial image, comprising the steps of:

(a) obtaining a digital image of a person'"'"'s face;

(b) selecting a digital eyeglass frame image, said frame image having left and right temple segments; and

(c) digitally combining said frame image with said face image to produce a composite image that portrays an eyeglass frame superimposed on a face, the eyeglass frame portrayed in said composite image having left and right temple segments that correspond the left and right temple segments of said frame image.

7. A method of detecting the irises and pupil centers of a person, in a digital image that includes the face of the person, said method comprising the steps of:

(a) providing a gray level image g(x,y) that encompasses the eyes of the person, one eye being in a left area of image g(x,y) and the other eye being in a right area of image g(x,y);

(b) converting the image to a binary image b(x,y) based on a dynamic threshold T determined by the expression, ##EQU14## where p is less than one, and is equal to a first value when the average brightness A of image g(x,y) is greater than a threshold brightness value, and is equal to a second value when A is less than or equal to the threshold brightness value, and where binary image b(x,y) is determined from the expression, ##EQU15## binary image b(x,y) being defined by the left and right areas of image g(x,y);

(c) searching the left area of the binary image for black objects and calculating the size and centroid coordinates of each black object found;

(d) searching the right area of the binary image for black objects and calculating the size and centroid coordinates of each black object found; and

(e) selecting a black object as a candidate for the iris in each of the left and right areas of the binary image, in accordance with the following selection criteria

(i) the black object selected in the left area is approximately the same size as the black object selected in the right area, and

(ii) the black object selected in the left area is approximately at the same level as the black object selected in the right area.

8. The method of claim 7, further comprising the steps of:

(f) locating, in the gray level image g(x,y), a plurality of circle edge points within a neighborhood of each iris candidate selected in step (e);

(g) converting the neighborhoods of each iris candidate to a binary image where the circle edge points are set to black and the other points in the neighborhood are set to white;

(h) comparing a plurality of circle patterns having known radii to the circle edge points in each of the neighborhoods, to determine the best match between the circle edge points and one of the plurality of circle patterns; and

(i) recording, for each neighborhood, the known radius of the circle pattern that best matches the circle edge points and recording the location of the pattern'"'"'s center where the best match occurred, whereby the recorded radii and center locations of the circle patterns are considered the iris radii and pupil centers, respectively, of the person'"'"'s eyes in the digital image g(x,y).

9. The method of claim 7, wherein the selection criteria in step (e) further includes

(iii) the distance between the black object selected in the left area is not too far or too close to the black object selected in the right area.

10. The method of claim 9, wherein the left and right areas of image g(x,y) are divided by a centerline, and wherein the selection criteria in step (e) further includes

(iv) the black object selected in the left area and the black object selected in the right area are the most symmetric about the centerline.

11. A method of detecting the shape of a person'"'"'s face in a digital image that includes the face of the person, said method comprising the steps of:

(a) determining the contour of the person'"'"'s face in the digital image;

(b) converting the person'"'"'s face contour into the frequency domain by a Fourier transformation of a space domain function representing the contour;

(c) providing a plurality of standard face contours represented in the frequency domain, each of the standard face contours having a name associated therewith;

(d) comparing the frequency domain representation of the person'"'"'s face contour with each of the plurality of standard face contours, to determine the best match; and

(e) assigning the name of the standard contour that was determined to be the best match in step (d), to the person'"'"'s face contour.

12. The method of claim 11, wherein said step of determining the person'"'"'s face contour includes the steps of:

(a) locating a nose point at the tip of the person'"'"'s nose in the digital image, the vertical coordinate of the nose point being situated on a vertical centerline;

(b) locating a mouth point at the person'"'"'s mouth in the digital image, the mouth point being substantially aligned with the nose point along the vertical centerline;

(c) locating a bottom chin point at the bottom of the person'"'"'s chin in the digital image, the bottom chin point being substantially aligned with the mouth point along the vertical centerline;

(d) locating left and right top chin points at the edge of the person'"'"'s face, the left chin point being to the left of the mouth point and the right chin point being to the right of the mouth point, the left and right top chin points and the mouth point being substantially horizontally aligned;

(e) constructing a contour of the chin in the digital image, by fitting a curve on the bottom chin point and the left and right top chin points;

(f) locating left and right face edge points at the edges of the person'"'"'s face in the digital image, the left and right face edge points being adjacent to the left and right temple areas, respectively, of the person'"'"'s face;

(g) constructing a contour of the left cheek in the digital image, by connecting the left temple point and the left top chin point with a substantially straight line;

(h) constructing a contour of the right cheek in the digital image, by connecting the right temple point and the right top chin point with a substantially straight line;

(i) locating a forehead point on the person'"'"'s forehead in a gray level representation of the digital image, by searching upward along the vertical centerline until a substantially black point is detected, the detected point being designated the forehead point; and

(j) locating a plurality of upper contour points around the person'"'"'s forehead in the gray level representation of the digital image, by searching for substantially black points left and right from the vertical centerline, line by line until reaching the forehead point, the detected points being designated the upper contour points, the upper contour points defining an upper contour of the person'"'"'s face;

whereby the person'"'"'s face contour is the composite of the chin contour, left and right cheek contours, and the upper contour.

13. A method of creating a composite digital image of an eyeglass frame being worn on a person'"'"'s face, said method comprising the steps of:

(a) providing a digital face image having first and second face edges;

(b) providing a digital frame image having first and second temple points, and further having first and second temple segments extending from the first and second temple points, respectively;

(c) superimposing said frame image on said face image such that the first and second temple points of said frame image are adjacent to the first and second face edges of said face image, respectively; and

(d) locating the first temple segment of said frame image between the first temple point of said frame image and the first face edge of said face image, and locating the second temple segment of said frame image between the second temple point of said frame image and the second face edge of said face image,

whereby a composite image is created showing an eyeglass frame worn on a person'"'"'s face, where the frame includes left and right temple portions extending back to the left and right ears of the person'"'"'s face.

14. The method of claim 13, wherein said step (c) includes aligning said frame image with said face image.

15. The method of claim 14, wherein said step (d) includes

(i) defining first and second temple points on said face image in accordance with the first and second temple points of said frame image, respectively,

(ii) locating the first temple segment of said frame image between the first temple point and the first face edge of said face image, and

(iii) locating the second temple segment of said frame image between the second temple point and the second face edge of said face image.

16. The method of claim 14, wherein said step (d) includes

(i) defining first and second temple points on said face image in accordance with the first and second temple points of said frame image, respectively,

(ii) scaling the first temple segment of said frame image to fit between the first temple point and the first face edge of said face image, and

(iii) scaling the second temple segment of said frame image to fit between the second temple point and the second face edge of said face image.

17. The method of claim 14, wherein the first and second face edges of said digital face image includes first and second face edge points, respectively;

and wherein said step (d) includes

(i) defining first and second temple points on said face image in accordance with the first and second temple points of said frame image, respectively,

(ii) scaling the first temple segment of said frame image to fit between the first temple point and the first face edge point of said face image, and

(iii) scaling the second temple segment of said frame image to fit between the second temple point and the second face edge point of said face image.

18. A method of creating a composite digital image of an eyeglass frame being worn on a person'"'"'s face, said method comprising the steps of:

(a) providing a digital face image having first and second face edge points and a pupil midpoint;

(b) providing a digital frame image having first and second temple points, a frame center point, and first and second temple segments;

(c) substantially aligning said frame image with said face image in accordance with the frame center point of said frame image and the pupil midpoint of said face image;

(d) defining first and second temple points on said face image based on the first and second temple points, respectively, of said digital frame image;

(e) determining a first temple length between the first temple point and the first face edge point of said face image, and determining a second temple length between the second temple point and the second face edge point of said face image;

(f) scaling said frame image to said face image, such that the first and the second temple segments of said frame image are scaled to the first and second temple lengths, respectively; and

(g) generating the composite digital image from said scaled frame image and said face image, said scaled frame image being superimposed on said face image, the scaled temple segments being superimposed on said face image between the temple points and the face edge points of said face image.

19. The method of claim 18, further comprising the step of:

(h) fading the image of the scaled temple segments of said frame image gradually from the temple points to the face edge points of said face image.

20. The method of claim 18, further comprising the step of:

(h) adjusting the alignment of said frame image with said face image.