System and method for 3-dimension simulation of glasses

• US 20050162419A1
• Filed: 03/26/2003
• Published: 07/28/2005
• Est. Priority Date: 03/26/2002
• Status: Abandoned Application

• Current Assignee: So Woon Kim
• Original Assignee: So Woon Kim
• Inventors: Cho, Hang Shin, Choi, Sung Il, Yi, Seung Won, Kim, So Woon
• Application Number:
  US10/509,257
• Publication Number:
  US 20050162419A1
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G02C 13/003 Measuring during assembly or fitting of spectacles
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering

Abstract

A 3D virtual simulation system and method that provide decision-making information for selection and purchase of eyeglasses is presented. The system is comprised of four major units: 3D graphic simulation unit, contents delivery unit intelligent, Customer Relation Management (CRM) unit and back-office unit. 3D graphic simulation unit generates 3D face models of a user face and eyeglasses, and fit those objects automatically on networked platforms at real-time. The 3D face model is created from photo images of the face with options to select hair models. The 3D eyeglasses model is generated by a systematic reverse engineering process with specially designed measuring device. Graphic simulation unit transacts with intelligent CRM unit, so that user behavior is tracked down for push-marketing activity. Contents are delivered in a form of service-on-demand and ASP (Application Service Provider). This system enables precise virtual simulation of wearing eyeglasses with real dimensions of face and eyeglasses models and provides data and tools for custom-made production of eyeglasses assisted by expert knowledge base.

Claims

1. A virtual simulation system connected to a computer network to generate a 3D face model of a user, and to fit the face model and 3D eyeglasses models selected by the user, and to simulate them graphically with a database that stores the information of users, products, 3D models and knowledge base comprising:

a user data processing unit to identify the user who needs to have an access to simulation system, and to generate a 3D face model of the user;

a graphic simulation unit where a user can visualize 3D eyeglasses model that is generated as the user selects a product in the database, and to place and to fit automatically in 3D space on user'"'"'s face model created in user data processing module;

an intelligent CRM (Customer Relation Management) unit that can advise the user by a knowledge base that provides consulting information acquired by knowledge of fashion expert, purchase history and customer behavior on various products.

2. A system for 3D simulation of eyeglasses according to claim 1, wherein the user data processing unit comprises:

A user information management operative to identify authorized user who have a legal access to the system and to maintain user information at each transaction with database;

A 3D face model generation operative to create a 3D face model of a user by the information retrieved by the user.

3. A system for 3D simulation of eyeglasses according to claim 2, wherein the 3D face model generation operative comprises a data acquisition operative to generate a 3D face model of a user:

by a image capturing device connected to a computer;

or by retrieving front or front-and-side view of photo images of the face;

or by manipulating 3D face model stored in the database of 3D eyeglasses simulation system.

4. A system for 3D simulation of eyeglasses according to claim 2, wherein the 3D face model generation operative comprises a facial feature extraction operative to generate feature points of a base 3D model as a user input a outline profile and feature points of the face on a device that displays acquired photo images of the face, and to generate a base 3D model.

5. A system for 3D simulation of eyeglasses according to claim 2, wherein the 3D face model generation operative further comprises a 3D face model deformation operative to retrieve precise coordinates points by user interaction, and to deform a base 3D model by relative displacement of reference points from default location by calculated movement of feature points and other points in the vicinity.

6. A system for 3D simulation of eyeglasses according to claim 4, wherein the feature points of a face comprises predefined reference points on outline profile, eyes, nose, mouth and ears of a face.

7. A system for 3D simulation of eyeglasses according to claim 4, wherein the facial feature extraction operative comprises:

a face profile extraction operative to extract outline profile of 3D face model from the reference points input by the user;

a facial feature points extraction operative to extract feature points that characterize the face of the user from the reference points on of eyes, nose, mouth and ears input by the user

8. A system for 3D simulation of eyeglasses according to claim 4, wherein the 3D face model generation operative further comprises a facial expression operative to deform a 3D face model at-real time to generate human expressions under user'"'"'s control.

9. A system for 3D simulation of eyeglasses according to claim 4, wherein the 3D face model generation operative further comprises a face composition operative to create a new virtual model by combining a 3D face model of a user generated by the face model deformation operative with that of the others.

10. A system for 3D simulation of eyeglasses according to claim 4, wherein the 3D face model generation operative further comprises a face texture generation operative:

to retrieve texture information from photo images provided by a user;

to combine textures acquired from front and side view of the photo images;

to generate textures for the unseen part of head and face on the photo images.

11. A system for 3D simulation of eyeglasses according to claim 4, wherein the 3D face model generation operative further comprises a real-time preview operative to display 3D face and eyeglasses models with texture over the network, and to display deformation process of the models.

12. A system for 3D simulation of eyeglasses according to claim 4, wherein the 3D face model generation operative further comprises a file managing operative to create and save 3D face model in proprietary format and to convert 3D face model data into industry standard formats.

13. A system for 3D simulation of eyeglasses according to claim 1, wherein the graphic simulation unit comprises:

a 3D eyeglasses model management operative to retrieve and store 3D model information on the database by user interaction;

a texture generation operative to create colors and texture pattern of 3D eyeglasses models, and to store the data in the database, and to display textures of 3D models on a monitor generated in user data processing unit and eyeglasses modeling operative;

a virtual-try-on operative to place 3D eyeglasses and face model in 3D space and to display.

14. A system for 3D simulation of eyeglasses according to claim 13, wherein a 3D eyeglasses model management operative comprise:

an eyeglasses modeling operative to create a 3D model and texture of eyeglasses and to generate fitting parameters for virtual-try-on that include reference points for the gap distance between the eyes and lenses, hinges in eyeglasses and contact points on ears;

a face model control operative to match fitting parameters generated in eyeglasses modeling operative.

15. A system for 3D simulation of eyeglasses according to claim 13, wherein a 3D virtual-try-on operative comprises:

an automatic eyeglasses model fitting operative to deform a 3D eyeglasses model to match a 3D face model automatically at real-time on precise location by using fitting parameters upon user'"'"'s selection of eyeglasses and face model;

an animation operative to display prescribed animation scenarios to illustrate major features of eyeglasses models;

a real-time rendering operative to rotate, move, pan, and zoom 3D models by user interaction or by prescribed series of interaction.

16. A system for 3D simulation of eyeglasses according to claim 13, wherein the 3D virtual-try-on operative further comprises a custom-made eyeglasses simulation operative:

to build user'"'"'s own design by combining components of eyeglasses that include lenses, frames, hinges, temples and bridges from built-in library of eyeglasses models and texture;

to place imported images of user'"'"'s name or character to a specific location to build user'"'"'s own design;

to store simulated design in user data processing unit.

17. A system for 3D simulation of eyeglasses according to claim 1 further comprises a commerce transaction unit to operate a merchant process so that a user can purchase the products after trying graphic simulation unit.

18. A system for 3D simulation of eyeglasses according to claim 17, wherein the commerce transaction unit comprises:

a purchase management operative to manage orders and purchase history of a user;

a delivery management operative to verify order status and to forward shipping information to delivery companies;

a inventory management operative to manage the status of inventory along with payment and delivery process.

19. A system for 3D simulation of eyeglasses according to claim 1, wherein the intelligent CRM unit comprises:

a product preference analysis operative to analyze the preference on individual product by demographic characteristics of a user and of a category, and to store the analysis result on knowledge base;

a customer behavior analysis operative to analyze the characteristics of a user'"'"'s action on commerce contents, and to store the analysis result on knowledge base;

an artificial intelligent learning operative to integrate analysis for product preference and customer behavior with fashion trend information provided by experts in fashion, and construct raw data for advising service dedicated to a customer;

a fashion advise generation operative to create advising data from the knowledge base and store it to the database of 3D eyeglasses simulation system, and to deliver dedicated consulting information upon user'"'"'s demand that include design, style and fashion trend suited for a specific user;

an 1;

1 marketing data generation operative to acquire and manage demographic information of the user including email address or phone numbers and to publish promotional contents using 3D simulative features;

an 1;

1 marketing data delivery operative to deliver promotional contents to the multiple telecommunication form factors of the customer.

20. A system for 3D simulation of eyeglasses according to claim 19, the knowledge base comprises a database for log analysis and for advise on fashion trend.

21. A method for 3D simulation of eyeglasses for a 3D eyeglasses simulation system connected to a computer network to generate a 3D face model of a user, and to fit the face model and 3D eyeglasses models selected by the user, and to simulate them graphically with a database that stores the information of users, products, 3D models and knowledge base comprising:

a step to generate 3D face model of the user as the user transmit photo images of his or her face to the 3D eyeglasses simulation system, or as the user select one of 3D face model stored in said database;

a step to generate 3D eyeglasses model that selects one of 3D models stored in said database and generates 3D model parameters of said eyeglasses model for simulation;

a step to simulate virtual-try-on on display monitor that fits said 3D eyeglasses and face model by deforming eyeglasses model at-real time, and that displays combined 3D mages of eyeglasses and face model at different angles.

22. A method for 3D simulation of eyeglasses according to claim 21, the step to generate a 3D face model of the user comprises:

a step to display image information from the input provided by the user;

a step to extract an outline profile and feature points of said face as the user input base feature points on displayed image information;

a step to create a 3D face model by deforming base 3D model with a movement of base feature points observed during user interaction.

23. A method for 3D simulation of eyeglasses according to claim 22, the step to extract an outline profile and feature points of said face comprises:

a step to create a base snake as the user input base feature points that include facial features points along outline and featured parts of the face;

a step to define vicinity of said snake to move on each points along the snake to vertical direction;

a step to move said snake to the direction where color maps of the face in said image information exist.

24. A method for 3D simulation of eyeglasses according to claim 22, the step to extract outline profile and feature points of said face extract similarity between image information of featured parts of the face input by the user and that of predefined generic model.

25. A method for 3D simulation of eyeglasses according to claim 22, the step to create a 3D face model comprises:

a step to generate Sibson coordinates of the base feature points;

a step to calculate movement of the base feature points to that of said image information;

a step to calculate a new coordinates of the base feature points as a summation of coordinates of the default position and the calculated movement.

26. A method for 3D simulation of eyeglasses according to claim 22, the step to create a 3D face model comprises:

a step to calculate movement coefficients as a function of movement of the base feature points;

a step to calculate new positions of feature points in the vicinity of base points by multiplying movement coefficient.

27. A method for 3D simulation of eyeglasses according to claim 22 further comprises a step to generate facial expressions by deforming said 3D face model generated from said step to create a 3D face model and by using additional information provided by the user.

28. A method for 3D simulation of eyeglasses according to claim 27, the step to generate facial expressions comprises:

a step to compute the first light intensity on the entire points over the 3D face model;

a step to compute the second light intensity of the image information provided by the user;

a step to calculate the ERI (Expression Ratio Intensity) value with the ratio of said second light intensity over that of said second;

a step to warp polygons of the face model by using the ERI value to generate human expressions.

29. A method for 3D simulation of eyeglasses according to claim 22 further comprises a step to combine photo image information of the front and side view of the face, and to generate textures of the remaining parts of the head that are unseen by said photo image.

30. A method for 3D simulation of eyeglasses according to claim 29, the generate textures of remaining parts of the head comprises:

a step to generate Cartesian coordinates of said 3D face model and to generate texture coordinates of the front and side image of the face;

a step to extract a border of said two images and to project the border onto the front and side views to generate textures in the vicinity of the border on the front and side views;

a step to blend textures from the front and side views by referencing acquired texture on the border.

31. A method for 3D simulation of eyeglasses according to claim 29, before the step to generate 3D face model of the user, comprises:

the first step to check whether the user'"'"'s 3D face model has been registered before or not;

the second step to check whether the user will update registered models or not;

the third step to check whether the registered model has been generated by photo image provided by the user or by built-in 3D face model library;

the fourth step to load the selected model when it is generated form the information provided by the user.

32. A method for 3D simulation of eyeglasses according to claim 31 further comprises:

the fifth step to confirm whether the user will generate a new face model or not when a stored model does not exist;

the sixth step to display built-in default models when the user does not want to generate a new model;

the seventh to create an avatar from 3D face model generated by photo image of the user by installing dedicated software on personal computer when the software has not been installed before in case the user wants to generate a 3D face model;

the eighth step to register the avatar information and to proceed to the third step to check whether the model has been registered or not.

33. A method for 3D simulation of eyeglasses according to claim 31 proceeds to the seventh step and to complete remaining process when the user wants to update the 3D face model in the second step.

34. A method for 3D simulation of eyeglasses according to claim 31 further comprises a step to display the last saved model that has been selected in said third step.

35. A method for 3D simulation of eyeglasses according to claim 31 that checks whether the user has been registered or not as in said first step and identifies that the user is the first visitor comprises:

a step to check whether the user select one of built-in default models or not after providing login procedure;

a step to display selected default models on the monitor;

a step to check to proceed to said seventh step if the user does not select any of built-in default model.

36. A method for 3D simulation of eyeglasses according to claim 21 further comprises a step to select a design of frame and lenses, brand, color, materials or pattern from built-in library for the user.

37. A method for 3D simulation of eyeglasses according to claim 21, the step to generate 3D eyeglasses model that selects one of 3D models stored in the database further comprises a step to provide fashion advise information to the user by intelligent CRM unit can advise the user by a knowledge base that provides consulting information acquired by knowledge of fashion expert, purchase history and customer behavior on various products.

38. A method for 3D simulation of eyeglasses according to claim 21, the step to simulate on display monitor comprises:

a step to scale eyeglasses model with respect to X-direction, that is the lateral direction of the 3D face model, by referencing fitting points at eyeglasses and face model that consists of the distance between face and far end part of eyeglasses, hinges in eyeglasses and contact points on ears;

a step to transform coordinates of Y-direction, that is up and downward direction to the 3D face model, and Z-direction, that is front and backward direction to the 3D face model, with the scale calculated in X-direction;

a step deform temple part of the 3D eyeglasses model to match corresponding fitting points between 3D face and eyeglasses model.

39. A method for 3D simulation of eyeglasses according to claim 38 comprises the scale factor that scales the size of 3D eyeglasses model for automatic fitting represented by:

SF=X_B/X_B′

,
g=SF·

G

Where, SF is the scale factor, X_B′

is the X-coordinate of the fitting point B′

for the hinge part of 3D eyeglasses model and X_B is the X-coordinate of the corresponding fitting point B for the 3D face model, G is the size of original 3D eyeglasses model and g is a scaled size of the model in X-direction.

40. A method for 3D simulation of eyeglasses according to claim 38 comprises the movement in Y-direction to close the gap between the fitting point B for 3D face model and the scaled fitting point b′

by said scale factor for the hinge part of 3D eyeglasses model represented by;

41. A method for 3D simulation of eyeglasses according to claim 38 comprises the movement in Z-direction to close the gap between the fitting point A for 3D face model and the scaled fitting point a′

by said scale factor for the hinge part of 3D eyeglasses model represented by;

42. A method for 3D simulation of eyeglasses according to claim 38 comprises the rotation angle θ

_y in X-Z plane with respect to Y-axis represented by the angle calculated from cosine function represented by;

Cos θ

_y=Cos(∠

CB′

C′

)_X-Z

where, C is the fitting point for the vertical top point in the ear of the 3D face model that contacts with temple part of the 3D eyeglasses model, C′

is the corresponding fitting point for the temple part of the 3D eyeglasses model and B′

is the fitting point for the hinge part of the 3D eyeglasses.

43. A method for 3D simulation of eyeglasses according to claim 38 comprises the rotation angle θ

_x in Y-Z plane with respect to X-axis represented by the angle calculated from cosine function represented by;

Cos θ

_x=Cos(∠

CB′

C′

)_Y-Z

where, C is the fitting point for the vertical top point in the ear of the 3D face model that contacts with temple part of the 3D eyeglasses model, C′

is the corresponding fitting point for the temple part of the 3D eyeglasses model and B′

is the fitting point for the hinge part of the 3D eyeglasses.

44. A storage media to read a program to from a computer network to generate a 3D face model of a user, and to fit the face model and 3D eyeglasses models selected by the user, and to simulate them graphically with a database that stores the information of users, products, 3D models and knowledge base, to execute a program comprising:

an operative to generate 3D face model of the user as the user transmit photo images of his or her face to the 3D eyeglasses simulation system, or as the user select one of 3D face model stored in said database;

an operative to generate 3D eyeglasses model that selects one of 3D models stored in said database and generates 3D model parameters of said eyeglasses model for simulation;

an operative to simulate virtual-try-on on display monitor that fits said 3D eyeglasses and face model by transforming the Y and Z-coordinates of 3D eyeglasses model with the scale factor calculated from X-direction, using the gap distance between the eyes and the lenses and the fitting points for the ear part of the face model and for the hinge and the temple part of the eyeglasses model, and that displays combined 3D images of eyeglasses and face model at different angles.

45. A method to generate a 3D face model comprising:

(a) a step to input a 2D photo image of a face in front view and to display said image;

(b) a step to input at least one base points, on the said image, that characterizes a human face;

(c) a step to extract an outline profile and feature points for eyes, nose, mouth and ears that construct feature shapes of said face;

(d) a step to convert said input image information to a 3D face model using said outline profile and feature points.

46. A method to generate a 3D face model according to claim 45, the base points include at least one points in the outline profile of the face, and the step (c) to extract the outline profile of the face comprises:

(c1) a step to generate a base snake on said face information on said image referencing said base points;

(c2) a step to extract the outline profile by moving snake of the said face to the direction where textures of the face exist.

47. A method to generate a 3D face model according to claim 45, the base points include at least one points that correspond to eyes, nose, mouth and ears, and the step (c) to extract the outline profile of the face comprises:

a step to comprise a standard image information for a standard 3D face model;

(c2) a step to extract feature points of said input image by analyzing the similarity in image information of the featured shape and that of the standard image.

48. A method to generate a 3D face model according to claim 45, the step (a) to input said 2D image provides a facility to zoom in, zoom out or rotate said image upon user'"'"'s demand, and the step (b) comprises:

(b1) a step to input the size and degree of rotation of the said image by the user;

(b2) a step to generate a vertical center line for the face and to input base points for outline profile of the face, the step (c) comprises;

(c1) a step to generate base snake of the face by the said base points of the said image of the face;

(c2) a step to extract outline profile of the face by moving said snake to the direction where texture of the face exist;

(c3) a step to comprise standard image information for 3D face model;

(c4) a step to extract feature points of said input image by analyzing the similarity in image information of the featured shape and that of the standard image;

(c5) a step to display the outline profile or the feature points along the outline profile to the user, and to provide a facility to modify said profile or feature points, and to finalize the outline profile and feature points of said face.

49. A method to generate a 3D face model according to claim 45 further comprises:

(e) a step to generate 3D face model by deforming said face image information using the movement of base feature points in the standard image information to extracted feature points by user interaction on said face image.

50. A method to generate a 3D face model according to claim 49, the step (e) comprises:

(e1) a step to generate Sibson coordinates on the original position of the base points extracted from the step to deform said face model;

(e2) a step to calculate movements of each base points to the corresponding position of said image information;

(e3) a step to calculate a new position with a summation of coordinates of the original positions and said movements;

(e4) a step to generate 3D face model that corresponds to adjusted image information, by new positions, of said face.

51. A method to generate a 3D face model according to claim 49, the step (e) comprises:

(e1) a step to calculate the movement of base points;

(e2) a step to calculate new positions of base points and their vicinity that have by using said movement;

(e3) a step to generate 3D face model that corresponds to adjusted image information, by new positions, of said face.

52. A method to generate a 3D face model according to claim 45 further comprises:

(f) a step to generate facial expressions by deforming said 3D face model generated from said step to create a 3D face model and by using additional information provided by the user.

53. A method to generate a 3D face model according to claim 52, the step (f) comprises:

(f1) a step to compute the first light intensity on the entire points over the 3D face model;

(f2) a step to compute the second light intensity of the image information provided by the user;

(f3) a step to calculate the ERI (Expression Ratio Intensity) value with the ratio of said second light intensity over that of said second;

(f4) a step to warp polygons of the face model by using the ERI value to generate human expressions.

54. A method to generate a 3D face model according to claim 45 further comprises:

(g) a step to combine photo image information of the front and side view of the face, and to generate textures of the remaining parts of the head that are unseen by said photo image.

55. A method to generate a 3D face model according to claim 54, the step (g) comprises:

(g1) a step to generate Cartesian coordinates of said 3D face model and to generate texture coordinates of the front and side image of the face;

(g2) a step to extract a border of said two images and to project the border onto the front and side views to generate textures in the vicinity of the border on the front and side views;

(g3) a step to blend textures from the front and side views by referencing acquired texture on the border.

56. A method to generate a 3D face model according to claim 45 further comprises:

(h) a step to provide a facility for the user to select a hair models from a built-in library of 3D hair models, and to fit said hair model onto said 3D face model.

57. A method to generate a 3D face model according to claim 54, the step (h) comprises:

(h1) a step to comprise a library of 3D hair models in at least one category in hair style;

(h2) a step for the user to select a hair model from the built-in library of 3D hair models;

(h3) a step to extract a fitting point for the 3D hair model that matches the top position of the scalp on the vertical center line of said 3D face model;

(h4) a step to calculate the scale that matches to said 3D face model, and to fit 3D hair and face model together by using said fitting point for the hair.

115. A storage media to read a program from a computer to execute a method in claim 45 by a computer.

58. A method for 3D simulation of eyeglasses comprising:

(a) a step to acquire photographic image information from front, side and top views of eyeglasses placed in a cubic box with a measure in transparent material;

(b) a step to generate a base 3D model for eyeglasses by using measured value from said images or by combining components from a built-in library for 3D eyeglasses component models and textures;

(c) a step to generate a 3D lens model parametrically with the geometric information about lens shape, curvature, slope and focus angle;

(d) a step to generate a shape of the bridge and frame of eyeglasses by using measured value from said image and to combine said lenses, bridge and frame model together to generate a 3D complete model for eyeglasses.

59. A method for 3D simulation of eyeglasses according to claim 58, the step (c) comprises:

(c1) a step to acquire curvature information from said images or by specification of the product, and to create a sphere model that matches said curvature or predefined curvature preference;

(c2) a step to project the outline profile the lens to the surface of the sphere model and to trim out inner part of the projected surface.

60. A method for 3D simulation of eyeglasses according to claim 59 further comprises:

(c3) a step to generate thickness on trimmed surface of the lens.

61. A method for 3D simulation of eyeglasses according to claim 58, the step (d) comprises:

(d1) a step to display the base 3D model to the user, and to acquire input parameters for adjusting the 3D frame model, and to deform said frame model with acquired parameters;

(d2) a step to mirror said 3D lens model with respect to center line defined by user input or measured by said photo images and generate a pair of lenses in symmetry, and to generate a 3D bridge model with the parameters defined by user input or measured by said photo images.

62. A method for 3D simulation of eyeglasses according to claim 61, the step (d) further comprises:

(d3) a step to generate a connection part of the 3D frame model between temple and lens frame with the parameters defined by user input or measured by said photo images, or by the built-in 3D component library.

63. A method for 3D simulation of eyeglasses according to claim 58 further comprises:

(e) a step to generate temple part of the 3D frame model with the parameters defined by user input or measured by said photo images, or by the built-in 3D component library, while matching topology of said connection part and to convert automatically in a format of polygons;

(f) a step to deform temple part of the 3D frame model to match the curvature measured by said photo images or predefined curvature preference;

(g) a step to mirror said 3D temple model with respect to center line defined by user input or measured by said photo images and generate a pair of lenses in symmetry.

64. A method for 3D simulation of eyeglasses according to claim 58 further comprises:

(h) a step to generate a nose part, a hinge part, screws, bolts and nuts from with the parameters defined by user input or built-in 3D component library.

65. A method for 3D simulation of eyeglasses comprising:

(a) a step to comprise at least one 3D eyeglasses and 3D face model information;

(b) a step to select a 3D face model and 3D eyeglasses model by a user from said model information;

(c) a step to fit automatically said face and eyeglasses model at-real time;

(d) a step to compose a 3D image of said face and eyeglasses model, and to display generated said 3D image upon the user'"'"'s demand.

66. A method for 3D simulation of eyeglasses according to claim 65, the step (c) comprises:

(c1) a step to adjust to the scale of the 3D eyeglasses model in X-direction, that is the lateral direction of the 3D face model, with the fitting points for hinge part of the 3D eyeglasses model, for corresponding fitting points in 3D face model, for top center of the ear part of the 3D face model, for gap distance between eyes and lenses;

(c2) a step to transform the coordinates and the location of 3D eyeglasses model in Y-direction, that is up and downward direction to the 3D face model, and Z-direction, that is front and backward direction to the 3D face model, with the scale calculated in X-direction;

(c3) a step to deform temple part of the 3D eyeglasses model to match corresponding fitting points between 3D face and eyeglasses model.

67. A method for 3D simulation of eyeglasses according to claim 66, the step (c1) comprises the scale factor that scales the size of 3D eyeglasses model for automatic fitting represented by:

SF=X_B/X_B′

,
g=SF·

G

Where, SF is the scale factor, X_B′

is the X-coordinate of the fitting point B′

for the hinge part of 3D eyeglasses model and X_B is the X-coordinate of the corresponding fitting point B for the 3D face model, G is the size of original 3D eyeglasses model and g is a scaled size of the model in X-direction.

68. A method for 3D simulation of eyeglasses according to claim 67 comprises the movement in Y-direction to close the gap between the fitting point B for 3D face model and the scaled fitting point b′

by said scale factor for the hinge part of 3D eyeglasses model represented by;

69. A method for 3D simulation of eyeglasses according to claim 65 comprises the movement in Z-direction to close the gap between the fitting point A for 3D face model and the scaled fitting point a′

by said scale factor for the hinge part of 3D eyeglasses model represented by;

70. A method for 3D simulation of eyeglasses according to claim 65 comprises the rotation angle θ

_y in X-Z plane with respect to Y-axis represented by the angle calculated from cosine function represented by;

Cos θ

_y=Cos(∠

CB′

C′

)_X-Z

where, C is the fitting point for the vertical top point in the ear of the 3D face model that contacts with temple part of the 3D eyeglasses model, C′

is the corresponding fitting point for the temple part of the 3D eyeglasses model and B′

is the fitting point for the hinge part of the 3D eyeglasses.

71. A method for 3D simulation of eyeglasses according to claim 65 comprises the rotation angle θ

_x in Y-Z plane with respect to X-axis represented by the angle calculated from cosine function represented by;

Cos θ

_x=Cos(∠

CB′

C′

)_Y-Z

where, C is the fitting point for the vertical top point in the ear of the 3D face model that contacts with temple part of the 3D eyeglasses model, C′

is the corresponding fitting point for the temple part of the 3D eyeglasses model and B′

is the fitting point for the hinge part of the 3D eyeglasses.

72. A method for 3D simulation of eyeglasses according to claim 65, the step (c) comprises:

(c1) a step to input center points of the fitting region, NF, CF, DF, NG, HG and CG, in that 3D eyeglasses model and 3D face model contact each other, where NF is the center point of said 3D face model, CF is the center top of the ear part of said 3D face model that contacts the temple part of the 3D eyeglasses model during virtual-try-on, DF is the point at the top of the scalp, NG is the center of the nose part of said 3D face model that contacts the nose pad part of the 3D eyeglasses model during virtual-try-on, HG is the rotational center of hinge part of the 3D eyeglasses model and CG is the center of inner side of the temple part of the 3D eyeglasses model that contact said ear part of the 3D face model;

(c2) a step to obtain new coordinates set for said 3D eyeglasses model using said value of NF, CF, DF, NG, HG and CG that are need to fit eyeglasses on face model;

(c3) a step to fit said 3D eyeglasses model on said 3D face model automatically at-real time.

73. A method for 3D simulation of eyeglasses according to claim 72, the step (c2) comprises;

(c2i) a step to move said 3D eyeglasses model to proper position by using the difference of said NF and said NG;

(c2ii) a step for the user to input his or her own PD, pupillary distance, and to calculate PD value of said 3D face and corresponding value of 3D eyeglasses model;

(c2iii) a step to calculate the rotation angles for the template part of said eyeglasses model in horizontal plane to be fitted on said 3D face model by using said CF and HG value;

(c2iv) a step to deform 3D eyeglasses model and to fit on said 3D face model by using said values and angles.

74. A method for 3D simulation of eyeglasses according to claim 73, the step (c2ii) comprises a step to define a value between 63 and 72 millimeters without having input from the user.

109. A device for 3D simulation of eyeglasses according to claim 73, the step (c2ii) comprises a step to define a value between 63 and 72 millimeters without having input from the user.

75. An eyeglasses marketing method comprising:

(a) a step to generate 3D face model of a user a with a photo image of the face, and to generate image information to combine said 3D face model and stored 3D eyeglasses model, and to deliver said image information to a customer;

(b) a step to retrieve at least one selection of the 3D eyeglasses model by the user, and to manage purchase inquiry information of the eyeglasses, that corresponds to 3D eyeglasses model, inputted by the user;

(c) a step to analyze the environment where said purchase inquiry occurs including analysis or occasion of customer behavior on the corresponding inquiry and eyeglass product;

(d) a step to analyze the customer'"'"'s preference on eyeglasses product inquired and to manage the preference result;

(e) a step to forecast trend future trend of fashion driven from said analysis step for product preference and analysis result for customer behavior and acquired information on eyeglasses fashion;

(f) a step to acquire future trend of fashion by an artificial intelligent learning tool dedicated to fashion trend forecast, and to generate a knowledge base that advise suited design or proper fashion trend upon customer'"'"'s request;

(g) a step to generate a promotional contents for eyeglasses for a specific customer based on the integrated information about customer preference obtained from said customer behavior analysis tool, advising information generated by said knowledge base and artificial intelligent learning tool;

(h) a step to acquire and manage demographic information of the user including email address or phone numbers and to publish promotional contents using 3D simulative features, and to deliver promotional contents to the multiple telecommunication form factors of the customer.

76. An eyeglasses marketing method according to claim 75, the step (g) comprises:

a step to categorize customers by a predefined rule and to generate promotional contents according to said category.

77. An eyeglasses marketing method according to claim 75, the step (d) and (e) comprises analysis for the customer that includes at least one parameter for hair texture of 3D face model of the customer, lighting of the face, skin tone, width of the face, length of the face, size of the mouth, interpupillary distance and race of the customer.

78. An eyeglasses marketing method according to claim 75, the step (d) comprises the analysis for the eyeglasses product that includes at least one parameter for size of the frame and lenses, shape of the frame and lenses, material of the frame and lenses, color of the frame, color of the lenses, model year, brand and price.

79. An eyeglasses marketing method according to claim 75, the step (d) comprises analysis for the product preference that includes at least one parameter for seasonal trend in fashion, seasonal trend of eyeglasses shape, width of the face, race, skin tone, interpupillary distance, and hair style in the 3D face model.

116. A storage media to read a program from a computer to execute a method in claim 79 by a computer.

80. A device to generate a 3D face model comprising:

an operative to input a 2D photo image of a face in front view and to display said image and to input at least one base points, on the said image, that characterizes a human face;

an operative to extract an outline profile and feature points for eyes, nose, mouth and ears that construct feature shapes of said face;

an operative to convert said input image information to a 3D face model using said outline profile and feature points.

81. A device to generate a 3D face model according to claim 80, the base points include at least one points in the outline profile of the face, and said operative to extract the outline profile of the face comprises:

an operative to generate a base snake on said face information on said image referencing said base points;

an operative to extract the outline profile by moving snake of the said face to the direction where textures of the face exist.

82. A device to generate a 3D face model according to claim 80, the base points include at least one points that correspond to eyes, nose, mouth and ears, and the operative to extract the outline profile of the face comprises:

a database to comprise a standard image information for a standard 3D face model;

an operative to extract feature points of said input image by analyzing the similarity in image information of the featured shape and that of the standard image.

83. A device to generate a 3D face model according to claim 80, the operative to input said 2D image provides a facility to zoom in, zoom out or rotate said image upon user'"'"'s demand, retrieves the size and degree of rotation of the said image by the user, and generates a vertical center line for the face and to input base points for outline profile of the face, the operative to extract the outline profile of the face comprises:

an operative to generate base snake of the face by the said base points of the said image of the face and to extract outline profile of the face by moving said snake to the direction where texture of the face exist;

an operative to comprise a database of standard image information for 3D face model;

an operative to extract feature points of said input image by analyzing the similarity in image information of the featured shape and that of the standard image;

an operative to display the outline profile or the feature points along the outline profile to the user, and to provide a facility to modify said profile or feature points, and to finalize the outline profile and feature points of said face.

84. A device to generate a 3D face model according to claim 80 further comprises:

an operative to generate 3D face model by deforming said face image information using the movement of base feature points in the standard image information to extracted feature points by user interaction on said face image.

85. A device to generate a 3D face model according to claim 84, the operative to deform 3D face model comprises:

an operative to generate Sibson coordinates on the original position of the base points extracted from the operative to deform said face model;

an operative to calculate movements of each base points to the corresponding position of said image information;

an operative to calculate a new position with a summation of coordinates of the original positions and said movements;

(e4) an operative to generate 3D face model that corresponds to adjusted image information, by new positions, of said face.

86. A device to generate a 3D face model according to claim 84, the operative to deform 3D face model:

an operative to calculate the movement of base points;

an operative to calculate new positions of base points and their vicinity that have by using said movement;

an operative to generate 3D face model that corresponds to adjusted image information, by new positions, of said face.

87. A device to generate a 3D face model according to claim 80 further comprises an operative to generate facial expressions by deforming said 3D face model generated from said operative to create a 3D face model and by using additional information provided by the user.

88. A device to generate a 3D face model according to claim 87, the operative to generate facial expressions comprises:

an operative to compute the first light intensity on the entire points over the 3D face model;

an operative to compute the second light intensity of the image information provided by the user;

(f3) an operative to calculate the ERI (Expression Ratio Intensity) value with the ratio of said second light intensity over that of said second;

(f4) an operative to warp polygons of the face model by using the ERI value to generate human expressions.

89. A device to generate a 3D face model according to claim 80 further comprises:

an operative to combine photo image information of the front and side view of the face, and to generate textures of the remaining parts of the head that are unseen by said photo image.

90. A device to generate a 3D face model according to claim 89, the operative comprises:

an operative to generate Cartesian coordinates of said 3D face model and to generate texture coordinates of the front and side image of the face;

an operative to extract a border of said two images and to project the border onto the front and side views to generate textures in the vicinity of the border on the front and side views;

an operative to blend textures from the front and side views by referencing acquired texture on the border.

91. A device to generate a 3D face model according to claim 80 further comprises:

an operative to provide a facility for the user to select a hair models from a built-in library of 3D hair models, and to fit said hair model onto said 3D face model.

92. A device to generate a 3D face model according to claim 91, the operative comprises:

an operative to comprise a library of 3D hair models in at least one category in hair style;

an operative for the user to select a hair model from the built-in library of 3D hair models;

an operative to extract a fitting point for the 3D hair model that matches the top position of the scalp on the vertical center line of said 3D face model;

an operative to calculate the scale that matches to said 3D face model, and to fit 3D hair and face model together by using said fitting point for the hair.

93. A device to generate a 3D eyeglasses model comprising:

an operative to acquire photographic image information from front, side and top views of eyeglasses placed in a cubic box with a measure in transparent material;

an operative to generate a base 3D model for eyeglasses by using measured value from said images;

an operative to generate a 3D lens model parametrically with the geometric information about lens shape, curvature, slope and focus angle;

an operative to generate a shape of the bridge and frame of eyeglasses by using measured value from said image and to combine said lenses, bridge and frame model together to generate a 3D complete model for eyeglasses.

94. A device to generate a 3D eyeglasses model according to claim 93, the operative to generate a 3D lens model comprises:

an operative to acquire curvature information from said images and to create a sphere model that matches said curvature or predefined curvature preference;

an operative to project the outline profile the lens to the surface of the sphere model and to trim out inner part of the projected surface.

95. A device to generate a 3D eyeglasses model according to claim 94 further comprises:

an operative to generate thickness on trimmed surface of the lens.

96. A device to generate a 3D eyeglasses model according to claim 93, the operative to generate a 3D model comprises:

an operative to display the base 3D model to the user, and to acquire input parameters for adjusting the 3D frame model, and to deform said frame model with acquired parameters;

an operative to mirror said 3D lens model with respect to center line defined by user input or measured by said photo images and generate a pair of lenses in symmetry, and to generate a 3D bridge model with the parameters defined by user input or measured by said photo images.

97. A device to generate a 3D eyeglasses model according to claim 96, the operative to generate a 3D model comprises further comprises:

an operative to generate a connection part of the 3D frame model between temple and lens frame with the parameters defined by user input or measured by said photo images, or by built-in 3D component library.

98. A device to generate a 3D eyeglasses model according to claim 93 further comprises:

an operative to generate temple part of the 3D frame model while matching topology of said connection part and to convert automatically in a format of polygons;

an operative a step to deform temple part of the 3D frame model to match the curvature measured by said photo images or predefined curvature preference;

an operative a step to mirror said 3D temple model with respect to center line defined by user input or measured by said photo images and generate a pair of lenses in symmetry.

99. A device to generate a 3D eyeglasses model according to claim 93 further comprises:

an operative to generate a nose part, a hinge part, screws, bolts and nuts from with the parameters defined by user input or built-in 3D component library.

100. A device for 3D simulation of eyeglasses comprising:

a database that comprises at least one 3D eyeglasses and 3D face model information;

an operative to select a 3D face model and 3D eyeglasses model by a user from said model information;

an operative to fit automatically said face and eyeglasses model at-real time;

an operative to compose a 3D image of said face and eyeglasses model, and to display generated said 3D image upon the user'"'"'s demand.

101. A device for 3D simulation of eyeglasses according to claim 100, the operative to fit eyeglasses model comprises:

an operative to adjust to the scale of the 3D eyeglasses model in X-direction, that is the lateral direction of the 3D face model, with the fitting points for hinge part of the 3D eyeglasses model, for corresponding fitting points in 3D face model, for top center of the ear part of the 3D face model, for gap distance between eyes and lenses;

an operative to transform the coordinates and the location of 3D eyeglasses model in Y-direction, that is up and downward direction to the 3D face model, and Z-direction, that is front and backward direction to the 3D face model, with the scale calculated in X-direction;

an operative to deform temple part of the 3D eyeglasses model to match corresponding fitting points between 3D face and eyeglasses model.

102. A device for 3D simulation of eyeglasses according to claim 101, the operative to adjust the scale comprises the scale factor that scales the size of 3D eyeglasses model for automatic fitting represented by:

SF=X_B/X_B′

,
g=SF·

G

Where, SF is the scale factor, X_B′

is the X-coordinate of the fitting point B′

for the hinge part of 3D eyeglasses model and X_B is the X-coordinate of the corresponding fitting point B for the 3D face model, G is the size of original 3D eyeglasses model and g is a scaled size of the model in X-direction.

103. A device for 3D simulation of eyeglasses according to claim 102 comprises the movement in Y-direction to close the gap between the fitting point B for 3D face model and the scaled fitting point b′

by said scale factor for the hinge part of 3D eyeglasses model represented by;

104. A device for 3D simulation of eyeglasses according to claim 101 comprises the movement in Z-direction to close the gap between the fitting point A for 3D face model and the scaled fitting point a′

by said scale factor for the hinge part of 3D eyeglasses model represented by;

105. A device for 3D simulation of eyeglasses according to claim 101 comprises the rotation angle θ

_y in X-Z plane with respect to Y-axis represented by the angle calculated from cosine function represented by;

Cos θ

_y=Cos(∠

CB′

C′

)_X-Z

where, C is the fitting point for the vertical top point in the ear of the 3D face model that contacts with temple part of the 3D eyeglasses model, C′

is the corresponding fitting point for the temple part of the 3D eyeglasses model and B′

is the fitting point for the hinge part of the 3D eyeglasses.

106. A device for 3D simulation of eyeglasses according to claim 101 comprises the rotation angle θ

_x in Y-Z plane with respect to X-axis represented by the angle calculated from cosine function represented by;

Cos θ

_x=Cos(∠

CB′

C′

)_Y-Z

where, C is the fitting point for the vertical top point in the ear of the 3D face model that contacts with temple part of the 3D eyeglasses model, C′

is the corresponding fitting point for the temple part of the 3D eyeglasses model and B′

is the fitting point for the hinge part of the 3D eyeglasses.

107. A device for 3D simulation of eyeglasses according to claim 100, the operative to fit 3D eyeglasses comprises:

an operative to input center points of the fitting region, NF, CF, DF, NG, HG and CG, in that 3D eyeglasses model and 3D face model contact each other, where NF is the center point of said 3D face model, CF is the center top of the ear part of said 3D face model that contacts the temple part of the 3D eyeglasses model during virtual-try-on, DF is the point at the top of the scalp, NG is the center of the nose part of said 3D face model that contacts the nose pad part of the 3D eyeglasses model during virtual-try-on, HG is the rotational center of hinge part of the 3D eyeglasses model and CG is the center of inner side of the temple part of the 3D eyeglasses model that contact said ear part of the 3D face model;

an operative to obtain new coordinates set for said 3D eyeglasses model using said value of NF, CF, DF, NG, HG and CG that are need to fit eyeglasses on face model;

an operative to fit said 3D eyeglasses model on said 3D face model automatically at-real time.

108. A device for 3D simulation of eyeglasses according to claim 107, the operative to obtain new coordinates comprises;

an operative to move said 3D eyeglasses model to proper position by using the difference of said NF and said NG;

an operative a step for the user to input his or her own PD, pupillary distance, and to calculate PD value of said 3D face and corresponding value of 3D eyeglasses model;

an operative a step to calculate the rotation angles for the template part of said eyeglasses model in horizontal plane to be fitted on said 3D face model by using said CF and HG value;

an operative a step to deform 3D eyeglasses model and to fit on said 3D face model by using said values and angles.

110. A device for marketing of eyeglasses comprising:

an operative to generate 3D face model of a user a with a photo image of the face, and to generate image information to combine said 3D face model and stored 3D eyeglasses model, and to deliver said image information to a customer;

an operative to retrieve at least one selection of the 3D eyeglasses model by the user, and to manage purchase inquiry information of the eyeglasses, that corresponds to 3D eyeglasses model, inputted by the user;

an operative to analyze the environment where said purchase inquiry occurs including analysis or occasion of customer behavior on the corresponding inquiry and eyeglass product;

an operative to analyze the customer'"'"'s preference on eyeglasses product inquired and to manage the preference result;

an operative to forecast trend future trend of fashion driven from said analysis step for product preference and analysis result for customer behavior and acquired information on eyeglasses fashion;

an operative to acquire future trend of fashion by an artificial intelligent learning tool dedicated to fashion trend forecast, and to generate a knowledge base that advise suited design or proper fashion trend upon customer'"'"'s request;

an operative to generate a promotional contents for eyeglasses for a specific customer based on the integrated information about customer preference obtained from said customer behavior analysis tool, advising information generated by said knowledge base and artificial intelligent learning tool;

an operative to acquire and manage demographic information of the user including email address or phone numbers, and to deliver promotional contents to the customer as an 1;

1 marketing tool.

111. A device for marketing of eyeglasses according to claim 110, the operative to provide 1:

1 marketing tool comprises;

an operative to categorize customers by a predefined rule and to generate promotional contents according to said category and to publish promotional contents using 3D simulative features for eyeglasses.

112. A device for marketing of eyeglasses according to claim 110 comprises analysis for the customer that includes at least one parameter for hair texture of 3D face model of the customer, lighting of the face, skin tone, width of the face, length of the face, size of the mouth, interpupillary distance and race of the customer.

113. A device for marketing of eyeglasses according to claim 110 comprises the analysis for the eyeglasses product that includes at least one parameter for size of the frame and lenses, shape of the frame and lenses, material of the frame and lenses, color of the frame, color of the lenses, model year, brand and price.

114. A device for marketing of eyeglasses according to claim 110 comprises analysis for the product preference that includes at least one parameter for seasonal trend in fashion, seasonal trend of eyeglasses shape, width of the face, race, skin tone, interpupillary distance, and hair style in the 3D face model