Three-dimensional beauty simulation client-server system

• US 20010037191A1
• Filed: 03/15/2001
• Published: 11/01/2001
• Est. Priority Date: 03/15/2000
• Status: Abandoned Application

• Current Assignee: Infiniteface Incorporated
• Original Assignee: Infiniteface Incorporated
• Inventors: Miyazawa, Takeo, Furuta, Hima
• Application Number:
  US09/808,207
• Publication Number:
  US 20010037191A1
• Time in Patent Office: Days
• US Class Current: 703/6
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes
• CPC Class Codes: G06T 7/596 from three or more stereo images
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

It is an object of the invention to provide a three-dimensional beauty simulation client-server system which is capable of handling a users face in a three-dimensional fashion and of providing more realistic beauty simulations. This system comprises a shop-based client that obtains and transmits three-dimensional shape data regarding a user, and a server comprising a makeup simulation unit that receives and stores the three-dimensional shape data from the shop-based client and carries out makeup simulation based on the three-dimensional shape data in response to requests from the user, and a data control unit that analyzes the users operation record and generates administrative information.

Claims

1. A three-dimensional beauty simulation client-server system comprising:

a shop-based client that obtains and transmits three-dimensional shape data regarding a user; and

a server that comprises a makeup simulation unit that receives and stores said three-dimensional shape data from said shop-based client and carries out makeup simulation based on said three-dimensional shape data in response to the users requests, and a data control unit that analyzes the users operation record and generates administrative information.

2. The three-dimensional beauty simulation client-server system according to

claim 1

, further comprising a client that can access said servers wherein said server provides a makeup simulation in response to requests from said client.

3. The three-dimensional beauty simulation client-server system according to

claim 1

, further comprising a cellular telephone that has a data transmission function and can access said server, wherein said server provides a makeup simulation in response to requests from said cellular telephone.

4. The three-dimensional beauty simulation client-server system according to

claim 1

, wherein said server further comprises a member registration unit that stores member registration information, and wherein said server provides makeup simulations to users registered beforehand in said member registration unit.

5. The three-dimensional beauty simulation client-server system according to

claim 1

, wherein said server transmits the operation record and/or administrative information regarding said users via a computer network.

6. The three-dimensional beauty simulation client-server system according to

claim 1

, wherein said shop-based client comprises:

a plurality of cameras to obtain images of the user as seen from a plurality of viewpoints;

a corresponding point search unit that receives each item of image data obtained from the plurality of cameras, analyzes the plurality of images, and searches for corresponding points that correspond to each other;

a three-dimensional shape recognition unit that analyzes the searched corresponding points and recognizes the three-dimensional shape of the target object;

a geometric calculation unit that sets a line of sight based on the recognition results from said three-dimensional shape recognition unit, and generates an image from a prescribed line of sight through geometric conversion of the data based on the set line of sight;

a display unit that displays the image generated by said geometric calculation unit; and

communication means that transmits the image data generated by said geometric calculation unit to said server.

7. The three-dimensional beauty simulation client-server system according to

claim 6

, wherein said corresponding point search unit and said geometric calculation unit comprises:

a feature point extraction unit that extracts feature points from each of said plurality of images;

a correlation calculating unit that seeks correlation among the feature points of said plurality of images and seeks combinations of said feature points;

a matching unit that discards combinations having a low feasibility from among said combinations of feature points based on the condition that the images were seen from said plurality of viewpoints;

a camera orientations determining unit that seeks the positions of said plurality of viewpoints and the directions of the lines of sight; and

a match propagation unit that, under the conditions imposed by the positions of said plurality of viewpoints and the direction of said lines of sight obtained by said camera orientations determining unit, selects combinations of feature points starting with those having superior geometric and statistical reliability and adjusts the analysis range of the images of said target object.

8. The three-dimensional beauty simulation client-server system according to

claim 6

, wherein said corresponding point search unit and said geometric calculation unit comprises:

a feature point extraction unit that extracts feature points from each of said plurality of images;

a correlation calculating unit that seeks correlation among the feature points of said plurality of images and seeks combinations of said feature points;

a matching unit that discards combinations having a low feasibility from among said combinations of feature points based on the condition that the images were seen from said plurality of viewpoints;

a camera orientations determining unit that seeks the positions of said plurality of viewpoints and the directions of the lines of sight;

a match propagation unit that, under the conditions imposed by the positions of said plurality of viewpoints and the direction of said lines of sight obtained by said camera orientations determining unit, selects combinations of feature points starting with those having superior geometric and statistical reliability and adjusts the analysis range of the images of said target object;

a resampling unit that normalizes the matching map obtained by said match propagation unit;

a three-dimensional position measurement unit that determines the position of said/target object in a three-dimensional space based on the normalized matching map; and

a view interpolation unit that generates images seen from viewpoints different from said plurality of viewpoints based on the determined three-dimensional position of said target object.

9. The three-dimensional beauty simulation client-server system according to

claim 6

, wherein said corresponding point search unit and said geometric calculation unit comprises:

a feature point extraction unit that extracts feature points from each of said plurality of images;

a correlation calculating unit that seeks correlation among the feature points of said plurality of images and seeks combinations of said feature points;

a matching unit that discards combinations having a low feasibility from among said combinations of feature points based on the condition that the images were seen from said plurality of viewpoints; and

a match propagation unit that, under the geometric constraints imposed by the lines of sight, selects combinations of feature points starting with those having superior geometric and statistical reliability and adjusts the analysis range of the images of said target object.

10. The three-dimensional beauty simulation client-server system according to

claim 6

, wherein said corresponding point search unit and said geometric calculation unit comprises:

a feature point extraction unit that extracts feature points from each of said plurality of images;

a correlation calculating unit that seeks correlation among the feature points of said plurality of images and seeks combinations of said feature points;

a matching unit that discards combinations having a low feasibility from among said combinations of feature points based on the condition that the images were seen from said plurality of viewpoints;

a match propagation unit that, under the geometric constraints imposed by the lines of sight, selects combinations of feature points starting with those having superior geometric and statistical reliability and adjusts the analysis range of the images of said target object;

a resampling unit that normalizes the matching map obtained by said match propagation unit;

a three-dimensional position measurement unit that determines the position of said target object in a three-dimensional space based on the normalized matching map; and

a view interpolation unit that generates images seen from viewpoints different from said plurality of viewpoints based on the determined three-dimensional position of said target object.

11. A three-dimensional beauty simulation server comprising a makeup simulation unit that receives and stores three-dimensional shape data of a user from a shop-based client and carries out makeup simulation based on the three-dimensional shape data in response to requests from the user, and a data control unit that analyzes the operation record for said user and generates administrative information,

wherein said makeup simulation unit comprises a receiving unit that receives said three-dimensional shape data, a database that stores the received three-dimensional shape data, and a makeup simulation providing unit that provides a makeup simulation in response to requests for such simulation;

and

wherein said data control unit of said server comprises a user information analyzer that receives the operation history of the user and analyzes the trends therein, a control database that stores the analyzed data, an information processing unit that reads out data from the control database in response to external requests and processes the data in accordance with said requests, and a transmitting/receiving unit that transmits the output of said information processing unit to the requesting source and receives requests from the requesting source.

12. The three-dimensional beauty simulation server according to

claim 11

, wherein said makeup simulation providing unit analyzes the condition of the users facial skin and the light and dark areas that indicate the protrusions and indentations thereon, and evaluates the users facial expression based on the results of such analysis.

13. The three-dimensional beauty simulation server according to

claim 11

, wherein said makeup simulation providing unit obtains a facial image of the user, displays for the user a plurality of target facial images stored beforehand for allowing the user to select one of these images, combines said user facial image and said target facial image in a plurality of predetermined ratios, and supplies a plurality of combined facial images to the user.

14. The three-dimensional beauty simulation server according to

claim 11

, wherein said makeup simulation providing unit supplies facial images seen from freely chosen viewpoints.