Virtual try-on apparatus, virtual try-on method, and computer program product

• US 10,423,220 B2
• Filed: 03/30/2015
• Issued: 09/24/2019
• Est. Priority Date: 08/08/2014
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation), Toshiba Solutions Corporation (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Osada, Kunio, Dobashi, Toshimasa, Yoshioka, Hisao, Mikami, Shigeru
• Primary Examiner(s): Martello, Edward
• Application Number:
  US14/672,339
• Publication Number:
  US 20160042564A1
• Time in Patent Office: 1,639 Days
• Field of Search: None
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2210/16 Cloth

Abstract

According to an embodiment, a virtual try-on apparatus includes a first acquisition unit, a first display controller, an acceptor, a generator, and a second display controller. The first acquisition unit is configured to acquire characteristic information on a try-on subject. The first display controller is configured to display on a first display, clothing images corresponding to the acquired characteristic information in first information in which the characteristic information and the clothing images are associated with each other. The acceptor is configured to accept from the try-on subject a selection of an image of clothing to be tried on from among the clothing images displayed on the first display. The generator is configured to generate a composite image of a try-on subject image of the try-on subject and the selected clothing image. The second display controller is configured to display the composite image on a second display.

Claims

1. A virtual try-on apparatus comprising:

a first acquisition unit configured to acquire first characteristic information relating to a try-on subject, wherein the first characteristic information represents an age bracket in which the try-on subject resides, the try-on subject'"'"'s personality, the try-on subject'"'"'s preference, and a personal color similar to a skin color of the try-on subject, an eye color of the try-on subject, or a hair color of the try-on subject;

a first display controller configured to display, on a first display, multiple clothing images corresponding to the first characteristic information in first information in which the first characteristic information and the multiple clothing images are associated with each other;

an acceptor configured to accept a selection of a subset of the multiple clothing images to be tried on in combination from among the multiple clothing images displayed on the first display;

a generator configured to generate a composite image comprising a try-on subject image of the try-on subject and the subset of the multiple clothing images superimposed on the try-on subject image according to a first order of layers defined by order of layers information associated with the subset of the multiple clothing images; and

a second display controller configured to display the composite image on a second display,

wherein

the virtual try-on apparatus further comprises a first transmitter configured to transmit, to a first server device, try-on information including clothing IDs representing product numbers or clothing names of the multiple clothing images included in the composite image displayed on the second display, a try-on subject ID for uniquely identifying the try-on subject in the try-on subject image included in the composite image, and the clothing images identified by the clothing ID,

the second display controller repeatedly executes, during displaying of the composite image, generating of the composite image by combining the try-on subject image continuously shot by an image-capturing unit with the clothing images corresponding to the clothing IDs included in second information and corresponding to posture information calculated from a depth map of the try-on subject, the second information including clothing IDs of clothing to be tried on

each time a new composite image is generated, the second display controller switches the composite image displayed on the second display with the new composite image,

during generation and regeneration of the composite image, after removing an overlapping area between the try-on subject image or a clothing image in a lower layer, and a clothing image superimposed in an upper layer, the generator sequentially superimposes the subset of the multiple clothing images from the lower layer to the upper layer, and

the first display controller is configured to further display, on the first display, a recommended image extracted from the multiple clothing images according to a pre-decided extraction condition that is at least one of an image of clothing previously selected by another try-on subject having second characteristic information fitted to or similar to the first characteristic information of the try-on subject, an image of clothing recommended by a store selling clothing, an image of clothing recommended by a famous person or a celebrity preferred by the try-on subject, or an image of clothing previously selected by another try-on subject with a preference fitted to or similar to the try-on subject'"'"'s preference.

2. The apparatus according to claim 1, further comprising a second acquisition unit configured to acquire body shape parameters indicative of a body shape of the try-on subject, wherein

the generator is configured to generate the composite image of corrected images in which the subset of the multiple clothing images are corrected according to the acquired body shape parameters and the try-on subject image.

3. The apparatus according to claim 1, wherein the second display controller is configured to display the composite image on the first display as the second display.

4. The apparatus according to claim 1, wherein the first characteristic information comprises at least one of inner characteristics or outer characteristics of the try-on subject.

5. The apparatus according to claim 1, wherein the first characteristic information further represents one or more body shape parameters indicative of a body shape of the try-on subject.

6. The apparatus according to claim 1, wherein the recommended image is represented by a combination of one or more of the multiple clothing images.

7. The apparatus according to claim 1, wherein the pre-decided extraction condition is further at least one of:

the first characteristic information, an image of clothing previously selected by the try-on subject, an image of clothing recommended by another try-on subject and selected in advance by the try-on subject, or an image of clothing fitting to a body shape of the try-on subject or according to a body shape similar to the body shape of the try-on subject.

8. The apparatus according to claim 1, wherein

the multiple clothing images corresponding to the first information comprise a first clothing image representing worn clothing and a second clothing image representing clothing placed and arranged in shape,

the first display controller is configured to display on the first display the second clothing image corresponding to the first characteristic information in the first information, and

the generator is configured to generate another composite image of the try-on subject image and the first clothing image.

9. A virtual try-on apparatus comprising:

a first acquisition unit configured to acquire first characteristic information of a try-on subject, the first characteristic information representing an age bracket in which the try-on subject resides, the try-on subject'"'"'s personality, the try-on subject'"'"'s preference, and a personal color similar to a skin color of the try-on subject, an eye color of the try-on subject, or a hair color of the try-on subject;

a first display controller configured to display, on a first display, attribute information corresponding to the first characteristic information in first information in which the first characteristic information, multiple clothing images, and the attribute information relating to the multiple clothing images are associated with each other;

an acceptor configured to accept a selection of attribute information corresponding to multiple clothing items to be tried on from among the attribute information displayed on the first display;

a generator configured to generate a composite image comprising a try-on subject image of the try-on subject and a subset of the multiple clothing images corresponding to the multiple clothing items and superimposed on the try-on subject image according to a first order of layers defined by order of layers information associated with the subset of the multiple clothing images; and

a second display controller configured to display the composite image on a second display,

wherein

the virtual try-on apparatus further comprises a first transmitter configured to transmit, to a first server device, try-on information comprising clothing IDs representing product numbers or clothing names of the multiple clothing images included in the composite image displayed on the second display, a try-on subject ID for uniquely identifying the try-on subject in the try-on subject image included in the composite image, and the clothing images identified by the clothing IDs,

the second display controller repeatedly executes, during displaying of the composite image, generating of the composite image by combining the try-on subject image continuously shot by an image-capturing unit with the clothing images corresponding to the clothing IDs included in second information and corresponding to posture information calculated from a depth map of the try-on subject, the second information including clothing IDs of clothing to be tried on,

each time a new composite image is generated, the second display controller switches the composite image displayed on the second display with the new composite image,

during generation and regeneration of the composite image, after removing an overlapping area between the try-on subject image or a clothing image in a lower layer, and a clothing image superimposed in an upper layer, the generator sequentially superimposes the subset of the multiple clothing images from the lower layer to the upper layer, and

the first display controller is configured to further display, on the first display, a recommended image extracted from the multiple clothing images according to a pre-decided extraction condition that is at least one of an image of clothing previously selected by another try-on subject having second characteristic information fitted to or similar to the first characteristic information of the try-on subject, an image of clothing recommended by a store selling clothing, an image of clothing recommended by a famous person or a celebrity preferred by the try-on subject, or an image of clothing previously selected by another try-on subject with a preference fitted to or similar to the try-on subject'"'"'s preference.

10. The apparatus of claim 9, further comprising a second acquisition unit configured to acquire body shape parameters indicative of a body shape of the try-on subject, wherein

the generator is configured to generate the composite image of corrected images in which the subset of the multiple clothing images are corrected according to the acquired body shape parameters and the try-on subject image.

11. A virtual try-on method comprising:

acquiring first characteristic information of a try-on subject, wherein the first characteristic information defines an age bracket in which the try-on subject resides, a personality of the try-on subject, a preference of the try-on subject, and a personal color similar to a skin color of the try-on subject, an eye color of the try-on subject, or a hair color of the try-on subject;

displaying, on a first display, multiple clothing images corresponding to the first characteristic information in first information in which the first characteristic information and the multiple clothing images are associated with each other, wherein the displaying comprises at least displaying, as one of the multiple clothing images, a recommended image selected according to a pre-decided extraction condition that is at least one of an image of clothing previously selected by another try-on subject having second characteristic information fitted to or similar to the first characteristic information of the try-on subject, an image of clothing recommended by a store selling clothing, an image of clothing recommended by a famous person or a celebrity preferred by the try-on subject, or an image of clothing previously selected by another try-on subject with a preference fitted to or similar to the preference of the try-on subject;

accepting a selection of a subset of the multiple clothing images to be tried on in combination;

generating a composite image comprising a try-on subject image of the try-on subject and the subset of the multiple clothing images superimposed on the try-on subject image according to a first order of layers defined by order of layers information associated with the subset of the multiple clothing images;

displaying the composite image on a second display; and

transmitting, to a first server device, try-on information including clothing IDs representing product numbers or clothing names of the multiple clothing images included in the composite image displayed on the second display, a try-on subject ID for uniquely identifying the try-on subject in the try-on subject image included in the composite image, and the clothing images identified by the clothing IDs, wherein

during displaying of the composite image, generating of the composite image is repeatedly executed by combining the try-on subject image continuously shot with the clothing images corresponding to the clothing IDs included in second information and corresponding to posture information calculated from a depth map of the try-on subject, the second information including clothing IDs of clothing to be tried on,

each time a new composite image is generated, switching of the composite image displayed on the second display for the new composite image is performed, and

during generation and regeneration of the composite image, after removing an overlapping area between the try-on subject image or a clothing image in a lower layer, and a clothing image superimposed in an upper layer, the subset of the multiple clothing images are sequentially superimposed from the lower layer to the upper layer.

12. A computer program product comprising a non-transitory computer-readable medium containing a program executed by a computer, the program causing the computer to execute:

acquiring first characteristic information relating to a try-on subject, the first characteristic information defining an age bracket in which the try-on subject resides, a personality of the try-on subject, a preference of the try-on subject, and a personal color similar to a skin color of the try-on subject, an eye color of the try-on subject, or a hair color of the try-on subject;

displaying, on a first display, multiple clothing images corresponding to the first characteristic information in first information in which the first characteristic information and the multiple clothing images are associated with each other, wherein the displaying comprises at least displaying, as one of the multiple clothing images, a recommended image selected according to a pre-decided extraction condition that is at least one of an image of clothing previously selected by another try-on subject having second characteristic information fitted to or similar to the first characteristic information of the try-on subject, an image of clothing recommended by a store selling clothing, an image of clothing recommended by a famous person or a celebrity preferred by the try-on subject, or an image of clothing previously selected by another try-on subject with a preference fitted to or similar to the preference of the try-on subject;

accepting a selection of a subset of the multiple clothing images to be tried on in combination;

generating a composite image comprising a try-on subject image of the try-on subject and the subset of the multiple clothing images superimposed on the try-on subject image according to a first order of layers defined by order of layers information associated with the subset of the multiple clothing images;

displaying the composite image on a second display; and

transmitting, to a first server device, try-on information including clothing IDs representing product numbers or clothing names of the multiple clothing images included in the composite image displayed on the second display, a try-on subject ID for uniquely identifying the try-on subject in the try-on subject image included in the composite image, and the clothing images identified by the clothing IDs, wherein

during displaying of the composite image, generating of the composite image is repeatedly executed by combining the try-on subject image continuously shot with the clothing images corresponding to the clothing IDs included in second information and corresponding to posture information calculated from a depth map of the try-on subject, the second information including clothing IDs of clothing to be tried on,

each time a new composite image is generated, switching of the composite image displayed on the second display for the new composite image is performed, and

during generation and regeneration of the composite image, after removing an overlapping area between the try-on subject image or a clothing image in a lower layer, and a clothing image superimposed in an upper layer, the subset of the multiple clothing images are sequentially superimposed from the lower layer to the upper layer.