Clothing image processing device, clothing image display method and program

• US 9,996,909 B2
• Filed: 05/10/2013
• Issued: 06/12/2018
• Est. Priority Date: 08/30/2012
• Status: Active Grant

• Current Assignee: Rakuten Group, Inc.
• Original Assignee: Rakuten, Inc. (Rakuten Group, Inc.)
• Inventors: Masuko, Soh, Hayashi, Yasuyuki, Hoshi, Tetsuya
• Primary Examiner(s): Fadok, Mark A
• Application Number:
  US14/424,444
• Publication Number:
  US 20150206292A1
• Time in Patent Office: 1,859 Days
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20004 Adaptive image processing
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2210/16 Cloth
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G09G 2320/10 Special adaptations of display systems for operation with variable images
• CPC Class Codes: G09G 2340/0464 Positioning
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: G09G 5/36 characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: G09G 5/38 with means for controlling the display position
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

A clothing image processing device acquires an image of a first item, the first item being placed in a projection direction of a projection device and worn on one of a plurality of parts of a human body; measures a position of a specific portion of the first item in the acquired image; determines a position of a second item based on the measured position of the specific portion and on a specific portion of the second item; adjusts, based on a position of a projection target onto which the projection device projects and the position of the second item, a size and position of an image of the second item that is to be projected onto the projection target; and controls the projection device so that the adjusted image of the second item is projected.

Claims

1. A clothing image processing system, comprising:

a camera;

a projector;

at least one processor; and

at least one memory device that stores a plurality of instructions, which when executed by the at least one processor, causes the at least one processor to;

acquire a photographed image of a first item from the camera, the first item being placed in a projection direction of the projector and worn on a first part of a human body, the first part being one of a plurality of parts of the human body;

measure a position of a specific portion of the first item in the acquired image, wherein the specific portion of the first item is determined by the first part;

acquire, in an image of a second item which is an item to be worn on a second part, a position of a specific portion of the second item, the second part being one of the plurality of parts of the human body and being different from the first part;

acquire a relative position of the second part from the first cart based on an information indicating human model which is a model of user'"'"'s body;

determine a position to which the specific portion of the second item is projected based on the measured position of the specific portion of the first item and on the acquired relative position;

recognize a projection target onto which the projector projects;

adjust, based on a distance from the recognized projection target and the position of the specific portion of the second item in the image of the second item, a size and position of the image of the second item that is read out of an item information storing unit and that is to be projected onto the projection target so that a projected position of the specific portion of the second item overlaps with the determined position; and

control the projector so that the adjusted image of the second item is projected.

2. The clothing image processing system according to claim 1,

wherein the first part and the second part are different parts,

wherein the processor of the clothing image processing system is further caused to:

acquire information that indicates a relation between a position of a portion of a human body that corresponds to the specific portion of the first item and a position of a portion of the human body that corresponds to the specific portion of the second item, and

determine, based on the position of the specific portion of the first item and on the information that indicates the relation, the position of the specific portion of the second item when the first item is worn on the human body.

3. The clothing image processing system according to claim 2,

wherein the processor is further caused to determine the position of the specific portion of the second item to be projected in longitudinal direction based on the position of the specific portion of the first item in longitudinal direction and on a distance to the projection target.

4. The clothing image processing system according to claim 1, wherein, based on information about a size of the second item, the processor is further caused to adjust the size of the image of the second item so that the size of the projected image of the second item matches the size of the second item.

5. The clothing image processing system according to claim 1, wherein the processor is further caused to:

detect a position and tilt of a projection surface onto which the image of the second item is projected, and

correct a shape of the image of the second item to be projected onto the projection surface, depending on the detected position and tilt of the projection surface.

6. The clothing image processing system according to claim 1, wherein the processor is further caused to processes the image of the second item so that the first item is prevented from overlapping with the projected image of the second item.

7. The clothing image processing system according to claim 1, wherein the processor is further caused to determine the first part on which the first item is to be worn, based on the photographed image of the first item.

8. The clothing image processing system according to claim 1, wherein the processor is further caused to change a mode of display of the second item based on a result of comparison between a size of a user with respect to clothing and the size of the second item.

9. The clothing image processing system according to claim 1, wherein the processor is further caused to switch the second item to a new second item based on photographed body movement of a user.

10. The clothing image processing system according to claim 1, wherein the processor is further caused to advance electronic commerce processing with respect to the second item, based on photographed body movement of a user.

11. The clothing image processing system according to claim 1,

wherein the processor is further caused to:

recognize a distance between the projection device and a position of the second item to be projected, and

adjust the size of the second item to be projected based on the recognized distance.

12. A clothing image display method which uses a camera, a processor and a memory device that stores a plurality of instructions, the method comprising the steps of:

acquiring a photographed image of a first item from the camera, the first item being placed in a projection direction of a projection device and worn on a first part of a human body, the first part being one of a plurality of parts of the human body;

measuring a position of a specific portion of the first item in the acquired photographed image, wherein the specific portion of the first item is determined by the first part;

acquiring, in an image of a second item which is an item to be worn on a second part, a position of a specific portion of the second item, the second part being one of the plurality of parts of the human body and being different from the first part;

acquiring a relative position of the second part from the first part based on an information indicating human model which is a model of user'"'"'s body;

determining a position to which the specific portion of the second item is projected based on the measured position of the specific portion of the first item and on the acquired relative position;

recognizing a projection target onto which the projection device projects;

adjusting, based on a distance from the recognized projection target and the position of the specific portion of the second item in the image of the second item, a size and position of the image of the second item that is read out of an item information storing unit and that is to be projected onto the projection target so that a projected position of the specific portion of the second item overlaps with the determined position; and

controlling the projection device so that the adjusted image of the second item is projected.

13. A computer-readable non-transitory storage medium having stored thereon a program for causing a computer to:

acquire a photographed image of a first item from a camera, the first item being placed in a projection direction of a projection device and worn on a first part of a human body, the first part being one of a plurality of parts of the human body;

measure a position of a specific portion of the first item in the acquired image, wherein the specific portion of the first item is determined by the first part;

acquire, in an image of a second item which is an item to be worn on a second part, a position of a specific portion of the second item, the second part being one of the plurality of parts of the human body and being different from the first part;

acquire a relative position of the second part from the first part based on an information indicating human model which is a model of user'"'"'s body;

determine a position to which the specific portion of the second item is projected based on the measured position of the specific portion of the first item and on the acquired relative position;

recognizing a projection target onto which the projection device projects;

adjust, based on a distance from the recognized projection target and the position of the specific portion of the second item in the image of the second item, a size and position of the image of the second item that is read out of an item information storing unit and that is to be projected onto the projection target so that a projected position of the specific portion of the second item overlaps with the determined position; and

control the projection device so that the adjusted image of the second item is projected.