Methods for extracting objects from digital images and for performing color change on the object

• US 9,269,157 B2
• Filed: 04/15/2014
• Issued: 02/23/2016
• Est. Priority Date: 03/01/2005
• Status: Active Grant

• Current Assignee: Eyesmatch, Ltd.
• Original Assignee: Eyesmatch, Ltd.
• Inventors: Saban, Ofer, Vilcovsky, Nissi
• Primary Examiner(s): Shankar, Vijay
• Application Number:
  US14/253,819
• Publication Number:
  US 20140226900A1
• Time in Patent Office: 679 Days
• Field of Search: 345204-215, 348/333.01, 434/395
• US Class Current: 1/1
• CPC Class Codes: G02B 5/08 Mirrors vehicle mirrors involving special optical features B60R1/08
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G09F 19/16 involving the use of mirrors
• CPC Class Codes: G09F 27/00 Combined visual and audible advertising or displaying, e.g. for public address
• CPC Class Codes: H04N 1/622 with simulation on a subsidiary picture reproducer
• CPC Class Codes: H04N 2005/2726 for simulating a person's appearance, e.g. hair style, glasses, clothes
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 5/2624 for obtaining an image which is composed of whole input images, e.g. splitscreen
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact
• CPC Class Codes: H04N 9/87 Regeneration of colour television signals H04N9/80 takes precedence

Abstract

Computerized method for separating an object in a digital image and for performing color change on an object within a digital image. The steps include: obtaining a digital image; receiving a selection of an object within the digital image; selecting a plurality of representative pixels estimated to be within the object; calculating a representative color from the plurality of representative pixels; selecting pixels of the digital image and for each pixel calculating a Euclidian distance to the representative color and, if the Euclidian distance is within a set threshold, identifying the pixel as belonging to the object. For color change, the steps include: generating a plurality of masks, each mask storing different property values of the collection of pixels; selecting a new color; applying the plurality of masks to the new color to generate new image of the object.

Claims

1. A computer system programmed for performing color change on an object within a digital image, comprising a memory storing a program for execution by a processor, the program including instructions for:

selecting pixels of the digital image and for each pixel calculating a Euclidian distance to the representative color and, if the Euclidian distance is within a set threshold, identifying the pixel as belonging to the object;

receiving a selection of object in the digital image to be recolored;

performing element selection to obtain collection of pixels that belong to the selected object;

generating a plurality of masks, each mask storing different property values of the collection of pixels;

selecting a new color;

applying the plurality of masks to the new color to generate new image of the object; and

, displaying the new image.

2. The computer system of claim 1, wherein the plurality of masks comprise:

a grayscale mask storing grayscale values of pixels belonging to the object, RGB mask storing RGB color values of pixels belonging to the object, XYZ mask storing XYZ values of pixels belonging to the object, and light reflection mask storing reflection values of pixels belonging to the object.

3. The computer system of claim 2, wherein the plurality of masks further comprise saturation mask storing saturation values of pixels belonging to the object.

4. The computer system of claim 1, further comprising instructions for calibration of the image and performing distortion transformation on the image prior to generating the plurality of masks.

5. The computer system of claim 1, wherein the program includes instructions for selection of the element by a user.

6. The computer system of claim 1, wherein the program includes instructions for changing the element appearance by modifying at least one of fabric, color, texture, reflection, absorption, brightness, and tones.

7. The computer system of claim 1, wherein the program includes instructions for rendering on a monitor augmented reality, comprising at least one of:

change of background, virtual dressing of user;

changing color of item in the image, and adding accessory.

8. The computer system of claim 1, wherein the program includes instructions for using infrared image to obtain user'"'"'s orientation body line.

9. The computer system of claim 1, wherein the program includes instructions for setting registration points on an image of a user.

10. The computer system of claim 9, wherein the program includes instructions for fetching a stored object from a database, and fitting the object on the image of the user by stretching or contracting the image of the stored object according to the registration points.

11. The computer system of claim 10, wherein the program includes instructions for modifying the image to render a mirrored image corresponding to an image obtained from a point of view of the user, to thereby mimic an image of a mirror.

12. The computer system of claim 1, wherein the program includes instructions for edge detection process.

13. The computer system of claim 1, wherein the program includes instructions for creating measurement signature of the body or body part.

14. The computer system of claim 1, wherein the program includes instructions for calibration of the image based on projective transformation of a camera used to obtain the digital image and based on a user distance to the camera, the height is being compensated so user will stay at the same size and orientation in the image a cross a wide range of distances from the camera.

15. The computer system of claim 1, wherein the program includes instructions for edge detection by one or more of the following edge-emphasize techniques:

color separation, depth information, gradient detection.

16. The computer system of claim 1, wherein the program includes instructions for storing analyzed data as a vector of data with weighting that will allow single or multiple attribute scoring comparison between users.

17. A non-transitory computer-readable storage medium storing a program for execution by a processor, the program including instructions for separating an object in a digital image by performing the steps comprising:

obtaining a digital image;

receiving a selection of an object within the digital image;

selecting a plurality of representative pixels estimated to be within the object;

calculating a representative color from the plurality of representative pixels;

selecting pixels of the digital image and for each pixel calculating a Euclidian distance to the representative color and, if the Euclidian distance is within a set threshold, identifying the pixel as belonging to the object.

18. The non-transitory computer-readable storage medium of claim 17, wherein the step of selecting pixels of the digital image comprises generating a vector from all of the pixels forming the digital image.

19. The non-transitory computer-readable storage medium of claim 17, wherein the step of calculating a representative color comprises transforming the RGB value of each selected pixel to an XYZ color space.

20. The non-transitory computer-readable storage medium of claim 19, wherein the step of calculating a representative color further comprises for each selected pixel calculating:

distance=√

((X−

xi(k))²+(Y−

yi(k))²+(Z−

zi(k))²)

wherein X, Y, and Z are average XYZ values of the selected pixels, xi, yi, and zi are the XYZ values of an i^th selected pixel, and k is the total number of selected pixels;

and wherein if a calculated distance of an i^th selected pixel is larger than a set threshold, the pixel is excluded from calculating the representative color.

21. The non-transitory computer-readable storage medium of claim 17, further comprising fetching an geometrical estimate of the object, and excluding from the calculation of Euclidian distance all pixels falling outside the geometrical estimate.

22. The non-transitory computer-readable storage medium of claim 17, further comprising obtaining an IR image and registering the IR image with the digital image so as to obtain an area in the digital image occupied by the object, and excluding from the calculation of Euclidian distance all pixels falling outside the area.

23. The non-transitory computer-readable storage medium of claim 17, further comprising an image texture of the object, and excluding from the calculation of Euclidian distance all pixels the image texture of the object.

24. The non-transitory computer-readable storage medium of claim 17, further comprising determining a boundary of the object, and labeling all pixels falling outside the boundary as not belonging to the object.

25. The non-transitory computer-readable storage medium of claim 17, further comprising determining a boundary of the object, and for each pixel having calculated Euclidian distance outside the set threshold, determining whether the pixel is within the boundary and, if within the boundary, re-labeling the pixel as belonging to the object.

26. The non-transitory computer-readable storage medium of claim 17, further comprising generating a plurality of masks, each corresponding to one property of image representing the object.

27. The non-transitory computer-readable storage medium of claim 26, wherein the masks are selected from:

grey scale, RGB, XYZ, black and white, hue, saturation, absorption, and brightness.