Augmented reality skin evaluation

• US 9,990,772 B2
• Filed: 01/31/2014
• Issued: 06/05/2018
• Est. Priority Date: 01/31/2014
• Status: Active Grant

• Current Assignee: Koninklijke Philips N.V.
• Original Assignee: Empire Technology Development LLC (Allied Inventors Management, LLC)
• Inventors: Ur, Shmuel, Medan, Yoav
• Primary Examiner(s): Buttram, Todd
• Application Number:
  US14/384,870
• Publication Number:
  US 20160253841A1
• Time in Patent Office: 1,586 Days
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2024 Style variation
• CPC Class Codes: G09G 5/026 Control of mixing and/or overlay of colours in general G09G5/022 and G09G5/024 take precedence

Abstract

Technologies are generally described for methods and systems effective to produce an evaluation image. In some examples, a processor may receive initial image data generated by light reflected from an object, that corresponds to a real object at a first instance of time. The processor may apply a first skin to modify pixels of features in the initial image data to produce intermediate data and apply a second skin to pixels of features in the intermediate data to produce augmented data. The skins may modify pixels in image data. The processor may receive evaluation image data, used to evaluate execution of the first or second skin, that corresponds to the real object at a second instance of time. The processor may apply the first skin to the evaluation image data to generate evaluation data different from the augmented data.

Claims

1. A method to produce an evaluation image, the method comprising, by a processor:

receiving initial image data at a first instance of time, wherein the initial image data corresponds to a real object;

applying a first skin to the received initial image data to produce first intermediate image data that corresponds to the first skin;

determine first difference image data between the first intermediate image data and the received initial image data for identification of the first skin;

applying a second skin to the first intermediate image data to produce augmented data;

determine second difference image data between the augmented data and the first intermediate image data for identification of the second skin;

rendering an augmented image based on the augmented data;

receiving a first input to evaluate at least one of the first skin or the second skin in the rendered augmented image;

displaying an evaluation image that corresponds to a first difference image related to the first difference image data or that corresponds to a second difference image related to the second difference image data, wherein the first difference image is displayed when the received first input corresponds to the evaluation of the first skin, and wherein the second difference image is displayed when the received first input corresponds to the evaluation of the second skin;

receiving a second input to modify the first skin in the first difference image when the first difference image is displayed, or to modify the second skin in the second difference image when the second difference image is displayed;

in response to receiving the second input, modifying the first skin in the first difference image when the first difference image is displayed, or modifying the second skin in the second difference image when the second difference image is displayed;

applying the modified first skin or the modified second skin to the augmented image; and

rendering the augmented image with one of the modified first skin or the modified second skin.

2. The method of claim 1, wherein receiving the first input includes identifying a position of a marker in a user interface.

3. The method of claim 1, wherein the first skin includes instructions and skin data.

4. The method of claim 1, wherein receiving the first input to evaluate at least one of the first skin or the second skin in the rendered augmented image comprises receiving the first input that corresponds to an area having an attribute that is different from an attribute of rest of the rendered augmented image.

5. The method of claim 1, wherein modifying the first skin in the first difference image or the second skin in the second difference image includes at least one of repositioning, resizing, and removing of the first skin or the second skin.

6. The method of claim 1, further comprising:

in response to receiving the first input to evaluate at least one of the first skin or the second skin, retrieving the first difference image or the second difference image.

7. A system effective to produce an evaluation image, the system comprising:

a memory;

a camera;

an augmented reality display; and

a processor configured to be in communication with the memory, the camera, and the augmented reality display, wherein;

the camera is configured to capture initial image data at a first instance of time, wherein the initial image data corresponds to a real object, and

the processor is configured to;

store the initial image data in the memory;

apply a first skin to the initial image data to produce first intermediate image data that corresponds to the first skin;

determine first difference image data between the first intermediate image data and the initial image data for identification of the first skin;

apply a second skin to the first intermediate image data to produce augmented data;

determine second difference image data between the augmented data and the first intermediate image data for identification of the second skin;

render an augmented image based on the augmented data on the augmented reality display;

receive a first input to evaluate at least one of the first skin or the second skin in the rendered augmented image;

display an evaluation image that corresponds to a first difference image related to the first difference image data or that corresponds to a second difference image related to the second difference image data, wherein the first difference image is displayed when the received first input corresponds to the evaluation of the first skin, and wherein the second difference image is displayed when the received first input corresponds to the evaluation of the second skin;

receive a second input to modify the first skin in the first difference image when the first difference image is displayed, or to modify the second skin in the second difference image when the second difference image is displayed;

in response to the received second input, modify the first skin in the first difference image when the first difference image is displayed, or modify the second skin in the second difference image when the second difference image is displayed;

apply the modified first skin or the modified second skin to the augmented image; and

render the augmented image with one of the modified first skin or the modified second skin.

8. The system of claim 7, wherein receipt of the first input includes identification of a position of a marker in a user interface.

9. The system of claim 7, wherein the augmented reality display includes a marker movable along a slider in a first direction and a second direction.

10. The system of claim 9, wherein the marker is movable along the slider to apply a particular skin to a particular image data.

11. The system of claim 7, wherein modification of the first skin in the first difference image or the second skin in the second difference image includes at least one of repositioning, resizing, and removing of the first skin or the second skin.

12. The system of claim 7, further comprising:

in response to receiving the first input to evaluate at least one of the first skin or the second skin, retrieving the first difference image or the second difference image.

13. A method to produce an evaluation image, the method comprising, by a processor:

identifying augmented reality instructions effective to;

apply a first skin to initial image data to produce first intermediate image data, wherein the initial image data corresponds to a real object; and

apply a second skin to the first intermediate image data to produce augmented data;

rendering an augmented image based on the augmented data;

receiving a first input to evaluate the augmented image, wherein the first input corresponds to an area in the rendered augmented image;

receiving evaluation image data, wherein the evaluation image data corresponds to the real object and includes the area of the rendered augmented image;

receiving a second input based on a positioning of a marker to a first position in a slider to apply one of the first skin and the second skin to the evaluation image data;

in response to the second input, applying one of the first skin and the second skin to the evaluation image data to generate evaluation data, wherein the evaluation data is different from the augmented data;

rendering the evaluation image based on the evaluation data, wherein the evaluation image corresponds to a first difference image that is displayed when the received first input corresponds to an evaluation of the first skin in the augmented image, and wherein the evaluation image corresponds to a second difference image that is displayed when the received second input corresponds to the evaluation of the second skin in the augmented image.

14. The method of claim 13, wherein:

the first skin includes instructions and skin data;

applying the first skin to the evaluation image data comprises;

identifying a reference point in the evaluation image data based on the instructions;

associating a portion of the evaluation image data with the reference point; and

replacing the portion of the evaluation image data with the skin data; and

receiving the first input includes;

receiving an identification of an area in the evaluation image data; and

determining that the reference point corresponds to the area identified in the first input.

15. The method of claim 13, wherein receiving the first input to evaluate the augmented image comprises receiving the first input that corresponds to an area having an attribute that is different from an attribute of rest of the rendered augmented image.