ATTRIBUTE DETECTION TOOLS FOR MIXED REALITY

• US 20180075658A1
• Filed: 12/30/2016
• Published: 03/15/2018
• Est. Priority Date: 09/15/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Lanier, Jaron, Amores Fernandez, Judith
• Granted Patent: US 10,325,407 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes

Abstract

Techniques described herein include mixed reality tools, referred to as HoloPaint, that allow use of any of a variety of sensors to determine physical parameters of real objects in a mixed reality environment. HoloPaint may correlate current measurements of the real world with past measurements to perform inventory management, analysis of changes of physical parameters of real objects and environments, and so on. A user may select which parameter to analyze by selecting a particular type of virtual paint, such as for drawing onto an object to be analyzed.

Claims

1. A method comprising:

receiving a first set of values for a physical attribute of an object measured at a first time;

receiving instructions to virtually paint a portion of the object in a mixed reality environment;

measuring a second set of values for the physical attribute of the portion of the object at a second time in the mixed reality environment;

comparing the second set of values to the first set of values to produce a comparison; and

determining a type of paint to apply onto the object, wherein the determining is based, at least in part, on the instructions and the comparison.

2. The method as claim 1 recites, wherein the type of paint is based at least in part on color, brightness, albedo, or hue.

3. The method as claim 1 recites, further comprising:

determining a placement location of the paint, wherein the determining is based, at least in part, on the instructions.

4. The method as claim 1 recites, wherein the instructions comprise drawing instructions that designate the portion of the object to be painted.

5. The method as claim 1 recites, wherein the object is a real object.

6. The method as claim 1 recites, wherein the physical attribute is a kinematic parameter.

7. The method as claim 1 recites, wherein the first set of values and the second set of values for the physical attribute comprise respective mappings of the physical attribute over at least a portion of the object.

8. A method comprising:

receiving stored inventory data for a set of objects in a first state;

virtually painting at least a portion of the set of objects in a second state in a mixed reality environment;

determining current inventory data for the virtually painted at least the portion of the set of objects in the second state;

comparing the current inventory data to the stored inventory data to produce a comparison; and

determining a change in inventory for the set of objects in the second state relative to the set of objects in the first state, wherein the determining is based, at least in part, on the comparison.

9. The method as claim 8 recites, wherein the set of objects in the first state comprises a first number of the objects and the set of objects in the second state comprises a second number of the objects, wherein the second number is different from the first number.

10. The method as claim 8 recites, wherein the stored inventory data and the current inventory data comprise a location and number of individual objects of the set of objects in the first state and the second state, respectively.

11. The method as claim 8 recites, wherein virtually painting at least a portion of the set of objects in the second state comprises drawing a virtual line across the at least the portion of the set of objects in the second state.

12. The method as claim 8 recites, further comprising:

causing virtual paint to be applied to portions of the set of objects in the second state that are different from corresponding portions of the set of objects in the first state to distinguish from portions of the set of objects in the second state that are same as corresponding portions of the set of objects in the first state.

13. The method as claim 8 recites, further comprising:

causing virtual paint to be applied to portions of the set of objects in the second state that are same as corresponding portions of the set of objects in the first state to distinguish from portions of the set of objects in the second state that are different from corresponding portions of the set of objects in the first state.

14. The method as claim 8 recites, wherein the stored inventory data comprises an image captured at a time that the set of objects is in the first state.

15. The method as claim 8 recites, wherein the set of objects are real objects.

16. The method as claim 8 recites, wherein the comparing the current inventory data to the stored inventory data comprises comparing a subset of the current inventory data to the stored inventory data, wherein the subset is based, at least in part, on at least one physical attribute of the set of objects in the second state.

17. A system comprising:

a mixed reality display device operable in a mixed reality environment; and

a device communicatively coupled to the mixed reality display device, the device comprising;

one or more processors;

memory; and

one or more modules stored in the memory and executable by the one or more processors to perform operations comprising;

determining differences between a first map of a physical parameter of an object and a second map of the physical parameter of the object; and

virtually painting a particular region of the object, wherein the location of the particular region is based, at least in part, on the determined differences.

18. The system as claim 17 recites, wherein the first map of the physical parameter of the object comprises first values of the physical parameter at a first time and the second map of the physical parameter of the object comprises second values of the physical parameter at a second time.

19. The system as claim 17 recites, wherein the physical parameter is a kinematic attribute of the object.

20. The system as claim 17 recites, wherein the physical attribute is a compound physical attribute comprising two or more parameters of the object.