ALTERING PROPERTIES OF RENDERED OBJECTS VIA CONTROL POINTS

• US 20170358144A1
• Filed: 06/13/2016
• Published: 12/14/2017
• Est. Priority Date: 06/13/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Schwarz, Julia, Ahluwalia, Bharat, Calabrese, David, Pengelly, Robert CJ, Sheri, Yasaman, Tichenor, James
• Granted Patent: US 10,140,776 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 15/60 Shadow generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 2219/20 Indexing scheme for editing of 3D models
• CPC Class Codes: G06T 2219/2004 Aligning objects, relative positioning of parts
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling

Abstract

Altering properties of rendered objects and/or mixed reality environments utilizing control points associated with the rendered objects and/or mixed reality environments is described. Techniques described can include detecting a gesture performed by or in association with a control object. Based at least in part on detecting the gesture, techniques described can identify a target control point that is associated with a rendered object and/or a mixed reality environment. As the control object moves within the mixed reality environment, the target control point can track the movement of the control object. Based at least in part on the movement of the control object, a property of the rendered object and/or the mixed reality environment can be altered. A rendering of the rendered object and/or the mixed reality environment can be modified to reflect any alterations to the property.

Claims

1. A device comprising:

one or more processors;

memory;

one or more modules stored in the memory and executable by the one or more processors to perform operations comprising;

presenting, in a three-dimensional coordinate space, a rendered object via a display of the device;

detecting, in the three-dimensional coordinate space, a gesture performed by or in association with a control object;

identifying, based at least in part on the gesture, a target control point from a plurality of control points, each control point of the plurality of control points displayed in a position within a threshold distance of the rendered object;

tracking movement of the control object in the three-dimensional coordinate space;

causing the target control point to move with the movement of the control object in the three-dimensional coordinate space;

determining a displacement of the target control point from an original position to a new position in the three-dimensional coordinate space, the new position based at least in part on the movement of the control object;

altering a property of the rendered object based at least in part on the displacement; and

modifying a rendering of the rendered object to reflect an alteration of the property.

2. The device as claim 1 recites, wherein a processor of the one or more processors is configured to identify a state of the control object based at least in part on a machine-learned algorithm and the operations further comprise:

receiving a signal from the processor indicating the state; and

detecting the gesture based at least in part on the signal.

3. The device as claim 1 recites, wherein the device comprises a head-mounted device and the gesture comprises a pinch gesture that pinches the target control point in the three-dimensional coordinate space.

4. The device as claim 1 recites, wherein the target control point is a vertex of the rendered object, an edge of the rendered object, or a control that is proximate to the rendered object.

5. The device as claim 1 recites, the operations further comprising:

determining distances between the control object and the plurality of control points; and

identifying the target control point based at least in part on determining that a distance between the control object and the target control point is a shortest distance of the distances.

6. The device as claim 1 recites, the operations further comprising:

determining that the target control point is a gaze target of a user associated with the device; and

identifying the target control point based at least in part on determining that the target control point is the gaze target.

7. The device as claim 1 recites, the operations further comprising:

determining contextual data associated with the gesture; and

identifying the target control point based at least in part on the contextual data.

8. The device as claim 1 recites, the operations further comprising:

determining a release of the gesture; and

terminating tracking the movement of the control object.

9. The device as claim 1 recites, wherein the movement comprises at least one of movement along an x-axis, movement along a y-axis, movement along a z-axis, rotation around the x-axis, rotation around the y-axis, or rotation around the z-axis.

10. A computer-implemented method for altering one or more properties of a rendered object in a mixed reality environment via one or more control points associated with the rendered object, the computer-implemented method comprising:

presenting the rendered object in a three-dimensional coordinate space;

detecting, in the three-dimensional coordinate space, a gesture performed by or in association with a control object;

identifying, by a processor, a target control point of the one or more control points, the target control point being within a threshold distance of the gesture performed by or in association with the control object;

tracking movement of the control object in the three-dimensional coordinate space;

causing the target control point to move with the movement of the control object in the three-dimensional coordinate space;

altering a property of the rendered object based at least in part on the movement of the control object; and

modifying a rendering of the rendered object to reflect an alteration of the property.

11. The computer-implemented method as claim 10 recites, wherein the property comprises a scale of the rendered object.

12. The computer-implemented method as claim 10 recites, wherein the property comprises a three-dimensional structure of the rendered object.

13. The computer-implemented method as claim 10 recites, wherein the property comprises a position of the rendered object in the three-dimensional coordinate space.

14. The computer-implemented method as claim 10 recites, wherein the property comprises a rotation of the rendered object in the three-dimensional coordinate space.

15. The computer-implemented method as claim 10 recites, wherein the property is mapped to or otherwise associated with the target control point.

16. The computer-implemented method as claim 10 recites, wherein:

a second property is mapped to or otherwise associated with the target control point; and

the computer-implemented method further comprises;

altering the second property of the rendered object based at least in part on the movement of the control object; and

modifying the rendering of the rendered object to reflect an alteration of the second property.

17. The computer-implemented method as claim 10 recites, wherein:

the target control point is associated with a second rendered object in the mixed reality environment; and

the computer-implemented method further comprises;

altering a property of the second rendered object based at least in part on the movement of the control object; and

modifying a rendering of the second rendered object to reflect an alteration to the property of the second rendered object.

18. The computer-implemented method as claim 10 recites, further comprising:

determining a position of the control object relative to the target control point; and

presenting feedback to guide the gesture performed by or in association with the control object to be within the threshold distance of the target control point.

19. A computer-implemented method for altering one or more properties of a mixed reality environment, the computer-implemented method comprising:

presenting an object in a three-dimensional coordinate space;

presenting a plurality of control points in a three-dimensional coordinate space, the plurality of control points positioned within a threshold distance of the object;

identifying, by a processor, a target control point of the plurality of control points, the target control point being within a threshold distance of a control object;

tracking movement of the control object in the three-dimensional coordinate space;

causing the target control point to move with the movement of the control object in the three-dimensional coordinate space;

altering a property of the object based at least in part on the movement of the control object; and

modifying a rendering of the object to reflect an alteration of the property.

20. The computer-implemented method as claim 19 recites, further comprising:

determining a displacement of the target control point from an original position to a new position, the new position based at least in part on the movement of the control object; and

altering the property of the object proportional to the displacement.