Multi-step placement of virtual objects

• US 10,692,287 B2
• Filed: 04/17/2017
• Issued: 06/23/2020
• Est. Priority Date: 04/17/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Evans, David Anthony, Scott, Jason Bradley, Chevrier, Jedd, Copic, John, Paulovich, Jonathan Gustav, Schutz, Timothy James
• Primary Examiner(s): Choi, David E
• Application Number:
  US15/489,682
• Publication Number:
  US 20180300952A1
• Time in Patent Office: 1,163 Days
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/453 Help systems
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]

Abstract

A technique is described herein for placing a virtual object within any type of modified-reality environment. The technique involves receiving the user'"'"'s specification of plural values in plural stages. The plural values collectively define an object display state. The technique places the virtual object in the modified-reality environment in accordance with the object display state. Overall, the technique allows the user to place the virtual object in the modified-reality environment with high precision and low ambiguity by virtue of its guided piecemeal specification of the object display state.

Claims

1. A method, implemented by one or more computing devices, the method comprising:

receiving first input information in response to a first input action performed by a user when engaging a modified-reality environment;

generating first placement information based at least on the first input information;

displaying a guide to the user within the modified-reality environment, the guide being located in the modified-reality environment based at least on the first placement information;

moving a virtual object along the guide in response to user input received while the guide is displayed and restricting movement of the virtual object to points along the guide;

receiving second input information in response to a second input action performed by the user;

generating second placement information based at least on the second input information, the second placement information specifying a particular point on the guide displayed in the modified-reality environment at which to place the virtual object; and

placing the virtual object in the modified-reality environment at the particular point on the guide displayed in the modified-reality environment, as specified by the second placement information.

2. The method of claim 1, further comprising receiving at least one of the first input information or the second input information by one or more of:

projecting a ray defined by a gaze of the user into the modified-reality environment;

interpreting a voice command issued by the user;

interpreting a bodily gesture made by the user based at least on image information that captures the bodily gesture; and

/or

interpreting a control signal emitted by a controller operated by the user.

3. The method of claim 1, further comprising:

prior to receiving the first input information, displaying a grid within the modified-reality environment,

wherein the first input information is received in response to interaction by the user with the grid.

4. The method of claim 1, further comprising:

based at least on the first placement information, placing the virtual object at an initial position on the guide in the modified-reality environment prior to receiving the second input information.

5. The method of claim 1, further comprising:

receiving the first input information in response to a selection by the user of another point on a selected surface of the modified-reality environment.

6. The method of claim 1, further comprising:

displaying the guide as a straight line that extends from a first point in the modified-reality environment, the first point being specified by the first placement information, and

receiving the second input information in response to selection by the user of a second point that lies on the straight line, the selection of the second point occurring after the virtual object has been moved along the straight line in response to the user input,

wherein the second point is the particular point on the guide where the virtual object is placed.

7. The method of claim 6,

wherein the first point lies on a surface of the modified-reality environment, and

wherein the straight line is normal to the surface at the first point.

8. The method of claim 1, further comprising:

presenting an additional guide;

receiving an instance of additional input information in response to interaction by the user with the additional guide;

generating an instance of additional value information in response to the instance of additional input information; and

governing a visual characteristic of the virtual object based at least on the instance of additional value information.

9. The method of claim 8, the visual characteristic being a size of the virtual object.

10. The method of claim 8, the visual characteristic being a rotation of the virtual object in the modified-reality environment with respect to at least one axis of rotation.

11. The method of claim 8,

wherein the additional guide corresponds to a control element having a range of selection points along an axis, and

wherein the instance of additional input information is received in response to selection by the user of a particular selection point that lies on the axis.

12. The method of claim 11, wherein the particular selection point governs a rate of change of the visual characteristic of the virtual object.

20. The method of claim 1, further comprising:

after moving the virtual object to the particular point on the guide and placing the virtual object, removing the guide from the modified-reality environment.

21. The method of claim 1, further comprising:

fixing the guide at a specific point along a planar surface in the modified-reality environment, the specific point specified by the first placement information, the guide extending away from the planar surface in a particular direction.

22. The method of claim 21, the particular direction being perpendicular to the planar surface.

13. One or more computing devices comprising:

a hardware processor; and

storage storing machine-readable instructions which, when executed by the hardware processor, cause the hardware processor to;

present a modified-reality environment for display on a display device;

receive first input information in response to a first input action performed by a user while engaging the modified-reality environment;

generate first placement information based at least on the first input information;

present a guide to the user within the modified-reality environment, the guide being located in the modified-reality environment based at least on the first placement information;

in response to movement inputs received while the guide is presented, move a virtual object along the guide while restricting movement of the virtual object to points along the guide;

receive second input information in response to a second input action performed by the user in response to interaction by the user with the guide presented in the modified-reality environment;

generate second placement information based at least on the second input information, the second placement information specifying a particular point on the guide presented in the modified-reality environment at which to place the virtual object; and

place the virtual object in the modified-reality environment at the particular point on the guide presented in the modified-reality environment, as specified by the second placement information.

14. The one or more computing devices of claim 13, wherein the machine-readable instructions, when executed by the hardware processor, cause the hardware processor to:

prior to receiving the first input information, present an initial guide within the modified-reality environment and receive the first input information in response to interaction by the user with the initial guide.

15. The one or more computing devices of claim 13, wherein the machine-readable instructions, when executed by the hardware processor, cause the hardware processor to:

receive the first input information based at least on a selection by the user of another point on a selected surface of the modified-reality environment.

16. The one or more computing devices of claim 13, wherein the guide corresponds to a straight line that extends vertically from a first point in the modified-reality environment, the first point being specified by the first placement information, and wherein the second input information designates a second point that lies on the straight line.

17. The one or more computing devices of claim 13, wherein the machine-readable instructions, when executed by the hardware processor, cause the hardware processor to:

present an additional guide within the modified-reality environment;

receive an instance of additional input information in response to interaction by the user with the additional guide;

generate an instance of additional value information in response to the instance of additional input information; and

adjust a visual characteristic of the virtual object based at least on the additional value information.

18. The one or more computing devices of claim 17, wherein the visual characteristic comprises a size or rotation of the virtual object in the modified-reality environment.

19. A computer-readable storage medium storing computer-readable instructions, the computer-readable instructions, when executed by one or more processor devices, causing the one or more processor devices to perform acts comprising:

presenting a modified-reality environment for display;

receiving first input information in response to a first user selection of a first point on a surface of the modified-reality environment;

generating first placement information based at least on the first input information;

presenting a guide within the modified-reality environment, the guide corresponding to a straight line that extends from the first point on the surface;

moving a virtual object along the straight line in response to user input received while the straight line is presented and restricting movement of the virtual object to points along the straight line when moving the virtual object;

receiving second input information in response to a second user selection of a second point on the straight line presented in the modified-reality environment;

generating second placement information based at least on the second input information, the second placement information specifying a position of the virtual object at the second point on the straight line presented in the modified-reality environment; and

placing the virtual object in the modified-reality environment at the second point on the straight line presented in the modified-reality environment.