Virtual object manipulation

• US 9,256,282 B2
• Filed: 03/20/2009
• Issued: 02/09/2016
• Est. Priority Date: 03/20/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Latta, Stephen G., Geisner, Kevin, Markovic, Relja, Bennett, Darren Alexander, Tomlin, Arthur Charles
• Primary Examiner(s): Chow, Doon
• Assistant Examiner(s): CHOI, DAVID E
• Application Number:
  US12/408,589
• Publication Number:
  US 20100241998A1
• Time in Patent Office: 2,517 Days
• Field of Search: 715/856, 345/419, 345/167, 345/164, 345/163, 345/161, 345/157, 345/156, 341/20, 108/56
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Systems, methods and computer readable media are disclosed for manipulating virtual objects. A user may utilize a controller, such as his hand, in physical space to associate with a cursor in a virtual environment. As the user manipulates the controller in physical space, this is captured by a depth camera. The image data from the depth camera is parsed to determine how the controller is manipulated, and a corresponding manipulation of the cursor is performed in virtual space. Where the cursor interacts with a virtual object in the virtual space, that virtual object is manipulated by the cursor.

Claims

1. A method for manipulating a first virtual object in a virtual space, the first virtual object being associated with a physical object in a first physical space, the physical object being separate from a first user, comprising:

receiving first depth image data indicative of the first physical space;

generating the first virtual object in the virtual space based on the physical object as represented in the first depth image data;

in response to receiving second depth image data indicative of the first physical space where the physical object is being moved by the first user, displaying a corresponding moving of the first virtual object in the virtual space; and

in response to receiving third depth image data indicative of a second user in a second physical space, the second physical space differing from the first physical space in which the physical object is located, and in response to determining that the third depth image data is indicative of the second user attempting to move a second virtual object in the virtual space and the first virtual object in the virtual space without the second user having moved the physical object in the first physical space, determining to display movement of the second virtual object and not to display movement of the first virtual object.

2. The method of claim 1, wherein the virtual space comprises a representation of the first physical space, a virtual sub-space, a representation of the second physical space, or a combination thereof.

3. The method of claim 1, wherein there is a scale of movement, and wherein displaying a corresponding moving of the first virtual object comprises:

displaying the corresponding moving of the first virtual object based on an amount that the physical object is moved modified by the scale of movement.

4. The method of claim 1, wherein the virtual space is presented to the first user in a first-person perspective or a third-person perspective.

13. The method of claim 1, further comprising:

in response to receiving fourth depth image data indicative of the physical object being moved in the first physical space by the first user, determining not to display a corresponding moving of the first virtual object where the moving of the first virtual object conflicts with a boundary of the virtual space.

5. A system for interacting with a first virtual object in a virtual environment, the first virtual object being associated with a physical object in a first physical space, the physical object being separate from a first user, comprising:

a memory bearing instructions that, upon execution by a processor, cause the system at least to;

generate the first virtual object based in the virtual environment on the physical object as represented in first depth image data;

in response to receiving second depth image data indicative of the first physical space where the physical object is being moved by the first user, display a corresponding moving of the first virtual object in the virtual environment; and

in response to receiving third depth image data indicative of a second user in a second physical space, the second physical space differing from the first physical space in which the physical object is located, and in response to determining that the third depth image data is indicative of the second user attempting to move a second virtual object in the virtual environment and the first virtual object in the virtual environment without the second user having moved the physical object in the first physical space, determine to display movement of the second virtual object and not to display movement of the first virtual object.

7. The system of claim 5, wherein the virtual environment comprises a representation of the first physical space, a virtual sub-space, a representation of the second physical space, or a combination thereof.

8. The system of claim 5, wherein there is a scale of movement, and wherein the instructions that, upon execution by the processor, cause the system to at least display a corresponding moving of the first virtual object further cause the system at least to:

display the corresponding moving of the first virtual object based on an amount that the physical object is moved modified by the scale of movement.

9. The system of claim 5, wherein the virtual environment is presented to the first user in a first-person perspective or a third-person perspective.

14. The system of claim 5, further comprising instructions that upon execution cause the system to perform further operations, comprising:

in response to receiving fourth depth image data indicative of the physical object being moved in the first physical space by the first user, determine not to display a corresponding moving of the first virtual object where the moving of the first virtual object conflicts with a boundary of the virtual environment.

6. A computer-readable storage device that is not a propagating signal comprising computer-readable instructions that upon execution on a computer, cause the computer to perform operations comprising:

receiving first depth image data indicative of a first physical space;

generating a first virtual object in a virtual space based on a physical object as represented in the first depth image data;

in response to receiving second depth image data indicative of the first physical space where the physical object is being moved by a first user, displaying a corresponding moving of the first virtual object in the virtual space; and

in response to receiving third depth image data indicative of a second user in a second physical space, the second physical space differing from the first physical space in which the physical object is located, and in response to determining that the third depth image data is indicative of the second user attempting to move a second virtual object in the virtual space and the first virtual object in the virtual space without the second user having moved the physical object in the first physical space, determining to display movement of the second virtual object and not to display movement of the first virtual object.

10. The computer-readable storage device of claim 6, wherein the virtual space comprises a representation of the first physical space, a virtual sub-space, a representation of the second physical space, or a combination thereof.

11. The computer-readable storage device of claim 6, wherein there is a scale of movement, and wherein displaying a corresponding moving of the first virtual object comprises:

displaying the corresponding moving of the first virtual object based on an amount that the physical object is moved modified by the scale of movement.

12. The computer-readable storage device of claim 6, wherein the virtual space is presented to the first user in a first-person perspective or a third-person perspective.

15. The computer-readable storage device of claim 6, further comprising instructions that upon execution cause the system to perform further operations, comprising:

in response to receiving fourth depth image data indicative of the physical object being moved in the first physical space by the first user, determining not to display a corresponding moving of the first virtual object where the moving of the first virtual object conflicts with a boundary of the virtual space.