HAPTIC INTERFACE FOR POPULATION OF A THREE-DIMENSIONAL VIRTUAL ENVIRONMENT

• US 20160124608A1
• Filed: 10/30/2014
• Published: 05/05/2016
• Est. Priority Date: 10/30/2014
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: SUMNER, Robert, NORIS, Gioacchino
• Granted Patent: US 9,733,790 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

The disclosure provides an approach for populating a virtual environment with objects. In one embodiment, an editing application may track a handheld device using sensor data from a camera, by following an image displayed on the handheld device'"'"'s screen. The editing application then updates the position of an object in the virtual environment according to the tracked position of the handheld device. Initially, the handheld device may be placed at a fixed location for calibration purposes, during which the editing application initializes a mapping between the virtual and physical environments. To add an object to the virtual environment, a user may select the object on the handheld device. The user may then place the object at a desired location and orientation in the virtual environment by moving the handheld device in the physical environment.

Claims

1. A computer-implemented method for populating a virtual environment, comprising:

receiving, from a handheld device, a selection of an object to add to the virtual environment;

tracking a position of the handheld device; and

updating a position of the object in the virtual environment based on the tracked position of the handheld device.

2. The method of claim 1, wherein tracking the position of the handheld device includes using one or more computer vision algorithms to compute the position of the handheld device based on an image displayed on a screen of the handheld device.

3. The method of claim 2, wherein the image is a predefined image used for tracking.

4. The method of claim 2, wherein the image depicts the object.

5. The method of claim 1, wherein tracking the position of the handheld device includes using one or more computer vision algorithms to compute the position of the handheld device based on near infrared (IR) markers positioned on the handheld device.

6. The method of claim 1, further comprising:

modifying one or more characteristics of the object in the virtual environment in response to receiving a touch-based gesture on a screen of the handheld device.

7. The method of claim 1, further comprising:

receiving a depth map of a physical environment in which the handheld device is located;

receiving one or more images of the physical environment; and

adding, to the one or more images of the physical environment, a depiction of the object based on the tracked position of the handheld device and the depth map of the physical environment.

8. The method of claim 1, wherein the handheld device is one of a mobile phone, a tablet computer, a laptop computer, a personal digital assistant (PDA), a video game device, and a wearable device.

9. The method of claim 1, further comprising:

snapping the position of the object to a plane in the virtual environment.

10. A non-transitory computer-readable storage medium storing a program, which, when executed by a processor performs operations for populating a virtual environment, the operations comprising:

receiving, from a handheld device, a selection of an object to add to the virtual environment;

tracking a position of the handheld device; and

updating a position of the object in the virtual environment based on the tracked position of the handheld device.

11. The computer-readable storage medium of claim 10, wherein tracking the position of the handheld device includes using one or more computer vision algorithms to compute the position of the handheld device based on an image displayed on a screen of the handheld device.

12. The computer-readable storage medium of claim 11, wherein the image is a predefined image used for tracking purposes.

13. The computer-readable storage medium of claim 11, wherein the image is an image of the object.

14. The computer-readable storage medium of claim 10, wherein tracking the position of the handheld device includes using one or more computer vision algorithms to compute the position of the handheld device based on near infrared (IR) markers positioned on the handheld device.

15. The computer-readable storage medium of claim 10, the operations further comprising:

modifying one or more characteristics of the object in the virtual environment in response to receiving a touch-based gesture on a screen of the handheld device.

16. The computer-readable storage medium of claim 10, the operations further comprising:

receiving a depth map of a physical environment in which the handheld device is located;

receiving one or more images of the physical environment; and

adding, to the one or more images of the physical environment, a depiction of the object based on the tracked position of the handheld device and the depth map of the physical environment.

17. The computer-readable storage medium of claim 11, wherein the handheld device is one of a mobile phone, a tablet computer, a laptop computer, a personal digital assistant (PDA), a video game device, and a wearable device.

18. The computer-readable storage medium of claim 18, the operations further comprising:

snapping the position of the object to a plane in the virtual environment.

19. A system, comprising:

a processor; and

a memory, wherein the memory includes an application program configured to perform operations for populating a virtual environment, the operations comprising;

receiving, from a handheld device, a selection of an object to add to the virtual environment,

tracking a position of the handheld device, and

updating a position of the object in the virtual environment based on the tracked position of the handheld device.

20. The system of claim 19, the operations further comprising:

modifying one or more characteristics of the object in the virtual environment in response to receiving a touch-based gesture on a screen of the handheld device.