CONTROLLING A VIRTUAL CAMERA

• US 20150130801A1
• Filed: 01/12/2015
• Published: 05/14/2015
• Est. Priority Date: 07/19/2010
• Status: Active Grant

• Current Assignee: Lucasfilm Limited (The Walt Disney Company)
• Original Assignee: Lucasfilm Limited (The Walt Disney Company)
• Inventors: WOOLEY, KEVIN, SANDERS, MICHAEL, SULLIVAN, STEVE, REYNOLDS, SPENCER, CANTWELL, BRIAN
• Granted Patent: US 9,324,179 B2
• Time in Patent Office: Days
• US Class Current: 345/420
• CPC Class Codes: G06F 3/0202 Constructional details or processes of manufacture of the input device
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0362 with detection of 1D translations or rotations of an operating part of the device, e.g. scroll wheels, sliders, knobs, rollers or belts
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G11B 27/00 Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 5/2224 related to virtual studio applications
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

Among other aspects, on computer-implemented method includes: receiving at least one command in a computer system from a handheld device; positioning a virtual camera and controlling a virtual scene according to the command; and in response to the command, generating an output to the handheld device for displaying a view of the virtual scene as controlled on a display of the handheld device, the view captured by the virtual camera as positioned.

Claims

1. A computer-implemented method comprising operating a computer system in one of a set of modes, wherein:

operating in a first mode comprises;

capturing motion capture information from an environment;

rendering a virtual scene from a first perspective based at least in part on the captured motion capture information and a virtual camera associated with the virtual scene, wherein the virtual camera is an abstract definition including a point in space and a direction of a field of view;

receiving a first input from a user specifying a modification of at least one of the point in space of the virtual camera or the field of view of the virtual camera; and

rendering in response to the first input, the virtual scene from a second perspective based on the virtual camera associated with the virtual scene, wherein the second perspective is different from the first perspective; and

operating in a second mode, wherein operation in the second mode comprises;

rendering the virtual scene from a third perspective; and

modifying virtual assets defined for the virtual scene in response to a second input, received at a handheld device, wherein the modifying includes adding, relocating, removing or altering an appearance of at least one of the virtual assets defined for the virtual scene.

2. The computer-implemented method of claim 1, wherein the handheld device is capable of recording images and comprises a visual display, and the motion capture information from the environment is captured by the handheld device.

3. The computer-implemented method of claim 2, wherein the rendering of the virtual scene from the second perspective is displayed on the visual display of the handheld device.

4. The computer-implemented method of claim 2, wherein motion of the handheld device is tracked by a motion capture system separate from the handheld device, and the tracked motion is used to determine the first perspective of the virtual scene.

5. The computer-implemented method of claim 4, wherein the handheld device comprises motion capture marks capable of detection by the separate motion capture system.

6. The computer-implemented method of claim 1, wherein the point in space of the virtual camera in the virtual scene specified by the first input comprises a location on one of the virtual assets.

7. The computer-implemented method of claim 6, wherein the first input further comprises a constraint on the field of view within the virtual scene of the virtual camera.

8. A computer-implemented method comprising operating a computer system in one of a set of modes, wherein:

operating in a first mode comprises;

rendering a virtual scene from a first perspective based at least in part on a virtual camera associated with the virtual scene, wherein the virtual camera is an abstract definition including a point in space and a direction for a field of view;

receiving motion information for a handheld device;

determining at least one of an updated point in space of the virtual camera in the virtual scene or an updated direction for a field of view within the virtual scene of the virtual camera based on the motion information for the handheld device and an association between the virtual camera and a position of the handheld device;

rendering the virtual scene from a second perspective based at least in part on the updated point in space of the virtual camera or the updated direction for a field of view of the virtual camera; and

operating in a second mode comprises;

rendering the virtual scene from the second perspective for display on the handheld device; and

modifying virtual assets defined for the virtual scene in response to input received at the handheld device, wherein the modifying includes adding, relocating, removing or altering an appearance of at least one of the virtual assets defined for the virtual scene.

9. The computer-implemented method of claim 8, wherein the handheld device comprises a visual display, is capable of recording images, and the motion information for the handheld device comprises recorded image data.

10. The computer-implemented method of claim 9, wherein the handheld device is operated in a motion capture stage, the virtual scene includes a representation of the motion capture stage, and the motion information received for the handheld device comprises images of the motion capture stage.

11. The computer-implemented method of claim 8, wherein motion of the handheld device is tracked by a motion capture system, and the tracked motion is used to determine the first perspective of the virtual scene.

12. The computer-implemented method of claim 11, wherein the motion capture system is separate from the handheld device, and the handheld device comprises motion capture marks capable of detection by the separate motion capture system.

13. The computer-implemented method of claim 8, wherein the point in space of the virtual camera in the virtual scene specified by the first input comprises a location on one of the virtual assets.

14. A system comprising:

a computing device comprising;

a processor; and

a memory communicatively linked to the processor, wherein the memory stores a representation of a virtual scene; and

a handheld device communicatively linked to the computing device;

wherein the memory comprises instructions that cause the computing device to operate in one of a set of modes, wherein;

operating in a first mode comprises;

rendering the virtual scene from a first perspective based at least in part on a virtual camera associated with the virtual scene, wherein the virtual camera is an abstract definition including a point in space and a direction for a field of view;

receiving motion information for the handheld device;

determining at least one of an updated point in space of the virtual camera in the virtual scene or an updated direction a field of view within the virtual scene of the virtual camera based on the motion information for the handheld device and an association between the virtual camera and a position of the handheld device;

rendering the virtual scene from a second perspective based at least in part on the updated point in space of the virtual camera or the updated direction for a field of view of the virtual camera; and

operating in a second mode comprises;

rendering the virtual scene from the second perspective for display on the handheld device; and

modifying virtual assets defined for the virtual scene in response to input received at the handheld device, wherein the modifying includes adding, relocating, removing or altering an appearance of at least one of the virtual assets defined for the virtual scene.

15. The system of claim 14, wherein the handheld device comprises a visual display, is capable of recording images, and the motion information for the handheld device comprises recorded image data.

16. The system of claim 14, wherein the handheld device is operated in a motion capture stage, the virtual scene includes a representation of the motion capture stage, and the motion information received for the handheld device comprises images of the motion capture stage.

17. The system of claim 14, further comprising a motion capture subsystem capable of tracking the handheld device, wherein tracked motion of the handheld device is used to determine the first perspective of the virtual scene.

18. The system of claim 17, wherein the motion capture system is separate from the handheld device, and the handheld device comprises motion capture marks capable of detection by the separate motion capture system.

19. A computer program product tangibly embodied in a non-transitory computer-readable medium and comprising instructions that when executed by a computer system cause the computer system to operate in one of a set of modes, wherein:

operating in a first mode comprises;

rendering a virtual scene from a first perspective based at least in part on a virtual camera associated with the virtual scene, wherein the virtual camera is an abstract definition including a point in space and a direction for a field of view;

receiving motion information for a handheld device;

determining at least one of an updated point in space of the virtual camera in the virtual scene or an updated direction-a field of view within the virtual scene of the virtual camera based on the motion information for the handheld device and an association between the virtual camera and a position of the handheld device;

rendering the virtual scene from a second perspective based at least in part on the updated point in space of the virtual camera or the updated direction for a field of view of the virtual camera; and

operating in a second mode comprises;

rendering the virtual scene from the second perspective for display on the handheld device; and

modifying virtual assets defined for the virtual scene in response to input received at the handheld device, wherein the modifying includes adding, relocating, removing or altering an appearance of at least one of the virtual assets defined for the virtual scene.

20. The computer-program product of claim 19, wherein the handheld device comprises a visual display and is capable of recording images and wherein the motion information for the handheld device comprises recorded image data.