CONTROLLING A VIRTUAL CAMERA

• US 20160227132A1
• Filed: 04/12/2016
• Published: 08/04/2016
• Est. Priority Date: 07/19/2010
• Status: Active Grant

• Current Assignee: Lucasfilm Limited (The Walt Disney Company)
• Original Assignee: Lucasfilm Limited (The Walt Disney Company)
• Inventors: Wooley, Kevin, Sanders, Michael, Sullivan, Steve, Reynolds, Spencer, Cantwell, Brian
• Granted Patent: US 9,781,354 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0202 Constructional details or processes of manufacture of the input device
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0362 with detection of 1D translations or rotations of an operating part of the device, e.g. scroll wheels, sliders, knobs, rollers or belts
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G11B 27/00 Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 5/2224 related to virtual studio applications
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

Among other aspects, on computer-implemented method includes: receiving at least one command in a computer system from a handheld device; positioning a virtual camera and controlling a virtual scene according to the command; and in response to the command, generating an output to the handheld device for displaying a view of the virtual scene as controlled on a display of the handheld device, the view captured by the virtual camera as positioned.

Claims

1. A computer-implemented method comprising:

obtaining motion capture information of a handheld device in an environment;

accessing a representation of a virtual scene comprising a virtual camera associated with the virtual scene;

determining a position of the virtual camera based at least in part on the obtained motion capture information;

modifying a virtual asset defined for the virtual scene in response to an input, received at the handheld device, wherein the modifying includes adding, relocating, removing or altering an appearance of the virtual asset defined for the virtual scene;

generating an image of the virtual scene based at least in part on the modification to the virtual asset and the determined position of the virtual camera; and

providing the image to the handheld device for display.

2. The method of claim 1 wherein the virtual camera is an abstract definition including a point in space and a direction for a field of view within the virtual scene.

3. The method of claim 1 further comprises generating an animation including the view, and editing the animation based on a user input made on the handheld device

4. The method of claim 1 wherein the virtual asset is a character or a set piece.

5. The method of claim 1 wherein the handheld device is capable of recording images and comprises a visual display.

6. The method of claim 1 wherein the motion capture information of the handheld device is received by the computer system from a motion capture system, separate from the handheld device, that tracks the motion of the handheld device, and the tracked motion is used to determine the first perspective of the virtual scene.

7. The method of claim 6 wherein the handheld device comprises motion capture marks capable of detection by the separate motion capture system.

8. The method of claim 1 wherein the motion capture information is generated by the handheld device in response to a user performing a gesture, and wherein the method further comprises performing at least one action on the virtual scene or the virtual camera based on the motion capture input.

9. The method of claim 1 further comprising receiving audio recorded using the handheld device, and storing the audio in association with the virtual scene

10. The method of claim 1 wherein the display includes a touchscreen.

11. A system comprising:

a computing device comprising;

a processor; and

a memory communicatively linked to the processor, wherein the memory is configured to store a representation of a virtual scene;

wherein the memory comprises instructions that cause the computing device to;

obtain motion capture information of a handheld device in an environment;

access a representation of a virtual scene comprising a virtual camera associated with the virtual scene;

determine a position of the virtual camera based at least in part on the obtained motion capture information;

modify a virtual asset defined for the virtual scene in response to an input, received at the handheld device, wherein the modifying includes adding, relocating, removing or altering an appearance of the virtual asset defined for the virtual scene;

generate an image of the virtual scene based at least in part on the modification to the virtual asset and the determined position of the virtual camera; and

provide the image to the handheld device for display.

12. The system of claim 11 further comprising the handheld device and wherein the handheld device is communicatively linked to the computing device.

13. The system of claim 12 wherein the handheld device comprises a visual display, is capable of recording images, and the motion information from the handheld device comprises recorded image data.

14. The system of claim 12 wherein the handheld device is operated in a motion capture stage, the virtual scene includes a representation of the motion capture stage, and the motion information received from the handheld device comprises images of the motion capture stage.

15. The system of claim 12 further comprising a motion capture subsystem capable of tracking the handheld device, wherein tracked motion of the handheld device is used to determine the first perspective of the virtual scene.

16. The system of claim 15 wherein the motion capture system is separate from the handheld device, and the handheld device comprises motion capture marks capable of detection by the separate motion capture system.

17. A computer program product tangibly embodied in a non-transitory computer-readable medium and comprising instructions that, when executed by a computer system, cause the computer system to perform a method comprising:

obtaining motion capture information of a handheld device in an environment;

accessing a representation of a virtual scene comprising a virtual camera associated with the virtual scene;

determining a position of the virtual camera based at least in part on the obtained motion capture information;

modifying a virtual asset defined for the virtual scene in response to an input, received at the handheld device, wherein the modifying includes adding, relocating, removing or altering an appearance of the virtual asset defined for the virtual scene;

generating an image of the virtual scene based at least in part on the modification to the virtual asset and the determined position of the virtual camera; and

providing the image to the handheld device for display.

18. The computer-program product of claim 17 wherein the virtual camera is an abstract definition including a point in space and a direction for a field of view within the virtual scene.

19. The computer-program product of claim 17 wherein the method further comprises generating an animation including the view, and editing the animation based on a user input made on the handheld device

20. The computer-program product of claim 17 wherein the virtual asset is a character or a set piece.