Virtual-scene control device

• US 9,626,786 B1
• Filed: 12/30/2010
• Issued: 04/18/2017
• Est. Priority Date: 07/19/2010
• Status: Active Grant

• Current Assignee: Lucasfilm Entertainment Co Ltd (The Walt Disney Company)
• Original Assignee: Lucasfilm Entertainment Co Ltd (The Walt Disney Company)
• Inventors: Reynolds, Spencer, Sanders, Michael, Wooley, Kevin, Sullivan, Steve, Schnitzer, Adam
• Primary Examiner(s): Lee, Benjamin C
• Assistant Examiner(s): Zheng, Xuemei
• Application Number:
  US12/982,437
• Time in Patent Office: 2,301 Days
• Field of Search: 345169
• CPC Class Codes: G06F 3/0202 Constructional details or processes of manufacture of the input device
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0362 with detection of 1D translations or rotations of an operating part of the device, e.g. scroll wheels, sliders, knobs, rollers or belts
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G11B 27/00 Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 5/2224 related to virtual studio applications
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

A handheld device includes: an input control configured to control and modify a virtual scene including a virtual camera; and a display that shows a representation of the controlled and modified virtual scene generated by the virtual camera. A system includes: a computer system configured to execute program instructions for generating a virtual scene including a virtual camera; and handheld device configured to communicate with the computer system for controlling and modifying the virtual scene, the handheld device comprising: an input control configured to control and modify the virtual scene; and a display that shows a representation of the controlled and modified virtual scene generated by the virtual camera.

Claims

1. A handheld device that allows a user to control a virtual scene generated by a computer system, the virtual scene including one or more characters, a plurality of non-character assets and at least one virtual camera, the plurality of assets including one or more immovable objects, moveable objects or self-propelled objects, the handheld device comprising:

a body having a camera form factor including a lens-like portion and a rotatable lens ring coupled to the lens-like portion;

an I/O channel configured to enable the handheld device to communicate with the computer system that generates the virtual scene;

input controls positioned on the body;

a display mounted on the body and configured to show a representation of the virtual scene as generated by the at least one virtual camera; and

a mode selection input control operable to place the handheld device into a first mode in which the handheld device is configured to enable the user to record images with a virtual camera of the at least one virtual camera in the virtual scene using the input controls, and a second mode in which the handheld device is configured to enable the user to use the input controls to select individual ones of the plurality of assets and move the selected assets within the virtual scene;

wherein when the device is in the first mode, the rotatable lens ring can be used by the user to control a zoom level of the virtual camera, and when the device is in the second mode, the user can use the input controls to select between different virtual lenses for the virtual camera.

2. The handheld device of claim 1 further comprising at least one virtual button configured to be presented on the display, the virtual button configured to control an aspect on an application program that generates the virtual scene including the at least one asset and the virtual camera.

3. The handheld device of claim 1, wherein the display comprises a touchscreen for control or modification of the at least one asset or the virtual camera included in the virtual scene.

4. The handheld device of claim 1, wherein the rotatable lens ring enables control of focus length of the virtual camera.

5. The handheld device of claim 1, further comprising at least one handle attached to the body of the handheld device, wherein the handle includes at least one physical input mechanism for controlling or modifying the at least one asset and the virtual camera included in the virtual scene.

6. The handheld device of claim 1, further comprising a motion capture apparatus configured so that motion of the handheld device is captured and used for controlling or modifying the at least one asset and the virtual camera included in the virtual scene, wherein the representation on the display is updated based on the captured motion.

7. The handheld device of claim 6, where in the motion capture apparatus comprises motion capture markers selectively attachable to the handheld device, wherein an outer motion capture system is configured to register the captured motion of the handheld device using the motion capture markers.

8. The handheld device of claim 7, wherein the motion capture markers include marker rods of different lengths and wherein the handheld device further comprises holders for attaching the marker rods, the holders oriented on the handheld device so that the marker rods extend in each of three spatial dimensions.

9. The handheld device of claim 6, further comprising at least one sensor on the handheld device configured to generate a signal for registering the captured motion of the handheld device.

10. The handheld device of claim 1, wherein the handheld device is modular and configured to have multiple modules selectively attached to a body of the handheld device, each of the modules providing at least one user-operable function for controlling or modifying the at least one asset or the virtual camera included in the virtual scene.

11. The handheld device of claim 1, wherein the input control comprises at least one physical input mechanism mounted on a body of the handheld device for controlling or modifying the at least one asset or the virtual camera included in the virtual scene.

12. The handheld device of claim 11, wherein the physical input mechanism comprises a lens ring for controlling or modifying the at least one asset or the virtual camera included in the virtual camera.

13. The handheld device of claim 11, wherein the physical input mechanism comprises control buttons on the body of the handheld device, at least two of the control buttons located in finger indentations on the body for placement of a user'"'"'s fingers.

14. The handheld device of claim 11, wherein the physical input mechanism comprises buttons arranged in a row on the body of the handheld device, and at least one scroll wheel between adjacent buttons.

15. The handheld device of claim 11, wherein the physical input mechanism comprises a mode control mounted on the body of the handheld device, the mode control comprising mode settings and respective custom settings for multiple users.

16. The handheld device of claim 1, further comprising:

a lens mounted on the body of the handheld device; and

an electronic light sensor operatively coupled to the lens for registering a physical-scene image captured by the lens; and

a memory operatively coupled to the electronic light sensor for storing the image.

17. The handheld device of claim 1 wherein the input controls include a scroll wheel that, after the asset has been selected, moves the asset closer to and further from the virtual camera depending on the scroll direction.

18. The handheld device of claim 17 wherein the input controls further include motion sensors that enable a selected asset to be moved left/right and up/down within the virtual scene.

19. The handheld device of claim 1 wherein the input controls are further configured to allow a user to place an aiming reticle at a chosen location in the virtual scene, and include a button that can be selected to place the selected asset at a location identified by the aiming reticle.

20. The handheld device of claim 1 wherein the input controls include button that can be selected to move the selected asset to the location of the virtual camera.

21. The handheld device of claim 1 wherein the input controls are further configured to, after the asset has been selected, enable a user to move the asset closer to and further from the virtual camera as well as left and right and up and down within the virtual scene.

22. The handheld device of claim 1 wherein the mode selection input control is further operable to place the handheld device into a third mode in which the input controls enable a user to attach a virtual camera to an object in the virtual scene such that the camera moves to the object and a temporary association is created so that the virtual camera undergoes the same or similar movement as the object.

23. The handheld device of claim 22 wherein the mode selection input control is further operable to place the handheld device into a fourth mode in which the input controls enable a user to play previously recorded images on the display.

24. A system comprising:

a computer system configured to execute program instructions for generating a virtual scene including one or more characters, a plurality of non-character assets, and at least one virtual camera, the plurality of assets including one or more immovable objects, moveable objects or self-propelled objects that can be recorded within the virtual scene by the at least one virtual camera; and

a handheld device configured to communicate with the computer system and allow a user to control and modify the virtual scene and the virtual camera, the handheld device comprising;

a body having a camera form factor including a lens-like portion and a rotatable lens ring coupled to the lens-like portion;

an I/O channel configured to enable the handheld device to communicate with the computer system;

input controls positioned on the body;

a display mounted on the body and configured to show a representation of the virtual scene; and

a mode selection input control operable to place the handheld device into a first mode in which the handheld device is configured to enable the user to record images with a virtual camera of the at least one virtual camera in the virtual scene using the input controls, and a second mode in which the handheld device is configured to enable the user to use the input controls to select individual ones of the plurality of assets and move the selected assets within the virtual scene;

wherein when the device is in the first mode, the rotatable lens ring can be used by the user to control a zoom level of the virtual camera, and when the device is in the second mode, the user can use the input controls to select between different virtual lenses for the virtual camera.

25. The system of claim 24 wherein the input controls are further configured to, after the asset has been selected, enable a user to move the asset closer to and further from the virtual camera as well as left and right and up and down within the virtual scene.