Methods and apparatus for simultaneous user inputs for three-dimensional animation

• US 8,836,646 B1
• Filed: 04/23/2009
• Issued: 09/16/2014
• Est. Priority Date: 04/24/2008
• Status: Active Grant

• Current Assignee: Pixar (The Walt Disney Company)
• Original Assignee: Pixar (The Walt Disney Company)
• Inventors: De Rose, Tony, Kin, Kenrick
• Primary Examiner(s): Wu, Xiao
• Assistant Examiner(s): He, Yingchun
• Application Number:
  US12/429,052
• Time in Patent Office: 1,972 Days
• Field of Search: 345/173, 345/427, 715/702, 715/863
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04109 FTIR in optical digitiser, i.e. touch detection by frustrating the total internal reflection within an optical waveguide due to changes of optical properties or deformation at the touch location
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A method for a computer system includes determining a plurality of positions of portions of a hand of a user simultaneously placed upon a user interface device of the computer system, retrieving a set of display icons in response to the plurality of positions of the portions of the user hand, displaying the display icons from the set of display icons on a display relative to the plurality of positions of the portions of the user hand; while displaying the display icons on the display, determining a user selection of a display icon from the display icons, and performing a function in response to the user selection of the display icon.

Claims

1. A method for a computer system comprising:

determining a first set of virtual camera parameters for a virtual camera in three-dimensional space relative to a three-dimensional scene, wherein the location of the camera is within the three-dimensional scene, wherein the three-dimensional scene comprises at least one three-dimensional object, and wherein the first set of virtual camera parameters include a first three-dimensional position of the virtual camera;

determining a first image of the three-dimensional scene in response to the first set of virtual camera parameters in three-dimensional space;

displaying the first image of the three-dimensional scene to the user on a user interface display of the computer system;

determining a first position of a first indicator of a user placed upon the first image on the user interface display of the computer system;

determining a first position of a second indicator of the user placed upon the first image on the user interface display of the computer system;

determining a second position of the first indicator of the user placed upon the user interface display of the computer system;

determining a second position of the second indicator of the user placed upon the user interface display of the computer system;

determining a second set of virtual camera parameters for the virtual camera in three-dimensional space relative to the three-dimensional scene in response to the first set of virtual camera parameters, to the first position and the second positions of the first indicator of the user, and to the first position and the second position of the second indicator, wherein the second set of virtual camera parameters include a second three-dimensional position of the virtual camera,

wherein when the second position of the first indicator is shifted in a vertical direction upon the user interface display from the first position of the first indicator concurrently with the second position of the second indicator being shifted in the vertical direction upon the user interface display from the first position of the second indicator, the virtual camera is moved forward parallel to a ground plane from the first three-dimensional position to the second three-dimensional position in the three-dimensional scene;

determining an updated three-dimensional scene based on the second three-dimensional position of the virtual camera;

determining a second image of the updated three-dimensional scene in response to the second set of virtual camera parameters in three-dimensional space; and

displaying the second image of the three-dimensional scene to the user on the user interface display of the computer system.

2. The method of claim 1 wherein determining the second set of virtual camera parameters in three-dimensional space comprises:

determining a first direction of movement between the first position and the second position of the first finger of the user;

determining a second direction of movement between the first position and the second position of the second finger of the user; and

determining the second set of virtual camera parameters is in response to the first direction, the second direction, and the first set of virtual camera parameters.

3. The method of claim 2 wherein determining the second set of virtual camera parameters comprises:

determining a relative direction between the first direction and the second direction; and

modifying the first set of virtual camera parameters to become the second set of virtual camera parameters in response to the relative direction, the first direction, and the second direction.

4. The method of claim 3 wherein when the relative direction is upwards or downwards, a virtual camera roll parameter is modified.

5. The method of claim 1 wherein the first indicator is selected from a group consisting of:

a finger of the user, a stylus, a pointing device, a laser spot.

6. The method of claim 1, wherein when the second position of the first indicator is shifted in a downward direction upon the user interface device from the first position of the first indicator, and the second position of the second indicator is shifted in the downward direction upon the user interface device from the first position of the second indicator, the virtual camera is moved backward along the plane parallel to a ground plane of the three-dimensional scene.

7. A computer program product comprising a non-transitory tangible medium including computer system executable code for a computer system including a processor, a memory and a user interface display, the computer program product comprising:

code that directs the processor to determine a first set of virtual camera parameters for a virtual camera in three-dimensional space relative to a three-dimensional scene, wherein the location of the camera is within the three-dimensional scene, wherein the three-dimensional scene comprises at least one three-dimensional object, and wherein the first set of virtual camera parameters include a first three-dimensional position of the virtual camera;

code that directs the processor to determine a first image of the three-dimensional scene in response to the first set of virtual camera parameters in three-dimensional space;

code that directs the processor to display the first image of the three-dimensional scene to the user on user interface display of the computer system;

code that directs the processor to determine a first position of a first indicator of a user placed upon the first image on the user interface display of the computer system;

code that directs the processor to determine a first position of a second indicator of the user placed upon the first image on the user interface display of the computer system;

code that directs the processor to determine a second position of the first indicator of the user placed upon the user interface display of the computer system;

code that directs the processor to determine a second position of the second indicator of the user placed upon the user interface display of the computer system;

code that directs the processor to determine a second set of virtual camera parameters for the virtual camera in three-dimensional space relative to the three-dimensional scene in response to the first set of virtual camera parameters, to the first position and the second positions of the first indicator of the user, and to the first position and the second position of the second indicator, wherein the second set of virtual camera parameters include a second three-dimensional position of the virtual camera, wherein when the second position of the first indicator is shifted in a vertical direction upon the user interface display from the first position of the first indicator concurrently with the second position of the second indicator being shifted in the vertical direction upon the user interface display from the first position of the second indicator, the virtual camera is moved forward parallel to a ground plane from the first three-dimensional position to the second three-dimensional position in the three-dimensional scene;

code that directs the processor to determine an updated three-dimensional scene based on the second three-dimensional position of the virtual camera;

code that directs the processor to determining a second image of the updated three-dimensional scene in response to the second set of virtual camera parameters in three-dimensional space; and

code that directs the processor to display the second image of the three-dimensional scene to the user on the user interface display of the computer system.

8. The computer program product of claim 7 further comprising:

code that directs the processor to determine a first direction of movement between the first position and the second position of the first finger of the user;

code that directs the processor to determine a second direction of movement between the first position and the second position of the second finger of the user; and

code that directs the processor to determine the second set of virtual camera parameters is in response to the first direction, the second direction, and the first set of virtual camera parameters.

9. The computer program product of claim 7 further comprising:

code that directs the processor to determine a relative direction between the first direction and the second direction; and

code that directs the processor to modify the first set of virtual camera parameters to become the second set of virtual camera parameters in response to the relative direction, the first direction, and the second direction.

10. The computer program product of claim 9

wherein when the relative direction is upwards or downwards, a virtual camera roll parameter is modified.

11. A computer system comprising:

a memory configured to store a first set of virtual camera parameters for a virtual camera in three-dimensional space relative to a three-dimensional scene, wherein the location of the camera is within the three-dimensional scene, wherein the three-dimensional scene comprises at least one three-dimensional object, and wherein the first set of virtual camera parameters include a first three-dimensional position of the virtual camera;

a processor coupled to the memory, wherein the processor is configured to;

determine a first image of the three-dimensional scene in response to the first set of virtual camera parameters in three-dimensional space,

determine a first position of a first indicator of a user placed upon the first image on a user interface display of the computer system,

determine a first position of a second indicator of the user placed upon the first image on the user interface display of the computer system,

determine a second position of the first indicator of a user placed upon the user interface display of the computer system,

determine a second position of the second indicator of the user placed upon the user interface display of the computer system,

determine a second set of virtual camera parameters for the virtual camera in three-dimensional space relative to the three-dimensional scene in response to the first set of virtual camera parameters, to the first position and the second position of the first indicator of the user, and to the first position and the second position of the second indicator, wherein the second set of virtual camera parameters include a second three-dimensional position of the virtual camera, wherein when the second position of the first indicator is shifted in a vertical direction upon the user interface display from the first position of the first indicator concurrently with the second position of the second indicator being shifted in the vertical direction upon the user interface display from the first position of the second indicator, the virtual camera is moved forward parallel to a ground plane from the first three-dimensional position to the second three-dimensional position in the three-dimensional scene,

determine an updated three-dimensional scene based on the second three-dimensional position of the virtual camera, and

determine a second image of the updated three-dimensional scene in response to the second set of virtual camera parameters in three-dimensional space; and

the user interface display coupled to the processor, wherein the user interface display is configured to display the first image of the three-dimensional scene to the user on the user interface display, and configured to display the second image of the three-dimensional scene to the user on the user interface display.

12. The computer system of claim 11 wherein the first indicator is selected from a group consisting of:

a finger of the user, a stylus, a pointing device, a laser spot.

13. A method for a computer system comprises

displaying a representation of a plurality of three-dimensional objects, wherein a first three-dimensional object is located at a first three-dimensional location at a first display position on a display to a user;

determining a first user interface position of a first indicator of a user placed upon a user interface device of the computer system;

determining a selection of the first three-dimensional object of the plurality of three-dimensional objects in response to the first user interface position of the first indicator being at the first display position;

displaying a representation of a plurality of geometric constraints associated with the first three-dimensional object at a plurality of display positions on the display to the user;

while the first indicator of the user is placed upon the user interface device, determining a second user interface position of a second indicator of the user placed upon the user interface device;

determining a selection of a geometric constraint from the plurality of geometric constraints in response to the second user interface position and the plurality of display positions;

determining movement of the second indicator of the user placed upon the user interface device from the second user interface position to a third user interface position;

moving the first three-dimensional object from the first three-dimensional location to a second three-dimensional location along the direction of the selected geometric constraint, in response to the movement of the second indicator and in response to the selected geometric constraint;

determining a representation of the first three-dimensional object located at the second three-dimensional location; and

displaying the representation of the first three-dimensional object located at the second three-dimensional location at a second display position on the display to the user.

14. The method of claim 13 wherein the geometric constraint from the plurality of geometric constraints is selected from the group consisting of:

a translational movement constraint along a geometric axis within three-dimensional space, a rotational constraint along a geometric axis within three-dimensional space.

15. The method of claim 14 wherein prior to determining movement of the second indicator of the user, the method comprises determining that the second indicator of the user is removed from the user interface device of the computer system.

16. The method of claim 13

wherein while the first indicator of the user is placed upon the user interface device, determining a fourth user interface position of a third indicator of the user placed upon the user interface device;

wherein determining the selection of the geometric constraint from the plurality of geometric constraints is also in response to the fourth user interface position.

17. The method of claim 16 wherein the geometric constraint from the plurality of geometric constraints comprises a translational movement constraint within a geometric plane within three-dimensional space.

18. The method of claim 13 wherein prior to determining movement of the second indicator of the user, the method comprises determining that the second indicator of the user is removed from the user interface device of the computer system.

19. The method of claim 13 wherein the first indicator is selected from a group consisting of:

a finger of the user, a stylus, a pointing device, a laser spot.

20. The method of claim 13 wherein differences in geometric placement of the first three-dimensional object are constrained between the first three-dimensional location and the second three-dimensional location.

21. The method of claim 20 wherein the first three-dimensional location and the second three-dimensional location have a similar ground plane.

22. The method of claim 13 wherein determining the second user interface position of a second indicator of the user placed upon the user interface device occurs while the first indicator of the user is placed upon the user interface device.

23. The method of claim 13 wherein moving the first three-dimensional object from the first three-dimensional location to a second three-dimensional location, in response to the second user interface position occurs before the first indicator of the user is removed from the user interface device.

24. The method of claim 13 wherein determining the second user interface position of a second indicator of the user placed upon the user interface device occurs after the first indicator of the user is removed from the user interface device.

25. The method of claim 13 wherein moving the first three-dimensional object from the first three-dimensional location to a second three-dimensional location, in response to the second user interface position occurs after the first indicator of the user is removed from the user interface device.

26. A computer program product comprising a non-transitory tangible medium including computer system executable code for a computer system including a processor, a memory and a display, the computer program product comprising:

code that directs the processor to display a representation of a plurality of three-dimensional objects, wherein a first three-dimensional object is located at a first three-dimensional location at a first display position on the display to a user;

code that directs the processor to determine a first user interface position of a first indicator of the user placed upon a user interface device of the computer system;

code that directs the processor to determine a selection of the first three-dimensional object of the plurality of three-dimensional objects in response to the first user interface of the first indicator being at the first display position;

code that directs the processor to display a representation of a plurality of geometric constraints associated with the first three-dimensional object at a plurality of display positions on the display to the user;

code that directs the processor to determine a second user interface position of a second indicator of the user placed upon the user interface device, while the first indicator of the user is placed upon the user interface device;

code that directs the processor to determine a selection of a geometric constraint from the plurality of geometric constraints in response to the second user interface position and the plurality of display positions;

code that directs the processor to determine movement of the second indicator of the user placed upon the user interface device from the second user interface position to a third user interface position;

code that directs the processor to move the first three-dimensional object from the first three-dimensional location to a second three-dimensional location along the direction of the selected geometric constraint, in response to the movement of the second indicator and in response to the selected geometric constraint;

code that directs the processor to determine a representation of a first three-dimensional object located at the second three-dimensional location; and

code that directs the processor to display the representation of the first three-dimensional object located at the second three-dimensional location at a second display position on the display to the user.

27. The computer program product of claim 26 wherein the geometric constraint from the plurality of geometric constraints is selected from the group consisting of:

a translational movement constraint along a geometric axis within three-dimensional space, a rotational constraint along a geometric axis within three-dimensional space.

28. The computer program product of claim 27 further comprising code that directs the processor to determine that the second indicator of the user is removed from the user interface device of the computer system.

29. The computer program product of claim 26 further comprising:

code that directs the processor to determine a fourth user interface position of a third indicator of the user placed upon the user interface device, while the first indicator of the user is placed upon the user interface device; and

code that directs the processor to determine the selection of the geometric constraint from the plurality of geometric constraints in response to the fourth user interface position.

30. The computer program product of claim 29 wherein the geometric constraint from the plurality of geometric constraints comprises a translational movement constraint within a geometric plane within three-dimensional space.

31. The computer program product of claim 26 further comprising code that directs the processor to determine that the second indicator of the user is removed from the user interface device of the computer system.

32. A computer system comprising:

a memory configured to store a model of a first three-dimensional object;

a processor coupled to the memory, wherein the processor is configured to;

determine a representation of a plurality of three-dimensional objects, wherein the first three-dimensional object is located at a first three-dimensional location in response to the model,

determine a first user interface position of a first indicator of a user placed upon a user interface device of the computer system,

determine a selection of the first three-dimensional object of the plurality of three-dimensional objects in response to the first user interface position of the first indicator being at a first display position,

determine a second user interface position of a second indicator of the user placed upon the user interface device, while the first indicator of the user is placed upon the user interface device,

determine a selection of a geometric constraint from the plurality of geometric constraints in response to the second user interface position, configured to

determine movement of the second indicator of the user placed upon the user interface device from the second user interface position to a third user interface position,

move the first three-dimensional object from the first three-dimensional location to a second three-dimensional location along the direction of the selected geometric constraint, in response to the movement of the second indicator and in response to the selected geometric constraint, and

determine a representation of a first three-dimensional object located at the second three-dimensional location; and

a display coupled to the processor, wherein the display is configured to;

display the representation of the first three-dimensional object located at the second three-dimensional location at a second display position on the display to the user,

display the representation of the first three-dimensional object located at the first three-dimensional location at the first display position on the display to the user, and

display a representation of a plurality of geometric constraints associated with the first three-dimensional object at a plurality of display positions on the display to the user.