Method and apparatus for direct manipulation of 3-D objects on computer displays

• US 5,588,098 A
• Filed: 06/07/1995
• Issued: 12/24/1996
• Est. Priority Date: 11/22/1991
• Status: Expired due to Term

• Current Assignee: Apple Computer Incorporated (Apple Inc.)
• Original Assignee: Apple Computer Incorporated (Apple Inc.)
• Inventors: Chen, Michael, Seidl, Robert H., Houde, Stephanie L.
• Primary Examiner(s): Zimmerman, Mark K.
• Application Number:
  US08/475,405
• Time in Patent Office: 566 Days
• Field of Search: 395/133, 395/136, 395/137, 395/138, 395/139
• US Class Current: 345/653
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2210/12 Bounding box
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling

Abstract

A method and apparatus for direct manipulation of 3-D objects on computer display. This method and apparatus provides for a 3-D bounding region wherein the 3-D bounding region has sensitive areas which the user can select with a reference pointer such that when the user manipulates the reference pointer after selecting a sensitive area the 3-D bounding region and the object within it are manipulated with direct kinesthetic correspondence between the user'"'"'s manipulation of the reference pointer and the bounding region and object manipulation.

Claims

1. A method for manipulating an object displayed in three-dimensional representation on a computer controlled display system having a computer and a display coupled to the computer, comprising the steps of:

a) providing a user actuated input controller signal for selectively positioning a reference indicator on the display;

b) positioning the reference indicator over the displayed object and signaling the computer to activate a control movement mode;

c) providing on the display a three-dimensional representation of a bounding region including at least two portions which specify respective predefined control movement types, one of said types being rotation around an axis parallel to the portion which specifies rotation around that axis;

d) positioning the reference indicator over one of said portions;

e) signaling the computer to activate a predefined control movement type specified by the portion of the bounding region under the reference indicator, and while the reference indicator is over said portion, repositioning the reference indicator to define a movement of the specified type; and

f) re-displaying the displayed object in accordance with the defined movement of the specified type.

2. The method of claim 1 wherein the three-dimensional representation of a bounding region is a generally rectilinear bounding box, and wherein the step of re-displaying further comprises re-displaying the bounding box in accordance with the defined movement of the predefined control movement type.

3. The method of claim 1 wherein the bounding region is reduced in size relative to the displayed object.

4. The method of claim 1 wherein the input controller signal is from a mouse.

5. The method of claim 4 wherein the reference indicator is a pointer displayed on the display.

6. The method of claim 5 wherein the step of signaling the computer to activate the control movement mode includes the step of activating a switch.

7. The method of claim 6 wherein the step of activating a switch comprises pressing a button on the mouse.

8. The method of claim 6 wherein the step of signaling the computer to activate a predefined control movement type includes the step of activating a switch.

9. The method of claim 8 wherein the step of activating a switch comprises pressing a button on the mouse.

10. The method of claim 1 further comprising, in step (e), after signaling the computer to activate a predefined control movement type, changing the shape of the reference indicator and thus indicating, to a user of the method, a type and direction of available object manipulations.

11. The method of claim 1 wherein the portion of the bounding region is an active zone.

12. The method of claim 1 wherein the portion of the bounding region defines a handle which graphically distinguishes said portion from the remainder of the bounding region.

13. The method of claim 12 wherein the handle indicates, to a user of the method, a type and direction of available object manipulations.

14. The method of claim 12 wherein the handle extends outwardly from the bounding region such that the outwardly extending handle presents an appearance of being available to be selected by the reference indicator.

15. The method of claim 12 wherein the handle is shaped like a rod.

16. The method of claim 12 wherein the handle is shaped like a cube.

17. The method of claim 12 wherein the handle is shaped like an arrow.

18. The method of claim 1 wherein the portion of the bounding region is an image of a hand.

19. The method of claim 18 wherein the hand indicates to a user of the method a type and direction of available object manipulations.

20. The method of claim 18 wherein the hand looks like a human hand.

21. The method of claim 20 wherein the hand appears to be grabbing the three-dimensional representation of a bounding region.

22. The method of claim 18 wherein the hand looks like a mechanical hand.

23. The method of claim 22 wherein the hand appears to be grabbing the three-dimensional representation of a bounding region.

24. The method of claim 1 wherein the predefined control movement type is a translation of the displayed object.

25. The method of claim 24 wherein the translation of the displayed object is along model space coordinate axes.

26. The method of claim 25 wherein the translation of the displayed object in model space is parallel to a plane of a face of the bounding region.

27. The method of claim 25 wherein the translation of the displayed object in model space is perpendicular to a plane of a face of the bounding region.

28. The method of claim 24 wherein the translation of the displayed object is performed by determining a difference in model space between a position of the reference indicator when the translation operation was activated and a current position of the reference indicator, and transforming the difference into scaled and rotated coordinates using a scale transform and a rotate transform, and adding the transformed difference to a translation transform to create a new translation transform, and wherein re-displaying the displayed object on the video display is in accordance with the scale transform, the rotate transform and the new translation transform.

29. The method of claim 24 wherein translation of the displayed object is parallel to a plane of a face of the bounding region and is limited to a predefined range of translation manipulations.

30. The method of claim 1 wherein the axis of rotation of the displayed object is parallel to model space coordinate axes.

31. The method of claim 30 wherein the rotation of the displayed object in model space is around an axis of the bounding region.

32. The method of claim 31 wherein the axis of rotation is a center line of the bounding region.

33. The method of claim 31 wherein the axis of rotation is an edge of the bounding region.

34. The method of claim 33 wherein the axis of rotation is around a bounding region edge opposite from the portion of the bounding region under the reference indicator.

35. The method of claim 30 wherein the axis of rotation is outside of the bounding region.

36. The method of claim 1 wherein the rotation of the displayed object is performed by transforming a position of the reference indicator when the rotation operation was activated by a scale transform, and transforming a current position of the reference indicator by the scale transform and determining an angle created thereby versus a center point of the displayed object, and concatenating a rotation matrix of the angle into a rotation transform to create a new rotation transform, and wherein re-displaying the displayed object on the video display is in accordance with the scale transform, the new rotation transform and a translation transform.

37. The method of claim 1 wherein a second of the predefined control movement types is a scaling of the displayed object.

38. The method of claim 37 wherein the scaling of the displayed object is along model space coordinate axes.

39. The method of claim 38 wherein the scaling in model space is parallel to a plane of a face of the bounding region.

40. The method of claim 38 wherein the scaling is homogenous.

41. The method of claim 37 wherein the scaling of the displayed object is performed by determining a ratio of a position of the reference indicator versus an origin of the bounding region when the scaling operation was activated and a current-position of the reference indicator versus the origin of the bounding region and multiplying the ratio with a scale transform to create a new scale transform and wherein re-displaying the displayed object on the video display is in accordance with the new scale transform, a rotate transform and a translation transform.

42. The method of claim 41 wherein scaling of the displayed object is parallel to a plane of a face of the bounding region and is limited to a predefined range of scaling manipulations.

43. A method for manipulating an object displayed in three-dimensional representation on a computer controlled video display system having a computer, a video display and a mouse, comprising the steps of:

a) selecting the displayed object by using the mouse to position a pointer over the displayed object and pressing a button on the mouse;

b) displaying a three-dimensional representation of a bounding box having at least two active zones which specify respective available movement types, one of said types being rotation around an axis parallel to the zone which specifies rotation around that axis, when the displayed object is selected;

c) defining a movement of a type predefined by one of the active zones by using the mouse to position the pointer over an active zone, pressing the button on the mouse, and, while the pointer is over the active zone, repositioning the pointer with the mouse; and

d) re-displaying the bounding box and the displayed object in accordance with the defined movement.

44. The method of claim 43 wherein the available movement types further comprise translation and scaling.

45. The method of claim 44 wherein if the predefined movement type is translation then the translation of the displayed object and the bounding box is performed by determining a difference in model space between a position of the pointer when the pointer was positioned over the active zone and the mouse button was pressed and a current position of the pointer, and transforming the difference into scaled and rotated coordinates using a scale transform and a rotate transform and adding the transformed difference to a translation transform to create a new translation transform, and wherein re-displaying the bounding box and the displayed object is in accordance with the scale transform, the rotate transform and the new translation transform.

46. The method of claim 44 wherein if the predefined movement type is rotation then the rotation of the displayed object and the bounding box is performed by transforming a position of the pointer when the pointer was positioned over the active zone and the mouse button was pressed by a scale transform and transforming a current position of the pointer by the scale transform and determining an angle created thereby versus a center point of the displayed object and concatenating a rotation matrix of the angle into a rotation transform to create a new rotation transform, and wherein re-displaying the displayed object and the bounding box on the video display is in accordance with the scale transform, the new rotation transform and a translation transform.

47. The method of claim 44 wherein if the predefined movement type is scaling then the scaling of the displayed object and the bounding box is performed by determining a ratio of a position of the pointer versus an origin of the bounding box when the pointer was positioned over the active zone and the mouse button was pressed and a current position of the pointer versus the origin of the bounding box and multiplying the ratio with a scaling transform to create a new scaling transform and wherein re-displaying the displayed object and the bounding box on the video display is in accordance with the new scaling transform, a rotate transform and a translation transform.

48. An apparatus for manipulating an object displayed in three-dimensional representation on a computer controlled display system having a computer and a display coupled to the computer, the apparatus comprising:

a) means for positioning a reference indicator over the displayed object and signaling the computer to activate a control movement mode;

b) means for generating on the display a three-dimensional representation of a bounding region including at least two portions which specify respective predefined control movement types, one of said types being rotation around an axis parallel to the portion which specifies rotation around that axis;

c) means for signaling the computer to activate a predefined control movement type specified by a portion of the bounding region under the reference indicator;

d) means for repositioning the reference indicator while the reference indicator is over the portion of the bounding region to define a movement of the predefined control movement type; and

e) means for re-displaying the displayed object in accordance with the defined movement of the predefined control movement type.

49. The apparatus of claim 48 wherein the three-dimensional representation of a bounding region is a generally rectilinear bounding box, and further comprising means for re-displaying the bounding box in accordance with the defined movement of the predefined control movement type.

50. The apparatus of claim 48 wherein the bounding region is reduced in size relative to the displayed object.

51. The apparatus of claim 48 wherein the positioning means and the repositioning means is a mouse.

52. The apparatus of claim 51 wherein the reference indicator is a pointer displayed on the display.

53. The apparatus of claim 52 wherein the means for signaling the computer to activate the control movement mode is a switch.

54. The apparatus of claim 53 wherein the switch is a button on the mouse.

55. The apparatus of claim 53 wherein the means for signaling the computer to activate a predefined control movement type includes a switch.

56. The apparatus of claim 55 wherein the switch is a button on the mouse.

57. The apparatus of claim 48 wherein the means for signaling the computer further comprises means for changing the shape of the reference indicator when the computer has been signaled to activate a predefined control movement type thus indicating to a user of the apparatus a type and direction of available object manipulations.

58. The apparatus of claim 48 wherein the portion of the bounding region is an active zone.

59. The apparatus of claim 48 wherein the portion of the bounding region defines a handle which graphically distinguishes said portion from the remainder of the bounding region.

60. The apparatus of claim 59 wherein the handle indicates, to a user of the apparatus, a type and direction of available object manipulations.

61. The apparatus of claim 59 wherein the handle extends outwardly from the bounding region such that the outwardly extending handle presents an appearance of being available to be selected by the reference indicator.

62. The apparatus of claim 59 wherein the handle is shaped like a rod.

63. The apparatus of claim 59 wherein the handle is shaped like a cube.

64. The apparatus of claim 59 wherein the handle is shaped like a arrow.

65. The apparatus of claim 48 wherein the portion of the bounding region is an image of a hand.

66. The apparatus of claim 65 wherein the hand indicates to a user of the apparatus a type and direction of available object manipulations.

67. The apparatus of claim 65 wherein the hand looks like a human hand.

68. The apparatus of claim 67 wherein the hand appears to be grabbing the three-dimensional representation of a bounding region.

69. The apparatus of claim 65 wherein the hand looks like a mechanical hand.

70. The apparatus of claim 69 wherein the hand appears to be grabbing the three-dimensional representation of a bounding region.

71. The apparatus of claim 48 wherein the predefined control movement type is a translation of the displayed object.

72. The apparatus of claim 71 wherein the translation of the displayed object is along model space coordinate axes.

73. The apparatus of claim 72 wherein the translation of the displayed object in model space is parallel to a plane of a face of the bounding region.

74. The apparatus of claim 72 wherein the translation of the displayed object in model space is perpendicular to a plane of a face of the bounding region.

75. The apparatus of claim 71 wherein the translation of the displayed object is performed by determining a difference in model space between a position of the reference indicator when the translation operation was activated and a current position of the reference indicator and transforming the difference into scaled and rotated coordinates using a scale transform and a rotate transform and adding the transformed difference to a translation transform to create a new translation transform and wherein re-displaying the displayed object on the video display is in accordance with the scale transform, the rotate transform and the new translation transform.

76. The apparatus of claim 71 wherein translation of the displayed object is parallel to a plane of a face of the bounding region and is limited to a predefined range of translation manipulations.

77. The apparatus of claim 48 wherein the axis of rotation of the displayed object is parallel to model space coordinate axes.

78. The apparatus of claim 77 wherein the rotation in model space of the displayed object is around an axis of the bounding region.

79. The apparatus of claim 78 wherein the axis of rotation is a center line of the bounding region.

80. The apparatus of claim 78 wherein the axis of rotation is an edge of the bounding region.

81. The apparatus of claim 80 wherein the axis of rotation is around a bounding region edge opposite from the portion of the bounding region under the reference indicator.

82. The apparatus of claim 77 wherein the axis of rotation is outside of the bounding region.

83. The apparatus of claim 48 wherein the rotation of the displayed object is performed by transforming a position of the reference indicator when the rotation operation was activated by a scale transform and transforming a current position of the reference indicator by the scale transform and determining an angle created thereby versus a center point of the displayed object, and concatenating a rotation matrix of the angle into a rotation transform to create a new rotation transform, and wherein re-displaying the displayed object on the video display is in accordance with the scale transform, the new rotation transform and a translation transform.

84. The apparatus of claim 48 wherein a second of the predefined control movement types is a scaling of the displayed object.

85. The apparatus of claim 84 wherein the scaling of the displayed object is along model space coordinate axes.

86. The apparatus of claim 85 wherein the scaling in model space is parallel to a plane of a face of the bounding region.

87. The apparatus of claim 84 wherein the scaling is homogenous.

88. The apparatus of claim 84 wherein the scaling of the displayed object is performed by determining a ratio of a position of the reference indicator versus an origin of the bounding region when the scaling operation was activated and a current position of the reference indicator versus the origin of the bounding region and multiplying the ratio with a scale transform to create a new scale transform and wherein re-displaying the displayed object on the video display is in accordance with the new scale transform, a rotate transform and a translation transform.

89. The apparatus of claim 88 wherein scaling of the displayed object is parallel to a plane of a face of the bounding region and is limited to a predefined range of scaling manipulations.

90. An apparatus for manipulating an object displayed in three-dimensional representation on a computer controlled video display system having a computer, a video display and a mouse, the apparatus comprising:

a) means for selecting the displayed object by using the mouse to position a pointer over the displayed object and pressing a button on the mouse;

b) means for displaying a three-dimensional representation of a bounding box having at least two active zones which specify respective available movement types, one of said types being rotation around an axis parallel to the zone which specifies rotation around that axis, when the displayed object is selected;

c) means for defining a movement of a type predefined by one of the active zones by using the mouse to position the pointer over an active zone, pressing the button on the mouse, and, while the pointer is over the active zone, repositioning the pointer with the mouse; and

d) means for re-displaying the bounding box and the displayed object in accordance with the defined movement.

91. The apparatus of claim 90 wherein the available movement types comprise translation and scaling.

92. The apparatus of claim 91 wherein if the predefined movement type is translation then the translation of the displayed object and the bounding box is performed by a means for determining a difference in model space between a position of the pointer when the pointer was positioned over the active zone and the mouse button was pressed and a current position of the pointer, and transforming the difference into scaled and rotated coordinates using a scale transform and a rotate transform and adding the transformed difference to a translation transform to create a new translation transform, and wherein re-displaying the bounding box and the displayed object is in accordance with the scale transform, the rotate transform and the new translation transform.

93. The apparatus of claim 91 wherein if the predefined movement type is rotation then the rotation of the displayed object and the bounding box is performed by a means for transforming a position of the pointer when the pointer was positioned over the active zone and the mouse button was pressed by a scale transform and transforming a current position of the pointer by the scale transform and determining an angle created thereby versus a center point of the displayed object and concatenating a rotation matrix of the angle into a rotation transform to create a new rotation transform and wherein re-displaying the displayed object and the bounding box on the video display is in accordance with the scale transform, the new rotation transform and a translation transform.

94. The apparatus of claim 91 wherein if the predefined movement type is scaling then the scaling of the displayed object and the bounding box is performed by a means for determining a ratio of a position of the pointer versus an origin of the bounding box when the pointer was positioned over the active zone and the mouse button was pressed and a current position of the pointer versus the origin of the bounding box and multiplying the ratio with a scaling transform to create a new scaling transform, and wherein re-displaying the displayed object and the bounding box on the video display is in accordance with the new scaling transform, a rotate transform and a translation transform.