Systems and methods for interacting with virtual objects in a haptic virtual reality environment

• US 6,421,048 B1
• Filed: 06/28/1999
• Issued: 07/16/2002
• Est. Priority Date: 07/17/1998
• Status: Expired due to Term

• Current Assignee: 3D Systems, Inc. (3D Systems Corporation)
• Original Assignee: SensAble Technologies, Inc. (3D Systems Corporation)
• Inventors: Tarr, Christopher M., Shih, Loren, Massie, Thomas H., Aviles, Walter A.
• Primary Examiner(s): Nguyen, Phu K.
• Application Number:
  US09/340,875
• Time in Patent Office: 1,114 Days
• Field of Search: 345/419, 345/420, 345/440, 345/441, 345/156, 345/157
• US Class Current: 345/419
• CPC Class Codes: B33Y 50/00 Data acquisition or data processing for additive manufacturing
• CPC Class Codes: B33Y 80/00 Products made by additive manufacturing
• CPC Class Codes: G06F 2203/013 Force feedback applied to a game
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2210/21 Collision detection, intersection
• CPC Class Codes: G06T 2210/28 Force feedback
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling

Abstract

A user of a modeling application uses a haptic interface device in the real world to manipulate a virtual tool in a virtual environment to interact with a virtual object. The user can use the tool to evaluate the shape of the virtual object and navigate its virtual surface, based on an interactive force feedback approach. When the user attempts to penetrate the virtual object with the virtual tool, a modeling application limits the movement of the virtual tool depending on the geometry of the surface, the position of the virtual tool, and a haptic interface location in the virtual environment, which represents the physical location of the haptic interface device in the real world. The user can evaluate different geometries of the virtual surface, including an edge geometry, such as occurs when the virtual tool is touching or moving along an edge of the virtual object.

Claims

1. A method for interfacing with a virtual object in a haptic virtual environment, comprising:

generating a virtual object comprising a virtual surface in the haptic virtual environment;

sensing a location of a user in real space;

determining a virtual tool comprising a plurality of discrete points for use by the user in the haptic virtual environment;

determining a haptic interface location in the haptic virtual environment in response to the location of the user in real space;

determining locations for the plurality of discrete points of the virtual tool in the haptic virtual environment in comparison to the haptic interface location and a location of the virtual surface;

determining if at least one of the plurality of discrete points of the virtual tool penetrates the virtual surface;

if at least one of the plurality of discrete points penetrates the virtual surface, determining a geometry for the virtual surface at an area of penetration of the virtual tool; and

limiting movement of the virtual tool based on (i) the geometry of the virtual surface, (ii) at least one location of at least one of the plurality of discrete points, and (iii) the haptic interface location.

2. The method of claim 1, wherein the step of limiting movement of the virtual tool further comprises moving a position of the virtual tool toward the haptic interface location.

3. The method of claim 2, further comprising the step of determining a surface contact point based on (i) the geometry of the virtual surface, (ii) at least one location of at least one of the plurality of the discrete points of the virtual tool, and (iii) the haptic interface location.

4. The method of claim 1, further comprising the step of determining a surface direction vector in response to the steps of determining the locations for the plurality of discrete points of the virtual tool in the haptic virtual environment and determining if at least one of the plurality of discrete points of the virtual tool penetrates the virtual surface.

5. The method of claim 4, further comprising the step of determining a constraint plane in response to the step of determining a surface direction vector, and wherein the step of limiting the movement of the virtual tool further comprises limiting the movement based on the constraint plane.

6. The method of claim 1, wherein the step of determining the geometry of the virtual surface comprises determining an edge geometry.

7. The method of claim 6, wherein the step of determining the edge geometry comprises determining if the virtual tool has penetrated two portions of the virtual surface;

and determining two edge surface direction vectors in response to (i) the two portions and (ii) at least one location of at least one of the plurality of discrete points of the virtual tool.

8. The method of claim 7, wherein the step of determining the geometry for the virtual surface comprises determining an edge line by calculating the cross product of the two edge surface direction vectors;

and the step of limiting the movement of the virtual tool comprises limiting the movement of the virtual tool based on the edge line.

9. The method of claim 1, further comprising calculating an interaction force between the virtual object and the virtual tool in response to the step of determining the locations of the plurality of discrete points of the virtual tool.

10. A system for interfacing with a virtual object in a haptic virtual environment, comprising:

the virtual object comprising a virtual surface;

a haptic interface device, wherein the haptic interface device senses a location of a user in real space;

a virtual tool comprising a plurality of discrete points for use by the user in the haptic virtual environment; and

a modeling application in communication with the haptic interface device, the virtual object, and the virtual tool, wherein the modeling application (a) determines a haptic interface location in the haptic virtual environment in response to the location of the user in real space;

(b) determines locations for the plurality of discrete points of the virtual tool in the haptic virtual environment in comparison to the haptic interface location and a location of the virtual surface;

(c) determines a geometry for the virtual surface at an area where at least one of the plurality of discrete points of the virtual tool penetrates the virtual surface; and

(d) limits movement of the virtual tool based on (i) the geometry of the virtual surface, (ii) at least one location of at least one of the plurality of discrete points, and (iii) the haptic interface location.

11. The system of claim 10, wherein the modeling application limits the movement of the virtual tool by moving a position of the virtual tool toward the haptic interface location.

12. The system of claim 11, further comprising a surface contact point, wherein the modeling application determines the surface contact point based on (i) the geometry of the virtual surface, (ii) at least one location of at least one of the plurality of the discrete points of the virtual tool, and (iii) the haptic interface location.

13. The system of claim 10, further comprising a surface direction vector, wherein at least one of the plurality of discrete points of the virtual tool penetrates the virtual surface and the modeling application determines the surface direction vector based on the locations for the plurality of discrete points of the virtual tool in the haptic virtual environment.

14. The system of claim 13, further comprising a constraint plane, wherein the modeling application determines the constraint plane based on the surface direction vector and limits the movement of the virtual tool based on the constraint plane.

15. The system of claim 10, wherein the geometry of the virtual surface is an edge geometry.

16. The system of claim 15, further comprising two edge surface direction vectors;

and the virtual surface comprising two portions, wherein the virtual tool has penetrated the two portions of the virtual surface and the modeling application determines the two edge surface direction vectors based on (i) the two portions and (ii) at least one location of at least one of the plurality of discrete points of the virtual tool.

17. The system of claim 16, wherein the edge geometry comprises an edge line, wherein the modeling application determines the edge line by calculating the cross product of the two edge surface direction vectors, and limits the movement of the virtual tool based on the edge line.

18. The system of claim 10, further comprising an interaction force between the virtual object and the virtual tool, wherein the modeling application calculates the interaction force based on the locations of the plurality of discrete points of the virtual tool in comparison to the haptic interface location and the location of the virtual surface.

19. A method for interfacing with a virtual surface in a haptic virtual environment, comprising:

generating a virtual surface in the haptic virtual environment;

sensing a location of a user in real space;

determining a virtual representation of the user in real space, the virtual representation comprising a plurality of discrete points;

determining a haptic interface location in the haptic virtual environment in response to the location of the user in real space;

determining a virtual representation location in the haptic virtual environment;

moving the virtual representation location toward the haptic interface location in the haptic virtual environment; and

limiting movement of the virtual representation based on a geometry of the surface and on preventing any one of the plurality of discrete points of the virtual representation from substantially penetrating the virtual surface.

20. The method of claim 19 wherein the virtual surface is derived from a set of scalar values arranged at spatial locations.

21. The method of claim 20 wherein the set of scalar values is arranged in a regularly spaced three dimensional grid.

22. The method of claim 19 wherein the virtual surface is an isosurface of a set of voxels.

23. The method of claim 19 further comprising a step of determining a force feedback vector and a step of sending the force feedback vector to the user through a haptic interface.

24. The method of claim 23 wherein the step of determining the force feedback vector comprises determining the force feedback vector derived from a difference in position of the haptic interface location and the virtual representation location.

25. The method of claim 24 wherein the step of determining the force feedback vector comprises determining the force feedback vector derived from a difference in velocity of the haptic interface location and the virtual representation location.

26. The method of claim 19 wherein the step of determining the virtual representation comprises determining a virtual tool.

27. The method of claim 26 wherein the step of determining the virtual tool comprises arranging the plurality of discrete points of the virtual tool to approximate a sphere.

28. The method of claim 19 wherein the step of generating a virtual surface comprises generating a virtual object from a closed surface.

29. The method of claim 28 further comprises steps of reading the virtual object from a file, modifying the virtual object in shape, and saving the virtual object to the file.

30. The method of claim 19 wherein the step of moving the virtual representation comprises moving the virtual representation iteratively.

31. The method of claim 19 wherein the step of limiting movement is achieved by determining a constraint plane based on the geometry of the virtual surface.

32. The method of claim 19 wherein the step of limiting movement is achieved by determining a constraint edge based on the geometry of the virtual surface.

33. The method of claim 19 wherein the step of determining the haptic interface location comprises determining a position and an orientation of a haptic interface.

34. The method of claim 33 further comprising a step of determining a force feedback vector;

a step of sending the force feedback to the user through the haptic interface, wherein the haptic interface is adapted for measuring six degrees of freedom.

35. The method of claim 34 further comprising the step of applying the force feedback vector through the haptic interface device with less than six degrees of freedom of force feedback.

36. The method of claim 34 wherein the step of determining the force feedback comprises a step of determining a force in at least one of three principal directions and a torque in at least one of three principal rotations.

37. A method for interfacing with a virtual surface in a haptic virtual environment, comprising:

generating a virtual surface in the haptic virtual environment;

sensing a location of a user in real space;

determining a virtual representation of the user in real space, the virtual representation comprising a plurality of discrete points;

determining a haptic interface location in the haptic virtual environment in response to the location of the user in real space;

determining a virtual representation location in the haptic virtual environment;

proposing to move the virtual representation to a first proposed location in the direction of the haptic interface location;

evaluating at least one of the plurality of discrete points of the virtual representation to detect interference of the virtual representation with the virtual surface for the first proposed location of the virtual representation;

moving the virtual representation to the first proposed location if no interference of the virtual representation with the virtual surface is detected; and

proposing a second location of the virtual representation if interference of the virtual representation with the virtual surface is detected at the first proposed location.

38. The method of claim 37 wherein the step of evaluating at least one of the plurality of discrete point comprises determining an interference point from the plurality of discrete points, the interference point interfering with the virtual surface at the first proposed location, and wherein the step of proposing the second proposed location comprises calculating the second proposed location based on local geometry information of the virtual surface near the interference point.

39. The method of claim 38 further comprising a step of determining a vector formed by projecting the haptic interface location onto a plane passing through a current location of the virtual representation, the plane being tangent to the virtual surface near the interference point for the first proposed position and wherein the step of proposing the second proposed location comprises proposing the second proposed location in the direction of the vector.

40. The method of claim 39 further comprising the steps of

evaluating at least one of the discrete set of points to check for interference of the virtual representation with the virtual surface for the second proposed location of the virtual representation;

moving the virtual representation to the second proposed location if no interference of the virtual representation with the virtual surface is detected; and

proposing a third location of the virtual representation if interference of the virtual representation with the virtual surface is detected at the second proposed location.

41. The method of claim 40 further comprising a step of determining a line formed by the intersection of two planes, the two planes derived from local geometry information in a region near the first proposed location and the second proposed location, and wherein the step of proposing the third proposed location comprises determining the third proposed location in the direction of the line.

42. The method of claim 40 further comprising a step of determining a resultant vector from the cross product of two vectors derived from local geometry information in a region near the first proposed location and second proposed location, and wherein the step of proposing the third proposed location comprises determining the third proposed location in the direction of the proposed resultant vector.

43. The method of claim 37 further comprising the steps of

evaluating at least one of the discrete set of points to detect interference of the virtual representation with the virtual surface for the second proposed location of the virtual representation;

moving the virtual representation to the second proposed location if no interference of the virtual representation with the virtual surface is detected; and

proposing a third location of the virtual representation if interference of the virtual representation with the virtual surface is detected at the second proposed location.

44. A method for interfacing with a virtual surface in a haptic virtual environment, comprising:

generating a virtual surface in the haptic environment based on a set of scalar values arranged in a three dimensional grid;

sensing a location of a user in real space;

determining a virtual representation of the user in real space;

determining a haptic interface location in the haptic virtual environment in response to the location of the user in real space;

determining a virtual representation location in the haptic virtual environment;

moving the virtual representation location in the haptic virtual environment based on the previous virtual representation location, the haptic interface location in the virtual environment, and the geometry of the virtual surface.