Method and apparatus for generating and interfacing with rigid and deformable surfaces in a haptic virtual reality environment

• US 6,191,796 B1
• Filed: 01/21/1998
• Issued: 02/20/2001
• Est. Priority Date: 01/21/1998
• Status: Expired due to Term

• Current Assignee: 3D Systems, Inc. (3D Systems Corporation)
• Original Assignee: SensAble Technologies, Inc. (3D Systems Corporation)
• Inventors: Tarr, Christopher
• Primary Examiner(s): Nguyen, Phu K.
• Application Number:
  US09/010,295
• Time in Patent Office: 1,126 Days
• Field of Search: 345/430, 345/419, 345/431, 345/433, 345/427, 345/439, 345/127, 345/136
• US Class Current: 345/581
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2210/21 Collision detection, intersection
• CPC Class Codes: G06T 2210/28 Force feedback
• CPC Class Codes: G06T 2219/2021 Shape modification

Abstract

A method for haptically deforming a virtual surface within a haptic virtual environment is used to plastically deform the virtual surface of a virtual object by sensing a user'"'"'s position in real space, determining a haptic interface location in the haptic environment based thereon, and determining whether the virtual surface collides with the haptic interface location. Upon detection of a collision above a threshold force, a visual representation of the virtual surface is plastically deformed and a corresponding force is calculated and fed back to the user. The virtual surface can be visco-elastically deformed.

Claims

1. A method for haptically deforming a virtual surface within a haptic virtual environment, comprising the steps of:

generating a haptic interactive representation comprising a virtual deformable surface in a haptic interaction space, the virtual deformable surface comprising a triangular mesh;

sensing a position of a user in real space;

determining a haptic interface location in the haptic interaction space in response to the position of the user in real space; and

deforming the virtual surface in response to the haptic interface location.

2. The method of claim 1 wherein the step of generating a haptic interactive representation further comprises the step of generating a haptic interactive representation comprising a virtual plastically deformable surface.

3. The method of claim 1 wherein the step of generating a haptic interactive representation further comprises the step of generating a haptic interactive representation comprising a virtual visco-elastically deformable surface.

4. The method of claim 1 wherein the step of generating a haptic interactive representation further comprises the step of generating a haptic interactive representation comprising a virtual visco-elastically and plastically deformable surface.

5. The method of claim 1 wherein the step of deforming the virtual surface further comprises the steps of:

determining whether the virtual surface collides with the haptic interface location; and

if the virtual surface collides with the haptic interface location, performing the steps of;

(i) calculating an interaction force between the virtual surface and the user; and

(ii) if the calculated interaction force exceeds a predetermined threshold force, deforming the virtual surface.

6. The method of claim 5 further comprising the step of calculating a force to be applied to the user in real space in response to the interaction force after the step of calculating the interaction force.

7. The method of claim 1 wherein the triangular mesh has a predetermined density of vertices and wherein the step of deforming the virtual surface further comprises the step of updating the triangular mesh to maintain the predetermined density of vertices substantially constant.

8. The method of claim 1 wherein the step of generating a haptic interactive representation further comprises the step of generating a virtual deformable surface comprising a single-layer triangular mesh.

9. The method of claim 1 wherein the step of generating a haptic interactive representation further comprises the step of generating a three-dimensional virtual deformable surface.

10. The method of claim 1 wherein the the step of determining the haptic interface location in the haptic interaction space comprises determining the haptic interface location in response to the position of the haptic interactive representation.

11. The method of claim 1 wherein the step of generating the haptic interactive representation comprises generating a plurality of haptic interactive representations comprising a plurality of virtual deformable surfaces and the step of deforming the virtual surface comprises deforming at least one of the plurality of virtual deformable surfaces.

12. The method of claim 1, wherein

the step of determining a haptic interface location further comprises determining a location of a virtual tool in response to the position of the user in real space and the position of the haptic interactive representation;

and

the step of deforming tho virtual surface further comprises deforming the virtual surface in response to the virtual tool.

13. A method for haptically deforming a virtual surface within a haptic virtual reality environment, comprising the steps of:

generating a haptic interactive representation comprising a plastically deformable virtual surface in a haptic interaction space;

sensing a position of a user in real space;

determining a haptic interface location in the haptic interaction space in response to the position of the user in real space; and

plastically deforming the plastically deformable virtual surface in response to the haptic interface location.

14. The method of claim 13 wherein the step of generating a haptic interactive representation further comprises the step of generating a plastically deformable virtual surface comprising a single-layer triangular mesh.

15. The method of claim 14 wherein the step of plastically deforming the virtual surface further comprises the step of adding triangles to the triangular mesh.

16. The method of claim 13 wherein the step of generating a haptic interactive representation further comprises the step of generating a three-dimensional plastically deformable virtual surface.

17. The method of claim 13 wherein the step of plastically deforming the plastically deformable virtual surface further comprises the steps of:

determining whether the virtual surface collides with the haptic interface location; and

if the virtual surface collides with the haptic interface location, performing the steps of;

(i) calculating an interaction force between the virtual surface and the user; and

(ii) if the calculated interaction force exceeds a predetermined threshold force, deforming the virtual surface.

18. The method of claim 17 further comprising the step of calculating a force to be applied to the user in real space in response to the interaction force after the step of calculating an interaction force.

19. A method for determining forces to be applied to a user through a haptic interface, comprising the steps of:

generating a haptic interactive representation comprising a virtual surface in a haptic interaction space;

sensing a position of a user in real space;

determining a haptic interface location in the haptic interaction space in response to the position of the user in real space and the position of the haptic interactive representation;

representing the haptic interface location as a sphere comprising a penetrable outer layer and a substantially rigid inner core; and

determining a force to be applied to the user in real space in response to an intersection between the sphere and the virtual surface.

20. A system for generating and interfacing with a virtual deformable surface in a virtual reality environment, comprising:

a sensor for sensing a position of a user in real space;

a haptic rendering processor in electrical communication with the sensor, the haptic rendering processor executing an algorithm to determine feedback forces to be applied to the user in real space, the algorithm comprising;

a module generating a haptic interactive representation in a haptic interaction space comprising a virtual deformable surface comprising a triangular mesh;

a module determining a haptic interface in the haptic interaction space;

a module determining a haptic interface location in the haptic interaction space in response to the position of the user in real space and the position of the haptic interactive representation; and

a module determining a force to be applied to the user in real space in response to the haptic interface location; and

a surface interaction processor in electrical communication with the haptic rendering processor, the surface interaction processor determining deformations of the virtual surface.

21. The system of claim 20 wherein

the haptic rendering processor determines whether the virtual surface collides with the haptic interface location and calculates an interaction force between the virtual surface and the user if the virtual surface collides with the haptic interface location;

if the calculated interaction force exceeds a predetermined threshold force, the surface interaction processor deforms the virtual surface; and

if the calculated interaction force does not exceed a predetermined threshold force, the surface interaction processor slides the haptic interface location across the virtual surface.

22. The system of claim 21 wherein the haptic rendering processor calculates the force to be applied to the user in real space in response to the interaction force.

23. The system of claim 20 wherein the haptic rendering processor and the surface interaction processor are a single processor.

24. The system of claim 20 wherein the haptic interface is a virtual tool.

25. The system of claim 20 wherein the algorithm further comprises a module generating a plurality of haptic interactive representations comprising a plurality of virtual surfaces, and the surface interaction processor determines at least one deformation of at least one of the plurality of virtual surfaces.

26. A system for generating and interfacing with a virtual deformable surface in a virtual reality environment, comprising:

a sensor for sensing a position of a user in real space;

a haptic rendering processor in electrical communication with the sensor, the haptic rendering processor generating a haptic interaction space comprising a haptic interactive representation and a virtual tool, the haptic interactive representation comprising a virtual deformable surface comprising a triangular mesh wherein the haptic rendering processor determines (i) a virtual tool location in response to the position of the user in real space and the position of the haptic interactive representation and (ii) a force to be applied to the user in real space in response to the virtual tool location; and

a surface interaction processor in electrical communication with the haptic rendering processor, the surface interaction processor determining deformations of the virtual surface in response to the virtual tool.

27. The system of claim 26 wherein

the haptic rendering processor determines whether the virtual surface collides with the virtual tool and calculates an interaction force between the virtual surface and the user if the virtual surface collides with the virtual tool;

if the calculated interaction force exceeds a predetermined threshold force, the surface interaction processor deforms the virtual surface; and

if the calculated interaction force does not exceed a predetermined threshold force, the surface interaction processor slides the virtual tool across the virtual surface.

28. The system of claim 26 wherein the haptic rendering processor calculates the force to be applied to the user in real space in response to the interaction force.

29. The system of claim 26 wherein the haptic rendering processor and the surface interaction processor are a single processor.

30. The system of claim 26 wherein the haptic rendering processor generates a plurality of haptic interactive representations comprising a plurality of virtual surfaces and the surface interaction processor determines at least one deformation of at least one of the plurality of virtual surfaces.

31. A method for haptically deforming a virtual surface within a haptic virtual reality environment, comprising the steps of:

generating a haptic interactive representation comprising a virtual surface in a haptic interaction space;

determining a state of the virtual surface;

sensing a position of a user in real space;

determining a virtual tool for use by the user in the haptic interaction space;

determining a haptic interface position in the haptic interaction space in response to the position of the user in real space;

determining a position of the virtual tool in the haptic interaction space in comparison to the haptic interface position; and

calculating an interaction force between the virtual surface and the user in response to the step of determining the position of the virtual tool.

32. The method of claim 31 wherein the step of determining the position of the virtual tool further comprises moving the position of the virtual tool to coincide with the haptic interface position.

33. The method of claim 31 further comprising the step of determining an orientation of the virtual tool in the haptic interaction space, and wherein the step of calculating the interaction force between the virtual surface and the user further comprises calculating the interaction force in response to the step of determining the orientation of the virtual tool.

34. The method of claim 31 wherein the step of determining the state of the virtual surface further comprises determining the state of the viral surface to be a rigid state and the step of determining a position of the virtual tool in haptic interaction-space further comprises sliding the virtual tool across the virtual surface.

35. The method of claim 31 further comprising the steps of

determining whether the virtual surface collides with the vial tool; and

if the virtual surface collides with the virtual tool, preventing any movement of the virtual tool into the virtual surface.

36. The method of claim 35 further comprising the step of:

if the calculated inaction force does not exceed a predetermined threshold force, treating the virtual surface as a rigid surface.

37. The method of claim 35 further comprising the step of:

if the calculated interaction force exceeds a predetermined threshold force, deforming the virtual surface.

38. The method of claim 31 wherein the step of determining the state of the virtual surface further comprises the step of determining the state of the virtual surface to be a deformable state.

39. The method of claim 38 further comprising the step of deforming the virtual surface in response to the virtual tool.

40. The method of claim 39 wherein the step of deforming the virtual surface further comprises the steps of

determining whether the virtual surface collides with the virtual tool;

and

if the virtual surface collides with the virtual tool and the calculated interaction force exceeds a predetermined threshold force, deforming the virtual surface.

41. The method of claim 39 wherein the step of deforming the virtual surface further comprises the step of plastically deforming the virtual surface in response to the virtual tool.

42. The method of claim 39 wherein the virtual surface comprises a triangular mesh and the step of deforming the virtual surface further comprises the step of visco-elastically deforming the virtual surface in response to the virtual tool.

43. The method of claim 39 wherein the virtual surface comprises a triangular mesh and the step of deforming the virtual surface further comprises the step of visco-elastically and plastically deforming the virtual surface in response to the virtual tool.

44. A system for haptically deforming a virtual surface within a haptic virtual reality environment, comprising:

a sensor for sensing a position of a user in real space;

a haptic rendering processor in electrical communication with the sensor, the haptic rendering processor generating a haptic interaction space comprising a haptic interactive representation and a virtual tool, the haptic interactive representation comprising a virtual surface, wherein the haptic rendering processor determines (i) a haptic interface position in the haptic interaction space in response to the position of the user in real space, and (ii) a position of the virtual tool in the haptic interaction space in comparison to the haptic interface position; and

a surface interaction processor in electrical communication with the haptic rendering processor, wherein the surface interaction processor determines a state of the virtual surface and the haptic rendering processor calculates an interaction force between the virtual surface and the user based on the position of the virtual tool and the state of the virtual surface.

45. The system of claim 44 wherein the haptic rendering processor determines the position of the virtual tool by moving the position of the virtual tool to coincide with the haptic interface position.

46. The system of claim 44 wherein the haptic rendering processor determines an orientation of the virtual tool in the haptic interaction space, and calculates the interaction force between the virtual surface and the user in response to the orientation of the virtual tool.

47. The system of claim 44 wherein the surface interaction processor determines the state of the virtual surface to be a rigid state, and the haptic rendering processor slides the virtual tool across the virtual surface.

48. The system of claim 44 wherein the haptic rendering processor determines whether the virtual surface collides with the virtual tool, and, if the virtual surface collides with the virtual tool, prevents any movement of the virtual tool into the virtual surface.

49. The system of claim 48 wherein, if the calculated interaction force does not exceed a predetermined threshold force, the surface interaction processor determines the state of the virtual surface to be a rigid state.

50. The system of claim 48 wherein, if the calculated interaction force exceeds a predetermined threshold force, the surface interaction processor deforms the virtual surface.

51. The system of claim 44 wherein the surface interaction processor determines the state of the virtual surface to be a deformable state.

52. The system of claim 51 wherein the surface interaction processor deforms the virtual surface in response to the virtual tool.

53. The system of claim 52 wherein the haptic rendering processor determines whether the virtual surface collides with the virtual tool, and, if the virtual surface collides with the virtual tool and the calculated interaction force exceeds a predetermined threshold force, the surface interaction processor deforms the virtual surface.

54. The system of claim 52 wherein the surface interaction processor plastically deforms the virtual surface in response to the virtual tool.

55. The system of claim 52 wherein the virtual surface comprises a triangular mesh and wherein the surface interaction processor visco-elastically deforms the virtual surface in response to the virtual tool.

56. The system of claim 52 wherein the virtual surface comprises a triangular mesh and wherein the surface interaction processor viscoelastically and plastically deforms the virtual surface in response to the virtual tool.

57. The system of claim 44 wherein the haptic interactive representation is defined by a volumetric representation.

58. The system of claim 44 wherein the haptic interactive representation comprises a triangular mesh.

59. The system of claim 44 wherein the haptic interactive representation is defined by at least one implicit equation.

60. The system of claim 44 wherein the virtual tool is defined by a volumetric representation.

61. The system of claim 44 wherein the virtual tool is defined by at least one implicit equation.

62. The system of claim 44 wherein the virtual tool comprises a triangular mesh.

63. The system of claim 44 wherein the virtual tool is defined by a series of discrete points.