Method and apparatus for determining forces to be applied to a user through a haptic interface

• US 6,111,577 A
• Filed: 04/04/1996
• Issued: 08/29/2000
• Est. Priority Date: 04/04/1996
• Status: Expired due to Term

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Massie, Thomas H., Salisbury, J. Kenneth Jr., Srinivasan, Mandayam A., Brock, David Lawrence, Zilles, Craig B., Morgenbesser, Hugh B.
• Primary Examiner(s): Breene, John E.
• Application Number:
  US08/627,432
• Time in Patent Office: 1,608 Days
• Field of Search: 395/326-358, 345/156-179, 345/326-358, 345/433, 345/438, 345/337, 345/437, 318/568-589, 318/568.11-568.25, 318/568.1
• US Class Current: 715/701
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

A method and apparatus for determining forces to be applied to a user through a haptic interface. The method includes the steps of generating a representation of an object in graphic space, sensing the position of the user in real space and calculating a force to be applied to a user in response to the user'"'"'s haptic interface and the user'"'"'s fiducial object. The user'"'"'s fiducial object represents the location in graphic space at which the user'"'"'s haptic interface would be located if the haptic interface could not penetrate the surfaces of virtual objects. In one embodiment, the method calculates a stiffness force to be applied to the user. In other embodiments, the method calculates damping and friction forces to be applied to the user. In one embodiment the step of generating a representation of an object in graphic space includes defining the object as a mesh of planar surfaces and associating surface condition values to each of the nodes defining the planar surfaces. In another embodiment, the step of generating a representation of an object in graphic space includes describing the surface of the object using a coordinate system and associating surface condition values with each set of coordinates of the coordinate system.

Claims

1. A method for determining forces to be applied to a user through a haptic interface, said method comprising the steps of:

(a) generating a representation of an object in graphic space;

(b) sensing a position of a user in real space;

(c) determining a haptic interface location in graphic space in response to said position of said user in real space;

(d) determining a fiducial object location in graphic space; and

(e) calculating a stiffness force to be applied to said user in real space in response to said haptic interface location and said fiducial object location in graphic space.

2. The method of claim 1 wherein the haptic interface is a point and wherein the fiducial object is a point.

3. The method of claim 1 wherein the step of calculating a stiffness force further comprises the step of:

determining a displacement between said fiducial object location and said haptic interface location; and

calculating said stiffness force in response to said displacement.

4. The method of claim 1 wherein said haptic interface has a velocity and said fiducial object has a velocity, said method further comprising the steps of:

(f) determining the velocity of said haptic interface relative to a common reference;

(g) determining the velocity of said fiducial object relative to said common reference;

(h) calculating a damping force to be applied to said user in real space in response to said velocity of said haptic interface and said velocity of said fiducial object; and

(i) calculating a feedback force to be applied to said user in real space in response to said stiffness force and said damping force.

5. The method of claim 4 wherein said common reference is said representation of said object in graphic space.

6. The method of claim 4 wherein the step of generating a representation of an object in graphic space further comprises the step of defining said object as a mesh of planar surfaces, each of said planar surfaces having nodes 1-n, wherein n is a positive integer.

7. The method of claim 4 wherein the step of generating a representation of an object in graphic space further comprises the step of defining said object as an n-noded polygon having nodes 1-n, wherein n is a positive integer.

8. The method of claim 6 wherein said planar surfaces are triangular, said triangular planar surfaces having a first node, a second node and a third node.

9. The method of claim 8 wherein the stiffness force has a magnitude and wherein the step of calculating a stiffness force to be applied to said user in real space further comprises the step of:

calculating the magnitude of the stiffness force, wherein the step of calculating the magnitude of the stiffness force further comprises the steps of;

associating a stiffness value with said first node, said second node and said third node of each said triangular planar surface;

determining on which of said triangular planar surfaces said fiducial object is located; and

computing a stiffness value of said fiducial object location by interpolating said stiffness values associated with said first node, said second node and said third node of each of said triangular planar surfaces on which said fiducial object is located.

10. The method of claim 8 wherein the stiffness force has a direction and wherein the step of calculating a stiffness force to be applied to said user in real space further comprises the step of:

calculating the direction of the stiffness force, wherein the step of calculating the direction of the stiffness force further comprises the steps of;

associating a surface normal with each said first node, said second node and said third node of each said triangular planar surface;

determining on which of said triangular planar surfaces said fiducial object is located; and

computing a surface normal for said fiducial object location by interpolating said surface normals associated with said first node, said second node and said third node of each of said triangular planar surfaces on which said fiducial object is located.

11. The method of claim 8 wherein the damping force has a magnitude and wherein the step of calculating a damping force to be applied to said user in real space further comprises the step of:

calculating the magnitude of the damping force, wherein the step of calculating the magnitude of the damping force further comprises the steps of;

associating a damping coefficient with each said first node, said second node and said third node of each said triangular planar surface;

determining on which of said planar surfaces said fiducial object is located; and

computing a damping coefficient of said fiducial object location by interpolating said damping coefficients associated with said first node, said second node and said third node of each of said triangular planar surfaces on which said fiducial object is located.

12. The method of claim 8 wherein the damping force has a direction and wherein the step of calculating a damping force to be applied to said user in real space further comprises the step of:

calculating the direction of the damping force, wherein the step of calculating the direction of the damping force further comprises the steps of;

associating a surface normal with each said first node, said second node and said third node of each said triangular planar surface;

determining on which of said triangular planar surfaces said fiducial object is located; and

computing a surface normal for said fiducial object location by interpolating said surface normals associated with said first node, said second node and said third node of each said triangular planar surfaces on which said fiducial object is located.

13. The method of claim 6 wherein the stiffness force has a magnitude and wherein the step of calculating a stiffness force to be applied to said user in real space further comprises the step of:

calculating the magnitude of the stiffness force, wherein the step of calculating the magnitude of the stiffness force further comprises the steps of;

associating a stiffness value with each said 1-n nodes of each said planar surface;

determining on which of said planar surfaces said fiducial object is located; and

computing a stiffness value of said fiducial object location by interpolating said stiffness values associated with said 1-n nodes of each of said planar surfaces on which said fiducial object is located.

14. The method of claim 6 wherein the stiffness force has a direction and wherein the step of calculating a stiffness force to be applied to said user in real space further comprises the step of:

calculating the direction of the stiffness force, wherein the step of calculating the direction of the stiffness force further comprises the steps of;

associating a surface normal with each said 1-n nodes of each said planar surface;

determining on which of said planar surfaces said fiducial object is located; and

computing a surface normal for said fiducial object location by interpolating said surface normals associated with said 1-n nodes of each of said planar surfaces on which said fiducial object is located.

15. The method of claim 6 wherein the damping force has a magnitude and wherein the step of calculating a damping force to be applied to said user in real space further comprises the step of:

calculating the magnitude of the damping force, wherein the step of calculating the magnitude of the damping force further comprises the steps of;

associating a damping coefficient with each said 1-n nodes of each said planar surface;

determining on which of said planar surfaces said fiducial object is located; and

computing a damping coefficient of said fiducial object location by interpolating said damping coefficients associated with said 1-n nodes of each of said planar surfaces on which said fiducial object is located.

16. The method of claim 6 wherein the damping force has a direction and wherein the step of calculating a damping force to be applied to said user in real space further comprises the step of:

calculating the direction of the damping force, wherein the step of calculating the direction of the damping force further comprises the steps of;

associating a surface normal with each said 1-n nodes of each said planar surface;

determining on which of said planar surfaces said fiducial object is located; and

computing a surface normal for said fiducial object location by interpolating said surface normals associated with said 1-n nodes of each said planar surfaces on which said fiducial object is located.

17. The method of claim 4 wherein said object has a surface and wherein the step of generating a representation of an object in graphic space further comprises the steps of:

describing the surface of said object using a coordinate system, said coordinate system comprising a plurality of coordinate sets; and

associating a parameter with each set of coordinates of said plurality of coordinate sets.

18. The method of claim 17 wherein the parameter is a stiffness value and wherein the step of calculating a stiffness force to be applied to said user in real space further comprises the steps of:

determining which set of coordinates of said plurality of coordinate sets describes said fiducial object location; and

determining a stiffness value of said fiducial object location in response to said set of coordinates describing said fiducial object location.

19. The method of claim 17 wherein the parameter is a surface normal and wherein the step of calculating a stiffness force to be applied to said user in real space further comprises the steps of:

determining which set of coordinates of said plurality of coordinate sets describes said fiducial object location; and

determining a surface normal for said fiducial object location in response to said set of coordinates describing said fiducial object location.

20. The method of claim 17 wherein the parameter is a surface normal and wherein the step of calculating a damping force to be applied to said user in real space further comprises the steps of:

determining which set of coordinates of said plurality of coordinate sets describes said fiducial object location; and

determining a surface normal for said fiducial object location in response to said set of coordinates describing said fiducial object location.

21. The method of claim 17 wherein the parameter is a displacement force, wherein each set of coordinates has a corresponding surface normal and wherein the step of calculating a damping force to be applied to said user in real space further comprises the step of:

adding each said displacement force of each said set of coordinates to said corresponding surface normal to calculate a total surface normal for each said set of coordinates;

determining which set of coordinates of said plurality of coordinate sets describes said fiducial object location; and

determining a total surface normal for said fiducial object location in response to said set of coordinates describing said fiducial object location.

22. The method of claim 4 further comprising the step of:

subsequent to calculating said feedback force to be applied to said user in real space, producing said feedback force by a force actuating device.

23. The method of claim 22 wherein said object has a surface and wherein said generated representation of said object includes a surface representation, said method further comprising the steps of:

defining a plane tangent to said surface representation at said fiducial object location,

wherein said produced feedback force is normal to said plane.

24. The method of claim 22 wherein said velocity of said fiducial object has a first direction and wherein said produced feedback force has a second direction, said second direction being opposite said first direction.

25. The method of claim 4 further comprising the steps of:

determining a first stiction location in graphic space;

calculating a friction force to be applied to said user in real space in response to said haptic interface location and said first stiction location in graphic space; and

calculating a total force to be applied to said user in real space in response to said friction force and said feedback force.

26. The method of claim 25 further comprising after the step of calculating a friction force and before the step of calculating a total force to be applied to said user in real space, the steps of:

multiplying said feedback force by a first coefficient of friction to determine a product;

determining whether said friction force is greater than or equal to said product of said feedback force and said first coefficient of friction;

updating said first stiction location to a second stiction location in graphic space if said friction force is greater than said product; and

re-calculating said friction force in response to said second stiction location in graphic space if said first stiction location was updated.

27. The method of claim 26 wherein the step of updating said first stiction location to a second stiction location in graphic space further comprises the steps of:

calculating a maximum friction force in response to said normal force to be applied to said user in real space and a second coefficient of friction;

placing said second stiction location along a line intersecting said first stiction location and a current haptic interface location; and

placing said second stiction location along said line at a distance from said current haptic interface location where said friction force to be applied to said user is substantially equal to said calculated maximum friction force.

28. The method claim 27 wherein said first coefficient of friction is substantially equal to said second coefficient of friction.

29. The method of claim 27 wherein said second coefficient of friction is lower than said first coefficient of friction.

30. The method of claim 25 further comprising after the step of calculating a total force to be applied to said user in real space, the steps of:

calculating a first distance in graphic space between said fiducial object location and said first stiction location;

calculating a second distance in graphic space between said haptic interface location and said fiducial object location;

determining whether said first distance is greater than said second distance multiplied by a third coefficient of friction;

if said first distance is greater than said second distance multiplied by said third coefficient of friction, moving said first stiction location in graphic space toward said fiducial object location until said first distance is not greater than said second distance multiplied by said third coefficient of friction; and

calculating a change in total force to be applied to said user in real space in response to said moved first stiction location and said haptic interface location.

31. The method of claim 4 further comprising the steps of:

determining a first stiction location in graphic space, said first stiction location having a velocity;

calculating a friction force to be applied to said user in real space in response to said fiducial object location, said first stiction location, said velocity of said fiducial object and said velocity of said first stiction location; and

calculating a total force to be applied to said user in real space in response to said friction force and said feedback force.

32. A method for determining forces to be applied to a user through a haptic interface, said method comprising the steps of:

(a) generating a representation of an object in graphic space;

(b) displaying said representation of said object on a display;

(c) sensing a first position of a user in real space;

(d) determining a first haptic interface location in graphic space in response to said first position of said user in real space;

(e) determining a first fiducial object location in graphic space;

(f) displaying said first position of said user relative to said object on said display; and

(g) calculating a first stiffness force to be applied to said user in real space in response to said first haptic interface location and said first fiducial object location in graphic space.

33. The method of claim 32 further comprising the steps of:

(h) storing a first set of state variables representing said first fiducial object location in graphic space;

(i) sensing a second position of said user in real space;

(j) determining a second haptic interface location in graphic space in response to said second position of said user in real space;

(k) calculating a second fiducial object location in graphic space in response to said first set of state variables and said second haptic interface location;

(l) removing said first position of said user relative to said object from said display and displaying said second position of said user relative to said object on said display; and

(m) repeating step (g) for said second haptic interface location and said second fiducial object location to determine a second stiffness force to be applied to said user in real space.

34. The method of claim 33 wherein the step of generating a representation of an object in graphic space further comprises the step of defining said object as a mesh of surfaces, and wherein the step of calculating a second fiducial object location in graphic space further comprises the steps of:

(n) determining a line intersecting said first fiducial object location and said second haptic interface location;

(o) determining which of said surfaces said line intersects;

(p) determining whether each said surface intersected by said line is located a negative distance from said first fiducial object location;

(q) determining whether each said surface intersected by said line is located a positive distance from said second haptic interface location;

(r) defining each said surface intersected by said line, located a negative distance from said first fiducial object location and located a positive distance from said second haptic interface location as an active constraint;

(s) computing said second fiducial object location in response to said second haptic interface location and said active constraints;

(t) repeating steps (n)-(r) using said computed second fiducial object location in place of said second haptic interface location to determine additional active constraints;

(u) re-computing said second fiducial object location in response to said additional active constraints; and

(v) repeating steps (t)-(u) until no new additional active constraints are determined.

35. The method of claim 33 wherein the step of generating a representation of an object in graphic space further comprises the step of defining said object as a mesh of triangular planar surfaces, each of said triangular planar surface having three boundary lines, and wherein the step of calculating a second fiducial object location in graphic space further comprises the steps of:

(n) determining a line intersecting said first fiducial object location and said second haptic interface location;

(o) determining which of said triangular planar surfaces said line intersects;

(p) determining whether each said triangular planar surface intersected by said line is located a negative distance from said first fiducial object location;

(q) determining whether each said triangular planar surface intersected by said line is located a positive distance from said second haptic interface location;

(r) defining each said triangular planar surface intersected by said line, located a negative distance from said first fiducial object location and located a positive distance from said second haptic interface location as an active constraint;

(s) computing said second fiducial object location in response to said second haptic interface location and said active constraints;

(t) repeating steps (n)-(r) using said computed second fiducial object location in place of said second haptic interface location to determine additional active constraints;

(u) re-computing said second fiducial object location in response to said additional active constraints; and

(v) repeating steps (t)-(u) until no new additional active constraints are determined.

36. An apparatus for determining and applying feedback forces to a user in real space through a haptic interface, said apparatus comprising:

a sensor sensing positions of a user in real space;

a first processor in electrical communication with said sensor executing an algorithm to determine feedback forces to be applied to said user in real space, said algorithm including;

a module generating a representation of an object in graphic space,

a module determining a haptic interface location in graphic space in response to a position of said user in real space,

a module determining a fiducial object location in graphic space, and

a module calculating a stiffness force to be applied to said user in real space in response to said haptic interface location and said fiducial object location;

a display processor in electrical communication with said first processor, said display processor displaying said representation of said object on a display and displaying said positions of said user relative to said object on said display; and

a force actuator in electrical communication with said first processor and said display processor, said force actuator producing said stiffness force to be applied to said user in real space.

37. The apparatus of claim 36 wherein said first processor and said display processor are the same processor.

38. The apparatus of claim 36 wherein said module generating a representation of an object in graphic space, said module determining a haptic interface location in graphic space, said module determining a fiducial object location in graphic space and said module calculating a stiffness force to be applied to said user in real space are separate devices.