Method and apparatus for providing force feedback to the user of an interactive computer simulation

• US 6,037,927 A
• Filed: 04/07/1997
• Issued: 03/14/2000
• Est. Priority Date: 07/14/1994
• Status: Expired due to Term

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Rosenberg, Louis B.
• Primary Examiner(s): Liang, Regina
• Application Number:
  US08/833,502
• Time in Patent Office: 1,072 Days
• Field of Search: 345/7, 345/8, 345/161, 345/166, 345/157, 345/156, 345/158, 345/184, 318/567, 318/628, 395/95, 395/99, 74/471, 364/578
• US Class Current: 345/156
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/76 Manipulators having means for providing feel, e.g. force or tactile feedback
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G05B 19/409 characterised by using manual data input [MDI] or by using control panel, e.g. controlling functions with the panel; characterised by control panel details or by setting parameters G05B19/408, G05B19/4093 take precedence
• CPC Class Codes: G05B 2219/35338 Display virtual tool, locus, part to check possibility of execution next block
• CPC Class Codes: G05B 2219/35441 Production design metaphore, tool, operation like input system
• CPC Class Codes: G05B 2219/40119 Virtual internal model, derive from forces on object, motion of end effector
• CPC Class Codes: G05B 2219/40137 Force sensation feedback from simulated tool
• CPC Class Codes: G05B 2219/45118 Endoscopic, laparoscopic manipulator
• CPC Class Codes: G16H 50/50 for simulation or modelling of medical disorders

Abstract

An apparatus for interfacing the movement of a shaft with a computer includes a support, a gimbal mechanism having two degrees of freedom, and three electromechanical transducers. When a shaft is engaged with the gimbal mechanism, it can move with three degrees of freedom in a spherical coordinate space, where each degree of freedom is sensed by one of the three transducers. A fourth transducer can be used to sense rotation of the shaft around an axis. The method includes the steps of defining an origin in 3-dimensional space, physically constraining a shaft in the 3-dimensional space such that a portion of the shaft always intersects the origin and such that a portion of the shaft extending beyond the origin defines a radius in a spherical coordinate system, transducing a first electrical signal related to a first angular coordinate of the radius with a first transducer, transducing a second electrical signal related to a second angular coordinate with a second transducer, transducing a third electrical signal related to the length of the radius with a third transducer, and coupling the transducers to a computer.

Claims

1. A method for providing force feedback to a user of an interactive computer simulation, the method comprising:

maintaining on a host computer having a display screen a virtual reality medical simulation suitable for representing a plurality of virtual reality objects within a graphical environment;

coupling a human/computer interface with the host computer, the human/computer interface including at least a portion of a medical instrument suitable for manipulation by a user, an actuator arranged to generate force feedback to the user through the portion of the medical instrument in response to signals from the host computer, and a local microprocessor separate from the host computer and executing a local software routine at the same time as the maintaining of the virtual reality medical simulation by the host computer said local software routine operable to decode force commands transmitted from the host computer and related to the control of the actuator;

representing the portion of the medical instrument as a virtual reality object within the virtual reality simulation, the virtual reality object having a simulated position corresponding to a position of the portion of the medical instrument;

displaying on the display screen a graphical representation of the virtual reality simulation including a graphical object representing the portion of the medical instrument;

detecting, at the human/computer interface, position information related to the portion of the medical instrument;

transmitting position data from the human/computer interface to the host computer, the position data representing the position information related to the portion of the medical instrument;

updating the virtual reality simulation such that the simulated position of the virtual reality object corresponds to the position data;

receiving at the human/computer interface a force command from the host computer; and

responding to the force command with the local microprocessor including decoding the force command and controlling said actuator in accordance with the force command thereby providing force feedback to the user, wherein said force feedback is provided in coordination with the updating of said virtual reality medical simulation, thereby creating a feel sensation that corresponds with the virtual reality object encountering a different virtual object in said virtual reality medical simulation.

2. A method as recited in claim 1 wherein the human/computer interface further includes a gimbal mechanism and a support, the gimbal mechanism coupling the portion of the medical instrument to the support such that the portion of the medical instrument has at least two degrees of freedom of motion with respect to the support.

3. A method as recited in claim 2 wherein the two degrees of freedom of motion are provided about a first axis and a second axis, the first and second axes being substantially mutually perpendicular and intersecting at an origin point.

4. A method as recited in claim 3 wherein the human/computer interface further includes first and second sensors, the first sensor for detecting rotary motion of the portion of the medical instrument about the first axis and the second sensor for detecting rotary motion of the portion of the medical instrument about the second axis.

5. A method as recited in claim 4 wherein the virtual reality simulation includes a representation of a body.

6. A method as recited in claim 5 wherein the force feedback to the user includes a resistance force that causes a pushing of the portion of the medical instrument by the user more difficult.

7. A method as recited in claim 6 wherein the portion of the medical instrument represents a laparoscopic instrument and is formed in the shape of a physical laparoscopic instrument handle.

8. A method as recited in claim 5 wherein the medical instrument being represented is a catheter and the portion of the medical instrument is formed in the shape of a catheter wire.

9. A method as recited in claim 6 wherein the force feedback to the user includes a twisting force on a shaft of the portion of the medical instrument when the virtual reality object encounters a virtual obstacle.

10. A method as recited in claim 2 wherein the actuator is coupled between the support and the gimbal mechanism, the human/computer interface further including a continuous belt loop that is turned when the portion of the medical instrument is rotated by the user, the continuous loop belt arranged such that a substantial force may be applied to said continuous loop belt without causing substantial slippage of the continuous loop belt.

11. A method as recited in claim 1 wherein the force command corresponds to the virtual reality object encountering an obstacle within the virtual reality medical simulation.

12. A method as recited in claim 11 wherein the virtual reality object is a simulated surgical tool, the virtual reality medical simulation includes a simulated patient body, and the obstacle encountered by the virtual reality object involves the simulated surgical tool interacting with the simulated patient body.

13. A method as recited in claim 1 wherein the actuator applies force through the portion of the medical instrument through the frictional engagement of a drive wheel.

14. A method as recited in claim 4 wherein the portion of the medical instrument has a third linear degree of freedom, wherein force is applied to said third linear degree of freedom through frictional engagement of a drive wheel with said portion of the medical instrument.

15. A virtual reality medical apparatus comprising:

a host computer having display means;

means for maintaining a virtual reality medical simulation on the host computer, the virtual reality medical simulation suitable for representing a plurality of virtual reality objects;

human/computer interface means for providing force feedback to a user of the virtual reality apparatus, the human/computer interface means coupled with the host computer, the human/computer interface means including,

a) at least a portion of a medical instrument suitable for manipulation by a user,

b) actuator means arranged to generate force feedback to the user through the portion of the medical instrument in response to signals from the host computer,

c) local microprocessor means separate from the host computer for executing a local software routine at the same time as the maintaining of the virtual reality medical simulation by the host computer the local software routine operable to decode force commands transmitted from the host computer in order to control the actuator such that the user experiences force sensations corresponding to interactions of virtual objects represented in the virtual reality medical simulation and in coordination with the maintaining of the virtual reality medical simulation,

d) means for detecting position information related to the portion of the medical instrument,

e) means for transmitting position data to the host computer, the position data representing the position information related to the portion of the medical instrument, and

f) means for receiving a force command from the host computer;

means for representing the portion of the medical instrument as a virtual reality object within the virtual reality medical simulation, the virtual reality object having a simulated position corresponding to a position of the portion of the medical instrument;

means for displaying on the display means a graphical representation of the virtual reality medical simulation including a graphical object representing the portion of the medical instrument; and

means for updating the virtual reality medical simulation such that the simulated position of the virtual reality object corresponds to the position data.

16. A virtual reality apparatus as described in claim 15 wherein the human/computer interface means further includes gimbal means and support means, the gimbal means for coupling the portion of the medical instrument to the support means such that the portion of the medical instrument has at least two degrees of freedom of motion with respect to the support.

17. A virtual reality apparatus as described in claim 16 wherein the two degrees of freedom of motion are provided about a first axis and a second axis, the first and second axes being substantially mutually perpendicular and intersecting at an origin point, the means for detecting position information including first and second sensor means, the first sensor means for detecting rotary motion of the portion of the medical instrument about the first axis and the second sensor means for detecting rotary motion of the portion of the medical instrument about the second axis.

18. A virtual reality apparatus as described in claim 16 wherein the actuator means is coupled between the support means and the gimbal means, the human/computer interface means further including a continuous belt loop that is turned when the portion of the medical instrument is rotated by the user, the continuous loop belt arranged such that a substantial force may be applied to said continuous loop belt without causing substantial slippage of the continuous loop belt.

19. A virtual reality apparatus as described in claim 15 wherein the virtual reality object represents a medical instrument and the virtual reality medical simulation is a representation of a body.

20. A virtual reality apparatus as described in claim 15 wherein the force command corresponds to the virtual reality object encountering an obstacle within the virtual reality medical simulation.