Force feedback cursor control interface

• US 6,366,273 B1
• Filed: 02/23/2000
• Issued: 04/02/2002
• Est. Priority Date: 07/16/1993
• Status: Expired due to Term

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Jackson, Bernard G., Rosenberg, Louis B.
• Primary Examiner(s): Luu, Matthew
• Application Number:
  US09/511,413
• Time in Patent Office: 769 Days
• Field of Search: 345/156, 345/157, 345/158, 345/162
• US Class Current: 345/156
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: B25J 9/1692 Calibration of manipulator
• CPC Class Codes: G01B 21/04 by measuring coordinates of points
• CPC Class Codes: G01B 5/008 using coordinate measuring machines
• CPC Class Codes: G01B 7/004 for measuring coordinates of points
• CPC Class Codes: G05B 19/4207 in which a model is traced or scanned and corresponding data recorded
• CPC Class Codes: G05G 5/03 Means for enhancing the operator's awareness of arrival of the controlling member at a command or datum position; Providing feel, e.g. means for creating a counterforce arrangements for indicating the position of the controlling member G05G1/015
• CPC Class Codes: G05G 9/04737 with six degrees of freedom
• CPC Class Codes: G06F 2203/015 Force feedback applied to a joystick
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0383 Signal control means within the pointing device
• CPC Class Codes: H01H 2003/008 with a haptic or a tactile feedback controlled by electrical means, e.g. a motor or magnetofriction

Abstract

A force feedback cursor control interface for use with a computer that provides locative data to a computer for tracking a user manipulatable physical object and provides feedback to the user through output forces. The physical object is movable in multiple degrees of freedom and is tracked by sensors for sensing the location and/or orientation of the object. A multi-processor system architecture is disclosed wherein the host computer is interfaced with a device microprocessor which is responsive to the output of the sensors and provides the host computer with information derived from the sensors. The host computer runs an application program and provides images on a display, where the program responds to the provided sensor information and force feedback is correlated with the displayed images via force feedback commands from the host computer.

Claims

1. A human interface device for enabling manual interactions with application software running on a host computer, said software providing images displayed on a display apparatus, said device comprising:

(a) a user manipulatable physical object physically contacted and movable by a user, said user manipulating said user manipulatable physical object in a plurality of rotational degrees of freedom;

(b) a sensor apparatus coupled to said user manipulatable physical object, said sensor apparatus producing a locative signal which is responsive to and corresponding with a position of said user manipulatable physical object at points in time during normal operation;

(c) a communication bus coupled to said host computer;

(d) a device controller separate from said host computer and coupled to said host computer by said communication bus, said device controller being coupled to said sensor apparatus, said device controller running a program contained at least in part in a non-volatile memory coupled to said device controller and separate from said host computer, said device controller providing information for use by said host computer an application program simultaneously with said device controller running said program, said information including a representation of said locative signal,

wherein said application program of said host computer can provide images on a computer display, said images updated on said computer display in response to said locative signal, and

wherein said host computer can provide host commands, said host commands being communicated to said device controller by said communication bus, wherein said device controller;

(i) monitors said communication bus for said host commands; and

(ii) decodes said host commands, wherein

at least one of said host commands causes information to be reported from said device controller to said host computer, and

at least one of said host commands causes said device controller to output control signals to cause a force to be output to said user, said at least one host command and said force being correlated with at least one of said images developed by said host computer on said computer display; and

(e) a force generator controlled by said device controller for providing a force to to user in response to at least one of said control signals.

2. A human interface device as recited in claim 1 further comprising a support mechanism which supports said user manipulatable physical object while allowing a plurality of degrees of freedom in the motion of said user manipulatable physical object.

3. A human interface device as recited in claim 2 wherein said support includes a linkage coupled between said user manipulatable physical object and a fixed surface and allowing at least five degrees of freedom in motion of said physical object.

4. A human interface device as recited in claim 1 wherein said user manipulatable physical object has a pencil-like stylus configuration that can be manually manipulated by a user of said device.

5. A human interface device as recited in claim 3 wherein three of said degrees of freedom closest to said user manipulatable object allow an orientation of said object to be adjusted in three dimensional space while three degrees of freedom closest to said fixed surface allow a location of said object to be adjusted in three dimensional space.

6. A human interface device as recited in claim 1 further comprising a switch coupled to said user manipulatable object, said switch capable of being in multiple states in response to user interaction, wherein a state of said switch being transmitted to said host computer and wherein an action is taken by said host computer in response to s aid state of said switch.

7. A human interface device as recited in claim 6 wherein at least one of said host commands causes said device controller to report a state of said switch to said host computer.

8. A human interface device as recited in claim 6 wherein said switch comprises a foot pedal unit.

9. A human interface device as recited in claim 1 further comprising a plurality of command routines stored in a local memory coupled to said device controller, at least one of said command routines allowing said controller to control said force generator in accordance with at least one of said decoded host commands, and at least one of said command routines reporting a representation of said locative signal to said host computer in accordance with at least one of said decoded host commands.

10. A human interface device as recited in claim 1 wherein said device controller computes the position or orientation of said user manipulatable object from said locative signal produced by said sensor apparatus, said controller reporting said position or orientation to said host computer.

11. A human interface device as recited in claim 2 wherein said device has access to information describing physical dimensions of said support mechanism and information of device kinematics such that said locative signal produced by said sensor apparatus can be converted into coordinate data describing a position of said physical object.

12. A human interface device as recited in claim 1 wherein said device controller executes a routine stored in a non-volatile memory accessible by said device controller and based on said host command, wherein said routine processes said locative signal into angle data, and wherein said angle data is sent to said host computer.

13. A human interface device as recited in claim 1 wherein at least one of said host commands sets parameters on said device controller.

14. A human interface device as recited in claim 2 wherein said force generator transmits a force via said support mechanism in response to said force signals, and wherein said force signals are correlated to information displayed on said computer display apparatus, said information including a cursor interacting with a graphical surface.

15. A human interface device as recited in claim 1 wherein said host command from said host computer causes said device controller to send control signals to a communications sub-unit of the device controller to provide data to said host computer.

16. A human interface device as recited in claim 2 wherein said host commands are decoded by said controller, at least one of said host commands calling a subroutine stored on said controller to change a resistance at a joint of said support mechanism.

17. A human interface device as recited in claim 1 wherein forces commanded by said host computer to said device and felt by said user correspond to images displayed on said computer display.

18. A human interface device as recited in claim 17 wherein said images include a cursor interacting with another object displayed on said display screen, wherein said cursor interacts with a surface image displayed on said display screen.

19. A human interface device as recited in claim 1 wherein said parameters are communication parameters for communicating with said host computer.

20. A multi-degree-of-freedom force feedback cursor control interface for use with a host computer running an application program which displays images on a display apparatus, said cursor control interface enabling a user to control the position of a cursor displayed by said host computer and enabling said host computer to convey force feedback to said user in accordance with said position of said cursor, said force feedback cursor control comprising:

a user manipulatable physical object physically contacted and manipulated by said user, where said user manipulatable physical can be manipulated by said user in a plurality of rotational degrees of freedom;

a sensor coupled to said user manipulatable physical object to provide a locative signal responsive to and corresponding with a user'"'"'s manipulation of said user manipulatable object;

a force generator coupled to said user manipulatable physical object providing a force to said user based on host commands received from said host computer running an application program which displays images on an associated display apparatus;

a controller separate from said host computer, coupled to said sensor, and interacting with said force generator, said controller supervising communication between said cursor control interface and said host computer by monitoring and decoding said host commands received from said host computer and by sending data to said host computer, where at least one of said host commands sent by said host computer to said controller is a force feedback command causing the output of force to said user, and wherein at least part of said data sent to said host computer is representative of the motion of said user manipulatable object, wherein said output of force occurs simultaneously with said display of images on said display apparatus by said host computer, thereby allowing force to be output from said force generator in response to and in conjunction with cursor position on said display apparatus;

memory locally coupled to said controller and separate from memory of said host computer for locally storing a representation of said locative signal; and

a communication bus for coupling said controller to said host computer, said bus being adapted to convey said force feedback commands from said host computer to said controller and adapted to convey said data from said controller to said host computer;

whereby said display apparatus of said host computer can display a cursor positioned in accordance with a position of said user manipulatable object such that force feedback commands are sent from said host computer to said cursor control interface in accordance with displayed cursor interaction with other displayed images on said display apparatus.

21. A force feedback cursor control interface as recited in claim 20 wherein said output of said force includes providing resistance to motion of said user manipulatable physical object in at least one of said degrees of freedom.

22. A force feedback cursor control interface as recited in claim 21 wherein said force generator provides force upon said user manipulatable physical object by applying resistance to a rotary joint.

23. A force feedback cursor control interface as recited in claim 20 wherein a plurality of command routines are stored in said local memory, at least one of said command routines allowing said controller to control said force generator in accordance with at least one of said decoded host commands, and at least one of said command routines reporting data to said host computer in accordance with at least one of said decoded host commands.

24. A force feedback cursor control interface as recited in claim 20 further comprising a peripheral switch coupled to said user manipulatable physical object for providing input signals to said controller when a user manipulates said peripheral switch, wherein said controller reports a state of said switch to said host computer, said state causing said host computer to modify a displayed process.

25. A force feedback cursor control interface as recited in claim 20 wherein said user manipulatable physical object includes a stylus that can be moved by said user in at least five degrees of freedom.

26. A force feedback cursor control interface as recited in claim 20 wherein said host command causes said controller execute a routine to set communication parameters for communication between said controller and said host computer.

27. A force feedback cursor control interface as recited in claim 20 wherein said host command is sent from said host computer and is based on said cursor position.

28. A multi-degree-of-freedom cursor control interface for use in conjunction with a host computer running an application program which displays images on a display apparatus, said cursor control interface enabling a user to control the position of a graphical image displayed by said host computer on said display apparatus and enabling said host computer to convey force feedback to a user in accordance with graphical image manipulation, said cursor control interface comprising:

a user manipulatable physical object grasped and manipulated by a user, said user manipulatable physical object being manipulatable in a plurality of degrees of freedom;

a force generator coupled to said user manipulatable physical object and providing force resistance to motion of said user manipulatable physical object in a plurality of said degrees of freedom in response to commands from said host computer and in coordination with said displayed graphical image;

a device controller separate from said host computer and interacting with said force generator, said device controller supervising communication between said cursor control interface and said host computer by monitoring and decoding said commands received from said host computer and by sending data to said host computer, wherein said host computer sends a command to said device controller that sets communication parameters on said controller for communicating with said host computer, wherein at least one of said commands sent from said host computer to said device controller is a force feedback command causing the output of force to said user by calling a routine to output said force, and wherein at least part of said data sent to said host computer is representative of motion of said user manipulatable physical object, said decoding, sending, and output of force occurring simultaneously with host computer control of said graphical image, thereby allowing force to be output from said force generator in response to and in conjunction with graphical image manipulation on said display apparatus;

a communication bus connecting said device controller and said host computer, said bus used in conveying said force feedback commands from said host computer to said device controller;

a sensor apparatus coupled to said device controller, said sensor apparatus providing said device controller with a locative signal responsive to and corresponding with manipulation of said user manipulatable object; and

local memory coupled to said device controller and separate from host computer memory, said local memory locally storing a representation of said locative signal and storing said routine causing said output of said force;

whereby said display apparatus coupled to said host computer displays a graphical image controlled by said host computer in accordance with said motion of said user manipulatable object, and wherein force feedback commands are sent from said host computer to said cursor control interface in accordance with said displayed graphical image.

29. A cursor control interface as recited in claim 28 further comprising a support mechanism providing said plurality of degrees of freedom to said physical object, wherein said device controller has access to information describing physical dimensions of said support mechanism and information of device kinematics such that said locative signal produced by said sensor apparatus can be converted into coordinate data describing a position of said user manipulatable physical object.

30. A cursor control interface as recited in claim 28 wherein at least one of said plurality of degrees of freedom of said user manipulatable physical object is a rotational degree of freedom.

31. A cursor control interface as recited in claim 28 wherein said device controller stores sensor data representative of said locative signal in said local memory including multiple locative values while said device controller waits to send at least a representation of said data to said host computer, wherein said multiple locative values include information such that said host computer can determine which of said host commands said multiple locative values are in response to.

32. A cursor control interface as recited in claim 28 wherein said graphical image on said display apparatus includes a cursor.

33. A cursor control interface as recited in claim 32 wherein forces commanded by said host computer to said device controller and felt by said user correspond to said cursor interacting wit another object displayed on said display screen.

34. A cursor control interface as recited in claim 33 wherein said cursor interacts with a surface image displayed on said display screen.