Safe and low cost computer peripherals with force feedback for consumer applications

• US 5,691,898 A
• Filed: 03/28/1996
• Issued: 11/25/1997
• Est. Priority Date: 09/27/1995
• Status: Expired due to Term

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Human Interface Corporation
• Inventors: Schena, Bruce M., Rosenberg, Louis B., Braun, Adam C.
• Primary Examiner(s): RUGGIERO, JOSEPH F
• Application Number:
  US08/623,660
• Time in Patent Office: 607 Days
• Field of Search: 364/167.01, 364/188, 364/189, 364/190, 364/578, 345/161, 345/179, 345/156, 395/95, 395/326, 395/961, 395/965, 463/30, 463/36-43, 414/5, 318/628
• US Class Current: 700/85
• CPC Class Codes: A63F 13/21 characterised by their sensors, purposes or types
• CPC Class Codes: A63F 13/23 for interfacing with the game device, e.g. specific interfaces between game controller and console
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: A63F 13/57 Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game A63F13/56 takes precedence
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
• CPC Class Codes: A63F 2300/1025 details of the interface with the game device, e.g. USB version detection
• CPC Class Codes: A63F 2300/1031 using a wireless connection, e.g. Bluetooth, infrared connections
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: A63F 2300/64 for computing dynamical parameters of game objects, e.g. motion determination or computation of frictional forces for a virtual car
• CPC Class Codes: A63F 2300/8017 Driving on land or water; Flying
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G05G 2009/04766 providing feel, e.g. indexing means, means to create counterforce
• CPC Class Codes: G05G 2009/04777 with additional push or pull action on the handle
• CPC Class Codes: G05G 9/047 the controlling member being movable by hand about orthogonal axes, e.g. joysticks for switches H01H25/04
• CPC Class Codes: G06F 1/266 Arrangements to supply power to external peripherals either directly from the computer or under computer control, e.g. supply of power through the communication port, computer controlled power-strips
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 2203/015 Force feedback applied to a joystick
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0383 : Signal control means within the pointing device
• CPC Class Codes: H01H 2003/008 : with a haptic or a tactile feedback controlled by electrical means, e.g. a motor or magnetofriction
• CPC Class Codes: H04L 67/131 : Protocols for games, networked simulations or virtual reality

Abstract

A method and apparatus for providing safe and low-cost force feedback peripherals for consumer applications. A device microprocessor local to an interface device is coupled to the host by a first interface bus. The microprocessor receives host commands from the host computer on the first interface bus, such as an RS-232 interface, and commands an actuator to apply a force to a user object, such as a joystick, in response to host commands. A sensor detects positions of the user object and outputs signals to the host on a second interface bus, such as a PC game port bus, separate from the first bus. In a "recoil" embodiment, a user initiates force feedback by pressing a button on the joystick, which sends an activation signal to the actuator. In other recoil embodiments, the host computer can transmit one or more enable signals and/or activation signals to the actuator to enable or command forces. A safety switch of the present invention disables the actuator when the interface device is not in use and enables the actuator when an amount of weight over a predetermined amount is placed on the joystick created by a user grasping the joystick. A circuit of the present invention includes a capacitor for storing power provided by an input signal and supplied to the actuator when forces are to be output.

Claims

1. A force feedback interface apparatus for use with a host computer displaying a graphical environment, said interface apparatus providing locative signals to said host computer which are used by said host computer to update said graphical environment in response to user manipulation of said interface apparatus, said interface apparatus providing force sensations to the user and button reflex capability wherein a force sensation is generated locally in response to a press of a button of said interface apparatus, said interface apparatus comprising:

a user object being physically contacted and manipulated by a user and having at least one degree of freedom of movement with respect to an origin;

an actuator coupled to said user object for applying a force along said degree of freedom to said user object,

a sensor for detecting motion of said user object along said degree of freedom and outputting a sensor signal, wherein a representation of said sensor signal being received by said host computer via an interface bus coupling said interface apparatus to said host computer wherein said host computer updates said graphical environment in response to said sensor signal; and

at least one button coupled to said interface apparatus and manipulable by said user, said button providing a state signal to said host computer via an interface bus, said state signal conveying a state of said button as selected by said user, thereby allowing said host computer to execute a process in response to said state of said button;

control electronics coupled to said actuator and to said at least one button, said control electronics locally varying said force over time, thereby generating a force sensation felt by said user in said degree of freedom, said control electronics having button-reflex capability allowing local generation of a force sensation to be initiated in response to a locally detected change in said state of said button and generated locally over a period of time after being so initiated.

2. An interface apparatus as recited in claim 1 wherein said force commands received by said actuator are host data sent by said host computer and which instruct said actuator to output said force in accordance with a software process running on said host computer.

3. An interface apparatus as recited in claim 2 wherein said interface bus transmits said host data serially to said interface apparatus from said host computer, and further comprising at least one shift register for providing said serial host data as control signals to said actuator.

4. An interface apparatus as recited in claim 3 wherein said interface bus transmits at least a portion of said host data in parallel in conjunction with serially transmitted host data.

5. An interface apparatus as recited in claim 3 wherein said interface bus is an interface bus in which a plurality of lines are used to transmit data to said interface apparatus as parallel information and in which at least one line is used to transmit data to said interface apparatus as serial information.

6. An interface apparatus as recited in claim 3 wherein said interface bus is an interface bus in which all lines included in said interface bus are used to transmit data to said interface apparatus as parallel information.

7. An interface apparatus as recited in claim 1 further comprising a plurality of said buttons, wherein said button-reflex capability of said control electronics detects a change in state of each of said buttons to initiate a force sensation associated with that button.

8. An interface apparatus as recited in claim 7 wherein a signal received by said interface apparatus from said host computer over an interface bus indicates if a particular force sensation is to be associated with a change in state of a particular button.

9. An interface apparatus as recited in claim 1 wherein said initiation of said force sensation in response to said locally detected change in said state of said button occurs only if an enable signal is received by said interface apparatus from said host computer prior to said change in state of said user adjustable button.

10. An interface apparatus as recited in claim 9 wherein said enable signal is used to coordinate events in said host computer graphical environment with said locally generated force sensations.

11. An interface apparatus as recited in claim 10 wherein said enable signal is a high-level host command indicating which of a plurality of said user adjustable buttons should initiate a given force sensation.

12. An interface apparatus as recited in claim 7 wherein the initiation of a particular force sensation in response to a locally detected change in said state of a particular button occurs only if previously enabled by a separate enable signal associated with said particular button, said enable signal received by said interface apparatus from said host computer.

13. An interface apparatus as recited in claim 1 wherein said interface bus connecting said interface apparatus and said host computer includes a PC Game Port.

14. An interface apparatus as recited in claim 1 wherein all signals passing between said interface apparatus and said host computer pass through a PC Game Port.

15. An interface apparatus as recited in claim 13 wherein said interface bus includes a serial port in parallel with said PC Game Port.

16. An interface apparatus as recited in claim 1 wherein said force sensation is a recoil force.

17. An interface apparatus as recited in claim 1 wherein said force sensation is a vibration force.

18. An interface apparatus as recited in claim 1 wherein said control electronics include a local microprocessor wherein a process running on said local microprocessor supports said button-reflex capability by initiating a force sensation in response to a change in said state of said button as locally detected by said local microprocessor.

19. An interface apparatus as detected in claim 1 further including a dead-man safety switch that prevents actuators from producing any forces upon said user object when said user is not properly engaging said user object.

20. An interface apparatus as recited in claim 1 wherein said period of time is derived from a signal sent to said interface apparatus from said host computer over an interface bus.

21. An interface apparatus as recited in claim 1 wherein said local generation of said force sensation is re-initiated when said button is held by said user in a particular state for more than a defined period of time after the previous generation of said force sensation.

22. An interface apparatus as recited in claim 1 wherein power is dram over time from a power source and stored in a power storage circuit local to said interface device, said stored power able to be discharged rapidly across said actuator to allow said actuator to produce a transient force sensation of greater magnitude than if powered directly by said power source.

23. An interface apparatus as recited in claim 22 wherein said power source is a universal serial bus connection between said host computer and said interface apparatus.

24. An interface apparatus as recited in claim 19 wherein said dead-man safety switch is a hand weight activated switch.

25. An interface apparatus as recited in claim 19 wherein said dead-man safety switch is an optical switch, said optical safety switch being closed when said user covers an optical sensor when grasping said user object.

26. An interface apparatus as recited in claim 19 wherein said safety switch includes two touch plates coupled to said force feedback interface apparatus, wherein said safety switch is closed when said user creates an electrical connection between said two touch plates by touching both of said touch plates.

27. An interface apparatus as recited in claim 1 wherein said host computer is a video game console and said interface bus includes parallel control port lines.

28. An interface apparatus as recited in claim 22 wherein said power storage circuit includes a storage capacitor.

29. A force feedback interface device for use with a host computer displaying a graphical environment, said interface device providing sensor signals to said host computer which are used by said host computer to update said graphical environment in response to user manipulation of said interface device, said interface device providing force sensations to the user and user-selected reflex capability wherein a force sensation is generated locally in response to a selection of a user-selectable control of said interface device, the interface device comprising:

a sensor for determining the position of a user object in a degree of freedom and for providing sensor signals representing said position to said host computer via an interface bus, wherein said host computer updates said graphical environment in response to said sensor signals;

an actuator for applying forces on said user object in said degree of freedom when said actuator receives an activation signal; and

a user-selectable control coupled to said interface device, wherein when said user selects said user-selectable control, a reflex process local to said interface device is initiated in which said selection of said user-selectable control is locally detected and in which said activation signal is sent to said actuator and commands said actuator to apply said forces on said user object in said degree of freedom by varying a magnitude of said forces over time, thereby initiating force feedback on said interface device independently of said host computer.

30. A force feedback interface device as recited in claim 29 wherein said user object is a joystick wherein said actuator applies said force on said joystick only when said user selects said user-selectable control.

31. A force feedback interface device as recited in claim 29 wherein said user selectable control includes a button.

32. A force feedback interface device as recited in claim 31 wherein said host computer is operative to transmit an enable signal to said interface apparatus for enabling said reflex process, wherein when said enable signal is provided by said host computer, said selection of said user-selectable control causes said actuator to output said forces, and wherein when said enable signal is not provided by said host computer, said selection of said user selectable control does not cause said actuator to output said forces.

33. A force feedback interface device as recited in claim 32 wherein said activation signal is a first activation signal, and wherein said host computer is operative to provide a second activation signal to said actuator to command said actuator to apply said force to said user object regardless of whether said first activation signal from said user selectable control has been provided to said actuator.

34. A force feedback interface device as recited in claim 33 wherein said second activation signal is provided at a desired baud rate to said actuator such that said second activation signal activates and deactivates said actuator at said desired baud rate.

35. A force feedback interface device as recited in claim 33 further comprising a user enable switch on said interface device which, when selected, enables said reflex process to output force in response to said user-selectable control and, when unselected, disables said reflex process from applying said force in response to said user-selectable control.

36. A force feedback interface device as recited in claim 31 further comprising a safety switch for said user, said user able to change a state of said switch during operation of said interface device, wherein said actuator is operative to output said force only when said user continually maintains said safety switch in a particular state during operation of said interface device.

37. A force feedback interface device as recited in claim 36 wherein said safety switch is a hand-weight safety switch coupled to said user object, said hand weight safety switch being maintained in said particular state by said user when said user applies weight to said user object by grasping said user object.

38. A force feedback interface device as recited in claim 36 wherein said safety switch includes two touch plates coupled to said force feedback interface device, wherein said safety switch is maintained in said state when said user creates an electrical connection between said two touch plates by touching both of said touch plates.

39. A force feedback interface device as recited in claim 37 wherein when said button is continually depressed by said user, a timer/pulse generator of said interface apparatus causes said actuator to repeatedly output forces to said user object.

40. A method for interfacing a force feedback interface device manipulated by a user with a host computer system, the method comprising the steps of:

providing a user manipulable joystick included in said force feedback interface device, said joystick having a degree of freedom;

sensing positions of said joystick along said degree of freedom with a sensor and producing electrical sensor signals therefrom, said sensor signals being sent to said host computer system over an interface bus to said host computer system, said host computer system updating application software implemented by said host computer system in accordance with said sensor signals;

utilizing a microprocessor local to said joystick and separate from said host computer system to communicate with said host computer system to receive host commands from said host computer system;

providing at least one input switch coupled to said microprocessor;

enabling said input switch by at least one of said host commands to initiate a specified force sensation on said user object when said input switch is activated by a user as locally detected by said microprocessor, said force sensation being specified by at least one of said host commands from said host computer system; and

outputting processor force commands from said microprocessor to an actuator when said input switch is activated by said user, said actuator being coupled to said joystick and being operative to apply said specified force sensation to said joystick in accordance with said processor force commands.

41. A method as recited in claim 40 wherein said sensor is an analog potentiometer.

42. A method as recited in claim 40 further comprising a step of further comprising providing multiple input switches, wherein each of said multiple switches is enabled with a different command from said host computer system, and wherein appropriate processor force commands to output a different force sensation associated with said switch on said user object are output when said user activates each of said switches.

43. An interface apparatus for interfacing motion of a user with a host computer and enabling said host computer to track motion of said user and to command feedback forces to said user, said interface apparatus comprising:

a user object being physically contacted and manipulable by a user and having at least one degree of freedom of movement;

a device microprocessor provided local to said interface apparatus and separate from said host computer and being controlled by software instructions stored in a memory device separate from said host computer, said microprocessor being coupled to said host computer by a serial interface bus and receiving host commands from said host computer on said first interface bus;

an actuator coupled to said user object for applying a force along said degree of freedom to said user object, wherein said actuator applies said force in response to microprocessor force commands produced by said device microprocessor;

a thermal sensor for detecting a temperature of said actuator, said thermal sensor providing a temperature signal to said microprocessor such that said microprocessor can deactivate said actuator when said temperature of said actuator rises above a predetermined threshold; and

a sensor coupled to said user object for detecting a position of said user object along said degree of freedom and for outputting a locative sensor signal which represents said position of said user manipulable object, said locative sensor signal being received by said host computer, and wherein said host computer modifies a displayed process in response to said locative sensor signal.

44. An interface apparatus for interfacing motion of a joystick with a host computer and enabling said host computer to track motion of said user manipulable object and to command feedback forces to said user manipulable object, said interface apparatus comprising:

a joystick being physically contacted and manipulable by a user and having at least one degree of freedom of movement with respect to a ground surface, wherein said joystick includes a safety break-away handle that breaks off from said interface apparatus when a force over a predetermined threshold is applied to said joystick;

a device microprocessor provided local to said interface apparatus and separate from said host computer and being controlled by software instructions stored in a memory device separate from said host computer, said microprocessor being coupled to said host computer by a serial interface bus and receiving host commands from said host computer on said first interface bus;

an actuator coupled to said user object for applying a force along said degree of freedom to said user object, wherein said actuator applies said force in response to microprocessor force commands produced by said device microprocessor; and

a sensor coupled to said user object for detecting a position of said user object along said degree of freedom and for outputting a locative sensor signal which represents said position of said user manipulable object, said locative sensor signal being received by said host computer, and wherein said host computer modifies a displayed process in response to said locative sensor signal.

45. An interface apparatus as recited in claim 44 wherein said safety break-away handle is attached to said interface apparatus by a magnet.

46. An interface apparatus for interfacing motion of a user with a host computer and enabling said host computer to track motion of said user, to update an execute software process in response to said motion, and to command feedback forces to said user, said interface apparatus providing a button-reflex capability wherein a force sensation is generated locally in response to a press of a button on said interface apparatus, said interface apparatus comprising:

a user object being physically contacted and manipulable by a user and having at least one degree of freedom of movement;

a device microprocessor provided local to said interface apparatus and separate from said host computer and being controlled by software instructions stored in a local device separate from said host computer, said microprocessor being coupled to said host computer by an interface bus and receiving host commands from said host computer on said interface bus, wherein said host commands include a supervisory command which affects implementation of said software instructions by said device microprocessor;

at least one button-actuated switch coupled to said device microprocessor, wherein said supervisory command enables said switch to initiate a force sensation specified by a parameter in said supervisory command on said user object when said button-actuated switch is manipulated to a particular state by said user;

a sensor coupled to said user object for detecting a position of said user object along said degree of freedom and for outputting a locative sensor signal which represents said position of said user manipulable object, said locative sensor signal being received by said host computer on said interface bus, and wherein said host computer modifies a displayed process in response to said locative sensor signal; and

an actuator coupled to said user object for applying a force along said degree of freedom to said user object, wherein said actuator applies said force in response to microprocessor force signals produced by said device microprocessor.

47. An interface apparatus as recited in claim 46 wherein said interface bus is a serial interface bus for bi-directionally transmitting and receiving data serially.

48. An interface apparatus as recited in claim 47 wherein said interface bus is a USB bus.

49. An interface apparatus as recited in claim 48 wherein said bus is a game port interface bus coupled to a personal computer game port of said host computer.

50. An interface apparatus as recited in claim 49 wherein said user object includes a joystick.

51. An interface apparatus as recited in claim 46 wherein said state of said button actuated switch is locally detected by said device microprocessor.

52. An interface apparatus as recited in claim 46 further comprising multiple button-actuated switches, each of said multiple switches initiating a different force sensation associated with said switch on said user object when said user closes said switch.

53. An interface apparatus as recited in claim 46 wherein said microprocessor is powered by said host computer using a power line of said interface bus.

54. An interface apparatus as recited in claim 46 further comprising a power storage device coupled to said actuator, said power storage device storing power from a power source and providing said stored power to said actuator when said actuator applies said force to said user object.

55. An interface apparatus as recited in claim 50 wherein said joystick includes a safety break-away handle that breaks off from said interface apparatus when a force having a magnitude over a predetermined threshold magnitude is applied to said joystick.

56. An interface apparatus as recited in claim 55 wherein said safety break-away handle is attached to said interface apparatus by a magnet.

57. An interface apparatus as recited in claim 46 wherein said local memory device stores force routines, wherein said microprocessor selects a force routine in accordance with said at least one of said host commands, said force routine determining said microprocessor force signals, at least in part, from said sensor signals of said microprocessor sensors.

58. An interface apparatus as recited in claim 57 wherein said selected force routine retrieves a plurality of digital force values stored in said local memory device and outputs said stored force values to said actuator.

59. A force feedback interface device for use with a host computer running a software application, said interface device providing a button-reflex capability wherein a force sensation is generated locally in response to a press of a button on said interface apparatus by a user, said interface device comprising:

a user object for manipulation by a user in at least one degree of freedom with respect to an origin;

a sensor for detecting motion of said user object in said degree of freedom and for producing a locative sensor signal, wherein a representation of said locative sensor signal is conveyed to said host computer, thereby allowing said software application to update a process in response to manipulation of said user object;

a user adjustable button local to said interface apparatus and electrically coupled to said local microprocessor, a state of said button being conveyed to said host computer to allow said software application to update a process in response to said state of said button;

an actuator physically coupled to said user object, said actuator applying forces to said user object in said degree of freedom; and

a local microprocessor separate from said host computer and electronically coupled to said actuator, said local microprocessor executing a local process, said local microprocessor coupled to said host computer by a communication bus and outputting signals to said actuator such that said actuator applies said forces in response to said signals produced by said local microprocessor, wherein said local process locally generates force sensations by modulating a magnitude of said forces produced by said actuator over a period of time, said local process having button-reflex capability wherein said local generation of a force sensation may be initiated in response to a locally detected change in said state of said user adjustable button.

60. An interface device as recited in claim 59 wherein said user adjustable button is one of a plurality of user adjustable buttons included in said interface device, and wherein button-reflex data is stored local to said interface device, said data indicating which of said plurality of user adjustable buttons have been associated with particular ones of a plurality of force sensations.

61. An interface device as recited in claim 60 wherein said button-reflex data is derived from parameters received from said host computer over said communication bus.

62. An interface device as recited in claim 60 wherein said button-reflex data includes a repeat rate indicating a length of a period of time after a force sensation has been output that said force sensation is to be re-executed if the initiating button is held in a given state.

63. An interface device as recited in claim 59 wherein said force sensation initiated in response to a locally detected change in state of a user adjustable button is a jolt sensation.

64. An interface device as recited in claim 59 wherein said force sensation initiated in response to a locally detected change in state of a user adjustable button is a damping sensation.

65. An interface device as recited in claim 59 wherein said force sensation initiated in response to a locally detected change in state of a user adjustable button is a force profile sensation wherein forces applied by said actuator are derived from a list of values in memory local to said interface apparatus.

66. An interface device as recited in claim 65 wherein said list of values is downloaded to said local microprocessor over said communication bus.

67. An interface device as recited in claim 59 wherein said button-reflex capability associated with a particular button on said interface device is enabled in response to a high level host command, wherein said host command includes a parameter representing which button is to trigger a force sensation, said force sensation being specified by a host command.

68. An interface device as recited in claim 59 wherein said interface bus connect said communication bus includes a PC Game Port.

69. An interface device as recited in claim 68 wherein said interface bus also includes a serial port in parallel with said PC Game Port.

70. An interface device as recited in claim 59 further including a dead-man safety switch that prevents actuators from producing any forces upon said user object when said user is not properly engaging said user object.

71. An interface device as recited in claim 59 wherein said period of time is derived from a host command sent to said interface device from said host computer over said communication bus.

72. An interface device as recited in claim 59 wherein said local generation of said force sensation is re-initiated when said button is held by said user in a particular state for more than a defined period of time after a previous generation of said force sensation.

73. An interface device as recited in claim 59 wherein power is drawn over time from a power source and stored in a power storage circuit local to said interface device, said stored power able to be discharged rapidly across said actuator, thereby allowing said actuator to produce a transient force sensation of greater magnitude than if powered directly by said power source.

74. An interface device as recited in claim 73 wherein said power source is included in said communication bus.

75. An interface device as recited in claim 74 wherein said power source is a universal serial bus connection between said host computer and said interface device.

76. An interface device as recited in claim 70 wherein said dead-man safety switch is a hand weight activated switch.

77. An interface device as recited in claim 70 wherein said dead-man safety switch is an optical switch, said optical safety switch being closed when said user covers an optical sensor when grasping said user object.

78. An interface device as recited in claim 70 wherein said safety switch includes a plurality of touch plates coupled to said interface device, wherein said safety switch is closed when said user creates an electrical connection between said plurality of touch plates by touching all of said touch plates.

79. An interface device as recited in claim 59 wherein said actuator is a motor.

80. An interface device as recited in claim 79 further including a temperature sensor electrically coupled to said local microprocessor for measuring heat produced by said motor.

81. An interface device as recited in claim 80 wherein said local process can deactivate said motor in response to said temperature sensor reporting a value above a threshold.

82. An interface device as recited in claim 79 wherein said local process performs a software estimation of motor heating and deactivates said motor when said estimation exceeds a threshold.

83. An interface device as recited in claim 59 wherein said host computer is a video game console and said communication bus includes game console control port lines.