Hybrid control of haptic feedback for host computer and interface device

• US 6,411,276 B1
• Filed: 10/13/2000
• Issued: 06/25/2002
• Est. Priority Date: 11/13/1996
• Status: Expired due to Term

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Martin, Kenneth M., Rosenberg, Louis B., Braun, Adam C.
• Primary Examiner(s): WU, XIAO MIN
• Application Number:
  US09/687,744
• Time in Patent Office: 620 Days
• Field of Search: 345/700, 345/701, 345/702, 345/156, 345/157, 345/158, 345/159, 345/160, 345/161, 345/162, 345/168, 345/173
• US Class Current: 345/156
• CPC Class Codes: G05G 2009/04766 providing feel, e.g. indexing means, means to create counterforce
• CPC Class Codes: G05G 9/047 the controlling member being movable by hand about orthogonal axes, e.g. joysticks for switches H01H25/04
• CPC Class Codes: G06F 2203/015 Force feedback applied to a joystick
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G09B 9/28 Simulation of stick forces or the like

Abstract

A hybrid haptic feedback system in which a host computer and haptic feedback device share processing loads to various degrees in the output of haptic sensations, and features for efficient output of haptic sensations in such a system. A haptic feedback interface device in communication with a host computer includes a device microcontroller outputting force values to the actuator to control output forces. In various embodiments, the microcontroller can determine force values for one type of force effect while receiving force values computed by the host computer for a different type of force effect. For example, the microcontroller can determine closed loop effect values and receive computed open loop effect values from the host; or the microcontroller can determine high frequency open loop effect values and receive low frequency open loop effect values from the host. Various features allow the host to efficiently stream computed force values to the device.

Claims

1. A haptic feedback interface device in communication with a host computer, said host computer running and displaying a graphical environment, said haptic feedback interface device comprising:

a user manipulatable object physically contacted by a user and moveable in at least one degree of freedom;

at least one actuator operative to output forces, said forces felt by said user;

at least one sensor operative to detect motion of said user manipulatable object in said at least one degree of freedom and outputting a sensor signal indicative of said motion; and

a device processor coupled to said at least one actuator and to said at least one sensor and local to said haptic feedback interface device, said processor operative to output force values to said at least one actuator to control said forces and receive said sensor signal from said at least one sensor, wherein said processor is operative to determine a closed loop force value based at least in part on said sensor signal and output said closed loop force value to said at least one actuator, and wherein said processor does not compute open loop force values but instead is operative to receive open loop force values from said host computer and direct said open loop force values to said at least one actuator, wherein said forces output by said at least one actuator are based on said closed loop force values and said open loop force values.

2. A haptic feedback interface device as recited in claim 1 wherein said open loop force values are computed on said host computer.

3. A haptic feedback interface device as recited in claim 1 wherein said open loop force values include periodic forces.

4. A haptic feedback interface device as recited in claim 1 wherein said open loop force values are received from said host computer over a streaming serial communication channel provided between said haptic feedback interface device and said host computer.

5. A haptic feedback interface device as recited in claim 1 wherein said closed loop forces include spring forces, damping forces, and texture forces.

6. A haptic feedback interface device as recited in claim 4 wherein said streaming channel is an isochronous channel.

7. A haptic feedback interface device as recited in claim 4 wherein said processor extrapolates a force value if a force value received from said host computer is corrupted or missing.

8. A haptic feedback interface device as recited in claim 4 wherein said processor outputs a previously-received force value if a force value received from said host computer is corrupted or missing.

9. A haptic feedback interface device as recited in claim 4 wherein said processor accesses a timer, said timer running for a time period starting when a force value is received from said host computer, wherein if said time period reaches a length greater than a predetermined length, a current force output is terminated, said predetermined length being a known length of time at least as long as a time required to receive a successive force value over said streaming channel.

10. A haptic feedback interface device as recited in claim 1 wherein said processor computes kinematics which convert said sensor signal to coordinates in a coordinate system.

11. A method for providing haptic feedback functionality on a host computer in a hybrid system including said host computer and a haptic feedback interface device, the method comprising:

receiving on a driver of said host computer a command to provide a force effect, said command provided by an application program running on said host computer;

determining whether said commanded force effect is an open loop effect or a closed loop effect;

if said commanded force effect is a closed loop effect, directing force information based on said command to said haptic feedback device to allow said haptic feedback device to compute force values for said closed loop effect; and

if said commanded force effect is an open loop effect,

storing information associated with said command in memory of said host computer;

computing a force value of said open loop force effect by said driver using said information associated with said command; and

providing said force value to said haptic feedback device to allow said haptic feedback device to output said force value as a force to a user of said haptic feedback device.

12. A method as recited in claim 11 wherein said driver is a low level emulation driver, functioning below an operating system and above a communication driver on said host computer.

13. A method as recited in claim 11 wherein said open loop effects are based primarily on time, and wherein said closed loop effects are based at least in part on a current position of a user manipulandum on said haptic feedback device.

14. A method as recited in claim 11 wherein a plurality of said force values for said open loop effect are provided to said haptic feedback device using a streaming channel.

15. A method as recited in claim 14 wherein said streaming channel is an isochronous channel.

16. A method as recited in claim 14 wherein said streaming channel is a Universal Serial Bus channel.

17. A method as recited in claim 14 wherein said streaming channel is a FireWire channel.

18. A method as recited in claim 11 wherein said command is a command to create said force effect in memory, and wherein said computing is performed when said force effect is to be output, wherein said computed force value is output to said device when a command to output said effect is received.

19. A method as recited in claim 11 wherein said command is a command to create said force effect in memory, wherein said computing is performed when said create command is received, wherein said computed force value is output to said device when a command to output said effect is received.

20. A method as recited in claim 11 wherein said driver emulates a haptic feedback device so that said application program sends said command as if it were sending said command to a haptic feedback device.

21. A method as recited in claim 11 wherein said force value is provided to said haptic feedback device over a streaming channel, and wherein said channel is kept continuously full of streaming data by providing a transfer request to a communication driver on said host computer while said communication driver is outputting data from a previous transfer request to said haptic feedback device.

22. A method as recited in claim 11 wherein said driver manages a memory model for said haptic feedback device and can determine when memory of said haptic feedback device is full or available.

23. A method as recited in claim 22 wherein said driver caches closed loop effects in host memory which cannot fit in device memory and provides a particular one of said cached closed loop effects to said device when said particular cached effect is commanded to be output by said device.

24. A method as recited in claim 11 wherein when said open loop effect is a repeating force effect, said driver computes a force value for an initial period of said effect, stores said force value, and retrieves said stored force value when outputting said value in successive periods of said effect.

25. A method as recited in claim 24 further comprising adjusting said stored force value based on an envelope that modifies a magnitude of said repeating force effect.

26. A method as recited in claim 11 wherein said driver stores created effects for said device in memory of said host computer in said memory model using effect caching, said memory model able to store a greater number of said effects than memory of said device.

27. A method as recited in claim 11 wherein said streaming force value is repeated in a successive packet sent to said haptic feedback device to allow error correction at said haptic feedback device.

28. A force feedback interface device, said device in communication with a host computer, said host computer running and displaying a graphical environment, said force feedback interface device comprising:

a user manipulatable object physically contacted by a user and moveable in at least one degree of freedom;

at least one actuator operative to output forces felt by said user;

at least one sensor operative to detect motion of said user manipulatable object in said at least one degree of freedom and outputting a sensor signal indicative of said motion; and

a device processor coupled to said at least one actuator and to said at least one sensor and local to said force feedback interface device, said processor operative to output signals to said at least one actuator to control said forces and receive said sensor signal from said at least one sensor, wherein said processor is operative to determine a force value for a high frequency open loop effect based at least in part on a command received from said host computer, and wherein said processor does not determine force values for low frequency open loop effects and is operative to receive force values for low frequency open loop effects from said host computer, said processor operative to direct said force values for said high frequency open loop effects and low frequency open loop effects to said at least one actuator.

29. A force feedback interface device as recited in claim 28 wherein said force values for said high frequency open loop effects and low frequency open loop effects define a vibration force effect.

30. A force feedback interface device as recited in claim 29 wherein said vibration force effect is a periodic force effect.

31. A force feedback interface device as recited in claim 28 wherein said processor outputs force values from said high frequency open loop effects simultaneously with outputting force values for low frequency open loop effects received from said host computer.

32. A force feedback interface device as recited in claim 31 wherein said processor sums force values from said high frequency open loop effect and from said low frequency open loop effect and outputs a total force value.

33. A force feedback interface device as recited in claim 28 wherein said device processor also determines closed loop force values based at least in part on a position of said user manipulatable object in said degree of freedom.

34. A force feedback interface device, said device in communication with a host computer, said host computer running and displaying a graphical environment, said force feedback interface device comprising:

a user manipulatable object physically contacted by a user and moveable in at least one degree of freedom;

at least one actuator operative to output forces, said forces felt by said user;

at least one sensor operative to detect motion of said user manipulatable object in said at least one degree of freedom and outputting a sensor signal indicative of said motion; and

a device processor coupled to said at least one actuator and to said at least one sensor and local to said force feedback interface device, said processor operative to output signals to said at least one actuator to control said forces and receive said sensor signal from said at least one sensor, wherein said processor is operative to determine a closed loop force value based at least in part on said sensor signal and outputs said closed loop force value, and wherein said processor does not compute any low frequency open loop force values and is operative to receive low frequency open loop force values from said host computer and direct said open loop force values to said at least one actuator.

35. A force feedback interface device as recited in claim 34 wherein said low frequency open loop force values describe a vibration having a frequency under a threshold frequency, and wherein said processor determines high frequency open loop values which describe a vibration having a frequency over said threshold frequency.

36. A force feedback interface device as recited in claim 34 wherein said open loop forces are computed on said host computer.

37. A force feedback interface device as recited in claim 34 wherein said open loop forces include periodic forces.

38. A force feedback interface device as recited in claim 34 wherein said open loop force values are received from said host computer over a streaming serial communication channel coupling said force feedback interface device to said host computer.

39. A force feedback interface device as recited in claim 37 wherein said closed loop forces include spring forces and damping forces.

40. A method for providing force feedback functionality on a host computer in a hybrid system including said host computer and a force feedback interface device, the method comprising:

receiving on a driver program of said host computer a command to provide a force effect, said command provided by an application program running on said host computer;

determining whether said commanded force effect is a high frequency open loop effect or a low frequency open loop effect;

if said commanded force effect is a high frequency open loop effect, directing force information based on said command to said force feedback device to allow said force feedback device to compute force values for said high frequency open-loop effect and output said force values as forces to said user; and

if said commanded force effect is a low frequency open loop effect,

storing information included with said command in memory of said host computer;

computing a force value of said low frequency open loop force effect by said driver using said information included with said command; and

providing said force value to said force feedback device to allow said force feedback device to output said force value as a force to a user of said force feedback device.

41. A method as recited in claim 40 wherein said low frequency open loop effect is a vibration having a frequency under a threshold frequency, and wherein said high frequency open loop effect is a vibration having a frequency over said threshold frequency.

42. A method for providing force feedback functionality on a host computer in a hybrid system including said host computer and a force feedback interface device, the method comprising:

receiving on a driver program of said host computer a command to provide a force effect, said command provided by an application program running on said host computer;

determining whether said commanded force effect is an open loop effect or a closed loop effect;

if said commanded force effect is a closed loop effect or a high frequency open-loop effect, directing force information based on said command to said force feedback device to allow said force feedback device to compute force values for said closed loop effect or said high frequency open-loop effect; and

if said commanded force effect is a low frequency open loop effect,

storing information included with said command in memory of said host computer;

computing a force value of said low frequency open loop force effect by said driver using said information included with said command; and

providing said force value to said force feedback device to allow said force feedback device to output said force value as a force to a user of said force feedback device.

43. A method as recited in claim 42 wherein said low frequency open loop effect is a vibration having a frequency under a threshold frequency, and wherein said high frequency open loop effect is a vibration having a frequency over said threshold frequency.

44. A haptic feedback interface device in communication with a host computer, said host computer running and displaying a graphical environment, said haptic feedback interface device comprising:

at least one actuator operative to output forces, said forces felt by a user of said haptic feedback interface device;

at least one sensor operative to detect manipulation of said haptic feedback interface device by said user and outputting a sensor signal indicative of said manipulation; and

a processor in communication with said at least one actuator and said at least one sensor, said processor operative to output force values to said at least one actuator to control said forces and receive said sensor signal from said at least one sensor, wherein said processor is operative to determine a closed loop force value based at least in part on said sensor signal and output said closed loop force value to said at least one actuator, and wherein said microcontroller does not compute open loop force values but instead is operative to receive open loop force values from said host computer and direct said open loop force values to said at least one actuator, wherein said forces output by said at least one actuator are based on said closed loop force values and said open loop force values.

45. A haptic feedback interface device as recited in claim 44 wherein said open loop force values are computed on said host computer.

46. A haptic feedback interface device as recited in claim 44 wherein said open loop force values include periodic forces.

47. A haptic feedback interface device as recited in claim 44 wherein said open loop force values are received from said host computer over a streaming serial communication channel between said haptic feedback interface device and said host computer.

48. A haptic feedback interface device as recited in claim 44 wherein said closed loop forces include spring forces, damping forces, and texture forces.

50. A haptic feedback interface device as recited in claim 47 wherein said processor extrapolates a force value if a force value received from said host computer is corrupted or missing.

51. A haptic feedback interface device as recited in claim 47 wherein said processor outputs a previously-received force value if a force value received from said host computer is corrupted or missing.

52. A haptic feedback interface device as recited in claim 47 wherein said processor accesses a timer, said timer running for a time period starting when a force value is received from said host computer, wherein if said time period reaches a length greater than a predetermined length, a current force output is terminated, said predetermined length being a known length of time at least as long as a time required to receive a successive force value over said streaming channel.

53. A haptic feedback interface device as recited in claim 44 wherein said processor computes kinematics which convert said sensor signal to coordinates in a coordinate system.

54. A haptic feedback interface device as recited in claim 44 wherein said manipulation of said haptic feedback interface device includes moving at least a portion of said haptic feedback interface device in at least one degree of freedom.

49. A haptic feedback interface device as recited in 47 claim wherein said streaming channel is an isochronous channel.