Three-dimensional cursor control interface with force feedback

• US 5,880,714 A
• Filed: 01/15/1997
• Issued: 03/09/1999
• Est. Priority Date: 07/16/1993
• Status: Expired due to Term

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Rosenberg, Louis B., Jackson, Bernard G.
• Primary Examiner(s): Luu, Matthew
• Application Number:
  US08/784,198
• Time in Patent Office: 783 Days
• Field of Search: 345/156, 345/157, 345/158, 345/162
• US Class Current: 345/156
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: B25J 9/1692 Calibration of manipulator
• CPC Class Codes: G01B 21/04 by measuring coordinates of points
• CPC Class Codes: G01B 5/008 using coordinate measuring machines
• CPC Class Codes: G01B 7/004 for measuring coordinates of points
• CPC Class Codes: G05B 19/4207 in which a model is traced or scanned and corresponding data recorded
• CPC Class Codes: G05G 5/03 Means for enhancing the operator's awareness of arrival of the controlling member at a command or datum position; Providing feel, e.g. means for creating a counterforce arrangements for indicating the position of the controlling member G05G1/015
• CPC Class Codes: G05G 9/04737 with six degrees of freedom
• CPC Class Codes: G06F 2203/015 Force feedback applied to a joystick
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0383 Signal control means within the pointing device
• CPC Class Codes: H01H 2003/008 with a haptic or a tactile feedback controlled by electrical means, e.g. a motor or magnetofriction

Abstract

A method and apparatus for use with a computer for providing commands to a computer through tracked manual gestures and for providing feedback to the user through forces applied to the interface. A user manipulatable object is coupled to a mechanical linkage which is, in turn, supportable on a fixed surface. The mechanical linkage or the user manipulatable object is tracked by sensors for sensing the location and/or orientation of the object. A multi-processor system architecture is disclosed wherein a host computer system is interfaced with a dedicated microprocessor which is responsive to the output of the sensors and provides the host computer with information derived from the sensors. The host computer runs an application program which responds to the information provided via the microprocessor and which can provide force-feedback commands back to the microprocessor, the force feedback being felt by a user via the user manipulatable object.

Claims

1. An interactive device for use in conjunction with a host computer, wherein images are displayed on a computer display apparatus coupled to said host computer, said interactive device comprising:

a user manipulatable object engaged by a user'"'"'s hand to allow dexterous manipulations by fingers of said user;

a mechanical linkage coupled to a fixed surface by a base rotary joint and coupled to said user manipulatable object by an object rotary joint, said linkage for supporting said object allowing at least five degrees of freedom in motion of said object with respect to said fixed surface, wherein said mechanical linkage provides said degrees of freedom through a structure of substantially rigid members joined by a plurality of rotary joints, said mechanical linkage providing said user the ability to manipulate both the location and orientation of said object in three dimensional space, and wherein a configuration of said degrees of freedom allow said user to rotate said object about a fixed point in space when three degrees of freedom closest to said fixed surface are held fixed and when remaining ones of said degrees of freedom are moved;

one or more sensors for producing an interactive object locative signal which is responsive to and corresponding with the position of the user manipulatable object, said object locative signal providing information about the location of said user manipulatable object for use by said host computer to manipulate images displayed by said computer display apparatus in accordance with said location of said user manipulatable object, said images including a cursor whose position on said computer display apparatus is influenced by said user manipulatable object locative signal; and

a force generator for generating a force on said user object in at least one of said five degrees of freedom in response to force signals provided to said interactive device, said force signals correlated to information displayed on said computer display apparatus including interaction of said cursor with other images on said computer display apparatus.

2. An interactive device as recited in claim 1 wherein said mechanical linkage further includes a counter weight for opposing a gravitational weight of said user manipulatable object.

3. An interactive device as recited in claim 1 wherein said mechanical linkage further includes a spring for opposing a gravitational weight of said user manipulatable object.

4. An interactive device as recited in claim 1 wherein said linkage allows six degrees of freedom in motion of said object, wherein three of said degrees of freedom closest to said user manipulatable object allow an orientation of said object to be adjusted in three dimensional space while three degrees of freedom closest to said fixed surface allow a location of said object to be adjusted in three dimensional space.

5. An interactive device as recited in claim 1 further including a switch coupled to said user manipulatable object, said switch capable of being in multiple states in response to user interaction, wherein a state of said switch being transmitted to said host computer and wherein an action is taken by said host computer in response to said state of said switch.

6. An interactive device as recited in claim 1 further including a foot pedal containing a switch capable of being in multiple states in response to user interaction, a state of said switch being transmitted to said host computer, wherein an action is taken by said host computer in response to said switch state.

7. An interactive device as recited in claim 1 wherein a configuration of said joints allows said user manipulatable object to spin freely about an axis extending through the length of said object while all others of said joints remain fixed in position.

8. An interactive device as recited in claim 1 wherein said object is a stylus having a pen-like configuration to allow writing-like manipulations between said fingers.

9. A cursor control interface for use in conjunction with a host computer, said interface enabling a user to control the position of a cursor displayed by said host computer, said interface also enabling said host computer to convey force feedback to a user in accordance with cursor position, said interface comprising:

a user manipulatable object engaged by a user'"'"'s hand to allow dexterous manipulations of said user manipulatable object;

a mechanical linkage coupled to a fixed surface by a rotary joint and coupled to said user manipulatable object, said linkage for supporting said object while allowing at least five degrees of freedom in the motion of said object with respect to said fixed surface, said mechanical linkage providing said degrees of freedom through a structure of substantially rigid members joined by rotary joints, said mechanical linkage providing said user the ability to manipulatable both a location and orientation of said object in three dimensional space;

a force generator for generating a force on said user object in at least one of said five degrees of freedom in response to signals provided by said host computer, said force signals correlated to information displayed on said computer display apparatus;

a device microprocessor separate from said host computer, said device microprocessor enabling communication between said cursor control interface and said host computer;

one or more sensors coupled to said device microprocessor for producing an interactive object locative signal which is responsive to and corresponding with the position of said user manipulatable object in three dimensional space, said object locative signal for use by said host computer to manipulate images displayed by said computer display apparatus in accordance with said location of said user manipulatable object, said displayed images including a cursor whose position on said computer display apparatus is influenced by said object locative signal;

memory local to said device microprocessor and separate from memory of said host computer for locally storing a representation of said locative signal; and

a communication bus connecting said device microprocessor and said host computer, said bus used in conveying force feedback commands from said host computer to said local microprocessor, said bus also used in conveying a representation of said object locative signal from said cursor control interface to said host computer.

10. A cursor control interface as recited in claim 9 wherein an order and configuration of said joints and said members allow said user to rotate said object about a fixed point in space when three degrees of freedom closest to said fixed surface are held fixed and when remaining ones of said degrees of freedom are moved.

11. A cursor control interface as recited in claim 9 wherein said device microprocessor computes said position and orientation of said user manipulatable object using data from said sensors and knowledge of link lengths and device kinematics.

12. A cursor control interface as recited in claim 9 wherein said device microprocessor monitors and decodes host commands including said force feedback commands from said host computer.

13. A cursor control interface as recited in claim 12 wherein said device microprocessor reads said one or more sensors in response to said decoded commands.

14. A cursor control interface as recited in claim 12 wherein said device microprocessor controls said force generators in response to said decoded commands.

15. A cursor control interface as recited in claim 9 wherein said memory includes non-volatile memory.

16. A cursor control interface as recited in claim 9 wherein said non-volatile memory stores program instructions for setting communication parameters for enabling communication between said microprocessor and said host computer and for decoding at least one of said host commands.

17. A cursor control interface as recited in claim 9 wherein said non-volatile memory includes a look-up table.

18. A cursor control interface as recited in claim 9 further including a switch physically coupled to said user manipulatable object and electrically coupled to said local microprocessor, said switch capable of being in one of multiple states in response to user interaction, and wherein the state of said switch is transmitted to said host computer by said local microprocessor.

19. A cursor control interface as recited in claim 9 wherein said communication bus is a serial interface.

20. A cursor control interface as recited in claim 12 further including a plurality of command routines stored in said local memory, at least one of said command routines allowing said microprocessor to control said force generator in accordance with at least one of said decoded host commands, and at least one of said command routines reporting a representation of said locative signal to said host computer in accordance with at least one of said decoded host commands.

21. A cursor control interface as recited in claim 20 further including a switch on said user objects wherein said local microprocessor can monitor and report the state of said switch, and wherein the state of said switch to said host is controlled by one of said command routines.

22. A cursor control interface as recited in claim 20, further including a foot pedal wherein said local microprocessor can monitor and report the state of said foot pedal and report said state to said host computer.

23. A cursor control interface for use in conjunction with a host computer, said interface enabling a user to control the position of a cursor displayed by said host computer, said interface also enabling said host computer to convey force feedback to a user in accordance with cursor position, said interface comprising:

a user manipulatable object engaged by a user'"'"'s hand to allow dexterous manipulations of said user manipulatable object;

a mechanical linkage coupled to a fixed surface by a rotary joint and coupled to said user manipulatable object, said linkage for supporting said object while allowing at least five degrees of freedom in the motion of said object with respect to said fixed surface, said mechanical linkage providing said degrees of freedom trough a structure of substantially rigid members joined by rotary joints, said mechanical linkage providing said user the ability to manipulatable both a location and orientation of said object in three dimensional space;

a force generator for generating a force on said user object in at least one of said five degrees of freedom in response to signals provided by said host computer, said force signals correlated to information displayed on said computer display apparatus;

a device microprocessor separate from said host computer, said device microprocessor enabling communication between said cursor control interface and said host computer;

one or more sensors coupled to said device microprocessor for producing an interactive object locative signal which is responsive to and corresponding with the position of said user manipulatable object in tree dimensional space, said object locative signal for use by said host computer to manipulate images displayed by said computer display apparatus in accordance with said location of said user manipulatable object, said displayed images including a cursor whose position on said computer display apparatus is influenced by said object locative signal;

memory local to said device microprocessor and separate from memory of said host computer for locally storing a representation of said locative signal; and

a communication bus connecting said device microprocessor and said host computer, said bus used in conveying force feedback data from said host computer to said local microprocessor, said bus also used in conveying a representation of said object locative signal from said cursor control interface to said host computer.

24. A cursor control interface as recited in claim 23 wherein an order and configuration of said joints and said members allow said user to rotate said object about a fixed point in space when three degrees of freedom closest to said fixed surface are held fixed and when remaining ones of said degrees of freedom are moved.

25. A cursor control interface as recited in claim 23 wherein said device microprocessor computes said position and orientation of said user manipulatable object using data from said sensors and knowledge of link lengths and device kinematics.

26. A cursor control interface as recited in claim 23 wherein said memory includes non-volatile memory.

27. A cursor control interface as recited in claim 23 wherein said non-volatile memory stores program instructions for setting communication parameters for enabling communication between said microprocessor and said host computer.

28. A cursor control interface as recited in claim 23 further including a switch physically coupled to said user manipulatable object and electrically coupled to said local microprocessor, said switch capable of being in one of multiple states in response to user interaction, wherein the state of said switch is transmitted to said host computer by said local microprocessor.

29. A cursor control interface as recited in claim 28 wherein said state of said switch provides a command to said host computer.

30. A cursor control interface as recited in claim 23 wherein said mechanical linkage includes at least one counterweight to ease manipulation of said user manipulatable object by said user.