Textures and other spatial sensations for a relative haptic interface device

• US 6,448,977 B1
• Filed: 02/15/2000
• Issued: 09/10/2002
• Est. Priority Date: 11/14/1997
• Status: Expired due to Term

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Martin, Kenneth M., Rosenberg, Louis B., Braun, Adam C.
• Primary Examiner(s): Hjerpe, Richard
• Assistant Examiner(s): Tran, Henry N.
• Application Number:
  US09/504,201
• Time in Patent Office: 938 Days
• Field of Search: 345/156, 345/157, 345/161, 345/163, 345/700, 345/701, 345/702
• US Class Current: 715/701
• CPC Class Codes: A63F 13/21 characterised by their sensors, purposes or types
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: A63F 2300/1043 being characterized by constructional details
• CPC Class Codes: A63F 2300/1062 being specially adapted to a type of game, e.g. steering wheel
• CPC Class Codes: A63F 2300/8017 Driving on land or water; Flying
• CPC Class Codes: G06F 2203/013 Force feedback applied to a game
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 2203/015 Force feedback applied to a joystick
• CPC Class Codes: G06F 2203/0333 Ergonomic shaped mouse for one hand
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0338 with detection of limited linear or angular displacement of an operating part of the device from a neutral position, e.g. isotonic or isometric joysticks
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/03548 Sliders, in which the moving part moves in a plane
• CPC Class Codes: G06F 3/0362 : with detection of 1D translations or rotations of an operating part of the device, e.g. scroll wheels, sliders, knobs, rollers or belts
• CPC Class Codes: H04L 67/75 : Indicating network or usage conditions on the user display
• CPC Class Codes: H04L 69/329 : in the application layer [OSI layer 7]

Abstract

A low-cost haptic feedback device that provides spatially-based sensations such as textures in correlation with a displayed graphical environment. The device includes a housing, a sensor device, and an actuator for applying a force to the user. A local processor reports relative sensor data to the host processor and receives force information from the host processor. The host force information causes a texture sensation to be output by the actuator, the texture sensation providing forces to the user at least approximately spatially correlated with predefined locations in the graphical environment as determined by a local processor. In some embodiments, the local processor can model a position of the cursor while the cursor interacts with the texture field, where the modeled position is used for determining the force output to the user, and the force information from the host can include a gating command to activate or deactivate the texture sensation when the cursor enters or exits the texture field.

Claims

1. A haptic feedback device capable of communicating with a host computer implementing a graphical environment, said device physically moved by a user in a planar workspace, the device comprising:

a housing that is physically contacted by said user;

a sensor device coupled to said housing of said device, said sensor device detecting said movement of said device in said planar workspace and outputting sensor signals representative of said movement;

an actuator coupled to said housing of said device, said actuator operative to apply a force along an axis approximately perpendicular to said planar workspace, wherein said force is transmitted to said user contacting said housing; and

a local processor separate from a host processor in said host computer and in communication with said host processor, said local processor reporting sensor data based on said sensor signals to said host processor, said sensor data representing a relative change in position of said device in said planar workspace from a previous position, and wherein said local processor receives force information from said host processor, wherein said force information causes a texture sensation to be output by said actuator, said texture sensation providing forces to said user based on an interaction of a cursor in said graphical environment with a texture field in said graphical environment, wherein said texture field includes predefined locations that are at least approximately spatially correlated with said forces from said texture sensation, and wherein said forces of said texture sensation include pulses, a pulse being output on said housing each time said cursor crosses one of a plurality of predefined regions in a texture field.

2. A haptic feedback device as recited in claim 1 wherein each of said regions includes two borders, wherein said pulse is output when said cursor crosses one of said borders and enters said region, and when said cursor crosses said other border and exits said region.

3. A haptic feedback device as recited in claim 1 wherein said actuator moves a cover portion of said housing approximately perpendicularly to said planar workspace, wherein said cover portion contacts a portion of said user'"'"'s hand.

4. A haptic feedback device as recited in claim 1 wherein said actuator moves a button of said haptic feedback device approximately perpendicularly to said planar workspace.

5. A haptic feedback device as recited in claim 1 wherein said device is a mouse.

6. A haptic feedback device as recited in claim 5 wherein said sensor device includes a ball that frictionally contacts a surface on which said mouse is moved by said user.

7. A haptic feedback device as recited in claim 5 wherein said sensor device includes an optical sensor that detects motion of a surface on which said mouse is moved relative to said housing of said mouse.

8. A haptic feedback device as recited in claim 1 wherein said actuator is a linear actuator.

9. A haptic feedback device as recited in claim 1 wherein said texture sensation is commanded by a command received from said host computer.

10. A haptic feedback device as recited in claim 1 wherein said texture sensation has a sense direction of up or down, said sense being commanded from said host computer, wherein said sense direction provides the direction of pulses output for said texture sensation.

11. A haptic feedback device as recited in claim 1 wherein said local processor determines when pulses are output by said actuator in correlation with said predefined locations, and wherein said host processor determines when to start and stop said texture sensation.

12. A haptic feedback device as recited in claim 1 wherein said local processor further receives ballistic information from said host processor, said ballistic information allowing said local processor to determine a cursor position with respect to said predefined locations associated with said texture sensation.

13. A haptic feedback device capable of communicating with a host computer implementing a graphical environment, said device physically moved by a user in a planar workspace, the device comprising:

a housing that is physically contacted by said user;

a sensor device coupled to said housing of said device, said sensor device detecting said movement of said device in said planar workspace and outputting sensor signals representative of said movement;

an actuator coupled to said housing of said device, said actuator operative to apply an inertial force approximately along an axis approximately perpendicular to said planar workspace, wherein said force is transmitted to said user contacting said housing; and

a local processor separate from a host processor in said host computer and in communication with said host processor, said local processor reporting sensor data based on said sensor signals to said host processor, said sensor data representing a relative change in position of said device in said planar workspace from a previous position, and wherein said local processor receives force information from said host processor, wherein said force information causes a texture sensation to be output by said actuator, said texture sensation providing forces to said user based on an interaction of a cursor in said graphical environment with a texture field in said graphical environment, wherein said texture field includes predefined locations that are at least approximately spatially correlated with said forces from said texture sensation.

14. A haptic feedback device as recited in claim 13 wherein said device is a mouse.

15. A haptic feedback device as recited in claim 13 wherein said local processor further receives ballistic information from said host processor, said ballistic information allowing processor to determine a cursor position with respect to said predefined locations with said texture sensation.

16. A haptic feedback device capable of communicating with a host computer implementing a graphical environment, said device physically moved by a user in a planar workspace, the device comprising:

a housing that is physically contacted by said user;

a sensor device coupled to said housing of said device, said sensor device detecting said movement of said device in said planar workspace and outputting sensor signals representative of said movement;

an actuator coupled to said housing of said device, said actuator operative to apply a force along an axis approximately perpendicular to said planar workspace, wherein said force is transmitted to said user contacting said housing; and

a local processor separate from a host processor in said host computer and in communication with said host processor, said local processor reporting sensor data based on said sensor signals to said host processor, said sensor data representing a relative change in position of said device in said planar workspace from a previous position, and wherein said local processor receives force information from said host processor, wherein said force information causes a texture sensation to be output by said actuator, said texture sensation providing forces to said user based on an interaction of a cursor in said graphical environment with a texture field in said graphical environment, wherein said texture field includes predefined locations that are at least approximately spatially correlated with said forces from said texture sensation, wherein said texture sensation is commanded by a command received from said host computer, and wherein said command includes parameters of width and spacing.

17. A method for providing haptic feedback to a user interacting with a graphical environment displayed by a host computer, the user manipulating a mouse device, the method comprising:

detecting the motion of said mouse device in a planar workspace and providing sensor data to said host computer, said sensor data representing a relative change in position of said mouse device in said planar workspace;

receiving information from said host computer indicating that a texture sensation is to be initiated, said texture sensation including a plurality of pulses, each of said pulses to be correlated with an interaction of a displayed cursor with a location in a texture field in said graphical environment;

determining, on a processor local to said mouse device, when said pulses are to be output while said cursor is interacting with said texture field, said determining based on motion of said mouse device in at least one degree of freedom of said planar workspace; and

outputting said pulses of said texture sensation to said user of said mouse device approximately along an axis substantially perpendicular to said planar workspace, wherein said user can contact said housing and experience said texture sensation.

18. A method as recited in claim 17 wherein said determining when said pulses are to be output includes modeling a position of said cursor, wherein said output of said texture sensation is based on said modeled position of said cursor.

19. A method as recited in claim 18 wherein said local processor adds said relative position changes of said mouse device to determine said modeled position.

20. A method as recited in claim 18 wherein said modeled position is a ballistic position providing a cursor position based on a velocity of said mouse device in said workspace.

21. A method as recited in claim 20 wherein said ballistic position modeled by said local processor is used only while said cursor is within said texture field and for determining when said pulses of said texture are to be output.

22. A method as recited in claim 20 wherein said modeled ballistic position is based on a velocity threshold, wherein if said mouse velocity is below said velocity threshold, said cursor is modeled to move a shorter distance across texture field than if said mouse velocity is above said velocity threshold.

23. A method as recited in claim 20 further comprising receiving ballistic information from said host processor, said ballistic information used in said modeling of said ballistic position.

24. A method as recited in claim 23 wherein said ballistic information from said host processor includes a ballistic scale factor for scaling motion of said cursor based on mouse velocity.

25. A method as recited in claim 18 wherein said modeled ballistic position is not used to determine when to cease said output of said texture sensation.

26. A method as recited in claim 18 wherein said host computer determines a host ballistic position of said cursor based on said sensor data, said host computer using said host ballistic position to determine when said cursor interacts with said texture field and to determine when to send said information to start said texture sensation.

27. A method as recited in claim 17 wherein said information from said host computer to output said texture sensation is sent from said host computer when said cursor has entered said texture field.

28. A method as recited in claim 17 further comprising receiving information from said host computer indicating that said texture sensation is to be ceased.

29. A method as recited in claim 17 wherein said host computer provides a texture command parsed by said local processor, wherein characteristics of said texture sensation are specified by said texture command and are stored in memory local to said local processor.

30. A method as recited in claim 29 wherein said characteristics include a spacing of said features in said texture field.

31. A method as recited in claim 29 wherein said characteristics include a sense of said texture, said sense providing texture pulses in either an up direction or a down direction.

32. A method as recited in claim 29 wherein said information from said host computer is a state command to start said output of said texture sensation, wherein said state command is separate from said texture command.

33. A method as recited in claim 17 wherein said force is an inertial force caused by moving an inertial mass coupled to an actuator.

34. A method as recited in claim 17 wherein said force is a contact force caused by moving a cover portion of said mouse device or a button of said mouse device against said user.

35. A haptic mouse device capable of communicating with a host computer implementing a graphical environment, said mouse device physically moved by a user in a planar workspace, the mouse device comprising:

a housing that is physically contacted by said user;

a sensor device coupled to said housing of said device, said sensor device detecting said movement of said device in said planar workspace and outputting sensor signals representative of said movement;

an actuator coupled to said housing of said device, said actuator operative to apply forces along an axis approximately perpendicular to said planar workspace, wherein said forces are transmitted to said user contacting said housing; and

a local processor separate from a host processor in said host computer and in communication with said host processor, said local processor reporting sensor data based on said sensor signals to said host processor, said sensor data representing a relative change in position of said device in said planar workspace, and wherein said local processor receives force information from said host processor, wherein said force information causes a texture sensation to be output by said actuator, said texture sensation providing said forces to said user based on an interaction of a cursor in said graphical environment with features of a texture field in said graphical environment associated with said texture sensation, and wherein said local processor determines when to output said forces to said user after said cursor is within said texture field, and wherein said force information received from said host computer is a gating command that indicates to said microprocessor to activate said texture sensation.

36. A haptic mouse device as recited in claim 35 wherein said host computer sends another gating command to indicate to said microprocessor to deactivate said texture sensation.

37. A haptic mouse device capable of communicating with a host computer implementing a graphical environment, said mouse device physically moved by a user in a planar workspace, the mouse device comprising:

a housing that is physically contacted by said user;

a sensor device coupled to said housing of said device, said sensor device detecting said movement of said device in said planar workspace and outputting sensor signals representative of said movement;

an actuator coupled to said housing of said device, said actuator operative to apply forces along an axis approximately perpendicular to said planar workspace, wherein said forces are transmitted to said user contacting said housing; and

a local processor separate from a host processor in said host computer and in communication with said host processor, said local processor reporting sensor data based on said sensor signals to said host processor, said sensor data representing a relative change in position of said device in said planar workspace, and wherein said local processor receives force information from said host processor, wherein said force information causes a texture sensation to be output by said actuator, said texture sensation providing said forces to said user based on an interaction of a cursor in said graphical environment with features of a texture field in said graphical environment associated with said texture sensation, and wherein said local processor determines when to output said forces to said user after said cursor is within said texture field, wherein said local processor models a position of said cursor when said cursor interacts with said texture field, said modeled position used for determining said force output to said user, and wherein said modeled position is an approximate ballistic position of said cursor, said ballistic position providing a cursor position based on a velocity of said mouse device in said workspace.

38. A haptic mouse device as recited in claim 37 wherein said host determines a host ballistic position based on said sensor data, said host using said host ballistic position to display said cursor in said graphical environment and determine when said texture sensation should be activated and deactivated.

39. A haptic mouse device capable of communicating with a host computer implementing a graphical environment, said mouse device physically moved by a user in a planar workspace, the mouse device comprising:

a housing that is physically contacted by said user;

a sensor device coupled to said housing of said device, said sensor device detecting said movement of said device in said planar workspace and outputting sensor signals representative of said movement;

an actuator coupled to said housing of said device, said actuator operative to apply forces along an axis approximately perpendicular to said planar workspace, wherein said forces are transmitted to said user contacting said housing; and

a local processor separate from a host processor in said host computer and in communication with said host processor, said local processor reporting sensor data based on said sensor signals to said host processor, said sensor data representing a relative change in position of said device in said planar workspace, and wherein said local processor receives force information from said host processor, wherein said force information causes a texture sensation to be output by said actuator, said texture sensation providing said forces to said user based on an interaction of a cursor in said graphical environment with features of a texture field in said graphical environment associated with said texture sensation, and wherein said local processor determines when to output said forces to said user after said cursor is within said texture field, and wherein each of said forces is one of a plurality of pulses in said texture sensation, each of said pulses being correlated with an interaction of a cursor with one of said features in said texture field.

40. A haptic mouse device as recited in claim 39 wherein said actuator provides a linear contact force to said user by moving a cover portion or a button of said mouse device.

41. A haptic mouse device as recited in claim 39 wherein said local processor is a microprocessor.

42. A haptic mouse device as recited in claim 39 wherein said mouse device has an unbounded workspace.

43. A haptic mouse device as recited in claim 39 wherein said local processor models a position of said cursor when said cursor interacts with said texture field, said modeled position used for determining said force output to said user.

44. A haptic mouse device capable of communicating with a host computer implementing a graphical environment, said mouse device physically moved by a user in a planar workspace, the mouse device comprising:

a housing that is physically contacted by said user;

a sensor device coupled to said housing of said device, said sensor device detecting said movement of said device in said planar workspace and outputting sensor signals representative of said movement;

an actuator coupled to said housing of said device, said actuator operative to apply forces along an axis approximately perpendicular to said planar workspace, wherein said forces are transmitted to said user contacting said housing; and

a local processor separate from a host processor in said host computer and in communication with said host processor, said local processor reporting sensor data based on said sensor signals to said host processor, said sensor data representing a relative change in position of said device in said planar workspace, and wherein said local processor receives force information from said host processor, wherein said force information causes a texture sensation to be output by said actuator, said texture sensation providing said forces to said user based on an interaction of a cursor in said graphical environment with features of a texture field in said graphical environment associated with said texture sensation, and wherein said local processor determines when to output said forces to said user after said cursor is within said texture field, and wherein said actuator provides a linear inertial force to said housing by moving an inertial mass in a linear direction.