Graphical click surfaces for force feedback applications to provide selection of functions using cursor interaction with a trigger position of a graphical object

• US 6,317,116 B1
• Filed: 06/08/2000
• Issued: 11/13/2001
• Est. Priority Date: 12/13/1995
• Status: Expired due to Term

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Rosenberg, Louis B., Chang, Dean C.
• Primary Examiner(s): Saras, Steven
• Assistant Examiner(s): Bell, Paul A
• Application Number:
  US09/590,856
• Time in Patent Office: 523 Days
• Field of Search: 345/145, 345/146, 345/156, 345/157, 345/163, 345/161, 345/339, 345/419, 345/355, 463/30, 463/38, 318/568.11, 318/568.17, 318/568.19, 318/568.25, 700/83, 700/84, 700/85
• US Class Current: 715/701
• CPC Class Codes: A63F 2300/1025 details of the interface with the game device, e.g. USB version detection
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: G05G 2009/0477 holding the member in a number of definite positions
• CPC Class Codes: G05G 9/047 the controlling member being movable by hand about orthogonal axes, e.g. joysticks for switches H01H25/04
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0354 with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/03548 Sliders, in which the moving part moves in a plane
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: H01H 2003/008 with a haptic or a tactile feedback controlled by electrical means, e.g. a motor or magnetofriction
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

A method and apparatus for providing a click surface in a graphical environment, such as a graphical user interface, implemented on a host computer for use with a force feedback interface device. A displayed cursor is controlled by a user-moveable user object, such as a mouse, of the interface device. A click surface is displayed with an associated graphical object, such as a graphical button or an edge of a window, icon, or other object. When the click surface is contacted by the cursor, a force is output opposing movement of the user object in a direction into the click surface and into the graphical object. When the user object has moved to or past a trigger position past the contact with the click surface, a command gesture signal is provided to the host computer indicating that the graphical object has been selected as if a physical input device on the user object, such as a button, has been activated by the user. Preferably, the host computer displays the graphical environment including the click surface and cursor, while a microprocessor local to the interface device controls the force output of the click surface in parallel with the host display.

Claims

1. A method for providing a force feedback click surface in a graphical environment provided by a host computer, said click surface for selecting a function in a graphical user interface implemented on said host computer, said host computer coupled to a force feedback device, the method comprising:

causing the display of said click surface on a display screen in said graphical environment and receiving position data from said force feedback device describing a position of at least a portion of the force feedback device adapted to be contacted and manipulated by a user in at least one degree of freedom, wherein a user-controlled cursor is displayed at a position in said graphical environment based on said position data;

providing a force command to said force feedback device causing said force feedback device to output a force to the user when said user-controlled cursor contacts said click surface displayed in said graphical environment;

maintaining a display of said click surface at a point where said cursor first engaged said click surface while said at least a portion of the force feedback device is moved to positions into and past said engagement point of said click surface; and

selecting said function when a trigger point of said click surface positioned past said engagement point of said click surface has been reached by said at least a portion of the force feedback device.

2. A method as recited in claim 1 further comprising changing said force output to said user to indicate to said user that said function has been selected.

3. A method as recited in claim 2 wherein said click surface is part of a graphical button having an on state and an off state, wherein said state is toggled when said trigger point is reached by said at least a portion of the force feedback device.

4. A method as recited in claim 1 wherein said click surface is a portion of a graphical icon in said graphical environment, and wherein said signal to said host computer indicates that said icon has been double-clicked by said user.

5. A method as recited in claim 1 wherein said click surface is at least a portion of an edge of a graphical window in said graphical environment.

6. A method as recited in claim 1 wherein said click surface is displayed at a new position after said at least a portion of the force feedback device reaches said trigger position.

7. A method as recited in claim 1 wherein said click surface is a positive action button in which said cursor remains displayed engaged with said stationary click surface while said at least a portion of the force feedback device moves past said click surface into said graphical object, and in which said click surface is displayed at a new position when said at least a portion of the force feedback device reaches said trigger position.

8. A method as recited in claim 1 wherein said click surface is a static selection surface in which said cursor always is displayed engaged with said stationary click surface while said at least a portion of the force feedback device is positioned into and past said click surface.

9. A method as recited in claim 1 wherein said click surface is a static selection surface in which said click surface is always displayed at said engagement position while said cursor tracks said movement of said at least a portion of the force feedback device and is displayed to move into and past said click surface.

10. A method as recited in claim 9 wherein said graphical object is a graphical button, and wherein said at least a portion of the force feedback device moving to said trigger position either toggles said button from an off state to an on state or from an on state to an off state, wherein said click surface is displayed at a greater distance from said trigger position at said original position when said button is in an off state than when said button is in said on state at said new position.

11. A method as recited in claim 1 wherein said at least a portion of the force feedback device comprises a user object.

12. A method as recited in claim 1 wherein the force output to the user comprises a force opposing motion of said at least a portion of the force feedback device that would move said cursor into said click surface.

13. A method as recited in claim 1 wherein the force output to the user comprises one or more of a vibration, a texture, and a jolt.

14. A method of using a force feedback interface device to select a function in a graphical user interface implemented on a host computer, the host computer in communication with the force feedback interface device, the method comprising:

updating a location of a cursor within said graphical user interface using sensor information received from said force feedback interface device, said sensor information representing motion of at least a portion of said force feedback interface device;

determining when said cursor engages a click surface of a graphical element displayed within said graphical user interface;

providing a force command to said force feedback interface device to cause a force to be output to a user when said cursor engages said click surface;

maintaining a display of said click surface of said graphical element at a point where said cursor first engaged said click surface while said at least a portion of said force feedback interface device is moved into said click surface;

determining when said cursor has moved to or past a trigger location positioned into said click surface;

when said cursor has moved to or past said trigger location, selecting said function within said graphical user interface; and

changing said force to indicate to said user that said function has been selected.

15. A method as recited in claim 14 wherein said cursor is determined to have moved to said trigger position when said cursor has moved a distance corresponding to predetermined distance past said click surface from said point of engagement.

16. A method as recited in claim 14 wherein said click surface is displayed at a new position when said cursor reaches said trigger position.

17. A method as recited in claim 14 wherein said click surface is a static selection surface in which said click surface and said cursor are always displayed at said original position of said click surface while said cursor moves past said click surface into said first graphical object and when said trigger position is reached.

18. A method as recited in claim 14 wherein said click surface is a static selection surface in which said click surface is always displayed at said original position while said cursor moves past said click surface and said cursor is displayed to move past said click surface into said graphical element in accordance with said motion of said at least a portion of said force feedback interface device.

19. A method as recited in claim 14 wherein said click surface hinders said cursor from moving out of contact with said click surface when said cursor moves to the ends of said click surface.

20. A method as recited in claim 14 wherein said host computer determines when said cursor has moved to or past said trigger position and when said function is selected.

21. A method as recited in claim 14 wherein a local microprocessor, separate from said host computer, is provided on said force feedback interface device and reports to said host computer when said at least a portion of said force feedback interface device is moved such that said cursor is positioned at or past said trigger position.

22. A method as recited in claim 14 wherein said graphical element is a window displayed in said graphical environment and said click surface is at least a portion of an edge of said window.

23. A method as recited in claim 14 wherein said graphical element is a graphical button displayed in a Web page graphical environment and said click surface is an edge of said button.

24. A method as recited in claim 14 further including changing the display properties of said graphical element after said trigger location has been exceeded to indicate to said user that said function has been selected.

25. A method as recited in claim 14 wherein said at least a portion of said force feedback interface device comprises a user object.

26. A method as recited in claim 14 wherein said force output to said user comprises a force in a direction that opposes cursor motion into said click surface.

27. A method as recited in claim 26 wherein said force in a direction that opposes cursor motion into said click surface is proportional to a distance between a present position of said cursor past said click surface and an original position of said click surface.

28. A method as recited in claim 26 wherein said force in a direction that opposes cursor motion into said click surface changes to be proportional to a distance between a present position of said cursor past said click surface and a new position of said click surface, said change occurring when said cursor moves to said trigger position.

29. A method as recited in claim 14 wherein the force output to said user comprises one or more of a vibration, a texture, and a jolt.

30. A method of providing a click surface in a graphical environment implemented on a host computer for use with a force feedback interface device coupled to said host computer, the method comprising:

determining when a click surface of a first graphical object displayed in a graphical environment is contacted by a user-controlled second graphical object at an original position of said click surface, said user-controlled second graphical object being displayed at a position corresponding to a position of at least a portion of said interface device in a physical object workspace, said at least a portion of said interface device physically contacted by a user and moveable in a degree of freedom;

causing the output of a force to said user, wherein said click surface remains displayed at said original position while said at least a portion of said interface device moves in a direction corresponding to movement into said click surface; and

when said at least a portion of said interface device has moved to or past a trigger position past said original position of said click surface, selecting said first graphical object as if said first graphical object had been selected by an activation of a physical input device on said interface device by said user.

31. A method as recited in claim 30 further comprising changing said force on said at least a portion of said interface device to indicate to said user that said first graphical object has been selected.

32. A method as recited in claim 30 wherein said click surface is part of a graphical button having an on state and an off state, wherein said state is toggled when said trigger point is reached by said at least a portion of said interface device.

33. A method as recited in claim 30 wherein said click surface is a portion of a graphical icon in said graphical environment, and wherein said signal to said host computer indicates that said icon has been double-clicked by said user.

34. A method as recited in claim 30 wherein said click surface is a static selection surface in which said cursor always is displayed engaged with said stationary click surface while said at least a portion of said interface device is positioned into and past said click surface.

35. A method as recited in claim 30 wherein said click surface is a static selection surface in which said click surface is always displayed at said engagement position while said cursor tracks said movement of said at least a portion of said interface device and is displayed to move into and past said click surface.

36. A method as recited in claim 30 wherein said at least a portion of said interface device comprises a user object.

37. A method as recited in claim 36 wherein said user object is a mouse and said interface device includes actuators for outputting said force on said mouse, and wherein said physical input device is a button on said mouse that is pressed by said user to be activated.

38. A method as recited in claim 30 wherein said force output to said user comprises a force opposing movement of said at least a portion of said interface device in said direction corresponding to movement into said click surface.

39. A method as recited in claim 30 wherein the force output to said user comprises one or more of a vibration, a texture, and a jolt.

40. A method for providing a force feedback click surface in a graphical environment provided by a host computer, said click surface for selecting a function in a graphical user interface implemented on said host computer, said host computer coupled to a force feedback device, the method comprising:

causing the display of said click surface on a display screen in said graphical environment and receiving position data from said force feedback device describing a position of at least a portion of the force feedback device adapted to be contacted and manipulated by a user in at least one degree of freedom, wherein a user-controlled cursor is displayed at a position in said graphical environment based on said position data;

providing a force command to said force feedback device causing said force feedback device to output a force to the user when said user-controlled cursor contacts said click surface displayed in said graphical environment;

moving a display of said click surface with said cursor when said cursor is moved to positions into and past a point where said cursor first engaged said click surface; and

selecting said function when a trigger point of said click surface positioned past said engagement point of said click surface has been reached by said cursor and thereafter maintaining a display of said click surface at a position other than at a point where said cursor first engaged said click surface, wherein said cursor can be disengaged from said click surface while said display of said click surface is maintained at said position.

41. A method as recited in claim 40 wherein said at least a portion of said force feedback device comprises a user object.

42. A method as recited in claim 40 wherein said user object is a mouse and said force feedback device includes actuators for outputting said force on said mouse.

43. A method as recited in claim 40 wherein said force output to said user comprises a force opposing movement of said at least a portion of said force feedback device in said direction corresponding to movement into said click surface.

44. A method as recited in claim 43 wherein said force opposing movement of said at least a portion of said force feedback device in said direction corresponding to movement into said click surface is proportional to a distance between a present position of said cursor past said click surface and an original position of said click surface.

45. A method as recited in claim 40 wherein the force output to said user comprises one or more of a vibration, a texture, and a jolt.

46. A method as recited in claim 40 further comprising turning off said function when a trigger point has been reached a second time and thereafter maintaining a display of said click surface at a point where said cursor first engaged said click surface.

47. A method as recited in claim 40 wherein said graphical element is a window displayed in said graphical environment and said click surface is at least a portion of an edge of said window.

48. A method as recited in claim 40 wherein said graphical element is a graphical button displayed in a Web page graphical environment and said click surface is an edge of said button.