Graphical click surfaces for force feedback applications to provide user selection using cursor interaction with a trigger position within a boundary of a graphical object

• US 6,078,308 A
• Filed: 06/18/1997
• Issued: 06/20/2000
• Est. Priority Date: 12/13/1995
• Status: Expired due to Term

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Rosenberg, Louis B., Chang, Dean C.
• Primary Examiner(s): Saras, Steven J.
• Assistant Examiner(s): Bell, Paul A.
• Application Number:
  US08/879,296
• Time in Patent Office: 1,098 Days
• Field of Search: 345/145, 345/146, 345/156, 345/157, 345/158, 345/161, 345/163, 345/339, 345/179, 463/38, 463/30, 318/568.11, 318/568.17, 318/568.19, 318/568.25
• US Class Current: 715/856
• CPC Class Codes: A63F 2300/1025 details of the interface with the game device, e.g. USB version detection
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: G05G 2009/0477 holding the member in a number of definite positions
• CPC Class Codes: G05G 9/047 the controlling member being movable by hand about orthogonal axes, e.g. joysticks for switches H01H25/04
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0354 with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/03548 Sliders, in which the moving part moves in a plane
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: H01H 2003/008 with a haptic or a tactile feedback controlled by electrical means, e.g. a motor or magnetofriction
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

A method and apparatus for providing a click surface in a graphical environment, such as a graphical user interface, implemented on a host computer for use with a force feedback interface device. A displayed cursor is controlled by a user-moveable user object, such as a mouse, of the interface device. A click surface is displayed with an associated graphical object, such as a graphical button or an edge of a window, icon, or other object. When the click surface is contacted by the cursor, a force is output opposing movement of the user object in a direction into the click surface and into the graphical object. When the user object has moved to or past a trigger position past the contact with the click surface, a command gesture signal is provided to the host computer indicating that the graphical object has been selected as if a physical input device on the user object, such as a button, has been activated by the user. Preferably, the host computer displays the graphical environment including the click surface and cursor, while a microprocessor local to the interface device controls the force output of the click surface in parallel with the host display.

Claims

1. A method of providing a click surface in a graphical environment implemented on a host computer for use with a force feedback interface device coupled to said host computer, the method comprising:

determining when a click surface of a first graphical object displayed in a graphical environment is contacted by a user-controlled second graphical object at an original position of said click surface, said user-controlled second graphical object being displayed at a position corresponding to a position of a physical user object of said interface device in a physical object workspace, said user object physically contacted by a user and moveable in a degree of freedom;

outputting a force opposing movement of said user object in a direction corresponding to movement into said click surface and into said first graphical object, wherein said click surface remains displayed at said original position while said user object moves in said direction corresponding to movement into said click surface; and

when said user object has moved to or past a trigger position past said original position said click surface, sending a command gesture signal to said host computer indicating said first graphical object has been selected as if a physical input device on said user object had been activated by said user.

2. A method as recited in claim 1 wherein said user controlled second graphical object is a cursor.

3. A method as recited in claim 2 wherein said click surface is a positive action button in which both said click surface and said cursor remain displayed at said original position while said user object moves past said click surface into said first graphical object, and in which said click surface is displayed at a new position when said user object reaches said trigger position.

4. A method as recited in claim 3 wherein said original position of said click surface is displayed at a greater distance from said trigger position than said new position of said click surface.

5. A method as recited in claim 3 wherein said first graphical object is a graphical button, and wherein said user object moving to said trigger position either toggles said button from an off state to an on state or from an on state to an off state, and wherein said click surface is displayed at a greater distance from said trigger position when said button is in an off state than when said button is in said on state.

6. A method as recited in claim 2 wherein said click surface is a static selection surface in which said click surface and said cursor are always displayed at said original position of said click surface while said user object moves past said click surface into said first graphical object and when said trigger position is reached.

7. A method as recited in claim 2 wherein said click surface is a static selection surface in which said click surface is always displayed at said original position while said user object moves past said click surface and said cursor is displayed to move past said click surface into said first graphical object in accordance with said user object.

8. A method as recited in claim 2 wherein said click surface hinders said cursor from moving out of contact with said click surface when said cursor moves to the ends of said click surface.

9. A method as recited in claim 1 wherein said user object is determined to have moved to said trigger position when said user object has moved a distance corresponding to predetermined distance past said click surface from said point of contact.

10. A method as recited in claim 1 wherein said force opposing said movement of said user object is proportional to a distance between a present position of said user object past said click surface and an original position of said click surface.

11. A method as recited in claim 10 wherein said force opposing said movement of said user object changes to be proportional to a distance between a present position of said user object past said click surface and a new position of said click surface, said change occurring when said user object moves to said trigger position.

12. A method as recited in claim 1 wherein said first graphical object is a graphical button, and wherein said user object moving to said trigger position either toggles said button from an off state to an on state or from an on state to an off state.

13. A method as recited in claim 1 wherein said user object is a mouse and said interface device includes actuators for outputting said force on said mouse.

14. A method as recited in claim 13 wherein said physical input device is a button on said mouse that is pressed by said user to be activated.

15. A method as recited in claim 1 wherein said signal sent to said host computer indicates that said physical input device has been double-clicked.

16. A method as recited in claim 1 wherein said first graphical object is a window displayed in said graphical environment and said click surface is at least a portion of an edge of said window.

17. A method as recited in claim 1 wherein said first graphical object is a graphical button displayed in a Web page graphical environment and said click surface is an edge of said button.

18. A method as recited in claim 1 wherein said host computer displays said graphical environment including said click surface and said user controlled second graphical object, and wherein a microprocessor controls said outputting said force in parallel with said display of said graphical environment, said microprocessor being local to said force feedback interface device and separate from said host computer.

19. A method as recited in claim 18 wherein said local microprocessor sends said command gesture signal to said host computer when said user object has moved to said trigger position.

20. A force feedback interface device allowing interaction with a click surface of a graphical object displayed in a graphical environment implemented by a host computer system, the force feedback interface device comprising:

a user manipulatable object physically contacted by a user and movable in physical space in at least two degrees of freedom with respect to a ground, said user manipulatable object controlling a position of a cursor in said graphical environment;

a sensor operative to detect said movement of said user manipulatable object in physical space in said two degree of freedom with respect to said ground;

an actuator coupled to said user manipulatable object operative to apply an output force in at least one degree of freedom of said user manipulatable object;

a local microprocessor, separate from said host computer system, said local microprocessor reading said sensor, reporting motion of said user manipulatable object to said host computer, and controlling said actuator, said microprocessor receiving an indication of a click surface of a graphical object from said host computer, and wherein said local microprocessor;

determines penetration of said cursor into said click surface and does not report data describing motion of said user manipulatable object in the direction into and past said click surface to said host computer when said user manipulatable object moves into said click surface, such that said click surface is displayed at a constant position and said cursor is displayed to remain at a point of contact with said click surface,

controls said actuator to output a force opposing motion of said user manipulatable object into said click surface, and

sends said host computer a signal indicating said graphical object has been selected as if a physical input device on said user manipulatable object has been activated by said user, said signal being sent when said user manipulatable object is moved to or past a trigger position into said click surface.

21. A force feedback interface device as recited in claim 20 wherein said indication received by said microprocessor includes parameters describing said click surface.

22. A force feedback interface device as recited in claim 21 wherein said parameters include a location of said click surface in said graphical environment and include a location of said trigger position past said click surface.

23. A force feedback interface device as recited in claim 22 wherein said microprocessor determines when said cursor contacts said click surface.

24. A force feedback interface device as recited in claim 20 wherein said host computer indicates to said microprocessor a direction and orientation of said click surface with respect to said current position of said cursor.

25. A force feedback interface device as recited in claim 20 wherein said user manipulatable object includes a mouse, and wherein said physical input device is a physical button on said mouse.

26. A force feedback interface device as recited in claim 20 wherein said force opposing said motion of said user manipulatable object has a magnitude proportional to a distance between a position of said user manipulatable object into said click surface and a position of said contact between said cursor and said click surface.

27. A force feedback interface device as recited in claim 20 further comprising a safety switch for deactivating said output force from said actuator when said safety switch is opened.

28. A force feedback interface device as recited in claim 27 wherein, in an indexing function, a mapping between said cursor and said user manipulatable object is disabled when said safety switch is opened.

29. A force feedback interface device as recited in claim 20 wherein said user manipulatable object is coupled to said actuator by a linkage having a plurality of members, said members rotatable coupled to each other.

30. A force feedback interface device as recited in claim 20 wherein said interface device and said host computer communicate using a Universal Serial Bus (USB), wherein said actuators receive power from said USB.

31. A method as recited in claim 20 wherein said click surface is displayed at a new position after said user object reaches said trigger position.

32. A method for providing a force feedback click surface in a graphical environment provided by a host computer, said host computer coupled to a force feedback device, the method comprising:

causing the display of said click surface on a display screen in said graphical environment and receiving position data from said force feedback device describing a position of a user object contacted and manipulated by a user in at least one degree of freedom, wherein a user-controlled cursor is displayed at a position in said graphical environment based on said position data;

providing a force command to said force feedback device causing said force feedback device to output a force on said user object when said cursor controlled by said user object contacts said click surface displayed in said graphical environment, said force opposing motion of said user object that would move said cursor into said click surface;

maintaining a display of said click surface at a point where said cursor first engaged said click surface while said user object is moved to positions into and past said engagement point of said click surface against said opposing force; and

receiving a signal from said force feedback device indicating a trigger point of said click surface positioned past said engagement point of said click surface has been reached by said user object.

33. A method as recited in claim 32 wherein said signal output to said host computer indicating said trigger point has been reached is equivalent to a signal sent to said host computer when a physical button on said user object is pressed by said user.

34. A method as recited in claim 33 wherein said click surface is part of a graphical button having an on state and an off state, wherein said state is toggled when said trigger point is reached by said user object.

35. A method as recited in claim 32 wherein said click surface is a portion of a graphical icon in said graphical environment, and wherein said signal to said host computer indicates that said icon has been double-clicked by said user.

36. A method as recited in claim 32 wherein said click surface is at least a portion of an edge of a graphical window in said graphical environment.

37. A method as recited in claim 32 wherein said click surface is a positive action button in which said cursor remains displayed engaged with said stationary click surface while said user object moves past said click surface into said graphical object, and in which said click surface is displayed at a new position when said user object reaches said trigger position.

38. A method as recited in claim 37 wherein said graphical object is a graphical button, and wherein said user object moving to said trigger position either toggles said button from an off state to an on state or from an on state to an off state, wherein said click surface is displayed at a greater distance from said trigger position at said original position when said button is in an off state than when said button is in said on state at said new position.

39. A method as recited in claim 32 wherein said click surface is a static selection surface in which said cursor always is displayed engaged with said stationary click surface while said user object is positioned into and past said click surface.

40. A method as recited in claim 32 wherein said click surface is a static selection surface in which said click surface is always displayed at said engagement position while said cursor tracks said movement of said user object and is displayed to move into and past said click surface.

41. An apparatus for providing force feedback click surfaces for a user using an interface device and a host computer system displaying a graphical environment, the apparatus comprising:

means for providing input to said host computer system, wherein said input describes a position of a physical user object in at least one degree of freedom and is used by said host computer system to update a position of a user-controlled graphical object in said graphical environment to correspond to a position of said user object;

means for applying a force to said user object in said at least one degree of freedom when said user-controlled graphical object engages a click surface displayed in said graphical environment at an original position, said force opposing motion of said user object in a direction that causes said graphical object to move into said click surface, wherein said click surface continues to be displayed at said original position when said user object is moved to cause said graphical object to move into said click surface; and

means for providing a click signal to said host computer when said user object is moved to a trigger position past said original position of said click surface in a direction opposing said force, said click signal being equivalent to a signal provided to said host computer when a physical input device on said user object is activated by said user.

42. An apparatus as recited in claim 41 wherein said user-controlled graphical object is a cursor and said graphical environment includes a graphical user interface.

43. An apparatus as recited in claim 42 wherein said click surface allows said user to select said at least one program function associated with a graphical object with which said click surface is associated.