Force feedback applications based on cursor engagement with graphical targets

• US 20020050978A1
• Filed: 11/13/2001
• Published: 05/02/2002
• Est. Priority Date: 12/13/1995
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Rosenberg, Louis B., Chang, Dean C.
• Granted Patent: US 7,131,073 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: A63F 2300/1025 details of the interface with the game device, e.g. USB version detection
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: G05G 2009/0477 holding the member in a number of definite positions
• CPC Class Codes: G05G 9/047 the controlling member being movable by hand about orthogonal axes, e.g. joysticks for switches H01H25/04
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0354 with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/03548 Sliders, in which the moving part moves in a plane
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: H01H 2003/008 with a haptic or a tactile feedback controlled by electrical means, e.g. a motor or magnetofriction
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

A method and apparatus for providing a click surface in a graphical environment, such as a graphical user interface, implemented on a host computer for use with a force feedback interface device. A displayed cursor is controlled by a user-moveable user object, such as a mouse, of the interface device. A click surface is displayed with an associated graphical object, such as a graphical button or an edge of a window, icon, or other object. When the click surface is contacted by the cursor, a force is output opposing movement of the user object in a direction into the click surface and into the graphical object. When the user object has moved to or past a trigger position past the contact with the click surface, a command gesture signal is provided to the host computer indicating that the graphical object has been selected as if a physical input device on the user object, such as a button, has been activated by the user. Preferably, the host computer displays the graphical environment including the click surface and cursor, while a microprocessor local to the interface device controls the force output of the click surface in parallel with the host display.

Claims

1. A method of providing a click surface in a graphical environment implemented on a host computer for use with a force feedback interface device coupled to said host computer, the method comprising:

determining when a click surface of a graphical object displayed in a graphical environment is contacted by a user-controlled graphical object, said user-controlled graphical object being displayed at a position corresponding to a position of a physical user object of said interface device grasped by a user and moveable in a degree of freedom;

outputting a force opposing movement of said user object in a direction corresponding to movement into said click surface and into said graphical object; and

when said user object has moved to or past a trigger position past said contact with said click surface, sending a command gesture signal to said host computer indicating said graphical object has been selected as if a physical input device on said user object had been activated by said user.

2. A method as recited in claim 1 wherein said user controlled graphical object is a cursor.

3. A method as recited in claim 1 wherein said user object is determined to have moved to said trigger position when said user object has moved a distance corresponding to predetermined distance past said click surface from said point of contact.

4. A method as recited in claim 1 wherein said force opposing said movement of said user object is proportional to a distance between a present position of said user object past said click surface and an original position of said click surface.

5. A method as recited in claim 4 wherein said force opposing said movement of said user object changes to be proportional to a distance between a present position of said user object past said click surface and a new position of said click surface, said change occurring when said user object moves to said trigger position.

6. A method as recited in claim 1 wherein said graphical object is a graphical button, and wherein said user object moving to said trigger position either toggles said button from an off state to an on state or from an on state to an off state.

7. A method as recited in claim 6 wherein said click surface is displayed at a greater distance from said trigger position when said button is in an off state than when said button is in said on state.

8. A method as recited in claim 1 wherein said user object is a mouse and said interface device includes actuators for outputting said force on said mouse.

9. A method as recited in claim 8 wherein said physical input device is a button on said mouse that is pressed by said user to be activated.

10. A method as recited in claim 1 wherein said click surface is an analog button in which said click surface is displayed moving with said user object when said user object moves into said graphical object.

11. A method as recited in claim 2 wherein said click surface is a positive action button in which said click surface and said cursor remain displayed at an original position while said user object moves past said click surface into said graphical object, and in which said click surface is displayed at a new position when said user object reaches said trigger position.

12. A method as recited in claim 2 wherein said click surface is a static selection surface in which said click surface and said cursor are always displayed at an original position of said click surface while said user object moves past said click surface into said graphical object and when said trigger position is reached.

13. A method as recited in claim 2 wherein said click surface is a static selection surface in which said click surface is always displayed at an original position while said user object and said cursor move past said click surface into said graphical object.

14. A method as recited in claim 2 wherein said click surface hinders said cursor from moving out of contact with said click surface when said cursor moves to the ends of said click surface.

15. A method as recited in claim 1 wherein said signal sent to said host computer indicates that said physical input device has been double-clicked.

16. A method as recited in claim 1 wherein said click surface is an edge of a window displayed in said graphical environment.

17. A method as recited in claim 1 wherein said click surface is a graphical button displayed in a Web page graphical environment.

18. A method as recited in claim 1 wherein said host computer displays said graphical environment including said click surface and said user controlled graphical object, and wherein a microprocessor controls said outputting said force in parallel with said display of said graphical environment, said microprocessor being local to said force feedback interface device and separate from said host computer.

19. A method as recited in claim 18 wherein said local microprocessor sends said command gesture signal to said host computer when said user object has moved to said trigger position.

20. A force feedback interface device allowing interaction with a click surface of a graphical object displayed in a graphical environment implemented by a host computer system, the force feedback interface device comprising:

a user manipulatable object physically contacted by a user and movable in physical space in at least two degrees of freedom with respect to a ground, said user manipulatable object controlling a position of a cursor in said graphical environment;

a sensor operative to detect said movement of said user manipulatable object in physical space in said two degree of freedom with respect to said ground;

an actuator coupled to said user manipulatable object operative to apply an output force in at least one degree of freedom of said user manipulatable object;

a local microprocessor, separate from said host computer system, said local microprocessor reading said sensor, reporting motion of said user object to said host computer, and controlling said actuator, said microprocessor receiving an indication of a click surface of a graphical object from said host computer, and wherein said local microprocessor determines penetration of said cursor into said click surface, wherein said microprocessor outputs a force opposing motion of said user object into said click surface, and wherein said microprocessor sends said host computer a signal indicating said graphical object has been selected as if a physical input device on said user object has been activated by said user, said signal being sent when said user object is moved to or past a trigger position into said click surface.

21. A force feedback interface device as recited in claim 20 wherein said indication received by said microprocessor includes parameters describing said click surface.

22. A force feedback interface device as recited in claim 21 wherein said parameters include a location of said click surface in said graphical environment.

23. A force feedback interface device as recited in claim 22 wherein said microprocessor determines when said cursor contacts said click surface.

24. A force feedback interface device as recited in claim 22 wherein said parameters include a location of said trigger position past said click surface.

25. A force feedback interface device as recited in claim 20 wherein said indication received by said microprocessor is a command from said host computer for said microprocessor to output forces for said click surface at a current position of said user object.

26. A force feedback interface device as recited in claim 25 wherein said host computer indicates a direction and orientation of said click surface with respect to said current position of said cursor.

27. A force feedback interface device as recited in claim 20 wherein said microprocessor clips data reported to said host computer when said user object moves into said click surface, such that data describing motion of said user object in the direction into said click surface is not reported to said host.

28. A force feedback interface device as recited in claim 20 wherein said user object includes a mouse, and wherein said physical input device is a physical button on said mouse.

29. A force feedback interface device as recited in claim 20 wherein said graphical object is a graphical button, and wherein said user object moving to said trigger position either toggles said graphical button from an off state to an on state or from an on state to an off state.

30. A force feedback interface device as recited in claim 20 wherein said force opposing said motion of said user object has a magnitude proportional to a distance between a position of said user object into said click surface and a position of said contact between said cursor and said click surface.

31. A force feedback interface device as recited in claim 20 further comprising a safety switch for deactivating said output force from said actuator when said safety switch is opened.

32. A force feedback interface device as recited in claim 31 wherein, in an indexing function, a mapping between said cursor and said user object is disabled when said safety switch is opened.

33. A force feedback interface device as recited in claim 20 wherein said actuator is a voice coil actuator.

34. A force feedback interface device as recited in claim 33 wherein said user object is coupled to said actuator by a linkage having a plurality of members.

35. A force feedback interface device as recited in claim 20 wherein said interface device and said host computer communicate using a Universal Serial Bus (USB), wherein said actuators receive power from said USB.

37. A method as recited in claim 36 wherein said signal output to said host computer indicating said trigger point has been reached is equivalent to a signal sent to said host computer when a physical button on said user object is pressed by said user.

38. A method as recited in claim 37 wherein said click surface is part of a graphical button having an on state and an off state, wherein said state is toggled when said trigger point is reached by said user object.

39. A method as recited in claim 36 wherein said click surface is a portion of a graphical icon in said graphical environment, and wherein said signal to said host computer indicates that said icon has been clicked or double-clicked by said user.

40. A method as recited in claim 36 wherein said method is performed by a microprocessor, separate from said host computer and local to an interface device that includes said user object.

36. A method for providing a force feedback click surface in a graphical environment provided by a host computer, the method comprising:

outputting a force on a user object grasped and manipulated by a user in a degree of freedom when a cursor controlled by said user object contacts said click surface displayed in said graphical environment, said force opposing motion of said user object that would move said cursor into said click surface;

determining when said user object is moved to a point that would correspond to movement to a trigger point positioned into said click surface past said point of contact between said cursor and said click surface; and

outputting a signal to said host computer indicating said trigger point of said click surface has been reached.

41. An apparatus for providing force feedback click surfaces for a user using an interface device and a host computer system displaying a graphical environment, the apparatus comprising:

means for providing input to said host computer system, wherein said input describes a position of a physical user object in at least one degree of freedom and is used by said host computer system to update a position of a user-controlled graphical object in said graphical environment to correspond to a position of said user object;

means for applying a force to said user object in said at least one degree of freedom when said user-controlled graphical object engages a click surface displayed in said graphical environment at an original position, said force opposing motion of said user object in a direction corresponding to a direction into said click surface; and

means for providing a click signal to said host computer when said user object is moved to a trigger position past said original position of said click surface in a direction opposing said force, said click signal being equivalent to a signal provided to said host computer when a physical input device on said user object is activated by said user.

42. An apparatus as recited in claim 41 wherein said user-controlled graphical object is a cursor and said graphical environment includes a graphical user interface.

43. An apparatus as recited in claim 42 wherein said click surface allows said user to select said at least one program function associated with a graphical object with which said click surface is associated.