Force effects for object types in a graphical user interface

• US 20030063064A1
• Filed: 10/09/2001
• Published: 04/03/2003
• Est. Priority Date: 11/14/1997
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Rosenberg, Louis B., Braun, Adam C., Beamer, Jonathan L., Chang, Dean C.
• Granted Patent: US 7,168,042 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G05B 19/00 Programme-control systems
• CPC Class Codes: G06F 2203/013 Force feedback applied to a game
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 2203/0333 Ergonomic shaped mouse for one hand
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0338 with detection of limited linear or angular displacement of an operating part of the device from a neutral position, e.g. isotonic or isometric joysticks
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/03548 Sliders, in which the moving part moves in a plane
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/75 Indicating network or usage conditions on the user display
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A force feedback system provides components for use in a force feedback system including a host computer and a force feedback interface device. An architecture for a host computer allows multi-tasking application programs to interface with the force feedback device without conflicts. One embodiment of a force feedback device provides both relative position reporting and absolute position reporting to allow great flexibility. A different device embodiment provides relative position reporting device allowing maximum compatibility with existing software. Information such as ballistic parameters and screen size sent from the host to the force feedback device allow accurate mouse positions and cursor positions to be determined in the force feedback environment. Force feedback effects and structures are further described, such as events and enclosures.

Claims

1. A method for interfacing a multi-tasking graphical environment implemented on a host computer with a force feedback interface device coupled to said host computer, wherein a plurality of application programs are running in said multi-tasking environment, the method comprising:

creating a context associated with each application program running in said multi-tasking graphical environment, said contexts being allocated in memory of said host computer;

receiving force effect commands from said application programs, said force effect commands commanding said force feedback interface device to output a force effect specified by said command;

storing said force effect commands into said contexts, wherein each of said contexts is associated with one of said application programs running on said host computer, and wherein each of said force effect commands is stored in a context associated with said application program that sent said force effect command; and

sending said force effect commands in said context of an application program to said force feedback device when said application program is active in said multi-tasking environment.

2. A method as recited in claim 1 wherein said force effect commands in said contexts of inactive application programs are not sent to said force feedback device.

3. A method as recited in claim 2 wherein when said application program becomes inactive and a new application program becomes active, new force effect commands are sent to said force feedback device to replace said force effect commands, said new force effect commands being included in a context associated with said new application program.

4. A method as recited in claim 1 further comprising a step of sending force effect commands in a context associated with a background application program to said force feedback interface device, wherein said background application program is not active in said multi-tasking environment, such that said force feedback device may output forces based on force effects from said active application program or said background application program.

5. A method as recited in claim 4 wherein only said background application program may send said force effect commands to said force feedback interface device when inactive.

6. A method as recited in claim 1 wherein said contexts further include at least one event that can be triggered by said force feedback device, wherein said event includes conditions based on interactions of a cursor in said graphical environment such that when said conditions are met, an event notification is sent to an application program associated with said event.

7. A method as recited in claim 1 wherein said force effect commands include commands for a spring, a damper, and a vibration.

8. A method as recited in claim 1 wherein said multi-tasking graphical environment includes a Windows™

operating system environment.

9. A method as recited in claim 8 wherein said force effect commands are created as effect objects in an application programming interface in said Windows operating system environment, and wherein said effect objects are sent to said force feedback device when created by said active application program.

10. A method for controlling a force feedback interface device coupled to a host computer displaying a graphical environment and a cursor within said graphical environment on a display screen, the method comprising:

reading sensor data and determining a position of a user manipulandum in a device frame of said force feedback interface device from said sensor data, said manipulandum being physically contacted and moveable by a user;

determining a device delta position of said manipulandum that includes the change in position of said manipulandum from a previous position, and reporting said device delta position to said host computer when said host computer is not enabled to receive a screen position of said cursor from said force feedback interface device; and

when said host computer is enabled to receive said screen position of said cursor from said force feedback interface device,

determining said screen position from said device delta position, said screen position provided in a screen frame of said display screen,

reporting said screen position to said host computer to allow said host computer to display said cursor in said screen frame at said screen position, and

using a scaled position of said manipulandum related to said screen position of said cursor in determining a force to be output by actuators of said force feedback interface device.

11. A method as recited in claim 10 wherein said scaled position of said manipulandum is determined from said delta position, and wherein said screen position of said cursor is determined based on said scaled position.

12. A method as recited in claim 10 wherein said scaled position of said manipulandum is a ballistic position in a ballistic frame, said ballistic frame having a resolution that is a multiple of a resolution of said screen frame.

13. A method as recited in claim 11 wherein said scaled position is determined based on a ballistic delta position, said ballistic delta position being derived from said delta position using a ballistics algorithm.

14. A method as recited in claim 13 wherein said ballistics algorithm uses a velocity of said manipulandum in said device frame in said determination of said ballistic delta position.

15. A method as recited in claim 14 wherein, in determining said ballistic delta position, said ballistics algorithm multiplies said device delta position by one if said velocity of said manipulandum is below a first threshold velocity, by two if said velocity of said manipulandum is between said first threshold and a second threshold velocity, and by four if said velocity of said manipulandum is above said second threshold velocity.

16. A method as recited in claim 13 further comprising receiving ballistic parameters from said host computer, said ballistic parameters characterizing said ballistic algorithm.

17. A method as recited in claim 11 further comprising receiving a current screen resolution of said display screen of said host computer to allow said microprocessor to adjust said screen position based on said screen resolution.

18. A method as recited in claim 11 further comprising receiving a current screen aspect ratio of said display screen of said host computer to allow said microprocessor to adjust said modeled screen position based on said screen aspect ratio.

19. A method as recited in claim 10 wherein said scaled position is determined according to a rate control method for indexing when said manipulandum is moved within a predetermined distance of a limit to said device frame.

20. A method as recited in claim 10 further comprising outputting said force on said manipulandum.

21. A method as recited in claim 10 wherein said delta position is determined from a current manipulandum position and a previous manipulandum position, wherein said current and previous manipulandum positions are determined from measured angles of joints of said force feedback device.

22. A force feedback interface device coupled to a host computer displaying a graphical environment and a user-controlled graphical object within said graphical environment on a display device, the interface device comprising:

a user manipulandum physically contacted by a user and movable in a planar workspace in two degrees of freedom;

a sensor that provides raw data describing a device position of said manipulandum in said workspace;

a plurality of actuators coupled to said manipulandum that provide a force on said manipulandum in said planar workspace; and

a local microprocessor, separate from said host computer and coupled to said sensor and to said actuators, said local microprocessor;

determining a delta position of said manipulandum that includes the change in position of said manipulandum from a previous position, said microprocessor reporting said delta position to said host computer when said host computer is not enabled to receive a screen position;

determining said screen position of said user-controlled graphical object from said delta position, reporting said screen position to said host computer to allow said host computer to display said user-controlled graphical object at said screen position, and using said screen position in determination of said force output by said actuators.

23. A force feedback interface device as recited in claim 22 wherein said screen position is determined from a ballistic position, said ballistic position being determined from said delta position by using a ballistic algorithm, and wherein said force is determined using said ballistic position.

24. A force feedback interface device as recited in claim 22 wherein said determining said screen position is not performed if said host computer is not enabled to receive said screen position.

25. A force feedback interface device as recited in claim 22 wherein said local microprocessor receives a current screen size of said display screen of said host computer to allow said microprocessor to adjust said screen position based on said screen size.

26. A force feedback interface device as recited in claim 23 wherein said local microprocessor receives ballistic parameters from said host computer which characterize said ballistic algorithm.

27. A force feedback interface device as recited in claim 22 wherein said local microprocessor modifies said screen position for indexing when said manipulandum and said cursor position have become offset and when said manipulandum is moved within a predetermined distance of a limit to said device workspace.

28. A force feedback interface device as recited in claim 22 wherein said manipulandum is a mouse.

29. A force feedback interface device coupled to a host computer displaying a graphical environment and a user-controlled graphical object within said graphical environment on a display device, the interface device comprising:

a user manipulandum physically contacted by a user and movable in a planar workspace in two degrees of freedom;

a sensor that provides raw data describing a device position of said manipulandum in said workspace;

a plurality of actuators coupled to said manipulandum that provide a force on said manipulandum in said planar workspace; and

a local microprocessor, separate from said host computer and coupled to said sensor and to said actuators, said local microprocessor;

determining a delta position of said manipulandum that includes the change in position of said manipulandum from a previous position, said microprocessor reporting said delta position to said host computer;

modeling a screen position of said user-controlled graphical object from said delta position and using said modeled screen position in determination of said force output by said actuators; and

receiving an actual screen position of said user-controlled graphical object from said host computer and correcting said modeled screen position using said actual screen position.

30. A force feedback interface device as recited in claim 29 wherein said local microprocessor corrects said modeled screen position by determining an error correction value as the difference between said actual screen position and said modeled screen position.

31. A force feedback interface device as recited in claim 30 wherein said local microprocessor corrects said modeled screen position incrementally over time to smooth out said correction and reduce a perception of said error correction by said user.

32. A force feedback interface device as recited in claim 29 wherein said local microprocessor receives a current screen resolution of said display screen of said host computer to allow said microprocessor to adjust said modeled screen position based on said screen resolution.

33. A force feedback interface device as recited in claim 29 wherein said local microprocessor receives a current screen aspect ratio of said display screen of said host computer to allow said microprocessor to adjust said modeled screen position based on said screen aspect ratio.

34. A force feedback interface device as recited in claim 29 wherein said local microprocessor models said screen position by modeling a delta screen position using a ballistic algorithm and applying said delta screen position to a previous modeled screen position.

35. A force feedback interface device as recited in claim 34 wherein said local microprocessor receives ballistic parameters from said host computer which characterize said ballistic algorithm.

36. A method for controlling a force feedback interface device coupled to a host computer displaying a graphical environment and a cursor within said graphical environment on a display screen, the method comprising:

reading sensor data and determining a position of a user manipulandum in a workspace of said force feedback interface device from said sensor data, said manipulandum being physically contacted and moveable by a user;

reporting a position of said manipulandum to said host computer, said position used by said host to display said cursor in said graphical environment; and

receiving information from said host computer describing a current screen size of said display screen of said host computer, wherein said current screen size is included in a determination of said position reported to said host.

37. A method as recited in claim 36 wherein said position is a change in position of said manipulandum in said workspace.

38. A method as recited in claim 36 wherein said position is an absolute position at which to display said cursor.

39. A method as recited in claim 36 wherein said screen size is a current screen resolution of said display screen.

40. A method as recited in claim 36 wherein said screen size includes information describing a current aspect ratio of said display screen.