Providing force feedback to a user of an interface device based on interactions of a user-controlled complaint paddle with a simulated object in a graphical environment

• US 9,582,077 B2
• Filed: 02/06/2007
• Issued: 02/28/2017
• Est. Priority Date: 12/01/1995
• Status: Expired due to Fees

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Rosenberg, Louis B., Brave, Scott B.
• Primary Examiner(s): HORNER, JONATHAN R
• Application Number:
  US11/703,322
• Publication Number:
  US 20070139375A1
• Time in Patent Office: 3,675 Days
• Field of Search: 345156-161, 345/184
• US Class Current: 1/1
• CPC Class Codes: A63F 13/21 characterised by their sensors, purposes or types
• CPC Class Codes: A63F 13/23 for interfacing with the game device, e.g. specific interfaces between game controller and console
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 13/57 Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game A63F13/56 takes precedence
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
• CPC Class Codes: A63F 2300/1025 details of the interface with the game device, e.g. USB version detection
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: A63F 2300/308 Details of the user interface
• CPC Class Codes: A63F 2300/64 for computing dynamical parameters of game objects, e.g. motion determination or computation of frictional forces for a virtual car
• CPC Class Codes: A63F 2300/8017 Driving on land or water; Flying
• CPC Class Codes: G05G 2009/04766 providing feel, e.g. indexing means, means to create counterforce
• CPC Class Codes: G05G 2009/04777 with additional push or pull action on the handle
• CPC Class Codes: G05G 9/047 the controlling member being movable by hand about orthogonal axes, e.g. joysticks for switches H01H25/04
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 2203/015 Force feedback applied to a joystick
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0383 Signal control means within the pointing device
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: H01H 2003/008 : with a haptic or a tactile feedback controlled by electrical means, e.g. a motor or magnetofriction
• CPC Class Codes: H04L 67/131 : Protocols for games, networked simulations or virtual reality

Abstract

A method and apparatus for providing force feedback to a user operating a human/computer interface device in conjunction with a graphical user interface (GUI) displayed by a host computer system. A physical object, such as a joystick or a mouse, controls a graphical object, such as a cursor, within the GUI. The GUI allows the user to interface with operating system functions implemented by the computer system. A signal is output from the host computer to the interface device to apply a force sensation to the physical object using one or more actuators. This desired force sensation is associated with at least one of the graphical objects and operating system functions of the graphical user interface and is determined by a location of the cursor in the GUI with respect to targets that are associated with the graphical objects. The graphical objects include icons, windows, pull-down menus and menu items, scroll bars (“sliders”), and buttons. The force sensation assists the user to select a desired operating system function or physically informs the user of the graphical objects encountered by the cursor within the GUI. A microprocessor local to the interface apparatus and separate from the host computer can be used to control forces on the physical object.

Claims

1. A method for providing a user of a manipulandum with haptic feedback simulating an interaction of a simulated object with a simulated compliant paddle whose movement is controlled by movement of the manipulandum, the method comprising:

determining a first haptic force during a first stage of the interaction, the first haptic force based on (i) the interaction, (ii) a simulated mass and stiffness of the simulated compliant paddle, and (iii) a mass of the simulated object, the first stage of the interaction beginning with a collision of the simulated object with the simulated compliant paddle and continuing to a maximum flexibility point of the simulated compliant paddle;

determining a second haptic force during a second stage of the interaction, the second haptic force based on (i) the interaction, (ii) a simulated mass and stiffness of the simulated compliant paddle, and (iii) the mass of the simulated object, the second stage of the interaction corresponding to a rebounding of the simulated compliant paddle and a release of the simulated object from the simulated compliant paddle, the second stage of the interaction beginning in response to the simulated compliant paddle reaching the maximum flexibility point;

actuating one or more actuators that are coupled to the manipulandum to impart the first haptic force to the manipulandum during the first stage of the interaction; and

actuating at least one of the one or more actuators to impart the second haptic force to the manipulandum during the second stage of the interaction.

2. The method of claim 1, further comprising:

rendering the simulated compliant paddle and the simulated object on a display.

3. The method of claim 2, wherein said rendering depicts the interaction of the simulated compliant paddle and the simulated object.

4. The method of claim 3, wherein said rendering depicts deformation of the simulated compliant paddle caused by the interaction.

9. The method of claim 1, wherein the simulated object comprises at least one of a ball or a puck.

16. The method of claim 1, wherein the collision includes a simulated compression of the simulated compliant paddle based on the simulated stiffness, and therein the release includes a decompression of the simulated compliant paddle.

5. A system comprising:

a manipulandum;

at least one actuator coupled to the manipulandum;

a sensor configured to generate sensor signals indicative of a position of the manipulandum; and

a processor configured to;

determine a first haptic force during a first stage of an interaction of a simulated object with a simulated compliant paddle controlled by the position of the manipulandum, the first haptic force based on (i) the interaction, (ii) a simulated mass and stiffness of the simulated compliant paddle, and (iii) a mass of the simulated object, the first stage of the interaction beginning with a collision of the simulated object with the simulated compliant paddle and continuing to a maximum flexibility point of the simulated compliant paddle;

determine a second haptic force during a second stage of the interaction, the second haptic force based on (i) the interaction, (ii) a simulated mass and stiffness of the simulated compliant paddle, and (iii) the mass of the simulated object, the second stage of the interaction corresponding to a rebounding of the simulated compliant paddle and a release of the simulated object from the simulated compliant paddle, the second stage of the interaction beginning in response to the simulated compliant paddle reaching the maximum flexibility point;

actuate the at least one actuator based on the sensor signals and on the interaction of the simulated object and the simulated compliant paddle, said actuation configured to impart to the manipulandum the first and second haptic forces corresponding to the first and second stages of the interaction, respectively.

6. The system of claim 5, wherein the processor is further configured to cause the simulated compliant paddle and the simulated object to be rendered on a display.

7. The system of claim 6, wherein said rendering depicts the interaction of the simulated compliant paddle and the simulated object.

8. The system of claim 7, wherein said rendering depicts deformation of the simulated compliant paddle caused by the interaction.

10. The system of claim 5, wherein the simulated object comprises at least one of a ball or a puck.

17. The system of claim 5, wherein the collision includes a simulated compression of the simulated compliant paddle based on the simulated stiffness, and therein the release includes a decompression of the simulated compliant paddle.

11. A non-transitory computer-readable medium comprising program code for executing a processor-executable method for providing a user of a manipulandum with haptic feedback simulating an interaction of a simulated object with a simulated compliant paddle whose movement is controlled by movement of the manipulandum, the method comprising:

determine a first haptic force during a first stage of the interaction, the first haptic force based on (i) the interaction, (ii) a simulated mass and stiffness of the simulated compliant paddle, and (iii) a mass of the simulated object, the first stage of the interaction beginning with a collision of the simulated object with the simulated compliant paddle and continuing to a maximum flexibility point of the simulated compliant paddle;

determine a second haptic force during a second stage of the interaction, the second haptic force based on (i) the interaction, (ii) a simulated mass and stiffness of the simulated compliant paddle, and (iii) the mass of the simulated object, the second stage of the interaction corresponding to a rebounding of the simulated compliant paddle and a release of the simulated object from the simulated compliant paddle, the second stage of the interaction beginning in response to the simulated compliant paddle reaching the maximum flexibility point;

actuating one or more actuators that are coupled to the manipulandum to impart the first haptic force to the manipulandum during the first stage of the interaction; and

actuating at least one of the one or more actuators to impart the second haptic force to the manipulandum during the second stage of the interaction.

12. The non-transitory computer-readable medium of claim 11, further comprising:

rendering the simulated compliant paddle and the simulated object on a display.

13. The non-transitory computer-readable medium of claim 12, wherein said rendering depicts the interaction of the simulated compliant paddle and the simulated object.

14. The non-transitory computer-readable medium of claim 13, wherein said rendering depicts deformation of the simulated compliant paddle caused by the interaction.

15. The non-transitory computer-readable medium of claim 11, wherein the simulated object comprises at least one of a ball or a puck.

18. The non-transitory computer-readable medium of claim 11, wherein the collision includes a simulated compression of the simulated compliant paddle based on the simulated stiffness, and therein the release includes a decompression of the simulated compliant paddle.