Providing interactions between simulated objects using force feedback

• US 6,366,272 B1
• Filed: 11/03/1999
• Issued: 04/02/2002
• Est. Priority Date: 12/01/1995
• Status: Expired due to Term

• Current Assignee: Immersion Corporation Delaware
• Original Assignee: Immersion Corporation
• Inventors: Rosenberg, Louis B., Brave, Scott B.
• Primary Examiner(s): Saras, Steven
• Assistant Examiner(s): Bell, Paul A.
• Application Number:
  US09/433,657
• Time in Patent Office: 881 Days
• Field of Search: 345/145, 345/146, 345/156, 345/157, 345/158, 345/161, 345/163, 345/326, 345/339, 345/348, 345/352, 345/353, 345/949, 345/952, 345/959, 345/967, 463/30, 463/36, 463/37, 463/38, 318/568.11, 318/568.17, 318/568.19, 318/568.25, 700/83, 700/84, 700/85
• US Class Current: 345/156
• CPC Class Codes: A63B 24/00 Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
• CPC Class Codes: A63F 11/0051 Indicators of values, e.g. score counters
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/573 using trajectories of game objects, e.g. of a golf ball according to the point of impact
• CPC Class Codes: A63F 13/577 using determination of contact between game characters or objects, e.g. to avoid collision between virtual racing cars
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: A63F 2300/1025 details of the interface with the game device, e.g. USB version detection
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: A63F 2300/64 for computing dynamical parameters of game objects, e.g. motion determination or computation of frictional forces for a virtual car
• CPC Class Codes: A63F 2300/8011 Ball
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G05G 2009/04766 providing feel, e.g. indexing means, means to create counterforce
• CPC Class Codes: G05G 2009/04777 with additional push or pull action on the handle
• CPC Class Codes: G05G 9/047 the controlling member being movable by hand about orthogonal axes, e.g. joysticks for switches H01H25/04
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 2203/015 Force feedback applied to a joystick
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0383 : Signal control means within the pointing device
• CPC Class Codes: H01H 2003/008 : with a haptic or a tactile feedback controlled by electrical means, e.g. a motor or magnetofriction
• CPC Class Codes: H04L 67/131 : Protocols for games, networked simulations or virtual reality

Abstract

A method and apparatus for providing force feedback to a user operating a human/computer interface device and interacting with a computer-generated simulation. In one aspect, a computer-implemented method simulates the interaction of simulated objects displayed to a user who controls one of the simulated objects manipulating a physical object of an interface device. The position of the simulated object, as provided within the simulation and as displayed, is mapped directly to the physical position of the user object. This mapping is broken under conditions that are effective to provide force feedback to the user which imparts a physical sensation corresponding to the interaction of the simulated objects. In another aspect, a ball simulated ball object interacts with a user-controlled simulated object in a simulation to allow the user to utilize a wide range of physical skill and dexterity in interacting with the simulation. In another aspect, a simulation apparatus provides a display device such as one or more display screens or a projection device, and which also provides an intuitive mechanical interface device for the user to skillfully and dexterously manipulate objects within a computer-generated simulation.

Claims

1. A computer readable medium including program instructions for simulating the spatial interaction of a displayed first simulated object with a displayed second simulated object in a computer-simulated spatial environment such that the user is provided with a force feedback that realistically represents said interaction, said program instructions performing the following on a computer system:

executing a simulation including a first simulated object, said simulation being configured to implement the motion of said first simulated object in response to motion of a physical object of an interface device controlled by a user, wherein said physical object has a physical position in a physical workspace, and wherein a position control mapping between said simulated location of said first simulated object and said physical position of said physical object exists, said simulation being further configured to generate a second simulated object having boundaries such that said second simulated object impedes the simulated motion of said first simulated object when the trajectory of said first simulated object intersects said boundaries of said second simulated object;

providing information causing a display device to display the location and motion of said first simulated object and said second simulated object; and

providing information causing a force feedback mechanism to impart to a user of said force feedback mechanism a physical sensation that corresponds to the simulated physical interaction of said first simulated object with said second simulated object when the trajectory of said first simulated object intersects the boundaries of said second simulated object, wherein said interaction is determined by examining a current location and a previous location of said first simulated object to determine said trajectory, and wherein said position control mapping between said physical object and said first simulated object is broken during said interaction when providing said physical sensation.

2. The computer readable medium of claim 1, wherein said physical sensation includes a restoring force that is proportional to the magnitude by which said mapping is broken when said first simulated object intersects said boundaries of said second simulated object.

3. The computer readable medium of claim 2, wherein said restoring force includes a spring force having the mathematical form:

 

F=kx

where F is said restoring force, x is a magnitude of a deviation of said spatial correlation including a deviation between the current location of the first simulated object and a location of said first simulated object had said mapping not been broken, and k is a spring constant parameter.

4. The computer readable medium of claim 3, wherein said restoring force includes a damping force and said restoring force has the mathematical form:

5. The computer readable medium of claim 4, wherein said restoring force includes an intertial force corresponding to the movement of said second simulated object in response to said interaction between said second simulated object and said first simulated object and said restoring force has the mathematical form:

6. The computer readable medium of claim 2, wherein said restoring force includes a component resulting from friction between said simulated object and said simulated spatial environment.

7. The computer readable medium of claim 2, wherein said second simulated object moves on said display device during said simulation in response to manipulations of a second physical object of a second interface device by said second user, said second interface device being coupled to a second computer system coupled to said computer system through a network interface.

8. The computer readable medium of claim 2, wherein said restoring force includes a weighting factor such that the location L on said display device of the simulated objects shown on said display device is determined by the equation:

$L=\frac{\left(w_1<br><br>x_1+w_2<br><br>x_2\right)}{\left(w_1+w_2\right)}.$

9. The computer readable medium of claim 1, wherein said processor is coupled with a second processor executing said simulation, said second processor being responsive to input from a second interface device, said processors being coupled such that said simulations communicate input information from said interface devices.

10. A method for providing an interaction between displayed objects in a graphical environment implemented by a host computer, wherein a user interfaces with said graphical environment using a force feedback device coupled to said host computer, the method comprising:

moving a first graphical object in response to movement of a user manipulatable object of said force feedback device by said user, said movement of said first graphical object provided according to a position control mapping to said movement of said user manipulatable object;

determining whether said first graphical object has engaged a second graphical object by examining a path of said first graphical object in said graphical environment, said path determined by examining a current location of said first graphical object and a previous location of said first graphical object;

breaking said position control mapping and providing an illusion of rigidity of said second graphical object by displaying said first graphical object as remaining engaged with said second graphical object when said path of said first graphical object has been determined to move through said second graphical object if said position control mapping were maintained; and

providing information that causes said force feedback device coupled to said host computer to output an opposing force on said user manipulatable object by at least one actuator in said force feedback device in a direction approximately opposite to said movement of said user manipulatable object while said first graphical object is engaged with said second graphical object.

11. A method as recited in claim 10 wherein said opposing force is a restoring spring force.

12. A method as recited in claim 10 wherein said second graphical object is fixed in location within said graphical environment.

13. A method as recited in claim 10 wherein said user is a first user, and wherein said second graphical object is moveable according to input from a second user of a second force feedback device coupled to said host computer.

14. A method as recited in claim 13 wherein said host computer is a first host computer, and wherein said second force feedback device is coupled to a second host computer which is coupled to said first host computer via a network.

15. A method as recited in claim 14 wherein said network is the World Wide Web.

16. A method as recited in claim 10 wherein a friction force is output on said user manipulatable object when said user manipulatable object is moved in a direction corresponding to a direction approximately perpendicular to said path of engagement of said first graphical object while said first and second graphical objects are engaged.

17. A method as recited in claim 16 wherein said friction force has a magnitude that is a function of said magnitude of said opposing spring force.

18. A method for providing an interaction between displayed objects in a graphical environment implemented by a host computer, wherein a user interfaces with said graphical environment using a force feedback device coupled to said host computer, the method comprising:

(a) moving a first graphical object in response to movement of a user manipulatable object of said force feedback device by said user, said movement of said first graphical object provided according to a position control mapping to said movement of said user manipulatable object;

(b) determining whether said first graphical object has engaged a second graphical object by examining a path of said first graphical object in said graphical environment, said path determined by examining a current location of said first graphical object and a previous location of said first graphical object; and

(c) providing information that causes said force feedback device coupled to said host computer to output;

(i) an opposing force on said user manipulatable object by at least one actuator in said force feedback device when said user manipulatable object is moved in a direction corresponding to a direction approximately opposite to said path of said first graphical object while said first graphical object is engaged with said second graphical object; and

(ii) a friction force on said user manipulatable object by at least one actuator in said force feedback device when said user manipulatable object is moved in a direction corresponding to a direction approximately perpendicular to said path of engagement of said first graphical object while said first and second graphical objects are engaged.

19. A method as recited in claim 18 wherein said opposing force is a spring force, and wherein said friction force has a magnitude that is a function of said magnitude of said opposing spring force.

20. A method as recited in claim 18 further comprising breaking said position control mapping and providing an illusion of rigidity of said second graphical object by displaying said first graphical object as remaining engaged with said second graphical object when said path of said first graphical object has been determined to move through said second graphical object if said position control mapping were maintained.