Networked applications including haptic feedback

• US 7,636,080 B2
• Filed: 07/10/2003
• Issued: 12/22/2009
• Est. Priority Date: 12/01/1995
• Status: Expired due to Fees

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Rosenberg, Louis B., Tan, Sian W., Brave, Scott B.
• Primary Examiner(s): Liang; Regina
• Application Number:
  US10/615,927
• Publication Number:
  US 20070279392A1
• Time in Patent Office: 2,357 Days
• Field of Search: 345156-158, 345/161, 273/148.B, 463 30- 32, 463 36- 42, 715/701, 715/702, 709/203, 709/217, 709/227, 709/250
• US Class Current: 345/156
• CPC Class Codes: A63B 24/00 Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
• CPC Class Codes: A63F 11/0051 Indicators of values, e.g. score counters
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/573 using trajectories of game objects, e.g. of a golf ball according to the point of impact
• CPC Class Codes: A63F 13/577 using determination of contact between game characters or objects, e.g. to avoid collision between virtual racing cars
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: A63F 2300/1025 details of the interface with the game device, e.g. USB version detection
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: A63F 2300/64 for computing dynamical parameters of game objects, e.g. motion determination or computation of frictional forces for a virtual car
• CPC Class Codes: A63F 2300/8011 Ball
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G05G 2009/04766 providing feel, e.g. indexing means, means to create counterforce
• CPC Class Codes: G05G 2009/04777 with additional push or pull action on the handle
• CPC Class Codes: G05G 9/047 the controlling member being movable by hand about orthogonal axes, e.g. joysticks for switches H01H25/04
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 2203/015 Force feedback applied to a joystick
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0383 : Signal control means within the pointing device
• CPC Class Codes: H01H 2003/008 : with a haptic or a tactile feedback controlled by electrical means, e.g. a motor or magnetofriction
• CPC Class Codes: H04L 67/131 : Protocols for games, networked simulations or virtual reality

Abstract

Method and apparatus for providing peer-to-peer force feedback over a computer network. A network force feedback system includes a network, a first computer coupled to the network, and a second computer coupled to the network. The first and second computers each include a visual display and a force feedback interface device. Each computer provides a force feedback signal to its force feedback device based on information received from the other, remote computer and in some cases also based on input from the local force feedback device. Positional information of each force feedback device and/or feel sensation information can be transmitted between the computers over the network. A graphical environment can be displayed to enhance the interaction between users. The present invention therefore permits two computer users to interact using force feedback provided over a network on a peer-to-peer basis.

Claims

1. An apparatus, comprising:

a network interface;

a peripheral interface; and

a processor coupled to the network interface and the peripheral interface,

the processor configured to receive from the network interface a web page comprising embedded force feedback information,

the processor configured to generate a virtual environment based at least in part on the web page,

the processor configured to execute a force feedback driver software, the force feedback driver software configured to interpret the embedded force feedback information, and

the processor configured to send to the peripheral interface a force feedback signal configured to cause a force feedback effect, the force feedback signal based at least in part on the interpreted force feedback information.

2. The apparatus of claim 1, wherein:

the processor is configured to receive from the peripheral interface a position signal associated with a position of a manipulandum, the processor is configured to send to the network interface a signal associated with a first virtual object in the virtual environment based at least in part on the position of the manipulandum.

3. The apparatus of claim 1, wherein the force feedback signal sent to the peripheral interface is configured to compensate within the virtual environment for a delay.

4. The apparatus of claim 1, the processor being a first processor, wherein:

the virtual environment is defined by the first processor and a second processor in communication with the first processor over a network, the first processor implementing a first simulation of the virtual environment, the second processor implementing a second simulation of the virtual environment, the first simulation substantially corresponding to the second simulation.

5. The apparatus of claim 1, the processor being a first processor, the apparatus further comprising:

a manipulandum;

an actuator coupled to the manipulandum; and

a second processor coupled to the actuator and the peripheral interface, the second processor configured to receive the force feedback signal from the peripheral interface, the second processor configured to send a signal to the actuator based at least in part on the force feedback signal, the actuator configured to generate the force feedback effect based at least in part on the second signal.

6. The apparatus of claim 1, the processor being a first processor, the apparatus further comprising:

a manipulandum having at least one degree of freedom;

an actuator coupled to the manipulandum;

a sensor configured to detect a position of the manipulandum in the at least one degree of freedom; and

a second processor coupled to the sensor and the peripheral interface, the second processor configured to send a position signal to the peripheral interface based at least in part on the position of the manipulandum,

the first processor configured to send to the network interface a signal associated with a first virtual object based at least in part on the position signal.

7. The apparatus of claim 1, wherein:

the network interface, the peripheral interface and the processor are included within a video game console system configured to generate a virtual game environment; and

the network interface comprises an Ethernet connection or a modem connection.

8. The apparatus of claim 1, wherein the force feedback signal comprises a high-level command, the high-level command configured to be interpreted by a local processor to implement a local force routine with a manipulandum.

9. The apparatus of claim 1, wherein:

the force feedback signal comprises a positional offset, the positional offset being associated with a difference between a first virtual object and a second virtual object within the virtual environment generated by the processor.

10. The apparatus of claim 1, wherein the force feedback driver software comprises a browser plug-in or a dynamically linked library.

11. The apparatus of claim 1, wherein the web page is an HTML web page, and the embedded force feedback information is embedded in the HTML web page.

12. The apparatus of claim 1, wherein the embedded force feedback information is included in a separate file.

13. An apparatus, comprising:

a manipulandum having at least one degree of freedom;

an actuator coupled to the manipulandum;

a sensor configured to detect a position of the manipulandum in the at least one degree of freedom, the position of the manipulandum being associated with a first virtual object within a virtual environment; and

a local processor coupled to the actuator and the sensor,

the local processor configured to receive from a host processor a force feedback signal generated by a force feedback driver software executing on the host processor, the force feedback signal based at least in part on a web page received from a remote processor,

the local processor configured to send an actuator signal to the actuator based at least in part on the force feedback signal from the host processor.

14. The apparatus of claim 13, wherein:

the local processor is configured to receive from the sensor a position signal associated with a position of a manipulandum, the position of the manipulandum being associated with a position of the first virtual object within the virtual environment.

15. The apparatus of claim 13, wherein:

the host processor is associated with a first simulation of the virtual environment;

the remote processor is associated with a second simulation of the virtual environment; and

the actuator signal is configured to compensate within the first simulation for a delay between the host processor and the remote processor.

16. The apparatus of claim 13, wherein:

the first processor implements a first simulation of the virtual environment, the second processor implements a second simulation of the virtual environment, the first simulation substantially corresponding to the second simulation.

17. The apparatus of claim 13, wherein:

the force feedback signal received from the host processor includes a high-level command, the local processor configured to implement a local force routine based on the high-level command, the actuator signal sent to the actuator being based at least in part on the local force routine.

18. The apparatus of claim 13, wherein:

the host processor is associated with a first simulation of the virtual environment;

the remote processor is associated with a second simulation of the virtual environment; and

the force feedback signal from the host processor includes a positional offset, the positional offset being associated with a difference between the first virtual object and the second virtual object within the first simulation.

19. A method, comprising:

providing a manipulandum having at least one degree of freedom;

providing an actuator coupled to the manipulandum;

providing a sensor configured to detect a position of the manipulandum in the at least one degree of freedom, the position of the manipulandum being associated with a first virtual object within a virtual environment; and

providing a local processor coupled to the actuator and the sensor, the local processor configured to receive from a host processor a force feedback signal generated by a force feedback driver software executing on the host processor, the force feedback signal based at least in part on a web page received from a remote processor, the local processor configured to send an actuator signal to the actuator based at least in part on the force feedback signal from the host processor.

20. The method of claim 19, wherein:

the local processor is configured to receive from the sensor a position signal associated with a position of a manipulandum, the position of the manipulandum being associated with a position of the first virtual object within the virtual environment executing on the host processor.

21. The method of claim 19, wherein:

the host processor is associated with a first simulation of the virtual environment;

the remote processor is associated with a second simulation of the virtual environment; and

the actuator signal sent to the actuator is configured to compensate within the first simulation for a delay.

22. The method of claim 19, wherein:

the first processor implements a first simulation of the virtual environment, the second processor implements a second simulation of the virtual environment, the first simulation substantially corresponding to the second simulation.

23. The method of claim 19, wherein:

the force feedback signal from the host processor includes a high-level command, the local processor configured to implement a local force routine based on the high-level command, the actuator signal sent to the actuator being based at least in part on the local force routine.

24. The method of claim 19, wherein:

the host processor is associated with a first simulation of the virtual environment;

the remote processor is associated with a second simulation of the virtual environment; and

the force feedback signal from the host processor includes a positional offset, the positional offset being associated with a difference between the first virtual object and the second virtual object within the first simulation.

25. A system, comprising:

a computer having

a network interface;

a peripheral interface; and

a host processor coupled to the network interface and the peripheral interface,

the host processor configured to generate a first simulation of a virtual environment including a first virtual object,

the host processor configured to receive from the network interface a web page comprising embedded force feedback information,

the host processor configured to execute a force feedback driver software, the force feedback driver software configured to interpret the embedded force feedback information, and

the host processor configured to send to the peripheral interface a force feedback signal configured to generate a force feedback effect, the force feedback signal based at least in part on the interpreted haptic feedback information; and

a controller having

a manipulandum having at least one degree of freedom;

an actuator coupled to the manipulandum;

a sensor configured to detect a position of the manipulandum in the at least one degree of freedom; and

a local processor coupled to the actuator, the sensor and the peripheral interface of the computer, the local processor configured to receive the force feedback signal, the local processor configured to send an actuator signal to the actuator based at least in part on the on the force feedback signal,

the actuator configured to generate a force feedback effect to the manipulandum based at least in part on the actuator signal.

26. A computer-readable medium comprising program code to cause a processor to perform the steps of:

receive a web page comprising embedded force feedback information from a network interface;

generate a virtual environment based at least in part on the web page;

execute a force feedback driver software, the force feedback driver software configured to interpret the embedded force feedback information, and

transmit a force feedback signal configured to cause an actuator to generate a haptic feedback effect, the peripheral signal based at least in part on the haptic feedback information.