Networked applications including haptic feedback

• US 20070279392A1
• Filed: 07/10/2003
• Published: 12/06/2007
• Est. Priority Date: 12/01/1995
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Rosenberg, Louis, Brave, Scott, Tan, Sian
• Granted Patent: US 7,636,080 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: A63B 24/00 Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
• CPC Class Codes: A63F 11/0051 Indicators of values, e.g. score counters
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/573 using trajectories of game objects, e.g. of a golf ball according to the point of impact
• CPC Class Codes: A63F 13/577 using determination of contact between game characters or objects, e.g. to avoid collision between virtual racing cars
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: A63F 2300/1025 details of the interface with the game device, e.g. USB version detection
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: A63F 2300/64 for computing dynamical parameters of game objects, e.g. motion determination or computation of frictional forces for a virtual car
• CPC Class Codes: A63F 2300/8011 Ball
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G05G 2009/04766 providing feel, e.g. indexing means, means to create counterforce
• CPC Class Codes: G05G 2009/04777 with additional push or pull action on the handle
• CPC Class Codes: G05G 9/047 the controlling member being movable by hand about orthogonal axes, e.g. joysticks for switches H01H25/04
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 2203/015 Force feedback applied to a joystick
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0383 : Signal control means within the pointing device
• CPC Class Codes: H01H 2003/008 : with a haptic or a tactile feedback controlled by electrical means, e.g. a motor or magnetofriction
• CPC Class Codes: H04L 67/131 : Protocols for games, networked simulations or virtual reality

Abstract

Method and apparatus for providing peer-to-peer force feedback over a computer network. A network force feedback system includes a network, a first computer coupled to the network, and a second computer coupled to the network. The first and second computers each include a visual display and a force feedback interface device. Each computer provides a force feedback signal to its force feedback device based on information received from the other, remote computer and in some cases also based on input from the local force feedback device. Positional information of each force feedback device and/or feel sensation information can be transmitted between the computers over the network. A graphical environment can be displayed to enhance the interaction between users. The present invention therefore permits two computer users to interact using force feedback provided over a network on a peer-to-peer basis.

Claims

1-43. -43. (canceled)

44. An apparatus, comprising:

a network interface;

a peripheral interface; and

a processor coupled to the network interface and the peripheral interface, the processor being associated with a first simulation of a virtual environment including a first virtual object, the processor configured to receive from the network interface a signal associated with a second virtual object within the virtual environment, the processor configured to send to the peripheral interface a signal associated with a haptic feedback based on a virtual interaction between the first virtual object and the second virtual object.

45. The apparatus of claim 44, wherein:

the processor is configured to receive from the peripheral interface a signal associated with a position of a manipulandum, the processor is configured to send to the network interface a signal associated with the first virtual object based on the position of the manipulandum.

46. The apparatus of claim 44, the processor being a first processor, wherein the signal associated with the haptic feedback is configured to compensate within the first simulation for a delay between signals associated with the first virtual object and the signal associated with the second virtual object.

47. The apparatus of claim 44, the processor being a first processor, wherein:

the virtual environment is defined by the first processor and a second processor in communication with the first processor over a network, the first processor defining the first simulation of the virtual environment, the second processor defining a second simulation of the virtual environment, the first simulation substantially corresponding to the second simulation.

48. The apparatus of claim 44, the processor being a first processor, the signal associated with the haptic feedback being a first signal, the apparatus further comprising:

a manipulandum;

an actuator coupled to the manipulandum; and

a second processor coupled to the actuator and the peripheral interface, the second processor configured to receive the first signal from the peripheral interface, the second processor configured to send a second signal to the actuator based on the first signal,

the actuator configured to provide haptic feedback based on the second signal.

49. The apparatus of claim 44, the processor being a first processor, the apparatus further comprising:

a manipulandum having at least one degree of freedom;

an actuator coupled to the manipulandum;

a sensor configured to detect a position of the manipulandum in the at least one degree of freedom; and

a second processor coupled to the sensor and the peripheral interface, the second processor configured to send a position signal to the peripheral interface based on the position of the manipulandum,

the first processor configured to send to the network interface a signal associated with the first virtual object based on the position signal.

50. The apparatus of claim 44, wherein:

the network interface, the peripheral interface and the processor are included within a video game console system, the first simulation and the second simulation being associated with a virtual game environment; and

the network interface being at least one of an Ethernet connection and a modem connection.

51. The apparatus of claim 44, wherein:

the signal associated with the haptic feedback includes a high-level command, the high-level command configured to be interpreted by a local processor to implement a local force routine with a manipulandum.

52. The apparatus of claim 44, wherein:

the signal associated with the haptic feedback includes a positional offset, the positional offset being associated with a difference between the first virtual object and the second virtual object within the first simulation.

53. An apparatus, comprising:

a manipulandum having at least one degree of freedom;

an actuator coupled to the manipulandum;

a sensor configured to detect a position of the manipulandum in the at least one degree of freedom, the position of the manipulandum being associated with a first virtual object within a virtual environment; and

a local processor coupled to the actuator and the sensor, the local processor configured to receive from a host processor a signal associated with a virtual interaction between the first virtual object and a second virtual object within the virtual environment, the local processor configured to send a signal to the actuator based on the signal from the host processor, the virtual environment being defined by the host processor and a remote processor in communication with the host processor over a network.

54. The apparatus of claim 53, wherein:

the local processor is configured to receive from the sensor a signal associated with a position of a manipulandum, the position of the manipulandum being associated with a position of the first virtual object within the virtual environment.

55. The apparatus of claim 53, wherein:

the host processor is associated with a first simulation of the virtual environment;

the remote processor is associated with a second simulation of the virtual environment; and

the signal sent to the actuator is configured to compensate within the first simulation for a delay between signals associated with the first virtual object and the signal associated with the second virtual object.

56. The apparatus of claim 53, wherein:

the virtual environment is defined by the host processor and the remote processor, the first processor defining the first simulation of the virtual environment, the second processor defining a second simulation of the virtual environment, the first simulation substantially corresponding to the second simulation.

57. The apparatus of claim 53, wherein:

the signal from the host processor includes a high-level command, the local processor configured to implement a local force routine based on the high-level command, the signal sent to the actuator being based on the local force routine.

58. The apparatus of claim 53, wherein:

the host processor is associated with a first simulation of the virtual environment;

the remote processor is associated with a second simulation of the virtual environment; and

the signal from the host processor includes a positional offset, the positional offset being associated with a difference between the first virtual object and the second virtual object within the first simulation.

59. A method, comprising:

providing a manipulandum having at least one degree of freedom;

providing an actuator coupled to the manipulandum;

providing a sensor configured to detect a position of the manipulandum in the at least one degree of freedom, the position of the manipulandum being associated with a first virtual object within a virtual environment; and

providing a local processor coupled to the actuator and the sensor, the local processor configured to receive from a host processor a signal associated with a virtual interaction between the first virtual object and a second virtual object within the virtual environment, the local processor configured to send a signal to the actuator based on the signal from the host processor, the virtual environment being defined by the host processor and a remote processor in communication with the host processor over a network.

60. The method of claim 59, wherein:

the local processor is configured to receive from the sensor a signal associated with a position of a manipulandum, the position of the manipulandum being associated with a position of the first virtual object within the virtual environment.

61. The method of claim 59, wherein:

the host processor is associated with a first simulation of the virtual environment;

the remote processor is associated with a second simulation of the virtual environment; and

the signal sent to the actuator is configured to compensate within the first simulation for a delay between signals associated with the first virtual object and the signal associated with the second virtual object.

62. The method of claim 59, wherein:

the virtual environment is defined by the host processor and the remote processor, the first processor defining the first simulation of the virtual environment, the second processor defining a second simulation of the virtual environment, the first simulation substantially corresponding to the second simulation.

63. The method of claim 59, wherein:

the signal from the host processor includes a high-level command, the local processor configured to implement a local force routine based on the high-level command, the signal sent to the actuator being based on the local force routine.

64. The method of claim 59, wherein:

the host processor is associated with a first simulation of the virtual environment;

the remote processor is associated with a second simulation of the virtual environment; and

the signal from the host processor includes a positional offset, the positional offset being associated with a difference between the first virtual object and the second virtual object within the first simulation.

65. A method, comprising:

enabling a first simulation of a virtual environment on a first processor and a second simulation of the virtual environment on a second processor, the first processor being in communication with a haptic feedback device, the second processor being in communication with a haptic feedback device;

enabling the first processor to provide a signal to its haptic feedback device based on an interaction between a first virtual object and a second virtual object within the first simulation, the interaction within the first simulation being based on a position signal from the haptic feedback device of the first processor and a signal associated with the second virtual object from the second processor; and

enabling the second processor to provide a signal to its haptic feedback device based on an interaction between the first virtual object and the second virtual object within the second simulation, the interaction within the second simulation being based on a position signal from the haptic feedback device of the second processor and a signal associated with the first virtual object from the first processor.

66. The method of claim 65, further comprising:

enabling synchronization between the first simulation and the second simulation based, at least in part, on the signal to the haptic feedback device of the first processor and the signal to the haptic feedback device of the second processor.

67. The method of claim 65, wherein:

the first processor is a first video-gaming console, the haptic feedback device associated with the first processor is a first controller; and

the second processor is a second video-gaming console, the haptic feedback device associated with the second processor is a second controller.

68. The method of claim 67, wherein:

the first controller includes a manipulandum, the position signal from the first controller being based on a position of the manipulandum of first controller; and

the second controller includes a manipulandum, the position signal from the second controller being based on a position of the manipulandum of first controller.

69. A system, comprising:

a video-gaming console having

a network interface;

a peripheral interface; and

a host processor coupled to the network interface and the peripheral interface, the host processor being associated with a first simulation of a virtual environment including a first virtual object, the host processor configured to receive from the network interface a signal associated with a second virtual object within the virtual environment, the host processor configured to send to the peripheral interface a signal associated with a haptic feedback based on a virtual interaction between the first virtual object and the second virtual object; and

a controller having

a manipulandum having at least one degree of freedom;

an actuator coupled to the manipulandum;

a sensor configured to detect a position of the manipulandum in the at least one degree of freedom, the position of the manipulandum being associated with the first virtual object within the virtual environment; and

a local processor coupled to the actuator, the sensor and the peripheral interface of the video-gaming console, the local processor configured to receive the signal associated with the haptic feedback from the peripheral interface of the video-gaming console, the local processor configured to send a signal to the actuator based on the signal associated with the haptic feedback,

the actuator configured to provide haptic feedback to the manipulandum based on the signal from the local processor.