Force feedback provided over a computer network

• US 6,101,530 A
• Filed: 09/16/1998
• Issued: 08/08/2000
• Est. Priority Date: 12/13/1995
• Status: Expired due to Term

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Rosenberg, Louis B., Tan, Sian W.
• Primary Examiner(s): Geckil, Mehmet B.
• Application Number:
  US09/153,784
• Time in Patent Office: 692 Days
• Field of Search: 709/203, 709/217, 709/230, 709/250, 709/227, 345/335
• US Class Current: 709/203
• CPC Class Codes: A63F 2300/1025 details of the interface with the game device, e.g. USB version detection
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: G06F 2203/013 Force feedback applied to a game
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/03548 Sliders, in which the moving part moves in a plane
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 67/75 Indicating network or usage conditions on the user display
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

A method for providing force feedback over a network supporting TCP/IP protocols by: (a) sending from a client computer over a network supporting TCP/IP protocols, a connection request to a web server connected to the network that is hosting a desired URL; (b) receiving and processing an HTML file at the client computer that was sent from the web server in response to the connection request, wherein the processing includes parsing an embedded force object reference having associated parameters and building a force object therefrom; (c) developing a force feedback signal with the force object; and (d) providing force feedback to a human/computer interface device coupled to the client computer in response to the force feedback signal. A networked force feedback system of the present invention includes a network, a first computer coupled to the network, and a second computer coupled to the network, where the second computer includes a visual display and a human/computer interface device capable of providing a second computer input and providing force feedback in response to a force feedback signal provided by the second computer. The second computer develops an image on the visual display that is associated with stored feedback information, such that the second computer produces the force feedback signal in response to at least one of information derived from the first computer and of the second computer input.

Claims

1. A computer-readable medium including program instructions for implementing force feedback over a network, the program instructions performing acts comprising:

causing a connection to be established between a server machine and a client machine over a network, said client machine including a visual display device and a force feedback device providing computer-controlled physical force feedback to a user of said force feedback device;

receiving from said server machine over said network web page information, said web page information including screen display information representing a visual layout of a web page and force feedback information related to said visual layout of said web page;

causing a display of said web page based upon said screen display information, said display provided on said visual display device of said client machine;

receiving input information from said force feedback device for positioning a pointer image with respect to said visual layout of said web page; and

causing a force feedback signal to be output to said force feedback device, said force feedback signal based upon said input information and based upon said web page information received over said network, wherein said force feedback signal causes said force feedback device to output computer-controlled physical force feedback to said user correlated with said visual layout of said web page on said visual display device, said force feedback being based at least in upon said force-feedback signal.

2. A computer-readable medium as recited in claim 1 wherein said server machine is a web server, said client machine is a network access computer.

3. A computer-readable medium as recited in claim 2 wherein said web server and said network access computer communicate over said network using TCP/IP protocols.

4. A computer-readable medium as recited in claim 2 wherein said input information includes information representing a button click of said force feedback device, said button click information affecting said force feedback.

5. A computer-readable medium as recited in claim 3 wherein said force feedback device includes a user manipulatable object having at least two degrees of freedom.

6. A computer-readable medium as recited in claim 5 wherein said force feedback information includes commands for providing said physical force feedback with said force feedback device.

7. A computer-readable medium as recited in claim 5 wherein said force feedback device is provided with a local microprocessor which communicates with said network access computer, said force feedback device including sensors and actuators coupled to said local microprocessor, and wherein said force-feedback signal is a force feedback command, said method further comprising:

parsing said force feedback command on said local microprocessor to control said actuators of said force feedback device in a control loop with said sensors of said force feedback device.

8. A computer-readable medium as recited in claim 5 further comprising:

developing on said server machine an HTML file including said screen display information and said force-feedback information.

9. A computer-readable medium as recited in claim 8 wherein said HTML file includes an embedded reference that can be used to create a force feedback object on said client machine, said force feedback object including information characterizing forces output by at least one actuator included in said force feedback device, the program instructions further performing:

parsing on said client machine said HTML file and creating on said client machine said force feedback object.

10. A computer-readable medium as recited in claim 9 wherein said causing a force feedback signal to be output includes:

monitoring for said input information from said force feedback device; and

using said force feedback object and said input information to provide said force feedback signal.

11. A computer-readable medium as recited in claim 5, the program instructions further performing:

developing on said server machine a VRML file including said screen display information and said force-feedback information.

12. A computer readable medium as recited in claim 11 further comprising using said input information from said force feedback device to allow said user to move a different object in 3-D space described by said VRML file.

13. A computer-readable medium as recited in claim 1 wherein said server machine is a web server, said client machine is a first network access computer, and further comprising a second network access computer coupled to said network, said second network access computer including a visual display device and a force feedback device, wherein said web server, said first network access computer, and said second network access computer can communicate with said network using TCP/IP protocols.

14. A computer-readable medium as recited in claim 13 wherein said input information allows said first network access computer to position a pointer image with respect to said web page on said visual display of said first network access computer, said input information including button information representing a button click of a force feedback device of at least one of said first network access computer and said second network access computer.

15. A computer readable medium as recited in claim 1 wherein said program instructions implement a network browser program.

16. A method for providing force feedback over a network supporting TCP/IP protocols comprising:

causing a sending of a connection request from a client computer over a network supporting TCP/IP protocols to a web server connected to said network that is hosting a desired URL;

receiving and parsing an HTML file at said client computer, said HTML file including a web page of said web server, said HTML file being sent from said web server in response to said connection request, wherein said parsing includes;

parsing a force object reference included in said HTML file and having associated parameters; and

building a force object based on said force object reference and said associated parameters; and

causing a force feedback signal to be developed and sent to a force feedback device coupled to said client computer, said force feedback signal being developed using said force object, wherein physical force feedback is provided to a user manipulatable object of said force feedback device in response to said force feedback signal, said force feedback being generated by at least one actuator included in said force feedback device.

17. A method for providing force feedback over a network supporting TCP/IP protocols as recited in claim 16 wherein said force object is built using information stored on said client machine in addition to using said force object reference and said associated parameters.

18. A method for providing force feedback over a network supporting TCP/IP protocols as recited in claim 17 wherein a network browser is provided on said client computer to perform said parsing and developing said force feedback signal.

19. A method for providing force feedback over a network supporting TCP/IP protocols as recited in claim 18 wherein said network browser is provided with a plug-in for said parsing of said embedded force object reference and building said force object, and to aid in said development of said force feedback signal.

20. A method for providing force feedback over a network supporting TCP/IP protocols as recited in claim 19 wherein said force feedback device is provided with a plurality of actuators, at least one sensor, and a local microprocessor that communicates with said client computer, and wherein said force feedback signal is a force feedback command sent from said client computer to said force feedback device, and further comprising:

parsing on said local microprocessor said force feedback command to control said actuators in a feedback loop with said sensors.

21. A method for providing force feedback over a network supporting TCP/IP protocols as recited in claim 20 wherein said local microprocessor can interpret JAVA instructions sent from said client computer.

22. A method for providing force feedback over a network supporting TCP/IP protocols as recited in claim 20 wherein said force object is a graphical object having associated forces, said force object being one of a link object, text object, slider object, window object, icon object, and drop-down menu object.

23. A force feedback device coupled to a host computer, said force feedback device providing force feedback from information received over a network, the force feedback device comprising:

a user manipulatable object physically contacted and manipulated by a user and moveable in a degree of freedom;

an actuator for outputting forces on said user manipulatable object in said degree of freedom;

a sensor for detecting a position of said user manipulatable object in said degree of freedom, a representation of said position being provided to said host computer to control a cursor displayed by said host computer; and

a local microprocessor, separate from and communicating with said host computer, and coupled to said actuator and said sensor, said local microprocessor interpreting force feedback commands received from said host computer, said force feedback commands coordinating a force to be output by said actuator with cursor interaction with a displayed element on a web page displayed by said host computer, wherein said force feedback commands and said displayed element are derived from said information received by said host computer from a different computer over said network.

24. A force feedback device as recited in claim 23 wherein said force output by said actuator is based at least in part on a position of said user object in said degree of freedom.

25. A force feedback device as recited in claim 23 wherein said information has been received by said host computer from a server machine connected to said network.

26. A force feedback device as recited in claim 23 wherein said server machine and said host computer communicate over said network using TCP/IP protocols.

27. A force feedback device as recited in claim 23 wherein said force feedback command designates a type of force sensation to output using said actuator.

28. A force feedback device as recited in claim 23 wherein said local microprocessor parses said force feedback command to control said actuator in a control loop with said sensor.

29. A force feedback device as recited in claim 23 further comprising a button input device having a state responsive to manipulation by said user, wherein said state of said button input device is provided to said local microprocessor and from said local microprocessor to said host computer.

30. A force feedback device as recited in claim 23 wherein movement of said user manipulatable object is used to control a position of said cursor displayed on said web page.

31. A force feedback device as recited in claim 30 wherein said user manipulatable object is constrained to move in two planar degrees of freedom, wherein said actuator is a first voice coil actuator, and further comprising a second voice coil actuator, wherein said cursor is controlled by said user manipulatable object to select said displayed element on said web page.

32. A force feedback device as recited in claim 23 wherein said host computer monitors for said position of said user manipulatable object, and wherein said position is used to provide said force feedback command to said local microprocessor.

33. Computer instructions embodied on a computer-readable medium for providing force feedback over a network, said computer instructions performing:

causing a connection request to be sent from a client machine to a server machine over a network, said client machine including a visual display and a force feedback interface device providing computer-controlled force feedback to a user of said force feedback interface device using at least one actuator;

receiving web page information from a server machine over a network, said web page information including screen display information representing a visual layout of a web page and force feedback information related to said visual layout of said web page information, wherein said web page is displayed on a client machine based upon said screen display information;

parsing a component from said received web page information; and

providing said component to a force feedback interpreting process if said component is a force feedback component, wherein said force feedback interpreting process instructs said force feedback device to output a physical force based on an interaction of a user-controlled pointer with objects displayed on said web page.

34. Computer instructions as recited in claim 33, wherein said web page information includes an HTML file, and wherein said component is an HTML component.

35. Computer instructions as recited in claim 33 wherein said force feedback interpreting process includes a plug-in extension.

36. Computer instructions as recited in claim 33 wherein said force feedback interpreting process creates a force object including parameters intepreted from said force feedback component.

37. Computer instructions as recited in claim 33 further comprising interpreting said parsed component if said component is not a force feedback component.

38. A method for implementing force feedback over a network, the method comprising:

causing a connection to be established between a server machine and a client machine over a network, said client machine including a visual display device and a force feedback device providing computer-controlled physical force feedback to a user of said force feedback device;

receiving from said server machine over said network web page information, said web page information including screen display information representing a visual layout of a web page and force feedback information related to said visual layout of said web page;

causing a display of said web page based upon said screen display information, said display provided on said visual display device of said client machine;

receiving input data derived from input information from said force feedback device for positioning a user-controlled graphical object with respect to said visual layout of said web page; and

causing a force feedback signal to be output to said force feedback device, said force feedback signal based upon said input data and based upon said web page information received over said network, wherein said force feedback signal causes said force feedback device to output computer-controlled physical force feedback to said user correlated with said visual layout of said web page on said visual display device, said force feedback being based at least in part upon said force-feedback signal.

39. A method as recited in claim 38 wherein said web page includes a two-dimensional visual layout, and wherein said input data causes said user-controlled graphical object to select a different displayed object in said visual layout.

40. A method as recited in claim 38 wherein said web page includes a 3-D environment, and wherein said input data causes said user-controlled graphical object to move a different displayed object in said 3-D environment.