Implementing force feedback over the World Wide Web and other computer networks

• US 6,161,126 A
• Filed: 02/02/1999
• Issued: 12/12/2000
• Est. Priority Date: 12/13/1995
• Status: Expired due to Term

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Rosenberg, Louis B., Chang, Dean C., Tan, Sian W., Mallett, Jeffrey R., Wies, Evan F.
• Primary Examiner(s): Geckil, Mehmet B.
• Application Number:
  US09/244,622
• Time in Patent Office: 679 Days
• Field of Search: 709/203, 709/217, 709/219, 709/218, 707/500, 707/501, 707/513, 345/163, 345/161, 345/160
• US Class Current: 709/203
• CPC Class Codes: A63F 2300/1025 details of the interface with the game device, e.g. USB version detection
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: G05B 19/00 Programme-control systems
• CPC Class Codes: G06F 2203/013 Force feedback applied to a game
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 2203/0333 Ergonomic shaped mouse for one hand
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/03548 Sliders, in which the moving part moves in a plane
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416
• CPC Class Codes: H04L 67/75 Indicating network or usage conditions on the user display
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

Force feedback is provided to a user of a client computer receiving information such as a web page over a network such as the World Wide Web from a server machine. The client machine has a force feedback interface device through which the user experiences physical force feedback. The web page may include force feedback information to provide authored force effects. Force feedback is correlated to web page objects by a force feedback program running on the client and based on input information from the interface device, the web page objects, and the force feedback information. Generic force effects can also be provided, which are applied uniformly at the client machine to all web page objects of a particular type as defined by user preferences at the client machine. A web page authoring interface is also described that includes the ability to add force sensations to a web page. The user may assign force effects to web page objects and immediately feel how the web page will feel to an end user. A web page is output by the interface, including force information to provide the force effects at a client. The authoring tool can include or access a force design interface for creating or modifying force effects.

Claims

1. A method for providing force feedback over a network comprising:

causing a connection to be established between a server machine and a client machine over a network, said client machine including a visual display device and an interface device providing computer-controlled physical force feedback to a user of said interface device;

receiving web page information from said server machine over said network, said web page information including screen display information representing a visual layout of a web page and force feedback information related to providing a feel sensation correlated with said visual layout;

causing a display of said web page based upon said screen display information, said display provided on said visual display device of said client machine;

receiving input information from said interface device for positioning a displayed cursor with respect to said visual layout of said web page; and

causing a force feedback signal to be output to said interface device, said force feedback signal based upon said input information and based upon said web page information received over said network, wherein said causing said force feedback signal to be output includes a call to a force feedback program running on said client machine that provides said force feedback signal received by said interface device, wherein said interface device outputs computer-controlled physical force feedback to said user correlated with said visual layout of said web page on said visual display device, said force feedback being based upon said force feedback signal.

2. A method for providing force effects for a web page, the method comprising:

determining which of a plurality of web page objects are force web page objects to be associated with at least one force effect, wherein said force web page objects are web page objects having a predefined type, said web page objects to be displayed in a web page, wherein said web page is derived from web page information received from a server machine over a network, said web page information including screen display information representing said web page objects; and

assigning at least one generic force effect to be associated with each of said force web page objects, each of said generic force effects being defined by effect information derived from a client machine, wherein at least one generic force effect associated with a particular one of said force web page objects causes a force signal to be output by said client machine when a user-controlled cursor interacts with said particular force web page object, said cursor and said force web page objects being displayed on said web page by said client machine, wherein said force signal causes an actuator of a force feedback interface device coupled to said client machine to output a force sensation to a user of said force feedback interface device, and wherein said force signal is based on at least a portion of said effect information.

3. A method as recited in claim 2 wherein a force control program running on said client machine detects whether said user-controlled cursor is contacting one of said force web page objects having said predefined type.

4. A method as recited in claim 3 wherein said force control program is a separate application from a web browser displaying said web page on said client machine.

5. A method as recited in claim 2 wherein information describing said predefined types of web page objects to be associated with force effects is stored as user preference information on said client machine.

6. A method as recited in claim 2 wherein said web page information is processed to add force information to said web page to implement said generic force effects, said force information being added before said web page is displayed by said client machine.

7. A method as recited in claim 6 wherein a proxy server processes said web page information to add said force information to said web page, said proxy server connected to said server machine and to said client machine over said network, said proxy server receiving said web page information, determining which of said web page objects are force web page objects, and assigning said at least one generic force effect to each of said force web page objects, wherein said proxy server sends said processed web page information to said client machine so that said web page is displayed on said client machine and said force signal is output to said actuator of said force feedback interface device coupled to said client machine, said force signal being based on said force information in said web page.

8. A method as recited in claim 7 wherein said proxy server receives said effect information from said client machine over said network, said effect information including preferences information to determine which of said web objects in said web page are force web page objects, and to determine the particular generic force effects to assign to said force web objects.

9. A method as recited in claim 7 wherein said proxy server receives a request for said web page information from said client machine, and wherein said proxy server requests said server machine for said web page information to be sent to said proxy server.

10. A method as recited in claim 6 wherein said client machine receives said web page information from said server, determines which of said web page objects are force web page objects, and processes said web page information to add force information to said web page to implement said generic force effects.

11. A method for providing force effects for a web page, the method comprising:

determining which of a plurality of web page objects are associated with one or more authored force effects, wherein effect information specifying said authored force effects is included in web page information received by a client machine from a server machine over a network, wherein said web page information also includes screen display information representing said web page objects to be displayed in a web page on said client machine;

determining which of said web page objects are associated with one or more generic force effects, wherein effect information specifying said generic force effects is stored on said client machine prior to receiving said web page information, and wherein said generic force effects are each applicable to a predefined type of web page object; and

providing force information when a user-controlled cursor interacts with a particular web page object on said web page associated with at least one of said authored force effects or associated with at least one of said generic force effects, wherein said force information causes a force signal to be output to a force feedback interface device coupled to said client machine and causes an actuator of said force feedback interface device to output a force sensation to a user of said force feedback interface device, and wherein said force information is based on said effect information associated with said particular web page object.

12. A method as recited in claim 11 wherein if said web page includes both an authored force effect and a generic force effect, further comprising overriding said authored force effect with said generic force effect such that said generic force effect is output instead of said authored force effect.

13. A method as recited in claim 12 wherein said authored force effect is overridden based on preferences of said user for said generic force effects, said preferences stored on said client machine.

14. A method as recited in claim 11 wherein if said web page includes both an authored force effect and a generic force effect, further comprising overriding said generic force effect with one of said authored force effects if said generic force effect and said authored force effect are associated with the same web page object.

15. A method as recited in claim 11 wherein said effect information for said authored force effect includes information specifying that said effect information specifying said generic effects and stored on said client machine is to be used in implementing said authored force effect.

16. A method as recited in claim 11 wherein said authored force effect is implemented using an ActiveX control referenced by said effect information for said authored force effect.

17. A method as recited in claim 11 wherein properties of said web page object associated with said generic force effect are accessed using Dynamic HTML.

18. A method for implementing a web page authoring interface including the ability to add force sensations to a web page, the method comprising:

causing said web page authoring interface to be displayed on a display device of a host computer, said web page authoring interface displaying a web page including a plurality of web page objects, at least one of said web page objects being a hyperlink object;

receiving input from a user to said web page authoring interface, said input selecting one of said web page objects and selecting a force effect to be associated with said selected web page object; and

outputting a web page including said web page objects and including force information to allow said force effect to be implemented when said web page is displayed by a client machine after being received by said client machine from a server machine over a network, wherein said force effect is commanded by said client machine and output as a force sensation by a force feedback interface device coupled to said client machine, said force feedback interface device including a user manipulatable object graspable and moveable by a user of said client machine.

19. A method as recited in claim 18 wherein said force information is a reference to a separate force effect file, wherein said force effect file is downloaded by said client machine to output said force aid force feedback interface device.

20. A method as recited in claim 18 wherein said force information includes force effect content data characterizing said force effect.

21. A method as recited in claim 18 wherein said web page authoring interface includes a force design interface for creating or modifying said force effect.

22. A method as recited in claim 19 further comprising creating said force effect in said force effect file using a force design interface separate from said web page authoring interface.

23. A method as recited in claim 18 wherein said force effect is output to a force feedback device and experienced by said user of said web page authoring interface when a user-controlled displayed cursor interacts with said web page object assigned to said force effect.

24. A method as recited in claim 18 wherein said web page is in HTML format.

25. A method as recited in claim 24 wherein said web page includes scripting instructions for accessing a force-only control program.

26. A method as recited in claim 24 wherein said web page portrays a two-dimensional image.

27. A method as recited in claim 18 further comprising receiving spatial input from said user to said web page authoring interface, said spatial input spatially designating an area of one of said web page objects to be associated with a selected force effect.

28. A method as recited in claim 27 wherein said spatial input spatially designates an area arbitrarily chosen by said user.

29. A method as recited in claim 18 wherein said user of said web page authoring interface may select a condition in said design interface, said condition determining when said force effect is output based on user input during display of said web page.

30. A method as recited in claim 29 wherein said condition includes a cursor controlled by said user moving over said web page object on said web page.

31. A method as recited in claim 29 wherein said condition includes a button on said user manipulatable object being pressed by said user.

32. A method as recited in claim 18 further comprising receiving input from said user of said web page authoring interface to associate sound data with said selected web page object, such that when said force effect is output, a sound defined by said sound data is output to said user synchronized with said force effect.

33. A method as recited in claim 32 wherein when said force effect is output, a start of said output sound is synchronized to a start of said force effect output.

34. A method for implementing a force-enabled web page authoring interface, the method comprising:

causing said force-enabled web page authoring interface to be displayed on a display device of a host computer, said force sensation design interface displaying a web page including a plurality of web page objects; and

receiving input from a user to said force-enabled web page authoring interface, said input spatially designating an area of at least one of said web page objects to be associated with at least one selected force effect, said at least one selected force effect to be commanded by a client machine receiving said web page and output as a force sensation by a force feedback interface device, said force feedback interface device including a user manipulatable object graspable and moveable by a user of said client machine.

35. A method as recited in claim 34 wherein said input spatially designating an area of said at least one web page object includes input providing a graphical mark on said at least one web page object, wherein said at least one associated force effect is to be output when a user of said client machine moves a user-controlled cursor over a location correlated with said graphical mark.

36. A method as recited in claim 35 wherein said graphical mark includes a graphical outline displayed within a sub-area of said at least one web page object.

37. A method as recited in claim 36 wherein said outline surrounds a visually distinguishable region of said at least one web page object so as to make said visually distinguishable region haptically distinguishable from other regions of said web page object and from other web objects.

38. A method as recited in claim 36 wherein a selected force effect associated with said graphical outline includes a texture effect, said texture effect to be output when a user-controlled cursor moves within an interior region of said outline.

39. A method as recited in claim 35 wherein said graphical mark includes a graphical line displayed on a sub-area of said at least one web page object, wherein a selected force effect associated with said graphical line includes a barrier force resisting motion of a user-controlled cursor through said graphical line.

40. A method as recited in claim 35 wherein said web page object is a two-dimensional pictorial image or text.

41. A method as recited in claim 35 wherein said graphical mark is visually perceptible by said user of said force-enabled web page authoring interface and is visually invisible to said user of said client machine.

42. A method as recited in claim 34 wherein said at least one selected force effect is output as a force sensation to said user of said force-enabled web page authoring interface when said user moves a cursor over said graphical mark.

43. A method as recited in claim 34 wherein said spatial input spatially designates an area arbitrarily chosen by said user.

44. A method as recited in claim 34 further comprising outputting web page information that describes said web page including said at least one web page object associated with said at least one selected force effect, wherein force information is included in said web page to allow said force effect to be implemented when said web page is displayed by a client machine after being received by said client machine from a server machine over a network, wherein said force effect is commanded by said client machine and output as a force sensation by said force feedback interface device coupled to said client machine.

45. A method for implementing a force sensation design interface for a web page, the method comprising:

causing said force sensation design interface to be displayed on a display device of a host computer, said force sensation design interface displaying a web page including a plurality of web page objects;

receiving input from a user to said force sensation design interface, said input associating at least one force effect with at least one of said web page objects, said force effect to be commanded by said host computer and output by a force feedback interface device, said force feedback interface device including a user manipulatable object graspable by a user and moveable in a degree of freedom; and

automatically inserting a predefined graphical identifier into said web page, said identifier indicating to a user that said web page provides force effects for use with a force feedback interface device.

46. A method as recited in claim 45 wherein said predefined graphical identifier is an image of a logo.

47. A method as recited in claim 45 wherein said predefined graphical identifier is a link object that causes a linked force-feedback-related web page to be downloaded on a client computer displaying said web page when said link object is selected with a user-controlled cursor.

48. A method as recited in claim 45 wherein said graphical identifier is automatically inserted into said web page if user preferences stored on said host computer allow said graphical identifier to be inserted.

49. An apparatus for implementing a web page authoring interface including the ability to add force sensations to a web page, the apparatus comprising:

means for receiving input from a user in a web page authoring interface displayed on a display device of a host computer, said web page authoring interface displaying a web page including a plurality of web page objects including a hyperlink object, and wherein said input selects one of said web page objects and selects a force effect to be associated with said selected web page object; and

means for outputting a web page including said web page objects and including force information to allow said force effect to be implemented when said web page is displayed by a client machine after being received by said client machine from a server machine over a network, wherein said force effect is commanded by said client machine and output as a force sensation by a force feedback interface device coupled to said client machine, said force feedback interface device including a user manipulatable object graspable and moveable by a user of said client machine.

50. An apparatus as recited in claim 49 wherein said means for outputting a web page outputs said web page as an HTML file stored in a computer readable medium.

51. An apparatus as recited in claim 49 wherein said means for receiving and said means for outputting include program instructions stored on a computer readable medium.

52. A method for providing force effects for a web page, the method comprising:

receiving web page information from a server machine over a network, said web page information including screen display information representing a plurality of web page objects to be displayed in said web page;

determining which of said web page objects are force web page objects to be associated with at least one force effect, wherein said force web page objects are web page objects having a predefined type; and

assigning a generic force effect for each of said force web page objects, each of said generic force effects being defined by associated effect information derived from a client machine, wherein a generic force effect associated with a particular one of said force web page objects causes a force signal to be output by said client machine when a user-controlled cursor interacts with said particular force web page object, said cursor and said force web page objects being displayed on said web page by said client machine, wherein said force signal causes an actuator of a force feedback interface device coupled to said client machine to output a force sensation to a user of said force feedback interface device, and wherein said force signal is based on said effect information associated with said generic force effect that is assigned to said particular force web page object.