Method and apparatus for controlling non-computer system devices by manipulating a graphical representation

• US 5,657,221 A
• Filed: 11/04/1994
• Issued: 08/12/1997
• Est. Priority Date: 09/16/1994
• Status: Expired due to Fees

• Current Assignee: Medialink Technologies Corporation
• Original Assignee: Medialink Technologies Corporation
• Inventors: Warman, David J., Lacas, Mark A., Coco, Geoffrey P.
• Primary Examiner(s): Gordon, Paul P.
• Application Number:
  US08/334,416
• Time in Patent Office: 1,012 Days
• Field of Search: 364/140, 364/141, 364/145, 364/146, 364/188, 364/189, 364/191, 395/155, 395/161, 395/169
• US Class Current: 700/83
• CPC Class Codes: G06F 8/34 Graphical or visual programming

Abstract

The graphical control system of the present invention includes a computer (20), a device interface (35) for a non-computer system device (21) having at least one feature control (29a) (or display (29b)), a bus network (28) connecting the computer (20) to the device interface (35), and a visual network operating system (78) based on an object-oriented programming paradigm. The device interface (35) connects the non-computer system device (21) to the bus network (28) and provides the mechanism for converting computer-generated commands into signals for controlling the operation of the feature control (29a) of the non-computer system device (21). The visual network operating system (78) is a distributed operating system that is partially stored on the computer (20) and partially stored in the device interface (35). A computer portion causes the computer (20) to generate or create a visual device control (40) that graphically replicates the feature control (29a) normally associated with the non-computer system device (21). The visual device control (40) is operated by a conventional graphical control device, such as a mouse (32), track ball, touch screen, joystick, etc. As the replicated controls are operated, the computer (20) sends messages to the device interfaces (35), which cause the feature control (29a) of the non-computer system device (21) to respond in the same way it would have responded had the feature control been operated manually, electrically or effected by another computer on the bus network (28).

Claims

1. A graphical control system for controlling at least one non-computer system device, the graphical control system comprising:

(a) a communication medium;

(b) a computer coupled to the communication medium, the computer including a display, a processor, a storage medium and a user manipulation device;

(c) an interface for coupling the non-computer system device to the communication medium, the interface including an interface processor and an interface storage medium; and

(d) a visual network operating system for controlling and monitoring operation of the computer and the interface by;

(i) causing the computer to display an adjustable visual device control, the visual device control graphically representing a feature control of a type normally associated with the non-computer system device, the visual device control having associated data stored in the computer storage medium that represents the adjustment of the visual device control;

(ii) enabling a user to adjust the visual device control by operating the user manipulation device upon the visual device control and effectuating a change in the data that represents the adjustment of the visual device control;

(iii) communicating the change in the data that represents the adjustment of the visual device control to the interface;

(iv) causing the interface to adjust the non-computer system device in accordance with the change in the data that represents the adjustment of the visual device control;

(v) detecting an adjustment to the feature control associated with the non-computer system device, the feature control having associated data stored in the interface storage medium that represents the adjustment to the feature control;

(vi) effectuating a change in the data that represents the adjustment of the feature control;

(vii) communicating the change in the data representing the adjustment of the feature control to the computer; and

(viii) causing the computer to regenerate any visual device control representing the feature control in accordance with the change in the data that represents the adjustment of the feature control.

2. The graphical control system of claim 1, wherein the visual network operating system enables the user to predefine a function for further manipulating the change in data that represents the adjustment of the visual device control.

3. The graphical control system of claim 2, wherein the visual network operating system enables the user to create a clone of the visual device control which also graphically represents the feature control, wherein the clone has the same associated data stored in the computer storage medium as the visual device control.

4. The graphical control system of claim 3, wherein the visual network operating system enables the user to effectuate a change in the data that represents the adjustment of the visual device control by operating the user manipulation device upon the clone.

5. The graphical control system of claim 3, wherein the visual network operating system enables the user to graphically alter the clone without effectuating a change in the data that represents the adjustment of the visual device control and without graphically altering the visual device control.

6. The graphical control system of claim 3, wherein the visual network operating system enables the user to create a plurality of clones of the visual device control so that each clone graphically represents the feature control, each clone having the same associated data as the visual device control.

7. The graphical control system of claim 1, wherein the visual device control has an adjustable visual device control element, wherein the adjustable visual device control element moves linearly, and wherein the adjustable visual device control element moves vertically when a height of the visual device control is greater than a width of the visual device control.

8. The graphical control system of claim 7, wherein the visual device control element of the visual device control moves horizontally when the width of the visual device control is greater than the height of the visual device control.

9. The graphical control system of claim 8, wherein the visual network operating system:

(a) enables the user to increase the width of the visual device control, while decreasing the height of the visual device control by operating the user manipulation device; and

(b) causes the computer to automatically regenerate the visual device control so that the adjustable visual device control element of the visual device control moves horizontally when the width of the visual device control becomes greater than the height.

10. The graphical control system of claim 9, wherein the visual network operating system:

(a) enables the user to increase the height of the visual device control, while decreasing the width of the visual device control by operating the user manipulation device; and

(b) causes the computer to automatically regenerate the visual device control so that the adjustable visual device control element of the visual device control moves vertically when the height of the visual device control becomes greater than the width.

11. The graphical control system of claim 1, wherein the visual network operating system causes the computer to display a template visual device control, the template visual device control graphically representing a hypothetical feature control of the type normally associated with a hypothetical non-computer system device.

12. The graphical control system of claim 11, wherein the visual network operating system enables the user to create an adjustable master visual device control from the template visual device control by operating the user manipulation device, the master visual device control graphically representing the hypothetical feature control, the master visual device control having associated data stored in said computer storage medium that represents the adjustment of the master visual device control.

13. The graphical control system of claim 12, wherein the visual network operating system enables the user to govern the visual device control using the master visual device control by:

(a) enabling the user to graphically create an interaction between the visual device control and the master visual device control by operating the user manipulation device;

(b) as a result of the interaction, establishing a relationship between the master visual device control and the visual device control so that;

(i) operating the user manipulation device upon the master visual device control effectuates a change in the data representing the adjustment of the master visual device control; and

(ii) the change in the data representing the adjustment of the master visual device control effectuates a similar change in the data representing the adjustment of the visual device control.

14. The graphical control system of claim 13, wherein the visual network operating system enables the user to govern a plurality of visual device controls using the master visual device control.

15. The graphical control system of claim 12, wherein the visual network operating system enables the user to predefine a function for further manipulating the change in data representing the adjustment of the master visual device control so that the manipulated change in the data representing the adjustment of the master visual device control effectuates a similar manipulated change in the data representing the adjustment of the visual device control.

16. The graphical control system of claim 1, wherein the visual network operating system employs an object-oriented programming paradigm in which a plurality of objects are used to control the computer, and wherein the objects are interoperable and organized into classes in a hierarchical fashion.

17. The graphical control system of claim 16, wherein five types of objects are employed by the visual network operating system for controlling the computer, the five types of objects comprising:

(a) a window object containing the data and methods necessary to display a window on the display of the computer;

(b) a visual reference object containing the data and methods necessary for displaying the visual device control in the window;

(c) a value control element object containing the data representing the adjustment of the visual device control, and methods necessary for interpreting and storing the data representing the adjustment of the visual device control;

(d) a device object containing the data and methods necessary for communicating with the non-computer system device and for managing a graphical representation of the non-computer system device on the display of the computer; and

(e) a packet object containing the data and methods necessary for communicating the change in data that represents the adjustment of the visual device control to the interface.

18. The graphical control system of claim 17, wherein each type of object may be divided into lower order subclasses of objects.

19. The graphical control system of claim 17, wherein the visual reference object further contains the data and methods necessary for displaying an icon representing the non-computer system device in a window on the display of the computer.

20. The graphical control system of claim 17, wherein the methods of each of the five types of objects further comprise generic methods and class-specific methods.

21. The graphical control system of claim 20, wherein the visual network operating system enables the user to load the device object, value control element object and visual reference object with the class-specific methods.

22. The graphical control system of claim 20, wherein the genetic methods provide for translation of the data contained by the window object, visual reference object and packet object into text that is stored in the computer storage medium.

23. The graphical control system of claim 20, wherein the genetic methods provide for communication between the five types of objects.

24. The graphical control system of claim 20, wherein the genetic methods provide for storage of the data contained by the window object, visual reference object and device object in the computer storage medium when the computer is disabled.

25. The graphical control system of claim 24, wherein the genetic methods provide for restoration of the data contained by the window object, visual reference object and device object from the computer storage medium when the computer is enabled.

26. The graphical control system of claim 20, wherein the class-specific methods of the window object provide for placement of the visual device control in the window displayed on the display of the computer.

27. The graphical control system of claim 20, wherein the class-specific methods of the visual reference object provide for unique graphical regeneration of the visual device control.

28. The graphical control system of claim 20, wherein the class-specific methods of the value control element object provide for further manipulation of the data representing the adjustment of the visual device control.

29. The graphical control system of claim 20, wherein the class-specific methods of the value control element object provide the logic for governing the visual device control using another visual device control.

30. The graphical control system of claim 20, wherein the class-specific methods of the device object provide for managing a graphical representation of the non-computer system device in the window.

31. The graphical control system of claim 20, wherein the class-specific methods of the packet object provide for communication of the data representing the adjustment of the visual device control between the computer and the interface of the non-computer system device via the communication medium.

32. The graphical control system of claim 1, further comprising a plurality of interfaces for coupling a plurality of non-computer system devices to the communication medium so that the graphical control system controls a plurality of non-computer system devices.

33. The graphical control system of claim 1, wherein the visual network operating system causes the computer to display a plurality of adjustable visual device controls, each visual device control graphically representing a feature control of a type normally associated with the non-computer system device, each visual device control having associated data stored in the computer storage medium that represents the adjustment of the visual device control.

34. The graphical control system of claim 1, wherein the communication medium is a network.

35. The graphical control system of claim 1, wherein the communication medium is a point-to point connection between a computer and a single non-computer system device.

36. The graphical control system of claim 1, wherein the visual network operating system employs an object-oriented programming paradigm in which a plurality of objects are used to control the interface.

37. The graphical control system of claim 36, wherein two types of objects are used for controlling the interface, said two types of objects comprising:

(a) a device control element object containing the data representing the adjustment of the feature control, and methods necessary for further interpreting and storing the data representing the adjustment of the feature control; and

(b) an interface packet object containing data and methods necessary for communicating the change in data that represents the adjustment of the feature control to the computer.

38. The graphical control system of claim 37, wherein the methods of each of the two types of objects further comprise generic methods and class-specific methods.

39. The graphical control system of claim 38, wherein the visual network operating system enables the user to load the class-specific methods into the device control element object.

40. The graphical control system of claim 39, wherein the class-specific methods of the device control element object provide for further manipulation of the data representing the adjustment of the feature control.

41. The graphical control system of claim 39, wherein the class-specific methods of the interface packet object provide for communication of the data representing the adjustment of the feature control between the interface and the computer via the communication medium.

42. A method for controlling at least one non-computer system device including at least one device control, the device control governing an aspect of the non-computer system device, wherein the non-computer system device is coupled to a communication medium by an interface and connected via the communication medium to a computer system, wherein the computer system comprises a display, a processor, a storage medium and a user manipulation device, the method comprising:

(a) causing the computer to display a graphical control display representing the non-computer system device, the graphical control display having at least one adjustable graphical control, the graphical control representing a feature control of a type normally associated with the non-computer system device, the graphical control having associated data stored in the computer storage medium that represents the adjustment of the graphical control;

(b) enabling a user to adjust the graphical control by operating the user manipulation device upon the graphical control and effectuating a change in the data that represents the adjustment of the graphical control;

(c) communicating the change in the data that represents the adjustment of the graphical control to the interface;

(d) causing the interface to adjust the non-computer system device in accordance with the change in the data that represents the adjustment of the graphical control;

(e) detecting an adjustment to the feature control associated with the non-computer system device, the feature control having associated data stored in the interface storage, medium that represents an adjustment of the feature control;

(f) effectuating a change in the data that represents the adjustment of the feature control when the adjustment to the feature control is detected;

(g) communicating the change in the data representing the adjustment of the, feature control to the computer; and

(h) causing the computer to regenerate any graphical controls representing the feature control in accordance with the change in the data that represents the adjustment of the feature control.

43. The method of claim 42, further comprising predefining a function for further manipulating the change in data that represents the adjustment of the graphical control before communicating the change in data to the interface.

44. The method of claim 43, wherein enabling the user to adjust the graphical control further comprises creating a clone of the graphical control so that the clone graphically represents the feature control, the clone having the same associated data as the graphical control.

45. The method of claim 44, wherein creating the clone further comprises enabling the user to effectuate a change in the data that represents the adjustment of the graphical control by operating the user manipulation device upon the clone.

46. The method of claim 45, wherein creating the clone further comprises enabling the user to graphically alter the clone in accordance with a predefined function, wherein the predefined function does not effectuate a change in the data that represents the adjustment of the graphical control, and wherein the predefined function does not graphically alter the graphical control.

47. The method of claim 44, wherein enabling the user to adjust the graphical control further comprises creating a plurality of clones of the graphical control so that each clone graphically represents the feature control, each clone having the same associated data as the graphical control.

48. The method of claim 42, further comprising causing the computer to display a template graphical control, said template graphical control graphically representing a hypothetical feature control of a type normally associated with a hypothetical non-computer system device.

49. The method of claim 48, further comprising enabling the user to create an adjustable master graphical control from the template graphical control by operating the user manipulation device, the master graphical control graphically representing the hypothetical feature control, the master graphical control having associated data stored in said computer storage medium that represents the adjustment of the master graphical control.

50. The method of claim 49, further comprising enabling the user to use the master graphical control to govern the graphical control, wherein enabling the user to use the master graphical control comprises:

(a) enabling the user to graphically create an interaction between the graphical control and the master graphical control by operating the user manipulation device;

(b) as a result of the interaction, establishing a relationship between the master graphical control and the graphical control;

(c) as a result of the relationship between the master graphical control and the graphical control, enabling the user to effectuate a change in the data representing the adjustment of the master graphical control by operating the user manipulation device, wherein the change in data representing the adjustment of the master graphical control also effectuates a similar change in the data representing the adjustment of the graphical control.

51. The method of claim 50, wherein enabling the user to effectuate a change in the data representing the adjustment of the master graphical control further comprises enabling the user to predefine a function for additionally manipulating the change in the data representing the adjustment of the master graphical control.

52. The method of claim 49, further comprising enabling the user to use the master graphical control to govern a plurality of graphical controls, wherein enabling the user to use the master graphical control comprises:

for each graphical control,

(a) enabling the user to graphically create an interaction between the master graphical control and the graphical control by operating the user manipulation device;

(b) as a result of the interaction, establishing a relationship between the master graphical control and the graphical control;

(c) as a result of the relationship between the master graphical control and the graphical control, enabling the user to effectuate a change in the data representing the adjustment of the master graphical control by operating the user manipulation device, wherein the change in data representing the adjustment of the master graphical control also effectuates a similar change in the data representing the adjustment of the graphical control.

53. The method of claim 42, further comprising:

(a) causing the computer to display the graphical control display representing the non-computer system device, the graphical control display having a plurality of adjustable graphical controls, each graphical control representing a feature control of a type normally associated with the non-computer system device, each graphical control having associated data stored in the computer storage medium that represents the adjustment of each graphical control; and

(b) enabling a user to adjust each graphical control by operating the user manipulation device upon each graphical control and effectuating a change in the data that represents the adjustment of each graphical control.

54. The method of claim 42, further comprising predefining a function for further manipulating the change in data representing the adjustment of the feature control before communicating the change in data to the computer system.

55. The method of claim 42, further comprising causing the computer to display a plurality of graphical control display each graphical control displays representing at least one of a plurality of non-computer system devices, wherein each non-computer system device is coupled to the communication medium by an interface.

56. A graphical control system for controlling at least one non-computer system device, the graphical control system comprising:

(a) a communication medium;

(b) a computer coupled to the communication medium, the computer including a display, a processor, a storage medium and a user manipulation device;

(c) an interface for coupling the non-computer system device to the communication medium, the interface including an interface processor mad an interface storage medium; and

(d) a visual network operating system for controlling and monitoring operation of the computer and the interface by;

(i) causing the computer to display an adjustable visual device control, the visual device control graphically representing a feature control of a type normally associated with the non-computer system device, the visual device control having associated data stored in the computer storage medium that represents the adjustment of the visual device control;

(ii) enabling a user to adjust the visual device control by operating the user manipulation device upon the visual device control and effectuating a change in the data that represents the adjustment of the visual device control;

(iii) communicating the change in the data that represents the adjustment of the visual device control to the interface;

(iv) causing the interface to adjust the non-computer system device in accordance with the change in the data that represents the adjustment of the visual device control; and

(v) causing the computer to display a template visual device control, the template visual device control graphically representing a hypothetical feature control of a non-computer system device that is not controlled and monitored by the visual network operating system.

57. The graphical control system of claim 56, wherein the visual network operating system enables the user to create an adjustable master visual device control from the template visual device control by operating the user manipulation device, the master visual device control graphically representing the hypothetical feature control, the master visual device control having associated data stored in said computer storage medium that represents the adjustment of the master visual device control.

58. The graphical control system of claim 57, wherein the visual network operating system enables the user to govern the visual device control using the master visual device control by:

(a) enabling the user to graphically create an interaction between the visual device control and the master visual device control by operating the user manipulation device;

(b) as a result of the interaction, establishing a relationship between the master visual device control and the visual device control so that;

(i) operating the user manipulation device upon the master visual device control effectuates a change in the data representing the adjustment of the master visual device control; and

(ii) the change in the data representing the adjustment of the master visual device control effectuates a similar change in the data representing the adjustment of the visual device control.

59. The graphical control system of claim 58, wherein the visual network operating system enables the user to govern a plurality of visual device controls using the master visual device control.

60. The graphical control system of claim 57, wherein the visual network operating system enables the user to predefine a function for further manipulating the change in data representing the adjustment of the master visual device control so that the manipulated change in the data representing the adjustment of the master visual device control effectuates a similar manipulated change in the data representing the adjustment of the visual device control.

61. A graphical control system for controlling at least one non-computer system device, the graphical control system comprising:

(a) a communication medium;

(b) a computer coupled to the communication medium, the computer including a display, a processor, a storage medium and a user manipulation device;

(c) an interface for coupling the non-computer system device to the communication medium, the interface including an interface processor and an interface storage medium; and

(d) a visual network operating system for controlling and monitoring operation of the computer and the interface by;

(i) causing the computer to display an adjustable visual device control, the visual device control graphically representing a feature control of a type normally associated with the non-computer system device, the visual device control having associated data stored in the computer storage medium that represents the adjustment of the visual device control;

(ii) enabling a user to adjust the visual device control by operating the user manipulation device upon the visual device control and effectuating a change in the data that represents the adjustment of the visual device control;

(iii) communicating the change in the data that represents the adjustment of the visual device control to the interface;

(iv) causing the interface to adjust the non-computer system device in accordance with the change in the data that represents the adjustment of the visual device control; and

(v) enabling the user to create a clone of the visual device control which also graphically represents and controls the feature control, wherein the clone has the same associated data stored in the computer storage medium as the visual device control.

62. The graphical control system of claim 61, wherein the visual network operating system enables the user to effectuate a change in the data that represents the adjustment of the visual device control by operating the user manipulation device upon the clone.

63. The graphical control system of claim 61, wherein the visual network operating system enables the user to graphically alter the clone without effectuating a change in the data that represents the adjustment of the visual device control and without graphically altering the visual device control.

64. The graphical control system of claim 61, wherein the visual network operating system enables the user to create a plurality of clones of the visual device control so that each clone graphically represents the feature control, each clone having the same associated data as the visual device control.

65. The graphical control system of claim 61, wherein the visual network operating system enables the user to predefine a function for further manipulating the change in data that represents the adjustment of the visual device control.

66. A method for controlling at least one non-computer system device including at least one device control, the device control governing an aspect of the non-computer system device, wherein the non-computer system device is coupled to a communication medium by an interface and connected via the communication medium to a computer system, wherein the computer system comprises a display, a processor, a storage medium and a user manipulation device, the method comprising:

(a) causing the computer to display a graphical control display representing the non-computer system device, the graphical control display having at least one adjustable graphical control, the graphical control representing a feature control of a type normally associated with the non-computer system device, the graphical control having associated data stored in the computer storage medium that represents the adjustment of the graphical control;

(b) enabling a user to adjust the graphical control by operating the user manipulation device upon the graphical control and effectuating a change in the data that represents the adjustment of the graphical control;

(c) communicating the change in the data that represents the adjustment of the graphical control to the interface;

(d) causing the interface to adjust the non-computer system device in accordance with the change in the data that represents the adjustment of the graphical control; and

(e) causing the computer to display a template graphical control, said template graphical control graphically representing a hypothetical feature control of a non-computer system device that is not connected to the communication medium.

67. The method of claim 66, further comprising enabling the user to create an adjustable master graphical control from the template graphical control by operating the user manipulation device, the master graphical control graphically representing the hypothetical feature control, the master graphical control having associated data stored in said computer storage medium that represents the adjustment of the master graphical control.

68. The method of claim 67, further comprising enabling the user to use the master graphical control to govern the graphical control, wherein enabling the user to use the master graphical control comprises:

(a) enabling the user to graphically create an interaction between the graphical control and the master graphical control by operating the user manipulation device;

(b) as a result of the interaction, establishing a relationship between the master graphical control and the graphical control;

(c) as a result of the relationship between the master graphical control and the graphical control, enabling the user to effectuate a change in the data representing the adjustment of the master graphical control by operating the user manipulation device, wherein the change in data representing the adjustment of the master graphical control also effectuates a similar change in the data representing the adjustment of the graphical control.

69. The method of claim 68, wherein enabling the user to effectuate a change in the data representing the adjustment of the master graphical control further comprises enabling the user to predefine a function for additionally manipulating the change in the data representing the adjustment of the master graphical control.

70. The method of claim 68, further comprising enabling the user to use the master graphical control to govern a plurality of graphical controls, wherein enabling the user to use the master graphical control comprises:

for each graphical control,

(a) enabling the user to graphically create an interaction between the master graphical control and the graphical control by operating the user manipulation device;

(b) as a result of the interaction, establishing a relationship between the master graphical control and the graphical control;

(c) as a result of the relationship between the master graphical control and the graphical control, enabling the user to effectuate a change in the data representing the adjustment of the master graphical control by operating the user manipulation device, wherein the change in data representing the adjustment of the master graphical control also effectuates a similar change in the data representing the adjustment of the graphical control.

71. A method for controlling at least one non-computer system device including at least one device control, the device control governing an aspect of the non-computer system device, wherein the non-computer system device is coupled to a communication medium by an interface and connected via the communication medium to a computer system, wherein the computer system comprises a display, a processor, a storage medium and a user manipulation device, the method comprising:

(a) causing the computer to display a graphical control display representing the non-computer system device, the graphical control display having at least one adjustable graphical control, the graphical control representing a feature control of a type normally associated with the, non-computer system device, the graphical control having associated data stored in the computer storage medium that represents the adjustment of the graphical control;

(b) enabling a user to adjust the graphical control by operating the user manipulation device upon the graphical control and effectuating a change in the data that represents the adjustment of the graphical control;

(c) communicating the change in the data that represents the adjustment of the graphical control to the interface;

(d) causing the interface to adjust the non-computer system device in accordance with the change in the data that represents the adjustment of the graphical control; and

(e) creating a clone of the graphical control which along with the graphical control, graphically represents and controls the feature control, the clone having the same associated data as the graphical control.

72. The method of claim 71, wherein creating the clone further comprising enabling the user to effectuate a change in the data that represents the adjustment of the graphical control by operating the user manipulation device upon the clone.

73. The method of claim 72, wherein creating the clone further comprising enabling the user to graphically alter the clone in accordance with a predefined function, wherein the predefined function does not effectuate a change in the data that represents the adjustment of the graphical control, and wherein the predefined function does not graphically alter the graphical control.

74. The method of claim 71, wherein enabling the user to adjust the graphical control further comprises creating a plurality of clones of the graphical control so that each clone graphically represents the feature control, each clone having the same associated data as the graphical control.