Computer modelling system and method for specifying the behavior of graphical operator interfaces

• US 5,485,600 A
• Filed: 11/09/1992
• Issued: 01/16/1996
• Est. Priority Date: 11/09/1992
• Status: Expired due to Term

• Current Assignee: Engenuity Technologies, Inc. (CAE Incorporated)
• Original Assignee: Virtual Prototypes, Inc. (CAE Incorporated)
• Inventors: Trachtman, Michael, Joseph, Eugene R.
• Primary Examiner(s): Lall, Parshotam S.
• Assistant Examiner(s): ELLIS, RICHARD L
• Application Number:
  US07/972,779
• Time in Patent Office: 1,163 Days
• Field of Search: 395/500, 395/159, 395/161, 364/578, 364/188, 364/189, 364/190, 364/191, 364/192, 364/146, 364/147, 364/474.22, 364/474.23, 364/474.24, 364/474.26, 364/474.27
• US Class Current: 703/13
• CPC Class Codes: G06F 8/34 Graphical or visual programming

Abstract

A computer simulator system allows the user to specify prototype reaction to events in a pictorial manner using a visual object environment. A spreadsheet like State Table and a visual object collection coincide on a common graphical display. The State Table is filled in by pointing to lists of events or actions associated with the different objects. The contents of these lists are dependent on the object class. Event or action descriptions are transported into the respective cells of the State Table in the form of descriptive strings of text. This text describes the event or action, and the event source, or action destination. Entries in the State Table define the operation of the simulation and are executed directly by an interpreter or are compiled to generate a program of instructions for performing the simulation.

Claims

1. A system for developing and executing and specifying interactive visual applications wherein dynamic data are mapped into coherently animated virtual objects and operator interaction facilities are provided to interact with the objects on the display, the system comprising:

a central processing unit responsive to operator control signals for deriving address and control signals;

a random access memory for providing instructions to said central processing unit in response to said address signals from said central processing unit;

a non-volatile storage medium for supplying said instructions to said random access memory in response to said control signals from said central processing unit;

a graphics display device responsive to said control signals from said central processing unit for displaying the objects; and

operator input means for supplying said operator control signals to said central processing unit, said operator control signals including positional and alphanumeric components, said operator input means including;

(i) a locator/trigger device for supplying said positional signals to the central processing unit, and

(ii) a data entry device for supplying said alphanumeric signals to said central processing unit;

said instructions including at least one functional module for defining operations of said central processing unit and including;

(i) an object editor module responsive to said operator control signals for deriving the virtual objects by relating a graphical appearance of said virtual objects to a behavior of said objects during execution by the system;

(ii) an integration editor module responsive to said operator control signals for mapping ones of the virtual objects into names of variables, the contents of memory locations of said random access memory corresponding to said named variables, said named variables being used for redrawing the virtual objects on the graphics display to reflect changes in the contents of the memory locations associated with said named variables and said named variables further supplying graphic access to data in response to said positional signals from said locator/trigger device and to screen locations of said virtual objects,

(iii) an execution facility for redrawing ones of said virtual objects in response to at least one of;

(i) dynamic data stored in said random access memory and (ii) operator control signals, the execution facility further being responsive to predetermined logic specification data;

(a) said logic specification data specifying sets of events and corresponding reactions which should occur in response to the respective events, a respective origin of the event, a state of the system wherein said state is a state in a finite state machine model of the system, wherein each of said events originating from at least one of (A) ones of said objects, (B) a system clock signal, (C) control signals from said central processing unit, (D) messages from other computer systems, or (E) changes in the contents of the named variables;

(b) said logic specification data further specifying the next state that the system should transition to after an event is received;

(c) said logic specification data further specifying the reactions that occur under control of the logic specification, ones of which affect ones of the objects displayed on the display device; and

a state table editor responsive to said operator control signals for defining said logic specification by specifying;

(1) system states;

(2) events to be recognized in respective ones of said states;

(3) actions to be taken in response to respective ones of said events and states; and

(4) next states to be entered in response to respective ones of said events and states.

2. The system of claim 1 wherein said logic specification is generated responsive to selection of the virtual objects displayed on the graphics display to fill in a state table including said events and conditions to be recognized and corresponding actions to be taken.

3. The system of claim 2 including means responsive to said operator control signals for entering state names of respective ones of said states into the state table.

4. The system of claim 3 wherein said means for entering said state names includes a data entry device.

5. The system of claim 3 wherein said means for entering said state names includes means for copying from one of said cells forming said state table to another of said cells forming said state table.

6. The system of claim 1 including means for specifying events and respective sources of said events by clicking with the locator/trigger device on the visual representation of the virtual object generating respective ones of said events.

7. The system of claim 6 including means responsive to said locator/trigger device for selectively inserting said logic specification data into said state table.

8. The system of claim 1 further comprising means for specifying said actions by clicking with the locator/trigger device on visual representations of the virtual objects to which said actions apply.

9. The system of claim 8 including means for selectively inserting ones of said actions into cells forming the state table.

10. The system of claim 9 wherein said means for selectively inserting comprises said locator/trigger device.

11. The system of claim 1 including non-volatile memory means for storing said logic specification in an open ASCII format and a description of objects aggregates in an open ASCII format, whereby for each object such description includes details of the object class, object interface to a controlling variable and object graphical primitives, attributes and transformations.

12. The system of claim 1 wherein said execution facility includes said logic specification and said execution facility is stored in said random access memory during a system execution mode of operation.

13. The system of claim 1 wherein said logic specification and object description is postprocessed and stored in the modeled system, wherein the modeled system exhibits a behavior substantially the same as a behavior of said model system.

14. The system of claim 13 wherein the logic specification and object description includes states and transition statements, each transition statement including:

an event, an action list and a next state, each event including an event type and an event descriptor, an action list being a list of names of programs that are to be executed, and the next state being a name,

the object description including, for any object, behavioral parameters, interface information and graphics information;

the behavioral information being class dependent and capturing salient real-time behavior of the object class, the interface information identifying at least one variable through which the application interfaces with the operator interface, the graphics information including (i) graphics primitives and coordinates thereof, (ii) graphics transformations such as scaling, rotation, and translation and (iii) graphics attributes applicable to any primitive, such as color, line style, texture and fonts.

15. The system of claim 14 wherein at least some of the transition statements include a condition, the condition being any expression involving memory addresses.

16. The system of claim 1 wherein said instructions further include:

(i) an object editor module responsive to said operator control signals for defining the virtual objects by relating a graphical appearance of said virtual objects to a behavior of said objects during execution by the system;

(ii) an integration editor module responsive to said operator control signals for mapping ones of the virtual objects into names of variables, the contents of memory locations of said random access memory corresponding to said named variables, said named variables being used for redrawing the virtual objects on the graphics display to reflect changes in the contents of the memory locations associated with said named variables and said named variables further supplying graphic access to data in response to said positional signals from said locator/trigger device and to screen locations of said virtual objects,

(iii) an execution facility for redrawing ones of said virtual objects in response to dynamic data stored in said random access memory and in response to operator control signals, the execution facility further responsive to predetermined logic specification data.

17. The system of claim 1 including means for the operator interface to communicate with an application by transferring data between the operator interface and the application.

18. The system of claim 1 wherein said logic specification data further specify predetermined conditions existing when the events occurred, and the state table editor further specifies conditions to be recognized in respective ones of said system states specified by the state table editor.

19. The system of claim 1 wherein the object editor module includes:

I. at least one of;

(A) a first arrangement for creating or modifying custom graphics including facilities for;

(i) precisely drawing graphic primitives, such as lines, circles, polygons and texts;

(ii) manipulating at least one of the color, texture, line style and font graphic attributes of said primitives;

(iii) combining such primitives to form hierarchical constructions; and

(iv) selectively applying graphic transformations, such as scaling, translating, rotating and combinations thereof, to the primitives or hierarchical constructions thereof; and

(B) a second arrangement for building custom graphics with another tool and importing custom graphics into the object editor; and

II. A third arrangement for mapping user defined graphics derived by at least one of the first and second arrangements into objects with execution time behaviors, the execution time behaviors being mapped to the user defined graphics by selecting these graphics through pointing and clicking with the graphics input device, and by filling in class specific information into a class specific form by pointing and clicking with the graphics input device on the displayed graphics and using an alphanumeric data entry device, resulting in an association of the execution time behaviors to the imported graphics, the execution time behaviors including at least one of;

a. displaying dynamic data of the application as a user designed bar chart,

b. displaying a finite, but dynamically varying, number of any of the user designed visual objects with execution behaviors specified herein in a clipped viewing area,

c. displaying and controlling a user defined cursor having an arbitrary graphical looks inside a clipped viewing area,

d. displaying dynamic real-time application data in a rotational manner, by rotating user defined or the imported graphics around a user defined center of rotation,

e. displaying as a graphics overlay user defined animated graphics over the graphics produced by a user program,

f. forming a correspondence between a set of user defined graphical representations and a finite set of natural numbers (1, 2, 3, . . . n), the natural numbers dynamically varying under application control and having only one representation of the set displayed at any given time based on the value of an index of the set,

g. displaying dynamic real-time application data in a textual fashion as an ASCII string, using a font,

h. displaying dynamic real-time application data as a user designed graph that is redrawn dynamically to illustrate the most recent data values,

i. displaying dynamic real time application data as a set of user designed graphics translating along a straight line segment,

j. forming mapping between a finite set of consecutive natural numbers and a rectangular collection of graphics and displaying the portion of the rectangular collection of graphics that corresponds to a dynamically computed, current value in the set of numbers inside a rectangular region,

k. sending to the application a value selected dynamically from a finite set of values, while concurrently providing visual feedback by displaying one of a set of user designed graphics mapped to that value, where such value set is traversed in a fixed order,

l. sending a user defined ASCII string to the application, the user defined ASCII string being statically associated with a user defined graphic appearing on the display, the user defined ASCII string being sent to the application in real-time whenever the user dynamically interfaces with the displayed graphics,

m. defining and referencing a collection of graphics to be displayed or removed together under application control,

n. sending to the application a character string generated by typing on a real hardware keyboard or by inputting through a virtual keyboard comprised of user defined graphics appearing on the display, after validation and display of the generated character string using a font, said character string being sent in real-time as the user manipulates the real or virtual keyboard,

o. sending to the application ASCII data associated with user defined graphics appearing in a set of non-overlapping rectangular regions on the display, such that for every region, a distinct character or character string is sent, the ASCII data associated with user defined graphics being sent in real-time as the user manipulates the user defined graphics,

p. sending to the application a continuous input value varying in a finite, continuous real-number range, and concurrently providing real-time visual feed-back by rotating a set of user defined graphics around a user defined center of rotation,

q. sending to the application an input signal including x,y coordinates derived by associating a user defined coordinate set with a user defined graphics region and selecting at execution time a point in the graphics region to thus map to the x,y coordinate set,

r. sending to the application lists of ASCII strings contained in pop-up or pull-down menus, said menus including user designed graphics and a user designed look and feel while being manipulated, the strings in the menus being sent to the application in real-time by manipulating the graphics through the graphic input device,

s. sending to the application a continuously varying input value from a finite continuous range while concurrently providing visual feedback by translating user designed graphics along a straight line segment mapped to a value range,

t. causing the application to display graphics primitives and alter graphical attributes thereof such as color, font, texture and transform them, such as by rotating, translating, scaling, making visible and invisible, based on real-time values of an application variable,

u. sending to the application a value from a finite set of discrete values, and providing concurrent visual feedback by rotating an image on the display around a point at predefined angles,

v. forming correspondence between a graphics object including simultaneously displayed user designed and first and second graphic objects, and sending to the application the identity of the second object when the user interfaces with graphics of the first object, such correspondence being of a temporal nature and existing only when the first and second objects are simultaneously displayed,

w. sending to the application a value from a finite set of discrete values and providing concurrent visual feedback by displaying mutually exclusive user designed graphics on the display, the traversal order of the set being determined at execution time by the user,

x. visually displaying and interacting with application data structured as a collection of nodes and connecting arcs (or links), each node and link being capable of being individually addressed and queried, and each node and link being visually displayed on the display as a network employing user designed graphics,

the data sent to the application being generated in response to a human interaction with at least one of the objects or being supplied to at least one of the objects through a simulated event generation mechanism, the application data being sent from the application to objects built with the subject object editor for display,

the application data being the contents of a finite set of named application variables of various types contained in computer memory accessed by the application program, the application data reflecting the latest results of computations and other data manipulations performed by the application program.

20. The system of claim 19 wherein the imported graphics are graphics custom designed by the user.

21. The system of claim 19 wherein the imported graphics are derived from an external editing tool.

22. The system of claim 19 wherein the fonts are designed by the user.

23. The system of claim 19 wherein the fonts are scalable vector fonts.

24. The system of claim 19 wherein the fonts are user designed raster fonts.

25. The system of claim 19 wherein the fonts are system-provided fonts.

26. The system of claim 19 wherein a new correspondence occurs in response to the second object being replaced on the display by another object.

27. The system of claim 19 further including an integration editor facility for visually defining a run-time correspondence between an object and a named application variable such that a change in application data causes a redrawing of the corresponding graphics object in accordance with its behavioral class.

28. The system of claim 27 wherein manipulation of the graphics object causes a change of the corresponding application data.

29. The system of claim 19 wherein the execution time behaviors include building new, more complex objects through a combination of the above behaviors by making an object part of another object.

30. The system of claim 19 wherein the execution time behaviors include manipulating a collection of objects including multiple hierarchical objects as an entity rather than as individual objects.

31. The system of claim 19 wherein the execution time behaviors include dynamically instancing any number of objects and altering the position thereof and number of instances displayed based on application data.

32. The system of claim 19 wherein all of the execution time behaviors a-x are included, as are the execution time behaviors of claims 29-31.

33. A system for enabling visual applications to be specified, comprising:

a central processing unit responsive to operator control signals for deriving address and control signals;

a random access memory for providing instructions to said central processing unit in response to said address signals from said central processing unit;

a non-volatile storage medium for supplying said instructions to said random access memory in response to said control signals from said central processing unit;

a graphics display device responsive to said control signals from said central processing unit for displaying the objects; and

operator input means for supplying said operator control signals to said central processing unit, said operator control signals including positional and alphanumeric components, said operator input means including;

(i) a locator/trigger device for supplying said positional signals to the central processing unit, and

(ii) a data entry device for supplying said alphanumeric signals to said central processing unit;

said instructions including at least one functional module for defining operations of said central processing unit and including;

an object editor module responsive to said operator control signals for deriving virtual objects by relating a graphical appearance of said virtual objects to a behavior of said objects during execution by the system,

the object editor module being arranged so a user can import graphics;

the object editor module assigning execution time behaviors (i) to said imported graphics or (ii) to user designed graphics;

the object editor module having;

a drawing capability for supporting creation of custom graphics providing the user with the capabilities of;

(i) precisely drawing graphic primitives, such as lines, circles, rectangles, polygons and texts;

(ii) manipulating attributes of at least one of color, texture, line style and font;

(iii) selectively applying graphic transformations, such as scaling, translation, rotation and combinations thereof, to the primitives; and

(iv) combining such primitives to form more complex constructions;

the execution time behaviors being mapped to the imported graphics by (i) selecting graphics on the display through pointing and clicking with the graphics input device, and (ii) filling in class specific information by pointing and clicking with the graphics input device on the displayed graphics and using an alphanumeric data entry device, resulting in an association of the execution time behaviors to the imported graphics, the execution time behaviors including at least one of;

a. displaying dynamic data of the application as a user designed bar chart,

b. displaying a finite, but dynamically varying, number of any of the user designed visual objects with execution behaviors specified herein in a clipped viewing area,

c. displaying and controlling a user defined cursor having an arbitrary graphical looks inside a clipped viewing area,

d. displaying dynamic real-time application data in a rotational manner, by rotating user defined or the imported graphics around a user defined center of rotation,

e. displaying as a graphics overlay user defined animated graphics over the graphics produced by a user program,

f. forming a correspondence between a set of user defined graphical representations and a finite set of natural numbers (1, 2, 3, . . . n), the natural numbers dynamically varying under application control and having only one representation of the set displayed at any given time based on the value of an index of the set,

g. displaying dynamic real-time application data in a textual fashion as an ASCII string, using a font,

h. displaying dynamic real-time application data as a user designed graph that is redrawn dynamically to illustrate the most recent data values,

i. displaying dynamic real time application data as a set of user designed graphics translating along a straight line segment,

j. forming mapping between a finite set of consecutive natural numbers and a rectangular collection of graphics and displaying the portion of the rectangular collection of graphics that corresponds to a dynamically computed, current value in the set of numbers inside a rectangular region,

k. sending to the application a value selected dynamically from a finite set of values, while concurrently providing visual feedback by displaying one of a set of user designed graphics mapped to that value, where such value set is traversed in a fixed order,

l. sending a user defined ASCII string to the application, the user defined ASCII string being statically associated with a user defined graphic appearing on the display, the user defined ASCII string being sent to the application in real-time whenever the user dynamically interfaces with the displayed graphics,

m. defining and referencing a collection of graphics to be displayed or removed together under application control,

n. sending to the application a character string generated by typing on a real hardware keyboard or by inputting through a virtual keyboard comprised of user defined graphics appearing on the display, after validation and display of the generated character string using a font, said character string being sent in real-time as the user manipulates the real or virtual keyboard,

o. sending to the application ASCII data associated with user defined graphics appearing in a set of non-overlapping rectangular regions on the display, such that for every region, a distinct character or character string is sent, the ASCII data associated with user defined graphics being sent in real-time as the user manipulates the user defined graphics,

p. sending to the application a continuous input value varying in a finite, continuous real-number range, and concurrently providing real-time visual feed-back by rotating a set of user defined graphics around a user defined center of rotation,

q. sending to the application an input signal including x,y coordinates derived by associating a user defined coordinate set with a user defined graphics region and selecting at execution time a point in the graphics region to thus map to the x,y coordinate set,

r. sending to the application lists of ASCII strings contained in pop-up or pull-down menus, said menus including user designed graphics and a user designed look and feel while being manipulated, the strings in the menus being sent to the application in real-time by manipulating the graphics through the graphic input device,

s. sending to the application a continuously varying input value from a finite continuous range while concurrently providing visual feedback by translating user designed graphics along a straight line segment mapped to a value range,

t. causing the application to display graphics primitives and alter graphical attributes thereof such as color, font, texture and transform them, such as by rotating, translating, scaling, making visible and invisible, based on real-time values of an application variable,

u. sending to the application a value from a finite set of discrete values, and providing concurrent visual feedback by rotating an image on the display around a point at predefined angles,

v. forming correspondence between a graphics object including simultaneously displayed user designed and first and second graphic objects, and sending to the application the identity of the second object when the user interfaces with graphics of the first object, such correspondence being of a temporal nature and existing only when the first and second objects are simultaneously displayed,

w. sending to the application a value from a finite set of discrete values and providing concurrent visual feedback by displaying mutually exclusive user designed graphics on the display, the traversal order of the set being determined at execution time by the user,

x. visually displaying and interacting with application data structured as a collection of nodes and connecting arcs (or links), each node and link being capable of being individually addressed and queried, and each node and link being visually displayed on the display as a network employing user designed graphics,

the data sent to the application means being generated in response to a human interaction or a simulated event generation mechanism responding to a graphical operator interface built with said modeling system, the application data that is displayed being sent from the application to the graphical operator interface built with the subject modelling system,

the application data being a finite set of named application variables of various types contained in computer memory accessed by the application program, the application data reflecting the latest results of computations and other data manipulations performed by the application program.

34. The system of claim 33 wherein all of the execution time behaviors a-x are included, as are the following execution time behaviors:

building new, more complex objects through a combination of the above behaviors by making an object part of another object;

manipulating a collection of objects including multiple hierarchical objects as an entity rather than as individual objects; and

dynamically initiating any number of objects and altering the position thereof and number of instances displayed based on application data.

35. A modeling, simulation, specification and source code generation system for at least one graphical, data driven operator interface, comprising:

a graphics input device responsive to user inputs for deriving a positional signal;

a processor unit including;

an object editor module responsive to said operator control signals for deriving virtual objects by relating a graphical appearance of said virtual objects to a behavior of said objects during execution by the system, the object editor module being arranged so a user can import graphics, the object editor module assigning execution time behaviors (i) to said imported graphics or (ii) to user designed graphics, the object editor module having;

I. at least one of;

(A) a first arrangement for creating and modifying custom graphics including facilities for;

(i) precisely drawing graphic primitives, such as lines, circles, polygons and texts;

(ii) manipulating at least one of the color, texture, line style, and font graphic attributes of said primitives;

(iii) combining such primitives to form hierarchical constructions; and

(iv) selectively applying graphic transformations, such as scaling, translating, rotating and combinations thereof, to the primitives or hierarchical constructions thereof and;

(B) a second arrangement for building custom graphics with another tool and importing custom graphics into the object editor; and

(II) a third arrangement for mapping user defined graphics derived by at least one of the first and second arrangements into objects with execution time behaviors, the execution time behaviors being mapped to the user defined graphics by selecting these graphics through pointing and clicking with the graphics input device, and by filling in class specific information into a class specific form by pointing and clicking with the graphics input device on the displayed graphics and using an alphanumeric data entry device, resulting in an association of the execution time behaviors to the imported graphics, the execution time behaviors including at least one of;

a. displaying dynamic data of the application as a user designed bar chart,

b. displaying a finite, but dynamically varying, number of any of the user designed visual objects with execution behaviors specified herein in a clipped viewing area,

c. displaying and controlling a user defined cursor having an arbitrary graphical look inside a clipped viewing area,

d. displaying dynamic real-time application data in a rotational manner, by rotating user defined or the imported graphics around a user defined center of rotation,

e. displaying as a graphics overlay user defined animated graphics over the graphics produced by a user program,

f. forming a correspondence between a set of user defined graphical representations and a finite set of natural numbers (1, 2, 3, . . . n), the natural numbers dynamically varying under application control and having only one representation of the set displayed at any given time based on the value of an index of the set,

g. displaying dynamic real-time application data in a textual fashion as an ASCII string, using a font,

h. displaying dynamic real-time application data as a user designed graph that is redrawn dynamically to illustrate the most recent data values,

i. displaying dynamic real time application data as a set of user designed graphics translating along a straight line segment,

j. forming mapping between a finite set of consecutive natural numbers and a rectangular collection of graphics and displaying the portion of the rectangular collection of graphics that corresponds to a dynamically computed, current value in the set of numbers inside a rectangular region,

k. sending to the application a value selected dynamically from a finite set of values, while concurrently providing visual feedback by displaying one of a set of user designed graphics mapped to that value, where such value set is traversed in a fixed order,

l. sending a user defined ASCII string to the application, the user defined ASCII string being statically associated with a user defined graphic appearing on the display, the user defined ASCII string being sent to the application in real-time whenever the user dynamically interfaces with the displayed graphics,

m. defining and referencing a collection of graphics to be displayed or removed together under application control,

n. sending to the application a character string generated by typing on a real hardware keyboard or by inputting through a virtual keyboard comprised of user defined graphics appearing on the display, after validation and display of the generated character string using a font, said character string being sent in real-time as the user manipulates the real or virtual keyboard,

o. sending to the application ASCII data associated with user defined graphics appearing in a set of non-overlapping rectangular regions on the display, such that for every region, a distinct character or character string is sent, the ASCII data associated with user defined graphics being sent in real-time as the user manipulates the user defined graphics,

p. sending to the application a continuous input value varying in a finite, continuous real-number range, and concurrently providing real-time visual feed-back by rotating a set of user defined graphics around a user defined center of rotation,

q. sending to the application an input signal including x,y coordinates derived by associating a user defined coordinate set with a user defined graphics region and selecting at execution time a point in the graphics region to thus map to the x,y coordinate set,

r. sending to the application lists of ASCII strings contained in pop-up or pull-down menus, said menus including user designed graphics and a user designed look and feel while being manipulated, the strings in the menus being sent to the application in real-time by manipulating the graphics through the graphic input device,

s. sending to the application a continuously varying input value from a finite continuous range while concurrently providing visual feedback by translating user designed graphics along a straight line segment mapped to a value range,

t. causing the application to display graphics primitives and alter graphical attributes thereof such as color, font, texture and transform them, such as by rotating, translating, scaling, making visible and invisible, based on real-time values of an application variable,

u. sending to the application a value from a finite set of discrete values, and providing concurrent visual feedback by rotating an image on the display around a point at predefined angles,

v. forming correspondence between a graphics object including simultaneously displayed user designed and first and second graphic objects, and sending to the application the identity of the second object when the user interfaces with graphics of the first object, such correspondence being of a temporal nature and existing only when the first and second objects are simultaneously displayed,

w. sending to the application a value from a finite set of discrete values and providing concurrent visual feedback by displaying mutually exclusive user designed graphics on the display, the traversal order of the set being determined at execution time by the user,

x. visually displaying and interacting with application data structured as a collection of nodes and connecting arcs (or links), each node and link being capable of being individually addressed and queried, and each node and link being visually displayed on the display as a network employing user designed graphics;

the data sent to the application being generated in response to a human interaction with at least one of the objects or being supplied to at least one of the objects through a simulated event generation mechanism, the application data being sent from the application to objects built with the subject object editor for display,

the application data being the contents of a finite set of named application variables of various types contained in computer memory accessed by the application program, the application data reflecting the latest results of computations and other data manipulations performed by the application program;

the processor unit further including;

a code generation module for (i) processing object description information including behavioral parameters, interface information and graphics information;

the behavioral information being class dependent and capturing the salient real-time behavior of the object class and (ii) generating graphics source code, which after suitable post-processing, in the form of assembly or compilation, link-editing or interpretation, causes an image similar to the one appearing on the model system to appear on the model(ed) system, where the two images have similar animation behaviors when subjected to similar data.

36. The system of claim 35 wherein the code generating module responds to object description information including behavioral parameters, interface information and graphics information to generate graphics source code in any language which can be described formally through a grammar, the behavioral information being class dependent and capturing the salient real-time behavior of the object class, the interface information identifying a variable through which the application interfaces with the operator interface, the graphics information including (i) graphics primitives and coordinates thereof, and (ii) graphics transformations such as scaling rotating, translating, and (iii) graphics attributes applicable to any primitive, such as color, line style, texture and fonts.

37. A modeling, simulation, specification and source code generation system for at least one graphical data driven, interactive operator interface capable of executing similarly on a model system and on a modeled system, comprising:

a graphic input device responsive to user inputs for deriving a positional signal;

a processor unit including;

(1) an input module;

(2) an object editor module responsive to said operator control signals for deriving virtual objects by relating a graphical appearance of said virtual objects to a behavior of said objects during execution by the system, the object editor module being arranged so a user can import graphics, the object editor module assigning execution time behaviors (i) to said imported graphics or (ii) to a user designed graphics, the object editor module having;

I. at least one of;

(A) a first arrangement for creating and modifying custom graphics including facilities for;

(i) precisely drawing graphic primitives, such as lines, circles, polygons and texts;

(ii) manipulating of at least one of the color, texture, line style, font and texture graphic attributes of said primitives;

(iii) combining such primitives to form hierarchical constructions; and

(iv) selectively applying graphic transformations, such as scaling, translating, rotating and combinations thereof, to the primitives or hierarchical constructions thereof; and

(B) a second arrangement for building custom design graphics with another tool and importing custom design graphics into the object editor; and

(II) a third arrangement for mapping user defined graphics derived by one of (i)-(iv) into objects with execution time behaviors being mapped to the user defined graphics by selecting these graphics through pointing and clicking with the graphics input device, and by filling in class specific information by pointing and clicking with the graphics input device on the displayed graphics and using an alphanumeric data entry device, resulting in an association of the execution time behaviors to the imported graphics, the execution time behaviors including at least one of;

a. displaying dynamic data of the application as a user designed bar chart,

b. displaying a finite, but dynamically varying, number of any of the user designed visual objects with execution behaviors specified herein in a clipped viewing area,

c. displaying and controlling a user defined cursor having an arbitrary graphical look inside a clipped viewing area,

d. displaying dynamic real-time application data in a rotational manner, by rotating user defined or the imported graphics around a user defined center of rotation,

e. displaying as a graphics overlay user defined animated graphics over the graphics produced by a user program,

f. forming a correspondence between a set of user defined graphical representations and a finite set of natural numbers (1, 2, 3, . . . n), the natural numbers dynamically varying under application control and having only one representation of the set displayed at any given time based on the value of an index of the set,

g. displaying dynamic real-time application data in a textual fashion as an ASCII string, using a font,

h. displaying dynamic real-time application data as a user designed graph that is redrawn dynamically to illustrate the most recent data values,

i. displaying dynamic real time application data as a set of user designed graphics translating along a straight line segment,

j. forming mapping between a finite set of consecutive natural numbers and a rectangular collection of graphics and displaying the portion of the rectangular collection of graphics that corresponds to a dynamically computed, current value in the set of numbers inside a rectangular region,

k. sending to the application a value selected dynamically from a finite set of values, while concurrently providing visual feedback by displaying one of a set of user designed graphics mapped to that value, where such value set is traversed in a fixed order,

l. sending a user defined ASCII string to the application, the user defined ASCII string being statically associated with a user defined graphic appearing on the display, the user defined ASCII string being sent to the application in real-time whenever the user dynamically interfaces with the displayed graphics,

m. defining and referencing a collection of graphics to be displayed or removed together under application control,

n. sending to the application a character string generated by typing on a real hardware keyboard or by inputting through a virtual keyboard comprised of user defined graphics appearing on the display, after validation and display of the generated character string using a font, said character string being sent in real-time as the user manipulates the real or virtual keyboard,

o. sending to the application ASCII data associated with user defined graphics appearing in a set of non-overlapping rectangular regions on the display, such that for every region, a distinct character or character string is sent, the ASCII data associated with user defined graphics being sent in real-time as the user manipulates the user defined graphics,

p. sending to the application a continuous input value varying in a finite, continuous real-number range, and concurrently providing real-time visual feed-back by rotating a set of user defined graphics around a user defined center of rotation,

q. sending to the application an input signal including x,y coordinates derived by associating a user defined coordinate set with a user defined graphics region and selecting at execution time a point in the graphics region to thus map to the x,y coordinate set,

r. sending to the application lists of ASCII strings contained in pop-up or pull-down menus, said menus including user designed graphics and a user designed look and feel while being manipulated, the strings in the menus being sent to the application in real-time by manipulating the graphics through the graphic input device,

s. sending to the application a continuously varying input value from a finite continuous range while concurrently providing visual feedback by translating user designed graphics along a straight line segment mapped to a value range,

t. causing the application to display graphics primitives and alter graphical attributes thereof such as color, font, texture and transform them, such as by rotating, translating, scaling, making visible and invisible, based on real-time values of an application variable,

u. sending to the application a value from a finite set of discrete values, and providing concurrent visual feedback by rotating an image on the display around a point at predefined angles,

v. forming correspondence between a graphics object including simultaneously displayed user designed and first and second graphic objects, and sending to the application the identity of the second object when the user interfaces with graphics of the first object, such correspondence being of a temporal nature and existing only when the first and second objects are simultaneously displayed,

w. sending to the application a value from a finite set of discrete values and providing concurrent visual feedback by displaying mutually exclusive user designed graphics on the display, the traversal order of the set being determined at execution time by the user,

x. visually displaying and interacting with application data structured as a collection of nodes and connecting arcs (or links), each node and link being capable of being individually addressed and queried, and each node and link being visually displayed on the display as a network employing user designed graphics;

the data sent to the application being generated in response to a human interaction with at least one of the objects or being supplied to at least one of the objects through a simulated event generation mechanism, the application data being sent from the application to objects built with the subject object editor for display,

the application data being the contents of a finite set of named application variables of various types contained in computer memory accessed by the application program, the application data reflecting the latest results of computations and other data manipulations performed by the application program;

(3) a state table editor responsive to operator control signals for defining a logic specification by specifying;

(a) system states;

(b) events to be recognized in respective ones of said states;

(c) actions to be taken in response to respective ones of said events and states; and

(d) next states to be entered in response to respective ones of said events and states; and

(4) a code generation module for generating (a) portable graphics code from an object description including behavioral parameters, interface information and graphics information;

the behavioral information being class dependent and capturing salient real-time behavior of the object class, the interface information identifying a variable through which the application interfaces with the operator interface, the graphics information including graphics primitives and coordinates thereof, graphics transformations such as scaling, rotating, translating and graphics attributes applicable to any primitive, such as color, line style, texture and fonts, and (b) logic control code from a logic specification including states and transition statements, each transition statement including;

an event, an action list and a next state;

each event including an event type and an event descriptor, a condition being any expression involving memory addresses, an action list being a list of names of programs that are to be executed, and the next state being a name;

which code, after suitable post-processing causes images with similar appearances, animation behaviors, and interactive behaviors to appear on the model(ed) system.