Visual software engineering system and method for developing visual prototypes and for connecting user code to them

• US 5,883,639 A
• Filed: 07/03/1997
• Issued: 03/16/1999
• Est. Priority Date: 03/06/1992
• Status: Expired due to Term

• Current Assignee: Keysight Technologies, Inc.
• Original Assignee: Hewlett-Packard Company (HP Inc.)
• Inventors: Johnson, Roy L., Walton, Thomas J., Kuzara, Eric J., Kraemer, Thomas F., Fulghum, Kipper K., Batcha, Thomas R., Juran, Michael T.
• Primary Examiner(s): FEILD, JOSEPH H
• Application Number:
  US08/887,828
• Time in Patent Office: 621 Days
• Field of Search: 345/418, 345/419, 345/473, 345/326, 345/474, 345/475, 345/333-335, 345/348-349, 345/351, 345/433
• US Class Current: 345/473
• CPC Class Codes: G06T 1/00 General purpose image data processing

Abstract

A system for providing a simple, easy to learn and flexible means of creating user interfaces to products under development without the need of a programming language or the need to learn a large set of complicated commands. The Visual Software Engineering ("VSE") system of the invention uses a simple concept of defining both input to and output from graphical objects in an object-oriented system by providing examples of what the user desires the graphical object to do. This technique is referred to herein as "animation by example". In accordance with this process, the user creates a user interface by drawing the user interface with a graphics editor and then defining the output behavior (i.e., graphics manipulation) of the user interface components by showing each state or frame as an animation. This is accomplished by changing the object using a graphic editor function such as move or rotate and storing each of the frames with the object as a behavior state. Just as with defining the output, the input is defined by giving the graphic object an example of what type of input to look for, and once it finds that input, it tells the object which frame to output or change to. Application code can then drive the animation or read the input by accessing the frame numbers assigned to each of the example frames.

Claims

1. A system for creating and animating graphical objects by directly manipulating said graphical objects on a display screen, the system comprising:

a graphics editor which permits a user to create and manipulate said graphical objects on said display screen;

means for animating said graphical objects by storing information representing respective manipulations of said graphical objects by said user using said graphics editor as respective behavior states of said graphical objects and recreating said manipulations by specifying stored behavior states of said graphical objects corresponding to said manipulations; and

means for displaying said animated graphical objects on said display screen as display states of said graphical objects.

2. A system as in claim 1, wherein said graphics editor comprises means for grouping at least two graphical objects into a composite graphical object and for manipulating said composite graphical object.

3. A system as in claim 2, wherein said graphics editor further comprises means for creating and managing a plurality of said graphical objects and composite graphical objects.

4. A system as in claim 1, wherein said animating means comprises a behavior editor which defines each graphical object'"'"'s input behavior by selectively assigning a user defined input region, behavior state name and value to a user input operation, said behavior editor further defining each graphical object'"'"'s output behavior by assigning a current behavior state of each graphical object to a user defined output state name and value.

5. A system as in claim 4, wherein said behavior editor comprises means for sending a behavior state to said graphical objects by specifying said output state name and said value.

6. A system as in claim 1, wherein said animating means comprises an example mouse input device with an associated example mouse pad for defining operations done with a hardware mouse input device with an associated hardware mouse pad, said example mouse comprising:

a graphical mouse object representing said hardware mouse with said associated hardware mouse pad on said display screen; and

means for manipulating said graphical mouse object so as to simulate operation of said hardware mouse.

7. A system as in claim 6, wherein said manipulating means comprises:

means for pressing and releasing one or more buttons on said example mouse pad under control of said hardware mouse and associated hardware mouse pad, a button to be depressed on said example mouse pad being selected by the user by using said hardware mouse and said button on said example mouse pad being depressed by the user by depressing a corresponding button on said hardware mouse pad; and

means for moving said example mouse on its associated example mouse pad under control of said hardware mouse and associated hardware mouse pad, said example mouse being selected by the user by using said hardware mouse and being moved by moving the hardware mouse such that the example mouse is moved a corresponding amount on its associated example mouse pad on said display screen relative to the movement of said hardware mouse.

8. A system as in claim 1, further comprising a library of graphical objects which are selected by the user and manipulated by said graphics editor for display on said display screen.

9. A system as in claim 1, further comprising an interface for linking the behavior states of said graphical objects created by said graphics editor to one of (a) user defined source code and (b) an external database.

10. A system as in claim 1, further comprising means for recording and playing back behavior states of said graphical objects.

11. A method of creating and animating graphical objects by directly manipulating said graphical objects on a display screen using a graphics editor, comprising the steps of:

drawing graphical objects with said graphics editor;

manipulating said graphical objects to different display states using said graphics editor;

defining output behavior for said graphical objects for said different display states of said graphical objects;

storing the output behavior defined in said defining step with the corresponding graphical object for said different display states; and

animating said graphical objects by designating display states to be displayed and then displaying the designated display states in a desired order as an animation.

12. A method as in claim 11, wherein said drawing step comprises a step of retrieving pre-drawn graphical objects having predetermined output behavior from a graphical object library.

13. A method as in claim 11, wherein said drawing step comprises a step of grouping at least two of said desired graphical objects into a composite graphical object and said manipulating step comprises a step of manipulating said composite graphical object.

14. A method as in claim 11, wherein said manipulating step comprises a step of performing at least one of moving, rotating, scaling, stretching, cutting, filling, pasting, mapping and unmapping said graphical object.

15. A method as in claim 11, wherein said defining step comprises a step of assigning respective behavior states to respective display states and said storing step comprises a step of storing said defined output behavior with a corresponding behavior state.

16. A method as in claim 15, including a further step of interpolating said output behavior for display states between respective behavior states of said animation.

17. A method as in claim 15, wherein said defining step comprises a step of defining what output behavior each graphical object has by assigning a current behavior state of each graphical object to a user defined output state name and value.

18. A method as in claim 17, wherein said animating step comprises a step of sending a behavior state to said graphical objects by specifying an output state name and said value.

19. A method as in claim 11, wherein said defining step comprises steps of defining input behavior to said graphics editor by giving a graphical object an example of a desired input to look for, and once the desired input is found, by telling the graphical object which behavior state to output or to change to.

20. A method as in claim 19, including a further step of interpolating said input behavior for input regions between respective user defined input regions.

21. A method as in claim 19, wherein said defining step comprises a step of defining what input behavior each graphical object has by assigning a user defined input region, input state name and value to a user input operation.

22. A method as in claim 11, wherein said animating step comprises a step of receiving the behavior states of said graphical objects as input from a user'"'"'s application program and linking the behavior states of said graphical objects created by said graphics editor to one of (a) user defined source code and (b) an external database.

23. A method as in claim 11, comprising a further step of creating an example mouse input device with an associated example mouse pad to define operations done with a hardware mouse input device with an associated hardware mouse pad, said example mouse creating step comprising the steps of:

creating a graphical mouse object representing said hardware mouse with said associated hardware mouse pad on said display screen; and

manipulating said graphical mouse object so as to simulate operation of said hardware mouse input device.

24. A method as in claim 23, wherein said manipulating step comprises the steps of:

pressing and releasing one or more buttons on said example mouse pad under control of said hardware mouse and associated hardware mouse pad, comprising the steps of selecting a button to be depressed on said example mouse pad using said hardware mouse and depressing said button on said example mouse pad by depressing a corresponding button on said hardware mouse pad; and

moving said example mouse on its associated example mouse pad under control of said hardware mouse and associated hardware mouse pad, comprising the steps of selecting said example mouse using said hardware mouse and moving the hardware mouse such that the example mouse is moved a corresponding amount on its associated example mouse pad on said display screen relative to the movement of said hardware mouse.

25. A method as in claim 11, comprising further steps of saving and replaying said animation.

26. A method of defining user input to graphical objects as respective behavior states of said graphical objects, comprising the steps of:

defining at least one region on a display screen relative to each graphical object which receives user input;

describing what input event types for said at least one region relative to each of said graphical objects simulate output behavior states of said graphical objects which control output behavior for said graphical objects;

defining desired input behavior states for said graphical objects which simulate said output behavior states of said graphical objects;

storing the input behavior states defined in said input behavior states defining step with corresponding graphical objects for respective output behavior states; and

applying the input behavior states defined in said input behavior states defining step to said graphical objects so as to cause a change in at least one output behavior state of at least one of said graphical objects, thereby changing a display state of said at least one of said graphical objects.

27. A method as in claim 26, wherein said region defining step comprises a step of having a user demarcate a region relative to each of said graphical objects.

28. A method as in claim 27, wherein said demarcating step comprises a step of using an example mouse input device with an associated example mouse pad which simulates operation of a hardware mouse input device with an associated hardware mouse pad for selecting said region relative to each of said graphical objects.

29. A method as in claim 26, comprising the further step of retrieving graphical objects with predetermined input behavior states from a library and assigning input behavior of retrieved graphical objects to said graphical objects being described in said describing step.

30. A method as in claim 26, wherein said describing step comprises a step of performing at least one of pressing an input device, releasing said input device, toggling said input device, and moving said input device.

31. A method as in claim 30, wherein said performing step comprises a step of selectively pressing, releasing, toggling and moving an example mouse input device with an associated example mouse pad which simulates operation of a hardware mouse input device with an associated hardware mouse pad.

32. A method as in claim 30, wherein said input behavior states defining step comprises a step of associating a graphical object with an input from said input device in said at least one region relative to said graphical object and with an input behavior state, and said storing step comprises a step of storing said association with said graphical object.

33. A method as in claim 26, wherein said applying step comprises steps of routing said user input to an appropriate graphical object and sending from said appropriate graphical object an associated input behavior state to all graphical objects so that all graphical objects have corresponding display states for said user input.

34. A method as in claim 26, comprising a further step of interpolating said defined input behavior states with said user input to create new input behavior states between previously defined input behavior states.

35. A method as in claim 26, comprising further steps of saving and replaying said user input.

36. A method of enabling a user to create and animate a graphical user interface, by providing examples of what the user intends one or more graphical objects in the graphical user interface to do in response to user input, the method comprising steps of:

creating an arbitrary graphical user interface, using a graphics editor, by first drawing the graphical user interface as a collection of first states of graphical objects, said first states including a first state of a first graphical object;

manipulating the first graphical object, via the graphics editor, to a second state of the first graphical object, wherein together, the first state of the first graphical object, the second state of the first graphical object, and the manipulation, define an exemplary output behavior for the first graphical object;

connecting user input to the exemplary output behavior for the first graphical object; and

animating the first graphical object in response to the user input.

37. A method as in claim 36, wherein the user input is application code.

38. A method as in claim 36, wherein:

the first state of the first graphical object corresponds to a first frame in an animation sequence; and

the second state of the first graphical object corresponds to a second frame in the animation sequence; and

wherein the method further comprises a step of;

generating additional frames of the animation sequence by interpolating the exemplary output behavior for the first graphical object.

39. A method as in claim 36, wherein the step of manipulating the first graphical object to a second state comprises a step of performing at least one of moving, rotating, scaling, stretching, cutting, filling, pasting, mapping and unmapping the first graphical object.