User interface having simulated devices

• US 5,491,495 A
• Filed: 11/13/1990
• Issued: 02/13/1996
• Est. Priority Date: 11/13/1990
• Status: Expired due to Term

• Current Assignee: 3M Innovative Properties Company (3M Company)
• Original Assignee: Wang Laboratories Incorporated
• Inventors: Martin, Patricia A., Barrett, David M., Ward, Jean R., Mokoski, Christopher D.
• Primary Examiner(s): Weldon, Ulysses
• Assistant Examiner(s): FATAHI YAR, MAHMOUD
• Application Number:
  US07/613,416
• Time in Patent Office: 1,918 Days
• Field of Search: 340/706, 340/707, 340/710, 340/711, 340/712, 178/18, 382/3, 382/13, 345/156, 345/157, 345/161-168, 345/173-175, 345/112, 345/179, 345/127, 345/145, 395/159
• US Class Current: 345/173
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A computer system hating a digitizing tablet overlaying the display screen. The tablet serves as a user'"'"'s primary input device. Various features of the system make it possible for the user to run and interact with standard programs designed for keystroke and mouse input and not designed for use with a tablet. In addition to the main processor, on which the user'"'"'s programs are executed, there is an interface processor. In addition to a standard display buffer, there is an ink plane buffer for interface display data that is combined with the data from the standard display buffer on a pixel-by-pixel basis according to data from a mask plane buffer. The interface processor manages input from the tablet, presents feedback to the user by means of the ink and mask planes, and provides keystroke and mouse data to the main processor as if from a standard keyboard controller. The interface processor presents the user with a collection of simulated devices, including standard devices such as a keyboard and a mouse. A nonstandard simulated device performs character recognition, permitting handwritten characters to be used for program input. During interaction with one of the user'"'"'s programs, the user can activate and deactivate simulated devices (by removing them from and returning them to a device tray) and can make adjustments in their operation and location on the screen.

Claims

1. A system for receiving input from a non-mouse pointing device, movements of which are controlled by a user of the system, and for providing the received input to a program having a means for receiving input from a mouse, the system comprising:

(A) user input means for use with a display screen, said user input means including a stylus and a digitizing tablet for generating absolute coordinate data in response to said stylus interacting with said digitizing tablet;

(B) means for displaying on the display screen a mouse icon, at least a portion of which represents a simulated mouse surface,

(C) means by which the user can visually correlate pointing device locations with locations on the display screen,

(D) means coupled to said user input means for converting to relative coordinate data only those stylus movements that begin within the simulated surface portion of the mouse icon,

(E) means for providing the relative coordinate data to the program'"'"'s means for receiving input from a mouse;

wherein

(F) the displayed mouse icon further includes a button portion, and wherein the system further includes means for converting a touch action of the stylus in the button portion of the mouse icon to mouse button data and for providing the mouse button data to the program'"'"'s means for receiving mouse data; and

wherein

there is a further portion of the displayed mouse icon, and wherein stylus movements beginning in the further portion control a scaling of a conversion from absolute coordinate data to relative coordinate data.

2. The system of claim 1 in which the digitizing tablet is transparent, and wherein the correlating means is implemented by placing the transparent tablet over the display screen.

3. The system of claim 1 in which the means for correlating includes means for displaying a cursor indicating a current stylus position.

4. The system of claim 1 wherein a touch action causes a mouse button state to toggle.

5. The system of claim 1 wherein a brief touch action causes a mouse button click and a relatively longer touch action causes a mouse button state to toggle.

6. The system of claim 1 in which the displayed mouse icon further includes a second button portion, and touch action in each of the button portions results in simulation of mouse button data for a different button.

8. The system of claim 1 in which the displayed mouse icon is movable and can be positioned by the user at various locations overlaying portions of the display screen used by the program.

9. The system of claim 8 in which the displayed mouse icon is resizable by the user.

7. A system for receiving input from a non-mouse pointing device, movements of which are controlled by a user of the system, and for providing the received input to a program having a means for receiving input from a mouse, the system comprising:

(A) user input means for use with a display screen, said user input means including a stylus and a digitizing tablet for generating absolute coordinate data in response to said stylus interacting with said digitizing tablet;

(B) means for displaying on the display screen a mouse icon, at least a portion of which represents a simulated mouse surface,

(C) means by which the user can visually correlate pointing device locations with locations on the display screen,

(D) means coupled to said user input means for converting to relative coordinate data only those stylus movements that begin within the simulated surface portion of the mouse icon,

(E) means for providing the relative coordinate data to the program'"'"'s means for receiving input from a mouse;

wherein

(F) the displayed mouse icon further includes a button portion, and wherein the system further includes means for converting a touch action of the stylus in the button portion of the mouse icon to mouse button data and for providing the mouse button data to the program'"'"'s means for receiving mouse data;

wherein the displayed mouse icon further includes a second button portion, and touch action in each of the button portions results in simulation of mouse button data for a different button, and wherein the displayed mouse icon further includes a third button portion, and touch action of the third button portion results in mouse button data simulating the pressing of two mouse buttons.

10. A system for receiving input from a non-mouse pointing device, movements of which are controlled by a user of the system, and for providing the received input to a program having a means for receiving input from a mouse, the system comprising:

(A) user input means for use with a display screen, said user input means including a stylus and a digitizing tablet for generating absolute coordinate data in response to said stylus interacting with said digitizing tablet;

(B) means for displaying on the display screen a mouse icon, at least a portion of which represents a simulated mouse surface,

(C) means by which the user can visually correlate pointing device locations with locations on the display screen,

(D) means coupled to said user input means for converting to relative coordinate data only those stylus movements that begin within the simulated surface portion of the mouse icon,

(E) means for providing the relative coordinate data to the program'"'"'s means for receiving input from a mouse;

wherein

(F) the displayed mouse icon further includes a button portion, and wherein the system further includes means for converting a touch action of the stylus in the button portion of the mouse icon to mouse button data and for providing the mouse button data to the program'"'"'s means for receiving mouse data;

wherein the displayed mouse icon is movable and can be positioned by the user at various locations overlaying portions of the display screen used by the program;

wherein the displayed mouse icon is resizable by the user, and wherein there is a further portion of the displayed mouse icon, and wherein stylus movements beginning in the further portion control a scaling of a conversion from absolute coordinate data to relative coordinate data.

11. A method, using a computer system having a display screen and an absolute position device having a stylus and being arranged such that when the stylus is positioned over the display screen the device provides X-Y coordinate data corresponding to the position on the display screen over which the stylus is positioned, for running mouse-based programs that are designed to receive relative position data, comprising the steps of:

(A) running a mouse-based program that presents data to a user on the display screen;

(B) presenting on the display screen a mouse icon, at least a portion of the displayed mouse icon defining a simulated mouse surface, the mouse icon being smaller than the display screen and being movable, by user manipulation of the stylus, to various locations on the display screen, including locations at which data from the mouse-based program is presented;

(C) converting movements of the stylus initiated only within the simulated surface portion of the mouse icon to mouse movement data, wherein there is a further portion of the displayed mouse icon, and wherein stylus movements beginning in the further portion control a scaling of a conversion from absolute coordinate data to relative coordinate data; and

(D) providing the mouse movement data to the mouse-based program;

wherein

the mouse icon further includes at least one button portion and, in response to the user positioning the stylus in a predetermined spatial relationship with the at least one button portion, the method further includes the steps of,

(E) generating mouse button data; and

(F) providing the mouse button data to a mouse-based program.

12. A computer system having a pointing device comprising:

(A) means for displaying a plurality of icons representing simulated input devices and being responsive to pointing device input to move individual icon displays,

(B) means for displaying a device tray having one or more simulated input devices and being responsive to pointing device input to move the device tray,

(C) means by which a user of the system can activate and deactivate the display of the device tray,

(D) means responsive to deactivation of the device tray to deactivate simulated input devices located on the device tray and to activate for use any simulated input devices that have moved off of the device tray.

13. The computer system of claim 12 in which at least one of the simulated device icons has two display forms, and where one form of that icon is displayed when the device tray is active and the other form of the icon is displayed when the device t,,ay is not active.

14. In a system including a display screen and a pointing device and having a simulated keyboard device and running an application program capable of running independently of, but receiving keystroke input from, the simulated keyboard device, a method of reconfiguring the simulated keyboard device comprising the steps of:

(A) presenting on the display screen a plurality of key-group icons, each key-group icon including an area corresponding to at least one key,

(B) receiving data from the pointing device and moving the presentation of a key-group icon on the display screen in response to the received data, whereby a user of the system can move individual key-group icons on the display screen,

(C) binding together a plurality of key-group icons, such that they can be moved by the user in a single unit which can be used to provide keystroke input to a program.

15. The method of claim 14 in which the reconfiguration can be performed while the application program to which keystroke input is being provided is running.

16. The method of claim 14 in which the step of presenting on the screen includes displaying a build space and a storage space.

17. A method for operating a tablet-based computer system comprising the steps of:

(A) displaying a keyboard icon for representing a simulated keyboard, the icon including at least an area representing a first shift-lock key and an area representing a second shift-lock key,

(B) generating keystroke data in response to tablet actions at locations on the keyboard icon corresponding to keys, such that

(1) a tablet action in the first shift-lock key sets that key to a shifted state, in which that key stays until a succeeding tablet action in that key sets the first shift-lock key back to an unshifted state, and

(2) a tablet action in the second shift-lock key sets that key to a shifted state, in which that key stays until a succeeding tablet action in a non-shift key sets the second shift-lock key back to an unshifted state, and

(C) presenting the generated keystroke data as input to a program as if the keystroke data had been generated by a physical keyboard;

whereby the first shift-lock key operates in a toggle mode and the second shift-lock key operates in a one-shot mode.

18. The method of claim 23 in which an output of the simulated keyboard is provided to the program as keycode data that is encoded in accordance with a user-selected keyboard encoding format.