Method and apparatus for user control by deriving next states of a process from a current state and by providing a visual presentation of the derived next states

• US 5,426,732 A
• Filed: 10/05/1994
• Issued: 06/20/1995
• Est. Priority Date: 04/17/1992
• Status: Expired due to Term

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ukelson, Jacob P., Spraragen, Susan L., Gould, John D., Comerford, Liam D., Boies, Stephen J.
• Primary Examiner(s): Zimmerman, Mark K.
• Assistant Examiner(s): Burraston, N. Kenneth
• Application Number:
  US08/320,891
• Time in Patent Office: 258 Days
• Field of Search: 395/155, 395/156, 395/159, 395/160, 395/161, 364/188, 364/190, 364/474.22, 364/474.27, 364/578, 345/121
• US Class Current: 715/839
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G08B 13/19682 Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
• CPC Class Codes: G08B 13/19689 Remote control of cameras, e.g. remote orientation or image zooming control for a PTZ camera

Abstract

A user interface includes a process model unit (34) for predicting one or more allowable next states, from a current state of a process, and a display processing unit (26) for deriving, for each of the allowable next states, a representation of the allowable next state. The display processing unit has an output coupled to a display screen (30) for displaying each of the representations (30b-30g) in conjunction with a representation (30a) of a current state of the process. The user interface further includes an actuator control unit (22) that is coupled to an input mechanism whereby a user selects one of the displayed representations of one of the allowable next states. The motor control unit controls the process to cause it to enter a new current state that corresponds to the selected derived representation. In one embodiment, the display screen has a touchscreen capability whereby the user selects one of the representations by physically touching the display screen within an area associated with a selected one of the derived allowable states.

Claims

1. A method for controlling a process, comprising the steps of:

(a) presenting a representation of a current state of a process;

(b) deriving in accordance with a predetermined transformation, from the current state of the process, one or more alternative states each representing a possible future state of the process that may be entered directly or indirectly from the current state, wherein each of the one or more derived alternative states of the process is not predetermined but is instead made manifest only as a result of executing the step of deriving;

(c) presenting a representation of the one or more derived alternative states; and

in response to a selection of one of the representations of the one or more derived alternative states,

(d) controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation.

2. A method as set forth in claim 1 wherein the representation of the current state of the process is also selectable and, in response to the selection of the representation of the current state of the process, further including a step of initiating a current state accept-specific sequence of actions.

3. A method as set forth in claim 1 wherein the step of controlling includes a step of energizing an actuator means to drive the process into the new current state.

4. A method as set forth in claim 3 wherein the step of energizing the actuator means energizes the actuator means to drive the process by a predetermined increment.

5. A method as set forth in claim 1 wherein the step of controlling includes a step of updating a model of the process to reflect an assumption of the new current state.

6. A method as set forth in claim 1 wherein the step of deriving derives the one or more alternative states in accordance with a predetermined increment.

7. A method as set forth in claim 5 wherein the step of controlling includes a step of updating the increment for use in deriving one or more alternative states from the new current state.

8. A method as set forth in claim 1 and further including the steps of:

periodically updating the current state of the process;

presenting a representation of the updated current state of the process;

deriving in accordance with the predetermined transformation, from the updated current state of the process, one or more alternative states each representing a possible future state of the process that may be entered directly or indirectly from the updated current state; and

presenting a representation of the one or more alternative states that are derived from the updated current state.

9. A method as set forth in claim 1 wherein the step of presenting a representation of a current state of a process, and the step of presenting a representation of the one or more derived alternative states, presents the representations in such a manner as to support a user'"'"'s physical intuition concerning the process.

10. A method as set forth in claim 1 wherein the presentations are presented on display means that includes a display screen having touchscreen capability, and wherein a user selects one of the representations of the one or more derived alternative states by physically touching the display screen within an area associated with the selected one of the derived alternative states.

11. A method as set forth in claim 1, wherein the steps (a), (b), (c), and (d) are repeated until the representation of the current state of the process is selected.

12. A method for controlling a process, comprising the steps of:

(a) presenting a presentation of a current state of a process;

(b) deriving, from the current state of the process, one or more alternative states;

(c) presenting a representation of the one or more derived alternative states; and

in response to a selection of one of the representations of the one or more derived alternative states,

(d) controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation, wherein the process controls the positioning of a camera along or about one or more axes.

13. A method for controlling a process, comprising the steps of:

(a) presenting a representation of a current state of a process;

(b) deriving, from the current state of the process, one or more alternative states;

(c) presenting a representation of the one or more derived alternative states; and

in response to a selection of one of the representations of the one or more derived alternative states,

(d) controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation, wherein the representation of the current state of the process is an image obtained by a camera at a current location, and wherein the representations of the one or more derived alternative states are each an image that would be obtained by the camera if the camera were translated, from the current position, about or along an axis.

14. A method for controlling a camera, comprising the steps of:

(a) presenting with a display means a representation of a current image obtained by the camera;

(b) deriving, from the current image, one or more alternative images that would be obtained by the camera if the camera were translated about or along an axis;

(c) presenting with the display means a representation of the one or more derived alternative images; and

in response to a user selecting one of the representations of the one or more derived alternative images,

(d) translating the camera to a position where the camera obtains a new current image that corresponds to the derived alternative image that is associated with the selected representation.

15. A method as set forth in claim 14 wherein derived alternative images include images that would be obtained if a pan angle of the camera were changed and if a tilt angle of the camera were changed.

16. A method as set forth in claim 15 wherein derived alternative images further include images that would be obtained if a camera zoom function were changed.

17. A method as set forth in claim 14 wherein each of the derived alternative images is derived so as to be located within an associated predetermined increment of camera displacement away from a current camera position.

18. A method as set forth in claim 17 wherein the step of translating includes a step of updating a model of the camera orientation to reflect the translation, and a step of updating, if required, one or more of the predetermined camera displacement increments.

19. A method as set forth in claim 18 wherein the step of updating the one or more predetermined camera displacement increments includes an initial step of determining, based on the updated model and on one or more predetermined allowed camera states, if a further translation of the camera would result in the camera entering a disallowed state.

20. A method as set forth in claim 14 wherein the steps (a) and (c) of displaying display the representations in such a manner as to support a user'"'"'s physical intuition concerning the control of the camera.

21. A method as set forth in claim 14 wherein the display means includes a display screen having touchscreen capability, and wherein the user selects one of the displayed representations by physically touching the display screen within an area associated with the selected one of the derived alternative images.

22. A method as set forth in claim 14 and, in response to the user selecting the representation of the current image, further including a step of initiating an accept-specific sequence of actions for accepting the current image.

23. A method as set forth in claim 14 and, in response to the user selecting the representation of the current image, further including a step of accepting the current image by reproducing the current image in a tangible form, and/or storing the current image for later reference, and/or transmitting the current image to a remote location.

24. A user interface for a process control system, comprising:

means for deriving in accordance with a predetermined transformation, from a current state of the process, one or more alternative states each representing a possible future state of the process that may be entered directly or indirectly from the current state, wherein each of the one or more derived alternative states of the process is not predetermined but is instead made manifest only as a result of an operation of said means for deriving;

means for presenting a representation of the current state of the process and for presenting a representation of the one or more derived alternative states; and

control means, responsive to a user selecting one of the representations of the one or more derived alternative states, for controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation.

25. A user interface as set forth in claim 24 wherein said control means is responsive to the user selecting the representation of the current state of the process, for initiating a current state accept-specific sequence of operations.

26. A user interface as set forth in claim 24 wherein said control means includes actuator means for driving the process into the new current state.

27. A user interface as forth in claim 24 wherein said control means includes means for storing and updating a model of the process to reflect an assumption of the new current state.

28. A user interface as set forth in claim 24 wherein said deriving means includes means for deriving the one or more alternative states in accordance with a predetermined increment, and wherein said predetermined increment is generated by said control means.

29. A user interface as set forth in claim 24 wherein said presenting means presents the representations in such a manner as to support a user'"'"'s physical intuition concerning the process.

30. A user interface as set forth in claim 24 wherein said presenting means includes a display screen, and wherein the user selects one of the representations of the one or more derived alternative states by physically touching the display screen to specify one of the derived alternative states, or by otherwise specifying one of the derived alternative states.

31. A user interface comprising:

process model means for predicting in accordance with a predetermined transformation one or more allowable future states of a process from a current state of the process, each of the predicted one or more future states, representing a future state of the process that may be entered directly or indirectly from the current state, wherein each of the predicted one or more future states of the process is not predetermined but is instead made manifest only as a result of an operation of said process model means;

display processing means, having an input coupled to an output of said process model means, for deriving, for each of the predicted allowable future states, a representation of said predicted allowable future state, said display processing means having an output coupled to display means for displaying each of said derived representations in conjunction with a representation of the current state of the process; and

control means, having a first input coupled to an output of said process model means and a second input coupled to a means whereby a user selects one of said derived representations of one of said predicted allowable future states, for controlling the process to enter a new current state that corresponds to the selected derived representation.

32. A user interface as set forth in claim 31 wherein said display means includes a display screen having touchscreen capability, and wherein the user selects one of the derived representations of the one or more predicted allowable future states by physically touching the display screen within an area associated with the selected one of the derived representations.

33. A user interface as set forth in claim 31 wherein said display means includes a display screen, and further including means for use by the user for specifying one of the derived representations of the one or more predicted allowable future states to select one of the predicted allowable future states.

34. A user interface as set forth in claim 31 wherein said display means displays the derived representations in such a manner as to support a user'"'"'s physical intuition concerning the process.

35. A user interface for use with a video image capture means, comprising:

display means having a display screen, said display screen displaying in a first region a representation of a captured video signal, said display screen displaying in a second region a transformed representation of said captured video signal;

video image transform means, having an input for receiving the captured video signal, for generating said transformed representation from the captured video signal, the transformed representation being generated to represent a video image that would be obtained from a predetermined allowable future state of the video image capture means; and

means for specifying to said user interface either the first region or the second region so as to select the representation of the captured video signal, or to cause the video image capture means to enter the predetermined allowable future state, respectively.

36. A method for controlling a process, comprising the steps of:

(a) presenting a representation of a current state of a process;

(b) deriving, from the current state of the process, one or more alternative states, wherein each of the one or more derived alternative states of the process is not predetermined but is instead made manifest only as a result of executing the step of deriving;

(c) presenting a representation of the one or more derived alternative states; and

in response to a selection of one of the representations of the one or more derived alternative states,

(d) controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation, wherein the process controls the movement of one or more physical objects within a space defined by a coordinate system.

37. A method for controlling a process, comprising the steps of:

(a) presenting a representation of a current state of a process;

(b) deriving, from the current state of the process, one or more alternative states, wherein each of the one or more derived alternative states of the process is not predetermined but is instead made manifest only as a result of executing the step of deriving;

(c) presenting a representation of the one or more derived alternative states; and

in response to a selection of one of the representations of the one or more derived alternative states,

(d) controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation, wherein the process controls the orientation of one or more physical objects within a space defined by a coordinate system.