System and method for providing and using a computer user interface with a view space having discrete portions

• US 6,061,064 A
• Filed: 08/27/1996
• Issued: 05/09/2000
• Est. Priority Date: 08/31/1993
• Status: Expired due to Term

• Current Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Reichlen, Bruce A.
• Primary Examiner(s): POWELL, MARK R
• Application Number:
  US08/708,103
• Time in Patent Office: 1,351 Days
• Field of Search: 345/418-20, 345/326, 345/339-46, 345/159-61, 345/8-9, 345/112-18, 345/121-26, 345/425, 345/435-38
• US Class Current: 345/418
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

A system and method associate each of a plurality of computer applications with a corresponding physical location external to the computer and display a given one of the applications when the user focuses attention on the physical location associated with that application. Preferably the display as a view window in a graphical user interface, and the user has means for moving that window relative to the given application. The computer can be a portable and display device can be head mounted. Preferably an input device enables the user to interact with the given application, and preferably the physical locations bring to mind their associated applications. In some embodiments, an identifier, such as a bar code or a coded transmitter, is placed near each of the physical locations to help detect when the user focuses attention on that particular location. The invention also provides a head mounted unit which projects a visual image to the user wearing it. The unit also includes an object detector for generating a signal when pointed at certain objects. The object detector is mounted so a user wearing the unit can point the detector by moving his or her head. Preferably the unit includes a see through window so the user can see his or her physical surroundings. The invention also provides for a computer which includes a display, a device for generating a display object on the display, such as a cursor, and a motion sensor for causing the display object to move within the display in response to motion of the display.

Claims

1. A computerized method for providing an improved graphical user interface comprising the steps of:

defining a view space in which graphical user interface windows can be selectively located by a user, said view space having a plurality of discrete portions each of which is large enough to hold an entire window;

selectively associating one of said view-space portions with each of one or more physical locations external to, and not physically connected to, said computing apparatus;

generating one or more of said windows for each of one or more computer processes;

enabling a user to selectively locate one or more of said windows in one of said view-space portions which has been associated with one of said physical locations;

detecting when a user motions toward a given one of said physical locations; and

responding to a detection that the user has motioned toward said given physical location by presenting a visual display to the user which shows the given view-space portion associated with the given physical location, together with any window located in the given view-space portion;

wherein;

an identifier object is associated with each of said physical locations; and

the step of detecting when a user motions toward a given physical location includes using an identifying device to detect when the user motions toward the identifier object associated with that location; and

the identifier objects are bar codes and the identifying device is a bar code reader.

2. A computer system with an improved graphical user interface comprising:

computing elements for defining a view space in which graphical user interface windows can be selectively located by a user, said view space having a plurality of discrete portions each of which is large enough to hold an entire window;

computing elements for selectively associating one of said view-space portions with each of one or more physical locations external to, and not physically connected to, said computing system;

computing elements for generating one or more of said windows for each of one or more computer processes;

computing elements for enabling a user to selectively locate one or more of said windows in one of said view-space portions which has been associated with one of said physical locations;

computing elements for detecting when a user motions toward a given one of said physical locations; and

computing elements for responding to a detection that the user has motioned toward said given physical location by presenting a visual display to the user which shows the given view-space portion associated with the given physical location, together with any window located in the given view-space portion;

wherein;

the physical locations with which view-space portions are associated are those of identifiers objects;

the computing elements for detecting when a user motions toward a given physical location includes an identifying device for detecting when the user motions toward the identifier object provided for that location; and

the identifiers objects are bar codes and the identifying device is a bar code reader.

3. A computing apparatus comprising:

computing elements for computing one or more graphical user interface windows for each of one or more computer processes;

computing elements for defining a view space in which said windows can be located, said view space having a plurality of discrete portions each of which is large enough to hold a plurality of said windows;

computing elements for enabling the user to selectively place individual windows at individual desired locations in said view space with a point and click interface, including moving such windows between different ones of said view space portions;

a head mounted display for projecting a visual image which can be seen by a user wearing said head mounted display;

computing elements for causing one or more images to be shown on said head mounted display which show to a user, at any given time, a subset of the portions of said view space;

a directional detector for generating an object detection signal when said detector receives a certain signals from a physical object in response to said detector being pointed at said object, said detector being mounted on said head mounted display so a user who is wearing the apparatus can point the detector by moving his or her head;

computing elements responsive to said directional detector for detecting when a user moves through each of a succession of specified zones of motion; and

computing elements making a succession of quantized changes, in units of whole view space portions, in which of said portions of said view space are shown in said head mounted display when said user is detected moving through said succession of zones of motion, without requiring any additional selection step by the user in association with each of said successive zones of motion.

4. A computing apparatus as in claim 3, wherein said head mounted display includes a see through window to enable a user to see his or her physical surroundings when wearing the head mounted display.

5. A computing apparatus as in claim 3, further comprising an additional input device to enable a user wearing the head mounted display to selectively generate computer readable signals in addition to those generated by said directional detector.

6. A computing apparatus as in claim 3 wherein said directional detector points in the same general direction as the face of a user wearing said apparatus.

7. A computerized method comprising the steps of:

computing one or more graphical user interface windows for each of one or more computer processes;

defining a view space in which said windows can be located, said view space having a plurality of discrete portions each of which is large enough to hold a plurality of said windows;

enabling a user to selectively associate each of a plurality of said view space portions with each of a plurality of zones of motion, each defined relative to a physical location external to, and not physically connected to, said computing apparatus;

enabling the user to selectively place individual windows at individual desired locations in said view space with a point and click interface, including moving such windows between different ones of said view space portions;

presenting a visual display to a user which shows a subset of the portions of said view space;

detecting when a user moves through each of a succession of said zones of motion;

responding to a detection that the user is moving through a given zone of motion by causing said visual display to show the given view space portion associated with the given zone of motion, together with any window located in the given view space portion, said step making a succession of quantized changes, in units of whole view space portions, in which of said view space portions are shown in said visual display when said user is detected moving through said succession of zones of motion, without requiring any additional selection step by the user in association with each of said successive zones of motion.

8. The method of claim 7, wherein said step of detecting when a user moves through a succession of zones of motion detects when a user points toward a given location.

9. The method of claim 8, wherein said step of detecting when a user moves through a succession of zones of motion detects when a user'"'"'s head points toward the given location.

10. The method of claim 7 wherein said step of enabling a user to selectively place individual windows includes enabling a user to drag a window from one view space portion associated with a first zone of motion to another view space portion associated with a second zone of motion through use of a pointing device.

11. The method of claim 7 wherein:

the step of detecting when a user moves through a given zone of motion also detects when the user moves out of the given zone of motion; and

the step of responding to a detection that the user is moving through a given zone of motion includes responding to a detection that the user has moved out of a given zone of motion by removing the visual display of the view space portion associated with the given zone of motion.

12. The method of claim 11 wherein the step of responding to a detection that the user has moved out of a given zone of motion includes removing the visual display of a view space portion associated with a first zone of motion in response to a detection that the user has moved out of said first zone of motion, but not removing the visual display of a view space portion associated with a second zone of motion in response to a detection that the user has move out of said second zone of motion.

13. A computer system with an improved graphical user interface comprising:

computing elements for computing one or more graphical user interface windows for each of one or more computer processes;

computing elements for defining a view space in which said windows can be located, said view space having a plurality of discrete portions each of which is large enough to hold a plurality of said windows;

computing elements for enabling a user to selectively associate each of a plurality of said view space portions with each of a plurality of zones of motion, each defined relative to a physical location external to, and not physically connected to, said computing system;

computing elements for enabling the user to selectively place individual windows at individual desired locations in said view space with a point and click interface, including moving such windows between different ones of said view space portions;

computing elements for presenting a visual display to a user which shows a subset of the portions of said view space;

computing elements for detecting when a user moves through each of a succession of said zones of motion;

computing elements for responding to a detection that the user is moving through a given zone of motion by causing said visual display to show the given view space portion associated with the given zone of motion, together with any window located in the given view space portion, said computing element making a succession of quantized changes, in units of whole view space portions, in which of said view space portions are shown in said visual display when said user is detected moving through said succession of zones of motion, without requiring any additional selection step by the user in association with each of said successive zones of motion.

14. The system of claim 13, further comprising a head mounted display for showing the visual display to the user.

15. The system of claim 14, wherein the head mounted display further comprises a see through window.

16. The system of claim 15 wherein:

said head mounted display further includes a directional detector for generating an object detection signal when said detector receives a signal radially emitted by a given physical object in response to said detector being pointed at said object, said detector being mounted on said head mounted display so a user who is wearing the apparatus can point the detector by moving his or her head;

said computing elements for enabling a use to associate one of said view space portion with a zone of motion defined relative to a physical location do so by associating said signal emitted by a given physical object with said zone of motion; and

said computing elements for detecting when a user moves through each of a succession of zones of motion does so in response to object detection signals generated in response to each of a plurality of said physical objects;

whereby said user can select the display of a given portion of said view space by pointing his or her head so said directional detector on her or her head mounted display is pointed at the physical object associated with that given view space portion'"'"'s corresponding zone of motion.

17. The system of claim 13, further comprising an input device to enable the user to interact with the graphical user interface windows shown in the visual display.

18. The system of claim 13, wherein:

the physical locations which said zones of motion are defined relative to are each identified by a location identifier object; and

wherein the computing elements for detecting when a user is moving through a given zone of motion includes an identifying device for detecting a certain signal from a given location identifier object when the user motions toward the given location identifier object.

19. The system of claim 18, wherein:

the location identifier objects are transmitters for transmitting codes which identify and distinguish between different such identifier objects; and

the identifying device is a receiver for receiving such codes and using such codes to identify their associated location identifier objects.

20. A method performed by computing apparatus comprising the steps of:

computing one or more graphical user interface windows for each of one or more computer processes;

defining a view space in which said windows can be located, said view space having a plurality of discrete portions each of which is large enough to hold a plurality of said windows;

enabling the user to selectively place individual windows at individual desired locations in said view space with a point and click interface, including moving such windows between different ones of said view space portions;

presenting a visual display to a user which shows a subset of the portions of said view space;

detecting when a user moves through each of a succession of specified zones of motion; and

making a succession of quantized changes, in units of whole view space portions, in which of said view space portions are shown in said visual display when said user is detected moving through said succession of zones of motion, without requiring any additional selection step by the user in association with each of said successive zones of motion.

21. The method of claim 20, further comprising the step of showing the visual display to the user on a head mounted display.

22. The method of claim 21, further comprising the step of detecting eye movements of the user and using the detection of said eye movements to enable the user to interact with windows in the portion of the view space displayed to him or her by the head mounted display.

23. The method of claim 21, wherein said head mounted display enables the user to see his or her surroundings while using the head mounted display.

24. The method of claim 20, further comprising the step of enabling the user to interact with the graphical user interface windows shown in the visual display through an input device.

25. The method of claim 24, wherein the step of enabling the user to interact with windows includes enabling the user to interact with such windows using at least one of the following input devices:

a mouse, a device for detecting eye movement of the user, a keyboard, a microphone and speech recognition software;

a computer network input port, and a radio signal port.

26. The method of claim 20, wherein the zones of motion each correspond to the location of a physical object related to subject matter of a window contained in the location'"'"'s associated view space portion.

27. The method of claim 26, wherein the physical object associated with a zone of motion is an image related to subject matter of a window contained in the zone of motion'"'"'s associated view space portion.

28. The method of claim 20, wherein the computer apparatus is a portable computer.

29. The method of claim 20:

further including the step of enabling a user to associate a zone of motion corresponding to a given view space portion with a given physical location external to, and not physically connected to, said computing system, by placing a location identifier object in the given physical location; and

wherein said step of detecting when a user moves through said succession of zones of motion includes detecting a certain signal from said location identifier object which indicates the user has motioned toward said given location.

30. The method of claim 29, wherein:

the step of enabling a user to associate a zone of motion with a given physical location enables a user to associate a plurality of zones of motion with a corresponding plurality of physical locations by placing a separate location identifier object in each such location;

said step of detecting when a user moves through said succession of zones of motion uses an identifier device which detects a signal from each of a succession of said location identifier objects when said user successively motions towards them; and

the location identifier objects each radiate a distinct code which identifies and distinguishes each of said location identifier objects from each other and the identifying device is a receiver for receiving such codes and using such codes to identify their associated location identifier objects.

31. The method of claim 20 wherein said detecting when a user moves through each of a succession of zones of motion performs such detection as a function of sensing motion of a user'"'"'s head.

32. The method of claim 31 wherein said step of enabling a user to selectively locate and move windows includes enabling a user to drag a window with a pointing device to a desired portion in said view space.

33. The method of claim 31 wherein:

said sensing motion of a user'"'"'s head includes sensing rotational motion of a user'"'"'s head; and

said step of making a succession of quantized changes in which of said view space portions are shown includes making such changes in response to the detection of said rotational head motion, so a user can change which view space portions are shown merely by rotating his or her head.

34. The method of claim 33, further comprising the step of detecting eye movements of the user and using the detections of said eye movements to enable the user to interact with windows in displayed portion of the view space.

35. A computer operating system stored in computer readable memory for providing an improved graphical user interface, said operating system comprising:

elements for computing one or more graphical user interface windows for each of one or more computer processes;

elements for defining a view space in which said windows can be located, said view space having a plurality of discrete portions each of which is large enough to hold a plurality of said windows;

elements for enabling the user to selectively place individual windows at individual desired locations in said view space with a point and click interface, including moving such windows between different ones of said view space portions;

elements for presenting a visual display to a user which shows a subset of the portions of said view space;

elements for detecting when a user moves through each of a succession of specified zones of motion; and

elements for making a succession of quantized changes, in units of whole view space portions, in which of said view space portions are shown in said visual display when said user is detected moving through said succession of zones of motion, without requiring any additional selection step by the user in association with each of said successive zones of motion.