Methods, apparatus and data structures for providing a user interface, which exploits spatial memory in three-dimensions, to objects and which visually groups proximally located objects

• US 6,414,677 B1
• Filed: 09/14/1998
• Issued: 07/02/2002
• Est. Priority Date: 09/14/1998
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Van Dantzich, Maarten Roderik, Robbins, Daniel C., Robertson, George G.
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): Padmanabhan, Mano
• Application Number:
  US09/152,712
• Time in Patent Office: 1,387 Days
• Field of Search: 345/419, 345/425, 345/334, 345/335, 345/339, 345/348, 345/355, 345/149, 345/967, 345/977, 345/716, 345/751, 345/752, 345/753, 345/694, 345/762, 345/763, 345/764, 345/765, 345/766, 345/767, 345/768, 345/769, 345/810, 345/812, 345/825, 345/835-839, 345/848
• US Class Current: 345/419
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

A graphical user interface in which object thumbnails are rendered on a simulated three-dimensional surface which (i) exploits spatial memory and (ii) allows more objects to be rendered on a given screen. The objects may be moved, continuously, on the surface with a two-dimensional input device.

Claims

1. A man-machine interface method for permitting a user to act on objects, for use with a machine having a video display device and a user input device, the man-machine interface method comprising the steps, performed by the machine, of:

a) generating a three-dimensional environment, having a three-dimensional surface, to be rendered on the video display device;

b) determining a virtual location environment of each of the objects in the three-dimensional environment;

c) generating visual representations of the objects, within the three-dimensional environment, at the determined locations, to be rendered on the video display device;

d) autonomously determining, in response to predefined criteria, a cluster of corresponding ones of the visual representations of a plurality of said objects, wherein said predefined criteria comprises a degree to which individual ones of the plurality of objects are related to each other by a predefined property of each of the objects themselves or prior user selection of a predefined property of each of said plurality of objects themselves and

e) generating a visual boundary, to be rendered on the video display device, which isolates the visual representations of said objects in the cluster from the visual representations of other ones of the objects then being displayed on the visual display device so as to visually segregate the visual representations of the objects associated with the cluster from the visual representations of the other objects.

2. The method of claim 1 wherein the cluster determining step is performed based on at least one factor selected from a group of factors consisting of:

(a) a distance between a location of a corresponding visual representation of a first one of the objects in the environment and a location of a corresponding visual representation of a second one of the objects in the environment, and (b) distances between the location of the corresponding visual representation of the first object and locations of corresponding visual representations of all other objects in the environment.

3. The method of claim 1 wherein the visual boundary is selected from a group consisting of:

(a) a translucent wall, (b) a translucent pool, (c) a border, (d) a valley and (e) an attached shape.

4. The method of claim 3 wherein the selected one of the:

(a) translucent wall, (b) translucent pool, (c) border and (d) valley, and (e) the attached shape have a color based on the cluster with which the visual boundary is associated.

5. The method of claim 1 wherein if a user moves a corresponding visual representation of one of the plurality of the objects belonging to the cluster to a position outside the boundary, the visual boundary of the cluster persists.

6. The method of claim 1 wherein if a user moves a corresponding visual representation of one of the plurality of the objects belonging to the cluster to a position outside of the boundary, the visual boundary of the cluster persists until a user instruction to re-cluster the corresponding visual representations of the objects in the environment is received.

7. The method of claim 1 wherein if a user moves a corresponding visual representation of one of the objects belonging to the cluster beyond the visual boundary, the visual boundary will stretch to bound the corresponding visual representation of said one object within the cluster.

8. The method of claim 1 wherein if a user moves a corresponding visual representation of one of the objects belonging to the cluster beyond the visual boundary, the visual boundary will separately and visually encompass the corresponding visual representation of said one object so as to define a first bounded display area and corresponding visual representations of remaining ones of the objects in the cluster so as to define a second bounded display area, and visually link the first and second bounded display areas.

17. A computer readable medium having computer executable instructions stored therein, said instructions being executed by a computer, for performing the steps of claim 1.

9. Apparatus for providing a man-machine interface which permits a user to act on objects, comprising:

a) a processor;

b) a memory connected to the processor and storing computer executable instructions therein; and

c) a video display device connected to the processor; and

d) wherein the processor, in response to execution of the instructions;

d1) generates a three-dimensional environment, having a three-dimensional surface, to be rendered on the video display device;

d2) determines a virtual location environment of each of the objects in the three-dimensional environment;

d3) generates visual representations of the objects, within the three-dimensional environment, at the determined locations, to be rendered on the video display device;

d4) autonomously determines, in response to predefined criteria, a cluster of corresponding ones of the visual representations of a plurality of said objects, wherein said predefined criteria comprises a degree to which individual ones of the plurality of objects are related to each other by a predefined property of each of the objects themselves or prior user selection of a predefined property of each of said plurality of objects themselves; and

d5) generates a visual boundary, to be rendered on the video display device, which isolates the visual representations of said objects in the cluster from the visual representations of other ones of the objects then being displayed on the visual display device so as to visually segregate the visual representations of the objects associated with the cluster from the visual representations of the other objects.

10. The apparatus of claim 9 wherein the processor, in response to execution of the instructions, determines the cluster based on at least one factor selected from a group of factors consisting of:

(a) a distance between a location of a corresponding visual representation of a first one of the objects in the environment and a location of a corresponding visual representation of a second one of the objects in the environment, and (b) distances between the location of the corresponding visual representation of the first object and locations of corresponding visual representations of all other objects in the environment.

11. The apparatus of claim 9 wherein the visual boundary is selected from a group consisting of:

(a) a translucent wall, (b) a translucent pool, (c) a border, (d) a valley and (e) an attached shape.

12. The apparatus of claim 11 wherein the selected one of the:

(a) translucent wall, (b) translucent pool, (c) border and (d) valley, and (e) the attached shape have a color based on the cluster with which the visual boundary is associated.

13. The apparatus of claim 9 wherein if a user moves a corresponding visual representation of one of the plurality of the objects belonging to the cluster to a position outside the boundary, the processor, in response to execution of the instructions, maintains display of the visual boundary of the cluster.

14. The apparatus of claim 9 wherein if a user moves a corresponding visual representation of one of the plurality of the objects belonging to the cluster to a position outside of the boundary, the visual boundary of the cluster persists until the processor receives a user instruction to re-cluster the corresponding visual representations of the objects in the environment.

15. The apparatus of claim 9 wherein if a user moves a corresponding visual representation of one of the objects belonging to the cluster beyond the visual boundary, the processor, in response to execution of the instructions, stretches the visual boundary to bound the corresponding visual representation of said one object within the cluster.

16. The apparatus of claim 9 wherein if a user moves a corresponding visual representation of one of the objects belonging to the cluster beyond the visual boundary, the processor, in response to execution of the stored instructions, manipulates the visual boundary in order as to separately and visually encompass the corresponding visual representation of said one object so as to define a first bounded display area and corresponding visual representations of remaining ones of the objects in the cluster so as to define a second bounded display area, and visually link the first and second bounded display areas.