Methods, apparatus and data structures for providing a user interface, which exploits spatial memory in three-dimensions, to objects and in which object occlusion is avoided

• US 6,160,553 A
• Filed: 09/14/1998
• Issued: 12/12/2000
• Est. Priority Date: 09/14/1998
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Robbins, Daniel C., Robertson, George G., Van Dantzich, Maarten Roderik
• Primary Examiner(s): Bayerl, Raymond J.
• Application Number:
  US09/152,527
• Time in Patent Office: 820 Days
• Field of Search: 345/348, 345/339, 345/145, 345/355, 345/421, 345/349, 345/438, 345/436, 345/121, 345/342, 345/344, 345/350, 345/351
• US Class Current: 715/767
• CPC Class Codes: G06F 16/904 Browsing; Visualisation therefor for navigating the web G06F16/954; browsing optimisation for the web G06F16/957
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

A graphical user interface in which object thumbnails are rendered on a simulated three-dimensional surface which (i) exploits spatial memory and (ii) allows more objects to be rendered on a given screen. The objects may be moved, continuously, on the surface with a two-dimensional input device.

Claims

1. A man-machine interface method for permitting a user to act on objects, for use with a machine having a video display device and a user input device, the man-machine interface method comprising steps of:

a) generating a surface, to be rendered on the video display device;

b) determining a location of each of the objects on the surface;

c) generating visual representations of the objects, on the surface, at the determined locations, to be rendered on the video display device;

d) accepting inputs from the user input device;

e) determining a cursor location based on the accepted inputs;

f) generating the cursor at the determined cursor location, to be rendered on the video display device;

g) if the cursor is located on one of the objects, defining that object as an active object;

h) if the user input provides a move input and if an active object exists, then

i) updating a location of the active object, and

ii) generating the visual representation of the active object at its updated location, to be rendered on the video display device; and

i) if the updated location of the active object is within a predetermined distance of another object, then

i) defining the other object as a pushed object, and

ii) updating a location of the other object to maintain a predetermined distance between it and the active object to prevent total occlusion of the visual representations of either object.

2. The man-machine interface method of claim 1 wherein, if the updated location of the active object is within a predetermined distance of another object, then further

iii) generating an audio cue associated with object occlusion avoidance.

3. The man-machine interface method of claim 1 wherein, if the updated location of the pushed object is within the predetermined distance of yet another object, then further

i) defining the yet other object as a pushed object, and

ii) updating a location of the yet other object to maintain the predetermined distance between it and the other pushed object to prevent total occlusion of the visual representation of either object.

4. The man-machine interface method of claim 1 further comprising a step of:

j) generating a bumper shadow, having a location beneath the active object, to be rendered on the video display device.

5. The man-machine interface method of claim 4 wherein the bumper shadow has a size corresponding to the predetermined distance.

6. The man-machine interface method of claim 5 wherein the bumper shadow is an oval.

7. A man-machine interface method for permitting a user to act on objects, for use with a machine having a video display device and a user input device, the man-machine interface method comprising steps of:

a) generating a surface, to be rendered on the video display device;

b) determining a location of each of the objects on the surface;

c) generating visual representations of the objects, on the surface, at the determined locations, to be rendered on the video display device;

d) accepting inputs from the user input device;

e) determining a cursor location based on the accepted inputs;

f) generating the cursor at the determined cursor location, to be rendered on the video display device;

g) if the cursor is located on one of the objects, defining that object as an active object;

h) if the user input provides a move input and if an active object exists, then

i) updating a location of the active object, and

ii) generating the visual representation of the active object at its updated location, to be rendered on the video display device; and

i) if the updated location of the active object is within a predetermined distance of another object, then

i) defining the other object as a pushed object,

ii) storing the location of the pushed object as a pre-push location, and

iii) after storing the pre-push location, updating a location of the other object to maintain a predetermined distance between it and the active object to prevent total occlusion of the visual representations of either object.

8. The man-machine interface method of claim 7 wherein, if the updated location of the active object is within a predetermined distance of another object, then further

iv) generating an audio cue associated with object occlusion avoidance.

9. The man-machine interface method of claim 7 wherein, if the updated location of the pushed object is within the predetermined distance of yet another object, then further

i) defining the yet other object as another pushed object,

ii) storing the location of the other pushed object as a pre-push location, and and

iii) updating a location of the yet other object to maintain the predetermined distance between it and the other pushed object to prevent total occlusion of the visual representation of either object.

10. The man-machine interface method of claim 7 further comprising a step of:

j) generating a bumper shadow, having a location beneath the active object, to be rendered on the video display device.

11. The man-machine interface method of claim 10 wherein the bumper shadow has a size corresponding to the predetermined distance.

12. The man-machine interface method of claim 11 wherein the bumper shadow is an oval.

13. The man-machine interface method of claim 7 further comprising steps of:

j) if the active object is released by the user, then,

i) determining whether the location of the active object is within the predetermined distance of the pre-push location, and if not, setting the location of the pushed object to the pre-push location, and

ii) defining the pushed object as a non-pushed object.

14. A system which permits a user to interact with objects, the system comprising:

a) an input facility for accepting user inputs;

b) a storage facility containing

i) location and state information for each of the objects,

ii) a visual representation of each of the objects,

iii) a cursor location, and

iv) a surface;

c) a processing unit which

i) accepts user inputs from the input facility,

ii) updates (a) the location and state information for each of the objects contained in the storage facility, and (b) the cursor location contained in the storage facility, based on the accepted user inputs, and

iii) generates video outputs based on

A) the location and state information for each of the objects,

B) the visual representation of each of the objects,

C) the cursor location, and

D) the surface,

contained in the storage facility; and

d) a video display unit for rendering the video outputs generated by the processing unit,

wherein the state information for each of the objects contained in the storage facility includes an indication of whether or not the object is active,

wherein the processing unit determines that an object is active if a cursor is on an object based on the cursor location and the location of the object,

wherein, if an object is active and the input facility accepts a move input, then

i) the processing unit updates the state and location of the object,

ii) the processing unit generates a video output based on the updated location of the object, and

iii) the video display device renders the video output generated by the processing unit, and

wherein, if an object is active and the input facility accepts a move input, then the processing unit further

determines whether the updated location of the object is within a predetermined distance of a location of another object, and

if the processor determines that the updated location of the object is within a predetermined distance of another object, the processor updates the location of the other object to avoid total occlusion of either the active object or the other object.

15. A system which permits a user to interact with objects, the system comprising:

a) an input facility for accepting user inputs;

b) a storage facility containing

i) location and state information for each of the objects,

ii) a visual representation of each of the objects,

iii) a cursor location, and

iv) a surface;

c) a processing unit which

i) accepts user inputs from the input facility,

ii) updates (a) the location and state information for each of the objects contained in the storage facility, and (b) the cursor location contained in the storage facility, based on the accepted user inputs, and

iii) generates video outputs based on

A) the location and state information for each of the objects,

B) the visual representation of each of the objects,

C) the cursor location, and

D) the surface,

contained in the storage facility; and

d) a video display unit for rendering the video outputs generated by the processing unit,

wherein the state information for each of the objects contained in the storage facility includes an indication of whether or not the object is active,

wherein the processing unit determines that an object is active if a cursor is on an object based on the cursor location and the location of the object,

wherein, if an object is active and the input facility accepts a move input, then

i) the processing unit updates the state and location of the object,

ii) the processing unit generates a video output based on the updated location of the object, and

iii) the video display device renders the video output generated by the processing unit, and

wherein, if an object is active and the input facility accepts a move input, then the processing unit further

determines whether the updated location of the object is within a predetermined distance of a location of another object, and

if the processor determines that the updated location of the object is within a predetermined distance of another object, the processor

i) defines the other object as a pushed object,

ii) stores, in the storage facility, the location of the pushed object as a pre-push location, and

iii) updates a location of the other object to maintain a predetermined distance between it and the active object to prevent total occlusion of the visual representations of either object.

16. The system claim 15 wherein if the processor determines that the active object is released by the user, then the processor,

i) determines whether the location of the active object is within the predetermined distance of the pre-push location, and if not, sets the location of the pushed object to the pre-push location, and

ii) defines the pushed object as a non-pushed object.

17. A machine readable medium containing data and machine executable instructions which, when executed by a machine having a video display device and a user input device, performs steps comprising:

a) generating a surface, to be rendered on the video display device;

b) determining a location of each of the objects on the surface;

c) generating visual representations of the objects, on the surface, at the determined locations, to be rendered on the video display device;

d) accepting inputs from the user input device;

e) determining a cursor location based on the accepted inputs;

f) generating the cursor at the determined cursor location, to be rendered on the video display device;

g) if the cursor is located on one of the objects, defining that object as an active object;

h) if the user input provides a move input and if an active object exists, then

i) updating a location of the active object, and

ii) generating the visual representation of the active object at its updated location, to be rendered on the video display device; and

i) if the updated location of the active object is within a predetermined distance of another object, then

i) defining the other object as a pushed object, and

ii) updating a location of the other object to maintain a predetermined distance between it and the active object to prevent total occlusion of the visual representations of either object.

18. A machine readable medium containing data and machine executable instructions which, when executed by a machine having a video display device and a user input device, performs steps comprising:

a) generating a surface, to be rendered on the video display device;

b) determining a location of each of the objects on the surface;

c) generating visual representations of the objects, on the surface, at the determined locations, to be rendered on the video display device;

d) accepting inputs from the user input device;

e) determining a cursor location based on the accepted inputs;

f) generating the cursor at the determined cursor location, to be rendered on the video display device;

g) if the cursor is located on one of the objects, defining that object as an active object;

h) if the user input provides a move input and if an active object exists, then

i) updating a location of the active object, and

ii) generating the visual representation of the active object at its updated location, to be rendered on the video display device; and

i) if the updated location of the active object is within a predetermined distance of another object, then

i) defining the other object as a pushed object,

ii) storing the location of the pushed object as a pre-push location, and

iii) after storing the pre-push location, updating a location of the other object to maintain a predetermined distance between it and the active object to prevent total occlusion of the visual representations of either object.

19. The machine readable medium of claim 18 wherein when the machine executable instructions are executed by a machine having a video display device and a user input device, perform a further step of:

j) if the active object is released by the user, then,

i) determining whether the location of the active object is within the predetermined distance of the pre-push location, and if not, setting the location of the pushed object to the pre-push location, and

ii) defining the pushed object as a non-pushed object.