Pressure-sensitive layering of displayed objects

• US 8,745,514 B1
• Filed: 04/13/2009
• Issued: 06/03/2014
• Est. Priority Date: 04/11/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Perceptive Pixel Incorporated (Microsoft Corporation)
• Inventors: Davidson, Philip L.
• Primary Examiner(s): Pesin, Boris
• Assistant Examiner(s): Wright, Elizabeth G
• Application Number:
  US12/422,974
• Time in Patent Office: 1,877 Days
• Field of Search: 715/727, 715/765, 715/863, 345/173, 345/179, 705/1
• US Class Current: 715/766
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

First and second objects are displayed on a pressure-sensitive touch-screen display device. An intersection is detected between the objects. Contact by one or more input mechanisms is detected in a region that corresponds to the first displayed object. Pressure applied by at least one input mechanisms is sensed. The depth of the first displayed object is adjusted as a function of the sensed pressure. The depth of the displayed objects are determined at their detected intersection. The determined depths of the displayed objects are compared. Based on a result of comparing the determined depths, data is stored indicating that one of the displayed objects is overlapping the other. In addition, the displayed objects are displayed such that the overlapping displayed object is displayed closer to a foreground of the pressure-sensitive touch-screen display device than the other displayed object.

Claims

1. A method for adjusting an ordering of objects displayed on a pressure-sensitive touch-screen display device, the method being implemented using a computing system and comprising:

causing two or more objects to be displayed on the pressure-sensitive touch-screen display device;

receiving user input through user interaction with the pressure-sensitive touch-screen display device;

adjusting one or more of a shape and a location of at least one of a first one of the two or more displayed objects and a second one of the two or more displayed objects as a function of the received user input;

detecting an intersection between the first displayed object and the second displayed object during the adjusting of one or more of the shape and location of at least one of a first one of the two or more displayed objects and a second one of the two or more displayed objects;

detecting contact with the pressure-sensitive touch-screen display device by one or more input mechanisms in a region of the pressure-sensitive touch-screen display device that corresponds to the first displayed object;

sensing pressure applied to the pressure-sensitive touch-screen display device by at least one of the input mechanisms detected to be contacting the pressure-sensitive touch-screen display device in the region of the pressure-sensitive touch-screen display device that corresponds to the first displayed object;

calculating a pressure-sensitive tilt plane for the first displayed object as a function of the sensed pressure applied to the pressure-sensitive display device by the at least one input mechanism detected to be contacting the pressure-sensitive touch-screen display device in the region of the pressure-sensitive touch-screen display device that corresponds to the first displayed object;

applying a tilt to the first displayed object as a function of the calculated pressure-sensitive tilt plane for the first displayed object;

determining a depth of the first displayed object at its detected intersection with the second displayed object based on the tilt applied to the first displayed object;

determining a depth of the second displayed object at its detected intersection with the first displayed object;

comparing the determined depths of the first and second displayed objects at their detected intersection;

based on a result of comparing the determined depths of the first and second displayed objects at their detected intersection, storing data indicating that one of the first and second displayed objects is overlapping the other of the first and second displayed objects; and

causing the first and second displayed objects to be displayed on the pressure-sensitive touch-screen display device in accordance with the stored data such that the overlapping displayed object is displayed closer to a foreground of the pressure-sensitive touch-screen display device than the other displayed object.

2. The computer-implemented method of claim 1 wherein:

applying a tilt to the first displayed object as a function of the calculated tilt plane for the first displayed object includes applying a tilt to the first displayed object such that, at its intersection with the second displayed object, the first displayed object is raised above the display plane of the pressure-sensitive touch-screen display device;

comparing the determined depths of the first and second displayed objects at their detected intersection includes determining that the depth of the first displayed object is higher than the depth of the second displayed object at the detected intersection between the first displayed object and the second displayed object;

storing data indicating that one of the first and second displayed objects is overlapping the other of the first and second displayed objects includes storing data indicating that the first displayed object is overlapping the second displayed object; and

causing the first and second displayed objects to be displayed on the pressure-sensitive touch-screen display device in accordance with the stored data includes causing the first displayed object to be displayed closer to the foreground of the pressure-sensitive touch-screen display device than the second displayed object.

3. The computer-implemented method of claim 1 wherein:

applying a tilt to the first displayed object as a function of the calculated tilt plane for the first displayed object includes applying a tilt to the first displayed object such that, at its intersection with the second displayed object, the first displayed object is depressed below the display plane of the pressure-sensitive touch-screen display device;

comparing the determined depths of the first and second displayed objects at their detected intersection includes determining that the depth of the first displayed object is lower than the depth of the second displayed object at the detected intersection between the first displayed object and the second displayed object;

storing data indicating that one of the first and second displayed objects is overlapping the other of the first and second displayed objects includes storing data indicating that the second displayed object is overlapping the first displayed object; and

causing the first and second displayed objects to be displayed on the pressure-sensitive touch-screen display device in accordance with the stored data includes causing the second displayed object to be displayed closer to the foreground of the pressure-sensitive touch-screen display device than the first displayed object.

4. The computer-implemented method of claim 1 further comprising:

detecting contact with the pressure-sensitive touch-screen display device by one or more additional input mechanisms in a region of the pressure-sensitive touch-screen display device that corresponds to the second displayed object;

sensing pressure applied to the pressure-sensitive touch-screen display device by at least one of the additional input mechanisms detected to be contacting the pressure-sensitive touch-screen display device in the region of the pressure-sensitive touch-screen display device that corresponds to the second displayed object;

calculating a tilt plane for the second displayed object as a function of the sensed pressure applied to the pressure-sensitive display device by the at least one additional input mechanism detected to be contacting the pressure-sensitive touch-screen display device in the region of the pressure-sensitive touch-screen display device that corresponds to the second-displayed object; and

applying a tilt to the second displayed object as a function of the calculated tilt plane for the second displayed object, wherein determining a depth of the second displayed object at its detected intersection with the first displayed object includes determining a depth of the second displayed object at its detected intersection with the first displayed object based on the tilt applied to the second displayed object.

5. The computer-implemented method of claim 4 wherein:

applying a tilt to the first displayed object as a function of the calculated tilt plane for the first displayed object includes applying a tilt to the first displayed object such that, at its intersection with the second displayed object, the first displayed object is raised above the display plane of the pressure-sensitive touch-screen display device;

applying a tilt to the second displayed object as a function of the calculated tilt plane for the second displayed object includes applying a tilt to the second displayed object such that, at its intersection with the first displayed object, the second displayed object is depressed below the display plane of the pressure-sensitive touch-screen display device;

comparing the determined depths of the first and second displayed objects at their detected intersection includes determining that the depth of the first displayed object is higher than the depth of the second displayed object at the detected intersection between the first displayed object and the second displayed object;

storing data indicating that one of the first and second displayed objects is overlapping the other of the first and second displayed objects includes storing data indicating that the first displayed object is overlapping the second displayed object; and

causing the first and second displayed objects to be displayed on the pressure-sensitive touch-screen display device in accordance with the stored data includes causing the first displayed object to be displayed closer to the foreground of the pressure-sensitive touch-screen display device than the second displayed object.

6. The computer-implemented method of claim 1 wherein:

detecting an intersection between the first displayed object and the second displayed object includes detecting an intersection between an edge of the first displayed object and an edge of the second displayed object;

determining a depth of the first displayed object at its detected intersection with the second displayed object based on the tilt applied to the first displayed object includes determining the depth of the edge of the first displayed object;

determining a depth of the second displayed object at its detected intersection with the first displayed object includes determining the depth of the edge of the second displayed object; and

comparing the determined depths of the first and second displayed objects at their detected intersection includes comparing the depth of the edge of the first displayed object to the depth of the second displayed object.

7. The computer-implemented method of claim 1 wherein:

detecting an intersection between the first displayed object and the second displayed object includes detecting an intersection between an edge of the first displayed object and an edge of the second displayed object;

determining a depth of the first displayed object at its detected intersection with the second displayed object based on the tilt applied to the first displayed object includes determining the depth of a portion of the first displayed object adjacent to the edge of the first displayed object;

determining a depth of the second displayed object at its detected intersection with the first displayed object includes determining the depth of a portion of the second displayed object adjacent to the edge of the second displayed object; and

comparing the determined depths of the first and second displayed objects at their detected intersection includes comparing the depth of the portion of the first displayed object adjacent to the edge of the first displayed object to the portion of the second displayed object adjacent to the edge of the second displayed object.

8. The computer-implemented method of claim 1 further comprising maintaining data reflecting a global ordering of objects displayed by the pressure-sensitive touch-screen display device, wherein:

comparing the determined depths of the first and second displayed objects at their detected intersection includes determining that the determined depths of the first and second displayed objects at their detected intersection are the same; and

storing data indicating that one of the first and second displayed objects is overlapping the other of the first and second displayed objects includes;

accessing the maintained global ordering of objects in response to determining that the determined depths of the first and second displayed objects at their detected intersection are the same,

determining that one of the first and second displayed objects is overlapping the other of the first and second displayed objects based on the accessed global ordering of objects, and

storing data indicating that one of the first and second displayed objects is overlapping the other of the first and second displayed objects based on determining that one of the first and second displayed objects is overlapping the other of the first and second displayed objects.

9. The computer-implemented method of claim 1 further comprising maintaining data reflecting a global ordering of objects displayed by the pressure-sensitive touch-screen display device, wherein:

comparing the determined depths of the first and second displayed objects at their detected intersection includes;

determining that, in a first region of the intersection between the first and second displayed objects, the depth of the first displayed object is higher than the depth of the second displayed object, and

determining that, in a second region of the intersection between the first and second displayed objects, the depth of the first displayed object is lower than the depth of the second displayed object; and

storing data indicating that one of the first and second displayed objects is overlapping the other of the first and second displayed objects includes;

accessing the maintained global ordering of objects in response to determining that, in a first region of the intersection between the first and second displayed objects, the depth of the first displayed object is higher than the depth of the second displayed object and that, in a second region of the intersection between the first and second displayed objects, the depth of the first displayed object is lower than the depth of the second displayed object,

determining that one of the first and second displayed objects is overlapping the other of the first and second displayed objects based on the accessed global ordering of objects, and

storing data indicating that one of the first and second displayed objects is overlapping the other of the first and second displayed objects based on determining that one of the first and second displayed objects is overlapping the other of the first and second displayed objects.

10. The computer-implemented method of claim 9 further comprising generating a visual or audio alert as a consequence of determining that, in a first region of the intersection between the first and second displayed objects, the depth of the first displayed object is higher than the depth of the second displayed object and that, in a second region of the intersection between the first and second displayed objects, the depth of the first displayed object is lower than the depth of the second displayed object.

11. The computer-implemented method of claim 1 further comprising:

maintaining an overlap list that is associated with the first displayed object for recording other displayed objects that are overlapped by the first displayed object; and

maintaining an underlap list that is associated with the second displayed object for recording other displayed objects that are underlapped by the second displayed object, wherein;

comparing the determined depths of the first and second displayed objects at their detected intersection includes determining that the depth of the first displayed object is higher than the depth of the second displayed object at the detected intersection between the first displayed object and the second displayed object;

storing data indicating that one of the first and second displayed objects is overlapping the other of the first and second displayed objects includes;

updating the overlap list that is associated with the first displayed object to reflect that the first displayed object is overlapping the second displayed object, and

updating the underlap list that is associated with the second displayed object to reflect that the second displayed object is underlapping the first displayed object.

12. The computer-implemented method of claim 11 further comprising:

maintaining data reflecting a global ordering of objects displayed by the pressure-sensitive touch-screen display device; and

after storing the data indicating that one of the first and second displayed objects is overlapping the other of the first and second displayed objects, updating the maintained data reflecting the global ordering of objects displayed by the pressure-sensitive touch-screen display device based on the updated overlap list that is associated with the first displayed object and the updated underlap list that is associated with the second displayed object, wherein;

causing the first and second displayed objects to be displayed on the pressure-sensitive touch-screen display device in accordance with the stored data includes;

accessing the updated data reflecting the global ordering of objects displayed by the pressure-sensitive touch-screen display device, and

causing the first and second displayed objects to be displayed on the pressure-sensitive touch-screen based on the updated data reflecting the global ordering of objects displayed by the pressure-sensitive touch-screen display device.

13. The computer-implemented method of claim 11 further comprising, in response to detecting an intersection between the first displayed object and the second displayed object, determining that no previous overlap relationship exists between the first and second object, wherein storing data indicating that one of the first and second displayed objects is overlapping the other of the first and second displayed objects includes storing data indicating that one of the first and second displayed objects is overlapping the other of the first and second displayed objects only after determining that no previous overlap relationship exists between the first and second object.

14. The computer-implemented method of claim 13 wherein determining that no previous overlap relationship exists between the first and second object includes:

accessing the overlap list that is associated with the first displayed object; and

determining that no previous overlap relationship exists between the first and second object based on the accessed overlap list that is associated with the first displayed object.

15. The computer-implemented method of claim 13 wherein determining that no previous overlap relationship exists between the first and second object includes:

accessing the underlap list that is associated with the second displayed object; and

determining that no previous overlap relationship exists between the first and second object based on the accessed underlap list that is associated with the second displayed object.

16. The computer-implemented method of claim 1 further comprising:

causing a third object to be displayed on the pressure-sensitive touch-screen display device;

detecting a new intersection between the first displayed object and the third displayed object;

in response to detecting an intersection between the first displayed object and the third displayed object;

accessing the overlap list that is associated with the first displayed object, and

determining that the first displayed object already overlaps the third displayed object based on the accessed overlap list that is associated with the first displayed object;

causing the first and third displayed objects to be displayed on the pressure-sensitive touch-screen display device such that the first displayed object is overlapping the third displayed object at their new intersection as a consequence of having determined that the first displayed object already overlaps the third displayed object.

17. The computer-implemented method of claim 1 wherein:

storing data indicating that one of the first and second displayed objects is overlapping the other of the first and second displayed objects includes storing data indicating that the first displayed object is overlapping the second displayed object;

causing the first and second displayed objects to be displayed on the pressure-sensitive touch-screen display device in accordance with the stored data such that the overlapping displayed object is displayed closer to a foreground of the pressure-sensitive touch-screen display device than the other displayed object includes causing the first and second displayed objects to be displayed on the pressure-sensitive touch-screen display device such that the first displayed object is overlapping the second displayed object; and

the method further comprises;

receiving additional user input through user interaction with the pressure-sensitive touch-screen display device;

storing an indication of a two-dimensional position of the second displayed object as a return position for the second displayed object;

performing a first translation on the second displayed object as a function of the received additional user input;

detecting that, as a consequence of the first translation performed on the second displayed object, the first displayed object and the second displayed object no longer are intersecting;

receiving further user input through user interaction with the pressure-sensitive touch-screen display device;

performing a second translation on the second displayed object as a function of the received further user input;

detecting a new intersection between the first displayed object and the second displayed object as a consequence of the second translation performed on the second displayed object;

detecting contact with the pressure-sensitive touch-screen display device by one or more input mechanisms in a region of the pressure-sensitive touch-screen display device that corresponds to the second displayed object;

sensing pressure applied to the pressure-sensitive touch-screen display device by at least one of the input mechanisms detected to be contacting the pressure-sensitive touch-screen display device in the region of the pressure-sensitive touch-screen display device that corresponds to the second displayed object;

adjusting the depth of the second displayed object relative to the display plane as a function of the sensed pressure applied to the pressure-sensitive display device by the at least one input mechanism detected to be contacting the pressure-sensitive touch-screen display device in the region of the pressure-sensitive touch-screen display device that corresponds to the second-displayed object;

determining the depth of the first displayed object at its new detected intersection with the second displayed object;

determining the depth of the second displayed object at its new detected intersection with the first displayed object;

comparing the determined depths of the first and second displayed objects at their new detected intersection;

based on a result of comparing the determined depths of the first and second displayed objects at their detected intersection;

determining that the depth of the second displayed object is higher than the depth of the first displayed object at their detected new intersection, and

storing data indicating that the second displayed object is overlapping the first displayed object;

determining that the second displayed object is within a threshold distance of the return position for the second displayed object as a consequence of the second translation performed on the second displayed object;

while the second displayed object remains separated from the return position for the second displayed object by less than the threshold distance, detecting that the further user input through user interaction with the pressure-sensitive touch-screen display device has ceased;

as a consequence of detecting that the further user input has ceased while the second displayed object remains separated from the return position for the second displayed object by less than the threshold distance, returning the second displayed object to the return position for the second displayed object; and

causing the first and second displayed objects to be displayed on the pressure-sensitive touch-screen display device such that the second displayed object is overlapping the first displayed object.

18. The computer-implemented method of claim 1 wherein detecting an intersection between the first displayed object and the second displayed object includes detecting an intersection between the first displayed object and the second displayed object as a consequence of adjusting one or more of a shape and location of at least one of the first and second displayed objects.

19. A pressure-sensitive touch-screen display device configured to provide a pressure-sensitive control for adjusting an ordering of objects displayed on the pressure-sensitive touch-screen display device, the pressure-sensitive touch-screen display device comprising:

a display for displaying output images;

an input surface for receiving user interaction with the pressure-sensitive touch-screen display device;

a processor, and

a memory storage device storing instructions that, when executed by the processor, cause the processor to;

cause two or more objects to be displayed on the display;

interpret user interaction received by the input surface;

perform a transformation on at least one of a first one of the two or more displayed objects and a second one of the two or more displayed objects as a function of the interpreted user interaction with the input surface;

detect an intersection between the first displayed object and the second displayed object;

detect contact with the input surface by one or more input mechanisms in a region of the input surface that corresponds to the first displayed object;

quantify pressure applied to the input surface by at least one of the input mechanisms detected to be contacting the input surface in the region of the input surface that corresponds to the first displayed object;

calculate a pressure-sensitive tilt plane for the first displayed object as a function of the quantified pressure applied to the input surface by the at least one input mechanism detected to be contacting the input surface in the region of the input surface that corresponds to the first displayed object;

apply a tilt to the first displayed object as a function of the calculated pressure-sensitive tilt plane for the first displayed object;

determine a depth of the first displayed object at its detected intersection with the second displayed object based on the tilt applied to the first displayed object;

determine a depth of the second displayed object at its detected intersection with the first displayed object;

compare the determined depths of the first and second displayed objects at their detected intersection;

based on a result of comparing the determined depths of the first and second displayed objects at their detected intersection, store data indicating that one of the first and second displayed objects is overlapping the other of the first and second displayed objects; and

cause the first and second displayed objects to be displayed on the pressure-sensitive touch-screen display device in accordance with the stored data such that the overlapping displayed object is displayed closer to a foreground of the pressure-sensitive touch-screen display device than the other displayed object.

20. A non-volatile computer readable storage memory having stored thereon instructions that, when executed, cause a computer to:

cause two or more objects to be displayed on a pressure-sensitive touch-screen display device;

receive user input through user interaction with the pressure sensitive touch-screen display device;

perform a transformation on at least one of a first one of the two or more displayed objects and a second one of the two or more displayed objects as a function of the received user input;

detect an intersection between the first displayed object and the second displayed object;

detect contact with the pressure-sensitive touch-screen display device by one or more input mechanisms in a region of the pressure-sensitive touch-screen display device that corresponds to the first displayed object;

sense pressure applied to the pressure-sensitive touch-screen display device by at least one of the input mechanisms detected to be contacting the pressure-sensitive touch-screen display device in the region of the pressure-sensitive touch-screen display device that corresponds to the first displayed object;

calculate a pressure-sensitive tilt plane for the first displayed object as a function of the sensed pressure applied to the pressure-sensitive display device by the at least one input mechanism detected to be contacting the pressure-sensitive touch-screen display device in the region of the pressure-sensitive touch-screen display device that corresponds to the first displayed object;

apply a tilt to the first displayed object as a function of the calculated pressure-sensitive tilt plane for the first displayed object;

determine a depth of the first displayed object at its detected intersection with the second displayed object based on the tilt applied to the first displayed object;

determine a depth of the second displayed object at its detected intersection with the first displayed object;

compare the determined depths of the first and second displayed objects at their detected intersection;

based on a result of comparing the determined depths of the first and second displayed objects at their detected intersection, store data indicating that one of the first and second displayed objects is overlapping the other of the first and second displayed objects; and

cause the first and second displayed objects to be displayed on the pressure-sensitive touch-screen display device in accordance with the stored data such that the overlapping displayed object is displayed closer to a foreground of the pressure-sensitive touch-screen display device than the other displayed object.