System and method for accessing remote screen content

• US 20040150664A1
• Filed: 03/05/2003
• Published: 08/05/2004
• Est. Priority Date: 02/03/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Baudisch, Patrick Markus
• Granted Patent: US 7,231,609 B2
• Time in Patent Office: Days
• US Class Current: 345/740
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/1462 with means for detecting differences between the image stored in the host and the images displayed on the remote displays

Abstract

A system and method for facilitating user access to remote content from a current location is presented. The invention monitors for and detects an activation gesture. The activation gesture may include an empty or nonempty selection, the nonempty selection having a source icon. Upon detecting the activation gesture, a target region is determined according to the activation gesture. Target icons capable of responding to activation gestures'"'"' selection are selected from those icons within the target region. If the activation gesture includes a nonempty selection having a source icon, only those icons within the target region and compatible with the source icon are selected. The target icons are displayed in a cluster in the proximity of the working location until a user action dismissing the cluster is detected.

Claims

1. A method for facilitating user access to remote icons on a display screen, the method comprising:

detecting an activation gesture, the activation gesture having an origin, a direction, a location, and a selection;

selecting target icons from the icons on the display screen according to the activation gesture; and

temporarily displaying the target icons in proximity to the activation gesture'"'"'s location until an action dismissing the target icons is detected.

2. The method of claim 1, wherein the activation gesture'"'"'s selection is a nonempty selection having a source icon or an empty selection.

3. The method of claim 2, wherein the activation gesture is a drag gesture.

4. The method of claim 1, wherein selecting target icons from the icons on the display screen according to the activation gesture comprises:

determining a central vector originating at the activation gesture'"'"'s origin and extending to the limits of the display screen in the activation gesture'"'"'s direction;

determining a target region defined by two boundary lines originating at the activation gesture'"'"'s origin and diverging away from and on either side of the central vector extending to the limits of the display screen; and

selecting target icons only from those icons located within the target region.

5. The method of claim 4, wherein selecting target icons from the icons on the display screen according to the activation gesture further comprises selecting target icons only from those icons located within the target region and at a distance greater than a predetermined value from the activation gesture'"'"'s origin.

6. The method of claim 5, wherein the predetermined value is user configurable.

7. The method of claim 5, wherein selecting target icons from the icons on the display screen according to the activation gesture further comprises:

determining whether the activation gesture'"'"'s selection included a source icon and, if so, selecting target icons only from those icons located within the target region and at a distance greater than the predetermined value from the activation gesture'"'"'s origin and representing content compatible with the source icon.

8. The method of claim 7, wherein selecting target icons from the icons on the display screen according to the activation gesture further comprises:

determining whether the number of target icons exceeds a predetermined limit and, if so, repeatedly;

reducing the area of the target region; and

selecting target icons only from those icons located within the reduced target region and at a distance greater than a predetermined distance from the activation gesture'"'"'s origin and compatible with the source icon if the activation gesture'"'"'s selection includes the source icon;

until the number of selected target icons is less than or equal to the predetermined limit.

9. The method of claim 8, wherein the two boundary lines diverge away from and on either side of the central vector extending to the limits of the display screen at a predetermined angle.

10. The method of claim 9, wherein the predetermined angle is user configurable.

11. The method of claim 9, wherein reducing the area of the target region comprises reducing the predetermined angle at which the boundary lines diverge away from and at either side of the central vector.

12. The method of claim 4, wherein the activation gesture is modified according to a modification condition, and wherein selecting target icons from the icons on the display screen according to the activation gesture further comprises:

selecting target icons only from those icons located within the target region and according to a predetermined criteria associated with the modification condition.

13. The method of claim 1, wherein temporarily displaying the target icons in proximity to the activation gesture'"'"'s location until an action dismissing the target icons is detected, comprises:

generating a target cluster comprising the selected target icons;

displaying the target cluster at a predetermined proximity distance from the activation gesture'"'"'s location;

monitoring for and detecting an action dismissing the target cluster; and

upon detecting an action dismissing the target cluster, removing the target cluster from the display screen.

14. The method of claim 13, wherein displaying the target cluster at the predetermined proximity distance from the activation gesture'"'"'s location comprises:

determining a center point of the target cluster, the center point determined according to the spatial arrangement of the target cluster;

determining a vector originating at the activation gesture'"'"'s origin and proceeding in the activation gesture'"'"'s direction; and

displaying the target cluster on the display screen such that the target cluster'"'"'s center point is located on the determined vector at the predetermined proximity distance from the activation gesture'"'"'s location.

15. The method of claim 13, wherein displaying the target cluster at the predetermined proximity distance from the activation gesture'"'"'s location further comprises displaying a band between each target icon in the target cluster and a corresponding base icon.

16. The method of claim 15, wherein displaying the target cluster at the displaying each base icon corresponding to a target icon in the target cluster in a manner indicative that the target icon has been temporarily moved to the target cluster.

17. The method of claim 15, wherein displaying a band between each target icon in the target cluster and the corresponding base icon comprises iteratively selecting each target icon in the target cluster in a closest-first manner and displaying the band between each target icon in the target cluster and its corresponding base icon.

18. The method of claim 15, wherein each band between a target icon in the target cluster and its corresponding base icon narrows on both sides of the band as the band approaches a midpoint between the target icon and its base icon.

19. The method of claim 18, wherein each band narrows according to the distance between a target icon and its base icon, such that the greater the distance between the target icon and its base icon, the more the band narrows.

20. The method of claim 13, wherein monitoring for and detecting an action dismissing the target cluster comprises, monitoring for and detecting a release gesture and, upon detecting the release gesture, determining whether the release gesture is made on a target icon in the target cluster, and if so:

determining the associated target icon upon which the release gesture is made; and

activating the content associated with the associated target icon in accordance with the activation gesture'"'"'s selection.

21. The method of claim 20, wherein activating the content associated with the associated target icon in accordance with the activation gesture'"'"'s selection comprises determining whether the activation gesture'"'"'s selection is a nonempty selection having a source icon and, if so, activating the content associated with the associated target icon with the source icon.

22. The method of claim 20, wherein the release gesture is a drop gesture.

23. The method of claim 20, wherein monitoring for and detecting an action dismissing the target cluster further comprises:

(a) monitoring for and detecting a movement gesture; and

(b) upon detecting a movement gesture, determining whether the movement gesture is an action dismissing the target cluster, such that;

(i) if the movement associated with the movement gesture is made in a direction toward any of the target icons in the target cluster, the movement gesture is not an action dismissing the target cluster;

(ii) if the movement associated with the movement gesture is made in a direction away from any of the target icons in the target cluster, and the amount of movement associated with the movement gesture is less than a predetermined tolerance amount, the movement gesture is not an action dismissing the target cluster;

(iii) otherwise, the movement gesture is an action dismissing the target cluster.

24. A computer system for facilitating user access to remote icons on a display device, the system comprising:

(a) a display device capable of displaying a plurality of icons;

(b) a user input device for interfacing with and interacting with the user; and

(c) a computing device, communicatively coupled to the display device and to the user input device, for;

(i) displaying a plurality of icons on the display device;

(ii) monitoring for and detecting an activation gesture, the activation gesture having an origin, a direction, a location, and a selection, and upon detecting the activation gesture;

(iii) selecting target icons from the plurality of icons displayed on the display device; and

(iv) temporarily displaying the target icons in proximity to the activation gesture'"'"'s location on the display device until an action dismissing the target icons is detected.

25. The computer system of claim 24, wherein the activation gesture'"'"'s selection is a nonempty selection having a source icon or an empty selection.

26. The computer system of claim 25, wherein the activation gesture is a drag gesture.

27. The computer system of claim 24, wherein the computing device selects target icons from the plurality of icons displayed on the display device by:

determining a central vector originating at the activation gesture'"'"'s origin and extending to the limits of the display device according to the activation gesture'"'"'s direction;

determining a target region defined by two boundary lines originating at the activation gesture'"'"'s origin and diverging away from and on either side of the central vector extending to the limits of the display device; and

selecting target icons only from those icons displayed within the target region.

28. The computer system of claim 27, wherein the computing device further selects target icons from the plurality of icons displayed on the display device by:

selecting target icons only from those icons displayed within the target region and at a distance greater than a predetermined value from the activation gesture'"'"'s origin.

29. The computer system of claim 28, wherein the predetermined value is user configurable.

30. The computer system of claim 28, wherein the computing device further selects target icons from the plurality of icons displayed on the display device by:

determining whether the activation gesture'"'"'s selection included a source icon and, if so, selecting target icons from those icons displayed within the target region and at a distance greater than a predetermined value from the activation gesture'"'"'s origin and representing content compatible with the source icon.

31. The computer system of claim 30, wherein the computing device further selects target icons from the plurality of icons displayed on the display device by determining whether the number of target icons exceeds a predetermined limit and, if so, repeatedly:

reducing the area of the target region; and

selecting target icons from those icons located within the reduced target region and at a distance greater than a predetermined distance from the activation gesture'"'"'s origin and compatible with the source icon if the activation gesture'"'"'s selection included a source icon;

until the number of selected target icons is less than or equal to the predetermined limit.

32. The computer system of claim 31, wherein the two boundary lines diverge away from and at either side of the central vector extending to the limits of the display screen at a predetermined angle.

33. The computer system of claim 32, wherein the predetermined angle is user configurable.

34. The computer system of claim 32, wherein the computing device reduces the area of the target region by reducing the predetermined angle at which the boundary lines diverge away from and at either side of the central vector.

35. The computer system of claim 27, wherein the activation gesture is modified according to a modification condition, and wherein the computer device further selects target icons from the icons on the display screen according to the activation gesture by:

selecting target icons only from those icons located within the target region and according to a predetermined criteria associated with the modification condition.

36. The computer system of claim 24, wherein the computing device temporarily displays the target icons in proximity to the activation gesture'"'"'s location on the display device until an action dismissing the target icons is detected by:

generating a target cluster comprising the selected target icons;

displaying the target cluster on the display device at a predetermined proximity distance from the activation gesture'"'"'s location;

monitoring for and detecting an action dismissing the target cluster; and

upon detecting an action dismissing the target cluster, removing the target cluster from the display device.

37. The computer system of claim 36, wherein the computing device displays the target cluster on the display device at the predetermined proximity distance from the activation gesture'"'"'s location by:

determining a center point of the target cluster, the center point determined according to the spatial arrangement of the target cluster;

determining a vector originating at the activation gesture'"'"'s origin and proceeding in the activation gesture'"'"'s direction; and

displaying the target cluster on the display device such that the target cluster'"'"'s center point is located on the determined vector at the predetermined proximity distance from the activation gesture'"'"'s location.

38. The computer system of claim 36, wherein the computing device further displays the target cluster on the display device at the predetermined proximity distance from the activation gesture'"'"'s location by:

displaying a band between each target icon in the target cluster and a corresponding base icon.

39. The computer system of claim 38, wherein the computing device further displays the target cluster on the display device at the predetermined proximity distance from

displaying each base icon corresponding to a target icon in the target cluster in a manner indicative that the target icon has been temporarily moved to the target cluster.

40. The computer system of claim 38, wherein the computing device displays the band between each target icon in the target cluster and the corresponding base icon by:

iteratively selecting each target icon in the target cluster in a closest-first manner and displaying the band between each target icon and its corresponding base icon.

41. The computer system of claim 38, wherein the computing device displays the band between each target icon in the target cluster and the corresponding base icon by:

displaying each band as narrowing on both sides as the band approaches a midpoint between the target icon and the corresponding base icon.

42. The computer system of claim 41, wherein the computing device further displays each band as narrowing on both sides as the band approaches a midpoint between the target icon and the corresponding base icon by:

narrowing the band according to the distance between the target icon and the corresponding base icon, such that the greater the distance between the target icon and the base icon, the more the band narrows.

43. The computer system of claim 36, wherein the computing device monitors for and detects a user action dismissing the target cluster by monitoring for and detecting a release gesture and, upon detecting a release gesture, determining whether the release gesture is made on a target icon in the target cluster and, if so:

determining the associated target icon on which the release gesture is made; and

activating the content associated with the associated target icon in accordance with the activation gesture'"'"'s selection.

44. The computer system of claim 43, wherein the computing device activates the content associated with the associated target icon in accordance with the activation gesture'"'"'s selection by:

determining whether the activation gesture'"'"'s selection is a nonempty selection having a source icon and, if so, activating the content associated with the associated target icon with the source icon.

45. The computer system of claim 43, wherein the release gesture is a drop gesture.

46. The computer system of claim 43, wherein the computing device further monitors for and detects an action dismissing the target cluster by:

(a) monitoring for and detecting a movement gesture; and

(b) upon detecting a movement gesture, determining whether the movement gesture is an action dismissing the target cluster, such that;

(i) if the movement associated with the movement gesture is made in a direction toward any of the target icons in the target cluster, the movement action is not an action dismissing the target cluster;

(ii) if the movement associated with the movement gesture is made in a direction away from any of the target icons in the target cluster, and the amount of movement associated with the movement gesture is less than a predetermined tolerance amount, the movement gesture is not an action dismissing the target cluster;

(iii) otherwise, the movement gesture is an action dismissing the target cluster.

47. The computer system of claim 24, wherein the display device comprises a plurality of display panels cooperating to present a single visual display area.

48. The computer system of claim 47, wherein the plurality of display panels include a pressure-sensitive display panel.

49. The computer system of claim 48, wherein the pressure-sensitive display panel is also the user input device.

50. The computer system of claim 24, wherein the display device is a pressure-sensitive display device.

51. The computer system of claim 50, wherein the pressure-sensitive display device is also the user input device.

52. The computer system of claim 24, wherein the user input device is a pen-based input device.

53. The computer system of claim 24, wherein the user input device is a touch-based input device.

54. A computer-readable medium having computer-executable instructions for carrying out the method comprising the steps of:

monitoring for and detecting an activation gesture, the activation gesture having an origin, a direction, a location, and a selection;

selecting target icons from the icons on the display screen according to the activation gesture; and

temporarily displaying the target icons in proximity to the activation gesture'"'"'s location until an action dismissing the target icons is detected.

55. The computer-readable medium of claim 54, wherein the step of selecting target icons from the icons on the display screen according to the activation gesture comprises:

determining a central vector originating at the activation gesture'"'"'s origin and extending to the limits of the display screen in the activation gesture'"'"'s direction;

determining a target region defined by two boundary lines originating at the activation gesture'"'"'s origin and diverging away from and on either side of the central vector extending to the limits of the display screen; and

selecting target icons only from those icons located within the target region.

56. The computer-readable medium of claim 55, wherein the step of selecting target icons from the icons on the display screen according to the activation gesture further comprises:

selecting target icons only from those icons located within the target region and at a distance greater than a predetermined value from the activation gesture'"'"'s origin.

57. The computer-readable medium of claim 56, wherein the step of selecting target icons from the icons on the display screen according to the activation gesture further comprises:

determining whether the activation gesture'"'"'s selection includes a source icon and, if so, selecting target icons only from those icons located within the target region and at a distance greater than the predetermined value from the activation gesture'"'"'s origin and representing content compatible with the source icon.

58. The computer-readable medium of claim 57, wherein the step of selecting target icons from the icons on the display screen according to the activation gesture further comprises determining whether the number of selected target icons exceeds a predetermined limit and, if so, repeatedly:

reducing the area of the target region; and

selecting target icons only from those icons located within the reduced target region and at a distance greater than the predetermined distance from the activation gesture'"'"'s origin and compatible with the source icon if the activation gesture'"'"'s selection includes a source icon;

until the number of selected target icons is less than or equal to the predetermined limit.

59. The computer-readable medium of claim 54, wherein the step of temporarily displaying the target icons in proximity to the activation gesture'"'"'s location until an action dismissing the target icons is detected comprises:

generating a target cluster comprising the selected target icons;

displaying the target cluster at a predetermined proximity distance from the activation gesture'"'"'s location;

monitoring for and detecting an action dismissing the target cluster; and

upon detecting an action dismissing the target cluster, removing the target cluster from the display screen.

60. The computer-readable medium of claim 59, wherein the step of displaying the target cluster at the predetermined proximity distance from the activation gesture'"'"'s location comprises:

determining a center point of the target cluster, the center point determined according to the spatial arrangement of the target cluster;

determining a vector originating at the activation gesture'"'"'s origin and proceeding in the activation gesture'"'"'s direction; and

displaying the target cluster on the display screen such that the target cluster'"'"'s center point is located on the determined vector at the predetermined proximity distance from the activation gesture'"'"'s location.

61. The computer-readable medium of claim 59, wherein the step of displaying the target cluster at a predetermined proximity distance from the activation gesture'"'"'s location further comprises:

displaying a band between each target icon in the target cluster and a corresponding base icon.

62. The computer-readable medium of claim 59, wherein the step of monitoring for and detecting a user action dismissing the target cluster comprises monitoring for and detecting a release gesture and, upon detecting a release gesture, determining whether the release gesture is made on a target icon in the target cluster and, if so:

determining the associated target icon upon which the release gesture is made; and

activating the content associated with the associated target icon in accordance with the activation gesture'"'"'s selection.

63. The computer-readable medium of claim 62, wherein the step of activating the content associated with the associated target icon in accordance with the activation gesture'"'"'s selection comprises:

determining whether the activation gesture'"'"'s selection is a nonempty selection having a source icon and, if so, activating the content associated with the associated target icon with the source icon.

64. The computer-readable medium of claim 62, wherein the step of monitoring for and detecting an action dismissing the target cluster further comprises:

(a) monitoring for and detecting a movement gesture; and

(b) upon detecting a movement gesture, determining whether the movement gesture is an action dismissing the target cluster, such that;

(i) if the movement associated with the movement gesture is made in a direction toward any of the target icons in the target cluster, the movement gesture is not an action dismissing the target cluster;

(ii) if the movement associated with the movement gesture is made in a direction away from any of the target icons in the target cluster, and the amount of movement associated with the movement gesture is less than a predetermined tolerance amount, the movement gesture is not an action dismissing the target cluster;

(iii) otherwise, the movement gesture is an action dismissing the target cluster.