System and method for accessing remote screen content

• US 7,231,609 B2
• Filed: 03/05/2003
• Issued: 06/12/2007
• Est. Priority Date: 02/03/2003
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Baudisch, Patrick Markus
• Primary Examiner(s): Lo; Weilun
• Assistant Examiner(s): PHANTANA ANGKOOL, DAVID
• Application Number:
  US10/384,869
• Publication Number:
  US 20040150664A1
• Time in Patent Office: 1,560 Days
• Field of Search: 345/700, 715/740, 715/799, 715/779, 715/863
• US Class Current: 715/769
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/1462 with means for detecting differences between the image stored in the host and the images displayed on the remote displays

Abstract

A system and method for facilitating user access to remote content from a current location is presented. The invention monitors for and detects an activation gesture. The activation gesture may include an empty or nonempty selection, the nonempty selection having a source icon. Upon detecting the activation gesture, a target region is determined according to the activation gesture. Target icons capable of responding to activation gestures'"'"' selection are selected from those icons within the target region. If the activation gesture includes a nonempty selection having a source icon, only those icons within the target region and compatible with the source icon are selected. The target icons are displayed in a cluster in the proximity of the working location until a user action dismissing the cluster is detected.

Claims

1. A method for facilitating user access to remote icons on a display screen, the method comprising:

detecting an activation gesture, the activation gesture having an origin, a direction, a location, and a selection;

selecting base icons on the display screen according to the activation gesture;

generating a target cluster of target icons, wherein each target icon corresponds to a selected base icon remotely displayed on the display screen;

displaying the target cluster at a predetermined proximity distance from the activation gesture'"'"'s location; and

displaying target icons in proximity to the activation gesture'"'"'s location until an action dismissing the target icons is detected; and

upon detecting an action dismissing the target cluster, removing the target cluster from the display screen.

2. The method of claim 1, wherein the activation gesture'"'"'s selection is a nonempty selection having a source icon or an empty selection.

3. The method of claim 2, wherein the activation gesture is a drag gesture.

4. The method of claim 1, wherein selecting base icons on the display screen according to the activation gesture comprises:

determining a central vector originating at the activation gesture'"'"'s origin and extending to the limits of the display screen in the activation gesture'"'"'s direction;

determining a target region defined by two boundary lines originating at the activation gesture'"'"'s origin and diverging away from and on either side of the central vector extending to the limits of the display screen; and

selecting base icons located within the target region according to the activation gesture.

5. The method of claim 4, wherein selecting base icons on the display screen according to the activation gesture further comprises selecting base icons located within the target region and at a distance greater than a predetermined distance from the activation gesture'"'"'s origin.

6. The method of claim 5, wherein the predetermined distance is user configurable.

7. The method of claim 5, wherein selecting base icons on the display screen according to the activation gesture further comprises:

determining whether the activation gesture'"'"'s selection included a source icon and, if so, selecting base icons located within the target region and at a distance greater than the predetermined distance from the activation gesture'"'"'s origin and representing content compatible with the source icon.

8. The method of claim 7, wherein selecting base icons on the display screen according to the activation gesture further comprises:

determining whether the number of selected base icons exceeds a predetermined limit and, if so, repeatedly;

reducing the area of the target region; and

selecting base icons located within the reduced target region and at a distance greater than the predetermined distance from the activation gesture'"'"'s origin and compatible with the source icon if the activation gesture'"'"'s selection includes the source icon;

until the number of selected base icons is less than or equal to the predetermined limit.

9. The method of claim 8, wherein the two boundary lines diverge away from and on either side of the central vector extending to the limits of the display screen at a predetermined angle.

10. The method of claim 9, wherein the predetermined angle is user configurable.

11. The method of claim 9, wherein reducing the area of the target region comprises reducing the predetermined angle at which the boundary lines diverge away from and at either side of the central vector.

12. The method of claim 4, wherein the activation gesture is modified according to a modification condition, and wherein selecting base icons on the display screen according to the activation gesture further comprises:

selecting base icons located within the target region and according to a predetermined criteria associated with the modification condition.

13. The method of claim 1, wherein displaying the target cluster at the predetermined proximity distance from the activation gesture'"'"'s location comprises:

determining a center point of the target cluster, the center point determined according to the spatial arrangement of the target cluster;

determining a vector originating at the activation gesture'"'"'s origin and proceeding in the activation gesture'"'"'s direction; and

displaying the target cluster on the display screen such that the target cluster'"'"'s center point is located on the determined vector at the predetermined proximity distance from the activation gesture'"'"'s location.

14. The method of claim 1, wherein displaying the target cluster at the predetermined proximity distance from the activation gesture'"'"'s location further comprises displaying a band between each target icon in the target cluster and its corresponding base icon.

15. The method of claim 14, wherein displaying the target cluster at the predetermined proximity distance from the activation gesture'"'"'s location further comprises temporarily displaying each base icon in an altered state until an action dismissing the target icons is detected.

16. The method of claim 14, wherein displaying a band between each target icon in the target cluster and the corresponding base icon comprises iteratively selecting each target icon in the target cluster in a closest-first manner and displaying the band between each target icon in the target cluster and its corresponding base icon.

17. The method of claim 14, wherein each band between a target icon in the target cluster and its corresponding base icon narrows on both sides of the band as the band approaches a midpoint between the target icon and its base icon.

18. The method of claim 17, wherein each band narrows according to the distance between a target icon and its base icon, such that the greater the distance between the target icon and its base icon, the more the band narrows.

19. The method of claim 1, wherein detecting an action dismissing the target cluster comprises detecting a release gesture and, upon detecting the release gesture, determining whether the release gesture is made on a target icon in the target cluster, and if so:

determining the associated target icon upon which the release gesture is made; and

activating the content associated with the associated target icon in accordance with the activation gesture'"'"'s selection.

20. The method of claim 19, wherein activating the content associated with the associated target icon in accordance with the activation gesture'"'"'s selection comprises determining whether the activation gesture'"'"'s selection is a nonempty selection having a source icon and, if so, activating the content associated with the associated target icon with the source icon.

21. The method of claim 19, wherein the release gesture is a drop gesture.

22. The method of claim 19, wherein detecting an action dismissing the target cluster further comprises:

detecting a movement gesture; and

determining whether the movement gesture is an action dismissing the target cluster, such that;

if the movement associated with the movement gesture is made in a direction toward any of the target icons in the target cluster, the movement gesture is not an action dismissing the target cluster;

if the movement associated with the movement gesture is made in a direction away from all of the target icons in the target cluster, and the amount of movement associated with the movement gesture is less than a predetermined tolerance amount, the movement gesture is not an action dismissing the target cluster;

otherwise, the movement gesture is an action dismissing the target cluster.

23. A computer system for facilitating user access to remote icons on a display device, the system comprising:

a display device capable of displaying a plurality of icons;

a user input device for interfacing with and interacting with the user; and

a computing device, communicatively coupled to the display device and to the user input device, for;

displaying a plurality of icons on the display device;

detecting an activation gesture, the activation gesture having an origin, a direction, a location, and a selection, and upon detecting the activation gesture;

selecting base icons from the plurality of icons displayed on the display device;

generating a target cluster of target icons, wherein each target icon corresponds to a selected base icon remotely displayed on the display screen;

displaying target icons in proximity to the activation gesture'"'"'s location on the display device until an action dismissing the target icons is detected; and

upon detecting an action dismissing the target cluster, removing the target cluster from the display screen.

24. The computer system of claim 23, wherein the activation gesture'"'"'s selection is a nonempty selection having a source icon or an empty selection.

25. The computer system of claim 24, wherein the activation gesture is a drag gesture.

26. The computer system of claim 23, wherein the computing device selects base icons from the plurality of icons displayed on the display device by:

determining a central vector originating at the activation gesture'"'"'s origin and extending to the limits of the display device according to the activation gesture'"'"'s direction;

determining a target region defined by two boundary lines originating at the activation gesture'"'"'s origin and diverging away from and on either side of the central vector extending to the limits of the display device; and

selecting base icons displayed within the target region.

27. The computer system of claim 26, wherein the computing device further selects base icons from the plurality of icons displayed on the display device by:

selecting base icons displayed within the target region and at a distance greater than a predetermined distance from the activation gesture'"'"'s origin.

28. The computer system of claim 27, wherein the predetermined distance is user configurable.

29. The computer system of claim 27, wherein the computing device further selects base icons from the plurality of icons displayed on the display device by:

determining whether the activation gesture'"'"'s selection included a source icon and, if so, selecting base icons displayed within the target region and at a distance greater than a predetermined distance from the activation gesture'"'"'s origin and representing content compatible with the source icon.

30. The computer system of claim 29, wherein the computing device further selects base icons from the plurality of icons displayed on the display device by determining whether the number of selected base icons exceeds a predetermined limit and, if so, repeatedly:

reducing the area of the target region; and

selecting base icons located within the reduced target region and at a distance greater than the predetermined distance from the activation gesture'"'"'s origin and compatible with the source icon if the activation gesture'"'"'s selection included a source icon;

until the number of selected base icons is less than or equal to the predetermined limit.

31. The computer system of claim 30, wherein the two boundary lines diverge away from and at either side of the central vector extending to the limits of the display screen at a predetermined angle.

32. The computer system of claim 31, wherein the predetermined angle is user configurable.

33. The computer system of claim 31, wherein the computing device reduces the area of the target region by reducing the predetermined angle at which the boundary lines diverge away from and at either side of the central vector.

34. The computer system of claim 26, wherein the activation gesture is modified according to a modification condition, and wherein the computer device further selects base icons from the plurality of icons on the display device by:

selecting base icons located within the target region and according to a predetermined criteria associated with the modification condition.

35. The computer system of claim 23, wherein the computing device displays the target cluster on the display device at the predetermined proximity distance from the activation gesture'"'"'s location by:

determining a center point of the target cluster, the center point determined according to the spatial arrangement of the target cluster;

determining a vector originating at the activation gesture'"'"'s origin and proceeding in the activation gesture'"'"'s direction; and

displaying the target cluster on the display device such that the target cluster'"'"'s center point is located on the determined vector at the predetermined proximity distance from the activation gesture'"'"'s location.

36. The computer system of claim 23, wherein the computing device further displays the target cluster on the display device at the predetermined proximity distance from the activation gesture'"'"'s location by:

displaying a band between each target icon in the target cluster and a corresponding base icon.

37. The computer system of claim 36, wherein the computing device further displays the target cluster on the display device at the predetermined proximity distance from the activation gesture'"'"'s location by:

temporarily displaying each base icon in an altered slate until an action dismissing the target icons is detected.

38. The computer system of claim 36, wherein the computing device displays the band between each target icon in the target cluster and the corresponding base icon by:

iteratively selecting each target icon in the target cluster in a closest-first manner and displaying the band between each target icon and its corresponding base icon.

39. The computer system of claim 36, wherein the computing device displays the band between each target icon in the target cluster and the corresponding base icon by:

displaying each band as narrowing on both sides as the band approaches a midpoint between the target icon and the corresponding base icon.

40. The computer system of claim 39, wherein the computing device further displays each band as narrowing on both sides as the band approaches a midpoint between the target icon and the corresponding base icon by:

narrowing the band according to the distance between the target icon and the corresponding base icon, such that the greater the distance between the target icon and the base icon, the more the band narrows.

41. The computer system of claim 23, wherein the computing device detects an action dismissing the target cluster by detecting a release gesture and, upon detecting a release gesture, determining whether the release gesture is made on a target icon in the target cluster and, if so:

determining the associated target icon on which the release gesture is made; and

activating the content associated with the associated target icon in accordance with the activation gesture'"'"'s selection.

42. The computer system of claim 41, wherein the computing device activates the content associated with the associated target icon in accordance with the activation gesture'"'"'s selection by:

determining whether the activation gesture'"'"'s selection is a nonempty selection having a source icon and, if so, activating the content associated with the associated target icon with the source icon.

43. The computer system of claim 41, wherein the release gesture is a drop gesture.

44. The computer system of claim 41, wherein the computing device further detects an action dismissing the target cluster by:

detecting a movement gesture; and

determining whether the movement gesture is an action dismissing the target cluster, such that;

if the movement associated with the movement gesture is made in a direction toward any of the target icons in the target cluster, the movement action is not an action dismissing the target cluster;

if the movement associated with the movement gesture is made in a direction away from all of the target icons in the target cluster, and the amount of movement associated with the movement gesture is less than a predetermined tolerance amount, the movement gesture is not an action dismissing the target cluster;

otherwise, the movement gesture is an action dismissing the target cluster.

45. The computer system of claim 23, wherein the display device comprises a plurality of display panels cooperating to present a single visual display area.

46. The computer system of claim 45, wherein the plurality of display panels include a pressure-sensitive display panel.

47. The computer system of claim 46, wherein the pressure-sensitive display panel is also the user input device.

48. The computer system of claim 23, wherein the display device is a pressure-sensitive display device.

49. The computer system of claim 48, wherein the pressure-sensitive display device is also the user input device.

50. The computer system of claim 23, wherein the user input device is a pen-based input device.

51. The computer system of claim 23, wherein the user input device is a touch-based input device.

52. A computer-readable medium having computer-executable instructions for carrying out the method comprising the steps of:

detecting an activation gesture, the activation gesture having an origin, a direction, a location, and a selection;

selecting base icons on a display screen according to the activation gesture;

generating a target cluster of target icons, wherein each target icon corresponds to a selected base icon remotely displayed on the display screen;

displaying target icons in proximity to the activation gesture'"'"'s location until an action dismissing the target icons is detected; and

upon detecting an action dismissing the target cluster, removing the target cluster from the display screen.

53. The computer-readable medium of claim 52, wherein the step of selecting base icons on the display screen according to the activation gesture comprises:

determining a central vector originating at the activation gesture'"'"'s origin and extending to the limits of the display screen in the activation gesture'"'"'s direction;

determining a target region defined by two boundary lines originating at the activation gesture'"'"'s origin and diverging away from and on either side of the central vector extending to the limits of the display screen; and

selecting base icons located within the target region according to the activation gesture.

54. The computer-readable medium of claim 53, wherein the step of selecting base icons on the display screen according to the activation gesture further comprises:

selecting base icons located within the target region and at a distance greater than a predetermined distance from the activation gesture'"'"'s origin.

55. The computer-readable medium of claim 54, wherein the step of selecting icons on the display screen according to the activation gesture further comprises:

determining whether the activation gesture'"'"'s selection includes a source icon and, if so, selecting base icons located within the target region and at a distance greater than the predetermined distance from the activation gesture'"'"'s origin and representing content compatible with the source icon.

56. The computer-readable medium of claim 55, wherein the step of selecting base icons on the display screen according to the activation gesture further comprises determining whether the number of selected target icons exceeds a predetermined limit and, if so, repeatedly:

reducing the area of the target region; and

selecting base icons located within the reduced target region and at a distance greater than the predetermined distance from the activation gesture'"'"'s origin and compatible with the source icon if the activation gesture'"'"'s selection includes a source icon;

until the number of selected base icons is less than or equal to the predetermined limit.

57. The computer-readable medium of claim 52, wherein the step of displaying the target cluster at the predetermined proximity distance from the activation gesture'"'"'s location comprises:

determining a center point of the target cluster, the center point determined according to the spatial arrangement of the target cluster;

determining a vector originating at the activation gesture'"'"'s origin and proceeding in the activation gesture'"'"'s direction; and

displaying the target cluster on the display screen such that the target cluster'"'"'s center point is located on the determined vector at the predetermined proximity distance from the activation gesture'"'"'s location.

58. The computer-readable medium of claim 52, wherein the step of displaying the target cluster at a predetermined proximity distance from the activation gesture'"'"'s location further comprises:

displaying a band between each target icon in the target cluster and a corresponding base icon.

59. The computer-readable medium of claim 52, wherein the step of detecting an action dismissing the target cluster comprises detecting a release gesture and, upon detecting the release gesture, determining whether the release gesture is made on a target icon in the target cluster and, if so:

determining the associated target icon upon which the release gesture is made; and

activating the content associated with the associated target icon in accordance with the activation gesture'"'"'s selection.

60. The computer-readable medium of claim 59, wherein the step of activating the content associated with the associated target icon in accordance with the activation gesture'"'"'s selection comprises:

determining whether the activation gesture'"'"'s selection is a nonempty selection having a source icon and, if so, activating the content associated with the associated target icon with the source icon.

61. The computer-readable medium of claim 59, wherein the step of detecting an action dismissing the target cluster further comprises:

detecting a movement gesture; and

determining whether the movement gesture is an action dismissing the target cluster, such that;

if the movement associated with the movement gesture is made in a direction toward any of the target icons in the target cluster, the movement gesture is not an action dismissing the target cluster;

if the movement associated with the movement gesture is made in a direction away from all of the target icons in the target cluster, and the amount of movement associated with the movement gesture is less than a predetermined tolerance amount, the movement gesture is not an action dismissing the target cluster;

otherwise, the movement gesture is an action dismissing the target cluster.