Apparatus and method for application sharing in a graphic user interface

• US 5,758,110 A
• Filed: 04/30/1997
• Issued: 05/26/1998
• Est. Priority Date: 06/17/1994
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Hicks, David G., Boss, Dale W.
• Primary Examiner(s): Feild, Joseph H.
• Assistant Examiner(s): HONG, STEPHEN S
• Application Number:
  US08/846,680
• Time in Patent Office: 391 Days
• Field of Search: 395/807, 395/788, 395/806, 395/329, 395/330, 395/331, 395/332, 395/335, 395/340, 395/346, 395/971, 395/200.01, 395/200.02, 395/200.03, 395/200.04
• US Class Current: 715/751
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

Methods and apparatus for task based application sharing in a graphic user interface such as for Windows® are provided. A host user designates an application to be shared, referred to as a shared application. Another user at a remote location, referred to as the client user, shares control of the shared application. The shared application runs and executes only on the host system. The client system has a rectangular area on the display screen within which all shared applications are displayed. The client system renders an image of all windows of a shared application including pop-up dialogs and Menus without also displaying unshared applications. Further, both the client and the host users continue to perform normal operations outside of the shared rectangular area, and the host user defines the tasks which are to be shared.

Claims

1. A method of remote control application sharing in a graphic user interface environment, said method comprising the steps of:

selecting applications to be shared, said selection of said applications to be shared performed only by a host user using a host system;

sharing control of said applications running on said host system, said sharing performed by said host user using said host system and a client user using a client system, without affecting operations performed on unshared applications on said host system by said host user;

displaying a rectangle on a display device of said client system within which shared applications are displayed and operated on by said client user, said shared applications on said host system being displayed on a display device of said host system without said rectangle, said display device of said host system able to display both said shared and said unshared applications at the same time;

determining whether user inputs to said client system are for a shared or for an unshared application, said determination being made on said client system; and

transmitting said inputs which are determined to be for a shared application to said host system to be held in a host system event queue for execution as if a user on said host system had caused the inputs locally.

2. The method of claim 1, further comprising the steps of:

processing a graphic user interface application by a CPU in said host system; and

processing a graphic user interface by a CPU in said client system.

3. The method of claim 1 further comprising the steps of:

creating a communication packet containing drawing information needed for a display driver to perform drawings prompted by a graphic user interface application on said host system; and

transmitting said communication packet from said host system to said client system;

displaying a duplicate image of said drawings performed on said host system on said display device of said client system.

4. The method of claim 1 further comprising the step of displaying operations on said shared applications.

5. The method of claim 1 further comprising the step of resizing said rectangle on said display device of said client system to match changes in size of shared application windows on said host system.

6. The method of claim 1 further comprising the step of detecting children of shared applications.

7. The method of claim 1 further comprising the step of detecting said shared application windows covered by an unshared application window on said host system.

8. The method of claim 1 further comprising the step of displaying a host cursor and a client cursor on said display device of said client system.

9. The method of claim 1 further comprising the step of restricting display driver rendering of fonts for text drawing performed on said shared applications.

10. The method of claim 1 further comprising the step of restricting capabilities of a display driver of said host system.

11. The method of claim 1 further comprising the step of scrolling and resizing said rectangle.

12. The method of claim 1 further comprising the step of transmitting keyboard movements performed on said shared applications from said client system to said host system.

13. The method of claim 1 further comprising the step of transmitting mouse movements performed on said shared applications from said client system to said host system.

14. The method of claim 3 wherein said step of transmitting further comprises the steps of:

constructing a graphic user interface call to perform said drawings on said client system; and

displaying said duplicate image of said drawings on said client system, said displaying performed by said display driver.

15. The method of claim 5 further comprising the steps of:

interfacing a graphic user interface application with a graphical device interface on said host system;

retrieving drawing information used by a display driver to perform drawings prompted by said graphic user interface application on said host system; and

drawing drawings prompted by said graphic user interface application on a said display device of said host system.

16. The method of claim 5 further comprising the steps of:

enumerating said windows belonging to a shared application on said host system;

determining if each new window from said windows belongs to said shared application on said host system;

adding said each new window to a shared windows list;

creating a communication packet listing said each new window in said shared windows list and the size of said each new window;

transmitting said communication packet from said host system to said client system;

computing a new rectangle size that encompass an old rectangle size and said each new window; and

resizing said old rectangle if said new rectangle size is not equal to said old rectangle size.

17. The method of claim 6 further comprising the steps of:

determining if a new application on said host system is a child of a shared application;

adding said new application to a shared application list if said new application is a child of a shared application;

determining if said new application has a parent application;

determining if said parent application is a child of a shared application;

adding said parent application to said shared application list if said parent application is a child of a shared application; and

determining if said parent application has a parent application.

18. The method of claim 7 further comprising the steps of:

enumerating windows on a display screen of said host system from top of said display screen to bottom of said display screen and detecting windows on said display screen;

adding said windows to a window list;

determining if each said windows is a shared application, said shared application is an application being shared by said host system and said client system;

marking a new window added to said window list as a shared application if said new window is a shared application;

marking said new window added to said window list as an unshared application if said new window is not a shared application;

creating a communication packet containing said window list;

transmitting said communication packet to said client system;

receiving said communication packet from said host system;

detecting windows in said data packet, said detecting performed in said client system;

determining if a new window in said communication packet is a shared application;

expanding said rectangle to encompass said window;

shading said new window visible in said rectangle if said new window is not a shared application; and

making visible an area on said window which is a shared application if said window is covered by a shaded window, wherein said shaded window is an application which is not shared.

19. The method of claim 8 further comprising the steps of:

detecting a mouse movement by a host cursor on said host system;

transmitting information on said mouse movement from said host system to said client system;

receiving said information on said mouse movement from said host system;

drawing an image of said host cursor on said rectangle of said client system using said information on said mouse movement; and

displaying said host cursor on said client system with a client cursor on said client system as one cursor if said client cursor is inside of said rectangle.

20. The method of claim 19 wherein said step of displaying further comprises the step of repositioning said client cursor to equal a position of said host cursor on said rectangle of said client system to display and move said host cursor and said client cursor as one.

21. The method of claim 9 further comprising the steps of:

drawing a character string on a shared application running on said host system;

selecting a font type for said character string;

modifying an internal table containing capabilities of said host system display driver to indicate extent of said capabilities available to said host system display driver;

producing a display of said character string on said display device of said host system;

sending a communication packet containing information on said drawing of said character string from said host system to said client system;

receiving said communication packet sent by said host system; and

reproducing said character string on said display device of said client system using a display driver directly and thus bypassing font limitations on said client system.

22. The method of claim 10 further comprising the steps of:

modifying an internal table containing capabilities of a host display driver to indicate that said host system display driver only has a set of required minimum functions and does not have any optional functions, said modifying occurring when said host system is sharing an application with said client system;

examining said internal table, and using said set of required minimum functions to perform drawing operations in said shared application; and

modifying said internal table into said internal table'"'"'s original state, said original state allowing for said host system to make full use of host display driver capabilities.

23. The method of claim 11 further comprising the steps of:

performing a mouse action on a shared application window;

capturing a cursor, said cursor captured by said shared application window;

scrolling said shared application in said rectangle on said client system, when said cursor reaches edge of said rectangle;

resizing said rectangle, when said cursor reaches edge of said rectangle and said shared application in said rectangle is fully scrolled in that direction; and

restoring size of said rectangle when said cursor is no longer captured.

24. The method of claim 12 further comprising the steps of:

performing keyboard movements on said client system;

transmitting communication packets containing said keyboard movements to said host system, if said keyboard movements are performed on a shared application; and

performing said keyboard movements on said shared application on said host system.

25. The method of claim 13 further comprising the steps of:

performing mouse movements on said client system;

transmitting communication packets containing said mouse movements to a sensor application of said host system, if said mouse movements are performed on said shared application; and

performing said mouse movements on said shared application on said host system.

26. An apparatus for remote control application sharing in a graphic user interface environment, said apparatus comprising:

means for selecting applications to be shared, said selection of said applications to be shared performed only by a host user using a host system;

means for sharing control of said applications running on said host system, said sharing performed by said host user using said host system and a client user using a client system, without affecting operations performed on unshared applications on said host system by said host user;

means for displaying a rectangle on a display device of said client system within which shared applications are displayed and operated on by said client user, said shared applications on said host system being displayed on a display device of said host system without said rectangle, said display device of said host system able to display both said shared and said unshared applications at the same time;

means for determining whether user inputs are for a shared or for an unshared application, said determination being made on said client system; and

means for transmitting said inputs which are determined to be for a shared application to said host system to be held in a host system event queue for execution as if a user on said host system had caused the inputs locally.

27. The apparatus of claim 26, further comprising:

means for processing a graphic user interface application by a CPU in said host system; and

means for processing a graphic user interface application by a CPU in said client system.

28. The apparatus of claim 26 further comprising:

means for creating a communication packet containing drawing information needed for a display driver to perform drawings prompted by a graphic user interface application on said host system;

means for transmitting said communication packet from said host system to said client system; and

means for displaying a duplicate image of said drawings performed on said host system on said display device of said client system.

29. The apparatus of claim 26 further comprising means for displaying operations on said shared applications.

30. The apparatus of claim 26 further comprising means for resizing said rectangle on said display device of said client system to match changes in size of shared application windows on said host system.

31. The apparatus of claim 26 further comprising means for detecting children of shared applications.

32. The apparatus of claim 26 further comprising means for detecting said shared application windows covered by an unshared application window on said host system.

33. The apparatus of claim 26 further comprising means for displaying a host cursor and a client cursor on said display device of said client system.

34. The apparatus of claim 26 further comprising means for restricting display driver rendering of fonts for text drawing performed on said shared applications.

35. The apparatus of claim 26 further comprising means for restricting capabilities of a display driver of said host system.

36. The apparatus of claim 26 further comprises means for scrolling and resizing said rectangle.

37. The apparatus of claim 26 further comprising means for transmitting keyboard movements performed on said shared applications from said client system to said host system.

38. The apparatus of claim 26 further comprising means for transmitting mouse movements performed on said shared applications from said client system to said host system.

39. The apparatus of claim 28 wherein said means for transmitting further comprises:

means for constructing a graphic user interface call to perform said drawings on said client system; and

means for displaying said duplicate image of said drawings on said client system, said displaying performed by said display driver.

40. The apparatus of claim 30 further comprising:

means for interfacing a graphic user interface application with a graphical device interface on said host system;

means for retrieving drawing information used by a display driver to perform drawings prompted by said graphic user interface application on said host system; and

means for drawing drawings prompted by said graphic user interface application on a said display device of said host system.

41. The apparatus of claim 30 further comprising:

means for enumerating said windows belonging to a shared application on said host system;

means for determining if each new window from said windows belongs to said shared application on said host system;

means for adding said each new window to a shared windows list;

means for creating a communication packet listing said each new window in said shared windows list and the size of said each new window;

means for transmitting said communication packet from said host system to said client system;

means for computing a new rectangle size that encompass an old rectangle size and said each new window; and

means for resizing said old rectangle if said new rectangle size is not equal to said old rectangle size.

42. The apparatus of claim 31 further comprising:

means for determining if a new application on said host system is a child of a shared application;

means for adding said new application to said shared application list if said new application is a child of a shared application;

means for determining if said new application has a parent application;

means for determining if said parent application is a child of a shared application;

means for adding said parent application to said shared application list if said parent application is a chile of a shared application; and

means for determining if said parent application has a parent application.

43. The apparatus of claim 32 further comprising:

means for enumerating windows on a display screen of said host system from top of said display screen to bottom of said display screen and detecting windows on said display screen;

means for adding said windows to a window list;

means for determining if each said windows is a shared application, said shared application is an application being shared by said host system and said client system;

means for marking a new window added to said window list as a shared application if said new window is a shared application;

means for marking said new window added to said window list as an unshared application if said new window is not a shared application;

means for creating a communication packet containing said window list;

means for transmitting said communication packet to said client system;

means for receiving said communication packet from said host system;

means for detecting windows in said data packet, said detecting performed in said client system;

means for determining if a new window in said communication packet is a shared application;

means for expanding said rectangle to encompass said window;

means for shading said new window visible in said rectangle if said new window is not a shared application; and

means for making visible an area on said window which is a shared application if said window is covered by a shaded window, wherein said shaded window is an application which is not shared.

44. The apparatus of claim 33 further comprising:

means for detecting a mouse movement by a host cursor on said host system;

means for transmitting information on said mouse movement from said host system to said client system;

means for receiving said information on said mouse movement from said host system;

means for drawing an image of said host cursor on said rectangle of said client system using said information on said mouse movement; and

means for displaying said host cursor on said client system with a client cursor on said client system as one cursor if said client cursor is inside of said rectangle.

45. The apparatus of claim 44 wherein said means for displaying further comprises means for repositioning said client cursor to equal a position of said host cursor on said rectangle of said client system to display and move said host cursor and said client cursor as one.

46. The apparatus of claim 35 further comprising:

means for drawing a character string on a shared application running on said host system;

means for selecting a font type for said character string;

means for modifying an internal table containing capabilities of said host system display driver to indicate the extent of said capabilities available to said host system display driver;

means for producing a display of said character string on said display device of said host system;

means for sending a communication packet containing information on said drawing of said character string from said host system to said client system;

means for receiving said communication packet sent by said host system; and

means for reproducing said character string on said display device of said client system using a display driver directly and thus bypassing font limitations on said client system.

47. The apparatus of claim 36 further comprising:

means for modifying an internal table containing capabilities of a host display driver to indicate that said host system display driver only has a set of required minimum functions and does not have any optional functions, said modifying occurring when said host system is sharing an application with said client system;

means for examining said internal table, and using said set of required minimum functions to perform text drawing operations in said shared application; and

means for modifying said internal table into said internal table'"'"'s original state, said original state allowing for said host system to make full use of host display driver capabilities.

48. The apparatus of claim 37 further comprising:

means for performing a mouse action on a shared application window;

means for capturing a cursor, said cursor captured by said shared application window;

means for scrolling said shared application in said rectangle on said client system, when said cursor reaches edge of said rectangle

means for resizing said rectangle, when said cursor reaches edge of said rectangle and said shared application in said rectangle is fully scrolled in that direction; and

means for restoring size of said rectangle when said cursor is no longer captured.

49. The apparatus of claim 38 further comprising:

means for performing keyboard movements on said client system;

means for transmitting communication packets containing said keyboard movements to said host system, if said keyboard movements are performed on a shared application; and

means for performing said keyboard movements on said shared application on said host system.

50. The apparatus of claim 39 further comprising:

means for performing mouse movements on said client system;

means for transmitting communication packets containing said mouse movements to a sensor application of said host system, if said mouse movements are performed on said shared application; and

means for performing said mouse movements on said shared application on said host system.

51. A computer system for remote control application sharing in a graphic user interface environment, said comprising:

means for selecting application to be shared, said selection of said application to be shared performed only by a host user using a host system;

means for sharing of said application running on said host system, said sharing performed by said host user using said host system and a client user using a client system, without affecting operations performed on unshared applications on said host system by said host user;

means for displaying a rectangle on a display device of said client system within which shared applications are displayed and operated on by said client user, said shared applications on said host system being displayed on a display device of said host system without said rectangle, said display device of said host system able to display both said shared and said unshared applications at the same time;

means for determining whether user inputs are for a shared or for an unshared application, said determination being made on said client system;

means for transmitting said inputs which are determined to be for a shared application to said host system to be held in a host system event queue for execution as if a user on said host system had caused the inputs locally; and

means for processing a graphic user interface application by a CPU.

52. The computer system of claim 51 further comprising:

means for creating a communication packet containing drawing information needed for a display driver to perform drawings prompted by a graphic user interface application on said host system;

means for transmitting said communication packet from said host system to said client system; and

means for displaying a duplicate image of said drawings performed on said host system on said display device of said client system.

53. The computer system of claim 51 further comprising means for displaying operations on said shared applications.

54. The computer system of claim 51 further comprising means for resizing said rectangle on said display device of said client system to match changes in size of shared application windows on said host system.

55. The computer system of claim 51 further comprising means for detecting children of shared applications.

56. The computer system of claim 51 further comprising means for detecting said shared application windows covered by an unshared application window on said host system.

57. The computer system of claim 51 further comprising means for displaying a host cursor and a client cursor on said display device of said client system.

58. The computer system of claim 51 further comprising means for restricting display driver rendering of fonts for text drawing performed on said shared applications.

59. The computer system of claim 51 further comprising means for restricting capabilities of a display driver of said host system.

60. The computer system of claim 51 further comprising means for scrolling and resizing said rectangle.

61. The computer system of claim 51 further comprising means for transmitting keyboard movements performed on said shared applications from said client system to said host system.

62. The computer system of claim 51 further comprising means for transmitting mouse movements performed on said shared applications from said client system to said host system.

63. The computer system of claim 52 wherein said means for transmitting further comprises:

means for constructing a graphic user interface call to perform said drawings on said client system; and

means for displaying said duplicate image of said drawings on said client system, said displaying performed by said display driver.

64. The computer system of claim 54 further comprising:

means for interfacing a graphic user interface application with a graphical device interface on said host system;

means for retrieving drawing information used by a display driver to perform drawings prompted by said graphic user interface application on said host system; and

means for drawing drawings prompted by said graphic user interface application on a said display device of said host system.

65. The computer system of claim 54 further comprising:

means for enumerating said windows belonging to a shared application on said host system;

means for determining if each new window from said windows belongs to said shared application on said host system;

means for adding said each new window to a shared windows list;

means for creating a communication packet listing said each new window in said shared windows list and the size of said each new window;

means for transmitting said communication packet from said host system to said client system;

means for computing a new rectangle size that encompass an old rectangle size and said each new window; and

means for resizing said old rectangle if said new rectangle size is not equal to said old rectangle size.

66. The computer system of claim 55 further comprising:

means for determining if a new application on said host system is a child of a shared application;

means for adding said new application to a shared application list if said new application is a child of a shared application;

means for determining if said new application has a parent application;

means for determining if said parent application is a child of a shared application;

means for adding said parent application to said shared application list if said parent application is a child of a shared application; and

means for determining if said parent application has a parent application.

67. The computer system of claim 56 further comprising:

means for enumerating windows on a display screen of said host system from top of said display screen to bottom of said display screen and detecting windows on said display screen;

means for adding said windows to a window list;

means for determining if each said windows is a shared application, said shared application is an application being shared by said host system and said client system;

means for marking a new window added to said window list as a shared application if said new window is a shared application;

means for marking said new window added to said window list as an unshared application if said new window is not a shared application;

means for creating a communication packet containing said window list;

means for transmitting said communication packet to said client system;

means for receiving said communication packet from said host system;

means for detecting windows in said data packet, said detecting performed in said client system;

means for determining if a new window in said communication packet is a shared application;

means for expanding said rectangle to encompass said window;

means for shading said new window visible in said rectangle if said new window is not a shared application; and

means for making visible an area on said window which is a shared application if said window is covered by a shaded window, wherein said shaded window is an application which is not shared.

68. The computer system of claim 57 further comprising:

means for detecting a mouse movement by a host cursor on said host system;

means for transmitting information on said mouse movement from said host system to said client system;

means for receiving said information on said mouse movement from said host system;

means for drawing an image of said host cursor on said rectangle of said client system using said information on said mouse movement; and

means for displaying said host cursor on said client system with a client cursor on said client system as one cursor if said client cursor is inside of said rectangle.

69. The computer system of claim 68 wherein said step of displaying further comprises means for repositioning said client cursor to equal a position of said host cursor on said rectangle of said client system to display and move said host cursor and said client cursor as one.

70. The computer system of claim 58 further comprising:

means for drawing a character string on a shared application running on said host system;

means for selecting a font type for said character string;

means for modifying an internal table containing capabilities of said host system display driver to indicate extent of said capabilities available to said host system display driver;

means for producing a display of said character string on said display device of said host system;

means for sending a communication packet containing information on said drawing of said character string from said host system to said client system;

means for receiving said communication packet sent by said host system; and

means for reproducing said character string on said display device of said client system using a display driver directly and thus bypassing font limitations on said client system.

71. The computer system of claim 59 further comprising:

means for modifying an internal table containing capabilities of a host display driver to indicate that said host system display driver only has a set of required minimum functions and does not have any optional functions, said modifying occurring when said host system is sharing an application with said client system;

means for examining said internal table, and using said set of required minimum functions to perform text drawing operations in said shared application; and

means for modifying said internal table into said internal table'"'"'s original state, said original state allowing for said host system to make full use of host display driver capabilities.

72. The computer system of claim 60 further comprising:

means for performing a mouse action on a shared application window;

means for capturing a cursor, said cursor captured by said shared application window;

means for scrolling shared application in said rectangle on said client system, when said cursor reaches edge of said rectangle;

means for resizing said rectangle, when said cursor reaches edge of said rectangle and said shared application in said rectangle is fully scrolled in that direction; and

means for restoring size of said rectangle when said cursor is no longer captured.

73. The computer system of claim 61 further comprising:

means for performing keyboard movements on said client system;

means for transmitting communication packets containing said keyboard movements to said host system, if said keyboard movements are performed on a shared application; and

means for performing said keyboard movements on said shared application on said host system.

74. The computer system of claim 62 further comprising:

means for performing mouse movements on said client system;

means for transmitting communication packets containing said mouse movements to a sensor application of said host system, if said mouse movements are performed on said shared application; and

means for performing said mouse movements on said shared application on said host system.