Method and architecture of an event transform oriented operating environment for a personal mobile display system

• US 20030218632A1
• Filed: 05/23/2002
• Published: 11/27/2003
• Est. Priority Date: 05/23/2002
• Status: Abandoned Application

• Current Assignee: AirSpeak, Inc. (Turbine Aviation, Inc.)
• Original Assignee: AirSpeak, Inc. (Turbine Aviation, Inc.)
• Inventors: Rioux, Tom, Wu, Jiangmin, Altwies, Tony, Hodzic, Migdat
• Application Number:
  US10/155,601
• Publication Number:
  US 20030218632A1
• Time in Patent Office: Days
• US Class Current: 715/740
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units

Abstract

Disclosed herein is a personal mobile display system that operate in cooperation with a remote system that include a processor for executing PC compatible application programs capable of generating and displaying a plurality of operational functions and data in response to user specified input. In addition, the present invention is directed to a method of providing event driven functionality that allows a personal mobile display system to system to cooperatively execute application programs in conjunction with a remote computer system by detecting that a change to the functionality, appearance and logical position of an application-oriented functional element and categorizing type of event. Once categorized the present invention encapsulates one or more events as an event transform to be transmitted to a corresponding remote computer system. Here the event transform is an encapsulation of a source event, method and or actions that may be moved in space and or instance in time that is transmitted from the personal mobile display system to the remote computer system. Once the corresponding system processes the event transform and transmits it back to the personal mobile display system to reconstruct and display the changes in the form of one or more rectangular display segments that were encapsulated in the received event transforms in accordance with the application program from the remote computer system.

Claims

1. A method for operating in cooperation with a desktop computer system executing an array of application programs, performed by a personal mobile display system comprising steps of:

detecting that an event transform has occurred;

determining type of event transform wherein an event transform is comprised of a user-directed event or alternatively a device event transform;

interpreting the operative function of each event transform;

converting event transform into the appropriate format for transmissions across network to the corresponding remote desktop computer system;

transmitting event transform across network via at least one communication protocol to the corresponding remote desktop computer system;

receiving packaged event transform from the corresponding computer system; and

displaying the results of event transform received from the corresponding computer system

2. The method as recited in claim 1, wherein the detected event is a user-directed event causes the cursor to move one position to another on the video display screen of the corresponding computer system, where a user-directed event transform that includes a keyboard, a mouse and a screen entry caused by a stylus moving across a touch sensitive screen.

3. The method as recited in claim 1, wherein the detected device event transform communicated to the corresponding computer system includes the operational state of the personal mobile display system, where the operative state includes the status and availability on-board resources of the personal mobile display system.

4. The method as recited in claim 1, wherein the personal mobile display system transmits the packaged event to the corresponding remote computer system across a network via a TCP/IP connected socket to the corresponding host system for processing.

5. The method as recited in claim 4, wherein the personal mobile display system transmits the packaged event across the network to the corresponding remote computer system across a network via a connectionless UDP datagram to the corresponding host system for processing.

6. The method as recited in claim 1, wherein the corresponding host system performs the steps for processing the event transform from received the personal mobile display system comprising:

determining type of event transform to process;

processing event transform in accordance to type of event

displaying wherein changes as a result of host resident application routines, a control event as well as user directed actions in accordance with the processed event transform are displayed on the host system;

storing rectangular display regions containing any changes resulting from an application process or user specified action that has occurred in a display stack, wherein the rectangular display region is used to refresh and update the personal mobile display system;

packaging the stored display regions for transfer to the personal mobile display system; and

transferring the updated rectangular regions across the network to the personal mobile display system.

7. The method as recited in claim 6, wherein if the event transform is data sent from the personal mobile display system in response made by a host resident application routine, then, the data is then returned to that routine.

8. The method as recited in claim 6, wherein if the event transform type is a command response message, the corresponding host system then issues a request to display the results of the command response.

9. The method as recited in claim 6, wherein if event transform type is a control event such as a keyboard input or mouse movement caused by the stylus moving across the touch sensitive screen of the personal mobile display system, the host system sends the keyboard or mouse input to the appropriate application for processing.

10. The method as recited in claim 6, wherein only when a number of updated rectangular regions have been accumulated in the stack does the host system package transfer the updated rectangular regions across the network to the personal mobile display system.

11. The method as recited in claim 6, If the host system determines that there are any commands and a program request for data needed to be transmitted to the personal mobile display system, the host system then repeats the steps of packaging and transmitting the commands and or any program requests for data across the network to the personal mobile display system.

12. The method as recited in claim 1, wherein the personal mobile display system receives a event transform from the corresponding host system, the personal mobile display system determines whether the event transform received is either at least one updated region, a control command or a program request for data.

13. The method as recited in claim 1, wherein if the received event transform is at least one updated region to be displayed, the personal mobile display system, then paints the appropriate rectangular areas of the display with the received updated region.

14. The method as recited in claim 1, wherein if the received event transform is a request for data, the personal mobile display system retrieves the requested data, prepares the data for transfer back to the host and then transmits it across the network to the host system.

15. The method as recited in claim 1, wherein if the received event transform is a control command, the personal mobile display system executes command or processes the request for system status, prepares the response that is due back to the host and then transmits it across the network to the host system.

16. A computer program product that includes a computer useable medium having computer readable code embodied therein that operates in cooperation with a remote system to execute an array of application programs, comprising:

a computer readable program code device of a personal mobile display system configured to detect that an event transform has occurred;

a computer readable program code device of a personal mobile display system configured to determine type of event transform wherein an event transform is comprised of a user-directed event or alternatively a device event transform;

a computer readable program code device of a personal mobile display system configured to interpret the operative function of each event transform;

a computer readable program code device of a personal mobile display system configured to package event transform in appropriate format for transmissions across network to the corresponding remote desktop computer system;

a computer readable program code device of a personal mobile display system configured to transmit packaged event transform across network via at least one communication protocol to the corresponding remote desktop computer system;

a computer readable program code device of a personal mobile display system configured to receive packaged event transform from the corresponding remote desktop computer system; and

a computer readable program code device of a personal mobile display system configured to display the results of event transform received from the corresponding remote desktop computer system

17. The computer program product as recited in claim 16, wherein the detected event is a user-directed event causes the cursor to move one position to another on the video display screen of the corresponding remote desktop computer system, where a user-directed event transform that includes a keyboard, a mouse and a screen entry caused by a stylus moving across a touch sensitive screen.

18. The computer program product as recited in claim 16, wherein the detected device event transform communicated to the corresponding computer system includes the operational state of the personal mobile display system, where the operative state includes the status and availability on-board resources of the personal mobile display system.

19. The computer program product as recited in claim 16, wherein the personal mobile display system transmits the packaged event to the corresponding remote computer system across a network via a TCP/IP connected socket to the corresponding host system for processing.

20. The computer program product as recited in claim 19, wherein the personal mobile display system transmits the packaged event across the network to the corresponding remote computer system across a network via a connectionless UDP datagram to the corresponding host system for processing.

21. The computer program product as recited in claim 16, wherein the corresponding host system performs the steps for processing the event transform from received the personal mobile display system comprising:

a computer readable program code device of a personal mobile display system configured to determine type of event transform to process;

a computer readable program code device of a personal mobile display system configured to process event transform in accordance to type of event

a computer readable program code device of a personal mobile display system configured to display wherein changes as a result of host resident application routines, a control event as well as user directed actions in accordance with the processed event transform are displayed on the host system;

a computer readable program code device of a personal mobile display system configured to store rectangular display regions containing any changes resulting from an application process or user specified action that has occurred in a display stack, wherein the rectangular display region is used to refresh and update the personal mobile display system;

a computer readable program code device of a personal mobile display system configured to package the stored display regions for transfer to the personal mobile display system; and

a computer readable program code device of a personal mobile display system configured to transfer the updated rectangular regions across the network to the personal mobile display system.

22. The computer program product as recited in claim 21, wherein if the event transform is data sent from the personal mobile display system in response made by a host resident application routine, then, the data is then returned to that routine.

23. The computer program product as recited in claim 21, wherein if the event transform type is a command response message, the corresponding host system then issues a request to display the results of the command response.

24. The computer program product as recited in claim 21, wherein if event transform type is a control event such as a keyboard input or mouse movement caused by the stylus moving across the touch sensitive screen of the personal mobile display system, the host system sends the keyboard or mouse input to the appropriate application for processing.

25. The computer program product as recited in claim 21, wherein only when a number of updated rectangular regions have been accumulated in the stack does the host system package transfer the updated rectangular regions across the network to the personal mobile display system.

26. The computer program product as recited in claim 21, wherein if the host system determines that there are any commands and a program request for data needed to be transmitted to the personal mobile display system, the host system then repeats the steps of packaging and transmitting the commands and or any program requests for data across the network to the personal mobile display system.

27. The computer program product as recited in claim 16, wherein the personal mobile display system receives a event transform from the corresponding host system, the personal mobile display system determines whether the event transform received is either at least one updated region, a control command or a program request for data.

28. The computer program product as recited in claim 16, wherein if the received event transform is at least one updated region to be displayed, the personal mobile display system, then paints the appropriate rectangular areas of the display with the received updated region.

29. The computer program product as recited in claim 16, wherein if the received event transform is a request for data, the personal mobile display system retrieves the requested data, prepares the data for transfer back to the host and then transmits it across the network to the host system.

30. The computer program product as recited in claim 16, wherein if the received event transform is a control command, the personal mobile display system executes command or processes the request for system status, prepares the response that is due back to the host and then transmits it across the network to the host system.

31. A system for operating in cooperation with a computer system to execute an array of application programs, performed by a personal mobile display system comprising steps of:

circuitry for detecting that an event transform has occurred;

circuitry for determining type of event transform wherein an event transform is comprised of a user-directed event or alternatively a device event transform;

circuitry for interpreting the operative function of each event transform;

circuitry for packaging event transform in appropriate format for transmissions across network to the corresponding remote desktop computer system;

circuitry for transmitting packaged event transform across network via at least one communication protocol to the corresponding remote desktop computer system;

circuitry for receiving packaged event transform from the corresponding remote desktop computer system; and

circuitry for displaying the results of event transform received from the corresponding remote desktop computer system

32. The system as recited in claim 31, wherein the detected event is a user-directed event causes the cursor to move one position to another on the video display screen of the corresponding remote desktop computer system, where a user-directed event transform that includes a keyboard, a mouse and a screen entry caused by a stylus moving across a touch sensitive screen.

33. The system as recited in claim 31, wherein the detected device event transform communicated to the corresponding computer system includes the operational state of the personal mobile display system, where the operative state includes the status and availability on-board resources of the personal mobile display system.

34. The system as recited in claim 31, wherein the personal mobile display system transmits the packaged event to the corresponding remote computer system across a network via a TCP/IP connected socket to the corresponding host system for processing.

35. The system as recited in claim 34, wherein the personal mobile display system transmits the packaged event across the network to the corresponding remote computer system across a network via a connectionless UDP datagram to the corresponding host system for processing.

36. The system as recited in claim 31, wherein the corresponding host system performs the steps for processing the event transform from received the personal mobile display system comprising:

circuitry for determining type of event transform to process;

circuitry for processing event transform in accordance to type of event;

circuitry for displaying wherein changes as a result of host resident application routines, a control event as well as user directed actions in accordance with the processed event transform are displayed on the host system;

circuitry for storing rectangular display regions containing any changes resulting from an application process or user specified action that has occurred in a display stack, wherein the rectangular display region is used to refresh and update the personal mobile display system;

circuitry for packaging the stored display regions for transfer to the personal mobile display system; and

circuitry for transferring the updated rectangular regions across the network to the personal mobile display system.

37. The system as recited in claim 36, wherein if the event transform is data sent from the personal mobile display system in response made by a host resident application routine, then, the data is then returned to that routine.

38. The system as recited in claim 36, wherein if the event transform type is a command response message, the corresponding host system then issues a request to display the results of the command response.

39. The system as recited in claim 36, wherein if event transform type is a control event such as a keyboard input or mouse movement caused by the stylus moving across the touch sensitive screen of the personal mobile display system, the host system sends the keyboard or mouse input to the appropriate application for processing.

40. The system as recited in claim 36, wherein only when a number of updated rectangular regions have been accumulated in the stack does the host system package transfer the updated rectangular regions across the network to the personal mobile display system.

41. The system as recited in claim 36, If the host system determines that there are any commands and a program request for data needed to be transmitted to the personal mobile display system, the host system then repeats the steps of packaging and transmitting the commands and or any program requests for data across the network to the personal mobile display system.

42. The system as recited in claim 36, wherein the personal mobile display system receives a event transform from the corresponding host system, the personal mobile display system determines whether the event transform received is either at least one updated region, a control command or a program request for data.

43. The system as recited in claim 36, wherein if the received event transform is at least one updated region to be displayed, the personal mobile display system, then paints the appropriate rectangular areas of the display with the received updated region.

44. The system as recited in claim 36, wherein if the received event transform is a request for data, the personal mobile display system retrieves the requested data, prepares the data for transfer back to the host and then transmits it across the network to the host system.

45. The system as recited in claim 36, wherein if the received event transform is a control command, the personal mobile display system executes command or processes the request for system status, prepares the response that is due back to the host and then transmits it across the network to the host system.

46. A method of providing event driven functionality that allows a personal mobile display system to system to cooperatively execute application programs in conjunction with a remote computer system, said method comprising the steps of:

displaying an array of application-oriented attributes of at least one application program on the video display screen of the personal mobile display system wherein the functionality, appearance and logical position of an application-oriented attributes is associated at least one rectangular display segment of the displayed desktop of the personal mobile display system detecting that a change to the functionality, appearance and logical position of an application-oriented functional element has occurred wherein a change to the functionality, appearance and logical position of an application-oriented functional element is defined as an event;

categorizing type of event wherein an event comprises an user-directed event or alternatively a device driven event;

encapsulating at one or more events as an event transform to be transmitted across a network to a corresponding remote computer system wherein the event transform is an encapsulation of at least one source event, method and or actions that may be moved in space and or instance in time between the personal mobile display system and the remote computer system.

transmitting event transform across network via at least one communication protocol to the corresponding remote desktop computer system;

receiving one or more event transforms from the corresponding remote desktop computer system;

reconstructing the refreshed and updated rectangular display segment that were encapsulated in the received event transforms wherein the revised rectangular display segments depicts any changes in functionality, appearance and or logical position of an application-oriented functional element; and

displaying those rectangular display segments that are required to revise the display on the personal mobile display system in accordance with the application program from the remote computer system.

47. The method as recited in claim 46, wherein a change of an application-oriented attribute may be invoked by a user-specified action that includes a keyboard, a mouse or an entry caused by a stylus moving across a touch sensitive display of the personal mobile display system causing the appearance and or logical position of the application-oriented display of the personal mobile display system to change.

48. The method as recited in claim 46, wherein the personal mobile display system transmits the event transform to the corresponding remote computer system through a network to the corresponding host system for processing.

49. The method as recited in claim 46, wherein the corresponding remote computer system performs the steps for processing the event transform from received the personal mobile display system comprising:

determining type of event transform to process;

interpreting and reconstructing appropriate event method and action from the event transform;

processing event transform in accordance to type of event;

displaying wherein changes as a result of host resident application routines, a control event as well as user directed actions in accordance with the processed event transform are displayed on the host system;

storing rectangular display segments containing any changes resulting from an application process or user specified action that has occurred in a display stack, wherein the rectangular display region is used to refresh and update the personal mobile display system;

encapsulating the refreshed and updated rectangular display segment into an event transform to be transferred to the personal mobile display system; and

transferring the updated rectangular regions across the network to the personal mobile display system.

50. The method as recited in claim 49, wherein only when a number of updated rectangular display segments have been accumulated does the host system encapsulate the updated rectangular display segments into an event transform and transfer it through the network to the personal mobile display system.

51. The method as recited in claim 49, If the remote computer system determines that there are any commands and a program request for data needed to be transmitted to the personal mobile display system, the host system then repeats the steps of packaging and transmitting the commands and or any program requests for data across the network to the personal mobile display system.

52. The method as recited in claim 49, wherein the personal mobile display system receives an event transform from the corresponding host system, the personal mobile display system then determines whether the received event transform contains either at least one updated rectangular display segment, a control command or a program request for data.

53. The method as recited in claim 49, wherein if the received event transform is at least one updated rectangular display segment to be displayed, the personal mobile display system, then reconstructs the source properties of the rectangular segment and paints the rectangular segment of the display with the appropriate attributes where these attributes can be source properties, appearance or logical position.

54. A computer program product that includes a computer useable medium having computer readable code embodied therein that provides event driven functionality that allows a personal mobile display system to cooperatively execute application programs in conjunction with a remote computer system, said method comprising the steps of:

a computer readable program code device of a personal mobile display system configured to display an array of application-oriented attributes of at least one application program on the video display screen of the personal mobile display system wherein the functionality, appearance and logical position of an application-oriented attributes is associated at least one rectangular display segment of the displayed desktop of the personal mobile display system;

a computer readable program code device of a personal mobile display system configured to detect that a change to the functionality, appearance and logical position of an application-oriented functional element has occurred wherein a change to the functionality, appearance and logical position of an application-oriented functional element is defined as an event;

a computer readable program code device of a personal mobile display system configured to categorize type of event wherein an event comprises an user-directed event or alternatively a device driven event;

a computer readable program code device of a personal mobile display system configured to encapsulate at one or more events as an event transform to be transmitted across a network to a corresponding remote computer system wherein the event transform is an encapsulation of at least one source event, method and or actions that may be moved in space and or instance in time between the personal mobile display system and the remote computer system.

a computer readable program code device of a personal mobile display system configured to transmit event transform across network via at least one communication protocol to the corresponding remote desktop computer system;

a computer readable program code device of a personal mobile display system configured to receive one or more event transforms from the corresponding remote desktop computer system;

a computer readable program code device of a personal mobile display system configured to reconstruct the refreshed and updated rectangular display segment that were encapsulated in the received event transforms wherein the revised rectangular display segments depicts any changes in functionality, appearance and or logical position of an application-oriented functional element; and

a computer readable program code device of a personal mobile display system configured to display those rectangular display segments that are required to revise the display on the personal mobile display system in accordance with the application program from the remote computer system.

55. The computer program product as recited in claim 54, wherein a change of an application-oriented attribute may be invoked by a user-specified action that includes a keyboard, a mouse or an entry caused by a stylus moving across a touch sensitive display of the personal mobile display system causing the appearance and or logical position of the application-oriented display of the personal mobile display system to change.

56. The computer program product as recited in claim 54, wherein the personal mobile display system transmits the event transform to the corresponding remote computer system through a network to the corresponding host system for processing.

57. The computer program product as recited in claim 54, wherein the corresponding remote computer system performs the steps for processing the event transform from received the personal mobile display system comprising:

a computer readable program code device of a personal mobile display system configured to determine type of event transform to process;

a computer readable program code device of a personal mobile display system configured to interpret and reconstructing appropriate event method and action from the event transform;

a computer readable program code device of a personal mobile display system configured to process event transform in accordance to type of event;

a computer readable program code device of a personal mobile display system configured to display wherein changes as a result of host resident application routines, a control event as well as user directed actions in accordance with the processed event transform are displayed on the host system;

a computer readable program code device of a personal mobile display system configured to store rectangular display segments containing any changes resulting from an application process or user specified action that has occurred in a display stack, wherein the rectangular display region is used to refresh and update the personal mobile display system;

a computer readable program code device of a personal mobile display system configured to encapsulate the refreshed and updated rectangular display segment into an event transform to be transferred to the personal mobile display system; and

a computer readable program code device of a personal mobile display system configured to transfer the updated rectangular regions across the network to the personal mobile display system.

58. The computer program product as recited in claim 57, wherein only when a number of updated rectangular display segments have been accumulated does the host system encapsulate the updated rectangular display segments into an event transform and transfer it through the network to the personal mobile display system.

59. The computer program product as recited in claim 57, If the remote computer system determines that there are any commands and a program request for data needed to be transmitted to the personal mobile display system, the host system then repeats the steps of packaging and transmitting the commands and or any program requests for data across the network to the personal mobile display system.

60. The computer program product as recited in claim 57, wherein the personal mobile display system receives an event transform from the corresponding host system, the personal mobile display system then determines whether the received event transform contains either at least one updated rectangular display segment, a control command or a program request for data.

61. The computer program product as recited in claim 57, wherein if the received event transform is at least one updated rectangular display segment to be displayed, the personal mobile display system, then reconstructs the source properties of the rectangular segment and paints the rectangular segment of the display with the appropriate attributes where these attributes can be source properties, appearance or logical position.

62. A system that provides event driven functionality that allows a personal mobile display system to system to cooperatively execute application programs in conjunction with a remote computer system, said method comprising the steps of:

circuitry for displaying an array of application-oriented attributes of at least one application program on the video display screen of the personal mobile display system wherein the functionality, appearance and logical position of an application-oriented attributes is associated at least one rectangular display segment of the displayed desktop of the personal mobile display system;

circuitry for detecting that a change to the functionality, appearance and logical position of an application-oriented functional element has occurred wherein a change to the functionality, appearance and logical position of an application-oriented functional element is defined as an event;

circuitry for categorizing type of event wherein an event comprises an user-directed event or alternatively a device driven event;

circuitry for encapsulating at one or more events as an event transform to be transmitted across a network to a corresponding remote computer system wherein the event transform is an encapsulation of at least one source event, method and or actions that may be moved in space and or instance in time between the personal mobile display system and the remote computer system circuitry for transmitting event transform across network via at least one communication protocol to the corresponding remote desktop computer system;

circuitry for receiving one or more event transforms from the corresponding remote desktop computer system;

circuitry for reconstructing the refreshed and updated rectangular display segment that were encapsulated in the received event transforms wherein the revised rectangular display segments depicts any changes in functionality, appearance and or logical position of an application-oriented functional element; and

circuitry for displaying those rectangular display segments that are required to revise the display on the personal mobile display system in accordance with the application program from the remote computer system.

63. The system as recited in claim 62, wherein a change of an application-oriented attribute may be invoked by a user-specified action that includes a keyboard, a mouse or an entry caused by a stylus moving across a touch sensitive display of the personal mobile display system causing the appearance and or logical position of the application-oriented display of the personal mobile display system to change.

64. The system as recited in claim 62, wherein the personal mobile display system transmits the event transform to the corresponding remote computer system through a network to the corresponding host system for processing.

65. The system as recited in claim 62, wherein the corresponding remote computer system performs the steps for processing the event transform from received the personal mobile display system comprising:

determining type of event transform to process;

interpreting and reconstructing appropriate event method and action from the event transform;

processing event transform in accordance to type of event;

displaying wherein changes as a result of host resident application routines, a control event as well as user directed actions in accordance with the processed event transform are displayed on the host system;

storing rectangular display segments containing any changes resulting from an application process or user specified action that has occurred in a display stack, wherein the rectangular display region is used to refresh and update the personal mobile display system;

encapsulating the refreshed and updated rectangular display segment into an event transform to be transferred to the personal mobile display system; and

transferring the updated rectangular regions across the network to the personal mobile display system.

66. The system as recited in claim 65, wherein only when a number of updated rectangular display segments have been accumulated does the host system encapsulate the updated rectangular display segments into an event transform and transfer it through the network to the personal mobile display system.

67. The system as recited in claim 65, If the remote computer system determines that there are any commands and a program request for data needed to be transmitted to the personal mobile display system, the host system then repeats the steps of packaging and transmitting the commands and or any program requests for data across the network to the personal mobile display system.

68. The system as recited in claim 65, wherein the personal mobile display system receives an event transform from the corresponding host system, the personal mobile display system then determines whether the received event transform contains either at least one updated rectangular display segment, a control command or a program request for data.

69. The system as recited in claim 65, wherein if the received event transform is at least one updated rectangular display segment to be displayed, the personal mobile display system, then reconstructs the source properties of the rectangular segment and paints the rectangular segment of the display with the appropriate attributes where these attributes can be source properties, appearance or logical position.