Efficient capture of computer screens

• US 6,573,915 B1
• Filed: 12/08/1999
• Issued: 06/03/2003
• Est. Priority Date: 12/08/1999
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kupeev, Konstantin Y., Sivan, Zohar
• Primary Examiner(s): Cabeca, John
• Assistant Examiner(s): Thai, Cuong T.
• Application Number:
  US09/456,343
• Time in Patent Office: 1,273 Days
• Field of Search: 345/781, 345/802, 345/803, 345/804, 345/788, 345/798, 345/790, 345/794, 345/800, 345/806
• US Class Current: 715/781
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

A method for capture of computer screens in a sequence of frames. A first set of one or more windows appearing in a first frame in the sequence is identified, each window in the set having respective first-frame window characteristics and window contents. A description of the first set of windows is encoded, indicative of the appearance of the computer screen in the first frame. In a second frame in the sequence, a second set of one or more windows is identified, having respective second-frame window characteristics and window contents, the second set including one or more windows corresponding respectively to one or more of the windows in the first set. One or more transformations are determined, which are applied to the first-frame window characteristics of the windows in the first set to generate the second-frame window characteristics of the corresponding windows in the second set. A description of the second set of windows is encoded, including the determined transformations, for use in reconstructing the computer screen as it appeared in the second frame.

Claims

1. A method for capture of computer screens in a sequence of frames, comprising:

identifying a first set of one or more windows appearing in a first frame in the sequence, each window in the set having respective first-frame window characteristics and window contents;

encoding a description of the first set of windows indicative of the appearance of the computer screen in the first frame;

identifying in a second frame in the sequence a second set of one or more windows having respective second-frame window characteristics and window contents, the second set including one or more windows corresponding respectively to one or more of the windows in the first set;

determining one or more transformations applied to the first-frame window characteristics of the windows in the first set to generate the second-frame window characteristics of the corresponding windows in the second set; and

encoding a description of the second set of windows including the determined transformations, for use in reconstructing the computer screen as it appeared in the second frame,

wherein identifying the first set of windows comprises identifying windows generated in accordance with an operating system of the computer, which associates each window with a respective function of the computer, such that the contents of the windows are determined by the respective functions.

2. A method according to claim 1, wherein the respective functions comprise applications running under the operating system.

3. A method according to claim 1, wherein determining the transformations comprises defining transformations applied by the operating system, and which are applicable to different windows associated with different functions, generally irrespectively of the functions.

4. A method according to claim 1, wherein identifying the second set of windows comprises querying the operating system regarding the characteristics of the windows.

5. A method according to claim 1, wherein identifying the second set of windows comprises intercepting events generated by the operating system.

6. A method according to claim 1, wherein identifying the first set of windows comprises processing an image of the screen to identify the windows.

7. A method according to claim 1, wherein determining the transformations comprises defining a set of typical transformations, which are applicable to alter the window characteristics of the one or more windows, generally irrespectively of the contents of the windows.

8. A method according to claim 7, wherein the typical transformations are selected from a group of transformations consisting of moving and resizing a window.

9. A method according to claim 7, wherein the typical transformations are selected from a group of transformations consisting of minimizing, restoring and maximizing the size of a window.

10. A method according to claim 7, wherein the typical transformations comprise changing a Z-order of the windows, according to which two or more of the windows are overlaid one upon another on the screen.

11. A method according to claim 7, wherein the typical transformations are selected from a group of transformations consisting of scrolling and panning the window contents.

12. A method according to claim 1, wherein identifying the first and second sets of windows comprises identifying windows associated with respective functions of the computer, wherein the identified windows include one or more user interface windows generated inside other identified windows for the purpose of controlling the functions associated therewith.

13. A method according to claim 1, and comprising identifying first and second sets of one or more icons in the first and second frames, respectively, and determining transformations applied to the icons in the first frame to generate the icons in the second frame, to be encoded along with the description of the second set of windows.

14. A method according to claim 13, wherein the first and second sets of icons comprise a cursor.

15. A method for capture of computer screens in a sequence of frames, comprising:

identifying a first set of one or more windows appearing in a first frame in the sequence, each window in the set having respective first-frame window characteristics and window contents;

encoding a description of the first set of windows indicative of the appearance of the computer screen in the first frame;

identifying in a second frame in the sequence a second set of one or more windows having respective second-frame window characteristics and window contents, the second set including one or more windows corresponding respectively to one or more of the windows in the first set;

determining one or more transformations applied to the first-frame window characteristics of the windows in the first set to generate the second-frame window characteristics of the corresponding windows in the second set; and

encoding a description of the second set of windows including the determined transformations, for use in reconstructing the computer screen as it appeared in the second frame,

wherein encoding the description of the first set of windows comprises encoding the first-frame window characteristics and the respective contents of the windows in the first set, and wherein encoding the description of the second set of windows comprises encoding the determined transformations and encoding changes in the contents of the windows in the second set with respect to the contents of the corresponding windows in the first set.

16. A method according to claim 15, wherein encoding the changes in the contents of the windows comprises applying different encoding schemes to the contents of different ones of the windows.

17. A method according to claim 16, wherein applying the different encoding schemes comprises applying a video compression scheme to the contents of at least one of the windows, and embedding resultant compressed video data in the encoded description of the windows.

18. A method according to claim 16, wherein applying the different encoding schemes comprises applying different levels of encoding resolution to different ones of the windows in the second set.

19. A method according to claim 15, wherein encoding the first and second descriptions comprises transferring the encoded descriptions over a communication link to a recipient computer.

20. A method according to claim 19, wherein encoding the descriptions comprises encoding the descriptions in a platform-independent format.

21. A method according to claim 15, wherein encoding the first and second descriptions comprises storing the encoded descriptions in a memory.

22. A method according to claim 15, and comprising reconstructing the second frame responsive to the encoded descriptions of the first and second sets of windows.

23. A method for reconstructing captured computer screens, comprising:

receiving an encoded description of a first set of one or more windows, having first-frame characteristics and window contents, which appeared on the computer screen in a first captured frame;

receiving an encoded description of a second set of one or more windows, having second-frame characteristics and window contents, which appeared on the computer screen in a second captured frame, subsequent to the first frame, the description of the second set of windows comprising a description of one or more transformations applied to the first-frame characteristics of at least one of the windows in the first set to derive the second-frame characteristics of a corresponding window in the second set; and

reconstructing the second captured frame responsive to the encoded descriptions of the first and second sets of windows,

wherein reconstructing the second captured frame comprises decoding the encoded description of the first set of windows to determine the first-frame characteristics thereof, and applying the one or more transformations described in the description of the second set of windows to transform the first-frame characteristics into the second-frame characteristics of the at least one corresponding window.

24. A method according to claim 23, wherein the encoded description of the second set of windows further comprises encoded changes in the contents of the windows in the second set with respect to the contents of the at least one corresponding window in the first set, and wherein reconstructing the second captured frame comprises reconstructing the contents of the windows in the second set responsive to the encoded changes.

25. A method according to claim 23, wherein the encoded description of the second set of windows comprises compressed video data in a standard media format, and wherein reconstructing the second captured frame comprises invoking a standard media player to reconstruct video images in one of the windows.

26. A method according to claim 23, wherein reconstructing the second captured frame comprises reconstructing the first and second sets of windows substantially independently of an operating system under which the windows were generated.

27. A method according to claim 26, wherein reconstructing the first and second sets of windows comprises operating a platform-independent screen player.

28. Apparatus for capture of computer screens in a sequence of frames, comprising:

a display; and

a processor, which is adapted to identify a first set of one or more windows appearing on the display in a first frame in the sequence, each window in the set having respective first-frame window characteristics and window contents, and to encode a description of the first set of windows, indicative of the appearance of the computer screen in the first frame, and

to identify, in a second frame in the sequence a second set of one or more windows appearing on the display, having respective second-frame window characteristics and window contents, the second set including one or more windows corresponding respectively to one or more of the windows in the first set, and to determine one or more transformations applied to the first-frame window characteristics of the windows in the first set to generate the second-frame window characteristics of the corresponding windows in the second set, and

to encode a description of the second set of windows including the determined transformations, for use in reconstructing the computer screen as it appeared in the second frame,

wherein the windows are generated in accordance with an operating system of the processor, which associates each window with a respective function of the processor, such that the contents of the windows are determined by the respective functions.

29. Apparatus according to claim 28, wherein the processor is further adapted to identify first and second sets of one or more icons in the first and second frames, respectively, and to determine transformations applied to the icons in the first frame to generate the icons in the second frame, to be encoded along with the description of the second set of windows.

30. Apparatus according to claim 28, wherein the processor is adapted to be coupled via a communication link to transfer the encoded descriptions to a recipient computer.

31. Apparatus according to claim 30, wherein the recipient computer reconstructs the second frame responsive to the encoded descriptions of the first and second sets of windows.

32. Apparatus according to claim 28, and comprising a memory adapted to store the encoded first and second descriptions.

33. A computer software product for capture of computer screens in a sequence of frames, the product comprising computer-readable media in which program instructions are stored, which instructions, when read by a computer, cause the computer:

to identify a first set of one or more windows appearing in a first frame in the sequence, each window in the set having respective first-frame window characteristics and window contents and to encode a description of the first set of windows, indicative of the appearance of the computer screen in the first frame, and

to identify in a second frame in the sequence a second set of one or more windows having respective second-frame window characteristics and window contents, the second set including one or more windows corresponding respectively to one or more of the windows in the first set and to determine one or more transformations applied to the first-frame window characteristics of the windows in the first set to generate the second-frame window characteristics of the corresponding windows in the second set, and

to encode a description of the second set of windows including the determined transformations, for use in reconstructing the computer screen as it appeared in the second frame,

wherein the windows are generated in accordance with an operating system of the computer, which associates each window with a respective function of the computer, such that the contents of the windows are determined by the respective functions.

34. A product according to claim 33, wherein the determined transformations are applied by the operating system, and are applicable to different windows associated with different functions, generally irrespectively of the functions.

35. A product according to claim 33, wherein the program instructions, when run by the computer, cause the computer to query the operating system regarding the characteristics of the windows.

36. A product according to claim 33, wherein the program instructions, when run by the computer, further cause the computer to reconstruct the second frame responsive to the encoded descriptions of the first and second sets of windows.