System and method for capturing images of screens which display multiple windows

• US 5,877,762 A
• Filed: 09/08/1997
• Issued: 03/02/1999
• Est. Priority Date: 02/27/1995
• Status: Expired due to Term

• Current Assignee: Apple Inc.
• Original Assignee: Apple Computer Incorporated (Apple Inc.)
• Inventors: Young, James A.
• Primary Examiner(s): Kim, Matthew M.
• Assistant Examiner(s): dela Torre, Crescelle N.
• Application Number:
  US08/925,270
• Time in Patent Office: 540 Days
• Field of Search: 345/340, 345/342, 345/343, 345/344, 345/345, 345/356, 345/434, 345/118
• US Class Current: 715/803
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

Methods and apparatus for capturing images on screens which display multiple windows. Exemplary embodiments eliminate unnecessary reading and writing of information pertaining to display data that is not visible. A user designates an arbitrary region of a screen to be captured, which can cross windows that are displayed at different layers within the image. A snapshot of the selected region is obtained by retrieving data for each of the pixels in the region, converting them into standard values, and storing them in an off-screen pixel map. Before the data for the pixels is obtained, a listing is made of all windows that are encompassed by the designated region. For each window a record is made of the portion of the window that is visible within the designated region. If no portion is visible, the window is removed from the list. After all of the windows within the region have been examined in this manner, only the pixels pertaining to the recorded visible portion of each window need to be read and stored in the pixel map. With this approach, redundancy is eliminated, since only the visible pixel data is retrieved and written to the pixel map.

Claims

1. A method for capturing data pertaining to an image displayed on a computer monitor, in which information is displayed in multiple windows that are located at different effective distances from the viewer in the image, comprising the steps of:

designating a boundary within the image, said boundary defining a capture region for which image data is to be stored for subsequent processing;

generating a list of each window that is at least partially contained within said capture region;

determining whether any window on said list is completely obscured from view within said capture region by other windows, and removing each such completely obscured window from said list;

obtaining pixel values for the windows which remain on said list; and

storing said pixel values in a map, wherein said determining and removing steps include the steps of

creating a viewing region which initially corresponds to the designated boundary and

iteratively updating the viewing region by selecting successive windows in the list and, for each selected window, subtracting from the viewing region an area of intersection between the selected window and the viewing region. wherein windows in the list are selected in order of effective distance from shortest effective distance to longest effective distance.

2. The method of claim 1 wherein the display of the image on the computer monitor is controlled by a display server, and further including the steps of locking the display server at the beginning of said method to prevent the image from being changed while data is being captured, and unlocking said server at the completion of data capture.

3. The method of claim 2 wherein said pixel values are obtained from said display server, and further including the step of converting said pixel values into a standardized format for storage in said map.

4. The method of claim 1 wherein said list generating step includes the steps of:

identifying a hierarchy of windows displayed on the computer monitor;

selecting windows in said hierarchy and, for each selected window, determining whether that window intersects the capture region; and

adding to the list each selected window which intersects the capture region, and determining whether the window has any descendant windows which also intersect the capture region.

5. The method of claim 4 further including the steps of:

clipping the area of each selected window to lie within the area of any window from which the selected window descends; and

disregarding descendant windows of any selected window which does not intersect the capture region.

6. A method according to claim 1, wherein a computer user manually designates said boundary.

7. The method of claim 1 wherein said determining and removing step includes the steps of:

a) creating a viewing region corresponding to said capture region;

b) selecting a window from said list;

c) calculating the area of intersection between said viewing region and the selected window;

d) removing the window from the list if the area of intersection is null;

e) updating the viewing region by subtracting the area of intersection from the viewing region; and

f) repeating steps b-e for each of the other windows on the list, wherein windows are selected in order of effective distance from shortest effective distance to longest effective distance.

8. The method of claim 7 including the further step of recording the area of intersection for each window that remains on the list, and wherein said step of obtaining pixel values comprises obtaining pixel values for the recorded area of intersection for each window that remains on the list.

9. A method for capturing data pertaining to an image displayed on a computer monitor, in which information is displayed in multiple windows that are located at different effective distances from the viewer in the image, comprising the steps of:

defining a capture region within the image;

generating a list of each window that is at least partially contained within said capture region;

determining whether any window on said list is completely obscured from view within said capture region by other windows, and removing each such completely obscured window from said list;

obtaining pixel values for the windows which remain on said list; and

storing said pixel values in a map, wherein said determining and removing step includes the steps of;

a) creating a viewing region which initially corresponds to said capture region;

b) selecting a window from said list;

c) calculating the area of intersection between said viewing region and the selected window;

d) removing the window from the list if the area of intersection is null;

e) updating the viewing region by subtracting the area of intersection from the viewing region; and

f) repeating steps b-e for each of the other windows on the list, wherein windows are selected in order of effective distance from shortest effective distance to longest effective distance.

10. The method of claim 9 including the further step of recording the area of intersection for each window that remains on the list, and wherein said step of obtaining pixel values comprises obtaining pixel values for the recorded area of intersection for each window that remains on the list.

11. A system for capturing a portion of an image displayed on a computer monitor, comprising:

a display server which receives information, and displays said information on the computer monitor as an image having the appearance of different effective layers which can overlap one another;

means for designating a boundary within said image, said boundary defining a capture region for which image data is to be stored for subsequent processing;

means for generating a list containing each layer having an area that is at least partially contained within said capture region;

means for removing from said list each layer whose information is completely obscured within said capture region by displayed information in another layer; and

means for obtaining from said display server data values pertaining to the displayed information in each layer remaining on said list and storing said values in a map, wherein said means for removing includes

means for creating a viewing region which initially corresponds to the designated boundary and

means for iteratively updating the viewing region by selecting successive windows in the list and, for each selected window, subtracting from the viewing region an area of intersection between the selected window and the viewing region, wherein windows in the list are selected in order of effective distance from shortest effective distance to longest effective distance.

12. The system of claim 11 wherein each display layer comprises a window which contains displayed information.

13. A system according to claim 11, wherein said means for designating a boundary enables a computer user to manually designate said boundary.

14. The system of claim 11 wherein said removing means includes:

means for storing a viewing region which initially corresponds to said capture region;

means for successively calculating the intersection of the area of the layers on said list with said viewing region, and removing a layer from the list if there is no intersecting area; and

means for updating the viewing region by subtracting the intersecting area from the stored viewing region as the intersecting area for each layer is calculated.

15. The system of claim 14 wherein said calculating means stores the intersecting area of each layer, and said obtaining means obtains data values pertaining to the information displayed within the stored intersecting areas.

16. The system of claim 11 further including means for converting data values obtained from said display server into standardized values before storage in said map.

17. A system for capturing a portion of an image displayed on a computer monitor, comprising:

a display server which receives information, and displays said information on the computer monitor as an image having the appearance of different effective layers which can overlap one another;

means for defining a capture region within said image;

means for generating a list containing each layer having an area that is at least partially contained within said capture region;

means for removing from said list each layer whose information is completely obscured within said capture region by displayed information in another layer; and

means for obtaining from said display server data values pertaining to the displayed information in each layer remaining on said list and storing said values in a map, wherein said removing means includes;

means for storing a viewing region which initially corresponds to said capture region;

means for successively calculating the intersection of the area of the layers on said list with said viewing region, and removing a layer from the list if there is no intersecting area; and

means for updating the viewing region by subtracting the intersecting area from the stored viewing region as the intersecting area for each layer is calculated.

18. The system of claim 17 wherein said calculating means stores the intersecting area of each layer, and said obtaining means obtains data values pertaining to the information displayed within the stored intersecting areas.

19. A method for capturing data pertaining to an image displayed on a computer monitor, in which information is displayed in multiple windows that are located at different effective distances from the viewer in the image, comprising the steps of:

designating a boundary within the image, said boundary defining a capture region for which image data is to be stored for subsequent processing;

generating a list of each window that is at least partially contained within said capture region;

determining whether any window on said list is completely obscured from view within said capture region by other windows, and removing each such completely obscured window from said list;

recording an intersecting area for each window that remains on the list, wherein an intersecting area for a particular window corresponds to a portion of the particular window which is visible within said capture region;

obtaining pixel values for the recorded intersecting area for each window that remains on the list; and

storing said pixel values in a map, wherein said determining and removing steps include the steps of

creating a viewing region which initially corresponds to the designated boundary and

iteratively updating the viewing region by selecting successive windows in the list and, for each selected window, subtracting from the viewing region an area of intersection between the selected window and the viewing region, wherein windows in the list are selected in order of effective distance from shortest effective distance to longest effective distance.

20. A system for capturing a portion of an image displayed on a computer monitor, comprising:

a display server receiving information and displaying the information on the computer monitor as an image having the appearance of different effective layers which can overlap one another;

means for designating a boundary within the image, said boundary defining a region for which image data is to be stored for subsequent processing;

means for generating a list containing each layer having an area that is at least partially contained within said capture region;

means for removing from said list each layer whose information is completely obscured within said capture region by displayed information in another layer;

means for recording an intersecting area for each layer that remains on the list, wherein an intersecting area for a particular layer corresponds to a portion of the particular layer which is visible within said capture region; and

means for obtaining from said display server data values corresponding to the intersecting area of each layer remaining on said list and storing said values in a map, wherein said means for removing includes

means for creating a viewing region which initially corresponds to the designated boundary and

means for iteratively updating the viewing region by selecting successive windows in the list and, for each selected window, subtracting from the viewing region an area of intersection between the selected window and the viewing region, wherein windows in the list are selected in order of effective distance from shortest effective distance to longest effective distance.