Image transfer using drawing command hooking

• US 20050278455A1
• Filed: 05/25/2005
• Published: 12/15/2005
• Est. Priority Date: 06/11/2004
• Status: Active Grant

• Current Assignee: Columbia Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Ichieda, Hiroyuki
• Granted Patent: US 7,817,156 B2
• Time in Patent Office: Days
• US Class Current: 709/233
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 3/1462 with means for detecting differences between the image stored in the host and the images displayed on the remote displays
• CPC Class Codes: G09G 2340/02 Handling of images in compressed format, e.g. JPEG, MPEG
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas

Abstract

A hook processing module 400 hooks and preempts a specific drawing command issued by an application program 122, and draws an image in an image data storage area 106b within the RAM 106 according to the acquired drawing command. VNC server 130 acquires the image from the image data storage area 106b, and transfers the acquired image to a projector via a network.

Claims

1. An image supply device for supplying an image to an image display device via a network, comprising:

an application program capable of issuing image drawing commands;

a drawing module configured to process drawing commands issued by the application program;

a hook processing module configured to hook and preempt a specific drawing command issued by the application program and to draw an image in a transfer image storage area within a general purpose memory according to the acquired drawing command; and

an image transfer processing module configured to acquire an image from the transfer image storage area and to transfer the acquired image to the image display device via the network.

2. An image supply device recited in claim 1, wherein

the hook processing module is configured to write, into the general purpose memory, change area information indicating a change area that is an area within the transfer image storage area in which an image is drawn according to the drawing command, and

the image transfer processing module is configured to reference the change area information stored within the general purpose memory and acquires the image within the image change area that is stored in the transfer image storage area, and to transfer to the image display device via the network the acquired image together with the change area information.

3. An image supply device recited in claim 1, wherein

the image transfer processing module is configured to transfer a first image stored in the transfer image storage area via the network to the image display device to make the image display device display a second image that is expanded from the first image on the screen of the image display device,

the hook processing module is configured to write, into the general purpose memory, transfer area information containing change area information and size information, the change area information indicating a change area that is an area within the transfer image storage area for which the first image is drawn according to the drawing command, the size information indicating size of the second image on the screen of the image display device, and

the image transfer processing module is configured to acquire the transfer area information stored within the general purpose memory, and to transfer the transfer area information together with the first image acquired from the transfer image storage area to the image display device via the network.

4. An image supply device recited in claim 1, wherein

the image transfer processing module is constituted such that the hook processing module is loaded into the application program responsive to activation of the image transfer processing module, and the hook processing module is unloaded from the application program responsive to deactivation of the image transfer processing module.

5. An image supply device recited in claim 1, wherein

when the image transfer processing module is executing a specified image transfer process including at least a process of acquiring an image from the transfer image storage area, the hook processing module waits in standby without starting processing according to the drawing command until the image transfer process is completed.

6. An image supply device recited in claim 5, wherein

the image transfer processing module sets a transfer flag within the general purpose memory to an on state while the image transfer process is being executed, and sets the transfer flag to an off state after the image transfer process is completed, and

when the transfer flag is in the on state, the hook processing module waits in standby without starting processing according to the drawing command until the transfer flag is turned off.

7. An image supply device recited in claim 5, wherein

the hook processing module is configured to write, into the general purpose memory, change area information for indicating a change area that is an area within the transfer image storage area for which an image is drawn according to the drawing command, and

the image transfer processing module executes:

(i) in the image transfer process, referencing the change area information stored within the general purpose memory, and acquiring the image within the image change area stored in the transfer image storage area, and

(ii) transferring the acquired image together with the change area information to the image display device via the network.

8. An image supply device recited in claim 1, wherein

the hook processing module is configured to write, into the general purpose memory, first change area information for indicating a first change area that is an area within the transfer image storage area for which the image is drawn according to the specific drawing command, and to supply to the drawing module the specific drawing command after processing the specific drawing command, and

the drawing module is configured to draw an image into the frame memory according to a drawing command received from the application program or the hook processing module, and to write, into the general purpose memory, second change area information for indicating a second change area that is an area within the frame memory for which the image is drawn according to the drawing command, and

the image transfer processing module executes:

(i) referencing the first and second change area information stored within the general purpose memory, and acquiring an image within an area of the second change area which is not contained in the first change area from the frame memory,

(ii) acquiring an image within the first change area from the transfer image storage area without accessing the frame memory, and

(iii) transferring the acquired images together with screen update area information to the image display device via the network, the screen update area information indicating a screen update area that is the total of the first and second change areas.

9. An image supply device recited in claim 1, wherein

the hook processing module is configured to write, into the general purpose memory, first change area information for indicating a first change area that is an area within the transfer image storage area for which an image is drawn according to the specific drawing command, and to supply to the drawing module the specific drawing command after processing the specific drawing command,

the drawing module is configured to draw an image into the frame memory according to a drawing command received from the application program or the hook processing module, and to write, into the general purpose memory, second change area information for indicating a second change area that is an area within the frame memory for which an image is drawn according to the drawing command, and

the image transfer processing module executes:

(i) referencing the first and second change area information stored within the general purpose memory, and acquiring from the frame memory an image within an area of the second change area which is not contained in the first change area,

(ii) acquiring an image within an area of the second change area that is also contained in the first change area from the transfer image storage area without accessing the frame memory, and

(iii) transferring the acquired images together with screen update area information to the image display device via the network, the screen update area information indicating a screen update area that is equivalent to the second change area.

10. An image supply device recited in claim 9, wherein

the image transfer processing module acquires the image within the area of the second change area that is not contained in the first change area from the frame memory to write the image into the transfer image storage area, and after the writing of the image, acquires the image within the screen update area from the transfer image storage area.

11. An image supply device recited in claim 1, wherein

the hook processing module is configured to write, into the general purpose memory, first change area information for indicating a first change area that is an area within the transfer image storage area for which the image is drawn according to the specific drawing command, and to supply to the drawing module the specific drawing command after processing the specific drawing command,

the drawing module is configured to draw an image into the frame memory according to a drawing command received from the application program or the hook processing module, and to write, into the general purpose memory, second change area information for indicating a second change area that is an area within the frame memory for which the image is drawn according to the drawing command, and

the image transfer processing module executes:

(i) setting as a history area the first change area that exists in the transfer process during a specified preceding time period, and writing into the general purpose memory transfer history information for representing the history area,

(ii) referencing the first and second change area information and the transfer history information stored in the general purpose memory, and acquiring from the frame memory an image within an area of the second change area that is not included in either the first change area or the history area,

(iii) acquiring from the transfer image storage area an image within the first change area and an image within an area of the second change area that is not contained in the first change area and that is contained in the history area, without accessing the frame memory, and

(iv) transferring the acquired images together with screen update area information to the image display area via the network, the screen update area information indicating a screen update area that is the total of the first and second change areas.

12. An image supply device recited in claim 1, wherein

the hook processing module is configured to write, into the general purpose memory, first change area information for indicating a first change area that is an area within the transfer image storage area for which the image is drawn according to the specific drawing command, and to supply to the drawing module the specific drawing command after processing the specific drawing command,

the drawing module is configured to draw an image into the frame memory according to a drawing command received from the application program or the hook processing module, and to write, into the general purpose memory, second change area information for indicating a second change area that is an area within the frame memory for which the image is drawn according to the drawing command, and

the image transfer processing module executes:

(i) setting as a history area the first change area that exists for the transfer process during a specified preceding time period, and writing into the general purpose memory transfer history information for representing the history area,

(ii) referencing the first and second change area information and the transfer history information stored in the general purpose memory, and acquiring from the frame memory an image within an area of the second change area that is not included in either the first change area or the history area,

(iii) acquiring from the transfer image storage area an image within an area of the second change area that is contained in at least one of the first change area and the history area without accessing the frame memory, and

(iv) transferring the acquired images together with screen update area information to the image display area via the network, the screen update area information indicating a screen update area that is equivalent to the second change area.

13. An image supply device recited in claim 12, wherein

the image transfer processing module acquires from the frame memory the image within the area of the second change area that is not contained in either the first change area or the history area to write the image into the transfer image storage area, and after the writing of the image, acquires the image within the screen update area from the transfer image storage area.

14. An image supply device recited in claim 1, wherein

the drawing module is configured to draw an image into the frame memory according to a drawing command received from the application program or the hook processing module,

and the image transfer processing module has:

a normal transfer mode for acquiring and transferring the image drawn in the transfer image storage area by the hook processing module when image drawing operation is performed, and

a resend mode for acquiring and transferring the image drawn in the frame memory by the drawing module after the image drawing operation stops.

15. An image supply device recited in claim 14, wherein

the hook processing module is configured to write, into the general purpose memory, first change area information for indicating a first change area that is an area within the transfer image storage area for which the image is drawn according to the specific drawing command, and

the image transfer processing module executes:

in the normal transfer mode, setting the first change area as a resend area for which transfer is completed, and writing into the general purpose memory resend area information for representing the resend area, and

in the resend mode, acquiring and transferring from the frame memory an image within at least part of the resend area.

16. An image supply device recited in claim 15, wherein

the image within at least part of the resend area is an image part correlating to a specified size area within the resend area, and

the image transfer processing module transfers the image part correlating to the specified size area within the resend area in the resend mode, and deletes the specified size area from the resend area.

17. An image supply device recited in claim 14, wherein

the resend mode includes a full screen resend mode for transferring an entire image of a full screen area within the fame memory after a specified time has elapsed from when the image drawing operation stopped.

18. An image supply device recited in claim 17, wherein

the image transfer processing module, in the full screen resend mode, divides the full screen area into a plurality of division areas of a specified size, and executes transfer of the entire image of the full screen area by transferring an image of one division area selected in sequence from among the plurality of division areas for each one time transfer process.

19. An image supply device recited in claim 18, wherein

the image transfer processing module is configured to execute:

compressing an image to be transferred in the normal transfer mode using a compression method selected from among a plurality of compression methods including a first compression method with relatively high image degradation and a second compression method with relatively low image degradation,

registering as a resend area an area of the image that is compressed by the first compression method and transferred in the normal transfer mode, the resend area being an area of an image to be transferred in the resend mode, and

compressing the image within the resend area using the second compression method and transferring the compressed image in the resend mode.

20. A method for supplying an image to the image display device via the network, comprising:

(a) drawing an image to a transfer image storage area within a general purpose memory according to a specific drawing command issued by an application program, and

(b) acquiring the image from the transfer image storage area, and transferring the acquired image to the image display device via the network.

21. An image supply method recited in claim 20, wherein the step (a) includes:

determining whether or not a specified image transfer process including at least a process of acquiring an image from the transfer image storage area is being executed, and

waiting in standby without starting processing according to the drawing command until the image transfer process is completed when the image transfer process is being executed.

22. An image supply method recited in claim 20, wherein

the step (a) includes:

providing a drawing module for processing drawing commands issued by the application program, and a hook processing module for hooking and preempting a specific drawing command issued by the application program to draw an image in a specific transfer image storage area within a general purpose memory according to the acquired drawing command,

drawing an image, by operation of the hook processing module, in the specific transfer image storage area within the general purpose memory according to the specific drawing command issued by the application program,

writing into the general purpose memory, by operation of the hook processing module, first change area information for indicating a first change area that is an area within the transfer image storage area for which the image is drawn according to the specific drawing command,

supplying, by operation of the hook processing module, the specific drawing command to the drawing module after the specific drawing command is processed,

drawing, by operation of the drawing module, an image within the frame memory according to the drawing command received from the application program or the hook processing module, and

writing into the general purpose memory, by operation of the drawing module, second change area information for indicating a second change area which is an area within the frame memory for which the image is drawn according to the drawing command,

and the step (b) includes;

referencing the first and second change area information stored in the general purpose memory, and acquiring from the frame memory an image within an area of the second change area that is not contained in the first change area,

acquiring an image part within the first change area from the transfer image storage area without accessing the frame memory, and

transferring the acquired images together with screen update area information to the image display device via the network, the screen update information indicating a screen update area which is the total of the first and second change areas.

23. An image supply method recited in claim 20, wherein

the step (a) includes:

providing a drawing module for processing drawing commands issued by the application program, and a hook processing module for hooking and preempting a specific drawing command issued by the application program to draw an image in a specific transfer image storage area within a general purpose memory according to the acquired drawing command,

drawing, by operation of the hook processing module, an image in the specific transfer image storage area within the general purpose memory according to the specific drawing command issued by the application program,

writing into the general purpose memory, by operation of the hook processing module, first change area information for indicating a first change area that is an area within the transfer image storage area for which the image is drawn according to the specific drawing command,

supplying, by operation of the hook processing module, the specific drawing command to the drawing module after the specific drawing command is processed,

drawing, by operation of the drawing module, an image within the frame memory according to the drawing command received from the application program or the hook processing module, and

writing into the general purpose memory, by operation of the drawing module, second change area information for indicating a second change area which is an area within the frame memory for which the image is drawn according to the drawing command,

and the step (b) includes;

referencing the first and second change area information stored in the general purpose memory, and acquiring from the frame memory an image within an area of the second change area that is not contained in the first change area,

acquiring an image within the first change area from the transfer image storage area without accessing the frame memory, and

transferring the acquired images together with screen update area information to the image display device via the network, the screen update information indicating a screen update area which is equivalent to the second change area.

24. An image supply method recited in claim 20, wherein

the step (a) includes:

providing a drawing module for processing drawing commands issued by the application program, and a hook processing module for hooking and preempting a specific drawing command issued by the application program to draw an image in a specific transfer image storage area within a general purpose memory according to the acquired drawing command,

drawing, by operation of the hook processing module, an image in the specific transfer image storage area within the general purpose memory according to the specific drawing command issued by the application program,

writing into the general purpose memory, by operation of the hook processing module, first change area information for indicating a first change area that is an area within the transfer image storage area for which the image is drawn according to the specific drawing command,

supplying, by operation of the hook processing module, the specific drawing command to the drawing module after the specific drawing command is processed,

drawing, by operation of the drawing module, an image within the frame memory according to the drawing command received from the application program or the hook processing module,

writing into the general purpose memory, by operation of the drawing module, second change area information for indicating a second change area which is an area within the frame memory for which the image is drawn according to the drawing command, and

setting as a history area the first change area that exists in the transfer process during a specified preceding time period, and writing, into the general purpose memory, transfer history information for representing the history area,

and the step (b) includes;

referencing the first and second change area information and the transfer history information stored in the general purpose memory, and acquiring from the frame memory an image within an area of the second change area that is not included in either the first change area or the history area,

acquiring from the transfer image storage area an image within the first change area and an image within an area of the second change area that is not contained in the first change area and that is contained in the history area, without accessing the frame memory, and

transferring the acquired images together with screen update area information to the image display area via the network, the screen update area information indicating a screen update area that is the total of the first and second change areas.

25. An image supply method recited in claim 20, wherein

the step (a) includes:

providing a drawing module for processing drawing commands issued by the application program, and a hook processing module for hooking and preempting a specific drawing command issued by the application program to draw an image in a specific transfer image storage area within a general purpose memory according to the acquired drawing command,

drawing, by operation of the hook processing module, an image in the specific transfer image storage area within the general purpose memory according to the specific drawing command issued by the application program,

writing into the general purpose memory, by operation of the hook processing module, first change area information for indicating a first change area that is an area within the transfer image storage area for which the image is drawn according to the specific drawing command,

supplying, by operation of the hook processing module, the specific drawing command to the drawing module after the specific drawing command is processed,

drawing, by operation of the drawing module, an image within the frame memory according to the drawing command received from the application program or the hook processing module,

writing into the general purpose memory, by operation of the drawing module, second change area information for indicating a second change area which is an area within the frame memory for which the image is drawn according to the drawing command, and

setting as a history area the first change area that exists in the transfer process during a specified preceding time period, and writing, into the general purpose memory, transfer history information for representing the history area,

and the step (b) includes;

referencing the first and second change area information and the transfer history information stored in the general purpose memory, and acquiring from the frame memory an image within an area of the second change area that is not included in either the first change area or the history area,

acquiring from the transfer image storage area an image within an area of the second change area that is contained in at least one of the first change area and the history area without accessing the frame memory, and

transferring the acquired images together with screen update area information to the image display area via the network, the screen update area information indicating a screen update area that is equivalent to the second change area.

26. An image supply method recited in claim 20, wherein

the step (a) includes:

providing a drawing module for processing drawing commands issued by the application program, and a hook processing module for hooking and preempting a specific drawing command issued by the application program to draw an image in a specific transfer image storage area within a general purpose memory according to the acquired drawing command,

drawing an image, by operation of the hook processing module, in the specific transfer image storage area within the general purpose memory according to the specific drawing command issued by the application program,

and the step (b) includes;

acquiring and transferring an image drawn in the transfer image storage area when image drawing operation is performed according to a drawing command, and

acquiring the image drawn in the frame memory, and transferring the acquired image via the network after the image drawing operation stops.

27. A hook processing module used for an image supply device for supplying an image to an image display device via a network, the image supply device including an application program capable of issuing image drawing commands, a drawing module for processing drawing commands issued by the application program, and an image transfer processing module for acquiring an image from a transfer image storage area within a general purpose memory and for transferring the acquired image to the image display device via the network, wherein

the hook processing module is configured to hook and preempt a specific drawing command issued by the application program, and to draw an image in the transfer image storage area according to the specific drawing command.

28. An hook processing module recited in claim 27, wherein when the image transfer processing module is executing a specified image transfer process including at least a process of acquiring an image from the transfer image storage area, the hook processing module waits in standby without starting processing according to the drawing command until the image transfer process is completed.

29. An image transfer processing module used for image supply devices for supplying an image to an image display device via a network, the image supply device including an application program capable of issuing image drawing commands, and a drawing module for processing drawing commands issued by the application program, wherein

the image transfer processing module is configured to acquire an image from a transfer image storage area within a general purpose memory, and to transfer the acquired image to the image display device via the network,

the image transfer processing module is further configured to load a hook processing module for hooking and preempting a specific drawing command issued by the application program onto the application program responsive to activation of the image transfer processing module, and to unload the hook processing module from the application program responsive to deactivation of the image transfer processing module, and

the hook processing module is configured to draw an image in the transfer image storage area according to a drawing command acquired by the hooking.

30. An image transfer processing module recited in claim 29, wherein

when the image transfer processing module is executing a specified image transfer process including at least a process of acquiring an image from the transfer image storage area, the hook processing module waits in standby without starting processing according to the drawing command until the image transfer process is completed.

31. An image transfer processing module recited in claim 29, wherein

the hook processing module is configured to write, into the general purpose memory, first change area information for indicating a first change area that is an area within the transfer image storage area for which the image is drawn according to the specific drawing command, and to supply to the drawing module the specific drawing command after processing the specific drawing command, and

the drawing module is configured to draw an image into the frame memory according to a drawing command received from the application program or the hook processing module, and to write, into the general purpose memory, second change area information for indicating a second change area that is an area within the frame memory for which the image is drawn according to the drawing command, and

the image transfer processing module executes:

(i) referencing the first and second change area information stored within the general purpose memory, and acquiring an image within an area of the second change area which is not contained in the first change area from the frame memory,

(ii) acquiring an image within the first change area from the transfer image storage area without accessing the frame memory, and

(iii) transferring the acquired images together with screen update area information to the image display device via the network, the screen update area information indicating a screen update area that is the total of the first and second change areas.

32. An image transfer processing module recited in claim 29, wherein

the hook processing module is configured to write, into the general purpose memory, first change area information for indicating a first change area that is an area within the transfer image storage area for which an image is drawn according to the specific drawing command, and to supply to the drawing module the specific drawing command after processing the specific drawing command,

the drawing module is configured to draw an image into the frame memory according to a drawing command received from the application program or the hook processing module, and to write, into the general purpose memory, second change area information for indicating a second change area that is an area within the frame memory for which an image is drawn according to the drawing command, and

the image transfer processing module executes:

(i) referencing the first and second change area information stored within the general purpose memory, and acquiring from the frame memory an image within an area of the second change area which is not contained in the first change area,

(ii) acquiring an image within an area of the second change area that is also contained in the first change area from the transfer image storage area without accessing the frame memory, and

(iii) transferring the acquired images together with screen update area information to the image display device via the network, the screen update area information indicating a screen update area that is equivalent to the second change area.

33. An image transfer processing module recited in claim 29, wherein

the hook processing module is configured to write, into the general purpose memory, first change area information for indicating a first change area that is an area within the transfer image storage area for which the image is drawn according to the specific drawing command, and to supply to the drawing module the specific drawing command after processing the specific drawing command,

the drawing module is configured to draw an image into the frame memory according to a drawing command received from the application program or the hook processing module, and to write, into the general purpose memory, second change area information for indicating a second change area that is an area within the frame memory for which the image is drawn according to the drawing command, and

the image transfer processing module executes:

(i) setting as a history area the first change area that exists in the transfer process during a specified preceding time period, and writing, into the general purpose memory, transfer history information for representing the history area,

(ii) referencing the first and second change area information and the transfer history information stored in the general purpose memory, and acquiring from the frame memory an image within an area of the second change area that is not included in either the first change area or the history area,

(iii) acquiring from the transfer image storage area an image within the first change area and an image within an area of the second change area that is not contained in the first change area and that is contained in the history area, without accessing the frame memory, and

(iv) transferring the acquired images together with screen update area information to the image display area via the network, the screen update area information indicating a screen update area that is the total of the first and second change areas.

34. An image transfer processing module recited in claim 29, wherein

the hook processing module is configured to write, into the general purpose memory, first change area information for indicating a first change area that is an area within the transfer image storage area for which the image is drawn according to the specific drawing command, and to supply to the drawing module the specific drawing command after processing the specific drawing command,

the drawing module is configured to draw an image into the frame memory according to a drawing command received from the application program or the hook processing module, and to write, into the general purpose memory, second change area information for indicating a second change area that is an area within the frame memory for which the image is drawn according to the drawing command, and

the image transfer processing module executes:

(i) setting as a history area the first change area that exists for the transfer process during a specified preceding time period, and writing, into the general purpose memory, transfer history information for representing the history area,

(ii) referencing the first and second change area information and the transfer history information stored in the general purpose memory, and acquiring from the frame memory an image within an area of the second change area that is not included in either the first change area or the history area,

(iii) acquiring from the transfer image storage area an image within an area of the second change area that is contained in at least one of the first change area and the history area without accessing the frame memory, and

(iv) transferring the acquired images together with screen update area information to the image display area via the network, the screen update area information indicating a screen update area that is equivalent to the second change area.

35. An image transfer processing module recited in claim 29, wherein

the drawing module is configured to draw an image into the frame memory according to a drawing command received from the application program or the hook processing module,

and the image transfer processing module has:

a normal transfer mode for acquiring and transferring the image drawn in the transfer image storage area by the hook processing module when image drawing operation is performed, and

a resend mode for acquiring and transferring the image drawn in the frame memory by the drawing module after the image drawing operation stops.