Capturing graphics primitives associated with any display object rendered to a graphical user interface

• US 6,802,055 B2
• Filed: 06/27/2001
• Issued: 10/05/2004
• Est. Priority Date: 06/27/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Thompson, Peter A., Jade, Shashidhar P.
• Primary Examiner(s): Dam, Tuan
• Assistant Examiner(s): Vo, Ted T.
• Application Number:
  US09/892,989
• Publication Number:
  US 20030001854A1
• Time in Patent Office: 1,196 Days
• Field of Search: 717/104-113, 717/120-135, 717/158-159, 717/174-178, 345/619, 345/771-773, 702/122-123, 709/223-224, 714/34-35, 714/38-39
• US Class Current: 717/130
• CPC Class Codes: G06F 9/454 Multi-language systems; Localisation; Internationalisation

Abstract

The present invention provides a mechanism for capturing the one or more graphics primitives drawn to a user interface by an application in execution is presented. A calling process invokes an injection component to inject a spy component into a target process. Once the spy component is injected, it installs patches and hook functions into the executable code of the operating system API'"'"'s that relate to the output of graphics primitives to the graphical user interface. The hook functions monitor operating system messages generated during the execution of the target process/application, while the patches allow for the capture of graphics primitives and attributes associated with such graphics primitives. Whenever a display object is rendered to an interface screen as a result of an invoked action within the target process, the hook functions are called to capture the operating system messages passed, and the patches capture the graphics primitives that are drawn to the screen to render the display object. This information is then packaged and delivered to the calling process for processing. Because the graphics primitives and attributes associated with such graphics primitives are captured in connection with the operating system messages passed during process execution, the calling process is able to obtain complete information about any viewable or executable object displayed by the target process.

Claims

1. A method for a calling process to manipulate information presented by a target process, the method comprising:

injecting a spy component into the target process by the calling process, wherein the spy component is an executable program module;

capturing one or more graphics primitives and attributes associated with such graphics primitives during execution of the target process;

returning the graphics primitives and attributes associated with such graphics primitives to the calling process; and

producing, by the calling process, an output, the output based, at least in part, on the returned graphics primitives and attributes associated with such graphics primitives.

2. The method of claim 1, wherein the step of injecting the spy component is performed by an injection component that is invoked by the calling process.

3. The method of claim 2, wherein the injection component is an executable program module comprising executable routines for injecting source code into a target process.

4. The method of claim 1, wherein the step of injecting includes inserting one or more function patches into one or more executable program modules that correspond to an operating system upon which the target process is being executed, the executable program modules having instructions for rendering graphics primitives to a graphical user interface.

5. The method of claim 4, wherein the step of inserting the function patches is performed by the spy component.

6. The method of claim 1, wherein the step of injecting includes installing one or more hook functions into the operating system APIs that generate system messages in the event of a display object being output to a user interface screen during the execution of the target process.

7. The method of claim 6, wherein the step of installing is performed by the spy component.

8. The method of claim 6, wherein the system messages provide context information that is descriptive of an invoked action within the target process.

9. The method of claim 6, wherein the one or more hook functions are installed by a hook management component that is called upon by the spy component after injection into the target process.

10. The method of claim 9, wherein the hook management component is responsible for uninstalling the one or more hook functions upon termination of the target process.

11. The method of claim 6, wherein the step of capturing includes calling the one or more hook functions to intercept any system messages generated as a result of an invoked action within the target process.

12. The method of claim 11, wherein the one or more hook functions set a flag to activate one or more function patches installed by the spy component.

13. The method of claim 1, wherein the step of capturing includes invalidating a display object that is output to a user interface as a result of an invoked action within the target process.

14. The method of claim 13, wherein the step of invalidating includes calling function patches to capture the graphics primitives and attributes associated with such graphics primitives as they are drawn to the user interface to render the display object.

15. The method of claim 1, wherein the step of returning includes sending context information captured by one or more hook functions to the calling process as a system message.

16. The method of claim 1, wherein the step of returning further includes sending the graphics primitives and attributes associated with such graphics primitives that are captured to the calling process as a system message.

17. The method of claim 1, wherein the calling process is selected from the group consisting of:

a localization tool, a language-processing application a text-to-speech application, and an operating system utility.

18. The method of claim 1, wherein producing an output comprises performing an operation selected from the group consisting of:

displaying an image on a display screen and producing an audio output.

19. A computer-readable medium having computer-executable instructions for a calling process to manipulate information presented by a target process, the computer-executable instructions performing steps comprising:

injecting a spy component into the target process by the calling process, wherein the spy component is an executable program module;

capturing one or more graphics primitives and attributes associated with such graphics primitives during execution of the target process;

returning the graphics primitives and attributes associated with such graphics primitives to the calling process; and

producing, by the calling process, an output, the output based, at least in part, on the returned graphics primitives and attributes associated with such graphics primitives.

20. A system for manipulating information presented by a target process, the system comprising:

an injection component for injecting a spy component into the target process residing on a computer;

a spy component for capturing graphics primitives and attributes associated with such graphics primitives in connection with system messages that are generated by the target process as a result of an invoked action within the target process;

a hook management component for installing and uninstalling one or more hook functions into one or more program modules that are executed by an operating system residing on the computer, the program modules having instructions for generating system messages during the execution of the target process; and

a calling process for producing an output based, at least in part, on the captured graphics primitives and attributes associated with such graphics primitives.

21. The system of claim 20, wherein the injection component is an executable program module consisting of executable instructions for injecting the spy component into the executable code of the target process.

22. The system of claim 20, wherein the injection component injects the spy component into the target process on behalf of the calling process.

23. The system of claim 22, wherein the calling process is a computer-executable application.

24. The system of claim 20, wherein the spy component inserts one or more function patches into one or more executable program modules that correspond to the operating system upon which the target process is being executed, the executable program modules having instructions for rendering graphics primitives to a graphical user interface.

25. The system of claim 24, wherein the function patches capture graphics primitives and associated attributes of such graphics primitives that are rendered to a user interface by a display object as a result of an action invoked within the target process.

26. The system of claim 25, wherein the spy component packages the graphics primitives and attributes associated with such graphics primitives and sends them to the calling process as a system message.

27. The system of claim 20, wherein the spy component calls the hook management component to insert one or more hook functions into one or more executable program modules that correspond to the operating system upon which the target process is being executed, the executable program modules having instructions for generating system messages in the event of a display object being output to a user interface during execution of the target process.

28. The system of claim 27, wherein the one or more hook functions set a flag to activate one or more function patches installed by the spy component.

29. The system of claim 27, wherein the one or more hook functions intercept any system messages generated as a result of an invoked action within the target process.

30. The method of claim 29, wherein the system message contain context information that is descriptive of an invoked action within the target process.

31. The system of claim 20, wherein the spy component packages context information and sends it to the calling process as an OS message.

32. The system of claim 20, wherein the hook management component is responsible for installing and uninstalling hook functions on behalf of the spy component.

33. The system of claim 32, wherein the hook functions are installed by the hook management component upon termination of the target process.

34. The system of claim 20, wherein the calling process is selected from the group consisting of:

a localization tool, a language-processing application a text-to-speech application, and an operating system utility.

35. The system of claim 20, wherein producing an output comprises performing an operation selected from the group consisting of:

displaying an image on a display screen and producing an audio output.