Integration of graphical application content into the graphical scene of another application

• US 8,624,892 B2
• Filed: 11/15/2012
• Issued: 01/07/2014
• Est. Priority Date: 03/07/2006
• Status: Expired due to Fees

• Current Assignee: RPX Corporation
• Original Assignee: RPX Corporation
• Inventors: Feth, William J., Hughes, David William, Boccara, Michael
• Primary Examiner(s): Nguyen, Hau
• Assistant Examiner(s): Gray, Ryan M
• Application Number:
  US13/677,920
• Publication Number:
  US 20130069963A1
• Time in Patent Office: 418 Days
• Field of Search: None
• US Class Current: 345/419
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06T 1/00 General purpose image data processing
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics

Abstract

This application describes a system that captures 3D geometry commands from a first 3D graphics process and stores them in a shared memory. A second 3D environment process creates a 3D display environment using a display and display hardware. A third process obtains the 3D commands and supplies them to the hardware to place 3D objects in the 3D environment. The result is a fused display environment where 3D objects are displayed along with other display elements. Input events in the environment are analyzed and mapped to the 3D graphics process or the environment where they affect corresponding processing.

Claims

1. A method, comprising:

intercepting, by an intercept module, buffer-based architecture three-dimensional (3D) imagery produced by a first application, wherein the 3D imagery is not provided by the first application for consumption by the intercept module;

extracting a two-dimensional (2D) imagery, color imagery and depth imagery from the intercepted 3D imagery;

combining the 2D imagery, the color imagery and the depth imagery with a 3D graphics command stream of a second application to drive display hardware in order to integrate a 3D object based on the two-dimensional (2D) imagery, the color imagery and the depth imagery into a 3D scene, using a same 3D coordinate system of the 3D scene, based on the 3D graphics command stream of the second application, wherein the 3D scene is independent of the first application, wherein the combining further comprises modifying the 2D imagery, the color imagery and the depth imagery in order to visually integrate the 3D object into the 3D scene.

2. The method as recited in claim 1, further comprising producing an integrated 3D scene comprising the integrated 3D object and the 3D scene generated from the 2D imagery, the color imagery and the depth imagery in the display hardware.

3. The method as recited in claim 1, wherein the modifying comprises correlating 3D coordinate systems and scene lighting of the 3D object and the 3D scene.

4. The method as recited in claim 1, wherein the modifying comprises correlating visual effects of the 3D object and the 3D scene.

5. The method as recited in claim 1, further comprising driving a same graphics accelerator hardware with the 2D imagery, the color imagery and the depth imagery and the 3D graphics command stream.

6. The method as recited in claim 1, further comprising driving another display hardware with the 2D imagery, the color imagery and the depth imagery.

7. The method as recited in claim 1, further comprising storing the 2D imagery, the color imagery and the depth imagery in a memory that is shared among different processing threads, or distributed over the network between several computer hosts.

8. The method as recited in claim 1, further comprising transforming data of the 2D imagery, the color imagery and the depth imagery from a first application data type to a second application data type.

9. The method as recited in claim 1, further comprising creating a 3D media fusion environment at the second application.

10. The method as recited in claim 1, wherein the intercepting comprises copying function calls arriving at call entry points of the first application.

11. The method as recited in claim 10, wherein the combining further comprises passing the copied function calls to entry points of the second application.

12. The method as recited in claim 11, wherein the combining further comprises modifying the passed function responsive to a 3D fusion environment.

13. A system, comprising:

graphics hardware configured to produce display signals;

a display configured to produce an image using the display signals;

a shared memory; and

a computer comprising;

a three-dimensional (3D) graphics process configured to generate buffer-based architecture 3D graphics imagery;

an intercept process configured to extract a two-dimensional (2D) graphics imagery, color imagery and depth imagery from the 3D graphics imagery produced by the 3D graphics process and, wherein the 3D graphics imagery is not provided by the 3D graphics process for consumption by the intercept process, and storing the 2D graphics imagery, the color imagery and the depth imagery in the shared memory;

a 3D environment process configured to generate a 3D graphics command stream for a 3D fusion environment and supply the 3D graphics command stream to the graphics hardware;

an integration process configured to integrate a 3D object based on the stored 2D graphics imagery, the color imagery and the depth imagery into the 3D fusion environment, using a same 3D coordinate system of the 3D fusion environment, based on the 3D graphics command stream of the second application, wherein the 3D fusion environment is independent of the 3D graphics process, wherein the integration process is further configured to modify the 2D imagery, the color imagery and the depth imagery in order to visually integrate the 3D object into the 3D fusion environment.

14. The system as recited in claim 13, further comprising:

second graphics hardware receiving the 2D imagery, the color imagery and the depth imagery; and

a second display producing a display for the 2D imagery, the color imagery and the depth imagery.

15. The system of claim 13, the computer further comprising a mapping process configured to map inputs to the 3D fusion environment to the 3D graphics process and the 3D environment process.

16. A display, comprising:

a three-dimensional (3D) fusion display environment displayed in a window, independent of a first process, created by a buffer-based architecture 3D graphics command stream generated by a second process;

a 3D object created by two-dimensional (2D) graphics imagery, color imagery and depth imagery generated from 3D graphics imagery generated by a first process, intercepted by an intercept module, and integrated into the 3D fusion display environment, using a same 3D coordinate system of the 3D fusion display environment, wherein the 2D graphics imagery, the color imagery and the depth imagery are not provided by the first process for consumption by the intercept module, wherein the combining further comprises modifying the 2D imagery, the color imagery and the depth imagery in order to visually integrate the 3D object into the 3D fusion display environment; and

display hardware configured to display the integrated 3D fusion display environment.

17. The display as recited in claim 16, wherein data can be dragged and dropped into the window.

18. The display as recited in claim 16, wherein transparent collaboration on the 3D object and the 3D fusion display environment displayed in the window can be performed without altering or recompiling source code of a source application.

19. A non-transitory computer-readable medium having stored thereon computer-executable instructions that, if executed by a computing device, cause the computing device to perform a method comprising:

intercepting, by an intercept module, buffer-based architecture three-dimensional (3D) imagery produced by a first application, wherein the 3D imagery is not provided by the first application for consumption by the intercept module;

extracting a two-dimensional (2D) imagery, the color imagery and the depth imagery from the intercepted 3D imagery;

combining the 2D imagery, color imagery and depth imagery with a 3D graphics command stream of a second application to drive display hardware in order to integrate a 3D object based on the two-dimensional (2D) imagery, the color imagery and the depth imagery into a 3D scene, using a same 3D coordinate system of the 3D scene, based on the 3D graphics command stream of the second application, wherein the 3D scene is independent of the first application, wherein the combining further comprises modifying the 2D imagery, the color imagery and the depth imagery in order to visually integrate the 3D object into the 3D scene.