System, method, and computer program product for modifying signals of a three-dimensional graphics application program based on a tracking algorithm

• US 8,659,590 B1
• Filed: 12/17/2008
• Issued: 02/25/2014
• Est. Priority Date: 12/17/2008
• Status: Active Grant

• Current Assignee: NVIDIA Corporation
• Original Assignee: NVIDIA Corporation
• Inventors: Stam, Joseph Scott, Luebke, David Patrick, Cook, David Robert, Slavenburg, Gerrit A.
• Primary Examiner(s): Wu, Xiao M.
• Assistant Examiner(s): He, Yingchun
• Application Number:
  US12/336,878
• Time in Patent Office: 1,896 Days
• Field of Search: 345/419, 345/619, 725/37, 463/36
• US Class Current: 345/419
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics

Abstract

A system, method, and computer program product are provided for modifying signals of a three-dimensional graphics application program based on a tracking algorithm. In operation, a plurality of signals are received from a three-dimensional graphics application program. Additionally, the signals, or a derivative thereof, are modified based on an algorithm that tracks at least a portion of an upper body of a person. Furthermore, the modified signals are output.

Claims

1. A method, comprising:

intercepting, by a driver, an application program interface (API) command stream including a plurality of API calls from a three-dimensional graphics application program;

modifying, by the driver, at least one of the plurality of API calls, based on an algorithm that tracks at least a portion of an upper body of a person, by displacing viewpoint data associated with the at least one of the plurality of API calls based on the at least a portion of the upper body of the person tracked utilizing the algorithm;

generating a pushbuffer utilizing the at least one modified API call, and splitting the pushbuffer into a first pushbuffer and a second pushbuffer; and

outputting, from the driver, the at least one modified API call to a graphics processing unit for generating at least one image utilizing the at least one modified API call, wherein the outputting of the at least one modified API call to the graphics processing unit includes communication data in each of the first pushbuffer and the second pushbuffer to the graphics processing unit, without communicating the at least one modified API call back to the three-dimensional graphics application program;

wherein the driver that intercepts the API command stream and the algorithm that tracks the at least a portion of the upper body of the person are separate from the three-dimensional graphics application program, such that the three-dimensional graphics application program is prevented from being aware of the interception and the modification of the at least one modified API call.

2. The method of claim 1, wherein the portion of the upper body includes a head of the person.

3. The method of claim 2, wherein the algorithm tracking the at least a portion of the upper body of the person determines that the head of the person is tilted with respect to a display, and, in response, a first image generated utilizing the at least one modified API call for a first eye of the person is positioned higher than a second image generated utilizing the at least one modified API call for a second eye of the person.

4. The method of claim 1, wherein the portion of the upper body includes at least a portion of a face of the person.

5. The method of claim 1, wherein the plurality of application program interface calls are associated with a standard application program interface.

6. The method of claim 1, wherein the three-dimensional graphics application program operates independent of the algorithm.

7. The method of claim 1, and further comprising processing the at least one modified API call for stereoscopic display.

8. The method of claim 1, wherein the modifying is performed utilizing computer code embodied on a computer readable medium.

9. The method of claim 1, wherein the modifying is performed utilizing hardware.

10. The method of claim 9, wherein the hardware includes the graphics processing unit.

11. The method of claim 1, wherein the modifying includes modifying the at least one of the plurality of API calls at a pushbuffer level.

12. The method of claim 1, wherein the modifying includes modifying the at least one of the plurality of API calls at a processing level defined by the driver.

13. The method of claim 1, wherein the portion of the upper body includes at least one eye of the person.

14. The method of claim 1, wherein the algorithm utilizes input from one or more cameras to track at least one of a head position, a body position, or an eye position of the person.

15. The method of claim 1, wherein the modification of the at least one of the plurality of API calls includes generating two new API calls for one or more different views corresponding to the at least one modified API call.

16. The method of claim 15, wherein the modification of the at least one of the plurality of API, calls generates a first view for a right eye and a second view for a left eye by displacing the viewpoint data associated with the at least one of the plurality of API cabs.

17. The method of claim 1, wherein the modification of the at least one of the plurality of API calls by displacing the viewpoint data includes displacing an original viewpoint specified by the three-dimensional graphics application program with a new viewpoint.

18. The method of claim 1, wherein outputting the at least one modified API call includes communicating the at least one modified API call to an external device for rendering and display of the at least one image, without communicating the at least one modified API call back to the three-dimensional graphics application program.

19. The method of claim 18, wherein the external device to which the at least one modified API call is communicated to is not capable of being used with the API command stream received from the three-dimensional graphics application program.

20. The method of claim 1, wherein the portion of the upper body of the person tracked utilizing the algorithm further includes an eye of the person, such that the algorithm determines a position of the eye of the person.

21. The method of claim 20, wherein the modification of the at least one of the plurality of API calls by displacing the viewpoint data includes:

displacing in one operation the viewpoint data associated with the at least one of the plurality of API calls based on a position of a head of the person, and

displacing in another operation the viewpoint data associated with the at least one of the plurality of API calls based on the position of the eye of the person.

22. A computer program product embodied on a non-transitory computer readable medium, comprising:

computer code for intercepting, by a driver, an application program interface (API) command stream including a plurality of API calls from a three-dimensional graphics application program;

computer code for modifying, by the driver, at least one of the plurality of API calls, based on an algorithm that tracks at least a portion of an upper body of a person, by displacing viewpoint data associated with the at least one of the plurality of API calls based on the at least a portion of the upper body of the person tracked utilizing the algorithm;

computer code for generating a pushbuffer utilizing the at least one modified API call, and splitting the pushbuffer into a first pushbuffer and a second pushbuffer; and

computer code for outputting, from the driver, the at least one modified API call to a graphics processing unit for generating at least one image utilizing the at least one modified API call, wherein the outputting of the at least one modified API call to the graphics processing unit includes communicating data in each of the first pushbuffer and the second pushbuffer to the graphics processing unit, without communicating the at least one modified API call back to the three-dimensional graphics application program;

wherein the computer program product is operable such that the driver that intercepts the API command stream and the algorithm that tracks the at least a portion of the upper body of the person are separate from the three-dimensional graphics application program, such that the three-dimensional graphics application program is prevented from being aware of the interception and the modification of the at least one modified API call.

23. An apparatus, comprising:

an interface included in a driver, the interface for intercepting an application program interface (API) command stream including a plurality of API calls from a three-dimensional graphics application program; and

a processor for;

modifying, utilizing the driver, at least one of the plurality of API calls, based on an algorithm that tracks at least a portion of an upper body of a person, by displacing viewpoint data associated with the at least one of the plurality of API calls based on the at least a portion of the upper body of the person tracked utilizing the algorithm,

generating a pushbuffer utilizing the at least one modified API call, and splitting the pushbuffer into a first pushbuffer and a second pushbuffer, and

outputting, from the driver, the at least one modified API call to a graphics processing unit for generating at least one image utilizing the at least one modified API call, wherein the outputting of the at least one modified API call to the graphics processing unit includes communicating data in each of the first pushbuffer and the second pushbuffer to the graphics processing unit, without communicating the at least one modified API call back to the three-dimensional graphics application program;

wherein the apparatus is operable such that the driver that intercepts the API command stream and the algorithm that tracks the at least a portion of the upper body of the person are separate from the three-dimensional graphics application program, such that the three-dimensional graphics application program is prevented from being aware of the interception and the modification of the at least one modified API call.

24. The apparatus of claim 23, wherein the processor remains in communication with memory and a display via a bus.