CENTRALISED INTERACTIVE GRAPHICAL APPLICATION SERVER

• US 20140111529A1
• Filed: 12/21/2013
• Published: 04/24/2014
• Est. Priority Date: 03/01/2002
• Status: Active Grant

• Current Assignee: T5 Labs Ltd.
• Original Assignee: T5 Labs Ltd.
• Inventors: Clemie, Graham, Duckett, Dedrick
• Granted Patent: US 9,117,285 B2
• Time in Patent Office: Days
• US Class Current: 345/522
• CPC Class Codes: A63F 13/327 using wireless networks, e.g. Wi-Fi® or piconet
• CPC Class Codes: A63F 13/332 using wireless networks, e.g. cellular phone networks
• CPC Class Codes: A63F 2300/203 Image generating hardware
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 2210/08 Bandwidth reduction
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/463 by compressing encoding parameters before transmission
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation

Abstract

A system for processing a plurality of graphical programs on a centralized computer system whereby the images produced by the programs are compressed and transmitted to a plurality of remote processing devices where they are decompressed. Compression assistance data (CAD) is produced by intercepting instructions outputted by the programs and the CAD is then used in the compression step.

Claims

1. A system comprising:

i) an interception module;

ii) a graphics processor module (GPM) accessible by the instruction interception module;

iii) memory accessible by the GPM and the instruction interception module;

iv) a network interface;

such that operation of the system causes the system to perform at least the following steps;

a) intercepting instructions outputted by a graphics generating program, the graphics generating program being responsive at least partially to the user input;

b) providing supplementary instructions responsive to the outputted instructions;

c) processing at least one of the outputted instructions to create a first plurality of pixels;

d) processing at least one of the supplementary instructions to create a second plurality of pixels;

e) processing at least one of the supplementary instructions to produce compression assistance data; and

f) processing at least one of the first plurality of pixels or at least one of the second plurality of pixels or a combination thereof using at least part of the compression assistance data to produce compressed image data;

wherein the memory and the GPM are integrated within one unit of hardware.

2. The system as claimed in claim 1 wherein the processing in step (b) further comprises the step of analysing the outputted instructions and whereby the producing of the compression assistance data is responsive to the analysis.

3. The system as claimed in claim 2 wherein the supplementary instructions comprise modified instructions wherein the modifying is responsive to the analysis.

4. A system comprising:

i) an image data compressor;

ii) an instruction interception module;

iii) a graphics processor module (GPM) accessible by the instruction interception module;

iv) memory accessible by the image data compressor and the GPM;

such that operation of the system causes the system to perform at least the following steps;

a) intercepting processor instructions by the instruction interception module;

b) identifying an attribute of at least one of the intercepted processor instructions;

c) generating compression assistance data (CAD) or compression assistance instructions (CAI) or a combination thereof that are responsive to the attribute;

d) modifying at least one of the intercepted processor instructions;

e) processing at least one of the modified processor instructions by the GPM to generate a plurality of pixels to be stored in the memory; and

f) processing at least a portion of the plurality of pixels using at least part of the CAD or at least part of the CAI or a combination thereof to generate compressed image data.

5. The system as claimed in claim 4 wherein step (c) occurs before step (d).

6. The system as claimed in claim 4 wherein the image data compressor and the GPM are integrated within one unit of graphics processor hardware.

7. The system as claimed in claim 6 wherein the generation of the compressed image data is at least partially performed by the image data compressor.

8. The system as claimed in claim 6 wherein the CAD comprises details of which of the pixels will be used to generate the compressed image data.

9. The system as claimed in claim 6 wherein the attribute is responsive to the type, the contents, or a combination thereof of at least one of the intercepted processor instructions.

10. The system as claimed in claim 4 wherein the generation of the compressed image data is at least partially performed by the image data compressor.

11. The system as claimed in claim 10 wherein the CAD comprises details of which of the pixels will be used to generate the compressed image data.

12. The system as claimed in claim 10 wherein the attribute is responsive to the type, the contents, or a combination thereof of at least one of the intercepted processor instructions.

13. The system as claimed in claim 4 wherein the CAD comprises details of which of the pixels will be used to generate the compressed image data.

14. The system as claimed in claim 13 wherein the attribute is responsive to the type, the contents, or a combination thereof of at least one of the intercepted processor instructions.

15. The system as claimed in claim 14 wherein the GPM comprises at least one GPU.

16. The system as claimed in claim 13 wherein the GPM comprises at least one GPU.

17. The system as claimed in claim 4 wherein the attribute is responsive to the type, the contents, or a combination thereof, of at least one of the intercepted processor instructions.

18. The system as claimed in claim 17 wherein the GPM comprises at least one GPU.

19. The system as claimed in claim 4 wherein the step of modifying comprises a step selected from a group of steps consisting of adding instructions, removing instructions, altering instructions, replacing instructions, and any combination thereof.

20. The system as claimed in claim 19 wherein the GPM comprises at least one GPU.

21. The system as claimed in claim 4 wherein the GPM comprises at least one GPU.

22. A system comprising:

i) an image data compressor;

ii) an instruction interception module;

iii) a graphics processor module (GPM) accessible by the instruction interception module;

iv) memory accessible by the image data compressor and the GPM;

such that operation of the system causes the system to perform at least the following steps;

a) intercepting a first set of processor instructions from a first graphics generating computer program by the instruction interception module to create a first set of intercepted processor instructions (IPI-1);

b) intercepting a second set of processor instructions from a second graphics generating computer program by the instruction interception module to create a second set of intercepted processor instructions (IPI-2);

c) identifying an attribute of at least one of the instructions of IPI-1 or at least one of the instructions of IPI-2 or a combination of instructions from IPI-1 and IPI-2;

d) generating compression assistance data (CAD) or compression assistance instructions (CAI) or a combination thereof that are responsive to the attribute;

e) modifying at least one of the instructions of IPI-1 to create a first set of modified processor instructions (MPI-1);

f) modifying at least one of the instructions of IPI-2 to create a second set of modified instructions (MPI-2);

g) processing at least one of the MPI-1 instructions by the GPM to generate a first frame of image pixels;

h) processing at least one of the MPI-2 instructions by the GPM to generate a second frame of image pixels;

i) storing the majority or all of the first frame of image pixels in a first set of locations in the memory;

j) storing the majority or all of the second frame of image pixels in a second set of locations in the memory; and

k) processing at least a portion of the first frame using at least part of the CAD or at least part of the CAI or a combination thereof to generate first compressed image data;

l) processing at least a portion of the second frame using at least part of the CAD or at least part of the CAI or a combination thereof to generate second compressed image data;

wherein the majority or all of the first set of memory locations do not overlap the second set of memory locations.

23. The system as claimed in claim 22 wherein step (d) occurs before step (e).

24. The system as claimed in claim 22 wherein the image data compressor and the GPM are integrated within one unit of graphics processor hardware.

25. The system as claimed in claim 24 wherein the generation of the compressed image data is at least partially performed by the image data compressor.

26. The system as claimed in claim 25 wherein the attribute is responsive to the type, the contents, or a combination thereof of at least one of the intercepted processor instructions.

27. The system as claimed in claim 25 wherein the GPM comprises at least one GPU.

28. The system as claimed in claim 24 wherein the CAD or the CAI or a combination thereof comprises details of which of the pixels will be used to generate the compressed image data.

29. The system as claimed in claim 24 wherein the attribute is responsive to the type, the contents, or a combination thereof of at least one of the intercepted processor instructions.

30. The system as claimed in claim 24 wherein the GPM comprises at least one GPU.

31. The system as claimed in claim 22 wherein the generation of the compressed image data is at least partially performed by the image data compressor.

32. The system as claimed in claim 31 wherein the CAD or the CAI or a combination thereof comprises details of which of the pixels will be used to generate the compressed image data.

33. The system as claimed in claim 31 wherein the attribute is responsive to the type, the contents, or a combination thereof of at least one of the intercepted processor instructions.

34. The system as claimed in claim 31 wherein the GPM comprises at least one GPU.

35. The system as claimed in claim 22 wherein the CAD or the CAI or a combination thereof comprises details of which of the pixels will be used to generate the compressed image data.

36. The system as claimed in claim 35 wherein the GPM comprises at least one GPU.

37. The system as claimed in claim 22 wherein the attribute is responsive to the type, the contents, or a combination thereof of at least one of the intercepted processor instructions.

38. The system as claimed in claim 22 wherein operation of the system causes the system to perform the step of delivering information specifying at least one capability of the GPM to the first graphics generating computer program wherein the information delivered differs at least in part from the actual capabilities of the GPM.

39. The system as claimed in claim 22 wherein the GPM comprises at least one GPU.

40. A system comprising:

i) an instruction interception module;

ii) a plurality of graphics processor units (GPUs) accessible by the instruction interception module;

iii) memory accessible by the plurality of GPUs;

such that operation of the system causes the system to perform at least the following steps;

a) intercepting a first set of processor instructions from a first graphics generating computer program by the instruction interception module to create a first set of intercepted processor instructions (IPI-1);

b) intercepting a second set of processor instructions from a second graphics generating computer program by the instruction interception module to create a second set of intercepted processor instructions (IPI-2);

c) modifying at least one of the instructions of IPI-1 to create a first set of modified processor instructions (MPI-1);

d) modifying at least one of the instructions of IPI-2 to create a second set of modified processor instructions (MPI-2);

e) processing at least one of the MPI-1 instructions by a first of the plurality of GPUs to generate a first frame of image pixels;

f) processing at least one of the MPI-2 instructions by a second of the plurality of GPUs to generate a second frame of image pixels; and

g) processing at least a portion of the first frame of image pixels to generate compressed image data.

41. The system as claimed in claim 40 wherein step (c) occurs before step (d).

42. The system as claimed in claim 40 wherein the system further comprises an image data compressor, and wherein at least part of the memory is accessible by the image data compressor.

43. The system as claimed in claim 42 wherein the generation of the compressed image data is at least partially performed by the image data compressor.

44. The system as claimed in claim 40 wherein the step of modifying comprises a step selected from a group of steps consisting of adding instructions, removing instructions, altering instructions, replacing instructions, and any combination thereof.

45. The system as claimed in claim 40 wherein the modifying in step (d) directs the processing in step (f) to occur in the second of the plurality of GPUs.