Providing a coherent user interface across multiple output devices

• US 20090309808A1
• Filed: 06/17/2008
• Published: 12/17/2009
• Est. Priority Date: 06/17/2008
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Swingler, Michael A.
• Granted Patent: US 8,803,896 B2
• Time in Patent Office: Days
• US Class Current: 345/1.300
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 3/1446 display composed of modules, e.g. video walls
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2360/02 Graphics controller able to handle multiple formats, e.g. input or output formats
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

Exemplary embodiments of methods and apparatuses to provide a coherent user interface across output devices having various characteristics are described. Rendering into a first window back buffer is performed to output a window on a first output device having first characteristics, and rendering into a second window back buffer is performed to display the window on a second output device having second characteristics. Pixel values of the window outputted on the first output device are stored in the first window back buffer, and the pixel values of the window outputted on the second output device are stored in the second window back buffer. The size of the first window back buffer is associated with the first characteristics of the first output device, and the size of the second window back buffer is associated with the second characteristics of the second output device.

Claims

1. A machine-implemented method, comprising:

rendering into a first window back buffer to output a window on a first output device having first characteristics; and

rendering into a second window back buffer to output the window on a second output device having second characteristics.

2. The machine-implemented method of claim 1, wherein the rendering into the first window back buffer and the rendering into the second window back buffer are synchronized.

3. The machine-implemented method of claim 1, further comprising:

storing first pixel values of the window in the first window back buffer; and

storing second pixel values of the window in the second window back buffer.

4. The machine-implemented method of claim 1, further comprising:

scaling first pixels of the window outputted on the first output device to provide a scaled version of the first pixels; and

allocating a third window back buffer for the scaled version of the first pixels.

5. The machine-implemented method of claim 4, wherein the third window back buffer is allocated in response to an event, which is one of:

(a) connecting a new display device;

(b) receiving an input to drag or move a window;

(c) receiving an input to resize a window;

(d) receiving an input to rotate the window;

(e) switching to another user;

(f) screen sharing;

(g) zooming;

(h) switching into a mirroring mode; and

any combination thereof.

6. The machine-implemented method of claim 1, wherein the first window back buffer has a first size associated with the first characteristics;

and the second window back buffer has a second size associated with the second characteristics.

7. The machine-implemented method of claim 1, further comprising:

releasing the first window back buffer.

8. The machine-implemented method of claim 1, wherein the first characteristics include a first display resolution;

and the second characteristics include a second display resolution.

9. The machine-implemented method of claim 1, wherein the first characteristics include a first video standard and the second characteristics include a second video standard.

10. The machine-implemented method of claim 1, further comprising:

allocating the first window back buffer according to the first characteristics; and

allocating the second window back buffer according to the second characteristics.

11. The machine-implemented method of claim 1, wherein the rendering into the first window back buffer is performed for a first portion of the window outputted on the first output device;

and the rendering into the second window back buffer is performed for a second portion of the window outputted on the second output device.

12. A machine-implemented method, comprising:

scaling first pixels of a first window on a first output device having first characteristics to generate a scaled version of the first pixels, wherein values of the first pixels are stored in a first window back buffer;

allocating a third window back buffer to store the values of the scaled version of the first pixels; and

rendering into the third window back buffer to output the window on a second display having second characteristics.

13. The machine-implemented method of claim 12, further comprising:

determining a scaling factor based on the first characteristics.

14. The machine-implemented method of claim 12, wherein the allocating the third window back buffer is performed in response to an event, which is one of:

(a) connecting a new display device;

(b) receiving an input to drag or move a window;

(c) receiving an input to resize a window;

(d) receiving an input to rotate the window;

(e) switching to another user;

(f) screen sharing;

(g) zooming;

(h) switching into a mirroring mode; and

any combination thereof.

15. The machine-implemented method of claim 12, further comprising:

sending a request to allocate a second window buffer according to the second characteristics of the second output device.

16. The machine-implemented method of claim 15, further comprising:

determining whether the second window back buffer is allocated, and

releasing the third window back buffer if the second window back buffer is allocated; and

continuing to maintain the third window back buffer to output the window on the second output device if the second window back buffer is not allocated.

17. The machine-implemented method of claim 16, further comprising:

storing the values of second pixels of the window in the second window back buffer; and

rendering into the second window back buffer according to the second characteristics of the second output device.

18. The machine-implemented method of claim 17, further comprising:

releasing the first window back buffer.

19. The machine-implemented method of claim 12, wherein the first characteristics include a first display resolution;

and the second characteristics include a second display resolution.

20. The machine-implemented method of claim 12, wherein the first characteristics include a first video standard and the second characteristics include a second video standard.

21. The machine-implemented method of claim 12, wherein the window is mirrored on the second output device.

22. The machine-implemented method of claim 12, wherein the window is straddled from the first output device to the second output device.

23. A machine-implemented method, comprising:

rendering into a first window back buffer to output a window on a first output device having first characteristics; and

allocating a second window back buffer associated with second characteristics of a second output device.

24. The machine-implemented method of claim 23, further comprising

receiving a request to allocate the second window back buffer.

25. The machine-implemented method of claim 23, further comprising:

storing values of pixels of the window in the second window back buffer.

26. The machine-implemented method of claim 23, further comprising:

rendering into the second window back buffer according to the second characteristics.

27. The machine-implemented method of claim 23, further comprising:

releasing the first window back buffer.

28. A machine-readable medium storing executable program instructions which cause a data processing system to perform operations, comprising:

rendering into a first window back buffer to output a window on a first output device having first characteristics; and

rendering into a second window back buffer to output the window on a second output device having second characteristics.

29. The machine-readable medium of claim 28, wherein the rendering into the first window back buffer and the rendering into the second window back buffer are synchronized.

30. The machine-readable medium of claim 28, further including instructions that cause the data processing system to perform operations comprising:

storing first pixel values of the window in the first window back buffer; and

storing second pixel values of the window in the second window back buffer.

31. The machine-readable medium of claim 28, further including instructions that cause the data processing system to perform operations comprising:

scaling first pixels of the window outputted on the first output device to provide a scaled version of the first pixels; and

allocating a third window back buffer for the scaled version of the first pixels.

32. The machine-implemented method of claim 31, wherein the third window back buffer is allocated in response to an event, which is one of:

(a) connecting a new display device;

(b) receiving an input to drag or move a window;

(c) receiving an input to resize a window;

(d) receiving an input to rotate the window;

(e) switching to another user;

(f) screen sharing;

(g) zooming;

(h) switching into a mirroring mode; and

any combination thereof.

33. The machine-readable medium of claim 28, wherein the first window back buffer has a first size associated with the first characteristics;

and the second window back buffer has a second size associated with the second characteristics.

34. The machine-readable medium of claim 28, further including instructions that cause the data processing system to perform operations comprising:

releasing the first window back buffer.

35. The machine-readable medium of claim 28, wherein the first characteristics include a first display resolution;

and the second characteristics include a second display resolution.

36. The machine-readable medium of claim 28, wherein the first characteristics include a first video standard and the second characteristics include a second video standard.

37. The machine-readable medium of claim 28, further including instructions that cause the data processing system to perform operations comprising:

allocating the first window back buffer according to the first characteristics; and

allocating the second window back buffer according to the second characteristics.

38. The machine-readable medium of claim 28, wherein the rendering into the first window back buffer is performed for a first portion of the window outputted on the first output device;

and the rendering into the second window back buffer is performed for a second portion of the window outputted on the second output device.

39. A machine-readable medium storing executable program instructions which cause a data processing system to perform operations, comprising:

scaling first pixels of a first window on a first output device having first characteristics to generate a scaled version of the first pixels, wherein values of the first pixels are stored in a first window back buffer;

allocating a third window back buffer to store the values of the scaled version of the first pixels; and

rendering into the third window back buffer to output the window on a second output device having second characteristics.

40. The machine-readable medium of claim 39, further including instructions that cause the data processing system to perform operations comprising:

determining a scaling factor based on the first characteristics.

41. The machine-readable medium of claim 39, wherein the allocating the third window back buffer is performed in response to an event, which is one of:

(a) connecting a new display device;

(b) receiving an input to drag or move a window;

(c) receiving an input to resize a window;

(d) receiving an input to rotate the window;

(e) switching to another user;

(f) screen sharing;

(g) zooming;

(h) switching into a mirroring mode; and

any combination thereof.

42. The machine-readable medium of claim 39, further including instructions that cause the data processing system to perform operations comprising:

sending a request to allocate a second window buffer according to the second characteristics of the second output device.

43. The machine-readable medium of claim 42, further including instructions that cause the data processing system to perform operations comprising:

determining whether the second window back buffer is allocated, and

releasing the third window back buffer if the second window back buffer is allocated; and

continuing to maintain the third window back buffer to output the window on the second output device if the second window back buffer is not allocated.

44. The machine-readable medium of claim 43, further including instructions that cause the data processing system to perform operations comprising:

storing the values of second pixels of the window in the second window back buffer; and

rendering into the second window back buffer according to the second characteristics of the second output device.

45. The machine-readable medium of claim 44, further including instructions that cause the data processing system to perform operations comprising:

releasing the first window back buffer.

46. The machine-readable medium of claim 39, wherein the first characteristics include a first display resolution;

and the second characteristics include a second display resolution.

47. The machine-readable medium of claim 39, wherein the first characteristics include a first video standard and the second characteristics include a second video standard.

48. The machine-readable medium of claim 39, wherein the window is mirrored on the second output device.

49. The machine-readable medium of claim 39, wherein the window is straddled from the first output device to the second output device.

50. A machine-readable medium storing executable program instructions which cause a data processing system to perform operations, comprising:

rendering into a first window back buffer to output a window on a first output device having first characteristics; and

allocating a second window back buffer associated with second characteristics of a second output device.

51. The machine-readable medium of claim 50, further including instructions that cause the data processing system to perform operations comprising:

receiving a request to allocate the second window back buffer.

52. The machine-readable medium of claim 50, further including instructions that cause the data processing system to perform operations comprising:

storing values of pixels of the window in the second window back buffer.

53. The machine-readable medium of claim 50, further including instructions that cause the data processing system to perform operations comprising:

rendering into the second window back buffer according to the second characteristics.

54. The machine-readable medium of claim 50, further including instructions that cause the data processing system to perform operations comprising:

releasing the first window back buffer.

55. A data processing system, comprising:

a processor;

a memory coupled to the processor, wherein the processor is configured to render into a first window back buffer to output a window on a first output device having first characteristics; and

to render into a second window back buffer to output the window on a second output device having second characteristics.

56. The data processing system of claim 55, wherein the rendering into the first window back buffer and the rendering into the second window back buffer are synchronized.

57. The data processing system of claim 55, wherein the processor is further configured to store first pixel values of the window in the first window back buffer;

and

to store second pixel values of the window in the second window back buffer.

58. The data processing system of claim 55, wherein the processor is further configured to scale first pixels of the window outputted on the first output device to provide a scaled version of the first pixels;

and

to allocate a third window back buffer for the scaled version of the first pixels.

59. The data processing system of claim 58, wherein the third window back buffer is allocated in response to an event, which is one of:

(a) connecting a new display device;

(b) receiving an input to drag or move a window;

(c) receiving an input to resize a window;

(d) receiving an input to rotate the window;

(e) switching to another user;

(f) screen sharing;

(g) zooming;

(h) switching into a mirroring mode; and

any combination thereof.

60. The data processing system of claim 55, wherein the first window back buffer has a first size associated with the first characteristics;

and

the second window back buffer has a second size associated with the second characteristics.

61. The data processing system of claim 55, wherein the processor is further configured to release the first window back buffer.

62. The data processing system of claim 55, wherein the first characteristics include a first display resolution;

and the second characteristics include a second display resolution.

63. The data processing system of claim 55, wherein the first characteristics include a first video standard and the second characteristics include a second video standard.

64. The data processing system of claim 55, wherein the processor is further configured to allocate the first window back buffer according to the first characteristics;

and

allocate the second window back buffer according to the second characteristics.

65. The data processing system of claim 55, wherein the rendering into the first window back buffer is performed for a first portion of the window outputted on the first output device;

and the rendering into the second window back buffer is performed for a second portion of the window outputted on the second output device.

66. A data processing system, comprising:

a processor;

a memory coupled to the processor, wherein the processor is configured to scale first pixels of a first window on a first output device having first characteristics to generate a scaled version of the first pixels, wherein values of the first pixels are stored in a first window back buffer;

to allocate a third window back buffer to store the values of the scaled version of the first pixels; and

to render into the third window back buffer to output the window on a second output device having second characteristics.

67. The data processing system of claim 66, wherein the processor is further configured to determine a scaling factor based on the first characteristics.

68. The data processing system of claim 66, wherein the allocating the third window back buffer is performed in response to an event, which is one of:

(a) connecting a new display device;

(b) receiving an input to drag or move a window;

(c) receiving an input to resize a window;

(d) receiving an input to rotate the window;

(e) switching to another user;

(f) screen sharing;

(g) zooming;

(h) switching into a mirroring mode; and

any combination thereof.

69. The data processing system of claim 66, wherein the processor is further configured to send a request to allocate a second window buffer according to the second characteristics of the second output device.

70. The data processing system of claim 69, wherein the processor is further configured to determine whether the second window back buffer is allocated, to release the third window back buffer if the second window back buffer is allocated;

and to continue to maintain the third window back buffer to output the window on the second output device if the second window back buffer is not allocated.

71. The data processing system of claim 70, wherein the processor is further configured to store the values of second pixels of the window in the second window back buffer;

and to render into the second window back buffer according to the second characteristics of the second output device.

72. The data processing system of claim 71, wherein the processor is further configured to release the first window back buffer.

73. The data processing system of claim 66, wherein the first characteristics include a first display resolution;

and the second characteristics include a second display resolution.

74. The data processing system of claim 66, wherein the first characteristics include a first video standard and the second characteristics include a second video standard.

75. The data processing system of claim 66, wherein the window is mirrored on the second output device.

76. The data processing system of claim 66, wherein the window is straddled from the first output device to the second output device.

77. A data processing system, comprising:

a processor;

a memory coupled to the processor, wherein the processor is configured to

render into a first window back buffer to output a window on a first output device having first characteristics, and to allocate a second window back buffer associated with second characteristics of a second output device.

78. The data processing system of claim 77, wherein the processor is further configured to receive a request to allocate the second window back buffer.

79. The data processing system of claim 77, wherein the processor is further configured to store values of pixels of the window in the second window back buffer.

80. The data processing system of claim 77, wherein the processor is further configured to render into the second window back buffer according to the second characteristics.

81. The data processing system of claim 77, wherein the processor is further configured to release the first window back buffer.