Display wall layout optimization

• US 9,177,402 B2
• Filed: 12/19/2012
• Issued: 11/03/2015
• Est. Priority Date: 12/19/2012
• Status: Active Grant

• Current Assignee: Barco NV
• Original Assignee: Barco NV
• Inventors: Demeyere, Kristof R., Demuytere, Pieter F. R.
• Primary Examiner(s): PHANTANA ANGKOOL, DAVID
• Application Number:
  US13/720,213
• Publication Number:
  US 20140173507A1
• Time in Patent Office: 1,049 Days
• Field of Search: 715/789, 715/780, 715/764, 715/763
• US Class Current: 1/1
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/1446 display composed of modules, e.g. video walls
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2370/20 Details of the management of multiple sources of image data
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: G09G 5/30 Control of display attribute

Abstract

The present invention provides a system, method, and apparatus for optimally determining, for a layout of windows on a display wall of a display system, the quality of display content to display in each window without exceeding system resources. The display system receives display content from at least one source, at least one of the at least one source providing display content at multiple qualities. Displaying display content uses system resources, and displaying high quality display content uses more system resources than displaying low quality display content. A received layout defines an arrangement of windows on the display wall, each window directed to displaying display content from one source. An objective function defines the quality of display content displayed in each window. The objective function is optimized to determine an optimized layout that does not exceed system resources.

Claims

1. A controller for optimally determining, for a layout of windows on a display wall of a display system, the quality of display content to display in each window without exceeding system resources of the display system, the display system receiving display content from at least one source, at least one of the at least one source providing display content at multiple qualities, the controller is arranged to:

receive the layout, the layout defining an arrangement of windows on the display wall, each window directed to displaying display content from one source;

construct an objective function defining the quality of display content displayed in each window to be low quality display content or high quality display content that uses more of the s stem resources than displaying the low quality display content;

wherein the controller includes a processor, the processor arranged to optimize the objective function to determine an optimized layout that does not exceed system resources of the display system; and

wherein the controller outputs the optimized layout for displaying display content using the system resources of the display system.

2. The controller of claim 1, wherein the objective function is weighted to favor the selection of high quality display content for windows having larger areas.

3. The controller of claim 2, wherein, in the objective function, the weight given to each window to favor the selection of high quality display content for windows having larger areas is proportional to the area of the window.

4. The controller of claim 1, wherein optimizing the objective function includes maximizing the objective function such that for the optimized layout a maximum of system resources are used without exceeding the system resources.

5. The controller of claim 4, wherein the objective function is a linear objective function that is maximized using integer programming.

6. The controller of claim 5, wherein the objective function is maximized using binary integer programming.

7. The controller of claim 1, wherein constraints to the objective function are constructed prior to optimizing the objective function and the objective function is optimized within the constraints.

8. The controller of claim 7, the display system receiving display content over a network, wherein one constraint is that a bandwidth required to receive all of the display content does not exceed a maximum allotted bandwidth of the network.

9. The controller of claim 1, wherein during optimization the objective function may not assign a quality level to the display content in a given window such that display content is not displayed in the given window.

10. The controller of claim 9, wherein constraints to the objective function are constructed prior to optimizing the objective function, the objective function is optimized within the constraints, and one constraint is that each window is assigned a quality level that includes displaying display content in the window.

11. The controller of claim 1, wherein the multiple qualities are differentiated by at least one of resolution, compression, color content, encryption, or further processing requirements.

12. The controller of claim 1, wherein the layout defines at least one of the number of windows, the position of each window, the shape of each window, and the area of each window.

13. The controller of claim 1, wherein the layout is a user defined layout.

14. The controller of claim 1, wherein each source is one of a camera or a computer.

15. The controller of claim 1, wherein the controller comprises a processor arranged to execute instructions stored in a memory.

16. The controller of claim 1, wherein the display system receives display content from more than one source.

17. The controller of claim 1, wherein the controller calculates the windows in the layout and an area for each window in the layout.

18. The controller of claim 1, wherein:

the controller is further arranged to receive a proposed layout; and

the processor is further arranged to use the proposed layout as a seed point for optimizing the objective function.

19. The controller of claim 1, wherein the controller is further arranged to provide instructions to the at least one source specifying the quality of display content to output.

20. A display system for optimally determining, for a layout of windows on a display wall of the display system, the quality of display content to display in each window without exceeding system resources, the display system comprising:

the controller of claim 1;

at least one display renderer, wherein;

each display renderer is responsible for displaying display content on a particular region of the display wall;

each display renderer receives display content from at least one of the at least one source; and

the display wall, wherein the layout of windows is displayed on the display wall and display content is displayed in each window at the quality level specified in the optimized layout.

21. The display system of claim 20, the at least one display renderer receiving display content over a network, wherein the system resources include at least one of a maximum network bandwidth, a maximum bandwidth of each display renderer to receive display content, a maximum decoding capacity of each display renderer to decode display content, and a maximum encoding capacity of each source to encode display content.

22. The display system of claim 21, wherein the objective function is constrained such that for each display renderer, a bandwidth required to receive all of the display content received by each display renderer over the network does not exceed the maximum bandwidth of that display renderer.

23. The display system of claim 22, further comprising at least two display renderers.

24. The display system of claim 23, wherein the objective function is constrained, for windows spanning multiple display renderers, to apply the same quality for each region of the window.

25. The display system of claim 23, wherein at least one window in the layout includes display content displayed by at least two display renderers.

26. The display system of claim 23, wherein at least two of the display renderers have at least one of a different screen resolution, number of pixels, or pixel density.

27. The display system of claim 21, wherein the objective function is constrained such that for each display renderer, a decoding capacity required to decode all of the display content received by that display renderer does not exceed the maximum decoding capacity available to that display renderer to decode display content.

28. The display system of claim 20, wherein the display system receives display content from more than one source.

29. The display system of claim 20, wherein the objection function M is defined as:

30. The display system of claim 29, wherein the controller calculates w_ij and a_ij for each window in the layout.

31. The display system of claim 29, wherein if adjacent windows w_ij and w_kl in the layout receive display content from the same source and window w_ij is on the i^th display renderer and w_kl is on the k^th display renderer, then the objective function M is constrained such that x_ij=x_kl and y_ij=y_kl.

32. The display system of claim 29, wherein the objective function M is constrained according to the following constraint:

33. The display system of claim 29, wherein the objective function M is constrained according to the following constraint:

b_xX+b_yY≦

B

wherein b_x is a bandwidth required to receive the low quality display content, b_y is a bandwidth required to receive the high quality display content, B is a maximum bandwidth of a network available to transfer display content to the display system, if the sources transfer the display content using unicast, X and Y are equal to;

34. The display system of claim 29, wherein the objective function M is constrained according to the following constraint:

35. The display system of claim 20, wherein the display wall is an IP display wall and the sources are streamed over a network to the display system.

36. The display system of claim 20, wherein each display renderer is one of a projector, a monitor, or a television.

37. A method for optimally determining, for a layout of windows on a display wall of a display system, the quality of display content to display in each window without exceeding system resources of the display system, the display system receiving display content from the at least one source, at least one of the at least one source providing display content at multiple qualities, the method comprising:

receiving the layout of the display wall, wherein the layout defines an arrangement of windows on the display wall and each window displays display content from one source;

constructing an objective function defining the quality of display content displayed in each window of the layout to be low quality display content or high quality display content that uses more of the system resources than displaying the low quality display content;

optimizing the objective function to determine an optimized layout that does not exceed system resources of the display system; and

outputting the optimized layout for displaying display content using the system resources of the display system.

38. The method of claim 37, wherein the objective function is weighted to favor the selection of high quality display content for windows having larger areas.