Method of and apparatus for partitioning, scaling and displaying video and/or graphics across several display devices

• US 6,501,441 B1
• Filed: 06/18/1999
• Issued: 12/31/2002
• Est. Priority Date: 06/18/1998
• Status: Expired due to Term

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Eyer, Mark Kenneth, Smyers, Scott D., Ludtke, Harold Aaron
• Primary Examiner(s): Luu, Matthew
• Assistant Examiner(s): Sajous, Wesner
• Application Number:
  US09/336,489
• Time in Patent Office: 1,292 Days
• Field of Search: 345/1, 345/503, 345/2, 345/504, 345/3, 345/520, 345/4, 345/543, 345/660, 345/544, 345/903, 345/547, 345/671, 345/548, 345/565, 345/699, 345/670, 345/686, 345/687, 345/723, 345/726, 345/730, 345/788, 345/764-767, 345/792, 345/798, 345/800-807, 345/815, 345/838, 345/505-506, 345/530-535, 345/902, 345/905, 345/555, 345/556, 345/561, 345/571, 345/204, 345/205, 345/213, 345/214, 345/1.1, 345/1.2, 345/1.3, 345/2.2, 345/3--, 345/5, 345/10, 345/29-30, 345/33, 348/13, 348/14, 348/15, 348/16, 348/383, 348/840, 340/716, 340/721, 358/56, 358/241, 358/87
• US Class Current: 345/1.1
• CPC Class Codes: G06F 3/1446 display composed of modules, e.g. video walls
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G09G 2340/02 Handling of images in compressed format, e.g. JPEG, MPEG
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: G09G 5/393 Arrangements for updating the contents of the bit-mapped memory
• CPC Class Codes: H04L 12/2805 Home Audio Video Interoperability [HAVI] networks
• CPC Class Codes: H04L 12/2809 indicating that an appliance service is present in a home automation network monitoring functionality H04L43/0817; discovery or management thereof, e.g. service location protocol [SLP] or web services, H04L67/51
• CPC Class Codes: H04L 12/2834 Switching of information between an external network and a home network access arrangements H04L12/2856
• CPC Class Codes: H04L 12/2838 Distribution of signals within a home automation network, e.g. involving splitting/multiplexing signals to/from different paths adaptations of television systems for transmission by electric cable for domestic distribution H04N7/106; hybrid transport H04L12/6418; home network arrangements specially adapted for distribution of digital video signals H04N7/24
• CPC Class Codes: H04L 12/40058 Isochronous transmission
• CPC Class Codes: H04L 12/40071 Packet processing; Packet format adaptation of digital video signals for transport over a specific network H04N21/2381, H04N21/4363, H04N21/4381; packet switches H04L49/00; intermediate storage or scheduling H04L49/90
• CPC Class Codes: H04L 12/40117 Interconnection of audio or video/imaging devices home automation networks H04L12/2803; bitstream network arrangements specially adapted for distribution of digital video signals H04N7/24
• CPC Class Codes: H04L 12/66 Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
• CPC Class Codes: H04L 2012/2849 Audio/video appliances
• CPC Class Codes: H04N 21/2381 Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/4363 Adapting the video stream to a specific local network, e.g. a Bluetooth® network
• CPC Class Codes: H04N 21/4381 Recovering the multiplex stream from a specific network, e.g. recovering MPEG packets from ATM cells transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 5/445 : for displaying additional information H04N5/50 takes precedence
• CPC Class Codes: H04N 9/12 : Picture reproducers H04N9/11 takes precedence

Abstract

An apparatus for partitioning, scaling and displaying video and/or graphics across several display devices includes a video source, a master device and multiple display devices arranged into a multiple display configuration. Preferably, the master device partitions the video stream into image sections and assigns each display device a corresponding image section based on its position within the multiple display configuration. Each display device then preferably receives the video stream and separates the data representing the corresponding image section. Preferably, once the data for the corresponding image section is separated, the display devices then scale their respective image sections to a full screen size image and display the scaled image simultaneously with the display of the entire frame on all display devices within the multiple display configuration. In an alternative embodiment, the master device is responsible for partitioning the video stream into image sections, scaling the image sections, encoding the scaled image sections and transmitting the scaled and encoded image sections to the appropriate display devices within the multiple display configuration. The display devices then simultaneously display the image at an appropriate time based on information in the video stream or as specified by the master device. This process is repeated for each frame within the video stream. In a further alternate embodiment, the multiple display configuration includes a combination of display devices capable of physically partitioning an image section from the video stream and display devices to which the master device partitions, scales, encodes and transmits the image for display.

Claims

1. A method of displaying images on a multiple display configuration including a plurality of display devices comprising the steps of:

a. determining capabilities and characteristics of the display devices;

b. partitioning an image into a plurality of image sections each corresponding to a display device within the multiple display configuration;

c. assigning each image section to a corresponding display device;

d. capturing each image section corresponding to each display device from an original data stream;

e. scaling each image section for each display device thereby forming scaled image sections corresponding to each display device;

f. displaying the corresponding scaled image sections at each display device at an appropriate time thereby forming a magnified image across the multiple display configuration representing the image;

g. determining a latency value for each of the display devices;

h. determining a worst case latency value for the display devices; and

i. communicating the worst case latency value to each of the display devices wherein the worst case latency value is a maximum delay value.

2. The method as claimed in claim 1 further comprising the steps of:

a. determining display dimensions of the multiple display configuration;

b. determining image dimensions of the image; and

c. calculating a size of each image section.

3. The method as claimed in claim 1 wherein the corresponding scaled image section represents a full screen of data for the display device.

4. The method as claimed in claim 1 wherein the steps of capturing and scaling are performed by each display device on the appropriate image section corresponding to the display device.

5. The method as claimed in claim 1 wherein the steps of capturing and scaling are performed by a master device on each appropriate image section.

6. The method as claimed in claim 5 further comprising the step of encoding each scaled image section into an encoded data stream representing the scaled image section.

7. The method as claimed in claim 6 wherein the step of encoding includes updating a display time value within the encoded data stream.

8. The method as claimed in claim 7 further comprising the step of transmitting the encoded data stream in a format across a high speed serial interface.

9. The method as claimed in claim 8 wherein the format is a selective one of an MPEG format and Digital Video format and the high speed serial interface is an IEEE 1394 serial bus network.

10. The method as claimed in claim 9 wherein the encoded data stream for each display device is transmitted over a separate isochronous channel.

11. The method as claimed in claim 6 further comprising the step of transmitting each encoded data stream to each appropriate display device.

12. The method as claimed in claim 11 wherein the step of transmitting includes combining data representing the scaled image section for an appropriate display device in a stream of data packets, each including an address value corresponding to a memory location within the appropriate display device.

17. The method as claimed in claim 1 further comprising the step of transmitting the original data stream to each appropriate display device over a high speed serial interface.

18. The method as claimed in claim 17 wherein the high speed serial interface is an IEEE 1394 serial bus network.

19. The method as claimed in claim 1 wherein the original data stream includes a selective one or more of video, graphics and text.

13. A method of displaying images on a multiple display configuration including a plurality of display devices comprising the steps of:

a. determining capabilities and characteristics of the display devices;

b. partitioning an image into a plurality of image sections each corresponding to a display device within the multiple display configuration;

c. assigning each image section to a corresponding display device;

d. capturing each image section corresponding to each display device from an original data stream;

e. scaling each image section for each display device thereby forming scaled image sections corresponding to each display device;

f. displaying the corresponding scaled image sections at each display device at an appropriate time thereby forming a magnified image across the multiple display configuration representing the image;

g. encoding each scaled image section into an encoded data stream representing the scaled image section;

h. transmitting each encoded data stream to each appropriate display device, wherein the step of transmitting each encoded data stream includes combining data representing the scaled image section for an appropriate display device in a stream of data packets, each including an address value corresponding to a memory location within the appropriate display device; and

i. transmitting a trigger packet on occurrence of a trigger event, the trigger packet including a trigger address value corresponding to a trigger memory location within the appropriate display device;

wherein the steps of capturing and scaling are performed by a master device on each appropriate image section.

14. The method as claimed in claim 13 wherein the trigger packet includes a trigger bit, which when written into the trigger memory location, signals that storage of a current scaled image section for display by the appropriate display device is complete.

15. The method as claimed in claim 14 wherein the data packets are isochronous packets.

16. The method as claimed in claim 13 wherein the trigger packet further includes a presentation time value specifying a display time of the scaled image section.

20. A method of transmitting images to display devices within a multiple display configuration comprising the steps of:

a. logically partitioning an image into a plurality of image sections each corresponding to a display device within the multiple display configuration;

b. instructing each display device to physically partition and capture an appropriate image section corresponding to the display device from an original data stream;

c. transmitting the original data stream to each display device;

d. determining a latency value for each of the display devices;

e. determining a worst case latency value for the display devices; and

f. communicating the worst case latency value to each of the display devices.

21. The method as claimed in claim 20 further comprising the steps of:

a. determining display dimensions of the multiple display configuration;

b. determining image dimensions of the image; and

c. calculating a size of each image section.

22. The method as claimed in claim 20 further comprising the step of instructing each display device to display its appropriate image section at an appropriate time.

23. The method as claimed in claim 22 wherein the appropriate time is based on the worst case latency.

24. The method as claimed in claim 20 wherein the step of transmitting is completed over a high speed serial interface.

25. The method as claimed in claim 24 wherein the high speed serial interface is an IEEE 1394 serial bus network.

26. The method as claimed in claim 20 wherein the original data stream includes a selective one or more of video, graphics and text.

27. A method of displaying images on a multiple display configuration including a plurality of display devices and a master device comprising the steps of:

a. partitioning an image into a plurality of image sections each corresponding to a display device within the multiple display configuration;

b. assigning each image section to a corresponding display device;

c. capturing each image section corresponding to each display device from an original data stream at the master device;

d. scaling each image section for each display device thereby forming scaled image sections corresponding to each display device at the master device;

e. transmitting each scaled image section to each appropriate display device, wherein the step of transmitting each scaled image section includes combining data representing the scaled image section for an appropriate display device in a stream of data packets, each including an address value corresponding to a memory location within the appropriate display device;

f. displaying the corresponding scaled image sections at each display device at an appropriate time thereby forming a magnified image across the multiple display configuration representing the image; and

g. transmitting a trigger packet on occurrence of a trigger event, the trigger packet including a trigger address value corresponding to a trigger memory location within the appropriate display device.

28. The method as claimed in claim 27 further comprising the step of encoding each scaled image section into an encoded data stream representing the scaled image section, before the step of transmitting is completed.

29. The method as claimed in claim 28 wherein the step of encoding includes updating a display time value within the encoded data stream.

30. The method as claimed in claim 29 wherein the step of transmitting includes transmitting the encoded data stream in a selective one of an MPEG format and DV format across an IEEE 1394 serial bus network.

31. The method as claimed in claim 27 wherein the trigger packet includes a trigger bit, which when written into the trigger memory location signals that storage of a current scaled image section for display by the appropriate display device is complete.

32. The method as claimed in claim 31 wherein the data packets are isochronous packets.

33. The method as claimed in claim 27 wherein the trigger packet further includes a presentation time value specifying a display time of the scaled image section.

34. The method as claimed in claim 27 wherein the step of transmitting is performed over a high speed serial interface.

35. The method as claimed in claim 34 wherein the high speed serial interface is an IEEE 1394 serial bus network.

36. The method as claimed in claim 27 further comprising the steps of:

a. determining display dimensions of the multiple display configuration;

b. determining image dimensions of the image; and

c. calculating a size of each image section.

37. The method as claimed in claim 27 wherein the corresponding scaled image section represents a full screen of data for the display device.

38. The method as claimed in claim 27 wherein the original data stream includes a selective one or more of video, graphics and text.

39. A multiple display configuration system comprising:

a. a plurality of display devices each comprising;

i. a display communications circuit configured for receiving and transmitting data;

ii. a capturing circuit coupled to the display communications circuit for capturing an image section assigned to the display for each image within an original data stream;

iii. a scaling circuit coupled to the capturing circuit for scaling each captured image section thereby forming a scaled image section; and

iv. a display circuit coupled to the scaling circuit for displaying the scaled image section; and

b. a master device coupled to the plurality of display devices comprising;

i. a master communications circuit configured for receiving and transmitting data; and

ii. a control circuit coupled to the master communications circuit for partitioning an image into a plurality of image sections each corresponding to one of the display devices and assigning each image section to a corresponding display device.

40. The multiple display configuration as claimed in claim 39 further comprising a video source coupled to the display devices and to the master device for providing a video stream as the original data stream to be displayed on the display devices.

41. The multiple display configuration as claimed in claim 39 wherein the display circuit displays the scaled image section at an appropriate time specified by the master device.

42. The multiple display configuration as claimed in claim 39 wherein the scaled image section represents a full screen of data for the display device.

43. The multiple display configuration as claimed in claim 39 wherein the display devices and the master device are all coupled together by a high speed serial interface.

44. The multiple display configuration as claimed in claim 43 wherein the high speed serial interface is an IEEE 1394 serial bus network.

45. The multiple display configuration as claimed in claim 39 wherein the original data stream includes a selective one or more of video, graphics and text.