Dynamically generating video streams for user interfaces based on device capabilities

• US 8,166,507 B2
• Filed: 09/30/2004
• Issued: 04/24/2012
• Est. Priority Date: 08/13/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: McDowell, Brian C., Evans, Stephen S.
• Primary Examiner(s): Tran, Hai V
• Assistant Examiner(s): THOMAS, JASON M
• Application Number:
  US10/955,211
• Publication Number:
  US 20060037053A1
• Time in Patent Office: 2,763 Days
• Field of Search: 348/552, 725/82, 725/37, 725/38, 725/78, 725/80, 725/85, 715716-719, 715/864, 715/866, 715/744, 709/203, 709217-219, 345/156, 345/157, 370/473
• US Class Current: 725/82
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/4363 Adapting the video stream to a specific local network, e.g. a Bluetooth® network
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

The present invention provides for transferring user interface information from a host computing device to an electronic device that typically has limited resources. A request to display a user interface on an electronic device is received. Further, information about the electronic device is received for determining the capabilities of the electronic device. Based on the determination, a video codec is utilized for dynamically generating a video data stream that includes user interface information corresponding to at least a portion of the user interface. The video data stream is then transmitted to the electronic device for rendering the user interface on a display of the electronic device.

Claims

1. In an entertainment network comprising a host computer system with a video codec and at least one electronic device, the host computer system and the electronic device connected by a video stream channel and a back channel, a method of transferring user interface information from the host computer to the electronic device, the method comprising acts of:

the host computer receiving a request from the electronic device, the request requesting to interact with an application running at the host computer system;

receiving capability information about the electronic device;

determining from the capability information that the electronic device can process a video data stream;

in response to the request, opening an application session for the electronic device at the host computer system, the application session for delivering user interface information for displaying the user interface of the application to the electronic device, the user interface information comprising graphics commands for rendering the user interface;

establishing a remote desktop protocol (RDP) connection between the application session and an executable abstraction process within the host computer, rather than with an RDP process running on the electronic device;

the application session generating user interface information, including graphics commands;

the application session sending the user interface information over the RDP connection to the executable abstraction process;

the executable abstraction process rendering the user interface information into a user interface and then encoding the user interface into a motion video stream;

transmitting the video stream over the video stream channel to the electronic device such that the electronic device can use the video processor to display the user interface on a display of the electronic device rather than requiring components for rendering the user interface from the graphics commands of the user interface information.

2. The method of claim 1, wherein the capabilities determined are one or more of central processing unit speed, available bandwidth, sustained bandwidth, resolution of a display, color depth, amount of memory available, memory access speed and graphics capabilities.

3. The method of claim 2, wherein the graphics capabilities are one or more of alpha blending and support for 3D image rendering.

4. The method of claim 1, wherein the user input is received directly from a remote device by one or more of inferred light, microwaves or radio waves.

5. The method of claim 1, wherein the object information is configured to be used in rendering portions of other user interfaces on the electronic device.

6. The method of claim 1, wherein prior to dynamically generating the video stream, the user interface information is formatted by one or more of a re-sampling or resizing based on the display capabilities of the electronic device.

7. The method of claim 1, wherein the video data stream includes an audio data stream portion.

8. The method of claim 1, wherein the video stream format is one of MPEG, MPEG 2, DMV or WMV.

9. The method of claim 1, wherein the electronic device is chosen from a set top box, DVD player, network television set, or digital media adapter.

10. The method of claim 1, wherein UPnP AV is used to detect the presence of the host computer, the electronic device, or both.

11. In an entertainment network comprising a host computer system including a video codec and at least one electronic device, the host computer system and the electronic device connected by a video stream channel and a back channel, a computer readable storage device storing computer executable instructions which when executed perform a method of transferring user interface information from the host computer to the electronic device for transferring user interface information from the host computer system to the electronic device, the method comprising:

the host computer receiving a request from the electronic device, the request requesting to interact with an application running at the host computer system;

receiving capability information about the electronic device;

determining from the capability information that the electronic device can process a video data stream;

in response to the request, opening an application session for the electronic device at the host computer system, the application session for delivering user interface information for displaying the user interface of the application to the electronic device, the user interface information comprising graphics commands for rendering the user interface;

establishing a remote desktop protocol (RDP) connection between the application session and an executable abstraction process within the host computer, rather than with an RDP process running on the electronic device;

the application session generating user interface information, including graphics device interface (GDI) commands;

the application session sending the user interface information, including the GDI commands, over the RDP connection to the executable abstraction process;

the executable abstraction process rendering the user interface information, including the GDI commands, into a user interface and then encoding the user interface into a motion video stream; and

transmitting the video stream over the video stream channel to the electronic device such that the electronic device can use the video processor to display the user interface on a display of the electronic device rather than requiring components for rendering the user interface from the graphics commands of the user interface information.

12. The computer storage medium of claim 11, wherein the capabilities determined are one or more of central processing unit speed, available bandwidth, sustained bandwidth, resolution of a display, color depth, amount of memory available, memory access speed and graphics capabilities.

13. The computer storage medium of claim 12, wherein the graphics capabilities are one or more of alpha blending and support for 3D image rendering.

14. The computer storage medium of claim 11, wherein the user input is received directly from a remote device by one or more of inferred light, microwaves or radio waves.

15. The computer storage medium of claim 11, wherein the object information is configured to be used in rendering portions of other user interfaces on the electronic device.

16. The computer storage medium of claim 12, wherein prior to dynamically generating the video stream, the user interface information is formatted by one or more of a re-sampling or resizing based on the display capabilities of the electronic device.

17. The computer storage medium of claim 12, wherein the video data stream includes an audio data stream portion.

18. The computer storage medium of claim 12, wherein the video stream format is one of MPEG, MPEG 2, DMV or WMV.

19. The computer storage medium of claim 12, wherein the electronic device is chosen from a set top box, DVD player, network television set, or digital media adapter.

20. The computer storage medium of claim 12, wherein UPnP AV is used to detect the presence of the host computer, the electronic device, or both.

21. In an entertainment network comprising a host computer system with a video codec and at least one electronic device, the host computer system and the electronic device connected by a video stream channel and a back channel, a method of transferring user interface information from the host computer to the electronic device, the method comprising acts of:

the host computer receiving a request from the electronic device, the request requesting to interact with an application running at the host computer system;

receiving capability information about the electronic device;

determining from the capability information that the electronic device can process a video data stream;

in response to the request, opening an application session for the electronic device at the host computer system, the application session for delivering user interface information for displaying the user interface of the application to the electronic device, the user interface information comprising graphics commands for rendering the user interface;

establishing a local procedure call connection between the application session and an executable abstraction process within the host computer, rather than with an RDP process running on the electronic device;

the application session generating user interface information, including graphics commands;

the application session sending the user interface information over the local procedure call connection to the executable abstraction process;

the executable abstraction process rendering the user interface information into a user interface and then encoding the user interface into a motion video stream;

transmitting the video stream over the video stream channel to the electronic device such that the electronic device can use the video processor to display the user interface on a display of the electronic device rather than requiring components for rendering the user interface from the graphics commands of the user interface information.

22. The method of claim 21, wherein the capabilities determined are one or more of central processing unit speed, available bandwidth, sustained bandwidth, resolution of a display, color depth, amount of memory available, memory access speed and graphics capabilities.

23. The method of claim 22, wherein the graphics capabilities are one or more of alpha blending and support for 3D image rendering.

24. The method of claim 21, wherein the user input is received directly from a remote device by one or more of inferred light, microwaves or radio waves.

25. The method of claim 21, wherein the object information is configured to be used in rendering portions of other user interfaces on the electronic device.

26. The method of claim 21, wherein prior to dynamically generating the video stream, the user interface information is formatted by one or more of a re-sampling or resizing based on the display capabilities of the electronic device.

27. The method of claim 21, wherein the video data stream includes an audio data stream portion.

28. The method of claim 21, wherein the video stream format is one of MPEG, MPEG 2, DMV or WMV.

29. The method of claim 21, wherein the electronic device is chosen from a set top box, DVD player, network television set, or digital media adapter.

30. The method of claim 21, wherein UPnP AV is used to detect the presence of the host computer, the electronic device, or both.

31. In an entertainment network comprising a host computer system including a video codec and at least one electronic device, the host computer system and the electronic device connected by a video stream channel and a back channel, a computer readable storage device storing computer executable instructions which when executed perform a method of transferring user interface information from the host computer to the electronic device for transferring user interface information from the host computer system to the electronic device, the method comprising:

the host computer receiving a request from the electronic device, the request requesting to interact with an application running at the host computer system;

receiving capability information about the electronic device;

determining from the capability information that the electronic device can process a video data stream;

in response to the request, opening an application session for the electronic device at the host computer system, the application session for delivering user interface information for displaying the user interface of the application to the electronic device, the user interface information comprising graphics commands for rendering the user interface;

establishing a local procedure call connection between the application session and an executable abstraction process within the host computer, rather than with an RDP process running on the electronic device;

the application session generating user interface information, including graphics commands;

the application session sending the user interface information over the local procedure call connection to the executable abstraction process;

the executable abstraction process rendering the user interface information into a user interface and then encoding the user interface into a motion video stream;

transmitting the video stream over the video stream channel to the electronic device such that the electronic device can use the video processor to display the user interface on a display of the electronic device rather than requiring components for rendering the user interface from the graphics commands of the user interface information.

32. The computer storage medium of claim 31, wherein the capabilities determined are one or more of central processing unit speed, available bandwidth, sustained bandwidth, resolution of a display, color depth, amount of memory available, memory access speed and graphics capabilities.

33. The computer storage medium of claim 32, wherein the graphics capabilities are one or more of alpha blending and support for 3D image rendering.

34. The computer storage medium of claim 31, wherein the user input is received directly from a remote device by one or more of inferred light, microwaves or radio waves.

35. The computer storage medium of claim 31, wherein the object information is configured to be used in rendering portions of other user interfaces on the electronic device.

36. The computer storage medium of claim 32, wherein prior to dynamically generating the video stream, the user interface information is formatted by one or more of a re-sampling or resizing based on the display capabilities of the electronic device.

37. The computer storage medium of claim 32, wherein the video data stream includes an audio data stream portion.

38. The computer storage medium of claim 32, wherein the video stream format is one of MPEG, MPEG 2, DMV or WMV.

39. The computer storage medium of claim 32, wherein the electronic device is chosen from a set top box, DVD player, network television set, or digital media adapter.

40. The computer storage medium of claim 32, wherein UPnP AV is used to detect the presence of the host computer, the electronic device, or both.