Rendering graphics/image data using dynamically generated video streams

• US 7,694,324 B2
• Filed: 09/30/2004
• Issued: 04/06/2010
• Est. Priority Date: 08/13/2004
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Evans, Stephen S., McDowell, Brian C.
• Primary Examiner(s): Hirl; Joseph P
• Assistant Examiner(s): Taylor; Joshua
• Application Number:
  US10/955,358
• Publication Number:
  US 20060037054A1
• Time in Patent Office: 2,014 Days
• Field of Search: 348/552, 725/82, 725/80
• US Class Current: 725/133
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/4363 Adapting the video stream to a specific local network, e.g. a Bluetooth® network
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

The present invention provides for rendering graphics/image data at an electronic device that typically has limited resources. A video data steam is received at an electronic device, the video data stream including either user interface information or slideshow presentation data coded in a video codec format. A video processor of the electronic device is used to decode at least a portion of the video data stream into video frame data. A portion of the user interface or slideshow presentation is then displayed as video from data on a display of the electronic device.

Claims

1. In an entertainment network comprising a host computer system with a video codec and an electronic device, the host computer system and the electronic device connected by a video stream channel and a back channel, a method of rendering a user interface for an application running at the host computer on a display of the electronic device, the method comprising acts of:

receiving at the electronic device, through the video stream channel, a video data stream, which includes displayable image user interface information coded in a video codec format, the video data stream having been generated by;

the host computer opening an application session for the electronic device at the host computer, the application session for delivering user interface information to the electronic device, the user interface information including displayable image data of a user interface for the application;

the host computer establishing an internal remote desktop protocol connection between the application session and an executable abstraction process within the host computer, the internal remote desktop protocol connection internal to the host computer, the internal remote desktop protocol connection configured for use with existing application program interfaces for sending user interface information from the application session to remote electronic devices, the application session connected to a first end of the internal remote desktop protocol connection and the executable abstraction process connected to a second end of the internal remote desktop protocol connection, the executable abstraction process abstracting the application session from the video codec such that the application session is relieved from having to interact directly with the video codec, the executable abstraction process configured to receive user interface information sent from the application at the second end of the internal remote desktop protocol connection and feed the user interface information to the video codec for conversion into streaming video;

the application session generating user interface information, including displayable image data, the displayable image data configured for rendering in a user interface window at the display of the electronic device;

the application session using remote desktop protocol to transfer a portion of the user interface information, including the displayable image data, through the first end of the internal remote desktop protocol connection;

the executable abstraction process using remote desktop protocol to receive the portion of the user interface information through the second end of the internal remote desktop protocol connection;

the executable abstraction process feeding the portion of user interface information, including the displayable image data, to the video codec for conversion into the video stream; and

the video codec converting the portion of user interface information, including the displayable image data, into the video data stream, the video data stream including video data that represents the displayable image data and which is capable of being processed at the electronic device by a video processor to remotely render the user interface for the application at the electronic device, the video data stream further comprising control information, including object information and command data, the object information representing objects of a user interface, the command data indicating how an instance of a represented object is to be manipulated on the electronic device, the command data including one or more of a sizing, a location, the graphical appearance and a behavior of the instance;

using the video processor for decoding at least a portion of the video data stream into video frame data;

displaying at least a portion the user interface, including the displayable image data, as video frame data in the user interface window at the display of the electronic device;

transmitting user input through the back channel that indicates a change in one or more portions of the user interface; and

receiving, through the video stream channel, an altered video data stream in accordance with the user input received.

2. The method of claim 1, wherein the user input is received at the electronic device from a remote device by one or more of inferred light waves, micro waves or radio waves.

3. The method of claim 1, wherein the object information is configured to be used in rendering portions of other user interfaces on the electronic device.

4. The method of claim 1, wherein the control information supersedes the video frame data such that when the control information is received after a portion of the video data steam corresponding to the video frame data as been received, but before the video frame data is fully processed, the control data is processed before the processing of the video frame data.

5. The method of claim 1, wherein the video data stream includes an audio data stream portion.

6. The method of claim 1, wherein the video data stream format is one of MPEG, MPEG 2, DMV or WMV.

7. The method of claim 1, wherein the electronic device is chosen from a set top box, DVD player, network television set, or digital media adapter.

8. The method of claim 1, wherein UPnP AV is used to detect the presence of the host computer, the electronic device, or both.

9. In an entertainment network comprising a host computer system with a video codec and an electronic device, the host computer system and the electronic device connected by a video stream channel and a back channel, a method of rendering slideshow presentation information on a display of the electronic device, the method comprising acts of:

receiving at the electronic device, through the video stream channel, a video data stream, which includes slideshow presentation data coded in a video codec format, the slideshow presentation data including still image data accessible by a host computer, the video data stream having been generated by;

the host computer opening an application session for the electronic device at the host computer, the application session for delivering the slideshow presentation data to the electronic device;

the host computer establishing an internal remote desktop protocol connection between the application session and executable abstraction process within the host computer, the internal remote desktop protocol connection internal to the host computer, the internal remote desktop protocol connection configured for use with existing application program interfaces for sending slideshow presentation information from the application session to remote electronic devices, the application session connected to a first end of the internal remote desktop protocol connection and the executable abstraction process connected to a second end of the internal remote desktop protocol connection the executable abstraction process abstracting the application session from the video codec such that the application session is relieved form having to interact directly with the video codec, the executable abstraction process configured to receive slideshow presentation information sent from the application at the second end of the internal remote desktop protocol connection and feed the slideshow presentation information to the video codec for conversion into streaming video;

the application session generating slideshow presentation data, including still image data that represents a series of bitmaps, the still image data configured for rendering at the display of the electronic device;

the application session using the remote desktop protocol to transfer a portion of the slideshow presentation data, including the still image data, through the first end of the internal remote desktop protocol connection;

the executable abstraction process feeding the portion of the slideshow presentation data, including the still image data, to the video codec for conversion into the video stream; and

the video codec converting the slideshow presentation data, including still image data, into the video data stream, the video data stream including video data that represents the slideshow presentation and which is capable of being processed at the electronic device by a video processor to remotely render the still image data at the electronic device, the video data stream further comprising control information, the control information including command data, which indicates how the image data is to be visually represented at one or more moments in time within the slideshow presentation, the command data including one or more an animate, pan, zoom, cross-fade or playing music;

using the video processor for decoding at least a portion of the video data stream into video frame data;

displaying at least a portion of the slideshow presentation data, including the still image data, as video frame data on the display of the electronic device;

transmitting user input through the back channel that indicates a change in the ordering or display of the series of bitmaps, the change being one of a pause, skip, stop, fast forward or reverse; and

receiving, through the video stream channel, an altered video data stream in accordance with the user input received.

10. The method of claim 9, wherein the slideshow presentation data further includes transitional information that indicates how the image data is to be visually represented at one or more moments in time within the slideshow presentation, the transitional information within the video data stream being processed by a central processing unit of the electronic device.

11. The method of claim 10, wherein the transitional information is one or more an animate, pan, zoom, cross-fade or playing of music.

12. The method of claim 9, wherein the command data is received by a remote device and processed by a central processing unit of the electronic device.

13. The method of claim 9, wherein the user input is received at the electronic device from a remote device by one or more of inferred light waves, micro waves or radio waves.

14. The method of claim 9, wherein the video data stream format is one of MPEG, MPEG 2, DMV or WMV.

15. The method of claim 9, wherein the electronic device is chosen from a set top box, DVD player, network television set, or digital media adapter.

16. The method of claim 9, wherein UPnP AV is used to detect the presence of the host computer, the electronic device, or both.

17. In an entertainment network comprising a host computer system with a video codec and an electronic device, the host computer system and the electronic device connected by a video stream channel and a back channel, a computer program product for implementing a method of rendering a user interface for an application running at the host computer on a display of the electronic device, the computer program product comprising one or more computer readable media having stored thereon computer executable instructions that, when executed by a processor, cause the electronic device to perform the following:

receive at the electronic device, through the video stream channel, a video data stream, which includes displayable image user interface information coded in a video codec format, the video data stream having been generated by;

the host computer opening an application session for the electronic device at the host computer, the application session for delivering user interface information to the electronic device, the user interface information, including displayable image data, of a user interface for the application;

the host computer establishing an internal remote desktop protocol connection between the application session and an executable abstraction process within the host computer, the internal remote desktop protocol connection internal to the host computer, the internal remote desktop protocol connection configured for use with existing application program interfaces for sending user interface information from the application session to remote electronic devices, the application session connected to a first end of the internal remote desktop protocol connection and the executable abstraction process connected to a second end of the internal remote desktop protocol connection the executable abstraction process abstracting the application session from the video codec such that the application session is relieved from having to interact directly with the video codec, the executable abstraction process configured to receive user interface information sent from the application at the second end of the internal remote desktop protocol connection and feed the user interface information to the video codec for conversion into streaming video;

the application session generating user interface information, including displayable image data, the displayable image data configured for rendering in a user interface window at the display of the electronic device;

the application session using remote desktop protocol to transfer a portion of the user interface information, including the displayable image data, through the first end of the internal remote desktop protocol connection;

the executable abstraction process using remote desktop protocol to receive the portion of the user interface information through the second end of the internal remote desktop protocol connection;

the executable abstraction process feeding the portion of user interface information, including the displayable image data, to the video codec for conversion into a video stream; and

the video codec converting the portion of user interface information, including the displayable image data, into the video data stream, the video data stream including video data that represents the displayable image data and which is capable of being processed at the electronic device by a video processor to remotely render the user interface for the application at the electronic device, the video data stream further comprising control information, including object information and command data, the object information representing objects of a user interface, the command data indicating how an instance of a represented object is to be manipulated on the electronic device, the command data including one or more of a sizing, a location, the graphical appearance and a behavior of the instance;

use the video processor for decoding at least a portion of the video data stream into video frame data;

display at least a portion the user interface, including the displayable image data, as video frame data in the user interface window at the display of the electronic device;

transmit user input through the back channel that indicates a change in one or more portions of the user interface; and

receive, through the back channel, an altered video data stream in accordance with the user input received.

18. The computer program product of claim 17, wherein the object information is configured to be used in rendering portions of other user interfaces on the electronic device.

19. The computer program product of claim 17, wherein the control information supersedes the video frame data such that when the control information is received after a portion of the video data steam corresponding to the video frame data as been received, but before the video frame data is fully processed, the control data is processed before the processing of the video frame data.

20. In an entertainment network comprising a host computer system with a video codec and an electronic device, the host computer system and the electronic device connected by a video stream channel, a computer program product for implementing a method of rendering slideshow presentation information on a display of the electronic device, the computer program product comprising one or more computer readable media having stored thereon computer executable instructions that, when executed by a processor, cause the electronic device to perform the following:

receive at the electronic device, through the video stream channel, a video data stream, which includes slideshow presentation data coded in a video codec format, the slideshow presentation data including still image data accessible by a host computer, the video data stream having been generated by;

the host computer opening an application session for the electronic device at the host computer, the application session for delivering the slideshow presentation data to the electronic device;

the host computer establishing an internal remote desktop protocol connection between the application session and executable abstraction process within the host computer, the internal remote desktop protocol connection internal to the host computer, the internal remote desktop protocol connection configured for use with existing application program interfaces for sending slideshow presentation information from the application session to remote electronic devices, the application session connected to a first end of the internal remote desktop protocol connection and the executable abstraction process connected to a second end of the internal remote desktop protocol connection the executable abstraction process abstracting the application session from the video codec such that the application session is relieved from having to interact directly with the video codec, the executable abstraction process configured to receive slideshow presentation information sent from the application at the second end of the internal remote desktop protocol connection and feed the slideshow presentation information to the video codec for conversion into streaming video;

the application session generating slideshow presentation data, including still image data that represents a series of bitmaps, the still image data configured for rendering at the display of the electronic device;

the application session using the remote desktop protocol to transfer a portion of the slideshow presentation data, including the still image data, through the first end of the internal remote desktop protocol connection;

the executable abstraction process feeding the portion of the slideshow presentation data, including the still image data, to the video codec for conversion into the video stream; and

the video codec converting the slideshow presentation data, including still image data, into the video data stream, the video data stream including video data that represents the slideshow presentation and which is capable of being processed at the electronic device by a video processor to remotely render the still image data at the electronic device, the video data stream further comprising control information, the control information including command data, which indicates how the image data is to be visually represented at one or more moments in time within the slideshow presentation, the command data including one or more an animate, pan, zoom, cross-fade or playing music;

use the video processor for decoding at least a portion of the video data stream into video frame data;

display at least a portion of the slideshow presentation data, including the still image data, as video frame data on the display of the electronic device;

transmit user input through the back channel that indicates a change in the ordering or display of the series of bitmaps, the change being one of a pause, skip, stop, fast forward or reverse; and

receive, through the video stream channel, and altered video data stream in accordance with the user input received.

21. The computer program product of claim 20, wherein the slideshow presentation data further includes transitional information that indicates how the image data is to be visually represented at one or more moments in time within the slideshow presentation, the transitional information within the video data stream being processed by a central processing unit of the electronic device.

22. The computer program product of claim 21, wherein the transitional information is one or more an animate, pan, zoom, cross-fade or playing of music.

23. The computer program product of claim 20, wherein the command data is received by a remote device and processed by a central processing unit of the electronic device.