Manipulating a compressed video stream

• US 7,162,531 B2
• Filed: 01/25/2001
• Issued: 01/09/2007
• Est. Priority Date: 07/27/1998
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Fenster, Maier, Paz, Ofir, Keren, Avishai
• Primary Examiner(s): Dharia; Rupal
• Assistant Examiner(s): Shingles; Kristie D.
• Application Number:
  US09/770,644
• Publication Number:
  US 20040205213A1
• Time in Patent Office: 2,175 Days
• Field of Search: 709/231, 709/208, 709/219, 709/217, 709/247, 375/240, 348/588, 705/1, 380/269, 382/232, 710/68, 345/1.2
• US Class Current: 709/231
• CPC Class Codes: A63F 2300/409 Data transfer via television network
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/5506 using advertisements
• CPC Class Codes: A63F 2300/577 for watching a game played by other players
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/134 characterised by the element, parameter or criterion affecting or controlling the adaptive coding
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234336 by media transcoding, e.g. video is transformed into a slideshow of still pictures or audio is converted into text
• CPC Class Codes: H04N 21/234354 by altering signal-to-noise ratio parameters, e.g. requantization
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/234381 by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• CPC Class Codes: H04N 21/23655 Statistical multiplexing, e.g. by controlling the encoder to alter its bitrate to optimize the bandwidth utilization
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/2407 Monitoring of transmitted content, e.g. distribution time, number of downloads arrangements for monitoring programmes for broadcast or for distribution combined with broadcast H04H20/14
• CPC Class Codes: H04N 21/241 Operating system [OS] processes, e.g. server setup
• CPC Class Codes: H04N 21/2543 Billing , e.g. for subscription services payment schemes, architectures or protocols G06Q20/00; e-commerce G06Q30/00; arrangements for billing for the use of broadcast information or broadcast-related information H04H60/21
• CPC Class Codes: H04N 21/262 : Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists scheduling strategies for dispatcher in multiprogramming arrangements G06F9/4881; arrangements for scheduling broadcast services or broadcast-related services H04H60/06; flow control in packet networks H04L47/10; establishing a time schedule or organising the servicing of application requests H04L67/62
• CPC Class Codes: H04N 21/2668 : Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/274 : Storing end-user multimedia data in response to end-user request , e.g. network recorder
• CPC Class Codes: H04N 21/478 : Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/488 : Data services, e.g. news ticker systems specially adapted for using meteorological information in broadcast systems H04H60/71
• CPC Class Codes: H04N 21/6168 : involving cable transmission, e.g. using a cable modem
• CPC Class Codes: H04N 21/6187 : involving transmission via a telephone network, e.g. POTS
• CPC Class Codes: H04N 21/6405 : Multicasting data broadcast and multicast in packet switching networks H04L12/18
• CPC Class Codes: H04N 21/6408 : Unicasting
• CPC Class Codes: H04N 21/6547 : comprising parameters, e.g. for client setup
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8173 : End-user applications, e.g. Web browser, game
• CPC Class Codes: H04N 21/8453 : by locking or enabling a set of features, e.g. optional functionalities in an executable program
• CPC Class Codes: H04N 21/8543 : using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80
• CPC Class Codes: H04N 21/8586 : by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02
• CPC Class Codes: H04N 7/165 : Centralised control of user terminal subsequent to an upstream request signal H04N7/17345; Registering at central by two-way working H04N7/17309

Abstract

A computer may be remotely accessed. At a first location, display commands are generated. The display commands are converted into a compressed video data stream. Each display element (50) is checked if it is encoded (52). If an object is encoded, it is transcribed into MPEG (54). The image is adjusted for display (56) and compression (58). Additional steps of motion determination (60), change detection (62), compression depth and frame determination (66, 68) are executed. Then the data is transmitted to a second location. This display commands are decompressed and displayed as an image at the second location.

Claims

1. In a computer system with a server that provides remote client access to a software program that creates a layout of elements from a display design based on limitation of the system, wherein the display will be transmitted to one or more clients as compressed video stream, one or more computer readable media having stored thereon:

the software program comprising;

a restriction module that when at a server receives one or more restrictions defining one or more limitations imposed by a compression method to be used in generating a display representing a user interface corresponding to a program running at the server and that is displayed at a remote client as a compressed video stream;

a design module that when running at the server lays out one or more display elements for said user interface responsive to said received one or more limitations in order to meet said one or more limitations when creating a compressed video stream of said display to be sent to said remote client, wherein the design module modifies the layout of the one or more display elements relative to a layout without the one or more limitations in order to make compression for the one or more display elements more efficient, and wherein the modifications includes at least one or more of moving an element, replacing an element, changing a property on an element, or removing an element;

a compression module for compressing the modified layout of the one or more display elements into the compressed video stream and such that the compressed video stream includes a modified layout that was modified by, at least one of, moving an element, replacing an element, changing a property on an element, or removing an element and responsive to the one or more limitations; and

a transmission module for sending the compressed video stream to one or more clients for viewing and interacting with the user interface of the program.

2. A software according to claim 1, wherein said one or more restrictions include a block size definition.

3. A software according to claim 1, wherein said one or more restrictions include a bandwidth restriction, and wherein laying out of said one or more display elements comprises selecting one or more display elements to match said bandwidth limitations.

4. A software according to claim 1, wherein said one or more computer readable media have stored thereon an automated WWW page generator for a WWW server.

5. A software according to claim 4, comprising a communication module for receiving said one or more restrictions from a server associated with said compression.

6. A software according to claim 1, wherein the modification includes at least replacing an element, and wherein the replacement is based on the type of the element.

7. A software according to claim 1, wherein the modification includes at least moving an element, and wherein the movement is based on one or more portions of the element straddling a boundary of a block.

8. A software according to claim 1, wherein the program running at the server displays a WWW page.

9. A software according to claim 1, wherein the modification includes one or more of:

manipulating colors so that color components can be compressed to a greater degree, using fonts with fewer high-frequency components, reducing flashing rates of colors and cursors, synchronizing flashing rates of colors and cursors, reducing animation rates of animated objects, low-pass filtering to reduce display resolution, reducing scrolling resolution, or changing an element to a similar element to a similar element.