Multimedia stream compression

• US 8,259,788 B2
• Filed: 01/25/2001
• Issued: 09/04/2012
• Est. Priority Date: 07/27/1998
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Keren, Avishai, Feder, Meir, Paz, Ofir
• Primary Examiner(s): DIEP, NHON THANH
• Application Number:
  US09/770,767
• Publication Number:
  US 20010026591A1
• Time in Patent Office: 4,240 Days
• Field of Search: 37524001-24003, 37524012-24016, 37524023-24024, 375/240.26, 725 86- 87, 725/91, 725/95, 725/100, 725/105, 725/110, 725/114, 348/384.1, 348/390.1, 348/420.1
• US Class Current: 375/240.01
• CPC Class Codes: A63F 2300/409 Data transfer via television network
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/5506 using advertisements
• CPC Class Codes: A63F 2300/577 for watching a game played by other players
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/134 characterised by the element, parameter or criterion affecting or controlling the adaptive coding
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234336 by media transcoding, e.g. video is transformed into a slideshow of still pictures or audio is converted into text
• CPC Class Codes: H04N 21/234354 by altering signal-to-noise ratio parameters, e.g. requantization
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/234381 by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• CPC Class Codes: H04N 21/23655 Statistical multiplexing, e.g. by controlling the encoder to alter its bitrate to optimize the bandwidth utilization
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/2407 Monitoring of transmitted content, e.g. distribution time, number of downloads arrangements for monitoring programmes for broadcast or for distribution combined with broadcast H04H20/14
• CPC Class Codes: H04N 21/241 Operating system [OS] processes, e.g. server setup
• CPC Class Codes: H04N 21/2543 Billing , e.g. for subscription services payment schemes, architectures or protocols G06Q20/00; e-commerce G06Q30/00; arrangements for billing for the use of broadcast information or broadcast-related information H04H60/21
• CPC Class Codes: H04N 21/262 : Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists scheduling strategies for dispatcher in multiprogramming arrangements G06F9/4881; arrangements for scheduling broadcast services or broadcast-related services H04H60/06; flow control in packet networks H04L47/10; establishing a time schedule or organising the servicing of application requests H04L67/62
• CPC Class Codes: H04N 21/2668 : Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/274 : Storing end-user multimedia data in response to end-user request , e.g. network recorder
• CPC Class Codes: H04N 21/478 : Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/488 : Data services, e.g. news ticker systems specially adapted for using meteorological information in broadcast systems H04H60/71
• CPC Class Codes: H04N 21/6168 : involving cable transmission, e.g. using a cable modem
• CPC Class Codes: H04N 21/6187 : involving transmission via a telephone network, e.g. POTS
• CPC Class Codes: H04N 21/6405 : Multicasting data broadcast and multicast in packet switching networks H04L12/18
• CPC Class Codes: H04N 21/6408 : Unicasting
• CPC Class Codes: H04N 21/6547 : comprising parameters, e.g. for client setup
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8173 : End-user applications, e.g. Web browser, game
• CPC Class Codes: H04N 21/8453 : by locking or enabling a set of features, e.g. optional functionalities in an executable program
• CPC Class Codes: H04N 21/8543 : using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80
• CPC Class Codes: H04N 21/8586 : by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02
• CPC Class Codes: H04N 7/165 : Centralised control of user terminal subsequent to an upstream request signal H04N7/17345; Registering at central by two-way working H04N7/17309

Abstract

A computer may be remotely accessed. At a first location, display commands are generated. The display commands are covered into a compressed video data stream. Each display element (5) is checked if it is encoded (52). If object is encoded, it is transcribed into MPEG (54). The image is adjusted for display (56) and compression (58). Additional steps of motion determination (60), change detection (62), compression depth and frame determination (66, 68) are executed. Then the data is transmitted to a second location. The display commands are decompressed and displayed as an image at the second location.

Claims

1. In a server that provides remote client access to one or more computer programs, a method of determining when to generate a compressed video stream representing a display for the one or more computer program s by monitoring changes to the display, the method comprising:

executing a computer program at a server, wherein the execution of the computer program generates display objects from a set of display commands, the display objects are at least a portion of a display for said computer program;

identifying changes of said display which are responsive to at least one type of continuous user interaction command received from a remote client;

determining whether said changes warrant an update to an image based at least in part on one or more of available bandwidth, available computing power, or type of user connection; and

upon determining said changes do warrant an update, processing said changes and converting said display commands into a compressed video stream, wherein said changes are inserted into said compressed video stream at an update frame rate corresponding to a priority assigned to other portions of the display that are unchanged and such that changes to said image are inserted into the compressed video stream at a faster rate than compressed data that does not include changes to said image.

2. A method according to claim 1, wherein said type of user command comprises a pointing device command.

3. A method according to claim 1, wherein said changes comprises an indication of a selection of a GUI (graphical user interface) element.

4. A method according to claim 1, comprising analyzing said user command to determine display commands which effect said identified changes.

5. A method as recited in claim 1, wherein the display commands are directly converted into the compressed video stream without first generating a display raster of the display commands.

6. The method of claim 1, wherein only changed portions of the image are processed and a new output frame is assembled that includes both newly compressed image portions and unchanged compressed image portions.