System for collaborative conferencing using streaming interactive video

• US 8,949,922 B2
• Filed: 12/05/2007
• Issued: 02/03/2015
• Est. Priority Date: 12/10/2002
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: OL2 Incorporated (Sony Group Corp.)
• Inventors: Perlman, Stephen G., van der Laan, Roger
• Primary Examiner(s): VU, NGOC K
• Application Number:
  US11/999,462
• Publication Number:
  US 20090119737A1
• Time in Patent Office: 2,617 Days
• Field of Search: 725/105, 725/114, 725/133, 725/141, 725/153, 463/40
• US Class Current: 725/133
• CPC Class Codes: A63F 13/12 involving interaction between a plurality of game devices, e.g. transmisison or distribution systems protocols for networked virtual reality, networked games H04L29/06034
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/358 Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
• CPC Class Codes: A63F 2300/402 Communication between platforms, i.e. physical link to protocol
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 21/2143 located in a single building, e.g. hotel, hospital or museum arrangements specially adapted for plural spots in a confined site in broadcast systems H04H20/63; adaptations for transmission by electric cable for domestic distribution in television systems H04N7/106
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/222 Secondary servers, e.g. proxy server, cable television Head-end provisioning of proxy services in data packet switching networks H04L67/56
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/23605 Creation or processing of packetized elementary streams [PES]
• CPC Class Codes: H04N 21/23805 Controlling the feeding rate to the network, e.g. by controlling the video pump
• CPC Class Codes: H04N 21/2387 Stream processing in response to a playback request from an end-user, e.g. for trick-play
• CPC Class Codes: H04N 21/2393 involving handling client requests scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/2743 Video hosting of uploaded data from client
• CPC Class Codes: H04N 21/2747 : Remote storage of video programs received via the downstream path, e.g. from the server
• CPC Class Codes: H04N 21/42607 : for processing the incoming bitstream
• CPC Class Codes: H04N 21/4781 : Games
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/64322 : IP
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/8173 : End-user applications, e.g. Web browser, game
• CPC Class Codes: H04N 21/8545 : for generating interactive applications
• CPC Class Codes: H04N 7/106 : for domestic distribution
• CPC Class Codes: H04N 7/17318 : Direct or substantially direct transmission and handling of requests
• CPC Class Codes: H04W 4/06 : Selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS]; Services to user groups; One-way selective calling services
• CPC Class Codes: H04W 84/12 : WLAN [Wireless Local Area Networks]
• CPC Class Codes: H04W 88/14 : Backbone network devices

Abstract

An apparatus comprising a server that generates a stream of streaming interactive video and/or audio. Means for multicasting the stream to a plurality of client devices accommodating different device video, audio and/or network characteristics, and means for allowing input from one or more of the client devices to control the streaming interactive video.

Claims

1. A computer-implemented method of operation for a hosting service system which includes a plurality of servers comprising:

running one or more twitch video games or applications on the servers, one or more streams of uncompressed streaming interactive video being generated therefrom;

compressing the one or more streams of uncompressed streaming interactive video, thereby producing one or more compressed streaming interactive video streams;

routing the one or more compressed streaming interactive video streams to a plurality of client devices over the Internet, each of the client devices being located remote to the plurality of servers;

receiving, by one or more of the servers, input transmitted from one or more of the client devices, the input including video input;

integrating the video input into the one or more streams of uncompressed streaming interactive video to produce one or more streams of uncompressed collaborative streaming interactive video;

compressing the one or more streams of uncompressed collaborative streaming interactive video, thereby producing one or more compressed collaborative streaming interactive video streams; and

routing the one or more compressed collaborative streaming interactive video streams to the client devices over the Internet.

2. The computer-implemented method of claim 1 wherein the video input is compressed.

3. The computer-implemented method of claim 1 wherein the input further includes audio input.

4. The computer-implemented method of claim 1 wherein at least one of the client devices comprises a mobile communication device having a camera.

5. The computer-implemented method of claim 1 further comprising sizing the video input appropriately for each of the client devices depending on characteristics of a display associated with each client device and a corresponding connection between each client device and the servers.

6. The computer-implemented method of claim 1 wherein the one or more compressed streaming interactive video streams are compressed with a finite latency of about 80 ms or less.

7. The computer-implemented method of claim 1 wherein the one or more compressed collaborative streaming interactive video streams are compressed with a finite latency of about 80 ms or less.

8. The computer-implemented method of claim 1 wherein the one or more compressed streaming interactive video streams are compressed with a finite latency of about 150 ms or less.

9. The computer-implemented method of claim 1 wherein the one or more compressed collaborative streaming interactive video streams are compressed with a finite latency of about 150 ms or less.

10. A computer-implemented method comprising:

running, at a hosting service center, a twitch video game or application as part of a collaborative conference among a plurality of users, the twitch video game or application running on a plurality of servers that generate a one or more streams of uncompressed streaming interactive video therefrom;

compressing the one or more streams of uncompressed streaming interactive video thereby producing one or more compressed streaming interactive video streams;

transmitting the one or more compressed of streaming interactive video streams over the Internet to a plurality of client devices correspondingly associated with the plurality of users, the client devices each having a display screen and each being located remote to the hosting service center, at least one of the client devices comprising a communication device that includes a camera;

receiving, at the hosting service center, input from the communication device that includes real-time video captured by the camera;

integrating the real-time video into the one or more streams of uncompressed streaming interactive video to produce one or more streams of uncompressed collaborative streaming interactive video;

compressing the one or more streams of uncompressed collaborative streaming interactive video, thereby producing one or more compressed collaborative streaming interactive video streams;

transmitting the one or more compressed collaborative streaming interactive video streams to the client devices over the Internet; and

wherein the one or more streams of collaborative streaming interactive video are compressed with a sufficiently low-latency as to make the real-time video appear on the display screen of each of the client devices without perceptual delay.

11. The computer-implemented method of claim 10 wherein the one or more compressed streaming interactive video streams are compressed with a finite latency of about 80 ms or less.

12. The computer-implemented method of claim 10 wherein the one or more compressed collaborative streaming interactive video streams are compressed with a finite latency of about 80 ms or less.

13. The computer-implemented method of claim 10 wherein the communication device comprises a mobile communication device.

14. The computer-implemented method of claim 10 wherein the input further includes audio input.

15. The computer-implemented method of claim 10 wherein the one or more compressed streaming interactive video streams are compressed with a finite latency of about 150 ms or less.

16. The computer-implemented method of claim 10 wherein the one or more compressed collaborative streaming interactive video streams are compressed with a finite latency of about 150 ms or less.