Method and devices for implementing highly interactive entertainment services using interactive media-streaming technology, enabling remote provisioning of virtual reality services

• US 20040064504A1
• Filed: 08/08/2003
• Published: 04/01/2004
• Est. Priority Date: 08/12/2002
• Status: Active Grant

• Current Assignee: Sound View Innovations, LLC (Sound View Innovation Holdings, LLC)
• Original Assignee: Alcatel SA (Nokia Corporation)
• Inventors: Domschitz, Peter
• Granted Patent: US 7,480,727 B2
• Time in Patent Office: Days
• US Class Current: 709/203
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/338 using television networks
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/71 using secure communication between game devices and game servers, e.g. by encrypting game data or authenticating players
• CPC Class Codes: A63F 2300/401 Secure communication, e.g. using encryption or authentication
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/409 Data transfer via television network
• CPC Class Codes: A63F 2300/50 characterized by details of game servers
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/4433 Implementing client middleware, e.g. Multimedia Home Platform [MHP]
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 21/4781 Games
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

The invention relates to a method for generating an interactive virtual reality with a network service using interactive media-streaming technology comprising the steps of establishing an action stream session comprising connection handling, quality of service handling, adapting the network environment by demanding network resources and control information, establishing media-streaming path from the service to the client and a user interaction control path in the reverse direction, controlling the network with respect to required quality of service, continuously, generating and transmitting individual media streams to the client (ASC) by embedding interaction into a virtual reality, and extracting and encoding a media stream at the service using a virtual reality description compressed motion picture stream, encoding and transmitting the user'"'"'s interaction to the service, as well as de-coding and playing the individual media data stream at the client side. Further it relates to a Action Streaming Service, Action Streaming Client (ASC), Action Streaming Server (ASS), Action Streaming System, Action Stream, Action Streaming Session, Action Streaming Protocol, and Computer Software Products for generating an interactive virtual reality.

Claims

1. A method for generating an interactive individual virtual reality with at least one action streaming client (ASC) and a network action streaming service system using interactive media-streaming technology comprising the steps of

establishing an action stream session comprising connection handling between said network service and said client, and comprising quality of service handling,

adapting the network environment by demanding network resources and control information in the user'"'"'s client and participated network elements (NE),

establishing a media-streaming path (CN) from the service to the client and a user interaction control path (CN) from the client to the service,

generating and transmitting individual media streams to the client by embedding interaction into a virtual reality, and extracting and encoding a media stream at the service using a virtual reality description compressed motion picture stream,

encoding and transmitting the user'"'"'s interaction to the service, as well as decoding and playing the individual media data stream at the client side.

2. The method according to claim 1, where said network environment and said media streaming path is coordinated for multiple possibly interacting user action streaming clients (ASC).

3. The method according to claim 1, further comprising the step of controlling the network for controlling the required quality of service, continuously.

4. The method according to claim 3, where said controlling the network for ensuring the required quality of service is coordinated between for multiple possibly interacting user action streaming clients (ASC) and based on the virtual reality scenario.

5. The method according to claim 3, where said quality of service are especially high data rates in the downstream direction as well as a minimal round trip delay and/or minimal delay variation in both directions.

6. The method according to claim 1, where said generating individual media streams by embedding interaction into a virtual reality, and extracting and encoding a media stream at the service using a virtual reality description compressed motion picture stream is performed by coding parts of the virtual reality description directly in the outgoing compressed data stream.

7. The method according to claim 1, where said media stream is on the basis of an application oriented graphic and/or sound description information without intermediate uncompressed video information.

8. The method according to claim 1, where said action stream session comprising an compatibility alignment, e.g. by updating and configuring software parts of the service and/or the client by uploading software.

9. An Action Streaming Client (ASC) for generating an interactive individual virtual reality invoking a network action streaming service system using interactive media-streaming technology comprising

a downstream interface (TP/PI) for receiving interactive media streams,

decoding means for viewing interactive media streams, and

controlling means (APP, ME-CT, ME-P, TP/PI, Ul) for encoding user interaction and demanding network resources,

an upstream interface (TP/PI) for transmitting the encoded interaction, leading to an instantaneous manipulation of the media downstream channel.

10. The Action Streaming Client (ASC) according to claim 9, realized by a device implementing a DSL access enabled digital TV user equipment.

11. The Action Streaming Client (ASC) according to claim 9, realized by a enabled personal computer with DSL access.

21. A computer software product for generating an interactive virtual reality with real time user interaction using interactive media-streaming technology realizing an action streaming client according to claim 9.

12. An Action Streaming Server (ASS) for generating an interactive individual virtual reality providing a network action streaming service system using interactive media-streaming technology comprising

at least one upstream interface (IN) for receiving user interaction and at least one downstream interface (IN) for providing an interactive media-stream, and

for at least one user sharing a media stream

an interpreter (INJ) for the received user interaction,

a virtual reality engine (VRE) for embedding the user interaction in the virtual reality,

a media extraction (ME) part for extracting an individual media stream,

an encoder (ME) for encoding the individual media stream, and

a session controlling unit (SES-CON) for controlling the required quality of service, continuously,

and an environment controller (ENV-CON) for coordinating multiple individual virtual realities, and multiple individual media streams.

13. An Action Streaming Service System (ASS) providing resources for generating an interactive individual virtual reality using interactive media-streaming technology comprising

at least one upstream interface (IN) for receiving user interaction and at least one downstream interface (IN) for providing an interactive media-stream, and

for at least one user sharing a media stream

an interpreter (INJ) for the received user interaction,

a virtual reality engine (VRE) for embedding the user interaction in the virtual reality,

a media extraction (ME) part for extracting an individual media stream,

an encoder (VSE) for encoding the individual media stream, and

a session controlling unit (SES-CON) for ensuring the required quality of service, continuously,

and an environment controller (ENV-CON) for coordinating multiple individual virtual realities, and multiple individual media streams.

20. A computer software product for generating an interactive virtual reality with real time user interaction using interactive media-streaming technology realizing an Action Streaming Service according to claim 13.

14. An Action Streaming System for generating an interactive virtual reality with real time user interaction using interactive media-streaming technology comprising

an access network (NW) providing at least one action streaming service system,

where said action streaming service system (ASS) comprises means for generating an interactive virtual reality,

at least one action streaming client (ASC) comprises means for consuming said interactive virtual reality,

where said action streaming service is located in a network at an action streaming server (ASS) and said network (NW) is controlled by the service.

15. An action stream comprising a data structure for encoding, decoding a virtual reality in a media data stream a data structure for embedding interaction, and a control structure for managing network resources ensuring the required quality of service.

16. The action stream realized by a digital video broadcast multimedia home platform compliant video/audio and control data stream.

17. The action stream realized MPEG compliant video/audio and control data stream.

18. An action streaming session comprising

a connection handling between service and at least one client

a perquisite quality of service handling ensuring that the network provides the required quality of service, and

a continuous quality of service handling according to the service'"'"'s quality of service demands,

a compatibility alignment between server and client,

a service authentication-authorization-and-accounting,

as well as action stream exchange.

19. An action streaming protocol comprising means for creating an action streaming service session, means for adaptation of the users'"'"' client and the service, means for authentication-authorization-and-accounting, means for controlling network resources according to quality of services demands, and means for coordinating and exchanging action streams.