System architecture and method for composing and directing participant experiences

• US 8,789,121 B2
• Filed: 02/07/2013
• Issued: 07/22/2014
• Est. Priority Date: 10/21/2010
• Status: Active Grant

• Current Assignee: Wickr Inc. (Amazon.com, Inc.)
• Original Assignee: Net Power & Light, Inc (Amazon.com, Inc.)
• Inventors: Vonog, Stanislav, Surin, Nikolay, Lemmey, Tara
• Primary Examiner(s): Bruckart, Benjamin R
• Assistant Examiner(s): Nguyen, Tien M
• Application Number:
  US13/762,149
• Publication Number:
  US 20130156093A1
• Time in Patent Office: 530 Days
• Field of Search: 725/24, 725/117
• US Class Current: 725/109
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04N 19/90 using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
• CPC Class Codes: H04N 21/2543 Billing , e.g. for subscription services payment schemes, architectures or protocols G06Q20/00; e-commerce G06Q30/00; arrangements for billing for the use of broadcast information or broadcast-related information H04H60/21
• CPC Class Codes: H04N 21/4131 home appliance, e.g. lighting, air conditioning system, metering devices home automation data switching networks exchanging configuration information on appliance services H04L12/2807
• CPC Class Codes: H04N 21/4627 Rights management associated to the content security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104

Abstract

The present invention contemplates a variety of improved methods and systems for providing an experience platform within a CDN, as well as sentio or experience codecs, and experience agents for supporting the experience platform. The experience platform may be provided by a service provider to enable an experience provider to compose and direct a participant experience. The service provider monetizes the experience by charging the experience provider and/or the participants for services. Alternatively, the experience platform may be provided by the CDN, either as an integrated component of the CDN, or a supplemental complimentary aspect. The participant experience can involve one or more experience participants. The experience provider can create an experience with a variety of dimensions and features. As will be appreciated, the following description provides one paradigm for understanding the multi-dimensional experience available to the participants. There are many suitable ways of describing, characterizing and implementing the experience platform contemplated herein.

Claims

1. A non-transitory storage medium for storing modules executable by a processor comprising:

a content player module presenting a streaming content received from a content distribution network (CDN);

a user interface module operative to receive an user input;

an experience agent module operative to receive one or more live realtime participant experiences as one or more separate layers transmitted from one or more realtime participant experience engines via a low-latency protocol and to present a real-time participant experience by composing the one or more separate layers and the streaming content into a single live stream;

wherein the low-latency protocol to transmit the one or more live realtime participant experiences is operative to;

divide each of the one or more live realtime participant experiences into a plurality of regions; and

encode a first region of the plurality of regions with a first codec and a second region of the plurality of regions with a second codec; and

wherein the storage medium further comprises a sentio codec operative to encode and decode data streams corresponding to different senses and dimensions of an experience.

2. The storage medium of claim 1, wherein the experience agent module is operative to overlay the one or more separate layers and streaming content.

3. The storage medium of claim 1, wherein the experience agent module is operative to communicate with one or more non-realtime services.

4. The storage medium of claim 1, wherein the experience agent module comprises a video ensemble application programming interface (API) and wherein the video ensemble API is operative to receive a video ensemble realtime participant experience from at least one of the one or more realtime participant experience engines.

5. The storage medium of claim 1, wherein the experience agent module comprises a synch digital video recorder (DVR) application programming interface (API) and wherein the synch DVR API is operative to receive a synch DVR realtime participant experience from at least one of the one or more realtime experience engines.

6. The storage medium of claim 1, wherein the experience agent module comprises a chalktalk application programming interface (API) and wherein the chalktalk API is operative to receive a chalktalk realtime participant experience from at least one of the one or more realtime experience engines.

7. The storage medium of claim 1, wherein the experience agent module comprises a virtual experience application programming interface (API) and wherein the virtual experience API is operative to receive a realtime participant virtual experience from at least one of the one or more realtime experience engines.

8. The storage medium of claim 3, wherein the experience agent module comprises a content application programming interface (API) and wherein the content API is operative to receive a streaming content search information from a non-realtime service.

9. The storage medium of claim 3, wherein the experience agent module comprises a friends application programming interface (API) and wherein the friends API is operative to receive friends information from a non-realtime service.

10. The storage medium of claim 1, the experience agent module comprises an explore engine to provide an interface to explore available contents.

11. The storage medium of claim 1, wherein the streaming content is a live streaming content received from the CDN.

12. The storage medium of claim 1, wherein the streaming content is an on-demand streaming content received from the CDN.

13. The storage medium of claim 1, the sentio codec is operative to synchronize a presentation of the one or more separate layers into the single live stream.

14. The storage medium of claim 1, wherein the experience agent module is operative to communicate with one or more non-realtime services via a human-readable data interchange format.

15. storage medium of claim 1, wherein the content player module is operative to present a streaming content controlled by a digital rights management (DRM).

16. A system for generating multi-dimensional realtime participant experiences, the system comprising:

a plurality of realtime experience engines; and

a plurality of non-realtime services; and

a sentio codec coupled to the plurality of the realtime experience engines operative to encode and decode multiple dimensions of perception related data streams for the plurality of the realtime experience engines;

wherein each of the realtime experience engines is operative to compose and transmit a realtime participant experience to an experience agent module on a remote device via a low-latency protocol and communicate with one or more non-realtime services;

wherein each of the non-realtime services is operative to communicate with the experience agent module, and is further operative to communicate with at least one of the plurality of realtime experience engines;

wherein the plurality of non-realtime services comprises a content service operative to receive a streaming content search result from a content distribution network (CDN); and

wherein the low-latency protocol to transmit the realtime participant experience is operative to;

divide the realtime participant experience into a plurality of regions; and

encode a first region of the plurality of regions with a first codec and a second region of the plurality of regions with a second codec.

17. The system of claim 16, wherein at least one of the plurality of realtime experience engines is operative to convert a plurality of realtime participant experiences into a single live stream for the experience agent module.

18. The system of claim 16, wherein the plurality of realtime experience engines includes an experience composition engine operative to compose a single live participant experience from experience layers received from the plurality of realtime experience engines, the plurality of non-realtime services, the CDN, or any combination thereof.

19. The system of claim 16, wherein the plurality of non-realtime services comprises an account service operative to maintain an account information including friends information and user devices information.

20. The system of claim 19, wherein the plurality of non-real-time services comprises a group formation service operative to form a group based on the account information from the account service.

21. The system of claim 16, wherein the plurality of non-real-time services comprises a presence service operative to generate and transmit a presence notification.

22. The system of claim 16, wherein the plurality of realtime experience engines comprises a video ensemble engine operative to transmit a video ensemble realtime participant experience.

23. The system of claim 16, wherein the plurality of realtime experience engines comprises a synch digital video recorder (DVR) engine operative to transmit a synch DVR realtime participant experience.

24. The system of claim 16, wherein the plurality of realtime experience engines comprises a synch chalktalk engine operative to transmit a synch chalktalk realtime participant experience, the synch chalktalk realtime experience providing an interactive drawing synchronized across multiple participant devices.

25. The system of claim 16, wherein the plurality of realtime experience engines comprises a virtual experience engine operative to transmit a realtime participant virtual experience.

26. The system of claim 16, wherein at least one of the realtime experience engines is operative to generate an ensemble of realtime participant experiences and transmit each realtime participant experience of the ensemble to the experience agent module.

27. A system for composing and directing multi-dimensional realtime participant experiences, the system comprising:

a content distribution network (CDN) infrastructure including an origin content server and a plurality of surrogate content servers, the CDN infrastructure operable to distribute content across various geographic locations;

a realtime experience engine;

a non-realtime service; and

a device including a content player presenting a streaming content received from the CDN infrastructure, the device further including an experience agent module presenting one or more live realtime participant experiences transmitted from the realtime experience engine via a low-latency protocol;

wherein the realtime experience engine is operative to transmit one or more realtime participant experiences to the experience agent module via a low-latency protocol and communicate with the non-realtime service;

wherein the non-realtime service is operative to communicate with the experience agent module, and communicate with the realtime experience engine;

wherein the experience agent module includes a sentio codec operative to encode or decode perception-related data streams associated with the one or more real-time participant experiences received from the realtime experience engine; and

wherein the low-latency protocol to transmit the realtime participant experience is operative to;

divide the realtime participant experience into a plurality of regions; and

encode a first region of the plurality of regions with a first codec and a second region of the plurality of regions with a second codec.

28. The system of claim 27, wherein the device further comprises a user interface operative to receive an input from a user of the device to interact with the one or more live participant experiences presented on the device.

29. The system of claim 27, wherein the realtime experience engine comprises a realtime experience engine operative to compose and direct a video ensemble realtime participant experience, a synch DVR realtime participant experience, synch chalktalk realtime participant experience, a realtime participant virtual experience, or any combination thereof.

30. A system for composing and directing multi-dimensional realtime participant experiences, the system provided by a service provider for use by an experience provider, the system comprising:

an experience server including;

a first experience agent, the first experience agent including a first sentio codec for encoding and decoding multi-dimensional data streams; and

an experience composition engine;

a plurality of devices, wherein each device includes a specific experience agent operative to communicate with the experience server;

a content server in communication with the experience server and operable to provide a content layer to the experience composition engine, the content server being part of a content distribution network (CDN) infrastructure including an origin content server and a plurality of surrogate content servers, the CDN infrastructure operable to distribute content to participants across various geographic locations;

a service platform in communication with and operable to provide services, for contributing experience dimensions to the experience composition engine, to the plurality of devices and the experience server, the service platform including;

a plurality of service engines; and

a second experience agent including an application programming interface (API) and a second sentio codec, the services of the plurality of service engines exposed via the API; and

wherein

the experience composition engine is operable to;

receive, via low latency transmission protocol, experience layers including the content layer from the content server and the plurality of devices;

request, via the first experience agent, the services from the plurality of service engines via the API; and

compose and direct a realtime participant experience using the layers, and services performed by the service platform;

wherein the low-latency transmission protocol is operative to;

divide the realtime participant experience into a plurality of regions;

encode a first region of the plurality of regions with a first codec and a second region of the plurality of regions with a second codec.