System and method for a media intelligence platform

• US 9,805,399 B2
• Filed: 09/12/2016
• Issued: 10/31/2017
• Est. Priority Date: 02/03/2015
• Status: Active Grant

• Current Assignee: Twilio, Inc.
• Original Assignee: Twilio, Inc.
• Inventors: Lawson, Jeffrey
• Primary Examiner(s): Eskandarnia, Arvin
• Application Number:
  US15/262,220
• Publication Number:
  US 20170032433A1
• Time in Patent Office: 414 Days
• Field of Search: 709224
• CPC Class Codes: G06Q 30/04 Billing or invoicing
• CPC Class Codes: H04L 12/1403 Architecture for metering, charging or billing
• CPC Class Codes: H04L 41/147 for predicting network behaviour
• CPC Class Codes: H04L 43/50 Testing arrangements
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04M 15/41 Billing record details, i.e. parameters, identifiers, structure of call data record [CDR]
• CPC Class Codes: H04M 15/765 Linked or grouped accounts, e.g. of users or devices
• CPC Class Codes: H04M 15/7652 shared by users
• CPC Class Codes: H04M 15/7655 shared by technologies
• CPC Class Codes: H04M 15/8066 According to the number of recipients
• CPC Class Codes: H04M 15/8228 Session based
• CPC Class Codes: H04W 4/24 Accounting or billing

Abstract

A multi-tenant media processing platform system and method. At least a first media analysis service of a plurality of media analysis services is activated for at least a portion of an active communication session of an entity in the platform system. The first activated media analysis service performs a first media analysis on media of the active communication session that is collected by the platform system. The first activated media analysis service performs the first media analysis on the collected media while the communication session is active to generate a first media analysis result. During the active communication session, at least one media analysis result is applied.

Claims

1. A method comprising:

at a multi-tenant media processing platform system that includes a plurality of media analysis services;

activating a first media analysis service of the plurality of media analysis services for at least a portion of an active communication session of an entity in the platform system, the platform system activating the first media analysis service responsive to at least one of;

a determination by the platform system that at least one property of media of the active communication session that is collected by the platform system maps to at least one setting of entity platform configuration that enables media analysis, the entity platform configuration being platform configuration of the entity in the platform system,

a communication application instruction,

a REST API call, and

a SIP message;

the activated first media analysis service performing a first media analysis on media of the active communication session that is collected by the activated first media analysis service, while the communication session is active, to generate a first media analysis result;

during the active communication session, applying the first media analysis result by transmitting at least one application layer protocol message to at least one callback URI of an external system of the entity, the at least one application layer protocol message providing the first media analysis result,

accounting for the use of the first activated media analysis service on behalf of the entity; and

generating billing information for the entity based on the accounting for the use of the first activated media analysis service,

wherein each external system of the entity is a system of an account holder of a platform account of the multi-tenant media processing platform system, each external system being external to the platform system.

2. The method of claim 1,

wherein the active communication session is a communication session between at least a first communication endpoint device and a second communication endpoint device, the first communication endpoint device and the second communication endpoint device each being external to the platform system, and

wherein the first media analysis service is constructed to perform media analysis on media of the communication session between at least the first communication endpoint device and the second communication endpoint device,

wherein the active communication session is at least one of:

a peer-to-peer communication session; and

a real-time communication session, and

wherein the media is a real-time media stream.

3. The method of claim 2,

the method further comprising:

at the multi-tenant media processing platform system;

activating a second media analysis service of the plurality of media analysis services; and

the second activated media analysis service performing a second media analysis to generate a second media analysis result.

4. The method of claim 3,

wherein the media is real-time media, and the first media analysis service performs real-time media analysis on the real-time media, and

wherein the second activated media analysis service performs real-time media analysis on at least one of the real-time media and the first media analysis result.

5. The method of claim 4,

the method further comprising:

at the multi-tenant media processing platform system;

during the active communication session, applying the second media analysis result by transmitting at least one application layer protocol message to a communication service of the platform system, the at least one application layer protocol message providing the second media analysis result.

6. The method of claim 5, wherein the second media analysis is different from the first media analysis.

7. The method of claim 3,

wherein the media is real-time media, and the first media analysis service performs real-time media analysis on the real-time media, and

wherein the second activated media analysis service performs asynchronous media analysis on stored media.

8. The method of claim 2, wherein activating the first media analysis service comprises:

the platform system selecting at least a sentiment and emotion detection service as the first media analysis service from the plurality of media analysis services of the platform system, the plurality of media analysis services including the sentiment and emotion detection service and at least one of a context analysis service, and a computer vision service; and

the platform system activating each selected media analysis service.

9. The method of claim 8, wherein a REST API call provided by the external system of the entity specifies at least the communication session and at least the first activated media analysis service.

10. The method of claim 8, the method further comprising at least one of:

at the multi-tenant media processing platform system, de-activating the first media analysis service responsive to a de-activation REST API call provided by an external system of the entity, the de-activation REST API call specifying the communication session; and

at the multi-tenant media processing platform system, configuring the first media analysis service responsive to a configuration REST API call provided by an external system of the entity, the configuration REST API call specifying the communication session.

11. The method of claim 9,

the method further comprising:

at the multi-tenant media processing platform system;

training at least one media analysis service by using the first media analysis result and user feedback provided by an external system via a feedback API of the platform system,

wherein the platform system receives user feedback via the feedback API, the feedback indicating a quality of media analysis provided by the first media analysis service, and the platform system trains the first media analysis service based on the received feedback, wherein the platform system maintains history data that includes media analysis results and corresponding media and user feedback of the first media analysis service, and the platform system trains the first media analysis service by using the history data.

12. The method of claim 1, wherein a REST API call provided by the external system of the entity specifies program instructions for a customized media analysis service, and responsive to the REST API call, the platform system performs at least one of:

generation of a customized media analysis service according to the received program instructions; and

control of a communication service of the platform system to provide a media analysis service according to the received program instructions.

13. The method of claim 1,

the method further comprising:

at the multi-tenant media processing platform system;

responsive to a request for communication records that is provided by the external system of the entity, providing the external system with communication records that match media analysis results properties specified in the request,

wherein the provided communication records include information for a plurality of communication sessions of the entity including the communication session of the entity for which the first media analysis is performed,

wherein the information for the communication session of the entity includes at least information of the first media analysis result,

wherein the platform system indexes media analysis results information of the communication records, and

wherein the matching communication records include indexed media analysis results information that matches media analysis results properties specified in the request.

14. The method of claim 1, wherein the platform system activates the first media analysis service responsive to a REST API call provided by an eternal system of the entity, and wherein the at least one callback URI is specified by the REST API call.

20. The method of claim 1,

wherein the platform system activates the first media analysis service responsive to a determination by the platform system that at least one property of media of the active communication session that is collected by the platform system maps to at least one setting of entity platform configuration that enables media analysis, the entity platform configuration being platform configuration of the entity in the platform system,

wherein the at least one property of media of the active communication session identifies at least one of an endpoint of the communication session, an account of the communication session, a type of communication, a time window associated with the communication session, and a location associated with the communication session, and

wherein the entity platform configuration specifies whether to activate the first media analysis service for the active communication session based on the at least one property.

15. A hardware system comprising:

a multi-tenant media processing platform system that includes a plurality of media analysis services, and that is constructed to;

activate a first media analysis service of the plurality of media analysis services for at least a portion of an active communication session of an entity in the platform system, the platform system being constructed to activate the first media analysis service responsive to at least one of;

a determination by the platform system that at least one property of media of the active communication session that is collected by the platform system maps to at least one setting of entity platform configuration that enables media analysis, the entity platform configuration being platform configuration of the entity in the platform system,

a communication application instruction,

a REST API call, and

a SIP message;

control the activated first media analysis service to perform a first media analysis on media of the active communication session that is collected by the activated first media analysis service, while the communication session is active, to generate a first media analysis result;

during the active communication session, apply the first media analysis result by transmitting at least one application layer protocol message to at least one callback URI of an external system of the entity, the at least one application layer protocol message providing the first media analysis result,

account for the use of the first activated media analysis service on behalf of the entity; and

generate billing information for the entity based on the accounting for the use of the first activated media analysis service,

wherein each external system of the entity is a system of an account holder of a platform account of the multi-tenant media processing platform system, each external system being external to the hardware system.

16. The system of claim 15, wherein the platform system is constructed to activate the first media analysis service responsive to a REST API call provided by an eternal system of the entity, and wherein the at least one callback URI is specified by the REST API call.

17. The system of claim 15,

wherein the active communication session is a communication session between at least a first communication endpoint device and a second communication endpoint device, the first communication endpoint device and the second communication endpoint device each being external to the hardware system, and

wherein the first media analysis service is constructed to perform media analysis on media of the communication session between at least the first communication endpoint device and the second communication endpoint device,

wherein the active communication session is at least one of:

a peer-to-peer communication session; and

a real-time communication session, and

wherein the media is a real-time media stream.

18. The system of claim 15, wherein a REST API call provided by the external system of the entity specifies program instructions for a customized media analysis service, and responsive to the REST API call, the platform system is constructed to perform at least one of:

generation of a customized media analysis service according to the received program instructions; and

control of a communication service of the platform system to provide a media analysis service according to the received program instructions.

19. The system of claim 15,

wherein the multi-tenant media processing platform system is constructed to:

responsive to a request for communication records that is provided by the external system of the entity, providing the external system with communication records that match media analysis results properties specified in the request,

wherein the provided communication records include information for a plurality of communication sessions of the entity including the communication session of the entity for which the first media analysis is performed,

wherein the information for the communication session of the entity includes at least information of the first media analysis result,

wherein the platform system indexes media analysis results information of the communication records, and

wherein the matching communication records include indexed media analysis results information that matches media analysis results properties specified in the request.