Apparatus and methods for data collection, analysis and validation including error correction in a content delivery network

• US 8,484,511 B2
• Filed: 07/01/2010
• Issued: 07/09/2013
• Est. Priority Date: 07/01/2010
• Status: Active Grant

• Current Assignee: Time Warner Cable Enterprises LLC (Charter Communications Incorporated)
• Original Assignee: Time Warner Cable Enterprises LLC (Charter Communications Incorporated)
• Inventors: Tidwell, Justin, Samame, Eduardo, Engel, Craig
• Primary Examiner(s): NGUYEN BA, HOANG VU A
• Application Number:
  US12/829,104
• Publication Number:
  US 20120005527A1
• Time in Patent Office: 1,104 Days
• Field of Search: 714/15, 714/57, 725/9, 725/14, 709/223, 709/224, 707687-690, 707/694, 707696-701, 707/761
• US Class Current: 714/15
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: H04H 60/33 Arrangements for monitoring the users' behaviour or opinions
• CPC Class Codes: H04H 60/46 for recognising users' preferences
• CPC Class Codes: H04H 60/64 for providing detail information
• CPC Class Codes: H04N 21/2408 Monitoring of the upstream path of the transmission network, e.g. client requests monitoring data switching networks utilization H04L43/0876; scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/44204 Monitoring of content usage, e.g. the number of times a movie has been viewed, copied or the amount which has been watched monitoring of user activities for profile generation for accessing a video database G06F16/739; protecting generic digital content where the protection is independent of the precise nature of the content G06F21/10; arrangements for monitoring the use made of the broadcast services in broadcast systems H04H60/31
• CPC Class Codes: H04N 21/44224 Monitoring of user activity on external systems, e.g. Internet browsing
• CPC Class Codes: H04N 21/4524 involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; systems specially adapted for using geographical information in broadcast systems H04H60/70; protocols in which the network application is adapted for the location of the user terminal in communication control or processing H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/458 Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10
• CPC Class Codes: H04N 21/4667 Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38

Abstract

Methods and apparatus for collection, validation, analysis, and automated error correction of data regarding user interaction with content. In one embodiment, statistical methods are used to arrive at expected values for the collected data. The data is compared to the expected value and must meet one or more acceptance criteria (e.g., be within a prescribed range) to be considered valid. The prescribed range is determined by the network operator, or a computer program adapted to generate this value. The invention enables a network operator to assess a large volume of data without requiring significant amounts of manual monitoring and/or error correction. The ability to collect, validate and analyze data across multiple platforms is also provided. Still further, an automated system capable of learning evaluation and error correction patterns is disclosed.

Claims

1. A method of providing data collection in a content delivery network, said method comprising:

collecting a plurality of data relating to interaction of a plurality of users with content or apparatus controlling said content;

deriving an expected value for said plurality of data utilizing statistical regression analysis;

based at least in part on said expected value, configuring a range of acceptable values for said plurality of data;

comparing said plurality of data to said range of acceptable values;

when said plurality of data is within said range, transmitting said plurality of data to at least one entity for analyzing said plurality of data; and

when said plurality of data is not within said range;

generating at least one error message;

implementing an error correction process to correct said plurality of data; and

transmitting corrected data to at least one entity for analyzing.

2. The method of claim 1, wherein said act of implementing said error correction process comprises checking for an existing error profile associated with said plurality of data.

3. The method of claim 2, wherein when said error profile does not exist, determining whether to create a new error profile based at least on previous similar corrections.

4. A method of providing data collection in a content delivery network, said method comprising:

collecting a plurality of data relating to interaction of a plurality of users with content or apparatus controlling said content;

deriving an expected value for said plurality of data;

based at least in part on said expected value, configuring a range of acceptable values for said plurality of data by utilizing sample data to determine instances of error given various operator-designated ranges;

comparing said plurality of data to said range of acceptable values;

when said plurality of data is within said range, transmitting said plurality of data to at least one entity for analyzing said plurality of data; and

when said plurality of data is not within said range;

generating at least one error message;

implementing an error correction process to correct said plurality of data; and

transmitting corrected data to at least one entity for analyzing.

5. The method of claim 4, wherein said act of configuring a range of acceptable values further comprises defining a sample period timeframe and a type of regression analysis to apply.

6. The method of claim 5, wherein said sample period timeframe and said type of regression analysis is used to determine whether a tolerance of said range of acceptable values exceeds a predetermined threshold.

7. A method of providing data collection in an operator-managed content delivery network, said method comprising:

collecting a plurality of data relating to interaction of a plurality of users with content or apparatus controlling said content;

deriving an expected value for said plurality of data;

based at least in part on said expected value, configuring a range of acceptable values for said plurality of data;

comparing said plurality of data to said range of acceptable values;

when said plurality of data is within said range, transmitting said plurality of data to at least one entity for analyzing said plurality of data; and

when said plurality of data is not within said range;

generating at least one error message;

implementing a manual error correction process to correct said plurality of data by said operator; and

transmitting corrected data to at least one entity for analyzing.

8. The method of claim 7, wherein said act of implementing said manual error correction process comprises checking for an existing error profile associated with said plurality of data;

and when said error profile does not exist, determining whether to create a new error profile based at least on previous similar corrections.

9. The method of claim 7, wherein said plurality of platforms include at least one of:

a video on demand (VOD) server, a linear broadcast content source, and an Internet content source.

10. The method of claim 7, wherein said error correction process further comprises:

generating a record of said manual correction of said plurality of data; and

when a second plurality of data is determined to not be within said range, utilizing said record of said manual correction to automatically correct said second plurality of data.

11. In a content delivery network comprising a plurality of users receiving a plurality of content from a plurality of platforms, said plurality of platforms including at least one of:

a video on demand (VOD) server, a linear broadcast content source, and an Internet content source, a method of collecting data comprising;

collecting data relating to an interaction of said plurality of users with said plurality of contents received from said plurality of platforms;

comparing values of individual ones of a plurality of data fields in said data to one or more acceptance criteria for each of said individual ones of said plurality of data fields; and

when said one or more criteria are met, transmitting said data to at least one entity for analyzing said data.

12. The method of claim 11, wherein said interaction of said plurality of users comprises trick mode operations performed in association with said plurality of content.

13. The method of claim 11, wherein said interaction of said plurality of users comprises tuning events.

14. In a content delivery network comprising a plurality of users receiving a plurality of content from a plurality of platforms, a method of collecting data comprising:

collecting data relating to an interaction of said plurality of users with said plurality of contents received from said plurality of platforms;

comparing values of individual ones of a plurality of data fields in said data to one or more acceptance criteria for each of said individual ones of said plurality of data fields;

when said one or more criteria are met;

anonymizing said data via at least one cryptographic hash function; and

transmitting said data to at least one entity for analyzing said data.

15. The method of claim 14, wherein said act of transmitting further comprises performing a data integrity check of said data.

16. The method of claim 14, wherein said act of transmitting further comprises performing a delivery integrity check of said data.

17. In a content delivery network comprising a plurality of users receiving a plurality of contents from a plurality of platforms, a method of collecting data comprising:

collecting data relating to an interaction of said plurality of users with said plurality of contents received from said plurality of platforms;

deriving expected values for each of individual ones of a plurality of data fields in the data;

based at least in part on said expected values, configuring a range of acceptable values for each of said individual ones of said plurality of data fields;

comparing values of individual ones of a plurality of data fields in said data to said range of acceptable values for each of said individual ones of said plurality of data fields; and

when said values of said individual ones of said plurality of data fields are within said range of acceptable values for each of said individual ones of said plurality of data fields, transmitting said data to at least one entity for analyzing said data.

18. The method of claim 17, wherein said content delivery network comprises a broadcast switched network.

19. The method of claim 17, wherein said content delivery network comprises a Hybrid Fiber coaxial (HFC) cable television network, and said plurality of contents is delivered according to a unicast session over said HFC network via one or more Quadrature Amplitude Modulation (QAM) channels.

20. In a content delivery network comprising a plurality of users receiving a plurality of contents from a plurality of platforms, a method of collecting data comprising:

collecting data relating to an interaction of said plurality of users with said plurality of contents received from said plurality of platforms;

comparing values of individual ones of a plurality of data fields in said data to one or more acceptance criteria for each of said individual ones of said plurality of data fields;

when said one or more criteria are met, transmitting said data to at least one entity for analyzing said data; and

implementing an error correction process to correct said data having values of individual ones of said plurality of data fields which do not meet said one or more acceptance criteria.

21. The method of claim 20, further comprising determining whether an error profile has been created.

22. The method of claim 21, further comprising testing the success of said error correction.

23. A method of correcting errors in one or more fields of a plurality of collected data, said method comprising:

accessing an error indication, said error indication identifying said one or more fields of an individual one of said plurality of collected data having one or more errors;

determining whether an error correction record exists for correction of said one or more fields;

when said error correction record exists for correction of said one or more fields, automatically correcting said one or more fields; and

when said error correction record does not exist for correction of said one or more fields;

generating said error correction record for said one or more fields; and

storing said error correction record for said one or more fields for subsequent substantially automatic correction of errors in said one or more fields.

24. The method of claim 23, wherein when a threshold of multiple instances of said storage of said error correction record for said one or more fields has been met, an error correction profile is created.

25. The method of claim 24, wherein said one or more fields comprises one or more of:

(i) record type, (ii) field type, and (iii) error type.

26. A method of correcting errors in one or more fields of a plurality of collected data, said method comprising:

accessing an error message identifying said one or more fields of an individual one of said plurality of collected data having one or more errors;

determining whether an error correction record exists for correction of said one or more fields;

when said error correction record exists for correction of said one or more fields, automatically correcting said one or more fields; and

when said error correction record does not exist for correction of said one or more fields, causing a manual error correction process to be performed by an operator.

27. The method of claim 26, wherein said manual error correction process comprises using a template to create an error correction profile.

28. The method of claim 27, wherein subsequent implementations of said error correction profile always use the same correction performed by said operator.

29. A non-transitory computer readable apparatus comprising a medium adapted to store a computer program thereon, said computer program comprising a plurality of instructions which are configured to, when executed:

collect data which relates to interaction of a plurality of users with a plurality of content received from a plurality of platforms, said data each comprising a plurality of data fields having values contained therein capable of being validated, and said plurality of platforms include at least one of a video on demand (VOD) server, a linear broadcast content source, and an Internet content source;

compare said values of said plurality of data fields for each of said data to one or more acceptance criteria;

when said one or more criteria of said plurality of data fields are met, transmit said plurality of data to at least one entity for analysis; and

when said one or more criteria of said plurality of data fields are not met, implement a substantially automated error correction process.

30. The apparatus of claim 29, wherein said at least one entity for analysis provides targeted secondary content insertion opportunities.

31. The apparatus of claim 29, wherein said collected data is used to predict tuning behavior of said plurality of users.

34. The apparatus of claim 29, wherein said error correction process comprises a check for an established error profile associated with said plurality of data.

32. A non-transitory computer readable apparatus comprising a medium adapted to store a computer program thereon, said computer program comprising a plurality of instructions which are configured to, when executed:

collect data from a plurality of user devices which relates to interaction of a plurality of users with a plurality of content received from a plurality of platforms, said data each comprising a plurality of data fields having values contained therein capable of being validated;

compare said values of said plurality of data fields for each of said data to one or more acceptance criteria;

when said one or more criteria of said plurality of data fields are met, transmit said plurality of data to at least one entity for analysis; and

when said one or more criteria of said plurality of data fields are not met, implement a substantially automated error correction process.

33. The apparatus of claim 32, wherein said interaction of said plurality of users comprises trick mode operations performed in association with said plurality of contents.

35. A non-transitory computer readable apparatus comprising a medium adapted to store a computer program thereon, said computer program comprising a plurality of instructions which are configured to, when executed:

collect data which relates to interaction of a plurality of users with a plurality of content received from a plurality of platforms, said data each comprising a plurality of data fields having values contained therein capable of being validated;

deriving one or more acceptance criteria using statistical log-linear, log-log, or linear regression analysis;

compare said values of said plurality of data fields for each of said data to said one or more acceptance criteria;

when said one or more criteria of said plurality of data fields are met, transmit said plurality of data to at least one entity for analysis; and

when said one or more criteria of said plurality of data fields are not met, implement a substantially automated error correction process.

36. The apparatus of claim 35, wherein said analysis comprises a determination of viewership for a particular program.

37. The apparatus of claim 35, further configured to perform a periodic audit to ensure accurate validation.

38. A non-transitory computer readable apparatus comprising a medium adapted to store a computer program thereon, said computer program comprising a plurality of instructions which are configured to, when executed:

collect data which relates to interaction of a plurality of users with a plurality of content received from a plurality of platforms, said data each comprising a plurality of data fields having values contained therein capable of being validated;

compare said values of said plurality of data fields for each of said data to one or more acceptance criteria;

when said one or more criteria of said plurality of data fields are met, transmit said plurality of data to at least one entity for analysis; and

when said one or more criteria of said plurality of data fields are not met, utilize an existing error correction profile to automatically correct said plurality of data fields.

39. The apparatus of claim 38, wherein said existing error correction profile is tested using an appropriate regression model.

40. The apparatus of claim 39, wherein when said existing error correction profile is determined to be unsuccessful, a record of a manual correction is logged and said existing error correction profile is updated.

41. A non-transitory computer readable apparatus comprising a medium adapted to store a computer program thereon, said computer program comprising a plurality of instructions which are configured to, when executed:

collect data which relates to interaction of a plurality of users with a plurality of content received from a plurality of platforms, said data each comprising a plurality of data fields having values contained therein capable of being validated;

compare said values of said plurality of data fields for each of said data to one or more acceptance criteria;

when said one or more criteria of said plurality of data fields are met, transmit said plurality of data to at least one entity for analysis; and

when said one or more criteria of said plurality of data fields are not met, notify an administrator that manual error correction is necessary.

42. The apparatus of claim 41, wherein said computer program is further configured to analyze said collected data to identify a frequency with which individual ones of said plurality of content was requested by a subgroup of said plurality of users over a period of time.

43. The apparatus of claim 42, wherein said collection and said analysis are performed substantially at individual ones of a plurality of service nodes.