Systems and methods for spacial and temporal experimentation on content effectiveness

• US 10,142,685 B2
• Filed: 01/28/2015
• Issued: 11/27/2018
• Est. Priority Date: 01/29/2014
• Status: Active Grant

• Current Assignee: 3M Innovative Properties Company (3M Company)
• Original Assignee: 3M Innovative Properties Company (3M Company)
• Inventors: Arsenault, Frederick J., Brooks, Brian E.
• Primary Examiner(s): Kumar, Pankaj
• Assistant Examiner(s): Newlin, Timothy R
• Application Number:
  US15/114,440
• Publication Number:
  US 20160353166A1
• Time in Patent Office: 1,399 Days
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: H04N 21/251 Learning process for intelligent management, e.g. learning user preferences for recommending movies details of learning user preferences for the retrieval of video data in a video database G06F16/739; computer systems using learning methods G06N3/08
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45

Abstract

Systems and methods for organizing and controlling the display of content, then measuring the effectiveness of that content in modifying behavior, within a particular temporal and special dimension, so as to minimize or eliminate confounding effects.

Claims

1. A computer-implemented method for controlling the display of content on a content delivery channel to mitigate potential confounds and structure an experiment for the content delivery channel, comprising:

receiving temporal reach data for the content delivery channel;

receiving spatial reach data for the channel for content delivery, wherein the spatial reach data relates to the region in which a content recipient will likely take action after receiving content from the content delivery channel;

receiving content data comprising information about a plurality of different pieces of content;

receiving confound data for at least some of the plurality of different pieces of content, wherein the content comprises rendered files, instructions for procedural generation of content, rules constraining content creation, or elements for use in content creation or percentages of play for content pieces;

receiving randomization constraint data wherein the randomization constraint data comprises at least one of blocking, balancing and counterbalancing;

using a processor of a computer, computing an experimental unit comprising a response area based on the spatial reach data and a response duration based on the temporal reach data, wherein the response area is computed by doubling the data that defines the spatial reach data;

using the processor, assigning a first piece of content from the plurality of different pieces of content, to the experimental unit using the randomization constraint data; and

,

using the processor, assigning additional pieces of content from the plurality of different pieces of content to the experimental unit using the randomization constraint data, wherein the additional pieces of content do not confound the first piece of content according to the received confound data; and

distributing the first piece of content and the additional pieces of content to the content delivery channel.

2. The computer-implemented method of claim 1, further comprising:

causing the selected content to be displayed on the channel for content delivery consistent with the experimental unit.

3. The computer-implemented method of claim 2, further comprising:

receiving response data indicative of the effects of the displayed content.

4. The computer-implemented method of claim 3, further comprising:

determining the effectiveness of the displayed content.

5. The computer-implemented method of claim 4, wherein determining comprises thinning the response data to create a subset of response data that are unconfounded by spacial and temporal carryover effects.

6. The computer-implemented method of claim 5, wherein analyzing the subset of response data using statistical computing rules.

7. The computer-implemented method of claim 1, further comprising:

receiving behavior data within the response area of the experimental unit during at least a portion of the response duration.

8. The computer-implemented method of claim 7, wherein the collecting of behavior data is associated with a location other than the location associated with the presenting of the displayed content.

9. The computer-implemented method of claim 4, further comprising:

parsing the received effectiveness data by dividing it up by the time and location and types of content being displayed and experimental condition.

10. The computer-implemented method of claim 9, wherein experimental condition additionally comprises the required state of an intermediate metric for each of the channels of content display.

11. The computer-implemented method of claim 1, wherein the content delivery channel comprises fixed-location digital displays, digital billboards, mobile devices, or web pages.

12. The computer-implemented method of claim 1, wherein the response duration is computed by doubling the data that defines the temporal reach data.

13. The computer-implemented method of claim 3, wherein response data is collected during the second half of the response duration.

14. The computer-implemented method of claim 1, wherein the randomization constraint data reference the content in other experimental units to implement balancing and counterbalancing.

15. A computer-implemented system for displaying content in accordance with temporal and spatial experimental units to mitigate potential confounds and structure an experiment for a content delivery channel, comprising:

at least one content delivery channel, comprising a plurality of displays;

a computer data store having temporal reach factor data and a spatial reach factor data for the content delivery channel, wherein the spatial reach data relates to the region in which a content recipient will likely take action after receiving content from the content delivery channel;

information defining a plurality of different content pieces, wherein the information comprises rendered files, instructions for procedural generation of content, rules constraining content creation, or elements for use in content creation or percentages of play for content pieces;

a processor communicatively coupled to the data store and configured to execute instructions that;

define an experimental unit comprising a response area based on the spatial reach data and a response duration, wherein the response area is computed by doubling the data that defines the spatial reach data;

associate channels for content delivery with experimental units; and

associate at least some of the content pieces with the experimental units using randomization constraint data, wherein the randomization constraint data comprises at least one of blocking, balancing and counterbalancing, such that pieces of content that confound one another are not assigned to the same experimental unit

distributing the first piece of content and the additional pieces of content to the content delivery channel.

16. The system of claim 15 wherein the content delivery channel comprises fixed-location digital displays, digital billboards, mobile devices, or web pages.

17. The system of claim 15, further comprising sensors, communicatively coupled to the processor, for collecting behavior data.

18. The system of claim 15, wherein the sensors for collecting behavior data are located at locations other than those associated with the content delivery channel.

19. The system of claim 15, wherein the processor is further configured to execute instructions that cause the processor to receive behavior-related data associated with an experimental unit, and parse the data based on the location of the experimental unit and the time that the experimental unit started or stopped.

20. The system of claim 17, wherein the sensors for collecting behavior data comprise intermediate variable sensors that are particular to the content delivery channel, and ultimate variable sensors configured to measure a variable influenced by a plurality of content delivery channels.