Location for secondary content based on data differential

• US 8,079,054 B1
• Filed: 04/14/2008
• Issued: 12/13/2011
• Est. Priority Date: 04/14/2008
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Dhawan, Anmol, Sahai, Ganesh, Soni, Sachin
• Primary Examiner(s): Huynh, Son P
• Application Number:
  US12/102,194
• Time in Patent Office: 1,338 Days
• Field of Search: 725/32, 725 34- 36, 725/109, 725/110, 725/112, 705/14, 705/14.4, 705/14.41, 705/14.49, 705/14.53, 705/14.6, 705/14.66, 705/14.67
• US Class Current: 725/105
• CPC Class Codes: G06Q 30/00 Commerce
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G06Q 30/0242 Determining effectiveness of advertisements
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 30/0255 based on user history
• CPC Class Codes: G06Q 30/0263 based upon Internet or website rating
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02

Abstract

Methods and apparatus provide for a Content Inserter to determine location(s) for secondary content in streaming video. The Content Inserter receives a set of video frames and analyzes data differential(s) occurring in the set of video frames to identify secondary content locations between adjacent video frames. The Content Inserter inserts secondary content at the location(s) for secondary content. A data differential can be a change in subtitle information between video frames, a change in audio data between video frames, a difference in content ratings between regions of video frames, and/or difference in activity levels between regions of video frames. Activity levels correspond to changes in pixel color data of video frames. Thus, the Content Inserter identifies secondary content locations based on characteristics of each video stream and ensures that inserted secondary content is placed in a video stream so that the secondary content will most likely be viewed.

Claims

1. A method comprising:

receiving, at a computerized system, a set of video frames, wherein receiving the set of video frames includes receiving metadata associated with content of the set of video frames, wherein receiving metadata associated with content of the set of video frames includes receiving a content rating for each video frame in the set of video frames, the content rating providing past viewer reaction for each video frame;

analyzing, by the computer system performing logic instructions, at least one data differential occurring in the set of video frames to determine at least one location for secondary content, wherein analyzing the at least one data differential comprises detecting at least one change in the metadata, the at least one change in the metadata indicating a change in the content, wherein detecting the at least one change in the metadata includes evaluating a difference of content rating between a first region of video frames and at least one other region of video frames, the first region of video frames and the at least one other region of video frames comprising;

i) a same number of video frames; and

ii) at least one different video frame;

associating, at the computer system, secondary content at the at least one location for secondary content, the at least one location for secondary content positioned between two adjacent video frames in the set of video frames.

2. The method as in claim 1, wherein:

receiving metadata associated with content of the set of video frames includes receiving subtitle information for at least two video frames; and

wherein detecting the at least one change in the metadata includes detecting at least one change in the subtitle information.

3. The method as in claim 2, wherein detecting the at least one change in the subtitle information includes:

detecting the at least one change in the subtitle information between a first segment of video frames and a second segment of video frames, the first and second segment of video frames each comprising at least one video frame;

identifying the at least one location for secondary content as one of;

i) before the first segment of video frames;

ii) after the second segment of video frames; and

iii) between the first segment of video frames and second segment of video frames.

4. The method as in claim 2, wherein detecting the at least one change in the subtitle information includes:

detecting at least one beginning of presentation of the subtitle information;

detecting at least one termination of presentation of the subtitle information;

identifying the at least one location for secondary content as one of;

i) before the at least one beginning of presentation of the subtitle information; and

ii) after the at least one termination of presentation of the subtitle information.

5. The method as in claim 1, wherein evaluating the difference of content rating between the first region of video frames and the at least one other region of video frames includes:

for each of the first region of video frames and the at least one other region of video frames;

aggregating the content rating of each video frame in a region of video frames; and

creating a rating density for the region of video frames by dividing the aggregated content rating for the region of video frames by the same number of video frames.

6. The method as in claim 5, further comprising:

upon creating the rating density for each of the first region of video frames and the at least one other region of video frames;

creating a ranking of each rating density associated with the first region of video frames and the at least one other region of video frames;

based on the ranking, determining a highest-ranked region of video frames, defining at least one first location for secondary content in the highest-ranked region of video frames; and

based on the ranking, determining at least one lower-ranked region of video frames, defining at least one additional location for secondary content in each lower-ranked region of video frames.

7. The method as in claim 6, wherein associating secondary content at the at least one location for secondary content includes:

inserting secondary content with respect to the at least one first location for secondary content in the highest-ranked region of video frames, an amount of secondary content inserted with respect to the at least one first location for secondary content proportionally determined according to a ratio between a rating density associated with the highest-ranked region of video frames and a rating density for each lower-ranked region; and

for each lower-ranked region of video frames;

inserting secondary content with respect to the at least one additional location for secondary content in the lower-ranked region of video frames;

an amount of secondary content inserted with respect to the at least one additional location for secondary content proportionally determined according to the ratio between the rating density associated with the highest-ranked region of video frames and the rating density for each lower-ranked region.

8. The method as in claim 1, further comprising:

wherein receiving the set of video frames includes receiving audio data associated with content of the set of video frames;

wherein analyzing the at least one data differential occurring in the set of video frames to determine the at least one location for secondary content includes;

detecting at least one change in the audio data, the at least one change in the audio data comprising an amount of audio data that is greater than a threshold level of audio data, the at least one change in the audio data associated with a plurality of video frames from the set of video frames; and

defining the at least one location for secondary content as occurring in the plurality of video frames that includes the at least one change in audio data.

9. The method as in claim 1, further comprising:

wherein receiving the set of video frames includes receiving pixel color data associated with each video frame in the set of video frames;

wherein analyzing at least one data differential occurring in the set of video frames to determine at least one location for secondary content includes;

partitioning the set of video frames into a first region of video frames and at least one other region of video frames, the first region of video frames and the at least one other region of video frames comprising i) a same number of video frames and ii) at least one different video frame; and

for every video frame and a corresponding preceding video frame in the first region of video frames and the at least one other region of video frames;

detecting a difference in the pixel color data associated with the video frame and the pixel color data associated with the corresponding preceding video frame to determine a frame color differential between the video frame and the corresponding preceding video frame.

10. The method as in claim 9, further comprising:

for the first region of video frames and the at least one other region of video frames;

aggregating the frame color differential between each video frame and the corresponding preceding video frame to determine a color differential sum for a region of video frames;

dividing the color differential sum by a same number to determine an activity level for the region of video frames; and

upon determining the activity level for the region of video frames is greater than a threshold level, identifying the region of video frames as suitable for the at least one location for secondary content; and

from a group including each region of video frames identified as suitable for the at least one location for secondary content, determining the at least one location for secondary content occurs in a region of video frames with a maximum activity level.

11. A non-transitory computer readable medium comprising executable instructions encoded thereon operable on a computerized device to perform processing comprising:

instructions for receiving a set of video frames, wherein receiving the set of video frames includes receiving metadata associated with content of the set of video frames, wherein the instructions for receiving metadata associated with content of the set of video frames include instructions for receiving a content rating for each video frame in the set of video frames, the content rating providing past viewer reaction for each video frame;

instructions for analyzing at least one data differential occurring in the set of video frames to determine at least one location for secondary content, wherein analyzing the at least one data differential comprises detecting at least one change in the metadata, the at least one change in the metadata indicating a change in the content, wherein the instructions for detecting the at least one change in the metadata include instructions for evaluating a difference of content rating between a first region of video frames and at least one other region of video frames, the first region of video frames and the at least one other region of video frames comprising;

i) a same number of video frames; and

ii) at least one different video frame;

instructions for inserting secondary content into the set of video frames at the at least one location for secondary content, the at least one location for secondary content occurring between two adjacent video frames in the set of video frames.

12. The non-transitory computer readable medium as in claim 11, further comprising:

wherein the instructions for receiving metadata associated with content of the set of video frames include instructions for receiving subtitle information for at least two video frames; and

wherein the instructions for detecting the at least one change in the metadata include instructions for detecting at least one change in the subtitle information.

13. The non-transitory computer readable medium as in claim 12, wherein the instructions for detecting the at least one change in the subtitle information include:

instructions for detecting the at least one change in the subtitle information between a first segment of video frames and a second segment of video frames, the first and second segment of video frames each comprising at least one video frame;

instructions for identifying the at least one location for secondary content as one of;

i) before the first segment of video frames;

ii) after the second segment of video frames; and

iii) between the first segment of video frames and second segment of video frames.

14. The non-transitory computer readable medium as in claim 12, wherein the instructions for detecting the at least one change in the subtitle information include:

instructions for detecting at least one beginning of presentation of the subtitle information;

instructions for detecting at least one termination of presentation of the subtitle information;

instructions for identifying the at least one location for secondary content as one of;

i) before the at least one beginning of presentation of the subtitle information; and

ii) after the at least one termination of presentation of the subtitle information.

15. The non-transitory computer readable medium as in claim 11, wherein the instructions for evaluating the difference of content rating between the first region of video frames and the at least one other region of video frames include:

for each of the first region of video frames and the at least one other region of video frames;

instructions for aggregating the content rating of each video frame in a region of video frames; and

instructions for creating a rating density for the region of video frames by dividing the aggregated content rating for the region of video frames by the same number of video frames.

16. The non-transitory computer readable medium as in claim 15, further comprising:

upon creating the rating density for each of the first region of video frames and the at least one other region of video frames;

instructions for creating a ranking of each rating density associated with the first region of video frames and the at least one other region of video frames;

based on the ranking, instructions for determining a highest-ranked region of video frames, defining at least one first location for secondary content in the highest-ranked region of video frames; and

based on the ranking, instructions for determining at least one lower-ranked region of video frames, defining at least one additional location for secondary content in each lower-ranked region of video frames.

17. The non-transitory computer readable medium as in claim 16, wherein the instructions for inserting secondary content into the set of video frames at the at least one location for secondary content include:

instructions for inserting secondary content into the at least one first location for secondary content in the highest-ranked region of video frames, an amount of secondary content inserted into the at least one first location for secondary content proportionally determined according to a ratio between a rating density associated with the highest-ranked region of video frames and a rating density for each lower-ranked region; and

for each lower-ranked region of video frames;

instructions for inserting secondary content into the at least one additional location for secondary content in the lower-ranked region of video frames;

an amount of secondary content inserted into the at least one additional location for secondary content proportionally determined according to the ratio between the rating density associated with the highest-ranked region of video frames and the rating density for each lower-ranked region.

18. The non-transitory computer readable medium as in claim 11, further comprising:

wherein the instructions for receiving the set of video frames include instructions for receiving audio data associated with content of the set of video frames;

wherein the instructions for analyzing the at least one data differential occurring in the set of video frames to determine the at least one location for secondary content include;

instructions for detecting at least one change in the audio data, the at least one change in the audio data comprising an amount of audio data that is greater than a threshold level of audio data, the at least one change in the audio data associated with a plurality of video frames from the set of video frames; and

instructions for defining the at least one location for secondary content as occurring in the plurality of video frames that includes the at least one change in audio data.

19. The non-transitory computer readable medium as in claim 11, further comprising:

wherein the instructions for receiving the set of video frames include instructions for receiving pixel color data associated with each video frame in the set of video frames;

wherein the instructions for analyzing at least one data differential occurring in the set of video frames to determine at least one location for secondary content include;

instructions for partitioning the set of video frames into a first region of video frames and at least one other region of video frames, the first region of video frames and the at least one other region of video frames comprising i) a same number of video frames and ii) at least one different video frame; and

for every video frame and a corresponding preceding video frame in the first region of video frames and the at least one other region of video frames;

instructions for detecting a difference in the pixel color data associated with the video frame and the pixel color data associated with the corresponding preceding video frame to determine a frame color differential between the video frame and the corresponding preceding video frame.

20. The non-transitory computer readable medium as in claim 19, further comprising:

for the first region of video frames and the at least one other region of video frames;

instructions for aggregating the frame color differential between each video frame and the corresponding preceding video frame to determine a color differential sum for a region of video frames;

instructions for dividing the color differential sum by the same number to determine an activity level for the region of video frames; and

upon determining the activity level for the region of video frames is greater than a threshold level, instructions for identifying the region of video frames as suitable for the at least one location for secondary content; and

from a group including each region of video frames identified as suitable for the at least one location for secondary content, instructions for determining the at least one location for secondary content occurs in a region of video frames with a maximum activity level.

21. The non-transitory computer readable medium as in claim 12, wherein the instructions for detecting the at least one change in the subtitle information includes:

instructions for partitioning the set of video frames into a first region of video frames and at least one other region of video frames, the first region of video frames and the at least one other region of video frames comprising i) a same number of video frames and ii) at least one different video frame; and

for each of the first region of video frames and the at least one other region of video frames;

instructions for aggregating the subtitle information of each video frame in a region of video frames;

instructions for creating a subtitle density for the region of video frames by dividing the aggregated subtitle information for the region of video frames by the same number of video frames.

22. The non-transitory computer readable medium as in claim 21, further comprising:

upon creating the subtitle density for each of the first region of video frames and the at least one other region of video frames;

instructions for creating a ranking of each subtitle density associated with the first region of video frames and the at least one other region of video frames;

instructions for determining a highest-ranked region of video frames and defining at least one first location for secondary content in the highest-ranked region of video frames based on the ranking, the at least one first location for secondary content proportionally determined according to a ratio between a subtitle density associated with the highest-ranked region of video frames and a subtitle density for each lower-ranked region; and

instructions for determining at least one lower-ranked region of video frames based on the ranking and defining at least one additional location for secondary content in each lower-ranked region of video frames according to the ratio.

23. A computer system comprising:

a processor;

a memory unit that stores instructions associated with an application executed by the processor; and

an interconnect coupling the processor and the memory unit, enabling the computer system to execute the application and perform operations of;

receiving a set of video frames, wherein receiving the set of video frames includes receiving metadata associated with content of the set of video frames, wherein receiving metadata associated with content of the set of video frames includes receiving a content rating for each video frame in the set of video frames, the content rating providing past viewer reaction for each video frame;

analyzing at least one data differential occurring in the set of video frames to determine at least one location for secondary content, wherein analyzing the at least one data differential comprises detecting at least one change in the metadata, the at least one change in the metadata indicating a change in the content, wherein detecting the at least one change in the metadata includes evaluating a difference of content rating between a first region of video frames and at least one other region of video frames, the first region of video frames and the at least one other region of video frames comprising;

i) a same number of video frames; and

ii) at least one different video frame;

inserting secondary content into the set of video frames at the at least one location for secondary content, the at least one location for secondary content occurring between two adjacent video frames in the set of video frames.