APPARATUS AND METHOD FOR OPTIMIZING ON-SCREEN LOCATION OF ADDITIONAL CONTENT OVERLAY ON VIDEO CONTENT

• US 20110321084A1
• Filed: 06/24/2011
• Published: 12/29/2011
• Est. Priority Date: 06/25/2010
• Status: Active Grant

• Current Assignee: KDDI Corporation
• Original Assignee: KDDI Corporation
• Inventors: Sugano, Masaru, Sakazawa, Shigeyuki, Takahashi, Tomohiko
• Granted Patent: US 8,849,063 B2
• Time in Patent Office: Days
• US Class Current: 725/32
• CPC Class Codes: H04N 5/2723 Insertion of virtual advertisement; Replacing advertisements physical present in the scene by virtual advertisement data processing systems or methods specially adapted for marketing G06Q30/02

Abstract

A method of providing a video content formed of a plurality of successive video frames is disclosed. The method includes: extracting a visual feature from an image of each of subject frames which are all or some of the plurality of video frames; and selecting one of a plurality of candidate overlay insertion regions which are pre-assigned on each of the subject frames at different positions for allowing an additional content as a content overlay to be overlaid on the subject frames for display, as a final overlay insertion region, based on distances of the plurality of candidate overlay insertion regions from the extracted feature.

Claims

1. An apparatus for providing a video content formed of a plurality of successive video frames, comprising:

a feature extractor configured to extract a visual feature from an image of each of subject frames which are all or some of the plurality of video frames; and

an overlay insertion region selector configured to select one of a plurality of candidate overlay insertion regions which are pre-assigned on each of the subject frames at different positions for allowing an additional content as a content overlay to be overlaid on each subject frame for display, as a final overlay insertion region, such that, on each subject frame, the final overlay insertion region has a distance from the extracted feature that is substantially the longest among distances of the plurality of candidate overlay insertion regions from the extracted feature.

2. The apparatus according to claim 1, wherein the overlay insertion region selector is configured to calculate, for each subject frame, a distance of the extracted feature from each candidate overlay insertion region, weighted using a weighting factor ω

which is equal to or larger than “

0” and

equal to or smaller than “

1,”

to thereby allow a pre-designated one of the plurality of candidate overlay insertion regions to be more likely to be selected as the final overlay insertion region than any other candidate overlay insertion region.

3. The apparatus according to claim 1, wherein the distance is defined as a distance between a centroid of each candidate overlay insertion region and a centroid of the extracted feature, or a distance between a circumferential edge of each candidate overlay insertion region and the centroid of the extracted feature.

4. The apparatus according to claim 1, wherein the feature includes a plurality of features of each subject frame,

the overlay insertion region selector is configured to calculate, for each candidate overlay insertion region, a plurality of distances of the plurality of features from the each candidate overlay insertion region, respectively,

to select, for each candidate overlay insertion region, one of the plurality of features which has the shortest one of the plurality of distances calculated, as a nearest feature, and

to select one of the plurality of candidate overlay insertion regions which has the longest one of a plurality of distances from the nearest feature, as the final overlay insertion region.

5. The apparatus according to claim 1, wherein the additional content includes a plurality of additional contents to be overlaid on the video content, and

the overlay insertion region selector configured to select one of the plurality of candidate overlay insertion regions in decreasing order of the distance from the extracted feature, in succession, such that a successively-selected one of the plurality candidate overlay insertion regions serves as the final overlay insertion region for a successively-selected one of the plurality of additional contents.

6. The apparatus according to claim 1, wherein the additional content includes a plurality of additional contents to be overlaid on the video content,

the apparatus further comprises an additional content selector configured to select, at least one valid additional content, at least one of the plurality of additional contents, such that a degree of similarity in visual characteristic between an image of the video content and an image of the selected at least one valid additional content satisfies a predetermined condition, and

the overlay insertion region selector is configured to select one of the plurality of candidate overlay insertion regions as the final overlay insertion region for the selected at least one valid additional content.

7. The apparatus according to claim 6, wherein the visual characteristic is defined as color tone of an image.

8. The apparatus according to claim 6, wherein the predetermined condition is defined so as to be satisfied if the degree of similarity is higher than a predetermined similarity level, is defined so as to be satisfied if the degree of similarity is lower than a predetermined similarity level, or is defined so as to be satisfied if the degree of similarity falls within a predetermined range.

9. The apparatus according to claim 1, wherein the plurality of video frames are segmented into a plurality of successive slots each in a temporal array of a sub-plurality of successive video frames,

the apparatus further comprises a candidate overlay insertion slot selector configured to select at least one of the plurality of successive slots as at least one candidate overlay insertion slot each of which is a candidate of a slot into which the additional content is finally inserted, and

the candidate overlay insertion slot selector is configured to exclude at least one of the plurality of successive slots, if the at least one slot is not suitable for use in overlay of the additional content on the video content,

to measure a visual and/or audio characteristic value of each slot,

to select at least one of the remaining slots as the at least one candidate overlay insertion slot, if the at least one slot has the visual and/or audio characteristic value measured so as to be equal to or lower than a predetermined threshold, and

to output the selected at least one candidate overlay insertion slot to the feature extractor.

10. The apparatus according to claim 9, wherein the feature extractor is configured to select, for each candidate overlay insertion slot, a plurality of provisional representative frames from the sub-plurality of successive video frames forming each candidate overlay insertion slot,

to extract a feature from an image of each provisional representative frame,

to select, for each candidate overlay insertion slot, one of the plurality of provisional representative frames which has a total area of the extracted feature that is the largest among total areas of features of the plurality of provisional representative frames, as a final representative frame, wherein, for each provisional representative frame, if the extracted feature includes only one feature, the total area is equal to an area of the only one feature, while if the extracted feature includes a plurality of features, the total area is equal to a sum of areas of the plurality of features, and

to exclude at least one of the plurality of candidate overlay insertion slots, which corresponds to the final representative frame that has a ratio of the total area of the extracted feature, to an area for display of the final representative frame.

11. The apparatus according to claim 1, wherein the feature extractor is configured to extract the feature from each subject frame, by at least one of the followings:

facial recognition,

slot into which extraction,

motion recognition using a motion attention model,

extraction of Region of Interest (ROI),

extraction of interest points,

feature extraction based on a total energy level of alternate-current high-frequency components of Discrete Cosine Transform (DCT) coefficients, and

feature extraction based on luminance/saturation.

12. The apparatus according to claim 1, wherein the video content includes digital broadcast video data to be delivered from a broadcast center to a plurality of potential or actual viewers,

the additional content includes an advertising content as an advertisement overlay to be delivered from an advertisement distribution server to the plurality of viewers, and

the apparatus further comprises an advertising content storage device, locally disposed for each viewer, and configured to store therein the advertising content received from the advertisement distribution server.

13. The apparatus according to claim 12, wherein the advertising content storage device stores therein a plurality of advertising contents, in association with a plurality of sets of information corresponding to the plurality of viewers, and

the apparatus further comprises an advertising content selector, locally disposed for each viewer, and configured to select at least one of the plurality of advertising contents stored in the advertising content storage device, based on the information corresponding to each viewer who has received the video content.

14. The apparatus according to claim 1, further comprising:

an additional content overlay insertion section configured to insert the additional content into the video content, so that the video content is displayed such that the additional content is overlaid on the selected final overlay insertion region, to thereby modify the video content, and

a video content output section configured to output the modified video content.

15. A method of providing a video content formed of a plurality of successive video frames, comprising:

extracting a visual feature from an image of each of subject frames which are all or some of the plurality of video frames; and

selecting one of a plurality of candidate overlay insertion regions which are pre-assigned on each of the subject frames at different positions for allowing an additional content as a content overlay to be overlaid on each subject frame for display, as a final overlay insertion region, such that, on each subject frame, the final overlay insertion region has a distance from the extracted feature that is substantially the longest among distances of the plurality of candidate overlay insertion regions from the extracted feature.

16. The method according to claim 15, further comprising:

inserting the additional content into the video content, so that the video content is displayed such that the additional content is overlaid on the selected final overlay insertion region, to thereby modify the video content, and

outputting the modified video content.

17. A method of providing a video content formed of a plurality of successive video frames, comprising:

extracting a visual feature from an image of each of subject frames which are all or some of the plurality of video frames; and

selecting one of a plurality of candidate overlay insertion regions which are pre-assigned on each of the subject frames at different positions for allowing an additional content as a content overlay to be overlaid on the subject frames for display, as a final overlay insertion region, based on distances of the plurality of candidate overlay insertion regions from the extracted feature.