Apparatus and method for identifying insertion zones in video material and for inserting additional material into the insertion zones

• US 9,477,965 B2
• Filed: 11/25/2014
• Issued: 10/25/2016
• Est. Priority Date: 05/28/2008
• Status: Active Grant

• Current Assignee: Mirriad Advertising PLC
• Original Assignee: Mirriad Advertising Limited
• Inventors: Popkiewicz, Mark, Mrsic-Flogel, Janko, McLauchlan, Philip
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): SMITH, STEPHEN R
• Application Number:
  US14/553,893
• Publication Number:
  US 20150078733A1
• Time in Patent Office: 700 Days
• Field of Search: 386/278, 386248-250, 725 32- 36, 725/42, 348/584, 382/209, 382/282, 705/14.73, 705/113
• US Class Current: 1/1
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

An apparatus and method for automatic detection of insertion zones within pre-recorded video material are provided. The apparatus includes a video analysis unit configured to automatically determine at least one candidate insertion zone within the pre-recorded video material suitable for receiving additional material to be inserted and configured to generate zone meta data defining attributes of the insertion zone, and an insertion module configured to receive additional material for insertion and arranged to generate an output representative of the pre-recorded video material with the additional material placed on an insertion zone such that the additional material adopts the image attributes of the insertion zone as the pre-recorded video material is played. The automatic determination of at least one insertion zone is based on one or more of feature shape, size, duration, movement, color.

Claims

1. An apparatus for inserting additional material into pre-recorded video material, the apparatus configured to:

determine one or more candidate insertion zones within the pre-recorded video material, the candidate insertion zones identifying portions of frames of the pre-recorded video material into which additional material may be inserted;

identify a first scene of the pre-recorded video material in which a selected insertion zone appears, the selected insertion zone being one of the one or more candidate insertion zones;

identify a second scene in which the selected insertion zone appears, wherein the first scene and the second scene are separated by a third scene in the pre-recorded video material in which the selected insertion zone does not appear;

receive additional material for insertion into the selected insertion zone in the pre-recorded video material;

generate insertion zone metadata for the selected insertion zone, the insertion zone metadata comprising;

attributes of the selected insertion zone;

timing data defining when the additional material is to be inserted into the selected insertion zone; and

manipulation instructions for manipulating the additional material; and

generate output video material, in response to processing the insertion zone metadata, the output video material including the pre-recorded video material and the additional material inserted into the selected insertion zone in at least the first scene and the second scene of the pre-recorded video.

2. The apparatus of claim 1, wherein the timing data includes a start point and a stop point defining when the additional material is to be inserted into the selected insertion zone.

3. The apparatus of claim 1, wherein manipulation instructions comprise instructions for embedding the additional material into the pre-recorded video material, and wherein the generated output video material comprises the additional material embedded into at least the first scene and the second scene of the pre-recorded video material.

4. The apparatus of claim 1, wherein the generated output material comprises:

the pre-recorded video material on a first video layer; and

the additional video material on a second video layer, wherein the second video layer is overlayed on the first video layer.

5. The apparatus of claim 1, wherein the additional video material comprises one or more of:

one or more 2D video objects;

one or more 3D video objects; and

one or more surface textures.

6. The apparatus of claim 5, wherein the one or more surface textures comprises shadows, highlights or other lighting effects.

7. The apparatus of claim 1, wherein the apparatus is configured to:

detect the presence of black bars around each frame of video material based on a variation of pixel content in the pre-recorded video material; and

remove detected black bars from the pre-recorded video material.

8. The apparatus of claim 1, wherein the apparatus is configured to:

determine a video format of the pre-recorded video material based on interlacing of the pre-recorded video material.

9. The apparatus of claim 1, wherein the apparatus is configured to:

identify occlusion zones which correspond to objects which obscure some or all of the one or more candidate insertion zones;

generate masks for the identified occlusion zones;

generate an occlusion video layer comprising the generated masks; and

overlay the pre-recorded video material with the occlusion video layer, wherein the transparency of the generated masks of the occlusion video layer can be manipulated.

10. The apparatus of claim 1, wherein the apparatus is configured to detect a scene change within the pre-recorded video by determining a breakdown in visual similarity between a group of consecutive frames of the pre-recorded video material, and wherein a scene change is determined where two sequential frames are no longer similar.

11. The apparatus of claim 1, wherein the apparatus is configured to detect a scene change within the pre-recorded video material by detecting and tracking a plurality of feature points in each frame of the pre-recorded video material, and wherein a scene change is determined between a first frame and a second frame where a large proportion of feature points in the first frame do not appear in the second frame.

12. The apparatus of claim 1, wherein the attributes of the selected insertion zone include one or more of tracking information, camera calibration information, lighting, color balance, occlusions and opacity, shadows, highlights, masking, which pixels belong to which object together with determined confidence levels, image grain, and motion blur.

13. The apparatus of claim 1, wherein the apparatus is configured to determine insertion zone metrics, for the one or more candidate insertion zones, including one or more of a size of the one or more candidate insertion zone, a duration of the one or more candidate insertion zones, and a monetary value of the one or more candidate insertion zones.

14. A method of inserting additional material into pre-recorded video material, the method comprising:

determining one or more candidate insertion zones within the pre-recorded video material, the candidate insertion zones identifying portions of frames of the pre-recorded video material into which additional material may be inserted;

identifying a first scene of the pre-recorded video material in which a selected insertion zone appears, the selected insertion zone being one of the one or more candidate insertion zones;

identifying a second scene in which the selected insertion zone appears, wherein the first scene and the second scene are separated by a third scene in the pre-recorded video material in which the selected insertion zone does not appear;

receiving additional material for insertion into the selected insertion zone in the pre-recorded video material;

generating insertion zone metadata for the selected insertion zone, the insertion zone metadata comprising;

attributes of the selected insertion zone;

timing data defining when the additional material is to be inserted into the selected insertion zone; and

manipulation instructions for manipulating the additional material; and

generating output video material, in response to processing the insertion zone metadata, the output video material including the pre-recorded video material and the additional material inserted into the selected insertion zone in at least the first scene and the second scene of the pre-recorded video.

15. The method of claim 14, wherein the timing data includes a start point and a stop point defining when the additional material is to be inserted into the selected insertion zone.

16. The method of claim 14, wherein manipulation instructions comprise instructions for embedding the additional material into the pre-recorded video material, and wherein the generated output video material comprises the additional material embedded into at least the first scene and the second scene of the pre-recorded video material.

17. The method of claim 14, wherein the generated output material comprises:

the pre-recorded video material on a first video layer; and

the additional video material on a second video layer, wherein the second video layer is overlayed on the first video layer.

18. The method of claim 14, wherein the additional video material comprises one or more of:

one or more 2D video objects;

one or more 3D video objects; and

one or more surface textures.

19. The method of claim 14, comprising:

identifying occlusion zones which correspond to objects which obscure some or all of the one or more candidate insertion zones;

generating masks for the identified occlusion zones;

generating an occlusion video layer comprising the generated masks; and

overlaying the pre-recorded video material with the occlusion video layer, wherein the transparency of the generated masks of the occlusion video layer can be manipulated.

20. A non-transitory computer-readable storage medium comprising computer-executable instructions which, when executed by a processor, cause a computing device to perform a method of inserting additional material into pre-recorded video material, the method comprising:

determining one or more candidate insertion zones within the pre-recorded video material, the candidate insertion zones identifying portions of frames of the pre-recorded video material into which additional material may be inserted;

identifying a first scene of the pre-recorded video material in which a selected insertion zone appears, the selected insertion zone being one of the one or more candidate insertion zones;

identifying a second scene in which the selected insertion zone appears, wherein the first scene and the second scene are separated by a third scene in the pre-recorded video material in which the selected insertion zone does not appear;

receiving additional material for insertion into the selected insertion zone in the pre-recorded video material;

generating insertion zone metadata for the selected insertion zone, the insertion zone metadata comprising;

attributes of the selected insertion zone;

timing data defining when the additional material is to be inserted into the selected insertion zone; and

manipulation instructions for manipulating the additional material; and

generating output video material, in response to processing the insertion zone metadata, the output video material including the pre-recorded video material and the additional material inserted into the selected insertion zone in at least the first scene and the second scene of the pre-recorded video.