Video content summary

• US 8,407,575 B1
• Filed: 11/26/2008
• Issued: 03/26/2013
• Est. Priority Date: 09/10/2008
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Goldman, Dan, Barnes, Connelly, Shechtman, Elya
• Primary Examiner(s): Bashore, William
• Assistant Examiner(s): Distefano, Gregory A
• Application Number:
  US12/315,038
• Time in Patent Office: 1,581 Days
• Field of Search: 345/629, 715/201, 715/204
• US Class Current: 715/204
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames

Abstract

Among other disclosed subject matter, a computer program product is tangibly embodied in a tangible program carrier and includes instructions that when executed by a processor perform a method. The method includes obtaining a plurality of layouts generated from video content, each of the layouts comprising a visual summary of the video content at a respective zoom level, each visual summary including a plurality of frames selected from the video content and blended into the corresponding layout without borders between the frames. The method includes creating an animation between the plurality of layouts so that a user can zoom continuously between the zoom levels to view the visual summaries of the video content.

Claims

1. A computer program product embodied in a non-transitory program carrier and comprising instructions that when executed by a processor perform a method comprising:

obtaining a plurality of layouts generated from video content, each of the layouts comprising a visual summary of the video content at a respective zoom level, each visual summary including a plurality of frames selected from the video content and blended into the corresponding layout without borders between the frames; and

creating an animation between the plurality of layouts so that a user can zoom continuously between the zoom levels to view the visual summaries of the video content;

wherein creating the animation comprises finding connected components that are present in both the first and second layouts, and constraining the connected components to move at a constant velocity in the animation.

2. The computer program product of claim 1, further comprising:

configuring each of the plurality of layouts to form a tapestry showing the corresponding visual summary; and

providing navigation from each tapestry to the video content such that the user can initiate presentation of the video content from a frame by selecting that frame in at least one of the tapestries.

3. The computer program product of claim 1, wherein obtaining the plurality of layouts comprises:

selecting the plurality of frames from the video content;

assembling the plurality of frames side-by-side in a pattern; and

blending the plurality of frames into the corresponding layout, including removing borders between the frames.

4. The computer program product of claim 3, wherein the pattern includes a brickwork-like organization of the frames in at least two rows, wherein the rows are merged into one in the blending.

5. The computer program product of claim 3, wherein the blending takes into account a bidirectional similarity measure between the video content and the layouts, including a completeness criterion of each layout with regard to the video content, and a coherence criterion of each layout with regard to the video content.

6. The computer program product of claim 3, further comprising:

selecting a visual feature present in at least part of the video content; and

weighting the visual feature in performing the blending.

7. The computer program product of claim 1, wherein creating the animation comprises:

filling in spacetime gaps between the first and second layouts, the first layout being smaller than the second layout and the first zoom level being lower than the second zoom level.

8. The computer program product of claim 7, wherein filling in the spacetime gaps comprises:

identifying corresponding color regions in the layouts, including performing an approximate nearest-neighbor matching of patches from the first layout to the second layout.

9. The computer program product of claim 8, further comprising:

verifying that matching color regions are stable under small perturbations, including;

(i) calculation of scalar values for points in the first layout depending on a difference in coordinates between nearest-neighbor patches;

(ii) selecting patches of the first layout based on the scalar values and at least one threshold value; and

(iii) generating at least one additional layout using the selected patches, the additional layout filling in the spacetime gap between the first and second layouts.

10. The computer program product of claim 9, wherein the selected patches form island images of color data that are present at different positions in both the first and second layouts, the method further comprising:

defining time values corresponding to the animation from the second layout to the first layout; and

for each of the time values;

(i) resizing the second layout so that its image width changes linearly from a second-layout width to a first-layout width;

(ii) inserting constrained color data for the island images; and

(iii) applying an iterative update rule to patches until convergence.

11. The method of claim 1, wherein creating the animation further comprises filling in color data outside the connected components so that as the user zooms continuously between the first and second zoom levels the visual summary changes continuously between the first and second layouts.

12. A method comprising:

executing instructions on a specific apparatus so that binary digital electronic signals representing a plurality of layouts generated from video content are obtained, each of the layouts comprising a visual summary of the video content at a respective zoom level, each visual summary including a plurality of frames selected from the video content and blended into the corresponding layout without borders between the frames;

executing instructions on the specific apparatus so that binary digital electronic signals representing an animation between the plurality of layouts are created, so that a user can zoom continuously between the zoom levels to view the visual summaries of the video content;

wherein creating the animation comprises finding connected components that are present in both the first and second layouts, and constraining the connected components to move at a constant velocity in the animation; and

storing the resulting binary digital electronic signals representing the animation between the plurality of layouts in a location in memory of said specific apparatus for later use.

13. A system comprising:

one or more processors; and

computer program product with instructions that when executed perform a method comprising;

obtaining a plurality of layouts generated from video content, each of the layouts comprising a visual summary of the video content at a respective zoom level, each visual summary including a plurality of frames selected from the video content and blended into the corresponding layout without borders between the frames; and

creating an animation between the plurality of layouts so that a user can zoom continuously between the zoom levels to view the visual summaries of the video content;

wherein creating the animation comprises finding connected components that are present in both the first and second layouts, and constraining the connected components to move at a constant velocity in the animation.