METHOD AND SYSTEM FOR PRODUCING A VIDEO SYNOPSIS

• US 20090219300A1
• Filed: 11/15/2006
• Published: 09/03/2009
• Est. Priority Date: 11/15/2005
• Status: Active Grant

• Current Assignee: BriefCam, Ltd. (Canon Inc.)
• Original Assignee: Yissum Research Development Company of the Hebrew University of Jerusalem Ltd. (Hebrew University of Jerusalem)
• Inventors: Rav-Acha, Alexander, Peleg, Shmuel
• Granted Patent: US 8,102,406 B2
• Time in Patent Office: Days
• US Class Current: 345/630
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06T 3/16 Spatio-temporal transformations, e.g. video cubism
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 5/2625 for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect sequence generated by event triggered capturing H04N7/188

Abstract

A computer-implemented method and system transforms a first sequence of video frames of a first dynamic scene to a second sequence of at least two video frames depicting a second dynamic scene. A subset of video frames in the first sequence is obtained that show movement of at least one object having a plurality of pixels located at respective x, y coordinates and portions from the subset are selected that show non-spatially overlapping appearances of the at least one object in the first dynamic scene. The portions are copied from at least three different input frames to at least two successive frames of the second sequence without changing the respective x, y coordinates of the pixels in the object and such that at least one of the frames of the second sequence contains at least two portions that appear at different frames in the first sequence.

Claims

1. A method for creating a synopsis video by transforming a source sequence of video frames of a first dynamic scene captured at regular time intervals to a shorter synopsis sequence of video frames depicting a second dynamic scene, the method comprising:

(a) source sequence that show movement of at least one object, each source object being a connected subset of image points from at least three different frames of the source video;

(b) selecting from said subset portions that show non-spatially overlapping appearances of the at least one object in the source sequence; and

(c) copying said portions from at least three different source frames to at least two successive frames of the synopsis sequence without changing the image location of said portions and such that at least one of the frames of the synopsis sequence contains at least two portions that appear at different frames in the source sequence.

2. The method according to claim 1, wherein the at least two portions relate to a common object.

3. The method according to claim 1, wherein selecting portions from said subset includes determining an interest score.

5. The method according to claim 1, wherein the synopsis sequence contains all objects of interest in the source sequence.

7. The method according to claim 1, wherein the source sequence is captured by a camera that is rotated relative to an axis at a fixed location and including spatially warping at least two of said portions prior to copying to the synopsis sequence.

9. The method according to claim 1, wherein the at least three different frames of the source sequence are temporally contiguous.

10. The method according to claim 1, including spatially warping at least two of said portions prior to copying to the second sequence.

12. The method according to claim 1, including pre-aligning the first sequence of video frames so as to produce an aligned space-time volume by:

(a) computing image motion parameters between frames in the first sequence;

(b) warping the video frames in the first sequence so that stationary objects in the first dynamic scene will be stationary in the video.

15. The method according to claim 1, wherein two events that occur simultaneously in the source video sequence are displayed at different times in the synopsis video sequence.

16. The method according to claim 1, being used for video synopsis.

17. The method according to claim 16, being used for surveillance.

18. The method according to claim 16, being used for increasing activity density of a movie.

19. The method according to claim 16, being used for video indexing.

20. The method according to claim 19, including maintaining for each pixel in the synopsis sequence a pointer to a corresponding pixel in the source sequence.

28. A computer program comprising computer program code means for performing the method of claim 1 when said program is run on a computer.

29. A computer program as claimed in claim 28 embodied on a computer readable medium.

4. (canceled)

6. (canceled)

8. (canceled)

11. (canceled)

13. (canceled)

14. (canceled)

21. A data carrier tangibly embodying a sequence of synopsis video frames depicting a dynamic scene, at least two successive frames of said synopsis video frames comprising a plurality of pixels being derived from portions of a source object being a connected subset of image points from at least three different frames of the source video without changing the respective image location of said object and such that at least one of the synopsis video frames contains at least two portions that appear at different source frames.

22. A system for transforming a source sequence of video frames of a first dynamic scene to a synopsis sequence of at least two video frames depicting a second dynamic scene, the system comprising:

a first memory for storing a subset of video frames in said source sequence that show movement of at least one object being a connected subset of image points from at least three different frames of the source video,

a selection unit coupled to the first memory for selecting from said subset portions that show non-spatially overlapping appearances of the at least one object in the source video,

a frame generator for copying said portions from at least three different input frames to at least two successive frames of the synopsis sequence without changing the image location of said object and such that at least one of the frames of the second sequence contains at least two portions that appear at different frames in the first sequence,

a second memory for storing frames of the synopsis sequence, and

means for coupling a display device to the second memory for displaying the second dynamic scene.

25. The system according to claim 22, wherein the frame generator includes a warping unit for spatially warping at least two of said portions prior to copying to the second sequence.

23. (canceled)

24. (canceled)

26. (canceled)

27. The system according to claim 26, including a time slice generator coupled to the selection unit for sweeping the aligned space-time volume by a “

time front”

surface and generating a sequence of time slices.