Video booklet

• US 20070101268A1
• Filed: 11/01/2005
• Published: 05/03/2007
• Est. Priority Date: 11/01/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hua, Xian-Sheng, Li, Shipeng, Zhu, Cai-Zhi
• Granted Patent: US 7,840,898 B2
• Time in Patent Office: Days
• US Class Current: 715/721
• CPC Class Codes: G06F 16/7328 Query by example, e.g. a complete video frame or video sequence graphical querying G06F16/7335
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/745 the internal structure of a single video sequence
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/329 on a disc [VTOC]

Abstract

Systems and methods are described for creating a video booklet that allows browsing and search of a video library. In one implementation, each video in the video library is divided into segments. Each segment is represented by a thumbnail image. Signatures of the representative thumbnails are extracted and stored in a database. The thumbnail images are then printed into an artistic paper booklet. A user can photograph one of the thumbnails in the paper booklet to automatically play the video segment corresponding to the thumbnail. Active shape modeling is used to identify and restore the photo information to the form of a thumbnail image from which a signature can be extracted for comparison with the database.

Claims

1. A method, comprising:

dividing a video recording into segments;

deriving, for each segment, a digital thumbnail image to represent the segment;

printing each digital thumbnail image on paper to create a paper collection of thumbnail images;

receiving one of the printed thumbnail images as digital image information captured via a camera or scanner;

matching the digital image information with one of the corresponding digital thumbnail images; and

retrieving and playing the segment represented by the corresponding digital thumbnail image.

2. The method as recited in claim 1, wherein the dividing the video recording into segments further includes dividing the video recording into scenes, shots, and sub-shots based on one of discontinuities of timestamps, discontinuities of color, discontinuities of intensity, or discontinuities of camera motion.

3. The method as recited in claim 1, wherein the deriving a digital thumbnail image to represent each segment further comprises:

selecting multiple video frames from the segment to each be a candidate thumbnail image for representing the segment; and

for each candidate thumbnail image, extracting a digital signature based on an intensity distribution.

4. The method as recited in claim 3, wherein the digital signatures are used for matching the digital image information with one of the corresponding digital thumbnail images.

5. The method as recited in claim 3, wherein the deriving the digital thumbnail image to represent the segment further comprises:

selecting the candidate thumbnail image that best maximizes signature differences between the representative thumbnail and other representative thumbnails selected to represent other respective segments;

selecting the candidate thumbnail image with the highest visual quality; and

selecting the candidate thumbnail image that best represents the corresponding segment.

6. The method as recited in claim 5, further comprising storing a signature of each thumbnail image selected to represent a corresponding segment, wherein each segment is capable of being retrieved via the corresponding signature.

7. The method as recited in claim 1, wherein the deriving a digital thumbnail image to represent each segment further comprises reshaping the thumbnail image using a pre-trained shape template.

8. The method as recited in claim 7, wherein the matching the digital image information with one of the corresponding digital thumbnail images further comprises identifying a contour of the pre-trained template of the captured thumbnail image using an active shape model search.

9. The method as recited in claim 7, wherein the matching the digital image information with one of the corresponding digital thumbnail images further comprises matching based on grayscale similarities or color similarities between the digital image information and one of the digital thumbnail images.

10. The method as recited in claim 8, wherein the matching the digital image information with one of the corresponding digital thumbnail images further comprises restoring a thumbnail shape indicated by the digital image information to a pre-trained template shape of the corresponding digital thumbnail image by applying principal component analysis (PCA) to vary the thumbnail shape along one or more principal axes.

11. The method as recited in claim 8, wherein the matching the digital image information with one of the corresponding digital thumbnail images further comprises restoring the thumbnail image indicated by the digital image information by applying a perspective transform to the thumbnail image.

12. The method as recited in claim 11, further comprising extracting a signature from the restored thumbnail image to compare with signatures of the digital thumbnail images used to create the paper collection of thumbnail images.

13. The method as recited in claim 1, further comprising dividing a collection of photos into segments instead of the dividing the video recording into segments, wherein each photo in the collection of photos constitutes one of the segments.

14. The method as recited in claim 1, wherein the deriving a digital thumbnail image to represent the segment further includes finding an attractive part of a video frame of the video recording to become the center of the thumbnail image.

15. The method as recited in claim 1, wherein printing each digital thumbnail image on paper to create a paper collection of thumbnail images further includes using one or more booklet layout templates to create an artistic layout of the paper collection of thumbnail images.

16. The method as recited in claim 1, further comprising dividing the video recording into segments according to user-selected segment characteristics.

17. The method as recited in claim 1, further comprising receiving a user-defined shape and then training a shape definition model for the user-defined shape, wherein the shape definition model is used for deriving a digital thumbnail image to represent one of the segments.

18. A method, comprising:

dividing each video of a video collection into segments;

representing each segment with a thumbnail image;

printing the thumbnail images into a paper booklet; and

wherein each thumbnail image in the paper booklet can be digitally photographed to play back the corresponding segment.

19. The method as recited in claim 18, wherein a digital photograph of one of the thumbnail images from the paper booklet can be manipulated through active shape modeling (ASM) to match the thumbnail image corresponding to one of the segments.

20. A system, comprising:

means for segmenting a video recording;

means for representing each video segment with a corresponding digital thumbnail image;

means for printing each digital thumbnail image on paper;

means for receiving one of the printed thumbnail images as digital image information captured by a camera; and

means for playing the video segment corresponding to the captured digital image information.