Context aware video conversion method and playback system

• US 20060228048A1
• Filed: 04/08/2005
• Published: 10/12/2006
• Est. Priority Date: 04/08/2005
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Forlines, Clifton L.
• Granted Patent: US 7,526,725 B2
• Time in Patent Office: Days
• US Class Current: 382/299
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

A computer implemented method converts and displays a video. An input video acquired of a scene is segmented into shots. Similar shots are combined sequentially into corresponding chains. Each frame of the input video is then rendered into a composite image according to the chains and templates.

Claims

1. A computer implemented method for converting and displaying a video, comprising:

segmenting an input video acquired of a scene into shots;

comparing the shots to determine similar shots;

combining similar shots into corresponding chains;

rendering, for each frame of the input video, a composite image according to the chains and a template; and

arranging sequentially the composite images in an output video.

2. The method of claim 1, further comprising:

playing back the output video on a display surface.

3. The method of claim 1, in which the input video is segmented using color histograms for each frame of the input video.

4. The method of claim 1, in which a resolution of the output video is substantially larger than a resolution of the input video.

5. The method of claim 1, in which the input video includes first shots of a first character talking, and second shots of a second character talking, and the template is designed to render concurrently frames of the first shots in a first region of the composite images, and frames of the second shot in a second region of the composite images.

6. The method of claim 5, in which a still frame of the first shots is rendered in the first region when the second character is talking and the frames of the second talking character are rendered in the second region, and a still frame of the second shots is rendered in the second region when the first character is talking and the frames of the first talking character are rendered in the first region.

7. The method of claim 1, further comprising:

detecting shot boundaries to segment the input video.

8. The method of claim 1, further comprising:

constructing a color histogram for each frame of the input video;

comparing the color histogram of adjacent frames to determine similar frames; and

combining similar frames into corresponding similar shots.

9. The method of claim 8, further comprising:

combining shots having less than a predetermined number of frames that overlap with an immediately following shot.

10. The method of claim 8, in which the color histograms are used to determine the similar shots.

11. The method of claim 10, further comprising:

comparing the color histogram of a first frame of a current shot with the color histograms of a predetermined number of last frames of a predetermined number of previous shots to determine the similar shots.

12. The method of claim 1, in which the chains overlap.

13. The method of claim 1, further comprising:

estimating camera motion in the input video; and

rendering the frames into the composite images according to the camera motion.

14. The method of claim 13, in which the camera motion is estimated from motion vectors in the input video.

15. The method of claim 1, in which selected images include a plurality of regions, there being one region for each one of a selected chain, and further comprising:

rendering concurrently frames from the selected chains into the corresponding regions.

16. The method of claim 1, in which the templates depend on a structure of the video, the structure including number and pattern of the chains, a number of shots in the chains, lengths of the chains, and an amount of overlap of the chains.

17. The method of claim 6, further comprising:

fading the still frames.

20. The method of claim 6, further comprising:

pixelating the still frames.

21. The method of claim 6, further comprising:

converting the still frames to a sepia tone.

22. The method of claim 15, arranging the regions according to gaze direction of characters in the shots.

23. The method of claim 15, further comprising:

animating the regions.

18. The method of 6, further comprising:

reducing a color saturation of the still frames over time.

19. The method of 6, further comprising:

blurring the still frames.