Method for representing contents of a single video shot using frames

• US 5,606,655 A
• Filed: 03/31/1994
• Issued: 02/25/1997
• Est. Priority Date: 03/31/1994
• Status: Expired due to Fees

• Current Assignee: Unify GmbH & Co. KG (Atos SE)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Arman, Farshid, Chiu, Ming-Yee, Hsu, Arding
• Primary Examiner(s): Jankus, Almis R.
• Application Number:
  US08/221,225
• Time in Patent Office: 1,062 Days
• Field of Search: 395/140, 395/152, 395/153, 395/154, 395/155, 395/157, 395/159, 395/161, 395/162, 352/129
• US Class Current: 345/440
• CPC Class Codes: G11B 2220/41 Flat as opposed to hierarchical combination, e.g. library of tapes or discs, CD changer, or groups of record carriers that together store one title
• CPC Class Codes: G11B 2220/65 wherein solid state memory is used for storing indexing information or metadata
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/179 the unit being a scene or a shot
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/87 involving scene cut or scene change detection in combination with video compression
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects

Abstract

A computer implemented method provides a representative frame (Rframe) for a group of frames in a video sequence. The method comprises selecting a reference frame from the group of frames and storing the reference frame in a computer memory. The method defines a peripheral motion tracking region along an edge of the reference frame and successively tracks movement of boundary pixels in the tracking region, symbolizing any of the length of the shot and the presence of any caption.

Claims

1. A computer implemented method for providing a representative frame (Rframe) for a group of frames in a video sequence, said method comprising:

(a) selecting a reference frame from said group of frames;

(b) storing said reference frame in a computer memory;

(c) defining a peripheral motion tracking region along an edge of said reference frame; and

(d) successively tracking movement of boundary pixels in said tracking region.

2. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 1, wherein in step (d) movement of boundary pixels is tracked successively from frame to frame in said group of frames.

3. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 2, wherein step (d) results in a track representative of camera or global motion.

4. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 1, wherein step (d) results in a track indicative of a missed frame in said group of frames.

5. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 1, wherein said group of frames comprises frames between consecutive scene changes, or a "shot".

6. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 1, wherein step (d) results in a track indicative of the duration or length of a respective shot.

7. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 6, wherein for relative time t=0, tracking starts substantially from the center of said Rframe.

8. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 6, wherein for relative time t=0, tracking starts substantially from the edge closest to the center of said Rframe.

9. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 1, wherein said reference frame is a frame taken from the fifth to the fifteenth frame of a shot.

10. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 1, wherein said reference frame is taken from the end of a shot.

11. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 1, wherein a missed scene change results in a line not in contact with a central portion of said Rframe.

12. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 1, wherein step (d) results in a track indicative of a missed video cut or scene change in said group of frames.

13. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 1, wherein said group of frames comprises frames between two consecutive scene changes, or a "shot".

14. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 1, wherein a track or "emblem" is provided indicative of the duration or length of a respective shot.

15. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 1, wherein said reference frame is a frame taken from any frame of a shot.

16. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 1, wherein said reference frame is constructed by combining frames, by stacking a number of slices, one from each side of each frame selected.

17. A computer implemented method for providing a representative frame (Rframe) for a group of frames in a video sequence, said group of frames consituting a shot, being frames between consecutive scene changes, said method comprising:

(a) selecting a reference frame from said shot;

(b) storing said reference frame in a computer memory;

(c) defining a peripheral motion tracking region along an edge of said reference frame, said tracking region being narrow relative to dimensions of said frame; and

(d) successively tracking movement of boundary pixels in said tracking region such that a tracked position for each boundary pixel following a previous pixel occupies a position juxtaposed to a tracked position for said previous pixel.

18. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 17, wherein said peripheral motion tracking region is substantially rectangularly annular around said Rframe.

19. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 17, wherein said peripheral motion tracking region includes a gap therein and a display is provided in said gap to indicate the presence of a caption in said Rframe.

20. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 17, wherein said peripheral motion tracking region includes a gap therein and a display is provided in said gap to indicate the length of a shot.

21. A computer implemented method for providing a representative frame (Rframe) for a group of frames in a video sequence, said group of frames consituting a shot, being frames between consecutive scene changes, said method comprising:

(a) selecting a reference frame from said shot;

(b) storing said reference frame in a computer memory;

(c) defining a peripheral motion tracking region along an edge of said reference frame, said tracking region being narrow relative to dimensions of said frame;

(d) sub-sampling said shot to select a number of frames therefrom; and

(e) stacking a smaller number of slices, one from each side of each selected frame are stacked and applying an edge detection algorithm to each of said smaller number of slices, whereby border pixels from one frame to the next are tracked, enabling a user to visualize motion associated with said shot.

22. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 21, wherein said edge detection algorithm uses the principles of convolution wherein a mask which is an m×

m matrix is convolved with the pixels in each of the motion tracking regions.

23. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 21, wherein said edge detection algorithm is a local operation performed using principles of convolution wherein a mask which is an m×

m matrix is convolved with the pixels in each of motion tracking region and wherein an output of the convolution highlights the pixels where there are changes in two neighboring pixels, where neighboring means left, right, top, or bottom.

24. A computer implemented method for providing a representative frame (Rframe) in accordance with claim 21, wherein said smaller number of stacks is about 4.

25. A computer implemented method for providing a representative frame (Rframe) for a group of frames in a video sequence, said method comprising:

(a) selecting a reference frame from said group of frames;

(b) storing said reference frame in a computer memory;

(c) defining a peripheral motion tracking region along an edge of said reference frame; and

(d) successively tracking movement of boundary pixels in said tracking region for providing an indication of the length of a shot and the presence of a caption.

26. A computer implemented method for providing a representative frame (Rframe) for a group of frames in a video sequence, said group of frames consituting a shot, being frames between consecutive scene changes, said method comprising:

(a) selecting a reference frame from said shot;

(b) storing said reference frame in a computer memory;

(c) defining four peripheral motion tracking regions along each edge of said reference frame, said tracking region being narrow relative to dimensions of said frame; and

(d) successively tracking movement of boundary pixels in said tracking region such that a tracked position for each boundary pixel following a previous pixel occupies a position juxtaposed to a tracked position for said previous pixel for providing any of time indicators and caption indicators.