Method and apparatus for identifying selected portions of a video stream

• US 7,802,188 B2
• Filed: 05/13/2004
• Issued: 09/21/2010
• Est. Priority Date: 05/13/2004
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Wu, Peng
• Primary Examiner(s): Lo; Weilun
• Assistant Examiner(s): Tran; Tuyetlien T
• Application Number:
  US10/844,386
• Publication Number:
  US 20050257151A1
• Time in Patent Office: 2,322 Days
• Field of Search: 715/719, 715/720, 715/721, 715/723, 715/716
• US Class Current: 715/723
• CPC Class Codes: G06F 16/745 the internal structure of a single video sequence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

A method is disclosed for identifying a selected portion of a video stream. A user interface is provided for designating a reference frame of a selected portion of a video stream. A processor is configured to compare the reference frame with other portions of the video stream to establish a similarity measure, process the similarity measure to identify a candidate region as a boundary of the selected portion of the video stream, and provide user access to the candidate region to designate the boundary for storage via the user interface.

Claims

1. Apparatus, comprising

a memory storing computer-readable instructions, and

a data processing unit coupled to the memory, operable to execute the instructions, and based at least in part on the execution of the instructions operable to perform operations comprising:

in response to user input, selecting a reference in a video comprising a sequence of frames;

determining a sequence of values of a similarity measure from respective comparisons between the reference and a respective ones of the frames of the video;

identifying in the video one or more candidate regions of ones of the frames of the video based on identification of one or more respective valleys in the sequence of similarity measure values, wherein each of the one or more identified candidate regions comprises a frame of the video that corresponds to a respective one of the one or more valleys in the sequence of similarity measure values; and

in response to user selection of at least one of the candidate regions, presenting one or more of the frames of the selected candidate region on a display; and

in response to user input, designating at least one of the presented frames as a boundary of a segment of the video.

2. The apparatus of claim 1, wherein based at least in part on the execution of the instructions the data processing unit is operable to perform operations comprising generating on the display a user interface that comprises:

a control panel having user actuatable software buttons;

a first display region displaying a user-selectable identification for each of the candidate regions; and

a second display region displaying user-selectable representations of the one or more frames of the selected candidate region.

3. The apparatus of claim 1, wherein in the identifying the data processing unit is operable to perform operations comprising determining the similarity measure values from respective frame-by-frame similarity comparisons between the reference and ones of the frames of the video.

4. The apparatus of claim 1, wherein in the determining the data processing unit is operable to perform operations comprising ascertaining feature values respectively characterizing ones of the frames of the video, and determining the similarity measure values from respective comparisons between at least one of the feature values characterizing the reference and at least one of the feature values characterizing other frames of the video.

5. The apparatus of claim 1, wherein based at least in part on the execution of the instructions the data processing unit is operable to perform operations comprising filtering the sequence of similarity measure values to produce a filtered sequence of similarity measure values.

6. The apparatus of claim 1, wherein each of the similarity measure values is determined from a respective comparison between at least one feature of the reference and at least one feature of a respective one of the frames of the video.

7. The apparatus of claim 1, wherein based at least in part on the execution of the instructions the data processing unit is operable to perform operations comprising normalizing multiple quantities used to determine the similarity measure values.

8. The apparatus of claim 1, wherein in the identifying the data processing unit is operable to perform operations comprising identifying the one or more valleys in the sequence of similarity measure values.

9. A computer-implemented method, comprising:

in response to user input, selecting a reference in a video comprising a sequence of frames;

determining a sequence of values of a similarity measure from respective comparisons between the reference and respective ones of the frames of the video;

identifying in the video one or more candidate regions of ones of the frames of the video based on identification of one or more respective valleys in the sequence of similarity measure values, wherein each of the one or more identified candidate regions comprises a frame of the video that corresponds to a respective one of the one or more valleys in the sequence of similarity measure values;

in response to user selection of at least one of the candidate regions, presenting one or more of the frames of the selected candidate region on a display; and

in response to user input, designating at least one of the presented frames as a boundary of a segment of the video.

10. The method of claim 9, wherein the reference is a frame in the segment of the video.

11. The method of claim 9, wherein the similarity measures are determined from a respective comparison between the reference and respective other ones of the frames in a predetermined shot of the video that comprises the reference.

12. The method of claim 9, wherein the determining comprises ascertaining the similarity measure values from respective comparisons between the reference and respective ones of the frames of the video on a frame-by-frame basis.

13. The method of claim 12, further comprising generating on the display a miser interface that enables a riser to designate a portion of the video as the reference.

14. The method of claim 9, further comprising generating on the display a user interface that enables a user to designate a portion of the video as the reference.

15. The method of claim 9, further comprising filtering the sequence of similarity measure values to produce a filtered sequence of similarity measure values.

16. The method of claim 9, wherein each of the similarity measure values is determined from a respective comparison between at least one feature of the reference and at least one feature of a respective one of the frames of the video.

17. The method of claim 16, wherein the at least one feature comprises a color histogram metric.

18. The method of claim 16, wherein the at least one feature comprises an edge energy metric.

19. The method of claim 16, wherein the at least one feature is represented as a vector quantity.

20. The method of claim 9, further comprising normalizing multiple quantities used to produce the similarity measure values.

21. The method of claim 9, wherein the identifying comprises ascertaining the one or more valleys in the sequence of the similarity measure values.

22. The method of claim 21, wherein the identifying comprises for each of the one or more ascertained valleys selecting frames of the video near the valley as a respective one of the candidate regions.

23. The method of claim 22, wherein the selecting comprises, for each of the one or more ascertained valleys, selecting as a respective one of the candidate regions a respective set of frames of the video that are between the reference and a boundary one of the frames that coincides with the valley and that are bounded at one end by the boundary frame.

24. The method of claim 22, wherein the selecting comprises, for each of the one or more ascertained valleys, selecting as a respective one of the candidate regions a respective set of frames of the video that are bounded at one end by a boundary one of the frames that coincides with the respective valley.

25. The method of claim 22, wherein the selecting, comprises, for each of the one or more ascertained valleys, selecting as a respective one of the candidate regions a respective set of frames that comprise a respective one of the frames that coincides with the respective valley.

26. The method of claim 9, comprising:

identifying one of the plural candidate regions as possibly containing a start frame of the selected portion of the video stream.

27. The method of claim 9, further comprising classifying each of the candidate regions as either a candidate end frame region or a candidate start frame region.

28. The method of claim 9, further comprising presenting on the display a list of identifiers corresponding to the candidate regions in order of proximity of the corresponding candidate regions to the reference.

29. At least one memory having computer program code embodied therein, the computer-readable program code adapted to be executed by a computer to perform a method comprising:

in response to user input, selecting a reference in a video comprising a sequence of frames;

determining a sequence of values of a similarity measure from respective comparisons between the reference and a respective ones of the frames of the video;

identifying in the video one or more candidate regions of ones of the frames of the video based on identification of one or more respective valleys in the sequence of similarity measure values, wherein each of the one or more identified candidate regions comprises a frame of the video that corresponds to a respective one of the one or more valleys in the sequence of similarity measure values;

in response to user selection of at least one of the candidate regions, presenting one or more of the frames of the selected candidate region on a display; and

in response to user input, designating at least one of the presented frames as a boundary of a segment of the video.