Method and system for annotating video material

• US 8,433,566 B2
• Filed: 02/07/2008
• Issued: 04/30/2013
• Est. Priority Date: 02/13/2007
• Status: Active Grant

• Current Assignee: British Telecommunications PLC (BT Group PLC)
• Original Assignee: British Telecommunications PLC (BT Group PLC)
• Inventors: Cui, Zhan, Azarmi, Nader, Ducatel, Gery M
• Primary Examiner(s): Lerner, Martin
• Application Number:
  US12/526,726
• Publication Number:
  US 20100039564A1
• Time in Patent Office: 1,909 Days
• Field of Search: 704/235, 704/236, 704/270, 704/278, 386/241
• US Class Current: 704/235
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

Video material is dividing into temporal segments. Each segment is examined to determine whether the soundtrack of the segment contains speech sufficient for analysis and if so, metadata are generated based on analysis of the speech. If not, the segment is analysed by comparing frames thereof with those of stored segments that already have metadata assigned to them. One then assigns to the segment under consideration stored metadata associated with one or more stored segments that are similar.

Claims

1. A method of assigning metadata to video material, comprising dividing the material into temporal portions of video frames and for each temporal portion:

(i) determining, using a computer processing system comprising a computer processor, whether a soundtrack of the portion contains speech sufficient for analysis;

(ii) if it has, generating metadata for that portion of video frames based on analysis of the speech and adding the portion, with the generated metadata, to a database of stored video frames;

(iii) if it has not, analysing the portion by comparing the video frames thereof with already annotated video frames stored in the database, and assigning to the portion stored metadata associated with one or more stored video frames that are similar.

2. A method according to claim 1 in which the step of comparing video frames comprises extracting key frames from the portion under consideration and comparing the key frames with key frames of the stored portions.

3. A method according to claim 2 in which a portion under consideration is deemed similar to a stored portion if the proportion of video frames in the portion under consideration that meet a criterion of similarity with video frames of the stored portion exceeds a threshold.

4. A method according to claim 1, further comprising, in the event that the comparison step identifies a plurality of stored portions, retrieving stored metadata associated with each portion identified and combining said metadata, including eliminating duplicate or similar items, to produce a single set of metadata for the portion under consideration.

5. A method according to claim 1, wherein said comparing the video frames thereof with already annotated video frames stored in the database uses color indexing to determine similarity of the video frames.

6. A method according to claim 1, wherein the already annotated video frames used in the comparing of step (iii) became stored in the database as a result of previously performing steps (i) and (ii) for those video frames.

7. A system comprising:

a computer processing system, comprising a computer processor, configured to;

assign metadata to video material; and

divide the material into temporal portions of video frames and for each temporal portion perform at least;

(i) determine whether the soundtrack of the portion contains speech sufficient for analysis;

(ii) if it has, generate metadata for that portion of video frames based on analysis of the speech and adding the portion, with the generated metadata, to a database of stored video frames; and

(iii) if it has not, analyze the portion by comparing the video frames thereof with already annotated video frames stored in the database, and assign to the portion stored metadata associated with one or more stored video frames that are similar.

8. The system according to claim 7, wherein the computer processing system is further configured to compare video frames by extracting frames from the portion under consideration and comparing the frames with frames of the stored portions.

9. The system according to claim 8, wherein a portion under consideration is deemed similar to a stored portion if the proportion of video frames in the portion under consideration that meet a criterion of similarity with video frames of the stored portion exceeds a threshold.

10. The system according to claim 7, wherein the computer processing system is further configured to, in the event that the comparing identifies a plurality of stored portions, retrieve stored metadata associated with each portion identified and combine said metadata, including eliminating duplicate or similar items, to produce a single set of metadata for the portion under consideration.

11. The system according to claim 7, wherein said comparing the video frames thereof with already annotated video frames stored in the database uses color indexing to determine similarity of the video frames.

12. The system according to claim 7, wherein the already annotated video frames used in the comparing of (iii) became stored in the database as a result of previously performing (i) and (ii) for those video frames.