System and method for creation of video annotations

• US 7,334,186 B2
• Filed: 04/30/2003
• Issued: 02/19/2008
• Est. Priority Date: 04/30/2003
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Wu, Peng
• Primary Examiner(s): Hutton; Doug
• Assistant Examiner(s): RUTLEDGE, AMELIA L
• Application Number:
  US10/426,775
• Publication Number:
  US 20040221227A1
• Time in Patent Office: 1,756 Days
• Field of Search: 715/512, 725/44
• US Class Current: 715/230
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G11B 2220/2562 DVDs [digital versatile discs]; Digital video discs; MMCDs; HDCDs
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: H04N 21/42202 environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4402 involving reformatting operations of video signals for household redistribution, storage or real-time display details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 5/85 on discs or drums
• CPC Class Codes: H04N 9/8233 the additional signal being a character code signal

Abstract

A system and method for annotating a video data stream integrates geographic location data into the video data stream. A site name is mapped from the geographic location data, and the site name is automatically assigned as a text label to the associated portion of the video data stream.

Claims

1. A method for annotating a video data stream, at least one portion of the video data stream having associated geographic location data, the method comprising:

creating a video document by integrating geographic location data into the video data stream, wherein the video document comprises a plurality of video sequences and wherein each of the plurality of video sequences has associated geographic location data;

organizing the video document into a title-and-chapter navigation structure, the navigation structure comprising at least one title and each title comprising at least one chapter, wherein each chapter is associated with at least one of the plurality of video sequences;

extracting a plurality of keyframes from the video data stream each keyframe corresponding to one of the titles and chapters and having associated geographic location data;

mapping a site name for each title and chapter from the geographic location data, wherein the site names for each title have a first level of abstraction, and wherein the site names for each chapter have a second level of abstraction different from the first level of abstraction; and

automatically assigning the site name as a text label to each keyframe of each title and chapter.

2. The method of claim 1, wherein mapping the geographic location data farther comprises translating the geographic location data to map location data and extracting the site name from the map location.

3. A method for creating a text label for a video sequence comprising:

integrating geographic location data into a video data stream;

organizing the video data stream into a title-and-chapter navigation structure, the navigation structure comprising at least one title and each title comprising at least one chapter;

extracting a plurality of keyframes from the video data stream, each keyframe corresponding to one of the titles and chapters and having associated geographic location data;

determining a site name for each title and chapter using the geographic location data, wherein the site names for each title have a first level of abstraction, and wherein the site names for each chapter have a second level of abstraction different from the first level of abstraction; and

using the site names as a text label for the keyframes corresponding to each title and chapter.

4. The method of claim 3, wherein determining a site name using the geographic location data includes providing the geographic location data to a mapping application.

5. The method of claim 3, further comprising interfacing a video camera with a global positioning system receiving device to provide geographic location data to the video data stream.

6. The method of claim 3, wherein integrating geographic location data includes recording latitude and longitude data in the video data stream.

7. The method of claim 3, wherein integrating geographic location data includes manually entering geographic location data into the video data stream.

8. The method of claim 3, wherein integrating geographic location data includes automatically entering geographic location data into the video data stream.

9. The method of claim 3, wherein integrating geographic location data comprises continuously integrating geographic location data into the video data stream.

10. A method for creating a summarization of a video document comprising:

organizing video data having associated geographic location data into a tide-and-chapter navigation structure, the navigation structure comprising at least one title and each title comprising at least one chapter;

extracting a plurality of keyframes from the video data, each keyframe associated with a corresponding one of the titles and chapters;

determining a site name for each of the extracted keyframes using the associated geographic location data, wherein the site names for keyframes associated with the tides have a first level of abstraction, and wherein the site names for keyframes associated with the chapter have a second level of abstraction different from the first level of abstraction; and

annotating the extracted keyframes with the corresponding site name.

11. The method of claim 10, further comprising:

recording a video data stream; and

integrating geographic location data into the video data stream during recording.

12. The method of claim 11, wherein integrating geographic location data into the video data stream during recording includes integrating geographic location data continuously into the video data stream.

13. The method of claim 11, wherein integrating geographic location data into the video data stream during recording includes integrating geographic location data intermittently into the video data stream.

14. The method of claim 11, wherein integrating geographic location data into the video data stream during recording includes:

receiving a GPS signal from a GPS satellite; and

calculating a geographic location from the GPS signal.

15. The method of claim 10, wherein determining a site name using the geographic location data includes providing the geographic location data to a mapping application.

16. A video summarization and annotation system comprising:

a processor for organizing a video document having geographic location data integrated into the video document into a title-and-chapter navigation structure, the navigation structure comprising at least one tide and each title comprising at least one chapter, extracting a plurality of keyframes from the video document, each keyframe associated with a corresponding one of the titles and chapters, and for mapping a site name for each keyframe using the integrated geographic location data, wherein the site names for keyframes associated with the titles have a first level of abstraction, and wherein the site names for keyframes associated with the chapter have a second level of abstraction different from the first level of abstraction;

a data storage system in communication with the processor for storing the extracted keyframes and corresponding site names as a video summary; and

a display device for displaying the video summary.

17. The system of claim 16, further comprising:

a video camera; and

a global positioning system communicating with the video camera and providing geographic location data to the video camera.

18. The system of claim 16, wherein the data storage system includes a DVD disc, a hard disc drive, or other persistent storage device.

19. The system of claim 16, wherein the display device comprises a video monitor.

20. A computer readable medium containing instructions for controlling a computer system to perform a method for annotating a video summarization comprising:

organizing video data having associated geographic location data into a title-and-chapter navigation structure, the navigation structure comprising at least one title and each title comprising at least one chapter;

extracting a plurality of keyframes from the video data, each keyframe associated with a corresponding one of the titles and chapters;

mapping a site name for each of the extracted keyframes using the geographic location data, wherein the site names for keyframes associated with the titles have a first level of abstraction, and wherein the site names for keyframes associated with the chapter have a second level of abstraction different from the first level of abstraction; and

annotating extracted keyframes with the corresponding site name.

21. The method of claim 20, wherein mapping a site name using the geographic location data includes providing the geographic location data to a mapping application.