Methods and apparatus for interactive map-based analysis of digital video content

• US 8,645,832 B2
• Filed: 07/03/2006
• Issued: 02/04/2014
• Est. Priority Date: 12/30/2002
• Status: Expired due to Fees

• Current Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Original Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Inventors: Pea, Roy D., Mills, Michael I., Hoffert, Eric, Rosen, Joseph H., Dauber, Kenneth
• Primary Examiner(s): Augustine, Nicholas
• Assistant Examiner(s): Wong, William
• Application Number:
  US11/480,727
• Publication Number:
  US 20060253781A1
• Time in Patent Office: 2,773 Days
• Field of Search: 715719-726, 715/704, 715/751
• US Class Current: 715/719
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/105 of operating discs

Abstract

The present invention provides electronic methods and apparatus for interactively authoring, sharing and analyzing digital video content. Methods for authoring include displaying visual data, defining each traversal as a time-based sequence of frames and annotating and storing a record of the traversal and its associated audio records. Defining the traversal includes interactively panning the visual data by positioning an overlay window relative to the visual data and zooming in or out by resizing the overlay window. In alternative embodiments, the visual data may be displayed in a rectangular layout or a cylindrical layout. The methods are practiced using an integrated graphical interface, including an overview region displaying the visual data, a detail region displaying current data within the overlay window, and a worksheet region displaying a list of previously stored annotated traversal records. In a further aspect, the worksheet region list of annotated traversal records is published in a web document accessible via network using a standard HTML browser, and further annotations may be added by a community of network users. Analytical methods are also provided in which data markers corresponding to traversal records are plotted against an interactive abstract map enabling users to shift between levels of abstraction in exploring the video record.

Claims

1. An interactive, electronic method for exploring and analyzing visual data of a scene using a plurality of traversal records, comprising:

displaying an abstract map of the visual data, wherein the abstract map comprises a panoramic overview of the visual data derived from the visual data for illustrating a plurality of markers, where each marker is associated with a traversal record of the plurality of traversal records;

plotting the plurality of markers selectively onto the abstract map, wherein each of at least two of the plurality of traversal records comprises a time-based sequence of frames captured by a different one of a plurality of users from the visual data via an overlay window positioned relative to the visual data, wherein each of the at least two of the plurality of traversal records further comprises a respective subset of spatial audio data associated with a spatial territory of the time-based sequence of frames of a respective one of the at least two of the plurality of traversal records, wherein each subset of spatial audio data is defined via virtual microphone icons graphically placed by a user within the panoramic overview, and wherein the visual data and the spatial audio data have previously been recorded prior to each of the plurality of users capturing a respective time-based sequence of frames associated with a respective one of the plurality of traversal records; and

playing back one of the plurality of traversal records associated with one of the plurality of markers in response to an interactive selection of the one of the plurality of markers.

2. The method of claim 1, wherein each of the plurality of markers comprises a thumbnail image of one of the plurality of traversal records.

3. The method of claim 1, wherein each of the plurality of markers denotes an annotation value specified for the associated one of the plurality of traversal records.

4. The method of claim 1, wherein said method is performed using a graphical user interface, said graphical user interface comprising a plurality of computer display regions including a map region displaying the abstract map and plotted markers, and a playback region for playing back one of the plurality of traversal records associated with one of the plurality of markers in response to interactive selection of the one of the plurality of markers.

5. The method of claim 4, wherein the graphical user interface further includes a worksheet region displaying a list of the plurality of traversal records and associated annotations.

6. The method of claim 5, further comprising interactively selecting a spatial area of interest within the abstract map, and displaying within the worksheet region a filtered list of those traversal records corresponding to the plurality of markers located within the spatial area of interest.

7. An interactive, electronic apparatus for exploring and analyzing visual data of a scene using a plurality of traversal records, comprising:

means for displaying an abstract map of the visual data, wherein the abstract map comprises a panoramic overview of the visual data derived from the visual data for illustrating a plurality of markers, where each marker is associated with a traversal record of the plurality of traversal records;

means for plotting the plurality of markers selectively onto the abstract map, wherein each of at least two of the plurality of traversal records comprises a time-based sequence of frames captured by a different one of a plurality of users from the visual data via an overlay window positioned relative to the visual data, wherein each of the at least two of the plurality of traversal records further comprises a respective subset of spatial audio data associated with a spatial territory of the time-based sequence of frames of a respective one of the at least two of the plurality of traversal records, wherein each subset of spatial audio data is defined via virtual microphone icons graphically placed by a user within the panoramic overview, and wherein the visual data and the spatial audio data have previously been recorded prior to each of the plurality of users capturing a respective time-based sequence of frames associated with a respective one of the plurality of traversal records; and

means for playing back one of the plurality of traversal records associated with one of the plurality of markers in response to an interactive selection of the one of the plurality of markers.

8. The apparatus of claim 7, wherein each of the plurality of markers comprises a thumbnail image of one of the plurality of traversal records.

9. The apparatus of claim 7, wherein each of the plurality of markers denotes an annotation value specified for the associated one of the plurality of traversal records.

10. The apparatus of claim 7, wherein said means for displaying, plotting, and playing back are configured to display an integrated graphical user interface, said graphical user interface comprising a plurality of computer display regions including a map region displaying the abstract map and plotted markers, and a playback region for playing back one of the plurality of traversal records associated with one of the plurality of markers in response to interactive selection of the one of the plurality of markers.

11. The apparatus of claim 10, wherein the graphical user interface further includes a worksheet region displaying a list of the plurality of traversal records and associated annotations.

12. The apparatus of claim 11, further comprising means for interactively selecting a spatial area of interest within the abstract map, and means for displaying within the worksheet region a filtered list of those traversal records corresponding to the plurality of markers located within the spatial area of interest.