Methods and apparatus for interactive point-of-view authoring of digital video content

• US 20040125148A1
• Filed: 12/30/2002
• Published: 07/01/2004
• Est. Priority Date: 12/30/2002
• Status: Active Grant

• Current Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Original Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Inventors: Mills, Michael I., Rosen, Joseph H., Pea, Roy D.
• Granted Patent: US 7,823,058 B2
• Time in Patent Office: Days
• US Class Current: 345/802
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects

Abstract

The present invention provides electronic methods and apparatus for interactively authoring, sharing and analyzing digital video content. Methods for authoring include displaying visual data, defining each traversal as a time-based sequence of frames and annotating and storing a record of the traversal and its associated audio records. Defining the traversal includes interactively panning the visual data by positioning an overlay window relative to the visual data and zooming in or out by resizing the overlay window. In alternative embodiments, the visual data may be displayed in a rectangular layout or a cylindrical layout. The methods are practiced using an integrated graphical interface, including an overview region displaying the visual data, a detail region displaying current data within the overlay window, and a worksheet region displaying a list of previously stored annotated traversal records. In a further aspect, the worksheet region list of annotated traversal records is published in a web document accessible via network using a standard HTML browser, and further annotations may be added by a community of network users. Analytical methods are also provided in which data markers corresponding to traversal records are plotted against an interactive abstract map enabling users to shift between levels of abstraction in exploring the video record.

Claims

1. An interactive, electronic method of authoring annotated traversals through visual data, the method comprising:

displaying the visual data;

interactively defining a traversal of the displayed visual data by positioning an overlay window relative to the displayed visual data, said traversal specifying a time-based sequence of frames, each of said frames comprising the visual data delineated by the overlay window;

annotating the traversal; and

storing a persistent record of the annotated traversal.

2. The method of claim 1, further comprising performing said method repeatedly, thereby creating a plurality of stored annotated traversals.

3. The method of claim 2, wherein at least two of said traversals are created with respect to different visual data sources.

4. The method of claim 1, wherein the visual data includes, at least in part, data selected from one or more of the following categories:

{motion video, still imagery, simulated imagery, animated imagery, panoramic imagery, live imagery}.

5. The method of claim 4, wherein the panoramic imagery includes imagery captured using a plurality of cameras facing inward from a perimeter of a scene, and wherein said traversal reflects a user-adjusted 3D-perspective.

6. The method of claim 1, wherein defining a traversal further comprises panning the visual data by positioning the overlay window relative to the displayed visual data, and interactively zooming by resizing the overlay window.

7. The method of claim 1, wherein displaying said visual data comprises displaying visual data in rectangular layout.

8. The method of claim 1, wherein displaying said visual data comprises displaying visual data in a cylindrical layout, and wherein said positioning of the overlay window is defined by the field of view of a virtual camera located centrally to said cylindrical layout.

9. The method of claim 1, wherein positioning said overlay window comprises moving said visual data, while said overlay window is fixed.

10. The method of claim 1, wherein positioning said overlay window comprises moving said overlay window, while said displayed visual data is fixed.

11. The method of claim 1, wherein said overlay window has a geometric shape that is user-selectable.

12. The method claim 1, further comprising concurrently defining a plurality of traversals by using a plurality of independently-positioned overlay windows.

13. The method of claim 1, wherein the annotations include, at least in part, data selected from one or more of the following categories:

{textual comments;

graphical symbols;

classification codes;

meta-data;

audio transcription}.

14. The method of claim 13, wherein the classification codes are interactively selected from user-definable code templates.

15. The method of claim 1, wherein the stored traversal record comprises a compressed portion of said visual data.

16. The method of claim 1, wherein the stored traversal record comprises a set of location coordinates defining said traversal.

17. The method of claim 2, further comprising using an integrated graphical user interface to perform said method, and wherein said graphical user interface comprises a plurality of computer display regions including:

an overview region displaying the visual data;

a detail region displaying current data within the overlay window; and

a worksheet region displaying a list of said plurality of stored annotated traversal records.

18. The method of claim 17, wherein the detail region displays data at a higher magnification or resolution, and wherein said higher magnification or resolution is based at least partly on one or more factors selected from the group comprising:

{user request, network bandwidth, other network conditions}.

19. The method of claim 17, further comprising initiating playback of a selected stored record in the detail region, at least in part, by interactively selecting a desired one of the listed traversal records.

20. The method of claim 19, wherein said interactively selecting a desired traversal record comprises dragging and dropping a graphical element corresponding to the selected record from the worksheet region onto the detail region.

21. The method of claim 1, further comprising specifying a compound traversal record, being a combination of a plurality of the stored traversal records.

22. The method of claim 21, further comprising playing back the compound traversal record by adjacently displaying the contents of each of the plurality of traversal records.

23. The method of claim 21, further comprising specifying the audio contribution from each of the plurality of traversal records.

24. The method of claim 23, wherein said audio contribution is specified using a selection mode selected from one or more of the following modes:

{select one source record;

turn off all audio;

specify a relative audio contribution for each record}.

25. The method of claim 17, further comprising publishing said worksheet region list of annotated traversal records to a document accessible via network using a standard HTML browser.

26. An interactive, electronic apparatus for authoring annotated traversals through visual data, the apparatus comprising:

means for displaying the visual data;

means for interactively defining a traversal of the displayed visual data by positioning an overlay window relative to the displayed visual data, said traversal specifying a time-based sequence of frames, each of said frames comprising the visual data delineated by the overlay window;

means for annotating the traversal; and

means for storing a persistent record of the annotated traversal.

27. A digital electronic apparatus for interactively authoring annotated traversals through visual data, the apparatus comprising:

a first display device for displaying the visual data; and

a handheld, remote control device in communication with the first display device and configured to control a graphical interaction on the first display device comprising the following operations;

defining a traversal of the displayed visual data, said traversal comprising a time-based sequence of frames, each of said frames comprising a spatial subset of the visual data;

annotating said traversal; and

storing a persistent record of the annotated traversal.

28. The apparatus of claim 27, wherein the remote control device is further configured such that said defining a traversal comprises panning the visual data by positioning an overlay window relative to the displayed visual data, and zooming the visual data by resizing the overlay window.

29. An interactive, electronic method of authoring traversals through visual data and associated spatial audio data, the method comprising:

displaying the visual data;

interactively defining a traversal of the displayed visual data, the traversal comprising a time-based sequence of frames, each of said frames comprising a spatial subset of the visual data; and

storing a record of the traversal, including the spatial audio data associated with the spatial subset of the visual data in each of the frames.

30. The method of claim 29, wherein the visual data and the associated audio data comprise an audiovisual recording of a scene.

31. The method of claim 29, wherein the visual data comprises a visual depiction of a location, and the audio data comprises a recording of sound from said location.

32. The method of claim 31, wherein the visual depiction is abstract.

33. The method of claim 31, wherein the scene is a concert hall, and the audio data comprises a recording of music performed in said concert hall.

34. The method of claim 31, wherein the scene is a conference room, and the audio data comprises recorded conversation from said conference room.

35. The method of claim 29, further comprising annotating said traversal record with a transcription of the associated spatial audio data included in the record.