Methods and apparatus for interactive point-of-view authoring of digital video content

• US 7,823,058 B2
• Filed: 12/30/2002
• Issued: 10/26/2010
• Est. Priority Date: 12/30/2002
• Status: Expired due to Fees

• Current Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Original Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Inventors: Pea, Roy D., Mills, Michael I., Rosen, Joseph H.
• Primary Examiner(s): Basehoar; Adam L
• Application Number:
  US10/334,162
• Publication Number:
  US 20040125148A1
• Time in Patent Office: 2,857 Days
• Field of Search: 715/530, 715/512, 715/500.1, 715/716, 715/719, 715/704, 715/730, 715/764, 715/856, 715/230, 715/233, 715/723, 715/202, 715/720, 345/440, 345620-628, 352/69, 348/36, 348/39
• US Class Current: 715/230
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects

Abstract

The present invention provides electronic methods and apparatus for interactively authoring, sharing and analyzing digital video content. Methods for authoring include displaying visual data, defining each traversal as a time-based sequence of frames and annotating and storing a record of the traversal and its associated audio records. Defining the traversal includes interactively panning the visual data by positioning an overlay window relative to the visual data and zooming in or out by resizing the overlay window. In alternative embodiments, the visual data may be displayed in a rectangular layout or a cylindrical layout. The methods are practiced using an integrated graphical interface, including an overview region displaying the visual data, a detail region displaying current data within the overlay window, and a worksheet region displaying a list of previously stored annotated traversal records.

Claims

1. An interactive, electronic method of authoring annotated traversals through visual data, the method comprising:

displaying the visual data, wherein the visual data comprises motion video;

interactively defining a traversal of the displayed visual data by positioning a resizable overlay window relative to the displayed visual data, wherein said resizable overlay window is resizable while the visual data is being displayed, said traversal comprising a subset of motion video that specifies a time-based sequence of frames of said motion video, each of said frames comprising the visual data delineated by the overlay window, wherein the displaying said visual data comprises displaying the visual data in a cylindrical layout, and wherein said positioning of the overlay window is defined by a field of view of a virtual camera located centrally to said cylindrical layout;

annotating the traversal;

storing a persistent record of the annotated traversal; and

using an integrated graphical user interface to perform said method, and wherein said graphical user interface comprises a plurality of computer display regions including;

an overview region displaying the visual data;

a detail region displaying current data within the overlay window; and

a worksheet region displaying a list of a plurality of stored annotated traversal records.

2. The method of claim 1, further comprising performing said method repeatedly, thereby creating said list of a plurality of stored annotated traversals.

3. The method of claim 2, wherein at least two of said traversals are created with respect to different visual data sources.

4. The method of claim 1, wherein the motion video includes panoramic video imagery captured using a plurality of cameras facing inward from a perimeter of a scene, and wherein said traversal reflects a user-adjusted 3D-perspective.

5. The method of claim 1, wherein the defining the traversal further comprises panning the visual data by positioning the overlay window relative to the displayed visual data, and interactively zooming by resizing the overlay window.

6. The method of claim 1, wherein the displaying said visual data comprises displaying visual data in a rectangular layout.

7. The method of claim 1, wherein the positioning said overlay window comprises moving said visual data, while said overlay window is fixed.

8. The method of claim 1, wherein the positioning said overlay window comprises moving said overlay window, while said displayed visual data is fixed.

9. The method of claim 1, wherein said overlay window has a geometric shape that is user-selectable.

10. The method of claim 1, further comprising concurrently defining a plurality of traversals by using a plurality of independently-positioned overlay windows.

11. The method of claim 1, wherein the annotating includes, at least in part, data selected from one or more of the following categories:

{textual comments;

graphical symbols;

classification codes;

meta-data;

audio transcription}.

12. The method of claim 11, wherein the classification codes are interactively selected from user-definable code templates.

13. The method of claim 1, wherein the stored traversal record comprises a compressed portion of said visual data.

14. The method of claim 1, wherein the stored traversal record comprises a set of location coordinates defining said traversals.

15. The method of claim 1, wherein the detail region displays data at a higher magnification or resolution, and wherein said higher magnification or resolution is based at least partly on one or more factors selected from the group comprising:

{user request, network bandwidth, other network conditions}.

16. The method of claim 1, further comprising initiating playback of a selected stored record in the detail region, at least in part, by interactively selecting a desired one of the listed traversal records.

17. The method of claim 16, wherein said interactively selecting a desired traversal record comprises dragging and dropping a graphical element corresponding to the selected record from the worksheet region onto the detail region.

18. The method of claim 1, further comprising specifying a compound traversal record, being a combination of a plurality of the stored traversal records.

19. The method of claim 18, further comprising playing back the compound traversal record by adjacently displaying the contents of each of the plurality of traversal records.

20. The method of claim 18, further comprising specifying an audio contribution from each of the plurality of traversal records.

21. The method of claim 20, wherein said audio contribution is specified using a selection mode selected from one or more of the following modes:

{select one source record;

turn off all audio;

specify a relative audio contribution for each record}.

22. The method of claim 1, further comprising publishing said worksheet region list of annotated traversal records to a document accessible via network using a standard HTML browser.

23. An interactive, electronic apparatus for authoring annotated traversals through visual data, the apparatus comprising:

means for displaying the visual data, wherein the visual data comprises motion video;

means for interactively defining a traversal of the displayed visual data by positioning a resizable overlay window relative to the displayed visual data, wherein said resizable overlay window is resizable while the visual data is being displayed, said traversal comprising a subset of motion video that specifies a time-based sequence of frames of said motion video, each of said frames comprising the visual data delineated by the overlay window, wherein the means for displaying said visual data displays the visual data in a cylindrical layout, and wherein said positioning of the overlay window is defined by a field of view of a virtual camera located centrally to said cylindrical layout;

means for annotating the traversal;

means for storing a persistent record of the annotated traversal; and

means for providing an integrated graphical user interface, wherein said graphical user interface comprises a plurality of computer display regions including;

an overview region displaying the visual data;

a detail region displaying current data within the overlay window; and

a worksheet region displaying a list of a plurality of stored annotated traversal records.

24. A digital electronic apparatus for interactively authoring annotated traversals through visual data, the apparatus comprising

a first display device for displaying the visual data, wherein the visual data comprises motion video, wherein the first display device comprises an integrated graphical user interface, wherein said graphical user interface comprises a plurality of computer display regions including:

an overview region displaying the visual data;

a detail region displaying current data within a resizable overlay window; and

a worksheet region displaying a list of a plurality of stored annotated traversal records; and

a handheld, remote control device in communication with the first display device and configured to control a graphical interaction on the first display device comprising the following operations;

defining a traversal of the displayed visual data, said traversal comprising a subset of motion video that specifies a time-based sequence of frames of said motion video, each of said frames comprising a spatial subset of the visual data, wherein defining a traversal comprises panning the visual data by positioning said resizable overlay window relative to the displayed visual data, wherein said resizable overlay window is resizable while the visual data is being displayed, and zooming the visual data by resizing the overlay window, wherein the displayed visual data comprises displaying the displayed visual data in a cylindrical layout, and wherein said positioning of the resizable overlay window is defined by a field of view of a virtual camera located centrally to said cylindrical layout;

annotating said traversal; and

storing a persistent record of the annotated traversal.