Interactive video object processing environment having concurrently active subordinate windows

• US 6,633,309 B2
• Filed: 08/16/1999
• Issued: 10/14/2003
• Est. Priority Date: 08/16/1999
• Status: Expired due to Term

• Current Assignee: University of Washington
• Original Assignee: University of Washington
• Inventors: Kim, Yongmin, Kim, Donglok, Lau, Christopher
• Primary Examiner(s): Kincaid, Kristine
• Assistant Examiner(s): Tran, Mylinh T.
• Application Number:
  US09/375,822
• Publication Number:
  US 20030052906A1
• Time in Patent Office: 1,520 Days
• Field of Search: 345/804, 345/725, 345/719, 345/726, 345/629, 345/634, 345/722, 345/781, 345/723, 345/853, 382/232, 382/235, 382/173, 382/180
• US Class Current: 715/723
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 19/20 using video object coding

Abstract

A video processing environment includes a user interface and processing shell from which various video processing ‘plug-in’ programs are accessed. The shell insulates the plug-ins from the intricacies of reading various file formats. The user interface allows an operator to load a video sequence, define and view one or more video objects on any one or more frames of the video sequence, edit existing video object segmentations, view video objects across a series of video frames, and encode video objects among a video sequence in a desired format. Various encoding parameters can be adjusted allowing the operator to view the video sequence encoded at the various parameter settings. The user interface includes a video window, a time-line window, a zoom window, a set of menus including a menu of plug-in programs, and a set of dialogue boxes, including encoding parameter dialogue boxes.

Claims

1. An apparatus embodying an interactive video processing environment, the apparatus comprising:

means for generating display content in a main display window of the interactive video processing environment;

means for generating display content, pertaining to a segmented video object of an image, in a first subordinate window displayed within the main window, the display content indicating a portion corresponding to an outlined part of the object by emphasis; and

means for generating encoding control parameter display content, pertaining to an image encoding process for the segmented video object in the first subordinate window, in a second subordinate window concurrently displayed with the first subordinate window within the main window.

2. The apparatus of claim 1, further comprising:

means for generating display content in the first subordinate window for tracking the segmented video object among a plurality of video frames.

3. The apparatus of claim 1, in which said means for generating display content in the second subordinate window generates encoding control parameter display content, including a desired bit rate and a desired intra-coding period.

4. The apparatus of claim 1, in which said means for generating display content in the second subordinate window generates encoding control parameter display content including encoding control parameters for encoding the segmented video object across a plurality of video frames.

5. An apparatus embodying an interactive video processing environment, the apparatus comprising:

means for generating display content in a main display window of the interactive video processing environment;

means for generating display content, pertaining to a segmented video object of an image, in a first subordinate window displayed within the main window, the display content indicating a portion corresponding to an outlined part of the object by emphasis; and

means for generating encoding status display content, pertaining to an image encoding process for the segmented video object in the first subordinate window, in a second subordinate window concurrently displayed with the first subordinate window within the main window, said encoding status display content comprising encoding status information for an encoding of a plurality of segmented video objects across a plurality of video frames.

6. The apparatus of claim 5, in which the display content in the second subordinate window comprises a bit rate for an encoding of each one of the plurality of segmented video objects.

7. The apparatus of claim 5, in which the display content in the second subordinate window comprises a signal to noise ratio for an encoding of each one of the plurality of segmented video objects.

8. The apparatus of claim 5, in which the second subordinate window comprises a plurality of display graphs, in which there is a one to one correspondence between respective ones of the plurality of display graphs and each one of the plurality of segmented video objects.

9. The apparatus of claim 5, further comprising:

means for logically combining the plurality of segmented video objects into a combined object, and wherein said means for generating display content pertaining to said encoding pertains to an encoding of the combined object.

10. An apparatus embodying an interactive video processing environment, the apparatus comprising:

means for generating display content in a main display window of the interactive video processing environment;

means for generating a plurality of first subordinate windows concurrently in the main window, a first one of the plurality of first subordinate windows displaying a time sequence of video frames, a second one of the plurality of first subordinate windows displaying a time line;

means for generating display content, pertaining to a segmented video object of an image, in one of the plurality of first subordinate windows displayed within the main window, the display content indicating a portion corresponding to an outlined part of the object by emphasis,

wherein said means for generating display content pertaining to the segmented video object generates a marking along the time-line for each video frame that has been segmented to define the segmented video object; and

means for generating display content, pertaining to an image encoding process in a second subordinate window concurrently displayed with the first subordinate window within the main window.

11. A digital processor readable storage medium for storing processor-executable instructions and processor-accessible data for maintaining an interactive video processing environment of display windows in response to user inputs, the medium comprising:

code means for generating display content in a main display window of the interactive video processing environment;

code means for generating display content, pertaining to a segmented video object of an image, in a first subordinate window displayed within the main window, the display content indicating a portion corresponding to an outlined part of the object by emphasis; and

code means for generating encoding control parameter display content, pertaining to an image encoding process, in a second subordinate window concurrently displayed with the first subordinate window within the main window.

12. The medium of claim 11 further comprising:

code means for generating display content in the first subordinate window for tracking the segmented video object among a plurality of video frames.

13. The medium of claim 11, in which the code means for generating display content in the second subordinate window comprises means for generating encoding control parameter display content including a desired bit rate and a desired intra-coding period.

14. The medium of claim 11, in which the code means for generating display content in the second subordinate window comprising code means for generating encoding control parameter display content including encoding control parameters for encoding the segmented video object across a plurality of video frames.

15. A digital processor readable storage medium for storing processor-executable instructions and processor-accessible data for maintaining an interactive video processing environment of display windows in response to user inputs, the medium comprising:

code means for generating display content in a main display window of the interactive video processing environment;

code means for generating display content, pertaining to a segmented video object of an image, in a first subordinate window displayed within the main window, the display content indicating a portion corresponding to an outlined part of the object by emphasis; and

code means for generating encoding status display content, pertaining to an image encoding process, in a second subordinate window concurrently displayed with the first subordinate window within the main window, said encoding status display content comprising encoding status information for an encoding of a plurality of segmented video objects across a plurality of video frames.

16. The medium of claim 15, in which the encoding status information display content generating code means comprises means for generating a bit rate for an encoding of each one of the plurality of segmented video objects.

17. The medium of claim 15, in which the encoding status information display content generating code means comprises means for generating a signal to noise ratio for an encoding of each one of the plurality of segmented video objects.

18. The medium of claim 15, in which the second subordinate window comprises a plurality of display graphs, in which there is a one to one correspondence between respective ones of the plurality of display graphs and ones of the plurality of segmented video objects.

19. The medium of claim 15, further comprising:

code means for logically combining the plurality of segmented video objects into a combined object, and in which the code means for generating display content in the second subordinate window generates display content pertaining to an encoding of the combined object.

20. A digital processor readable storage medium for storing processor-executable instructions and processor-accessible data for maintaining an interactive video processing environment of display windows in response to user inputs, the medium comprising:

code means for generating display content in a main display window of the interactive video processing environment;

code means for generating a plurality of first subordinate windows concurrently in the main window, a first one of the plurality of first subordinate windows displaying a time sequence of video frames, a second one of the plurality of first subordinate windows displaying a time line;

code means for generating display content, pertaining to a segmented video object of an image, in one of the plurality of first subordinate windows displayed within the main window, the display content indicating a portion corresponding to an outlined part of the object by emphasis,

wherein said means for generating display content pertaining to the segmented video object generates a marking along the time-line for each video frame that has been segmented to define the segmented video object; and

code means for generating display content, pertaining to an image encoding process, in a second subordinate window concurrently displayed with the first subordinate window within the main window.

21. A method for interactively processing a video sequence on a system having a display, an input device, and a processor, the method comprising the steps of:

generating a main display window of a video processing environment;

generating a first subordinate window within the main display window for displaying a motion video sequence of video frames;

responding to a user input to hold the motion video sequence at a select video frame;

outlining a boundary of a semantically meaningful video object portion of an image displayed within the first subordinate window, in which the boundary is displayed at least while being selected, wherein the boundary distinguishes the semantically meaningful portion from a background portion around the semantically meaningful portion;

segmenting the outlined video object;

playing at least a portion of the motion video sequence, during which the video object is tracked to define the segmented video object among a plurality of frames of the motion video sequence;

selectively generating a second subordinate window within the main display window which is concurrently active with the first subordinate window, the second subordinate window displaying encoding information pertaining to an encoding process of said plurality of frames of the motion video sequence; and

encoding said plurality of video frames.

22. The method of claim 21, further comprising the steps of:

selectively generating a third subordinate window within the main display which is concurrently active with the first subordinate window, the third subordinate window displaying a time line;

generating a marking along the time-line for each video frame in which the segmented video object has been tracked.

23. The method of claim 21, in which a plurality of video objects are segmented and tracked, and further comprising the steps of:

selectively generating a third subordinate window within the main display which is concurrently active with the first subordinate window, the third subordinate window displaying a respective time line for each one of the plurality of video objects;

for each respective time-line, generating a marking along said respective time-line for each video frame in which a corresponding one of the plurality of video objects has been tracked.

24. The method of claim 21, in which a plurality of video objects are segmented and tracked, and further comprising the step of:

logically combining the plurality of objects into a combined object prior to the step for encoding, and wherein the plurality of objects are treated as the combined object during the encoding step.

25. The method for claim 21, in which a plurality of video objects are segmented and tracked, and further comprising the step of:

generating display content in the second subordinate window pertaining to an encoding of a plurality of segmented video objects, said display content comprising a bit rate for the encoding of each one of the plurality of segmented video objects.

26. The method for claim 21, in which a plurality of video objects are segmented and tracked, and further comprising the step of:

generating display content in the second subordinate window pertaining to an encoding of a plurality of segmented video objects, said display content comprising a signal to noise ratio for the encoding of each one of the plurality of segmented video objects.

27. The method of claim 21, in which a plurality of video objects are segmented and tracked, and further comprising the step of:

generating a display graph in the second subordinate window for each one of the segmented video objects.

28. A method for interactively processing a video sequence, the method comprising the steps of:

generating display content in a main window of the interactive video processing environment;

generating display content, pertaining to a segmented video object of an image, in a first subordinate window displayed within the main window, the display content indicating a portion corresponding to an outlined part of the object by emphasis; and

generating encoding control parameter display content, pertaining to an image encoding process for the segmented video object in the first subordinate window, in a second subordinate window concurrently displayed with the first subordinate window within the main window.

29. A method for interactively processing a video sequence, the method comprising the steps of:

generating display content in a main window of the interactive video processing environment;

generating display content, pertaining to a segmented video object of an image, in a first subordinate window displayed within the main window, the display content indicating a portion corresponding to an outlined part of the object by emphasis; and

generating encoding status display content, pertaining to an image encoding process for the image in the first subordinate window, in a second subordinate window concurrently displayed with the first subordinate window within the main window, said encoding status display content comprising encoding status information for an encoding of a plurality of segmented video objects across a plurality of video frames.

30. A method for interactively processing a video sequence, the method comprising the steps of:

generating display content in a main window of the interactive video processing environment;

generating a plurality of first subordinate windows concurrently in the main window, a first one of the plurality of first subordinate windows displaying a time sequence of video frames, a second one of the plurality of first subordinate windows displaying a time line;

generating display content, pertaining to a segmented video object of an image, in one of the plurality of first subordinate windows displayed within the main window, the display content indicating a portion corresponding to an outlined part of the object by emphasis, wherein said means for generating display content pertaining to the segmented video object generates a marking along the time-line for each video frame that has been segmented to define the segmented video object; and

generating display content, pertaining to an image encoding process in a second subordinate window concurrently displayed with the first subordinate window within the main window.