INTERACTIVE VIDEO OBJECT PROCESSING ENVIRONMENT HAVING CONCURRENTLY ACTIVE SUBORDINATE WINDOWS

• US 20030052906A1
• Filed: 08/16/1999
• Published: 03/20/2003
• Est. Priority Date: 08/16/1999
• Status: Active Grant

• Current Assignee: University of Washington
• Original Assignee: University of Washington
• Inventors: KIM, YONGMIN, KIM, DONGLOK, LAU, CHRISTOPHER
• Granted Patent: US 6,633,309 B2
• Time in Patent Office: Days
• US Class Current: 345/700
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 19/20 using video object coding

Abstract

A video processing environment includes a user interface and processing shell from which various video processing ‘plug-in’ programs are accessed. The shell insulates the plug-ins from the intricacies of reading various file formats. The user interface allows an operator to load a video sequence, define and view one or more video objects on any one or more frames of the video sequence, edit existing video object segmentations, view video objects across a series of video frames, and encode video objects among a video sequence in a desired format. Various encoding parameters can be adjusted allowing the operator to view the video sequence encoded at the various parameter settings. The user interface includes a video window, a time-line window, a zoom window, a set of menus including a menu of plug-in programs, and a set of dialogue boxes, including encoding parameter dialogue boxes.

Claims

1. A system including a display, an input device, a processor and a memory storage media and embodying an interactive video processing environment, the system further comprising:

code means for generating display content in a main display window of the interactive video processing environment;

code means for generating display content, pertaining to a segmented video object of an image, in a first subordinate window displayed within the main window; and

code means for generating display content, pertaining to an image encoding process, in a second subordinate window concurrently displayed with the first subordinate window within the main window.

2. The system of claim 1, further comprising:

code means for generating display content in the first subordinate window for tracking the segmented video object among a plurality of video frames.

3. The system of claim 1, in which the code means for generating display content in the first subordinate window generates an image in the first subordinate window, the image including the segmented video object, and in which the code means for generating display content in the second subordinate window generates display content including encoding control parameters for encoding the image.

4. The system of claim 3, in which the code means for generating display content in the second subordinate window generates display content including a desired bit rate and a desired intra-coding period.

5. The system of claim 1, in which the code means for generating display content in the second subordinate window generates display content including encoding control parameters for encoding the segmented video object across a plurality of video frames.

6. The system of claim 1, in which the code means for generating display content in the second subordinate window generates display content including encoding status information for an encoding of a plurality of segmented video objects across a plurality of video frames.

7. The system of claim 6, in which the display content in the second subordinate window comprises a bit rate for an encoding of each one of the plurality of segmented video objects.

8. The system of claim 6, in which the display content in the second subordinate window comprises a signal to noise ratio for an encoding of each one of the plurality of segmented video objects.

9. The system of claim 6, in which the second subordinate window comprises a plurality of display graphs, in which there is a one to one correspondence between respective ones of the plurality of display graphs and each one of the plurality of segmented video objects.

10. The system of claim 6, further comprising:

code means for logically combining the plurality of segmented video objects into a combined object, and wherein the code means for generating display content pertaining to said encoding pertains to an encoding of the combined object.

11. The system of claim 1, in which the input device comprises a pointing device, the system displaying a plurality of first subordinate windows concurrently in the main window, a first one of the plurality of first subordinate windows displaying a video frame, a second one of the plurality of first subordinate windows displaying a zoomed in area of the video frame currently displayed in the first one, wherein a cursor of the pointing device is concurrently displayed in the first one and the second one of the plurality of first subordinate windows, and wherein the second one automatically scrolls to maintain the pointing device cursor in view.

12. The system of claim 11, in which the code means for generating display content pertaining to the segmented video object generates an outline in response to user-selected points in the first one of the plurality of first subordinate windows.

13. The system of claim 11, in which the video frame displayed in the first one of the plurality of first subordinate windows comprises video content including the segmented video object and other video content, in which the code means for generating display content pertaining to the segmented video object generates a translucent object which overlays the segmented video object, the translucent object distinguishing the segmented video object from the other video content in the first one of the plurality of first subordinate windows.

14. The system of claim 1, in which the input device comprises a pointing device, the system displaying a plurality of first subordinate windows concurrently in the main window, a first one of the plurality of first subordinate windows displaying a time sequence of video frames, a second one of the plurality of first subordinate windows displaying a time line, wherein the code means for generating display content pertaining to the segmented video object generates a marking along the time-line for each video frame that has been segmented to define the segmented video object.

15. A digital processor readable storage medium for storing processor-executable instructions and processor-accessible data for maintaining an interactive video processing environment of display windows in response to user inputs, the medium comprising:

code means for generating display content in a main display window of the interactive video processing environment;

code means for generating display content pertaining to a segmented video object in a first subordinate window displayed within the main window; and

code means for generating display content, pertaining to an image encoding process, in a second subordinate window concurrently displayed with the first subordinate window within the main window.

16. The medium of claim 15 further comprising:

code means for generating display content in the first subordinate window for tracking the segmented video object among a plurality of video frames.

17. The medium of claim 15, in which the code means for generating display content in the first subordinate window generates an image in the first subordinate window, the image including the segmented video object, and in which the code means for generating display content in the second subordinate window generates display content including encoding control parameters for encoding the image.

18. The medium of claim 15, in which the code means for generating display content in the second subordinate window comprises means for generating display content including a desired bit rate and a desired intra-coding period.

19. The medium of claim 15, in which the code means for generating display content in the second subordinate window comprising code means for generating display content including encoding control parameters for encoding the segmented video object across a plurality of video frames.

20. The medium of claim 15, in which the code means for generating display content in the second subordinate window comprises code means for generating display content including encoding status information for an encoding of a plurality of segmented video objects across a plurality of video frames.

21. The medium of claim 20, in which the encoding status information display content generating code means comprises means for generating a bit rate for an encoding of each one of the plurality of segmented video objects.

22. The medium of claim 20, in which the encoding status information display content generating code means comprises means for generating a signal to noise ratio for an encoding of each one of the plurality of segmented video objects.

23. The medium of claim 20, in which the second subordinate window comprises a plurality of display graphs, in which there is a one to one correspondence between respective ones of the plurality of display graphs and ones of the plurality of segmented video objects.

24. The medium of claim 20, further comprising:

code means for logically combining the plurality of segmented video objects into a combined object, and in which the code means for generating display content in the second subordinate window generates display content pertaining to an encoding of the combined object.

25. The medium of claim 15, further comprising:

code means for generating a plurality of first subordinate windows concurrently in the main window, a first one of the plurality of first subordinate windows displaying a video frame, a second one of the plurality of first subordinate windows displaying a zoomed in area of the video frame currently displayed in the first one, wherein a cursor of a pointing device is concurrently displayed in the first one and the second one of the plurality of first subordinate windows, and wherein the second one automatically scrolls to maintain the pointing device cursor in view.

26. The medium of claim 25, in which the code means for generating display content pertaining to the segmented video object generates an outline in response to user-selected points in the first one of the plurality of first subordinate windows.

27. The medium of claim 25, in which the video frame displayed in the first one of the plurality of first subordinate windows comprises video content including the segmented video object and other video content, in which the code means for generating display content pertaining to the segmented video object generates a translucent object which overlays the segmented video object, the translucent object distinguishing the segmented video object from the other video content in the first one of the plurality of first subordinate windows.

28. The medium of claim 15, further comprising:

code means for generating a plurality of first subordinate windows concurrently in the main window, a first one of the plurality of first subordinate windows displaying a time sequence of video frames, a second one of the plurality of first subordinate windows displaying a time line, wherein the code means for generating display content pertaining to the segmented video object generates a marking along the time-line for each video frame that has been segmented to define the segmented video object.

29. A method for interactively processing a video sequence on a system having a display, an input device, and a processor, the method comprising the steps of:

generating a main display window of a video processing environment;

generating a first subordinate window within the main display window for displaying a motion video sequence of video frames;

responding to a user input to hold the motion video sequence at a select video frame;

outlining a boundary of a video object;

segmenting the outlined video object;

playing at least a portion of the motion video sequence, during which the video object is tracked to define the segmented video object among a plurality of frames of the motion video sequence;

selectively generating a second subordinate window within the main display window which is concurrently active with the first subordinate window, the second subordinate window displaying information pertaining to an encoding process of said plurality of frames of the motion video sequence; and

encoding said plurality of video frames.

30. The method of claim 29, further comprising the steps of:

selectively generating a third subordinate window within the main display which is concurrently active with the first subordinate window, the third subordinate window displaying a zoomed in area of the first subordinate window, wherein a cursor of the input device is concurrently displayed in the first subordinate window and the third subordinate window;

automatically scrolling the third subordinate window to maintain the pointing device cursor in view during the step of outlining the boundary of the video object.

31. The method of claim 29, further comprising the step of:

overlaying a translucent object onto the segmented video object.

32. The method of claim 29, further comprising the steps of:

selectively generating a third subordinate window within the main display which is concurrently active with the first subordinate window, the third subordinate window displaying a time line;

generating a marking along the time-line for each video frame in which the segmented video object has been tracked.

33. The method of claim 29, in which a plurality of video objects are segmented and tracked, and further comprising the steps of:

selectively generating a third subordinate window within the main display which is concurrently active with the first subordinate window, the third subordinate window displaying a respective time line for each one of the plurality of video objects;

for each respective time-line, generating a marking along said respective time-line for each video frame in which a corresponding one of the plurality of video objects has been tracked.

34. The method of claim 29, in which a plurality of video objects are segmented and tracked, and further comprising the step of:

logically combining the plurality of objects into a combined object prior to the step for encoding, and wherein the plurality of objects are treated as the combined object during the encoding step.

35. The method for claim 29, in which a plurality of video objects are segmented and tracked, and further comprising the step of:

generating display content in the second subordinate window pertaining to an encoding of a plurality of segmented video objects, said display content comprising a bit rate for the encoding of each one of the plurality of segmented video objects.

36. The method for claim 29, in which a plurality of video objects are segmented and tracked, and further comprising the step of:

generating display content in the second subordinate window pertaining to an encoding of a plurality of segmented video objects, said display content comprising a signal to noise ratio for the encoding of each one of the plurality of segmented video objects.

37. The method of claim 29, in which a plurality of video objects are segmented and tracked, and further comprising the step of:

generating a display graph in the second subordinate window for each one of the segmented video objects.