METHODS AND APPARATUS TO SPECIFY REGIONS OF INTEREST IN VIDEO FRAMES

• US 20090123069A1
• Filed: 09/26/2008
• Published: 05/14/2009
• Est. Priority Date: 11/09/2007
• Status: Active Grant

• Current Assignee: Nielsen Company (US) LLC (RR Donnelley & Sons Company)
• Original Assignee: Nielsen Company (US) LLC (RR Donnelley & Sons Company)
• Inventors: Deng, Kevin Keqiang, Greenberg, Barry
• Granted Patent: US 8,059,865 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/635 Overlay text, e.g. embedded captions in a TV program
• CPC Class Codes: G06V 2201/09 Recognition of logos

Abstract

Methods and apparatus to specify regions of interest in video frames are disclosed. An example disclosed method comprises determining an initial template region to represent a region of interest whose location is based on a first point selected in a graphical presentation, determining a first modification to perform on the initial template region in response to a second point selected in the graphical presentation, detecting the second selected point in the graphical presentation, and reshaping the initial template region toward the second selected point, the reshaping corresponding to the first modification, the reshaping being performed in response to detecting the second selected point without also requiring the user to select any point substantially on the boundary defining the initial template region to initiate the reshaping.

Claims

1. A method to specify a region of interest in a graphical presentation, the method comprising:

determining an initial template region to represent the region of interest, a location of the initial template region based on a first point selected in the graphical presentation by a user;

determining a first modification to perform on the initial template region in response to a second point selected in the graphical presentation by the user, the first modification at least one of selected by the user or corresponding to a default modification;

detecting the second selected point in the graphical presentation, the second selected point at least one of outside or inside the initial template region, but not substantially on a boundary defining the initial template region; and

reshaping the initial template region toward the second selected point, the reshaping corresponding to the first modification, the reshaping being performed in response to detecting the second selected point without also requiring the user to select any point substantially on the boundary defining the initial template region to initiate the reshaping.

2. A method as defined in claim 1 wherein determining the initial template region to represent the region of interest comprises:

determining a set of pixels in the graphical presentation that are connected to a starting pixel by adjacent pixels that are similar to the starting pixel, wherein the starting pixel corresponds to the first selected point within the region of interest; and

determining the initial template region to be at least one or a polygon or an ellipse bounding the set of pixels determined to be connected to the starting pixel.

3. A method as defined in claim 2 wherein two pixels are similar when the two pixels have at least one of substantially similar luminance values or substantially similar chrominance values.

4. A method as defined in claim 2 wherein the set of pixels is a second set of pixels, and determining the second set of pixels in the graphical presentation that are connected to the starting pixel comprises:

identifying a first set of pixels adjacent to the starting pixel of the graphical presentation and comparing the first set of adjacent pixels to the starting pixel to determine whether each one of the first set of adjacent pixels is similar to the starting pixel; and

recursively comparing adjacent pixels beginning with each one of the first set of adjacent pixels determined to be similar to the starting pixel to determine whether the adjacent pixels are similar, wherein the recursion ends when no more adjacent pixels are determined to be similar, and wherein the second set includes all pixels determined recursively to be similar to the starting pixel.

5. A method as defined in claim 1 wherein the first modification comprises a GROW_TO_POINT modification, and reshaping the initial template region toward the second selected point comprises:

expanding the boundary defining the initial template region uniformly to bound the second selected point when the second selected point is outside the initial template region; and

shrinking the boundary defining the initial template region uniformly to bound the second selected point when the second selected point is inside the initial template region.

6. A method as defined in claim 1 wherein the first modification comprises a GROW_ONE_STEP modification, and reshaping the initial template region toward the second selected point comprises:

expanding the boundary defining the initial template region uniformly by a unit step size when the second selected point is outside the initial template region; and

shrinking the boundary defining the initial template region uniformly by the unit step size when the second selected point is inside the initial template region.

7. A method as defined in claim 1 wherein the first modification comprises a GROW_ONE_DIRECTIONAL_STEP modification, and reshaping the initial template region toward the second selected point comprises:

expanding the boundary defining the initial template region in a first direction toward the second selected point and by a unit step size when the second selected point is outside the initial template region; and

shrinking the boundary defining the initial template region in a second direction toward the second selected point and by the unit step size when the second selected point is inside the initial template region.

8. A method as defined in claim 1 wherein the determined first modification comprises a GROW_TO_POINT_DIRECTIONAL modification, and reshaping the initial template region toward the second selected point comprises:

expanding the boundary defining the initial template region in a first direction toward the second selected point when the second selected point is outside the initial template region; and

shrinking the boundary defining the initial template region in a second direction toward the second selected point when the second selected point is inside the initial template region.

9. A method as defined in claim 1 further comprising returning a reshaped initial template region to a previous shape upon determining that a second modification selected by the user corresponds to an UNDO operation.

10. A method as defined in claim 1 wherein the region of interest is a first region of interest and the reshaped template region is a first reshaped template region, and further comprising:

determining a second reshaped template region to represent a second region of interest according to the method defined in claim 1;

determining a third template region bounding the first reshaped template region representing the first region of interest and the second reshaped template region representing the second region of interest; and

using the third template region to represent an overall region of interest.

11. A method as defined in claim 1 wherein the region of interest is a first region of interest and the reshaped template region is a first reshaped template region, and further comprising:

determining a second reshaped template region to represent a second region of interest according to the method defined in claim 1;

determining a third template region by excluding at least a portion of the second reshaped template region from the first reshaped template region; and

using the third reshaped template region to represent an overall region of interest.

12. A method as defined in claim 1 wherein the region of interest is a first region of interest and the initial template region is a first initial template region, and wherein determining the first initial template region used to represent the first region of interest comprises:

determining a second region of interest that most closely matches the set of reference points detecting as being selected by the user, the set of reference points selecting substantially on a boundary of the first region of interest, the second region of interest at least one of stored or previously created; and

determining the first initial template region used to represent the first region of interest to be a second template region used to represent the determined second region of interest.

13. An article of manufacture storing machine readable instructions which, when executed, cause a machine to:

determine an initial template region to represent a region of interest in a graphical presentation, a location of the initial template region based on a first point selected in the graphical presentation by a user;

determine a first modification from a set of modifications to perform on the initial template region in response to a second point selected in the graphical presentation by the user, the first modification at least one of selected by the user or corresponding to a default modification;

detect the second selected point in the graphical presentation, the second selected point at least one of outside or inside the initial template region, but not substantially on a boundary defining the initial template region; and

reshape the initial template region toward the second selected point, the reshaping corresponding to the determined type of modification, the reshaping being performed in response to detecting the second selected point without also requiring the user to select any point substantially on the boundary defining the initial template region to initiate the reshaping.

14. An article of manufacture as defined in claim 13 wherein the machine readable instructions, when executed, further cause the machine to determine the initial template region to represent the region of interest by:

determining a set of pixels in the graphical presentation that are connected to a starting pixel by adjacent pixels that are similar to the starting pixel, wherein the starting pixel corresponds to the first selected point within the region of interest, and wherein two pixels are similar when the two pixels have at least one of substantially similar luminance values or substantially similar chrominance values; and

determining the initial template region to be at least one or a polygon or an ellipse bounding the set of pixels determined to be connected to the starting pixel.

15. An article of manufacture as defined in claim 13 wherein the set of modifications comprises:

a GROW_TO_POINT modification which, to reshape the initial template region, causes the machine to expand the boundary defining the initial template region uniformly to bound the second selected point when the second selected point is outside the initial template region and to shrink the boundary defining the initial template region uniformly to bound the second selected point when the second selected point is inside the initial template region;

a GROW_ONE_STEP modification which, to reshape the initial template region, causes the machine to expand the boundary defining the initial template region uniformly by a unit step size when the second selected point is outside the initial template region and to shrink the boundary defining the initial template region uniformly by the unit step size when the second selected point is inside the initial template region;

a GROW_ONE_DIRECTIONAL_STEP modification which, to reshape the initial template region, causes the machine to expand the boundary defining the initial template region in a first direction toward the second selected point and by a unit step size when the second selected point is outside the initial template region and to shrink the boundary defining the initial template region in a second direction toward the second selected point and by the unit step size when the second selected point is inside the initial template region; and

a GROW_TO_POINT_DIRECTIONAL modification which, to reshape the initial template region, causes the machine to expand the boundary defining the initial template region in a first direction toward the second selected point when the second selected point is outside the initial template region and to shrink the boundary defining the initial template region in a second direction toward the second selected point when the second selected point is inside the initial template region.

16. A system for marking a region of interest in a graphical presentation, the system comprising:

an output device to display the graphical presentation to a user;

an input device for manipulation by the user to select points within the displayed graphical presentation; and

a graphical user interface (GUI) processing element to;

determine an initial template region to represent the region of interest in the graphical presentation, a location of the initial template region based on a first point selected in the graphical presentation by the user;

determine a first modification from a set of modifications to perform on the initial template region in response to a second point selected in the graphical presentation by the user, the first modification at least one of selected by the user or corresponding to a default modification;

detect the second selected point in the graphical presentation, the second selected point at least one of outside or inside the initial template region, but not substantially on a boundary defining the initial template region; and

reshape the initial template region toward the second selected point, the reshaping corresponding to the determined type of modification, the reshaping being performed in response to detecting the second selected point without also requiring the user to select any point substantially on the boundary defining the initial template region to initiate the reshaping.

17. A system as defined in claim 16 wherein the GUI processing element is to cause the initial and reshaped template regions to be overlaid on the graphical presentation displayed by the output device.

18. A system as defined in claim 16 wherein the GUI processing element is further to determine the initial template region to represent the region of interest by:

determining a set of pixels in the graphical presentation that are connected to a starting pixel by adjacent pixels that are similar to the starting pixel, wherein the starting pixel corresponds to the first selected point within the region of interest, and wherein two pixels are similar when the two pixels have at least one of substantially similar luminance values or substantially similar chrominance values; and

determining the initial template region to be at least one or a polygon or an ellipse bounding the set of pixels determined to be connected to the starting pixel.

19. A system as defined in claim 16 wherein the set of modifications comprises:

a GROW_TO_POINT modification which, to reshape the initial template region, causes the GUI processing element to expand the boundary defining the initial template region uniformly to bound the second selected point when the second selected point is outside the initial template region and to shrink the boundary defining the initial template region uniformly to bound the second selected point when the second selected point is inside the initial template region;

a GROW_ONE_STEP modification which, to reshape the initial template region, causes the GUI processing element to expand the boundary defining the initial template region uniformly by a unit step size when the second selected point is outside the initial template region and to shrink the boundary defining the initial template region uniformly by the unit step size when the second selected point is inside the initial template region;

a GROW_ONE_DIRECTIONAL_STEP modification which, to reshape the initial template region, causes the GUI processing element to expand the boundary defining the initial template region in a first direction toward the second selected point and by a unit step size when the second selected point is outside the initial template region and to shrink the boundary defining the initial template region in a second direction toward the second selected point and by the unit step size when the second selected point is inside the initial template region; and

a GROW_TO_POINT_DIRECTIONAL modification which, to reshape the initial template region, causes the GUI processing element to expand the boundary defining the initial template region in a first direction toward the second selected point when the second selected point is outside the initial template region and to shrink the boundary defining the initial template region in a second direction toward the second selected point when the second selected point is inside the initial template region.

20. A system as defined in claim 16 wherein the region of interest is a first region of interest and the reshaped template region is a first reshaped template region, and wherein the GUI processing element is further to:

determine a second reshaped template region to represent a second region of interest, the second reshaped template region determined by reshaping a second initial template region toward a third selected point, the reshaping corresponding to one of the set of modifications, the reshaping being performed in response to detecting the third selected point at least one of outside or inside the second initial template region without also requiring the user to select any point substantially on a boundary defining the second initial template region to initiate the reshaping; and

determining a third template region by excluding at least a portion of the second reshaped template region from the first reshaped template region; and

using the third reshaped template region to represent an overall region of interest.