Method and device for bounding an object in a video

• US 9,847,102 B2
• Filed: 05/15/2016
• Issued: 12/19/2017
• Est. Priority Date: 05/18/2015
• Status: Active Grant

• Current Assignee: Thomson Licensing (Vantiva SA)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Sirot, Joel, Chevallier, Louis, Vigouroux, Jean-Ronan
• Primary Examiner(s): DANG, HUNG Q
• Application Number:
  US15/155,059
• Publication Number:
  US 20160343411A1
• Time in Patent Office: 583 Days
• Field of Search: 386278-290
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

The invention relates to a method for bounding an object in a video sequence F_x,y,t. The method includes obtaining a subset of pixels located in the object to annotate, in each frame of the video sequence. Spatio-temporal slicing is performed on the video sequence F_x,y,t, centered on the obtained subsets of pixels, resulting in a first image F_y,t obtained by an horizontal concatenation of first slices, comprising the obtained subsets of pixels, and resulting in a second image F_x,t obtained by a vertical concatenation of second slices. A trajectory of the obtained subsets of pixels is displayed on both the first F_y,t and second F_x,t image. A bounding form around the object to annotate is obtained out of four points in each frame of the video sequence, wherein the coordinates of the four points of a frame t are obtained from the coordinates of the points located in the first and second boundary of the first and second image for that frame t.

Claims

1. A method, comprising:

obtaining a position of a first subset of pixels in an object in at least one frame of a video sequence according to selection data received from a user interface;

obtaining a subset of pixels per frame of the video sequence, resulting in a plurality of subsets of pixels, by interpolating the position of the first subset of pixels to the video sequence;

obtaining a first image from a first spatio-temporal slicing, wherein said first image is a horizontal concatenation of first slices comprising the subset of pixels for frames along said video sequence;

obtaining a second image from a second spatio-temporal slicing, wherein said second image is a vertical concatenation of second slices comprising the subset of pixels for said frames along said video sequence, each of said second slices being orthogonal to the first slice of a same frame;

obtaining on each of said first and second images a first and a second boundary around the plurality of subsets of pixels per frame by means of a contour detection method;

wherein the coordinates of said four points in a frame t are obtained from the coordinates of the points located in the first and second boundary of the first and second image for that frame t.

2. The method according to claim 1, wherein each of said first slices is a vertical slice.

3. The method according to claim 1, wherein said subset of pixels is selected among:

a single pixel,

a block of four pixels,

a block of eight pixels,

a block of sixteen pixels.

4. The method according to claim 1, wherein said bounding form is selected among:

a rectangle drawn out of said four points,

an ellipse comprising said four points,

the inscribed ellipse of a rectangle drawn out of said four points.

5. The method according to claim 1, further comprising obtaining a first and a second trajectory of the subsets of pixels per frame on each of said first and second images, adjusting the trajectory of said subsets of pixels in said first image, obtaining an updated version of said second image, obtaining an updated version of said second trajectory, obtaining an updated version of said first and second boundary around said updated version of said second trajectory on said updated version of said second image, and obtaining an updated version of said bounding form around said object.

6. The method according to claim 5, wherein said first trajectory is adjusted by a user.

7. The method according to claim 1, wherein each of said first slices is inclined with respect to the vertical.

8. The method according to claim 7, wherein the inclination α

of said first slices with respect to the vertical, is constant for a set of successive frames of said video sequence.

9. The method according to any of claims 7 to 8, wherein the inclination α

of said first slices with respect to the vertical is adjustable by a user for a set of successive frames of said video sequence.

10. The method according to any of claims 7 to 8, wherein the inclination α

of the first slice with respect to the vertical, is adjustable by a user on a plurality of frames of said video sequence, said inclination α

being interpolated to the rest of the frames of said video sequence.

11. A device configured to:

obtain a position of a first subset of pixels in an object in at least one frame of a video sequence according to selection data received from a user interface;

obtaining a subset of pixels per frame of the video sequence, resulting in a plurality of subsets of pixels, by interpolating the position of the first subset of pixels to the video sequence;

obtain a first image from a first spatio-temporal slicing, wherein said first image is a horizontal concatenation of first slices comprising the subset of pixels for frames along said video sequence;

obtain a second image from a second spatio-temporal slicing, wherein said second image is a vertical concatenation of second slices comprising the subset of pixels for said frames along said video sequence, each of said second slices being orthogonal to the first slice of a same frame;

obtain on each of said first and second images a first and second boundary around the plurality of subsets of pixels by means of a contour detection method;

obtain a bounding form out of four points, around said object in each frame of the video sequence, wherein the coordinates of said four points in a frame t are obtained from the coordinates of the points located in the first and second boundary of the first and second image for that frame t.

12. The device according to claim 11, wherein each of said first slices is inclined with respect to the vertical.

13. The device according to claim 12, wherein the inclination α

of said first slices with respect to the vertical is adjustable by a user for a set of successive frames of said video sequence.

14. A non-transitory computer program product stored on a non-transitory computer readable medium, and comprising program code instructions executable by a processor for:

obtaining a position of a first subset of pixels in an object in at least one frame of a video sequence according to selection data received from a user interface;

obtaining a subset of pixels per frame of the video sequence, resulting in a plurality of subsets of pixels, by interpolating the position of the first subset of pixels to the video sequence;

obtaining a first image from a first spatio-temporal slicing, wherein said first image is a horizontal concatenation of first slices comprising the subset of pixels for frames along said video sequence;

obtaining a second image from a second spatio-temporal slicing, wherein said second image is a vertical concatenation of second slices comprising the subset of pixels for said frames along said video sequence, each of said second slices being orthogonal to the first slice of a same frame;

obtaining on each of said first and second images a first and second boundary around the plurality of subsets of pixels by means of a contour detection method;

obtaining a bounding form out of four points, around said object in each frame of the video sequence, wherein the coordinates of said four points in a frame t are obtained from the coordinates of the points located in the first and second boundary of the first and second image for that frame t.