METHOD AND DEVICE FOR BOUNDING AN OBJECT IN A VIDEO

• US 20160343411A1
• Filed: 05/15/2016
• Published: 11/24/2016
• Est. Priority Date: 05/18/2015
• Status: Active Grant

• Current Assignee: Thomson Licensing (Vantiva SA)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: SIROT, Joel, CHEVALLIER, Louis, VIGOUROUX, Jean-Ronan
• Granted Patent: US 9,847,102 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

The invention relates to a method for bounding an object in a video sequence F_x,y,t. The method includes obtaining a subset of pixels located in the object to annotate, in each frame of the video sequence. Spatio-temporal slicing is performed on the video sequence F_x,y,t, centered on the obtained subsets of pixels, resulting in a first image F_y,t obtained by an horizontal concatenation of first slices, comprising the obtained subsets of pixels, and resulting in a second image F_x,t obtained by a vertical concatenation of second slices. A trajectory of the obtained subsets of pixels is displayed on both the first F_y,t and second F_x,t image. A bounding form around the object to annotate is obtained out of four points in each frame of the video sequence, wherein the coordinates of the four points of a frame t are obtained from the coordinates of the points located in the first and second boundary of the first and second image for that frame t.

Claims

1. A method, for bounding an object in a video sequence from a subset of pixels per frame of said video sequence, the method being characterized in that it further comprises:

obtaining a first image from a first spatio-temporal slicing, wherein said first image is an horizontal concatenation of first slices comprising the subset of pixels for frames along said video sequence;

obtaining a second image from a second spatio-temporal slicing, wherein said second image is a vertical concatenation of second slices comprising the subset of pixels for frames along said video sequence, each of said second slices being orthogonal to the first slice of a same frame;

obtaining on each of said first and second images a first and a second boundary around the subsets of pixels per frame by means of a contour detection method;

obtaining a bounding form out of four points, around said object in each frame of the video sequence, wherein the coordinates of said four points in a frame t are obtained from the coordinates of the points located in the first and second boundary of the first and second image for that frame t.

2. The method according to claim 1, wherein each of said first slices is a vertical slice.

3. The method according to claim 1, wherein each of said first slices is inclined with respect to the vertical.

4. The method according to claim 3, wherein the inclination α

of said first slices with respect to the vertical, is constant for a set of successive frames of said video sequence.

5. The method according to claim 3, wherein the inclination α

of said first slices with respect to the vertical is adjustable by a user for a set of successive frames of said video sequence.

6. The method according to claim 3, wherein the inclination α

of the first slice with respect to the vertical, is adjustable by a user on a plurality of frames of said video sequence, said inclination α

being interpolated to the rest of the frames of said video sequence.

7. The method according to claim 1, wherein said subset of pixels is selected among:

a single pixel,

a block of four pixels,

a block of eight pixels,

a block of sixteen pixels.

8. The method according to claim 1, wherein said bounding form is selected among:

a rectangle drawn out of said four points,

an ellipse comprising said four points,

the inscribed ellipse of a rectangle drawn out of said four points.

9. The method according to claim 1, further comprising obtaining a first and a second trajectory of the subsets of pixels per frame on each of said first and second images, adjusting the trajectory of said subsets of pixels in said first image, obtaining an updated version of said second image, obtaining an updated version of said second trajectory, obtaining an updated version of said first and second boundary around said updated version of said second trajectory on said updated version of said second image, and obtaining an updated version of said bounding form around said object.

10. The method according to claim 9, wherein said first trajectory is adjusted by a user.

11. A device for bounding an object in a video sequence from a subset of pixels per frame, said device comprising a processor, characterized in that the processor is configured to:

obtain a first image from a first spatio-temporal slicing, wherein said first image is an horizontal concatenation of first slices comprising the subset of pixels for frames along said video sequence;

obtain a second image from a second spatio-temporal slicing, wherein said second image is a vertical concatenation of second slices comprising the subset of pixels for frames along said video sequence, each of said second slices being orthogonal to the first slice of a same frame;

obtain on each of said first and second images a first and second boundary around the subsets of pixels per frame by means of a contour detection method;

obtain a bounding form out of four points, around said object in each frame of the video sequence, wherein the coordinates of said four points in a frame t are obtained from the coordinates of the points located in the first and second boundary of the first and second image for that frame t.

12. The device according to claim 11, wherein each of said first slices is inclined with respect to the vertical.

13. The device according to claim 12, wherein the inclination α

of said first slices with respect to the vertical is adjustable by a user for a set of successive frames of said video sequence.

14. A computer program for bounding an object in a video sequence from a subset of pixels per frame, said computer program characterized in that it comprises program code instructions executable by a processor for:

obtaining a first image from a first spatio-temporal slicing, wherein said first image is an horizontal concatenation of first slices comprising the subset of pixels for frames along said video sequence;

obtaining a second image from a second spatio-temporal slicing, wherein said second image is a vertical concatenation of second slices comprising the subset of pixels for frames along said video sequence, each of said second slices being orthogonal to the first slice of a same frame;

obtaining on each of said first and second images a first and second boundary around the subsets of pixels per frame by means of a contour detection method;

obtaining a bounding form out of four points, around said object in each frame of the video sequence, wherein the coordinates of said four points in a frame t are obtained from the coordinates of the points located in the first and second boundary of the first and second image for that frame t.

15. A computer program product for bounding an object in a video sequence from a subset of pixels per frame, said computer program product being stored on a non-transitory computer readable medium, and characterized in that it comprises program code instructions executable by a processor for:

obtaining a first image from a first spatio-temporal slicing, wherein said first image is an horizontal concatenation of first slices comprising the subset of pixels for frames along said video sequence;

obtaining a second image from a second spatio-temporal slicing, wherein said second image is a vertical concatenation of second slices comprising the subset of pixels for frames along said video sequence, each of said second slices being orthogonal to the first slice of a same frame;

obtaining on each of said first and second images a first and second boundary around the subsets of pixels per frame by means of a contour detection method;

obtaining a bounding form out of four points, around said object in each frame of the video sequence, wherein the coordinates of said four points in a frame t are obtained from the coordinates of the points located in the first and second boundary of the first and second image for that frame t.