METHOD OF TRACKING SHAPE IN A SCENE OBSERVED BY AN ASYNCHRONOUS LIGHT SENSOR

• US 20170053407A1
• Filed: 04/24/2015
• Published: 02/23/2017
• Est. Priority Date: 04/30/2014
• Status: Active Grant

• Current Assignee: Centre National De La Recherche Scientifique, Sorbonne Universite
• Original Assignee: Centre National De La Recherche Scientifique, University Pierre et Marie Curie
• Inventors: Ieng, Siohoi, Regnier, Stephane, Zhenjiang, Ni, Benosman, Ryad
• Granted Patent: US 10,109,057 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 2207/10141 Special mode during image acquisition
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

Asynchronous information is provided by a sensor having a matrix of pixels disposed opposite the scene. The asynchronous information includes, for each pixel of the matrix, successive events originating from this pixel, that may depend on variations of light in the scene. A model representing the tracked shape of an object is updated after detecting events attributed to this object in the asynchronous information. Following detection, the updating of the model includes an association of a point of the model with the event detected by minimizing a criterion of distance with respect to the pixel of the matrix from which the detected event originates. The updated model is then determined as a function of the pixel of the matrix from which the detected event originates and attributed to the object and of the associated point in the model, independently of the associations performed before the detection of this event.

Claims

1-18. -18. (canceled)

19. A method of tracking a shape in a scene, comprising:

receiving asynchronous information from a light sensor having a matrix of pixels arranged opposite the scene, wherein the asynchronous information comprises, for each pixel of the matrix, successive events originating asynchronously from said pixel; and

updating a model comprising a set of points and representing a tracked shape of an object, after detecting events attributed to said object in the asynchronous information, wherein the updating comprises, following detection of an event;

associating with the detected event a point of the model having a minimum distance with respect to the pixel of the matrix from which the detected event originates; and

determining the updated model according to the pixel of the matrix from which the detected event originates and the associated point, and independently of the associations performed before the detection of said event.

20. The method of claim 19, wherein when no point of the model is located at a distance less than a threshold with respect to the pixel from which a detected event originates, the model is not updated following the detection of said event which is not attributed to the object.

21. The method of claim 19, further comprising:

estimating a plane of displacement of the object, wherein the plane of displacement is estimated in a space with three dimensions, namely two spatial dimensions corresponding to two directions of the matrix of pixels and one time dimension, by minimizing a distance criterion in relation to a set of points that mark detected events which were attributed to the object during a time window ending at a current instant; and

following detection of an event, taking into account the estimated plane of displacement of the object in order to decide whether or not to attribute said event to the object.

22. The method of claim 21, wherein a plurality of objects have respective shapes tracked in the scene, wherein each one of the plurality of objects has a respective model updated after detection of events attributed thereto and an estimated plane of displacement, wherein the method further comprises, following detection of an event attributable to at least two of the objects:

calculating respective distances, in the three-dimensional space, between a point marking said detected event and the planes of displacement respectively estimated for said objects; and

attributing said detected event to the object for which the calculated distance is minimal.

23. The method of claim 21, further comprising:

estimating a statistical distribution of distances between the plane of displacement of the object and points marking detected events which were attributed to the object; and

following detection of an event, taking into account the estimated plane of displacement of the object and the estimated statistical distribution in order to decide whether or not to attribute said event to the object.

24. The method of claim 23, further comprising:

determining an interval of admissible distance values based on the estimated statistical distribution; and

not attributing a detected event to the object if the point marking said detected event in the three-dimensional space has, in relation to the estimated plane of displacement, a distance falling outside the interval of the admissible distance values.

25. The method of claim 19, wherein a plurality of objects have respective shapes tracked in the scene, wherein each of the objects has a respective model updated after detection of events attributed thereto, wherein the method further comprises, following detection of an event:

if only one of the objects satisfies a condition of having in its model a point that has a distance less than a threshold with respect to the pixel of the matrix from which the detected event originates, attributing the detected event to said one of the objects.

26. The method of claim 19, wherein a plurality of objects have respective shapes tracked in the scene, wherein each one of the objects has a respective model updated after detection of events attributed thereto, wherein the method further comprises, following detection of an event attributable to at least two of the objects:

for each object to which the detected event is attributable, associating with the detected event a point of the model of said object by minimizing a respective distance criterion with respect to the pixel of the matrix from which the detected event originates; and

attributing the detected event to the object for which the minimized criterion of distance is lowest.

27. The method of claim 19, wherein a plurality of objects have respective shapes tracked in the scene, wherein each one of the plurality of objects has a respective model updated after detection of events attributed thereto, and wherein, following detection of an event attributable to at least two of the objects, the detected event is not assigned to any of said at least two of the objects.

28. The method of claim 19, wherein a plurality of objects have respective shapes tracked in the scene, wherein each one of the plurality of objects has a respective model updated after detection of events attributed to it, and wherein, following detection of an event attributable to at least two of the objects, the detected event is assigned to each one of the objects to which the detected event is attributable.

29. The method of claim 28, wherein updating the models of objects to which the detected event is attributed is carried out with weightings that depend on the distance criteria respectively minimized for said objects.

30. The method of claim 19, wherein a plurality of objects have respective shapes tracked in the scene, wherein each one of the plurality of objects has a respective model updated after detection of events attributed thereto, the method further comprising:

for each object, estimating a rate of events attributed to said object and memorizing an instant at which the last event attributed to said object was detected; and

following detection of an event attributable to at least two of the objects, attributing the detected event to the one of the objects for which the product of the estimated rate of events by the time interval between the memorized instant and the instant of detection of said event is closest to 1.

31. The method of claim 19, wherein determining the updated model comprises estimating a spatial transformation defined by a set of parameters, and applying the estimated spatial transformation to the model, and wherein estimating the spatial transformation comprises calculating said parameters as a function of a gradient of a distance, in the plane of the matrix of pixels, between the pixel of the matrix from which the detected event originates and a point obtained by applying the spatial transformation to the point of the model associated with the detected event.

32. The method of claim 31, wherein the spatial transformation includes a translation and a rotation in the plane of the matrix of pixels, wherein the translation has a vector Δ

T taken equal to −

η

₁·

∇

_Tf(Δ

θ

₀, Δ

T₀) and the rotation has an angle Δ

θ

taken equal to −

η

₂·

∇

_θ

f(Δ

θ

₀, Δ

T₀) where η

₁ and n₂ are predefined positive convergence steps and Δ

θ

₀ and Δ

T₀ are particular values of the angle of rotation and of the translation vector.

33. The method of claim 32, wherein said particular values of the angle of rotation and of the translation vector are respectively Δ

θ

₀=mĉ

p and Δ

T₀=cp−

R<sub>Δ

θ

0</sub>[cm], where p designates the pixel of the matrix from which the detected event originates, m designates the point of the model associated with the detected event, c designates the center of the rotation, cp designates a vector pointing from c to p, cm designates a vector pointing from c to m, and R<sub>Δ

θ

0</sub>[.] is the rotation of center c and of angle Δ

θ

₀.

34. The method of claim 32, wherein the spatial transformation is an affine transformation further including application of respective scaling factors according to two axes included in the matrix of pixels.

35. The method of claim 34, wherein the scaling factors s_x, s_y according to the two axes x, y are respectively calculated according to s_x=1+η

₃·

(|p_x|−

|m_x|) and s_y=1+η

₃·

(|p_y|−

|m_y|) where η

₃ is a predefined positive convergence step, p_x and p_y are respective coordinates, according to the axes x and y, of the pixel of the matrix from which the detected event originates, and m_x and m_y are respective coordinates, according to the axes x and y, of the point of the model associated with the detected event.

36. A shape tracking device, comprising a computer configured to:

receive asynchronous information from a light sensor having a matrix of pixels arranged opposite a scene, wherein the asynchronous information comprises, for each pixel of the matrix, successive events originating asynchronously from said pixel; and

update a model comprising a set of points and representing a tracked shape of an object, after detecting events attributed to said object in the asynchronous information, wherein the updating comprises, following detection of an event;

associating with the detected event a point of the model having a minimum distance with respect to the pixel of the matrix from which the detected event originates; and

determining the updated model according to the pixel of the matrix from which the detected event originates and the associated point, and independently of the associations performed before the detection of said event.