VOLUME RECOGNITION METHOD AND SYSTEM

• US 20100208035A1
• Filed: 04/20/2007
• Published: 08/19/2010
• Est. Priority Date: 04/20/2007
• Status: Active Grant

• Current Assignee: SoftKinetic Software
• Original Assignee: Softkinetic S.A. (Sony Group Corp.)
• Inventors: Baele, Xavier, Roy, Jeremie, Desmecht, Laurent, Pinault, Gilles
• Granted Patent: US 8,175,374 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

The present invention relates to a volume recognition method comprising the steps of:

a) capturing three-dimensional image data using a 3D imaging system 3, wherein said image data represent a plurality of points 5, each point 5 having at least a set of coordinates in a three-dimensional space;
b) grouping at least some of the points 5 in a set of clusters 6;
c) selecting, according to a first set of parameters such as position and size, a cluster 6 corresponding to an object of interest 1 located in range of said imaging system 3;
d) grouping at least some of the points 5 of the selected cluster 6 in a set of sub-clusters according to a second set of parameters comprising their positions in the three-dimensional space, wherein each sub-cluster has a centroid 11 in the three-dimensional space; and
e) associating a volume 12 to each of at least some of said sub-clusters, wherein said volume 12 is fixed to the centroid 11 of said sub-cluster. The present invention also relates to a volume recognition system for carrying out this method.

Claims

1. Volume recognition method comprising the steps of:

a) capturing three-dimensional image data using a 3D imaging system (3), wherein said image data represent a plurality of points (5), each point (5) having at least a set of coordinates in a three-dimensional space;

b) grouping at least some of the points (5) in a set of clusters (6); and

c) selecting, according to a first set of parameters such as position and size, a cluster (6) corresponding to an object of interest (1) located in range of said imaging system (3);

and characterised in that it further comprises the steps of;

d) grouping at least some of the points (5) of the selected cluster (6) in a set of sub-clusters according to a second set of parameters comprising their positions in the three-dimensional space, wherein each sub-cluster has a centroid (11) in the three-dimensional space; and

e) associating a volume (12) to each of at least some of said sub-clusters, wherein said volume (12) is fixed to the centroid (11) of said sub-cluster.

2. Volume recognition method according to claim 1, wherein a K-means algorithm is used to group said points of the selected cluster (6) in a predetermined number K of sub-clusters.

3. Volume recognition method according to claim 1, wherein the volume (12) associated to a sub-cluster is a sphere, preferably centred on the centroid (11) of said sub-cluster.

4. Volume recognition method according to claim 1, wherein said grouping of points (5) in clusters (6) is carried out according to a method comprising the following steps:

a) creating a first cluster (6) comprising a first point (5); and

b) executing the following operations for each other point (5);

i) finding the cluster (6) whose centroid (7) is closest to said other point in the three-dimensional space; and

ii) creating an additional cluster (6) comprising said other point if the absolute distance in the three-dimensional space between said other point (5) and said closest cluster centroid (7) is higher than a predetermined threshold θ

, and the number of clusters (6) is still under a predetermined maximum q;

or

iii) adding said other point (5) to the cluster (6) whose centroid (7) is closest to said other point (5) if said absolute distance is not higher than the predetermined threshold θ

, or the number of clusters (6) has already reached said predetermined maximum q.

5. Volume recognition method according to claim 4, wherein said grouping of points (5) in clusters (6) further comprises the steps of:

a) determining whether two of said clusters (6) are connected; and

b) merging connected clusters (6).

6. Volume recognition method according to claim 5, wherein determining whether two of said clusters (6) are connected comprises the steps of:

a) calculating the standard deviation of the distribution of the projections of the points (5) of each one of said two clusters (6) along an axis (8) linking the centroids (7) of the two clusters (6); and

b) checking whether the sum of the standard deviations multiplied by a predetermined factor S, for example 2, is higher than the absolute distance between the centroids (7) of the two clusters (6).

7. Volume recognition method according to claim 1, wherein said imaging system (3) comprises a time-of-flight 3D camera, a stereo camera, a plurality of cameras located in different positions in the three-dimensional space, or a LIDAR, sonar or radar system.

8. Volume recognition method according to claim 1, wherein said image data comprise at least depth and zenith and azimuth angles of each point (5), and further comprising a step of transforming depth and zenith and azimuth angles of at least some of these points (5) into three-dimensional Cartesian coordinates.

9. Volume recognition method according to claim 1, wherein said object of interest (1) is at least part of a human body, preferably standing.

10. Volume recognition method according to claim 9, further comprising the step of calculating the approximated centre of mass (17) and main axis (18) of the torso (19) of said body.

11. Volume recognition method according to claim 10, wherein said approximated centre of mass (17) and main axis (18) of the torso (19) are calculated by executing the following steps:

a) calculating the centroid (7) and main axis (16) of said selected cluster (6);

b) calculating the distribution curve (20) of the distances of the points (5) of the selected cluster (6) with respect to said main axis (16) of the selected cluster (6);

c) calculating an inflection point (21) in said distribution curve (20);

d) selecting the points (5) with distances with respect to said main axis (16) of the selected cluster (6) inferior to D·

s, wherein s is the distance of said inflection point (21) to said main axis (16) of the selected cluster (6) and D is a factor of at most 1.25, preferably at most 1; and

e) calculating said centre of mass (17) and main axis (18) of the torso (19) as the centroid and main axis of the selected points (5).

12. Volume recognition method according to claim 10, wherein signals are transmitted to a data processing system (2) according to the position of the centre of mass (17) of said torso (19) and/or its main axis (18) and/or the orientation of said main axis (18) of said torso (1).

13. Volume recognition method according to claim 10, further comprising the step of measuring the height of the body.

14. Volume recognition method according to claim 13, wherein said height of the body is measured by calculating the heights of the points (5) among those of said selected cluster (6) that are closer than a predetermined distance to the main axis (18) of the torso (19), filtering said heights, preferably by median filtering, and selecting the maximum value of said heights after filtering.

15. Volume recognition method according to claim 14, wherein said measure of the height of the body is only considered as valid if a set of conditions is met, such as said main axis (18) of the torso (19) being substantially vertical.

16. Volume recognition method according to claim 1, wherein the volumes (12) associated with said set of sub-clusters are represented in a virtual environment generated by a data processing system (2).

17. Volume recognition method according to claim 16, wherein there is a collision and/or proximity check between the representation of the volumes (12) of said set of sub-clusters and a set of elements (14) of said virtual environment, so as to interact with said set of elements (14) of the virtual environment.

18. Volume recognition method according to claim 1, wherein a set of links (28) between sub-clusters is established using criteria such as absolute distance between the centroids (11) of the sub-clusters, the presence of points (5) between sub-cluster, etc.

19. Volume recognition method according to claim 18, where a set of extremities (29) of said object of interest (1) is identified according to said links (28).

20. Volume recognition method according to claim 19, wherein at least one of said extremities (29) is labelled according to a predetermined pattern, for example that of a human body.

21. Volume recognition method according to claim 19, wherein signals are transmitted to a data processing system (2) according to an absolute and/or relative position and/or movement of at least one of said extremities (29).

22. Volume recognition system comprising an imaging system (3) for capturing three-dimensional image data representing a plurality of points (5), each point (5) having at least a set of coordinates in a three-dimensional space, and at least some of said points (5) corresponding to an object of interest (1) located in range of said imaging system (3), and a data processing system (2) connected to said imaging system (3) and programmed for carrying out, in cooperation with said imaging system (3), a volume recognition method according to any one of the previous claims.