Method for the real-time-capable, computer-assisted analysis of an image sequence containing a variable pose
First Claim
1. A method for real-time-capable computer-assisted analysis of an image sequence containing a variable pose of an object composed of interconnected elements movable in relation to each other, wherein the frames of the image sequence have been recorded by a distance measuring camera and are processed by a computer and the frames have brightness and distance data as functions of pixel coordinates of the camera for each frame of the sequence;
- the method comprising;
(a) the computer detecting the pixels of a frame which map the object;
(b) the computer calculating a three-dimensional (3D) aggregate of points within a virtual space, which represents that surface of the object which is visible for the camera, by calculated projection of object-mapping pixels into such a space, taking into account acquired data of a distance from the object;
(c) the computer fitting a model of the object, which consists of nodes and edges, into the computer-generated 3D aggregate of points for the frame, the nodes representing a selection of elements of the object and the edges representing interconnections of these elements;
(d) the computer iteratively updating all node positions by using a learning rule for training a self-organizing map with a predetermined number of randomly sampled points of the aggregate of points;
(e) repeating at least 25 times per second the steps (a) through (d) for each subsequent frame of the sequence, the result of step (d) of the preceding frame being used respectively for the fitting process in step (c);
(f) the computer determining the changing pose from positions of predetermined nodes of the model, which have been detected in at least representative frames of the image sequence;
(g) wherein the node positions of the model are updated in iteration steps, one point x being randomly sampled for the predetermined number of points of the 3D aggregate of points in each iteration step and all nodes being shifted towards x; and
(h) the degree of the shift being largest for the node which had the smallest distance from x prior to the iteration step wherein the number of the randomly sampled points x or that of the iteration steps is about 10% of the total number of the points within the aggregate of points.
3 Assignments
0 Petitions
Accused Products
Abstract
The invention relates to a method for the real-time-capable, computer-assisted analysis of an image sequence of an object consisting of elements that can be moved relative to each other and are interconnected, said sequence containing a variable pose, wherein the individual images of the image sequence are recorded by way of a time-of-flight (TOF) camera such that said images can be processed by a computer, and contain brightness and distance data as functions of the pixel coordinates of the camera for each image of the sequence, comprising the following steps: a. Capturing the pixels of an individual image forming the object, b. calculating a three-dimensional (3D) point cloud in a virtual space, said point cloud representing the surface of the object that is visible to the camera, by a computational projection of object-depicting pixels in such a space, while taking captured distance data to the object into consideration, c. fitting a model of the object consisting of nodes and edges into the computer-generated 3D point cloud for the individual images, wherein the nodes represent a selection of elements of the object and the edges represent the connections of said elements amount each other, d. iteratively updating all node positions by applying a learning rule for training a self-organizing map having a previously defined number of randomly selected dots of the point cloud, e. repeating steps a. to d. for each subsequent individual image of the sequence, wherein for the fitting in step c. the result of step e. of the preceding image is used in each case, and f. determining the varying pose from the positions of predetermined nodes of the model which have been captured in at least representative images of the image sequence.
-
Citations
14 Claims
-
1. A method for real-time-capable computer-assisted analysis of an image sequence containing a variable pose of an object composed of interconnected elements movable in relation to each other, wherein the frames of the image sequence have been recorded by a distance measuring camera and are processed by a computer and the frames have brightness and distance data as functions of pixel coordinates of the camera for each frame of the sequence;
- the method comprising;
(a) the computer detecting the pixels of a frame which map the object; (b) the computer calculating a three-dimensional (3D) aggregate of points within a virtual space, which represents that surface of the object which is visible for the camera, by calculated projection of object-mapping pixels into such a space, taking into account acquired data of a distance from the object; (c) the computer fitting a model of the object, which consists of nodes and edges, into the computer-generated 3D aggregate of points for the frame, the nodes representing a selection of elements of the object and the edges representing interconnections of these elements; (d) the computer iteratively updating all node positions by using a learning rule for training a self-organizing map with a predetermined number of randomly sampled points of the aggregate of points; (e) repeating at least 25 times per second the steps (a) through (d) for each subsequent frame of the sequence, the result of step (d) of the preceding frame being used respectively for the fitting process in step (c); (f) the computer determining the changing pose from positions of predetermined nodes of the model, which have been detected in at least representative frames of the image sequence; (g) wherein the node positions of the model are updated in iteration steps, one point x being randomly sampled for the predetermined number of points of the 3D aggregate of points in each iteration step and all nodes being shifted towards x; and (h) the degree of the shift being largest for the node which had the smallest distance from x prior to the iteration step wherein the number of the randomly sampled points x or that of the iteration steps is about 10% of the total number of the points within the aggregate of points. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
- the method comprising;
-
13. A computer system for real-time-capable computer-assisted analysis of an image sequence containing a variable pose of an object composed of interconnected elements movable in relation to each other, wherein the frames of the image sequence have been recorded by a distance measuring camera so that they are processed by the computer system and the frames have brightness and
distance data as functions of pixel coordinates of the camera for each frame of the sequence, the computer system comprising: -
a detecting unit adapted to perform the step of (a) detecting the pixels of a frame which map the object; a calculating unit adapted to perform the step of (b) calculating a three-dimensional (3D) aggregate of points within a virtual space, which represents that surface of the object which is visible for the camera, by calculated projection of object-mapping pixels into such a space, taking into account acquired data of a distance from the object; a fitting unit adapted to perform the step of (c) fitting a model of the object, which consists of nodes and edges, into the computer-generated 3D aggregate of points for the frame, the nodes representing a selection of elements of the object and the edges representing interconnections of these elements; an updating unit adapted to perform the step of (d) iteratively updating all node positions by using a learning rule for training a self-organizing map with a predetermined number of randomly sampled points of the aggregate of points; a determination unit adapted to perform the step of (f) determining the changing pose from positions of predetermined nodes of the model, which have been detected in at least representative frames of the image sequence, wherein the computer system is adapted to cause its units to (e) repeat at least 25 times per second the steps (a) through (d) for each subsequent frame of the sequence, and to cause the fitting unit to use the result of the repetition of steps (a) through (d) of the preceding frame being used respectively for performing process in step (c); wherein the node positions of the model are (q) updated in iteration steps, one point x being randomly sampled for the predetermined number of points of the 3D aggregate of points in each iteration step and all nodes being shifted towards x; and the degree of the shift being largest for the node which (h) had the smallest distance from x prior to the iteration step wherein the number of the randomly sampled points x or that of the iteration steps is about 10% of the total number of the points within the aggregate of points.
-
-
14. A non-transitory computer-readable medium storing instructions that, when executed by a processor of a computer system for real-time-capable computer-assisted analysis of an image sequence, cause the computer system to:
-
(a) detect the pixels of a frame which map an object, said object being comprised in said image sequence which contains a variable pose of the object composed of interconnected elements movable in relation to each other wherein the frames of the image sequence have been recorded by a distance measuring camera so that they are processed by a computer and the frames have brightness and distance data as functions of pixel coordinates of the camera for each frame of the sequence (b) calculate a three-dimensional (3D) aggregate of points within a virtual space, which represents that surface of the object which is visible for the camera, by calculated projection of object-mapping pixels into such a space, taking into account acquired data of a distance from the object; (c) fit a model of the object, which consists of nodes and edges, into the computer-generated 3D aggregate of points for the frame, the nodes representing a selection of elements of the object and the edges representing interconnections of these elements; (d) iteratively update all node positions by using a learning rule for training a self-organizing map with a predetermined number of randomly sampled points of the aggregate of points; (e) repeat at least 25 times per second the steps (a) through (d) for each subsequent frame of the sequence, the result of step (e) of the preceding frame being used respectively for the fitting process in step (c); (f) determine the changing pose from positions of predetermined nodes of the model, which have been detected in at least representative frames of the image sequence (g) wherein the node positions of the model are updated in iteration steps, one point x being randomly sampled for the predetermined number of points of the 3D aggregate of points in each iteration step and all nodes being shifted towards x; and (h) the degree of the shift being largest for the node which had the smallest distance from x prior to the iteration step wherein the number of the randomly sampled points x or that of the iteration steps is about 10% of the total number of the points within the aggregate of points.
-
Specification