3D image analyzer for determining the gaze direction
First Claim
1. A 3D image analyzer for determination of a gaze direction, wherein the 3D image analyzer is configured to receive at least one first set of image data, which is determined on the basis of a first image, and a further set of information, which is determined on the basis of the first image or of a further image, wherein the first image comprises a pattern resulting from the display of a three-dimensional object from a first perspective into a first image plane, and wherein the further set comprises an image with a pattern resulting from the display of the same three-dimensional object from a further perspective into a further image plane, or wherein the further set comprises information which describes a relation between at least one point of the three-dimensional object and the first image plane, wherein the 3D image analyzer comprises the following features:
- a position calculator which is configured to calculate a position of the pattern within a three-dimensional room based on the first set, a further set, a further set, which is determined on the basis of the further image, and a geometric relation between the perspectives of the first and the further image or to calculate the position of the pattern within a three-dimensional room based on the first set and a statistically determined relation between at least two characterizing features towards each other in the first image, or to calculate the position of the pattern within the three-dimensional room based on the first set and on a position relation between at least one point of the three-dimensional object and the first image plane; and
an alignment calculator which is configured to calculate at least two possible 3D gaze vectors per image and to determine from these two possible 3D gaze vectors the 3D gaze vector according to which the pattern in the three-dimensional room is aligned, wherein the calculation and determination is based on the first set, the further set and on the calculated position of the pattern.
1 Assignment
0 Petitions
Accused Products
Abstract
A 3D image analyzer for the determination of a gaze direction or a line of sight (having a gaze direction vector and a location vector, which e.g. indicates the pupil midpoint and where the gaze direction vector starts) in a 3D room is configured to receive one first set of image data and a further set of image information, wherein the first image contains a pattern, which displays a three-dimensional object from a first perspective into a first image plane, and wherein the further set contains an image having a pattern, which displays the same three-dimensional object from a further perspective into a further image plane, or wherein the further set has an image information and/or a relation between at least two points in the first image and/or at least a position information. The 3D image analyzer has a position calculator and an alignment calculator and calculates therewith a gaze direction in a 3D room.
-
Citations
31 Claims
-
1. A 3D image analyzer for determination of a gaze direction, wherein the 3D image analyzer is configured to receive at least one first set of image data, which is determined on the basis of a first image, and a further set of information, which is determined on the basis of the first image or of a further image, wherein the first image comprises a pattern resulting from the display of a three-dimensional object from a first perspective into a first image plane, and wherein the further set comprises an image with a pattern resulting from the display of the same three-dimensional object from a further perspective into a further image plane, or wherein the further set comprises information which describes a relation between at least one point of the three-dimensional object and the first image plane, wherein the 3D image analyzer comprises the following features:
-
a position calculator which is configured to calculate a position of the pattern within a three-dimensional room based on the first set, a further set, a further set, which is determined on the basis of the further image, and a geometric relation between the perspectives of the first and the further image or to calculate the position of the pattern within a three-dimensional room based on the first set and a statistically determined relation between at least two characterizing features towards each other in the first image, or to calculate the position of the pattern within the three-dimensional room based on the first set and on a position relation between at least one point of the three-dimensional object and the first image plane; and an alignment calculator which is configured to calculate at least two possible 3D gaze vectors per image and to determine from these two possible 3D gaze vectors the 3D gaze vector according to which the pattern in the three-dimensional room is aligned, wherein the calculation and determination is based on the first set, the further set and on the calculated position of the pattern. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27)
-
-
28. A method for the determination of a gaze direction, comprising:
-
receiving of at least one first set of image data, which is determined on the basis of a first image, and a further set of image data, which is determined on the basis of a further image, wherein the first image displays a pattern of a three-dimensional object from a first perspective into a first image plane and wherein the further set comprises a further image or an information, which describes a relation between at least one point of the three-dimensional object and the first image plane; calculating a position of the pattern in a three-dimensional room based on the first set, a further set, and a geometric relation between the perspectives of the first and the further image or calculating of the position of the pattern in a three-dimensional room based on a first set and a statistically evaluated relation between at least two characteristic features in the first image or calculating the position of the pattern in a three-dimensional room based on the first set and a position relation between at least one point of the three-dimensional object and the first image plane, and calculating a 3D gaze vector according to which the pattern is aligned in the three-dimensional room based on the first set and the further set.
-
-
29. A non-transitory digital storage medium having stored thereon a computer program for performing a method for the determination of a gaze direction, comprising:
-
receiving of at least one first set of image data, which is determined on the basis of a first image, and a further set of image data, which is determined on the basis of a further image, wherein the first image displays a pattern of a three-dimensional object from a first perspective into a first image plane and wherein the further set comprises a further image or an information, which describes a relation between at least one point of the three-dimensional object and the first image plane; calculating a position of the pattern in a three-dimensional room based on the first set, a further set, and a geometric relation between the perspectives of the first and the further image or calculating of the position of the pattern in a three-dimensional room based on a first set and a statistically evaluated relation between at least two characteristic features in the first image or calculating the position of the pattern in a three-dimensional room based on the first set and a position relation between at least one point of the three-dimensional object and the first image plane, and calculating a 3D gaze vector according to which the pattern is aligned in the three-dimensional room based on the first set and the further set, when said computer program is run by a computer.
-
-
30. A 3D image analyzer for determination of a gaze direction, wherein the 3D image analyzer is configured to receive at least one first set of image data, which is determined on the basis of a first image, and a further set of information, which is determined on the basis of the first image or of a further image, wherein the first image comprises a pattern resulting from the display of a three-dimensional object from a first perspective into a first image plane, and wherein the further set comprises an image with a pattern resulting from the display of the same three-dimensional object from a further perspective into a further image plane, or wherein the further set comprises information which describes a relation between at least one point of the three-dimensional object and the first image plane, wherein the 3D image analyzer comprises the following features:
-
a position calculator which is configured to calculate a position of the pattern within a three-dimensional room based on the first set, a further set, a further set, which is determined on the basis of the further image, and a geometric relation between the perspectives of the first and the further image or to calculate the position of the pattern within a three-dimensional room based on the first set and a statistically determined relation between at least two characterizing features towards each other in the first image, or to calculate the position of the pattern within the three-dimensional room based on the first set and on a position relation between at least one point of the three-dimensional object and the first image plane; and an alignment calculator which is configured to calculate at least two possible 3D gaze vectors per image and to determine from these two possible 3D gaze vectors the 3D gaze vector according to which the pattern in the three-dimensional room is aligned, wherein the calculation and the determination is based on the first set, the further set and on the calculated position of the pattern, wherein the further set of image information comprises information how many pixel are scanned from the sclera displayed in first and/or the further image by the projections, which result from the pupil midpoint in the first and/or further image and the display of the two possible 3D gaze vectors into the image;
orthe further set comprises a further image, and wherein the alignment calculator is configured to calculate two further possible 3D gaze vectors and to compare the two further possible 3D gaze vectors to the two possible 3D gaze vectors and to determine on the basis of the comparison the 3D gaze vector according to which the pattern within the three-dimensional room is aligned;
wherein the alignment calculator is configured to select from the two possible 3D gaze vectors the 3D gaze vector, according to which the pattern is aligned in the three-dimensional room, wherein in this 3D gaze vector its rear projection into the image based on the pupil midpoint scans less sclera pixels than the rear projection of the other 3D gaze vector;
orthe alignment calculator is configured to determine a distance respectively between the recognized pupil midpoint and a recognized edge of the eye along the two possible 3D gaze vectors projected into the image and to select the 3D gaze vector, according to which the pattern is aligned in the three-dimensional room from the two possible 3D gaze vectors, wherein the 3D gaze vector is selected, the projection of which into the image there scans the smaller distance between the pupil midpoint and the edge of the eye opening;
orthe further set of image information comprises an information on the relation between a pupil position within the eye recognized in the first image to a reference pupil position and the two possible 3D gaze vectors;
orthe statistically evaluated relation comprises a distance between two characteristic facial features, a proportion between the two characteristic facial features and/or a proportion between one characteristic facial feature and one image edge;
orthe position calculator is configured to detect the two or more characteristic features and to compare their position relation with the previously statistically determined and stored data and to determine therefrom the distance and/or the alignment of the pattern towards the camera.
-
-
31. A method for the determination of a gaze direction, comprising:
-
receiving of at least one first set of image data, which is determined on the basis of a first image, and a further set of image data, which is determined on the basis of the first image or of a further image, wherein the first image displays a pattern of a three-dimensional object from a first perspective into a first image plane and wherein the further set comprises a further image or an information, which describes a relation between at least one point of the three-dimensional object and the first image plane; calculating a position of the pattern in a three-dimensional room based on the first set, a further set, and a geometric relation between the perspectives of the first and the further image or calculating of the position of the pattern in the three-dimensional room based on a first set and a statistically evaluated relation between at least two characteristic features in the first image or calculating the position of the pattern in the three-dimensional room based on the first set and a position relation between at least one point of the three-dimensional object and the first image plane, and calculating a 3D gaze vector according to which the pattern is aligned in the three-dimensional room based on the first set and the further set; wherein the further set of image information comprises information how many pixel are scanned from the sclera displayed in first and/or the further image by the projections, which result from the pupil midpoint in the first and/or further image and the display of the two possible 3D gaze vectors into the image;
orthe further set comprises a further image so as to calculate two further possible 3D gaze vectors and to compare the two further possible 3D gaze vectors to the two possible 3D gaze vectors and to determine on the basis of the comparison the 3D gaze vector according to which the pattern within the three-dimensional room is aligned; and
to select from the two possible 3D gaze vectors the 3D gaze vector, according to which the pattern is aligned in the three-dimensional room, wherein in this 3D gaze vector its rear projection into the image based on the pupil midpoint scans less sclera pixels than the rear projection of the other 3D gaze vector;
ora distance respectively is determined between the recognized pupil midpoint and a recognized edge of the eye along the two possible 3D gaze vectors projected into the image and the 3D gaze vector, according to which the pattern is aligned in the three-dimensional room is selected from the two possible 3D gaze vectors, wherein the 3D gaze vector is selected, the projection of which into the image there scans the smaller distance between the pupil midpoint and the edge of the eye opening;
orthe further set of image information comprises an information on a relation between a pupil position within the eye recognized in the first image to a reference pupil position and the two possible 3D gaze vectors;
orthe statistically evaluated relation comprises a distance between two characteristic facial features, a proportion between the two characteristic facial features and/or a proportion between one characteristic facial feature and one image edge;
orthe two or more characteristic features are detected and their position relations are compared with the previously statistically determined and stored data and therefrom the distance and/or the alignment of the pattern towards the camera is determined.
-
Specification