3D Positioning Apparatus and Method

• US 20100309289A1
• Filed: 12/29/2009
• Published: 12/09/2010
• Est. Priority Date: 06/09/2009
• Status: Active Grant

• Current Assignee: National Taiwan University (Government of The Republic of China)
• Original Assignee: National Taiwan University (Government of The Republic of China)
• Inventors: Chi, Ping-Cheng, Chen, Homer H.
• Granted Patent: US 8,866,888 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A 3D positioning apparatus is used for an object that includes feature points and a reference point. The object undergoes movement from a first to a second position. The 3D positioning apparatus includes: an image sensor for capturing images of the object; and a processor for calculating, based on the captured images, initial coordinates of each feature point when the object is in the first position, initial coordinates of the reference point, final coordinates of the reference point when the object is in the second position, and final coordinates of each feature point. The processor calculates 3D translational information of the feature points using the initial and final coordinates of the reference point, and 3D rotational information of the feature points using the initial and final coordinates of each feature point. A 3D positioning method is also disclosed.

Claims

1. A three-dimensional (3D) positioning apparatus for an object that includes a plurality of feature points, a reference point being formed by the feature points, the object undergoing movement from a first position to a second position, said 3D positioning apparatus comprising:

an image sensor for capturing images of the object; and

a processor coupled to said image sensor for receiving and processing the images captured thereby, said processor being configured to calculate on the basis of the images initial coordinates of each of the feature points when the object is in the first position, initial coordinates of the reference point when the object is in the first position, final coordinates of the reference point when the object is in the second position, and final coordinates of each of the feature points when the object is in the second position;

wherein said processor is further configured to calculate 3D translational information of the feature points on the basis of the initial and final coordinates of the reference point; and

wherein said processor is further configured to calculate 3D rotational information of the feature points on the basis of the initial and final coordinates of each of the feature points.

2. The 3D positioning apparatus of claim 1, wherein said processor calculates the initial coordinates of each of the feature points by:

calculating from at least one image of the object when it is in the first position a depth coordinate of said each of the feature points when the object is in the first position; and

calculating a horizontal coordinate and a vertical coordinate of said each of the feature points when the object is in the first position on the basis of a relation among the horizontal coordinate, the vertical coordinate, and the depth coordinate of said each of the feature points.

3. The 3D positioning apparatus of claim 2, wherein said processor calculates the depth coordinate of each of the feature points when the object in the first position by:

calculating a first distance between one of the feature points and a projection center of said 3D positioning apparatus;

calculating a second distance between another one of the feature points and the projection center of said 3D positioning apparatus; and

if a difference between the first and second distances is less than a threshold distance, calculating an actual distance between the feature points and the projection center based on a distance between the feature points on an imaging plane of said 3D positioning apparatus, and setting the depth coordinate of said each of the feature points when the object is in the first position as the actual distance between the feature points and the projection center.

4. The 3D positioning apparatus of claim 2, wherein said processor calculates the depth coordinate of each of the feature points when the object is in the first position by:

calculating a first distance between one of the feature points and a projection center of said 3D positioning apparatus;

calculating a second distance between another one of the feature points and the projection center of said 3D positioning apparatus; and

if a difference between the first and second distances is greater than or equal to a threshold distance, calculating a third distance from the projection center to a point between said one of the feature points and said another one of the feature points according to the first distance and the second distance, and calculating the depth coordinate of said each of the feature points when the object is in the first position according to the third distance.

5. The 3D positioning apparatus of claim 1, wherein said processor calculates the final coordinates of the reference point by:

calculating a horizontal motion amount and a vertical motion amount of the reference point when the object is in the second position based on an image of the object in the first position and an image of the object in the second position;

calculating a depth coordinate of the reference point when the object is in the second position based on the image of the object in the second position; and

calculating the coordinates of the reference point when the object is in the second position using the horizontal motion amount, the vertical motion amount, a focal distance of said image sensor, and the depth coordinate of the reference point when the object is in the second position.

6. The 3D positioning apparatus of claim 5, wherein said processor calculates the depth coordinate of the reference point when the object is in the second position by:

calculating a distance between a projection center of said 3D positioning apparatus and the reference point; and

calculating the depth coordinate of the reference point when the object is in the second position according to the distance, and a relation among the horizontal motion amount, the vertical motion amount, and the depth coordinate of the reference point when the object is in the second position.

7. The 3D positioning apparatus of claim 1, wherein the final coordinates of each of the feature points are proportional coordinates, in which a coordinate of one of the feature points is used to determine coordinates for the remainder of the feature points.

8. The 3D positioning apparatus of claim 7, wherein said processor calculates the final coordinates of each of the feature points by:

taking a depth coordinate of said one of the feature points as a standard unit length, and determining depth coordinates of said remainder of the feature points to be multiples of the standard unit length; and

recursively updating the depth coordinates of said remainder of the feature points until values of the depth coordinates of said remainder of the feature points converge.

9. The 3D positioning apparatus of claim 7, wherein said processor calculates the 3D rotational information of the feature points using a Horn algorithm, and the initial and proportional coordinates of each of the feature points.

10. A three-dimensional (3D) positioning method for an object that includes a plurality of feature points, a reference point being formed by the feature points, the object undergoing movement from a first position to a second position, said 3D positioning method being implemented by an image sensor and a processor of a 3D positioning apparatus, said 3D positioning method comprising:

(a) configuring the image sensor to capture images of the object;

(b) configuring the processor, which is coupled to the image sensor for receiving the images captured thereby, to calculate on the basis of the images initial coordinates of each of the feature points when the object is in the first position, initial coordinates of the reference point when the object is in the first position, final coordinates of the reference point when the object is in the second position, and final coordinates of each of the feature points when the object is in the second position;

(c) configuring the processor to calculate 3D translational information of the feature points on the basis of the initial and final coordinates of the reference point; and

(d) configuring the processor to calculate 3D rotational information of the feature points on the basis of the initial and final coordinates of each of the feature points.

11. The 3D positioning method of claim 10, wherein, in step (b), the processor is configured to calculate the initial coordinates of each of the feature points by:

(b1) calculating from at least one image of when the object is in the first position a depth coordinate of said each of the feature points when the object is in the first position; and

(b2) calculating a horizontal coordinate and a vertical coordinate of said each of the feature points when the object is in the first position on the basis of a relation among the horizontal coordinate, the vertical coordinate, and the depth coordinate of said each of the feature points.

12. The 3D positioning method of claim 11, wherein, in step (b1), the processor is configured to calculate the depth coordinate of each of the feature points when the object in the first position by:

calculating a first distance between one of the feature points and a projection center of the 3D positioning apparatus;

calculating a second distance between another one of the feature points and the projection center of the 3D positioning apparatus; and

if a difference between the first and second distances is less than a threshold distance, calculating an actual distance between the feature points and the projection center based on a distance between the feature points on an imaging plane of said 3D positioning apparatus, and setting the depth coordinate of said each of the feature points when the object is in the first position as the actual distance between the feature points and the projection center.

13. The 3D positioning method of claim 11, wherein, in step (b1), the processor is configured to calculate the depth coordinate of each of the feature points when the object is in the first position by:

calculating a first distance between one of the feature points and a projection center of the 3D positioning apparatus;

calculating a second distance between another one of the feature points and the projection center of the 3D positioning apparatus; and

if a difference between the first and second distances is greater than or equal to a threshold distance, calculating a third distance from the projection center to a point between said one of the feature points and said another one of the feature points according to the first distance and the second distance, and calculating the depth coordinate of said each of the feature points when the object is in the first position according to the third distance.

14. The 3D positioning method of claim 10, wherein, in step (b), the processor is configured to calculate the final coordinates of the reference point by:

(b1) calculating a horizontal motion amount and a vertical motion amount of the reference point when the object is in the second position based on an image of the object in the first position and an image of the object in the second position;

(b2) calculating a depth coordinate of the reference point when the object is in the second position based on the image of the object in the second position; and

(b3) calculating the coordinates of the reference point when the object is in the second position using the horizontal motion amount, the vertical motion amount, a focal distance of said image sensor, and the depth coordinate of the reference point when the object is in the second position.

15. The 3D positioning method of claim 14, wherein, in step (b2), the processor is configured to calculate the depth coordinate of the reference point when the object is in the second position by:

calculating a distance between a projection center of the 3D positioning apparatus and the reference point; and

calculating the depth coordinate of the reference point when the object is in the second position according to the distance, and a relation among the horizontal motion amount, the vertical motion amount, and the depth coordinate of the reference point when the object is in the second position.

16. The 3D positioning method of claim 10, wherein, in step (b), the final coordinates of each of the feature points are proportional coordinates, in which a coordinate of one of the feature points is used to determine coordinates for the remainder of the feature points.

17. The 3D positioning method of claim 16, wherein, in step (b), the processor is configured to calculate the final coordinates of each of the feature points by:

taking a depth coordinate of said one of the feature points as a standard unit length, and determining depth coordinates of said remainder of the feature points to be multiples of the standard unit length; and

recursively updating the depth coordinates of said remainder of the feature points until values of the depth coordinates of said remainder of the feature points converge.

18. The 3D positioning method of claim 17, wherein the depth coordinates of said remainder of the feature points are updated according to a Jacobian matrix and a residual difference matrix.

19. The 3D positioning method of claim 16, wherein, in step (d), the processor is configured to calculate the 3D rotational information of the feature points using a Horn algorithm, and the initial and proportional coordinates of each of the feature points.

20. The 3D positioning method of claim 10, wherein, in step (d), the processor is configured to calculate the 3D rotational information of the feature points on the basis of the initial and final coordinates of each of the feature points by calculating a rotation matrix corresponding to the feature points, and calculating a 3D rotation angle of each of the feature points according to the rotation matrix.

21. The 3D positioning method of claim 10, wherein, in step (b), the processor is configured to calculate the initial coordinates of each of the feature points by:

for each of a predetermined number of images of when the object is in the first position, calculating coordinates for said each of the feature points; and

averaging the coordinates for said each of the feature points to obtain the initial coordinates of said each of the feature points.