3D positioning apparatus and method

• US 8,866,888 B2
• Filed: 12/29/2009
• Issued: 10/21/2014
• Est. Priority Date: 06/09/2009
• Status: Active Grant

• Current Assignee: National Taiwan University (Government of The Republic of China)
• Original Assignee: National Taiwan University (Government of The Republic of China)
• Inventors: Chen, Homer H., Chi, Ping-Cheng
• Primary Examiner(s): Senfi, Behrooz
• Assistant Examiner(s): VAZQUEZ COLON, MARIA E
• Application Number:
  US12/649,029
• Publication Number:
  US 20100309289A1
• Time in Patent Office: 1,757 Days
• Field of Search: 348/46, 348/42, 348/135, 348/137, 348/141, 382/106, 382/154, 382/190, 382/201
• US Class Current: 348/46
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A 3D positioning apparatus is used for an object that includes feature points and a reference point. The object undergoes movement from a first to a second position. The 3D positioning apparatus includes: an image sensor for capturing images of the object; and a processor for calculating, based on the captured images, initial coordinates of each feature point when the object is in the first position, initial coordinates of the reference point, final coordinates of the reference point when the object is in the second position, and final coordinates of each feature point. The processor calculates 3D translational information of the feature points using the initial and final coordinates of the reference point, and 3D rotational information of the feature points using the initial and final coordinates of each feature point. A 3D positioning method is also disclosed.

Claims

1. An apparatus for positioning an object, the object undergoing movement from a first position to a second position in a three-dimensional (3D) space, said apparatus comprising:

an image sensor for capturing images of the object, wherein said object has more than two feature points and a reference point derived from said feature points, said feature points having predetermined geometric relationships thereamong in the 3D space, said reference point and said feature points being on a common plane in the 3D space; and

a processor coupled to said image sensor for receiving and processing the images captured thereby, said processor being configured to calculate, on the basis of the images and the predetermined geometric relationships among said feature points, initial coordinates of each of the feature points when the object is in the first position, initial coordinates of the reference point when the object is in the first position, final coordinates of the reference point when the object is in the second position, and final coordinates of each of the feature points when the object is in the second position;

wherein said processor is further configured to calculate 3D translational information of the feature points on the basis of the initial and final coordinates of the reference point; and

wherein said processor is further configured to calculate 3D rotational information of the feature points on the basis of the initial and final coordinates of each of the feature points;

wherein said processor calculates the initial coordinates of each of the feature points by;

calculating from at least one image of the object when it is in the first position a depth coordinate of said each of the feature points when the object is in the first position; and

calculating a horizontal coordinate and a vertical coordinate of said each of the feature points when the object is in the first position on the basis of a relation among the horizontal coordinate, the vertical coordinate, and the depth coordinate of said each of the feature points; and

wherein said processor calculates the depth coordinate of each of the feature points when the object is in the first position by;

calculating a first distance between one of the feature points and a projection center of said apparatus;

calculating a second distance between another one of the feature points and the projection center of said apparatus; and

if a difference between the first and second distances is greater than or equal to a threshold distance, calculating a third distance from the projection center to a point between said one of the feature points and said another one of the feature points according to the first distance and the second distance, and calculating the depth coordinate of said each of the feature points when the object is in the first position according to the third distance.

2. The apparatus of claim 1, wherein, if the difference between the first and second distances is less than a threshold distance, calculating an actual distance between the feature points and the projection center based on a distance between the feature points on an imaging plane of said apparatus, and setting the depth coordinate of said each of the feature points when the object is in the first position as the actual distance between the feature points and the projection center.

3. The apparatus of claim 1, wherein said processor calculates the final coordinates of the reference point by:

calculating a horizontal motion amount and a vertical motion amount of the reference point when the object is in the second position based on an image of the object in the first position and an image of the object in the second position;

calculating a depth coordinate of the reference point when the object is in the second position based on the image of the object in the second position; and

calculating the coordinates of the reference point when the object is in the second position using the horizontal motion amount, the vertical motion amount, a focal distance of said image sensor, and the depth coordinate of the reference point when the object is in the second position.

4. The apparatus of claim 3, wherein said processor calculates the depth coordinate of the reference point when the object is in the second position by:

calculating a distance between a projection center of said apparatus and the reference point; and

calculating the depth coordinate of the reference point when the object is in the second position according to the distance, and a relation among the horizontal motion amount, the vertical motion amount, and the depth coordinate of the reference point when the object is in the second position.

5. The apparatus of claim 1, wherein the final coordinates of each of the feature points are proportional coordinates, in which a coordinate of one of the feature points is used to determine coordinates for the remainder of the feature points.

6. The apparatus of claim 5, wherein said processor calculates the final coordinates of each of the feature points by:

taking a depth coordinate of said one of the feature points as a standard unit length, and determining depth coordinates of said remainder of the feature points to be multiples of the standard unit length; and

recursively updating the depth coordinates of said remainder of the feature points until values of the depth coordinates of said remainder of the feature points converge.

7. The apparatus of claim 5, wherein said processor calculates the 3D rotational information of the feature points using a Horn algorithm, and the initial and proportional coordinates of each of the feature points.

8. A method for causing an image sensor and a processor of a 3D positioning apparatus to determine a position of an object that undergoes movement from a first position to a second position in a three-dimensional (3D) space, said method comprising:

(a) using the image sensor, capturing images of the object, the object having more than two identifiable feature points and a reference point derived from the feature points, the feature points having predetermined geometric relationships thereamong in the 3D space, the reference point and the feature points being on a common plane in the 3D space;

(b) using the processor, which is coupled to the image sensor for receiving the images captured thereby, calculating, on the basis of the images and the predetermined geometric relationships among the feature points, initial coordinates of each of the feature points when the object is in the first position, initial coordinates of the reference point when the object is in the first position, final coordinates of the reference point when the object is in the second position, and final coordinates of each of the feature points when the object is in the second position;

(c) using the processor, calculating 3D translational information of the feature points on the basis of the initial and final coordinates of the reference point; and

(d) using the processor, calculating 3D rotational information of the feature points on the basis of the initial and final coordinates of each of the feature points;

wherein, in step (b), the processor calculates the initial coordinates of each of the feature points by;

(b1) calculating from at least one image of when the object is in the first position a depth coordinate of said each of the feature points when the object is in the first position; and

(b2) calculating a horizontal coordinate and a vertical coordinate of said each of the feature points when the object is in the first position on the basis of a relation among the horizontal coordinate, the vertical coordinate, and the depth coordinate of said each of the feature points; and

wherein, in step (b1), the processor calculates the depth coordinate of each of the feature points when the object is in the first position by;

calculating a first distance between one of the feature points and a projection center of the 3D positioning apparatus;

calculating a second distance between another one of the feature points and the projection center of the 3D positioning apparatus; and

if a difference between the first and second distances is greater than or equal to a threshold distance, calculating a third distance from the projection center to a point between said one of the feature points and said another one of the feature points according to the first distance and the second distance, and calculating the depth coordinate of said each of the feature points when the object is in the first position according to the third distance.

9. The method of claim 8, wherein, in step (b1), if the difference between the first and second distances is less than a threshold distance, calculating an actual distance between the feature points and the projection center based on a distance between the feature points on an imaging plane of the 3D positioning apparatus, and setting the depth coordinate of said each of the feature points when the object is in the first position as the actual distance between the feature points and the projection center.

10. The-method of claim 8, wherein, in step (b), the processor calculates the final coordinates of the reference point by:

(b3) calculating a horizontal motion amount and a vertical motion amount of the reference point when the object is in the second position based on an image of the object in the first position and an image of the object in the second position;

(b4) calculating a depth coordinate of the reference point when the object is in the second position based on the image of the object in the second position; and

(b5) calculating the coordinates of the reference point when the object is in the second position using the horizontal motion amount, the vertical motion amount, a focal distance of said image sensor, and the depth coordinate of the reference point when the object is in the second position.

11. The method of claim 10, wherein, in step (b4), the processor calculates the depth coordinate of the reference point when the object is in the second position by:

calculating a distance between a projection center of the 3D positioning apparatus and the reference point; and

calculating the depth coordinate of the reference point when the object is in the second position according to the distance, and a relation among the horizontal motion amount, the vertical motion amount, and the depth coordinate of the reference point when the object is in the second position.

12. The method of claim 8, wherein, in step (b), the final coordinates of each of the feature points are proportional coordinates, in which a coordinate of one of the feature points is used to determine coordinates for the remainder of the feature points.

13. The method of claim 12, wherein, in step (b), the processor calculates the final coordinates of each of the feature points by:

taking a depth coordinate of said one of the feature points as a standard unit length, and determining depth coordinates of said remainder of the feature points to be multiples of the standard unit length; and

recursively updating the depth coordinates of said remainder of the feature points until values of the depth coordinates of said remainder of the feature points converge.

14. The method of claim 13, wherein the depth coordinates of said remainder of the feature points are updated according to a Jacobian matrix and a residual difference matrix.

15. The method of claim 12, wherein, in step (d), the processor calculates the 3D rotational information of the feature points using a Horn algorithm, and the initial and proportional coordinates of each of the feature points.

16. The method of claim 8, wherein, in step (d), the processor calculates the 3D rotational information of the feature points on the basis of the initial and final coordinates of each of the feature points by calculating a rotation matrix corresponding to the feature points, and calculating a 3D rotation angle of each of the feature points according to the rotation matrix.

17. The method of claim 8, wherein, in step (b), the processor calculates the initial coordinates of each of the feature points by:

for each of a predetermined number of images of when the object is in the first position, calculating coordinates for said each of the feature points; and

averaging the coordinates for said each of the feature points to obtain the initial coordinates of said each of the feature points.