Systems and methods for tracking location of movable target object

• US 9,197,810 B2
• Filed: 03/30/2015
• Issued: 11/24/2015
• Est. Priority Date: 06/19/2013
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Troy, James J., Lea, Scott W., Wright, Daniel James, Georgeson, Gary E., Nelson, Karl Edward
• Primary Examiner(s): Chen, Shelley
• Application Number:
  US14/672,952
• Publication Number:
  US 20150207987A1
• Time in Patent Office: 239 Days
• Field of Search: 382/103
• US Class Current: 1/1
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 17/46 Indirect determination of position data
• CPC Class Codes: G01S 17/88 Lidar systems specially adapted for specific applications
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49
• CPC Class Codes: G06F 17/16 Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source
• CPC Class Codes: Y10S 901/46 Sensing device

Abstract

An automated process uses a local positioning system to acquire location (i.e., position and orientation) data for one or more movable target objects. In cases where the target objects have the capability to move under computer control, this automated process can use the measured location data to control the position and orientation of such target objects. The system leverages the measurement and image capture capability of the local positioning system, and integrates controllable marker lights, image processing, and coordinate transformation computation to provide tracking information for vehicle location control. The resulting system enables position and orientation tracking of objects in a reference coordinate system.

Claims

1. A method for controlling motion of a target object equipped with at least three active target markers arranged in a known pattern, comprising:

(a) defining a location of a camera with respect to a reference coordinate system;

(b) while the target object is stationary, aiming the camera so that the at least three active target markers on the target object are within a field-of-view of the camera;

(c) while the camera and target object are in states recited in step (b) and the active target markers are not on, capturing an image of the target object;

(d) while the camera and target object are in the states recited in step (b), activating the at least three active target markers at the same time or at different times and then capturing an image or images of the target object;

(e) processing the images captured in steps (c) and (d) to compute respective average pixel coordinates corresponding to respective active target markers;

(f) for respective active target markers and while the camera and target object are in the states recited in step (b), aiming a laser range finder and transmitting a laser beam in a direction defined by respective pan and tilt angles which are a function of at least pan and tilt angles at which the field-of-view of the camera was aimed at the target object and differences between the respective average pixel coordinates and pixel coordinates of a center of the images;

(g) for respective active target markers and while the camera and target object are in the states recited in step (b), acquiring respective range, pan and tilt data subsequent to the transmission of a respective laser beam;

(h) computing coordinates of respective points corresponding to the at least three active target markers in the reference coordinate system based on the measured range, pan and tilt data;

(i) comparing respective positions of the respective points whose coordinates were computed from measured data to respective positions of respective points arranged in the known pattern to determine a current position and orientation of the target object defined in terms of the reference coordinate system;

(j) comparing the current position and orientation of the target object to a desired position and orientation of the target object;

(k) activating the target object to move incrementally from the current position and orientation toward the desired position and orientation;

(l) tracking the incremental movements of the target object; and

(m) stopping the target object when results of step (l) indicate that the target object has attained the desired position and orientation.

2. The method as recited in claim 1, wherein step (e) comprises:

processing the images to compute a difference image representing differences between the image captured while all of the at least three active target markers were not on and images captured while one or more of the at least three active target markers were on; and

computing respective average pixel coordinates for the differences in the difference image corresponding to respective active target markers.

3. The method as recited in claim 2, wherein processing the images to compute a difference image comprises:

segmenting the difference image to include separate areas corresponding to the active target markers based on said differences; and

computing respective average pixel coordinates in the difference image for a respective centroid of each separate area corresponding to the active target markers.

4. The method as recited in claim 1, wherein the known pattern is asymmetric.

5. The method as recited in claim 1, generating encoder pulses in response to incremental motion of the target object during movement from the current position and orientation to the desired position and orientation.

6. The method as recited in claim 1, wherein incremental movements of the target object are tracked using differential odometry.

7. A system for controlling motion of a target object equipped with at least three active target markers arranged in a known pattern comprising:

a pan-tilt mechanism;

a camera and a laser range finder mounted to said pan-tilt mechanism; and

a computer system programmed to execute the following operations;

adjusting pan and tilt angles of said pan-tilt mechanism to center said camera on said target object with said active target markers within an image field-of-view of said camera;

commanding said camera to capture an image while said camera is centered on said target object and said active target markers are not on;

commanding said camera to capture one or more images while said camera is centered on said target object and one or more of said active target markers are on;

processing the images to compute a difference image representing differences between an image captured while all of said active target markers were not on and an image or images captured while said active target markers were on at a same time or different times;

computing respective average pixel coordinates for the differences in the difference image corresponding to respective active target markers;

for respective active target markers, commanding said laser range finder to transmit respective laser beams directed at respective pan and tilt angles which are a function of at least pan and tilt angles at which said target object was centered in the image field-of-view and differences between the respective average pixel coordinates and pixel coordinates of a center of the indicator image;

for respective active target markers, commanding said pan-tilt mechanism to acquire respective pan and tilt data and commanding said laser range finder to acquire respective range data subsequent to transmission of a respective laser beam;

computing coordinates of respective points corresponding to said active target markers in a reference coordinate system based on measured range, pan and tilt data;

comparing respective positions of the respective points whose coordinates were computed from measured data to respective positions of respective points arranged in the known pattern to determine a current position and orientation of the target object defined in terms of the reference coordinate system;

comparing the current position and orientation of the target object to a desired position and orientation of the target object;

activating the target object to move incrementally from the current position and orientation toward the desired position and orientation;

tracking changes in position and orientation of the target object during the incremental movement; and

stopping the target object when tracking results indicate that the target object has attained the desired position and orientation.

8. The system as recited in claim 7, wherein said active target markers are light-emitting diodes.

9. The system as recited in claim 7, wherein said target object comprises a crawler vehicle.

10. The system as recited in claim 9, wherein said crawler vehicle is a holonomic motion crawler vehicle.

11. The system as recited in claim 7, wherein said target object comprises a moveable base of a robotic arm.

12. The system as recited in claim 7, wherein changes in the position and orientation of the target object are tracked using differential odometry.