Systems, devices, and methods for tracking moving targets

• US 9,305,365 B2
• Filed: 03/14/2013
• Issued: 04/05/2016
• Est. Priority Date: 01/24/2013
• Status: Active Grant

• Current Assignee: KinetiCor, Inc.
• Original Assignee: KinetiCor, Inc.
• Inventors: Lovberg, John Arthur, Pan, Xiaojiang Jason
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): Mistry, Oneal R
• Application Number:
  US13/831,115
• Publication Number:
  US 20140205140A1
• Time in Patent Office: 1,118 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10088 Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A system for tracking a moving target having up to six degrees of freedom and rapidly determining positions of the target, said system includes an easy to locate precision optical target fixed to the target. This system includes at least two cameras positioned so as to view the optical camera from different directions with each of the at least two cameras being adapted to record two dimensional images of the precision optical target defining precise target point. A computer processor is programmed to determine the target position of x, y and z and pitch, roll and yaw. In an embodiment, the system can be configured to utilize an iteration procedure whereby an approximate first-order solution is proposed and tested against the identified precise target points to determine residual errors which can be divided by the local derivatives with respect to each component of rotation and translation, to determine an iterative correction.

Claims

1. A system for tracking a moving target having up to six degrees of freedom and rapidly determining positions of the moving target, said system comprising:

A) A precision optical target fixed to the moving target,

B) at least two cameras positioned so as to view the optical target from different directions with each of the at least two cameras being adapted to record two dimensional images of the precision optical target defining a precise target point,

C) a computer processor programmed to determine a target position in Cartesian coordinates of x, y, and z and pitch, roll and yaw utilizing an algorithm adapted to;

1) identify the precise target point on the precision optical target and the x, y and z displacement of the precise target point based on optical images collected by the at least two cameras,

2) utilize an iteration procedure whereby an approximate first-order solution is proposed and tested against the identified precise target points to determine residual errors which are then divided by the local derivatives with respect to each component of rotation and translation, to determine an iterative correction,

3) repeat step 2 until residual error becomes smaller than desired accuracy, and

4) utilize the results of steps

     1),

     2) and

     3) to determine the position of the target at rates of at least 100 times per second.

2. The system as in claim 1 wherein the moving target is a human head.

3. The system as in claim 1 and configured to interface as a component of an MRI device.

4. The system as in claim 1 wherein the iteration procedure is a variant of the Newton-Raphson method.

5. The system as in claim 2 wherein movements are measured relative to a pivot point in the patient'"'"'s neck.

6. The system as in claim 2 wherein measurements are updated at a rate of at least 100 solutions per second with a latency of less than 10 milliseconds.

7. The system as in claim 2 wherein measurements are updated at a rate of at least 200 solutions per second with a latency of less than 10 milliseconds.

8. The system as in claim 3 wherein the system is adapted to report to MRI systems the exact position of the head with accuracies better than 0.1 mm in distances and 0.1 degree in angles.

9. The system as in claim 1 wherein the precision optical target comprises at least three concentric sub-targets.

10. The system as in claim 2 wherein the precision optical target is fixed to at least one of the patient'"'"'s upper teeth.

11. The system as in claim 9 wherein the three concentric sub-targets are each concentric ellipses.

12. The system as in claim 9 wherein the computer processor is programmed to calculate centroids of each sub-target by dividing (a) a sum of a product of pixel intensity and pixel position by (b) a sum of pixel intensity in a sub-pixel array.

13. The system as in claim 11 wherein the pixel array is about 48×

48 pixels.

26. The system as in claim 4 wherein the computer processor is programmed with algorithms adapted to perform the iteration procedure in steps comprising the following steps:

I) Locate the positions of the three target centroids on each of the two Cameras and calculate six principal quantities;

Σ

_HD Δ

_HD Σ

_VD Δ

_BL Σ

_MT Δ

_MT;

II) Make an initial guess of (0, 0, 0, 0, 0,

     0) for the subject displacements and rotations (φ

, θ

, ψ

, Δ

x, Δ

y, Δ

z) leading to those centroid positions;

III) Enter the guess values for (φ

, θ

, ψ

, Δ

x, Δ

y, Δ

z) into a translation matrix and calculate the corresponding translated 3-D target position (x_i, y_i, z_i) for each of the three target centroids;

IV) Calculate the position (horizontal and vertical pixel number) of the projection of each of the three target centroids calculated in Step III on each camera system;

V) Calculate six principal quantities using Step IV data from the two cameras;

Σ

_HD Δ

_HD Σ

_VD Δ

_BL Σ

_MT Δ

_MT;

VI) Compare these six calculated principal quantities with measured values from Step I, List the differences as a matrix of errors in the guessed/calculated quantities relative to the measured quantities;

(σ

<sub>Σ

HD</sub>,σ

<sub>Δ

HD</sub>,σ

<sub>Σ

VD</sub>,σ

<sub>Δ

BL</sub>,σ

<sub>Σ

MT</sub>,σ

<sub>Δ

MT</sub>);

VII) Determine the local partial derivatives of;

Σ

_HD Δ

_HD Σ

_VD Δ

_BL Σ

_MT Δ

_MT by repeating Steps III-V six times with small added displacements in each degree of freedom, one at a time;

VIII) Determine a coarse correction matrix by dividing the error matrix by the derivative matrix, to improve the initial guess and reduce the residual error, Create a better displacement matrix;

[φ

+Δ

φ

,θ

+Δ

θ

,ψ

+Δ

ψ

,Δ

x+Δ

(Δ

x),Δ

y+Δ

(Δ

y),Δ

z+Δ

(Δ

z)];

IX) Repeat steps III through VIII for a second and final iteration, starting with the coarse-corrected displacement matrix computed in Step VIII as the input for Step III; and

X) After each successive camera frame increment, repeat Steps I-IX but use the result of previous Step IX for the guess in Step II.

14. A method for tracking a moving target having up to six degrees of freedom and rapidly determining positions of the target comprising the steps of:

A) attaching a precision optical target fixed to the moving target,

B) positioning at least two cameras positioned so as to view the optical target from different directions with each of the at least two cameras being adapted to record two dimensional images of the precision optical target defining a precise target point,

C) programming a computer processor to determine the target position in Cartesian coordinates of x, y and z and pitch, roll and yaw utilizing an algorithm adapted to

1) identify a precise target point on the precision optical target and the x, y and z displacement of the precise target point based on optical images collected by the at least two cameras,

2) utilize an iteration procedure whereby an approximate first-order solution is proposed and tested against the identified precise target points to determine residual errors which are then divided by the local derivatives with respect to each component of rotation and translation, to determine an iterative correction,

3) repeat step 2 until residual error becomes smaller than desired accuracy, and

4) utilize the results of steps

     1),

     2) and

     3) to determine the position of the target at rates of at least 100 times per second.

15. The method as in claim 14 wherein the target is a human head.

16. The method as in claim 15 wherein tracking results are utilized as an input to an MRI device so as to adjust its magnetic fields to compensate for movements of the human head.

17. The method as in claim 14 wherein the iteration procedure is a variant of the Newton-Raphson method.

18. The method as in claim 15 wherein movements are measured relative to a pivot point in the patient'"'"'s neck.

19. The method as in claim 14 wherein measurements are up-dated at a rate of at least 100 solutions per second with a latency of less than 10 milliseconds.

20. The method as in claim 14 wherein measurements are up-dated at a rate of at least 200 solutions per second with a latency of less than 10 milliseconds.

21. The method as in claim 16 wherein the system is adapted to report to MRI systems the exact position of the head with accuracies better than 0.1 mm in distances and 0.1 degree in angles.

22. The method as in claim 14 wherein the precision optical target comprises at least three concentric sub-targets.

23. The method as in claim 22 wherein the three concentric sub-targets are each concentric ellipses.

24. The method as in claim 22 wherein the computer processor is programmed to calculate centroids of each sub-target by dividing (a) a sum of a product of pixel intensity and pixel position by (b) a sum of pixel intensity in a sub-pixel array.

25. The method as in claim 24 wherein the pixel array is about 48×

48 pixels.

27. The method as in claim 14 wherein the computer processor is programmed with algorithms adapted to perform the iteration procedure in steps comprising the following steps:

I) Locate the positions of the three target centroids on each of the two Cameras and calculate six principal quantities;

Σ

_HD Δ

_HD Σ

_VD Δ

_BL Σ

_MT Δ

_MT;

II) Make an initial guess of (0, 0, 0, 0, 0,

     0) for the subject displacements and rotations (φ

, θ

, ψ

, Δ

x, Δ

y, Δ

z) leading to those centroid positions;

III) Enter the guess values for (φ

, θ

, Δ

x, Δ

x, Δ

y, Δ

z) into a translation matrix and calculate the corresponding translated 3-D target position (x_i, y_i, z_i) for each of the three target centroids;

IV) Calculate the position (horizontal and vertical pixel number) of the projection of each of the three target centroids calculated in Step III on each camera system;

V) Calculate six principal quantities using Step IV data from the two cameras;

Σ

_HD Δ

_HD Σ

_VD Δ

_BL Σ

_MT Δ

_MT;

VI) Compare these six calculated principal quantities with measured values from Step I, List the differences as a matrix of errors in the guessed/calculated quantities relative to the measured quantities;

(σ

<sub>Σ

HD</sub>,σ

<sub>Δ

HD</sub>,σ

<sub>Σ

VD</sub>,σ

<sub>Δ

BL</sub>,σ

<sub>Σ

MT</sub>,σ

<sub>Δ

MT</sub>);

VII) Determine the local partial derivatives of;

Σ

_HD Δ

_HD Σ

_VD Δ

_BL Σ

_MT Δ

_MT by repeating Steps III-V six times with small added displacements in each degree of freedom, one at a time;

VIII) Determine a coarse correction matrix by dividing the error matrix by the derivative matrix, to improve the initial guess and reduce the residual error, Create a better displacement matrix;

[φ

+Δ

φ

,θ

+Δ

θ

,ψ

+Δ

ψ

,Δ

x+Δ

(Δ

x),Δ

y+Δ

(Δ

y),Δ

z+Δ

(Δ

z)];

IX) Repeat steps III through VIII for a second and final iteration, starting with the coarse-corrected displacement matrix computed in Step VIII as the input for Step III; and

After each successive camera frame increment, repeat Steps I-IX but use the result of previous Step IX for the guess in Step II.