Video pose tracking system and method

• US 20040002642A1
• Filed: 07/01/2002
• Published: 01/01/2004
• Est. Priority Date: 07/01/2002
• Status: Active Grant

• Current Assignee: Claronav Inc. (Kofax Incorporated)
• Original Assignee: Claudio Gatti, Doron Dekel
• Inventors: Dekel, Doron, Gatti, Claudio
• Granted Patent: US 6,978,167 B2
• Time in Patent Office: Days
• US Class Current: 600/407
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/363 Use of fiducial points
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/3983 Reference marker arrangements for use with image guided surgery
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: G01B 11/002 for measuring two or more coordinates
• CPC Class Codes: G01B 11/26 for measuring angles or tapers; for testing the alignment of axes
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning

Abstract

A measurement system for tracking a pose of an object displaceable in a coordinate reference frame, such as a three dimensional coordinate system. The system comprises a predefined physical marker adapted for rigidly coupling to the object. The marker can be two or three dimensional in shape. The marker includes a visible target pattern with a series of contrast regions for providing at least two distributed feature points. The contrast regions include an arrangement of alternating dark and bright regions defining an intensity saddle point at their intersection. Each feature point is located at a respective intensity saddle point. The system also has an optical sensor assembly for generating a pair of digital images of the target pattern containing the feature points, wherein the pair of images generated are from dissimilar viewing angles. The system also has an identifier for testing the pixels of the digital images to determine the spatial location in the reference frame for each of the feature points of the target pattern. A marker template including a set of reference characteristics is used to uniquely define a pose relationship between the features points. Further, a processor is used for comparing the determined spatial locations of the feature points to the set of reference characteristics to confirm the recognition of the predefined marker. A method is also described to identify the intensity saddles contained in the digital images.

Claims

1. A measurement system for tracking a pose of a physical marker displaceable in a coordinate reference frame, the system comprising:

a) the physical marker including a visible target pattern with an arrangement of contrast regions for providing at least two distributed feature points;

b) an optical sensor assembly for generating a pair of digital images of the target pattern containing the feature points, the pair of images generated from dissimilar viewing angles;

c) a marker template comprising a unique set of reference characteristics including a relationship between the feature points; and

d) a processor for testing the pixels of the digital images to determine the spatial location in the reference frame for at least two of the feature points of the target pattern, for comparing the determined spatial locations of the feature points to the set of reference characteristics to confirm the recognition of the physical marker, and for calculating the physical marker pose from the spatial location of the feature points.

2. The system of claim 1 further comprising at least three feature points to define a pair of non-collinear target vectors.

3. The system of claim 2, wherein at least four feature points define the pair of non-collinear target vectors.

4. The system of claim 2, wherein a plurality of the feature points defines at least three non-collinear target vectors.

5. The system of claim 2 further comprising the marker having a plurality of target patterns with at least one of the feature points present on each one of the plurality of target patterns.

6. The system of claim 2, wherein the reference characteristics are selected from the group comprising a distance between adjacent feature points, the number of feature points, a distance between adjacent target vectors, and an angle between adjacent target vectors.

7. The system of claim 1 further comprising the contrast regions including a plurality of alternating dark and bright regions.

8. The system of claim 7 further comprising an intensity gradient between adjacent pixels in the digital images, one of the adjacent pixels being in one of the dark regions and the other of the adjacent pixels being in one of the bright regions.

9. The system of claim 8, wherein a distribution of intensity gradients corresponding to the alternating dark and bright regions defines an arrangement of alternating intensity peaks and valleys.

10. The system of claim 9, wherein each of the feature points is an intensity saddle located in the center of the arrangement of alternating peaks and valleys.

11. The system of claim 10, wherein the type of the feature points is determined by the number of peaks and valleys associated with each of the feature points.

12. The system of claim 11, wherein the feature point type is included in the reference characteristics.

13. The system of claim 1 further comprising a plurality of target patterns associated with the marker, each of the target patterns having at least one feature point.

14. The system of claim 13, wherein each of the target patterns contain a respective set of at least two of the feature points.

15. The system of claim 14, wherein the respective set of feature points defines a corresponding target vector.

16. The system of claim 15, wherein each of the target vectors is oriented in a unique direction.

17. The system of claim 16, wherein the plurality of target patterns are located on a planar surface.

18. The system of claim 16, wherein at least one of the target patterns is located on a three dimensional surface.

19. The system of claim 16, wherein at least one of the target patterns is positioned on a corresponding one of a plurality of planes, each of the planes is oriented in a different direction.

34. The system according to claim 1 further comprising a second target pattern located on a second marker, the second marker movable within a predefined path in relation to the target pattern of the physical marker.

35. The system according to claim 34, wherein the relative position of the second marker along the path is determined in relation to the pose of the physical marker.

36. The system according to claim 1, wherein the testing and comparing by the processor includes the relative orientation of the feature points with respect to one another.

20. A measurement method for tracking a pose of a predefined physical marker displaceable in a coordinate reference frame, the marker including a visible target pattern with a series of contrast regions for providing at least two distributed feature points, the method comprising the steps of:

a) generating a pair of digital images of the target pattern containing the feature points, the pair of images generated from dissimilar viewing angles;

b) testing the pixels of the digital images to determine the spatial location in the reference frame for at least two of the feature points of the target pattern;

c) comparing the determined spatial locations of the feature points to a unique set of reference characteristics for recognising the physical marker in the coordinate frame; and

d) calculating the physical marker pose from the determined spatial locations of the feature points.

21. The method of claim 20, wherein the step of testing the pixels includes a first point test for rejecting a majority of the pixels not containing candidate feature points.

22. The method of claim 21 further comprising the step of applying a second test for identifying the feature points from the remaining pixels containing the candidate feature points.

23. The method of claim 21, wherein the first test for the image pixels is selected from the group comprising multi-resolution processing, change detection, edge orientation template, edge orientation histogram, and fast tracking slow detection.

24. The method of claim 22, wherein the second test for the image pixels is selected from the group comprising multi-resolution processing, change detection, edge orientation template, edge orientation histogram, and fast tracking slow detection.

25. The method of claim 20, wherein the step of testing further comprises estimating a pair of edge lines intersecting in the vicinity of a feature point candidate.

26. The method of claim 25 further comprising the step of selecting a sequence of pixels in a direction different to the direction of one of said pair of edge lines according to intensity gradients between adjacent contrast regions.

27. The method of claim 26, wherein said sequence is selected at least two pixel units away from the feature point candidate.

28. The method of claim 27 further comprising the step of selecting a plurality of intensity values on each of said pair of edge lines contained in said sequence.

29. The method of claim 28 further comprising the step of fitting a curve equation to the intensity values for determining the location of each of said pair of edge lines to sub pixel accuracy.

30. The method of claim 29 further comprising the step of performing a linear regression on the intensity values of sub pixel accuracy for each of said pair of edge lines.

31. The method of claim 30 further comprising the step of calculating the intersections of the edge lines resulting from the regression for determining the sub pixel location of the feature point candidate at an intensity saddle point.

32. The method of claim 29, wherein the curve fit is second order or higher.

33. A computer readable medium storing an algorithm for performing the method steps of claim 20.

37. The method according to claim 20, wherein the steps of testing and comparing includes the relative orientation of the feature points with respect to one another.