Coded visual markers for tracking and camera calibration in mobile computing systems

• US 7,809,194 B2
• Filed: 02/08/2007
• Issued: 10/05/2010
• Est. Priority Date: 10/04/2001
• Status: Active Grant

• Current Assignee: Siemens Medical Solutions USA Incorporated (Siemens AG)
• Original Assignee: Siemens Corp. (Siemens AG)
• Inventors: Zhang, Xiang, Navab, Nassir
• Primary Examiner(s): Lu; Tom Y
• Application Number:
  US11/704,137
• Publication Number:
  US 20070133841A1
• Time in Patent Office: 1,335 Days
• Field of Search: 382/103, 382/181
• US Class Current: 382/181
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components

Abstract

A method for determining a pose of a user is provided including the steps of capturing a video image sequence of an environment including at least one coded marker; detecting if the coded marker is present in the video images; if the marker is present, extracting feature correspondences of the coded marker; determining a code of the coded marker using the feature correspondences; and comparing the determined code with a database of predetermined codes to determine the pose of the user. According to an embodiment, the coded marker includes four color blocks arranged in a square formation and the determining a code of the at least one marker further includes determining a color of each of the four blocks. According to another embodiment, the marker includes a coding matrix and a code of the marker being determined by numbered squares of the coding matrix being covered by a circle.

Claims

1. A method for determining a pose of a user comprising the steps of:

capturing a video image sequence of an environment including at least one coded marker, wherein the at least one coded marker comprises a coding matrix including a plurality of columns and rows with a numbered square at intersections of the columns and rows, the coding matrix being surrounded by a rectangular frame and a code of the at least one marker being deteimined by the numbered squares being covered by a circle;

detecting if the at least one coded marker is present in the video images;

extracting feature correspondences of the at least one coded marker if the at least one marker is present;

determining a code of the at least one coded marker using the feature correspondences; and

comparing the determined code with a database of predetermined codes to determine the pose of the user.

2. The method as in claim 1, wherein the coding matrix includes m columns and n rows, where m and n are whole number, resulting in 3×

2<sup>m×

n−

4</sup> codes.

3. The method as in claim 1, wherein the detecting step further comprises applying a watershed transformation to the at least one coded marker to extract a plurality of closed-edge strings that form a contour of the at least one marker.

4. The method as in claim 3, wherein the detecting step further comprises locating at least two closed-edge strings that have close weight centers.

5. The method as in claim 4, wherein the detecting step further comprises locating a corner of the rectangular frame of the at least one marker by determining a cross-point of the at least two closed-edge strings.

6. The method as in claim 3, wherein the detecting step comprises locating corners of the rectangular frame of the at least one marker by locating cross- points of the plurality of closed-edge strings.

7. The method as in claim 6, wherein the extracting step further comprises applying a 1-d Canny edge detection to locate the edge points of the rectangular frame.

8. The method as in claim 7, wherein the extracting step further comprises computing a homography from the corners and edge points;

extracting image feature correspondences of the at least one marker; and

determining locations of the circles in the coding matrix by the image correspondences.

9. The method as in claim 8, wherein the extracting image feature correspondences is performed by linear interpolation.

10. The method as in claim 8, further comprising the step of calibrating a camera used to capture the video image sequence with the image correspondences of the least one marker.

11. The method as in claim 10, further comprising the step of determining a position and orientation of the camera relative to the at least one marker.

12. A system comprising:

a plurality of coded markers located throughout an environment, each of the plurality of coded markers relating to a location in the environment, codes of the plurality of coded markers being stored in a database, wherein a marker comprises a coding matrix including a plurality of columns and rows with a numbered square at intersections of the columns and rows, the coding matrix being surrounded by a rectangular frame and a code of a marker being determined by the numbered squares being covered by a circle;

a camera for capturing a video image sequence of the environment;

a processor coupled to the camera, the processor adapted for detecting if at least one coded marker is present in the video images extracting feature correspondences of the at least one coded marker if the at least one marker is present, determining a code of the at least one coded marker using the feature correspondences, and comparing the determined code with the database to determine the pose of the user.