Tracking objects with markers

• US 20070081695A1
• Filed: 10/04/2006
• Published: 04/12/2007
• Est. Priority Date: 10/04/2005
• Status: Active Grant

• Current Assignee: Thales Visionix Incorporated (Thales SA)
• Original Assignee: Thales Visionix Incorporated (Thales SA)
• Inventors: Foxlin, Eric, Naimark, Leonid
• Granted Patent: US 8,224,024 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

The spatial location and azimuth of an object are computed from the locations, in a single camera image, of exactly two points on the object and information about an orientation of the object. One or more groups of four or more collinear markers are located in an image, and for each group, first and second outer markers are determined, the distances from each outer marker to the nearest marker in the same group are compared, and the outer marker with a closer nearest marker is identified as the first outer marker. Based on known distances between the outer markers and the marker nearest the first outer marker, an amount of perspective distortion of the group of markers in the image is estimated. Based on the perspective distortion, relative distances from each other point in the group to one of the outer markers are determined. Based on the relative distances, the group is identified.

Claims

1. A method comprising computing the spatial location and azimuth of an object from the locations, in a single camera image, of exactly two points on the object, and information about an orientation of the object.

2. The method of claim 1 wherein the information about an orientation of the object comes from a first inertial sensor mounted on the object.

3. The method of claim 1, further comprising using image locations in said camera image of one or more additional points on the object to identify said two points.

4. The method of claim 1, further comprising obtaining said camera image from a camera whose optical axis is collinear with the axis of gravity.

5. The method of claim 1, further comprising obtaining said camera image from a camera whose optical axis has a known tilt with respect to gravity.

6. The method of claim 1 further comprising using information from a second inertial sensor measuring orientation of said camera to compute said spatial location and azimuth of said object.

7. The method of claim 6, wherein information from said second inertial sensor comprises information about pitch and roll with respect to gravity.

8. The method of claim 7, wherein azimuth of the camera, if measured, is not used to compute the relative location and azimuth of the object with respect to the camera.

9. The method of claim 2, wherein information from said first inertial sensor comprises pitch with respect to gravity.

10. The method of claim 9, wherein information from said first inertial sensor also comprises roll with respect to gravity.

11. The method of claim 3, wherein said one or more additional points includes a third point that is closer to one of said two points than to the other, and wherein identifying said two points comprises using the location in said image of said third point to distinguish said two points from each other.

12. The method of claim 11, wherein said one or more additional points includes a fourth point that is collinear with said two points, and wherein identifying said two points further comprises using the location in said image of said fourth point to distinguish the linear array of points to which it belongs from other linear arrays of points.

13. The method of claim 2, wherein information from said first inertial sensor comprises pitch, and the pitch of the line containing the two points is equivalent to the pitch of the object.

14. The method of claim 9, further comprising calculating pitch of the line containing the two points using the measured pitch and roll of the object and a known orientation of said line on the object.

15. The method of claim 2 further comprising updating the computed spatial location and orientation of the object based on the information about the angular velocity and linear acceleration of the object from the first inertial sensor.

16. The method of claim 15 further comprising correcting for drift in the updated spatial location or orientation of the object using an updated image from the camera.

17. The method of claim 1 wherein the camera image is provided by a camera mounted on the head of a person.

18. The method of claim 17 wherein the object is a marking apparatus on the person'"'"'s hand.

19. The method of claim 17 wherein the object is a tool.

20. The method of claim 17 wherein the object is a weapon.

21. The method of claim 17 further comprising

projecting a visual display representing a virtual world, and

including in the visual display a representation of the object, including locating and positioning the representation of the object based on the computed spatial location and azimuth of the object.

22. The method of claim 17 further comprising

in a display visible to the person, projecting images that supplement the person'"'"'s view of their surroundings, in which the images include a representation of the object positioned in the display based on the computed spatial location and azimuth of the object.

23. The method of claim 1 further comprising

inverting the spatial location and azimuth of the object, and

from the inverted location and azimuth, determining the spatial location and orientation of a camera that produced the image.

24. The method of claim 23 further comprising

from an updated image from the camera and inertial sensors associated with the camera, determining an updated spatial location and orientation of the camera.

25. A set of groups of markers, each group comprising four or more collinear markers, arranged such that the ratio of the distances between three fixed base markers is the same for each group in the set.

26. The set of groups of markers of claim 25, wherein each group has a different arrangement of one or more encoding markers than each other group.

27. An apparatus comprising a group of markers arranged in a pattern that is uniquely identifiable by a single camera, comprising a linear pattern of four or more markers in which three base markers at the ends have a fixed geometry and one or more additional markers between the ends uniquely distinguish the group from other groups having the same base marker geometry.

28. The apparatus of claim 27 in which the markers are LEDs.

29. The apparatus of claim 28 also comprising a circuit board having a plurality of locations where LEDs can be installed.

30. The apparatus of claim 29 wherein at least some of the locations comprise connections for at least two separate circuits for energizing LEDs installed at the locations.

31. The apparatus of claim 27 wherein the markers comprise a printed pattern.

32. The apparatus of claim 31 wherein the printed pattern comprises white dots on a black background.

33. The apparatus of claim 31 wherein the printed pattern comprises black dots on a white background.

34. The apparatus of claim 27 wherein the printed pattern is attached to an adhesive strip.

35. The apparatus of claim 27 wherein the printed pattern is attached to a magnetic strip.

36. A method comprising

in an image, locating one or more groups of four or more collinear markers, and for each group,

determining first and second outer markers,

comparing the distance from each outer marker to the nearest marker in the same group, and identifying as the first outer marker the outer marker with a closer nearest marker,

based on known distances between the outer markers and the marker nearest the first outer marker, estimating an amount of perspective distortion of the group of markers in the image,

based on the perspective distortion, determining relative distances from each other point in the group to one of the outer markers, and

based on the relative distances, identifying the group.

38. The method of claim 36 further comprising

based on the identified group, identifying a location of a source of the image.

39. The method of claim 38 further comprising

based on a location of the identified group in an updated image, updating the location of the source of the image.

40. The method of claim 38 further comprising

based on an identification of a second group in an updated image, updating the location of the source of the image.

37. The method of 36 wherein identifying the group comprises identifying a code value based on the distances from each other point in the group to one of the outer markers, and identifying one of a known set of groups that most closely corresponds to the code value.

41. A method comprising

receiving coordinate information for images, on an imaging device of a camera, of two points on an object,

receiving pitch information from a sensor on the object,

using the coordinate information and the pitch information to obtain candidate values for the azimuth of the object,

selecting one azimuth value based on an evaluation of the candidate azimuth values in equations relating the coordinate information and pitch information to distances of the points from the camera.

42. The method of claim 41 in which the candidate azimuth values are calculated by solving a system of equations relating the coordinate information, pitch information, and known geometry of the object to the distances of the points from the camera.