Transformation between image and map coordinates

• US 9,749,594 B2
• Filed: 12/22/2011
• Issued: 08/29/2017
• Est. Priority Date: 12/22/2011
• Status: Active Grant

• Current Assignee: Pelco (Schneider Electric SE)
• Original Assignee: Pelco (Schneider Electric SE)
• Inventors: Huang, Chien-Min, Su, Wei, Vaughn, Frank
• Primary Examiner(s): Patel, Jay Au
• Assistant Examiner(s): Haghani, Shadan E
• Application Number:
  US13/335,442
• Publication Number:
  US 20130162838A1
• Time in Patent Office: 2,077 Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20101 Interactive definition of point of interest, landmark or seed
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Systems and methods for transformations between image and map coordinates, such as those associated with a video surveillance system, are described herein. An example of a method described herein includes selecting a reference point within the image with known image coordinates and map coordinates, computing at least one transformation parameter with respect to a location and a height of the camera and the reference point, detecting a target location to be tracked within the image, determining image coordinates of the target location, and computing map coordinates of the target location based on the image coordinates of the target location and the at least one transformation parameter.

Claims

1. A method of identifying map coordinates of a location within an image captured by a camera, the method comprising:

obtaining, at a location server communicatively coupled to the camera, a first image captured by the camera;

selecting, at the location server, a single reference point appearing within the first image captured by the camera, the single reference point having known image coordinates and depicting a location with known two-dimensional map coordinates in an associated two-dimensional map;

computing, at the location server, at least one transformation parameter to transform two-dimensional camera coordinate locations for a camera frame of reference to two-dimensional map coordinate locations for the associated two-dimensional map, with the at least one transformation parameter including a rotation angle and a tilt angle of the camera relative to the single reference point, based on location data for the camera and the single reference point appearing within the image captured by the camera, with the location data expressed in terms of image coordinates and map coordinates and including a location and an elevation of the camera in the map coordinates and the image coordinates and the known two-dimensional map coordinates of the single reference point;

obtaining, at the location server, a second image captured by the camera;

detecting, within the second image, a target location to be tracked;

determining, at the location server, image coordinates of the target location; and

computing, at the location server, map coordinates of the target location based on the image coordinates of the target location and the at least one transformation parameter computed based on the location data for the camera and the single reference point appearing within the image captured by the camera, including the location and the elevation of the camera in the map coordinates and the image coordinates and the known two-dimensional map coordinates of the single reference point.

2. The method of claim 1 wherein computing the map coordinates of the target location comprises computing a horizontal displacement angle and a vertical displacement angle from a center of the second image to the target location within the second image.

3. The method of claim 2 wherein computing the map coordinates of the target location comprises determining a position of the target location relative to an Earth-based coordinate system based on the horizontal displacement angle, the vertical displacement angle, the location and elevation of the camera, the rotation angle and the tilt angle.

4. The method of claim 3 wherein computing the at least one transformation parameter comprises computing a rotation matrix based at least in part on the rotation angle.

5. The method of claim 4 wherein the information relating to location and elevation of the camera comprises map coordinates of the camera, and wherein computing the map coordinates of the target location further comprises computing the map coordinates of the target location based on the rotation matrix, the map coordinates of the camera, and the map coordinates of the single reference point.

6. The method of claim 1 wherein the target location is a location of an object detected in the second image, and wherein the method further comprises tracking image coordinates and map coordinates of the object.

7. The method of claim 1 wherein the camera comprises a varifocal optical system, and wherein computing the at least one transformation parameter comprises:

determining a focal length of the camera; and

computing the at least one transformation parameter based on horizontal and vertical fields of view associated with the focal length of the camera.

8. The method of claim 7 wherein determining the focal length of the camera comprises:

obtaining, at the location server, a third image captured by the camera;

selecting a first reference point and a second reference point within the third image, the first reference point and the second reference point each having known image coordinates and depicting a location with known map coordinates; and

determining the focal length of the camera as a function of the location and elevation of the camera, the image coordinates and map coordinates of the first reference point, and the image coordinates and map coordinates of the second reference point.

9. The method of claim 1 comprising obtaining, at the location server, a field of view value associated with the camera, wherein the at least one transformation parameter is further based on the field of view value.

10. The method of claim 1, wherein computing the map coordinates of the target location comprises:

transforming the image coordinates of the target location to world ground coordinates of the target location; and

transforming the world ground coordinates of the target location to the map coordinates of the target location.

11. A target coordinate translation system comprising:

a camera configured to capture a first image and a second image, wherein the camera has a known location and elevation and a system of image coordinates is defined with respect to the first image and the second image;

a calibration module communicatively coupled to the camera and configured to identify a single reference point appearing within the first image captured by the camera, the single reference point having known image coordinates and depicting a location with known two-dimensional map coordinates in an associated two-dimensional map, and to compute at least one transformation parameter to transform two-dimensional camera coordinate locations for a camera frame of reference to two-dimensional map coordinate locations for the associated two-dimensional map, the at least one transformation parameter including a rotation angle and a tilt angle of the camera relative to the single reference point, based on location data for the camera and the single reference point appearing within the image captured by the camera, with the location data expressed in terms of image coordinates and map coordinates and including the location and elevation of the camera in the map coordinates and the image coordinates and the known two-dimensional map coordinates of the single reference point;

an object tracking module communicatively coupled to the camera and configured to select a target location within the second image and to identify image coordinates of the target location; and

a coordinate transformation module communicatively coupled to the calibration module and the object tracking module and configured to compute map coordinates of the target location based on the image coordinates of the target location and the at least one transformation parameter computed based on the location data for the camera and the single reference point appearing within the image captured by the camera, including the location and the elevation of the camera in the map coordinates and the image coordinates and the known two-dimensional map coordinates of the single reference point.

12. The system of claim 11 wherein the coordinate transformation module is further configured to compute a horizontal displacement angle and a vertical displacement angle from a center of the second image to the target location within the second image.

13. The system of claim 12 wherein the coordinate transformation module is further configured to determine coordinates of the target location relative to an Earth-based coordinate system based on the horizontal displacement angle, the vertical displacement angle, the location and elevation of the camera, the rotation angle and the tilt angle.

14. The system of claim 13 wherein:

the calibration module is further configured to compute a rotation matrix based at least in part on the rotation angle; and

the system further comprises a mapping module communicatively coupled to the coordinate transformation module and configured to identify map coordinates of the camera and to compute the map coordinates of the target location based on the rotation matrix, the map coordinates of the camera, and the map coordinates of the single reference point.

15. The system of claim 11 wherein:

the camera comprises a varifocal optical system; and

the calibration module is further configured to determine a focal length of the camera and to compute the at least one transformation parameter based on horizontal and vertical fields of view associated with the focal length of the camera.

16. The system of claim 15 wherein:

the camera is further configured to capture a third image, the third image being associated with the system of image coordinates; and

the calibration module is further configured to select a first reference point and a second reference point within the third image, the first reference point and the second reference point each having known image coordinates and depicting a location with known map coordinates, and to determine the focal length of the camera as a function of the location and elevation of the camera, the image coordinates and map coordinates of the first reference point, and the image coordinates and map coordinates of the second reference point.

17. A system for identifying map coordinates corresponding to a location within a captured image, the system comprising:

a camera configured to capture a first image and a second image, wherein the camera has a known location and elevation and a system of image coordinates is defined with respect to the first image and the second image;

calibration means, communicatively coupled to the camera, for selecting a single reference point appearing within the first image captured by the camera, the single reference point having known image coordinates and depicting a location with known two-dimensional map coordinates in an associated two-dimensional map, and computing at least one transformation parameter to transform two-dimensional camera coordinate locations for a camera frame of reference to two-dimensional map coordinate locations for the associated two-dimensional map, with the at least one transformation parameter including a rotation angle and a tilt angle of the camera relative to the single reference point, based on location data for the camera and the single reference point appearing within the image captured by the camera, with the location data expressed in terms of image coordinates and map coordinates and including the location and elevation of the camera in the map coordinates and the image coordinates and the known two-dimensional map coordinates of the single reference point;

tracking means, communicatively coupled to the camera, for detecting, within the second image, a target location and determining image coordinates of the target location; and

mapping means, communicatively coupled to the calibration means and the tracking means, for computing map coordinates of the target location based on the image coordinates of the target location and the at least one transformation parameter computed based on the location data for the camera and the single reference point appearing within the image captured by the camera, including the location and the elevation of the camera in the map coordinates and the image coordinates and the known two-dimensional map coordinates of the single reference point.

18. The system of claim 17 wherein the mapping means is further configured to compute a horizontal displacement angle and a vertical displacement angle from a center of the second image to the target location within the second image and to determine a position of the target location relative to an Earth-based coordinate system based on the horizontal displacement angle, the vertical displacement angle, the location and elevation of the camera, the rotation angle and the tilt angle.

19. The system of claim 18 wherein:

the calibration means is further configured to compute a rotation matrix based at least in part on the rotation angle; and

the mapping means is further configured to identify map coordinates of the camera and to compute the map coordinates of the target location based on the rotation matrix, the map coordinates of the camera, and the map coordinates of the single reference point.

20. The system of claim 17 wherein:

the camera comprises a varifocal optical system; and

the calibration means is further configured to determine a focal length of the camera and to compute the at least one transformation parameter based on horizontal and vertical fields of view associated with the focal length of the camera.

21. The system of claim 20 wherein:

the camera is further configured to capture a third image, the third image being associated with the system of image coordinates; and

the calibration means is further configured to select a first reference point and a second reference point within the third image, the first reference point and the second reference point each having known image coordinates and depicting a location with known map coordinates, and to determine the focal length of the camera as a function of the location and elevation of the camera, the image coordinates and map coordinates of the first reference point, and the image coordinates and map coordinates of the second reference point.

22. A computer program product residing on a non-transitory processor-executable computer storage medium, the computer program product comprising processor-executable instructions configured to cause a processor to:

identify information relating to a camera and a system of image coordinates relative to a first image and a second image captured by the camera;

select a single reference point appearing within the first image captured by the camera, the single reference point having known image coordinates and depicting a location with known two-dimensional map coordinates in an associated two-dimensional map;

compute at least one transformation parameter to transform two-dimensional camera coordinate locations for a camera frame of reference to two-dimensional map coordinate locations for the associated two-dimensional map, with the at least one transformation parameter including a rotation angle and a tilt angle of the camera relative to the single reference point, based on location data for the camera and the single reference point appearing within the image captured by the camera, with the location data expressed in terms of image coordinates and map coordinates and including location and elevation of the camera in the map coordinates and the image coordinates and the known two-dimensional map coordinates of the single reference point;

detect, within the second image, a target location to be tracked;

determine image coordinates of the target location; and

compute map coordinates of the target location based on the image coordinates of the target location and the at least one transformation parameter computed based on the location data for the camera and the single reference point appearing within the image captured by the camera, including the location and the elevation of the camera in the map coordinates and the image coordinates and the known two-dimensional map coordinates of the single reference point.

23. The computer program product of claim 22 wherein:

the instructions configured to cause the processor to compute the map coordinates of the target location are configured to cause the processor to;

compute a horizontal displacement angle and a vertical displacement angle from a center of the second image to the target location within the second image; and

determine a position of the target location relative to an Earth-based coordinate system based on the horizontal displacement angle, the vertical displacement angle, the location and elevation of the camera, the rotation angle and the tilt angle.

24. The computer program product of claim 23 wherein:

the instructions configured to cause the processor to compute the at least one transformation parameter are configured to cause the processor to compute a rotation matrix based at least in part on the rotation angle;

the information relating to location and elevation of the camera comprises map coordinates of the camera; and

the instructions configured to cause the processor to compute the map coordinates of the target location are configured to cause the processor to compute the map coordinates of the target location based on the rotation matrix, the map coordinates of the camera, and the map coordinates of the single reference point.