Apparatus and method for extracting movement path of mutual geometric relationship fixed camera group

• US 9,619,892 B2
• Filed: 05/06/2014
• Issued: 04/11/2017
• Est. Priority Date: 05/13/2013
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Kim, Jae Hean, Chu, Chang Woo, Park, Il Kyu, Cha, Young Mi, Choi, Jin Sung, Koo, Bon Ki
• Primary Examiner(s): Yentrapati, Avinash
• Application Number:
  US14/271,029
• Publication Number:
  US 20140334675A1
• Time in Patent Office: 1,071 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

Provided is an apparatus and method for extracting a movement path, the movement path extracting apparatus including an image receiver to receive an image from a camera group in which a mutual positional relationship among cameras is fixed, a geographic coordinates receiver to receive geographic coordinates of a moving object on which the camera group is fixed, and a movement path extractor to extract a movement path of the camera group based on a direction and a position of a reference camera of the camera group using the image and the geographic coordinates.

Claims

1. An apparatus for extracting a movement path, the apparatus comprising:

a processor configured to;

receive an image from a camera group in which a mutual positional relationship among cameras is fixed;

receive geographic coordinates of a moving object on which the camera group is fixed, wherein the geographic coordinates receiver receives the geographic coordinates of the moving object using a position receiving apparatus to verify a position of the moving object; and

extract a movement path of the camera group based on a direction and a position of a reference camera of the camera group and a direction and a position of a remaining cameras and the geographic coordinates, extract feature points of images photographed by the reference camera from an image sequence photographed by the camera group, match the feature points of images photographed by the reference camera, detect the direction and the position of the reference camera based on a result of the matching, extract feature points of images photographed by cameras remaining after excluding the reference camera from the camera group from an image sequence photographed by the camera group, match the feature points of the images photographed by the remaining cameras using a fundamental matrix or an essential matrix based on the direction and the position of the reference camera and a relative positional relationship between the remaining cameras and the reference camera, and detect the direction and the position of the remaining cameras based on a result of the matching.

2. The apparatus of claim 1, wherein the camera group comprises:

a camera group in which projection origins of the cameras are identical; and

a camera group in which projection origins of the cameras are fixed at predetermined intervals.

3. The apparatus of claim 1, wherein the processor comprises:

an image receiver configured to receive the image from the camera group;

a geographic coordinates receiver configured to receive geographic coordinates of the moving object; and

a movement path extractor configured to extract the movement path of the camera group.

4. The apparatus of claim 3, wherein the geographic coordinates receiver is configured to receive the geographic coordinates of the moving object in order to convert a position of the camera group to latitudinal and longitudinal coordinates.

5. The apparatus of claim 3, wherein the movement path extractor is configured to remove a result of matching feature points improperly matched during a process of matching the feature points of the images photographed by the remaining cameras, using an outlier removal algorithm.

6. The apparatus of claim 3, wherein the movement path extractor is configured to extract the movement path of the camera group based on a matrix between neighboring image frames as time elapses using a movement path of the reference camera and a relative positional relationship between the remaining cameras and the reference camera.

7. The apparatus of claim 3, wherein the movement path extractor is further configured to

calculate the directions and the positions of the reference camera and the remaining cameras constituting the camera group,

calculate identical three dimensional (3D) geographic coordinates for the reference camera and the remaining cameras from the calculated direction and the position of the reference camera and the calculated directions and the positions of the remaining cameras, wherein the directions and the positions of the reference camera in the camera group are optimized using a bundle adjustment by adding a relative position of the reference camera and a direction of each of the remaining cameras with respect to the reference camera to an optimization parameter.

8. The apparatus of claim 3, wherein the movement path extractor is configured to convert, to an absolute path of the camera group, the geographic coordinates and a relative movement path of the camera group based on a corresponding relationship of the feature points of the images photographed by the reference camera and the remaining cameras.

9. A method of extracting a movement path, the method comprising:

receiving an image from a camera group in which a mutual positional relationship among cameras is fixed;

receiving geographic coordinates of a moving object on which the camera group is fixed, wherein the receiving is based on verifying a position of the moving object; and

extracting a movement path of the camera group based on a direction and a position of a reference camera of the camera group and a direction and a position of a remaining cameras and the geographic coordinates,

wherein the extracting extracts feature points of images photographed by the reference camera from an image sequence photographed by the camera group, matches the feature points of images photographed by the reference camera, and detects the direction and the position of the reference camera based on a result of the matching, and

wherein the extracting extracts feature points of images photographed by cameras remaining after excluding the reference camera from the camera group from an image sequence photographed by the camera group, matches the feature points of the images photographed by the remaining cameras using a fundamental matrix or an essential matrix based on the direction and the position of the reference camera and a relative positional relationship between the remaining cameras and the reference camera, and detects the direction and the position of the remaining cameras based on a result of the matching.

10. The method of claim 9, wherein the camera group comprises:

a camera group in which projection origins of the cameras are identical; and

a camera group in which projection origins of the cameras are fixed at predetermined intervals.

11. The method of claim 9, wherein the receiving comprises receiving the geographic coordinates of the moving object in order to convert a position of the camera group to latitudinal and longitudinal coordinates.

12. The method of claim 9, wherein the extracting comprises removing a result of matching feature points improperly matched during a process of matching the feature points of the images photographed by the remaining cameras, using an outlier removal algorithm.

13. The method of claim 9, wherein the extracting comprises extracting the movement path of the camera group based on a matrix between neighboring image frames as time elapses using a movement path of the reference camera and a relative positional relationship between the remaining cameras and the reference camera.

14. The method of claim 9, wherein the extracting further comprises:

extracting the movement path of the camera group by calculating the directions and the positions of the reference camera and the remaining cameras constituting the camera group;

calculating identical three dimensional (3D) geographic coordinates for the reference camera and the remaining cameras from the calculated direction and the position of the reference camera and the calculated directions and the positions of the remaining cameras; and

optimizing the directions and the positions of the reference camera in the camera group by applying a bundle adjustment by adding a relative position of the reference camera and a direction of each of the remaining cameras with respect to the reference camera to an optimization parameter.

15. The method of claim 9, wherein the extracting comprises converting, to an absolute path of the camera group, the geographic coordinates and a relative movement path of the camera group based on a corresponding relationship of the feature points of the images photographed by the reference camera and the remaining cameras.

16. A non-transitory computer-readable medium storing program instructions that, when executed by a processor, causes the processor to perform the method of claim 9.

17. An apparatus for extracting a movement path, the apparatus comprising:

one or more processors configured to;

to receive an image from a camera group in which a mutual positional relationship among cameras is fixed;

to receive geographic coordinates of a moving object on which the camera group is fixed, wherein the geographic coordinates receiver receives the geographic coordinates of the moving object using a position receiving apparatus to verify a position of the moving object; and

to extract a movement path of the camera group based on a direction and a position of a reference camera of the camera group and a direction and a position of a remaining cameras and the geographic coordinates,

wherein the one or more processors extracts feature points of images photographed by the reference camera from an image sequence photographed by the camera group, matches the feature points of images photographed by the reference camera, and detects the direction and the position of the reference camera based on a result of the matching, and

wherein the one or more processors extracts feature points of images photographed by cameras remaining after excluding the reference camera from the camera group from an image sequence photographed by the camera group, matches the feature points of the images photographed by the remaining cameras using a fundamental matrix or an essential matrix based on the direction and the position of the reference camera and a relative positional relationship between the remaining cameras and the reference camera, and detects the direction and the position of the remaining cameras based on a result of the matching.