VEHICLE LOCALIZATION IN OPEN-PIT MINING USING GPS AND MONOCULAR CAMERA

• US 20110299730A1
• Filed: 03/16/2011
• Published: 12/08/2011
• Est. Priority Date: 03/16/2010
• Status: Active Grant

• Current Assignee: University of Sydney
• Original Assignee: University of Sydney
• Inventors: DURRANT-WHYTE, Hugh, ELINAS, Pantelis, PERERA, Linthotage Dushantha Lochana, NETTLETON, Eric
• Granted Patent: US 9,224,050 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles

Abstract

Described herein is a method and system for vehicle localization in an open pit mining environment having intermittent or incomplete GPS coverage. The system comprises GPS receivers associated with the vehicles and providing GPS measurements when available, as well as one or more cameras 50,55 overlooking the mine region 10. The cameras 50,55 are at a known location and are used for generating a sequence of images in a field of view with predetermined calibration in a fixed coordinate system. The system further comprises a vehicle recognition processor 120 for analysing individual images from the camera to identify and locate within an image a vehicle in the mine region, as well as a vehicle tracking processor 130 for analysing a sequence of images from the camera to track the identified vehicle location in the sequence of images. A data fusion processor 160 is coupled to receive GPS measurements, when available, from the vehicle GPS receivers, to fuse the received GPS measurement and corresponding vehicle image location, and to output a vehicle localization output 125.

Claims

1. A method for vehicle localization in an open pit mining environment, comprising:

providing a camera calibrated in a fixed coordinate system with a field of view encompassing a region of vehicle operation in the mining environment;

analysing at least one picture frame from the camera using a vehicle recognition algorithm to determine the presence of a vehicle in the camera field of view;

analysing a sequence of picture frames from the camera using a vehicle tracking algorithm to determine an image position of the vehicle within the region of vehicle operation;

ascertaining at least one independent measure of the vehicle location within the region of vehicle operation relative to the fixed coordinate system; and

providing an output of vehicle localization over time using the image position of the vehicle and the at least one independent vehicle location measurement.

2. A method according to claim 1 wherein the independent measure of vehicle location is available intermittently and if the independent measure is unavailable the method comprises:

providing the output of vehicle localization over time dependent on the image position of the vehicle.

3. A method according to claim 1 wherein the independent measure of vehicle location is a measurement from a GPS receiver.

4. A method according to claim 1 comprising:

providing a plurality of cameras calibrated in the fixed coordinate system with respective fields of view encompassing the region of vehicle operation in the mining environment;

determining respective image positions of the vehicle from picture frames of at least two cameras; and

providing the output of vehicle localization over time dependent on the respective image positions and, if available, the independent vehicle location measurement.

5. A method according to claim 1, comprising:

applying Bayesian filtering to provide the output of vehicle localization.

6. A method according to claim 5 wherein the Bayesian filtering uses a Kalman filter or a particle filter.

7. A method according to claim 1 comprising:

calibrating the camera field of view in the fixed coordinate system.

8. A method according to claim 1 wherein at least one camera is mounted on a camera-support vehicle and the method comprises calibrating a field of view of the mounted camera in the fixed coordinate system.

9. A method for continuous vehicle localization in an open pit mining environment in which vehicles carrying GPS receivers operate in a region of intermittent or incomplete GPS coverage, comprising:

generating digital images with a field of view overlooking the mine region from a fixed location as vehicles move about the mine;

processing the digital images by analysing the image data to identify the presence and location of a vehicle in an image; and

fusing the identified image location of the vehicle with a measurement from the corresponding vehicle GPS receiver to provide continuous vehicle localization data.

10. A system for localization of vehicles in an open pit mining environment having intermittent or incomplete GPS coverage, comprising:

GPS receivers associated with the vehicles and providing GPS measurements when available;

a camera overlooking the mine region from a known location for generating a sequence of images in a field of view with predetermined calibration in a fixed coordinate system;

a vehicle recognition processor for analysing individual images from the camera to identify and locate within an image a vehicle in the mine region;

a vehicle tracking processor for analysing a sequence of images from the camera to track the identified vehicle location in the sequence of images; and

a data fusion processor coupled to receive GPS measurements, when available, from the vehicle GPS receivers, to fuse the received GPS measurement and corresponding vehicle image location, and to output a vehicle localization output.

11. A system according to claim 10 comprising a plurality of cameras, wherein the data fusion processor fuses the received GPS measurement with corresponding vehicle image locations derived from images from the plurality of cameras.

12. A system according to claim 10 wherein the data fusion processor comprises a Bayesian filter selected from the group consisting of a Kalman filter and a particle filter.

13. A system according to claim 10 wherein at least one camera is mobile.