Self-calibration of a static camera from vehicle information

• US 10,029,622 B2
• Filed: 07/23/2015
• Issued: 07/24/2018
• Est. Priority Date: 07/23/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brown, Lisa M., Fan, Quanfu, Zhai, Yun
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): BELAI, NAOD W
• Application Number:
  US14/807,448
• Publication Number:
  US 20170024889A1
• Time in Patent Office: 1,097 Days
• Field of Search: 348148
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2300/402 Image calibration
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

A method to automatically calibrate a static camera in a vehicle is provided. The method may include receiving a captured image file. The method may also include detecting a plurality of vehicles in the captured image file. The method may further include determining a 2D projected shape, size, location, direction of travel, and a plurality of features for each vehicle at various locations in the captured image file. The method may additionally include inferring a plurality of geometry scenes associated with the captured image file, whereby the plurality of geometry scenes is inferred based on the determined 2D projected shape, size, location, direction of travel, and the plurality of features of each vehicle within the detected plurality of vehicles as projected onto the captured image file. The method may include calibrating the static camera based on the inferred plurality of geometry scenes associated with the captured image file.

Claims

1. A processor-implemented method to automatically calibrate a static camera in a vehicle, the method comprising:

receiving, by a processor, a captured image file;

detecting a plurality of vehicles in the captured image file;

inferring a pose associated with each vehicle within the detected plurality of vehicles;

inferring a height of the static camera and a tilt of the static camera to a ground plane based on the inferred pose associated with each vehicle within the detected plurality of vehicles and a variation in a projection of a 3D model of each vehicle within the detected plurality of vehicles;

determining a 2D projected shape, size, location, direction of travel, and a plurality of features for each vehicle within the detected plurality of vehicles at more than one location in the captured image file based on the inferred height and tilt of the static camera;

inferring a plurality of scene geometry associated with the captured image file, wherein the plurality of scene geometry is inferred based on the determined 2D projected shape, size, location, direction of travel, and the plurality of features of each vehicle within the detected plurality of vehicles as projected onto the captured image file; and

calibrating the static camera based on the inferred plurality of scene geometry associated with the captured image file.

2. The method of claim 1, further comprising:

estimating a size of a plurality of objects in the captured image file; and

estimating a speed of the detected plurality of vehicles in the captured image file.

3. The method of claim 1, further comprising:

estimating a plurality of distances travelled by each vehicle in the captured image file.

4. The method of claim 1, wherein the camera is a pin-hole camera and the captured image file comprises at least one of a camera feed, a video, a set of images, or a single image.

5. The method of claim 1, wherein detecting the plurality of vehicles in the captured image file includes the use of a standard Adaboost algorithm.

6. The method of claim 1, wherein determining the 2D projected shape, size, location, direction of travel, and the plurality of features for each vehicle within the plurality of detected vehicles at various locations in the captured image file includes the use of at least one of a vehicle detection, a background subtraction, an object classification, a feature-based tracking, a pose-dependent vehicle detection, and a foreground detection.

7. The method of claim 1, wherein detecting the plurality of vehicles in the captured image file comprises creating a bounding box in the captured image file together with the inferred pose associated with each vehicle.

8. The method of claim 7, wherein the inferred pose comprises determining a direction each vehicle is facing.

9. The method of claim 7, wherein creating the bounding box in the captured image file comprises taking each vehicle to rest horizontally on a ground plane and modelling each vehicle as a rectangular prism of a fixed fiducial size.

10. The method of claim 1, wherein determining the calibration comprises determining a height of the static camera, a focal length of the static camera, and an angle of the static camera.

11. A computer system to automatically calibrate a static camera in a vehicle the computer system comprising:

one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage devices, and program instructions stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising;

receiving a captured image file;

detecting a plurality of vehicles in the captured image file;

inferring a pose associated with each vehicle within the detected plurality of vehicles;

inferring a height of the static camera and a tilt of the static camera to a ground plane based on the inferred pose associated with each vehicle within the detected plurality of vehicles and a variation in a projection of a 3D model of each vehicle within the detected plurality of vehicles;

determining a 2D projected shape, size, location, direction of travel, and a plurality of features for each vehicle within the detected plurality of vehicles at more than one location in the captured image file based on the inferred height and tilt of the static camera;

inferring a plurality of scene geometry associated with the captured image file, wherein the plurality of scene geometry is inferred based on the determined 2D projected shape, size, location, direction of travel, and the plurality of features of each vehicle within the detected plurality of vehicles as projected onto the captured image file; and

calibrating the static camera based on the inferred plurality of scene geometry associated with the captured image file.

12. The computer system of claim 11, further comprising:

estimating a size of a plurality of objects in the captured image file; and

estimating a speed of the detected plurality of vehicles in the captured image file.

13. The computer system of claim 11, further comprising:

estimating a plurality of distances travelled by each vehicle in the captured image file.

14. The computer system of claim 11, wherein the camera is a pin-hole camera and the captured image file comprises at least one of a camera feed, a video, a set of images, or a single image.

15. The computer system of claim 11, wherein detecting the plurality of vehicles in the captured image file includes the use of a standard Adaboost algorithm.

16. The computer system of claim 11, wherein determining the 2D projected shape, size, location, direction of travel, and the plurality of features for each vehicle within the plurality of detected vehicles at various locations in the captured image file includes the use of at least one of a vehicle detection, a background subtraction, an object classification, a feature-based tracking, a pose-dependent vehicle detection, and a foreground detection.

17. The computer system of claim 11, wherein detecting the plurality of vehicles in the captured image file comprises creating a bounding box in the captured image file together with the inferred pose associated with each vehicle.

18. The computer system of claim 17, wherein the inferred pose comprises determining a direction each vehicle is facing.

19. The computer system of claim 17, wherein creating the bounding box in the captured image file comprises taking each vehicle to rest horizontally on a ground plane and modelling each vehicle as a rectangular prism of a fixed fiducial size.

20. A computer program product to automatically calibrate a static camera in a vehicle, the computer program product comprising:

one or more computer-readable storage devices and program instructions stored on at least one of the one or more tangible storage devices, the program instructions executable by a processor, the program instructions comprising;

program instructions to receive a captured image file;

program instructions to detect a plurality of vehicles in the captured image file;

program instructions to infer a pose associated with each vehicle within the detected plurality of vehicles;

program instructions to infer a height of the static camera and a tilt of the static camera to a ground plane based on the inferred pose associated with each vehicle within the detected plurality of vehicles and a variation in a projection of a 3D model of each vehicle within the detected plurality of vehicles;

program instructions to determine a 2D projected shape, size, location, direction of travel, and a plurality of features for each vehicle within the detected plurality of vehicles at more than one location in the captured image file based on the inferred height and tilt of the static camera;

program instructions to infer a plurality of scene geometry associated with the captured image file, wherein the plurality of scene geometry is inferred based on the determined 2D projected shape, size, location, direction of travel, and the plurality of features of each vehicle within the detected plurality of vehicles as projected onto the captured image file; and

program instructions to calibrate the static camera based on the inferred plurality of scene geometry associated with the captured image file.