METHOD AND SYSTEM FOR RECALIBRATING SENSING DEVICES WITHOUT FAMILIAR TARGETS

• US 20170280131A1
• Filed: 06/09/2017
• Published: 09/28/2017
• Est. Priority Date: 10/01/2015
• Status: Active Grant

• Current Assignee: Alibaba DAMO Technology Company Limited
• Original Assignee: Infinity Augmented Reality Israel, Ltd. (Alibaba Group Holding Ltd.)
• Inventors: PROTTER, Matan, KUSHNIR, Motti
• Granted Patent: US 10,499,038 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 17/002 for television cameras

Abstract

Methods and systems for recalibrating sensing devices without using familiar targets are provided herein. The method may include: capturing a scene using a sensing device; determining whether or not the device is calibrated; in a case that said sensing device is calibrated, adding at least one new landmark to the known landmarks; in a case that said sensing device is not calibrated, determining at least some of the captured objects as objects stored on a database as known landmarks at the scene whose position is known; and calibrating the sensing device based on the known landmarks. The system may have various architectures that include a sensing device which captures images of the scene and further derive 3D data on the scene of some form.

Claims

1. A method comprising:

providing captured data relating to one or more captured 2D points in a frame of a scene captured by a sensing device;

providing landmark data relating to a landmark in the scene from a landmark database;

mapping the landmark data to one or more calculated 2D points in a frame of the scene;

comparing the captured 2D points in the frame with the data relating to the calculated 2D points in the frame to yield a delta being a difference between the data relating to the captured 2D points and the calculated 2D points;

minimizing the delta by adjusting one or more calibration parameters; and

calibrating the sensing device based on the adjusted calibration parameters.

2. The method of claim 1 comprising minimizing the delta by minimizing an objective function measuring projection errors.

3. The method of claim 1 comprising minimizing the delta by using gradient descent.

4. The method of claim 1 comprising minimizing the delta by using the Levenberg-Marquardt optimization technique.

5. The method of claim 1, wherein the captured data is captured by a 3D sensing device that captures 3D images.

6. The method of claim 1, wherein the captured data is captured by a 2D sensing device coupled to an inertial measurement unit (IMU).

7. The method of claim 1, wherein the captured data is captured by a 2D sensing device mounted on a moving platform.

8. A system comprising:

a sensing device configured to provide captured data relating to one or more captured 2D points in a frame of a scene captured;

a database configured to provide landmark data relating to a landmark in the scene; and

a computer processor configured to;

map the landmark data to one or more calculated 2D points in a frame of the scene,

compare the captured 2D points in the frame with the data relating to the calculated 2D points in the frame to yield a delta being a difference between the data relating to the captured 2D points and the calculated 2D points,

minimize the delta by adjusting one or more calibration parameters, and

calibrate the sensing device based on the adjusted calibration parameters.

9. The system of claim 8, wherein the computer processor is configured to minimize the delta by minimizing an objective function measuring projection errors.

10. The system of claim 8, wherein the computer processor is configured to minimize the delta by using gradient descent.

11. The system of claim 8, wherein the computer processor is configured to minimize the delta by using the Levenberg-Marquardt optimization technique.

12. The system of claim 8, wherein the sensing device is a 3D sensing device that captures 3D images.

13. The system of claim 8, wherein the sensing device is a 2D sensing device coupled to an inertial measurement unit (IMU).

14. The system of claim 8, wherein the sensing device is a 2D sensing device mounted on a moving platform.