Calibration of large camera networks

• US 8,760,521 B2
• Filed: 05/14/2010
• Issued: 06/24/2014
• Est. Priority Date: 05/15/2009
• Status: Expired due to Fees

• Current Assignee: Olympus Corporation, Purdue Research Foundation (The Trustees of Purdue University (d/b/a Purdue University))
• Original Assignee: Olympus Corporation, Purdue Research Foundation (The Trustees of Purdue University (d/b/a Purdue University))
• Inventors: Medeiros, Henry Ponti, Park, Johnny, Kak, Avinash, Iwaki, Hidekazu
• Primary Examiner(s): Harold, Jefferey
• Assistant Examiner(s): Khalid, Omer
• Application Number:
  US13/201,064
• Publication Number:
  US 20110310255A1
• Time in Patent Office: 1,502 Days
• US Class Current: 348/187
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04B 17/40 of relay systems
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/15 Image signal generation with circuitry for avoiding or correcting image misregistration
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation

Abstract

The present disclosure relates to a sensor network including a plurality of nodes, each node having a directional sensor, a communication module, and a processor configured to receive local measurements of a calibration object from the directional sensor, receive additional measurements of the calibration object from neighboring nodes via the communication module, estimate an initial set of calibration parameters in response to the local and additional measurements, receive additional sets of calibration parameters from neighboring nodes via the communication module, and recursively estimate an updated set of calibration parameters in response to the additional sets of calibration parameters. Additional systems and methods for calibrating a large network of camera nodes are disclosed.

Claims

1. A method comprising:

sensing a calibration object with a plurality of directional sensor nodes;

exchanging measurement data regarding the calibration object between the plurality of directional sensor nodes;

estimating an initial set of calibration parameters in each of the plurality of directional sensor nodes, each of the plurality of directional sensor nodes estimating its initial set of calibration parameters in response to the measurement data which that directional sensor node sent to other directional sensor nodes of the plurality of directional sensor nodes and in response to the exchanged measurement data which that directional sensor node received from other directional sensor nodes of the plurality of directional sensor nodes;

exchanging the initial sets of calibration parameters between the plurality of directional sensor nodes; and

estimating an updated set of calibration parameters in each of the plurality of directional sensor nodes, each of the plurality of directional sensor nodes estimating its updated set of calibration parameters in response to the initial set of calibration parameters which that directional sensor node sent to other directional sensor nodes of the plurality of directional sensor nodes and in response to the exchanged initial sets of calibration parameters which that directional sensor node received from other directional sensor nodes of the plurality of directional sensor nodes.

2. The method of claim 1, wherein the steps of estimating the initial set of calibration parameters and estimating the updated set of calibration parameters are performed in a distributed manner by each of the plurality of directional sensor nodes.

3. The method of claim 1, wherein sensing the calibration object with the plurality of directional sensor nodes comprises simultaneously observing a calibration object having at least two distinctive features with a plurality of camera nodes.

4. The method of claim 1, wherein exchanging measurement data regarding the calibration object comprises each of the plurality of directional sensor nodes broadcasting measurement data to and receiving measurement data from all of its one-hop communication neighbors.

5. The method of claim 1, wherein exchanging measurement data regarding the calibration object comprises forming a dynamic cluster of directional sensor nodes and aggregating measurement data at a cluster head of the dynamic cluster.

6. The method of claim 5, wherein the cluster head performs the step of estimating an initial set of calibration parameters for each of the plurality of directional sensor nodes in the dynamic cluster after the cluster head leaves the dynamic cluster.

7. The method of claim 1, wherein estimating the initial set of calibration parameters for each of the plurality of directional sensor nodes comprises determining a relative position vector and a corresponding covariance between each pair of directional sensor nodes having eight or more corresponding data points regarding the calibration object among the measurement data.

8. The method of claim 1, wherein exchanging the initial sets of calibration parameters comprises each of the plurality of directional sensor nodes broadcasting its set of calibration parameters to and receiving a set of calibration parameters from all of its one-hop communication neighbors.

9. The method of claim 1, wherein estimating the updated set of calibration parameters for each of the plurality of directional sensor nodes comprises refining the initial set of calibration parameters for each of the plurality of directional sensor nodes using recursive least squares.

10. The method of claim 1, further comprising adjusting the updated set of calibration parameters for each of the plurality of directional sensor nodes to conform to a global coordinate frame.

11. The method of claim 10, wherein adjusting the updated set of calibration parameters for each of the plurality of directional sensor nodes comprises dynamically aligning local coordinate frames of the plurality of directional sensor nodes to one another.

12. The method of claim 1, further comprising refining the updated sets of calibration parameters using a bundle adjustment scheme.

13. The method of claim 1, further comprising performing the steps of exchanging the initial sets of calibration parameters and estimating an updated set of calibration parameters recursively until the respective directional sensor node achieves a predetermined level of calibration accuracy.

14. The method of claim 13, wherein the respective directional sensor node achieves a predetermined level of calibration accuracy when the difference between two consecutively estimated sets of calibration parameters is less than a predetermined value.

15. The method of claim 13, further comprising reporting to an operator that the respective directional sensor node has achieved a predetermined level of calibration accuracy.

16. The method of claim 15, further comprising placing the calibration object in a viewing area of a directional sensor node that has not yet achieved the predetermined level of calibration accuracy.

17. One or more tangible, non-transitory machine-readable media comprising a plurality of instructions that, in response to being executed, result in a plurality of directional sensor nodes:

sensing a calibration object;

exchanging measurement data regarding the calibration object between the plurality of directional sensor nodes;

estimating an initial set of calibration parameters in each of the plurality of directional sensor nodes, each of the plurality of directional sensor nodes estimating its initial set of calibration parameters in response to the measurement data which that directional sensor node sent to other directional sensor nodes of the plurality of directional sensor nodes and in response to the exchanged measurement data which that directional sensor node received from other directional sensor nodes of the plurality of directional sensor nodes;

exchanging the initial sets of calibration parameters between the plurality of directional sensor nodes; and

estimating an updated set of calibration parameters in each of the plurality of directional sensor nodes, each of the plurality of directional sensor nodes estimating its updated set of calibration parameters in response to the initial set of calibration parameters which that directional sensor node sent to other directional sensor nodes of the plurality of directional sensor nodes and in response to the exchanged initial sets of calibration parameters which that directional sensor node received from other directional sensor nodes of the plurality of directional sensor nodes.

18. The one or more tangible, non-transitory machine-readable media of claim 17, wherein the steps of estimating the initial set of calibration parameters and estimating the updated set of calibration parameters are performed in a distributed manner by each of the plurality of directional sensor nodes.

19. The one or more tangible, non-transitory machine-readable media of claim 17, wherein the plurality of instructions, in response to being executed, result in a plurality of camera nodes observing a calibration object having at least two distinctive features.

20. The one or more tangible, non-transitory machine-readable media of claim 17, wherein exchanging measurement data regarding the calibration object comprises each of the plurality of directional sensor nodes broadcasting measurement data to and receiving measurement data from all of its one-hop communication neighbors.

21. The one or more tangible, non-transitory machine-readable media of claim 17, wherein exchanging measurement data regarding the calibration object comprises forming a dynamic cluster of directional sensor nodes and aggregating measurement data at a cluster head of the dynamic cluster.

22. The one or more tangible, non-transitory machine-readable media of claim 21, wherein the cluster head performs the step of estimating an initial set of calibration parameters for each of the plurality of directional sensor nodes in the dynamic cluster after the cluster head leaves the dynamic cluster.

23. The one or more tangible, non-transitory machine-readable media of claim 17, wherein estimating the initial set of calibration parameters for each of the plurality of directional sensor nodes comprises determining a relative position vector and a corresponding covariance between each pair of directional sensor nodes having eight or more corresponding data points regarding the calibration object among the measurement data.

24. The one or more tangible, non-transitory machine-readable media of claim 17, wherein exchanging the initial sets of calibration parameters comprises each of the plurality of directional sensor nodes broadcasting its set of calibration parameters to and receiving a set of calibration parameters from all of its one-hop communication neighbors.

25. The one or more tangible, non-transitory machine-readable media of claim 17, wherein estimating the updated set of calibration parameters for each of the plurality of directional sensor nodes comprises refining the initial set of calibration parameters for each of the plurality of directional sensor nodes using recursive least squares.

26. The one or more tangible, non-transitory machine-readable media of claim 17, wherein the plurality of instructions, in response to being executed, further result in the plurality of directional sensor nodes adjusting the updated set of calibration parameters for each of the plurality of directional sensor nodes to conform to a global coordinate frame.

27. The one or more tangible, non-transitory machine-readable media of claim 26, wherein adjusting the updated set of calibration parameters for each of the plurality of directional sensor nodes comprises dynamically aligning local coordinate frames of the plurality of directional sensor nodes to one another.

28. The one or more tangible, non-transitory machine-readable media of claim 17, wherein the plurality of instructions, in response to being executed, further result in the plurality of directional sensor nodes refining the updated sets of calibration parameters using a bundle adjustment scheme.

29. The one or more tangible, non-transitory machine-readable media of claim 17, wherein the plurality of instructions, in response to being executed, further result in the plurality of directional sensor nodes performing the steps of exchanging the initial sets of calibration parameters and estimating an updated set of calibration parameters recursively until the respective directional sensor node achieves a predetermined level of calibration accuracy.

30. The one or more tangible, non-transitory machine-readable media of claim 29, wherein the respective directional sensor node achieves a predetermined level of calibration accuracy when the difference between two consecutively estimated sets of calibration parameters is less than a predetermined value.

31. The one or more tangible, non-transitory machine-readable media of claim 29, wherein the plurality of instructions, in response to being executed, further result in the plurality of directional sensor nodes reporting to an operator that the respective directional sensor node has achieved a predetermined level of calibration accuracy.

32. The one or more tangible, non-transitory machine-readable media of claim 31, wherein the plurality of instructions, in response to being executed, further result in the plurality of directional sensor nodes placing the calibration object in a viewing area of a directional sensor node that has not yet achieved the predetermined level of calibration accuracy.