Calibrating depth cameras using natural objects with expected shapes

• US 10,154,176 B1
• Filed: 05/30/2017
• Issued: 12/11/2018
• Est. Priority Date: 05/30/2017
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Hicks, Richmond F., Kumar, Avinash
• Primary Examiner(s): Laballe, Clayton E
• Assistant Examiner(s): Fenwick, Warren K
• Application Number:
  US15/608,200
• Publication Number:
  US 20180352119A1
• Time in Patent Office: 560 Days
• Field of Search: 396104
• CPC Class Codes: G01B 11/22 for measuring depth
• CPC Class Codes: G01B 11/2504 Calibration devices
• CPC Class Codes: G01B 21/042 Calibration or calibration artifacts G01B3/30, G01B9/02072 take precedence
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 2201/07 Target detection
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 5/2226 Determination of depth image, e.g. for foreground/background separation determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10

Abstract

An example apparatus for calibrating depth cameras includes an image receiver to receive an image from a depth camera. The apparatus also includes an object detector to detect a natural object with an expected shape in the image. The apparatus further includes a parameter adjuster to adjust a parameter of the depth camera to reduce a detected difference between the expected shape and a measured characteristic of the natural object.

Claims

1. An apparatus for calibrating depth cameras, comprising:

an image receiver to receive an image from a depth camera;

an object detector to detect a natural object with an expected shape in the image; and

a parameter adjuster to adjust a parameter of the depth camera to reduce a detected difference between the expected shape and a measured characteristic of the natural object.

2. The apparatus of claim 1, comprising:

a pose estimator to estimate a pose of the natural object based on the expected shape;

a gradient generator to generate a disparity-based depth gradient and a homography-based depth gradient;

a gradient comparer to detect a difference between the disparity-based depth gradient and the homography-based depth gradient exceeds a predetermined threshold, wherein the parameter adjuster is to adjust a parameter to reduce the difference between the disparity-based depth gradient and the homography-based depth gradient below the predetermined threshold.

3. The apparatus of claim 2, comprising an edge segmenter to segment edges of the expected shape of the natural object.

4. The apparatus of claim 3, wherein the gradient generator is to generate the disparity-based depth gradient based on a disparity along segmented edges calculated between two stereo images.

5. The apparatus of claim 3, wherein the gradient generator is to generate the homography-based depth gradient using multi-view geometric projections to calculate a percentage change in depth along the segmented edges.

6. The apparatus of claim 1, wherein the parameter adjuster is to limit modifications of one or more depth camera parameters based on expected system changes.

7. The apparatus of claim 1, wherein the expected shape comprises a polygonal shape.

8. The apparatus of claim 1, wherein the expected shape comprises a planar geometric shape to be used to compute a homography.

9. The apparatus of claim 1, wherein the parameter comprises a parameter corresponding to alignment of the depth camera, wherein the depth camera comprises a stereo camera.

10. The apparatus of claim 1, wherein the apparatus comprises a node that provides a calibration service over a network.

11. A method for calibrating depth cameras, comprising:

receiving, via a processor, an image from a depth camera;

detecting, via the processor, a natural object with an expected shape in the image; and

adjusting, via the processor, a parameter to reduce a difference between the expected shape and a measured characteristic of the natural object.

12. The method of claim 11, comprising:

estimating, via the processor, a pose of the natural object based on the expected shape;

generating, via the processor, a disparity-based depth gradient and a homography-based depth gradient; and

detecting, via the processor, a difference between the disparity-based depth gradient and the homography-based depth gradient exceeds a predetermined threshold, wherein adjusting the parameter comprises adjusting to reduce the difference between the disparity-based depth gradient and the homography-based depth gradient below the predetermined threshold.

13. The method of claim 12, comprising segmenting, via the processor, edges of the expected shape of the natural object, wherein generating the disparity-based depth gradient comprises calculating a disparity along segmented edges between two stereo images comprising the image and converting the disparity to depth based on calibration parameters of the depth camera.

14. The method of claim 13, wherein generating the homography-based depth gradient comprises calculating a percentage change in depth along the segmented edges.

15. The method of claim 11, comprising limiting modifications of the parameter based on an expected system change.

16. The method of claim 11, comprising qualifying, via the processor, the natural object by comparing the estimated pose of the natural object to an expected pose for the natural object and disregarding natural objects with unnatural poses.

17. The method of claim 11, comprising applying, via the processor, a constraint that a detected edge of the natural object is straight.

18. The method of claim 11, comprising selecting a band on an image region at the boundary of the natural object and performing edge detection on the band.

19. The method of claim 11, comprising detecting, via the processor, the expected shape based on a template.

20. The method of claim 11, comprising computing a homograph based on the expected shape.

21. At least one non-transitory computer readable medium for calibrating depth cameras having instructions stored therein that, in response to being executed on a computing device, cause the computing device to:

receive an image from a depth camera;

detect a natural object with an expected shape in the image;

estimate a pose of the natural object based on the expected shape;

generate a disparity-based depth gradient and a homography-based depth gradient;

detect a difference between the disparity-based depth gradient and the homography-based depth gradient exceeds a predetermined threshold; and

adjust a parameter to reduce the difference between the disparity-based depth gradient and the homography-based depth gradient below the predetermined threshold.

22. The at least one non-transitory computer readable medium of claim 21, comprising instructions to segment edges of the expected shape of the natural object and generate the disparity-based depth gradient based on a disparity along segmented edges calculated between two stereo images.

23. The at least one non-transitory computer readable medium of claim 21, comprising instructions to generate the homography-based depth gradient using multi-view geometric projections to calculate a percentage change in depth along the segmented edges.

24. The at least one non-transitory computer readable medium of claim 21, comprising instructions to detect the natural object based on an object template.

25. The at least one non-transitory computer readable medium of claim 21, comprising instructions to limit modification of one or more depth camera parameters based on an expected system change.