Device for acquiring depth image, calibrating method and measuring method therefor

• US 9,319,660 B2
• Filed: 04/26/2013
• Issued: 04/19/2016
• Est. Priority Date: 12/27/2012
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Liou, Tung-Fa, Luo, Wen-Shiou, Chen, Chia-Chen
• Primary Examiner(s): Lee, Michael
• Application Number:
  US13/871,000
• Publication Number:
  US 20140184751A1
• Time in Patent Office: 1,089 Days
• Field of Search: 348/47
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/571 from focus
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2213/001 Constructional or mechanical details

Abstract

A device for acquiring depth image, a calibrating method and a measuring method therefore are provided. The device includes at least one projecting device, at least one image sensing device, a mechanism device and a processing unit. The projecting device projects a projection pattern to a measured object. The image sensing device is controlled to adjust a focal length and focus position, and therefore sense real images. The mechanism device adjusts a location and/or a convergence angle of the image sensing device. The processing unit calibrates the at least one image sensing device and generates a three dimension (3D) measuring parameter set at a model focal length according to a plurality of image setting parameter reference sets corresponding to a model focal length and a plurality of default node distances, respectively, and then estimates a depth map or depth information of the measured object.

Claims

1. A depth image acquiring device, comprising:

at least one projecting device, projecting a projection pattern to a measured object;

a first and a second image sensing devices, receiving focal length adjusting signals to adjust focal lengths thereof, and respectively sensing the projection pattern projected to the measured object to generate a first real image and a second real image;

a mechanism device, coupled to the first and the second image sensing devices, and receiving lens adjusting signals to adjust locations and/or convergence angles of the first and the second image sensing devices;

a processing unit, coupled to the at least one projecting device, the first and the second image sensing devices and the mechanism device, providing the focal length adjusting signals and the lens adjusting signals to adjust the focal lengths, locations and/or convergence angles of the first and the second image sensing devices, and performing a depth operation on the first and the second real images to generate depth information,

wherein the processing unit calibrates and obtains a three-dimensional (3D) measuring parameter set of the first and the second image sensing devices at a model focal length according to a plurality of image setting parameter sets corresponding to the model focal length and a plurality of predetermined node distances corresponding to each of the image sensing devices, and accordingly performs a depth operation.

2. The depth image acquiring device as claimed in claim 1, wherein the processing unit respectively sets the focal lengths of the first and the second image sensing devices to the corresponding model focal length,

sets a plurality of node distances to serve as positions of a plurality of nodes,

sets the image setting parameter set corresponding to each of the nodes,

during calibration, the processing unit sets the first and the second image sensing devices to the respective model focal lengths, and sets states of the first and the second image sensing devices according to the image setting parameter set of each of the nodes, and performs calibration on each of the nodes to obtain the respective 3D measuring parameter sets of each of the nodes, wherein the 3D measuring parameter sets are used to perform 3D measuring calculation.

3. The depth image acquiring device as claimed in claim 2, wherein the image setting parameter sets or the 3D measuring parameter sets comprise baseline distances, convergence angles and/or focus positions of the first and the second image sensing devices.

4. The depth image acquiring device as claimed in claim 2, wherein the processing unit selects different focal lengths for the first and the second image sensing devices to serve as the model focal lengths.

5. The depth image acquiring device as claimed in claim 2, wherein the processing unit selects focal lengths within a zooming range of the first and the second image sensing devices to serve as the model focal lengths.

6. The depth image acquiring device as claimed in claim 1, wherein the processing unit receives a specified focal length and a specified measuring distance,

compares the specified measuring distance with the corresponding node distances to calculate the image setting parameter set and the 3D measuring parameter set of the first and the second image sensing devices at the specified focal length,

sets the first and the second image sensing devices according to the image setting parameter set of the specified focal length, wherein the first and the second image sensing devices respectively sense the projection pattern projected to the measured object, so as to generate the first and the second real images,

the processing unit calculates an image conversion equation according to the specified focal length and the model focal length, and

converts the first and the second real images according to the image conversion equation and calculates according to the 3D measuring parameter set of the specified focal length, so as to generate the depth information.

7. The depth image acquiring device as claimed in claim 1, wherein the depth information is a depth image, 3D image coordinates or a depth image distribution map.

8. The depth image acquiring device as claimed in claim 1, wherein the at least one projecting device serves as a virtual image sensing device, and an original image of the projection pattern is taken as a virtual image, wherein the processing unit performs the depth operation on the first and the second real images and the virtual image to generate the depth information.

9. The depth image acquiring device as claimed in claim 1, wherein the at least one projecting device comprises:

a light source, sending a projection beam; and

a projection pattern generation device, disposed on an optical path of the light source, and irradiated by the light source to generate the projection pattern.

10. The depth image acquiring device as claimed in claim 9, wherein the light source is a laser light source, an infrared light source, an ultraviolet light source, a visible light source or a light emitting diode light source.

11. The depth image acquiring device as claimed in claim 9, wherein the projection pattern generation device is a diffractive optical device or a frosted glass device.

12. The depth image acquiring device as claimed in claim 9, wherein the at least one projecting device further comprising:

a projecting lens, disposed on an optical path of the projection pattern generation device, and projecting the projection pattern generated by the light source through the projection pattern generation device to the measured object.

13. The depth image acquiring device as claimed in claim 12, wherein the projecting lens is a zoom lens or a fixed focal length lens.

14. The depth image acquiring device as claimed in claim 13, wherein when the projecting lens is the zoom lens, the processing unit provides the focal lens adjusting signals to synchronously adjust focal lengths of the projecting lens and the first and the second image sensing devices.

15. The depth image acquiring device as claimed in claim 1, wherein the projection pattern is pre-designed or randomly generated pattern, or a randomly distributed stray light spot pattern.

16. The depth image acquiring device as claimed in claim 1, wherein the first and the first and the second image sensing devices respectively comprise:

a zoom lens, receiving the focal length adjusting signal to adjust a focal length thereof; and

an image sensor, sensing the projection pattern projected to the measured object through the zoom lens.

17. The depth image acquiring device as claimed in claim 16, wherein when the light source of the at least one projecting device is a light source with a specific wavelength, the first and the second image sensing devices respectively comprise:

an optical filter, disposed on an optical path of the zoom lens, wherein the optical filter matches the specific wavelength and is pervious to light beams with the specified wavelength and filters the other light beams.

18. The depth image acquiring device as claimed in claim 1, wherein the mechanism device comprises:

a first and second carrying members, respectively carrying the first and the second image sensing devices;

a first and second rotating components, respectively adjusting the convergence angles of the first and the second image sensing devices; and

a shifting member, connected to the first and the second carrying members to adjust a baseline distance between the first and the second image sensing devices.

19. The depth image acquiring device as claimed in claim 18, wherein the shifting member comprises:

a screw, comprising a first region and a second region having threads of a first direction and a second direction, wherein the first region and the second region are respectively connected to the first and the second carrying members, and when the screw rotates, the first and the second carrying members move relative to each other along the screw.

20. The depth image acquiring device as claimed in claim 18, wherein a rotation shaft of the first rotating component comprises a non-circular cross-section long groove accommodating space with an opening facing towards a rotation shaft of the second rotating component, and the rotation shaft of the second rotating component comprises a loose transmission member protruding corresponding to the non-circular cross-section long groove accommodating space, such that when the rotation shaft of the first rotating component rotates, the rotation shaft drives the rotation shaft of the second rotating component, so as to simultaneously adjust the convergence angles of the first and the second image sensing devices.

21. The depth image acquiring device as claimed in claim 18, wherein the shifting member comprises:

a first and second screws, respectively connected to the first and the second carrying members, wherein the first and the second carrying members respectively move along the first and the second screws.

22. A depth image acquiring device, comprises:

at least one projecting device, projecting a projection pattern to a measured object, wherein the at least one projecting device serves as a virtual image sensing device, and an original image of the projection pattern is taken as a virtual image;

at least one image sensing device, receiving a focal length adjusting signal to adjust a focal length thereof, and sensing the projection pattern projected to the measured object to generate at least one real image;

a mechanism device, coupled to the at least one image sensing device, and receives a lens adjusting signal to adjust a location and/or a convergence angle of the at least one image sensing device;

a processing unit, coupled to the at least one projecting device, the at least one image sensing device and the mechanism device, providing the focal length adjusting signal and the lens adjusting signal to adjust the focal length, the location and/or the convergence angle of the at least one image sensing device, and performing a depth operation on the at least one real image and the virtual image to generate depth information,

wherein the processing unit calibrates and obtains a three dimension (3D) measuring parameter set of the at least one image sensing device at a model focal length according to a plurality of image setting parameter sets corresponding to a model focal length and a plurality of predetermined node distances corresponding to each of the image sensing devices, and accordingly performs the depth operation.

23. The depth image acquiring device as claimed in claim 22, wherein the processing unit respectively sets the focal length of the at least one image sensing device to the corresponding model focal length,

sets a plurality of node distances to serve as positions of a plurality of nodes,

sets the image setting parameter set corresponding to each of the nodes,

during calibration, the processing unit sets the at least one image sensing device to the respective model focal lengths, and sets a state of the at least one image sensing device according to the image setting parameter set of each of the nodes, and performs calibration on each of the nodes to obtain the respective 3D measuring parameter sets of each of the nodes, wherein the 3D measuring parameter sets are used to perform 3D measuring calculation.

24. The depth image acquiring device as claimed in claim 23, wherein the image setting parameter set or the 3D measuring parameter set comprises baseline distances, convergence angles and/or focus positions of the at least one image sensing device and the at least one projecting device.

25. The depth image acquiring device as claimed in claim 23, wherein the processing unit selects different focal lengths for the at least one image sensing device to serve as the model focal lengths.

26. The depth image acquiring device as claimed in claim 23, wherein the processing unit selects focal lengths within a zooming range of the at least one image sensing device to serve as the model focal lengths.

27. The depth image acquiring device as claimed in claim 22, wherein the processing unit receives a specified focal length and a specified measuring distance,

compares the specified measuring distance with the corresponding predetermined node distances to calculate the image setting parameter set and the 3D measuring parameter set of the at least one image sensing device at the specified focal length,

sets the at least one image sensing device according to the image setting parameter set of the specified focal length, wherein the at least one image sensing device respectively senses the projection pattern projected to the measured object, so as to generate the at least one real image,

the processing unit calculates an image conversion equation according to the specified focal length and the model focal length, and

converts the at least one real image and the virtual image according to the image conversion equation and calculates according to the 3D measuring parameter set of the specified focal length, so as to generate the depth information.

28. A calibrating method of a depth image acquiring device, wherein the depth image acquiring device comprises at least one projecting device, at least one image sensing device and a mechanism device, the at least one projecting device projects a projection pattern to a measured object, wherein the at least one projecting device serves as a virtual image sensing device, the at least one image sensing device senses the projection pattern projected to the measured object to generate at least one real image, the mechanism device adjusts a location and/or a convergence angle of the at least one image sensing device, and the depth image acquiring device performs a depth operation on the at least one real image to generate depth information, the calibrating method of the depth image acquiring device comprising:

respectively setting the at least one image sensing device to a model focal length;

setting a plurality of node distances to serve as locations of a plurality of nodes;

setting an image setting parameter set corresponding to each of the nodes;

setting the at least one image sensing device to the respective focal length,

setting a state of the at least one image sensing device according to the image setting parameter set of each of the nodes; and

calibrating each of the nodes to obtain a 3D measuring parameter set of each of the nodes, wherein the 3D measuring parameter set is used to perform a 3D measuring calculation.

29. The calibrating method of the depth image acquiring device as claimed in claim 28, wherein the image setting parameter set or the 3D measuring parameter set comprises baseline distances, convergence angles and/or focus positions of the at least one image sensing devices.

30. The calibrating method of the depth image acquiring device as claimed in claim 28, wherein the model focal lengths are different according to a zooming range setting of the at least one image sensing device.

31. The calibrating method of the depth image acquiring device as claimed in claim 28, wherein each of the model focal lengths is a focal length within a zooming range of the at least one image sensing device.

32. The calibrating method of the depth image acquiring device as claimed in claim 28, wherein the 3D measuring calculation comprises:

receiving the specified focal length and the specified measuring distance;

comparing the specified measuring distance with the corresponding predetermined node distances to calculate the image setting parameter set and the 3D measuring parameter set of the at least one image sensing devices at the specified focal length;

setting a state of the at least one image sensing device according to the image setting parameter set of the specified focal length, wherein the at least one image sensing device respectively senses the projection pattern projected to the measured object, so as to generate the at least one real image;

calculating an image conversion equation according to the specified focal length and the model focal length; and

converting the at least one real image according to the image conversion equation and calculating according to the 3D measuring parameter set to generate the depth information.

33. A measuring method of a depth image acquiring device, wherein the depth image acquiring device comprises at least one projecting device, at least one image sensing device and a mechanism device, the at least one projecting device projects a projection pattern to a measured object, wherein the at least one projecting device serves as a virtual image sensing device, the at least one image sensing device senses the projection pattern projected to the measured object to generate at least one real image, the mechanism device adjusts a location and/or a convergence angle of the at least one image sensing device, and the depth image acquiring device performs a depth operation on the at least one real image to generate depth information, the measuring method of the depth image acquiring device comprising:

receiving a specific focal length and a specific measuring distance, wherein the at least one image sensing device is set to have a model focal length, a plurality of node distances corresponding to locations of a plurality of nodes, an image setting parameter set and a 3D measuring parameter set corresponding to each of the nodes;

comparing the specified measuring distance with the corresponding node distances to calculate the image setting parameter set and the 3D measuring parameter set of the at least one image sensing device at the specified focal length;

setting a state of the at least one image sensing device according to the image setting parameter set of the specified focal length, wherein the at least one image sensing device senses the projection pattern projected to the measured object to generate the at least one real image;

calculating an image conversion equation according to the specified focal length and the at least one model focal length; and

converting the at least one real image according to the image conversion equation, and performing calculation according to the 3D measuring parameter set of the specified focal length to generate the depth information.

34. The measuring method of the depth image acquiring device as claimed in claim 33, wherein the image conversion equation is:

35. The measuring method of the depth image acquiring device as claimed in claim 33, wherein when the at least one image sensing device does not cause image offset and distortion, the image conversion equation is: