System for three-dimensional image capture while moving

• US 10,142,615 B2
• Filed: 12/16/2015
• Issued: 11/27/2018
• Est. Priority Date: 12/19/2014
• Status: Active Grant

• Current Assignee: Stereolabs
• Original Assignee: Stereolabs
• Inventors: Schmollgruber, Cecile, Azzam, Edwin Michel, Braun, Olivier Antoine, Beudet, Marc
• Primary Examiner(s): Lee, Michael
• Application Number:
  US15/536,762
• Publication Number:
  US 20170332068A1
• Time in Patent Office: 1,077 Days
• Field of Search: 348194, 348113, 348154, 348155, 348159, 348169, 348 40, 348 42, 348 44, 348 47, 348 48, 348 49, 348 50, 348 51, 3484254, 3484281, 348464, 348495, 348500, 348513, 348516, 348521, 348525, 348540, 348608, 348636, 348680, 348682, 348693, 348723, 348 46, 345 6, 345419, 345652, 345663, 345678, 382103, 382154, 382173, 382275
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0085 Motion estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/684 performed by controlling the image sensor readout, e.g. by controlling the integration time

Abstract

A system for three-dimensional image capture while moving. The system includes: an image capture device including at least two digital image sensors placed to capture a stereoscopic image; a processor for obtaining disparity information, associated with the digital images obtained by the image capture device and movement speed of elements in the digital images; a transmitter for sending the digital images to enable control of movement based on the digital images and the associated disparity information; and a controller for controlling the image capture device.

Claims

1. A system mounted on an object capable of movement for capturing three-dimensional images from the object, wherein the system comprises:

an image capture device including at least two digital image sensors, synchronized with each other, producing digital images, and capturing a stereoscopic image,

a processor obtaining disparity information associated with the digital images captured by the image capture device and movement speed of elements in the digital images,

a transmission connector controlling movement of the object, based on the digital images and the disparity information associated with the digital images, and

a first controller commanding the image capture device, by modifying, to facilitate controlling the movement of the object, dimensions of fields of view of the digital image sensors, and frequency of stereoscope image capture, taking into account maximum throughput tolerated by the transmission connector and the movement speed of elements in the digital images, wherein,

if the elements in the digital images are moving relatively quickly, the frequency of stereoscopic image capture is increased and the field of view is retracted, and,

if the elements in the digital images are moving relatively slowly, the frequency of stereoscopic image capture is decreased and the field of view is expanded.

2. The system mounted on an object capable of movement for capturing three-dimensional images from the object according to claim 1, wherein the movement speed of elements in the digital images is assessed by a contrast gradient calculation on the digital images.

3. The system mounted on an object capable of movement for capturing three-dimensional images from the object according to claim 1, wherein the transmission connector comprises one of a USB 3.0 connection, a Giga Ethernet connection, and a Thunderbolt connection.

4. The system mounted on an object capable of movement for capturing three-dimensional images from the object according to claim 1, wherein the processor comprises a computer program to be run on a microcomputer.

5. The system mounted on an object capable of movement for capturing three-dimensional images from the object according to claim 1, wherein the processing means processor comprises a chipset on a printed circuit substrate.

6. The system mounted on an object capable of movement for capturing three-dimensional images from the object according to claim 1, wherein the dimensions of the fields of view of the digital image sensors and the frequency of stereoscopic image capture can be set by a user.

7. The system mounted on an object capable of movement for capturing three-dimensional images from the object according to claim 1, wherein the two digital image sensors are master and slave, respectively, for synchronization purposes.

8. The system mounted on an object capable of movement for capturing three-dimensional images from the object according to claim 1, wherein

the image capture device comprises a digital image processing controller and a printed circuit substrate, and

the controller and the two digital image sensors are located on a printed circuit substrate of a shared substrate.

9. The system mounted on an object capable of movement for capturing three-dimensional images from the object according claim 1, wherein the transmission connector comprises a wire for transmitting the digital images between the image capture device and the processor.

10. The system mounted on an object capable of movement for capturing three-dimensional images from the object according to claim 1, further comprising a second controller controlling the movement of the object, wherein the transmission connector comprises a wire for transmitting the digital images and the disparity information associated with the digital images to second controller.

11. An autonomously navigating vehicle comprising:

an object capable of moving; and

a system mounted on the object and capturing three-dimensional images from the object, wherein

the system comprises

an image capture device including at least two digital image sensors, synchronized with each other, producing digital images, and capturing a stereoscopic image,

a processor obtaining disparity information associated with the digital images captured by the image capture device and movement speed of elements in the digital images,

a transmission connector controlling movement of the object, based on the digital images and the disparity information associated with the digital images, and

a first controller commanding the image capture device, by modifying, to facilitate controlling the movement of the object, dimensions of fields of view of the digital image sensors, and frequency of stereoscope image capture, taking into account maximum throughput tolerated by the transmission connector and the movement speed of elements in the digital images, wherein,

if the elements in the digital images are moving relatively quickly, the frequency of stereoscopic image capture is increased and the field of view is retracted, and,

if the elements in the digital images are moving relatively slowly, the frequency of stereoscopic image capture is decreased and the field of view is expanded, and

the object comprises an autonomous navigator sending a human operator or an electronic module the disparity information associated with the digital images produced by the system and controlling the movement of the object.

12. The autonomously navigating vehicle according to claim 11, wherein the autonomous navigator moves the object in autonomous navigation using the disparity information associated with the digital images.

13. A method for controlling movement of an object including an on-board system for three-dimensional image capture, the method comprising:

capturing digital images using at least two digital image sensors synchronized with each other and capturing a stereoscopic image,

obtaining movement speed of elements in the digital images,

obtaining disparity information associated with pairs of synchronized digital images,

controlling movement of the object based on the digital images and the disparity information associated with the digital images,

modifying, to facilitate control of the movement of the object, dimensions of fields of view of the sensors and frequency of capture of the digital images, taking into account maximum throughput tolerated and the movement speed of elements in the digital images, wherein,

if the elements in the digital images are moving relatively quickly, the frequency of stereoscopic image capture is increased, and the field of view is retracted, and,

if the elements in the digital images are moving relatively slowly, the frequency of stereoscopic image capture is decreased, and the field of view is expanded,

sending the digital images and the disparity information associated with the digital images to a navigation unit, and

controlling the movement of the object, based on the digital images and the disparity information associated with the pairs of synchronized digital images, with the navigation unit.

14. The method for controlling movement of the object according to claim 13, wherein the navigation unit is an autonomous navigation unit.