Dual-mode data capture system for collision detection and object dimensioning

• US 10,611,372 B2
• Filed: 09/06/2018
• Issued: 04/07/2020
• Est. Priority Date: 09/06/2018
• Status: Active Grant

• Current Assignee: Zebra Technologies Corporation
• Original Assignee: Zebra Technologies Corporation
• Inventors: Swope, Charles Burton
• Primary Examiner(s): Ghulamali, Qutbuddin
• Application Number:
  US16/123,594
• Publication Number:
  US 20200079370A1
• Time in Patent Office: 579 Days
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2420/54 Audio sensitive means, e.g. ultrasound
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/596 from three or more stereo images
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume

Abstract

A dual-mode data capture system includes a capture controller, a point cloud generator, a collision detector, a plurality of cameras viewing a capture volume, and a motion sensor to generate a detection signal when an object arrives at a capture position within the volume. The controller: activates a subset of cameras in a collision detection mode to capture sequences of images of the volume; responsive to receiving the detection signal, activates the cameras in a dimensioning mode to capture a synchronous set of images of the capture position. The collision detector: determines whether the sequences of images indicate a potential collision; and responsive to detection of a potential collision, generates a warning. The point cloud generator: receives the synchronous set of images and generates a point cloud representing the object based on the synchronous set of images, for use in determining dimensions of the object.

Claims

1. A dual-mode data capture system, comprising:

a plurality of cameras disposed to view a capture volume;

a motion sensor configured to generate a detection signal responsive to detecting an object at a capture position within the capture volume;

a capture controller connected to the motion sensor and configured to;

activate at least a subset of cameras in a collision detection mode causing the subset of cameras to capture respective sequences of images of the capture volume, the subset of cameras being a subset of the plurality of cameras;

responsive to receiving the detection signal, activate the plurality of cameras in a dimensioning mode causing each camera to capture a respective synchronous set of images of the capture position;

a collision detector connected to at least the subset of cameras and configured to;

receive the respective sequences of images;

determine whether the respective sequences of images from at least the subset of the plurality of cameras indicate a potential collision of one or more objects in the capture volume; and

responsive to detection of the potential collision, generate a warning; and

a point cloud generator connected to each of the plurality of cameras and configured to receive the synchronous set of images of the capture position from the plurality of cameras and generate a point cloud representing the object based on the synchronous set of images of the capture position from the plurality of cameras, wherein dimensions of the object are determined based on the point cloud.

2. The dual-mode data capture system of claim 1, wherein the capture controller is further configured, following activation of the cameras in the dimensioning mode, to return the subset of cameras to the collision detection mode.

3. The dual-mode data capture system of claim 2, wherein the collision detection mode is activated for a first time period.

4. The dual-mode data capture system of claim 3, wherein the dimensioning mode is activated for a second time period, the second time period being shorter than the first time period.

5. The dual-mode data capture system of claim 1, wherein the capture controller is further configured, for activating the cameras in the dimensioning mode, to:

generate and transmit a shutter command substantially simultaneously to each of a plurality of cameras that causes each camera to capture a respective synchronous set of images of the capture volume.

6. The dual-mode data capture system of claim 1, wherein the point cloud generator is configured to receive the synchronous set of images of the capture position from the plurality of cameras, and to store the synchronous set of images in a first repository.

7. The dual-mode data capture system of claim 6, wherein the collision detector is configured to receive the respective sequences of images from the subset of cameras, and to store the respective sequences of images in a second repository.

8. The dual-mode data capture system of claim 1, further comprising an output device for generating a warning signal;

wherein the collision detector is configured to generate the warning by transmitting a command to the output device.

9. The dual-mode data capture system of claim 1, wherein the collision detector is configured to determine whether the respective sequences of images indicate a potential collision by:

identifying a first object and a second object in the sequences of images; and

determining a first movement vector corresponding to the first object, and a second movement vector corresponding to the second object.

10. The dual-mode data capture system of claim 1, wherein the collision detector includes a collision detection server connected to the subset of cameras;

and

wherein the point cloud generator includes an image processing server connected to each of the plurality of cameras.

11. The dual-mode data capture system of claim 1, further comprising a server executing (i) a collision detection application to implement the collision detector, and (ii) a point cloud generation application to implement the point cloud generator.

12. The dual-mode data capture system of claim 1, wherein the capture controller is configured to activate the subset of the cameras in the collision detection mode by transmitting a command to each of the subset of cameras to switch from a single-frame capture mode to a continuous capture mode.

13. A method of dual-mode data capture, comprising:

at a capture controller connected to a plurality of cameras disposed to view a capture volume, activate at least a subset of cameras in a collision detection mode causing the subset of cameras to capture respective sequences of images of the capture volume, the subset of cameras being a subset of the plurality of cameras;

at a collision detector connected to at least the subset of cameras, receiving the respective sequences of images and determining whether the respective sequences of images from at least the subset of the plurality of cameras indicate a potential collision of one or more objects in the capture volume;

responsive to detection of the potential collision at the collision detector, generating a warning;

at a motion sensor connected to the capture controller, generating a detection signal responsive to detecting an object at a capture position within the capture volume;

at the capture controller, responsive to receiving the detection signal, activating the plurality of cameras in a dimensioning mode causing each camera to capture a respective synchronous set of images of the capture position; and

at a point cloud generator connected to each of the plurality of cameras, receiving the synchronous set of images of the capture position from the plurality of cameras and generating a point cloud representing the object based on the synchronous set of images of the capture position from the plurality of cameras, wherein dimensions of the object are determined based on the point cloud.

14. The method of claim 13, further comprising at the capture controller, following activation of at least the subset of the cameras in the dimensioning mode, returning at least the subset of the cameras to the collision detection mode.

15. The method of claim 14, further comprising activating the collision detection mode for a first time period.

16. The method of claim 15, further comprising activating the dimensioning mode for a second time period, the second time period being shorter than the first time period.

17. The method of claim 13, wherein activating the cameras in the dimensioning mode further comprises:

generating and transmitting a shutter command substantially simultaneously to each of a plurality of cameras that causes each camera to capture a respective synchronous set of images of the capture volume.

18. The method of claim 13, further comprising, at the point cloud generator:

receiving the synchronous set of images of the capture position from the plurality of cameras, and storing the synchronous set of images in a first repository.

19. The method of claim 18, further comprising, at the collision detector, receiving the respective sequences of images from the subset of cameras, and storing the respective sequences of images in a second repository.

20. The method of claim 13, wherein generating the warning comprises transmitting a command from the collision detector to an output device.

21. The method of claim 13, wherein determining whether the respective sequences of images indicate a potential collision comprises:

identifying a first object and a second object in the sequences of images; and

determining a first movement vector corresponding to the first object, and a second movement vector corresponding to the second object.

22. The method of claim 13, wherein activating the subset of the cameras in the collision detection mode comprises transmitting a command to each of the subset of cameras to switch from a single-frame capture mode to a continuous capture mode.

23. A non-transitory computer-readable medium storing a plurality of computer-readable instructions executable in a dual-mode data capture system, the instructions comprising:

at a capture controller connected to a plurality of cameras disposed to view a capture volume, activating at least a subset of cameras in a collision detection mode causing the subset of cameras to capture respective sequences of images of the capture volume, the subset of cameras being a subset of the plurality of cameras;

at a collision detector connected to at least the subset of cameras, receiving the respective sequences of images and determining whether the respective sequences of images from at least the subset of the plurality of cameras indicate a potential collision of one or more objects in the capture volume;

responsive to detection of the potential collision at the collision detector, generating a warning;

at a motion sensor connected to the capture controller, generating a detection signal responsive to detecting an object at a capture position within the capture volume;

at the capture controller, responsive to receiving the detection signal, activating the plurality of cameras in a dimensioning mode causing each camera to capture a respective synchronous set of images of the capture position; and

at a point cloud generator connected to each of the plurality of cameras, receiving the synchronous set of images of the capture position from the plurality of cameras and generating a point cloud representing the object based on the synchronous set of images of the capture position from the plurality of cameras, wherein dimensions of the object are determined based on the point cloud.