Dimensioning system calibration systems and methods

• US 9,292,969 B2
• Filed: 04/06/2015
• Issued: 03/22/2016
• Est. Priority Date: 05/07/2012
• Status: Active Grant

• Current Assignee: Intermec IP Corporation (Honeywell International Inc.)
• Original Assignee: Intermec IP Corporation (Honeywell International Inc.)
• Inventors: Laffargue, Franck, Thebault, Patrice, Hunt, Jeffrey Mark
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): Chin, Michelle
• Application Number:
  US14/679,275
• Publication Number:
  US 20150213647A1
• Time in Patent Office: 351 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G01B 11/00 Measuring arrangements characterised by the use of optical techniques
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting

Abstract

Systems and methods of determining the volume and dimensions of a three-dimensional object using a dimensioning system are provided. The dimensioning system can include an image sensor, a non-transitory, machine-readable, storage, and a processor. The dimensioning system can select and fit a three-dimensional packaging wireframe model about each three-dimensional object located within a first point of view of the image sensor. Calibration is performed to calibrate between image sensors of the dimensioning system and those of the imaging system. Calibration may occur pre-run time, in a calibration mode or period. Calibration may occur during a routine. Calibration may be automatically triggered on detection of a coupling between the dimensioning and the imaging systems.

Claims

1. A dimensioning system device selectively couplable to an imaging system device that has at least one imaging system image sensor and at least one imaging system display coupled to present display images acquired by the at least one imaging system image sensor, the dimensioning system device comprising:

at least one dimensioning system sensor, where;

in a calibration mode the dimensioning system device;

captures a depth map;

captures an intensity image;

receives imaging system image data display image from the imaging system, the imaging system image data representative of an image captured via the at least one imaging sensor; and

determines a number of extrinsic parameters that provide a correspondence between a three-dimensional point in each of the depth map and the intensity image as viewed by the dimensioning system sensor with a same three-dimensional point viewed by the at least one imaging system image sensor of the imaging system device, including;

determines a number of points positioned on at least one feature in the depth map and the intensity image including;

determines a minimum of at least eight points positioned on at least one feature in the depth map and the intensity image, the at least one feature including an item other than a calibration target; and

determines an equal number of respective points similarly positioned on the at least one feature in the imaging system image data.

2. The dimensioning system device of claim 1, wherein in a run-time mode the dimensioning system device:

generates a packaging wireframe image to be presented by the at least one imaging system display of the imaging system positioned to encompass an object in a display of the image represented by the imaging system image data concurrently presented with the package wireframe image by the at least one imaging system display as correlated spatially therewith based at least in part on the determined extrinsic parameters.

3. The dimensioning system device of claim 1, wherein the dimensioning system device selectively couples to the imaging system device wirelessly.

4. The dimensioning system device of claim 1, wherein responsive to the receipt of at least one input when in run-time mode, the dimensioning system device:

captures a second depth map;

captures a second intensity image;

receives a second display image from the imaging system; and

determines a number of extrinsic parameters that provide a correspondence between a three-dimensional point in each of the second depth map and the second intensity image as viewed by the dimensioning system sensor with a same three-dimensional point viewed by the at least one imaging system image sensor of the imaging system device.

5. The dimensioning system device of claim 1, comprising a timer configured to provide an output at one or more intervals, and wherein responsive to the at least one output by the timer, the dimensioning system device:

captures a second depth map;

captures a second intensity image;

receives a second display image from the imaging system; and

determines a number of extrinsic parameters that provide a correspondence between a three-dimensional point in each of the second depth map and the second intensity image as viewed by the dimensioning system sensor with a same three-dimensional point viewed by the at least one imaging system image sensor of the imaging system device.

6. The dimensioning system device of claim 1, comprising a depth lighting system to selectively illuminate an area within a field of view of the dimensioning system sensor, and wherein the dimensioning system device selectively illuminates the area contemporaneous with the capture of the depth map and the capture of the intensity image by the dimensioning system sensor.

7. The dimensioning system device of claim 6, wherein the depth lighting system provides a structured light pattern and/or a modulated light pattern.

8. The dimensioning system device of claim 6, comprising:

an automatic exposure control system, and wherein the automatic exposure control system adjusts an exposure of at least one of the depth map and the intensity image contemporaneous with the selective illumination of the area within the field of view of the dimensioning system sensor; and

/or

an automatic gain control system, and wherein the automatic gain control system adjusts a gain of at least one of the depth map and the intensity image contemporaneous with the selective illumination of the area within the field of view of the dimensioning system sensor.

9. The dimensioning system device of claim 1, wherein in response to an input when in calibration mode, the dimensioning system device establishes a correspondence between a three-dimensional point in each of the depth map and the intensity image as viewed by the dimensioning system sensor with a same three-dimensional point viewed by the at least one imaging system image sensor of the imaging system device.

10. A method of operation of a dimensioning system device, the method comprising:

detecting a selective coupling of an imaging system device to the dimensioning system device, the imaging system including at least one imaging system image sensor and at least one imaging system display;

capturing by the dimensioning system device a depth map via the dimensioning system sensor;

capturing by the dimensioning system device an intensity image via the dimensioning system sensor;

receiving by the dimensioning system device imaging system image data, the imaging system image data representing an image captured by the at least one imaging system image sensor; and

determining by the dimensioning system device a number of extrinsic parameters that provide a correspondence between a point in the depth map and intensity image as viewed by the dimensioning system sensor with a same point as viewed by the at least one imaging system image sensor of the imaging system device including;

determining by the dimensioning system device a minimum of at least eight points positioned on at least one feature in the depth map and the intensity image, the at least one feature including an item other than a calibration target; and

determining by the dimensioning system device an equal number of respective points similarly positioned on the at least one feature in the imaging system image data.

11. The method of claim 10, wherein the dimensioning system device selectively couples to the imaging system device wirelessly.

12. The method of claim 10, comprising generating by the dimensioning system processor a packaging wireframe model for presentation by the at least one imaging system display of the imaging system, the packaging wireframe model positioned to encompass an object in the imaging system image data concurrently presented by the at least one imaging system display and spatially correlated therewith based at least in part on the determined extrinsic parameters.

13. The method of claim 10, wherein:

the imaging system image sensor includes a known back focus value (f_c); and

determining by the dimensioning system processor a number of extrinsic parameters that provide a correspondence between a point in the depth map and intensity image as viewed by the dimensioning system sensor with a same point as viewed by the at least one imaging system image sensor of the imaging system device includes for each of the minimum of at least eight points;

determining by the dimensioning system device a point coordinate location (x_i, y_i) within the intensity image;

determining by the dimensioning system device a point depth (z_i) within the depth map for the determined point coordinate location (x_i, y_i); and

determining by the dimensioning system device a respective point coordinate location (x_c, y_c) within the imaging system image data.

14. The method of claim 13, wherein determining by the dimensioning system processor a number of extrinsic parameters that provide a correspondence between a point in the depth map and intensity image as viewed by the dimensioning system sensor with a same point as viewed by the at least one imaging system image sensor of the imaging system device further includes:

determining via the dimensioning system device all of the components of a three-by-three, nine component, Rotation Matrix (R_11-33) and a three element Translation Vector (T_0-2) that relate the depth map and intensity image to the imaging system image data using the following equations;

15. The method of claim 10, wherein capturing by the dimensioning system device a depth map via the dimensioning system sensor includes selectively illuminating an area within a first field of view of the dimensioning system sensor with at least one of a structured light pattern or a modulated light pattern.

16. The method of claim 10, wherein determining by the dimensioning system processor a number of extrinsic parameters that provide a correspondence between a point in the depth map and intensity image as viewed by the dimensioning system sensor with a same point as viewed by the at least one imaging system image sensor of the imaging system device includes:

receiving by the dimensioning system device, an input identifying a point located on at least on feature in the depth map and the intensity image; and

receiving by the dimensioning system device, a second input identifying a point similarly located on the at least one feature in the imaging system image data.

17. A method of calibrating a dimensioning system device, the method comprising:

determining that the dimensioning system device is communicably couple to an imaging system including an imaging system display and an imaging system image sensor;

receiving by the dimensioning system device a depth map and an intensity image of a scene provided by a dimensioning system sensor in the dimensioning system device, the depth map and the intensity image including a common plurality of three-dimensional points;

receiving by the dimensioning system device an imaging system image data from the imaging system image sensor, the imaging system image data comprising a plurality of visible points, at least a portion of the plurality of visible points corresponding to at least a portion of the plurality of three-dimensional points;

determining via the dimensioning system device a number of points positioned on at least on feature appearing in the depth map and intensity image with a same number of points similarly positioned on the at least one feature appearing in the imaging system image data, the at least one feature not including a calibration target; and

determining by the dimensioning system device a number of extrinsic parameters spatially relating each of the number of points in the depth map and the intensity image with each of the respective, same number of points in the imaging system image data.

18. The method of claim 17, comprising:

identifying, by the dimensioning system device, at least one three-dimensional object in the depth map and the intensity image;

generating by the dimensioning system device and based at least in part on the determined number of extrinsic parameters, a packaging wireframe for presentation on the imaging system display, the packaging wireframe substantially encompassing the at least one three-dimensional object;

generating by the dimensioning system device, a composite image signal including the three-dimensional wireframe substantially encompassing an image of the three-dimensional object appearing in the imaging system image data; and

displaying the composite image signal on the imaging system display.

19. The method of claim 17, wherein:

the imaging system image sensor includes a back focus value (f_c); and

determining a number of extrinsic parameters spatially relating each of the number of points in the depth map and the intensity image with each of the respective, same number of points in the imaging system image data includes;

determining by the dimensioning system device at least eight points located on at least one feature in the depth map and the intensity image; and

determining by the dimensioning system device an equal number of respective points similarly located on the at least one feature in the imaging system image data;

determining by the dimensioning system device for each of the at least eight points a point coordinate location (x_i, y_i) within the intensity image;

determining by the dimensioning system device for each of the at least eight points a point depth (z_i) within the depth map for the determined point coordinate location (x_i, y_i); and

determining by the dimensioning system device a respective point coordinate location for each of the at least eight points (x_c, y_c) within the imaging system image data.

20. The method of claim 19, wherein determining a number of extrinsic parameters spatially relating each of the number of points in the depth map and the intensity image with each of the respective, same number of points in the imaging system image data includes:

determining via the dimensioning system device all of the components of a three-by-three, nine component, Rotation Matrix (R_11-33) and a three element Translation Vector (T_0-2) that relate the depth map and intensity image to the imaging system image data using the following equations;