Method, system and apparatus for correcting translucency artifacts in data representing a support structure

• US 10,740,911 B2
• Filed: 04/05/2018
• Issued: 08/11/2020
• Est. Priority Date: 04/05/2018
• Status: Active Grant

• Current Assignee: Symbol Technologies, LLC (Zebra Technologies Corporation)
• Original Assignee: Symbol Technologies, LLC (Zebra Technologies Corporation)
• Inventors: Phan, Raymond, Rzeszutek, Richard Jeffrey, Segall, Iaacov Coby
• Primary Examiner(s): Johns, Andrew W
• Application Number:
  US15/946,165
• Publication Number:
  US 20190311486A1
• Time in Patent Office: 859 Days
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/68 Food, e.g. fruit or vegetables

Abstract

A method in an imaging controller of correcting translucency artifacts in data representing one or more objects disposed on a shelf includes: obtaining a plurality of depth measurements captured by a depth sensor and corresponding to an area containing the shelf; obtaining (i) a definition of a plane containing edges of the shelf, (ii) a location in the plane of an upper shelf edge, and (iii) a location in the plane of a lower shelf edge adjacent to the upper shelf edge; generating a depth map containing, for each of a plurality of positions in the plane, a nearest object depth; detecting an upper object boundary in the depth map between the upper and lower support surface edges; updating each nearest object depth between the upper object boundary and the lower shelf edge to contain a depth of the upper object boundary; and storing the corrected depth map.

Claims

1. A method in an imaging controller of correcting translucency artifacts in data representing one or more objects disposed on a shelf, comprising:

obtaining a plurality of depth measurements captured by a depth sensor and corresponding to an area containing the shelf;

obtaining (i) a definition of a plane containing edges of the shelf, (ii) a location in the plane of an upper shelf edge, and (iii) a location in the plane of a lower shelf edge adjacent to the upper shelf edge;

generating a depth map containing, for each of a plurality of positions in the plane, a nearest object depth;

detecting an upper object boundary in the depth map between the upper and lower shelf edges;

generating a corrected depth map by updating each nearest object depth between the upper object boundary and the lower shelf edge to contain a depth of the upper object boundary; and

storing the corrected depth map.

2. The method of claim 1, further comprising presenting the corrected depth map to a gap detector for use in detecting gaps on the shelf.

3. The method of claim 1, further comprising:

prior to generating the depth map, aligning the depth measurements with a frame of reference based on the plane.

4. The method of claim 1, wherein generating the depth map includes generating each nearest object depth by:

assigning each depth measurement to one of a plurality of bins arranged in a three-dimensional grid, to generate a count of depth measurements falling within each bin;

for each of the plurality of positions in the plane, traversing a subset of the bins in a direction perpendicular to the plane and accumulating the respective counts of the subset of bins until the accumulated counts reach a threshold; and

setting the nearest object depth as a depth of a final one of the subset of bins traversed.

5. The method of claim 1, wherein detecting the upper object boundary comprises:

beginning at the location of the upper shelf edge, traversing a strip of the depth map from the location of the upper shelf edge toward the location of the lower shelf edge; and

determining whether a change in depth between traversed positions in the strip exceeds a predefined threshold.

6. The method of claim 5, wherein the predefined threshold defines a decrease in depth.

7. The method of claim 5, wherein generating the corrected depth map comprises setting the nearest object depths of each position in the strip between the upper object boundary and the location of the lower shelf edge to the depth of the upper object boundary.

8. The method of claim 5, wherein the strip is a line.

9. The method of claim 5, wherein the strip has a predefined width greater than one nearest object depth value.

10. The method of claim 1, further comprising:

prior to generating the corrected depth map, correcting null values in the depth map by performing a leaky convolution on the depth map.

11. A computing device for correcting translucency artifacts in data representing one or more objects disposed on a shelf, comprising:

a memory; and

an imaging controller connected to the memory, the imaging controller including;

a preprocessor configured to obtain a plurality of depth measurements captured by a depth sensor and corresponding to an area containing the shelf;

the preprocessor further configured to obtain (i) a definition of a plane containing edges of the shelf, (ii) a location in the plane of an upper shelf edge, and (iii) a location in the plane of a lower shelf edge adjacent to the upper shelf edge;

a map generator configured to generate a depth map containing, for each of a plurality of positions in the plane, a nearest object depth;

a corrector configured to detect an upper object boundary in the depth map between the upper and lower shelf edges; and

the corrector further configured to generate a corrected depth map by updating each nearest object depth between the upper object boundary and the lower shelf edge to contain a depth of the upper object boundary; and

the imaging controller further configured to store the corrected depth map in the memory.

12. The computing device of claim 11, wherein the imaging controller is further configured to present the corrected depth map to a gap detector for use in detecting gaps on the shelf.

13. The computing device of claim 11, wherein the preprocessor is further configured, prior to generation of the depth map, to aligning the depth measurements with a frame of reference based on the plane.

14. The computing device of claim 11, wherein the map generator is configured to generate each nearest object depth by:

assigning each depth measurement to one of a plurality of bins arranged in a three-dimensional grid, to generate a count of depth measurements falling within each bin;

for each of the plurality of positions in the plane, traversing a subset of the bins in a direction perpendicular to the plane and accumulating the respective counts of the subset of bins until the accumulated counts reach a threshold; and

setting the nearest object depth as a depth of a final one of the subset of bins traversed.

15. The computing device of claim 11, wherein the corrector is further configured to detect the upper object boundary by:

beginning at the location of the upper shelf edge, traversing a strip of the depth map from the location of the upper shelf edge toward the location of the lower shelf edge; and

determining whether a change in depth between traversed positions in the strip exceeds a predefined threshold.

16. The computing device of claim 15, wherein the predefined threshold defines a decrease in depth.

17. The computing device of claim 15, wherein the corrector is further configured to generate the corrected depth map by setting the nearest object depths of each position in the strip between the upper object boundary and the location of the lower shelf edge to the depth of the upper object boundary.

18. The computing device of claim 15, wherein the strip is a line.

19. The computing device of claim 15, wherein the strip has a predefined width greater than one nearest object depth value.

20. The computing device of claim 11, wherein the map generator is further configured to:

prior to generating the corrected depth map, correct null values in the depth map by performing a leaky convolution on the depth map.