DEVICE AND METHOD FOR DETERMINING THE VOLUME OF AN OBJECT MOVED BY AN INDUSTRIAL TRUCK

• US 20170228885A1
• Filed: 07/15/2015
• Published: 08/10/2017
• Est. Priority Date: 08/08/2014
• Status: Active Application

• Current Assignee: Cargometer GmbH
• Original Assignee: Cargometer GmbH
• Inventors: BAUMGARTNER, Michael
• Granted Patent: US 10,290,115 B2
• Time in Patent Office: Days
• CPC Class Codes: G01B 11/00 Measuring arrangements characterised by the use of optical techniques
• CPC Class Codes: G01B 11/04 specially adapted for measuring length or width of objects while moving
• CPC Class Codes: G01B 11/245 using a plurality of fixed, simultaneously operating transducers G01B11/2408 - G01B11/2425, G01B11/255 take precedence
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00

Abstract

The invention relates to a device for determining the volume of an object moved by an industrial truck and to a corresponding method. The device comprises a first depth image sensor (3) and a second depth image sensor (4), which are arranged in such a way that the object (1) can be sensed from two different directions as the object passes through the passage region (5), wherein the device (10) is designed to produce a sequence of individual images in a first resolution by means of each of the depth image sensors (3, 4), whereby the industrial truck (2) and the object (1) are sensed from different viewing angles as the industrial truck and the object pass through the passage region (5). The device also comprises a volume-determining apparatus (11), which is designed to subject the sequences of the individual images in the first resolution of the first depth image sensor (3) and of the second depth image sensor (4) to a super resolution processing in order to create a 3-D model (17) comprising the industrial truck (2) and the object (1) in a second resolution that is higher than the first resolution and to determine the volume of the object (1) on the basis of the produced 3-D model (17).

Claims

1. -17. (canceled)

18. A device for determining a volume of an object moved by an industrial truck, comprising:

a first depth image sensor and a second depth image sensor, which are arranged such that the object can be sensed from two different directions as the object passes through the passage region, wherein

the device is designed to produce sequences of single images in a first resolution with each of the depth image sensors, whereby the industrial truck and the object are sensed from different viewing angles as they pass through the passage region; and

a volume-determining apparatus, which is designed to subject the sequences of the single images in the first resolution of the first depth image sensor and of the second depth image sensor to super resolution processing so as to create a 3-D model comprising the industrial truck and the object in a second resolution that is higher than the first resolution, and to determine the volume of the object on the basis of the 3-D model produced.

19. The device in accordance with claim 18,

wherein the two depth image sensors are arranged in the passage region such that a forward region, an upper region, a first lateral region and a second lateral region located opposite to the first lateral region of a surface shape of the object can be sensed, and

wherein with the first depth image sensor at least the first lateral region can be sensed, and with the second depth image sensor at least the second lateral region can be sensed.

20. The device in accordance with claim 18, wherein the two depth image sensors are arranged and aligned in the passage region such that the moving object can be sensed obliquely from above.

21. The device in accordance with claim 18, wherein the two depth image sensors are arranged in the passage region such that the object, as it passes through the passage region, moves past underneath and between the two depth image sensors.

22. The device in accordance with claim 18,

wherein the volume-determining apparatus is designed to determine a region of interest (RoI) in the single images comprising the industrial truck and the object, for which super resolution processing is executed by ascertaining elements moving in the image sequences, and

wherein moving elements are ascertained on the basis of pixel regions for which an item of distance information alters relative to a predecessor image or relative to a background image that does not contain the industrial truck and the object.

23. The device in accordance with claim 22, wherein for the determination of the region of interest (RoI) only those moving elements are used that exceed a predetermined minimum size, and/or move along a prescribed direction or trajectory.

24. The device in accordance with claim 18, wherein for the super resolution processing firstly data from the sequence of single images of the first depth image sensor are combined in time, and data from the sequence of single images of the second depth image sensor are combined in time, and time-wise combined image data of the first depth image sensor are then spatially combined with time-wise combined image data of the second depth image sensor.

25. The device in accordance with claim 18, wherein the volume-determining apparatus is designed so as to supplement and/or to correct image information that is missing or subject to errors in the single images when creating the 3-D model in the super resolution processing.

26. The device in accordance with claim 18, wherein axes of visual cones of the two depth image sensors are inclined towards one another.

27. The device in accordance with claim 18, wherein a visual cone of at least one depth image sensor is designed such that a rear face of the object, as it passes through the passage region, can be sensed for a short period, at least as the object is again exiting from the visual cone.

28. The device in accordance with claim 18, wherein the volume-determining apparatus is designed so as to use a predetermined labelling on the industrial truck, attached at a point that cannot be concealed by the object, for purposes of demarcating the object from the industrial truck.

29. The device in accordance with claim 28, wherein

(a) the labelling is arranged laterally with respect to a direction of travel on a lift mast of the industrial truck;

and

(b) the volume-determining apparatus is set up,

(b1) on a basis of the sensed labelling to determine an inclination of the lift mast, and a demarcation plane parallel to the lift mast, together with a base plane of the load at right angles to the latter, wherein the demarcation plane and the base plane bound a possible location of the object; and

/or

(b2) on a basis of the sensed labelling to determine a vertical movement of the object executed relative to the industrial truck.

30. The device in accordance with claim 29, wherein the labelling is designed in the form of strips.

31. The device in accordance with claim 28, further comprising a synchronized RGB camera for purposes of detecting the labelling.

32. The device in accordance with claim 18, wherein

(a) the device is designed to sense a sequence of single images of the unloaded industrial truck as it travels through the passage region;

and

(b) the volume-determining apparatus is designed, on a basis of the sequence of single images, to determine a surface shape of the industrial truck, which is used for purposes of demarcating a surface shape of the object from the surface shape of the industrial truck in the 3-D model.

33. The device in accordance with claim 18, further comprising a database, in which 3-D models of various industrial trucks are stored, wherein for purposes of demarcating a surface shape of the object from the surface shape of the industrial truck, the volume-determining apparatus is designed such that the surface shape of the industrial truck in the 3-D model can be ascertained by use of the stored 3-D models and prescribed identification features.

34. The device in accordance with claim 18, wherein

(a) the first depth image sensor and the second depth image sensor operate in accordance with a method providing a pixel image, such as for example, the structured light principle, and/or the time of flight principle, and/or the stereo vision principle;

and/or

(b) the first depth image sensor and the second depth image sensor are arranged at a height above the floor of 2 to 4 meters, and/or are arranged spaced apart from one another by 2.5 to 4 meters transverse to a direction of movement of the object in the passage region.

35. The device in accordance with claim 18, wherein

(a) for purposes of creating the 3-D model of the object depth image data from the first depth image sensor and the second depth image sensor are exclusively used;

or

(b) one or two additional depth image sensors are arranged in a lower section of the passage region; and

/or

(c) an additional depth image sensor is provided, which for purposes of detecting a rear edge and/or rear face of the object is aligned essentially vertically downwards.

36. The device in accordance with claim 18, wherein

(a) the object is a shipment or goods loaded onto an industrial truck;

and/or

(b) the industrial truck is a forklift truck, an electric lift truck, a low-lift truck, a high-lift truck, a hand pallet truck, an order-picking truck, a high-reach forklift truck, a platform truck, a trolley, or a wire mesh cart; and

/or

(c) the passage region is a loading gate, a loading passage, an industrial door, a garage door, an interior door, a loading bay, a tunnel, a portal, or a marked path section.

37. A method for determining the volume of an object moved by an industrial truck, comprising the steps:

(a) arrangement of a first depth image sensor and a second depth image sensor in a passage region of the object being moved, such that the object can be sensed from two different directions as it passes through the passage region;

(b) production of sequences of single images in a first resolution with each of the depth image sensors, which show the industrial truck and the object from different viewing angles as the latter pass through the passage region;

(c) production of a 3-D model comprising the industrial truck and the object with a second resolution, which is higher than the first resolution, by executing super resolution processing on a basis of the sequences of single images in a first resolution of the first depth image sensor and the second depth image sensor; and

(d) determination of the volume of the object on the basis of the 3-D model produced.