Vision-based method of determining cargo status by boundary detection

• US 7,940,955 B2
• Filed: 07/26/2006
• Issued: 05/10/2011
• Est. Priority Date: 07/26/2006
• Status: Active Grant

• Current Assignee: Aptiv Technologies AG (Aptiv PLC)
• Original Assignee: Delphi Technologies, Inc. (BorgWarner Incorporated)
• Inventors: Zhang, Yan, Kiselewich, Stephen J., Shen, Junqiang
• Primary Examiner(s): Hung; Yubin
• Assistant Examiner(s): Yentrapati; Avinash
• Application Number:
  US11/493,279
• Publication Number:
  US 20080025565A1
• Time in Patent Office: 1,749 Days
• Field of Search: 382/181, 382/199, 382/100, 382/103
• US Class Current: 382/100
• CPC Class Codes: G06T 7/0008 checking presence/absence
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles

Abstract

The empty vs. non-empty status of a cargo container is detected based on boundary analysis of a wide-angle image obtained by a monocular vision system. The wide-angle image is warped to remove distortion created by the vision system optics, and the resulting image is edge-processed to identify the boundaries of the container floor. If package boundaries are detected within the floor space, or a large foreground package is blocking the floor boundaries, the cargo status is set to non-empty. If floor boundaries are detected and no package boundaries are detected within the floor space, the cargo status is set to empty.

Claims

1. A method of determining a cargo status of a container having a floor, sidewalls, and a rear wall supporting a door that can be open or closed, comprising the steps of:

producing a digital image encompassing said floor and boundaries between said floor and said sidewalls, and at least a portion of said door;

edge-processing said image, and identifying a floor portion of said image bounded by detected edges that correspond to the boundaries between said floor and said sidewalls;

evaluating pixel brightness in a portion of said image corresponding to said door to determine whether said door is open;

locating a lower edge of said door when it is determined that said door is open;

bounding said floor portion of said image by the located lower edge of said door;

evaluating the detected edges within the identified floor portion of said image to detect edges of cargo; and

determining the cargo status of said container based on said evaluation.

2. The method of claim 1, including the steps of:

producing the digital image with a wide-angle lens; and

warping the produced image to remove optical distortion caused by the wide-angle lens so that the edges that correspond to the boundaries between said floor and said sidewalls are detected by said edge-processing as linear segments.

3. The method of claim 2, including the steps of:

edge-processing said image at a first resolution and determining if a linear segment was detected; and

edge-processing said image at a second resolution lower than said first resolution if said linear segment was not detected.

4. The method of claim 2, including the step of:

determining if a linear segment detected by said edge-processing corresponds to a boundary between said floor and a sidewall based on a length of said segment and its location and orientation in said image.

5. The method of claim 1, including the steps of:

adaptively updating floor boundary information stored a data archive based on said detected edges that correspond to the boundaries between said floor and said sidewalls; and

subsequently identifying the floor portion of said image based on the adaptively updated floor boundary data stored in said data archive when the edges that correspond to the boundaries between said floor and said sidewalls are not detected by said edge-processing.

6. The method of claim 1, including the steps of:

storing door edge data pertaining to the located lower edge of said door in a data archive; and

subsequently bounding said floor portion of said image based on the door edge data stored in said data archive when it is determined that said door is closed.

7. The method of claim 1, where the step of evaluating the detected edges within the identified floor portion includes the steps of:

measuring a length of a detected edge; and

determining that the detected edge is a cargo edge if the measured length exceeds a threshold length.

8. The method of claim 7, where said digital image is produced by a imaging device mounted at one end of said container, and said threshold length decreases with increasing distance of said detected edge from said imaging device.

9. The method of claim 7, including the step of:

setting said cargo-status to non-empty when said detected edge is determined to be a cargo edge.

10. The method of claim 7, including the step of:

setting said cargo-status to empty when none of the detected edges are determined to be cargo edges.

11. The method of claim 1, including the step of:

detecting a large item of cargo in a foreground of said digital image when an average brightness of said digital image exceeds a threshold and the boundaries between said floor and said sidewalls are not detected by said edge-processing; and

setting said cargo status to non-empty if said large item of cargo is detected.