Industrial vehicles with point fix based localization

• US 9,174,830 B1
• Filed: 09/29/2014
• Issued: 11/03/2015
• Est. Priority Date: 09/29/2014
• Status: Active Grant

• Current Assignee: Crown Equipment Corporation
• Original Assignee: Crown Equipment Corporation
• Inventors: Bell, Mark
• Primary Examiner(s): Cheung, Mary
• Assistant Examiner(s): MAZZARA, ANNE MARIE
• Application Number:
  US14/499,700
• Time in Patent Office: 400 Days
• Field of Search: 701/28
• US Class Current: 1/1
• CPC Class Codes: B66F 9/063 Automatically guided
• CPC Class Codes: B66F 9/0755 Position control; Position detectors
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06T 2207/20061 Hough transform
• CPC Class Codes: G06T 2207/30172 Centreline of tubular or elongated structure
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/13 Edge detection

Abstract

According to the embodiments described herein, an industrial vehicle can include a camera and one or more processors. The camera can capture an input image of ceiling lights of the ceiling of the warehouse. The one or more processors can execute machine readable instructions to detect transversal edges from the ceiling lights of the input image. The machine readable instructions executed by the one or more processors can determine points of convergence between the transversal edges and a first edge line and a second edge line. The machine readable instructions executed by the one or more processors can select a set of point fixes from the points of convergence. The machine readable instructions executed by the one or more processors can navigate the industrial vehicle through the warehouse utilizing the centerline and the set of point fixes.

Claims

1. An industrial vehicle comprising a camera and one or more processors, wherein:

the camera is communicatively coupled to the one or more processors;

the camera is mounted to the industrial vehicle and focused to a ceiling of a warehouse;

the camera captures an input image of ceiling lights of the ceiling of the warehouse;

the one or more processors execute machine readable instructions to;

provide a first edge line and a second edge line of the ceiling lights of the input image;

provide a centerline of the ceiling lights of the input image, wherein the centerline is disposed between the first edge line and the second edge line of the ceiling lights;

detect transversal edges from the ceiling lights of the input image;

determine points of convergence between the transversal edges and the first edge line and the second edge line;

select a set of point fixes from the points of convergence; and

navigate the industrial vehicle through the warehouse utilizing the centerline and the set of point fixes.

2. The industrial vehicle of claim 1, wherein the one or more processors execute the machine readable instructions to:

set a region of interest that is coincident with a usable portion of the ceiling lights, wherein the transversal edges are detected from the usable portion of the ceiling lights.

3. The industrial vehicle of claim 2, wherein the region of interest is a bounding box that is axis aligned with the centerline.

4. The industrial vehicle of claim 3, wherein:

the first edge line and the second edge line are line segments demarcated by end points; and

the region of interest is demarcated by one or more of the end points of the first edge line and the second edge line.

5. The industrial vehicle of claim 3, wherein:

the centerline is demarcated by end points; and

the one or more processors execute the machine readable instructions to project the end points of the centerline upon the first edge line and the second edge line to create intersection points, and wherein the region of interest is demarcated by one or more of the intersection points.

6. The industrial vehicle of claim 2, wherein:

the first edge line and the second edge line overlap to define an overlapping portion; and

the region of interest is demarcated by the overlapping portion of the first edge line and the second edge line.

7. The industrial vehicle of claim 2, wherein:

the first edge line and the second edge line converge at a vanishing point;

the first edge line and the second edge line are line segments demarcated by end points; and

the region of interest is determined utilizing the vanishing point.

8. The industrial vehicle of claim 1, wherein the transversal edges are detected using a Hough transform.

9. The industrial vehicle of claim 1, wherein the one or more processors execute the machine readable instructions to:

determine an average angle of the transversal edges; and

remove unusable edges from the transversal edges that are outside of a restricted range of angles on either side of the average angle of the transversal edges, whereby usable edges of the transversal edges remain, and wherein the set of point fixes are selected from the usable edges of the transversal edges.

10. The industrial vehicle of claim 1, wherein the points of convergence are determined by extrapolating the transversal edges.

11. The industrial vehicle of claim 10, wherein the points of convergence are determined by extrapolating the first edge line, the second edge line, or both.

12. The industrial vehicle of claim 10, wherein:

the set of point fixes are selected from the transversal edges having a length ratio above threshold level; and

the length ratio is given by a ratio of a length of each of the transversal edges to a length of a span between corresponding points of convergence.

13. The industrial vehicle of claim 1, wherein the one or more processors execute the machine readable instructions to:

label a point fix of the set of point fixes as corresponding to a transition of dark-to-bright or bright-to-dark in the input image.

14. A method for navigating an industrial vehicle comprising:

capturing an input image of skylights of a ceiling of a warehouse, wherein the input image is captured with a camera coupled to an industrial vehicle;

extracting a first edge line and a second edge line from the skylights of the input image;

determining a centerline of the skylights from the first edge line and the second edge line of the skylights, wherein the centerline is disposed between the first edge line and the second edge line of the skylights;

detecting transversal edges from the skylights of the input image;

determining points of convergence between the transversal edges and the first edge line and the second edge line of the skylights;

selecting a set of point fixes from the points of convergence;

determining a pose of the industrial vehicle, a position of the industrial vehicle, or both based upon the centerline and the set of point fixes; and

navigating the industrial vehicle through the warehouse utilizing the pose, the position, or both.

15. The method of claim 14, further comprising:

extracting raw feature contours from the skylights of the input image;

determining a convex hull from the skylights of the input image, wherein the convex hull comprises hull line segments;

generating a mask from the hull line segments of the convex hull; and

selecting a selected portion from the raw feature contours, wherein the selected portion of the raw feature contours is coincident with the mask, and wherein the first edge line and the second edge line are extracted from the selected portion of the raw feature contours.

16. The method of claim 14, wherein extracting the first edge line and the second edge line from the skylights of the input image further comprising:

extracting line segments from the skylights of the input image;

selecting the first edge line from the line segments;

weighting the line segments based at least in part upon overlap with the first edge line; and

selecting the second edge line from the line segments based upon the weighting of the line segments.

17. The method of claim 14, further comprising:

setting a region of interest that is coincident with a usable portion of the skylights, wherein the transversal edges are detected from the usable portion of the skylights.

18. The method of claim 17, further comprising using the centerline as a reference to set the region of interest.

19. The method of claim 14, further comprising:

determining an average angle of the transversal edges; and

removing unusable edges from the transversal edges that are outside of a restricted range of angles on either side of the average angle of the transversal edges, whereby usable edges of the transversal edges remain, and wherein the set of point fixes are selected from the usable edges of the transversal edges.

20. The method of claim 14, further comprising:

labeling a point fix of the set of point fixes as corresponding to a transition of dark-to-bright or bright-to-dark in the input image.