Vehicle positioning and object avoidance

• US 10,025,314 B2
• Filed: 01/27/2016
• Issued: 07/17/2018
• Est. Priority Date: 01/27/2016
• Status: Active Grant

• Current Assignee: Hand Held Products Incorporated (Honeywell International Inc.)
• Original Assignee: Hand Held Products Incorporated (Honeywell International Inc.)
• Inventors: Houle, Scott Xavier, Hunt, Jeffrey Mark
• Primary Examiner(s): Sample, Jonathan L
• Application Number:
  US15/007,522
• Publication Number:
  US 20170212517A1
• Time in Patent Office: 902 Days
• Field of Search: 701 50, 701 70
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2720/10 Longitudinal speed
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B66F 17/003 for fork-lift trucks
• CPC Class Codes: B66F 9/07509 Braking
• CPC Class Codes: B66F 9/0755 Position control; Position detectors
• CPC Class Codes: G05D 1/0214 in accordance with safety or protection criteria, e.g. avoiding hazardous areas monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593

Abstract

A system is described for presenting information relating to lifting and moving a load object with a vehicle. Upon the lifting, a dimensioner determines a size and a shape of the load object, computes a corresponding spatial representation, and generates a corresponding video signal. During the moving, an imager observes a scene in front of the vehicle, relative to its forward motion direction, and generates a video signal corresponding to the observed scene. The imager has at least one element moveable vertically, relative to the lifting. A display renders a real time visual representation of the scene observed in front of the vehicle based on the corresponding video signal and superimposes a representation of the computed spatial representation of the load object.

Claims

1. A system, comprising:

a dimensioner operable, upon lifting of a load object, for determining a size and a shape of the load object, computing a corresponding spatial representation of the load object, and generating a first video signal corresponding to the computed spatial representation;

an imager operable, during moving of the load object, for observing a scene disposed before a front of a vehicle, relative to a forward direction of motion, and generating a second video signal corresponding to the observed scene, the imager comprising at least one element moveable vertically in relation to the lifting; and

a display operable for rendering, a real time visual representation of the observed scene disposed before the front of the vehicle based on the corresponding second video signal, the visual representation transposed to a perspective consistent with the direct view of an operator of the vehicle, and superimposed with the visual representation of the observed scene, a transparent representation of the computed spatial representation of the load object based on the corresponding first video signal.

2. The system as described in claim 1 wherein the vehicle comprises a forklift, the forklift comprising a member operable in relation to the lifting of the load, and wherein the at least one element of the imager moveable vertically in relation to the lifting is positioned on a portion of the member disposed proximate to the front of the vehicle.

3. The system as described in claim 1 wherein the observing the scene comprises capturing a real time three dimensional (3D) image of the scene disposed before the front of the vehicle, and wherein the rendering of the real time visual representation of the observed scene disposed before the front of the vehicle is presented at least in relation to a perspective corresponding to the at least one vertically moveable element.

4. The system as described in claim 1 wherein the spatial representation corresponding to the load object comprises a wireframe computed based on the determined size and shape of the load object.

5. The system as described in claim 1 wherein the dimensioner is operable, further, and prior to the lifting of the load item, for computing a distance between the front of the vehicle and the load item, and wherein the display is operable, further, for rendering a representation corresponding to the computed distance.

6. The system as described in claim 1 wherein the imager comprises a trajectory analyzer operable, upon a detection of one or more avoidable objects positioned over a range within the observed scene disposed before the front of the vehicle, for computing a trajectory relating to the forward motion of the vehicle in relation to each of the avoidable objects and generating a trajectory signal corresponding to each of the avoidable objects.

7. The system as described in claim 6 wherein, upon the detection of the one or more objects, the rendering of the real time visual representation of the observed scene disposed before the front of the vehicle comprises presenting a visual representation of the one or more avoidable objects and data relating to the computed trajectory.

8. The system as described in claim 7 wherein, upon the computed trajectory comprising an imminent risk of a collision with at least one of the avoidable objects, the trajectory analyzer is operable, further, for performing at least one action related to avoiding the collision.

9. The system as described in claim 1, further comprising a plurality of cameras, the cameras operable in relation to the dimensioner or the imager, and comprising the at least one element moveable vertically in relation to the lifting.

10. The system as described in claim 1 wherein the display is observable to an operator of the vehicle during the lifting and the moving.

11. A method comprising the steps of:

determining, upon lifting a load object with a vehicle, a size and a shape of the load object;

computing a spatial representation of the load object corresponding to the determined size and shape of the load object;

generating a first video signal corresponding to the computed spatial representation;

observing, during moving the load object with the vehicle and using at least one element moveable vertically in relation to the lifting, a scene disposed before a front of the vehicle, relative to a forward direction of motion;

generating a second video signal corresponding to the observed scene;

rendering a real time visual representation of the observed scene disposed before the front of the vehicle based on the corresponding second video signal, the visual representation transposed to a perspective consistent with the direct view of an operator of the vehicle; and

rendering a transparent representation of the computed spatial representation of the load object based on the corresponding first video signal, the rendered transparent representation of the computed spatial representation of the load object comprising a wireframe corresponding to the determined size and shape of the load object superimposed in relation to the rendered real time visual representation of the observed scene disposed before the front of the vehicle.

12. The method as described in claim 11 wherein the vehicle comprises a forklift, the forklift comprising a member operable in relation to the lifting of the load, and wherein the at least one element of the imager moveable vertically in relation to the lifting is positioned on a portion of the member disposed proximate to the front of the vehicle.

13. The method as described in claim 11 wherein the observing the scene step comprises capturing a real time three dimensional (3D) image of the scene disposed before the front of the vehicle, and wherein the step of rendering of the real time visual representation of the observed scene disposed before the front of the vehicle is presented at least in relation to a perspective corresponding to the at least one vertically moveable element.

14. The method as described in claim 11 wherein the step of computing the spatial representation corresponding to the load object comprises computing a wireframe representation of the load item based on the determined size and shape thereof.

15. The method as described in claim 11, further comprising the steps of:

computing, prior to the lifting of the load item, a distance between the front of the vehicle and the load item; and

rendering a representation of data corresponding to the computed distance.

16. The method as described in claim 15 wherein the determining the size and a shape of the load object step, the observing the scene disposed before the front of the vehicle step, and the computing the distance between the front of the vehicle and the load item step, comprise a step of processing image data captured with a plurality of cameras, the cameras comprising the at least one element moveable vertically in relation to the lifting.

17. The method as described in claim 11, further comprising the steps of:

analyzing the observed scene disposed before the front of the vehicle;

detecting, based on the analysis of the observed scene, a presence of one or more avoidable objects positioned over a range within the observed scene disposed before the front of the vehicle;

computing a trajectory relating to the forward motion of the vehicle in relation to each of the avoidable objects;

generating a trajectory signal corresponding to each of the avoidable objects;

rendering, further, a visual representation of the one or more avoidable objects and data relating to the computed trajectory; and

performing, upon the computed trajectory comprising a data indicative of an imminent risk of a collision with at least one of the avoidable objects, at least one action related to avoiding the collision.

18. The method as described in claim 17 wherein the step of performing the at least one action related to avoiding the collision comprises one or more of:

annunciating an alarm related to the avoiding of the collision;

initiating an evasive action;

or

one or more of braking, slowing, or stopping the vehicle.

19. A vehicle, comprising:

a structure suspended on a movable frame;

a lift member movably coupled to the structure and operable for lifting a load object;

a drive coupled to the moveable frame and operable for providing a mechanical force for operating the lift member and for moving the vehicle and the lifted load object; and

a system for presenting information relating to the lifting and the moving of the load object and the vehicle, the system comprising;

a dimensioner operable, upon the lifting, for determining a size and a shape of the load object, computing a corresponding spatial representation of the load object, and generating a first video signal corresponding to the computed spatial representation;

an imager operable, during the moving, for observing a scene disposed before a front of the vehicle, relative to a forward direction of motion, and generating a second video signal corresponding to the observed scene, the imager comprising at least one element moveable vertically in relation to the lifting; and

a display operable for rendering, a real time visual representation of the observed scene disposed before the front of the vehicle based on the corresponding second video signal, the visual representation transposed to a perspective consistent with the direct view of an operator of the vehicle, and superimposed with the visual representation of the observed scene, a transparent representation of the computed spatial representation of the load object based on the corresponding first video signal.

20. The vehicle as described in claim 19 wherein the wherein the vehicle comprises a forklift, and wherein the at least one element of the imager moveable vertically in relation to the lifting is positioned on a portion of the lift member disposed proximate to the front of the vehicle.