Pallet Tracking During Engagement and Disengagement

• US 20190194005A1
• Filed: 12/22/2017
• Published: 06/27/2019
• Est. Priority Date: 12/22/2017
• Status: Active Grant

• Current Assignee: Boston Dynamics, Inc. (SoftBank Group Corp.)
• Original Assignee: X Development LLC (Alphabet Inc.)
• Inventors: Shah, Hemal, Gulliver, Ian
• Granted Patent: US 10,640,347 B2
• Time in Patent Office: Days
• CPC Class Codes: B66F 9/0755 Position control; Position detectors
• CPC Class Codes: B66F 9/24 Electrical devices or systems
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G05D 1/0225 involving docking at a fixed facility, e.g. base station or loading bay parking aids B62D15/027
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

An example method may include receiving, from a sensor on a vehicle, an initial plurality of sensor data points representing a position of a face of a pallet. The vehicle may include tines configured to engage the pallet. A baseline geometric representation of the face of the pallet may be determined based on the initial plurality of sensor data points. The vehicle may be caused to reposition the tines relative to the pallet. A subsequent plurality of sensor data points representing the position of the face of the pallet after repositioning the tines may be received from the sensor. An updated geometric representation of the face of the pallet may be determined based on the subsequent sensor data points. It may be determined that the updated geometric representation deviates from the baseline geometric representation by more than a threshold value and, in response, motion of the vehicle may be adjusted.

Claims

1. A method comprising:

receiving, by a control system and from a sensor on a vehicle, an initial plurality of sensor data points representing a position of a face of a pallet within an environment, wherein the vehicle comprises one or more tines configured to engage the pallet;

determining, by the control system, a baseline geometric representation of the face of the pallet based on the initial plurality of sensor data points;

causing, by the control system, the vehicle to reposition the one or more tines relative to the pallet;

receiving, by the control system and from the sensor, a subsequent plurality of sensor data points representing the position of the face of the pallet after repositioning the one or more tines;

determining, by the control system, an updated geometric representation of the face of the pallet based on the subsequent sensor data points;

determining, by the control system, that the updated geometric representation deviates from the baseline geometric representation by more than a threshold value; and

in response to determining that the updated geometric representation deviates from the baseline geometric representation by more than the threshold value, adjusting, by the control system, motion of the vehicle.

2. The method of claim 1, wherein the sensor data points represent the position of the face of the pallet in at least two dimensions, wherein the baseline geometric representation comprises a first line fitted to the initial plurality of sensor data points, and wherein the updated geometric representation comprises a second line fitted to the subsequent plurality of sensor data points.

3. The method of claim 1, wherein the sensor data points represent the position of the face of the pallet in three dimensions, wherein the baseline geometric representation comprises a first plane fitted to the initial plurality of sensor data points, and wherein the updated geometric representation comprises a second plane fitted to the subsequent plurality of sensor data points.

4. The method of claim 1, wherein the sensor data points represent the position of the face of the pallet in at least two dimensions, wherein the baseline geometric representation comprises a first center of mass of the initial plurality of sensor data points, and wherein the updated geometric representation comprises a second center of mass of the subsequent plurality of sensor data points.

5. The method of claim 1, wherein the sensor data points represent distances between the sensor and points along the face of the pallet, and wherein the method further comprises:

determining, based on the distances between the sensor and the points along the face of the pallet, the position of the face of the pallet in a reference frame of the vehicle; and

transforming, based on a location of the vehicle within the environment, the position of the face of the pallet from the reference frame of the vehicle to a reference frame of the environment.

6. The method of claim 1, wherein the sensor data points additionally represent positions of other physical features within the environment, and wherein the method further comprises:

determining a zone of interest within the environment occupied by the face of the pallet;

filtering the initial plurality of sensor data points by removing positions outside of the determined zone of interest;

determining the baseline geometric representation based on the filtered initial plurality of sensor data points;

filtering the subsequent plurality of sensor data points by removing positions outside of the determined zone of interest; and

determining the updated geometric representation based on the filtered subsequent plurality of sensor data points.

7. The method of claim 1, wherein determining that the updated geometric representation deviates from the baseline geometric representation by more than the threshold value comprises:

determining a plurality of residuals indicating differences between positions along the face of the pallet represented by the updated geometric representation and positions along the face of the pallet represented by the baseline geometric representation; and

determining that the plurality of determined residuals exceeds the threshold value.

8. The method of claim 7, wherein determining that the plurality of determined residuals exceeds the threshold value comprises:

determining (i) that one or more of the plurality of determined residuals exceed a first threshold value, (ii) that a sum of absolute values of the determined residuals exceeds a second threshold value, or (iii) that a sum of squares of the determined residuals exceeds a third threshold value.

9. The method of claim 1, wherein causing the vehicle to reposition the one or more tines relative to the pallet comprises:

causing the vehicle to drive towards the pallet to cause the one or more tines to penetrate the pallet while the pallet is disposed on a ground surface of the environment.

10. The method of claim 1, wherein causing the vehicle to reposition the one or more tines relative to the pallet comprises:

causing the vehicle to drive away from the pallet to remove the one or more tines from the pallet while the pallet is disposed on a ground surface of the environment.

11. The method of claim 1, wherein the face of the pallet comprises a first vertical surface defined by a side of the pallet and a second vertical surface defined by a load disposed on the pallet.

12. The method of claim 1, further comprising:

in response to determining that the updated geometric representation deviates from the baseline geometric representation by more than the threshold value, adjusting at least one of (i) a vertical position of the one or more tines or (ii) a tilt of the one or more tines.

13. The method of claim 1, wherein adjusting motion of the vehicle comprises stopping the vehicle, and wherein the method further comprises:

in response to determining that the updated geometric representation deviates from the baseline geometric representation by more than the threshold value, generating a warning indicating unexpected motion of the pallet.

14. A method comprising:

receiving, by a control system and from a sensor on a vehicle, an initial plurality of sensor data points representing a position of a face of a pallet within an environment, wherein the vehicle comprises one or more tines configured to engage the pallet;

determining, by the control system, a baseline geometric representation of the face of the pallet based on the initial plurality of sensor data points;

determining, by the control system, an exclusion field between the vehicle and the baseline geometric representation based on a distance between the baseline geometric representation and the vehicle;

causing, by the control system, the vehicle to reposition the one or more tines relative to the pallet;

receiving, by the control system and from the sensor, a subsequent plurality of sensor data points representing the position of the face of a pallet after repositioning the one or more tines relative to the pallet;

determining, by the control system, that more than a threshold number of sensor data points of the subsequent plurality of sensor data points are within the exclusion field; and

in response to determining that more than the threshold number of sensor data points of the subsequent plurality of sensor data points are within the exclusion field, adjusting, by the control system, motion of the vehicle.

15. The method of claim 14, wherein causing the vehicle to reposition the one or more tines relative to the pallet comprises causing the vehicle to drive away from the pallet to remove the one or more tines from the pallet while the pallet is disposed on a ground surface of the environment, and wherein the method further comprises:

increasing a size of the exclusion field as the vehicle drives away from the pallet.

16. The method of claim 14, wherein causing the vehicle to reposition the one or more tines relative to the pallet comprises causing the vehicle to drive towards the pallet to cause the one or more tines to penetrate the pallet while the pallet is disposed on a ground surface of the environment, and wherein the method further comprises:

decreasing a size of the exclusion field as the vehicle drives towards the pallet.

17. The method of claim 14, wherein the sensor data points represent the position of the face of the pallet in at least two dimensions, wherein the baseline geometric representation comprises a first line fitted to the initial plurality of sensor data points, and wherein the exclusion field comprises an area between the first line and the vehicle.

18. The method of claim 14, wherein the sensor data points represent the position of the face of the pallet in three dimensions, wherein the baseline geometric representation comprises a first plane fitted to the initial plurality of sensor data points, and wherein the exclusion field comprises a volume between the first plane and the vehicle.

19. The method of claim 14, wherein the sensor data points additionally represent positions of other physical features within the environment, and wherein the method further comprises:

determining a zone of interest within the environment occupied by the face of the pallet;

filtering the initial plurality of sensor data points by removing positions outside of the determined zone of interest; and

determining the baseline geometric representation based on the filtered initial plurality of sensor data points.

20. A system comprising:

a vehicle comprising one or more tines configured to engage pallets;

a sensor on the vehicle; and

a control system configured to;

receive, from the sensor, an initial plurality of sensor data points representing a position of a face of a pallet within an environment;

determine a baseline geometric representation of the face of the pallet based on the initial plurality of sensor data points;

cause the vehicle to reposition the one or more tines relative to the pallet;

receive, from the sensor, a subsequent plurality of sensor data points representing the position of the face of the pallet after repositioning the one or more tines;

determine an updated geometric representation of the face of the pallet based on the subsequent sensor data points;

determine that the updated geometric representation deviates from the baseline geometric representation by more than a threshold value; and

in response to determining that the updated geometric representation deviates from the baseline geometric representation by more than the threshold value, adjust motion of the vehicle.