Detecting sensor orientation characteristics using marker-based localization

• US 10,641,899 B2
• Filed: 11/16/2018
• Issued: 05/05/2020
• Est. Priority Date: 12/23/2016
• Status: Active Grant

• Current Assignee: Intrinsic Innovation LLC (Alphabet Inc.)
• Original Assignee: X Development LLC (Alphabet Inc.)
• Inventors: Shaffer, Daniel, Holz, Dirk
• Primary Examiner(s): Le, Toan M
• Application Number:
  US16/193,601
• Publication Number:
  US 20190086547A1
• Time in Patent Office: 536 Days
• CPC Class Codes: G01B 11/002 for measuring two or more coordinates
• CPC Class Codes: G01S 17/04 Systems determining the presence of a target
• CPC Class Codes: G01S 17/06 Systems determining position data of a target
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G05D 1/0236 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0244 using reflecting strips

Abstract

Methods and systems for detecting sensor orientation characteristics using marker-based localization are disclosed herein. In one aspect, a robotic device can: receive a map of a horizontal marker plane that includes mapped positions of a first marker and a second marker arranged in the horizontal marker plane; receive, from a sensor configured to scan a two-dimensional sensor plane, sensor data indicative of positions of the first and second markers relative to the sensor; determine measured positions of the first and second markers based on the sensor data and a current position of the sensor; determine a difference vector between a first vector that connects the mapped positions of the first and second markers and a second vector that connects the measured positions of the first and second markers; and determine, based on the difference vector, an orientation of the two-dimensional sensor plane relative to the horizontal marker plane.

Claims

1. A method comprising:

receiving a map of a horizontal marker plane that includes mapped positions of a first marker and a second marker arranged in the horizontal marker plane;

receiving, from a sensor configured to scan a two-dimensional sensor plane, sensor data indicative of positions of the first and second markers relative to the sensor;

determining measured positions of the first and second markers relative to the map based on the sensor data and a current position of the sensor relative to the map;

determining a difference vector between a first vector that connects the mapped positions of the first and second markers and a second vector that connects the measured positions of the first and second markers;

determining, based on the determined difference vector, an orientation of the two-dimensional sensor plane relative to the horizontal marker plane;

detecting, based on the determined orientation of the two-dimensional sensor plane relative to the horizontal marker plane, an uneven ground surface; and

navigating a robotic device based on detecting the uneven ground surface.

2. The method of claim 1, wherein the map of the horizontal marker plane further includes mapped positions of at least a third marker arranged in the horizontal marker plane, the method further comprising:

receiving, from the sensor, sensor data indicative of a position of the third marker relative to the sensor; and

determining the current position of the sensor relative to the map based on the sensor data indicative of the positions of the first, second, and third markers.

3. The method of claim 1, further comprising determining, based on a non-zero magnitude of the determined difference vector, that the sensor is vertically tilted relative to the horizontal marker plane.

4. The method of claim 1, further comprising determining, based on the determined difference vector, a pitch angle of the sensor relative to the horizontal marker plane and a roll angle of the sensor relative to the horizontal marker plane.

5. The method of claim 1, further comprising:

after receiving the sensor data from the sensor at a first time, receiving second sensor data indicative of positions of the first and second markers relative to the sensor at a second time; and

determining, based on the second sensor data, a second orientation of the two-dimensional sensor plane relative to the horizontal marker plane.

6. The method of claim 5, further comprising determining, based on a similarity between the determined orientation of the two-dimensional sensor plane relative to the horizontal marker plane and the determined second orientation of the two-dimensional sensor plane relative to the horizontal marker plane, that the sensor is improperly mounted to the robotic device.

7. The method of claim 5, further comprising determining, based on at least one of the determined orientation and the determined second orientation, a mounting angle of the sensor relative to the horizontal marker plane.

8. The method of claim 5, further comprising determining, based on a difference between the determined orientation of the two-dimensional sensor plane relative to the horizontal marker plane and the determined second orientation of the two-dimensional sensor plane relative to the horizontal marker plane, that the sensor is moving relative to the horizontal marker plane.

9. The method of claim 5, further comprising:

after receiving the second sensor data from the sensor at the second time, receiving third sensor data indicative of positions of the first and second markers relative to the sensor at a third time;

determining, based on the third sensor data, a third orientation of the two-dimensional sensor plane relative to the horizontal marker plane; and

determining, based on a second difference between the determined second orientation of the two-dimensional sensor plane relative to the horizontal marker plane and the determined third orientation of the two-dimensional sensor plane relative to the horizontal marker plane, that the sensor is oscillating relative to the horizontal marker plane.

10. The method of claim 1, wherein the robotic device is a first robotic device, and wherein the sensor data is received when the first robotic device is at a particular location relative to the map, the method further comprising:

receiving, from a second sensor configured to scan a second two-dimensional sensor plane, second sensor data indicative of positions of the first and second markers relative to the second sensor, wherein the second sensor is coupled to a second robotic device, and wherein the second sensor data is received when the second robotic device is at the particular location relative to the map; and

determining, based on the second sensor data, an orientation of the second two-dimensional sensor plane relative to the horizontal marker plane.

11. The method of claim 10, wherein the uneven ground surface is detected based on a similarity between the determined orientations of the two-dimensional sensor plane and the second two-dimensional sensor plane relative to the horizontal marker plane.

12. The method of claim 10, further comprising determining, based on at least one of the determined orientations of the two-dimensional sensor plane and the second two-dimensional sensor plane relative to the horizontal marker plane, a slope of the uneven ground surface at the particular location.

13. A robotic device comprising:

a sensor configured to scan a two-dimensional sensor plane; and

a control system configured to;

receive a map of a horizontal marker plane that includes mapped positions of a first marker and a second marker arranged in the horizontal marker plane;

receive, from the sensor, sensor data indicative of positions of the first and second markers relative to the sensor;

determine measured positions of the first and second markers relative to the map based on the sensor data and a current position of the sensor relative to the map;

determine a difference vector between a first vector that connects the mapped positions of the first and second markers and a second vector that connects the measured positions of the first and second markers;

determine, based on the determined difference vector, an orientation of the two-dimensional sensor plane relative to the horizontal marker plane;

detect, based on the determined orientation of the two-dimensional sensor plane relative to the horizontal marker plane, an uneven ground surface; and

navigate based on detecting the uneven ground surface.

14. The robotic device of claim 13, wherein the control system is further configured to determine, based on the determined difference vector, a pitch angle of the sensor relative to the horizontal marker plane and a roll angle of the sensor relative to the horizontal marker plane.

15. The robotic device of claim 13, wherein the control system is further configured to:

after receiving the sensor data from the sensor at a first time, receive second sensor data indicative of positions of the first and second markers relative to the sensor at a second time; and

determine, based on the second sensor data, a second orientation of the two-dimensional sensor plane relative to the horizontal marker plane.

16. The robotic device of claim 15, wherein the control system is further configured to determine, based on a difference between the determined orientation of the two-dimensional sensor plane relative to the horizontal marker plane and the determined second orientation of the two-dimensional sensor plane relative to the horizontal marker plane, that the sensor is oscillating relative to the horizontal marker plane.

17. The robotic device of claim 13, wherein the robotic device is configured to transport a load across a ground surface, and wherein the control system is further configured to determine, based on the determined difference vector, that the load is causing the robotic device to tilt relative to a ground surface.

18. A system comprising:

a first marker and a second marker arranged in a horizontal marker plane;

a robotic device comprising a sensor configured to scan a two-dimensional sensor plane; and

a control system configured to;

receive a map of a horizontal marker plane that includes mapped positions of the first marker and the second marker arranged in the horizontal marker plane;

receive, from the sensor, sensor data indicative of positions of the first and second markers relative to the sensor;

determine measured positions of the first and second markers relative to the map based on the sensor data and a current position of the sensor relative to the map;

determine a difference vector between a first vector that connects the mapped positions of the first and second markers and a second vector that connects the measured positions of the first and second markers;

determine, based on the determined difference vector, an orientation of the two-dimensional sensor plane relative to the horizontal marker plane;

detect, based on the determined orientation of the two-dimensional sensor plane relative to the horizontal marker plane, an uneven ground surface; and

cause the robotic device to navigate based on detecting the uneven ground surface.

19. The system of claim 18, wherein the control system is further configured to determine, based on the determined difference vector, a pitch angle of the sensor relative to the horizontal marker plane and a roll angle of the sensor relative to the horizontal marker plane.

20. The system of claim 18, wherein the control system is further configured to:

after receiving the sensor data from the sensor at a first time, receive second sensor data indicative of positions of the first and second markers relative to the sensor at a second time; and

determine, based on the second sensor data, a second orientation of the two-dimensional sensor plane relative to the horizontal marker plane.