Sensor Trajectory Planning for a Vehicle

• US 20170357270A1
• Filed: 06/09/2016
• Published: 12/14/2017
• Est. Priority Date: 06/09/2016
• Status: Active Grant

• Current Assignee: Intrinsic Innovation LLC (Alphabet Inc.)
• Original Assignee: X Development LLC (Alphabet Inc.)
• Inventors: Russell, Jared
• Granted Patent: US 9,880,561 B2
• Time in Patent Office: Days
• CPC Class Codes: B25J 9/162 Mobile manipulator, movable base with manipulator arm mounted on it
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/0242 using non-visible light signals, e.g. IR or UV signals
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/0257 using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G05D 1/0297 by controlling means in a control room
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 2201/06 Recognition of objects for industrial automation

Abstract

An example system includes a vehicle and a sensor connected to the vehicle. The system may receive a predetermined path for the vehicle to follow. The system may also receive a plurality of objectives, associated with a corresponding set of sensor data, for which to collect sensor data. The system may determine, for each of the plurality of objectives, a portion of the environment for the sensor to scan to acquire the corresponding set of sensor data. The system may determine, based on the portion of the environment determined for each of the plurality of objectives, a sensor trajectory through which to move the sensor. The system may cause the sensor to move through the determined sensor trajectory and scan portions of the environment corresponding to the determined sensor trajectory as the vehicle moves along the predetermined path.

Claims

1. A system comprising:

a vehicle;

a sensor connected to the vehicle, wherein the sensor is configured to move with respect to the vehicle to control a portion of an environment observable by the sensor; and

a control system configured to;

receive a predetermined path for the vehicle to follow;

receive a plurality of objectives for which to collect sensor data as the vehicle moves along the predetermined path, wherein each of the plurality of objectives is associated with a corresponding set of sensor data;

determine, for each of the plurality of objectives, at least one portion of the environment for the sensor to scan to acquire the corresponding set of sensor data;

determine, based on the at least one portion of the environment determined for each of the plurality of objectives, a sensor trajectory through which to move the sensor with respect to the vehicle while the vehicle moves along the predetermined path; and

provide instructions to cause the sensor to move through the determined sensor trajectory and scan portions of the environment corresponding to the determined sensor trajectory as the vehicle moves along the predetermined path.

2. The system of claim 1, wherein the control system is further configured to:

determine a plurality of priority levels corresponding to the plurality of objectives; and

determine, based on (i) the at least one portion of the environment determined for each of the plurality of objectives and (ii) the determined plurality of priority levels, the sensor trajectory through which to move the sensor with respect to the vehicle while the vehicle moves along the predetermined path.

3. The system of claim 1, wherein the control system is further configured to:

determine, for each of the plurality of objectives, a cost associated with scanning the at least one determined portion of the environment based on a position of the at least one determined portion of the environment within the environment;

determine an overall cost function for the plurality of objectives based on the cost determined for each of the plurality of objectives; and

determine the sensor trajectory through which to move the sensor with respect to the vehicle by determining a sensor trajectory that minimizes the overall cost function.

4. The system of claim 3, wherein the cost associated with scanning the at least one determined portion of the environment is an amount of time to move the sensor from a current sensor position to a sensor position corresponding to the at least one determined portion of the environment.

5. The system of claim 1, wherein the control system is further configured to:

determine, for one or more of the portions of the environment corresponding to the determined sensor trajectory, a minimum scan resolution;

modulate a speed with which the sensor moves along the determined sensor trajectory based on the determined minimum scan resolution for the one or more of the portions of the environment corresponding to the determined sensor trajectory.

6. The system of claim 1, wherein the control system is further configured to:

determine, for one or more of the portions of the environment corresponding to the determined sensor trajectory, a minimum scan resolution; and

modulate a speed with which the vehicle moves along the predetermined path based on the determined minimum scan resolution for the one or more of the portions of the environment corresponding to the determined sensor trajectory.

7. The system of claim 1, wherein the control system is configured to:

determine, for a particular objective of the plurality of objectives, the at least one portion of the environment for the sensor to scan in order to acquire the corresponding set of sensor data by determining an expected position of an object within the environment, wherein scanning a portion of the environment corresponding to the expected position provides the corresponding set of sensor data associated with the particular objective; and

determine the sensor trajectory by determining a range of angular positions of the sensor, wherein an extent of a field of view of the sensor intersects the expected position of the object when an angular position of the sensor is within the determined range of angular positions of the sensor.

8. The system of claim 1, wherein the control system is configured to determine the sensor trajectory through which to move the sensor with respect to the vehicle while the vehicle moves along the predetermined path by:

determining an objective function indicative of at least one parameter associated with the plurality of objectives;

determining a plurality of candidate sensor trajectories through which to move the sensor with respect to the vehicle while the vehicle moves along the predetermined path; and

selecting a sensor trajectory from the plurality of candidate sensor trajectories that maximizes the determined objective function.

9. The system of claim 8, wherein the objective function indicates a number of distinct features in the environment observable by the sensor while moving along the determined sensor trajectory.

10. The system of claim 1, wherein the control system is configured to determine the sensor trajectory through which to move the sensor with respect to the vehicle while the vehicle moves along the predetermined path by:

determining a plurality of candidate sensor trajectories between (i) a sensor position corresponding to a start of the predetermined path and (ii) a plurality of candidate sensor positions corresponding to an end of the predetermined path;

for each respective candidate sensor trajectory of the plurality of candidate sensor trajectories, determining a set of portions of the environment observable by the sensor along the respective candidate sensor trajectory; and

selecting a sensor trajectory from the plurality of candidate sensor trajectories based on (i) the determined set of portions of the environment associated with each of the plurality of candidate sensor trajectories and (ii) the at least one portion of the environment determined for each of the plurality of objectives.

11. The system of claim 10, wherein the control system is configured to determine the plurality of candidate sensor trajectories by:

determining a plurality of time points along the predetermined path, wherein each time point of the plurality of time points corresponds to a different position of the vehicle along the predetermined path;

for each time point of the plurality of time points, determining a plurality of candidate angular positions of the sensor based on (i) a maximum rotational velocity of the sensor and (ii) an angular position of the sensor at a prior time point; and

determining a plurality of sequences of the candidate angular positions of the sensor between (i) a time point corresponding to a start of the predetermined path and (ii) a time point corresponding to an end of the predetermined path, wherein the plurality of candidate sensor trajectories comprise the determined plurality of sequences of the candidate angular positions of the sensor.

12. The system of claim 10, wherein the control system is configured to determine a sensor trajectory through which to move the sensor with respect to the vehicle while the vehicle moves along the predetermined path by:

for each respective portion of the set of portions of the environment observable by the sensor along the respective candidate sensor trajectory, determining one or more candidate sensor parameters for scanning the respective portion of the set of portions of the environment; and

selecting a sensor trajectory from the plurality of candidate sensor trajectories based on (i) the determined set of portions of the environment associated with each of the plurality of candidate sensor trajectories, (ii) the at least one portion of the environment determined for each of the plurality of objectives, and (iii) the determined one or more candidate sensor parameters for scanning each respective portion of the set of portions of the environment.

13. The system of claim 1, wherein the determined sensor trajectory comprises:

an angular position trajectory through which to move the sensor as the vehicle moves along the predetermined path; and

a velocity profile corresponding to the angular position trajectory indicating a velocity with which to move the sensor along the angular position trajectory.

14. The system of claim 1, wherein the plurality of objectives comprise:

detection of obstacles on the predetermined path;

localization of the vehicle within the environment; and

detection of objects of interests in portions of the environment adjacent to the predetermined path.

15. The system of claim 1, wherein:

the vehicle is an autonomous fork truck configured to move a pallet within the environment; and

the plurality of objectives comprise detection of obstacles in a specific area of the environment, wherein the autonomous fork truck is scheduled to move the pallet to the specific area of the environment.

16. The system of claim 1, wherein the control system is further configured to:

determine, based on the determined sensor trajectory, one or more modifications to the predetermined path to place the at least one portion of the environment determined for each of the plurality of objectives within a field of view of the sensor as the sensor moves through the determined sensor trajectory.

17. The system of claim 1, wherein the control system is further configured to:

determine one or more sensor parameters for scanning the portions of the environment corresponding to the determined sensor trajectory; and

based on the one or more determined sensor parameters, provide instructions to adjust parameters of the sensor as the sensor moves through the determined sensor trajectory and scans portions of the environment corresponding to the determined sensor trajectory.

18. A method comprising:

receiving a predetermined path for a vehicle to follow;

receiving a plurality of objectives for which to collect sensor data from a sensor connected to the vehicle as the vehicle moves along the predetermined path, wherein each of the plurality of objectives is associated with a corresponding set of sensor data, and wherein the sensor is configured to move with respect to the vehicle to control a portion of an environment visible to the sensor;

determining, for each of the plurality of objectives, at least one portion of the environment for the sensor to scan to acquire the corresponding set of sensor data;

determining, based on the at least one portion of the environment determined for each of the plurality of objectives, a sensor trajectory through which to move the sensor with respect to the vehicle while the vehicle moves along the predetermined path; and

causing the sensor to move through the determined sensor trajectory and scan portions of the environment corresponding to the determined sensor trajectory as the vehicle moves along the predetermined path.

19. The method of claim 18, further comprising:

determining a plurality of priority levels corresponding to the plurality of objectives; and

determining, based on (i) the at least one portion of the environment determined for each of the plurality of objectives and (ii) the determined plurality of priority levels, the sensor trajectory through which to move the sensor with respect to the vehicle while the vehicle moves along the predetermined path.

20. A non-transitory computer readable storage medium having stored thereon instructions that, when executed by a computing device, cause the computing device to perform operations comprising:

receiving a predetermined path for a vehicle to follow;

receiving a plurality of objectives for which to collect sensor data from a sensor connected to the vehicle as the vehicle moves along the predetermined path, wherein each of the plurality of objectives is associated with a corresponding set of sensor data, and wherein the sensor is configured to move with respect to the vehicle to control a portion of an environment visible to the sensor;

determining, for each of the plurality of objectives, at least one portion of the environment for the sensor to scan to acquire the corresponding set of sensor data;

determining, based on the at least one portion of the environment determined for each of the plurality of objectives, a sensor trajectory through which to move the sensor with respect to the vehicle while the vehicle moves along the predetermined path; and

causing the sensor to move through the determined sensor trajectory and scan portions of the environment corresponding to the determined sensor trajectory as the vehicle moves along the predetermined path.