Tracking objects within a dynamic environment for improved localization

• US 9,707,961 B1
• Filed: 01/29/2016
• Issued: 07/18/2017
• Est. Priority Date: 01/29/2016
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Varnhagen, Scott, Halder, Bibhrajit
• Primary Examiner(s): Lee, Tyler J
• Application Number:
  US15/010,303
• Publication Number:
  US 20170217433A1
• Time in Patent Office: 536 Days
• Field of Search: 701 23, 701 26, 701 28, 701 36, 701300-302
• CPC Class Codes: B60W 2050/0075 Automatic parameter input, automatic initialising or calibrating means
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4041 Position
• CPC Class Codes: B60W 2554/4044 Direction of movement, e.g. backwards
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2556/10 Historical data
• CPC Class Codes: B60W 30/08 Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
• CPC Class Codes: B60W 30/0953 the prediction being responsive to vehicle dynamic parameters
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: B60W 50/0097 Predicting future conditions
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0257 using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30

Abstract

The present invention extends to methods, systems, and computer program products for tracking objects within a dynamic environment for improved localization. Sensing devices are utilized to gather data about a vehicle'"'"'s environment. In cases where the sensor data has become degraded, such as data indicating that lane lines have become degraded, obscured, or nonexistent, the vehicle computer system uses previously detected sensor data to estimate the speed and direction of travel of moving objects. The computer system then estimates the location of the moving objects after a specified period of time based on the estimated speed and direction of the moving object. The computer system utilizes this information to localize the vehicle within the dynamic environment and to control the configuration of the vehicle.

Claims

1. A method for tracking objects within a dynamic environment for improved localization, the method comprising:

detecting that sensor data for objects within the dynamic environment has degraded;

estimating a speed and direction of travel for moving objects from previously detected sensor data;

estimating a location of moving objects after a specified period of time, including for each moving object calculating a new position of the moving object based on an initial position and an initial velocity of the moving object, and the specified period of time;

localizing a vehicle within the dynamic environment; and

using the localization to control a configuration of the vehicle.

2. The method of claim 1, wherein detecting that sensor data for objects within the dynamic environment has degraded comprises detecting that lane lines on a roadway have become one or more of:

degraded, obscured, or nonexistent.

3. The method of claim 1, wherein using the localization to control a configuration of the vehicle comprises using localization to control one or more of:

acceleration, speed, or direction for the vehicle.

4. The method of claim 1, wherein localizing a vehicle within the dynamic environment comprises localizing the vehicle within the dynamic environment within a specified confidence interval.

5. The method of claim 1, wherein localizing the vehicle within the dynamic environment comprises calculating the configuration of the vehicle to maintain safe operation of the vehicle.

6. The method of claim 1, wherein using the localization to control the configuration of the vehicle comprises utilizing the vehicle control system to place the vehicle in a safe configuration.

7. A method for use at a vehicle computer system, the computer system including one or more processors and system memory, the method for tracking objects within a dynamic environment for improved localization, the method comprising the processor:

detecting that sensor data for objects within the dynamic environment has degraded, the sensor data having been gathered by a plurality of sensors at the vehicle, the sensor data indicating the configuration of objects within the dynamic environment, the objects including one or more static objects and one or more dynamic objects;

in response to detecting that the sensor data has become degraded, for each of the one or more dynamic objects;

estimating a speed and direction of travel for the dynamic object from previously detected sensor data; and

estimating a location of the dynamic object after a specified period of time, including calculating a new position for the dynamic object based on an initial position and an initial velocity of the dynamic object, and the specified period of time;

using a localization to control a configuration of the vehicle within the dynamic environment.

8. The method of claim 7, further comprising obtaining the sensor data from one or more sensors, the one or more sensors selected from among:

a camera, a global positioning systems (GPS), a LIDAR, a radar, an ultrasonic sensor, an infrared sensor, and a inertial measurement unit (IMU).

9. The method of claim 7, wherein detecting that sensor data for objects within the dynamic environment has degraded comprises detecting that lane lines on a roadway have become one or more of:

degraded, obscured, or nonexistent.

10. The method of claim 7, wherein using a localization to control the configuration of the vehicle within the dynamic environment comprises localizing the vehicle within the dynamic environment within a specified confidence interval.

11. The method of claim 7, wherein using a localization to control the configuration of the vehicle within the dynamic environment comprises indicating that there was a fault in using the localization to control the configuration of the vehicle.

12. The method of claim 7, wherein estimating a speed and direction of travel for the dynamic object from previously detected sensor data comprises identifying an object that is traveling in essentially the same direction as the vehicle.

13. The method of claim 7, wherein estimating a speed and direction of travel for the dynamic object from previously detected sensor data comprises identifying an object that is traveling in essentially the opposite direction as the vehicle.

14. The method of claim 7, further comprising, creating a map of the dynamic environment of the vehicle wherein the map is based on at least one of:

a lane marking on the road, a geographic location of the vehicle, and a predetermined map of the road.

15. The method of claim 14, further comprising localizing the vehicle within the dynamic environment based on the estimated locations for the one or more dynamic objects, including calculating a configuration for the vehicle to maintain safe autonomous operations in a roadway environment.

16. A computer system, the computer system comprising:

one or more processors;

system memory coupled to the one or more processors, the system memory storing instructions that are executable by the one or more processors; and

the one or more processors executing the instructions stored in the system memory to track objects within a dynamic environment, including the following;

detect that sensor data for objects within the dynamic environment has degraded;

estimate a speed and direction of travel for moving objects from previously detected sensor data;

estimate a location of moving objects after a specified period of time;

localize a vehicle within the dynamic environment within a specified confidence interval; and

use the localization to control a configuration of the vehicle.

17. The computer system of claim 16, further comprising the one or more processors executing the instructions stored in the system memory to obtain the sensor data for the objects within the dynamic environment from one or more sensors, the one or more sensors selected from among:

a camera, a global positioning systems (GPS), a LIDAR, a radar, an ultrasonic sensor, an infrared sensor, and a inertial measurement unit (IMU).

18. The computer system of claim 16, wherein the one or more processors executing the instructions stored in the system memory to use the localization to control a configuration of the vehicle comprises the one or more processors executing the instructions stored in the system memory to indicate that there was a fault in using the localization to control the configuration of the vehicle within the dynamic environment.

19. The computer system of claim 16, wherein the one or more processors executing the instructions stored in the system memory to use the localization to estimate a speed and direction of travel for moving objects from previously detected sensor data comprises the one or more processors executing the instructions stored in the system memory to identify an object that is traveling in essentially the opposite direction as the vehicle.

20. The computer system of claim 16, wherein the one or more processors executing the instructions stored in the system memory to estimate a location of moving objects after a specified period of time comprises the one or more processors executing the instructions stored in the system memory to, for each moving object, calculate a new position of the moving object based on an initial position and an initial velocity of the moving object, and the specified period of time.