Vehicle with high integrity perception system

• US 9,188,980 B2
• Filed: 09/11/2008
• Issued: 11/17/2015
• Est. Priority Date: 09/11/2008
• Status: Active Grant

• Current Assignee: Deere & Company
• Original Assignee: Deere & Company
• Inventors: Anderson, Noel Wayne
• Primary Examiner(s): MANCHO, RONNIE M
• Application Number:
  US12/208,851
• Publication Number:
  US 20100063672A1
• Time in Patent Office: 2,623 Days
• Field of Search: 701/25, 701/400, 701/408, 701/409, 701/454, 701/461, 701/462, 701/29.1, 701/29.2, 701/29.7, 701/450, 701/463, 701/532, 700/245, 455/456.1, 455/411
• US Class Current: 1/1
• CPC Class Codes: G05D 1/0077 using redundant signals or controls
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0278 using satellite positioning signals, e.g. GPS

Abstract

The illustrative embodiments provide an apparatus for processing sensor data and controlling the movement of a vehicle. In an illustrative embodiment, an operating environment around the vehicle is identified and sensor data is selected from a set of sensors. The movement of the vehicle is controlled based on the operating environment identified. In another illustrative embodiment, a sensor system has some sensors that are more accurate in a particular environment than other sensors. A dynamic condition is identified by the sensor system and commands are sent to the steering system, the propulsion system, and the braking system to move the vehicle using the sensor data detected by the sensor system. The environment is identified using the plurality of different types of sensors on the vehicle.

Claims

1. A vehicle comprising:

a machine controller;

a steering system;

a propulsion system;

a braking system;

a sensor system having a plurality of redundant sensors; and

wherein the machine controller is connected to the steering system, the propulsion system, the braking system; and

the sensor system, wherein the machine controller identifies an environment around the vehicle to form an operating environment and selects data from a set of sensors within a plurality of redundant sensors in the sensor system based on the operating environment and sends commands to the steering system, the propulsion system, and the braking system to control movement of the vehicle using sensor data, wherein the machine controller identifies the environment by obtaining the environment around the vehicle from a knowledge base.

2. A vehicle comprising:

a machine controller;

a steering system;

a propulsion system;

a braking system;

a sensor system having a plurality of redundant sensors; and

wherein the machine controller is connected to the steering system, the propulsion system, the braking system; and

the sensor system, wherein the machine controller identifies an environment around the vehicle to form an operating environment and selects data from a set of sensors within a plurality of redundant sensors in the sensor system based on the operating environment and sends commands to the steering system, the propulsion system, and the braking system to control movement of the vehicle using sensor data, wherein the machine controller identifies weather conditions associated with the environment around the vehicle to form the operating environment and selects data from the set of sensors in the sensor system based on the weather conditions identified.

3. The vehicle of claim 2, wherein the machine controller identifies the weather conditions associated with the environment by obtaining information regarding weather patterns for a present time in the environment around the vehicle from a knowledge base.

4. The vehicle of claim 2, wherein the machine controller identifies the weather conditions associated with the environment by obtaining data regarding current weather conditions in the environment around the vehicle from one or more of the plurality of redundant sensors on the vehicle.

5. A vehicle comprising:

a machine controller;

a steering system;

a propulsion system;

a braking system;

a sensor system having a plurality of different types of sensors, wherein some types of sensors are more accurate that other types of sensors within the plurality of different types of sensors in a particular environment; and

wherein the machine controller is connected to the steering system, the propulsion system, the braking system; and

the sensor system, wherein the machine controller identifies a dynamic condition using the sensor system and sends commands to the steering system, the propulsion system, and the braking system to move the vehicle using sensor data detected by the sensor system, and wherein the machine controller identifies an environment using the plurality of different types of sensors on the vehicle, wherein the machine controller receives data from the plurality of different types of sensors for the vehicle to form received data and generates a thematic map using the received data and a knowledge base.

6. The vehicle of claim 5, wherein the machine controller identifies a location of the vehicle, identifies a map based on the location of the vehicle, and places thematic features into the map using the data received and the knowledge base.

7. The vehicle of claim 6, wherein the machine controller moves the vehicle using the thematic map.

8. The vehicle of claim 5, wherein the knowledge base further comprises an online knowledge base, an a priori knowledge base, and a learned knowledge base.

9. The vehicle of claim 8, wherein the machine controller receives sensor data from the plurality of sensors for the vehicle and applies weights to the sensor data using the online knowledge base to form weighted sensor data.

10. The vehicle of claim 8, wherein the online knowledge base is located at a remote location and accessible to control the vehicle.

11. The vehicle of claim 9, wherein the machine controller selects a portion of the weighted sensor data to form selected sensor data and performs localization using the selected sensor data.

12. The vehicle of claim 11, wherein the machine controller identifies a set of objects in an environment around the vehicle using the selected sensor data and identifies a location of the vehicle using the location of the set of objects.

13. The vehicle of claim 12, wherein the machine controller identifies the set of objects using the sensor data and the knowledge base.

14. The vehicle of claim 12, wherein the knowledge base specifies the set of objects based on the environment.

15. The vehicle of claim 5, wherein the machine controller identifies a set of differences between an environment as detected by the plurality of sensors for the vehicle and a description of the environment in the knowledge base.

16. The vehicle of claim 15, wherein the set of differences comprises at least one of a change in an object in the environment, a new object in the environment, and a missing object from the environment.

17. A vehicle comprising:

a machine controller;

a steering system;

a propulsion system;

a braking system;

a sensor system having a plurality of different types of sensors, wherein some types of sensors are more accurate than other types of sensors within the plurality of different types of sensors in a particular environment; and

wherein the machine controller is connected to the steering system, the propulsion system, the braking system; and

the sensor system, wherein the machine controller identifies a dynamic condition using the sensor system and sends commands to the steering system, the propulsion system, and the braking system to move the vehicle using sensor data detected by the sensor system, and wherein the machine controller identifies an environment using the plurality of different types of sensors on the vehicle, wherein the machine controller monitors the plurality of different types of sensors for sensor integrity, wherein the machine controller identifies a fault in at least one sensor in the plurality of different types of sensors.

18. The vehicle of claim 17, wherein the fault in at least one sensor is sensor degradation.

19. The vehicle of claim 17, wherein the fault in at least one sensor is sensor failure.

20. The vehicle of claim 17, wherein the machine controller, responsive to identifying the fault in at least one sensor in the plurality of different types of sensors, generates an alert displaying the fault in at least one sensor in the plurality of different types of sensors.

21. The vehicle of claim 17, wherein the machine controller, responsive to identifying the fault in at least one sensor in the plurality of different types of sensors, selects a different sensor to compensate for the fault in at least one sensor.

22. The vehicle of claim 17, wherein the machine controller, responsive to identifying the fault in at least one sensor in the plurality of different types of sensors, transmits a request for sensor data to a sensor system of another vehicle.

23. The vehicle of claim 22, wherein the machine controller receives the sensor data from the sensor system of another vehicle and performs localization based on the sensor data from the sensor system of another vehicle.

24. A vehicle comprising:

a machine controller;

a steering system;

a propulsion system;

a braking system;

a sensor system; and

a knowledge base used by the machine controller, wherein the machine controller is connected to the steering system, the propulsion system, the braking system, and the sensor system; and

wherein the machine controller identifies an environment using a plurality of sensors on the vehicle, wherein the plurality of sensors comprises at least one of a global positioning system, structured light sensor, two dimensional/three dimensional lidar, dead reckoning, far and medium infrared camera, visible light camera, radar, ultrasonic sonar, and radio frequency identification reader;

identifies a location of the vehicle;

identifies a map based on the location of the vehicle;

identifies a set of objects in the environment using data received from the plurality of sensors and a knowledge base, wherein the knowledge base specifies the set of objects based on the environment;

places thematic features into the map using the set of objects identified and the map identified;

controls the vehicle using the knowledge base, wherein the knowledge base further comprises an online knowledge base, an a priori knowledge base, and a learned knowledge base;

monitors the plurality of sensors for sensor integrity;

identifies a fault in at least one sensor in the plurality of sensors, wherein the fault in at least one sensor is one of sensor degradation and sensor failure; and

responsive to identifying a fault in at least one sensor in the plurality of sensors, generates an alert displaying the fault in at least one sensor in the plurality of sensors, selects a different sensor to compensate for the fault in at least one sensor, transmits a request for sensor data to a sensor system of another vehicle, receives the sensor data from the sensor system of another vehicle, and performs localization based on the sensor data from the sensor system of another vehicle.