Obstacle detection system providing context awareness

• US 9,472,109 B2
• Filed: 01/07/2014
• Issued: 10/18/2016
• Est. Priority Date: 01/07/2014
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Starr, Jeffrey W., Lamkin, Andrew F., Buster, Duke
• Primary Examiner(s): Tarcza, Thomas
• Assistant Examiner(s): Lee, Tyler J
• Application Number:
  US14/149,482
• Publication Number:
  US 20150194059A1
• Time in Patent Office: 1,015 Days
• Field of Search: 701/3, 701300-302
• US Class Current: 1/1
• CPC Class Codes: G01S 13/934 on airport surfaces, e.g. while taxiing
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G08G 5/0017 Arrangements for implementing traffic-related aircraft activities, e.g. arrangements for generating, displaying, acquiring or managing traffic information head-up displays G02B27/01; ground or aircraft-carrier-deck-installations B64F
• CPC Class Codes: G08G 5/0073 Surveillance aids scene image recognition G06V20/00
• CPC Class Codes: G08G 5/04 Anti-collision systems
• CPC Class Codes: G08G 5/065 Navigation or guidance aids, e.g. for taxiing or rolling

Abstract

In sonic examples, an obstacle detection system is configured to generate and display a graphical user interface (GUI) that includes an overhead image of an area in which a vehicle is positioned, a graphical representation of the vehicle, and graphical representations of one or more obstacles, The graphical representations of the one or more obstacles and vehicle can be arranged relative to the overhead image to indicate determined real-world positions of the one or more obstacles and vehicle, respectively, relative to other features shown in the overhead (e.g., airport structure or other buildings).

Claims

1. A method comprising:

determining, by a processor, a location of at least one obstacle proximate an aircraft;

determining, by the processor, a location of the aircraft;

generating, by the processor, a graphical user interface based on the location of the at least one obstacle and the location of the aircraft, the graphical user interface comprising a graphical representation of the aircraft and a graphical representation of the at least one obstacle overlaid on an overhead image of an area in which the aircraft and the at least one obstacle are positioned, wherein the overhead image is an image of a real-world environment in which the aircraft is positioned, and wherein generating the graphical user interface comprises spatially registering, by the processor, the graphical representation of the aircraft and the graphical representation of the at least one obstacle with the overhead image to indicate a position of the aircraft and the at least one obstacle relative to a feature shown in the overhead image; and

displaying, by the processor, the graphical user interface via a display device on the aircraft or on a device external to the aircraft.

2. The method of claim 1, wherein determining the location of the at least one obstacle comprises receiving, by the processor, sensor data indicative of the location of the at least one obstacle, and wherein spatially registering the graphical representation of the aircraft and the graphical representation of the at least one obstacle with the overhead image comprises geo-registering the graphical representation of the at least one obstacle to a portion of the overhead image based on the sensor data.

3. The method of claim 2, wherein receiving sensor data indicative of the location of at least one obstacle comprises receiving at least one of Global Positioning System (GPS) sensor data, radar sensor data, or data from a camera.

4. The method of claim 1, wherein determining the location of the aircraft comprises receiving, by the processor, sensor data indicative of the location of the aircraft, and wherein spatially registering the graphical representation of the aircraft and the graphical representation of the at least one obstacle with the overhead image comprises geo-registering the graphical representation of the aircraft to a portion of the overhead image based on the sensor data indicative of the location of the portion of the aircraft.

5. The method of claim 4, wherein the sensor data indicates the location of each of a plurality of different portions of the aircraft, and wherein geo-registering the graphical representation of the aircraft to the portion of the overhead image comprises geo-registering each portion of the plurality of different portions of the aircraft to respective portions of the overhead image based on the sensor data.

6. The method of claim 4, wherein receiving sensor data indicative of the location the aircraft comprises receiving at least one of Global Positioning System (GPS) sensor data or an inertial navigation system of the aircraft.

7. The method of claim 1, wherein the overhead image comprises a satellite image.

8. The method of claim 1, further comprising receiving, by the processor, sensor data indicative of a location of a user, wherein generating the graphical user interface comprises generating, based on the sensor data, the graphical user interface comprising a graphical representation of the user on the overhead image.

9. The method of claim 1, further comprising:

determining, by processor, a threat level of the at least one obstacle based on the location of the at least one obstacle; and

generating, by the processor, the graphical representation of the at least one obstacle to include a characteristic that represents the determined threat level of the at least one of the at least one obstacle or the area of unknown.

10. The method of claim 1, wherein determining the location of the at least one obstacle comprises receiving, by the processor, sensor data indicative of the location of the at least one obstacle or the aircraft, wherein spatially registering the graphical representation of the aircraft and the graphical representation of the at least one obstacle with the overhead image comprises:

determining global coordinates of at least a portion of the aircraft or the at least one obstacle based on the sensor data;

determining global coordinates of two or more reference points in the overhead image or a coordinate system of the overhead image; and

mapping the graphical representation of the aircraft or the graphical representation of the at least one obstacle to the overhead image by at least matching up the global coordinates of a respective one of the aircraft or the at least one obstacle to the overhead image based on the determined global coordinates of the two or more reference points in the overhead image or the coordinate system of the overhead image.

11. An aircraft system comprising:

a display; and

a processor configured to determine a location of at least one obstacle proximate an aircraft, determine a location of the aircraft, generate a graphical user interface based on the location of the at least one obstacle and the location of the aircraft, and display the graphical user interface via the display, the graphical user interface comprising a graphical representation of the aircraft and a graphical representation of the at least one obstacle overlaid on an overhead image of area in which the aircraft and the at least one obstacle are positioned, wherein the overhead image is an image of a real-world environment in which the aircraft is positioned, and wherein the processor is configured to generate the graphical user interface by at least spatially registering the graphical representation of the aircraft and the graphical representation of the at least one obstacle with the overhead image to indicate a position of the aircraft and the at least one obstacle relative to a feature shown in the overhead image.

12. The aircraft system of claim 11, further comprising at least one sensor configured to generate sensor data indicative of the location of the at least one obstacle, wherein the processor is configured to determine the location of the at least one obstacle based on the sensor data, and spatially register the graphical representation of the aircraft and the graphical representation of the at least one obstacle with the overhead image by at least geo-registering the graphical representation of the at least one obstacle to a portion of the overhead image based on the sensor data.

13. The aircraft system of claim 12, wherein e at least one sensor comprises at least one of a Global Positioning System (GPS) sensor, a radar sensor, or a camera.

14. The aircraft system of claim 11, further comprising at least one sensor configured to generate sensor data indicative of the location of the aircraft, and wherein the processor is configured to determine the location of the aircraft based on the sensor data, and spatially register the graphical representation of the aircraft and the graphical representation of the at least one obstacle with the overhead image by at least geo-registering the graphical representation of the aircraft to a portion of the overhead image based on the sensor data indicative of the location of the portion of the aircraft.

15. The aircraft system of claim 14, wherein the sensor data indicates the location of each of a plurality of different portions of the aircraft, and wherein the processor is configured to geo-register the graphical representation of the aircraft to the portion of the overhead image by at least geo-registering each portion of the plurality of different portions of the aircraft to respective portions of the overhead image based on the sensor data.

16. The aircraft system of claim 14, wherein the at least one sensor comprises at least one of Global Positioning System (GPS) sensor or an inertial navigation system of the aircraft.

17. The aircraft system of claim 11, further comprising at least one sensor configured to generate sensor data indicative of a location of a user, wherein the processor is configured to generate the graphical user interface by at least generating, based on the sensor data, the graphical user interface comprising a graphical representation of the user on the overhead image.

18. The aircraft system of claim 11, wherein the processor is further configured to determine a threat level of the at least one obstacle based on the location of the at least one obstacle, and generate the graphical representation of the at least one obstacle to include a characteristic that represents the determined threat level of the at least one of the at least one obstacle or the area of unknown.

19. The aircraft system of claim 11, wherein the processor is configured to determine the location of the at least one obstacle by at least receiving sensor data indicative of the location of the at least one obstacle or the aircraft, and wherein the processor is configured to spatially register the graphical representation of the aircraft and the graphical representation of the at least one obstacle with the overhead image by at least:

determining global coordinates of at least a portion of the aircraft or the at least one obstacle based on the sensor data;

determining global coordinates of two or more reference points in the overhead image or a coordinate system of the overhead image; and

mapping the graphical representation of the aircraft or the graphical representation of the at least one obstacle to the overhead image by at least matching up the global coordinates of a respective one of the aircraft or the at least one obstacle to the overhead image based on the determined global coordinates of the two or more reference points in the overhead image or the coordinate system of the overhead image.

20. An aircraft system comprising:

means for determining a location of at least one obstacle proximate an aircraft;

means for determining a location of the aircraft;

means for receiving an overhead image of area in which the aircraft and the at least one obstacle are positioned;

means for generating a graphical user interface based on the location of the at least one obstacle and the location of the aircraft, the graphical user interface comprising a graphical representation of the aircraft and a graphical representation of the at least one obstacle on the overhead image, wherein the overhead image is an image of a real-world environment in which the aircraft is positioned, and wherein the means for generating the graphical user interface comprises means for spatially registering the graphical representation of the aircraft and the graphical representation of the at least one obstacle with the overhead image to indicate a position of the aircraft and the at least one obstacle relative to a feature shown in the overhead image.