System and method for onboard vision processing

• US 8,855,846 B2
• Filed: 10/20/2006
• Issued: 10/07/2014
• Est. Priority Date: 10/20/2005
• Status: Expired due to Fees

• Current Assignee: Prioria Robotics
• Original Assignee: Prioria Robotics
• Inventors: Grzywna, Jason W.
• Primary Examiner(s): Nguyen, John Q
• Assistant Examiner(s): Fei, Jordan S
• Application Number:
  US11/584,448
• Publication Number:
  US 20070093945A1
• Time in Patent Office: 2,909 Days
• Field of Search: 701/2, 701/23, 701/24, 701/25, 701/28, 701/4, 340/990, 340/995.25, 244/3.11, 244/3.14, 244/3.16, 244/3.17, 244/76.R, 244/189, 244/190, 382/104
• US Class Current: 701/23
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G05D 1/106 Change initiated in response to external conditions, e.g. avoidance of elevated terrain or of no-fly zones

Abstract

An unmanned aerial vehicle with a camera and conventional sensors, where the processor navigates the vehicle based at least in part on the image data and the sensor data. A method for navigating an unmanned aerial vehicle where a processor navigates the vehicle based at least in part on image data corrected by traditional sensor data.

Claims

1. An unmanned aerial vehicle (UAV) comprising:

a sensor;

a camera;

wings having a wingspan of less than twelve feet;

a plurality of actuators;

a plurality of control surfaces;

a digital signal processor (DSP) vision processor;

a field-programmable gate array (FPGA) device comprising a softcore decision-making processor;

a frame buffer coupled to the DSP vision processor and the FPGA device;

means for using the FPGA device to synchronously capture image data from the camera and sensor data from the sensor, wherein the captured image data represents a sequence of images of the UAV'"'"'s operating environment, and wherein the sensor data represents a current physical state of the UAV;

means for using the FPGA to pre-process the captured image data to produce pre-processed image data; and

means for buffering the pre-processed image data in the frame buffer;

wherein the DSP vision processor comprises means for performing targeting on the pre-processed image data to produce targeting output and performing geo-location on the targeting output to produce geo-location output;

wherein the FPGA device comprises means for processing the targeting output to perform target tracking by providing signals to the plurality of actuators;

wherein the plurality of actuators comprises means for physically moving the plurality of control surfaces of the UAV in response to the signals; and

wherein the means for processing the targeting output comprises means for processing the targeting output and the geo-location output to perform target tracking by providing signals to the plurality of actuators.

2. The UAV of claim 1, further comprising a gimbal, and wherein the plurality of actuators further comprises means for physically moving the gimbal in response to the signals.

3. The UAV of claim 1, wherein the sensor data comprises data from at least one of an accelerometer, an inclinometer, an altimeter, a gyroscope, a pressure sensor, a temperature sensor, and a GPS receiver.

4. The UAV of claim 1, wherein the image data comprises data from the non-visible spectrum.

5. The UAV of claim 1, wherein the image data comprises data from the ultra-violet spectrum.

6. The UAV of claim 1, wherein the image data comprises data from the infrared spectrum.

7. The UAV of claim 1, further comprising:

means for recording the sensor data and the camera data on-board the unmanned aerial vehicle during a state in which the unmanned aerial vehicle is out-of-range of a remote control station; and

means for transmitting the recorded sensor data and camera data with the remote control station when the unmanned aerial vehicle becomes in-range of the remote control station.