METHOD AND SYSTEM FOR ALIGNING A TAXI-ASSIST CAMERA

• US 20170334578A1
• Filed: 05/23/2016
• Published: 11/23/2017
• Est. Priority Date: 05/23/2016
• Status: Active Grant

• Current Assignee: Rosemount Aerospace, Inc. (Raytheon Technologies Corporation d/b/a RTX)
• Original Assignee: Rosemount Aerospace, Inc. (Raytheon Technologies Corporation d/b/a RTX)
• Inventors: Fleck, Travis W., Brown, Gerald L.
• Granted Patent: US 11,319,086 B2
• Time in Patent Office: Days
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: H04N 17/002 for television cameras

Abstract

Apparatus and associated methods relate to aligning a taxi-assist camera such that each image frame of real-time video that the camera generates has a standard presentation format. The taxi-assist camera is configured to be mounted on an aircraft and oriented such that each image frame includes both a specific feature of the aircraft and of nearby objects external to the aircraft. The specific feature of the aircraft is detected and a location within the image frame of the specific feature is determined. The determined location within the image frame is compared with a reference location. A transformation operator is generated to transform the image frame such that the specific feature of the aircraft will be located within the image at a location corresponding to the reference location. The transformation operator is then applied to each of the image frames of the real-time video that the camera generates.

Claims

1. An alignment system comprising:

a taxi-assist camera configured to be mounted to an aircraft and oriented to provide real-time video during taxi operations of both a specific feature(s) of the aircraft and of nearby objects external to the aircraft, the real-time video comprising a time sequence of image frames, each image frame including a two-dimensional array of pixel data;

a feature detector configured to detect the specific feature(s) of the aircraft within at least one of the image frames;

a feature comparator configured to compare the detected specific feature(s) within the at least one of the image frames with a reference feature within a reference image; and

an image transformation operator configured to transform each of the image frames of the real-time video into a transformed image frame such that the detected specific feature(s) is located at a two-dimensional location within each of the transformed image frames, the two-dimensional location within each of the transformed image frames corresponding to a two-dimensional reference location corresponding to the reference feature within the reference image.

2. The alignment system of claim 1, wherein the feature detector comprises:

an edge detector configured to generate at least one edge image corresponding to the at least one of the image frames, the edge image comprising a two-dimensional array of pixel data, each of the pixel data being a function of a difference between pixels located within a distance of threshold of one another.

3. The alignment system of claim 2, wherein the edge detector comprises:

a Sobel filter to measure a two-dimensional spatial gradient of the at least one image frame.

4. The alignment system of claim 1, wherein the feature detector comprises:

a noise reducer configured to zero the pixel data of the at least one edge image for pixels that have an amplitude that is less than a threshold.

5. The alignment system of claim 4, wherein the noise reducer is further configured to adaptively set the threshold so that fewer than a predetermined number of pixels have amplitudes that are not less than the threshold, and thereby are not zeroed.

6. The alignment system of claim 1, wherein the feature detector is further configured to detect a first feature having a first orientation within the two-dimensional array of pixel data, and a second feature having a second orientation not parallel to the first orientation within the two-dimensional array of pixel data.

7. The alignment system of claim 6, wherein the feature comparator is further configured to compare the first feature within the at least one of the image frames with a first reference feature within the reference image, and to compare the second feature within the at least one of the image frames with a second reference feature within the reference image.

8. The alignment system of claim 7, wherein the image transformation operator is an image translation operator, the image translation operator configured to translate each of the image frames of the real-time video both horizontally and vertically so as to translate both the first and second image features to locations within the image frames corresponding to the first and second reference features, respectively.

9. The alignment system of claim 7, wherein the image transformation operator is an image translation and rotation operator, the image translation and rotation operator configured to translate both horizontally and vertically and rotate each of the image frames of the real-time video so as to transform both the first and second image features to locations within the image frames corresponding to the first and second reference features, respectively.

10. The alignment system of claim 7, wherein the image transformation operator is an image translation, rotation, and shear operator, the image translation, rotation and shear operator configured to translate both horizontally and vertically, rotate and shear each of the image frames of the real-time video so as to transform both the first and second image features to locations within the image frames corresponding to the first and second reference features, respectively.

11. The alignment system of claim 1, wherein the feature detector is configured to only detect features located within a two-dimensional subarray of pixel data.

12. The alignment system of claim 1, wherein the feature(s) comprise the landing gear of the aircraft.

13. The alignment system of claim 1, wherein the feature(s) comprise the wing of the aircraft.

14. The alignment system of claim 11, further comprising an image cropper configured to crop the transformed image frames of the real-time video so as make a standard presentation of the image frames, the standard presentation having the feature(s) located in a standard location with respect to edges of the image frames.

15. The alignment system of claim 2, wherein the edge detector is further configured to reduce a thickness of edge image features to less than a predetermined thickness.

17. The method of claim 15, wherein generating a transformation operator comprises:

calculating a translation operation.

18. The method of claim 15, wherein generating a transformation operator comprises:

calculating a rotation operation.

19. The method of claim 15, wherein generating a transformation operator comprises:

calculating a shear operation.

20. The method of claim 15, further comprising:

thinning the calculated edges that have a thickness greater than a threshold number of pixel widths.

16. A method for aligning a taxi-assist camera of an aircraft to a standard view, the method comprising the steps of:

obtaining a first image from a camera mounted on an aircraft, the first image comprising a two-dimensional array of pixel intensity data;

calculating edge intensity data using the obtained first image, wherein the calculating uses a function of differences between pixel intensity data of pixels within a local vicinity;

creating a second image comprising a two-dimensional array of the calculated edge intensity data;

thresholding the second image to zero the magnitude of edge intensity data that are below a predetermined threshold, while edges intensity data that are above the predetermined threshold survive;

selecting, from the surviving edges, a first target edge oriented in a first orientation and a second target edge oriented in a second orientation, the first and second orientations providing a basis that spans the two-dimensional array of edge intensity data;

calculating a first offset between the selected first edge and a first reference location and a second offset between the selected second edge and a second reference location;

generating a transformation operator that transforms the second image such that the selected first edge is located at the first reference location and the selected second edge is located at the second reference location; and

applying the generated transformation operator in real-time to imagery obtained by the camera mounted on the aircraft.