Vision based zero velocity and zero attitude rate update

• US 8,948,446 B2
• Filed: 01/19/2011
• Issued: 02/03/2015
• Est. Priority Date: 01/19/2011
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Susca, Sara, Talasila, Viswanath, Rao, Shrikant
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): BROOKS, JULIAN D
• Application Number:
  US13/009,368
• Publication Number:
  US 20120183179A1
• Time in Patent Office: 1,476 Days
• Field of Search: 382/103, 382/107, 382/190.278, 702/150, 701/28
• US Class Current: 382/103
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01C 23/005 Flight directors indicating arrangements specially adapted for rotary gyroscopes G01C19/32
• CPC Class Codes: G01S 19/485 whereby the further system is an optical system or imaging system

Abstract

A method comprises receiving a first frame from at least one imaging device, receiving a second frame from the at least one imaging device, analyzing at least a portion of the first frame and at least a portion of the second frame, and indicating when at least one of a zero velocity update and a zero attitude update should be performed based on at least in part on the analysis of the at least a portion of the first frame and the at least a portion of the second frame. The first frame is captured at a first vantage point and the second frame is captured at a second vantage point.

Claims

1. A method comprising:

receiving a first frame from at least one imaging device, the first frame captured at a first vantage point;

receiving a second frame from the at least one imaging device, the second frame captured at a second vantage point;

analyzing at least a portion of the first frame and at least a portion of the second frame; and

indicating when at least one of a zero velocity update and a zero attitude rate update should be performed based at least in part on the analysis of the at least a portion of the first frame and the at least a portion of the second frame.

2. The method of claim 1, wherein indicating when at least one of the zero velocity update and the zero attitude rate update should be performed includes outputting at least one of:

a first signal representing a velocity of zero to a velocity input of an estimator; and

a second signal representing an attitude rate of zero to an attitude rate input of an estimator, wherein the estimator estimates position using an input from an inertial measurement unit.

3. The method of claim 1, wherein analyzing at least a portion of the first frame and at least a portion of the second frame comprises:

extracting features from the first frame and the second frame; and

matching one or more features from the first frame with respective one or more corresponding features of the second frame.

4. The method of claim 3, wherein analyzing at least a portion of the first frame and at least a portion of the second frame further comprises, for each of the one or more matched features, calculating a respective distance between the respective feature in the first frame and the respective corresponding feature in the second frame and flagging that feature as stationary if the distance is less than a threshold;

and

wherein indicating when at least one of a zero velocity update and a zero attitude rate update should be performed comprises indicating that at least one of a zero velocity update and a zero attitude rate update should be performed when more than a threshold amount of the one or more matched features are flagged as stationary.

5. The method of claim 3, wherein analyzing at least a portion of the first frame and at least a portion of the second frame further comprises, for each of the one or more matched features, performing an intensity correlation to generate a respective set of correlation numbers for that matched feature that comprises a zero-shifted correlation number and a plurality of shifted correlation numbers;

and

wherein indicating when at least one of the zero velocity update and the zero attitude rate update should be performed comprises indicating that at least one of the zero velocity update and the zero attitude rate update should be performed when no correlation number sets for any of the one or more matched features have a pattern of increasing correlation along any direction from any zero-shifted correlation number.

6. The method of claim 5, further comprising, for each of the one or more matched features, extracting pixel intensities for a plurality of pixels from the first frame and the second frame;

and

wherein the intensity correlation that is performed for each of the one or more matched features comprises;

(a) calculating the respective zero-shifted correlation number in the respective set of correlation numbers for the respective matched feature by;

(i) positioning a respective first window at a respective first position in the first frame, wherein the respective first window covers a respective first subset of the respective first plurality of pixels;

(ii) positioning a respective second window at the respective first position in the second frame, wherein the respective second window covers a respective first subset of the respective second plurality of pixels, wherein each pixel in the respective first window matches a pixel at an identical relative pixel position in the second window;

(iii) multiplying the pixel intensity of each pixel in the respective first window by the pixel intensity of each corresponding pixel in an identical relative pixel position in the respective second window to create a first plurality of pixel intensity products; and

(iv) summing each of the first plurality of pixel intensity products into the respective zero-shifted correlation number; and

(b) calculating each of the respective plurality of shifted correlation numbers in the respective set of correlation numbers for the respective matched feature by;

(i) positioning the first window at a respective new position in the first frame, wherein the respective first window covers a respective second subset of the respective first plurality of pixels;

(ii) positioning the respective second window at the respective first position in the second frame, wherein the respective second window covers the respective first subset of the respective second plurality of pixels;

(iii) multiplying the pixel intensity of each pixel in the respective first window by the pixel intensity of each corresponding pixel in the identical relative pixel position in the respective second window to create a second plurality of pixel intensity products; and

(iv) summing each of the respective second plurality of pixel intensity products into the respective shifted correlation number and storing the respective shifted correlation number; and

wherein indicating when at least one of the zero velocity update and the zero attitude rate update should be performed comprises indicating that at least one of the zero velocity update and the zero attitude rate update should be performed when the plurality of shifted correlation numbers indicates a pattern of increasing correlation away from the zero-shifted correlation number.

7. The method of claim 1, wherein analyzing at least a portion of the first frame and at least a portion of the second frame comprises performing an intensity correlation to generate a set of correlation numbers that comprises a zero-shifted correlation number and a plurality of shifted correlation numbers;

and

wherein indicating when at least one of the zero velocity update and the zero attitude rate update should be performed comprises indicating that at least one of the zero velocity update and the zero attitude rate update should be performed when the correlation number set does not have a pattern of increasing correlation along any direction from any zero-shifted correlation number.

8. A system comprising:

at least one imaging device;

at least one processor communicatively coupled to the at least one imaging device; and

wherein the at least one processor is configured to;

receive a first frame from the at least one imaging device;

receive a second frame from the at least one imaging device;

analyze at least a portion of the first frame and at least a portion of the second frame; and

indicate when at least one of a zero velocity update and a zero attitude rate update should be performed based at least in part on the analysis of the at least a portion of the first frame and the at least a portion of the second frame.

9. The system of claim 8, wherein the processor is configured to indicate that at least one of the zero velocity update and the zero attitude rate update should be performed by outputting at least one of:

a first signal representing a velocity of zero to a velocity input of an estimator; and

a second signal representing an attitude rate of zero to an attitude rate input of an estimator, wherein the estimator estimates position using input from an inertial measurement unit.

10. The system of claim 9, wherein the estimator comprises a Kalman filter.

11. The system of claim 8, wherein the processor is configured to do at least the following in order to analyze at least a portion of the first frame and at least a portion of the second frame:

extract features from the first frame and the second frame; and

match one or more features from the first frame with respective one or more corresponding features of the second frame.

12. The system of claim 11, wherein each feature is represented by a single pixel.

13. The system of claim 11, wherein the processor is further configured to do at least the following in order to analyze at least a portion of the first frame and at least a portion of the second frame, for each of the one or more matched features:

calculate a respective distance between the respective feature in the first frame and the respective corresponding feature in the second frame and flag that feature as stationary if the distance is less than a threshold; and

wherein the processor is configured to indicate that at least one of the zero velocity update and the zero attitude rate update should be performed when more than a threshold amount of the one or more matched features are flagged as stationary.

14. The system of claim 11, wherein the processor is further configured to do at least the following in order to analyze at least a portion of the first frame and at least a portion of the second frame, for each of the one or more matched features:

perform an intensity correlation to generate a respective set of correlation numbers for the respective matched feature that comprises a zero-shifted correlation number and a plurality of shifted correlation numbers; and

wherein the processor is configured to indicate that at least one of the zero velocity update and the zero attitude rate update should be performed when no correlation number sets for any of the one or more matched features have a pattern of increasing correlation along any direction from any zero-shifted correlation number.

15. The system of claim 14, wherein the processor is further configured to, for each of the one or more matched features, extract pixel intensities for a plurality of pixels from the first frame and the second frame;

wherein the processor is further configured to do at least the following in order to analyze at least a portion of the first frame and at least a portion of the second frame;

(a) calculate the respective zero-shifted correlation number in the respective set of correlation numbers for the respective matched feature by;

(i) positioning a respective first window at a respective first position in the first frame, wherein the respective first window covers a respective first subset of the respective first plurality of pixels;

(ii) positioning a respective second window at the respective first position in the second frame, wherein the respective second window covers a respective first subset of the respective second plurality of pixels, wherein each pixel in the respective first window matches a pixel at an identical relative pixel position in the second window;

(iii) multiplying the pixel intensity of each pixel in the respective first window by the pixel intensity of each corresponding pixel in an identical relative pixel position in the respective second window to create a first plurality of pixel intensity products; and

(iv) summing each of the first plurality of pixel intensity products into the respective zero-shifted correlation number;

(b) calculate each of the respective plurality of shifted correlation numbers in the respective set of correlation numbers for the respective matched feature by;

(i) positioning the first window at a respective new position in the first frame, wherein the respective first window covers a respective second subset of the respective first plurality of pixels;

(ii) positioning the respective second window at the respective first position in the second frame, wherein the respective second window covers the respective first subset of the respective second plurality of pixels;

(iii) multiplying the pixel intensity of each pixel in the respective first window by the pixel intensity of each corresponding pixel in the identical relative pixel position in the respective second window to create a second plurality of pixel intensity products; and

(iv) summing each of the respective second plurality of pixel intensity products into the respective shifted correlation number and storing the respective shifted correlation number; and

wherein the processor is configured to indicate that at least one of the zero velocity update and the zero attitude rate update should be performed when the plurality of shifted correlation numbers indicates a pattern of increasing correlation away from the zero-shifted correlation number.

16. The system of claim 8, wherein the processor is configured to do at least the following in order to analyze at least a portion of the first frame and at least a portion of the second frame:

perform an intensity correlation to generate a set of correlation numbers that comprises a zero-shifted correlation number and a plurality of shifted correlation numbers; and

wherein the processor is configured to indicate that at least one of the zero velocity update and the zero attitude rate update should be performed when the correlation number set does not have a pattern of increasing correlation along any direction from any zero-shifted correlation number.

17. The system of claim 8 further comprising a global positioning system receiver communicatively coupled to the at least one processor, wherein the processor receives current absolute position data from the global positioning receiver when available.

18. A program-product for triggering at least one of a vision based zero velocity update and a vision based zero attitude rate update at a system comprising at least one imaging device, the program-product comprising a non-transitory processor-readable storage medium on which program instructions are embodied, wherein the program instructions are operable to, when executed by at least one processor included in the system, cause the system to:

receive a first frame from the at least one imaging device;

receive a second frame from the at least one imaging device;

analyze at least a portion of the first frame and at least a portion of the second frame; and

indicate when at least one of a zero velocity update and a zero attitude rate update should be performed based at least in part on the analysis of the at least a portion of the first frame and the at least a portion of the second frame.

19. The program product of claim 18, wherein the program instructions are operable to, when executed by the at least one processor included in the system, cause the system to analyze at least a portion of the first frame and at least a portion of the second frame by doing at least the following:

extracting features from the first frame and the second frame;

matching one or more features from the first frame with respective one or more corresponding features of the second frame; and

for each of the one or more matched features, calculating a respective distance between the respective feature in the first frame and the respective corresponding feature in the second frame and flagging that feature as stationary if the distance is less than a threshold; and

wherein the program instructions are operable to, when executed by the at least one processor included in the system, cause the system to indicate that at least one of the zero velocity update and the zero attitude rate update should performed when more than a threshold amount of the one or more matched features are flagged as stationary.

20. The program product of claim 18, wherein the program instructions are operable to, when executed by the at least one processor included in the system, cause the system to analyze at least a portion of the first frame and at least a portion of the second frame by doing at least the following:

performing an intensity correlation to generate a set of correlation numbers that comprises a zero-shifted correlation number and a plurality of shifted correlation numbers; and

wherein the program instructions are operable to, when executed by the at least one processor included in the system, cause the system to indicate that at least one of the zero velocity update and the zero attitude rate update should be performed when the correlation number set does not have a pattern of increasing correlation along any direction from any zero-shifted correlation number.