Camera Aided Motion Direction and Speed Estimation

• US 20140226864A1
• Filed: 02/14/2013
• Published: 08/14/2014
• Est. Priority Date: 02/14/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Venkatraman, Subramaniam, Kulik, Victor
• Granted Patent: US 9,330,471 B2
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: G01C 21/12 executed aboard the object being navigated; Dead reckoning
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

Methods and devices for camera aided motion direction and speed estimation are disclosed. The method of determining position characteristics of a mobile device comprises capturing a plurality of images that represent views from the mobile device, adjusting perspectives of the plurality of images based at least in part on an orientation of the mobile device, determining a misalignment angle with respect to a direction of motion of the mobile device using the plurality of images, and storing the misalignment angle and the direction of motion in a storage device.

Claims

1. A method of determining position characteristics of a mobile device, comprising:

capturing a plurality of images that represent views from the mobile device;

adjusting perspectives of the plurality of images based at least in part on an orientation of the mobile device;

determining a misalignment angle with respect to a direction of motion of the mobile device using the plurality of images; and

storing the misalignment angle and the direction of motion in a storage device.

2. The method of claim 1, wherein adjusting perspectives of the plurality of images comprises at least one of:

adjusting perspectives of the plurality of images based on the orientation of the mobile device calculated using data collected from one or more sensors;

compensating for perspectives of the plurality of images using an area near centers of the plurality of images; and

compensating for perspectives of the plurality of images based at least in part on a weighted average of locations of features in the plurality of images.

3. The method of claim 1, wherein determining the misalignment angle comprises:

tracking features from the plurality of images;

estimating direction of motion of the mobile device;

estimating the orientation of the mobile device using sensor data; and

determining the misalignment angle based at least in part on the direction of motion and the orientation of the mobile device.

4. The method of claim 3, wherein the tracking features from the plurality of images comprises:

rejecting outliers in features of the plurality of images to eliminate at least one moving object in the plurality of images.

5. The method of claim 1, further comprising at least one of:

determining a confidence of the misalignment angle with respect to the direction of motion of the mobile device using information provided by a gyroscope of the mobile device;

determining the confidence of the misalignment angle with respect to the direction of motion of the mobile device using information provided by a magnetometer of the mobile device; and

determining the confidence of the misalignment angle with respect to the direction of motion of the mobile device using features of the plurality of images.

6. The method of claim 1 further comprises:

determining a speed estimation of the mobile device;

determining a confidence of the speed estimation of the mobile device; and

applying the speed estimation and the confidence of the speed estimation to navigate a user of the mobile device.

7. The method of claim 6, wherein determining a speed estimate comprises:

extracting features from the plurality of images;

computing an average displacement of the mobile device using the features from the plurality of images; and

computing the speed estimate based at least in part on the average displacement of the mobile device.

8. The method of claim 7, wherein computing the speed estimate comprises:

comparing the features from the plurality of images;

determining a separation of pixels between two consecutive images in the plurality of images;

determining a time interval between the two consecutive images; and

calculating the speed estimate of the mobile device in accordance with the separation of pixels and the time interval between the two consecutive images.

9. The method of claim 8 further comprises:

calibrating a height of the mobile device according to at least one of GPS location information and WIFI location information of the mobile device.

10. The method of claim 1 further comprises:

applying the misalignment angle and a confidence of the misalignment angle to navigate a user of the mobile device.

11. An apparatus, comprising:

a control unit including processing logic, the processing logic comprising;

logic configured to capture a plurality of images that represent views from a mobile device;

logic configured to adjust perspectives of the plurality of images based at least in part on an orientation of the mobile device;

logic configured to determine a misalignment angle with respect to a direction of motion of the mobile device using the plurality of images; and

logic configured to store the misalignment angle and the direction of motion in a storage device.

12. The apparatus of claim 11, wherein the logic configured to adjust perspectives of the plurality of images comprises at least one of:

logic configured to adjust perspectives of the plurality of images based on the orientation of the mobile device calculated using data collected from one or more sensors;

logic configured to compensate for perspectives of the plurality of images using an area near centers of the plurality of images; and

logic configured to compensate for perspectives of the plurality of images based at least in part on a weighted average of locations of features in the plurality of images.

13. The apparatus of claim 11, wherein the logic configured to determine the misalignment angle comprises:

logic configured to track features from the plurality of images;

logic configured to estimate direction of motion of the mobile device;

logic configured to estimate the orientation of the mobile device using sensor data; and

logic configured to determine the misalignment angle based at least in part on the direction of motion and the orientation of the mobile device.

14. The apparatus of claim 13, wherein the logic configured to track features from the plurality of images comprises:

logic configured to reject outliers in features of the plurality of images to eliminate at least one moving object in the plurality of images.

15. The apparatus of claim 11, further comprising at least one of:

logic configured to determine a confidence of the misalignment angle with respect to the direction of motion of the mobile device using information provided by a gyroscope of the mobile device;

logic configured to determine the confidence of the misalignment angle with respect to the direction of motion of the mobile device using information provided by a magnetometer of the mobile device; and

logic configured to determine the confidence of the misalignment angle with respect to the direction of motion of the mobile device using features of the plurality of images.

16. The apparatus of claim 11 further comprises:

logic configured to determine a speed estimation of the mobile device;

logic configured to determine a confidence of the speed estimation of the mobile device; and

logic configured to apply the speed estimation and the confidence of the speed estimation to navigate a user of the mobile device.

17. The apparatus of claim 16, wherein the logic configured to determine a speed estimate comprises:

logic configured to extract features from the plurality of images;

logic configured to compute an average displacement of the mobile device using the features from the plurality of images; and

logic configured to compute the speed estimate based at least in part on the average displacement of the mobile device.

18. The apparatus of claim 17, wherein the logic configured to compute the speed estimate comprises:

logic configured to compare the features from the plurality of images;

logic configured to determine a separation of pixels between two consecutive images in the plurality of images;

logic configured to determine a time interval between the two consecutive images; and

logic configured to calculate the speed estimate of the mobile device in accordance with the separation of pixels and the time interval between the two consecutive images.

19. The apparatus of claim 18 further comprises:

logic configured to calibrate a height of the mobile device according to at least one of GPS location information and WIFI location information of the mobile device.

20. The apparatus of claim 11 further comprises:

logic configured to apply the misalignment angle and a confidence of the misalignment angle to navigate a user of the mobile device.

21. A computer program product comprising non-transitory medium storing instructions for execution by one or more computer systems, the instructions comprising:

instructions for capturing a plurality of images that represent views from a mobile device;

instructions for adjusting perspectives of the plurality of images based at least in part on an orientation of the mobile device;

instructions for determining a misalignment angle with respect to a direction of motion of the mobile device using the plurality of images; and

instructions for storing the misalignment angle and the direction of motion in a storage device.

22. The computer program product of claim 21, wherein the instructions for adjusting perspectives of the plurality of images comprises at least one of:

instructions for adjusting perspectives of the plurality of images based on the orientation of the mobile device calculated using data collected from one or more sensors;

instructions for compensating for perspectives of the plurality of images using an area near centers of the plurality of images; and

instructions for compensating for perspectives of the plurality of images based at least in part on a weighted average of locations of features in the plurality of images.

23. The computer program product of claim 21, wherein the instructions for determining the misalignment angle comprises:

instructions for tracking features from the plurality of images;

instructions for estimating direction of motion of the mobile device;

instructions for estimating the orientation of the mobile device using sensor data; and

instructions for determining the misalignment angle based at least in part on the direction of motion and the orientation of the mobile device.

24. The computer program product of claim 23, wherein the instructions for tracking features from the plurality of images comprises:

instructions for rejecting outliers in features of the plurality of images to eliminate at least one moving object in the plurality of images.

25. The computer program product of claim 21, further comprising at least one of:

instructions for determining a confidence of the misalignment angle with respect to the direction of motion of the mobile device using information provided by a gyroscope of the mobile device;

instructions for determining the confidence of the misalignment angle with respect to the direction of motion of the mobile device using information provided by a magnetometer of the mobile device; and

instructions for determining the confidence of the misalignment angle with respect to the direction of motion of the mobile device using features of the plurality of images.

26. The computer program product of claim 21 further comprises:

instructions for determining a speed estimation of the mobile device;

instructions for determining a confidence of the speed estimation of the mobile device; and

instructions for applying the speed estimation and the confidence of the speed estimation to navigate a user of the mobile device.

27. The computer program product of claim 26, wherein the instructions for determining a speed estimate comprises:

instructions for extracting features from the plurality of images;

instructions for computing an average displacement of the mobile device using the features from the plurality of images; and

instructions for computing the speed estimate based at least in part on the average displacement of the mobile device.

28. The computer program product of claim 27, wherein the instructions for computing the speed estimate comprises:

instructions for comparing the features from the plurality of images;

instructions for determining a separation of pixels between two consecutive images in the plurality of images;

instructions for determining a time interval between the two consecutive images; and

instructions for calculating the speed estimate of the mobile device in accordance with the separation of pixels and the time interval between the two consecutive images.

29. The computer program product of claim 28 further comprises:

instructions for calibrating a height of the mobile device according to at least one of GPS location information and WIFI location information of the mobile device.

30. The computer program product of claim 21 further comprises:

instructions for applying the misalignment angle and a confidence of the misalignment angle to navigate a user of the mobile device.

31. A system, comprising:

means for capturing a plurality of images that represent views from a mobile device;

means for adjusting perspectives of the plurality of images based at least in part on an orientation of the mobile device;

means for determining a misalignment angle with respect to a direction of motion of the mobile device using the plurality of images; and

means for storing the misalignment angle and the direction of motion in a storage device.

32. The system of claim 31, wherein the means for adjusting perspectives of the plurality of images comprises at least one of:

means for adjusting perspectives of the plurality of images based on the orientation of the mobile device calculated using data collected from one or more sensors;

means for compensating for perspectives of the plurality of images using an area near centers of the plurality of images; and

means for compensating for perspectives of the plurality of images based at least in part on a weighted average of locations of features in the plurality of images.

33. The system of claim 31, wherein the means for determining the misalignment angle comprises:

means for tracking features from the plurality of images;

means for estimating direction of motion of the mobile device;

means for estimating the orientation of the mobile device using sensor data; and

means for determining the misalignment angle based at least in part on the direction of motion and the orientation of the mobile device.

34. The system of claim 31, further comprising at least one of:

means for determining a confidence of the misalignment angle with respect to the direction of motion of the mobile device using information provided by a gyroscope of the mobile device;

means for determining the confidence of the misalignment angle with respect to the direction of motion of the mobile device using information provided by a magnetometer of the mobile device; and

means for determining the confidence of the misalignment angle with respect to the direction of motion of the mobile device using features of the plurality of images.

35. The system of claim 31 further comprises:

means for determining a speed estimation of the mobile device;

means for determining a confidence of the speed estimation of the mobile device; and

means for applying the speed estimation and the confidence of the speed estimation to navigate a user of the mobile device.