Camera-based position location and navigation based on image processing

• US 9,341,720 B2
• Filed: 06/30/2011
• Issued: 05/17/2016
• Est. Priority Date: 01/11/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Naquib, Ayman, Garin, Lionel J., Bergan, Charles A.
• Primary Examiner(s): Patel, Jay
• Assistant Examiner(s): GEROLEO, FRANCIS
• Application Number:
  US13/173,984
• Publication Number:
  US 20120176491A1
• Time in Patent Office: 1,783 Days
• Field of Search: 348/113
• US Class Current: 1/1
• CPC Class Codes: G01S 19/235 Calibration of receiver components
• CPC Class Codes: G01S 19/485 whereby the further system is an optical system or imaging system
• CPC Class Codes: G01S 19/49 whereby the further system is an inertial position system, e.g. loosely-coupled

Abstract

An apparatus and method for estimating a position of a mobile device based on one or more images is disclosed. Positioning information is derived from an image containing an SAS (self-addressing source such as a QR code), thereby setting a first vector V₁ and an SAS pose. The image is also used to determine a displacement between the mobile device and the self-addressing source, thereby setting a second vector V₂. Using the first vector V₁, the SAS pose and the second vector V₂, the mobile device may estimate its position with high accuracy and may also recalibrate dead-reckoning navigation and a gyrometer.

Claims

1. A method for calibrating navigation in a mobile device comprising a camera and an inertial sensor, the method comprising:

providing an initial position without using any SASs (self-addressing sources);

detecting an SAS (self-addressing source);

determining a first vector V₁ between the SAS and a reference point;

determining an SAS pose relative to the reference point;

determining a second vector V₂ of the mobile device relative to the SAS, wherein the second vector V₂ is determined based from an image of the SAS captured by the camera and wherein the second vector V₂ is represented by a displacement from the mobile device to the SAS and an angle from the mobile device to the SAS;

computing a third vector V₃ of the mobile device based on the first vector V₁ and the second vector V₂, wherein V₃=V₁−

V₂;

computing a current position of the mobile device based on the third vector V₃; and

recalibrating the inertial sensor based on the current position.

2. The method of claim 1, wherein the initial position comprises an origin (0, 0, 0).

3. The method of claim 1, wherein the initial position comprises an image-based position estimate.

4. The method of claim 1, wherein the initial position comprises a global navigation satellite systems based position (GNSS-based position).

5. The method of claim 1, wherein the SAS comprises a quick response code (QR code).

6. The method of claim 1, wherein the SAS comprises a bar code.

7. The method of claim 1, wherein the SAS comprises text.

8. The method of claim 1, wherein the SAS comprises a link to a database.

9. The method of claim 1, wherein the SAS comprises a wireless transmitter.

10. The method of claim 9, wherein the wireless transmitter comprises an RF ID tag.

11. The method of claim 1, further comprising determining a mobile pose relative to the SAS.

12. The method of claim 1, wherein detecting the SAS comprises capturing an image of the SAS and wherein determining the second vector V₂ comprises determining geometry of the SAS in the image.

13. The method of claim 1, wherein the inertial sensor comprises a gyrometer.

14. The method of claim 1, wherein the detecting the SAS comprises using a scale-invariant feature transform (SIFT).

15. The method of claim 1, further comprising

sending an SAS information request to a server having a database, wherein the database includes the first vector V₁;

and

receiving an SAS information response, from the server, comprising the first vector V₁.

16. The method of claim 15, wherein the SAS information request comprises a position estimate of the SAS determined by the mobile device.

17. The method of claim 16, wherein the SAS information request further comprises an estimated SAS pose determined by the mobile device.

18. The method of claim 16, wherein the position estimate of the SAS comprises an estimated position of the mobile device.

19. The method of claim 15, wherein the SAS information response further comprises an SAS pose from the database.

20. The method of claim 1, further comprising using dead reckoning to determine a first updated position relative to the initial position.

21. The method of claim 20, wherein using dead reckoning comprises using a simultaneous localization and mapping (SLAM) algorithm.

22. The method of claim 20, wherein using dead reckoning comprises:

determining a direction using a magnetometer; and

determining a step using an accelerometer.

23. The method of claim 1, further comprising determining an estimated position of the SAS based on the second vector V₂ and the initial position.

24. A mobile device comprising a camera and a navigation system including an inertial sensor, the mobile device comprising:

means for providing an initial position without using any SASs (self-addressing sources);

means for detecting an SAS (self-addressing source);

means for determining a first vector V₁ between the SAS and a reference point;

means for determining an SAS pose relative to the reference point;

means for determining a second vector V₂ of the mobile device relative to the SAS, wherein the second vector V₂ is determined based from an image of the SAS captured by the camera and wherein the second vector V₂ is represented by a displacement from the mobile device to the SAS and an angle from the mobile device to the SAS;

means for computing a third vector V₃ of the mobile device based on the first vector V₁ and the second vector V₂, wherein V₃=V₁−

V₂;

means for computing a current position of the mobile device based on the third vector V₃; and

means for recalibrating the inertial sensor based on the current position.

25. A mobile device comprising a processor and a memory and further comprising a camera and an inertial sensor, wherein the memory includes software instructions to:

provide an initial position without using any SASs (self-addressing sources);

detect an SAS (self-addressing source);

determine a first vector V₁ between the SAS and a reference point;

determine an SAS pose relative to the reference point;

determine a second vector V₂ of the mobile device relative to the SAS, wherein the second vector V₂ is determined based from an image of the SAS captured by the camera and wherein the second vector V₂ is represented by a displacement from the mobile device to the SAS and an angle from the mobile device to the SAS;

compute a third vector V₃ of the mobile device based on the first vector V₁ and the second vector V₂, wherein V₃=V₁−

V₂;

compute a current position of the mobile device based on the third vector V₃; and

recalibrate the inertial sensor based on the current position.

26. A non-transitory computer-readable storage medium including program code stored thereon for a mobile device comprising a camera and an inertial sensor, the program code comprising code to:

provide an initial position without using any SASs (self-addressing sources);

detect an SAS (self-addressing source);

determine a first vector V₁ between the SAS and a reference point;

determine an SAS pose relative to the reference point;

determine a second vector V₂ of the mobile device relative to the SAS, wherein the second vector V₂ is determined based from an image of the SAS captured by the camera and wherein the second vector V₂ is represented by a displacement from the mobile device to the SAS and an angle from the mobile device to the SAS;

compute a third vector V₃ of the mobile device based on the first vector V₁ and the second vector V₂, wherein V₃=V₁−

V₂;

compute a current position of the mobile device based on the third vector V₃; and

recalibrate the inertial sensor based on the current position.

27. A method for determining a position of a mobile device comprising a camera and an inertial sensor, the method comprising:

providing an initial position without using any SASs (self-addressing sources);

detecting an SAS (self-addressing source);

determining a first vector V₁ between the SAS and a reference point;

determining an SAS pose relative to the reference point;

determining a second vector V₂ of the mobile device relative to the SAS, wherein the second vector V₂ is determined based from an image of the SAS captured by the camera and wherein the second vector V₂ is represented by a displacement from the mobile device to the SAS and an angle from the mobile device to the SAS;

computing a third vector V₃ of the mobile device based on the first vector V₁ and the second vector V₂, wherein V₃=V₁−

V₂;

computing a current position of the mobile device based on the third vector V₃; and

recalibrating the inertial sensor based on the current position.

28. The method of claim 27, wherein the SAS comprises a quick response code (QR code).

29. The method of claim 27, wherein the SAS comprises a bar code.

30. The method of claim 27, wherein the SAS comprises text.

31. The method of claim 27, wherein the SAS comprises a link to a database.

32. The method of claim 27, wherein the SAS comprises a wireless transmitter.

33. The method of claim 32, wherein the wireless transmitter comprises an RF ID tag.

34. The method of claim 27, further comprising determining a mobile pose relative to the SAS.

35. The method of claim 27, wherein detecting the SAS comprises capturing an image of the SAS and wherein determining the second vector V₂ comprises determining geometry of the SAS in the image.

36. The method of claim 27, wherein the detecting the SAS comprises using a scale-invariant feature transform (SIFT).

37. The method of claim 27, further comprising

sending an SAS information request to a server having a database, wherein the database includes the first vector V₁;

and

receiving an SAS information response, from the server, comprising the first vector V₁.

38. The method of claim 37, wherein the SAS information request comprises a position estimate of the SAS determined by the mobile device.

39. The method of claim 38, wherein the SAS information request further comprises an estimated SAS pose determined by the mobile device.

40. The method of claim 38, wherein the position estimate of the SAS comprises an estimated position of the mobile device.

41. The method of claim 37, wherein the SAS information response further comprises an SAS pose from the database.

42. The method of claim 27, further comprising determining an estimated position of the SAS based on the second vector V₂ and the initial position.

43. A mobile device comprising a camera and an inertial sensor, the mobile device comprising:

means for providing an initial position without using any SASs (self-addressing sources);

means for detecting an SAS (self-addressing source);

means for determining a first vector V₁ between the SAS and a reference point;

means for determining an SAS pose relative to the reference point;

means for determining a second vector V₂ of the mobile device relative to the SAS, wherein the second vector V₂ is determined based from an image of the SAS captured by the camera and wherein the second vector V₂ is represented by a displacement from the mobile device to the SAS and an angle from the mobile device to the SAS;

means for computing a third vector V₃ of the mobile device based on the first vector V₁ and the second vector V₂, wherein V₃=V₁−

V₂;

means for computing a current position of the mobile device based on the third vector V₃; and

recalibrating the inertial sensor based on the current position.

44. A mobile device comprising a processor and a memory and further comprising a camera and an inertial sensor, wherein the memory includes software instructions to:

provide an initial position without using any SASs (self-addressing sources);

detect an SAS (self-addressing source);

determine a first vector V₁ between the SAS and a reference point;

determine an SAS pose relative to the reference point;

determine a second vector V₂ of the mobile device relative to the SAS, wherein the second vector V2 is determined based from an image of the SAS captured by the camera and wherein the second vector V₂ is represented by a displacement from the mobile device to the SAS and an angle from the mobile device to the SAS;

compute a third vector V₃ of the mobile device based on the first vector V₁ and the second vector V₂, wherein V₃=V₁−

V₂;

compute a current position of the mobile device based on the third vector V₃; and

recalibrate the inertial sensor based on the current position.

45. A non-transitory computer-readable storage medium including program code stored thereon for a mobile device comprising a camera and an inertial sensor, the program code comprising code to:

provide an initial position without using any SASs (self-addressing sources);

detect an SAS (self-addressing source);

determine a first vector V₁ between the SAS and a reference point;

determine an SAS pose relative to the reference point;

determine a second vector V₂ of the mobile device relative to the SAS, wherein the second vector V₂ is determined based from an image of the SAS captured by the camera and wherein the second vector V₂ is represented by a displacement from the mobile device to the SAS and an angle from the mobile device to the SAS;

compute a third vector V₃ of the mobile device based on the first vector V₁ and the second vector V₂, wherein V₃=V₁−

V₂;

compute a current position of the mobile device based on the third vector V₃; and

recalibrating the inertial sensor based on the current position.