NAVIGATION IN BUILDINGS WITH RECTANGULAR FLOOR PLAN

• US 20130002857A1
• Filed: 06/30/2011
• Published: 01/03/2013
• Est. Priority Date: 06/30/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Kulik, Victor
• Granted Patent: US 8,810,649 B2
• Time in Patent Office: Days
• US Class Current: 348/135
• CPC Class Codes: G01C 17/38 Testing, calibrating, or compensating of compasses
• CPC Class Codes: G01C 21/1654 with electromagnetic compass
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation

Abstract

An apparatus and method for providing a direction based on an angle of a reference wall is provided. A mobile device uses an angle of a horizontal feature from an image to calibrate a sensor and future sensor measurements. The angle of the horizontal feature is determined by image processing and this angle is mapped to one of four assumed parallel or perpendicular angles of an interior of a building. A sensor correction value is determined from a difference between the sensor-determined angle and the image-processing determined angle. The image processing determined angle is assumed to be very accurate and without accumulated errors or offsets that the sensor measurements may contain.

Claims

1. A method of navigation in a mobile device using a reference wall, the method comprising:

determining a reference angle (θ

_REF) of the reference wall;

capturing, with a camera in the mobile device, a local image containing a horizontal feature;

processing the local image to determine a horizontal feature angle (θ

_HF), wherein the horizontal feature angle (θ

_HF) measures an angle from a current orientation of the mobile device to the horizontal feature;

selecting a wall angle, of the horizontal feature, equal to an integer multiple of 90°

from the reference angle (θ

_REF); and

determining an angle of the current orientation of the mobile device based on a combination of the horizontal feature angle (θ

_HF) and the wall angle.

2. The method of claim 1, wherein the reference angle (θ

_REF) is an absolute angle with respect to North.

3. The method of claim 1, wherein the reference wall is an exterior wall.

4. The method of claim 1, wherein the local image comprises an interior wall.

5. The method of claim 1, wherein determining the reference angle (θ

_REF) comprises;

capturing an initial image; and

computing the reference angle (θ

_REF) based on the initial image.

6. The method of claim 1, wherein determining the reference angle (θ

_REF) comprises;

determining a building containing the reference wall; and

determining the reference angle (θ

_REF) based on the building.

7. The method of claim 1, wherein determining the reference angle (θ

_REF) comprises;

determining a building containing the reference wall;

finding the building on an overhead map; and

computing the reference angle (θ

_REF) from the overhead map.

8. The method of claim 1, wherein determining the reference angle (θ

_REF) comprises;

determining a building containing the reference wall, wherein the building is indexed in a database;

looking up the reference angle (θ

_REF) from the database.

9. The method of claim 1, wherein the horizontal feature comprises a horizontal edge of a door frame.

10. The method of claim 1, wherein the horizontal feature comprises a horizontal edge of a window.

11. The method of claim 1, wherein the horizontal feature comprises an interface edge between a wall and one of a floor and a ceiling.

12. The method of claim 1, wherein the wall angle is a modulo of {the reference angle (θ

_REF)+n*90°

, 360°

}, wherein n is an integer.

13. The method of claim 1, wherein n is 0.

14. The method of claim 1, wherein n is −

1.

15. The method of claim 1, wherein selecting the wall angle is based on inertial sensor measurements.

16. The method of claim 15, further comprising calibrating an inertial sensor.

17. The method of claim 16, wherein calibrating the inertial sensor comprises:

deriving a sensor misalignment (θ

_ERROR) based on

a current orientation (θ

_SENSOR) of the mobile device based on the inertial sensor measurements from the inertial sensor; and

the angle of the current orientation of the mobile device; and

calibrating the inertial sensor based on the sensor misalignment (θ

_ERROR).

18. The method of claim 1, further comprising determining a position of the mobile device based on the angle of the current orientation of the mobile device.

19. The method of claim 1, further comprising determining motion of the mobile device based on the angle of the current orientation of the mobile device.

20. The method of claim 1, further comprising setting a direction of motion of the mobile device to the wall angle, thereby performing wall matching.

21. A method in a mobile device of navigation using wall matching with a horizontal feature to estimate motion, the method comprising:

capturing, with a camera in the mobile device, a local image containing the horizontal feature;

processing the local image to determine a horizontal feature angle (θ

_HF), wherein the horizontal feature angle (θ

_HF) is an angle from a current orientation of the mobile device to the horizontal feature; and

estimating a direction of motion of the mobile device, based on the horizontal feature angle (θ

_HO, as parallel to a wall angle of the horizontal feature.

22. The method of claim 21, wherein the local image comprises an interior wall.

23. The method of claim 21, further comprising:

determining a reference angle (θ

_REF) of a reference wall, wherein the reference angle (θ

_REF) is an absolute angle with respect to North; and

selecting a wall angle, of the horizontal feature, equal to an integer multiple of 90°

from the reference angle (θ

_REF).

24. The method of claim 23, wherein the reference wall is an exterior wall.

25. The method of claim 23, wherein determining the reference angle (θ

_REF) comprises;

capturing an initial image; and

computing the reference angle (θ

_REF) based on the initial image.

26. The method of claim 23, wherein determining the reference angle (θ

_REF) comprises;

determining a building containing the reference wall; and

determining the reference angle (θ

_REF) based on the building.

27. The method of claim 23, wherein determining the reference angle (θ

_REF) comprises;

determining a building containing the reference wall;

finding the building on an overhead map; and

computing the reference angle (θ

_REF) from the overhead map.

28. The method of claim 23, wherein determining the reference angle (θ

_REF) comprises;

determining a building containing the reference wall, wherein the building is indexed in a database;

looking up the reference angle (θ

_REF) from the database.

29. The method of claim 23, wherein the wall angle is a modulo of {the reference angle (θ

_REF)+n*90°

, 360°

}, wherein n is an integer.

30. The method of claim 23, wherein selecting the wall angle is based on inertial sensor measurements.

31. A method of calibrating an inertial sensor in a mobile device, the method comprising:

capturing a first image, at a first time from the mobile device in a first orientation, wherein the first image contains a horizontal feature;

determining a first horizontal feature angle (θ

<sub>HF—

1</sub>) of the horizontal feature in the first image, wherein the first horizontal feature angle (θ

<sub>HF—

1</sub>) is an angle from the mobile device in the first orientation at the first time to the horizontal feature in the first image;

capturing a second image, at a second time from the mobile device in a second orientation, wherein the second image contains a horizontal feature;

determining a second horizontal feature angle (θ

<sub>HF—

2</sub>) of the horizontal feature in the second image, wherein the second horizontal feature angle (θ

<sub>HF—

2</sub>) is an angle from the mobile device in the second orientation at the second time to the horizontal feature in the second image;

computing a difference between the first horizontal feature angle (θ

<sub>HF—

1</sub>) and the second horizontal feature angle (θ

<sub>HF—

2</sub>) to form an optical sensor rotation;

measuring, with the inertial sensor, a rotation of the mobile device from the first orientation at the first time to the second orientation at the second time to form an inertial sensor rotation; and

deriving a sensor misalignment (θ

_ERROR) based on a difference between the optical sensor rotation and the inertial sensor rotation.

32. The method of claim 31, wherein the horizontal feature in the first image is the horizontal feature in the second image.

33. The method of claim 31, wherein the horizontal feature in the first image is oriented 90°

from the horizontal feature in the second image.

34. The method of claim 31, wherein the horizontal feature in the first image is oriented 180°

from the horizontal feature in the second image.

35. The method of claim 31, wherein the horizontal feature in the first image is oriented from the horizontal feature in the second image at a modulo of {n*90°

, 360°

}, wherein n is an integer.

36. The method of claim 31, wherein computing the difference between the first horizontal feature angle (θ

<sub>HF—

1</sub>) and the second horizontal feature angle (θ

<sub>HF—

2</sub>) comprises;

selecting a wall angle limited to {n*90°

}, wherein n is an integer; and

computing the optical sensor rotation from

the first horizontal feature angle (θ

<sub>HF—

1</sub>);

the second horizontal feature angle (θ

<sub>HF—

2</sub>); and

the wall angle.

37. A method of navigation in a mobile device using a reference wall, the method comprising:

determining a reference angle (θ

_REF) of the reference wall;

determining an estimated direction of motion of the mobile device based on inertial sensor measurements;

selecting a wall angle, closest to the estimated direction, equal to an integer multiple of 90°

from the reference angle (θ

_REF); and

determining a direction of motion of the mobile device based on the wall angle.

38. A method of navigation in a mobile device using a reference wall, the method comprising:

capturing, with a camera in the mobile device, a local image containing a horizontal feature and the reference wall;

determining a reference angle (θ

_REF) of the reference wall;

processing the local image to determine a horizontal feature angle (θ

_HF), wherein the horizontal feature angle (θ

_HF) measures an angle from a current orientation of the mobile device to the horizontal feature; and

determining an angle of the current orientation of the mobile device based on a combination of the horizontal feature angle (θ

_HF) and the reference angle (θ

_REF).

39. A method of providing calibrated inertial sensor measurements, the method comprising:

determining a first reference angle (θ

<sub>REF—

1</sub>);

capturing, with a camera, a final image containing a horizontal feature from a first perspective;

processing the final image to determine a first horizontal feature angle (θ

<sub>HF—

1</sub>) of the horizontal feature;

processing sensor measurements from an inertial sensor to form a sensor angle (θ

_SENSOR);

computing a sensor misalignment (θ

_ERROR) based on;

the sensor angle (θ

_SENSOR);

the first reference angle (θ

<sub>REF—

1</sub>); and

the first horizontal feature angle (θ

<sub>HF—

1</sub>); and

correcting future sensor measurements from the sensor based on the sensor misalignment (θ

_ERROR).

40. The method of claim 39, wherein determining the first reference angle (θ

<sub>REF—

1</sub>) comprises;

capturing an initial image containing a second horizontal feature; and

determining a second angle (θ

<sub>HF—

2</sub>) of the second horizontal feature based on the initial image;

wherein the first reference angle (θ

<sub>REF—

1</sub>) is set based on the second angle (θ

<sub>HF—

2</sub>).

41. The method of claim 40, wherein the second horizontal feature is perpendicular to the horizontal feature.

42. The method of claim 40, wherein the sensor angle (θ

_SENSOR) comprises an accumulation of the sensor measurements.

43. The method of claim 40, wherein the sensor angle (θ

_SENSOR) comprises a difference between the sensor measurements.

44. The method of claim 39, wherein determining the first reference angle (θ

<sub>REF—

1</sub>) comprises;

capturing an initial image, from a different angle, containing the horizontal feature; and

determining a second angle (θ

<sub>HF—

2</sub>) of the horizontal feature based on the initial image;

wherein the first reference angle (θ

<sub>REF—

1</sub>) is set based on the second angle (θ

<sub>HF —

2</sub>).

45. A mobile device for navigation using a reference wall, the mobile device comprising:

a camera;

an inertial sensor; and

a processor and memory comprising code for;

determining a reference angle (θ

_REF) of the reference wall;

capturing, with the camera in the mobile device, a local image containing a horizontal feature;

processing the local image to determine a horizontal feature angle (θ

_HF), wherein the horizontal feature angle (θ

_HF) measures an angle from a current orientation of the mobile device to the horizontal feature;

selecting a wall angle, of the horizontal feature, equal to an integer multiple of 90°

from the reference angle (θ

_REF); and

determining an angle of the current orientation of the mobile device based on a combination of the horizontal feature angle (θ

_HF) and the wall angle.

46. A mobile device for navigation using a reference wall, the mobile device comprising:

means for determining a reference angle (θ

_REF) of the reference wall;

means for capturing a local image containing a horizontal feature;

means for processing the local image to determine a horizontal feature angle (θ

_HF), wherein the horizontal feature angle (θ

_HF) measures an angle from a current orientation of the mobile device to the horizontal feature;

means for selecting a wall angle, of the horizontal feature, equal to an integer multiple of 90°

from the reference angle (θ

_REF); and

means for determining an angle of the current orientation of the mobile device based on a combination of the horizontal feature angle (θ

_HF) and the wall angle.

47. A non-transient computer-readable storage medium including program code stored thereon, for navigation using a reference wall, comprising program code for:

determining a reference angle (θ

_REF) of the reference wall;

receiving, from a camera in the mobile device, a local image containing a horizontal feature;

processing the local image to determine a horizontal feature angle (θ

_HF), wherein the horizontal feature angle (θ

_HF) measures an angle from a current orientation of the mobile device to the horizontal feature;

selecting a wall angle, of the horizontal feature, equal to an integer multiple of 90°

from the reference angle (θ

_REF); and

determining an angle of the current orientation of the mobile device based on a combination of the horizontal feature angle (θ

_HF) and the wall angle.

48. A mobile device for navigation using wall matching with a horizontal feature to estimate motion, the mobile device comprising:

a camera;

an inertial sensor; and

a processor and memory comprising code for;

capturing, with the camera in the mobile device, a local image containing the horizontal feature;

processing the local image to determine a horizontal feature angle (θ

_HF), wherein the horizontal feature angle (θ

_HF) is an angle from a current orientation of the mobile device to the horizontal feature; and

estimating a direction of motion of the mobile device, based on the horizontal feature angle (θ

_HF), as parallel to a wall angle of the horizontal feature.

49. A mobile device for navigation using wall matching with a horizontal feature to estimate motion, the mobile device comprising:

means for capturing a local image containing the horizontal feature;

means for processing the local image to determine a horizontal feature angle (θ

_HF), wherein the horizontal feature angle (θ

_HF) is an angle from a current orientation of the mobile device to the horizontal feature; and

means for estimating a direction of motion of the mobile device, based on the horizontal feature angle (θ

_HF), as parallel to a wall angle of the horizontal feature.

50. A non-transient computer-readable storage medium including program code stored thereon, navigation using wall matching with a horizontal feature to estimate motion, comprising program code for:

receiving, from a camera in the mobile device, a local image containing the horizontal feature;

processing the local image to determine a horizontal feature angle (θ

_HF), wherein the horizontal feature angle (θ

_HF) is an angle from a current orientation of the mobile device to the horizontal feature; and

estimating a direction of motion of the mobile device, based on the horizontal feature angle (θ

_HF), as parallel to a wall angle of the horizontal feature.

51. A mobile device for calibrating an inertial sensor, the mobile device comprising:

a camera;

the inertial sensor; and

a processor and memory comprising code for;

capturing a first image, at a first time from the mobile device in a first orientation, wherein the first image contains a horizontal feature;

determining a first horizontal feature angle (θ

<sub>HF—

1</sub>) of the horizontal feature in the first image, wherein the first horizontal feature angle (θ

<sub>HF—

1</sub>) is an angle from the mobile device in the first orientation at the first time to the horizontal feature in the first image;

capturing a second image, at a second time from the mobile device in a second orientation, wherein the second image contains a horizontal feature;

determining a second horizontal feature angle (θ

<sub>HF—

2</sub>) of the horizontal feature in the second image, wherein the second horizontal feature angle (θ

<sub>HF—

2</sub>) is an angle from the mobile device in the second orientation at the second time to the horizontal feature in the second image;

computing a difference between the first horizontal feature angle (θ

<sub>HF—

1</sub>) and the second horizontal feature angle (θ

<sub>HF—

2</sub>) to form an optical sensor rotation;

measuring, with the inertial sensor, a rotation of the mobile device from the first orientation at the first time to the second orientation at the second time to form an inertial sensor rotation; and

deriving a sensor misalignment (θ

_ERROR) based on a difference between the optical sensor rotation and the inertial sensor rotation.

52. A mobile device for calibrating an inertial sensor, the mobile device comprising:

means for capturing a first image, at a first time from the mobile device in a first orientation, wherein the first image contains a horizontal feature;

means for determining a first horizontal feature angle (θ

<sub>HF—

1</sub>) of the horizontal feature in the first image, wherein the first horizontal feature angle (θ

<sub>HF—

1</sub>) is an angle from the mobile device in the first orientation at the first time to the horizontal feature in the first image;

means for capturing a second image, at a second time from the mobile device in a second orientation, wherein the second image contains a horizontal feature;

means for determining a second horizontal feature angle (θ

<sub>HF—

2</sub>) of the horizontal feature in the second image, wherein the second horizontal feature angle (θ

<sub>HF—

2</sub>) is an angle from the mobile device in the second orientation at the second time to the horizontal feature in the second image;

means for computing a difference between the first horizontal feature angle (θ

<sub>HF—

1</sub>) and the second horizontal feature angle (θ

<sub>HF—

2</sub>) to form an optical sensor rotation;

means for measuring, with the inertial sensor, a rotation of the mobile device from the first orientation at the first time to the second orientation at the second time to form an inertial sensor rotation; and

means for deriving a sensor misalignment (θ

_ERROR) based on a difference between the optical sensor rotation and the inertial sensor rotation.

53. A non-transient computer-readable storage medium including program code stored thereon, for calibrating an inertial sensor, comprising program code for:

capturing a first image, at a first time from the mobile device in a first orientation, wherein the first image contains a horizontal feature;

determining a first horizontal feature angle (θ

<sub>HF—

1</sub>) of the horizontal feature in the first image, wherein the first horizontal feature angle (θ

<sub>HF—

1</sub>) is an angle from the mobile device in the first orientation at the first time to the horizontal feature in the first image;

capturing a second image, at a second time from the mobile device in a second orientation, wherein the second image contains a horizontal feature;

determining a second horizontal feature angle (θ

<sub>HF—

2</sub>) of the horizontal feature in the second image, wherein the second horizontal feature angle (θ

<sub>HF—

2</sub>) is an angle from the mobile device in the second orientation at the second time to the horizontal feature in the second image;

computing a difference between the first horizontal feature angle (θ

<sub>HF—

1</sub>) and the second horizontal feature angle (θ

<sub>HF—

2</sub>) to form an optical sensor rotation;

measuring, with the inertial sensor, a rotation of the mobile device from the first orientation at the first time to the second orientation at the second time to form an inertial sensor rotation; and

deriving a sensor misalignment (θ

_ERROR) based on a difference between the optical sensor rotation and the inertial sensor rotation.

54. A mobile device for navigation using a reference wall, the mobile device comprising:

an inertial sensor; and

a processor and memory comprising code for;

determining a reference angle (θ

_REF) of the reference wall;

determining an estimated direction of motion of the mobile device based on inertial sensor measurements from the inertial sensor;

selecting a wall angle, closest to the estimated direction, equal to an integer multiple of 90°

from the reference angle (θ

_REF); and

determining a direction of motion of the mobile device based on the wall angle.

55. A mobile device for navigation using a reference wall, the mobile device comprising:

means for determining a reference angle (θ

_REF) of the reference wall;

means for determining an estimated direction of motion of the mobile device based on inertial sensor measurements;

means for selecting a wall angle, closest to the estimated direction, equal to an integer multiple of 90°

from the reference angle (θ

_REF); and

means for determining a direction of motion of the mobile device based on the wall angle.

56. A non-transient computer-readable storage medium including program code stored thereon, for navigation using a reference wall, comprising program code for:

determining a reference angle (θ

_REF) of the reference wall;

determining an estimated direction of motion of the mobile device based on inertial sensor measurements;

selecting a wall angle, closest to the estimated direction, equal to an integer multiple of 90°

from the reference angle (θ

_REF); and

determining a direction of motion of the mobile device based on the wall angle.

57. A mobile device for navigation using a reference wall, the mobile device comprising:

a camera;

an inertial sensor; and

a processor and memory comprising code for;

capturing, with the camera in the mobile device, a local image containing a horizontal feature and the reference wall;

determining a reference angle (θ

_REF) of the reference wall;

processing the local image to determine a horizontal feature angle (θ

_HF), wherein the horizontal feature angle (θ

_HF) measures an angle from a current orientation of the mobile device to the horizontal feature; and

determining an angle of the current orientation of the mobile device based on a combination of the horizontal feature angle (θ

_HF) and the reference angle (θ

_REF).

58. A mobile device for navigation using a reference wall, the mobile device comprising:

means for capturing a local image containing a horizontal feature and the reference wall;

means for determining a reference angle (θ

_REF) of the reference wall;

means for processing the local image to determine a horizontal feature angle (θ

_HF), wherein the horizontal feature angle (θ

_HF) measures an angle from a current orientation of the mobile device to the horizontal feature; and

means for determining an angle of the current orientation of the mobile device based on a combination of the horizontal feature angle (θ

_HF) and the reference angle (θ

_REF).

59. A non-transient computer-readable storage medium including program code stored thereon, for navigation in a mobile device using a reference wall, comprising program code for:

receiving, from a camera in the mobile device, a local image containing a horizontal feature and the reference wall;

determining a reference angle (θ

_REF) of the reference wall;

processing the local image to determine a horizontal feature angle (θ

_HF), wherein the horizontal feature angle (θ

_HF) measures an angle from a current orientation of the mobile device to the horizontal feature; and

determining an angle of the current orientation of the mobile device based on a combination of the horizontal feature angle (θ

_HF) and the reference angle (θ

_REF).

60. A mobile device for providing calibrated inertial sensor measurements, the mobile device comprising:

a camera;

an inertial sensor; and

a processor and memory comprising code for;

determining a first reference angle (θ

<sub>REF—

1</sub>);

capturing, with the camera, a final image containing a horizontal feature from a first perspective;

processing the final image to determine a first horizontal feature angle (θ

<sub>HF—

1</sub>) of the horizontal feature;

processing sensor measurements from the inertial sensor to form a sensor angle (θ

_SENSOR);

computing a sensor misalignment (θ

_ERROR) based on;

the sensor angle (θ

_SENSOR);

the first reference angle (θ

<sub>REF—

1</sub>); and

the first horizontal feature angle (θ

<sub>HF—

1</sub>); and

correcting future sensor measurements from the sensor based on the sensor misalignment (θ

_ERRoR).

61. A mobile device for providing calibrated inertial sensor measurements, the mobile device comprising:

means for determining a first reference angle (θ

<sub>REF—

1</sub>);

means for capturing a final image containing a horizontal feature from a first perspective;

means for processing the final image to determine a first horizontal feature angle (θ

<sub>HF—

1</sub>) of the horizontal feature;

means for processing sensor measurements from the inertial sensor to form a sensor angle (θ

_SENSOR);

means for computing a sensor misalignment (θ

_ERROR) based on;

the sensor angle (θ

_SENSOR);

the first reference angle (θ

<sub>REF—

1</sub>); and

the first horizontal feature angle (θ

<sub>HF—

1</sub>); and

means for correcting future sensor measurements from the sensor based on the sensor misalignment (θ

_ERRoR).

62. A non-transient computer-readable storage medium including program code stored thereon, for providing calibrated inertial sensor measurements, comprising program code for:

determining a first reference angle (θ

<sub>REF—

1</sub>);

capturing, with a camera, a final image containing a horizontal feature from a first perspective;

processing the final image to determine a first horizontal feature angle (θ

<sub>HF—

1</sub>) of the horizontal feature;

processing sensor measurements from the inertial sensor to form a sensor angle (θ

_SENSOR);

computing a sensor misalignment (θ

_ERROR) based on;

the sensor angle (θ

_SENSOR);

the first reference angle (θ

<sub>REF—

1</sub>); and

the first horizontal feature angle (θ

<sub>HF—

1</sub>); and

correcting future sensor measurements from the sensor based on the sensor misalignment (θ

_ERRoR).