Non-line-of-sight (NLoS) satellite detection at a vehicle using a camera

• US 10,495,762 B2
• Filed: 05/19/2017
• Issued: 12/03/2019
• Est. Priority Date: 05/19/2017
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Niesen, Urs, Ramanandan, Arvind
• Primary Examiner(s): Refai, Ramsey
• Application Number:
  US15/599,712
• Publication Number:
  US 20180335525A1
• Time in Patent Office: 928 Days
• CPC Class Codes: G01C 21/362 received from an external device or application, e.g. PDA, mobile phone or calendar application
• CPC Class Codes: G01C 21/3623 using a camera or code reader, e.g. for optical or magnetic codes
• CPC Class Codes: G01C 21/3676 Overview of the route on the road map
• CPC Class Codes: G01S 19/22 Multipath-related issues
• CPC Class Codes: G01S 19/28 Satellite selection
• CPC Class Codes: G01S 19/45 by combining measurements of signals from the satellite radio beacon positioning system with a supplementary measurement
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04B 7/18513 Transmission in a satellite or space-based system

Abstract

Techniques provided herein are directed toward using a camera, such as a forward-facing camera, to identify non-line-of-sight (NLoS) satellites in a satellite positioning system. In particular, successive images captured by the camera of the vehicle can be used to create a three-dimensional (3-D) skyline model of one or more objects that may be obstructing the view of a satellite (from the perspective of the vehicle). Accordingly, this allows for the determination of NLoS satellites and exclusion of data from the NLoS satellites in the determination of the location of the vehicle. Techniques may further include providing the determined location of the vehicle.

Claims

1. A method of identifying, with respect to a vehicle, non-line-of-sight (NLoS) satellites in a satellite positioning system, wherein the method comprises:

obtaining a plurality of successive images captured by a camera of the vehicle;

creating, from the plurality of successive images, a three-dimensional (3-D) model representative of one or more objects obstructing a view of the sky from the perspective of the vehicle;

using the 3-D model to simulate a view of a virtual upward-facing camera;

determining one or more NLoS satellites of the satellite positioning system with respect to the vehicle, based on simulated view of the virtual upward-facing camera;

determining a location of the vehicle by excluding data from the one or more NLoS satellites of the satellite positioning system, based on the determination of the one or more NLoS satellites of the satellite positioning system; and

providing the determined location of the vehicle.

2. The method of claim 1, wherein the camera comprises a forward-facing camera configured to capture images in a direction of travel of the vehicle.

3. The method of claim 1, wherein obtaining the plurality of successive images is in response to a triggering event.

4. The method of claim 3, wherein the triggering event comprises:

a determination that the vehicle is within a threshold distance of a certain region, or

a determination that an image captured by the camera includes an image of an obstruction, or

any combination thereof.

5. The method of claim 1, wherein the plurality of successive images are captured over a certain period of time, the method further comprising determining a movement of the vehicle during the certain period of time.

6. The method of claim 5, wherein determining the movement of the vehicle during the certain period of time comprises obtaining motion sensor data from one or more motion sensors, the motion sensor data indicative of sensor measurements of the one or more motion sensors captured during the certain period of time.

7. The method of claim 1, wherein determining the one or more NLoS satellites with respect to the vehicle is further based on satellite information indicative of where, from the perspective of the vehicle, the one or more NLoS satellites is in the sky.

8. The method of claim 1, wherein the determining the 3-D model representative of one or more objects obstructing a view of the sky further comprises tracking, over the plurality of successive images, one or more features of a building.

9. The method of claim 1, wherein providing the determined location of the vehicle comprises:

showing, on a display of the vehicle, the determined location of the vehicle,

sending, via wireless communication, the determined location of the vehicle to a device separate from the vehicle,

providing the determined location of the vehicle from one computing device of the vehicle to another, or

any combination thereof.

10. A computing device comprising:

a camera configured to capture a plurality of successive images from a vehicle;

a satellite positioning system receiver;

a memory; and

a processing unit communicatively connected with the camera, the memory, and the satellite positioning system receiver, the processing unit configured to;

create, from the plurality of successive images, a three-dimensional (3-D) model representative of one or more objects obstructing a view of the sky from the perspective of the vehicle;

use the 3-D model to simulate a view of a virtual upward-facing camera;

determine one or more NLoS satellites of the satellite positioning system with respect to the vehicle, based on the simulated view of the virtual upward-facing camera;

determine a location of the vehicle by excluding data from the one or more NLoS satellites of the satellite positioning system, based on the determination of the one or more non-line-of-sight (NLoS) satellites of the satellite positioning system; and

provide the determined location of the vehicle.

11. The computing device of claim 10, wherein the camera comprises a forward-facing camera configured to capture images in a direction of travel of the vehicle.

12. The computing device of claim 10, wherein, t to cause the camera to capture the plurality of successive images, the processing unit is configured to first determine a triggering event has occurred.

13. The computing device of claim 12, wherein, to determine the triggering event has occurred, the processing unit is configured to determine:

the vehicle is within a threshold distance of a certain region, or

an image captured by the camera includes an image of an obstruction, or

any combination thereof.

14. The computing device of claim 10, wherein the processing unit is configured to:

cause the camera to capture the plurality of successive images over a certain period of time, and

determine a movement of the vehicle during the certain period of time.

15. The computing device of claim 14, further comprising one or more motion sensors, wherein to determine the movement of the vehicle during the certain period of time the processing unit is configured to obtain motion sensor data indicative of sensor measurements of the one or more motion sensors captured during the certain period of time.

16. The computing device of claim 10, wherein, to determine the one or more NLoS satellites with respect to the vehicle, the processing unit is configured to use satellite information indicative of where, from the perspective of the vehicle, the one or more NLoS satellites is in the sky.

17. The computing device of claim 10, wherein, to determine the 3-D model representative of one or more objects obstructing a view of the sky, the processing unit is configured to track, over the plurality of successive images, one or more features of a building.

18. The computing device of claim 10, wherein, to provide the determined location, the processing unit is configured to:

cause a display of the vehicle to show the determined location of the vehicle,

cause a wireless transmitter to send, via wireless communication, the determined location of the vehicle to a device separate from the vehicle,

provide the determined location of the vehicle to a second computing device, or

any combination thereof.

19. An apparatus comprising:

means for obtaining a plurality of successive images captured by a camera of a vehicle;

means for creating, from the plurality of successive images, a three-dimensional (3-D) model representative of one or more objects obstructing a view of the sky from the perspective of the vehicle;

means for using the 3-D model to simulate a view of a virtual upward-facing camera;

means for determining one or more NLoS satellites of the satellite positioning system with respect to the vehicle, based on the simulated view of the virtual upward-facing camera;

means for determining a location of the vehicle by excluding data from the one or more NLoS satellites of the satellite positioning system, based on the determination of the one or more non-line-of-sight (NLoS) satellites of the satellite positioning system; and

means for providing the determined location of the vehicle.

20. The apparatus of claim 19, wherein the plurality of successive images are captured over a certain period of time, the apparatus further comprising means for determining a movement of the vehicle during the certain period of time.

21. The apparatus of claim 20, wherein the means for determining the movement of the vehicle during the certain period of time comprises means for obtaining motion sensor data from motion sensing means, the motion sensor data indicative of sensor measurements of the motion sensing means captured during the certain period of time.

22. The apparatus of claim 19, wherein the means for determining the 3-D model representative of one or more objects obstructing a view of the sky further comprises means for tracking, over the plurality of successive images, one or more features of a building.

23. A non-transitory computer-readable medium apparatus having instructions embedded thereon for identifying, with respect to a vehicle, non-line-of-sight (NLoS) satellites in a satellite positioning system, the instructions including computer code for:

obtaining a plurality of successive images captured by a camera of a vehicle;

creating, from the plurality of successive images, a three-dimensional (3-D) model representative of one or more objects obstructing a view of the sky from the perspective of the vehicle;

using the 3-D model to simulate a view of a virtual upward-facing camera;

determining one or more NLoS satellites of the satellite positioning system with respect to the vehicle, based on the simulated view of the virtual upward-facing camera;

determining a location of the vehicle by excluding data from the one or more NLoS satellites of the satellite positioning system, based on the determination of the one or more non-line-of-sight (NLoS) satellites of the satellite positioning system; and

providing the determined location of the vehicle.

24. The non-transitory computer-readable medium of claim 23, wherein the plurality of successive images are captured over a certain period of time, the instructions further including computer code for determining a movement of the vehicle during the certain period of time.

25. The non-transitory computer-readable medium of claim 24, wherein the computer code for determining the movement of the vehicle during the certain period of time comprises computer code for obtaining motion sensor data from motion sensing means, the motion sensor data indicative of sensor measurements of the motion sensing means captured during the certain period of time.

26. The non-transitory computer-readable medium of claim 23, wherein the computer code for determining the 3-D model representative of one or more objects obstructing a view of the sky further comprises computer code for tracking, over the plurality of successive images, one or more features of a building.