Maintaining heatmaps using tagged visual data

• US 9,824,481 B2
• Filed: 12/30/2014
• Issued: 11/21/2017
• Est. Priority Date: 12/30/2014
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Naguib, Ayman Fawzy, Chao, Hui, Das, Saumitra Mohan
• Primary Examiner(s): Yang, Ryan R
• Application Number:
  US14/585,538
• Publication Number:
  US 20160189416A1
• Time in Patent Office: 1,057 Days
• Field of Search: 345441
• CPC Class Codes: G01S 5/02522 The radio-map containing measured values of non-radio values
• CPC Class Codes: G01S 5/02524 Creating or updating the radio-map
• CPC Class Codes: G01S 5/02526 using non-dedicated equipment, e.g. user equipment or crowd-sourcing
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: H04N 5/00 Details of television systems scanning details or combination thereof with generation of supply voltages H04N3/00
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

Techniques describe tagging visual data (e.g., image and/or video data) with wireless and sensor measurement information by a mobile device. Tagged visual data may be sent to a server, such as a crowdsourcing server. Techniques further describe receiving visual data from a device, wherein the visual data is tagged with information comprising source identifying information associated with an at least one signal emitting device, identifying at least one visual feature from the visual data, determining a coordinate on a map at which the visual data was acquired based on identifying the at least one visual feature from the visual data, and associating the coordinate on the map with the information associated with the at least one signal emitting device.

Claims

1. A method for generating map data, the method comprising:

receiving visual data captured by a camera coupled to a device, wherein the visual data is tagged with source identifying information associated with a wireless access point other than the device by including the source identifying information in metadata attached to the visual data, wherein the metadata describes attributes associated with the visual data;

acquiring the source identifying information associated with the wireless access point from the metadata attached to the visual data;

identifying at least one visual feature from the visual data;

determining a coordinate on a map at which the visual data was acquired based on identifying the at least one visual feature from the visual data; and

associating the coordinate on the map with the source identifying information associated with the wireless access point acquired from the metadata attached to the visual data.

2. The method of claim 1, wherein the visual data is further tagged with signal strength of an at least one wireless signal from the wireless access point measured at the device, and further comprising

acquiring the signal strength of the at least one wireless signal from the wireless access point from the metadata attached to the visual data;

and

associating the coordinate on the map with the signal strength of the at least one wireless signal.

3. The method of claim 1, wherein the visual data is further tagged with a round-trip time derived using at least one wireless signal from the wireless access point, and further comprising

acquiring the round-trip time derived using the at least one wireless signal from the wireless access point from the meta data attached to the visual data;

and

associating the coordinate on the map with the round-trip time derived using the at least one wireless signal.

4. The method of claim 1, wherein the visual data is further tagged with barometer information associated with the device and wherein determining the coordinate on the map at which the visual data was captured by the camera coupled to the device comprises using the barometer information associated with the device.

5. The method of claim 1, wherein the visual data is further tagged with magnetometer information associated with the device and wherein determining the coordinate on the map at which the visual data was captured by the camera coupled to the device comprises using the magnetometer information associated with the device.

6. The method of claim 1, wherein the visual data is further tagged with motion data associated with motion of the device and wherein determining the coordinate on the map at which the visual data was captured by the camera coupled to the device comprises using the motion data associated with the device.

7. The method of claim 1, wherein the source identifying information associated with the wireless access point is a media access control (MAC) address.

8. The method of claim 1, wherein the map is for a GNSS-denied environment.

9. The method of claim 1, wherein a location of the wireless access point is unknown at least at a time the visual data is captured by the camera.

10. The method of claim 1, wherein the visual data is included in an image file formatted according to Exchangeable Image File Format (Exif).

11. The method of claim 1, wherein the visual data is a video.

12. A server computing device comprising:

a transceiver configured to receive visual data captured by a camera coupled to a device, wherein the visual data is tagged with source identifying information associated with a wireless access point other than the device by including the source identifying information in metadata attached to the visual data, wherein the metadata describes attributes associated with the visual data;

a memory configured to store the visual data and the metadata;

one or more processors configured to;

acquire the source identifying information associated with the wireless access point from the metadata attached to the visual data;

identify at least one visual feature from the visual data;

determine a coordinate on a map at which the visual data was acquired based on identifying the at least one visual feature from the visual data; and

associate the coordinate on the map with the source identifying information associated with the wireless access point acquired from the metadata attached to the visual data.

13. The server computing device of claim 12, wherein the visual data is further tagged with one or more of signal strength of at least one wireless signal from the wireless access point measured at the device or a round-trip time derived using the at least one wireless signal from the wireless access point, and further comprising the server computing device configured to:

acquire, from the metadata attached to the visual data, the one or more of signal strength of the at least one wireless signal from the wireless access point or the round-trip time derived using the at least one wireless signal from the wireless access point; and

associate the coordinate on the map with the one or more of the signal strength of the at least one wireless signal from the wireless access point or the round-trip time derived using the at least one wireless signal from the wireless access point.

14. The server computing device of claim 12, wherein the map is for a GNSS-denied environment.

15. The server computing device of claim 12, wherein the visual data is included in an image file formatted according to Exchangeable image file format (Exif).

16. The server computing device of claim 12, wherein the visual data is a video.

17. An apparatus comprising:

means for receiving visual data captured by a camera coupled to a device, wherein the visual data is tagged with source identifying information associated with a wireless access point other than the device by including the source identifying information in metadata attached to the visual data, wherein the metadata describes attributes associated with the visual data;

means for acquiring the source identifying information associated with the wireless access point from the metadata attached to the visual data;

means for identifying at least one visual feature from the visual data;

means for determining a coordinate on a map at which the visual data was acquired based on identifying the at least one visual feature from the visual data; and

means for associating the coordinate on the map with the source identifying information associated with the wireless access point acquired from the metadata attached to the visual data.

18. The apparatus of claim 17, wherein the visual data is further tagged with one or more of signal strength of at least one wireless signal from the wireless access point measured at the device or a round-trip time derived using the at least one wireless signal from the wireless access point, and the apparatus further comprising

means for acquiring, from the metadata attached to the visual data, the one or more of signal strength of the at least one wireless signal from the wireless access point or the round-trip time derived using the at least one wireless signal from the wireless access point;

and

means for associating the coordinate on the map with the one or more of the signal strength of the at least one wireless signal from the wireless access point or the round-trip time derived using the at least one wireless signal from the wireless access point.

19. The apparatus of claim 17, wherein the map is for a GNSS-denied environment.

20. The apparatus of claim 17, wherein the visual data is included in an image file formatted according to Exchangeable Image File Format (Exif).

21. The apparatus of claim 17, wherein the visual data is a video.

22. A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises instructions executable by a processor, the instructions comprising instructions to:

receive visual data captured by a camera coupled to a device, wherein the visual data is tagged with source identifying information associated with a wireless access point other than the device by including the source identifying information in metadata attached to the visual data, wherein the metadata describes attributes associated with the visual data;

acquire the source identifying information associated with the wireless access point from the metadata attached to the visual data;

identify at least one visual feature from the visual data;

determine a coordinate on a map at which the visual data was acquired based on identifying the at least one visual feature from the visual data; and

associate the coordinate on the map with the source identifying information associated with the wireless access point acquired from the metadata attached to the visual data.

23. The non-transitory computer-readable medium of claim 22, wherein the visual data is further tagged with one or more of signal strength of at least one wireless signal from the wireless access point measured at the device or a round-trip time derived using the at least one wireless signal from the wireless access point, and the non-transitory computer-readable medium further comprising instructions to:

acquire, from the metadata attached to the visual data, the one or more of signal strength of the at least one wireless signal from the wireless access point or the round-trip time derived using the at least one wireless signal from the wireless access point; and

associate the coordinate on the map with the one or more of the signal strength of the at least one wireless signal from the wireless access point or the round-trip time derived using the at least one wireless signal from the wireless access point.

24. The non-transitory computer-readable medium of claim 22, wherein the map is for a GNSS-denied environment.

25. The non-transitory computer-readable medium of claim 22, wherein the visual data is included in an image file formatted according to Exchangeable Image File Format (Exif).

26. The non-transitory computer-readable medium of claim 22, wherein the visual data is a video.