System and method for data collection to validate location data

• US 8,972,357 B2
• Filed: 02/24/2012
• Issued: 03/03/2015
• Est. Priority Date: 02/24/2012
• Status: Active Grant

• Current Assignee: Foursquare Labs, Inc.
• Original Assignee: Placed, Inc. (Foursquare Labs, Inc.)
• Inventors: Shim, David, Bicknell, Brad, Varghese, George, Gerner, Nick, Yi, Weilie
• Primary Examiner(s): Elhag, Magdi
• Application Number:
  US13/405,182
• Publication Number:
  US 20130225202A1
• Time in Patent Office: 1,103 Days
• Field of Search: None
• US Class Current: 707/690
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

A data collection system that provides the means to collect, store and make data available for a location analytics inference pipeline. The system incorporates a feedback mechanism enabling algorithms produced from the inference pipeline to drive the collection strategy to yield higher data quality and to produce reference data for model validation.

Claims

1. An apparatus for providing geolocation data, the apparatus coupled to receive data from multiple mobile devices wirelessly coupled to a network, the apparatus comprising:

at least one data storage device for storing geolocation data received from the multiple mobile devices,

wherein the geolocation data includes place data and corresponding latitude and longitude coordinate data, and

wherein at least some of the place data includes names of buildings located at corresponding latitude and longitude coordinates;

at least one server computer coupled to the data storage device, wherein the server computer is configured to receive observation data from the multiple mobile devices and to

infer place names for locations of the multiple mobile devices based on the received observation data;

provide sets of questions to individual users of the multiple mobile devices,

wherein each set of questions includes a request for a user response and includes multiple inferred place names that may correspond to a latitude and longitude location where one of the mobile devices is or was located;

receive, from individual users of the multiple mobile devices, place name responses to the sets of questions,

wherein each response includes a selection of one of the multiple place names for the location where the one mobile device is or was located;

store, in the data storage device, the received place name responses from the individual users in response to the sets of questions,

wherein the received place name responses are associated in the data storage device with corresponding latitude and longitude coordinate data, and

wherein the received place name responses are utilized in improving accuracy with which place names for locations of the multiple mobile devices are inferred.

2. The apparatus of claim 1,

wherein the observation data includes

device data identifying an individual mobile device;

timestamp data associated with the received observation data;

location data for the individual mobile device, wherein the location data includes at least three of;

latitude, longitude, altitude, accuracy of measurement, bearing, or speed of the mobile device; and

sensor data from sensors of the individual mobile device, wherein the sensor data includes at least one of;

acceleration, magnetic orientation, proximity, light intensity, battery status, gyroscope or temperature;

infer place names for locations of the multiple mobile devices based on the received observation data, wherein at least some of the inferred place names are rank ordered based on an inferred probability; and

,

provide in each of set of questions the multiple place names in a predetermined order.

3. The apparatus of claim 1,

wherein the observation data includes at least two of

device data identifying an individual mobile device;

timestamp data associated with the received observation data;

location data for the individual mobile device;

or

sensor data from sensors of the individual mobile device.

4. The apparatus of claim 1, wherein the server computer is further configured to provide compensation to at least some users who provide responses to the provided sets of questions.

5. The apparatus of claim 2, wherein the predetermined order includes ordering the multiple place names based on an estimated accuracy.

6. The apparatus of claim 2, wherein the predetermined order includes ordering the multiple place names based on a frequency of past visits.

7. A non-transitory computer readable storage medium storing instructions for collecting data that validates a geolocation of a user of a mobile device, the computer readable storage medium comprising:

instructions for storing geolocation data received from the mobile device, wherein the geolocation data includes latitude and longitude coordinate data;

instructions for prompting the user of the mobile device for a place confirmation,

wherein the place confirmation includes a place name selected from a list of possible place names that may correspond to a latitude and longitude location where the mobile device is or was located, and

wherein the place name includes a name of a venue located at corresponding latitude and longitude coordinates;

instructions for storing, in a database, the place confirmation,

wherein the received place confirmation is associated in the database with corresponding latitude and longitude coordinate data;

instructions for collecting sensor data from sensors of the mobile device to validate the associated latitude and the associated longitude coordinate data,

wherein the sensor data includes at least one of;

acceleration, magnetic orientation, microphone, proximity, image sensor, ambient light sensor, battery status, gyroscope, or temperature; and

instructions for providing a feedback to the mobile device based on characteristics of the sensor data to adjust collection of the sensor data and the geolocation data.

8. The computer readable storage medium of claim 7, further comprising instructions for extracting device activity data to validate the associated latitude and the associated longitude coordinate data from one of:

a geo-tagged image, a user profile, email, calendar appointment, call log, or SMS activity.

9. The computer readable storage medium of claim 7, further comprising instructions for extracting purchase data to validate the associated latitude and the associate longitude coordinate data, wherein purchase data includes at least one of:

bar code scans, credit card activity, or club membership activity.

10. The computer readable storage medium of claim 7, further comprising instructions for extracting social networking activity and content data to validate the associated latitude and the associated longitude coordinate data.

11. The computer readable storage medium of claim 7, further comprising instructions for utilizing digital log data to validate the associated latitude and the associate longitude coordinate data, wherein the digital log data includes place, date, and time information.

12. An apparatus for providing geolocation data to a server, comprising:

at least one memory;

at least one radio;

input/output components;

at least one processor coupled to the memory, radio and input/output components, wherein the apparatus is configured to;

transmit geolocation data to the server, wherein the geolocation data includes latitude and longitude coordinate data;

receive from the server a set of questions,

wherein the set of questions includes a request for a user to respond and includes multiple place names that may correspond to a latitude and longitude location where the apparatus is or was located,

wherein included among the multiple place names in the set of questions is a place name that is an incorrect choice; and

wherein at least some of the place names includes names of buildings located at corresponding latitude and longitude coordinates; and

transmit a selection of one of the place names to the server in response to the set of questions,

wherein the response includes a selection of one of the multiple place names for the location where the apparatus is or was located; and

wherein when the response corresponds to the place name that is an incorrect choice, the response is disregarded; and

wherein when the response corresponds to a place name other than the place name that is an incorrect choice, the response is stored in the data storage device in association with the latitude and longitude location where the apparatus is or was located.

13. The apparatus of claim 12, further configured to:

transmit observation data, wherein the observation data includes at least two of

device data identifying the apparatus;

timestamp data associated with received observation data;

location data for the apparatus;

or

sensor data from sensors of the apparatus.

14. The apparatus of claim 12, further configured to receive an indication of compensation in response to providing to the server a response to the provided set of questions.

15. A method for receiving data from multiple mobile devices wirelessly coupled to a network, comprising:

storing, in a data storage device, geolocation data received from multiple mobile devices, wherein the geolocation data includes place data and corresponding latitude and longitude coordinate data;

receiving observation data from the multiple mobile devices;

generating, based on an analysis of the observation data, possible place names for locations of the multiple mobile devices;

providing sets of questions to individual users of the multiple mobile devices,

wherein each set of questions includes a request for a user response and includes multiple possible place names corresponding to a latitude and longitude location where one of the multiple mobile devices is or was located;

receiving, from individual users of the multiple mobile devices, place name responses to the sets of questions,

wherein each response includes a selection of one of the multiple possible place names for the location where the one mobile device is or was located;

storing, in the data storage device, the received place name responses from the individual users in response to the sets of questions in association with corresponding latitude and longitude coordinate data; and

utilizing the place names responses to improve accuracy with which possible place names for locations of the multiple mobile devices are generated.

16. The method of claim 15, wherein at least some of the place data includes names of buildings located at corresponding latitude and longitude coordinates.

17. The method of claim 15,

wherein at least some of the possible place names are rank ordered based on an inferred probability;

and,

providing in each of set of questions the multiple place names in a predetermined order.

18. The method of claim 17, wherein the observation data includes:

device data identifying an individual mobile device;

timestamp data associated with the received observation data;

location data for the individual mobile device; and

sensor data from sensors of the individual mobile device.

19. The method of claim 18, wherein the location data includes at least three of:

latitude, longitude, altitude, accuracy of measurement, bearing, or speed of the mobile device.

20. The method of claim 18, wherein the sensor data includes at least one of:

acceleration, magnetic orientation, proximity, light intensity, battery status, gyroscope or temperature.

21. The method of claim 17,

wherein the observation

data includes at least two of

device data identifying an individual mobile device;

timestamp data associated with the received observation data;

location data for the individual mobile device;

or

sensor data from sensors of the individual mobile device.

22. The method of claim 15, further comprising:

providing compensation to at least some users who provide responses to the provided sets of questions.

23. The method of claim 17, wherein the predetermined order includes ordering the multiple place names based on an estimated accuracy.

24. The method of claim 17, wherein the predetermined order includes ordering the multiple place names based on a frequency of past visits.

25. The method of claim 15, further comprising:

including a place name that is an incorrect choice among the multiple place names included in at least one of the sets of questions.

26. A system for receiving data from multiple mobile devices wirelessly coupled to a network, comprising:

at least one memory;

at least one processor disposed in communication with the memory and configured to execute multiple instructions stored in the memory to;

store, in a data storage device, geolocation data received from multiple mobile devices,

wherein the geolocation data includes place data and corresponding latitude and longitude coordinate data, and

receive observation data from the multiple mobile devices, wherein the observation data includes at least two of;

device data identifying an individual mobile device;

timestamp data associated with the received observation data;

location data for the individual mobile device; and

sensor data from sensors of the individual mobile device;

generate possible place names for locations of the multiple mobile devices based on the received observation data;

provide sets of questions to individual users of the multiple mobile devices;

wherein each set of questions includes a request for a user response and includes multiple possible place names corresponding to a latitude and longitude location where one of the mobile devices is or was located;

receive, from individual users of the multiple mobile devices, place name responses to the sets of questions,

wherein each response includes a selection of one of the multiple place names for the location where the one mobile device is or was located;

store, in the data storage device, the received place name responses from the individual users in response to the sets of questions,

wherein the received place name responses are associated in the data storage device with corresponding latitude and longitude coordinate data;

utilize the place name responses to improve accuracy with which possible place names for locations of the multiple mobile devices are generated.

27. The system of claim 26, further comprising:

provide in each of set of questions the multiple place names in a predetermined order, wherein the predetermined order includes ordering the multiple place names based on an estimated accuracy.

28. The system of claim 26, wherein a place name that is an incorrect choice is included among the multiple place names to identify users that provide false data.

29. The system of claim 26, wherein the location data includes at least three of:

latitude, longitude, altitude, accuracy of measurement, bearing, or speed of the mobile device.

30. The system of claim 26, wherein the sensor data includes at least one of:

acceleration, magnetic orientation, proximity, light intensity, battery status, gyroscope or temperature.

31. The system of claim 26, wherein a place name that is an incorrect choice is included among the multiple place names to identify users that provide false data.