Inferring positions with content item matching

• US 9,641,977 B2
• Filed: 07/24/2015
• Issued: 05/02/2017
• Est. Priority Date: 12/02/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: DiAcetis, Stephen P., Lin, Jyh-Han
• Primary Examiner(s): Brandt, Christopher M
• Application Number:
  US14/809,157
• Publication Number:
  US 20160029170A1
• Time in Patent Office: 648 Days
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04W 4/023 using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/33 for indoor environments, e.g. buildings
• CPC Class Codes: H04W 4/60 Subscription-based services using application servers or record carriers, e.g. SIM application toolkits

Abstract

Embodiments infer position information using captured content items. A computing device such as a mobile computing or a cloud service analyzes one or more content items to generate information describing the content items. For content items that include photographs, for example, the photographs are processed to identify faces, objects, places, text, etc. The generated information is compared to reference content items (e.g., image comparison), user data (e.g., calendar data, contact information, etc.), and/or public data (e.g., address information, event information, etc.) to identify at least one match. Location information associated with the matched reference content items, user data, and/or public data is obtained. A position is determined for the content items based on the location information obtained from the matched reference content item, user data, and/or public data. In some embodiments, the obtained location information is used to refine a position determined using a beacon-based positioning service.

Claims

1. A system for supplementing a beacon-based position determination with image matching, said system comprising:

a memory area associated with a mobile computing device, said memory area configured to store one or more images captured by a user of the mobile computing device; and

a processor programmed to;

capture, at a device position, an image using the mobile computing device;

observe, at the device position, at least one beacon proximate to the mobile computing device;

compare location information of the captured image with the at least one observed beacon to determine a location of the device position;

based on the comparing, receive, from a cloud service, a determination of the location of the device position; and

associate the determined location of the device position with the captured image.

2. The system of claim 1, wherein the processor is programmed to observe the at least one beacon without using a global positioning system.

3. The system of claim 1, wherein the mobile computing device is indoors.

4. The system of claim 1, wherein the processor is further programmed to present the received determination of the location of the device position to the user for confirmation before associating the determined location of the device position with the captured image.

5. The system of claim 1, further comprising means for inferring the device position based on captured images and observed beacons.

6. The system of claim 1, further comprising means for determining the device position based on information extracted from the captured images.

7. A method comprising:

capturing, at a device, an image using a mobile computing device;

observing, at the device position, at least one beacon proximate to the mobile computing device;

compare location information of the captured image with at least one observed beacon to determine a location of the device position;

based on comparing, receiving, from a cloud service, a determination of the location of the device position; and

associating the determined location of the device position with the captured image.

8. The method of claim 7, further comprising observing the at least one beacon without using a global positioning system.

9. The method of claim 7, wherein the capturing of the image using a mobile computing device is executed indoors.

10. The method of claim 7, further comprising presenting the received determination of the location of the device position to a user for confirmation before associating the determined location of the device position with the one or more captured images.

11. The method of claim 7, further comprising associating the determined location of the device position with the one or more other images stored in a memory area.

12. The method of claim 7, wherein the determination of the location of the device position is further based on a distance of an identified object in the image from the mobile computing device at a time the image was captured.

13. The method of claim 7, refining the determined location of the device position by:

associating the captured image with one or more images stored on the mobile computing device;

accessing location information for the one or more images stored on the mobile computing device; and

refining the determined location of the device position based on the location information for the one or more images stored on the mobile computing device.

14. The method of claim 13, further comprising displaying the refined device position on a map in a user interface of the mobile computing device.

15. The method of claim 7, further comprising

capturing, at a second device position, one or more additional images using the mobile computing device;

observing, at the second device position, a second beacon proximate to the mobile computing device;

providing the one or more additional captured images and identifying the second observed beacon to the cloud service, the cloud service determining the second device position based on the observed second beacon and the one or more additional captured images;

receiving, from the cloud service, a refined device position based on the device position and the second device position; and

associating the refined device position with the captured image and the one or more additional captured images.

16. One or more computer storage media embodying computer-executable components, said components comprising:

a memory component that causes at least one processor to store a plurality of reference content items, the memory component further storing location information associated with the plurality of reference content items;

an interface component that causes at least one processor to access a content item captured by a computing device;

a signal processing component that causes at least one processor to compare the content item accessed by the interface component to one or more of the plurality of reference content items stored by the memory component, the signal processing component further identifying at least one of the plurality of reference content items based on the comparison; and

a positioning service component that causes at least one processor to determine a position for the accessed content item based on the location information associated with the identified content item, and determining a location of the computing device based on the location information for the identified at least one of the plurality of reference content items and from a position of a beacon corresponding to the computing device.

17. The computer storage media of claim 16, wherein the interface component accesses the content item by receiving the content item from the computing device, and wherein the interface component further sends the position determined by the positioning service component to the computing device.

18. The computer storage media of claim 16, wherein the location information for at least one of the reference content items is calculated from a position of a beacon corresponding to the reference content item.

19. The computer storage media of claim 16, wherein the content items comprise one or more of the following:

image data, video data, audio data, and accelerometer data.

20. The computer storage media of claim 16, wherein the memory component stores the plurality of reference content items received from a plurality of computing devices.

21. The computer storage media of claim 16, wherein the positioning service component further associates the determined location of the computing device with the accessed content item.