Enhancing indoor positioning using RF multilateration and optical sensing

• US 10,375,667 B2
• Filed: 12/07/2017
• Issued: 08/06/2019
• Est. Priority Date: 12/07/2017
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Kothari, Shanay Ravin, Snyder, Ian Matthew
• Primary Examiner(s): Blount, Eric
• Application Number:
  US15/834,990
• Publication Number:
  US 20190182790A1
• Time in Patent Office: 607 Days
• Field of Search: 3405391-53918
• CPC Class Codes: G01S 5/0252 Radio frequency fingerprinting
• CPC Class Codes: G01S 5/0264 at least one of the systems being a non-radio wave positioning system
• CPC Class Codes: G01S 5/14 Determining absolute distances from a plurality of spaced points of known location
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04B 7/0857 using maximum ratio combining techniques, e.g. signal-to- interference ratio [SIR], received signal strenght indication [RSS]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04W 64/00 Locating users or terminals or network equipment for network management purposes, e.g. mobility management
• CPC Class Codes: H04W 88/085 Access point devices with remote components

Abstract

Systems and methods are disclosed for enhancing target positioning. A broadcast signal from a device is received by at one or more access points, where a first position area of the device is determined from an analysis of the broadcast signal. A second position area of the target, which is within the first position area, is determined by scanning pixels within the first position area in an image captured by the camera and, based on the scanned pixels, at least one target comprising a portion of the pixels is detected. The target within the image is classified, and based on the classification and portion of the pixels comprising the target, the second position area of the target within the first position of the image is triangulated.

Claims

1. A system for enhancing target positioning comprising:

an access point for capturing a broadcast signal from a device, wherein a first position area of the device is determined by analyzing the broadcast signal; and

a camera in communication with the access point to receive the first position area from the access point, the camera including computer readable instructions stored thereon that are effective to;

scan pixels within the received first position area in an image captured by the camera;

based on the scanned pixels, detect at least one target within a portion of the pixels, wherein the target can be comprised of at least one of the device or a person associated with the device;

classify the target within the image; and

based on the classification and portion of the pixels including the target, triangulate a second position area of the at least one target within the first position area of the image.

2. The system of claim 1, wherein at least two access points capture the broadcast signal from the device, and wherein the first position area of the device is determined by analyzing the broadcast signal according to a multilateration model.

3. The system of claim 1, wherein the classification of the target further comprises instructions to:

identify a broadcast signature within the broadcast signal; and

identify a type of the device based on comparing the broadcast signature with a plurality of stored broadcast signatures with associated device information.

4. The system of claim 1, wherein the classification of the target further comprises instructions to:

identify a type of the device based on machine vision applied to at least one of the first position area or the second position area.

5. The system of claim 3, wherein an identifier that identifies the vendor of the device is determined from the broadcast signature, and wherein the type of the device is determined from the vendor.

6. The system of claim 1, the instructions effective to:

identify the device based on an application installed on the device, wherein the application is associated with a user assigned to the device;

classify the target in the second area as a first person; and

based on machine vision applied to the first person, determine that there is a discrepancy in possession of the device by identifying that the first person is not the user assigned to the device.

7. The system of claim 1, the instructions effective to:

receive a broadcast signature within the broadcast signal from the first position area;

identify a type of the device based on machine vision applied to the first position area; and

update a broadcast signature store based on the received broadcast signature and identified type of device.

8. The system of claim 7, wherein the updated broadcast signature dynamically updates a multilateration model.

9. The system of claim 1, the instructions effective to:

identify, based on the broadcast signal captured by the access point, that a plurality of devices associated with a plurality of targets are within the first position area;

determine that at least one device within the plurality of devices cannot be visually differentiated by machine vision;

track movement of the plurality of targets within the image as a cohesive block; and

apply a differencing model to the broadcast signal to extract broadcast signals for each device.

10. The system of claim 1, the instructions effective to:

identify, based on the broadcast signal captured by the access point, that a plurality of devices associated with a plurality of targets are within the first position area;

determine that at least one device within the plurality of devices cannot be visually differentiated using a machine vision technique;

track movement of the plurality of targets within the image;

determine, using the machine vision technique, that at least one target within the plurality of targets is moving in a direction that diverges from the plurality of targets;

for the at least one target, visually identify the associated device; and

remove, from the broadcast signal, the broadcast signature associated with the visually identified device.

11. The system of claim 1, the instructions effective to:

identify, based on the broadcast signal captured by the access point, that a plurality of devices associated with a plurality of targets are within the first position area;

determine that at least one device within the plurality of devices cannot be visually differentiated using a machine vision technique;

based on no diverging movement within the plurality of targets, assign all broadcast signals a group identifier and average the broadcast signal.

12. The system of claim 1, the instructions effective to:

determine that a machine vision technique cannot detect the device associated with the target;

track the target; and

when the device is detectable using the machine vision technique, identify the device.

13. The system of claim 1, wherein the classification of the target is based on a model in the cloud.

14. A method of enhancing target positioning comprising:

receiving, from an access point, a broadcast signal from a device, wherein a first position area of the device is determined from an analysis of the broadcast signal;

determining a second position area of the target within the first position area by;

scanning pixels within the received first position area in an image captured by the camera;

based on the scanned pixels, detecting at least one target comprising a portion of the pixels, wherein the target can be comprised of at least one of the device or a person associated with the device;

classifying the target within the image; and

based on the classification and portion of the pixels comprising the target, triangulating the second position area of the target within the first position area of the image.

15. The method of claim 14, wherein receiving the broadcast signal comprises capturing the broadcast signal from the device with at least two access points, and wherein the first position area of the device is determined by analyzing the broadcast signal according to a multilateration model.

16. The method of claim 14, the method further comprising:

identifying the device based on an application installed on the device, wherein the application is associated with a user assigned to the device;

classifying the target in the second area as a first person; and

based on machine vision applied to the first person, determining that there is a discrepancy in possession of the device by identifying that the first person is not the user assigned to the device.

17. The method of claim 14, the method further comprising:

identifying, based on the broadcast signal, that a plurality of devices associated with a plurality of targets are within the first position area;

determining that at least one device within the plurality of devices cannot be visually differentiated by machine vision;

tracking movement of the plurality of targets within the image;

determining, by machine vision, that at least one target within the plurality of targets is moving in a direction that diverges from the plurality of targets;

for the at least one target, visually identifying the associated device; and

removing, from the broadcast signal, the broadcast signature associated with the visually identified device.

18. A camera comprising:

at least one receiver for receiving, from an access point, a broadcast signal from a device, wherein a first position area of the device is determined from an analysis of the broadcast signal; and

at least one processor and at least one memory containing instructions that, when executed, cause the at least one processor to determine a second position area of the target within the first position area by;

scanning pixels within the received first position area in an image captured by the camera;

based on the scanned pixels, detecting at least one target comprising a portion of the pixels, wherein the target can be comprised of at least one of the device or a person associated with the device;

classifying the target within the image; and

based on the classification and portion of the pixels comprising the target, triangulating the second position area of the target within the first position of the image.

19. The camera of claim 18, wherein at least two access points capture the broadcast signal from the device, and wherein the first position area of the device is determined by analyzing the broadcast signal according to a multilateration model.

20. The camera of claim 18, wherein the at least one processor is further configured to:

identify the device based on an application installed on the device, wherein the application is associated with a user assigned to the device;

classify the target in the second area as a first person; and

based on machine vision applied to the first person, determine that there is a discrepancy in possession of the device by identifying that the first person is not the user assigned to the device.