System and method of personalized navigation inside a business enterprise

• US 10,634,506 B2
• Filed: 10/21/2019
• Issued: 04/28/2020
• Est. Priority Date: 12/12/2016
• Status: Active Grant

• Current Assignee: Position Imaging, Inc.
• Original Assignee: Position Imaging, Inc.
• Inventors: Hill, Edward L.
• Primary Examiner(s): Perez, Julio R
• Application Number:
  US16/658,951
• Publication Number:
  US 20200049509A1
• Time in Patent Office: 190 Days
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 30/0259 based on store location
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: H04B 17/27 for locating or positioning the transmitter
• CPC Class Codes: H04B 17/318 Received signal strength
• CPC Class Codes: H04W 4/021 Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/33 for indoor environments, e.g. buildings

Abstract

Systems and methods for tracking movement of individuals through a building receive, by one or more RF nodes disposed near an entrance to the building, RF signals from RF-transmitting mobile devices carried by persons near the entrance, capture an image of the persons while they are near the entrance, determine an identity and relative distance of each RF-transmitting mobile device from each RF node based on information associated with the RF signals received by that RF node, detect humans in the image, determine a relative depth of each human in the image, and assign the identity of each RF-transmitting mobile device to one of the humans detected in the image based on the relative distance of each RF-transmitting mobile device from each RF node and the relative depth of each human in the image, thereby identifying each individual who to be tracked optically as that individual moves throughout the building.

Claims

1. A system for tracking locations of individuals in a building, the system comprising:

at least one radiofrequency (RF) node disposed near an entrance to the building, the at least one RF node having an RF receiver to receive RF signals from RF-transmitting devices near the entrance to the building;

at least one optical device disposed near the entrance to the building, the at least one optical device capturing an image of a plurality of persons while the plurality of persons is near the entrance to the building; and

a controller in communication with the at least one RF node to obtain therefrom information associated with the RF signals received by the RF receiver of that at least one RF node and in communication with the at least one optical device to obtain therefrom the captured image,

the controller being configured to determine an identity of each RF-transmitting device and an angular position of that RF-transmitting device with respect to each RF node of the at least one RF node based on the information associated with the RF signals obtained by the controller from that RF node of the at least one RF node,

the controller being further configured to detect a plurality of humans in the image obtained by the controller from the at least one optical device, to detect in the image an orientation marker, and to assign the identity of each RF-transmitting device to one of the plurality of humans detected in the image based on a position of the orientation marker in the image relative to each human detected in the image and on the determined angular position of that RF-transmitting device with respect to each RF node of the at least one RF node, thereby identifying each individual who is to be located optically as that individual moves throughout the building.

2. The system of claim 1, wherein the at least one RF node is configured to determine a relative signal strength indicator (RSSI) value for the RF signals received from each RF-transmitting device, the information obtained by the controller from the at least one RF node includes the RSSI values, and the controller is configured to estimate a distance of each RF-transmitting device from each RF node based on the RSSI values for the RF signals received by that RF node from that RF-transmitting device and to use the estimated distance of each RF-transmitting device from each RF node when assigning the identity of each RF-transmitting device to one of the plurality of humans detected in the image.

3. The system of claim 1, wherein the controller is further configured to detect in the image a mobile phone held by one of the plurality of humans detected in the image and to assign the identity of each RF-transmitting device to one of the plurality of humans detected in the captured image based on the which of the plurality of humans detected in the image is holding the detected mobile phone.

4. The system of claim 1, wherein the controller is further configured to:

arrange, for each RF node, the RF-transmitting mobile devices in an angular order based on the angular positions of the RF-transmitting mobile devices from that RF node;

arrange the humans captured in the image into an angular order based on relative positions of the plurality of humans detected in the captured image; and

compare the angular order of humans detected in the captured image with the angular order of the RF-transmitting mobile devices when assigning the identity of each RF-transmitting mobile device to one of the plurality of humans detected in the captured image.

5. The system of claim 4, wherein the controller is further configured to arrange the angular order of the plurality of humans in the captured image to be in respect to the at least one RF node before comparing the angular order of humans detected in the image with the angular order of the RF-transmitting mobile devices.

6. The system of claim 1, further comprising a plurality of optical devices disposed throughout the building, and wherein the controller is further configured to optically track each identified individual through the building based on detecting that identified individual in images received from at least some of the plurality of optical devices over time.

7. The system of claim 1, wherein the information carried by RF signals transmitted by a given RF-transmitting mobile device includes a shopping list.

8. The system of claim 7, wherein the controller is further configured to:

determine a route through the building based on items on the shopping list; and

transmit the route to the given RF-transmitting mobile device for display on a screen of the given RF-transmitting mobile device.

9. The system of claim 7, wherein the controller is further configured to:

compute a location for each RF-transmitting mobile device based on the angular positions of the RF-transmitting mobile devices received from the at least one RF node;

arrange the RF-transmitting mobile devices in a depth order based on the computed locations of the RF-transmitting mobile devices with respect to the at least one RF node;

arrange the humans captured in the image into a depth order based on relative positions of the plurality of humans detected in the captured image; and

assign the identity of each RF-transmitting mobile device to one of the plurality of humans detected in the captured image by comparing the depth order of humans detected in the image with the depth order of the RF-transmitting mobile devices.

10. A method for tracking locations of individuals in a building, the method comprising:

receiving, by at least one radiofrequency (RF) node disposed near an entrance to the building, RF signals from RF-transmitting mobile devices carried by a plurality of persons near the entrance to the building;

capturing an image of the plurality of persons while the plurality of persons is near the entrance to the building;

determining an identity of each RF-transmitting mobile device and an angular position of each RF-transmitting mobile device from each RF node based on information associated with the RF signals received by the at least one RF node;

detecting a plurality of humans and an orientation marker in the captured image;

assigning the identity of each RF-transmitting mobile device to one of the plurality of humans detected in the captured image based on a position of the orientation marker in the image relative to each human detected in the image and on the determined angular position of each RF-transmitting mobile device with respect to each RF node, thereby identifying each individual who is to be located optically as that individual moves throughout the building.

11. The method of claim 10, further comprising:

determining a relative signal strength indicator (RSSI) value for the RF signals received from each RF-transmitting device;

computing an estimated distance of each RF-transmitting device from each RF node based on the RSSI values for the RF signals received by that RF node from that RF-transmitting device; and

using the estimated distance of each RF-transmitting device from each RF node when assigning the identity of each RF-transmitting device to one of the plurality of humans detected in the image.

12. The method of claim 10, further comprising detecting in the image a mobile phone held by one of the plurality of humans detected in the image;

and wherein the step of assigning the identity of each RF-transmitting device to one of the plurality of humans detected in the captured image is based on the which human detected in the image is holding the detected mobile phone.

13. The method of claim 11, further comprising:

arranging the RF-transmitting mobile devices in an angular order based on the angular positions of the RF-transmitting mobile devices with respect to the at least one RF node; and

arranging the humans captured in the image into an angular order based on relative positions of the plurality of humans detected in the captured image; and

wherein the step of assigning the identity of each RF-transmitting mobile device to one of the plurality of humans detected in the captured image includes comparing the angular order of humans detected in the image with the angular order of the RF-transmitting mobile devices.

14. The method of claim 13, further comprising arranging the angular order of humans in the captured image to be in respect to the at least one RF node before comparing the angular order of humans detected in the captured image with the angular order of the RF-transmitting mobile devices.

15. The method of claim 10, further comprising optically tracking a location of each identified individual through the building by comparing successive images that capture that identified individual.

16. The method of claim 10, wherein the information associated with the RF signals transmitted by a given RF-transmitting mobile device includes a shopping list.

17. The method of claim 16, further comprising:

determining a route through the building based on items on the shopping list; and

transmitting the route to the given RF-transmitting mobile device for display on a screen of the given RF-transmitting mobile device.

18. The method of claim 10, further comprising transmitting from the at least one RF node radio signals carrying a current location of a given identified individual within the building to the RF-transmitting mobile device carried by that given identified individual.

19. The method of claim 10, further comprising:

computing a location for each RF-transmitting mobile device based on the angular positions of the RF-transmitting mobile devices received from the at least one RF node;

arranging the RF-transmitting mobile devices in a depth order based on the computed locations of the RF-transmitting mobile devices with respect to the at least one RF node; and

arranging the humans captured in the image into a depth order based on relative positions of the plurality of humans detected in the captured image; and

wherein the step of assigning the identity of each RF-transmitting mobile device to one of the plurality of humans detected in the captured image includes comparing the depth order of humans detected in the image with the depth order of the RF-transmitting mobile devices.

20. A system for tracking locations of individuals in a building, the system comprising:

at least one radiofrequency (RF) node disposed near an entrance to the building, the at least one RF node having an RF receiver channel to receive RF signals from a plurality of RF-transmitting devices near the entrance to the building;

at least one camera disposed near the entrance to the building, the at least one camera capturing an image of a plurality of persons while the plurality of persons is near the entrance to the building; and

a controller in communication with the at least one RF node to obtain therefrom information associated with the RF signals received by the RF receiver channel of that at least one RF node and in communication with the at least one camera to obtain therefrom the captured image, the controller being configured to determine an identity of each RF-transmitting device and a position of that RF-transmitting device relative to each RF node of the at least one RF node based on the information associated with the RF signals obtained by the controller from that RF node of the at least one RF node, to detect a plurality of humans in the image obtained by the controller from the at least one camera, to detect an orientation marker in the image, and to assign the identity of each RF-transmitting device to one of the plurality of humans detected in the image based on a position of the orientation marker in the image relative to each human detected in the image and on the determined position of that RF-transmitting device relative to each RF node of the at least one RF node.