Target localization utilizing wireless and camera sensor fusion

• US 8,615,254 B2
• Filed: 08/17/2011
• Issued: 12/24/2013
• Est. Priority Date: 08/18/2010
• Status: Active Grant

• Current Assignee: RetailNext, Inc.
• Original Assignee: Nearbuy Systems, Inc (RetailNext, Inc.)
• Inventors: Jamtgaard, Mark, Mueller, Nathan
• Primary Examiner(s): PHAM, TIMOTHY X
• Application Number:
  US13/211,969
• Publication Number:
  US 20120046044A1
• Time in Patent Office: 860 Days
• Field of Search: 455/456.1
• US Class Current: 455/456.1
• CPC Class Codes: G01S 17/00 Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G01S 5/02585 at least one of the measurements being a non-radio measurement
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/025 using location based information parameters
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 84/12 WLAN [Wireless Local Area Networks]

Abstract

According to some implementations, an estimate of a target'"'"'s location can be calculated by correlating Wi-Fi and video location measurements. This spatio-temporal correlation combines the Wi-Fi and video measurements to determine an identity and location of an object. The accuracy of the video localization and the identity from the Wi-Fi network provide an accurate location of the Wi-Fi identified object.

Claims

1. A method comprising:

obtaining, by a computing device, image data associated with an object;

estimating, by the computing device, a first location of the object based on the image data and a first probability grid, where the image data is used to generate a value for each cell of the first probability grid that indicates a probability that a first probability grid cell is occupied by the object;

obtaining, by the computing device, wireless signal data from a device;

estimating, by the computing device, a second location of the device based on the wireless signal data and a second probability grid, where the wireless signal data is used to generate a value for each cell of the second probability grid that indicates a probability that a second probability grid cell is occupied by the device; and

multiplying each of the first probability grid cells with respective cells of the second probability grid to determine which two corresponding grid cells produce a largest probability across the first and second probability grids; and

estimating, by the computing device, a current location of the device based on corresponding grid cells that produce the largest probability.

2. The method of claim 1, where estimating the first location of the object based on the image data comprises:

transforming the image data through a ground plane homography projection.

3. The method of claim 1, wherein the image data comprises images captured by different cameras having overlapping fields of view.

4. The method of claim 1, wherein estimating the first location of the object comprises calculating an object track corresponding to the object.

5. The method of claim 1, wherein obtaining wireless signal data from the device includes obtaining device identification information from the wireless signal data.

6. A system comprising:

one or more processors; and

a computer-readable medium including one or more sequences of instructions which,

when executed by the one or more processors, causes;

obtaining image data associated with an object;

estimating a first location of the object based on the image data and a first probability grid, where the image data is used to generate a value for each cell of the first probability grid that indicates a probability that a first probability grid cell is occupied by the object;

obtaining wireless signal data from a device;

estimating a second location of the device based on the wireless signal data and a second probability grid, where the wireless signal data is used to generate a value for each cell of the second probability grid that indicates a probability that a second probability grid cell is occupied by the device;

multiplying each of the first probability grid cells with respective cells of the second probability grid to determine which two corresponding grid cells produce a largest probability across the first and second probability grids; and

estimating a current location of the device based on the corresponding grid cells that produce the largest probability.

7. The system of claim 6, wherein the instructions that cause estimating the first location of the object based on the image data comprise instructions that cause transforming the image data through a ground plane homography projection.

8. The system of claim 6, wherein the image data comprises images captured by different cameras having overlapping fields of view.

9. The system of claim 6, wherein the instructions that cause estimating the first location of the object comprise instructions that cause calculating an object track corresponding to the object.

10. The system of claim 6, wherein the instructions that cause obtaining wireless signal data from the device comprise instructions that cause obtaining device identification information from the wireless signal data.

11. A non-transitory computer-readable medium including one or more sequences of instructions which, when executed by one or more processors, causes:

obtaining image data associated with an object;

estimating a first location of the object based on the image data and a first probability grid, where the image data is used to generate a value for each cell of the first probability grid that indicates a probability that a first probability grid cell is occupied by the object;

obtaining wireless signal data from a device;

estimating a second location of the device based on the wireless signal data and a second probability grid, where the wireless signal data is used to generate a value for each cell of the second probability grid that indicates a probability that a second probability grid cell is occupied by the device;

multiplying each of the first probability grid cells with respective cells of the second probability grid to determine which two corresponding grid cells produce a largest probability across the first and second probability grids; and

estimating a current location of the device based on the corresponding grid cells that produce the largest probability.

12. The non-transitory computer-readable medium of claim 11, wherein the instructions that cause estimating the first location of the object based on the image data comprise instructions that cause transforming the image data through a ground plane homography projection.

13. The non-transitory computer-readable medium of claim 11, wherein the image data comprises images captured by different cameras having overlapping fields of view.

14. The non-transitory computer-readable medium of claim 11, wherein the instructions that cause estimating the first location of the object comprise instructions that cause calculating an object track corresponding to the object.

15. The non-transitory computer-readable medium of claim 11, wherein the instructions that cause obtaining wireless signal data from the device comprise instructions that cause obtaining device identification information from the wireless signal data.