Multi-Sensor Location and Identification

• US 20110134240A1
• Filed: 12/08/2009
• Published: 06/09/2011
• Est. Priority Date: 12/08/2009
• Status: Active Grant

• Current Assignee: Skyhook Holding, Inc. (Liberty Broadband Corp.)
• Original Assignee: TruePosition Incorporated
• Inventors: Bolon, Brian R., Anderson, Robert J.
• Granted Patent: US 8,531,523 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G01S 2205/01 specially adapted for specific applications
• CPC Class Codes: G01S 5/0264 at least one of the systems being a non-radio wave positioning system
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/38 for collecting sensor information

Abstract

By combining imaging systems with wireless location functionality, a subject'"'"'s videometric signature can be linked to a public identity, thus enabling continuous surveillance outside or between the coverage area of video surveillance networks. In addition to extending the surveillance coverage area, the combination of computerized video surveillance with wireless location determination may also allow for identification of mobile device users via the existing mobile equipment and user identifiers used in the wireless network.

Claims

1. A method of tracking a subject associated with a mobile device, comprising:

processing an image data set and a mobile device location data set to determine a match, wherein the image data set comprises image, time and location information, and the mobile device location data set comprises mobile device identification, time and location information, and wherein a match is indicated when the time and location information from the image and mobile device data sets meet a prescribed degree of correlation; and

in response to determining a match between the image and mobile device data sets, associating a specific mobile device with a specific image.

2. The method of claim 1, further comprising analyzing the specific image to determine a specific personal identity to associate with the specific image and the specific mobile device.

3. The method of claim 2, further comprising determining that the subject to be tracked corresponds to the specific personal identify, and tracking the location of the specific mobile device as a proxy for tracking the subject.

4. The method of claim 1, wherein the image data set comprises video surveillance data.

5. The method of claim 1, wherein the location data set is received from a network-based wireless location system configured to locate wireless devices using radio emissions from the wireless devices.

6. The method of claim 1, wherein the mobile device comprises a wireless communications transceiver for communicating with a wireless communications network.

7. The method of claim 6, wherein the wireless communications network is a local wireless communications network.

8. The method of claim 6, wherein the wireless communications network is a wide area wireless communications network.

9. The method of claim 5, further comprising monitoring entry and exit of the subject with respect to a geo-fence boundary based on the image data set and a mobile device location data set.

10. The method of claim 2, further comprising storing the analyzed specific image, image data set and a mobile device location data set and accumulating statistical data based on the analyzed specific image, image data set and a mobile device location data set.

11. The method of claim 5, further comprising using the image data set to track the subject when the mobile device is not located by the wireless location system.

12. The method of claim 5, further comprising using the mobile device location data set to track the subject when the image data set is not available.

13. The method of claim 3, wherein the subject is a vehicle.

14. The method of claim 3, wherein said image data set is received from a first and second image capture device, further comprising tracking the subject as the subject moves from the first image capture device'"'"'s field of view to the second image capture device'"'"'s field of view, and using image data from the first and second image capture devices to track the subject.

15. The method of claim 1, further comprising determining a direction of movement of the mobile device based on the image data set.

16. The method of claim 1, wherein said image data set is received from a first and second image capture device with overlapping fields of view.

17. The method of claim 16, further comprising determining a velocity of the mobile device based on the image data set received from the first and second image capture devices.

18. The method of claim 1, wherein the method is performed at a choke point.

19. A computer-implemented method of tracking a subject associated with a mobile device, the mobile device having a wireless communications transceiver for communicating with a wireless communications network, the method comprising:

receiving image data from at least one image capture device, wherein the image data comprises at least a portion of an image representing said subject;

receiving identification information and location information for the mobile device, wherein the identification information comprises a mobile device ID and a public ID associated with the mobile device; and

associating the mobile device to the identification information and the location information.

20. The method of claim 19, wherein said associating comprises:

analyzing the received image data; and

determining that the received image data is associated with said mobile device based on the analyzed image data and previously stored image data.

21. The method of claim 20, wherein said determining comprises determining that the received image data is correlated to previously stored image data according to a predetermined correlation level.

22. The method of claim 19, wherein the image data comprises video surveillance data.

23. The method of claim 19, wherein the location information is received from a network-based wireless location system configured to locate wireless devices using radio emissions from the wireless devices.

24. The method of claim 19, wherein the wireless communications network is a local wireless communications network.

25. The method of claim 19, wherein the wireless communications network is a wide area wireless communications network.

26. The method of claim 19, further comprising monitoring entry and exit of the subject with respect to a geo-fence boundary based on the location information and the identification information.

27. The method of claim 21, further comprising storing the analyzed image data, identification information, and location information and accumulating statistical data based on the stored image data, identification information, and location information.

28. The method of claim 23, further comprising using the received image data to track the mobile device when the mobile device is not located by the wireless location system.

29. The method of claim 23, further comprising using the identification information and location information to track the mobile device when the subject is out of a field of view of the image capture device.

30. The method of claim 19, wherein the subject is a vehicle.

31. The method of claim 19, wherein said at least one image capture device comprises a first and second image capture device, further comprising tracking the subject as the subject moves from the first image capture device'"'"'s field of view to the second image capture device'"'"'s field of view, and using image data from the first and second image capture devices to track the subject.

32. The method of claim 19, further comprising determining a direction of movement of the mobile device based on the received image data.

33. The method of claim 19, wherein said at least one image capture device comprises a first and second image capture device with overlapping fields of view.

34. The method of claim 33, further comprising determining a velocity of the mobile device based on the received image data from the first and second image capture devices.

35. The method of claim 19, wherein the method is performed at a choke point.

36. A computer-implemented method of tracking a subject'"'"'s location, wherein the subject is associated with a mobile device having a wireless communications transceiver for communicating with a wireless communications network, the method comprising:

receiving image data from an image capture device associated with a geographic location, wherein the image data comprises an image capturing at least a portion of said subject;

analyzing the received image data and determining that the received image data is associated with the subject;

determining that location information from a location determination service is not available for the mobile device; and

using the geographic location of the image capture device as a location for the subject until said location information becomes available.

37. The method of claim 36, wherein the location information is received from a network-based wireless location system configured to locate wireless devices using radio emissions from the wireless devices.

38. A method of correlating a subject'"'"'s location with a mobile device, the mobile device having a wireless communications transceiver for communicating with a wireless communications network, the method comprising:

receiving image data from at least one image capture device, wherein the image data comprises an image capturing at least a portion of said subject;

analyzing the received image data and determining a signature for said subject based on the analyzed image data;

requesting identification information and location information for the mobile device; and

associating the signature with said identification information and location information.

39. A system configured to track a subject associated with a mobile device, the system comprising at least one processor and at least one storage medium communicatively coupled to said at least one processor, the storage medium having stored therein computer-executable instructions for instructing the processor in causing the following steps:

processing an image data set and a mobile device location data set to determine a match, wherein the image data set comprises image, time and location information, and the mobile device location data set comprises mobile device identification, time and location information, and wherein a match is indicated when the time and location information from the image and mobile device data sets meet a prescribed degree of correlation;

in response to determining a match between the image and mobile device data sets, associating a specific mobile device with a specific image.

40. The system of claim 39, further comprising analyzing the specific image to determine a specific personal identity to associate with the specific image and the specific mobile device.

41. The system of claim 40, further comprising determining that the subject to be tracked corresponds to the specific personal identify, and tracking the location of the specific mobile device as a proxy for tracking the subject.

42. The system of claim 39, wherein the image data set comprises video surveillance data.

43. The system of claim 39, wherein the location data set is received from a network-based wireless location system configured to locate wireless devices using radio emissions from the wireless devices.

44. The system of claim 39, wherein the mobile device comprises a wireless communications transceiver for communicating with a wireless communications network.

45. The system of claim 44, wherein the wireless communications network is a local wireless communications network.

46. The system of claim 44, wherein the wireless communications network is a wide area wireless communications network.

47. The system of claim 43, further comprising monitoring entry and exit of the subject with respect to a geo-fence boundary based on the image data set and a mobile device location data set.

48. The system of claim 40, further comprising storing the analyzed specific image, image data set and a mobile device location data set and accumulating statistical data based on the analyzed specific image, image data set and a mobile device location data set.

49. The system of claim 43, further comprising using the image data set to track the subject when the mobile device is not located by the wireless location system.

50. The system of claim 43, further comprising using the mobile device location data set to track the subject when the image data set is not available.

51. The system of claim 41, wherein the subject is a vehicle.

52. The system of claim 41, wherein said image data set is received from a first and second image capture device, further comprising tracking the subject as the subject moves from the first image capture device'"'"'s field of view to the second image capture device'"'"'s field of view, and using image data from the first and second image capture devices to track the subject.

53. The system of claim 39, further comprising determining a direction of movement of the mobile device based on the image data set.

54. The system of claim 39, wherein said image data set is received from a first and second image capture device with overlapping fields of view.

55. The system of claim 54, further comprising determining a velocity of the mobile device based on the image data set received from the first and second image capture devices.

56. The system of claim 39, wherein the method is performed at a choke point.

57. A system configured to track a subject associated with a mobile device, the mobile device having a wireless communications transceiver for communicating with a wireless communications network, the system comprising at least one processor and at least one storage medium communicatively coupled to said at least one processor, the storage medium having stored therein computer-executable instructions for instructing the processor in causing the following steps:

receiving image data from at least one image capture device, wherein the image data comprises at least a portion of an image representing said subject;

receiving identification information and location information for the mobile device, wherein the identification information comprises a mobile device ID and a public ID associated with the mobile device; and

associating the mobile device to the identification information and the location information.

58. The system of claim 57, wherein said associating comprises:

analyzing the received image data; and

determining that the received image data is associated with said mobile device based on the analyzed image data and previously stored image data.

59. The system of claim 58, wherein said determining comprises determining that the received image data is correlated to previously stored image data according to a predetermined correlation level.

60. The system of claim 57, wherein the image data comprises video surveillance data.

61. The system of claim 57, wherein the location information is received from a network-based wireless location system configured to locate wireless devices using radio emissions from the wireless devices.

62. The system of claim 57, wherein the wireless communications network is a local wireless communications network.

63. The system of claim 57, wherein the wireless communications network is a wide area wireless communications network.

64. The system of claim 57, further comprising monitoring entry and exit of the subject with respect to a geo-fence boundary based on the location information and the identification information.

65. The system of claim 59, further comprising storing the analyzed image data, identification information, and location information and accumulating statistical data based on the stored image data, identification information, and location information.

66. The system of claim 61, further comprising using the received image data to track the mobile device when the mobile device is not located by the wireless location system.

67. The system of claim 61, further comprising using the identification information and location information to track the mobile device when the subject is out of a field of view of the image capture device.

68. The system of claim 57, wherein the subject is a vehicle.

69. The system of claim 57, wherein said at least one image capture device comprises a first and second image capture device, further comprising tracking the subject as the subject moves from the first image capture device'"'"'s field of view to the second image capture device'"'"'s field of view, and using image data from the first and second image capture devices to track the subject.

70. The system of claim 57, further comprising determining a direction of movement of the mobile device based on the received image data.

71. The system of claim 57, wherein said at least one image capture device comprises a first and second image capture device with overlapping fields of view.

72. The system of claim 71, further comprising determining a velocity of the mobile device based on the received image data from the first and second image capture devices.

73. The system of claim 57, wherein the method is performed at a choke point.

74. A system configured to track a subject'"'"'s location, wherein the subject is associated with a mobile device having a wireless communications transceiver for communicating with a wireless communications network, the system comprising at least one processor and at least one storage medium communicatively coupled to said at least one processor, the storage medium having stored therein computer-executable instructions for instructing the processor in causing the following steps:

receiving image data from an image capture device, wherein the image data comprises an image capturing at least a portion of said subject;

analyzing the received image data and determining that the received image data is associated with the subject;

determining that location information from a location determination service is not available for the mobile device; and

using the geographic location of the image capture device as a location for the subject until said location information becomes available.

75. The system of claim 74, wherein the location information is received from a network-based wireless location system configured to locate wireless devices using radio emissions from the wireless devices.

76. A system configured to correlate a subject'"'"'s location with a mobile device, the mobile device having a wireless communications transceiver for communicating with a wireless communications network, the system comprising at least one processor and at least one storage medium communicatively coupled to said at least one processor, the storage medium having stored therein computer-executable instructions for instructing the processor in causing the following steps:

receiving image data from at least one image capture device, wherein the image data comprises an image capturing at least a portion of said subject;

analyzing the received image data and determining a signature for said subject based on the analyzed image data;

requesting identification information and location information for the mobile device; and

associating the signature with said identification information and location information.

77. A computer readable storage medium storing thereon computer executable instructions for tracking a subject associated with a mobile device, said computer executable instructions for:

processing an image data set and a mobile device location data set to determine a match, wherein the image data set comprises image, time and location information, and the mobile device location data set comprises mobile device identification, time and location information, and wherein a match is indicated when the time and location information from the image and mobile device data sets meet a prescribed degree of correlation;

in response to determining a match between the image and mobile device data sets, associating a specific mobile device with a specific image.

78. The computer readable storage medium of claim 77, further comprising instructions for analyzing the specific image to determine a specific personal identity to associate with the specific image with the specific mobile device.

79. The computer readable storage medium of claim 78, further comprising instructions for determining that the subject to be tracked corresponds to the specific personal identify, and tracking the location of the specific mobile device as a proxy for tracking the subject.

80. The computer readable storage medium of claim 77, wherein the image data set comprises video surveillance data.

81. The computer readable storage medium of claim 77, wherein the location data set is received from a network-based wireless location system configured to locate wireless devices using radio emissions from the wireless devices.

82. The computer readable storage medium of claim 77, wherein the mobile device comprises a wireless communications transceiver for communicating with a wireless communications network.

83. The computer readable storage medium of claim 82, wherein the wireless communications network is a local wireless communications network.

84. The computer readable storage medium of claim 82, wherein the wireless communications network is a wide area wireless communications network.

85. The computer readable storage medium of claim 81, further comprising instructions for monitoring entry and exit of the subject with respect to a geo-fence boundary based on the image data set and a mobile device location data set.

86. The computer readable storage medium of claim 78, further comprising instructions for analyzed specific image, image data set and a mobile device location data set and accumulating statistical data based on the analyzed specific image, image data set and a mobile device location data set.

87. The computer readable storage medium of claim 81, further comprising instructions for using the image data set to track the mobile device when the mobile device is not located by the wireless location system.

88. The computer readable storage medium of claim 81, further comprising instructions for using the mobile device location data set to track the mobile device when the image data set is not available.

89. The computer readable storage medium of claim 79, wherein the subject is a vehicle.

90. The computer readable storage medium of claim 79, wherein said image data set is received from a first and second image capture device, further comprising tracking the subject as the subject moves from the first image capture device'"'"'s field of view to the second image capture device'"'"'s field of view, and using image data from the first and second image capture devices to track the subject.

91. The computer readable storage medium of claim 77, further comprising instructions for determining a direction of movement of the mobile device based on the image data set.

92. The computer readable storage medium of claim 77, wherein said image data set is received from a first and second image capture device with overlapping fields of view.

93. The computer readable storage medium of claim 92, further comprising instructions for determining a velocity of the mobile device based on the received image data set from the first and second image capture devices.

94. The computer readable storage medium of claim 77, wherein the method is performed at a choke point.

95. A computer readable storage medium storing thereon computer executable instructions for tracking a subject associated with a mobile device, the mobile device having a wireless communications transceiver for communicating with a wireless communications network, said computer executable instructions for:

receiving image data from at least one image capture device, wherein the image data comprises at least a portion of an image representing said subject;

receiving identification information and location information for the mobile device, wherein the identification information comprises a mobile device ID and a public ID associated with the mobile device; and

associating the mobile device to the identification information and the location information.

96. The computer readable storage medium of claim 95, wherein said associating comprises:

analyzing the received image data; and

determining that the received image data is associated with said mobile device based on the analyzed image data and previously stored image data.

97. The computer readable storage medium of claim 96, wherein said determining comprises determining that the received image data is correlated to previously stored image data according to a predetermined correlation level.

98. The computer readable storage medium of claim 95, wherein the image data comprises video surveillance data.

99. The computer readable storage medium of claim 95, wherein the location information is received from a network-based wireless location system configured to locate wireless devices using radio emissions from the wireless devices.

100. The computer readable storage medium of claim 95, wherein the wireless communications network is a local wireless communications network.

101. The computer readable storage medium of claim 95, wherein the wireless communications network is a wide area wireless communications network.

102. The computer readable storage medium of claim 95, further comprising instructions for monitoring entry and exit of the subject with respect to a geo-fence boundary based on the location information and the identification information.

103. The computer readable storage medium of claim 97, further comprising instructions for storing the analyzed image data, identification information, and location information and accumulating statistical data based on the stored image data, identification information, and location information.

104. The computer readable storage medium of claim 99, further comprising instructions for using the received image data to track the mobile device when the mobile device is not located by the wireless location system.

105. The computer readable storage medium of claim 99, further comprising instructions for using the identification information and location information to track the mobile device when the subject is out of a field of view of the image capture device.

106. The computer readable storage medium of claim 95, wherein the subject is a vehicle.

107. The computer readable storage medium of claim 95, wherein said at least one image capture device comprises a first and second image capture device, further comprising tracking the subject as the subject moves from the first image capture device'"'"'s field of view to the second image capture device'"'"'s field of view, and using image data from the first and second image capture devices to track the subject.

108. The computer readable storage medium of claim 95, further comprising instructions for determining a direction of movement of the mobile device based on the received image data.

109. The computer readable storage medium of claim 95, wherein said at least one image capture device comprises a first and second image capture device with overlapping fields of view.

110. The computer readable storage medium of claim 109, further comprising instructions for determining a velocity of the mobile device based on the received image data from the first and second image capture devices.

111. The computer readable storage medium of claim 110, wherein the method is performed at a choke point.

112. A computer readable storage medium storing thereon computer executable instructions for tracking a subject'"'"'s location, wherein the subject is associated with a mobile device having a wireless communications transceiver for communicating with a wireless communications network, said computer executable instructions for:

receiving image data from an image capture device, wherein the image data comprises an image capturing at least a portion of said subject;

analyzing the received image data and determining that the received image data is associated with the subject;

determining that location information from a location determination service is not available for the mobile device; and

using the geographic location of the image capture device as a location for the subject until said location information becomes available.

113. The computer readable storage medium of claim 112, wherein the location information is received from a network-based wireless location system configured to locate wireless devices using radio emissions from the wireless devices.

114. A computer readable storage medium storing thereon computer executable instructions for correlating a subject'"'"'s location with a mobile device, the mobile device having a wireless communications transceiver for communicating with a wireless communications network, said computer executable instructions for:

receiving image data from at least one image capture device, wherein the image data comprises an image capturing at least a portion of said subject;

analyzing the received image data and determining a signature for said subject based on the analyzed image data;

requesting identification information and location information for the mobile device; and

associating the signature with said identification information and location information.