LiDAR sensing system

• US 10,288,737 B2
• Filed: 09/19/2017
• Issued: 05/14/2019
• Est. Priority Date: 09/19/2017
• Status: Active Grant

• Current Assignee: CrowdVision, Inc.
• Original Assignee: Wirelesswerx International, Inc.
• Inventors: Mooney, Patrick, Gibson, Raymond V.
• Primary Examiner(s): Abraham, Samantha K
• Application Number:
  US15/708,219
• Publication Number:
  US 20190086545A1
• Time in Patent Office: 602 Days
• CPC Class Codes: G01S 17/06 Systems determining position data of a target
• CPC Class Codes: G01S 17/66 Tracking systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/88 Lidar systems specially adapted for specific applications
• CPC Class Codes: G01S 7/484 Transmitters
• CPC Class Codes: G01S 7/486 Receivers

Abstract

LiDAR ranging devices include an optical transmitter, and an optical receiver assembly. There is one LiDAR, and there can be more LiDARs each covering a field of sensing. The LiDAR devices are mounted within a building on a support structure in spaced relationship from each other. The fields of sensing from each LiDAR collectively provide an enhanced field of sensing of an area in the building. Each LiDAR and its base are tuned to sense human traffic. A base collects sensed data from each LiDAR such that tracking information and movement of human traffic over the enhanced area is received by the base. The fields of sensing of each LiDAR can overlap, and an analytic program stitches the sensed data from each LiDAR to obtain a stitched map of sensed data over the enhanced area. Barriers to line of site between humans to be sensed and LiDAR devices are minimized by the LiDARs.

Claims

1. A three-dimensional LiDAR device sensing system comprising at least two time-of-flight LiDAR ranging devices, each LiDAR device includes an optical transmitter, an optical receiver assembly, power regulation electronics, control electronics, data conversion electronics, and processing electronics;

and each LiDAR device covering a field or area of sensing of human traffic subjects, the two devices being mounted within a building on a support structure in spaced relationship from each other, the fields of sensing from each LiDAR device being at least adjacent the other whereby collectively an enhanced field or area of sensing of an area in the building is achieved, each LiDAR device being tuned to sense human traffic, a unit for collecting sensed data from each LiDAR device such that tracking information and movement of human traffic over the enhanced area is received by the unit, the sensing data of each LiDAR device being stitched by a stitching analytical process with data from the other data from the associated LiDAR device to provide an enlarged sensing profile, and wherein the stitching analytic process is effected on a principal of providing a predicted movement of sensing of the human traffic subjects being sensed in a queue, such that when movement of separate sensing of movement of the human traffic subjects does not occur within a predetermined time, the stitching analytic process projects representative movement of the subjects in the queue including inputs of tracking information being (i) counts the number of subjects that enter and exit a queue, (ii) generating real time velocity vectors for selected parts of the queue, (iii) detecting subject population density of the selected parts of the queue, and (iv) rendering sensed subjects to follow a path based on this processed analysis of the inputs (i);

(ii) and (iii), thereby to obtain data on queue performance without the need to track each subject all of the time.

2. The apparatus as claimed in claim 1 wherein such analytic process includes a first in/first out (FIFO) analysis.

3. The apparatus as claimed in claim 1 wherein the fields of sensing of each LiDAR device overlap each other, and an analytic program for stitching the sensed data from each LiDAR device to obtain a stitched map of sensed data over the enhanced area.

4. The apparatus as claimed in claim 1 wherein in at least one of the sensed areas there is barrier to a first line of sight to at least one LiDAR device, such that a human located on a side of the barrier remote from the location of the one LiDAR device is not sensed, and wherein the second LiDAR device is located with a second line of sight relative to the barrier to cover at least part of the blocked first line of sight of the first LiDAR device.

5. The apparatus as claimed in claim 4 wherein in at least one of the sensed areas the barrier to a first line of sight is a security body scanner in an area being an airport terminal.

6. The apparatus as claimed in claim 4 wherein in at least one of the sensed areas the barrier to a first line of sight is selectively at least one of wall or pillar.

7. The apparatus of claim 1 including a network comprising a single or a plurality of LiDAR device lasers, LiDAR laser drivers, LiDAR laser controllers, LiDAR optical amplifiers, LiDAR optical detectors, LiDAR receiver electronics, LiDAR power regulation electronics, LiDAR control electronics, LiDAR data conversion electronics, and LiDAR data processing electronics, each LiDAR device is located in a different geographical area of a building to provide the enhanced area of the building, and each LiDAR device is connected to transmit sensed data to the unit, whereby the unit analyzes the data to obtain tracking information of the movement of human traffic over the enhanced area.

8. The apparatus of claim 1 including separately from the LiDAR and for separate sensing of movement of the human traffic subjects at least a plurality of a visible video camera, an infrared video camera, an ultrasonic sensor, a cell modem and a Wi-Fi node.

9. The apparatus of claim 8 wherein the output is a relatively smooth queue line data of human traffic subjects including at least one of where they are in the queue, how fast they are moving in the queue, and how many subjects are in the queue relative to unprocessed unstitched input data of the subjects to the LiDAR system.

10. The apparatus of claim 1 wherein the output is a relatively smooth queue line data of human traffic subjects including at least one of where they are in the queue, how fast they are moving in the queue, and how many subjects are in the queue relative to unprocessed unstitched input data of the subjects to the LiDAR system.

11. A three-dimensional LiDAR device sensing system comprising at least two time-of-flight LiDAR ranging devices, each LiDAR device includes an optical transmitter, an optical receiver assembly, power regulation electronics, control electronics, data conversion electronics, and processing electronics;

and each LiDAR device covering a field or area of sensing, the two devices being mounted within a building on a support structure in spaced relationship from each other, the fields of sensing from each LiDAR device being at least adjacent the other whereby collectively an enhanced field or area of sensing of an area in the building is achieved, each LiDAR device being tuned to sense human traffic, a unit for collecting sensed data from each LiDAR device such that tracking information and movement of human traffic over the enhanced area is received by the unit, the sensing data of each LiDAR device being stitched by an analytical process with data from the other data from the associated LiDAR device to provide an enlarged sensing profile, for providing a tracking service to an operator of a facility;

the devices providing data about the movement of the humans in the facility relative to the location such data including at least one of entry into the location, departure from the location, amount of time spent in the vicinity of the location;

the unit receiving the data from the devices and photographs from a camera to constitute flow control information;

a location server for receiving location data of a communication device associated with a human, the location data defining the detected position of the communication device at a number of different points in time, the location server further being arranged to receive data associated with the location data indicative of the order in which the location data was determined;

wherein the server is further arranged to compare the received location data and area data defining a plurality of areas and to associate the received location data with one of the plurality of areas; and

determining the human'"'"'s path passing through an area; and

the tracking service receiving data from the unit, the flow control information of the photographs and the location data from the location server.

12. The apparatus of claim 11 wherein the facility includes multiple checking locations wherein humans leaving the facility need to pass through at least one of the checking locations prior to leaving the facility, and including locating the LiDAR devices in a physical location to be related to different respective checking locations of the multiple checking locations and wherein data from the location LiDAR devices includes at least one of the number of entities passing through different checking locations, the speed with which entities pass through the different checking locations, the number of entities in the vicinity of the different respective checking locations.

13. The apparatus of claim 12 wherein the facility includes a sensor for counting humans at or passing through or past selected locations.

14. The apparatus of claim 12 wherein the facility is an airport terminal and the checking facility is a security checking station, and selectively there are multiple stations.

15. A three-dimensional LiDAR device sensing system comprising at least two time-of-flight LiDAR ranging devices, each LiDAR device includes an optical transmitter, an optical receiver assembly, power regulation electronics, control electronics, data conversion electronics, and processing electronics;

and each LiDAR device covering a field or area of sensing, the two devices being mounted within a building on a support structure in spaced relationship from each other, the fields of sensing from each LiDAR device being at least adjacent the other whereby collectively an enhanced field or area of sensing of an area in the building is achieved, each LiDAR device being tuned to sense human traffic, a unit for collecting sensed data from each LiDAR device such that tracking information and movement of human traffic over the enhanced area is received by the unit, the sensing data of each LiDAR device being stitched by an analytical process with data from the other data from the associated LiDAR device to provide an enlarged sensing profile, wherein the plurality of areas having LiDAR device search areas are located on different floors of a building, the floors being in different vertically disposed relationship to each other, and wherein the analytic process effects stitching of data from the different floors, thereby to track humans moving between the different floors.

16. The apparatus of claim 15 and in addition to the analytical process effecting stitching of the data from the different floor levels, the analytic process effects stitching of data on each of respective multiple floor levels thereby to track humans moving between the different floors and on each floor.

17. A three-dimensional LiDAR device sensing system for providing a tracking service to an operator of an airport facility-building, the airport facility comprising a door for entry into the facility from an outside position from the facility, a first area between the door and a security checking location within the facility, the security checking location, and a second area beyond the security checking location and providing information of movement of humans from the first area through the security checking location and into the second area, including applying data from:

at least one time-of-flight light detection and ranging sensor (‘

LiDAR”

) ranging device, the LiDAR device includes an optical transmitter, an optical receiver assembly, power regulation electronics, control electronics, data conversion electronics, and processing electronics; and

the LiDAR device covering a field or area of sensing of a zone, the device being mounted within a building on a support structure, the LiDAR device providing data from the field or area of sensing of the zone, such data including from the LiDAR device being for sensing human traffic from the sensed data, and providing a sensing profile of tracking information and movement of human traffic over the field or area, the sensing data of the LiDAR device being in an analytical process to provide a sensing profile, the profile additionally including data relating to characteristics of the field or area of sensing in the building, and wherein the profile additionally includes data relating to characteristics of a normal regular expected pattern or sequence of activity in the building, applying artificial intelligence software to enable a perception of objects based on data collected by one or more LiDAR sensors, and software for enabling detection, measurement, tracking, identification and classification of objects, and further selectively triggering actions based on real-time scenario analysis in 3D.

18. The three-dimensional LiDAR device sensing system as claimed in claim 17 comprising at least two time-of-flight LiDAR ranging devices, and each LiDAR device covering a field or area of sensing, the two devices being mounted within a building on a support structure in spaced relationship from each other, the fields of sensing from each LiDAR device being at least adjacent the other whereby collectively an enhanced field or area of sensing of an area in the building is achieved, the sensing data of each LiDAR device being stitched by an analytical process with data from the other data from the associated LiDAR device to provide an enlarged sensing profile,

a location node within the facility, the location node including a transceiver of wireless signals, and transmitting the signals received to a base for processing the received signals;

the location node being located in a specific security checking location within the facility, the node wireless communicating with movable humans within the facility thereby to establish the location and movement of humans within the facility, the humans having wireless communicating units transmitting and receiving signals wirelessly with the node, each unit being unique for each human and communicating with the node when the human enters at least one area;

the location node including a detection device, the device including a radio transmitter/receiver, the device being capable of detecting the first area entry, dwell and exit events of the human respectively through a communication with the wireless communicating unit of respective human entities, the wireless communicating units equipped with selectively a Bluetooth and WI-FI radio; and

the tracking service receiving data from the unit, and the location data from the location node.

19. The apparatus of claim 18 wherein the enhanced area of the airport is a defined area effectively covered for tracking collectively by using a lesser number of LiDAR devices and a greater number of nodes over the defined area, the lesser number being by at least a factor of 2 and preferably, a factor of 4.

20. The apparatus of claim 18 wherein the node is for receiving location data of a communication device associated with a human, the location data defining the detected position of the communication device at a number of different points in time, the location server further being arranged to receive data associated with the location data indicative of the order in which the location data was determined;

and a server connected with t the node and arranged to compare the received location data and area data defining a plurality of areas and to associate the received location data with one of the plurality of areas; and

determining the human'"'"'s path passing through an area.

21. The system of claim 17 wherein the LiDAR device is tuned to sense human traffic, a unit for collecting sensed data from the LiDAR device such that tracking information and movement of human traffic over the enhanced area is received by the unit, the stitching analytic process being effected on a principal of providing a predicted movement of sensing of the human traffic subjects being sensed in a queue, such that when movement of separate sensing of movement of the human traffic subjects does not occur within a predetermined time, the stitching analytic process projects representative movement of the subjects in the queue including inputs of tracking information being (i) counts the number of subjects that enter and exit a queue, (ii) generating real time velocity vectors for selected parts of the queue, (iii) detecting subject population density of the selected parts of the queue, and (iv) rendering sensed subjects to follow a path based on this processed analysis of the inputs (i);

(ii) and (iii), thereby to obtain data on queue performance without the need to track each subject all of the time.