Video data and GIS mapping for traffic monitoring, event detection and change prediction

• US 10,223,911 B2
• Filed: 10/31/2016
• Issued: 03/05/2019
• Est. Priority Date: 10/31/2016
• Status: Active Grant

• Current Assignee: Echelon Corporation (Renesas Electronics Corporation)
• Original Assignee: Echelon Corporation (Renesas Electronics Corporation)
• Inventors: Modi, Sohrab, Singh, Shalender
• Primary Examiner(s): Sweeney, Brian P
• Application Number:
  US15/338,633
• Publication Number:
  US 20180122231A1
• Time in Patent Office: 855 Days
• Field of Search: 701117
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06F 16/738 Presentation of query results
• CPC Class Codes: G08G 1/0116 from roadside infrastructure, e.g. beacons
• CPC Class Codes: G08G 1/0129 for creating historical data or processing based on historical data
• CPC Class Codes: G08G 1/0133 for classifying traffic situation
• CPC Class Codes: G08G 1/0145 for active traffic flow control
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/048 with provision for compensation of environmental or other condition, e.g. snow, vehicle stopped at detector
• CPC Class Codes: G08G 1/08 according to detected number or speed of vehicles
• CPC Class Codes: G08G 1/095 Traffic lights
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04W 4/70 Services for machine-to-machine communication [M2M] or machine type communication [MTC]
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
• CPC Class Codes: H04W 64/006 with additional information processing, e.g. for direction or speed determination
• CPC Class Codes: H04W 84/042 Public Land Mobile systems, e.g. cellular systems
• CPC Class Codes: H04W 84/12 WLAN [Wireless Local Area Networks]

Abstract

A distributed system for traffic monitoring and event detection includes a plurality of video units within viewing distance of a thoroughfare to be monitored. Each video unit processes a video stream while monitoring events at the thoroughfare, identifies and analyzes events, encodes traffic meta data characterizing the analysis, and transmits the meta data to a central controller. The central controller stores a geographic map representing the thoroughfare, correlates the meta data received from the video units, and forms a composite representation of the geographic map and the correlated meta data. The central controller replays previously stored composite representations of the geographic map and the correlated meta data, which were monitored under mutually different event conditions. The central controller is also configured to allow changes to previously stored composite representations of the geographic map and to the correlated meta data for the purpose of understanding the impact of change.

Claims

1. A distributed system for traffic monitoring and event detection, comprising:

a plurality of video units located within viewing distance of at least one thoroughfare to be monitored, each video unit including a video camera, video frame processing logic, a processor and memory including computer program code configured to cause the video frame processing logic to process a video stream from at least one video camera while monitoring predefined categories of events at the at least one thoroughfare, to identify traffic monitoring events associated with the at least one thoroughfare, to analyze the traffic monitoring events, and to encode traffic meta data characterizing the analysis of the traffic monitoring events, each-video unit including a communications unit configured to transmit to a central controller the meta data characterizing analysis of the traffic monitoring events;

a central controller, including a processor and memory including computer program code configured to cause the central controller store at least one geographic map representing the at least one thoroughfare to be monitored, to cause the controller to correlate the traffic monitoring meta data received from the plurality of video units, and to cause the controller to form a composite representation of the at least one geographic map and a representation of the correlated traffic monitoring meta data received from the plurality of video units; and

the central controller including an archive storage configured to store a plurality of records of the composite representations of the at least one geographic map and representations of the correlated traffic monitoring meta data, each of the plurality of records corresponding to the at least one thoroughfare being monitored under mutually different event conditions.

2. The system of claim 1, further comprising:

the central controller including a playback unit configured to replay a sub-plurality of the a plurality of records of the composite representations of the at least one geographic map and representations of the correlated traffic monitoring meta data, monitored under mutually different event conditions.

3. The system of claim 2, wherein the mutually different event conditions include changing timing of traffic lights, replacing traffic lights with roundabouts, changing traffic flows, changing the direction or magnitude of traffic flow, widening the road, adding another road to spread out traffic, changing stop signs to traffic lights, making streets one-way, or changing ambient weather conditions.

4. The system of claim 1, further comprising:

the communications unit in the at least one video unit being configured to transmit the low bandwidth event message to the central controller over a communications medium selected from the group consisting of twisted pair, coax cable, Ethernet, WiFi, Bluetooth, cellular, IEEE 802.15.4 RF, Bluetooth Low Energy, ultra narrow band communications protocol from Sigfox, LTE-M, any Low Power Wireless Area Network (LPWAN) protocol, any M2M communication protocols or LoRa Low Power Wide Area Network; and

the central controller including a communications unit configured to receive the low bandwidth event message from the at least one video unit over the communications medium.

5. The system of claim 1, further comprising:

the communications unit in the at least one video unit being a power line communications unit configured to transmit a low bandwidth event message over installed power lines to the central controller;

the central controller including a communications unit being a power line communications unit, configured to receive the low bandwidth event message over installed power lines from the at least one video unit; and

the power line communications unit in the central controller further configured to transmit or receive data communications over installed power lines to electrically powered devices in the thoroughfare where power is needed and control desired.

6. The system of claim 1, further comprising:

a sensor located at the at least one video unit, configured to sense distance and motion of a vehicle, the at least one video unit, configured to prepare a low bandwidth event message representing the vehicle'"'"'s position and motion sensed by the sensor.

7. The system of claim 1, further comprising:

a plurality of video cameras located at the at least one video unit, configured to analyze and combine multiple views of a same event and only send a reduced amount of information to the central controller, the at least one video unit, configured to prepare a low bandwidth event message representing the reduced amount of information.

8. The system of claim 1, further comprising:

the central controller including artificial intelligence (AI) components to enable simulation of changes to environmental factors or events and the meta data replayed as a simulation of original events.

9. The system of claim 8, wherein the artificial intelligence (AI) components include a reference background model and an inference engine logic.

10. The system of claim 8, wherein the artificial intelligence (AI) components enable simulation of changing timing of traffic lights, replacing traffic lights with roundabouts, changing traffic flows, changing the direction or magnitude of traffic flow, widening the road, adding another road to spread out traffic, changing stop signs to traffic lights, making streets one-way, changing ambient weather conditions, changing street intersections, simulating nighttime, simulating daytime, or simulating vehicle activities.

11. The system of claim 8, wherein meta data directly from the video unit are combined with simulation of changes to environmental factors or events to do what-ifs investigations.

12. A method for traffic monitoring and event detection, comprising:

processing a video stream from at least one video camera in a video unit while monitoring predefined categories of events at an at least one thoroughfare, to identify traffic monitoring events associated with the at least one thoroughfare, to analyze the traffic monitoring events, and to encode traffic meta data characterizing the analysis of the traffic monitoring events, the at least one video unit including a communications unit configured to transmit to a central controller the meta data characterizing analysis of the traffic monitoring events;

storing at least one geographic map representing the at least one thoroughfare to be monitored, correlating the traffic monitoring meta data received from a plurality of video units, and forming a composite representation of the at least one geographic map and a representation of the correlated traffic monitoring meta data received from the plurality of video units;

storing a plurality of records of the composite representations of the at least one geographic map and representations of the correlated traffic monitoring meta data, each of the plurality of records corresponding to the at least one thoroughfare being monitored under mutually different event conditions; and

replaying a sub-plurality of the a plurality of records of the composite representations of the at least one geographic map and representations of the correlated traffic monitoring meta data, monitored under mutually different event conditions.