Pedestrian tracking at a traffic intersection to identify vulnerable roadway users for traffic signal timing, pedestrian safety, and traffic intersection control

• US 9,805,474 B1
• Filed: 03/27/2017
• Issued: 10/31/2017
• Est. Priority Date: 05/09/2016
• Status: Active Grant

• Current Assignee: Iteris Incorporated
• Original Assignee: Iteris Incorporated
• Inventors: Whiting, Michael T., Gao, Yan, Swaminathan, Dilip, Shivakumar, Shashank Jayaram, Kreter, Todd W.
• Primary Examiner(s): Azarian, Seyed
• Application Number:
  US15/470,627
• Publication Number:
  US 20170323448A1
• Time in Patent Office: 218 Days
• Field of Search: 382100, 382103-107, 382151, 382155, 382162, 382168, 382172-173, 382181, 382199, 382203, 382209, 382219, 382224, 382232, 382254, 382274, 382276, 382286-291, 382312, 382305, 701 93, 701118, 340902, 340922
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G08G 1/005 including pedestrian guidance indicator
• CPC Class Codes: G08G 1/0116 from roadside infrastructure, e.g. beacons
• CPC Class Codes: G08G 1/0129 for creating historical data or processing based on historical data
• CPC Class Codes: G08G 1/015 with provision for distinguishing between two or more types of vehicles, e.g. between motor-cars and cycles
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/091 Traffic information broadcasting broadcasting communication H04H
• CPC Class Codes: G08G 1/096 provided with indicators in which a mark progresses showing the time elapsed, e.g. of green phase
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

Pedestrian detection and counting for traffic intersection control analyzes characteristics of a field of view of a traffic detection zone to determine a location and size of a pedestrian area, and applies protocols for evaluating pixel content in the field of view to identify individual pedestrians. The location and size of a pedestrian area is determined based either on locations of vehicle and bicycle detection areas or on movement of various objects within the field of view. Automatic pedestrian speed calibration with a region of interest for pedestrian detection is accomplished using lane and other intersection markings in the field of view. Detection and counting further includes identifying a presence, volume, velocity and trajectory of pedestrians in the pedestrian area of the traffic detection zone.

Claims

1. A method, comprising:

receiving input data representing a) a field of view of a traffic detection zone at a roadway intersection and defining at least a portion of a safe passage zone for vulnerable roadway users at the roadway intersection, and b) traffic signal timing information for the roadway intersection that includes an allotted time for crossing the roadway intersection, the allotted time defined by one or both of a total pedestrian signal crossing time and a start time of a pedestrian signal indicating safe passage for a pedestrian at the roadway intersection;

analyzing the input data within a computing environment in one or more hardware and software components executed in conjunction with at least one specifically-configured processor, the one or more hardware and software components configured to perform data processing functions to determine a state of the pedestrian'"'"'s presence in the field of view, by;

analyzing a plurality of pixels in the field of view to identify the safe passage zone, wherein a left edge and a right edge of the field of view define at least a portion of the safe passage zone,

measuring pedestrian movement through the safe passage zone, by continually tracking pedestrian position as the pedestrian moves through the field of view, and continually estimating a current pedestrian speed as the pedestrian moves through the field of view,

determining a distance to the end of the safe passage zone from the pedestrian position, and dividing the distance to end of the safe passage zone by the current pedestrian speed to calculate an estimated pedestrian time of arrival at the end of the safe passage zone,

comparing the estimated pedestrian time of arrival at the end of the safe passage zone with an estimated time remaining for the pedestrian signal determined from the allotted time for crossing the traffic intersection, to predict a pedestrian'"'"'s ability to cross the safe passage zone within the allotted time; and

generating, as output data, one or more instructions to a traffic signal controller in response to the state of the pedestrian'"'"'s presence in the field of view.

2. The method of claim 1, wherein the one or more instructions include at least one controller signal to set a state of one or more traffic signal lights at each approach of the roadway intersection.

3. The method of claim 2, further comprising comparing the current pedestrian speed with one or more of a normal safe passage zone crossing speed and an average safe passage zone crossing speed to identify a type of pedestrian in the field of view, and further identify the type of pedestrian as an adult moving a regular speed, or as a member of a protected group, the protected group including an adult moving at a reduced speed, a child, a pedestrian moving with an external object, and a pedestrian using a mechanical movement aid.

4. The method of claim 3, wherein the one or more instructions include an instruction to the traffic signal controller to allow the member of the protected group to complete passage through the safe passage zone, and an instruction to the traffic signal controller to warn traffic approaching the roadway intersection that a member of a protected group remains in the safe passage zone.

5. The method of claim 1, wherein the one or more instructions include an instruction to the traffic signal controller to change a state of one or more traffic signal lights at the roadway intersection to allow the pedestrian to complete passage through the safe passage zone, an instruction to the traffic signal controller to hold a state of one or more traffic signal lights at the roadway intersection to allow the pedestrian to complete passage through the safe passage zone, and an instruction to the traffic signal controller to warn traffic approaching the roadway intersection that a pedestrian remains in the safe passage zone.

6. The method of claim 1, further comprising generating, as output data, one or more traffic indicators representing the state of the pedestrian'"'"'s presence in the field of view, wherein the one or more indicators include an indicator that a pedestrian will pass through the safe passage zone before the allotted time expires, an indicator that a pedestrian will not pass through the safe passage zone before the allotted time expires, an indicator that the pedestrian changed direction and exited the safe passage zone before reaching the end of the safe passage zone, an indicator that the pedestrian reversed direction and returned to the beginning of the safe passage zone, an indicator that the pedestrian stopped moving, and an indicator the pedestrian remains in the safe passage zone after the allotted time expires.

7. The method of claim 1, wherein the safe passage zone is identified within the field of view by a) ascertaining a region of interest for pedestrian tracks based on at least one of lane structures and intersection road markings, and on movement of pixels representing moving objects relative to the at least one of lane structures and intersection road markings, b) determining accumulated tracks of the moving objects in the field of view by analyzing motion strength and frequency of activity of pixels representing the moving objects in field of view, and c) tracking pedestrian characteristics that include one or more of size, gestures, speed, entry points, and exit points in the region of interest to distinguish the accumulated tracks of the moving objects from the pedestrian tracks.

8. The method of claim 1, wherein the safe passage zone is identified within the field of view by a) identifying a position of at least one vehicle detection zone in nearest proximity to a stop bar, each at least one vehicle detection zone having a height that extends to or near to the stop bar, and b) calculating a height of a pedestrian area in the field of view from the height of the at least one vehicle detection zone.

9. The method of claim 1, further comprising detecting a pedestrian in the safe passage zone within the field of view, by a) analyzing portions of the safe passage zone in the field of view, b) computing features of current pixel content identified in the analyzed portions by identifying part-based features defining an individual pedestrian that include one or more of body structure combinations, body shape, body width or walking gestures, c) developing a model of a single walking pedestrian to separate each individual pedestrian in a group of moving pedestrians in the field of view, d) computing pedestrian features using pixels defining a pedestrian contour, and e) determining a matching confidence between an individual pedestrian and a group of moving pedestrians by calculating a mathematical similarity between the computed features of current pixel content and the model of the single walking pedestrian.

10. The method of claim 1, further comprising detecting a non-pedestrian object within the safe passage zone during the allotted time for crossing the roadway intersection, comparing the non-pedestrian object with the state of the pedestrian'"'"'s presence in the field of view, and generating a warning where the non-pedestrian object is within the safe passage zone at a same time as a pedestrian.

11. The method of claim 1, wherein the input data representing the field of view of the traffic detection zone is captured by one or more sensors in or near a traffic intersection, the one or more sensors including at least one of video cameras, a radar system, and a magnetometer.

12. A method of pedestrian tracking for traffic signal control at a roadway intersection, comprising:

within a computing environment including one or more hardware and software components, executed in conjunction with at least one specifically-configured processor configured to perform a plurality of data processing functions, the plurality of data processing functions including;

analyzing a plurality of pixels in a field of view of a traffic detection zone at a roadway intersection to a) identify a safe passage zone for vulnerable roadway users at the roadway intersection, wherein a left edge and a right edge of the field of view define at least a portion of the safe passage zone, and b) to detect a pedestrian in the field of view;

measuring pedestrian movement through the safe passage zone, by continually tracking pedestrian position as the pedestrian moves through the field of view, and continually estimating a current pedestrian speed as the pedestrian moves through the field of view;

determining a distance to the end of the safe passage zone from the pedestrian position, and dividing the distance to end of the safe passage zone by the current pedestrian speed to calculate an estimated pedestrian time of arrival at the end of the safe passage zone;

comparing the estimated pedestrian time of arrival at the end of the safe passage zone with an estimated time remaining for a pedestrian signal determined from an allotted time for crossing the roadway intersection, the allotted time defined by one or both of a total pedestrian signal crossing time and a start time of a pedestrian signal indicating safe passage for a pedestrian at the roadway intersection, to predict a pedestrian'"'"'s ability to cross the safe passage zone within the allotted time;

determining a state of the pedestrian'"'"'s presence in the field of view from the predicted ability to cross the safe passage zone within the allotted time; and

sending one or more instructions to a traffic signal controller, the one or more instructions including at least one controller signal to set a state of one or more traffic signal lights at each approach of the roadway intersection in response to the state of the pedestrian'"'"'s presence in the field of view.

13. The method of claim 12, wherein the one or more instructions include an instruction to the traffic signal controller to change the state of at least one of the traffic signal lights to allow the pedestrian to complete passage through the safe passage zone, an instruction to the traffic signal controller to hold the state of at least of the traffic signal lights to allow the pedestrian to complete passage through the safe passage zone, and an instruction to the traffic signal controller to warn traffic approaching the roadway intersection that a pedestrian remains in the safe passage zone.

14. The method of claim 12, further comprising generating, as output data, one or more traffic indicators representing the state of the pedestrian'"'"'s presence in the field of view, wherein the one or more indicators include an indicator that a pedestrian will pass through the safe passage zone before the allotted time expires, an indicator that a pedestrian will not pass through the safe passage zone before the allotted time expires, an indicator that the pedestrian changed direction and exited the safe passage zone before reaching the end of the safe passage zone, an indicator that the pedestrian reversed direction and returned to the beginning of the safe passage zone, an indicator that the pedestrian stopped moving, and an indicator the pedestrian remains in the safe passage zone after the allotted time expires.

15. The method of claim 12, wherein the safe passage zone is identified within the field of view by ascertaining a region of interest for pedestrian tracks based on at least one of lane structures and intersection road markings, and on movement of pixels representing moving objects relative to the at least one of lane structures and intersection road markings, determining accumulated tracks of the moving objects in the field of view, by analyzing motion strength and frequency of activity of pixels representing the moving objects in field of view, and tracking pedestrian characteristics that include one or more of size, gestures, speed, entry points, and exit points in the region of interest to distinguish the accumulated tracks of the moving objects from the pedestrian tracks.

16. The method of claim 12, wherein the safe passage zone is identified within the field of view by identifying a position of at least one vehicle detection zone in nearest proximity to a stop bar, each at least one vehicle detection zone having a height that extends to or near to the stop bar, and calculating a height of a pedestrian area in the field of view from the height of the at least one vehicle detection zone.

17. The method of claim 12, wherein the analyzing the plurality of pixels in the field of view to detect the pedestrian in the field of view further comprises analyzing portions of the safe passage zone in the field of view, computing features of current pixel content identified in the analyzed portions by identifying part-based features defining an individual pedestrian that include one or more of body structure combinations, body shape, body width or walking gestures, developing a model of a single walking pedestrian to separate each individual pedestrian in a group of moving pedestrians in the field of view, computing pedestrian features using pixels defining a pedestrian contour, and determining a matching confidence between an individual pedestrian and a group of moving pedestrians by calculating a mathematical similarity between the computed features of current pixel content and the model of the single walking pedestrian.

18. The method of claim 12, further comprising detecting a non-pedestrian object within the safe passage zone during the allotted time for crossing the roadway intersection, comparing the non-pedestrian object with the state of the pedestrian'"'"'s presence in the field of view, and generating a warning where the non-pedestrian object is within the safe passage zone at a same time as the pedestrian.

19. The method of claim 12, further comprising comparing the current pedestrian speed with one or more of a normal safe passage zone crossing speed and an average safe passage zone crossing speed to identify a type of pedestrian in the field of view, and further identifying the type of pedestrian as an adult moving a regular speed, or as a member of a protected group, the protected group including an adult moving at a reduced speed, a child, a pedestrian moving with an external object, and a pedestrian using a mechanical movement aid.

20. The method of claim 19, wherein the one or more instructions include an instruction to the traffic signal controller to adjust the state of at least one of the traffic signal lights to allow the member of the protected group to complete passage through the safe passage zone, and an instruction to the traffic signal controller to warn traffic approaching the roadway intersection that a member of a protected group remains in the safe passage zone.

21. The method of claim 12, further comprising ingesting input data representing the field of view of the traffic detection zone at the roadway intersection and traffic signal timing information for the roadway intersection, the input data representing the field of view of the traffic detection zone at the roadway intersection captured by one or more sensors in or near a traffic intersection that include at least one of video cameras, a radar system, and a magnetometer.

22. A system, comprising:

a computing environment including at least one non-transitory computer-readable storage medium having program instructions stored therein and a computer processor operable to execute the program instructions within one or more hardware and software components configured to perform data processing functions to track pedestrian activity for traffic signal control at a roadway intersection, the one or more hardware and software components including;

a data ingest component configured to receive input data representing a field of view of a traffic detection zone at the roadway intersection, and traffic signal timing information for the roadway intersection, the input data representing the field of view of the traffic detection zone at the roadway intersection captured by one or more sensors in or near a traffic intersection;

a pedestrian zone identification component configured to analyze a plurality of pixels in the field of view to identify a safe passage zone for vulnerable roadway users at the roadway intersection, wherein a left edge and a right edge of the field of view define at least a portion of the safe passage zone;

a pedestrian detection and prediction component configured to a) detect a pedestrian in the field of view, b) measure pedestrian movement through the safe passage zone by continually tracking pedestrian position as the pedestrian moves through the field of view and continually estimating a current pedestrian speed as the pedestrian moves through the field of view, c) determine a distance to the end of the safe passage zone from the pedestrian position by dividing the distance to end of the safe passage zone by the current pedestrian speed to calculate an estimated pedestrian time of arrival at the end of the safe passage zone, and d) compare the estimated pedestrian time of arrival at the end of the safe passage zone with an estimated time remaining for a pedestrian signal determined from the traffic signal timing information, the traffic signal timing information including an allotted time for crossing the roadway intersection that is defined by one or both of a total pedestrian signal crossing time and a start time of a pedestrian signal indicating safe passage for a pedestrian at the roadway intersection, to predict a pedestrian'"'"'s ability to cross the safe passage zone within the allotted time; and

an output data component configured to determine a state of the pedestrian'"'"'s presence in the field of view from the predicted ability to cross the safe passage zone within the allotted time, and send one or more instructions to a traffic signal controller, the one or more instructions including at least one controller signal to set a state of one or more traffic signal lights at each approach of the roadway intersection in response to the state of the pedestrian'"'"'s presence in the field of view.

23. The system of claim 22, wherein the one or more instructions include an instruction to the traffic signal controller to change the state of at least one of the traffic signal lights to allow the pedestrian to complete passage through the safe passage zone, an instruction to the traffic signal controller to hold the state of at least one of the traffic signal lights to allow the pedestrian to complete passage through the safe passage zone, and an instruction to the traffic signal controller to warn traffic approaching the roadway intersection that a pedestrian remains in the safe passage zone.

24. The system of claim 22, wherein the output data component is further configured to generate one or more traffic indicators representing the state of the pedestrian'"'"'s presence in the field of view, wherein the one or more indicators include an indicator that a pedestrian will pass through the safe passage zone before the allotted time expires, an indicator that a pedestrian will not pass through the safe passage zone before the allotted time expires, an indicator that the pedestrian changed direction and exited the safe passage zone before reaching the end of the safe passage zone, an indicator that the pedestrian reversed direction and returned to the beginning of the safe passage zone, an indicator that the pedestrian stopped moving, and an indicator the pedestrian remains in the safe passage zone after the allotted time expires.

25. The system of claim 22, wherein the pedestrian zone identification component is further configured to ascertain a region of interest for pedestrian tracks based on at least one of lane structures and intersection road markings, and on movement of pixels representing moving objects relative to the at least one of lane structures and intersection road markings, determine accumulated tracks of the moving objects in the field of view by analyzing motion strength and frequency of activity of pixels representing the moving objects in field of view, and track pedestrian characteristics that include one or more of size, gestures, speed, entry points, and exit points in the region of interest to distinguish the accumulated tracks of the moving objects from the pedestrian tracks.

26. The system of claim 22, wherein the pedestrian zone identification component is further configured to identify a position of at least one vehicle detection zone in nearest proximity to a stop bar, each at least one vehicle detection zone having a height that extends to or near to the stop bar, and calculate a height of a pedestrian area in the field of view from the height of the at least one vehicle detection zone.

27. The system of claim 22, wherein the pedestrian detection and prediction component is further configured to analyze portions of the safe passage zone in the field of view to detect the pedestrian in the field of view, by computing features of current pixel content identified in the analyzed portions by identifying part-based features defining an individual pedestrian that include one or more of body structure combinations, body shape, body width or walking gestures, developing a model of a single walking pedestrian to separate each individual pedestrian in a group of moving pedestrians in the field of view, computing pedestrian features using pixels defining a pedestrian contour, and determining a matching confidence between an individual pedestrian and a group of moving pedestrians by calculating a mathematical similarity between the computed features of current pixel content and the model of the single walking pedestrian.

28. The system of claim 22, wherein the one or more data processing modules further include a component configured to detect a non-pedestrian object within the safe passage zone during the allotted time for crossing the roadway intersection, compare the non-pedestrian object with the state of the pedestrian'"'"'s presence in the field of view, and generate a warning where the non-pedestrian object is within the safe passage zone at a same time as the pedestrian.

29. The system of claim 22, wherein the pedestrian detection and prediction component is further configured to compare the current pedestrian speed with one or more of a normal safe passage zone crossing speed and an average safe passage zone crossing speed to identify a type of pedestrian in the field of view, and further identify the type of pedestrian as an adult moving a regular speed, or as a member of a protected group, the protected group including an adult moving at a reduced speed, a child, a pedestrian moving with an external object, and a pedestrian using a mechanical movement aid.

30. The method of claim 29, wherein the one or more instructions include an instruction to the traffic signal controller to adjust the state of at least one of the traffic signal lights to allow the member of the protected group to complete passage through the safe passage zone, and an instruction to the traffic signal controller to warn traffic approaching the roadway intersection that a member of a protected group remains in the safe passage zone.