Pedestrian counting and detection at a traffic intersection based on object movement within a field of view

• US 9,460,613 B1
• Filed: 05/09/2016
• Issued: 10/04/2016
• Est. Priority Date: 05/09/2016
• Status: Active Grant

• Current Assignee: Iteris Incorporated
• Original Assignee: Iteris Incorporated
• Inventors: Whiting, Michael T., Gao, Yan, Swaminathan, Dilip, Shivakumar, Shashank Jayaram, Hwang, Robert J., Kreter, Todd W.
• Primary Examiner(s): Previl, Daniel
• Application Number:
  US15/150,267
• Time in Patent Office: 148 Days
• Field of Search: 340/944, 340/545.3, 340/907, 340/928, 340/989, 340/554, 340/552, 340/545.1, 340/573.1, 340/573.3, 340/573.7
• US Class Current: 1/1
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/91 for traffic control G01S13/93 takes precedence
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G08G 1/005 including pedestrian guidance indicator
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/052 with provision for determining speed or overspeed speed measuring in general G01P
• CPC Class Codes: G08G 1/07 Controlling traffic signals
• CPC Class Codes: G08G 1/087 Override of traffic control, e.g. by signal transmitted by an emergency vehicle
• CPC Class Codes: G08G 1/205 Indicating the location of the monitored vehicles as destination, e.g. accidents, stolen, rental

Abstract

Pedestrian detection and counting for traffic intersection control analyzes characteristics of a field of view of a traffic detection zone to determine a location and size of a pedestrian area, and applies protocols for evaluating pixel content in the field of view to identify individual pedestrians. The location and size of a pedestrian area is determined based either on locations of vehicle and bicycle detection areas or on movement of various objects within the field of view. Automatic pedestrian speed calibration with a region of interest for pedestrian detection is accomplished using lane and other intersection markings in the field of view. Detection and counting further includes identifying a presence, volume, velocity and trajectory of pedestrians in the pedestrian area of the traffic detection zone.

Claims

1. A method, comprising:

receiving input data representing a field of view of a traffic detection zone;

analyzing the input data within a computing environment in one or more data processing modules executed in conjunction with at least one specifically-configured processor, the one or more data processing modules configured to a) identify a region in the field of view of the traffic detection zone used by one or more pedestrians, and b) accurately count the one or more pedestrians in the traffic detection zone, by

1) defining a pedestrian zone in a field of view of a traffic detection zone, by a) ascertaining a region of interest in the field of view for pedestrian tracks based on at least one of lane structures and intersection road markings, and on movement of pixels representing moving objects relative to the at least one of lane structures and intersection road markings, b) determining accumulated tracks of the moving objects in the field of view, by analyzing motion strength and frequency of activity of each pixel representing the moving objects in field of view, and c) tracking pedestrian characteristics that include one or more of size, gestures, speed, entry points, and exit points in the region of interest to distinguish the accumulated tracks of the moving objects from the pedestrian tracks;

2) counting the one or more pedestrians in the pedestrian zone by, a) analyzing portions of the pedestrian zone in the field of view, b) computing features of current pixel content identified in the analyzed portions by identifying part-based features defining an individual pedestrian that include one or more of body structure combinations, body shape, body width or walking gestures, c) developing a model of a single walking pedestrian to separate each individual pedestrian in a group of moving pedestrians in the field of view, by computing pedestrian features using pixels defining a pedestrian contour, d) determining a matching confidence between an individual pedestrian and a group of moving pedestrians by calculating a mathematical similarity between the computed features of current pixel content and the model of the single walking pedestrian, and e) incrementing a pedestrian count where a high matching confidence indicates that an individual pedestrian has been identified; and

generating, as output data, the pedestrian count.

2. The method of claim 1, wherein the determining a matching confidence between an individual pedestrian and a group of moving pedestrians further comprises identifying an individual pedestrian where a matching confidence is high, and analyzing a next portion of the field of view where a matching confidence is low.

3. The method of claim 1, wherein the identifying part-based features defining an individual pedestrian further comprises one or more of determining a width and a height of one or more object parts, comparing body structure combinations with one or more predetermined templates, and applying one or more geometric constraints to separate the part-based features.

4. The method of claim 1, further comprising creating and continually refining a boundary of the pedestrian zone based on the accumulated tracks of the moving objects in the region of interest.

5. The method of claim 1, further comprising adjusting sensitivity for detecting the one or more pedestrians in the pedestrian zone by applying a vehicle detection status, the vehicle detection status including one or more of vehicular speed and traffic detection zone saturation, and dynamically changing the sensitivity for detecting the one or more pedestrians in the pedestrian zone by one or both of increasing a likelihood of a pedestrian crossing when a stopped vehicle is detected or no vehicle in present, and decreasing a likelihood of a pedestrian crossing while the vehicle traffic is free flowing.

6. The method of claim 1, wherein the input data is captured by one or more sensors in or near a traffic intersection, the one or more sensors including at least one of video cameras, a radar system, and a magnetometer.

7. The method of claim 1, further comprising providing the output data to a traffic management tool configured to enable a user to select a size and location of one or more of the field of view and the at least one of lane structures and intersection road markings, the lane structures and intersection road markings defining one or more vehicle detection zones and a stop bar.

8. The method of claim 1, wherein the defining a pedestrian zone in a field of view of a traffic detection zone further comprises detecting pixel motion by identifying an initial motion threshold for pixel activity in the field of view, and updating the initial motion threshold for pixel activity in the field of view as pixels move through the field of view.

9. The method of claim 1, wherein the defining a pedestrian zone in a field of view of a traffic detection zone further comprises computing a binary thresholded image defining a histogram of oriented gradient features from the pixel activity, and updating the histogram of orient gradient features of the computed image as activity progresses in the field of view.

10. The method of claim 1, wherein the defining a pedestrian zone in a field of view of a traffic detection zone further comprises computing a frequency of the pixel activity and determining tracks of pixel motion based on the frequency of pixel activity.

11. The method of claim 1, wherein the determining accumulated tracks of the moving objects in the field of view further comprises determining vehicle and bicycle tracks in the traffic detection zone from the at least one of lane structures and intersection road markings.

12. A method of pedestrian detection and counting for traffic intersection control, comprising:

distinguishing one or more pedestrians from moving pixels representing other objects in a field of view of a traffic detection zone to define a pedestrian zone, by a) ascertaining a region of interest for pedestrian tracks within the field of view based on at least one of lane structures and intersection road markings, and on movement of pixels representing moving objects relative to the at least one of lane structures and intersection road markings, b) detecting movement of the moving objects within the traffic detection zone from pixel activity by computing a binary thresholded image defining a histogram of oriented gradient features to analyze motion strength, and computing a frequency of activity of each pixel representing the moving objects in field of view to identify accumulated tracks of the moving objects, and c) differentiating the accumulated tracks of the moving objects from the pedestrian tracks by tracking pedestrian characteristics that include one or more of size, gestures, speed, entry points, and exit points in the region of interest; and

detecting the one or more pedestrians in the pedestrian area from similarities of a model of a single walking pedestrian with part-based object recognition of individual pedestrians, by a) computing features of current pixel content identified by analyzing portions of the pedestrian zone in the field of view to define individual pedestrians from the part-based object recognition of one or more of body structure combinations, body shape, body width or walking gestures, c) computing pedestrian features using pixels defining a pedestrian contour to develop the model of the single walking pedestrian to separate each individual pedestrian from a group of moving pedestrians in the field of view, d) determining a matching confidence between an individual pedestrian and a group of moving pedestrians by calculating a mathematical similarity between the computed features of current pixel content and the model of the single walking pedestrian, and identifying an individual pedestrian where a matching confidence is high, and analyzing a next portion of the pedestrian zone in the field of view where a matching confidence is low.

13. The method of claim 12, further comprising incrementing a pedestrian count where a high matching confidence indicates that an individual pedestrian has been identified.

14. The method of claim 13, further comprising generating an output representing the pedestrian count to an external device or location for storage or use.

15. The method of claim 12, further comprising providing the pedestrian count to a traffic management tool, wherein the traffic management tool is configured to enable a user to select a size and location of one or more of the field of view and the at least one of lane structures and intersection road markings, the lane structures and intersection road markings defining one or more vehicle detection zones and a stop bar.

16. The method of claim 12, wherein the identifying part-based features defining an individual pedestrian further comprises one or more of determining a width and a height of one or more object parts, comparing body structure combinations with one or more predetermined templates, and applying one or more geometric constraints to separate the part-based features.

17. The method of claim 12, further comprising creating and continually refining a boundary of the pedestrian zone based on the accumulated tracks of the moving objects in the region of interest.

18. The method of claim 12, further comprising adjusting sensitivity for detecting the one or more pedestrians in the pedestrian zone by applying a vehicle detection status, the vehicle detection status including one or more of vehicular speed and traffic detection zone saturation, and dynamically changing the sensitivity for detecting the one or more pedestrians in the pedestrian zone by one or both of increasing a likelihood of a pedestrian crossing when a stopped vehicle is detected or no vehicle in present, and decreasing a likelihood of a pedestrian crossing while the vehicle traffic is free flowing.

19. The method of claim 12, wherein the field of view is captured by one or more sensors in or near a traffic intersection, the one or more sensors including at least one of video cameras, a radar system, and a magnetometer.

20. The method of claim 12, wherein the differentiating the accumulated tracks of the moving objects from the pedestrian tracks further comprises determining vehicle and bicycle tracks in the traffic detection zone from the at least one of lane structures and intersection road markings.

21. A system, comprising:

a computing environment including at least one non-transitory computer-readable storage medium having program instructions stored therein and a computer processor operable to execute the program instructions within one or more data processing modules configured to a) identify a region in a field of view of a traffic detection zone used by one or more pedestrians, and b) accurately count the one or more pedestrians in the traffic detection zone, the one or more data processing modules including;

a pedestrian count zone identification module configured to define a pedestrian zone in the field of view, by a) ascertaining a region of interest in the field of view for pedestrian tracks based on at least one of lane structures and intersection road markings, and on movement of pixels representing moving objects relative to the at least one of lane structures and intersection road markings, b) determining accumulated tracks of the moving objects in the field of view, by analyzing motion strength and frequency of activity of each pixel representing the moving objects in field of view, and c) tracking pedestrian characteristics that include one or more of size, gestures, speed, entry points, and exit points in the region of interest to distinguish the accumulated tracks of the moving objects from the pedestrian tracks;

a pedestrian detector learning module configured to count the one or more pedestrians in the pedestrian zone by, a) analyzing portions of the pedestrian zone in the field of view, b) computing features of current pixel content identified in the analyzed portions by identifying part-based features defining an individual pedestrian that include one or more of body structure combinations, body shape, body width or walking gestures, c) developing a model of a single walking pedestrian to separate each individual pedestrian in a group of moving pedestrians in the field of view, by computing pedestrian features using pixels defining a pedestrian contour, d) determining a matching confidence between an individual pedestrian and a group of moving pedestrians by calculating a mathematical similarity between the computed features of current pixel content and the model of the single walking pedestrian, and e) incrementing a pedestrian count where a high matching confidence indicates that an individual pedestrian has been identified; and

an output module configured to communicate the pedestrian count to an external device or location.

22. The system of claim 21, wherein the pedestrian detector learning module is further configured to identify an individual pedestrian where a matching confidence is high, and analyzing a next portion of the pedestrian zone in the field of view where a matching confidence is low.

23. The system of claim 21, wherein the pedestrian detector learning module is further configured to count the one or more pedestrians in the pedestrian zone by one or more of determining a width and a height of one or more object parts, comparing body structure combinations with one or more predetermined templates, and applying one or more geometric constraints to separate the part-based features.

24. The system of claim 21, wherein the pedestrian detector learning module is further configured to adjust sensitivity for detecting the one or more pedestrians in the pedestrian zone by applying a vehicle detection status that includes one or more of vehicular speed and traffic detection zone saturation, and dynamically change the sensitivity for detecting the one or more pedestrians in the pedestrian zone by one or both of increasing a likelihood of a pedestrian crossing when a stopped vehicle is detected or no vehicle in present, and decreasing a likelihood of a pedestrian crossing while the vehicle traffic is free flowing.

25. The system of claim 21, wherein the field of view is captured by one or more sensors in or near a traffic intersection, the one or more sensors including at least one of video cameras, a radar system, and a magnetometer.

26. The system of claim 21, wherein the pedestrian count is provided to a traffic management tool configured to enable a user to select a size and location of one or more of the field of view and the at least one of lane structures and intersection road markings, the lane structures and intersection road markings defining one or more vehicle detection zones and a stop bar.

27. The system of claim 21, wherein the pedestrian count zone identification module is further configured to detect pixel motion by identifying an initial motion threshold for pixel activity in the field of view, update the initial motion threshold for pixel activity in the field of view as pixels move through the field of view, compute a binary thresholded image defining a histogram of oriented gradient features from the pixel activity, and update the histogram of orient gradient features of the computed image as activity progresses in the field of view.

28. The system of claim 21, wherein the pedestrian count zone identification module is further configured to compute a frequency of the pixel activity and determine tracks of pixel motion based on the frequency of pixel activity.

29. The system of claim 21, wherein the pedestrian count zone identification module is further configured to determine vehicle and bicycle tracks in the traffic detection zone from the at least one of lane structures and intersection road markings.

30. The system of claim 21, wherein the pedestrian count zone identification module is further configured to create and continually refine a boundary of the pedestrian zone based on the accumulated tracks of the moving objects in the region of interest.