Traffic congestion measuring method and apparatus and image processing method and apparatus

• US 5,999,635 A
• Filed: 07/10/1998
• Issued: 12/07/1999
• Est. Priority Date: 01/12/1996
• Status: Expired due to Term

• Current Assignee: Sumitomo Electric Industries Limited (Sumitomo Corporation)
• Original Assignee: Sumitomo Electric Industries Limited (Sumitomo Corporation)
• Inventors: Ito, Yoshiyuki, Higashikubo, Masakatsu
• Primary Examiner(s): Johns, Andrew W.
• Application Number:
  US09/112,989
• Time in Patent Office: 515 Days
• Field of Search: 382/104, 382/107, 382/130, 382/193, 382/270, 348/148, 348/149, 340/933, 340/934, 340/937, 701/117, 701/118, 701/119
• US Class Current: 382/104
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G08G 1/0104 Measuring and analyzing of parameters relative to traffic conditions
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors

Abstract

The present invention measures traffic congestion by utilizing a video camera to capture images of vehicles travelling on a road and analyzing sample points that are assigned to different aspects of the images. For example, presence and movement sample points correspond to the expected location and motion of the vehicles respectively. The sample points adjacent to one another form either movement or congestion blocks. The blocks that have a ratio of movement sample points to all other sample points within the block greater than a predetermined level are movement blocks. The remaining presence sample points in the images signify congestion sample points and form congestion blocks. Congestion blocks have a ratio of congestion sample points to all other sample points within the block greater than a predetermined level. Therefore, the state of traffic congestion is measured based upon the resultant movement and congestion blocks.

Claims

1. A traffic congestion measuring method of measuring a state of traffic congestion on the basis of a motion picture which represents a road and vehicles coming and going on said road and is formed by a series of images taken by a video camera, comprising the steps of:

(a) setting a plurality of sample lines in said motion picture, each of said sample lines being formed by a plurality of sample points aligned on a line perpendicular to a direction in which said road extends, and each of said sample points having a brightness level;

(b) forming a criterion brightness image, in which no vehicle is expected to be present, taken by said video camera;

(c) calculating a difference image indicative of a difference in brightness between one of said series of images taken at a predetermined time and said criterion brightness image by calculating brightness difference values each indicative of a difference in brightness between each of the sample points in the image taken at said predetermined time and each of the sample points in said criterion brightness image corresponding in position to each other;

(d) detecting, as difference sample lines, the sample lines which are situated in the image taken at said predetermined time and each of which has a brightness level having a difference larger than a predetermined level from each of the sample lines in said criterion brightness image, on the basis of the brightness difference values of the sample points in said difference image;

(e) spatially differentiating said difference image to calculate differentiation values respectively belonging to the sample points in said difference image;

(f) detecting, as edge sample lines, the sample lines which are situated in the image taken at said predetermined time and each of which has a brightness level having a difference larger than a predetermined level from one of two sample lines in the direct neighborhood of each of the sample lines, on the basis of the differentiation values of the sample points in said spatially differentiated difference image;

(g) detecting, as presence sample lines on each of which the vehicle is expected to be present, the difference sample lines detected in said step (d) and the edge sample lines detected in said step (f); and

(h) measuring the state of traffic congestion on the basis of said presence sample lines.

2. A traffic congestion measuring method as set forth in claim 1, in which said sample lines set in said step (a) are aligned at space intervals in the image, the space interval between two adjoining sample lines becoming smaller as two actual locations on said road respectively corresponding to said two adjoining sample lines are more remote from said video camera.

3. A traffic congestion measuring method as set forth in claim 1, in which said steps (b) through (h) are repeated at regular time intervals,

said step (b) comprising the step of forming the criterion brightness image which is utilized for detecting said presence sample lines in the sample point image taken at time t1 corresponding to said predetermined time, on the basis of the sample point image taken at said time t1 and said criterion brightness image utilized for detecting said presence sample lines in said sample point image taken at time tO prior to said time t1 by one regular time interval.

4. A traffic congestion measuring method as set forth in claim 1, in which said step (d) comprises the steps of:

(d1) detecting the sample points having the positive brightness difference values each of which is larger than a positive predetermined level;

(d2) detecting the sample points having the negative brightness difference values each of which is smaller than a negative predetermined level; and

(d3) detecting, as said difference sample lines, the sample lines each including part of the sample points detected in said steps (d1) and (d2), a ratio of the sample points detected in said steps (d1) and (d2) to all of the sample points contained in each of the difference sample lines being larger than a predetermined level.

5. A traffic congestion measuring method as set forth in claim 4, in which said step (d) comprises the step (d4) of varying said positive and negative predetermined levels, which are utilized for judging each of the sample points, in accordance with an actual route distance from said video camera to an actual location on said road corresponding to a position of each of the sample points in the image.

6. A traffic congestion measuring method as set forth in claim 1, in which said step (d) comprises the steps of:

(d1) calculating mean brightness difference values each indicative of a mean of the brightness difference values of the sample points contained in each of the sample lines;

(d2) detecting, as said difference sample lines, the sample lines having the positive mean brightness difference values each of which is larger than a positive predetermined level; and

(d3) detecting, as said difference sample lines, the sample lines having the negative mean brightness difference values each of which is smaller than a negative predetermined level.

7. A traffic congestion measuring method as set forth in claim 6, in which said step (d) comprises the step (d4) of varying said positive and negative predetermined levels, which are utilized for judging each of the sample lines, in accordance with an actual route distance from said video camera to an actual location on said road corresponding to a position of each of the sample lines in the image.

8. A traffic congestion measuring method as set forth in claim 1, comprising the step (i) of adding, when there are two presence sample lines sandwiching no presence sample line and sandwiching at least one sample line therebetween and when said two presence sample lines are spaced apart from each other at a distance in the image corresponding to an actual route distance along said road smaller than a predetermined distance level, the sample lines between said two presence sample lines as new presence sample lines to the previously detected presence sample lines in said step (g).

9. A traffic congestion measuring method as set forth in claim 1, comprising the step (i) of eliminating, when there is a line group containing a series of presence sample lines which have no sample line interposed therebetween and when both end presence sample lines of said line group are spaced apart from each other at a distance in the image corresponding to an actual route distance along said road smaller than a predetermined distance level, said series of presence sample lines of said line group from the previously detected presence sample lines in said step (g).

10. A traffic congestion measuring method as set forth in claim 1, comprising the step (i) of eliminating, when there is a line group containing only a series of presence sample lines which have no sample line interposed therebetween and consist of the difference sample lines, when both end presence sample lines of said line group are spaced apart from each other at a distance in the image corresponding to an actual route distance along said road larger than a predetermined distance level, and when a mean of brightness levels of said series of presence sample lines of said line group is smaller than a predetermined level, said series of presence sample lines of said line group from the previously detected presence sample lines in said step (g).

11. A traffic congestion measuring method as set forth in claim 1, comprising the steps of:

(i) detecting, as movement sample lines, the sample lines on each of which the moving vehicle is expected to be present in the image taken at said predetermined time; and

(j) adding, when there are two presence sample lines sandwiching no presence sample line and sandwiching at least one sample line therebetween, when said two presence sample lines are spaced apart from each other at a distance in the image corresponding to an actual route distance along said road smaller than a predetermined distance level, and when there is at least one movement sample line between said two presence sample lines, the sample lines including the movement sample line interposed between said two presence sample lines as new presence sample lines to the previously detected presence sample lines in said step (g).

12. A traffic congestion measuring method as set forth in claim 11, comprising the step (k) of varying said predetermined distance level, which is utilized for judging said two presence sample lines, in accordance with at least one of two actual route distance along said road, one of said two actual route distances being between said video camera to an actual location on said road corresponding to a position of one of said two presence sample lines in the image, and the other of said two actual route distances being between said video camera to an actual location on said road corresponding to position of the other of said two presence sample lines in the image.

13. A traffic congestion measuring method of measuring a state of traffic congestion on the basis of a motion picture which represents a road and vehicles coming and going on said road and is formed by a series of images taken by a video camera, comprising the steps of:

(a) setting a plurality of sample lines in said motion picture, each of said sample lines being formed by a plurality of sample points aligned on a line perpendicular to a direction in which said road extends, and each of said sample points having a brightness level;

(b) detecting, as presence sample lines, the sample lines, on each of which the vehicle is expected to be present, on the basis of the brightness levels of the sample points on one of said series of images taken at a predetermined time;

(c) estimating shadow areas including shadows of the vehicles represented in the image taken at said predetermined time;

(d) correcting said presence sample lines on the basis of said shadow areas; and

(e) measuring the state of traffic congestion on the basis of the presence sample lines corrected in said step (d).

14. A traffic congestion measuring method of measuring a state of traffic congestion on the basis of a motion picture which represents a road and vehicles coming and going on said road and is formed by a series of images taken by a video camera, comprising the steps of:

(a) setting a plurality of sample lines in said motion picture, each of said sample lines being formed by a plurality of sample points aligned on a line perpendicular to a direction in which said road extends, and each of said sample points having a brightness level;

(b) detecting, as movement sample lines, the sample lines on each of which the vehicle in motion is expected to be present; and

(c) measuring the state of traffic congestion on the basis of said movement sample lines.

15. A traffic congestion measuring method as set forth in claim 14, in which said step (b) comprises the steps of:

(b1) calculating a first difference image indicative of a difference in brightness between the image taken at time t corresponding to said predetermined time and the image taken at time t-α

, α

being a positive time interval;

(b2) binarizing said first difference image with a first predetermined threshold level to form a first feature abstractive image containing only picture elements each having a level of "0" or "1";

(b3) calculating a second difference image indicative of a difference in brightness between the image taken at the time t and the image taken at time t+β

, β

being a positive time interval;

(b4) binarizing said second difference image with a second predetermined threshold level to form a second feature abstractive image containing only picture elements each having a level of "0" or "1";

(b5) calculating a product of said first feature abstractive image and said second feature abstractive image to form a product image;

(b6) detecting, as movement sample points, the sample points corresponding to picture elements each having the level of "1" in said product image; and

(b7) detecting, as said movement sample lines, the sample lines each including part of the movement sample points, a ratio of the movement sample points to all of the sample points on each movement sample line being larger than a predetermined level.

16. A traffic congestion measuring method as set forth in claim 15, further comprising the steps of:

(b8) varying said first and second predetermine threshold levels, which are utilized for judging each of the picture elements in said first and second difference images in said steps (b2) and (b4), respectively, in accordance with an actual route distance from said video camera to an actual location on said road corresponding to a position of each of the picture elements in the image; and

(b9) varying said predetermined level, which is utilized for judging each of the sample lines in said step (b7), in accordance with an actual route distance from said video camera to an actual location on said road corresponding to a position of each of the sample lines in the image.

17. A traffic congestion measuring method of measuring a state of traffic congestion on the basis of a motion picture which represents a road and vehicles coming and going on said road and is formed by a series of images taken by a video camera, comprising the steps of:

(a) setting a plurality of sample lines in said motion picture, each of said sample lines being formed by a plurality of sample points aligned on a line perpendicular to a direction in which said road extends, and each of said sample points having a brightness level;

(b) detecting, as presence sample lines, the sample lines on each of which the vehicle is expected to be present on the basis of the brightness levels of the sample points in one of said series of images taken at a predetermined time;

(c) calculating a density of the vehicles lying on said road at said predetermined time on the basis of said presence sample lines;

(d) estimating a large-sized vehicle ratio indicative of a ratio of large-sized vehicles larger than a predetermined vehicle size to all of the vehicles represented in the image taken at said predetermined time;

(e) correcting the density of the vehicles on the basis of said large-sized vehicle ratio estimated in said step (d); and

(f) measuring the state of traffic congestion on the basis of the density of the vehicles corrected in said step (e).

18. A traffic congestion measuring method as set forth in claim 17, further comprising the step (g) of estimating inter vehicle gaps of vehicles lying on said road at said predetermined time,

said density of the vehicles calculated in said step (c) being corrected in said step (e) on the basis of not only said large-sized vehicle ratio estimated in said step (d) but also said inter vehicle gaps estimated in said step (g).

19. A traffic congestion measuring method of measuring a state of traffic congestion on the basis of a motion picture which represents a road and vehicles coming and going on said road and is formed by a series of images taken by a video camera, comprising the steps of:

(a) setting a plurality of sample lines in said motion picture, each of said sample lines being formed by a plurality of sample points aligned on a line perpendicular to a direction in which said road extends, and each of said sample points having a brightness level;

(b) calculating mean brightness levels each indicative of a mean of brightness levels of the sample points situated on each of the sample lines in the image taken at time t;

(c) calculating mean brightness levels each indicative of a mean of brightness levels of the sample points contained in each of the sample lines in the image taken at time t+γ

, γ

being a positive time interval;

(d) detecting, as movement sample lines, the sample lines which are included in the image taken at said time t and in each of which the vehicle in motion is expected to be present;

(e) detecting, as prospective post-movement sample lines, the sample lines which are included in the image taken at said time t+γ and

to each of which each of the movement sample lines in the image taken at said time t is expected to be transferred;

(f) calculating line velocities each indicative of a velocity of the movement sample line based on said positive time interval γ and

an actual route distance between two locations on said road respectively corresponding to each of the movement sample lines and each of the post-movement sample lines; and

(g) measuring the state of traffic congestion on said road on the basis of the line velocities of the movement sample lines calculated in said step (f).

20. A traffic congestion measuring method as set forth in claim 19, in which said step (e) comprises the steps of:

(e1) determining a prospective post-movement area in the image taken at said time t+γ

, into which the movement sample line is expected to be transferred from the image taken at said time t, on the basis of said positive time interval γ and

an actual route distance from said video camera to an actual location on said road corresponding to a position of the movement sample line;

(e2) calculating absolute values each indicative of a difference between the mean brightness level of the movement sample line in the image taken at said time t and the mean brightness level of each of the sample lines contained in said prospective post-movement area in the image taken at said time t+γ

;

(e3) judging whether each of said absolute values calculated in said step (e2) is smaller than a predetermined level; and

(e4) detecting, as said prospective post-movement sample line, the sample line which is included in said prospective post-movement area of the image at said time t+γ and

which takes part in production of the smallest one of said absolute values judged to be smaller than said predetermined level in said step (e3).

21. A traffic congestion measuring method as set forth in claim 19, in which said step (b) comprises the step of calculating two mean brightness levels, one of said two mean brightness levels being indicative of a mean of the brightness levels of the sample points situated on the sample line which lies directly on an upstream side of each of the movement sample lines in the image taken at said time t, and the other of said two mean brightness levels being indicative of a mean of the brightness levels of the sample points situated on the sample line which lies directly on a downstream side of each of the movement sample lines in the image taken at said time t, and

said step (e) comprising the steps of:

(e1) determining a first prospective post-movement area in the image taken at said time t+γ

, into which the movement sample line is expected to be transferred from the image taken at said time t, on the basis of said positive time interval γ and

an actual route distance from said video camera to an actual location on said road corresponding to a position of the movement sample line;

(e2) determining a second prospective post-movement area in the image taken at said time t+γ

, which is deviated upstream by one line pitch from said first prospective post-movement area;

(e3) determining a third prospective post-movement area in the image taken said time t+γ

, which is deviated downstream by one line pitch from said first prospective post-movement area;

(e4) calculating absolute difference values each indicative of an absolute value of difference between the mean brightness level of the movement sample line in the image taken at said time t and the mean brightness level of each of the sample lines contained in said first prospective post-movement area in the image taken at said time t+γ

;

(e5) calculating absolute difference values each indicative of an absolute value of difference between the mean brightness level of the sample line which lies directly on an upstream side of the movement sample line in the image taken at said time t and the mean brightness level of each of the sample lines contained in said second prospective post-movement areas in the image taken at said time t+γ

;

(e6) calculating absolute difference values each indicative of an absolute value of difference between the mean brightness level of the sample line which lies directly on a downstream side of the movement sample line in the image taken at said time t and the mean brightness level of each of the sample lines contained in said third prospective post-movement area in the image taken at said time t+γ

;

(e7) calculating sum values each indicative of a sum of each of the absolute values calculated in said step (e4), each of the absolute values calculated in said step (e5) and each of the absolute values calculated in said step (e6);

(e8) judging whether the smallest one of said sum values calculated in said step (e7) is smaller than a predetermined level; and

(e9) detecting, as the post-movement sample line, the sample line which is included in said first post-movement area of the image taken at said time t+γ and

which takes part in production of the smallest one of said sum values judged to be smaller than said predetermined level in said step (e8).

22. A traffic congestion measuring method as set forth in claim 19, further comprising the steps of:

(h) dividing said road represented in the image taken at said time t into a plurality of mesh sections; and

(i) calculating mean line velocities each indicative of a mean of line velocities of the movement sample lines contained in each of said mesh sections and detecting, as spatial velocities of the mesh sections, the calculated mean line velocities, respectively.

23. A traffic congestion measuring method as set forth in claim 22, further comprising the step (j) of setting, to zero, the spatial velocities of the mesh sections smaller than a predetermined level.

24. A traffic congestion measuring apparatus for measuring a state of traffic congestion on the basis of a motion picture which represents a road and vehicles coming and going on said road and is formed by a series of images taken by a video camera, comprising:

means for setting a plurality of sample lines in said motion picture, each of said sample lines being formed by a plurality of sample points aligned on a line perpendicular to a direction in which said road extends, and each of said sample points having a brightness level;

means for forming a criterion brightness image, in which no vehicle is expected to be present, taken by said video camera;

means for calculating a difference image indicative of a difference in brightness between one of said series of images taken at a predetermined time and said criterion brightness image by calculating brightness difference values each indicative of a difference in brightness between each of the sample points in the image taken at said predetermined time and each of the sample points in said criterion brightness image corresponding in position to each other;

means for detecting, as difference sample lines, the sample lines which are situated in the image taken at said predetermined time and each of which has a brightness level having a difference larger than a predetermined level from each of the sample lines in said criterion brightness image, on the basis of the brightness difference values of the sample points in said difference image;

means for spatially differentiating said difference image to calculate differentiation values respectively belonging to the sample points in said difference image;

means for detecting, as edge sample lines, the sample lines which are situated in the image taken at said predetermined time and each of which has a brightness level having a difference larger than a predetermined level from one of two sample lines in the direct neighborhood of each of the sample lines, on the basis of the differentiation values of the sample points in said spatially differentiated difference image;

means for detecting, as presence sample lines on each of which the vehicle is expected to be present, said difference sample lines and said edge sample lines; and

means for measuring the state of traffic congestion on the basis of said presence sample lines.

25. A traffic congestion measuring apparatus for measuring a state of traffic congestion on the basis of a motion picture which represents a road and vehicles coming and going on said road and is formed by a series of images taken by a video camera, comprising:

means for setting a plurality of sample lines in said motion picture, each of said sample lines being formed by a plurality of sample points aligned on a line perpendicular to a direction in which said road extends, and each of said sample points having a brightness level;

mean for detecting, as movement sample lines, the sample lines on each of which the vehicle in motion is expected to be present in the image taken at a predetermined time; and

means for measuring the state of traffic congestion on the basis of said movement sample lines.

26. A traffic congestion measuring apparatus for measuring a state of traffic congestion on the basis of a motion picture which represents a road and vehicles coming and going on said road and is formed by a series of images taken by a video camera, comprising:

means for setting a plurality of sample lines in said motion picture, each of said sample lines being formed by a plurality of sample points aligned on a line perpendicular to a direction in which said road extends, and each of said sample points having a brightness level;

means for detecting, as presence sample lines, the sample lines on each of which the vehicle is expected to be present on the basis of the brightness levels of the sample points in one of said series of images taken at a predetermined time;

means for calculating a density of the vehicles lying on said road at said predetermined time on the basis of said presence sample lines;

means for estimating a large-sized vehicle ratio indicative of a ratio of large-sized vehicles larger than a predetermined vehicle size to all of the vehicles represented in the image taken at said predetermined time; and

means for correcting the density of the vehicles on the basis of said large-sized vehicle ratio.

27. A traffic congestion measuring apparatus for measuring a state of traffic congestion on the basis of a motion picture which represents a road and vehicles coming and going on said road and is formed by a series of images taken by a video camera, comprising:

means for setting a plurality of sample lines in said motion picture, each of said sample lines being formed by a plurality of sample points aligned on a line perpendicular to a direction in which said road extends, and each of said sample points having a brightness level;

means for calculating mean brightness levels each indicative of a mean of brightness levels of the sample points situated on each of the sample lines in the image taken at time t;

means for calculating mean brightness levels each indicative of a mean of brightness levels of the sample points situated on each of the sample lines in the image taken at time t+γ

, γ

being a positive time interval;

means for detecting, as movement sample lines, the sample lines which are included in the image taken at said time t and in each of which the vehicle in motion is expected to be present;

means for detecting, as prospective post-movement sample lines, the sample lines which are included in the image taken at said time t+γ and

to each of which each of the movement sample lines in the image taken at said time t is expected to be transferred;

means for calculating line velocities each indicative of a velocity of the movement sample line based on said positive time interval γ and

an actual route distance between two locations on said road respectively corresponding to each of the movement sample lines and each of the post-movement sample lines; and

means for measuring the state of traffic congestion on said road on the basis of the line velocities of the movement sample lines.