Method and apparatus for the surveillance of objects in images

• US 20030235327A1
• Filed: 12/23/2002
• Published: 12/25/2003
• Est. Priority Date: 06/20/2002
• Status: Active Grant

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Srinivasa, Narayan
• Granted Patent: US 7,409,092 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

Object detection and tracking operations on images that may be performed independently are presented. A detection module receives images, extracts edges in horizontal and vertical directions, and generates an edge map where object-regions are ranked by their immediacy. Filters remove attached edges and ensure regions have proper proportions/size. The regions are tested using a geometric constraint to ensure proper shape, and are fit with best-fit rectangles, which are merged or deleted depending on their relationships. Remaining rectangles are objects. A tracking module receives images in which objects are detected and uses Euclidean distance/edge density criterion to match objects. If objects are matched, clustering determines whether the object is new; if not, a sum-of-squared-difference in intensity test locates matching objects. If this succeeds, clustering is performed and rectangles are applied to regions and merged or deleted depending on their relationships and are considered objects; if not the regions are rejected.

Claims

1. A method for ranking edge regions representing objects in an image based on their immediacy with respect to a reference, the method comprising steps of:

receiving an image including edge regions;

determining a ranking reference in the image with respect to which ranks the edge regions;

determining an edge region reference on each edge region;

determining an offset of the edge region reference with respect to the ranking reference;

normalizing the offset for each edge region to determine a set of normalized offsets;

ranking the set of normalized offsets, whereby the ranked normalized offsets provide a ranking of the immediacy of objects represented by the edge regions.

2. A method for ranking edge regions as set forth in claim 1, wherein the edge region reference is a centroid of the area of the edge region to which it corresponds.

3. A method for ranking edge regions as set forth in claim 2, wherein the image comprises a plurality of pixels, wherein the edge regions each comprise areas represented by a number of pixels, and where each offset is represented by a distance measured as a number of pixels, and where each normalized offset is determined by dividing the number of pixels representing the offset for an edge region by the number of pixels representing the area of the edge region.

4. A method for ranking edge regions as set forth in claim 3, wherein the objects represented by edge regions are automobiles and the ranking reference is the hood of an automobile from which the image was taken.

5. A method for ranking edge regions as set forth in claim 4, wherein the offset includes only a vertical component measured as a normal line from a point on the hood of the automobile to the centroid of the area of the edge region.

6. A method for ranking edge regions as set forth in claim 4, wherein the immediacy is indicative of a risk of collision between the automobile from which the image was taken and the object represented by the edge region.

7. A method for ranking edge regions as set forth in claim 5, further comprising a step of selecting a fixed-size set of edge regions based on their ranking and discarding all other edge regions, whereby only the edge regions representing objects having the greatest immediacy are kept.

8. A method for ranking edge regions as set forth in claim 7, further comprising a step of providing the rankings of the edge regions representing objects kept to an automotive safety system.

9. A method for detecting objects in an image comprising a step of applying a compactness filter to the image, where the compactness filter comprises steps of:

receiving an image including edge regions representing potential objects;

determining a geometric shape to apply as a compactness filter, with the geometric shape including a reference point;

determining an edge region reference on each edge region in the image, and aligning the reference point on each edge region with the reference point of the geometric shape;

extending search lines from the edge region reference in at least one direction; and

determining intersections of the search lines with the geometric shape, and if a predetermined number of intersections exists, classifying the edge region as a compact object.

10. A method for detecting objects in an image as set forth in claim 9, wherein the edge region reference is a centroid of an area of an edge region representing a potential object.

11. A method for detecting objects in an image as set forth in claim 10, wherein the objects are cars, and wherein the geometric shape is substantially “

U”

-shaped.

12. A method for detecting objects in an image as set forth in claim 11, wherein the search lines are extended in eight directions with one search line residing on a horizontal axis, with a 45 degree angle between each search line and each other search line, and wherein at least four intersections between search lines and the geometric shape, including an intersection in a bottom vertical direction, must occur for the edge region to be classified a compact object.

13. A method for detecting objects in an image as set forth in claim 12, wherein the method further comprises a step of eliminating all edge regions not classified as compact objects.

14. A method for detecting objects in an image as set forth in claim 13, wherein the objects are automobiles, and wherein the method further comprises a step of providing information regarding the compact objects to an automotive safety system.

15. A method for the surveillance of objects in an image, the method comprising a step of detecting objects comprising sub-steps of:

receiving an image including edge regions;

independently extracting horizontal edges and vertical edges from the image;

eliminating horizontal edges and vertical edges that exceed at least one length-based threshold;

combining the remaining horizontal edges and vertical edges into edge regions to generate an integrated edge map; and

applying a ranking filter to rank relative immediacies of objects represented by the edge regions in the integrated edge map.

16. A method for the surveillance of objects in an image as set forth in claim 15, wherein the sub-step of independently extracting horizontal and vertical edges is performed by use of a Sobel operator.

17. A method for the surveillance of objects in an image as set forth in claim 16, wherein after the sub-step of independently extracting horizontal and vertical edges, the horizontal and vertical edges are grouped into connected edge regions based on proximity.

18. A method for the surveillance of objects in an image as set forth in claim 15, wherein after the sub-step of independently extracting horizontal and vertical edges, the horizontal and vertical edges are grouped into connected edge regions based on proximity.

19. A method for the surveillance of objects in an image as set forth in claim 18, further comprising a step of selecting a fixed-size set of objects based on their ranking and discarding all other objects, whereby a modified image is generated.

20. A method for the surveillance of objects in an image as set forth in claim 19, further comprising a step of applying a compactness filter to the modified image to detect edge regions representing objects likely to be of interest.

21. A method for the surveillance of objects in an image as set forth in claim 20, wherein the step of applying the compactness filter comprises sub-steps of:

receiving an image including edge regions representing potential objects;

determining a geometric shape to apply as a compactness filter, with the geometric shape including a reference point;

determining an edge region reference on each edge region in the image, and aligning the reference point on each edge region with the reference point of the geometric shape;

extending search lines from the edge region reference in at least one direction; and

determining intersections of the search lines with the geometric shape, and if a predetermined number of intersections exists, classifying the edge region as a compact object.

22. A method for the surveillance of objects in an image as set forth in claim 21, wherein the edge region reference is a centroid of an area of an edge region representing a potential object.

23. A method for the surveillance of objects in an image as set forth in claim 22, wherein the objects are cars, and wherein the geometric shape is substantially “

U”

-shaped.

24. A method for the surveillance of objects in an image as set forth in claim 23, wherein the search lines are extended in eight directions with one search line residing on a horizontal axis, with a 45 degree angle between each search line and each other search line, and wherein at least four intersections between search lines and the geometric shape, including an intersection in a bottom vertical direction, must occur for the edge region to be classified a compact object.

25. A method for the surveillance of objects in an image as set forth in claim 24, wherein the method further comprises a step of eliminating all edge regions representing objects not classified as compact objects.

26. A method for the surveillance of objects in an image as set forth in claim 25, wherein the objects are automobiles, and wherein the method further comprises a step of providing information regarding the compact objects to an automotive safety system.

27. A method for the surveillance of objects in an image as set forth in claim 19, further comprising steps of:

applying an attached line edge filter to the modified image to remove remaining oriented lines from edge regions in the modified image;

applying an aspect ratio and size filter to remove edge regions from the image that do not fit a pre-determined aspect ratio or size criterion;

fitting each of the remaining edge regions within a minimum enclosing rectangle; and

removing rectangles that are fully enclosed in a larger rectangle;

removing rectangles that are vertically stacked above another rectangle;

merging rectangles that overlap in excess of a pre-determined threshold; and

labeling each remaining rectangle as an object of interest.

28. A method for surveillance of objects in an image as set forth in claim 27, wherein the step of applying an attached line edge filter uses a Hough Transform to eliminate oriented edge segments that match patterns for expected undesired edge segments.

29. A method for surveillance of objects in an image as set forth in claim 27, wherein the aspect ratio has a threshold of 3.5 and the size criterion has a threshold of about {fraction (1/30)} of the pixels in the image, and where edge regions that do not match both of these criterion are excluded from the image.

30. A method for the surveillance objects in an image as set forth in claim 27, further comprising a step, between the step of applying an aspect ratio and size filter and the step of fitting each of the remaining edge regions within a minimum enclosing rectangle, of applying a compactness filter to detect edge regions representing objects likely to be of interest based on a pre-determined geometrical constraint, and removing those edge regions from the image that do not match the pre-determined geometrical constraint.

31. A method for the surveillance of objects in an image as set forth in claim 30, wherein the step of applying the compactness filter comprises sub-steps of:

receiving an image including edge regions representing potential objects;

determining a geometric shape to apply as a compactness filter, with the geometric shape including a reference point;

determining an edge region reference on each edge region in the image, and aligning the reference point on each edge region with the reference point of the geometric shape;

extending search lines from the edge region reference in at least one direction; and

determining intersections of the search lines with the geometric shape, and if a predetermined number of intersections exists, classifying the edge region as a compact object.

32. A method for the surveillance of objects in an image as set forth in claim 31, wherein the edge region reference is a centroid of an area of an edge region representing a potential object.

33. A method for the surveillance of objects in an image as set forth in claim 32, wherein the objects are cars, and wherein the geometric shape is substantially “

U”

-shaped.

34. A method for the surveillance of objects in an image as set forth in claim 33, wherein the search lines are extended in eight directions with one search line residing on a horizontal axis, with a 45 degree angle between each search line and each other search line, and wherein at least four intersections between search lines and the geometric shape, including an intersection in a bottom vertical direction, must occur for the edge region to be classified a compact object.

35. A method for the surveillance of objects in an image as set forth in claim 34, wherein the method further comprises a step of eliminating all edge regions representing objects not classified as compact objects.

36. A method for the surveillance of objects in an image as set forth in claim 35, wherein the objects are automobiles, and wherein the method further comprises a step of providing information regarding the compact objects to an automotive safety system.

37. A method for the surveillance of objects in an image as set forth in claim 36, wherein the step of applying the ranking filter comprises sub-steps of:

receiving an image including edge regions;

determining a ranking reference in the image with respect to which to rank the edge regions;

determining an edge region reference on each edge region;

determining an offset of the edge region reference with respect to the ranking reference;

normalizing the offset for each edge region to determine a set of normalized offsets;

ranking the set of normalized offsets, whereby the ranked normalized offsets provide a ranking of the immediacy of objects represented by the edge regions.

38. A method for the surveillance of objects in an image as set forth in claim 37, wherein the edge region reference is a centroid of the area of the edge region to which it corresponds.

39. A method for the surveillance of objects in an image as set forth in claim 38, wherein the image comprises a plurality of pixels, wherein the edge regions each comprise areas represented by a number of pixels, and where each offset is represented by a distance measured as a number of pixels, and where each normalized offset is determined by dividing the number of pixels representing the offset for an edge region by the number of pixels representing the area of the edge region.

40. A method for the surveillance of objects in an image as set forth in claim 39, wherein the objects represented by edge regions are automobiles and the ranking reference is the hood of an automobile from which the image was taken.

41. A method for the surveillance of objects in an image as set forth in claim 40, wherein the offset includes only a vertical component measured as a normal line from a point on the hood of the automobile to the centroid of the area of the edge region.

42. A method for the surveillance of objects in an image as set forth in claim 40, wherein the immediacy is indicative of a risk of collision between the automobile from which the image was taken and the object represented by the edge region.

43. A method for the surveillance of objects in an image as set forth in claim 41, further comprising a step of selecting a fixed-size set of edge regions based on their ranking and discarding all other edge regions, whereby only the edge regions representing objects having the greatest immediacy are kept.

44. A method for the surveillance of objects in an image as set forth in claim 43, further comprising a step of providing the rankings of the edge regions representing objects kept to an automotive safety system.

45. A method for the surveillance of objects in an image as set forth in claim 15, wherein the step of applying the ranking filter comprises sub-steps of:

receiving an image including edge regions;

determining a ranking reference in the image with respect to which to rank the edge regions;

determining an edge region reference on each edge region;

determining an offset of the edge region reference with respect to the ranking reference;

normalizing the offset for each edge region to determine a set of normalized offsets;

ranking the set of normalized offsets, whereby the ranked normalized offsets provide a ranking of the immediacy of objects represented by the edge regions.

46. A method for the surveillance of objects in an image comprising a step of detecting objects in the image, with the step of detecting objects comprising substeps of:

receiving an image including edge regions;

independently extracting horizontal edges and vertical edges from the image;

removing horizontal edges and vertical edges that exceed a particular threshold;

combining the remaining horizontal edges and vertical edges into edge regions to generate an integrated edge map; and

applying a compactness filter to the edge regions detect objects likely to be of interest.

47. A method for the surveillance of objects in an image as set forth in claim 46, wherein in the step of independently extracting horizontal and vertical edges, the extracting is performed by use of a Sobel operator.

48. A method for the surveillance of objects in an image as set forth in claim 46, further comprising steps of:

after combining the remaining horizontal edges and vertical edges to generate an integrated edge map and prior to applying the compactness filter;

applying an attached line edge filter to the image to remove remaining oriented lines from edge regions in the image;

applying an aspect ratio and size filter to remove edge regions from the image that do not fit a pre-determined aspect ratio or size criterion;

fitting each of the remaining edge regions within a minimum enclosing rectangle; and

removing rectangles that are fully enclosed in a larger rectangle;

removing rectangles that are vertically stacked above another rectangle;

merging rectangles that overlap in excess of a pre-determined threshold; and

labeling each remaining rectangle as an object of interest.

49. A method for the surveillance of objects in an image as set forth in claim 48, wherein the step of applying the compactness filter comprises sub-steps of:

receiving an image including edge regions representing potential objects;

determining a geometric shape to apply as a compactness filter, with the geometric shape including a reference point;

determining an edge region reference on each edge region in the image, and aligning the reference point on each edge region with the reference point of the geometric shape;

extending search lines from the edge region reference in at least one direction; and

determining intersections of the search lines with the geometric shape, and if a predetermined number of intersections exists, classifying the edge region as a compact object.

50. A method for the surveillance of objects in an image as set forth in claim 49, wherein the edge region reference is a centroid of an area of an edge region representing a potential object.

51. A method for the surveillance of objects in an image as set forth in claim 50, wherein the objects are cars, and wherein the geometric shape is substantially “

U”

-shaped.

52. A method for the surveillance of objects in an image as set forth in claim 51, wherein the search lines are extended in eight directions with one search line residing on a horizontal axis, with a 45 degree angle between each search line and each other search line, and wherein at least four intersections between search lines and the geometric shape, including an intersection in a bottom vertical direction, must occur for the edge region to be classified a compact object.

53. A method for the surveillance of objects in an image as set forth in claim 52, wherein the method further comprises a step of eliminating all edge regions representing objects not classified as compact objects.

54. A method for the surveillance of objects in an image as set forth in claim 53, wherein the objects are automobiles, and wherein the method further comprises a step of providing information regarding the compact objects to an automotive safety system.

55. A method for the surveillance of objects in an image comprising a step of tracking objects from a first frame to a next frame in a series of images, where each image includes at least one region, with the step of tracking objects comprising sub-steps of:

detecting objects within the first frame;

detecting objects within the next frame; and

matching objects of the first frame and the next frame using at least one geometric criterion selected from a group consisting of a Euclidean distance criterion and an edge density criterion, in order to find clusters of objects.

56. A method for the surveillance of objects in an image as set forth in claim 55, wherein in the step of matching regions of the first frame and the next frame:

the Euclidean distance criterion is used to determine a centroid of the area of an object in the first frame and a centroid of the area of the same object in the next frame, and;

when the position of the centroid in the first frame is within a predetermined distance of the centroid in the next frame, the object in the first frame is considered matched with the object in the next frame; and

for objects with centroids that are matched, the edge density criterion is applied to determine a density of edges in regions of the objects and a size of the objects, and;

if a size difference between the matched objects is below a predetermined threshold, then the object with the greater density of edges is considered to be the object and a step of determining object clusters is performed to determine if the object is a new object or part of an existing cluster of objects;

when the position of the centroid in the first frame is not within a predetermined distance of the centroid in the next frame, or when the size difference between the matched objects is above a predetermined threshold, applying a sum-of-square-of-difference in intensity criterion to determine whether to consider the object in the first image or the object in the next image as the object;

if the sum-of-square-of-difference in intensity criterion fails, eliminating the object from the image; and

if the sum-of square-of-difference in intensity criterion passes, the step of determining object clusters is performed to determine if the object is a new object or part of an existing cluster of objects, whereby objects may be excluded, may be considered as new objects, or may be considered as an object in an existing cluster of objects.

57. A method for the surveillance of objects in an image as set forth in claim 56, wherein the step of determining object clusters comprises sub-steps of:

clustering objects in the first frame into at least one cluster;

determining a centroid of the area of the at least one cluster;

determining a standard deviation in distance of each object in the second image from a point in the next frame corresponding to the centroid of the area of the at least one cluster; and

when the distance between an individual object and the point corresponding to the centroid exceeds a threshold, considering the individual object as a new object;

when the distance between an individual object and the point corresponding to the centroid is less than the threshold, considering the individual object as part of the cluster.

58. A method for the surveillance of objects in an image as set forth in claim 57, wherein image includes a plurality of pixels, with each pixel including a Cartesian vertical “

Y”

component, and the predetermined geometric constraint is whether a pixel representing a centroid of the individual object has a greater “

Y”

component than the centroid of the cluster, and when the individual object has a greater “

Y”

component, excluding the individual object from further consideration.

59. A method for the surveillance of objects in an image as set forth in claim 58, further comprising steps of:

merging best-fit rectangles for regions that are fully enclosed, are vertically stacked, or are overlapping; and

labeling each remaining region as a region of interest.

60. A method for the surveillance of objects in an image as set forth in claim 59, further comprising steps of:

calculating a sum-of-square-of-difference in intensity of pixels over an area of the next frame associated with an area of an object in the first frame over all positions allowed by a search area constraint;

determining the position which yields the least sum-of-square-of-difference in intensity of pixels; and

if the least sum-of-square-of-difference in intensity is beneath a predetermined threshold, shifting the area of the next frame into alignment based on the determined position, whereby errors in positions of objects are minimized by aligning them properly in the next frame.

61. A method for the surveillance of objects in an image as set forth in claim 55, the method further comprising steps, when the geometric criterion are satisfied, of:

clustering objects in the first frame into at least one cluster;

determining a centroid of the area of the at least one cluster;

determining a standard deviation in distance of each object in the second image from a point in the next frame corresponding to the centroid of the area of the at least one cluster; and

when the distance between an individual object and the point corresponding to the centroid exceeds a threshold, considering the individual object as a new object;

when the distance between an individual object and the point corresponding to the centroid is less than the threshold, considering the individual object as part of the cluster.

62. A method for the surveillance of objects in an image as set forth in claim 61, further comprising a step of excluding the individual object from further consideration if it fails to meet a predetermined geometric constraint with respect to the point corresponding to the centroid.

63. A method for the surveillance of objects in an image as set forth in claim 62, wherein image includes a plurality of pixels, with each pixel including a Cartesian vertical “

Y”

component, and the predetermined geometric constraint is whether a pixel representing a centroid of the individual object has a greater “

Y”

component than the centroid of the cluster, and when the individual object has a greater “

Y”

component, excluding the individual object from further consideration.

64. A method for the surveillance of objects in an image as set forth in claim 61, further comprising steps of:

merging best fit rectangles for regions that are fully enclosed, are vertically stacked, or are overlapping; and

labeling each remaining region as a region of interest.

65. A method for the surveillance of objects in an image as set forth in claim 55, further comprising steps of:

calculating a sum-of-square-of-difference in intensity of pixels over an area of the next frame associated with an area of an object in the first frame over all positions allowed by a search area constraint;

determining the position which yields the least sum-of-square-of-difference in intensity of pixels; and

if the least sum-of-square-of-difference in intensity is beneath a predetermined threshold, shifting the area of the next frame into alignment based on the determined position, whereby errors in positions of objects are minimized by aligning them properly in the next frame.

66. An apparatus for ranking edge regions representing objects in an image based on their immediacy with respect to a reference, the apparatus comprising a computer system including a processor, a memory coupled with the processor, an input coupled with the processor for receiving images from an image source, and an output coupled with the processor for outputting information regarding objects in the image, wherein the computer system further comprises means, residing in its processor and memory, for:

receiving an image including edge regions;

determining a ranking reference in the image with respect to which to rank the edge regions;

determining an edge region reference on each edge region;

determining an offset of the edge region reference with respect to the ranking reference;

normalizing the offset for each edge region to determine a set of normalized offsets;

ranking the set of normalized offsets, whereby the ranked normalized offsets provide a ranking of the immediacy of objects represented by the edge regions.

67. An apparatus for ranking edge regions as set forth in claim 66, wherein the edge region reference is a centroid of the area of the edge region to which it corresponds.

68. An apparatus for ranking edge regions as set forth in claim 67, wherein the image comprises a plurality of pixels, wherein the edge regions each comprise areas represented by a number of pixels, and where each offset is represented by a distance measured as a number of pixels, and where each normalized offset is determined by dividing the number of pixels representing the offset for an edge region by the number of pixels representing the area of the edge region.

69. An apparatus for ranking edge regions as set forth in claim 68, wherein the objects represented by edge regions are automobiles and the ranking reference is the hood of an automobile from which the image was taken.

70. An apparatus for ranking edge regions as set forth in claim 69, wherein the offset includes only a vertical component measured as a normal line from a point on the hood of the automobile to the centroid of the area of the edge region.

71. An apparatus for ranking edge regions as set forth in claim 69, wherein the immediacy is indicative of a risk of collision between the automobile from which the image was taken and the object represented by the edge region.

72. An apparatus for ranking edge regions as set forth in claim 70, further comprising means for selecting a fixed-size set of edge regions based on their ranking and discarding all other edge regions, whereby only the edge regions representing objects having the greatest immediacy are kept.

73. An apparatus for ranking edge regions as set forth in claim 72, further comprising means for providing the rankings of the edge regions representing objects kept to an automotive safety system.

74. An apparatus for detecting objects in an image comprising means for applying a compactness filter to the image, the apparatus comprising a computer system including a processor, a memory coupled with the processor, an input coupled with the processor for receiving images from an image source, and an output coupled with the processor for outputting information regarding objects in the image, wherein the computer system further comprises means, residing in its processor and memory, for:

receiving an image including edge regions representing potential objects;

determining a geometric shape to apply as a compactness filter, with the geometric shape including a reference point;

determining an edge region reference on each edge region in the image, and aligning the reference point on each edge region with the reference point of the geometric shape;

extending search lines from the edge region reference in at least one direction; and

determining intersections of the search lines with the geometric shape, and if a predetermined number of intersections exists, classifying the edge region as a compact object.

75. An apparatus for detecting objects in an image as set forth in claim 74, wherein the edge region reference is a centroid of an area of an edge region representing a potential object.

76. An apparatus for detecting objects in an image as set forth in claim 75, wherein the objects are cars, and wherein the geometric shape is substantially “

U”

-shaped.

77. An apparatus for detecting objects in an image as set forth in claim 76, wherein the search lines are extended in eight directions with one search line residing on a horizontal axis, with a 45 degree angle between each search line and each other search line, and wherein at least four intersections between search lines and the geometric shape, including an intersection in a bottom vertical direction, must occur for the edge region to be classified a compact object.

78. An apparatus for detecting objects in an image as set forth in claim 77, wherein the apparatus further comprises means for eliminating all edge regions representing objects not classified as compact objects.

79. An apparatus for detecting objects in an image as set forth in claim 78, wherein the objects are automobiles, and wherein the apparatus further comprises means for providing information regarding the compact objects to an automotive safety system.

80. An apparatus for the surveillance of objects in an image, the apparatus comprising a computer system including a processor, a memory coupled with the processor, an input coupled with the processor for receiving images from an image source, and an output coupled with the processor for outputting information regarding objects in the image, wherein the computer system further comprises means, residing in its processor and memory, for detecting objects including means for:

receiving an image including edge regions;

independently extracting horizontal edges and vertical edges from the image;

eliminating horizontal edges and vertical edges that exceed at least one length-based threshold;

combining the remaining horizontal edges and vertical edges into edge regions to generate an integrated edge map; and

applying a ranking filter to rank relative immediacies of objects represented by the edge regions in the integrated edge map.

81. An apparatus for the surveillance of objects in an image as set forth in claim 80, wherein the means for independently extracting horizontal and vertical edges is a Sobel operator.

82. An apparatus for the surveillance of objects in an image as set forth in claim 81, wherein after the means for independently extracting horizontal and vertical edges is operated, the horizontal and vertical edges are grouped into connected edge regions based on proximity.

83. An apparatus for the surveillance of objects in an image as set forth in claim 80, wherein after the means for independently extracting horizontal and vertical edges is operated, the horizontal and vertical edges are grouped into connected edge regions based on proximity.

84. An apparatus for the surveillance of objects in an image as set forth in claim 83, further comprising means for selecting a fixed-size set of objects based on their ranking and discarding all other objects, whereby a modified image is generated.

85. An apparatus for the surveillance of objects in an image as set forth in claim 84, further comprising means for applying a compactness filter to the modified image to detect edge regions representing objects likely to be of interest.

86. An apparatus for the surveillance of objects in an image as set forth in claim 85, wherein the means for applying the compactness filter comprises means for:

receiving an image including edge regions representing potential objects;

determining a geometric shape to apply as a compactness filter, with the geometric shape including a reference point;

determining an edge region reference on each edge region in the image, and aligning the reference point on each edge region with the reference point of the geometric shape;

extending search lines from the edge region reference in at least one direction; and

determining intersections of the search lines with the geometric shape, and if a predetermined number of intersections exists, classifying the edge region as a compact object.

87. An apparatus for the surveillance of objects in an image as set forth in claim 86, wherein the edge region reference is a centroid of an area of an edge region representing a potential object.

88. An apparatus for the surveillance of objects in an image as set forth in claim 87, wherein the objects are cars, and wherein the geometric shape is substantially “

U”

-shaped.

89. An apparatus for the surveillance of objects in an image as set forth in claim 88, wherein the search lines are extended in eight directions with one search line residing on a horizontal axis, with a 45 degree angle between each search line and each other search line, and wherein at least four intersections between search lines and the geometric shape, including an intersection in a bottom vertical direction, must occur for the edge region to be classified a compact object.

90. An apparatus for the surveillance of objects in an image as set forth in claim 89, wherein the apparatus further comprises means for eliminating all edge regions representing objects not classified as compact objects.

91. An apparatus for the surveillance of objects in an image as set forth in claim 90, wherein the objects are automobiles, and wherein the apparatus further comprises means for providing information regarding the compact objects to an automotive safety system.

92. An apparatus for the surveillance of objects in an image as set forth in claim 84, further comprising means for:

applying an attached line edge filter to the modified image to remove remaining oriented lines from edge regions in the modified image;

applying an aspect ratio and size filter to remove edge regions from the image that do not fit a pre-determined aspect ratio or size criterion;

fitting each of the remaining edge regions within a minimum enclosing rectangle; and

removing rectangles that are fully enclosed in a larger rectangle;

removing rectangles that are vertically stacked above another rectangle;

merging rectangles that overlap in excess of a pre-determined threshold; and

labeling each remaining rectangle as an object of interest.

93. An apparatus for surveillance of objects in an image as set forth in claim 92, wherein the means for applying an attached line edge filter uses a Hough Transform to eliminate oriented edge segments that match patterns for expected undesired edge segments.

94. An apparatus for surveillance of objects in an image as set forth in claim 92, wherein the aspect ratio has a threshold of 3.5 and the size criterion has a threshold of about {fraction (1/30)} of the pixels in the image, and where edge regions that do not match both of these criterion are excluded from the image.

95. An apparatus for the surveillance objects in an image as set forth in claim 92, further comprising means, operating temporally between the means for applying an aspect ratio and size filter and the means for fitting each of the remaining edge regions within a minimum enclosing rectangle, of applying a compactness filter to detect edge regions representing objects likely to be of interest based on a predetermined geometrical constraint, and removing those edge regions from the image that do not match the pre-determined geometrical constraint.

96. An apparatus for the surveillance of objects in an image as set forth in claim 95, wherein the means for applying the compactness filter comprises means for:

receiving an image including edge regions representing potential objects;

determining a geometric shape to apply as a compactness filter, with the geometric shape including a reference point;

determining an edge region reference on each edge region in the image, and aligning the reference point on each edge region with the reference point of the geometric shape;

extending search lines from the edge region reference in at least one direction; and

determining intersections of the search lines with the geometric shape, and if a predetermined number of intersections exists, classifying the edge region as a compact object.

97. An apparatus for the surveillance of objects in an image as set forth in claim 96, wherein the edge region reference is a centroid of an area of an edge region representing a potential object.

98. An apparatus for the surveillance of objects in an image as set forth in claim 97, wherein the objects are cars, and wherein the geometric shape is substantially “

U”

-shaped.

99. An apparatus for the surveillance of objects in an image as set forth in claim 98, wherein the search lines are extended in eight directions with one search line residing on a horizontal axis, with a 45 degree angle between each search line and each other search line, and wherein at least four intersections between search lines and the geometric shape, including an intersection in a bottom vertical direction, must occur for the edge region to be classified a compact object.

100. An apparatus for the surveillance of objects in an image as set forth in claim 99, wherein the apparatus further comprises means for eliminating all edge regions representing objects not classified as compact objects.

101. An apparatus for the surveillance of objects in an image as set forth in claim 100, wherein the objects are automobiles, and wherein the apparatus further comprises means for providing information regarding the compact objects to an automotive safety system.

102. An apparatus for the surveillance of objects in an image as set forth in claim 101, wherein the means for applying the ranking filter comprises means for:

receiving an image including edge regions;

determining a ranking reference in the image with respect to which to rank the edge regions;

determining an edge region reference on each edge region;

determining an offset of the edge region reference with respect to the ranking reference;

normalizing the offset for each edge region to determine a set of normalized offsets;

ranking the set of normalized offsets, whereby the ranked normalized offsets provide a ranking of the immediacy of objects represented by the edge regions.

103. An apparatus for the surveillance of objects in an image as set forth in claim 102, wherein the edge region reference is a centroid of the area of the edge region to which it corresponds.

104. An apparatus for the surveillance of objects in an image as set forth in claim 103, wherein the image comprises a plurality of pixels, wherein the edge regions each comprise areas represented by a number of pixels, and where each offset is represented by a distance measured as a number of pixels, and where each normalized offset is determined by dividing the number of pixels representing the offset for an edge region by the number of pixels representing the area of the edge region.

105. An apparatus for the surveillance of objects in an image as set forth in claim 104, wherein the objects represented by edge regions are automobiles and the ranking reference is the hood of an automobile from which the image was taken.

106. An apparatus for the surveillance of objects in an image as set forth in claim 105, wherein the offset includes only a vertical component measured as a normal line from a point on the hood of the automobile to the centroid of the area of the edge region.

107. An apparatus for the surveillance of objects in an image as set forth in claim 105, wherein the immediacy is indicative of a risk of collision between the automobile from which the image was taken and the object represented by the edge region.

108. An apparatus for the surveillance of objects in an image as set forth in claim 106, further comprising means for selecting a fixed-size set of edge regions based on their ranking and discarding all other edge regions, whereby only the edge regions representing objects having the greatest immediacy are kept.

109. An apparatus for the surveillance of objects in an image as set forth in claim 108, further comprising means for providing the rankings of the edge regions representing objects kept to an automotive safety system.

110. An apparatus for the surveillance of objects in an image as set forth in claim 80, wherein the means for applying the ranking filter comprises means for:

receiving an image including edge regions;

determining a ranking reference in the image with respect to which to rank the edge regions;

determining an edge region reference on each edge region;

determining an offset of the edge region reference with respect to the ranking reference;

normalizing the offset for each edge region to determine a set of normalized offsets;

ranking the set of normalized offsets, whereby the ranked normalized offsets provide a ranking of the immediacy of objects represented by the edge regions.

111. An apparatus for the surveillance of objects in an image, the apparatus comprising a computer system including a processor, a memory coupled with the processor, an input coupled with the processor for receiving images from an image source, and an output coupled with the processor for outputting information regarding objects in the image, wherein the computer system further comprises means, residing in its processor and memory, for detecting objects in the image, including means for:

receiving an image including edge regions;

independently extracting horizontal edges and vertical edges from the image;

removing horizontal edges and vertical edges that exceed a particular threshold;

combining the remaining horizontal edges and vertical edges into edge regions to generate an integrated edge map; and

applying a compactness filter to the edge regions detect objects likely to be of interest.

112. An apparatus for the surveillance of objects in an image as set forth in claim 111, wherein in the means for independently extracting horizontal and vertical edges is a Sobel operator.

113. An apparatus for the surveillance of objects in an image as set forth in claim 111, further comprising means for:

generating an integrated edge map after combining the remaining and prior to applying the compactness filter;

applying an attached line edge filter to the image to remove remaining oriented lines from edge regions in the image;

applying an aspect ratio and size filter to remove edge regions from the image that do not fit a pre-determined aspect ratio or size criterion;

fitting each of the remaining edge regions within a minimum enclosing rectangle; and

removing rectangles that are fully enclosed in a larger rectangle;

removing rectangles that are vertically stacked above another rectangle;

merging rectangles that overlap in excess of a pre-determined threshold; and

labeling each remaining rectangle as an object of interest.

114. An apparatus for the surveillance of objects in an image as set forth in claim 113, wherein the means for applying the compactness filter includes means for:

receiving an image including edge regions representing potential objects;

determining a geometric shape to apply as a compactness filter, with the geometric shape including a reference point;

determining an edge region reference on each edge region in the image, and aligning the reference point on each edge region with the reference point of the geometric shape;

extending search lines from the edge region reference in at least one direction; and

determining intersections of the search lines with the geometric shape, and if a predetermined number of intersections exists, classifying the edge region as a compact object.

115. An apparatus for the surveillance of objects in an image as set forth in claim 114, wherein the edge region reference is a centroid of an area of an edge region representing a potential object.

116. An apparatus for the surveillance of objects in an image as set forth in claim 115, wherein the objects are cars, and wherein the geometric shape is substantially “

U”

-shaped.

117. An apparatus for the surveillance of objects in an image as set forth in claim 116, wherein the search lines are extended in eight directions with one search line residing on a horizontal axis, with a 45 degree angle between each search line and each other search line, and wherein at least four intersections between search lines and the geometric shape, including an intersection in a bottom vertical direction, must occur for the edge region to be classified a compact object.

118. An apparatus for the surveillance of objects in an image as set forth in claim 117, wherein the apparatus further comprises means for eliminating all edge regions representing objects not classified as compact objects.

119. An apparatus for the surveillance of objects in an image as set forth in claim 118, wherein the objects are automobiles, and wherein the apparatus further comprises means for providing information regarding the compact objects to an automotive safety system.

120. An apparatus for the surveillance of objects in an image, the apparatus comprising a computer system including a processor, a memory coupled with the processor, an input coupled with the processor for receiving images from an image source, and an output coupled with the processor for outputting information regarding objects in the image, wherein the computer system further comprises means, residing in its processor and memory, for tracking objects from a first frame to a next frame in a series of images, where each image includes at least one region, with the means for tracking objects comprising means for:

detecting objects within the first frame;

detecting objects within the next frame; and

matching objects of the first frame and the next frame using at least one geometric criterion selected from a group consisting of a Euclidean distance criterion and an edge density criterion, in order to find clusters of objects.

121. An apparatus for the surveillance of objects in an image as set forth in claim 120, wherein in the means for matching regions of the first frame and the next frame:

the Euclidean distance criterion is used to determine a centroid of the area of an object in the first frame and a centroid of the area of the same object in the next frame, and;

when the position of the centroid in the first frame is within a predetermined distance of the centroid in the next frame, the object in the first frame is considered matched with the object in the next frame; and

for objects with centroids that are matched, the edge density criterion is applied to determine a density of edges in regions of the objects and a size of the objects, and;

if a size difference between the matched objects is below a predetermined threshold, then the object with the greater density of edges is considered to be the object and means for determining object clusters is performed to determine if the object is a new object or part of an existing cluster of objects;

when the position of the centroid in the first frame is not within a predetermined distance of the centroid in the next frame, or when the size difference between the matched objects is above a predetermined threshold, applying a sum-of-square-of-difference in intensity criterion to determine whether to consider the object in the first image or the object in the next image as the object;

if the sum-of-square-of-difference in intensity criterion fails, eliminating the object from the image; and

if the sum-of square-of-difference in intensity criterion passes, the means for determining object clusters is performed to determine if the object is a new object or part of an existing cluster of objects, whereby objects may be excluded, may be considered as new objects, or may be considered as an object in an existing cluster of objects.

122. An apparatus for the surveillance of objects in an image as set forth in claim 121, wherein the means for determining object clusters comprises means for:

clustering objects in the first frame into at least one cluster;

determining a centroid of the area of the at least one cluster;

determining a standard deviation in distance of each object in the second image from a point in the next frame corresponding to the centroid of the area of the at least one cluster; and

when the distance between an individual object and the point corresponding to the centroid exceeds a threshold, considering the individual object as a new object;

when the distance between an individual object and the point corresponding to the centroid is less than the threshold, considering the individual object as part of the cluster.

123. An apparatus for the surveillance of objects in an image as set forth in claim 122, wherein image includes a plurality of pixels, with each pixel including a Cartesian vertical “

Y”

component, and the predetermined geometric constraint is whether a pixel representing a centroid of the individual object has a greater “

Y”

component than the centroid of the cluster, and when the individual object has a greater “

Y”

component, excluding the individual object from further consideration.

124. An apparatus for the surveillance of objects in an image as set forth in claim 123, further comprising means for:

merging best-fit rectangles for regions that are fully enclosed, are vertically stacked, or are overlapping; and

labeling each remaining region as a region of interest.

125. An apparatus for the surveillance of objects in an image as set forth in claim 124, further comprising means for:

calculating a sum-of-square-of-difference in intensity of pixels over an area of the next frame associated with an area of an object in the first frame over all positions allowed by a search area constraint;

determining the position which yields the least sum-of-square-of-difference in intensity of pixels; and

if the least sum-of-square-of-difference in intensity is beneath a predetermined threshold, shifting the area of the next frame into alignment based on the determined position, whereby errors in positions of objects are minimized by aligning them properly in the next frame.

126. An apparatus for the surveillance of objects in an image as set forth in claim 120, the apparatus further comprising means for, when the geometric criterion are satisfied:

clustering objects in the first frame into at least one cluster;

determining a centroid of the area of the at least one cluster;

determining a standard deviation in distance of each object in the second image from a point in the next frame corresponding to the centroid of the area of the at least one cluster; and

when the distance between an individual object and the point corresponding to the centroid exceeds a threshold, considering the individual object as a new object;

when the distance between an individual object and the point corresponding to the centroid is less than the threshold, considering the individual object as part of the cluster.

127. An apparatus for the surveillance of objects in an image as set forth in claim 126, further comprising means for excluding the individual object from further consideration if it fails to meet a predetermined geometric constraint with respect to the point corresponding to the centroid.

128. An apparatus for the surveillance of objects in an image as set forth in claim 127, wherein image includes a plurality of pixels, with each pixel including a Cartesian vertical “

Y”

component, and the predetermined geometric constraint is whether a pixel representing a centroid of the individual object has a greater “

Y”

component than the centroid of the cluster, and when the individual object has a greater “

Y”

component, excluding the individual object from further consideration.

129. An apparatus for the surveillance of objects in an image as set forth in claim 126, further comprising means for:

merging best fit rectangles for regions that are fully enclosed, are vertically stacked, or are overlapping; and

labeling each remaining region as a region of interest.

130. An apparatus for the surveillance of objects in an image as set forth in claim 120, further comprising means for:

calculating a sum-of-square-of-difference in intensity of pixels over an area of the next frame associated with an area of an object in the first frame over all positions allowed by a search area constraint;

determining the position which yields the least sum-of-square-of-difference in intensity of pixels; and

if the least sum-of-square-of-difference in intensity is beneath a predetermined threshold, shifting the area of the next frame into alignment based on the determined position, whereby errors in positions of objects are minimized by aligning them properly in the next frame.

131. A computer program product for ranking edge regions representing objects in an image based on their immediacy with respect to a reference, the computer program product comprising a computer-readable medium further comprising computer-readable means for:

receiving an image including edge regions;

determining a ranking reference in the image with respect to which to rank the edge regions;

determining an edge region reference on each edge region;

determining an offset of the edge region reference with respect to the ranking reference;

normalizing the offset for each edge region to determine a set of normalized offsets;

ranking the set of normalized offsets, whereby the ranked normalized offsets provide a ranking of the immediacy of objects represented by the edge regions.

132. A computer program product for ranking edge regions as set forth in claim 131, wherein the edge region reference is a centroid of the area of the edge region to which it corresponds.

133. A computer program product for ranking edge regions as set forth in claim 132, wherein the image comprises a plurality of pixels, wherein the edge regions each comprise areas represented by a number of pixels, and where each offset is represented by a distance measured as a number of pixels, and where each normalized offset is determined by dividing the number of pixels representing the offset for an edge region by the number of pixels representing the area of the edge region.

134. A computer program product for ranking edge regions as set forth in claim 133, wherein the objects represented by edge regions are automobiles and the ranking reference is the hood of an automobile from which the image was taken.

135. A computer program product for ranking edge regions as set forth in claim 134, wherein the offset includes only a vertical component measured as a normal line from a point on the hood of the automobile to the centroid of the area of the edge region.

136. A computer program product for ranking edge regions as set forth in claim 134, wherein the immediacy is indicative of a risk of collision between the automobile from which the image was taken and the object represented by the edge region.

137. A computer program product for ranking edge regions as set forth in claim 135, further comprising means for selecting a fixed-size set of edge regions based on their ranking and discarding all other edge regions, whereby only the edge regions representing objects having the greatest immediacy are kept.

138. A computer program product for ranking edge regions as set forth in claim 137, further comprising means for providing the rankings of the edge regions representing objects kept to an automotive safety system.

139. A computer program product for detecting objects in an image comprising means for applying a compactness filter to the image, the computer program product comprising a computer-readable medium further comprising computer-readable means for:

receiving an image including edge regions representing potential objects;

determining a geometric shape to apply as a compactness filter, with the geometric shape including a reference point;

determining an edge region reference on each edge region in the image, and aligning the reference point on each edge region with the reference point of the geometric shape;

extending search lines from the edge region reference in at least one direction; and

determining intersections of the search lines with the geometric shape, and if a predetermined number of intersections exists, classifying the edge region as a compact object.

140. A computer program product for detecting objects in an image as set forth in claim 139, wherein the edge region reference is a centroid of an area of an edge region representing a potential object.

141. A computer program product for detecting objects in an image as set forth in claim 140, wherein the objects are cars, and wherein the geometric shape is substantially “

U”

-shaped.

142. A computer program product for detecting objects in an image as set forth in claim 141, wherein the search lines are extended in eight directions with one search line residing on a horizontal axis, with a 45 degree angle between each search line and each other search line, and wherein at least four intersections between search lines and the geometric shape, including an intersection in a bottom vertical direction, must occur for the edge region to be classified a compact object.

143. A computer program product for detecting objects in an image as set forth in claim 142, wherein the computer program product further comprises means for eliminating all edge regions representing objects not classified as compact objects.

144. A computer program product for detecting objects in an image as set forth in claim 143, wherein the objects are automobiles, and wherein the computer program product further comprises means for providing information regarding the compact objects to an automotive safety system.

145. A computer program product for the surveillance of objects in an image, the computer program product comprising a computer-readable medium further comprising computer-readable means for detecting objects including means for:

receiving an image including edge regions;

independently extracting horizontal edges and vertical edges from the image;

eliminating horizontal edges and vertical edges that exceed at least one length-based threshold;

combining the remaining horizontal edges and vertical edges into edge regions to generate an integrated edge map; and

applying a ranking filter to rank relative immediacies of objects represented by the edge regions in the integrated edge map.

146. A computer program product for the surveillance of objects in an image as set forth in claim 145, wherein the means for independently extracting horizontal and vertical edges is a Sobel operator.

147. A computer program product for the surveillance of objects in an image as set forth in claim 146, wherein the means for independently extracting horizontal and vertical edges is configured such that after it is operated, the horizontal and vertical edges are grouped into connected edge regions based on proximity.

148. A computer program product for the surveillance of objects in an image as set forth in claim 145, wherein the means for independently extracting horizontal and vertical edges is configured such that after it is operated, the horizontal and vertical edges are grouped into connected edge regions based on proximity.

149. A computer program product for the surveillance of objects in an image as set forth in claim 148, further comprising means for selecting a fixed-size set of objects based on their ranking and discarding all other objects, whereby a modified image is generated.

150. A computer program product for the surveillance of objects in an image as set forth in claim 149, further comprising means for applying a compactness filter to the modified image to detect edge regions representing objects likely to be of interest.

151. A computer program product for the surveillance of objects in an image as set forth in claim 150, wherein the means for applying the compactness filter comprises means for:

receiving an image including edge regions representing potential objects;

determining a geometric shape to apply as a compactness filter, with the geometric shape including a reference point;

determining an edge region reference on each edge region in the image, and aligning the reference point on each edge region with the reference point of the geometric shape;

extending search lines from the edge region reference in at least one direction; and

determining intersections of the search lines with the geometric shape, and if a predetermined number of intersections exists, classifying the edge region as a compact object.

152. A computer program product for the surveillance of objects in an image as set forth in claim 151, wherein the edge region reference is a centroid of an area of an edge region representing a potential object.

153. A computer program product for the surveillance of objects in an image as set forth in claim 152, wherein the objects are cars, and wherein the geometric shape is substantially “

U”

-shaped.

154. A computer program product for the surveillance of objects in an image as set forth in claim 153, wherein the search lines are extended in eight directions with one search line residing on a horizontal axis, with a 45 degree angle between each search line and each other search line, and wherein at least four intersections between search lines and the geometric shape, including an intersection in a bottom vertical direction, must occur for the edge region to be classified a compact object.

155. A computer program product for the surveillance of objects in an image as set forth in claim 154, wherein the computer program product further comprises means for eliminating all edge regions representing objects not classified as compact objects.

156. A computer program product for the surveillance of objects in an image as set forth in claim 155, wherein the objects are automobiles, and wherein the computer program product further comprises means for providing information regarding the compact objects to an automotive safety system.

157. A computer program product for the surveillance of objects in an image as set forth in claim 149, further comprising means for:

applying an attached line edge filter to the modified image to remove remaining oriented lines from edge regions in the modified image;

applying an aspect ratio and size filter to remove edge regions from the image that do not fit a pre-determined aspect ratio or size criterion;

fitting each of the remaining edge regions within a minimum enclosing rectangle; and

removing rectangles that are fully enclosed in a larger rectangle;

removing rectangles that are vertically stacked above another rectangle;

merging rectangles that overlap in excess of a pre-determined threshold; and

labeling each remaining rectangle as an object of interest.

158. A computer program product for surveillance of objects in an image as set forth in claim 157, wherein the means for applying an attached line edge filter uses a Hough Transform to eliminate oriented edge segments that match patterns for expected undesired edge segments.

159. A computer program product for surveillance of objects in an image as set forth in claim 157, wherein the aspect ratio has a threshold of 3.5 and the size criterion has a threshold of about {fraction (1/30)} of the pixels in the image, and where edge regions that do not match both of these criterion are excluded from the image.

160. A computer program product for the surveillance objects in an image as set forth in claim 157, further comprising means, to operate temporally between the means for applying an aspect ratio and size filter and the means for fitting each of the remaining edge regions within a minimum enclosing rectangle, of applying a compactness filter to detect edge regions representing objects likely to be of interest based on a pre-determined geometrical constraint, and removing those edge regions from the image that do not match the pre-determined geometrical constraint.

161. A computer program product for the surveillance of objects in an image as set forth in claim 160, wherein the means for applying the compactness filter comprises means for:

receiving an image including edge regions representing potential objects;

determining a geometric shape to apply as a compactness filter, with the geometric shape including a reference point;

determining an edge region reference on each edge region in the image, and aligning the reference point on each edge region with the reference point of the geometric shape;

extending search lines from the edge region reference in at least one direction; and

determining intersections of the search lines with the geometric shape, and if a predetermined number of intersections exists, classifying the edge region as a compact object.

162. A computer program product for the surveillance of objects in an image as set forth in claim 161, wherein the edge region reference is a centroid of an area of an edge region representing a potential object.

163. A computer program product for the surveillance of objects in an image as set forth in claim 162, wherein the objects are cars, and wherein the geometric shape is substantially “

U”

-shaped.

164. A computer program product for the surveillance of objects in an image as set forth in claim 163, wherein the search lines are extended in eight directions with one search line residing on a horizontal axis, with a 45 degree angle between each search line and each other search line, and wherein at least four intersections between search lines and the geometric shape, including an intersection in a bottom vertical direction, must occur for the edge region to be classified a compact object.

165. A computer program product for the surveillance of objects in an image as set forth in claim 164, wherein the computer program product further comprises means for eliminating all edge regions representing objects not classified as compact objects.

166. A computer program product for the surveillance of objects in an image as set forth in claim 165, wherein the objects are automobiles, and wherein the computer program product further comprises means for providing information regarding the compact objects to an automotive safety system.

167. A computer program product for the surveillance of objects in an image as set forth in claim 166, wherein the means for applying the ranking filter comprises means for:

receiving an image including edge regions;

determining a ranking reference in the image with respect to which to rank the edge regions;

determining an edge region reference on each edge region;

determining an offset of the edge region reference with respect to the ranking reference;

normalizing the offset for each edge region to determine a set of normalized offsets;

ranking the set of normalized offsets, whereby the ranked normalized offsets provide a ranking of the immediacy of objects represented by the edge regions.

168. A computer program product for the surveillance of objects in an image as set forth in claim 167, wherein the object reference is a centroid of the area of the object.

169. A computer program product for the surveillance of objects in an image as set forth in claim 168, wherein the image comprises a plurality of pixels, wherein the edge regions each comprise areas represented by a number of pixels, and where each offset is represented by a distance measured as a number of pixels, and where each normalized offset is determined by dividing the number of pixels representing the offset for an edge region by the number of pixels representing the area of the edge region.

170. A computer program product for the surveillance of objects in an image as set forth in claim 169, wherein the objects represented by edge regions are automobiles and the ranking reference is the hood of an automobile from which the image was taken.

171. A computer program product for the surveillance of objects in an image as set forth in claim 170, wherein the offset includes only a vertical component measured as a normal line from a point on the hood of the automobile to the centroid of the area of the edge region.

172. A computer program product for the surveillance of objects in an image as set forth in claim 170, wherein the immediacy is indicative of a risk of collision between the automobile from which the image was taken and the object represented by the edge region.

173. A computer program product for the surveillance of objects in an image as set forth in claim 171, further comprising means for selecting a fixed-size set of edge regions based on their ranking and discarding all other edge regions, whereby only the edge regions representing objects having the greatest immediacy are kept.

174. A computer program product for the surveillance of objects in an image as set forth in claim 173, further comprising means for providing the rankings of the edge regions representing objects kept to an automotive safety system.

175. A computer program product for the surveillance of objects in an image as set forth in claim 145, wherein the means for applying the ranking filter comprises means for:

receiving an image including edge regions;

determining a ranking reference in the image with respect to which to rank the edge regions;

determining an edge region reference on each edge region;

determining an offset of the edge region reference with respect to the ranking reference;

normalizing the offset for each edge region to determine a set of normalized offsets;

ranking the set of normalized offsets, whereby the ranked normalized offsets provide a ranking of the immediacy of objects represented by the edge regions.

176. A computer program product for the surveillance of objects in an image, the computer program product comprising a computer-readable medium further comprising computer-readable means for detecting objects including means for:

receiving an image including edge regions;

independently extracting horizontal edges and vertical edges from the image;

removing horizontal edges and vertical edges that exceed a particular threshold;

combining the remaining horizontal edges and vertical edges into edge regions to generate an integrated edge map; and

applying a compactness filter to the edge regions detect objects likely to be of interest.

177. A computer program product for the surveillance of objects in an image as set forth in claim 176, wherein in the means for independently extracting horizontal and vertical edges is a Sobel operator.

178. A computer program product for the surveillance of objects in an image as set forth in claim 176, further comprising means for:

generating an integrated edge map after combining the remaining and

prior to applying the compactness filter;

applying an attached line edge filter to the image to remove remaining oriented lines from edge regions in the image;

applying an aspect ratio and size filter to remove edge regions from the image that do not fit a pre-determined aspect ratio or size criterion;

fitting each of the remaining edge regions within a minimum enclosing rectangle; and

removing rectangles that are fully enclosed in a larger rectangle;

removing rectangles that are vertically stacked above another rectangle;

merging rectangles that overlap in excess of a pre-determined threshold; and

labeling each remaining rectangle as an object of interest.

179. A computer program product for the surveillance of objects in an image as set forth in claim 178, wherein the means for applying the compactness filter includes means for:

receiving an image including edge regions representing potential objects;

determining a geometric shape to apply as a compactness filter, with the geometric shape including a reference point;

determining an edge region reference on each edge region in the image, and aligning the reference point on each edge region with the reference point of the geometric shape;

extending search lines from the edge region reference in at least one direction; and

determining intersections of the search lines with the geometric shape, and if a predetermined number of intersections exists, classifying the edge region as a compact object.

180. A computer program product for the surveillance of objects in an image as set forth in claim 179, wherein the edge region reference is a centroid of an area of an edge region representing a potential object.

181. A computer program product for the surveillance of objects in an image as set forth in claim 180, wherein the objects are cars, and wherein the geometric shape is substantially “

U”

-shaped.

182. A computer program product for the surveillance of objects in an image as set forth in claim 181, wherein the search lines are extended in eight directions with one search line residing on a horizontal axis, with a 45 degree angle between each search line and each other search line, and wherein at least four intersections between search lines and the geometric shape, including an intersection in a bottom vertical direction, must occur for the edge region to be classified a compact object.

183. A computer program product for the surveillance of objects in an image as set forth in claim 182, wherein the computer program product further comprises means for eliminating all edge regions representing objects not classified as compact objects.

184. A computer program product for the surveillance of objects in an image as set forth in claim 183, wherein the objects are automobiles, and wherein the computer program product further comprises means for providing information regarding the compact objects to an automotive safety system.

185. A computer program product for the surveillance of objects in an image, the computer program product comprising a computer-readable medium further comprising computer-readable means for detecting objects including means for tracking objects from a first frame to a next frame in a series of images, where each image includes at least one region, with the means for tracking objects comprising means for:

detecting objects within the first frame;

detecting objects within the next frame; and

matching objects of the first frame and the next frame using at least one geometric criterion selected from a group consisting of a Euclidean distance criterion and an edge density criterion, in order to find clusters of objects.

186. A computer program product for the surveillance of objects in an image as set forth in claim 185, wherein in the means for matching regions of the first frame and the next frame:

the Euclidean distance criterion is used to determine a centroid of the area of an object in the first frame and a centroid of the area of the same object in the next frame, and;

when the position of the centroid in the first frame is within a predetermined distance of the centroid in the next frame, the object in the first frame is considered matched with the object in the next frame; and

for objects with centroids that are matched, the edge density criterion is applied to determine a density of edges in regions of the objects and a size of the objects, and;

if a size difference between the matched objects is below a predetermined threshold, then the object with the greater density of edges is considered to be the object and means for determining object clusters is performed to determine if the object is a new object or part of an existing cluster of objects;

when the position of the centroid in the first frame is not within a predetermined distance of the centroid in the next frame, or when the size difference between the matched objects is above a predetermined threshold, applying a sum-of-square-of-difference in intensity criterion to determine whether to consider the object in the first image or the object in the next image as the object;

if the sum-of-square-of-difference in intensity criterion fails, eliminating the object from the image; and

if the sum-of square-of-difference in intensity criterion passes, the means for determining object clusters is performed to determine if the object is a new object or part of an existing cluster of objects, whereby objects may be excluded, may be considered as new objects, or may be considered as an object in an existing cluster of objects.

187. A computer program product for the surveillance of objects in an image as set forth in claim 186, wherein the means for determining object clusters comprises means for:

clustering objects in the first frame into at least one cluster;

determining a centroid of the area of the at least one cluster;

determining a standard deviation in distance of each object in the second image from a point in the next frame corresponding to the centroid of the area of the at least one cluster; and

when the distance between an individual object and the point corresponding to the centroid exceeds a threshold, considering the individual object as a new object;

when the distance between an individual object and the point corresponding to the centroid is less than the threshold, considering the individual object as part of the cluster.

188. A computer program product for the surveillance of objects in an image as set forth in claim 187, wherein image includes a plurality of pixels, with each pixel including a Cartesian vertical “

Y”

component, and the predetermined geometric constraint is whether a pixel representing a centroid of the individual object has a greater “

Y”

component than the centroid of the cluster, and when the individual object has a greater “

Y”

component, excluding the individual object from further consideration.

189. A computer program product for the surveillance of objects in an image as set forth in claim 188, further comprising means for:

merging best-fit rectangles for regions that are fully enclosed, are vertically stacked, or are overlapping; and

labeling each remaining region as a region of interest.

190. A computer program product for the surveillance of objects in an image as set forth in claim 189, further comprising means for:

calculating a sum-of-square-of-difference in intensity of pixels over an area of the next frame associated with an area of an object in the first frame over all positions allowed by a search area constraint;

determining the position which yields the least sum-of-square-of-difference in intensity of pixels; and

if the least sum-of-square-of-difference in intensity is beneath a predetermined threshold, shifting the area of the next frame into alignment based on the determined position, whereby errors in positions of objects are minimized by aligning them properly in the next frame.

191. A computer program product for the surveillance of objects in an image as set forth in claim 185, the computer program product further comprising means for, when the geometric criterion are satisfied:

clustering objects in the first frame into at least one cluster;

determining a centroid of the area of the at least one cluster;

determining a standard deviation in distance of each object in the second image from a point in the next frame corresponding to the centroid of the area of the at least one cluster; and

when the distance between an individual object and the point corresponding to the centroid exceeds a threshold, considering the individual object as a new object;

when the distance between an individual object and the point corresponding to the centroid is less than the threshold, considering the individual object as part of the cluster.

192. A computer program product for the surveillance of objects in an image as set forth in claim 191, further comprising means for excluding the individual object from further consideration if it fails to meet a predetermined geometric constraint with respect to the point corresponding to the centroid.

193. A computer program product for the surveillance of objects in an image as set forth in claim 192, wherein image includes a plurality of pixels, with each pixel including a Cartesian vertical “

Y”

component, and the predetermined geometric constraint is whether a pixel representing a centroid of the individual object has a greater “

Y”

component than the centroid of the cluster, and when the individual object has a greater “

Y”

component, excluding the individual object from further consideration.

194. A computer program product for the surveillance of objects in an image as set forth in claim 191, further comprising means for:

merging best fit rectangles for regions that are fully enclosed, are vertically stacked, or are overlapping; and

labeling each remaining region as a region of interest.

195. A computer program product for the surveillance of objects in an image as set forth in claim 185, further comprising means for:

calculating a sum-of-square-of-difference in intensity of pixels over an area of the next frame associated with an area of an object in the first frame over all positions allowed by a search area constraint;

determining the position which yields the least sum-of-square-of-difference in intensity of pixels; and

if the least sum-of-square-of-difference in intensity is beneath a predetermined threshold, shifting the area of the next frame into alignment based on the determined position, whereby errors in positions of objects are minimized by aligning them properly in the next frame.