Object recognition system and abnormality detection system using image processing

• US 5,554,983 A
• Filed: 04/23/1993
• Issued: 09/10/1996
• Est. Priority Date: 04/24/1992
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Yamamoto, Katsuyuki, Nakanishi, Kunio, Inoue, Haruki, Horie, Takeshi, Kobayashi, Yoshiki, Satoh, Yoshiyuki, Shibata, Toshiro, Asada, Kazuyoshi, Kitamura, Tadaaki, Yahiro, Masakazu, Takatoo, Masao
• Primary Examiner(s): Peng, John K.
• Assistant Examiner(s): Pope, Daryl C.
• Application Number:
  US08/051,692
• Time in Patent Office: 1,236 Days
• Field of Search: 340/905, 340/937, 382/30, 382/31, 348/135, 348/149
• US Class Current: 340/937
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors

Abstract

An object recognition system using the image processing in which an area having a unique feature is extracted from an input image of an object, the unique image is registered in a shade template memory circuit as a shade template, the input image is searched for an image similar to the shade template registered by a shade pattern matching circuit, the position of an object is determined for each template, the speed and direction of movement of the object is determined from the positional information, and the results thereof are integrated by a separation/integration circuit, thereby recognizing the whole of the moving object.

Claims

1. An object recognition system for photographing an object by a TV camera and recognizing movement of the object by processing images from the TV camera, comprising:

a template extracting circuit extracting partial templates from an image of an object;

a following means for following the object by correlation calculation between the partial templates and an input image; and

means for separating/integrating the partial templates on a basis of a selected one of a history of coordinates followed and a knowledge relating to the object.

2. An object recognition system according to claim 1, wherein said template extracting circuit extracts images of a plurality of partial areas formed on an object-incoming side of an input image as templates.

3. An object recognition system according to claim 1, wherein said template extracting circuit extracts an input image corresponding to an area having selected one of a large spatial frequency feature and a large time frequency feature as a template of predetermined size.

4. An object recognition system according to claim 1, wherein the template extracting circuit extracts an area representing a similarity of a moving object from an input image and extracts an input image corresponding to a particular area as said templates.

5. An object recognition system according to claim 1, wherein said separation/integration circuit separates or integrates templates having a similar speed and direction of movement of an object on a basis of the trace of movement determined for each template.

6. An object recognition system according to claim 1, wherein said following means comprises:

a means for learning a past behavior pattern by a neural network;

a means for predicting future behavior from the past behavior pattern and a present behavior pattern; and

a means for determining a range of correlation calculation with respect to templates.

7. An object recognition system according to claim 1, further comprising a template updating circuit for updating a template image stored by use of an input image.

8. An object recognition system according to claim 7, wherein the template updating circuit matches patterns between an input image f(t) at time point t and a template g, and in a case where a coordinate P with a similarity degree larger than a predetermined threshold value is determined, an image in the vicinity of the coordinate P of input image f(t) is registered newly as a gray scale template g'"'"'.

9. An object recognition system according to claim 7, wherein the template updating circuit matches patterns between an input image f(t) at time point t and a template g, and in a case where a coordinate P with a similarity degree larger than a predetermined threshold value is determined, a new template expressed by g'"'"'=α

f'"'"'(t)+β

g (σ

, β

;

constants) using an image f'"'"'(t) in the vicinity of the coordinate P of all the input images f(t) is registered.

10. An object recognition system for photographing an object by a TV camera and recognizing movement of the object by processing input images from the TV camera, comprising:

a template extracting circuit extracting a plurality of images in a template area of predetermined size from an input image f(t) at a time point t;

a template memory circuit storing a plurality of images obtained by the template extracting circuit;

a pattern matching circuit determining a movement point of each template by shade pattern matching between each template stored in the template memory circuit and the input images at incrementing time points obtained from the TV camera;

a trace management table storing a relationship between a time point and a movement point for each gray scale template obtained at the gray scale pattern matching circuit; and

a separation/integration circuit deciding which templates represent an image of a same object by use of the trace management table.

11. A system for measuring an intersection traffic flow comprising:

a template extracting circuit extracting a plurality of images of a template area of predetermined size from an image at time point t;

a gray scale template memory circuit storing a plurality of templates obtained with the template extracting circuit;

a pattern matching circuit matching patterns between each of the templates stored in said template memory circuit and input images at time points (t+dt), (t+2dt), . . . (t+ndt) (n;

integral number) obtained from a TV camera and determining a movement point of each of said templates;

a template updating circuit sequentially updating the templates of said template memory circuit using an input image;

a trace management table storing a relationship between a time point and a moving point for each template obtained with the pattern matching circuit;

a separation/integration circuit deciding on which templates represent a same object by use of the trace management table; and

a traffic parameter measurement section measuring a number and speed of vehicles passed, a number of vehicles turned to a right/left by following a vehicle position obtained.

12. A vehicle recognition system for photographing a road by a TV camera and recognizing movement of vehicles by processing images from the TV camera, comprising:

a template memory circuit for storing several types of vehicles images as images in advance; and

a pattern matching circuit for determining a vehicle position within an input image by pattern matching between the templates stored in the template memory circuit and the input image obtained from the TV camera.

13. A vehicle recognition system according to claim 12, further comprising a template pattern preparation circuit having a plurality of template patterns T(0) to T(n) for determining a similarity degree between the template patterns and for preparing a new template pattern from patterns similar to each other.

14. A vehicle recognition system for photographing a road by a TV camera and recognizing vehicle movement by processing images from the TV camera, comprising:

a template extracting circuit for extracting an image pattern corresponding to a vehicle as a template from an input image obtained by the TV camera;

a template memory circuit for storing a template obtained with the template extracting circuit;

a pattern matching circuit for determining a vehicle position within a subsequent input image by pattern matching between the template stored in the template memory circuit and said subsequent input image; and

a template extracting circuit for cutting out an area of predetermined size from an image entered by a vehicle, determining an extracting position in such a manner as to locate the vehicle at a center thereof from features of an extracted image, and registering an input image of the extracting position and a vicinity thereof as templates.

15. A traffic flow measurement system for photographing a road by a TV camera and recognizing vehicle movement by processing images from the TV camera, comprising:

a template memory circuit storing a vehicle image corresponding to a vehicle as a template

a pattern matching circuit pattern matching between the template stored in the template memory circuit and a subsequent input image obtained from the TV camera; and

a vehicle speed and direction determining means for determining a vehicle position by pattern matching with respect to the input image at a given time point, further determining the vehicle position with respect to the input image at a given time point and determining at least a speed and direction of movement of vehicles, in accordance with a change in coordinates of determined positions;

wherein said vehicle speed and direction determining means determines a range of pattern matching of the template against the input image f(t+Δ

t) at a time point t+Δ

t on a basis of the speed and direction of movement obtained from a result of pattern matching of images f(t-Δ

t), f(t) at time points (t-Δ

t) and t respectively.

16. A pattern matching system for executing pattern matching between a template and an input image, comprising:

a template memory circuit storing a pattern of an image;

a pattern matching circuit pattern matching between a template stored in the template memory circuit and a subsequent input image obtained from a TV camera; and

a template updating circuit sequentially updating the templates of the template memory circuit using input images;

wherein pattern matching is effected with a given template g against an input image f(t) at time point t, and in a case where a coordinate P is determined to have a similarity degree obtained by pattern matching larger than a predetermined threshold value, an input image f(t) of the coordinate P and a vicinity thereof is registered as a new template g'"'"'.

17. A pattern matching system for executing pattern matching between a template and an input image, comprising:

a template memory circuit storing a pattern of an image;

a pattern matching circuit pattern matching between a template stored in the template memory circuit and a subsequent input image obtained from a TV camera; and

a template updating circuit sequentially updating the templates of the template memory circuit using input images;

wherein pattern matching is effected with an input image f(t) at a time point t against a template g, and in a case where a coordinate P is determined to have a similarity degree larger than a predetermined threshold value, an input image f(t) of the coordinate P and a vicinity thereof is used to register a template expressed as g'"'"'=α

f'"'"'(t)+β

g (α

, β

;

constants) as a new template.

18. A pattern matching system for executing pattern matching between a template and an input image, comprising:

a template memory circuit storing a pattern of an image;

a pattern matching circuit pattern matching between a template stored in the template memory circuit and a subsequent input image obtained from a TV camera; and

a template updating circuit sequentially updating the templates of the template memory circuit using input images;

wherein a template is updated while changing a template size in accordance with a size of an object.

19. A system for following, in a picked-up image, a moving object of a shape and size changing with the direction of movement and distance from the image pick-up means, comprising:

a template memory circuit for storing a pattern of an image of the moving object as a template;

a pattern matching circuit for determining a position of the moving object by pattern matching between the template stored in the template memory circuit and an input image obtained from the image pick-up means; and

a template updating circuit for preparing an updated template for a next pattern matching from the template of the moving object cut out of a present input image and a preceding template of the moving object and storing the updated template in the template memory circuit.

20. An abnormality detection system for detecting an abnormal phenomenon by photographing a road with a plurality of TV cameras installed at predetermined intervals on the road and processing images from the TV cameras, comprising:

a plurality of image processing units including a local overall decision section having functions of processing images from the TV cameras and measuring a speed and number of vehicles, monitoring abnormally-running vehicles, detecting a stationary object and measuring a congestion degree, and correcting data on the speed and number of vehicles and the abnormally-running vehicle monitor functions in accordance with the congestion degree; and

a central overall decision section making an overall decision by spatial and temporal interpolation of a result of measurement at the image processing units.

21. An abnormality detection system according to claim 20, wherein the central overall decision section includes a means for collecting several types of information from a plurality of image processing units, a spatial interpolation means for grasping said information spatially and making an abnormality decision only when said information undergoes a local change, and a time interpolation means for grasping a plurality of information in time series and deciding on an initial time of abnormality occurrence and an instantaneous abnormality.

22. An abnormality detection system for detecting an abnormal phenomenon by photographing a road with a TV camera and by processing images from the TV camera, comprising:

a local overall decision section having functions of measuring a speed and number of vehicles, monitoring abnormally-running vehicles, detecting a stationary object and measuring a congestion degree for correcting data on the speed and number of vehicles and abnormally-running vehicle monitor functions in accordance with the congestion degree;

wherein the function of monitoring abnormally-running vehicles is performed using a means for extracting vehicles from an input image, a means for determining an extreme left coordinate, extreme right coordinate and end coordinate of vehicles from a vehicle image extracted, a means for calculating an amount of vehicle movement from coordinates thus determined and an image inputted at a next time point, a means for accumulating resulting coordinate values and speed and direction of movement for a number of vehicles involved, and means for making an abnormality decision when coordinates exceed a threshold value of a frequency of vehicles running outside a predetermined position, a frequency of vehicles running at other than a predetermined speed and a frequency of vehicles moving in other than a predetermined direction, respectively.

23. An abnormality detection system for detecting an abnormal phenomenon by photographing a road with a TV camera and by processing images from the TV camera, comprising:

a local overall decision section having functions of measuring a speed and number of vehicles, monitoring abnormally-running vehicles, detecting a stationary object and measuring a congestion degree for correcting data on the speed and number of vehicles and abnormally-running vehicle monitor functions in accordance with the congestion degree;

wherein the local overall decision section cancels a speed data obtained with a high congestion degree, when the speed data is higher than a predetermined threshold value.

24. An abnormality detection system for detecting an abnormal phenomenon by photographing a road with a TV camera and by processing images from the TV camera, comprising:

a local overall decision section having functions of measuring a speed and number of vehicles, monitoring abnormally-running vehicles, detecting a stationary object and measuring a congestion degree for correcting data on the speed and number of vehicles and abnormally-running vehicle monitor functions in accordance with the congestion degree;

wherein the local overall decision section corrects data on the speed and the number of vehicles determined, in accordance with the congestion degree.