Neural network-controlled automatic tracking and recognizing system and method

• US 8,036,425 B2
• Filed: 06/26/2008
• Issued: 10/11/2011
• Est. Priority Date: 06/26/2008
• Status: Active Grant

• Current Assignee: InnoSys Industries Limited
• Original Assignee: InnoSys Industries Limited
• Inventors: Hou, Billy
• Primary Examiner(s): Tabatabai; Abolfazl
• Application Number:
  US12/213,902
• Publication Number:
  US 20090324010A1
• Time in Patent Office: 1,202 Days
• Field of Search: 382/100, 382/103, 382/104, 382/107, 340/287, 340/425.5, 340/426.18, 340/438, 340/506, 340/577, 340/578, 340/584, 340/628, 340/901, 348/154, 348/155, 348169-172, 348/208.14, 348/352, 700/48, 704/202, 704/232, 704/259, 706/2, 706/6, 706/15, 706/33, 706/41, 706/44
• US Class Current: 382/103
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A neural network-controlled automatic tracking and recognizing system includes a fixed field of view collection module, a full functions variable field of view collection module, a video image recognition algorithm module, a neural network control module, a suspect object track-tracking module, a database comparison and alarm judgment module, a monitored characteristic recording and rule setting module, a light monitoring and control module, a backlight module, an alarm output/display/storage module, and security monitoring sensors. The invention relates also to the operation method of the system.

Claims

1. A neural network-controlled automatic tracking and recognizing system, comprising:

a fixed field of view collection module, said fixed field of view collection module comprising a plurality of sensing cameras for collecting view images of fixed spots;

a full functions variable field of view collection module, said full functions variable field of view collection module comprising a plurality of full functions tracking camera for catching images of a suspect moving object appeared in the coverage of said fixed field of view collection module within 360-degrees;

a video image recognition algorithm module comprising a series of algorithms adapted for picking up a target object from video images collected by said full functions variable field of view collection module for analysis to identify the characteristic parts of the target object such as human face and car license number;

a neural network control module for controlling angle matching of said full functions tracking cameras with said sensing cameras and for controlling the angle of rotation, focus and aperture of said full functions tracking cameras subject to the allowable moving object moving direction in the fixed field of view so that said full functions cameras are controlled to track every suspect moving object and to catch the detail characteristics of every suspect moving object;

a suspect object track-tracking module adapted for tracking the track of the suspect target such as human face or object gravity center subject the images obtained through said sensing cameras, and recording/building up the motion track of the suspect object subject to the recognition results of the algorithms of said video image recognition algorithm module;

a database comparison and alarm judgment module adapted for fetching human face data, suspect object characteristic data, and other related database data for comparison subject to the recognition results of the algorithms of said video image recognition algorithm module, and determining a report of alarm of “

cross border”

, “

enter restricted area”

, “

wrong moving direction”

, etc., subject to set rules;

a monitored characteristic recording and rule setting module adapted for the input of characteristics of target object, such as human face image, to establish a database, and for the setting of alarm rule and sensitivity grade subject to requirements of the monitored area;

a light monitoring and control module adapted for analyzing surrounding light status subject to video images obtained through the sensing cameras and full functions tracking cameras of said fixed field of view collection module and said full functions variable field of view collection module, and controlling a backlight module to provide back light when the surrounding light is insufficient for monitoring;

a backlight module controllable by said light monitoring and control module to turn on infrared light source means and artificial light source means thereof to provide a backlight subject the condition of the monitored site;

an alarm output/display/storage module adapted for displaying alarm information and relay output and for management and storage of monitored video images and alarm information; and

security monitoring sensors linked with other security monitoring systems in such a manner that when the security monitoring sensors are started, said full functions tracking cameras are controlled to catch the desired data.

2. The neural network-controlled automatic tracking and recognizing system as claimed in claim 1, wherein said sensing cameras are pistol/ball type cameras;

said full functions tracking cameras are high-speed pan-tilt-zoom cameras/dome cameras;

said sensing cameras and said full functions tracking cameras are selectable from the group of CCD (charge coupled device), CMOS (complementary metal-oxide semiconductor) and thermal imaging cameras.

3. The neural network-controlled automatic tracking and recognizing system as claimed in claim 1, wherein said video image recognition algorithm module comprises a human figure recognition algorithm, a human head positioning algorithm, a moving object recognition algorithm, a behavior recognition algorithm, and a characteristic recognition algorithm.

4. The neural network-controlled automatic tracking and recognizing system as claimed in claim 1, which adopts a neural network control method to catch the target object like the functioning of a human eye by means of driving said sensing cameras to lock the suspect object and then driving said full functions tracking cameras to track the target object, catching the important characteristics of the target object including human face and car license number.

5. The neural network-controlled automatic tracking and recognizing system as claimed in claim 1, which said video image recognition algorithm module comprises a behavior recognition algorithm adapted for recognizing different behaviors including cross border, moving direction, moving speed, stealing behavior, properties left behind, and etc.

6. The neural network-controlled automatic tracking and recognizing system as claimed in claim 1, which said backlight module comprises near infrared light source means and visible light source means;

said sensing cameras and said full functions tracking cameras have a night vision function for catching near infrared images of a moving object.

7. A neural network-controlled automatic tracking and recognizing method comprising the steps of:

(1) driving sensing cameras and full functions tracking cameras to catch target images and driving predetermined algorithms to pick up target characteristic parameters including human face image and characteristics and car license number for storing in a system database, guiding pre-recorded images or target characteristic parameters into the system database, and setting behavior rules including cross border, moving direction abnormality, moving speed abnormality and moving direction abnormality;

(2) Camera position correction to have the monitoring area of said full functions tracking cameras and sensing cameras be combined together for associating mapping, by means of manual adjustment of the angle and focus of said full functions tracking cameras, selected points of images obtained from said full functions tracking cameras being corresponded to corresponding points of images obtained from said sensing cameras, so that a parameter is provided for the control of the viewing angle of said full functions tracking cameras in neural network control for enabling a neural network control module to control the default viewing angle of said full functions tracking cameras in matching with said sensing cameras in coarse adjustment. After through a further fine adjustment and control, the full functions tracking cameras catch the detail characteristics of the suspect target.

(3) The system continuously collecting the video images from said sensing cameras for analysis in such a manner that when a suspect target is found, said neural network control module controls the rotation, focus adjustment and aperture of said full functions tracking cameras to catch the images of the suspect target, keeping the characteristic part of the suspect target on the center area of each image and the area of the characteristic part of the suspect target in each image to be not less than 15%;

following movement of the target, said neural network control module controls said full functions tracking cameras to rotate and to adjust the focus, continuously tracking the characteristic part of the suspect object;

(4) The system performs an intelligent image analysis on the video images collected from said sensing cameras, checking every behavior against the behavior rule, such as cross border, wrong moving direction, abnormal moving speed, stealing behavior, etc;

when a violation behavior is found, the target is regarded as a suspect target, and at the same time the system automatically tracks the motion track of the suspect target such as the gravity center or human head subject to the type of the monitored object;

(5) The system performs an intelligent image analysis on the video images obtained through said full functions tracking cameras by means of a biological identification technology to pick up a characteristic part of the suspect target and the related parameters, the related parameters including skin color of human face, interpupillary width, skeleton features, features of five sense organs, car license number and car license color.

(6) The system compares the characteristic parameters of the suspect target thus obtained with the characteristic parameters stored in the database, and then gives a respective alarm report to every item that is not in conformity with the set conditions;

the alarm rules and conditions include alarms in conformity with database characteristics, and alarms not conformity with database characteristics, with respect to any recognized characteristic, the system performs a self-learning program and stores the newly obtained new characteristic parameter in the database.

8. The neural network-controlled automatic tracking and recognizing method as claimed in claim 7, wherein the establishment of target characteristics database is done by means of recording of system monitored target characteristics and real time recording by a monitored characteristic recording and rule setting module, for example, recording different angles and profiles of human face images on the real time or storing pre-recorded images into the database.

9. The neural network-controlled automatic tracking and recognizing method as claimed in claim 7, wherein during step (4) in tracking the motion track of the suspect target is to complete track calculation, tracking and marking within the coverage of the video images and to map track coordinates onto architecture plan coordinates so as to mark the track of the suspect target on the architecture plan.

10. The neural network-controlled automatic tracking and recognizing method as claimed in claim 7, wherein the step (5) in picking up a characteristic part of the suspect target and the related parameters comprises the sub-steps of:

(a) analyzing every image of the series of images obtained from the sensing cameras so as to pick up moving objects and to calculate parameters of proportional characteristics, color distribution characteristics, marks of violation to rule, gravity center, motion track and direction predication of moving objects;

(b) catching enlarged detail image of the suspect object against rules through a full functions variable field of view collection module so as to pick up a characteristic part of the subject object such as human face or car license number;

(c) calculating the parameters of the characteristic part of the suspect object, including body-size ratio, human face skeleton features, interpupillary width, futures of five sense organs, skin color, car license number and car color; and

(d) Roll-polling the sensing cameras to lock each of suspect targets appeared in the monitoring area, and running cluster analysis when a cluster of moving objects appeared where the system catches the nearest suspect target whose characteristic part is clear, and the system adopts roll polling to catch the characteristic part of every suspect target for recognition and calculation when distributed suspect targets are discovered.