Supervision and penalty method and system for expressway emergency lane occupancy

• US 10,147,153 B2
• Filed: 03/03/2017
• Issued: 12/04/2018
• Est. Priority Date: 09/07/2016
• Status: Active Grant

• Current Assignee: Southeast University
• Original Assignee: Southeast University
• Inventors: Ye, Zhirui, Yan, Yu, Zhu, Jiaojiao, Huang, Yingzi, Sun, Cuicui
• Primary Examiner(s): AHMED, SAMIR ANWAR
• Application Number:
  US15/570,340
• Publication Number:
  US 20180293684A1
• Time in Patent Office: 641 Days
• CPC Class Codes: G06Q 50/265 Personal security, identity or safety
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/153 using recognition of characters or words
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules

Abstract

A supervision and penalty method and system for expressway emergency lane occupancy based on public supervision and reporting. The system includes a mobile terminal and a central computer. When a citizen notices a violation behavior of emergency lane occupancy of another vehicle, evidence is collected by photographing and images are uploaded to the central computer; meanwhile, user location information is automatically read and uploaded. The central computer acquires, through network transmission, image information and location information uploaded by a client, distinguishes whether it is a malicious reporting behavior and performs technical processing and license plate recognition on the uploaded images, checks the road section where the violation behavior occurs, and determines whether the violation behavior is an emergency lane violation behavior. The computer automatically searches for and matches owner information corresponding to the license plate number of the violation vehicle, and automatically sends a penalty decision from the background.

Claims

1. A supervision and penalty system for expressway emergency lane occupancy, comprising a mobile terminal and a central computer, wherein

the mobile terminal further comprises a real-name authentication module, an image uploading module, and a positioning, storing and uploading module, wherein the real-name authentication module is used to receive personal identity information of a registered user;

the image uploading module is used to receive a plurality of violation photos taken by the user and upload the plurality of violation photos to a background central computer; and

the positioning, storing and uploading module is used to read current location information of the user when taking a photo, and automatically store and match the plurality of violation photos taken by the user;

the central computer further comprises an image identification unit, an image processing unit, an image recognition unit, a violation determining unit, an information matching unit, and a data storage unit;

wherein

the image identification unit is used to parse coded information of a plurality of images and determine whether each of the plurality of images is an original image without a later-stage editing;

the image processing unit is used to pre-process a plurality of captured images, so as to improve the image quality, eliminate noises, and improve the image definition;

the image recognition unit recognizes and records a license plate number of a violation vehicle through a license plate positioning module, a license plate character segmentation module, and a license plate character recognition module;

the violation determining unit further comprises a lane boundary determining module and a vehicle state determining module;

wherein the lane boundary determining module determines whether a vehicle is on an emergency lane according to an emergency lane boundary recognized by the image recognition unit;

the vehicle state determining module automatically compares two or more processed photos to determine whether the vehicle is traveling, pulling over or stopping on the emergency lane;

the vehicle state determining module automatically selects a feature point of each of the plurality of images and compares a relative position between the feature points to determine a state of the vehicle, and if the vehicle is in a stopping state, the vehicle state determining module continues to automatically search for a red feature point in the each of the plurality of images, and determines whether a double flashing light of the vehicle is turned on to indicate an emergency breakdown situation, and if it is impossible to make a determination, the vehicle state determining module records a serial number of the each of the plurality of images and feeds back the same to a manual processing side;

the information matching unit is used to access a provincial registered vehicle owner information database, and upon obtaining the license plate number of the violation vehicle, search for corresponding owner information according to the license plate number;

the data storage unit is used to maximize a data extraction convenience and storage security in a mixture of an online storage and a nearline storage;

the mobile terminal stores and sends, through an image transmission by a reporter and a positioning of the reporter, information of a vehicle traveling on and occupying the emergency lane;

the central computer receives the plurality of images and location information from the mobile terminal through a network transmission under a network data transmission encryption protocol, examines, through a computer built-in examining module, whether the plurality of images are properly captured, checks a road section where a violation behavior occurs, and determines whether the violation behavior is an emergency lane violation behavior, recognizes and records the license plate, accesses owner information for a fine and point deduction notification, and rewards the reporter.

2. The supervision and penalty system for expressway emergency lane occupancy according to claim 1, wherein in the image recognition unit, the license plate positioning module is used to perform an image feature search to find a plurality of areas conforming to a plurality of license plate features as a plurality of candidate areas, analyze the plurality of candidate areas to select a most matching area as an area to be recognized, and separate the area to be recognized from the image;

the license plate character segmentation module is used to segment a selected license plate area into a plurality of individual characters to be recognized respectively, wherein a vertical projection method is adopted for a character segmentation;

the license plate character recognition module performs a feature extraction on a plurality of segmented characters mainly based on an artificial neural network, and trains a neural network distributor using a plurality of obtained features, so as to obtain a result by matching; and

the image recognition unit recognizes the emergency lane boundary using a pixel-level feature by extracting a road edge and a characteristic curve.

3. The supervision and penalty system for expressway emergency lane occupancy according to claim 1, wherein the mobile terminal is a smartphone or tablet computer running an operating system selected form the group consisting of Android, iOS, windows phone, Symbian, BlackBerry OS, and any other operating system.

4. A supervision and penalty method for expressway emergency lane occupancy, using a supervision and penalty system for expressway emergency lane occupancy, wherein, the supervision and penalty system comprises a mobile terminal and a central computer, wherein

the mobile terminal further comprises a real-name authentication module, an image uploading module, and a positioning, storing and uploading module, wherein the real-name authentication module is used to receive personal identity information of a registered user;

the image uploading module is used to receive a plurality of violation photos taken by the user and upload the plurality of violation photos to a background central computer; and

the positioning, storing and uploading module is used to read current location information of the user when taking a photo, and automatically store and match the plurality of violation photos taken by the user;

the central computer further comprises an image identification unit, an image processing unit, an image recognition unit, a violation determining unit, an information matching unit, and a data storage unit wherein

the image identification unit is used to parse coded information of a plurality of images and determine whether each of the plurality of images is an original image without a later-stage editing;

the image processing unit is used to pre-process a plurality of captured images, so as to improve the image quality, eliminate noises, and improve the image definition;

the image recognition unit recognizes and records a license plate number of a violation vehicle through a license plate positioning module, a license plate character segmentation module, and a license plate character recognition module;

the violation determining unit further comprises a lane boundary determining module and a vehicle state determining module;

wherein the lane boundary determining module determines whether a vehicle is on an emergency lane according to an emergency lane boundary recognized by the image recognition unit the vehicle state determining module automatically compares two or more processed photos to determine whether the vehicle is traveling, pulling over or stopping on the emergency lane;

the vehicle state determining module automatically selects a feature point of each of the plurality of images and compares a relative position between the feature points to determine a state of the vehicle, and if the vehicle is in a stopping state, the vehicle state determining module continues to automatically search for a red feature point in the each of the plurality of images, and determines whether a double flashing light of the vehicle is turned on to indicate an emergency breakdown situation, and if it is impossible to make a determination, the vehicle state determining module records a serial number of the each of the plurality of images and feeds back the same to a manual processing side;

the information matching unit is used to access a provincial registered vehicle owner information database, and upon obtaining the license plate number of the violation vehicle search for corresponding owner information according to the license plate number;

the data storage unit is used to maximize a data extraction convenience and storage security in a mixture of an online storage and a nearline storage;

the mobile terminal stores and sends, through an image transmission by a reporter and a positioning of the reporter, information of a vehicle traveling on and occupying the emergency lane;

the central computer receives the plurality of images and location information from the mobile terminal through a network transmission under a network data transmission encryption protocol, examines, through a computer built-in examining module, whether the plurality of images are properly captured, checks a road section where a violation behavior occurs, and determines whether the violation behavior is an emergency lane violation behavior, recognizes and records the license plate, accesses owner information for a fine and point deduction notification, and rewards the reporter;

the method comprising the following steps;

Step 1;

when a first vehicle is traveling on or occupying an emergency lane and a second vehicle passing by notices a violation behavior, turning on the mobile terminal wherein the mobile terminal is allowed to read the user location, photographing a driving state of the first vehicle, and obtaining a plurality of photos capable of clearly showing the license plate number and the emergency lane line, wherein at least two photos are taken;

after taking the plurality of photos, selecting a plurality of images from the plurality of photos and uploading the plurality of images, and determining whether to add a text description according to the situation;

Step 2;

after obtaining the plurality of violation images and the location information, preliminarily distinguishing, by the central computer, an image source through the image identification unit to determine whether the plurality of violation images are taken in a normal condition, and if yes, performing Step 3;

otherwise, sending a warning to the reporter, providing feedback to the mobile terminal through an information feedback mechanism, and imposing a penalty;

Step 3;

performing, by the central computer, a compression, an enhancement and restoration, and a segmentation and description processing on a plurality of uploaded violation images through the image processing unit, so as to eliminate noises, improve the image definition, and enable the captured image to be more easily recognized and distinguished;

Step 4;

performing, by the central computer, an accurate recognition on the license plate in the plurality of images by applying the license plate positioning module, the license plate character segmentation module, and the license plate character recognition module in the image recognition unit, and recognizing the emergency lane boundary using the pixel-level feature by extracting the road edge and the characteristic curve;

Step 5;

performing, by the central computer, a violation determination on a reported vehicle through the violation determining unit, and if it is determined that the reported vehicle occupies the emergency lane in a non-emergency situation, performing Step 6;

otherwise, explaining the situation to the reporter, and sending feedback to a client end through the information feedback mechanism;

Step 6;

invoking, by the central computer, a database in the system through the information matching unit to automatically search for registered owner information corresponding to the license plate of the violation vehicle, sending a violation-handling decision, and specifying a time limit within which the owner should deal with the violation at a specific public security department;

meanwhile, checking, by the central computer, reporter information and rewarding the reporter after determining that the report is effective; and

Step 7;

storing, in the data storage unit for a future reference, the reporter information and corresponding reported violation place information, license plate information of the violation vehicle, and owner information of the violation vehicle as an entire information block.

6. The supervision and penalty method for expressway emergency lane occupancy, using a supervision and penalty system for expressway emergency lane occupancy according to claim 4,

wherein in the image recognition unit, the license plate positioning module is used to perform an image feature search to find a plurality of areas conforming to a plurality of license plate features as a plurality of candidate areas, analyze the plurality of candidate areas to select a most matching area as an area to be recognized, and separate the area to be recognized from the image;

the license plate character segmentation module is used to segment a selected license plate area into a plurality of individual characters to be recognized respectively, wherein a vertical projection method is adopted for a character segmentation;

the license plate character recognition module performs a feature extraction on a plurality of segmented characters mainly based on an artificial neural network, and trains a neural network distributor using a plurality of obtained features, so as to obtain a result by matching; and

the image recognition unit recognizes the emergency lane boundary using a pixel-level feature by extracting a road edge and a characteristic curve.

7. The supervision and penalty method for expressway emergency lane occupancy, using a supervision and penalty system for expressway emergency lane occupancy according to claim 4,

wherein the mobile terminal is a smartphone or tablet computer running an operating system selected form the group consisting of Android, iOS, windows phone, Symbian, BlackBerry OS, and any other operating system.

5. A supervision and penalty method for expressway emergency lane occupancy, comprising the following steps:

Step 1;

when a first vehicle is traveling on or occupying an emergency lane and a second vehicle passing by notices a violation behavior, turning on a mobile terminal wherein the mobile terminal is allowed to read a user location, photographing a driving state of the first vehicle, and obtaining a plurality of photos capable of clearly showing a license plate number and an emergency lane line, wherein at least two photos are taken;

after taking the plurality of photos, selecting a plurality of images from the plurality of photoes and uploading the plurality of images, and determining whether to add a text description according to the situation;

Step 2;

after obtaining the plurality of violation images and location information, preliminarily distinguishing, by a central computer, an image source through an image identification unit to determine whether the plurality of images are taken in a normal condition, and if yes, performing Step 3;

otherwise, sending a warning to a reporter, providing feedback to the mobile terminal through an information feedback mechanism, and imposing a penalty;

Step 3;

performing, by the central computer, a compression, an enhancement and restoration, a segmentation and description processing on a plurality of uploaded violation images through an image processing unit, so as to eliminate noises, improve the image definition, and enable the plurality of captured images to be more easily recognized and distinguished;

Step 4;

performing, by the central computer, an accurate recognition on a license plate in the plurality of images by applying a license plate positioning module, a license plate character segmentation module, and a license plate character recognition module in an image recognition unit, and recognizing an emergency lane boundary using a pixel-level feature by extracting a road edge and a characteristic curve;

Step 5;

performing, by the central computer, a violation determination on a reported vehicle through a violation determining unit, and if it is determined that the reported vehicle occupies the emergency lane in a non-emergency situation, performing Step 6;

otherwise, explaining the situation to the reporter, and sending feedback to a client end through the information feedback mechanism;

Step 6;

invoking, by the central computer, a database in the system through an information matching unit to automatically search for registered owner information corresponding to the license plate of a violation vehicle, sending a violation-handling decision, and specifying a time limit within which the owner should deal with the violation at a specific public security department;

meanwhile, checking, by the central computer, reporter information and rewarding the reporter after determining that the report is effective; and

Step 7;

storing, in a data storage unit for a future reference, each of the reporter information and corresponding reported violation place information, license plate information of the violation vehicle, and owner information of the violation vehicle as an entire information block.