Coarse-to-fine cascade adaptations for license plate recognition with convolutional neural networks

• US 9,785,855 B2
• Filed: 12/17/2015
• Issued: 10/10/2017
• Est. Priority Date: 12/17/2015
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Inventors: Gordo Soldevila, Albert, Almazan, Jon
• Primary Examiner(s): Liew, Alex
• Application Number:
  US14/972,481
• Publication Number:
  US 20170177965A1
• Time in Patent Office: 663 Days
• Field of Search: 382224-228
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/18057 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]

Abstract

Methods and systems for license plate recognition utilizing a trained neural network. In an example embodiment, a neural network can be subject to operations involving iteratively training and adapting the neural network for a particular task such as, for example, text recognition in the context of a license plate recognition application. The neural network can be trained to perform generic text recognition utilizing a plurality of training samples. The neural network can be applied to a cropped image of a license plate in order to recognize text and produce a license plate transcription with respect to the license plate. An example of such a neural network is a CNN (Convolutional Neural. Network).

Claims

1. A method for license plate, recognition, said method comprising:

generating a neural network having a plurality of convolutional filters and at least one fully connected layer and wherein said neural network ends in a plurality of independent classifiers, wherein although said classifiers among said plurality of independent classifiers are independent of one another, said classifiers are capable of being trained jointly together with remaining parameters of said neural network;

training said neural network in a coarse-to-fine manner to perform generic text recognition utilizing a plurality of training samples;

iteratively learning and adapting said neural network; and

applying said neural network to a cropped image of a license plate in order to recognize text associated with said license plate and produce a license plate transcription with respect to said license plate.

2. The method of claim 1 wherein said neural network comprises a CNN (Convolutional Neural Network).

3. The method of claim 2 further comprising adapting said neural network to perform tasks increasingly similar to license plate recognition in a target set as a part of training said neural network and wherein said plurality of independent classifiers utilize a cross-entropy loss for said training of said neural network.

4. The method of claim 1 wherein iteratively learning and adapting said neural network, further comprises:

iteratively learning and adapting said neural network by fine tuning said neural network.

5. The method of claim 3 wherein said fine tuning said neural network comprises utilizing a cascade of coarse-to-fine adaptations so as to iteratively learn and adapt said neural network.

6. The method of claim 2 further comprising adapting said neural network to perform tasks increasingly similar to license plate recognition in a target set as a part of training said neural network.

7. The method of claim 2 wherein:

iteratively learning and adapting said neural network, further comprises;

iteratively learning and adapting said neural network by fine tuning said neural network wherein said fine tuning said neural network comprises utilizing a cascade of coarse-to-fine adaptations so as to iteratively learn and adapt said neural network, wherein said coarse-to-fine manner includes said cascade of coarse-to-fine adaptations and wherein said fine tuning further includes utilizing all available samples from a target domain even if said all available samples are derived from different scenarios, and wherein said neural network is subsequently fine-tuned again with respect to only a target subdomain that is desired to be recognized.

8. A system for license plate recognition, said system comprising:

at least one processor; and

a computer-usable medium embodying computer program code, said computer-usable medium capable of communicating with said at least one processor, said computer program code comprising instructions executable by said at least one processor and configured for;

generating a neural network having a plurality of convolutional filters and at least one fully connected layer and wherein said neural network ends in a plurality of independent classifiers, wherein although said classifiers among said plurality of independent classifiers are independent of one another, said classifiers are capable of being trained jointly together with remaining parameters of said neural network;

training said neural network in a coarse-to-fine manner to perform generic text recognition utilizing a plurality of training samples;

iteratively learning and adapting said neural network; and

applying said neural network to a cropped image of a license plate in order to recognize text associated with said license plate and produce a license plate transcription with respect to said license plate.

9. The system of claim 8 wherein said neural network comprises a CNN (Convolutional Neural Network).

10. The system of claim 9 wherein said instructions are further configured for adapting said neural network to perform tasks increasingly similar to license plate recognition in a target set as a part of training said neural network and wherein said plurality of independent classifiers utilize a cross-entropy loss for said training of said neural network.

11. The system of claim 8 wherein said instructions for iteratively learning and adapting said neural network, further comprises instructions configured for iteratively learning and adapting said neural network by fine tuning said neural network.

12. The system of claim 11 wherein said fine tuning said neural network comprises utilizing a cascade of coarse-to-fine adaptations so as to iteratively learn and adapt said neural network.

13. The system of claim 9 wherein said instructions are further configured for adapting said neural network to perform tasks increasingly similar to license plate recognition ire a target set as a part of training said neural network.

14. The system of claim 9 wherein:

said instructions for iteratively learning and adapting said neural network are further configured for;

iteratively learning and adapting said neural network by fine tuning said neural network wherein said fine tuning said neural network comprises utilizing a cascade of coarse-to-fine adaptations so as to iteratively learn and adapt said neural network, wherein said coarse-to-fine manner includes said cascade of coarse-to-fine adaptations and wherein said fine tuning further includes utilizing all available samples from a target domain even if said all available samples are derived from different scenarios, and wherein said neural network is subsequently fine-tuned again with respect to only a target subdomain that is desired to be recognized.

15. A non-transitory computer-readable medium storing code representing instructions to cause a process for license plate recognition, said code including code to:

generate a neural network having a plurality of convolutional filters and at least one fully connected layer and wherein said neural network ends in a plurality of independent classifiers, wherein although said classifiers among said plurality of independent classifiers are independent of one another, said classifiers are capable of being trained jointly together with remaining parameters of said neural network;

train said neural network in a coarse-to-fine manner to perform generic text recognition utilizing a plurality of training samples;

iteratively learn and adapt said neural network; and

apply said neural network to a cropped image of a license plate in order to recognize text associated with said license plate and produce a license plate transcription with respect to said license plate.

16. The non-transitory computer-readable of claim 15 wherein said neural network comprises a CNN (Convolutional Neural Network).

17. The non-transitory computer-readable of claim 16 wherein said code further includes code to adapt said neural network to perform tasks increasingly similar to license plate recognition in a target set as a part of training said neural network and wherein said plurality of independent classifiers utilize a cross-entropy loss for said training of said neural network.

18. The non-transitory computer-readable of claim 15 wherein iteratively learning and adapting said neural network, further comprises:

iteratively learning and adapting said neural network by fine tuning said neural network.

19. The non-transitory computer-readable of claim 18 wherein said fine tuning said neural network comprises utilizing, a cascade of coarse-to-fine adaptations so as to iteratively learn and adapt said neural network.

20. The non-transitory computer-readable of claim 16 wherein said code further includes code to adapt said neural network to perform tasks increasingly similar to license plate recognition in a target set as a part of training said neural network and wherein said iteratively learn and adapt said neural network further comprises iteratively learning and adapting said neural network by fine tuning said neural network wherein said fine tuning said neural network comprises utilizing a cascade of coarse-to-fine adaptations so as to iteratively learn and adapt said neural network, wherein said coarse-to-fine manner includes said cascade of coarse-to-fine adaptations and wherein said fine tuning further includes utilizing all available samples from a target domain even if said all available samples are derived from different scenarios, and wherein said neural network is subsequently fine-tuned again with respect to only a target subdomain that is desired to be recognized.