LOGO DETECTION

• US 20200134377A1
• Filed: 10/25/2018
• Published: 04/30/2020
• Est. Priority Date: 10/25/2018
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Attorre, Brunno Fidel Maciel, Thien, Nicolas Huynh
• Granted Patent: US 10,769,496 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24143 Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 2201/09 Recognition of logos

Abstract

Disclosed herein are techniques for detecting logos in images or video. In one embodiment, a first logo detection model detects, from an image, candidate regions for determining logos in the image. A feature vector is then extracted from each candidate region and is compared with reference feature vectors stored in a database. The logo corresponding to the best matching reference feature vector is determined to be the logo in the candidate region if the best matching meets a certain criterion. In some embodiments, a second logo detection model trained using synthetic training images is used in combination with the first logo detection model to detect logos in a same image.

Claims

1. A method comprising:

receiving a source image at one or more computing devices;

detecting, in the source image and using a first logo detection model implemented by the one or more computing devices, a candidate region for determining a logo in the source image;

extracting, from the candidate region and by a neural network implemented using the one or more computing devices, a feature vector of the candidate region;

determining, for each reference feature vector from a set of reference feature vectors stored in a database, a respective matching score between the reference feature vector and the feature vector of the candidate region, wherein each reference feature vector in the set of reference feature vectors is extracted from a respective image of a target logo in a set of target logos;

selecting a first reference feature vector associated with a highest matching score among the set of reference feature vectors, the first reference feature vector extracted from a first image of a first target logo in the set of target logos; and

determining that the candidate region includes an image of the first target logo based on determining that the highest matching score is greater than a threshold value.

2. The method of claim 1, further comprising:

receiving an image of a new target logo to be added to the set of target logos;

extracting, using the neural network implemented using the one or more computing devices, a new reference feature vector from the image of the new target logo; and

storing the new reference feature vector in the database.

3. The method of claim 1, wherein the first logo detection model includes a fast region-based convolutional neural network (R-CNN).

4. The method of claim 1, wherein the neural network includes a convolutional neural network.

5. The method of claim 1, further comprising:

detecting, from the source image and using a second logo detection model implemented by the one or more computing devices, a second target logo in the set of target logos,

wherein the second logo detection model is trained using training images including the set of target logos to detect the set of target logos.

6. The method of claim 5, wherein the second logo detection model includes a Fast R-CNN.

7. The method of claim 5, wherein:

the first logo detection model and the second logo detection model are trained using synthetic training images;

each synthetic training image includes a background image and an image of a target logo in the set of target logos; and

the image of the target logo is superimposed on the background image.

8. The method of claim 7, wherein the image of the target logo is transformed before being superimposed on the background image.

9. The method of claim 7, wherein the image of the target logo is superimposed in a region of the background image, the region of the background image determined to be suitable for logo superposition.

10. The method of claim 9, wherein determining that the region of the background image is suitable for logo superposition comprises:

classifying each pixel in the background image as suitable or unsuitable for logo superposition; and

grouping neighboring pixels that are classified as suitable for logo superposition to form the region of the background image for logo superposition.

11. The method of claim 1, wherein the one or more computing devices include one or more cloud computing servers.

12. The method of claim 1, wherein the source image includes images of two or more logos.

13. A system comprising:

a processing device; and

a non-transitory computer-readable medium communicatively coupled to the processing device, wherein the processing device is configured to execute program code stored in the non-transitory computer-readable medium and thereby perform operations comprising;

receiving a source image at one or more computing devices;

detecting, in the source image and using a first logo detection model implemented by the one or more computing devices, a candidate region for determining a logo in the source image;

extracting, from the candidate region and by a neural network implemented using the one or more computing devices, a feature vector of the candidate region;

determining, for each reference feature vector from a set of reference feature vectors stored in a database, a respective matching score between the reference feature vector and the feature vector of the candidate region, wherein each reference feature vector in the set of reference feature vectors is extracted from a respective image of a target logo in a set of target logos;

selecting a first reference feature vector associated with a highest matching score among the set of reference feature vectors, the first reference feature vector extracted from a first image of a first target logo in the set of target logos; and

determining that the candidate region includes an image of the first target logo based on determining that the highest matching score is greater than a threshold value.

14. The system of claim 13, wherein the operations further comprise:

receiving an image of a new target logo to be added to the set of target logos;

extracting, using the neural network implemented using the one or more computing devices, a new reference feature vector from the image of the new target logo; and

storing the new reference feature vector in the database.

15. The system of claim 13, wherein the operations further comprise:

detecting, from the source image and using a second logo detection model implemented by the one or more computing devices, a second target logo in the set of target logos,

wherein the second logo detection model is trained using training images including the set of target logos to detect the set of target logos.

16. The system of claim 15, wherein:

the first logo detection model and the second logo detection model are trained using synthetic training images;

each synthetic training image includes a background image and an image of a target logo in the set of target logos; and

the image of the target logo is superimposed on the background image.

17. The system of claim 16, wherein:

the image of the target logo is superimposed in a region of the background image; and

the region of the background image is determined by;

classifying each pixel in the background image as suitable or unsuitable for logo superposition; and

grouping neighboring pixels that are classified as suitable for logo superposition to form the region of the background image for logo superposition.

18. A system comprising:

means for receiving a source image;

means for detecting, in the source image and using a first logo detection model, a candidate region for determining a logo in the source image;

means for extracting, from the candidate region and using a neural network, a feature vector of the candidate region;

means for determining, for each reference feature vector from a set of reference feature vectors stored in a database, a respective matching score between the reference feature vector and the feature vector of the candidate region, wherein each reference feature vector in the set of reference feature vectors is extracted from a respective image of a target logo in a set of target logos;

means for selecting a first reference feature vector associated with a highest matching score among the set of reference feature vectors, the first reference feature vector extracted from a first image of a first target logo in the set of target logos; and

means for determining that the candidate region includes an image of the first target logo based on determining that the highest matching score is greater than a threshold value.

19. The system of claim 18, further comprising:

means for receiving an image of a new target logo to be added to the set of target logos;

means for extracting, using the neural network, a new reference feature vector from the image of the new target logo; and

means for storing the new reference feature vector in the database.

20. The system of claim 18, further comprising:

means for detecting, from the source image and using a second logo detection model, a second target logo in the set of target logos,

wherein the second logo detection model is trained using training images including the set of target logos to detect the set of target logos.