METHOD OF VEHICLE IMAGE COMPARISON AND SYSTEM THEREOF

• US 20190304099A1
• Filed: 03/29/2018
• Published: 10/03/2019
• Est. Priority Date: 03/29/2018
• Status: Active Grant

• Current Assignee: UVeye Ltd.
• Original Assignee: UVeye Ltd.
• Inventors: HEVER, Amir, HEVER, Ohad, BOGOMOLNY, Ilya
• Granted Patent: US 10,643,332 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24143 Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30136 Metal
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06V 10/26 Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 2201/06 Recognition of objects for industrial automation
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

There are provided a system and method of vehicle image comparison, the method including: obtaining an input image comprising a plurality of image portions; retrieving a set of reference images; for each image portion, searching for a best matching reference portion in the set of reference images, comprising: i) for each given reference image: identifying a reference region; using a similarity model on the given image portion and the reference region to obtain a similarity map indicating a similarity between the image portion and a respective reference image portion; and selecting a reference image portion with the best similarity as a reference portion candidate; and ii) selecting the best matching reference portion; and comparing each given image portion with the best matching reference portion using a comparison model, giving rise to a difference map indicating probability of presence of DOI in the given image portion.

Claims

1. A computerized method of vehicle image comparison, comprising:

obtaining an input image acquired by an imaging device capturing at least part of a vehicle, the input image comprising a plurality of image portions;

retrieving a set of reference images;

for each given image portion of the plurality of image portions, searching for a best matching reference portion in the set of reference images, comprising;

i) obtaining a set of reference portion candidates corresponding to the set of reference images, comprising, for each given reference image within the set;

identifying a reference region in the given reference image according to location of the given image portion within the input image, the reference region having a size that is not smaller than the given image portion;

using a similarity model on the given image portion and the reference region to obtain a similarity map comprising one or more sections corresponding to one or more reference image portions with one or more respective locations within the reference region, each section in the similarity map indicating a similarity between the given image portion and a respective reference image portion; and

selecting, from the one or more reference image portions, a reference image portion with the best similarity as a reference portion candidate for the given reference image; and

ii) selecting the best matching reference portion from the set of reference portion candidates; and

comparing each given image portion with the best matching reference portion using a comparison model, giving rise to a difference map corresponding to the given image portion, the difference map indicating probability of presence of difference of interest (DOI) in the given image portion, thereby obtaining a plurality of difference maps corresponding to the plurality of image portions of the input image.

2. The computerized method of claim 1, further comprising combining the plurality of difference maps to a composite difference map indicating probability of presence of difference of interest (DOI) in the input image.

3. The computerized method of claim 1, wherein the plurality of image portions are obtained using a sliding window.

4. The computerized method of claim 1, wherein the plurality of image portions are obtained by segmenting the input image into one or more input segments corresponding to one or more mechanical components comprised in the at least part of the vehicle using a segmentation model.

5. The computerized method of claim 1, wherein the input image is acquired by the imaging device at a first imaging condition, and wherein at least one reference image within the set of reference images is acquired at a second imaging condition different from the first imaging condition, thereby causing false alarm (FA) difference between the input image and the at least one reference image resulting from difference between the first and second imaging conditions.

6. The computerized method of claim 5, wherein the first and second imaging conditions comprise one or more of the following:

relative position between the vehicle and the imaging device, illumination condition, and speed of the vehicle.

7. The computerized method of claim 1, wherein the set of reference images are retrieved using an instance descriptor uniquely identifying the vehicle in the input image.

8. The computerized method of claim 7, wherein the set of reference images are selected from a group comprising:

a first set of images capturing the same instance of the vehicle, and a second set of images capturing similar instances of the vehicle.

9. The computerized method of claim 8, wherein the second set of images are selected by:

using an encoding model to obtain vector representation for each of the input image and candidate reference images, and selecting the second set of images using a similarity metric between the vector representation of the input image and each candidate reference image.

10. The computerized method of claim 1, wherein the comparison model is a comparison deep learning model trained using a training dataset comprising a set of image pairs each including a target image and a reference image so as to be capable of identifying the DOI and excluding the FA difference in a difference map between each image pair.

11. The computerized method of claim 1, wherein the similarity model is a similarity deep learning model trained using training data comprising a plurality of sets of image portions extracted from registered images, and the similarity model is trained so as to reflect similarity within each pair of image portions within each set of the plurality of sets of image portions.

12. The computerized method of claim 1, wherein the segmentation model is a segmentation deep learning model trained using a training dataset comprising a set of pre-segmented vehicle images according to mechanical components comprised therein.

13. The computerized method of claim 1, wherein the difference map corresponding to the given image portion indicates portion-wise probability of presence of DOI in the given image portion.

14. The computerized method of claim 1, wherein the difference map corresponding to the given image portion indicates pixel-wise probability of presence of DOI in the given image portion.

15. The computerized method of claim 2, wherein at least some of pixels in the composite difference map result from comparisons with different reference images.

16. A computerized system of vehicle image comparison, the system comprising a processor and memory circuitry (PMC) operatively connected to an I/O interface, wherein:

the I/O interface is configured to;

obtain an input image acquired by an imaging device capturing at least part of a vehicle, the input image comprising a plurality of image portions; and

retrieve a set of reference images; and

the PMC is configured to;

for each given image portion of the plurality of image portions, search for a best matching reference portion in the set of reference images, comprising;

i) obtaining a set of reference portion candidates corresponding to the set of reference images, comprising, for each given reference image within the set;

identifying a reference region in the given reference image according to location of the given image portion within the input image, the reference region having a size that is not smaller than the given image portion;

using a similarity model on the given image portion and the reference region to obtain a similarity map comprising one or more sections corresponding to one or more reference image portions with one or more respective locations within the reference region, each section in the similarity map indicating a similarity between the given image portion and a respective reference image portion; and

selecting, from the one or more reference image portions, a reference image portion with the best similarity as a reference portion candidate for the given reference image; and

ii) selecting the best matching reference portion from the set of reference portion candidates; and

compare each given image portion with the best matching reference portion using a comparison model, giving rise to a difference map corresponding to the given image portion, the difference map indicating probability of presence of difference of interest (DOI) in the given image portion, thereby obtaining a plurality of difference maps corresponding to the plurality of image portions of the input image.

17. The computerized system of claim 16, wherein the PMC is further configured to combine the plurality of difference maps to a composite difference map indicating probability of presence of difference of interest (DOI) in the input image.

18. The computerized system of claim 16, wherein the plurality of image portions are obtained using a sliding window.

19. The computerized system of claim 16, wherein the plurality of image portions are obtained by segmenting the input image into one or more input segments corresponding to one or more mechanical components comprised in the at least part of the vehicle using a segmentation model.

20. The computerized system of claim 16, wherein the input image is acquired by the imaging device at a first imaging condition, and wherein at least one reference image within the set of reference images is acquired at a second imaging condition different from the first imaging condition, thereby causing false alarm (FA) difference between the input image and the at least one reference image resulting from difference between the first and second imaging conditions.

21. The computerized system of claim 20, wherein the first and second imaging conditions comprise one or more of the following:

relative position between the vehicle and the imaging device, illumination condition, and speed of the vehicle.

22. The computerized system of claim 16, wherein the set of reference images are retrieved using an instance descriptor uniquely identifying the vehicle in the input image.

23. The computerized system of claim 22, wherein the set of reference images are selected from a group comprising:

a first set of images capturing the same instance of the vehicle, and a second set of images capturing similar instances of the vehicle.

24. The computerized system of claim 23, wherein the second set of images are selected by:

using an encoding model to obtain vector representation for each of the input image and candidate reference images, and selecting the second set of images using a similarity metric between the vector representation of the input image and each candidate reference image.

25. The computerized system of claim 16, wherein the comparison model is a comparison deep learning model trained using a training dataset comprising a set of image pairs each including a target image and a reference image so as to be capable of identifying the DOI and excluding the FA difference in a difference map between each image pair.

26. The computerized system of claim 16, wherein the similarity model is a similarity deep learning model trained using a training dataset comprising a plurality of sets of image portions extracted from registered images, and the similarity model is trained so as to reflect similarity within each pair of image portions within each set of the plurality of sets of image portions.

27. The computerized system of claim 16, wherein the segmentation model is a segmentation deep learning model trained using a training dataset comprising a set of pre-segmented vehicle images according to mechanical components comprised therein.

28. The computerized system of claim 16, wherein the difference map corresponding to the given image portion indicates portion-wise probability of presence of DOI in the given image portion.

29. The computerized system of claim 16, wherein the difference map corresponding to the given image portion indicates pixel-wise probability of presence of DOI in the given image portion.

30. The computerized system of claim 17, wherein at least some of pixels in the composite difference map result from comparisons with different reference images.

31. A non-transitory computer readable medium comprising instructions that, when executed by a computer, cause the computer to perform a method of vehicle image comparison, the method comprising:

obtaining an input image acquired by an imaging device capturing at least part of a vehicle, the input image comprising a plurality of image portions;

retrieving a set of reference images;

for each given image portion of the plurality of image portions, searching for a best matching reference portion in the set of reference images, comprising;

i) obtaining a set of reference portion candidates corresponding to the set of reference images, comprising, for each given reference image within the set;

identifying a reference region in the given reference image according to location of the given image portion within the input image, the reference region having a size that is not smaller than the given image portion;

using a similarity model on the given image portion and the reference region to obtain a similarity map comprising one or more sections corresponding to one or more reference image portions with one or more respective locations within the reference region, each section in the similarity map indicating a similarity between the given image portion and a respective reference image portion; and

selecting, from the one or more reference image portions, a reference image portion with the best similarity as a reference portion candidate for the given reference image; and

ii) selecting the best matching reference portion from the set of reference portion candidates; and

comparing each given image portion with the best matching reference portion using a comparison model, giving rise to a difference map corresponding to the given image portion, the difference map indicating probability of presence of difference of interest (DOI) in the given image portion, thereby obtaining a plurality of difference maps corresponding to the plurality of image portions of the input image.