Signature generation and object detection that refer to rare scenes

• US 10,776,669 B1
• Filed: 10/31/2019
• Issued: 09/15/2020
• Est. Priority Date: 03/31/2019
• Status: Active Grant

• Current Assignee: Cortica Ltd.
• Original Assignee: Cortica Ltd.
• Inventors: Raichelgauz, Igal, Harel, Adam
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US16/669,933
• Publication Number:
  US 20200311484A1
• Time in Patent Office: 320 Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/231 Hierarchical techniques, i.e. dividing or merging pattern sets so as to obtain a dendrogram
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/513 Sparse representations
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions
• CPC Class Codes: G06V 10/82 using neural networks

Abstract

Systems, and method and computer readable media that store instructions for calculating signatures, utilizing signatures and the like.

Claims

1. A method for updating an object detector, the method comprises:

receiving an image of a rare scene and a request to update the object detector to detect a rare object that appears in the rare scene;

calculating, by a signature generator, a signature of the image of the rare scene;

wherein the signature of the rare scene includes a certain number of object identifiers;

calculating, by the signature generator, reference signatures of a large number of reference images;

searching for reference signatures that match the signature of the rare scene to provide matching reference signatures;

generating a cluster that comprises the signature of the rare scene and the matching reference signatures;

generating one or more cluster identifiers, each cluster identifier comprises a set of object identifiers that are shared by at least a predefined number of signatures of the cluster;

wherein a number of object identifiers per set is smaller than the certain number; and

associating the one or more cluster identifiers with the object detector.

2. The method according to claim 1 wherein the object identifiers are retrieval information for retrieving the significant portions.

3. The method according to claim 1 wherein each cluster identifier comprises do not care object identifiers and object identifiers of the set of object identifiers that are shared by at least the predefined number of signatures of the cluster.

4. The method according to claim 1 comprising evaluating object detection capabilities of the concept.

5. The method according to claim 1 comprising:

receiving an indication that a first image that has a first signature was falsely detected, by the object detector, as including the rare object;

wherein the false detection occurred after the associating of the one or more cluster identifiers with the object detector;

searching for first reference signatures that match the first signature to provide first reference signatures;

generating a first false positive cluster that comprises the first signature and the first reference signatures;

generating one or more false positive cluster identifiers, each false positive cluster identifier comprises a set of object identifiers that are shared by at least a predefined number of signatures of the false positive cluster;

removing from the cluster at least one part of at least one signature based on the one or more false positive cluster identifiers.

6. The method according to claim 5 wherein the removing comprises removing at least a part of a signature of the compressed concept that comprises at least one set of object identifiers that are shared by the at least predefined number of signatures of the false positive cluster.

7. The method according to claim 5 wherein the removing comprises removing at least a part of a signature of the compressed concept that comprises at least one subset out of a set of object identifiers that are shared by the at least predefined number of signatures of the false positive cluster.

8. The method according to claim 5 comprising validating object detection capabilities of the cluster.

9. The method according to claim 1 comprising compressing the cluster to provide a compressed cluster by replacing the signature of the cluster by the one or more cluster identifiers;

and wherein the associating of the one or more cluster identifiers with the object detector comprises associating the compressed cluster with the object detector.

10. The method according to claim 9 comprising:

receiving an indication that a first image that has a first signature was falsely detected, by the object detector, as including the rare object;

wherein the false detection occurred after the associating of the one or more cluster identifiers with the object detector;

searching for first reference signatures that match the first signature to provide first reference signatures;

generating a first false positive cluster that comprises the first signature and the first reference signatures;

generating one or more false positive cluster identifiers, each false positive cluster identifier comprises a set of object identifiers that are shared by at least a predefined number of signatures of the false positive cluster; and

removing from the compressed cluster at least a subset of the one or more sets of object identifiers that are shared by at least the predefined number of signatures of the false positive cluster.

11. The method according to claim 1 comprising:

receiving a miss-detection indication that indicates that an image that comprises the rate object and has a second signature was not detected, by the object detector;

wherein the miss-detection occurred after the associating of the one or more cluster identifiers with the object detector;

searching for second reference signatures that match the second signature to provide second reference signatures;

generating a second false negative cluster that comprises the second signature and the second reference signatures;

generating one or more false negative cluster identifiers, each false negative cluster identifier comprises a set of object identifiers that are shared by at least a predefined number of signatures of the false positive cluster;

adding to the cluster at least one false negative cluster identifier.

12. The method according to claim 11 comprising validating object detection capabilities of the cluster.

13. The method according to claim 1 comprising:

compressing the cluster to provide a compressed cluster by replacing the signature of the cluster by the one or more cluster identifiers;

wherein the associating of the one or more cluster identifiers with the object detector comprises associating the compressed cluster with the object detector;

receiving a miss-detection indication that indicates that an image that comprises the rate object and has a second signature was not detected, by the object detector;

wherein the miss-detection occurred after the associating of the one or more cluster identifiers with the object detector;

searching for second reference signatures that match the second signature to provide second reference signatures;

generating a second false negative cluster that comprises the second signature and the second reference signatures;

generating one or more false negative cluster identifiers, each false negative cluster identifier comprises a set of object identifiers that are shared by at least a predefined number of signatures of the false positive cluster;

adding to the compressed cluster at least one false negative cluster identifier.

14. The method according to claim 13 comprising validating object detection capabilities of the compressed cluster.

15. The method according to claim 1 further comprising

receiving an input image;

calculating by a signature generator an input image signature;

searching for at least one matching concept structure; and

determining that the input image comprises an object identified by a matching concept—

when a matching concept is found.

16. A non-transitory computer readable medium for updating an object detector, the non-transitory computer readable medium stores instructions for:

receiving an image of a rare scene and a request to update the object detector to detect a rare object that appears in the rare scene;

calculating, by a signature generator, a signature of the image of the rare scene;

wherein the signature of the rare scene includes a certain number of object identifiers;

calculating, by the signature generator, reference signatures of a large number of reference images;

searching for reference signatures that match the signature of the rare scene to provide matching reference signatures;

generating a cluster that comprises the signature of the rare scene and the matching reference signatures;

generating one or more cluster identifiers, each cluster identifier comprises a set of object identifiers that are shared by at least a predefined number of signatures of the cluster;

wherein a number of object identifiers per set is smaller than the certain number;

and

associating the one or more cluster identifiers with the object detector.

17. The non-transitory computer readable medium according to claim 16 wherein the object identifiers are retrieval information for retrieving the significant portions.

18. The non-transitory computer readable medium according to claim 16 wherein each cluster identifier comprises do not care object identifiers and object identifiers of the set of object identifiers that are shared by at least the predefined number of signatures of the cluster.

19. The non-transitory computer readable medium according to claim 16 that stores instructions for evaluating object detection capabilities of the concept.

20. The non-transitory computer readable medium according to claim 16 that stores instructions for:

receiving an indication that a first image that has a first signature was falsely detected, by the object detector, as including the rare object;

wherein the false detection occurred after the associating of the one or more cluster identifiers with the object detector;

searching for first reference signatures that match the first signature to provide first reference signatures;

generating a first false positive cluster that comprises the first signature and the first reference signatures;

generating one or more false positive cluster identifiers, each false positive cluster identifier comprises a set of object identifiers that are shared by at least a predefined number of signatures of the false positive cluster;

removing from the cluster at least one part of at least one signature based on the one or more false positive cluster identifiers.

21. The non-transitory computer readable medium according to claim 20 wherein the removing comprises removing at least a part of a signature of the compressed concept that comprises at least one set of object identifiers that are shared by the at least predefined number of signatures of the false positive cluster.

22. The non-transitory computer readable medium according to claim 20 wherein the removing comprises removing at least a part of a signature of the compressed concept that comprises at least one subset out of a set of object identifiers that are shared by the at least predefined number of signatures of the false positive cluster.

23. The non-transitory computer readable medium according to claim 20 that stores instructions for validating object detection capabilities of the cluster.

24. The non-transitory computer readable medium according to claim 16 that stores instructions for compressing the cluster to provide a compressed cluster by replacing the signature of the cluster by the one or more cluster identifiers;

and

wherein the associating of the one or more cluster identifiers with the object detector comprises associating the compressed cluster with the object detector.

25. The non-transitory computer readable medium according to claim 24 that stores instructions for:

receiving an indication that a first image that has a first signature was falsely detected, by the object detector, as including the rare object;

wherein the false detection occurred after the associating of the one or more cluster identifiers with the object detector;

searching for first reference signatures that match the first signature to provide first reference signatures;

generating a first false positive cluster that comprises the first signature and the first reference signatures;

generating one or more false positive cluster identifiers, each false positive cluster identifier comprises a set of object identifiers that are shared by at least a predefined number of signatures of the false positive cluster; and

removing from the compressed cluster at least a subset of the one or more sets of object identifiers that are shared by at least the predefined number of signatures of the false positive cluster.

26. The non-transitory computer readable medium according to claim 16 that stores instructions for:

receiving a miss-detection indication that indicates that an image that comprises the rate object and has a second signature was not detected, by the object detector;

wherein the miss-detection occurred after the associating of the one or more cluster identifiers with the object detector;

searching for second reference signatures that match the second signature to provide second reference signatures;

generating a second false negative cluster that comprises the second signature and the second reference signatures;

generating one or more false negative cluster identifiers, each false negative cluster identifier comprises a set of object identifiers that are shared by at least a predefined number of signatures of the false positive cluster;

adding to the cluster at least one false negative cluster identifier.

27. The non-transitory computer readable medium according to claim 26 that stores instructions for validating object detection capabilities of the cluster.

28. The non-transitory computer readable medium according to claim 16 that stores instructions for:

compressing the cluster to provide a compressed cluster by replacing the signature of the cluster by the one or more cluster identifiers;

wherein the associating of the one or more cluster identifiers with the object detector comprises associating the compressed cluster with the object detector;

receiving a miss-detection indication that indicates that an image that comprises the rate object and has a second signature was not detected, by the object detector;

wherein the miss-detection occurred after the associating of the one or more cluster identifiers with the object detector;

searching for second reference signatures that match the second signature to provide second reference signatures;

generating a second false negative cluster that comprises the second signature and the second reference signatures;

generating one or more false negative cluster identifiers, each false negative cluster identifier comprises a set of object identifiers that are shared by at least a predefined number of signatures of the false positive cluster;

adding to the compressed cluster at least one false negative cluster identifier.

29. A system that comprises a communication unit and a processor, wherein the communication unit is configured to receive an image of a rare scene and a request to update the object detector to detect a rare object that appears in the rare scene;

calculate, by a signature generator of the processor, a signature of the image of the rare scene;

wherein the signature of the rare scene includes a certain number of object identifiers;

calculate, by the signature generator, reference signatures of a large number of reference images;

search for reference signatures that match the signature of the rare scene to provide matching reference signatures;

generate a cluster that comprises the signature of the rare scene and the matching reference signatures;

generate one or more cluster identifiers, each cluster identifier comprises a set of object identifiers that are shared by at least a predefined number of signatures of the cluster;

wherein a number of object identifiers per set is smaller than the certain number; and

associate the one or more cluster identifiers with the object detector.

30. The system according to claim 29 comprising a sensing unit that is configured to obtain an input image;

wherein the signature generator is configured to calculate an input image signature;

wherein the processor is configured to search for at least one matching concept structure; and

determine that the input image comprises an object identified by a matching concept—

when a matching concept is found.