Efficient calculation of a robust signature of a media unit

• US 10,748,038 B1
• Filed: 08/16/2019
• Issued: 08/18/2020
• Est. Priority Date: 03/31/2019
• Status: Active Grant

• Current Assignee: Cortica Ltd.
• Original Assignee: Cortica Ltd.
• Inventors: Raichelgauz, Igal, Odinaev, Karina
• Primary Examiner(s): Li, Ruiping
• Application Number:
  US16/542,327
• Time in Patent Office: 368 Days
• CPC Class Codes: G06F 18/211 Selection of the most significant subset of features
• CPC Class Codes: G06F 18/2155 characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/422 for representing the structure of the pattern or shape of an object therefor
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames

Abstract

Systems, and method and computer readable media that store instructions for calculating signatures, utilizing signatures and the like.

Claims

1. A method for low-power calculation of a signature, the method comprises:

receiving or generating a media unit of multiple objects;

processing the media unit by performing multiple iterations, wherein each iteration out of at least some of the multiple iterations is associated with spanning elements of the iteration, the each iteration comprises applying, by spanning elements of the iteration, dimension expansion process that are followed by a merge operation;

wherein the dimension expansion processes are filtering operations that differ from each other, and wherein spanning elements of the iteration are filters;

wherein each spanning element is associated with a unique set of values;

wherein the applying of the dimension expansion process of each iteration of the each iteration out of at least some of the multiple iterations comprises;

determining a relevancy of the spanning elements of the iteration;

wherein the determining of a relevancy of a spanning element of the iteration comprises comparing (a) a unique set of values associated with the spanning element of the iteration to (b) merge results of a last iteration that preceded the iteration;

completing the dimension expansion process of the iteration by relevant spanning elements of the iteration and reducing a power consumption of irrelevant spanning elements until, at least, a completion of the applying of the dimension expansion process of the iteration;

determining identifiers that are associated with significant portions of an output of the multiple iterations;

wherein significant portions of the output have a higher intensity response than other portions of the output; and

providing a signature that comprises the identifiers and represents the multiple objects.

2. The method according to claim 1 wherein the identifiers are retrieval information for retrieving the significant portions.

3. The method according to claim 1 wherein the at least some of the multiple iterations are a majority of the multiple iterations.

4. The method according to claim 1 wherein the output of the multiple iteration comprises multiple property attributes for each segment out of multiple segments of the media unit;

and wherein the significant portions of an output of the multiple iterations comprises more impactful property attributes.

5. The method according to claim 1 wherein the merge operation of the each iteration out of the at least some of the multiple iterations comprises searching for regions of interest in an outcome of a dimension expansion process of the iteration.

6. The method according to claim 5 wherein the at least some of the multiple iteration exclude at least a first iteration of the multiple iterations.

7. The method according to claim 5 wherein the merge operation comprises searching for overlaps between regions of interest of the outcome of the dimension expansion process of the iteration, and defining regions of interest that are related to the overlaps.

8. The method according to claim 5 wherein the merge operation comprises searching for proximate regions of interest of the outcome of the dimension expansion process of the iteration, and defining regions of interest that are related to a proximity between regions of interest that are tested for proximity.

9. The method according to claim 1 wherein the determining the relevancy of the spanning elements of the iteration is based on properties of the media unit.

10. The method according to claim 1 comprising performing different merge operations during different iterations.

11. The method according to claim 1 comprising a neural network processing operation that is executed by one or more layers of a neural network and does not belong to the at least some of the multiple iterations.

12. The method according to claim 11, wherein the at least one iteration is executed without reducing power consumption of irrelevant neurons of the one or more layers.

13. The method according to claim 11 wherein the one or more layers output information about properties of the media unit, wherein the information differs from a recognition of the multiple objects.

14. A non-transitory computer readable medium for low-power calculation of a signature, the non-transitory computer readable medium stores instructions for:

receiving or generating a media unit of multiple objects;

processing the media unit by performing multiple iterations, wherein each iteration out of at least some of the multiple iterations is associated with spanning elements of the iteration, the each iteration comprises applying, by spanning elements of the iteration, dimension expansion process that are followed by a merge operation;

wherein the dimension expansion processes are filtering operations that differ from each other, and wherein spanning elements of the iteration are filters;

wherein each spanning element is associated with a unique set of values;

wherein the applying of the dimension expansion process of each iteration of the each iteration out of at least some of the multiple iterations comprises;

determining a relevancy of the spanning elements of the iteration;

wherein the determining of a relevancy of a spanning element of the iteration comprises comparing (a) a unique set of values associated with the spanning element of the iteration to (b) merge results of a last iteration that preceded the iteration;

completing the dimension expansion process of the iteration by relevant spanning elements of the iteration and reducing a power consumption of irrelevant spanning elements until, at least, a completion of the applying of the dimension expansion process of the iteration;

determining identifiers that are associated with significant portions of an output of the multiple iterations;

wherein significant portions of the output have a higher intensity response than other portions of the output; and

providing a signature that comprises the identifiers and represents the multiple objects.

15. The non-transitory computer readable medium according to claim 14 wherein the identifiers are retrieval information for retrieving the significant portions.

16. The non-transitory computer readable medium according to claim 14 wherein the at least some of the multiple iterations are a majority of the multiple iterations.

17. The non-transitory computer readable medium according to claim 14 wherein the output of the multiple iteration comprises multiple property attributes for each segment out of multiple segments of the media unit;

and wherein the significant portions of an output of the multiple iterations comprises more impactful property attributes.

18. The non-transitory computer readable medium according to claim 14 wherein the merge operation of the each iteration out of the at least some of the multiple iterations comprises searching for regions of interest in an outcome of a dimension expansion process of the iteration.

19. The non-transitory computer readable medium according to claim 18 wherein the at least some of the multiple iteration exclude at least a first iteration of the multiple iterations.

20. The non-transitory computer readable medium according to claim 18 wherein the merge operation comprises searching for overlaps between regions of interest of the outcome of the dimension expansion process of the iteration, and defining regions of interest that are related to the overlaps.

21. The non-transitory computer readable medium according to claim 18 wherein the merge operation comprises searching for proximate regions of interest of the outcome of the dimension expansion process of the iteration, and defining regions of interest that are related to a proximity between regions of interest that are tested for proximity.

22. The non-transitory computer readable medium according to claim 14 wherein the determining the relevancy of the spanning elements of the iteration is based on properties of the media unit.

24. The non-transitory computer readable medium according to claim 14 that stores instructions for performing a neural network processing operation, by one or more layers of a neural network, wherein the neural network processing operation and does not belong to the at least some of the multiple iterations.

25. The non-transitory computer readable medium according to claim 24, wherein the at least one iteration is executed without reducing power consumption of irrelevant neurons of the one or more layers.

26. The non-transitory computer readable medium according to claim 24 wherein the one or more layers output information about properties of the media unit, wherein the information differs from a recognition of the multiple objects.

27. The non-transitory computer readable medium according to claim 14 wherein the applying, by spanning elements of an iteration that differs from the first iteration, the dimension expansion process comprises assigning output values that are indicative of an identity of the relevant spanning elements of the iteration.

28. The non-transitory computer readable medium according to claim 14 wherein the applying, by spanning elements of an iteration that differs from the first iteration, the dimension expansion process comprises assigning output values that are indicative a history of dimension expansion processes until the iteration that differs from the first iteration.

29. The non-transitory computer readable medium according to claim 14 wherein each spanning element is associated with a subset of reference identifiers;

and wherein the determining of the relevancy of each spanning elements of the iteration is based a relationship between the subset of the reference identifiers of the spanning element and an output of a last merge operation before the iteration.

23. The non-transitory computer readable medium according to claim comprising performing different merge operations during different iterations.

30. A signature generator that comprises:

an input that is configured to receive or generate a media unit of multiple objects;

a processor that is configured to process the media unit by performing multiple iterations, wherein each iteration out of at least some of the multiple iterations is associated with spanning elements of the iteration, wherein at least some of the multiple iterations comprises applying, by spanning elements of the iteration, dimension expansion process that are followed by a merge operation;

wherein the dimension expansion processes are filtering operations that differ from each other, and wherein spanning elements of the iteration are filters;

wherein each spanning element is associated with a unique set of values;

wherein the applying of the dimension expansion process of each iteration of the each iteration out of at least some of the multiple iterations comprises;

determining a relevancy of the spanning elements of the iteration;

wherein the determining of a relevancy of a spanning element of the iteration comprises comparing (a) a unique set of values associated with the spanning element of the iteration to (b) merge results of a last iteration that preceded the iteration;

completing the dimension expansion process of the iteration by relevant spanning elements of the iteration and reducing a power consumption of irrelevant spanning elements until, at least, a completion of the applying of the dimension expansion process of the iteration;

determining identifiers that are associated with significant portions of an output of the multiple iterations;

wherein significant portions of the output have a higher intensity response than other portions of the output;

and

an identifier unit that is configured to determine identifiers that are associated with significant portions of an output of the multiple iterations; and

an output that is configured to provide a signature that comprises the identifiers and represents the multiple objects.