Method and apparatus for analyzing sensor data

• US 10,488,912 B1
• Filed: 01/29/2018
• Issued: 11/26/2019
• Est. Priority Date: 01/27/2017
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Sinclair, Eoin C.
• Primary Examiner(s): Wang, Albert
• Application Number:
  US15/882,963
• Time in Patent Office: 666 Days
• CPC Class Codes: G06F 1/3234 Power saving characterised by the action undertaken
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06F 18/285 Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
• CPC Class Codes: G06F 9/4893 taking into account power or heat criteria power management in computers in general G06F1/3203; thermal management in computers in general G06F1/206
• CPC Class Codes: G06T 7/77 using statistical methods
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04N 23/65 Control of camera operation in relation to power supply
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Mobile devices, such as smartphones, are severely battery-limited. The capabilities of mobile device processors have increased at exponential rates, but battery technologies have improved at much slower rates. As a consequence, it is ever more important that mobile devices be operated in battery-preserving manners. An aspect of the present technology concerns methods and arrangements enabling on-going recognition-processing of imagery and audio by different detectors, without the high battery drain that has characterized the prior art. A variety of other features and arrangements are also detailed.

Claims

1. A method of operating multiple differently-configured signal detectors in a battery-preserving manner, comprising the acts:

receiving a set of audio and/or image content data in a battery-powered device;

for each of N differently-configured signal detectors, where N>

2, generating a probability datum indicating an estimate of whether that signal detector will achieve a positive detection result if applied to the received set of content data; and

based at least in part on said probability data, selecting M of said N differently-configured signal detectors, where 1<

M<

N, and invoking the selected signal detectors on the received set of content data;

wherein the method includes limiting processing resources employed by the invoked M detectors so that, in the aggregate, a previously-established processing budget is not exceeded.

2. The method of claim 1 that includes:

for a first set of content data, invoking M1 differently-configured signal detectors;

for a second set of content data, invoking M2 differently-configured signal detectors, where M2>

M1; and

for a first signal detector that is invoked for both the first and second sets of content data, limiting said first signal detector to a first allocation of processing resources when applied to the first set of content data, and limiting said first signal detector to a second, smaller, allocation of processing resources when applied to the second set of content data.

3. The method of claim 1 in which the received set of content data comprises a frame of image data, and said N differently-configured signal detectors include at least two detectors drawn from the list consisting of:

a digital watermark decoder, a 1D barcode decoder, a 2D barcode detector, a face recognizer, an optical character recognizer, a logo recognizer, and an image recognition engine.

4. The method of claim 1 in which the received set of content data comprises a frame of image data, and said N differently-configured signal detectors include a watermark detector configured to examine different blocks within said frame, or blocks with different scales or perspective distortions within said frame.

5. The method of claim 4 in which said N differently-configured signal detectors include a watermark detector configured to examine different blocks within said frame.

6. The method of claim 4 in which said N differently-configured signal detectors include a watermark detector configured to examine blocks at different scales within said frame.

7. The method of claim 1 in which the act of estimating the probability that a first of said signal detectors will achieve a positive detection result, is performed by a neural network.

8. The method of claim 1 in which the previously-established processing budget comprises a time interval permitted to be consumed by operation of said invoked detectors.

9. The method of claim 1 in which the selecting of the M differently-configured signal detectors is also based on history data concerning previous processing of one or more previously-received sets of audio or image content data by one or more of said N signal detectors.

10. The method of claim 9 in which the history data includes data indicating one of said N signal detectors that was previously applied to a previously-received set of content data, and whether said detector previously achieved a positive detection result.

11. The method of claim 9 in which the history data includes data indicating an elapsed interval, either of time or intervening sets of received content data, since a previous processing of one of said previously-received sets of content data by one or said N signal detectors.

12. A method of applying N different detectors, D1 . . . DN, to different frames of image content data, where N is at least three, comprising the acts:

in a round “

A”

of signal processing;

(a) processing between 1 and N of said different frames of image content data by applying, collectively, all N of the detectors thereto, wherein no detector is applied to more than one buffered frame;

(b) identifying at least two of said detectors, including D1 and D2, that succeed in extracting information in act (a); and

(c) identifying at least one of said detectors, DN, that fail in extracting information in act (a);

in a subsequent round “

B”

of signal processing;

(d) applying said at least two detectors, including D1 and D2, that succeeded in extracting information in round “

A”

to a single further buffer of content data; and

(e) applying detector, DN, which failed in extracting information in round “

A,”

to a different further buffer of content data;

wherein the foregoing acts are performed by a hardware processor.

13. The method of claim 12 in which act (c) comprises identifying plural detectors that fail in extracting information in act (a), and applying each of said plural detectors to a respectively different further frame of of content data in act (e);

wherein each of the N detectors is applied in said subsequent round “

B”

.

14. The method of claim 13 in which, in round “

B,”

at least a pair of said detectors succeeded in extracting information from the frame of content data to which they were respectively applied, and in which the method further includes;

in a round “

C”

of signal processing that follows round “

B;

”

(f) applying those detectors that succeeded in extracting information in round “

B”

to yet a different single frame of content data; and

(g) applying those detectors that failed in extracting information in round “

B”

each, respectively, to still a different frame of content data;

wherein application of plural detectors to a single frame occurs only following a plurality of detectors succeeding in extracting information in an immediately-preceding round of signal processing.

15. The method of claim 12 in which a number of frames of image data processed in round “

B”

is equal to the lesser of N, and N+1 minus a number of detectors that succeeded in extracting information in act (a).

16. A method of applying N different detectors, D1 . . . DN, to buffered frames of image content data, comprising the acts:

in a phase “

A”

of signal processing, processing a single buffered frame of image content with between 2 and N−

1 of said N detectors;

in a phase “

B”

of signal processing that immediately follows phase “

A,”

processing a different single buffered frame of image content with plural of said detectors, including all of the detectors that succeeded in extracting information in phase “

A,”

together with one detector that was not applied—

or did not succeed in extracting information, in phase “

A,”

wherein the foregoing acts are performed by a hardware processor.

17. The method of claim 16 that further includes, in a phase “

C”

of signal processing applying only a single detector, except when a detector applied to in an immediately-preceding phase of signal processing succeeded in extracting information.

18. The method of claim 16 wherein, for each group of N successive buffered frames of image content data, each of the N different detectors is applied.