Multi-classifier selection and monitoring for MMR-based image recognition

• US 8,073,263 B2
• Filed: 10/07/2008
• Issued: 12/06/2011
• Est. Priority Date: 07/31/2006
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Hull, Jonathan J., Erol, Berna, Moraleda, Jorge
• Primary Examiner(s): Strege, John
• Application Number:
  US12/247,202
• Publication Number:
  US 20090076996A1
• Time in Patent Office: 1,155 Days
• Field of Search: 707/705, 707/706, 707/711, 707/713, 707/725, 707/737, 707/741, 707/751, 382/224
• US Class Current: 382/224
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 18/285 Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
• CPC Class Codes: G06V 10/96 Management of image or video recognition tasks
• CPC Class Codes: G06V 30/19113 Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
• CPC Class Codes: G06V 30/19147 Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 30/1916 Validation; Performance evaluation
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

A MMR system that uses multiple classifiers for predicting, monitoring, and adjusting index tables for image recognition comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit includes a plurality of recognition unit and index table pairs corresponding to classifiers to be applied to received image queries, as well as an image registration unit for storing and monitoring performance data for the classifiers. The MMR matching unit receives the image query and identifies, using a classifier set, a result including a document, the page, and the location on the page corresponding to the image query. The present invention also includes methods for monitoring online performance of a multiple classifier image recognition system, for classifier selection and comparison, and for offline classifier prediction.

Claims

1. A computer-implemented method of classifier set prediction, comprising:

dividing, by a computer, a future time interval into a plurality of minimum subintervals; and

for a selected minimum subinterval;

retrieving, by the computer, data for one or more historic time intervals corresponding to the selected minimum subinterval; and

determining, by the computer, a best performing classifier set for the one or more historic time intervals corresponding to the selected minimum subinterval by comparing performing classifier sets for each historic time interval.

2. The method of claim 1, wherein determining the best performing classifier set for the one or more historic time intervals comprises:

for each of the one or more historic time intervals, applying a complete set of classifiers to a set of image queries received during the historic time interval; and

determining the best performing classifier set for the historic time interval.

3. The method of claim 2, wherein the complete set of classifiers is all available classifiers.

4. The method of claim 2, wherein a current classifier set corresponds to an immediately previous minimum subinterval, further comprising calculating a potential performance gain for changing from the current classifier set to the best performing classifier set.

5. The method of claim 4, wherein calculating the potential performance gain further comprises:

determining performance data from an application of the current classifier set to the set of image queries received during the historic time interval;

determining performance data from application of the best performing classifier set to the set of image queries received during the historic time interval; and

calculating a difference between the best set performance data and the current classifier set performance data.

6. The method of claim 5, further comprising, in response to the potential performance gain exceeding a switching threshold, changing from the current classifier set to the best performing classifier set for the minimum subinterval.

7. The method of claim 6, further comprising, in response to a determination that additional subintervals exist, performing the method for the additional subintervals.

8. There method of claim 7, wherein the method is performed chronologically on the additional subintervals.

9. The method of claim 1, wherein the method is performed offline.

10. The method of claim 5, further comprising, in response to the potential performance gain being equal to or less than a switching threshold, maintaining the current classifier set.

11. The method of claim 1, further comprising determining whether the selected minimal subinterval is a last minimal subinterval.

12. The method of 11, further comprising, in response to the selected minimal subinterval being the last minimal subinterval, selecting a next minimal subinterval.