Method for boosting the performance of machine-learning classifiers

• US 7,016,881 B2
• Filed: 02/25/2005
• Issued: 03/21/2006
• Est. Priority Date: 12/08/2001
• Status: Expired due to Term

• Current Assignee: Zhigu Holdings Limited
• Original Assignee: Microsoft Corporation
• Inventors: Li, ZiQing, Zhang, ZhenQiu, Zhu, Long
• Primary Examiner(s): Hirl, Joseph P.
• Application Number:
  US11/067,284
• Publication Number:
  US 20050144149A1
• Time in Patent Office: 389 Days
• Field of Search: 706/12, 706/25, 706/47, 706/48, 382/157, 700/48
• US Class Current: 706/12
• CPC Class Codes: G06F 18/2115 by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/24765 Rule-based classification
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

A novel statistical learning procedure that can be applied to many machine-learning applications is presented. Although this boosting learning procedure is described with respect to its applicability to face detection, it can be applied to speech recognition, text classification, image retrieval, document routing, online learning and medical diagnosis classification problems.

Claims

1. A computer-readable medium having computer-executable instructions for boosting the performance of a classifier in a statistical based machine learning system, said computer executable instructions comprising:

identifying a set of weak classifiers each of which is associated with a feature found in a plurality of training examples, said weak classifiers collectively best classifying the training examples;

linearly combining each of the weak classifiers in the identified set of weak classifiers to define a strong classifier,

wherein the action of identifying the set of weak classifiers comprises using a sequential forward search for optimal weak classifiers with backtracking to ensure the inclusion of a weak classifier in the set of weak classifiers does not result in lower overall performance in the form of increased processing time.