Directed Ad-Hoc Inference

Monday 25 April 2005
Vladimir Vapnik, Royal Holloway, University of London

Directed Ad-Hoc Inference (DAHI) is a new machine learning technique that occupies an intermediate position between the inductive-deductive and types of inferences.

The main idea of DAHI is a reconsideration of the roles of the training and testing stages during the inference process. The classical inductive-deductive model of inference contains two different stages:

1. The training (inductive) stage, where one constructs a rule for classification using training, and
2. The testing (deductive) stage where one classifies the test data using the constructed rule.

The transductive model of inference solves the classification problem in one step:

1. Given a set of training data and a set of test data, it finds the labels for the test data directly.

DAHI works differently. During the training stage, DAHI looks for a principal direction (concept) used to construct different rules for future inferences. This is different from the inductive stage of inference where the goal is to find one fixed rule. During the test stage DAHI uses this principal direction to construct a specific rule for each given test vector (the ad-hoc rule). It constructs one specific rule for each specific example, rather than one rule for all test data. Therefore, DAHI contains elements of both inductive and transductive inference:

1. It constructs one general direction of inference (as in inductive inference).
2. It constructs an individual (ad-hoc) rule for each given test example (as in transductive inference).

From a technical point of view this method is a combination of ideas from Statistical Learning Theory (in particular, Support Vector Machines), and non-parametric statistics.