pharmine

The previous post already provides the steps for model validation using cross-validation. So how do we validate a model using an independent validation set?

Validate model on an independent validation set

1. Put ArffLoader component (DataSources) to layout area and configure it to load a training set from a file.
2. Put ClassAssigner component (Filters) to layout area and connect the dataSet connection from the ArffLoader component to it.
• Configure it by setting the classIndex to the class column.
3. Put TrainingSetMaker component (Evaluation) to layout area and connect the dataSet connection from the ClassAssigner component to it.
4. Put ArffLoader component (DataSources) to layout area and configure it to load an independent validation set from a file.
5. Put ClassAssigner component (Filters) to layout area and connect the dataSet connection from the ArffLoader component to it.
• Configure it by setting the classIndex to the class column.
6. Put TestSetMaker component (Evaluation) to layout area and connect the dataSet connection from the ClassAssigner component to it.
7. Put SMO component (Classifiers) to layout area and connect the trainingSet connection from the TrainingSetMaker component and the testSet connection from the TestSetMaker component to it.
• Configure it by choosing RBFKernel and setting the gamma value for the kernel to 0.01.
8. Put ClassifierPerformanceEvaluator component (Evaluation) to layout and connect the batchClassifier connection from the SMO component to it.
9. Put TextViewer component (Visualization) to layout and connect the text connection from the ClassifierPerformanceEvaluator component to it.
10. Run.

It can be seen that KnowledgeFlow is able to validate a model using either cross-validation or an independent validation set. However, it seems that KnowledgeFlow is unable to save a model and thus the model has to be rebuild each time it is to be used for validating an independent validation set.

This entry was posted by Yap Chun Wei on Saturday, April 19th, 2008 at 1:48 pm and is filed under Data mining, Review. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.