pharmine

The previous post already provides the steps for model validation using cross-validation. So how do we validate a model using an independent validation set?

Validate model on an independent validation set

1. Put File widget (Data) to canvas and configure it to load a training set from a file.
2. Put Select Attributes widget (Data) to canvas and connect the output port from the File widget to its input port.
• Specify the attributes and class for the training set.
• Click on the Apply button.
3. Put K Nearest Neighbours widget (Classify) to canvas and connect the output port from the Select Attributes widget to its input port.
• Configure it by setting Number of neighbours to 3.
• Click on the Apply button.
4. Put File widget (Data) to canvas and configure it to load an independent validation set from a file.
5. Put Select Attributes widget (Data) to canvas and connect the output port from the second File widget to its input port.
• Specify the attributes and class for the independent validation set.
• Click on the Apply button.
6. Put Test Learners widget (Evaluate) to canvas.
• Connect the output port from K Nearest Neighbours widget to its Learner input port.
• Connect the output port from the first Select Attributes widget to its Data input port.
• Connect the output port from the second Select Attributes widget to its Separate Test Data input port.
• Configure it by choosing Test on test data.

It can be seen that Orange is able to validate a model using either cross-validation or an independent validation set. However, it seems that Orange is unable to save a model and thus the model has to be rebuild each time it is to be used for validating an independent validation set.

This entry was posted by Yap Chun Wei on Friday, May 16th, 2008 at 10:22 pm and is filed under Data mining, Review. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.