1 / 22

Evaluating Classification Performance: Data Usage and Trust in Results

This document discusses essential considerations in measuring classification performance focusing on data selection, evaluation techniques, and reliability of outcomes. It highlights the significance of using appropriate training and test datasets, exploring methods like Resubstitution, Hold-out, Leave-one-out, K-fold cross-validation, and Bootstrap approaches. Each technique is examined for its advantages and disadvantages, particularly concerning the size of the dataset and trustworthiness of performance estimates.

ninon
Télécharger la présentation

Evaluating Classification Performance: Data Usage and Trust in Results

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. Measuring Classification Performance D. Pokrajac 2003

    2. Main Issues What Data to use? How to measure performance? How do we trust our measured results?

    3. What data to use? Typically we learn our model on some data set, known as training set Subsequently, we evaluate, model on dataset known as test set Main issue: given available data, how to generate training and test set

    4. Major technique to choose test set Re-substitution Hold-out Leave-one-out K-fold cross validation Bootstrap

    5. Re-substitution Use all available data for training set Test set=training set=available dataset Problems: If available data set is small our results on unseen data will be poor Estimation of actual performance can be poor

    6. Hold-Out Split available dataset into two halves Use one half for training set and the other for test set Problem: Poor use of data (half data discarded from training!)

More Related