Model Evaluation Dashboard

COMPSCI 713 Week 3 — Basic Lesson 7: Model Evaluation and Performance Metrics

Lesson Map

Step 1 — B07 Part 1: Accuracy Trap Step 2 — B07 Part 2: Threshold & Predictions Step 3 — B07 Part 2: Confusion Matrix Step 4 — B07 Part 2: Classification Metrics Step 5 — B07 Part 3: ROC Curve & AUC Step 6 — B07 Part 3: Precision-Recall Curve Step 7 — B07 Part 4: Threshold Optimisation Step 8 — B07 Part 5: Cross-Validation Step 9 — B07 Part 7: Exam Reference
1
The Accuracy TrapB07 Part 1
2
Your Data & Decision ThresholdB07 Part 2
3
Confusion MatrixB07 Part 2
Correct prediction
Error (misclassification)
4
Classification MetricsB07 Part 2
What does each metric mean?
5
ROC Curve & AUCB07 Part 3
The ROC curve plots True Positive Rate (Recall) vs False Positive Rate across all possible thresholds. AUC summarises this into one number: 1.0 = perfect, 0.5 = random coin flip. The red dot shows where your current threshold sits on the curve.
6
Precision-Recall CurveB07 Part 3
7
Threshold OptimisationB07 Part 4
Different thresholds optimise different metrics. This chart sweeps all thresholds and shows how each metric changes. Use this to find the best threshold for your specific problem.
8
Cross-Validation SimulatorB07 Part 5
K-Fold Cross-Validation splits data into K parts, tests on each fold in turn, and averages the results. This gives a more reliable performance estimate. Stratified K-Fold ensures each fold has the same class distribution as the full dataset.
9
Exam Quick ReferenceB07 Part 7
Practice Problem (try it yourself)

Given this confusion matrix, calculate all metrics:

                Predicted
              Neg    Pos
Actual Neg  [ 90     10 ]
Actual Pos  [  5     95 ]
Click to reveal answer
  • TP = 95, TN = 90, FP = 10, FN = 5
  • Accuracy = (95+90)/200 = 92.5%
  • Precision = 95/(95+10) = 90.5%
  • Recall = 95/(95+5) = 95.0%
  • F1 = 2(0.905 × 0.950)/(0.905+0.950) = 92.7%
  • Specificity = 90/(90+10) = 90.0%
University of Auckland | COMPSCI 713 | Model Evaluation and Performance Metrics