4.1. Classification Metrics

Below is a detailed list of metrics commonly used to evaluate the accuracy and performance of classification and regression models in machine learning, including neural networks. The metrics are categorized based on their applicability to classification or regression tasks, with explanations of their purpose and mathematical formulations where relevant.

Classification Metrics

Classification tasks involve predicting discrete class labels. The following metrics assess the accuracy and effectiveness of such models:

1. Accuracy
2. Purpose: Measures the proportion of correct predictions across all classes.
3. Formula: \( \text{Accuracy} = \frac{\text{Number of Correct Predictions}}{\text{Total Number of Predictions}} = \frac{TP + TN}{TP + TN + FP + FN} \)
   • \( TP \): True Positives, \( TN \): True Negatives, \( FP \): False Positives, \( FN \): False Negatives.

4. Use Case: Suitable for balanced datasets but misleading for imbalanced ones.

5. Precision

6. Purpose: Evaluates the proportion of positive predictions that are actually correct.
7. Formula: \( \text{Precision} = \frac{TP}{TP + FP} \)

8. Use Case: Important when false positives are costly (e.g., spam detection).

9. Recall (Sensitivity or True Positive Rate)

10. Purpose: Measures the proportion of actual positives correctly identified.
11. Formula: \( \text{Recall} = \frac{TP}{TP + FN} \)

12. Use Case: Critical when false negatives are costly (e.g., disease detection).

13. F1-Score

14. Purpose: Harmonic mean of precision and recall, balancing both metrics.
15. Formula: \( \text{F1-Score} = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}} \)

16. Use Case: Useful for imbalanced datasets where both precision and recall matter.

17. Area Under the ROC Curve (AUC-ROC)

18. Purpose: Measures the model’s ability to distinguish between classes across all thresholds.
19. Formula: Area under the curve plotting True Positive Rate (Recall) vs. False Positive Rate (\( \frac{FP}{FP + TN} \)).

20. Use Case: Effective for binary classification and assessing model robustness.

21. Area Under the Precision-Recall Curve (AUC-PR)

22. Purpose: Focuses on precision and recall trade-off, especially for imbalanced datasets.
23. Formula: Area under the curve plotting Precision vs. Recall.

24. Use Case: Preferred when positive class is rare (e.g., fraud detection).

25. Confusion Matrix

26. Purpose: Provides a tabular summary of prediction outcomes (TP, TN, FP, FN).

27. Use Case: Offers detailed insights into class-specific performance, especially for multi-class problems.

28. Log Loss (Logarithmic Loss or Cross-Entropy Loss)

29. Purpose: Penalizes incorrect predictions based on predicted probabilities.
30. Formula: \( \text{Log Loss} = -\frac{1}{N} \sum_{i=1}^N [y_i \log(\hat{y}_i) + (1 - y_i) \log(1 - \hat{y}_i)] \)
    • \( y_i \): True label, \( \hat{y}_i \): Predicted probability.

31. Use Case: Common in probabilistic classifiers like neural networks with softmax outputs.

32. Matthews Correlation Coefficient (MCC)

33. Purpose: Balances all four confusion matrix quadrants, robust for imbalanced data.
34. Formula: \( \text{MCC} = \frac{TP \cdot TN - FP \cdot FN}{\sqrt{(TP + FP)(TP + FN)(TN + FP)(TN + FN)}} \)

35. Use Case: Preferred for a single, comprehensive metric in binary classification.

36. Cohen’s Kappa

    • Purpose: Measures agreement between predicted and true labels, adjusted for chance.
    • Formula: \( \kappa = \frac{p_o - p_e}{1 - p_e} \)
    • \( p_o \): Observed agreement, \( p_e \): Expected agreement by chance.
    • Use Case: Useful for multi-class problems or when chance agreement is a concern.