Skip to contents

Evaluate classification performance

Source: R/evaluate_performance.R
evaluate_performance.Rd

Computes diagnostic performance metrics based on classification results from cutoff functions (percentile-based or ROC-based).

Usage

evaluate_performance(cutoff_result, outcome)

Arguments

cutoff_result

Output list from derive_cutoffs_percentile() or derive_cutoffs_sensspec(), containing classification vector.

outcome

Character, name of binary disease outcome column (0/1).

Value

Data frame with performance metrics.

Details

The function uses the classification vector returned by derive_cutoffs_percentile() or derive_cutoffs_sensspec() and compares it against the true disease labels to compute:

  • NPV (Negative Predictive Value): TN / (TN + FN), proportion of true negatives among predicted negatives.

  • PPV (Positive Predictive Value): TP / (TP + FP), proportion of true positives among predicted positives.

  • Overall Sensitivity: TP / (TP + FN) across the full cohort.

  • Overall Specificity: TN / (TN + FP) across the full cohort.

  • % in Middle Zone: Proportion of subjects classified as indeterminate (those with biomarker values between c_L and c_H).

Indeterminate subjects (those with biomarker values between the rule-out cutoff c_L and the rule-in cutoff c_H) are excluded from the sensitivity and specificity calculations. They do not contribute to the counts of true positives, true negatives, false positives, or false negatives. This design is intentional to reflect clinical practice, where patients in the gray zone are not classified as definitively diseased or non-diseased. As a result, sensitivity and specificity are computed only on subjects outside the indeterminate zone, while the proportion of indeterminate cases is reported separately.

Examples

n <- 500
set.seed(123)

# outcome
DISEASE <- rbinom(n, 1, 0.35)

# biomarker depends on disease
PTAU <- rnorm(n, mean = 25 + 10 * DISEASE, sd = 8)

# confounders (optional noise)
AGE <- round(rnorm(n, 73, 7))
SEX <- sample(c(0, 1), n, replace = TRUE)
BMI <- rnorm(n, 25, 4)
HTN <- rbinom(n, 1, 0.3)
DM <- rbinom(n, 1, 0.2)

df <- data.frame(PTAU, AGE, SEX, BMI, HTN, DM, DISEASE)

res <- adjust_score(
df,
biomarker = "PTAU",
outcome = "DISEASE",
confounders = c("AGE","SEX","BMI","HTN","DM"))
#> number of iterations= 704 
#> Setting levels: control = 0, case = 1
#> Setting direction: controls < cases

cutoffs <- derive_cutoffs_percentile(res, biomarker = "PTAU")
#> Number of cases below low_thresh = 0.05 is too small, so fallback to 5th percentile of biomarker was used. Consider using another low_thresh value if your dataset is small.
# cutoffs <- derive_cutoffs_sensspec(res, biomarker = "PTAU")

perf <- evaluate_performance(cutoffs, outcome = "DISEASE")