Stepwise Covariate Modelling with runSCM()

Introduction

Stepwise Covariate Modelling (SCM) is the standard automated covariate selection procedure in population PK/PD analysis. It uses a likelihood-ratio test to evaluate whether adding a covariate-parameter relationship significantly improves model fit, and proceeds in two phases:

  1. Forward inclusion: starting from the base model, the candidate relationship that produces the largest statistically significant ΔOFV (OFV decrease) is added at each step. The process repeats until no remaining candidate meets the threshold.
  2. Backward elimination: starting from the forward-final model, each included relationship is tested for removal. A relationship is dropped if its removal does not produce a statistically significant ΔOFV increase (at a stricter threshold than forward).

runSCM() implements both phases for nlmixr2 fits. It automatically generates the covariate expressions in the model body — centering continuous covariates at their observed median, creating indicator columns for categorical covariates — so no data pre-processing is required.

Output files

When saveModels = TRUE (the default), runSCM() writes the following files to the output directory (auto-named <fitName>_scm_<N> in the working directory).

Two CSV summaries and one log file are written at the end of the search:

File Contents
scm_step_summary.csv Best candidate per step (one row per step)
scm_all_candidates.csv Every candidate model tested, across all steps and both phases
scm_log.txt Human-readable run log with header metadata

Three .rds files are written per accepted step (not per candidate). The <key> segment is <covar>_<var> for the covariate that was added (forward) or removed (backward) at that step:

File Contents
forward_step_<N>_fit_<key>.rds Fit object for the model accepted at forward step N
forward_step_<N>_table_<key>.rds One-row data frame for the accepted best candidate
forward_step_<N>_completetable_<key>.rds All candidates tested across forward steps 1..N (cumulative)
backward_step_<N>_fit_<key>.rds Fit object after the covariate is removed at backward step N
backward_step_<N>_table_<key>.rds One-row data frame for the removed covariate
backward_step_<N>_completetable_<key>.rds All candidates tested across backward steps 1..N (cumulative)

Only the accepted (best-at-step) candidate is persisted to .rds; per-candidate fit results are not written individually but are available in scm_all_candidates.csv and in the cumulative _completetable_ snapshots.

The _table_ and _completetable_ .rds files duplicate information in the CSVs but at full numeric precision and as per-step checkpoints (useful for resume/inspection mid-run). Routine post-hoc analysis can rely on the CSVs alone.

Set saveModels = FALSE to run without writing any files, for example when exploring the search space in a scratch session.

Tips for a robust SCM

Sample size: the SCM uses a chi-squared approximation with 1 degree of freedom. The rule of thumb is at least 50–100 subjects for a reliable test; sparse data leads to inflated type I error in forward inclusion.

p-value thresholds: the conventional PsN defaults are fwd = 0.05 and bck = 0.01. In exploratory analyses, a more liberal forward threshold (e.g. 0.10) avoids missing important relationships at the cost of higher false-positive rates.

Shape choice: the power shape (log(cov/median)) is appropriate for weight and other allometric predictors. Use shapes = c("power", "lin") to test both and let the data decide.

Warm-starting: when a covariate is accepted, the estimated theta from that step is automatically used as the starting estimate for all subsequent steps that involve the same covariate shape, which speeds convergence and improves numerical stability.

Stalled candidates: with ODE models, solver noise can flatten the outer objective enough that a candidate’s fit never leaves its zero-effect initial estimate. By default (profileInitOnStall = TRUE), a candidate whose OFV improvement over its parent is <= stallTol is automatically rescued with a one-shot frozen 1-D profile (Brent method) that supplies a gradient-informative starting value, then refit. Set profileInit = TRUE to warm-start every forward candidate this way rather than only stalled ones.

Unrealistic OFVs: runSCM() retries a candidate (up to maxRetries, default 3) when its fit produces an implausible OFV — using a perturbed or near-zero covariate init on alternate attempts — before falling back to the best attempt seen (or, with retryFailOnExhaustion = TRUE, excluding the candidate as failed). Stochastic estimators (SAEM) get a wider tolerance automatically to avoid spurious retries from Monte Carlo noise. A p-value that underflows to zero also counts as implausible by default, which means genuinely strong effects (an OFV drop above about 70 for one parameter) are retried too; set retryOnUnderflow = FALSE to skip those extra fits.

Categorical covariates: pass catvarsVec rather than pre-creating dummy columns. runSCM() handles level detection, reference-level selection, and indicator column creation automatically.

References

Jonsson, E.N. & Karlsson, M.O. (1998). Automated covariate model building within NONMEM. Pharmaceutical Research, 15(9), 1463–1468.

Lindbom, L., Ribbing, J., & Jonsson, E.N. (2004). Perl-speaks-NONMEM (PsN) — a Perl module for NONMEM related programming. Computer Methods and Programs in Biomedicine, 75, 85–94.

Ribbing, J. & Jonsson, E.N. (2004). Power, selection bias and predictive performance of the population pharmacokinetic covariate model. Journal of Pharmacokinetics and Pharmacodynamics, 31(2), 109–134.