Advanced weasel: Selection Constraints and Diagnostics

Custom scenarios

weasel_plan() accepts a custom scenario table. It must contain the columns scenario, require_endpoints, max_missing, n_gap_max, and max_gap_len; the table is validated and a missing column raises an immediate error rather than silently selecting nobody.

my_scenarios <- data.frame(
  scenario          = c("complete_case", "one_gap_ok", "very_lenient"),
  require_endpoints = c(TRUE, TRUE, FALSE),
  max_missing       = c(0, 2, 4),
  n_gap_max         = c(0, 1, 4),
  max_gap_len       = c(0, 2, 4)
)

plan_custom <- weasel_plan(
  long_data,
  id = "id",
  wave = "time",
  span = "core",
  core_len = 8,
  scenarios = my_scenarios
)
#> rows not used for participation: 0 with missing id, 0 with missing time, 1368 outside span 1:8.
#> ✔ plan ready: span 1:8 (core, L = 8)

weasel_print_table(
  weasel_compare_scenarios(plan_custom),
  title = "Custom scenarios", digits = 3
)
#> ── Custom scenarios ────────────────────────────────────────────────────────────
#>       scenario require_endpoints max_missing n_gap_max max_gap_len n_ids
#>  complete_case              TRUE           0         0           0   228
#>     one_gap_ok              TRUE           2         1           2   334
#>   very_lenient             FALSE           4         4           4   391
#>  mean_present mean_prop_present endpoint_rate mean_n_gap mean_max_gap
#>         8.000             1.000         1.000      0.000        0.000
#>         7.671             0.959         1.000      0.317        0.329
#>         7.453             0.932         0.882      0.348        0.358
#>  worst_missing lower upper L span_reason note score recommended near_tie
#>              0     1     8 8        core      3.666        TRUE    FALSE
#>              2     1     8 8        core      3.619       FALSE    FALSE
#>              4     1     8 8        core      3.387       FALSE    FALSE
#> (list column(s) not shown: ids)

Scoring and weights

The composite score is built only from observed subset quality. With default weights it is:

score = 2.0 * mean_prop_present      (coverage)
      + 1.2 * endpoint_rate          (endpoints)
      + 0.8 * n_ids / max(n_ids)     (relative sample size)
      - 0.6 * worst_missing / L      (worst retained respondent)
      - 0.4 * (mean_n_gap + mean_max_gap) / L   (gap structure)

Scenarios that retain no respondents receive NA and are never recommended; exact ties go to the larger sample, then to input-table order, and tie_tolerance flags near-ties without changing the recommendation. You can reweight any term:

size_first <- weasel_compare_scenarios(
  plan_custom,
  weights = c(size = 3, coverage = 1)
)
size_first$scenario[size_first$recommended]
#> [1] "very_lenient"

Non-consecutive wave schedules

Both pipelines evaluate presence over a wave grid. The default, grid = "consecutive", treats every integer between the span bounds as a scheduled wave. That is wrong for biennial panels or panels whose waves are recorded as years: every skipped year would count as a wave missed by everyone.

biennial <- generate_weasel_dummy_data(
  n_ids = 200,
  waves = seq(2008, 2022, by = 2),
  seed = 11
)
#> seed: 11
sort(unique(biennial$time))
#> [1] 2008 2010 2012 2014 2016 2018 2020 2022

With the consecutive default, weasel now warns (a weasel_empty_waves warning) and every scenario collapses:

p_wrong <- weasel_plan(biennial, "id", "time", span = "full")
#> Warning: 7 of 15 wave(s) in the span 2008:2022 have no observations for any
#> respondent (2009, 2011, 2013, 2015, 2017, ...); they count as missed by
#> everyone. if waves are non-consecutive by design (e.g. biennial), use grid =
#> "observed".
#> ✔ plan ready: span 2008:2022 (full, L = 15)
weasel_compare_scenarios(p_wrong)$n_ids
#> [1] 0 0 0

grid = "observed" evaluates presence on the schedule that actually occurs in the data. Interior gaps then mean skipped scheduled waves, and the span length L counts observed waves:

p_biennial <- weasel_plan(biennial, "id", "time",
                          span = "full", grid = "observed")
#> ✔ plan ready: span 2008:2022 (full, 8 observed waves)
p_biennial
#> <weasel_plan>
#>   span: 2008:2022 (full, 8 observed waves)
#>   respondents observed in span: 200
#>   population: 200 of 200 distinct ids observed in span (denominator: observed_in_span)
#>           scenario n_ids mean_prop_present endpoint_rate score recommended
#>    anchored_strict   114             1.000         1.000 3.693       FALSE
#>  anchored_balanced   156             0.966         1.000 3.705        TRUE
#>   lenient_info_max   185             0.941         0.892 3.570       FALSE
#>   data: 1439 row(s) attached

span = "core" combines with grid = "observed" as you would expect: the window is a block of core_len consecutive scheduled waves. The same grid argument exists in set_weasel_scope() for the exploratory pipeline.

Sensitivity of the sample size to the tolerances

How strongly does the retained sample react to the tolerance settings? weasel_sensitivity() sweeps every combination of the supplied values against the per-respondent metrics already stored in the plan, so it is cheap even for large panels:

sens <- weasel_sensitivity(
  plan_custom,
  require_endpoints = c(TRUE, FALSE),
  max_missing = 0:2,
  n_gap_max = 0:1,
  max_gap_len = 0:2
)
weasel_print_table(sens, title = "Tolerance sweep", n = 10)
#> ── Tolerance sweep ─────────────────────────────────────────────────────────────
#>  require_endpoints max_missing n_gap_max max_gap_len n_ids prop_ids
#>               TRUE           0         0           0   228     0.57
#>               TRUE           0         0           1   228     0.57
#>               TRUE           0         0           2   228     0.57
#>               TRUE           0         1           0   228     0.57
#>               TRUE           0         1           1   228     0.57
#>               TRUE           0         1           2   228     0.57
#>               TRUE           1         0           0   228     0.57
#>               TRUE           1         0           1   228     0.57
#>               TRUE           1         0           2   228     0.57
#>               TRUE           1         1           0   228     0.57
#>  mean_prop_present
#>                  1
#>                  1
#>                  1
#>                  1
#>                  1
#>                  1
#>                  1
#>                  1
#>                  1
#>                  1

A steep jump in n_ids between adjacent rows tells you exactly which constraint is binding; reviewers tend to ask for this table.

Selectivity: who gets excluded?

Selecting on participation completeness can bias a sample if retention relates to the covariates you study. weasel_selectivity() compares retained respondents with those observed in the span but excluded, using each respondent’s value at their first observed wave (or their within-span mean with at = "mean"), and reports standardized mean differences:

First observed waves can differ between respondents, so this need not compare a common baseline. A missing item at the first observed wave remains missing, even if a later wave has a value. Within-span means can also average different sets of waves. For covariates that change over time, differing observation times can contribute to the comparison.

sel <- weasel_selectivity(plan_custom, "one_gap_ok")
weasel_print_table(sel, title = "Retained vs excluded", digits = 3)
#> ── Retained vs excluded ────────────────────────────────────────────────────────
#>  variable n_retained n_excluded mean_retained mean_excluded   diff    smd
#>      var2        331         64        18.379        16.809  1.569  0.224
#>      var1        331         65         7.592         7.000  0.592  0.126
#>      var5        326         65         9.900         9.672  0.228  0.076
#>      var4        328         65         3.930         3.815  0.114  0.058
#>      var3        324         64         0.287         0.312 -0.025 -0.055

Absolute SMD values around 0.1 are a descriptive balance heuristic, rather than a universal cutoff (see Austin, 2009). Participation in this example is generated without using the covariates, but finite-sample differences can still exceed that value, as var2 and var1 do here. These comparisons describe observed covariate means; small SMDs do not establish similar distributions, ignorable attrition or unbiased estimates.

Plans without attached data

By default the plan object carries the original data, which is convenient but heavy for large panels (for example when saving plans with saveRDS()). Use keep_data = FALSE and pass the data back in where needed:

p_light <- weasel_plan(long_data, "id", "time",
                       span = "full", keep_data = FALSE)
#> ✔ plan ready: span 1:12 (full, L = 12)
nrow(weasel_apply(p_light, "lenient", data = long_data))
#> [1] 3981

weasel_sensitivity() and weasel_justify_subset() keep working on such plans because they only need the stored metrics.

New plans store a structural fingerprint with encoding_version = 2: aggregate counts plus an order-invariant digest of the deduplicated (id, wave) assignments across the full supplied panel, including waves outside the selected window. The encoding preserves supported ID values exactly, including distinct numeric values and strings containing separator characters, and does not depend on numeric display options. Covariate values are excluded. The guard checks explicitly supplied data; reuse of attached data assumes the original attached object.

A structural mismatch, including reassigned participation with unchanged aggregate counts, produces one classed warning (weasel_data_mismatch). The result still uses the stored scenario IDs: the guard does not replan, change the saved plan or ensure that changed data satisfy the original scenario. Review the warning before using the result:

d_other <- long_data[-(1:10), ]
msg <- tryCatch(
  weasel_apply(p_light, "lenient", data = d_other),
  warning = function(w) conditionMessage(w)
)
substr(msg, 1, 78)
#> [1] "the supplied data do not structurally match the data this plan was built from "

Saved plans are checked using their recorded encoding version. Fingerprints without a version retain the historical v1 encoding, including its loss of some numeric ID distinctions, ambiguous separator framing and dependence on numeric display options. Older count-only fingerprints compare their stored fields; plans without a fingerprint have no structural guard. No saved fingerprint is silently upgraded. Rebuild a plan from its verified original data to create a new fingerprint. An unsupported or malformed encoding version raises weasel_error_fingerprint_version rather than falling back to v1.

The fingerprint is an accidental-drift diagnostic, not authentication of covariate values or protection against deliberate tampering.

Span selection in detail

span = "core" slides a window of core_len consecutive grid waves over the full observed range and scores each window by mean per-wave respondent coverage; with grid = "consecutive", waves with no observations contribute zero, so the chosen window is always a contiguous block of core_len integer waves. Use span = "full" to keep the entire range:

plan_full <- weasel_plan(long_data, "id", "time", span = "full")
#> ✔ plan ready: span 1:12 (full, L = 12)
c(lower = plan_full$lower, upper = plan_full$upper)
#> lower upper 
#>     1    12

Explicit analysis windows

To supply your own analysis window, pass explicit bounds. The plan records span_reason = "explicit", and weasel_justify_subset() describes the bounds as user-supplied without inferring when or why they were chosen:

plan_fixed <- weasel_plan(long_data, "id", "time", lower = 3, upper = 10)
#> rows not used for participation: 0 with missing id, 0 with missing time, 1438 outside span 3:10.
#> ✔ plan ready: span 3:10 (explicit, L = 8)
plan_fixed$span_reason
#> [1] "explicit"

For automatic core windows, the selection is inspectable: every candidate window and its coverage is stored, the chosen one is flagged, and exact coverage ties raise a classed warning (weasel_tied_windows) instead of being resolved silently:

weasel_print_table(plan_custom$span_candidates, digits = 0)
#>  lower upper coverage chosen
#>      1     8     2936   TRUE
#>      2     9     2894  FALSE
#>      3    10     2866  FALSE
#>      4    11     2831  FALSE
#>      5    12     2797  FALSE

Auditing the recommendation

recommended marks the highest composite score under the declared weights, nothing more. The active weights and the per-scenario score decomposition travel with the comparison table, near_tie flags scenarios the score cannot meaningfully separate (tune with tie_tolerance), and because the size term is normalised within the supplied scenario set, scores are comparison-relative rather than absolute properties of a scenario. Existing scores move only when the largest retained sample in the set changes, which can reverse a ranking between two otherwise unchanged scenarios; a zero size weight removes the effect:

cmp_audit <- weasel_compare_scenarios(plan_custom)
attr(cmp_audit, "weights")
#>  coverage endpoints      size   missing      gaps 
#>       2.0       1.2       0.8       0.6       0.4
weasel_print_table(attr(cmp_audit, "score_components"), digits = 3)
#>       scenario coverage endpoints  size missing   gaps
#>  complete_case    2.000     1.200 0.466    0.00  0.000
#>     one_gap_ok    1.918     1.200 0.683   -0.15 -0.032
#>   very_lenient    1.863     1.059 0.800   -0.30 -0.035
cmp_audit[c("scenario", "score", "recommended", "near_tie")]
#>        scenario    score recommended near_tie
#> 1 complete_case 3.666496        TRUE    FALSE
#> 2    one_gap_ok 3.618705       FALSE    FALSE
#> 3  very_lenient 3.386701       FALSE    FALSE

The planning population

Respondents enter the planning population when they are observed at least once inside the span; every retention proportion is relative to that population, not to the full panel. Plans store and print the counts, and the justification text states the denominator explicitly:

plan_custom$population
#> $denominator
#> [1] "observed_in_span"
#> 
#> $n_rows_data
#> [1] 4304
#> 
#> $n_ids_data
#> [1] 400
#> 
#> $n_ids_in_span
#> [1] 400
#> 
#> $n_pairs_in_span
#> [1] 2936

Auditing per-respondent metrics

The plan object exposes the per-respondent table that the scenarios filter on, which is useful for custom diagnostics:

m <- plan_custom$id_metrics
weasel_print_table(head(m, 8), title = "Per-respondent metrics")
#> ── Per-respondent metrics ──────────────────────────────────────────────────────
#>      id n_present n_missing prop_present has_lower has_upper n_gap max_gap
#>  800001         8         0        1.000      TRUE      TRUE     0       0
#>  800002         8         0        1.000      TRUE      TRUE     0       0
#>  800003         8         0        1.000      TRUE      TRUE     0       0
#>  800004         8         0        1.000      TRUE      TRUE     0       0
#>  800005         8         0        1.000      TRUE      TRUE     0       0
#>  800006         7         1        0.875      TRUE      TRUE     1       1
#>  800007         8         0        1.000      TRUE      TRUE     0       0
#>  800008         7         1        0.875      TRUE      TRUE     1       1

# distribution of interior gap counts
table(m$n_gap)
#> 
#>   0   1   2 
#> 270 121   9

Scope pipeline constraints

The exploratory pipeline accepts the same structural vocabulary as the scenario tables. min_present sets the minimum number of observed waves a respondent needs to be kept (default 1, so exploration shows everyone); max_missing, max_gap_len, n_gap_max, and require_endpoints apply the same constraints a plan scenario would:

set_weasel_scope(
  long_data,
  id = "id", wave = "time",
  lower = 2, upper = 11,
  min_present = 6,   # at least 6 observed waves within the span
  max_gap_len = 1,   # no interior gap longer than 1 wave
  n_gap_max = 2      # at most 2 interior gaps
)
#> ✔ WEASEL scope set
evaluate_weasel_scope()
weasel_reshape_to_wide()
#> rows not used for participation: 0 with missing id, 0 with missing time, 722 outside span 2:11.
#> 28 respondent(s) dropped by the scope constraints; 371 kept.
weasel_summarize_waves()
weasel_print_table(weasel_filter_wave_summary(),
                   title = "Constrained patterns", n = 8)
#> ── Constrained patterns ────────────────────────────────────────────────────────
#>  pattern                 waves  n ids
#>        1 2 3 4 5 6 7 8 9 10 11 10 179
#>        2 2 3 4 5 . 7 8 9 10 11  9  20
#>        3  2 3 4 5 6 7 8 9 10 .  9  18
#>        4 2 3 4 5 6 . 8 9 10 11  9  16
#>        5 2 3 4 5 6 7 8 . 10 11  9  16
#>        6  2 3 4 5 6 7 8 9 . 11  9  16
#>        7 2 . 4 5 6 7 8 9 10 11  9  12
#>        8 2 3 4 . 6 7 8 9 10 11  9  11
weasel_clear_scope()

Justification styles

weasel_justify_subset() writes the methods paragraph in three registers:

cat(weasel_justify_subset(plan_custom, "one_gap_ok", style = "concise"),
    "\n")
#> We selected a longitudinal analysis subset using the WEASEL framework (Wave-based Extraction and Selection for Longitudinal Data) (van den Bosch, 2026). The analysis window was selected using the core span rule: the highest total count of distinct respondent-wave pairs among contiguous windows of the same length on the wave grid; ties are resolved in favour of the earliest window. Within waves 1 to 8, we required observed endpoints to ensure temporal anchoring, allowed up to 2 missing waves within the window and restricted the missingness structure (at most 1 interior missing block(s), each no longer than 2 wave(s)). This strategy retained 334 respondent(s). The planning population comprised 400 respondent(s) observed at least once within the analysis window, out of 400 distinct respondent(s) in the supplied data; retention figures are relative to this in-window population. In the resulting subset, mean within-window coverage was 0.959; endpoint coverage was 1.000.

The in-text citation (package author and year) is derived automatically from the package’s own citation metadata, so the zero-argument call is already fully citable; supply author/year only to override the derived reference:

cat(weasel_justify_subset(plan_custom, "one_gap_ok"), "\n\n")
#> To construct a longitudinal analysis sample, we selected respondents whose wave participation satisfied explicit structural criteria using the WEASEL framework (Wave-based Extraction and Selection for Longitudinal Data) (van den Bosch, 2026). Specifically, we focused on waves 1 to 8 (L = 8) and required observed endpoints to ensure temporal anchoring, allowed up to 2 missing waves within the window, and restricted the missingness structure (at most 1 interior missing block(s), each no longer than 2 wave(s)). This strategy retained 334 respondent(s), reflecting an explicit trade-off between sample size and within-window completeness. The planning population comprised 400 respondent(s) observed at least once within the analysis window, out of 400 distinct respondent(s) in the supplied data; retention figures are relative to this in-window population. In the resulting subset, mean within-window coverage was 0.959, endpoint coverage was 1.000. The analysis window was selected using the core span rule: the highest total count of distinct respondent-wave pairs among contiguous windows of the same length on the wave grid; ties are resolved in favour of the earliest window. All selection decisions were rule-based and reproducible, and can be regenerated from the same inputs and parameters using the weasel workflow.
cat(weasel_justify_subset(plan_custom, "one_gap_ok", style = "methods",
                          author = "van den Bosch", year = "2026"),
    "\n")
#> To construct a longitudinal analysis sample, we selected respondents whose wave participation satisfied explicit structural criteria using the WEASEL framework (Wave-based Extraction and Selection for Longitudinal Data) (van den Bosch, 2026). Specifically, we focused on waves 1 to 8 (L = 8) and required observed endpoints to ensure temporal anchoring, allowed up to 2 missing waves within the window, and restricted the missingness structure (at most 1 interior missing block(s), each no longer than 2 wave(s)). This strategy retained 334 respondent(s), reflecting an explicit trade-off between sample size and within-window completeness. The planning population comprised 400 respondent(s) observed at least once within the analysis window, out of 400 distinct respondent(s) in the supplied data; retention figures are relative to this in-window population. In the resulting subset, mean within-window coverage was 0.959, endpoint coverage was 1.000. The analysis window was selected using the core span rule: the highest total count of distinct respondent-wave pairs among contiguous windows of the same length on the wave grid; ties are resolved in favour of the earliest window. All selection decisions were rule-based and reproducible, and can be regenerated from the same inputs and parameters using the weasel workflow.

The extended style adds a fuller design rationale:

cat(weasel_justify_subset(plan_custom, "one_gap_ok", style = "extended"),
    "\n")
#> We used the WEASEL framework (Wave-based Extraction and Selection for Longitudinal Data) (van den Bosch, 2026) to derive a reproducible longitudinal analysis subset from the available panel waves. Admissible participation patterns were defined explicitly, and the retained subset follows from those definitions and the supplied data. We defined a target window of waves 1 to 8 (L = 8) and applied structural constraints to respondent trajectories. Within this window, we required observed endpoints to ensure temporal anchoring, allowed up to 2 missing waves within the window, and restricted the missingness structure (at most 1 interior missing block(s), each no longer than 2 wave(s)). Under these criteria, 334 respondent(s) were retained. The planning population comprised 400 respondent(s) observed at least once within the analysis window, out of 400 distinct respondent(s) in the supplied data; retention figures are relative to this in-window population. Subset diagnostics indicated that mean within-window coverage was 0.959 and endpoint coverage was 1.000. The analysis window was selected using the core span rule: the highest total count of distinct respondent-wave pairs among contiguous windows of the same length on the wave grid; ties are resolved in favour of the earliest window. The inclusion set is fully determined by the declared constraints (window bounds, endpoint handling, and permitted missingness structure) and the supplied data, and can be regenerated from them.

Quiet mode and reproducibility

All status messages respect a single option:

options(weasel.verbose = FALSE)
p_quiet <- weasel_plan(long_data, "id", "time", span = "full")
options(weasel.verbose = TRUE)

The data generator runs synthetic draws in a separate R process, while console colours rotate using package-private state without random draws. Neither writes to the caller’s global workspace. The caller’s random-number stream, RNG configuration and cached Box-Muller normal deviate remain untouched. Explicit seeds reproduce the same panels as before. Without a seed, each generation process draws and reports its own seed; repeated calls can produce different panels. Each call incurs R process startup time, and temporary files are removed on exit, including errors.

You can also use suppressMessages() for the logo, scope report and full demonstration. Quiet mode suppresses explicit table display too, while retaining its formatted return value. For example, generation leaves your analysis stream unchanged:

set.seed(99)
before <- rnorm(2)

set.seed(99)
tmp <- generate_weasel_dummy_data(n_ids = 20, n_times = 5, seed = 1)
#> seed: 1
after <- rnorm(2)

identical(before, after)
#> [1] TRUE