Preparing and checking data

Step 02 prepares questionnaire data and auxiliary variables; Step 03 checks the prepared data against the canonical routing DAG. A missing response may be valid for an inactive question and invalid for an active question: the diagnosis depends on the graph structure and observed response categories.

source(file.path("scripts", "02_prepare_data.R"))
source(file.path("scripts", "03_run_routing_check.R"))

03_run_routing_check.R saves routing_check_results.rds, full_report.csv, summary_by_record.csv, summary_by_variable.csv and execution_time.csv in output/routing-check/. Use the detailed report to identify the variables involved and the summaries to investigate systematic metadata errors.

Validating localization

06_validate_violation_localization.R compares localized violations with a reference report when one is available in the configuration. Comparing results with known truth is distinct from simply counting detected errors.

source(file.path("scripts", "06_validate_violation_localization.R"))

Before attributing a violation to a record, inspect the item mapping and external sources used by auxiliary variables: an inconsistent rule can produce many apparent errors.

Representation of prepared data

Retain the original questionnaire data. Preparation may replace OR/XOR members with canonical nodes and convert text columns into presence indicators. Matching the column names to the DAG is necessary, but does not verify that identifier and code semantics have been preserved. See Building the routing file for these conventions.

A ground-truth report is usable only when it refers to the actual records and violations in the analysed dataset. A matching key-column name is not enough. Do not associate synthetic violation ground truth with the real questionnaire data. Without a consistent reference, report localization results without supervised accuracy metrics.

Reading the check object

routing_check() returns full_report, summary_by_rec and summary_by_var. The full report contains one row per reported violation, including the record key, varname_label, routevar_label, errcode and message. When logical edits are requested, violation_source distinguishes ROUTING from LOGICAL_EDIT. This column is not added in routing-only mode.

The record summary groups by identifier and message. Therefore its row count is not necessarily the number of inconsistent records. Count distinct keys in full_report instead. Consistent records are absent from these violation summaries. See the overview’s standalone example for a complete call.

The current checker requires a mapping CSV even when no recoding is needed; a header-only file with varname,item_route,item_dataset is sufficient. Numeric IDs must remain unique and unchanged throughout preparation.