02b — Building the routing file

Pierpaolo Massoli

Scope and reference

Routing metadata describe variable order, domains represented in the graph, logical groups and jumps. A missing response may reflect an inactive question or an omission: the path distinguishes these cases.

This guide describes the routing format supported by SSPICE 0.1.0. Examples are independent of the bundled questionnaire. Auxiliary variables and semantic edits are documented in Metadata and external sources. The framework is described in Massoli (2026), An Integrated Framework for Rule-Based Data Editing, https://arxiv.org/abs/2610.16947.

Physical format and row order

The project uses a semicolon-separated CSV file without a header. Each row represents an original variable; V1, V2, etc. are column names assigned by the reader and must not be added as the first row. build_DAG() also accepts a data frame with these columns.

Use the same number of fields on every row and pad unused jump pairs with empty fields or .. Supply the separator through separator and missing-value tokens through na_strings. In the project configuration, "", ".", "NA" and "NULL" denote missing values. Do not use these tokens as names or labels that must be retained.

Variable names must be unique, non-empty, case-consistent and match the data. Prefer syntactically valid R names; avoid entirely numeric names and collisions with canonical names such as VOR1 and VXOR1. Some functions construct expressions from these names.

Row order is part of the specification. Without a jump, processing continues to the next row, accounting for group contraction. Place each group’s members in a contiguous block. Do not sort the file alphabetically after construction.

In the sequential model used here, jump targets follow the originating variable. The last variable generates no outgoing incidence rows: do not use it to define another jump. stop_on_cycle = TRUE checks for cycles, but does not by itself establish the validity of all metadata.

Field dictionary

Field Descriptive name Meaning
V1 varname Original variable name
V2 nitems Represented categories; special conventions are explained below
V3 vmin Lower bound of the represented domain
V4 vmax Upper bound of the represented domain
V5 lgroup 0: single variable; positive: OR group; negative: XOR group
V6 nroutes 0: ordinary continuation; positive other than 98: number of jump pairs; 98: unconditional jump
V7 item1 Category triggering the first jump; missing for an unconditional jump
V8 target1 Target variable name for the first jump
V9, V10 item2, target2 Second pair, when specified
V(2j+5), V(2j+6) itemj, targetj Jump pair j

Construction requires at least six columns and two rows. Files containing jumps must include every column needed for the declared pairs. Set V5 and V6 to 0, not to a missing value, when no group or jump applies.

Variable types

Case V2 V3 V4 Structural interpretation
Qualitative, multiple categories n > 1 first code last code One structural category for each equidistant code
Qualitative, single category 1 c c One declared value
Quantitative 1 minimum maximum > minimum One presence state in routing; a declared numeric interval
Free text 0 . 1 Text presence/absence
Territorial code or numeric category without code-specific branching -1 minimum maximum One graph state, without enumerating every code

Qualitative variables and nonconsecutive codes

For n > 1, routing categories are equidistant:

\[d = (v_{max}-v_{min})/(n-1).\]

For example, SCELTA;3;1;3;0;0;.;. declares 1, 2 and 3, while FASE;3;10;30;0;0;.;. declares 10, 20 and 30. It does not represent 10, 21 and 30. A single-category variable can be specified as FLAG;1;1;1;0;0;.;.. In routing, an observed zero does not generally mean missing; OR members, however, follow a specific convention.

The dedicated Item mapping guide provides a complete example with irregular codes, a jump, checks, correction and exports.

items_mapping.csv stores a correspondence between codes of the same variable, not between items and column names:

varname,item_route,item_dataset
FASE,10,10
FASE,20,20
FASE,30,21

This example declares the pairing of structural category 30 with observed value 21. The project contains a more extensive mapping for FASE. build_DAG() does not receive this file: it constructs the graph from routing metadata. Subsequent functions accepting item mapping use it separately.

Encoding contract. Keep observed codes in the data and in VAUX/LEDIT predicates. Declare the equidistant structural codes in the routing metadata, including jump conditions. For example, observed codes {10, 21, 99} may map to structural codes {10, 20, 30}; the routing row then declares three modes, minimum 10 and maximum 30. A jump for observed 21 uses structural code 20. The alternative structural domain {1, 2, 3} uses minimum 1 and maximum 3.

At the checking boundary, each observed code is looked up once in column 3 and translated to column 2, then scaled to a dummy position. Missing values remain missing. The records and routing table are not rewritten. The canonical checker and correction oracle use this same encoding. Correction domains and generated RULES/VARDOM retain observed codes; gate selection translates them internally when comparing routing jump conditions.

For each mapped variable present in the DAG, supply a complete one-to-one mapping covering its structural domain. Duplicate codes, incompatible domains, and observed codes absent from the mapping raise an error. A metadata file may also contain variables outside the current DAG. Header-only mapping files remain valid when no recoding is needed. Mapping does not turn the routing check into a general numeric-domain validator.

After changing code or metadata, run scripts/13_verify_item_mapping.R before reusing results. The mapping patch changes the cache schema and correction checkpoint fingerprint so that results computed with the earlier encoding are not silently reused.

Quantitative variables

IMPORTO;1;0;100000;0;0;.;. declares an interval but does not create 100001 branches. The variable has a single structural presence state. A jump associated with that state is not a threshold comparison: writing a numeric item in V7 does not by itself implement IMPORTO > threshold.

Numeric routing conversion uses as.integer(): do not specify decimal bounds expecting them to be preserved. This limitation concerns metadata; it does not mean that every observed quantitative value must be an integer. Checking presence along a path does not replace complete numeric-domain validation: dummize() activates the single state for any non-missing value when nitems = 1.

Strings, leading zeros and identifiers

PROFESSIONE;0;.;1;0;0;.;. represents an open text field. During preparation, clean_data() converts each character/factor column to 1 if non-empty text remains after ASCII conversion and trimming, and to NA otherwise. The original text is not retained in the prepared column. This is not a mechanism for imputing or comparing the content of phrases.

Conversion depends on the column’s actual R class, not just V2. A code such as "001" read as numeric becomes 1 and loses its leading zeros; read as character, it becomes a presence indicator in clean_data(). Retain the original representation separately when formatting is informative. Alphanumeric identifiers require a one-to-one numeric encoding consistent across all linked files; do not supply them as text expecting the key to survive cleaning.

PROVINCIA;-1;1;107;0;0;.;. retains numeric coding in the data but avoids 107 graph branches. It does not validate membership in an official code list. If the path depends on a specific code, use an appropriate structural-category representation: -1 does not create separate branches for individual codes.

An identifier may be represented, as DB030 is in the project, by V2 = 1 and an interval. Its key role comes from record_id, not from a statistical property inferred from routing. Similarly, -1 does not automatically freeze a variable during correction. Identifier role, stratification and immutability are separate settings.

Continuation, conditional jumps and unconditional jumps

No jump: V6 = 0

Every category continues to the next node. Jump-pair fields remain missing. A target written after V6 is not used when V6 is zero.

Conditional jump

SCELTA;2;1;2;0;1;1;FINE
DETTAGLIO;1;0;100;0;0;.;.
FINE;1;1;1;0;0;.;.

If SCELTA is 1, the path jumps directly to FINE; if it is 2, the path continues to DETTAGLIO and then FINE. DETTAGLIO must be missing when its branch is skipped. Declare only one target per category and do not repeat the same category in different pairs. For two jumps, V6 is 2 and V9/V10 are also required. Do not use 98 as an ordinary count: it is reserved.

UNCONDITIONAL (absolute) jump: V6 = 98

SCELTA;3;1;3;0;98;.;FINE
DETTAGLIO;1;0;100;0;0;.;.
FINE;1;1;1;0;0;.;.

The jump is independent of the originating variable’s item value. When SCELTA is active, all its categories lead to FINE. Set V7 to missing (. in the CSV) and put FINE in V8; one pair is sufficient.

In the current code, 98 restricts reading to the first pair, while a missing V7 selects all categories. Do not enter a specific item in V7: for a variable with multiple categories this can restrict the jump to that item despite V6 being 98. A missing V7 can also extend a conditional jump to all categories, so fill these fields carefully.

Unconditional does not mean that the target must be filled when the originating variable is inactive. Independence concerns the item value, not activation of the path. The isolated DETTAGLIO node in this small example is intentional; in a complete questionnaire it may receive other incoming edges.

Historical V6 = -1 convention

A historical convention assigns -1 the meaning of no edge to the next row. create_incidence_matrix() instead creates the ordinary edge first and only processes jumps when V6 > 0. Do not use -1 to suppress the edge in the current version: this historical semantics is not implemented.

Logical groups

OR: at least one selected member

Members share a positive identifier in V5 and occupy contiguous rows. A group must contain at least two members. Example:

INIZIO;1;1;1;0;0;.;.
MEZZO_A;1;1;1;1;0;.;.
MEZZO_B;1;1;1;1;0;.;.
FINE;1;1;1;0;0;.;.

When the group is required, at least one member must be selected; multiple selections are allowed. The canonical DAG uses a single VOR1 node. Preparation replaces the members with this node.

Current data convention: generate_groups_OR() evaluates member > 0. Use a positive value for selected and NA for unselected. Zero or negative values do not activate the group. Do not describe this implementation as a generic non-missingness test suitable for any domain.

OR coarsening resets jumps on the resulting node and does not preserve member-specific jumps. In this representation, members should be declared without individual jumps. For an incoming jump, use the canonical name VOR1 as its target: original member names are removed and OR coarsening does not automatically rewrite all references to them. Inspect the grouped routing table after construction.

XOR: exactly one of two alternatives

The current version handles a group of two members sharing a negative V5 label:

This is not an arbitrary XOR among three or more variables or two quantitative variables. The following example represents the choice between reporting an expense and reporting no expense; the no-expense branch skips the detail question:

INIZIO;1;1;1;0;0;.;.
NESSUNA_SPESA;1;1;1;-1;98;.;FINE
IMPORTO;1;0;100000;-1;0;.;.
DETTAGLIO;2;1;2;0;0;.;.
FINE;1;1;1;0;0;.;.

VXOR1 is 1 when only the categorical member is present and 2 when only the quantitative member is present. Both missing produce NA: this is compatible with an inactive group, but not a required group. Both present constitute a violation and, with strict = TRUE, cause generate_groups_XOR() to stop. These members are checked for non-missingness: IMPORTO = 0 is also present.

The natural successor is the first row after the entire group. Each branch may have its own target. The quantitative member accepts ordinary continuation or an unconditional jump, not value-conditional jumps; the categorical member must have a single effective target. Coarsening then constructs jumps on categories 1 and 2 of the canonical node.

Standalone construction examples

These examples require only an installed SSPICE package. They do not load PROJECT data and are executed when the vignette is built. Example variable names are kept unchanged to match the CSV fragments: SCELTA means choice, DETTAGLIO means detail, INIZIO/FINE mean start/end, IMPORTO means amount, NESSUNA_SPESA means no expense, and MEZZO_A/MEZZO_B identify two options.

make_routes <- function(lines) {
  read.table(text = paste(lines, collapse = "\n"),
             sep = ";", header = FALSE,
             na.strings = ".", stringsAsFactors = FALSE)
}
r <- make_routes(c(
  "SCELTA;2;1;2;0;1;1;FINE",
  "DETTAGLIO;1;0;100;0;0;.;.",
  "FINE;1;1;1;0;0;.;."
))
dag <- SSPICE::build_DAG(r, analyze = FALSE)
stopifnot(isTRUE(dag$graph_diagnostics$is_DAG))
dag$Bt
#>      SCELTA DETTAGLIO FINE
#> [1,]      1         0   -1
#> [2,]      1        -1    0
#> [3,]      0         1   -1

absolute <- r
absolute$V6[1] <- 98L
absolute$V7[1] <- NA
absolute$V8[1] <- "FINE"
dag_absolute <- SSPICE::build_DAG(absolute, analyze = FALSE)
stopifnot(all(dag_absolute$Bt[1:2, "FINE"] == -1),
          all(dag_absolute$Bt[1:2, "DETTAGLIO"] == 0))
dag_absolute$Bt
#>      SCELTA DETTAGLIO FINE
#> [1,]      1         0   -1
#> [2,]      1         0   -1
#> [3,]      0         1   -1

or_routes <- make_routes(c(
  "INIZIO;1;1;1;0;0;.;.",
  "MEZZO_A;1;1;1;1;0;.;.",
  "MEZZO_B;1;1;1;1;0;.;.",
  "FINE;1;1;1;0;0;.;."
))
dag_or <- SSPICE::build_DAG(or_routes, analyze = FALSE)
stopifnot("VOR1" %in% dag_or$varnames)
dag_or$routes_grouped
#>       V1 V2 V3 V4 V5 V6 V7 V8
#> 1 INIZIO  1  1  1  0  0 NA NA
#> 2   VOR1  1  1  1  0  0 NA NA
#> 4   FINE  1  1  1  0  0 NA NA

xor_routes <- make_routes(c(
  "INIZIO;1;1;1;0;0;.;.",
  "NESSUNA_SPESA;1;1;1;-1;98;.;FINE",
  "IMPORTO;1;0;100000;-1;0;.;.",
  "DETTAGLIO;2;1;2;0;0;.;.",
  "FINE;1;1;1;0;0;.;."
))
dag_xor <- SSPICE::build_DAG(xor_routes, analyze = FALSE)
stopifnot("VXOR1" %in% dag_xor$varnames)
dag_xor$routes_grouped
#>          V1 V2 V3 V4 V5 V6 V7   V8
#> 1    INIZIO  1  1  1  0  0 NA <NA>
#> 2     VXOR1  2  1  2  0  1  1 FINE
#> 3 DETTAGLIO  2  1  2  0  0 NA <NA>
#> 4      FINE  1  1  1  0  0 NA <NA>

Checks before using routing metadata

  1. Check names, order, group contiguity and forward targets.
  2. Check types, integer metadata bounds and equidistant categories.
  3. For each jump, check its category, target and pair count.
  4. For each unconditional jump, check V6 = 98, missing V7 and a target in V8.
  5. Inspect routes_grouped, routes_numeric, Bt and isolated nodes.
  6. Prepare a few records with known expected results: ordinary continuation, each conditional jump, every category of an unconditional jump, OR with one/multiple/no selections, and XOR with each alternative and both present.
  7. Compare expectations with the actual record-checking results.

build_DAG() checks missing/duplicate names, some group constraints and acyclicity. It does not exhaustively validate repeated pairs, out-of-domain categories, unknown references or all conventions listed here. Numeric conversion may turn unrecognized references into NA; absence of an error is not a guarantee of correctness.

From questionnaire instructions to routing metadata

The following adapted fragments illustrate conditional jumps, unconditional jumps and multiple-response groups. Question wording, identifiers and numbering have been simplified for this tutorial. These examples are independent of the bundled questionnaire.

Conditional and unconditional jumps

Q1. During the reference period, did your household receive a housing allowance?

  • Yes (1): continue to Q2.
  • No (2): go to Q3.

Q2. What was the total amount received?

Amount: __________

After Q2, go directly to Q3, regardless of the amount reported.

Q3. Continue with the next questionnaire section.

The jump from Q1 to Q3 is conditional on response code 2. When this response is selected, Q2 is skipped and must be blank. When Q1 is answered Yes, Q2 is reached and an amount is required.

The jump from Q2 to Q3 is unconditional: its destination does not depend on the amount reported. This routing instruction does not make Q2 optional or remove its domain restrictions.

In SSPICE routing metadata, the conditional jump is encoded using the triggering item and its target. The unconditional jump uses nroutes = 98, an empty first routing-item field, and the target variable in the corresponding destination field.

In this short fragment, Q3 immediately follows Q2. The unconditional instruction is therefore equivalent to ordinary progression, but makes the value-independent destination explicit. The same mechanism can target a later variable and skip intervening questions.

A multiple-response question represented by an OR group

Q4. Which sources did you use to estimate the amount?

Select at least one option. More than one option may be selected.

  • Personal experience.
  • Information from other households.
  • Published advertisements.

Represent the three options by consecutive variables SRC1, SRC2 and SRC3. In this example, a selected option is coded 1 and an unselected option is blank. Assign the same positive logical-group identifier to the three routing rows.

When this question is reached, at least one member must be filled. Several filled members are allowed: this is an OR group. If routing skips the whole question, all three members must be blank.

The requirement to select at least one option is explicit in this adapted example. The instruction ‘multiple answers allowed’ alone would not establish that requirement.

OR groups and single-choice questions

Q1 is a single categorical variable with two response codes. Its mutually exclusive answers do not, by themselves, require an SSPICE XOR group. XOR groups concern the supported representation across separate variables described earlier in this guide.