Overview

RDML7 provides two levels for writing file importers.

The simple form is intended for formats that can be represented as one fluorescence table plus one metadata table:

vendor file
    |
    v
parser
    |
    +-- fdata
    |
    `-- description
            |
            v
    buildRDMLFromFData()
            |
            v
         rdmlType

The complex form is intended for formats that contain several independent fluorescence-series groups, for example amplification and melting data, or several data blocks that should be parsed first and converted to RDML in one central step:

vendor file
    |
    v
parser
    |
    +-- rdmlImportSeries (adp)
    +-- rdmlImportSeries (mdp)
    `-- ...
            |
            v
       rdmlImportData
            |
            v
       buildRDMLImport()
            |
            v
         rdmlType

Both forms keep the vendor-specific parser separate from the nested RDML/S7 schema construction.

This separation is useful because a parser should primarily answer questions such as:

where are the fluorescence values in this file?
which reaction belongs to which sample?
which target and dye were used?
which run and reaction identifiers are present?
does the file contain amplification (adp), melting (mdp), or both?

The common RDML builder then handles creation of experimentType, runType, reactType, dataType, sampleType, targetType, dyes, references, and fluorescence-point objects.

Registering a reader

After a reader has been written, register it with the minimal format registry:

registerRDMLFormat(
  name = "myformat",
  extensions = "abc",
  reader = readMyFormat
)

A format may provide a reader, a writer, or both:

registerRDMLFormat(
  name,
  extensions = character(),
  reader = NULL,
  writer = NULL
)

Automatic dispatch uses the file extension. A different registered format can be selected explicitly with format =.

x <- readRDML(
  "experiment.abc",
  format = "myformat"
)

A reader may return either:

a finished rdmlType; or
an rdmlImportData intermediate object.

When a reader returns rdmlImportData, readRDML() converts it to rdmlType with the common importer builder.

The common table contract

Both simple and complex importers ultimately use the same table representation.

Fluorescence table

fdata is a wide table. Its first column is the coordinate:

cyc for amplification data;
tmp for melting data.

All remaining columns are fluorescence series.

For example:

fdata <- data.frame(
  cyc = 1:5,
  r1 = c(100, 102, 110, 180, 650),
  r2 = c(101, 103, 112, 190, 690)
)

fdata
#>   cyc  r1  r2
#> 1   1 100 101
#> 2   2 102 103
#> 3   3 110 112
#> 4   4 180 190
#> 5   5 650 690

The fluorescence column names (r1, r2) are arbitrary identifiers used to join the fluorescence data to the description table.

Description table

The corresponding description table contains one row per fluorescence series.

description <- data.frame(
  fdataName = c("r1", "r2"),
  expId = c("exp1", "exp1"),
  runId = c("run1", "run1"),
  reactId = c("1", "2"),
  sample = c("sample1", "sample2"),
  sampleType = c("std", "unkn"),
  target = c("CD53", "CD53"),
  targetDyeId = c("FAM", "FAM"),
  stringsAsFactors = FALSE
)

description
#>   fdataName expId runId reactId  sample sampleType target targetDyeId
#> 1        r1  exp1  run1       1 sample1        std   CD53         FAM
#> 2        r2  exp1  run1       2 sample2       unkn   CD53         FAM

The required columns are:

fdataName
expId
runId
reactId
target

In practice, importers commonly also provide:

sample
sampleType
targetDyeId

Other supported metadata can be added when present in the source file, for example:

targetType
quantity
cq
meltTemp
meltTemp

The exact optional fields accepted by the installed version of RDML7 are documented by setFData().

The key rule is:

description$fdataName <-> names(fdata)[-1]

Each fluorescence column should have exactly one matching description row.

Simple importer

When to use the simple form

Use buildRDMLFromFData() when the file naturally contains one fluorescence series group:

one amplification table; or
one melting table.

This is the preferred interface for most third-party readers.

Typical examples are:

a CSV export containing amplification curves;
a simple text export with one curve table;
a custom laboratory format containing one qPCR run;
a format that already has one row of metadata per fluorescence series.

The parser only needs to produce fdata and description.

Minimal simple reader

readMyFormat <- function(fileName, ...) {

  # Vendor-specific parsing
  fdata <- ...
  description <- ...

  buildRDMLFromFData(
    fdata = fdata,
    description = description,
    fdataType = "adp"
  )
}

The reader does not need to construct nested S7 objects manually.

It should not need code such as:

# Do not do this in a normal file parser:
experimentType(
  run = list(
    runType(
      react = list(
        reactType(
          data = list(
            dataType(...)
          )
        )
      )
    )
  )
)

That construction belongs to the common RDML builder.

Example: a simple text qPCR format

Consider a deliberately small text format with three sections:

# Simple qPCR format

[meta]
experiment=cd53_efficiency
run=run1
target=CD53
dye=FAM

[reactions]
fdataName  reactId  sample       sampleType  quantity
r1         1        CD53_1       std         1
r2         2        CD53_0.1     std         0.1
r3         3        CD53_0.01    std         0.01
r4         4        CD53_0.001   std         0.001

[adp]
cyc  r1   r2   r3   r4
1    100  100  100  100
2    101  100  101  100
3    103  102  101  101
...

The format deliberately stores quantity as metadata rather than encoding it only in the sample name.

A reader can parse these sections and then convert them to the common tables.

readSimpleQPCR <- function(fileName, ...) {

  lines <- readLines(
    fileName,
    warn = FALSE,
    encoding = "UTF-8"
  )

  getSection <- function(name) {
    marker <- paste0("[", name, "]")
    start <- match(marker, lines)

    if (is.na(start)) {
      stop(
        "Missing section ",
        marker,
        call. = FALSE
      )
    }

    sections <- grep(
      "^\\[[^]]+\\]$",
      lines
    )

    nextSection <- sections[
      sections > start
    ]

    end <- if (length(nextSection)) {
      nextSection[[1L]] - 1L
    } else {
      length(lines)
    }

    out <- lines[
      seq.int(
        start + 1L,
        end
      )
    ]

    out[
      nzchar(trimws(out))
    ]
  }

  # Metadata -------------------------------------------------------------
  metaParts <- strsplit(
    getSection("meta"),
    "=",
    fixed = TRUE
  )

  meta <- stats::setNames(
    vapply(
      metaParts,
      function(x) {
        paste(
          x[-1L],
          collapse = "="
        )
      },
      character(1)
    ),
    vapply(
      metaParts,
      `[[`,
      character(1),
      1L
    )
  )

  # Reaction metadata ----------------------------------------------------
  reactions <- utils::read.delim(
    text = paste(
      getSection("reactions"),
      collapse = "\n"
    ),
    stringsAsFactors = FALSE,
    check.names = FALSE
  )

  # Amplification curves -------------------------------------------------
  fdata <- utils::read.delim(
    text = paste(
      getSection("adp"),
      collapse = "\n"
    ),
    stringsAsFactors = FALSE,
    check.names = FALSE
  )

  description <- data.frame(
    fdataName = reactions$fdataName,
    expId = meta[["experiment"]],
    runId = meta[["run"]],
    reactId = as.character(
      reactions$reactId
    ),
    sample = reactions$sample,
    sampleType = reactions$sampleType,
    target = meta[["target"]],
    targetDyeId = meta[["dye"]],
    quantity = as.numeric(
      reactions$quantity
    ),
    stringsAsFactors = FALSE
  )

  buildRDMLFromFData(
    fdata = fdata,
    description = description,
    fdataType = "adp",
    publisher = "simpleqpcr"
  )
}

registerRDMLFormat(
  name = "simpleqpcr",
  extensions = "sqpcr",
  reader = readSimpleQPCR
)

It can then be used through the normal RDML7 interface:

x <- readRDML(
  "cd53_efficiency.sqpcr"
)

summary(x)

asTable(x)

amp <- getFData(
  x,
  dpType = "adp",
  longTable = TRUE
)

The parser knows nothing about the internal structure of sampleType, quantityType, targetType, experimentType, or dataType. It only maps the source file into the standard fluorescence and description tables.

Complex importer

When to use the complex form

Use the canonical intermediate representation when a single source file contains more than one independently described fluorescence-series group.

The most common case is:

one source file
    |
    +-- amplification curves (adp)
    |
    `-- melting curves (mdp)

Other reasons include:

separate curve blocks with different coordinate systems;
several tables that should be validated before any RDML object is built;
source metadata that belongs to the import as a whole rather than to one fluorescence table;
a parser that should preserve original reaction identifiers;
an importer that needs to declare conversion information centrally.

In this situation it is useful to parse the complete vendor file first and only then construct the RDML hierarchy.

`rdmlImportSeries`

One rdmlImportSeries represents one homogeneous fluorescence-series group.

For amplification data:

ampSeries <- newRDMLImportSeries(
  fdataType = "adp",
  fdata = ampFdata,
  description = ampDescription
)

For melting data:

meltSeries <- newRDMLImportSeries(
  fdataType = "mdp",
  fdata = meltFdata,
  description = meltDescription
)

Each series has its own:

fdataType
fdata
description

This is important because adp and mdp have different coordinates and may contain different subsets of reactions.

`rdmlImportData`

rdmlImportData represents the complete result of parsing one source file.

parsed <- newRDMLImportData(
  series = list(
    ampSeries,
    meltSeries
  ),
  publisher = "Example vendor",
  serialNumber = "12345",
  format = "myformat"
)

The object groups the fluorescence-series blocks and source-level metadata without yet constructing the nested RDML hierarchy.

This gives the parser a clean boundary:

everything above rdmlImportData = vendor parsing
everything below rdmlImportData = RDML construction

Building the RDML object explicitly

The parsed representation can be converted explicitly:

x <- buildRDMLImport(
  parsed
)

The builder processes each series with the common fluorescence-data machinery and returns one rdmlType.

Conceptually:

rdmlImportData
    |
    +-- series 1 -> setFData(..., fdataType = "adp")
    |
    +-- series 2 -> setFData(..., fdataType = "mdp")
    |
    `-- ...
            |
            v
         rdmlType

This means all complex importers use the same RDML construction rules.

Returning intermediate data directly from a reader

A complex reader does not have to call buildRDMLImport() itself.

It may return rdmlImportData:

readMyComplexFormat <- function(fileName, ...) {

  # Parse the source file.
  ampFdata <- ...
  ampDescription <- ...

  meltFdata <- ...
  meltDescription <- ...

  ampSeries <- newRDMLImportSeries(
    fdataType = "adp",
    fdata = ampFdata,
    description = ampDescription
  )

  meltSeries <- newRDMLImportSeries(
    fdataType = "mdp",
    fdata = meltFdata,
    description = meltDescription
  )

  newRDMLImportData(
    series = list(
      ampSeries,
      meltSeries
    ),
    publisher = "Example vendor",
    serialNumber = "12345",
    format = "mycomplexformat"
  )
}

registerRDMLFormat(
  name = "mycomplexformat",
  extensions = "mcf",
  reader = readMyComplexFormat
)

Then:

x <- readRDML(
  "experiment.mcf"
)

readRDML() detects that the reader returned rdmlImportData and runs the common builder before returning the final rdmlType.

This is the preferred complex-reader pattern because the reader remains a pure parser.

Why not always use the complex form?

The intermediate representation is useful, but it should not be required for simple formats.

For a single amplification table, this:

buildRDMLFromFData(
  fdata,
  description,
  fdataType = "adp"
)

is clearer than:

series <- newRDMLImportSeries(
  "adp",
  fdata,
  description
)

parsed <- newRDMLImportData(
  series = list(series)
)

buildRDMLImport(parsed)

The second form adds objects without adding information.

Use the lowest-level interface that faithfully represents the source format.

Choosing between the two forms

Source format	Recommended form
One amplification table	`buildRDMLFromFData()`
One melting table	`buildRDMLFromFData()`
Simple CSV or TSV curve export	`buildRDMLFromFData()`
One table plus straightforward metadata	`buildRDMLFromFData()`
Amplification + melting in one file	canonical importer
Several independent fluorescence blocks	canonical importer
Need to parse/validate all blocks before building RDML	canonical importer
Need source-level importer metadata	canonical importer

A useful rule is:

If the source can be described by one fdata table and one description table, use buildRDMLFromFData(). If it naturally requires a list of such pairs, use rdmlImportData.

Relationship between the two APIs

The simple API is not a separate implementation.

Conceptually:

buildRDMLFromFData()
        |
        v
newRDMLImportSeries()
        |
        v
newRDMLImportData()
        |
        v
buildRDMLImport()

In other words, the simple function is a convenience layer over the same canonical construction path.

This is important because both simple and complex third-party importers get the same behavior for:

creation of experiments and runs;
creation of reactions;
sample and target references;
dyes;
sample types;
quantities;
Cq and melting metadata;
amplification and melting curve storage;
conflict handling.

Reader responsibilities

A custom reader should:

parse the external file;
normalize identifiers to character values;
construct the standard fdata and description tables;
preserve meaningful source metadata;
report malformed input clearly;
return either rdmlType or rdmlImportData.

A custom reader normally should not:

manually construct the complete nested RDML hierarchy;
duplicate setFData() logic;
invent missing biological metadata;
silently change sample or target identifiers;
encode format detection logic inside the parser.

Format selection is handled by the registry; RDML construction is handled by the common builder.

Validating an importer

A useful development loop is:

registerRDMLFormat(
  name = "myformat",
  extensions = "abc",
  reader = readMyFormat
)

x <- readRDML(
  "example.abc"
)

summary(x)

tbl <- asTable(x)
tbl

validateRDML(x)

amp <- getFData(
  x,
  dpType = "adp",
  longTable = TRUE
)

For a format containing melting data, inspect those separately:

melt <- getFData(
  x,
  dpType = "mdp",
  longTable = TRUE
)

During importer development, verify at least:

number of experiments and runs;
reaction identifiers;
sample identifiers;
target identifiers;
sample types;
quantities, if provided;
dye references;
number of fluorescence curves;
number and order of curve coordinates;
adp versus mdp assignment.

Testing a reader

A small test should verify both dispatch and biological metadata.

test_that("myformat reader imports amplification curves", {

  x <- readRDML(
    test_path(
      "data",
      "example.abc"
    )
  )

  expect_true(
    S7::S7_inherits(
      x,
      rdmlType
    )
  )

  tbl <- asTable(x)

  expect_true(
    all(tbl$adp)
  )

  expect_setequal(
    unique(tbl$target),
    "CD53"
  )

  amp <- getFData(
    x,
    dpType = "adp"
  )

  expect_gt(
    nrow(amp),
    0L
  )
})

For a complex reader:

test_that("complex reader imports both amplification and melting data", {

  x <- readRDML(
    test_path(
      "data",
      "example.mcf"
    )
  )

  tbl <- asTable(x)

  expect_true(
    any(tbl$adp)
  )

  expect_true(
    any(tbl$mdp)
  )
})

Summary

RDML7 intentionally provides two importer interfaces.

The simple form:

buildRDMLFromFData(
  fdata,
  description,
  fdataType
)

is the preferred choice for a source represented by one fluorescence table.

The complex form:

series <- newRDMLImportSeries(...)
parsed <- newRDMLImportData(
  series = list(...)
)

buildRDMLImport(parsed)

is useful when a file contains several independently described fluorescence groups.

The format registry remains independent of both approaches:

registerRDMLFormat(
  name,
  extensions,
  reader,
  writer
)

This keeps the three responsibilities separate:

registry       -> which reader handles this file?
parser         -> what data are stored in this vendor format?
RDML builder   -> how are those data represented as rdmlType?

That separation is the main reason the importer API exists.

Writing custom RDML importers

RDML7 authors

2026-10-14