Adding a custom mark

library(glyph)

This guide adds a new mark type to glyph, from the R function to the drawing code in the browser. The example is a lollipop mark: a stem from zero to each value, with a dot at the end. glyph doesn’t have one yet, so you can follow along and end up with a working mark_lollipop().

A lollipop chart of fruit sold: stems rise from zero to 12, 7 and 9 for apple, fig and kiwi, and one hangs down to -3 for pear.

Be clear about one thing before you start. Adding a mark means editing glyph’s own source code. There is no way yet to add a mark from another package or from a script. The last section explains why, and what would have to change.

You need a copy of the glyph source, and R with devtools. To try the browser tests at the end, you also need chromote and Chrome.

How a mark gets from R to the screen

A mark passes through five places on its way to the screen:

  1. A function such as mark_point() adds a mark to the spec. The spec is a plain R list.
  2. compile() resolves the mark’s mappings against the data and turns the spec into JSON.
  3. htmlwidgets sends the JSON to the browser with the plot.
  4. inflateSpec() in inst/htmlwidgets/glyph.js turns the data back into one object per row.
  5. drawMark() in the same file calls the drawing function for the mark’s type.

Steps 2 to 4 are the same for every mark. A new mark needs code in steps 1 and 5, plus a few hooks for axes and interactions.

Step 1: the R function

Every mark function is a thin wrapper around add_mark() in R/marks.R. Add mark_lollipop() to that file, next to the other marks:

#' Add a lollipop mark
#'
#' Draws a stem from zero to each value, with a dot at the end.
#' @inheritParams mark_point
#' @return Modified \code{glyph_spec} object with the lollipop mark added
#' @export
#'
#' @examples
#' df <- data.frame(fruit = c("apple", "fig", "kiwi"), sold = c(12, 7, 9))
#' glyph(df, x = fruit, y = sold) |> mark_lollipop()
mark_lollipop <- function(spec, ..., data = NULL, style = list()) {
  add_mark(spec, "lollipop", rlang::enquos(...), data, style,
           defaults = list(size = 5, stroke_width = 1.5, opacity = 0.9))
}

add_mark() takes:

Run devtools::document() to add the help page and the export to NAMESPACE. Then add mark_lollipop to the “Marks” section of _pkgdown.yml so the reference page lists it.

The mark is now stored in the spec like any other. You can see the same structure for an existing mark:

sales <- data.frame(
  fruit = c("apple", "fig", "kiwi", "pear"),
  sold = c(12, 7, 9, -3)
)
p <- glyph(sales, x = fruit, y = sold * 2) |> mark_point()
str(p$marks[[1]][c("type", "style", "id", "stat")])
#> List of 4
#>  $ type : chr "point"
#>  $ style:List of 2
#>   ..$ size   : num 5
#>   ..$ opacity: num 0.8
#>  $ id   : chr "point_1"
#>  $ stat : NULL

Each mark gets an id from its type and position, such as point_1. The id is used to name computed columns, as the next step shows.

Step 2: what compile() does with the mark

compile() handles mappings the same way for every mark, so a new mark gets this for free. In resolve_mark() in R/render.R:

The result is the mark’s encoding:

compiled <- compile(p)
str(compiled$spec$marks[[1]]$encoding)
#> List of 2
#>  $ x:List of 3
#>   ..$ field: chr "fruit"
#>   ..$ type : chr "nominal"
#>   ..$ label: chr "fruit"
#>  $ y:List of 3
#>   ..$ field: chr ".g_point_1_y"
#>   ..$ type : chr "quantitative"
#>   ..$ label: chr "sold * 2"

Your drawing code reads values through these fields. It never needs to know whether a mapping was a plain column or an expression.

Step 3: how data reaches the browser

Plot data is sent as columns, not rows. This makes compile() fast on large data. The plot’s own data looks like this:

str(compiled$spec$data)
#> List of 2
#>  $ columns:List of 3
#>   ..$ fruit       : chr [1:4] "apple" "fig" "kiwi" "pear"
#>   ..$ sold        : num [1:4] 12 7 9 -3
#>   ..$ .g_point_1_y: num [1:4] 24 14 18 -6
#>  $ n      : int 4

A mark that draws the plot’s data doesn’t carry a copy. It names the columns it uses and, if any rows were dropped for missing values, which rows it keeps:

str(compiled$spec$marks[[1]]$data)
#> List of 3
#>  $ from  : chr "plot"
#>  $ fields: 'AsIs' chr [1:3] "fruit" "sold" ".g_point_1_y"
#>  $ rows  : NULL

A mark with its own data sends its own columns instead. A statistical mark sends its summaries as values, one record per box or bin.

You don’t have to deal with any of this. Before anything is drawn, inflateSpec() in glyph.js turns every mark’s data back into an array of row objects. So your drawing function gets data as rows like {fruit: "apple", sold: 12, ".g_point_1_y": 24}, with exactly the rows it should draw.

Step 4: the drawing code

The browser side lives in inst/htmlwidgets/glyph.js. First add a case to the switch in drawMark():

    case "lollipop":
      drawLollipop(g, data, enc, scales, style, w, h, interact, theme, markRegistry);
      break;

Then add the drawing function. A good place is next to drawBars():

// A stem from zero to each value, with a dot at the end. Each lollipop is
// one <g>, so tooltips, hover, brushing and selection all act on the stem
// and the dot together.
function drawLollipop(g, data, enc, scales, style, w, h, interact, theme, markRegistry) {
  const baseOpacity = style.opacity ?? 0.9;
  const r = style.size || 5;
  const slotWidth = groupSlotWidth(scales, enc, 40);
  const px = d => xPos(scales, enc, d, slotWidth);
  const py = d => yPos(scales, enc, d);
  // Stems start at zero, or at the edge of the plot when a domain set with
  // set_scale() leaves zero outside it.
  const base = scales.y ? Math.max(0, Math.min(h, scales.y(0))) : h;
  const colour = d => scales.color ? scales.color(getVal(d, enc.color)) : (theme.accent || "#4269d0");

  const pops = g.append("g").attr("class", "marks-lollipop")
    .selectAll("g")
    .data(data)
    .join("g")
    .attr("class", "glyph-lollipop")
    .attr("opacity", baseOpacity);
  pops.append("line")
    .attr("x1", px).attr("x2", px)
    .attr("y1", base).attr("y2", py)
    .attr("stroke", colour)
    .attr("stroke-width", style.stroke_width || 1.5);
  pops.append("circle")
    .attr("cx", px).attr("cy", py)
    .attr("r", r)
    .attr("fill", colour);

  // Brushing selects a lollipop when its dot is inside the brushed area.
  const finder = pointFinder(pops, px, py);
  registerMark(markRegistry, pops, baseOpacity, {
    brush: {
      rowLevel: true,
      nearest: (mx, my, tolerance) => finder.nearest(mx, my, tolerance),
      hit: (d, i, node, [[x0, y0], [x1, y1]]) => {
        const x = px(d), y = py(d);
        return x >= x0 && x <= x1 && y >= y0 && y <= y1;
      },
      opacity: hit => hit ? 1 : 0.15
    }
  });

  if (interact.tooltip?.enabled) addTooltip(pops, data, enc, interact.tooltip, theme);
  if (interact.hover?.enabled) {
    pops
      .on("mouseover.hover", function() { d3.select(this).select("circle").attr("r", r * 1.6); })
      .on("mouseout.hover", function() { d3.select(this).select("circle").attr("r", r); });
  }
  if (interact.click?.enabled) addClickSelect(pops, interact.click.action, baseOpacity, markRegistry);
}

The arguments are the same for every drawing function:

xPos() places a value on the x scale. On a categorical scale it puts it in the middle of its band, which is what most marks want. yPos() does the same for y.

Wrap each element in a class of its own, such as marks-lollipop and glyph-lollipop. That gives tests and stylesheets something to find.

Step 5: the axes

The x and y scales cover every mark’s values. markExtent() in glyph.js reports the range each mark draws along an axis. For plain mappings it uses the mapped column, which is enough for most marks.

A lollipop measures from zero, like a bar, so its value axis should include zero. Add "lollipop" to the rule that does this for bars and areas:

    if (axis === "y" && (mark.type === "bar" || mark.type === "area" || mark.type === "lollipop")) vals.push(0);

If your mark draws something other than its mapped values, such as a whisker or a bin edge, add a branch to markExtent() that pushes those values instead. The statistical marks do this. Without it, part of the mark can be drawn outside the axes.

Step 6: tooltips, hover, brush and click

Interactions are where most of the work in a new mark goes. glyph has helpers for each of them.

Tooltips. addTooltip(selection, data, enc, interact.tooltip, theme) shows the mapped fields when the pointer is over an element, and on tap on touch screens. It uses the widget’s one shared tooltip. Don’t create your own tooltip elements: each redraw would leave another one behind.

Hover. The built-in addHoverEffect() changes the radius or outline of the element itself. A lollipop is a group, so the example writes its own small handler that enlarges the dot inside the group.

Brushing. When brushing is on, an overlay covers the plot so a drag can start anywhere. The overlay also takes the pointer events the marks would get. glyph forwards them to the element under the pointer, but only for marks registered with registerMark(). The brush part of the registration says:

Forwarded events are sent to the element that nearest returns. Attach every listener (tooltip, hover, click) to that same element. A listener on a child, such as the circle inside a lollipop’s group, would never see them.

Registering the mark also connects it to linked selections in compose(), to legend selections, to the “Clear selection” button and to the Escape key. These all change the opacity of the registered elements, so put the opacity on the group, not on its children.

Click. addClickSelect() handles interact(click = ...) and selection from linked panels.

Step 7: fitting the container

Plots are drawn at the size of the container they are in, and redrawn when it changes size (#17). Fitting a plot can take up to three draws, and every resize redraws the whole plot. Your drawing function will run many times. Three rules keep it working:

Facets, compose() and inset() call the same drawing function once per panel, each with its own g, w, h and scales. A mark that follows these rules works in all of them without extra code. Zoom also works without extra code, because it transforms the whole plot area.

Step 8: tests

Add tests in tests/testthat/test-lollipop.R. An R test checks what compile() produces:

sales <- data.frame(fruit = c("apple", "fig", "kiwi", "pear"), sold = c(12, 7, 9, -3))

test_that("mark_lollipop() adds a lollipop layer", {
  spec <- glyph(sales, x = fruit, y = sold) |> mark_lollipop()
  mark <- spec$marks[[1]]
  expect_equal(mark$type, "lollipop")
  expect_equal(mark$style$size, 5)
  compiled <- compile(spec)$spec$marks[[1]]
  expect_equal(compiled$encoding$y$field, "sold")
  expect_equal(compiled$encoding$x$type, "nominal")
})

A browser test checks what was drawn. The helpers in tests/testthat/helper-browser.R open the plot in headless Chrome through chromote. axisMap() converts drawn positions back to data values through the axis ticks, so the test compares the drawing with the data, not with glyph’s own calculations:

test_that("lollipops are drawn from zero to their value", {
  skip_if_no_browser()
  s <- open_page(save_page(
    glyph(sales, x = fruit, y = sold) |> mark_lollipop() |> interact(tooltip = TRUE, brush = TRUE)
  ))
  js(s, AXIS_JS)
  drawn <- js(s, "(() => { const y = axisMap('y');
    return [...document.querySelectorAll('.glyph-lollipop')].map(p => ({
      base: y(+p.querySelector('line').getAttribute('y1')),
      tip: y(+p.querySelector('line').getAttribute('y2')),
      dot: y(+p.querySelector('circle').getAttribute('cy')) })); })()")
  expect_length(drawn, 4)
  expect_equal(vapply(drawn, `[[`, numeric(1), "base"), rep(0, 4), tolerance = 0.05)
  expect_equal(vapply(drawn, `[[`, numeric(1), "tip"), sales$sold, tolerance = 0.05)
  expect_equal(vapply(drawn, `[[`, numeric(1), "dot"), sales$sold, tolerance = 0.05)
  expect_length(page_errors(s), 0)
})

Browser tests skip on CRAN and wherever chromote or Chrome is missing. The browser-tests and R CMD check workflows run them on every pull request.

Then try the mark by hand:

devtools::load_all()
glyph(sales, x = fruit, y = sold) |>
  mark_lollipop(color = sold > 0) |>
  interact(tooltip = TRUE, hover = "enlarge", brush = TRUE, zoom = TRUE)

Hover a dot, brush a few lollipops, clear the selection, zoom and reset, and resize the window. Add facet() and try it again.

Marks that summarise data

Some marks draw a summary of the data rather than one element per row, as boxplots and histograms do. glyph computes those summaries in R when the plot is compiled (#6), so the browser only draws what it is sent. To add one:

  1. Pass stat to add_mark(), for example stat = list(type = "errorbar", level = 0.95). Settings for the statistic go in the same list.
  2. Write a stat_errorbar() function in R/stats.R and add it to the switch in compute_stat(). It gets the mark’s rows after mappings are resolved and missing values dropped. It returns one record per summary element, with the encoding it should be drawn with.
  3. Give each record a .rows field: the 0-based indices of the rows it summarises. Brushing and linked selections use it to connect a summary to the rows behind it.
  4. Group by facet fields as well as by x and color, so each facet panel gets its own summaries. The helpers in R/stats.R do this.
  5. In glyph.js, add a markExtent() branch for the values the summary draws, and a drawing function as in step 4. Register the elements with addStatInteractions(), which sets up tooltips, hover, click and a brush that selects by bounding box.

mark_polygon() uses the same route without computing a statistic. It collects its vertex rows into one record per shape, and gives the brush its own hit test for the shape. Read stat_polygon() and drawPolygon() for a complete, short example.

Limits

Adding a mark means editing four places in glyph itself: R/marks.R, drawMark() and a drawing function in glyph.js, markExtent() when the axes need it, and R/stats.R for a summarising mark. You can’t do it from another package or from your own script:

Extending glyph from outside would need at least:

None of these exist yet.

Some of what this guide relies on is more settled than the rest. Likely to stay:

Likely to change:

A few features don’t know about new marks:

Checklist