HDR images

High Dynamic Range (HDR) images contain information about the luminosity in a scene that can be well outside the range of standard file formats like PNG or JPEG.

In non-HDR images (Low Dynamic Range (LDR) images), the value for any particular pixel usually lie in the range [0, 1] or [0, 255]. In HDR images, there is no real limit to the upper value.

Other considerations for HDR image

There may be more than 4 channels of data. EXR images are often used to stored extra information such as rendering depth, UV texture coordinates, coordinates etc
It will often be necessary to extract the channels of interest for plotting. In the EXR image format, channels are stored alphabetically, which means that careful selection will be needed to extract RGBA channels in order.

Tone mapping is the process of manipulating the HDR values into a lower dynamic range - usually with values in the range [0, 1].

There is no “correct” tone mapping operation - just different techniques depending on your requirements.

Example image

When an EXR image is loaded it is just a numeric array of data.

file <- system.file("image/rstats.exr", package = "picohdr")
im <- picohdr::read_exr(file)
dim(im)
#> [1] 270 360  25

The names on the array indicate the channel names. EXR stored all channels in alphabetical order.

Here there are RGBA channels but also extra information from the 3D renderer which created this image e.g. the u and v texture coordinates at each output pixel.

dimnames(im)[[3]]
#>  [1] "Albedo.B"           "Albedo.G"           "Albedo.R"          
#>  [4] "B"                  "G"                  "N.X"               
#>  [7] "N.Y"                "N.Z"                "Ns.X"              
#> [10] "Ns.Y"               "Ns.Z"               "P.X"               
#> [13] "P.Y"                "P.Z"                "R"                 
#> [16] "RelativeVariance.B" "RelativeVariance.G" "RelativeVariance.R"
#> [19] "Variance.B"         "Variance.G"         "Variance.R"        
#> [22] "dzdx"               "dzdy"               "u"                 
#> [25] "v"

A peek at the first plane in the array shows that it is just numeric data

im[1:5, 1:5, 1]
#>           [,1]      [,2]      [,3]      [,4]      [,5]
#> [1,] 0.4577637 0.4567871 0.4526367 0.4565430 0.4577637
#> [2,] 0.4567871 0.4560547 0.4548340 0.4538574 0.4550781
#> [3,] 0.4560547 0.4521484 0.4553223 0.4533691 0.4553223
#> [4,] 0.4562988 0.4567871 0.4541016 0.4521484 0.4543457
#> [5,] 0.4548340 0.4567871 0.4548340 0.4528809 0.4577637

We can create a standard RGB array from this image data

rgb_arr <- im[, , c('R', 'G', 'B')]

If this was a normal image loaded from PNG or JPEG, we could do the following to view it in R - but this code will cause an error with an HDR image as there is no guarantee that the pixel values lie between 0 and 1 (which is what as.raster() requires)

plot(as.raster(rgb_arr))
#> Error in rgb(t(x[, , 1L]), t(x[, , 2L]), t(x[, , 3L]), maxColorValue = max): color intensity 1.00977, not in [0,1]

library(ggplot2)
df <- array_to_df(rgb_arr)

ggplot(df) + 
  geom_density(aes(value, group = channel, colour = channel)) + 
  theme_bw() + 
  theme(legend.position = 'bottom') + 
  coord_cartesian(xlim = c(0, 1.5), ylim = c(0, 5)) +
  scale_color_manual(values = c('blue', 'green', 'red')) +
  labs(
    title = "Raw HDR pixel values",
    subtitle = "Some values greater than 1.0 in this image"
  )

Tone mapping is then the process by which these pixel values can be shifted, truncated, adapted to squeeze the pixel values into the range [0, 1] so that we can view it properly in our low dynamic range R session.

Tone-map by clamping

A simple technique for tone-mapping is to just clamp the values at [0, 1] with values outside this range being pulled back to these limits.

If we do this, then the image is now viewable as a raster, but some parts of it look blown out and overexposed.

rgb_clamped <- rgb_arr |>
  adj_clamp(lo = 0, hi = 1) |>
  adj_gamma()

oldpar <- par(mai = c(0, 0, 0, 0))
plot(as.raster(rgb_clamped))

par(oldpar)

Tone-map by linear rescaling

rgb_rescale <- rgb_arr |>
  adj_rescale(lo = 0, hi = 1) |>
  adj_gamma()

oldpar <- par(mai = c(0, 0, 0, 0))
plot(as.raster(rgb_rescale))

par(oldpar)

Tone-map with Reinhard’s technique

At its core, Reinhard’s technique is a non-linear rescaling of the values back into the range [0, 1].

rgb_rh1<- rgb_arr |>
  tm_reinhard() |>
  adj_gamma() 

oldpar <- par(mai = c(0, 0, 0, 0))
plot(as.raster(rgb_rh1))

par(oldpar)

Tone-map with Reinhard’s technique (extended)

This basic version of Reinhard’s technique does uses a simpler version of the algorithm. Results may be a bit darker and/or washed out.

rgb_rh2 <- rgb_arr |>
  tm_reinhard_basic() |>
  adj_gamma() 

oldpar <- par(mai = c(0, 0, 0, 0))
plot(as.raster(rgb_rh2))

par(oldpar)

Tone-map with Reinhard’s technique (variant)

This variant on Reinhard’s technique is a hybrid between the standard and basic techniques.

rgb_rh3 <- rgb_arr |>
  tm_reinhard_variant() |>
  adj_gamma() 

oldpar <- par(mai = c(0, 0, 0, 0))
plot(as.raster(rgb_rh3))

par(oldpar)