Getting Started with evoFE

What is evoFE?

evoFE (Evolutionary Feature Engineering) uses a genetic algorithm to automatically discover useful feature transformations for tabular data. Instead of manually crafting interaction terms, ratios, or binning strategies, you let evolution explore the space of possible transformations and keep the ones that improve predictive performance.

The result is an evo_recipe – a reusable transformation pipeline that can be applied to new data at prediction time.

How it works

  1. Initialisation – A population of individuals is created. Each individual is a “recipe” containing a set of feature transformations (genes) and an active feature mask that controls which raw input columns are visible.
  2. Evaluation – Every individual is scored via cross-validated or split model performance (LightGBM, XGBoost, or any custom evaluator).
  3. Selection – The top 50 % survive to breed.
  4. Breeding – Survivors are combined (crossover) and randomly altered (mutation) to produce the next generation. Mutations may add/remove/modify genes or toggle which raw features are active.
  5. Repeat – The cycle continues until the fitness plateaus or the generation budget is exhausted.

Installation

# Install the released version from CRAN
install.packages("evoFE")

# Or install the development version directly from GitHub
# devtools::install_github("tanopereira/evoFE")

Quick Start – Binary Classification

Let’s classify whether a car has an automatic or manual transmission using the mtcars dataset.

library(evoFE)

data(mtcars)
df <- mtcars
df$am <- as.integer(df$am)   # target: 0 = automatic, 1 = manual

set.seed(42)
res <- evolve_features(
  data      = df,
  target_col = "am",
  task       = "classification",
  evaluator  = "xgboost",
  generations = 5,
  pop_size    = 8,
  cv_folds    = 3,
  early_stopping_generations = 3,
  verbose     = TRUE
)
#> Starting Evolutionary Feature Engineering...
#>   Task: classification
#>   Evaluators (1 islands): xgboost
#>   Generations: 5, Population Size: 8, CV Folds: 3
#>   Original Numeric columns: mpg, cyl, disp, hp, drat, wt, qsec, vs, gear, carb
#>   Original Categorical columns:
#> 
#> --- Generation 0 (Baseline) ---
#>   Individual 1: [Original features only] (active: 10/10 raw, 0 genes)
#>   Tested Individual 1 -> Fitness: 0.7147
#> 
#> --- Generation 1 / 5 (Current Best Fitness: 0.7147) ---
#>   Individual 1 (0.7147): [Original features only] (active: 10/10 raw, 0 genes)
#>   Individual 2 (Unevaluated): [divide(hp, wt), random_projection(gear, drat, vs)] (active: 3/10 raw, 2 genes)
#>   Individual 3 (Unevaluated): [normalized_difference(gear, vs), umap_genie_k5_t0.17_nn16_d0.56(drat, vs, wt, ... + 3 more)] (active: 3/10 raw, 2 genes)
#>   Individual 4 (Unevaluated): [log_binning3(qsec), lumbermark_k3(mpg, cyl)] (active: 4/10 raw, 2 genes)
#>   Individual 5 (Unevaluated): [reciprocal(hp), umap_lumbermark_k2_nn13_d0.96(disp, qsec, vs)] (active: 3/10 raw, 2 genes)
#>   Individual 6 (Unevaluated): [umap1_nn17_d0.82(wt, vs), umap2_nn17_d0.82(wt, vs)] (active: 2/10 raw, 2 genes)
#>   Individual 7 (Unevaluated): [sqrt(gear), log_binning_cat6(drat)] (active: 4/10 raw, 2 genes)
#>   Individual 8 (Unevaluated): [sqrt(carb), umap1_nn18_d0.13(mpg, cyl, wt, ... + 2 more), umap2_nn18_d0.13(mpg, cyl, wt, ... + 2 more)] (active: 2/10 raw, 3 genes)
#>   Tested Individual 2 (New Best!) -> Fitness: 0.7153
#>   Tested Individual 3 -> Fitness: 0.6928
#>   Tested Individual 4 -> Fitness: 0.6996
#>   Tested Individual 5 -> Fitness: 0.7007
#>   Tested Individual 6 -> Fitness: 0.6996
#>   Tested Individual 7 -> Fitness: 0.6995
#>   Tested Individual 8 -> Fitness: 0.6980
#>   Gen 1 Best Fitness: 0.7153
#>   Gen 1 Best Recipe: [divide(hp, wt), random_projection(gear, drat, vs)] (active: 3/10 raw, 2 genes)
#> 
#> --- Generation 2 / 5 (Current Best Fitness: 0.7153) ---
#>   Individual 1 (0.7153): [divide(hp, wt), random_projection(gear, drat, vs)] (active: 3/10 raw, 2 genes)
#>   Individual 2 (Unevaluated): [random_projection(gear, drat, vs)] (active: 3/10 raw, 1 gene)
#>   Individual 3 (Unevaluated): [reciprocal(hp), umap_lumbermark_k2_nn13_d0.96(disp, qsec, vs)] (active: 10/10 raw, 2 genes)
#>   Individual 4 (Unevaluated): [umap1_nn17_d0.82(wt, vs), umap2_nn17_d0.82(wt, vs)] (active: 4/10 raw, 2 genes)
#>   Individual 5 (0.7147): [Original features only] (active: 10/10 raw, 0 genes)
#>   Individual 6 (Unevaluated): [divide(hp, wt), random_projection(gear, drat, vs), dlog383.84(wt)] (active: 10/10 raw, 3 genes)
#>   Individual 7 (Unevaluated): [umap1_nn17_d0.82(wt, vs), umap2_nn17_d0.82(wt, ump_20f14c)] (active: 2/10 raw, 2 genes)
#>   Individual 8 (Unevaluated): [umap_lumbermark_k2_nn13_d0.96(disp, qsec, vs), divide(hp, wt)] (active: 3/10 raw, 2 genes)
#>   Tested Individual 2 -> Fitness: 0.7026
#>   Tested Individual 3 -> Fitness: 0.7145
#>   Tested Individual 4 -> Fitness: 0.6982
#>   Tested Individual 6 -> Fitness: 0.7147
#>   Tested Individual 7 -> Fitness: 0.6842
#>   Tested Individual 8 -> Fitness: 0.7136
#>   Gen 2 Best Fitness: 0.7153
#>   Gen 2 Best Recipe: [divide(hp, wt), random_projection(gear, drat, vs)] (active: 3/10 raw, 2 genes)
#> 
#> --- Generation 3 / 5 (Current Best Fitness: 0.7153) ---
#>   Individual 1 (0.7153): [divide(hp, wt), random_projection(gear, drat, vs)] (active: 3/10 raw, 2 genes)
#>   Individual 2 (Unevaluated): [divide(hp, wt)] (active: 3/10 raw, 1 gene)
#>   Individual 3 (Unevaluated): [divide(hp, wt), log_binning_cat6(wt)] (active: 3/10 raw, 2 genes)
#>   Individual 4 (Unevaluated): [divide(hp, wt), random_projection(gear, drat, vs), umap_lumbermark_k2_nn13_d0.96(disp, qsec, vs)] (active: 3/10 raw, 3 genes)
#>   Individual 5 (Unevaluated): [divide(hp, wt), sqrt(carb)] (active: 7/10 raw, 2 genes)
#>   Individual 6 (Unevaluated): [Original features only] (active: 6/10 raw, 0 genes)
#>   Individual 7 (Unevaluated): [random_projection(gear, drat, vs)] (active: 3/10 raw, 1 gene)
#>   Individual 8 (Unevaluated): [random_projection(gear, drat, vs), quantile_binning7(wt)] (active: 3/10 raw, 2 genes)
#>   Individual 9 (Unevaluated): [divide(hp, wt), random_projection(gear, drat, vs), umap_lumbermark_k3_nn16_d0.14(hp, disp, qsec, ... + 2 more)] (active: 10/10 raw, 3 genes)
#>   Individual 10 (Unevaluated): [divide(hp, wt), random_projection(gear, drat, vs), lumb_cdist1_k3(rp_728f8e, div_693059), lumb_cdist2_k3(rp_728f8e, div_693059), lumb_cdist3_k3(rp_728f8e, div_693059)] (active: 3/10 raw, 5 genes)
#>   Individual 11 (Unevaluated): [divide(hp, wt), random_projection(gear, drat, vs), truncated_svd1(carb, cyl, wt, ... + 1 more), truncated_svd2(carb, cyl, wt, ... + 1 more)] (active: 10/10 raw, 4 genes)
#>   Individual 12 (Unevaluated): [quantile_binning_cat3(wt)] (active: 10/10 raw, 1 gene)
#>   Tested Individual 2 -> Fitness: 0.7136
#>   Tested Individual 3 -> Fitness: 0.7136
#>   Tested Individual 4 -> Fitness: 0.7153
#>   Tested Individual 5 -> Fitness: 0.7125
#>   Tested Individual 6 (New Best!) -> Fitness: 0.7162
#>   Tested Individual 7 -> Fitness: 0.7026 (cached)
#>   Tested Individual 8 -> Fitness: 0.7026
#>   Tested Individual 9 -> Fitness: 0.7147
#>   Tested Individual 10 -> Fitness: 0.7133
#>   Tested Individual 11 (New Best!) -> Fitness: 0.7203
#>   Tested Individual 12 -> Fitness: 0.7147
#>   Gen 3 Best Fitness: 0.7203
#>   Gen 3 Best Recipe: [divide(hp, wt), random_projection(gear, drat, vs), truncated_svd1(carb, cyl, wt, ... + 1 more), truncated_svd2(carb, cyl, wt, ... + 1 more)] (active: 10/10 raw, 4 genes)
#> 
#> --- Generation 4 / 5 (Current Best Fitness: 0.7203) ---
#>   Individual 1 (0.7203): [divide(hp, wt), random_projection(gear, drat, vs), truncated_svd1(carb, cyl, wt, ... + 1 more), truncated_svd2(carb, cyl, wt, ... + 1 more)] (active: 10/10 raw, 4 genes)
#>   Individual 2 (Unevaluated): [divide(hp, wt), truncated_svd1(carb, cyl, wt, ... + 1 more), truncated_svd2(carb, cyl, wt, ... + 1 more), random_projection(gear, drat, vs), umap_lumbermark_k2_nn13_d0.96(disp, qsec, vs)] (active: 7/10 raw, 5 genes)
#>   Individual 3 (Unevaluated): [divide(hp, wt), random_projection(gear, drat, vs), truncated_svd1(carb, cyl, wt, ... + 1 more), truncated_svd2(carb, cyl, wt, ... + 1 more), mst_score(wt, carb)] (active: 10/10 raw, 5 genes)
#>   Individual 4 (Unevaluated): [genie_k5_t0.90(wt, gear)] (active: 6/10 raw, 1 gene)
#>   Individual 5 (Unevaluated): [divide(hp, wt), truncated_svd2(carb, cyl, wt, ... + 1 more)] (active: 10/10 raw, 2 genes)
#>   Individual 6 (Unevaluated): [umap_lumbermark_k2_nn13_d0.96(disp, qsec, vs)] (active: 7/10 raw, 1 gene)
#>   Individual 7 (Unevaluated): [divide(hp, wt), random_projection(gear, drat, vs), umap_lumbermark_k2_nn13_d0.96(disp, qsec, vs), truncated_svd1(carb, cyl, wt, ... + 1 more), truncated_svd2(carb, cyl, wt, ... + 1 more), rank_transform(wt)] (active: 10/10 raw, 6 genes)
#>   Individual 8 (Unevaluated): [random_projection(gear, drat, vs), divide(hp, wt)] (active: 3/10 raw, 2 genes)
#>   Tested Individual 2 (New Best!) -> Fitness: 0.7210
#>   Tested Individual 3 -> Fitness: 0.7197
#>   Tested Individual 4 -> Fitness: 0.7162
#>   Tested Individual 5 -> Fitness: 0.7150
#>   Tested Individual 6 -> Fitness: 0.7150
#>   Tested Individual 7 -> Fitness: 0.7203
#>   Tested Individual 8 -> Fitness: 0.7153
#>   Gen 4 Best Fitness: 0.7210
#>   Gen 4 Best Recipe: [divide(hp, wt), truncated_svd1(carb, cyl, wt, ... + 1 more), truncated_svd2(carb, cyl, wt, ... + 1 more), random_projection(gear, drat, vs), umap_lumbermark_k2_nn13_d0.96(disp, qsec, vs)] (active: 7/10 raw, 5 genes)
#> 
#> --- Generation 5 / 5 (Current Best Fitness: 0.7210) ---
#>   Individual 1 (0.7210): [divide(hp, wt), truncated_svd1(carb, cyl, wt, ... + 1 more), truncated_svd2(carb, cyl, wt, ... + 1 more), random_projection(gear, drat, vs), umap_lumbermark_k2_nn13_d0.96(disp, qsec, vs)] (active: 7/10 raw, 5 genes)
#>   Individual 2 (Unevaluated): [divide(hp, wt), random_projection(gear, drat, vs), truncated_svd1(carb, cyl, wt, ... + 1 more), truncated_svd2(carb, cyl, wt, ... + 1 more), mst_score(wt, carb)] (active: 10/10 raw, 5 genes)
#>   Individual 3 (Unevaluated): [random_projection(gear, drat, vs), divide(hp, gear), umap_lumbermark_k2_nn13_d0.96(disp, qsec, vs), truncated_svd2(carb, cyl, wt, ... + 1 more)] (active: 9/10 raw, 4 genes)
#>   Individual 4 (Unevaluated): [divide(hp, wt), random_projection(gear, drat, vs), umap_lumbermark_k2_nn13_d0.96(disp, qsec, vs), truncated_svd1(carb, cyl, wt, ... + 1 more), truncated_svd2(carb, cyl, wt, ... + 1 more), rank_transform(wt), pow3(wt)] (active: 10/10 raw, 7 genes)
#>   Individual 5 (0.7203): [divide(hp, wt), random_projection(gear, drat, vs), truncated_svd1(carb, cyl, wt, ... + 1 more), truncated_svd2(carb, cyl, wt, ... + 1 more)] (active: 10/10 raw, 4 genes)
#>   Individual 6 (Unevaluated): [divide(hp, wt), random_projection(gear, drat, vs), truncated_svd2(carb, cyl, wt, ... + 1 more), rank_transform(wt), truncated_svd1(carb, cyl, wt, ... + 1 more)] (active: 9/10 raw, 5 genes)
#>   Individual 7 (Unevaluated): [divide(hp, wt), random_projection(gear, drat, vs), truncated_svd1(carb, cyl, wt, ... + 1 more), mst_score(wt, carb)] (active: 8/10 raw, 4 genes)
#>   Individual 8 (Unevaluated): [truncated_svd1(carb, cyl, wt, ... + 1 more), truncated_svd2(carb, cyl, wt, ... + 1 more), random_projection(gear, drat, vs), umap_lumbermark_k2_nn13_d0.96(disp, qsec, vs)] (active: 5/10 raw, 4 genes)
#>   Tested Individual 2 -> Fitness: 0.7197 (cached)
#>   Tested Individual 3 -> Fitness: 0.7140
#>   Tested Individual 4 -> Fitness: 0.7203
#>   Tested Individual 6 -> Fitness: 0.7203
#>   Tested Individual 7 -> Fitness: 0.7196
#>   Tested Individual 8 (New Best!) -> Fitness: 0.7211
#>   Gen 5 Best Fitness: 0.7211
#>   Gen 5 Best Recipe: [truncated_svd1(carb, cyl, wt, ... + 1 more), truncated_svd2(carb, cyl, wt, ... + 1 more), random_projection(gear, drat, vs), umap_lumbermark_k2_nn13_d0.96(disp, qsec, vs)] (active: 5/10 raw, 4 genes)
#> 
#> Evolution Complete. Best Validation Score: 0.7211
#> Best recipe: [truncated_svd1(carb, cyl, wt, ... + 1 more), truncated_svd2(carb, cyl, wt, ... + 1 more), random_projection(gear, drat, vs), umap_lumbermark_k2_nn13_d0.96(disp, qsec, vs)] (active: 5/10 raw, 4 genes)
#> Generated columns: svd_c46bbf, svd_570a2a, rp_728f8e, ulm_2882e9
#> Training final model on full dataset...

The returned evo_recipe object contains the best individual (feature recipe), the fitted model, and the evolution history.

# Print high-level overview of the recipe
print(res)
#> An evoFE Recipe
#>   Evaluator:    xgboost
#>   Task:         classification
#>   Metric:       default
#>   Validation Score: 0.7211
#>   Evolved Features: 4
#>   Winning Recipe:
#>     [1] truncated_svd1(carb, cyl, wt, ... + 1 more) -> svd_c46bbf
#>     [2] truncated_svd2(carb, cyl, wt, ... + 1 more) -> svd_570a2a
#>     [3] random_projection(gear, drat, vs) -> rp_728f8e
#>     [4] umap_lumbermark_k2_nn13_d0.96(disp, qsec, vs) -> ulm_2882e9

# View a detailed structured summary
summary(res)
#> === Evolutionary Feature Engineering Summary ===
#> ML Evaluator:             xgboost
#> Task type:                classification
#> Optimization Metric:      default
#> Validation Metric Score:  0.721096
#> Number of Evolved Features: 4
#> 
#> Feature Transformation Details:
#>        Transformer              Inputs     Output
#>      truncated_svd carb, cyl, wt, qsec svd_c46bbf
#>      truncated_svd carb, cyl, wt, qsec svd_570a2a
#>  random_projection      gear, drat, vs  rp_728f8e
#>    umap_lumbermark      disp, qsec, vs ulm_2882e9

Applying the recipe to new data

predict() applies the evolved transformations to new data and returns the engineered feature matrix:

engineered <- predict(res, df[1:5, ])
head(engineered)
#>      mpg  disp    hp    wt  gear svd_c46bbf svd_570a2a rp_728f8e ulm_2882e9
#>    <num> <num> <num> <num> <num>      <num>      <num>     <num>     <fctr>
#> 1:  21.0   160   110 2.620     4  -18.11833  0.6446314  2.076332          1
#> 2:  21.0   160   110 2.875     4  -18.67409  0.4928655  2.076332          1
#> 3:  22.8   108    93 2.320     4  -18.90399 -3.3689989  1.447684          1
#> 4:  21.4   258   110 3.215     3  -20.47404 -2.1557412  1.086996          2
#> 5:  18.7   360   175 3.440     3  -19.11326  0.7419507  1.731097          2

predict_model() goes one step further – it applies the transformations and runs the trained model to produce predictions:

preds <- predict_model(res, df[1:5, ])
preds
#> [1] 0.96124077 0.95224512 0.91079277 0.02653036 0.05989696

Regression

Predict petal length from the iris dataset:

data(iris)

set.seed(123)
res_reg <- evolve_features(
  data       = iris[, c("Sepal.Length", "Sepal.Width", "Petal.Width", "Petal.Length")],
  target_col = "Petal.Length",
  task       = "regression",
  evaluator  = "xgboost",
  generations = 5,
  pop_size    = 8,
  cv_folds    = 3,
  early_stopping_generations = 3,
  verbose     = TRUE
)
#> Starting Evolutionary Feature Engineering...
#>   Task: regression
#>   Evaluators (1 islands): xgboost
#>   Generations: 5, Population Size: 8, CV Folds: 3
#>   Original Numeric columns: Sepal.Length, Sepal.Width, Petal.Width
#>   Original Categorical columns:
#> 
#> --- Generation 0 (Baseline) ---
#>   Individual 1: [Original features only] (active: 3/3 raw, 0 genes)
#>   Tested Individual 1 -> Fitness: -0.3333
#> 
#> --- Generation 1 / 5 (Current Best Fitness: -0.3333) ---
#>   Individual 1 (-0.3333): [Original features only] (active: 3/3 raw, 0 genes)
#>   Individual 2 (Unevaluated): [quantile_binning_cat3(Sepal.Width), normalized_difference(Petal.Width, Petal.Width)] (active: 2/3 raw, 2 genes)
#>   Individual 3 (Unevaluated): [genie_k2_t0.31(Petal.Width, Sepal.Length), pca1(Petal.Width, Sepal.Width), pca2(Petal.Width, Sepal.Width)] (active: 2/3 raw, 3 genes)
#>   Individual 4 (Unevaluated): [quantile_binning3(Sepal.Width), umap_genie_k4_t0.48_nn15_d0.17(Sepal.Width, Petal.Width)] (active: 2/3 raw, 2 genes)
#>   Individual 5 (Unevaluated): [log_binning2(Sepal.Width), pow0.5(Sepal.Length)] (active: 2/3 raw, 2 genes)
#>   Individual 6 (Unevaluated): [log_binning10(Sepal.Width), deadwood(Petal.Width, Sepal.Length)] (active: 2/3 raw, 2 genes)
#>   Individual 7 (Unevaluated): [lumbermark_k4(Petal.Width, Sepal.Width), truncated_svd1(Sepal.Width, Sepal.Length), truncated_svd2(Sepal.Width, Sepal.Length)] (active: 2/3 raw, 3 genes)
#>   Individual 8 (Unevaluated): [reciprocal(Petal.Width), mst_score(Sepal.Width, Petal.Width)] (active: 2/3 raw, 2 genes)
#>   Tested Individual 2 (New Best!) -> Fitness: -0.3259
#>   Tested Individual 3 -> Fitness: -0.3377
#>   Tested Individual 4 -> Fitness: -0.5082
#>   Tested Individual 5 -> Fitness: -0.5081
#>   Tested Individual 6 -> Fitness: -0.3267
#>   Tested Individual 7 -> Fitness: -0.3395
#>   Tested Individual 8 -> Fitness: -0.4895
#>   Gen 1 Best Fitness: -0.3259
#>   Gen 1 Best Recipe: [quantile_binning_cat3(Sepal.Width)] (active: 2/3 raw, 1 gene)
#> 
#> --- Generation 2 / 5 (Current Best Fitness: -0.3259) ---
#>   Individual 1 (-0.3259): [quantile_binning_cat3(Sepal.Width)] (active: 2/3 raw, 1 gene)
#>   Individual 2 (Unevaluated): [Original features only] (active: 2/3 raw, 0 genes)
#>   Individual 3 (Unevaluated): [deadwood(Petal.Width, Sepal.Length), quantile_binning_cat3(Sepal.Width)] (active: 2/3 raw, 2 genes)
#>   Individual 4 (Unevaluated): [quantile_binning_cat3(Sepal.Width), add(Petal.Width, Petal.Width, Petal.Width)] (active: 2/3 raw, 2 genes)
#>   Individual 5 (Unevaluated): [pca1(Petal.Width, Sepal.Width), pca2(Petal.Width, Sepal.Width)] (active: 2/3 raw, 2 genes)
#>   Individual 6 (Unevaluated): [truncated_svd1(Petal.Width, Sepal.Length), truncated_svd2(Petal.Width, Sepal.Length)] (active: 2/3 raw, 2 genes)
#>   Individual 7 (Unevaluated): [quantile_binning_cat3(Sepal.Width), deadwood(Petal.Width, Sepal.Length), reciprocal(Petal.Width)] (active: 2/3 raw, 3 genes)
#>   Individual 8 (Unevaluated): [sqrt(Petal.Width)] (active: 2/3 raw, 1 gene)
#>   Tested Individual 2 (New Best!) -> Fitness: -0.3236
#>   Tested Individual 3 -> Fitness: -0.3276
#>   Tested Individual 4 -> Fitness: -0.5081
#>   Tested Individual 5 -> Fitness: -0.3356
#>   Tested Individual 6 -> Fitness: -0.3236
#>   Tested Individual 7 -> Fitness: -0.3276
#>   Tested Individual 8 -> Fitness: -0.3236
#>   Gen 2 Best Fitness: -0.3236
#>   Gen 2 Best Recipe: [Original features only] (active: 2/3 raw, 0 genes)
#> 
#> --- Generation 3 / 5 (Current Best Fitness: -0.3236) ---
#>   Individual 1 (-0.3236): [Original features only] (active: 2/3 raw, 0 genes)
#>   Individual 2 (Unevaluated): [reciprocal(Petal.Width)] (active: 2/3 raw, 1 gene)
#>   Individual 3 (Unevaluated): [reciprocal(Petal.Width), dlog676.41(Petal.Width)] (active: 2/3 raw, 2 genes)
#>   Individual 4 (Unevaluated): [supervised_bgpca1(Petal.Width, Sepal.Length), supervised_bgpca2(Petal.Width, Sepal.Length), supervised_bgpca3(Petal.Width, Sepal.Length), supervised_bgpca4(Petal.Width, Sepal.Length)] (active: 2/3 raw, 4 genes)
#>   Individual 5 (Unevaluated): [normalized_difference(Petal.Width, Petal.Width)] (active: 2/3 raw, 1 gene)
#>   Individual 6 (Unevaluated): [quantile_binning_cat3(Sepal.Width)] (active: 2/3 raw, 1 gene)
#>   Individual 7 (Unevaluated): [quantile_binning_cat3(Sepal.Width), umap_genie_k2_t0.50_nn9_d0.85(Petal.Width, Sepal.Width)] (active: 2/3 raw, 2 genes)
#>   Individual 8 (Unevaluated): [quantile_binning_cat3(Petal.Width)] (active: 2/3 raw, 1 gene)
#>   Tested Individual 2 -> Fitness: -0.3236
#>   Tested Individual 3 -> Fitness: -0.3236
#>   Tested Individual 4 -> Fitness: -0.3276
#>   Tested Individual 5 -> Fitness: -0.3236
#>   Tested Individual 6 -> Fitness: -0.3259
#>   Tested Individual 7 -> Fitness: -0.3259
#>   Tested Individual 8 -> Fitness: -0.3236
#>   Gen 3 Best Fitness: -0.3236
#>   Gen 3 Best Recipe: [Original features only] (active: 2/3 raw, 0 genes)
#> 
#> --- Generation 4 / 5 (Current Best Fitness: -0.3236) ---
#>   Individual 1 (-0.3236): [Original features only] (active: 2/3 raw, 0 genes)
#>   Individual 2 (Unevaluated): [genie_k3_t0.55(Petal.Width, Sepal.Length)] (active: 2/3 raw, 1 gene)
#>   Individual 3 (Unevaluated): [reciprocal(Sepal.Length)] (active: 2/3 raw, 1 gene)
#>   Individual 4 (Unevaluated): [multiply(Petal.Width, Petal.Width)] (active: 2/3 raw, 1 gene)
#>   Individual 5 (Unevaluated): [umap1_nn13_d0.42(Sepal.Width, Petal.Width), umap2_nn13_d0.42(Sepal.Width, Petal.Width)] (active: 2/3 raw, 2 genes)
#>   Individual 6 (Unevaluated): [reciprocal(Petal.Width)] (active: 2/3 raw, 1 gene)
#>   Individual 7 (Unevaluated): [rank_transform(Sepal.Length)] (active: 2/3 raw, 1 gene)
#>   Individual 8 (Unevaluated): [reciprocal(Petal.Width), umap_lumbermark_k2_nn16_d0.6(Petal.Width, rec_9c5690)] (active: 2/3 raw, 2 genes)
#>   Individual 9 (Unevaluated): [quantile_binning_cat7(Sepal.Length)] (active: 2/3 raw, 1 gene)
#>   Individual 10 (Unevaluated): [reciprocal(Petal.Width), add(Petal.Width, rec_9c5690)] (active: 2/3 raw, 2 genes)
#>   Individual 11 (Unevaluated): [add(Sepal.Length, Sepal.Width, Sepal.Length)] (active: 2/3 raw, 1 gene)
#>   Individual 12 (Unevaluated): [quantile_binning4(Petal.Width)] (active: 2/3 raw, 1 gene)
#>   Tested Individual 2 (New Best!) -> Fitness: -0.3231
#>   Tested Individual 3 -> Fitness: -0.3236
#>   Tested Individual 4 -> Fitness: -0.3236
#>   Tested Individual 5 (New Best!) -> Fitness: -0.3135
#>   Tested Individual 6 -> Fitness: -0.3236 (cached)
#>   Tested Individual 7 -> Fitness: -0.3236
#>   Tested Individual 8 -> Fitness: -0.3237
#>   Tested Individual 9 -> Fitness: -0.3236
#>   Tested Individual 10 -> Fitness: -0.3236
#>   Tested Individual 11 -> Fitness: -0.3236
#>   Tested Individual 12 -> Fitness: -0.3236
#>   Gen 4 Best Fitness: -0.3135
#>   Gen 4 Best Recipe: [Original features only] (active: 2/3 raw, 0 genes)
#> 
#> --- Generation 5 / 5 (Current Best Fitness: -0.3135) ---
#>   Individual 1 (-0.3135): [Original features only] (active: 2/3 raw, 0 genes)
#>   Individual 2 (Unevaluated): [normalized_difference(Petal.Width, Petal.Width)] (active: 2/3 raw, 1 gene)
#>   Individual 3 (Unevaluated): [genie_k3_t0.55(Petal.Width, Sepal.Length)] (active: 2/3 raw, 1 gene)
#>   Individual 4 (Unevaluated): [deadwood(Petal.Width, Sepal.Length)] (active: 2/3 raw, 1 gene)
#>   Individual 5 (Unevaluated): [genie_k3_t0.55(Petal.Width, Sepal.Length), supervised_famd1(gnie_6780bc, Petal.Width, Sepal.Length, ... + 1 more), supervised_famd2(gnie_6780bc, Petal.Width, Sepal.Length, ... + 1 more)] (active: 2/3 raw, 3 genes)
#>   Individual 6 (Unevaluated): [reciprocal(Petal.Width)] (active: 2/3 raw, 1 gene)
#>   Individual 7 (Unevaluated): [quantile_binning_cat8(Petal.Width)] (active: 2/3 raw, 1 gene)
#>   Individual 8 (Unevaluated): [genie_k3_t0.55(Petal.Width, Sepal.Length), famd1(gnie_6780bc, Petal.Width, Sepal.Length, ... + 1 more), famd2(gnie_6780bc, Petal.Width, Sepal.Length, ... + 1 more), famd3(gnie_6780bc, Petal.Width, Sepal.Length, ... + 1 more), famd4(gnie_6780bc, Petal.Width, Sepal.Length, ... + 1 more)] (active: 2/3 raw, 5 genes)
#>   Tested Individual 2 -> Fitness: -0.3236 (cached)
#>   Tested Individual 3 -> Fitness: -0.3231 (cached)
#>   Tested Individual 4 -> Fitness: -0.3267
#>   Tested Individual 5 -> Fitness: -0.3225
#>   Tested Individual 6 -> Fitness: -0.3236 (cached)
#>   Tested Individual 7 -> Fitness: -0.3236
#>   Tested Individual 8 -> Fitness: -0.3136
#>   Gen 5 Best Fitness: -0.3135
#>   Gen 5 Best Recipe: [Original features only] (active: 2/3 raw, 0 genes)
#> 
#> Evolution Complete. Best Validation Score: -0.3135
#> Best recipe: [Original features only] (active: 2/3 raw, 0 genes)
#> Training final model on full dataset...

cat("Best recipe:", individual_to_recipe_string(res_reg$best_individual), "\n")
#> Best recipe: [Original features only] (active: 2/3 raw, 0 genes)
cat("Fitness (neg RMSE):", res_reg$best_individual$fitness, "\n")
#> Fitness (neg RMSE): -0.3135399
preds_reg <- predict_model(res_reg, iris[1:10, ])
# Compare predictions to actuals
data.frame(
  actual    = iris$Petal.Length[1:10],
  predicted = round(preds_reg, 2)
)
#>    actual predicted
#> 1     1.4      1.50
#> 2     1.4      1.47
#> 3     1.3      1.47
#> 4     1.5      1.35
#> 5     1.4      1.46
#> 6     1.7      1.55
#> 7     1.4      1.36
#> 8     1.5      1.46
#> 9     1.4      1.35
#> 10    1.5      1.47

Multiclass Classification

Classify iris species (3 classes). Note task = "multiclass":

iris_mc <- iris
iris_mc$Species <- as.character(iris_mc$Species)

set.seed(99)
res_mc <- evolve_features(
  data       = iris,
  target_col = "Species",
  task       = "multiclass",
  evaluator  = "xgboost",
  generations = 5,
  pop_size    = 8,
  cv_folds    = 3,
  early_stopping_generations = 3,
  verbose     = TRUE
)
#> Starting Evolutionary Feature Engineering...
#>   Task: multiclass
#>   Evaluators (1 islands): xgboost
#>   Generations: 5, Population Size: 8, CV Folds: 3
#>   Original Numeric columns: Sepal.Length, Sepal.Width, Petal.Length, Petal.Width
#>   Original Categorical columns:
#> 
#> --- Generation 0 (Baseline) ---
#>   Individual 1: [Original features only] (active: 4/4 raw, 0 genes)
#>   Tested Individual 1 -> Fitness: 0.8405
#> 
#> --- Generation 1 / 5 (Current Best Fitness: 0.8405) ---
#>   Individual 1 (0.8405): [Original features only] (active: 4/4 raw, 0 genes)
#>   Individual 2 (Unevaluated): [supervised_bgpca1(Sepal.Length, Petal.Length), supervised_bgpca2(Sepal.Length, Petal.Length), supervised_bgpca3(Sepal.Length, Petal.Length), supervised_bgpca4(Sepal.Length, Petal.Length)] (active: 2/4 raw, 4 genes)
#>   Individual 3 (Unevaluated): [lumb_cdist1_k4(Petal.Length, Sepal.Width), lumb_cdist2_k4(Petal.Length, Sepal.Width), lumb_cdist3_k4(Petal.Length, Sepal.Width), lumb_cdist4_k4(Petal.Length, Sepal.Width)] (active: 2/4 raw, 4 genes)
#>   Individual 4 (Unevaluated): [normalized_difference(Petal.Length, Petal.Length), deadwood(Sepal.Width, Petal.Length)] (active: 2/4 raw, 2 genes)
#>   Individual 5 (Unevaluated): [umap_lumbermark_k4_nn13_d0.05(Sepal.Length, Petal.Length), umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width)] (active: 2/4 raw, 2 genes)
#>   Individual 6 (Unevaluated): [pow2(Sepal.Width), truncated_svd1(Petal.Width, Sepal.Length), truncated_svd2(Petal.Width, Sepal.Length)] (active: 2/4 raw, 3 genes)
#>   Individual 7 (Unevaluated): [deadwood(Petal.Width, Sepal.Length), subtract(Sepal.Width, Petal.Length)] (active: 2/4 raw, 2 genes)
#>   Individual 8 (Unevaluated): [pca1(Petal.Width, Sepal.Width), pca2(Petal.Width, Sepal.Width)] (active: 2/4 raw, 2 genes)
#>   Tested Individual 2 -> Fitness: 0.8388
#>   Tested Individual 3 (New Best!) -> Fitness: 0.8432
#>   Tested Individual 4 (New Best!) -> Fitness: 0.8477
#>   Tested Individual 5 -> Fitness: 0.8437
#>   Tested Individual 6 -> Fitness: 0.8000
#>   Tested Individual 7 (New Best!) -> Fitness: 0.8477
#>   Tested Individual 8 -> Fitness: 0.8293
#>   Gen 1 Best Fitness: 0.8477
#>   Gen 1 Best Recipe: [deadwood(Petal.Width, Sepal.Length)] (active: 2/4 raw, 1 gene)
#> 
#> --- Generation 2 / 5 (Current Best Fitness: 0.8477) ---
#>   Individual 1 (0.8477): [deadwood(Petal.Width, Sepal.Length)] (active: 2/4 raw, 1 gene)
#>   Individual 2 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd1(ug_ac976b, Petal.Length, Petal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), famd3(ug_ac976b, Petal.Length, Petal.Width), famd4(ug_ac976b, Petal.Length, Petal.Width)] (active: 2/4 raw, 5 genes)
#>   Individual 3 (Unevaluated): [deadwood(Petal.Width, Sepal.Length), quantile_binning3(Petal.Length)] (active: 2/4 raw, 2 genes)
#>   Individual 4 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width)] (active: 2/4 raw, 1 gene)
#>   Individual 5 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), groupby_ratio(ug_ac976b, Petal.Length)] (active: 2/4 raw, 2 genes)
#>   Individual 6 (Unevaluated): [deadwood(Petal.Width, Sepal.Length), deadwood(Sepal.Width, Petal.Length)] (active: 2/4 raw, 2 genes)
#>   Individual 7 (Unevaluated): [deadwood(Petal.Width, Sepal.Length), multiply(Petal.Length, Petal.Length, Petal.Length)] (active: 2/4 raw, 2 genes)
#>   Individual 8 (Unevaluated): [deadwood(Petal.Width, Petal.Length)] (active: 2/4 raw, 1 gene)
#>   Tested Individual 2 (New Best!) -> Fitness: 0.8841
#>   Tested Individual 3 -> Fitness: 0.8477
#>   Tested Individual 4 -> Fitness: 0.8478
#>   Tested Individual 5 -> Fitness: 0.8570
#>   Tested Individual 6 -> Fitness: 0.8479
#>   Tested Individual 7 -> Fitness: 0.8477
#>   Tested Individual 8 -> Fitness: 0.8477
#>   Gen 2 Best Fitness: 0.8841
#>   Gen 2 Best Recipe: [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width)] (active: 2/4 raw, 2 genes)
#> 
#> --- Generation 3 / 5 (Current Best Fitness: 0.8841) ---
#>   Individual 1 (0.8841): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width)] (active: 2/4 raw, 2 genes)
#>   Individual 2 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), groupby_ratio(ug_ac976b, Petal.Length), lumbermark_k2(Petal.Length, Petal.Width)] (active: 2/4 raw, 3 genes)
#>   Individual 3 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), deadwood(Sepal.Width, Petal.Length)] (active: 2/4 raw, 3 genes)
#>   Individual 4 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width)] (active: 2/4 raw, 1 gene)
#>   Individual 5 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), groupby_ratio(ug_ac976b, Petal.Length), deadwood(Petal.Width, Sepal.Length)] (active: 2/4 raw, 3 genes)
#>   Individual 6 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), log(Petal.Length)] (active: 2/4 raw, 2 genes)
#>   Individual 7 (Unevaluated): [deadwood(Petal.Width, Sepal.Length), deadwood(Sepal.Width, Petal.Length)] (active: 2/4 raw, 2 genes)
#>   Individual 8 (Unevaluated): [Original features only] (active: 2/4 raw, 0 genes)
#>   Tested Individual 2 -> Fitness: 0.8570
#>   Tested Individual 3 -> Fitness: 0.8814
#>   Tested Individual 4 -> Fitness: 0.8478 (cached)
#>   Tested Individual 5 -> Fitness: 0.8578
#>   Tested Individual 6 -> Fitness: 0.8478
#>   Tested Individual 7 -> Fitness: 0.8479 (cached)
#>   Tested Individual 8 -> Fitness: 0.8477
#>   Gen 3 Best Fitness: 0.8841
#>   Gen 3 Best Recipe: [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width)] (active: 2/4 raw, 2 genes)
#> 
#> --- Generation 4 / 5 (Current Best Fitness: 0.8841) ---
#>   Individual 1 (0.8841): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width)] (active: 2/4 raw, 2 genes)
#>   Individual 2 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), deadwood(Sepal.Width, Petal.Length), famd2(dwd_dc0e0f, Petal.Length, Petal.Width)] (active: 2/4 raw, 3 genes)
#>   Individual 3 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), groupby_ratio(ug_ac976b, Petal.Length), lumbermark_k2(Petal.Length, Petal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), deadwood(Sepal.Width, Petal.Length), genie_k5_t0.47(Petal.Width, Petal.Length)] (active: 2/4 raw, 6 genes)
#>   Individual 4 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), groupby_ratio(ug_ac976b, Petal.Length), lumbermark_k2(Petal.Length, Petal.Width)] (active: 3/4 raw, 3 genes)
#>   Individual 5 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width, Petal.Width), deadwood(Sepal.Width, Petal.Length)] (active: 2/4 raw, 2 genes)
#>   Individual 6 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), groupby_ratio(ug_ac976b, Petal.Length), deadwood(Petal.Width, Sepal.Length), famd2(ug_ac976b, Petal.Length, Petal.Width), deadwood(Sepal.Width, Petal.Length)] (active: 2/4 raw, 5 genes)
#>   Individual 7 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), deadwood(Sepal.Width, Petal.Length), deadwood(Petal.Width, Sepal.Length)] (active: 2/4 raw, 4 genes)
#>   Individual 8 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), minhash_encode3(ug_ac976b)] (active: 2/4 raw, 3 genes)
#>   Individual 9 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), dlog793.55(Sepal.Length)] (active: 2/4 raw, 3 genes)
#>   Individual 10 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), groupby_ratio(ug_ac976b, Petal.Length), deadwood(Petal.Width, Sepal.Length), genie_k2_t0.71(Petal.Length, gbr_927cce)] (active: 2/4 raw, 4 genes)
#>   Individual 11 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), truncated_svd1(famd_4ba13e, Sepal.Length), truncated_svd2(famd_4ba13e, Sepal.Length)] (active: 2/4 raw, 4 genes)
#>   Individual 12 (Unevaluated): [deadwood(Petal.Width, Sepal.Length), deadwood(Sepal.Width, Petal.Length), groupby_max(dwd_3f517f, Sepal.Length)] (active: 2/4 raw, 3 genes)
#>   Tested Individual 2 -> Fitness: 0.8632
#>   Tested Individual 3 -> Fitness: 0.8791
#>   Tested Individual 4 -> Fitness: 0.8476
#>   Tested Individual 5 -> Fitness: 0.8477
#>   Tested Individual 6 -> Fitness: 0.8786
#>   Tested Individual 7 -> Fitness: 0.8814
#>   Tested Individual 8 -> Fitness: 0.8841
#>   Tested Individual 9 -> Fitness: 0.8841
#>   Tested Individual 10 -> Fitness: 0.8578
#>   Tested Individual 11 -> Fitness: 0.8841
#>   Tested Individual 12 -> Fitness: 0.8479
#>   Gen 4 Best Fitness: 0.8841
#>   Gen 4 Best Recipe: [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width)] (active: 2/4 raw, 2 genes)
#> 
#> --- Generation 5 / 5 (Current Best Fitness: 0.8841) ---
#>   Individual 1 (0.8841): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width)] (active: 2/4 raw, 2 genes)
#>   Individual 2 (Unevaluated): [Original features only] (active: 2/4 raw, 0 genes)
#>   Individual 3 (Unevaluated): [mst_score(Petal.Length, Petal.Width)] (active: 2/4 raw, 1 gene)
#>   Individual 4 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), minhash_encode3(ug_ac976b), dlog410.18(Sepal.Length)] (active: 2/4 raw, 4 genes)
#>   Individual 5 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), deadwood(Sepal.Width, Petal.Length), deadwood(Petal.Width, Sepal.Length)] (active: 3/4 raw, 4 genes)
#>   Individual 6 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), minhash_encode3(ug_ac976b)] (active: 2/4 raw, 3 genes)
#>   Individual 7 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(Sepal.Length, Petal.Length, Petal.Width)] (active: 2/4 raw, 2 genes)
#>   Individual 8 (Unevaluated): [lumb_cdist1_k5(Petal.Length, Petal.Width), lumb_cdist2_k5(Petal.Length, Petal.Width), lumb_cdist3_k5(Petal.Length, Petal.Width), lumb_cdist4_k5(Petal.Length, Petal.Width), lumb_cdist5_k5(Petal.Length, Petal.Width)] (active: 2/4 raw, 5 genes)
#>   Individual 9 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), minhash_encode3(ug_ac976b), sqrt(minhash_3b09a6)] (active: 2/4 raw, 4 genes)
#>   Individual 10 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), truncated_svd1(Petal.Length, Petal.Width), truncated_svd2(Petal.Length, Petal.Width)] (active: 2/4 raw, 4 genes)
#>   Individual 11 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), ohe_other(ug_ac976b)] (active: 2/4 raw, 3 genes)
#>   Individual 12 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), quantile_binning_cat4(Sepal.Width)] (active: 2/4 raw, 3 genes)
#>   Individual 13 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), woe_encode(ug_ac976b)] (active: 2/4 raw, 3 genes)
#>   Individual 14 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), groupby_zscore(ug_ac976b, Sepal.Width)] (active: 2/4 raw, 3 genes)
#>   Individual 15 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), groupby_ratio(ug_ac976b, Petal.Length), deadwood(Petal.Width, Sepal.Length), famd2(ug_ac976b, Petal.Length, Petal.Width), deadwood(Sepal.Width, Petal.Length), concat(ug_ac976b, dwd_3f517f)] (active: 2/4 raw, 6 genes)
#>   Individual 16 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), fh12_bin16(ug_ac976b)] (active: 2/4 raw, 3 genes)
#>   Individual 17 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), groupby_quantile_q0.25(ug_ac976b, Petal.Width)] (active: 2/4 raw, 3 genes)
#>   Individual 18 (Unevaluated): [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), minhash_encode3(ug_ac976b), groupby_sd(ug_ac976b, minhash_3b09a6)] (active: 2/4 raw, 4 genes)
#>   Tested Individual 2 -> Fitness: 0.8477 (cached)
#>   Tested Individual 3 -> Fitness: 0.8487
#>   Tested Individual 4 -> Fitness: 0.8841
#>   Tested Individual 5 -> Fitness: 0.8781
#>   Tested Individual 6 -> Fitness: 0.8841 (cached)
#>   Tested Individual 7 -> Fitness: 0.8394
#>   Tested Individual 8 -> Fitness: 0.8710
#>   Tested Individual 9 -> Fitness: 0.8841
#>   Tested Individual 10 -> Fitness: 0.8832
#>   Tested Individual 11 -> Fitness: 0.8841
#>   Tested Individual 12 -> Fitness: 0.8802
#>   Tested Individual 13 -> Fitness: 0.8841
#>   Tested Individual 14 (New Best!) -> Fitness: 0.8863
#>   Tested Individual 15 -> Fitness: 0.8788
#>   Tested Individual 16 -> Fitness: 0.8841
#>   Tested Individual 17 -> Fitness: 0.8841
#>   Tested Individual 18 -> Fitness: 0.8841
#>   Gen 5 Best Fitness: 0.8863
#>   Gen 5 Best Recipe: [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), groupby_zscore(ug_ac976b, Sepal.Width)] (active: 2/4 raw, 3 genes)
#> 
#> Evolution Complete. Best Validation Score: 0.8863
#> Best recipe: [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), groupby_zscore(ug_ac976b, Sepal.Width)] (active: 2/4 raw, 3 genes)
#> Generated columns: ug_ac976b, famd_4ba13e, gbz_464bb0
#> Training final model on full dataset...

cat("Best recipe:", individual_to_recipe_string(res_mc$best_individual), "\n")
#> Best recipe: [umap_genie_k3_t0.78_nn12_d0.4(Petal.Length, Sepal.Width), famd2(ug_ac976b, Petal.Length, Petal.Width), groupby_zscore(ug_ac976b, Sepal.Width)] (active: 2/4 raw, 3 genes)

For multiclass, predict_model() returns a probability matrix – one column per class:

probs <- predict_model(res_mc, iris_mc[c(1, 51, 101), ])
round(probs, 3)
#>      setosa versicolor virginica
#> [1,]  0.987      0.007     0.006
#> [2,]  0.005      0.989     0.006
#> [3,]  0.006      0.006     0.988

Transformer Reference

evoFE ships with 42 built-in transformers that the genetic algorithm can select from during evolution. The table below groups them by category.

Arithmetic (numeric -> numeric)

Transformer Arity Description
log unary Safe natural logarithm: log1p(abs(x))
sqrt unary Safe square root: sqrt(abs(x))
reciprocal unary 1/x (0 where x == 0)
power unary Signed exponentiation: sign(x) * |x|^p, \(p \in \{0.5, 1/3, 2, 3\}\)
displaced_log unary log1p(|x + d|) where d is sampled from [10, 1000]
add multi Element-wise sum of 2+ columns
subtract binary \(x_1 - x_2\)
multiply multi Element-wise product of 2+ columns
divide binary \(x_1 / x_2\) (0 where denominator is 0)
normalized_difference binary \((x_1 - x_2) / (|x_1| + |x_2| + 10^{-6})\)
log_ratio binary \(\log(1+|x_1|) - \log(1+|x_2|)\)

Rank / Distribution (numeric -> numeric)

Transformer Description
rank_transform ECDF-based percentile rank mapped to [0, 1]; fit on training data, robust to outliers

Group-by Aggregations (mixed cat x num -> numeric)

These combine a categorical grouping column with a numeric value column. All are stateful (fit on training data).

Transformer Description
groupby_mean Per-group mean
groupby_sd Per-group standard deviation
groupby_max Per-group maximum
groupby_min Per-group minimum
groupby_median Per-group median (robust to outliers)
groupby_quantile Per-group Q1 or Q3 (\(q \in \{0.25, 0.75\}\))
groupby_ratio value / group_mean
groupby_zscore \((value - group\_mean) / group\_sd\)

Supervised Categorical Encodings (categorical -> numeric)

All are stateful (fit on training data only – no leakage).

Transformer Description
target_encode Smoothed mean-target encoding for binary classification / regression
pooled_target_encode Empirical Bayes pooled target encoding with dynamic shrinkage based on target variance
target_encode_multiclass Class-wise smoothed target encoding for multiclass tasks
woe_encode Weight of Evidence: ln(P(event|cat) / P(non-event|cat)) with Laplace smoothing; binary classification only

Unsupervised Encoding & Binning

Transformer Input -> Output Description
frequency_encode cat -> num Count of each category level in training data
one_hot_encode cat -> num Binary indicator for up to 5 top categories plus an “other” bucket
concat cat x cat -> cat Concatenates 2 or 3 categorical columns with underscore separator
quantile_binning num -> num Quantile-based bin index (numeric output)
quantile_binning_cat num -> cat Quantile-based bin label (categorical output)
log_binning num -> num Log-scale bin index (numeric output)
log_binning_cat num -> cat Log-scale bin label (categorical output)
datetime_extract date -> num Extracted datetime component: year, month, day, hour, day-of-week, or weekend indicator
date_diff date x date -> num Signed difference in days between two datetime columns

Dimensionality Reduction (numeric -> numeric)

All are stateful (fit on training data).

Transformer Description
pca Selected principal component from prcomp
truncated_svd Selected component from truncated SVD
random_projection Random unit-vector linear combination
umap UMAP projection component (requires uwot)

Manifold & Graph Learning (numeric -> categorical or numeric)

All are stateful. Clustering is fit on a (optionally downsampled) training set; new data is assigned to clusters via 1-NN lookup.

Transformer Output Description
genie categorical Genie robust hierarchical cluster label (requires genieclust)
genie_centroid_dist numeric Distance to each Genie cluster centroid
umap_genie categorical Genie cluster labels on UMAP embedding (requires uwot + genieclust)
lumbermark categorical Lumbermark MST-based hierarchical cluster label (requires lumbermark)
lumbermark_centroid_dist numeric Distance to each Lumbermark cluster centroid
umap_lumbermark categorical Lumbermark cluster labels on UMAP embedding (requires uwot + lumbermark)
mst_score numeric MST-based anomaly score (requires quitefastmst)
deadwood categorical Deadwood outlier indicator (requires deadwood)

Hierarchical Features (Gene Chaining)

One of evoFE’s powerful capabilities is hierarchical feature construction. After a gene has been evaluated and proven useful, subsequent generations can build on top of its output.

For example:

Gen 1: log_ratio(Sepal.Length, Petal.Width)        -> tested [OK]
Gen 2: divide(Petal.Width, logratio(...))            -> chains from tested gene [OK]

Important safety rule: a gene can only chain from outputs that have been evaluated in a previous generation. A brand-new untested gene is never used as input for another gene in the same individual. This prevents fragile dependency chains built on unproven transformations.

Hybrid Active Feature Mask

Each individual in evoFE not only carries a set of transformation genes, but also an active feature mask – a subset of the original raw input columns that the individual sees. This allows the evolutionary search to simultaneously optimise which features to include and how to transform them.

How the mask works

At initialisation, the mask is seeded using the baseline model importances. Features with higher importance (relative to the average) have a higher probability of being included via a sigmoid function:

\[P(\text{include } x_i) = \sigma\!\left(\frac{\text{importance}(x_i) - \bar{\text{importance}}}{\tau}\right)\]

where \(\tau\) is the temperature controlled by mask_temp_factor.

Mask mutation operators

Three mask-specific mutation operators are randomly triggered during each mutation event:

Operator Trigger probability Effect
Recalculate mask recalculate_mask_prob (default 0.05) Redraws the entire mask from scratch using current importances
Toggle raw feature(s) raw_toggle_prob (default 0.15) Activates or deactivates one or more features; count drawn from a geometric distribution
Gene mutation 1 - recalculate_mask_prob - raw_toggle_prob Standard add/remove/modify gene operation

Tuning advice

# More aggressive feature selection (wider exploration)
recipe <- evolve_features(
  data = df, target_col = "y", task = "classification",
  raw_toggle_prob = 0.25,       # toggle features more often
  recalculate_mask_prob = 0.10, # recalculate mask more often
  mask_temp_factor = 1.0        # flatter importance distribution
)

# Conservative -- mostly stick to all features, rely on gene mutations
recipe <- evolve_features(
  data = df, target_col = "y", task = "classification",
  raw_toggle_prob = 0.05,
  recalculate_mask_prob = 0.02,
  mask_temp_factor = 0.3  # sharper -- concentrate on highest-importance features
)

Island Model

The island model partitions the population into independent sub-populations (islands) that evolve in parallel. Periodic migration exchanges successful recipes and genes between islands, helping the overall search escape local optima while preserving diversity.

Recipe-level migration (Ring topology)

Every migration_interval generations, the top migration_rate individuals from each island are copied into the neighbouring island’s population (ring topology). This preserves co-adapted gene interactions.

Gene-level migration (Injection)

In addition to whole-recipe migration, individual genes from each island’s best individual are periodically injected into a neighbour island’s mutation pool. When a mutation event fires on the receiving island, there is a gene_migration_prob chance that a migrated gene is injected instead of a random new gene.

Basic island example

recipe <- evolve_features(
  data = df,
  target_col = "am",
  task = "classification",
  evaluator = "xgboost",
  generations = 10,
  pop_size = 8,
  islands = 4,                  # 4 independent sub-populations
  migration_interval = 5,       # migrate every 5 generations
  migration_rate = 2,           # top-2 individuals migrate
  gene_migration_prob = 0.2,    # 20% chance to inject a migrated gene
  verbose = TRUE
)

Row-split islands

When row_split_islands = TRUE, each island receives a distinct subset of the training rows. This is useful for very large datasets where evaluating every individual on the full dataset is prohibitively slow.

recipe <- evolve_features(
  data = big_df,
  target_col = "y",
  task = "regression",
  islands = 4,
  row_split_islands = TRUE,     # each island sees ~25% of rows
  evaluation_strategy = "split",
  split_ratio = c(0.6, 0.2, 0.2)
)

When combined with per_island_validation = TRUE, each island also uses its own local validation split during the evolutionary search. Only the final tournament (between islands) uses the global validation set. This prevents cross-island data leakage during evolution.

Heterogeneous transformer pools

Each island can search over a different transformer vocabulary:

recipe <- evolve_features(
  data = df,
  target_col = "am",
  task = "classification",
  islands = 2,
  allowed_transformers = list(
    "basic",       # island 1: arithmetic + encodings only
    "clustering"   # island 2: UMAP, Genie, MST, etc.
  )
)

Multi-Evaluator Island Portfolios

Islands can also run different machine learning backends simultaneously. Passing a vector of evaluator names assigns each model family to a dedicated island:

recipe <- evolve_features(
  data = df,
  target_col = "am",
  task = "classification",
  islands = 2,
  evaluator = c("lightgbm", "xgboost"),
  generations = 5,
  pop_size = 6
)

Island 1 discovers features optimized for LightGBM’s histogram-based splitting, while Island 2 discovers features optimized for XGBoost’s exact/approximate greedy trees.

Advanced Topologies & Migration Policies

For large-scale or structured exploration, evoFE provides graph-theoretic island topologies and adaptive probabilistic migration policies via migration_config():

# Torus topology with demand-driven Gibbs pull migration
mig_cfg <- migration_config(
  topology = topology_torus(c(2, 2)), # 4 islands arranged on a 2x2 toroidal grid
  policy = policy_gibbs_pull(),        # stagnant islands actively pull innovations from leading islands
  interval = 4,
  rate = 1,
  gene_prob = 0.25
)

recipe <- evolve_features(
  data = df,
  target_col = "am",
  task = "classification",
  migration = mig_cfg,
  generations = 10
)

Available topologies include: - topology_ring(k): Unidirectional ring (default) - topology_complete(k): Fully connected mesh - topology_torus(dims): N-dimensional torus with wrap-around boundaries - topology_hypercube(dim): \(2^d\)-vertex binary hypercube - topology_grid(dims): N-dimensional Cartesian grid with fixed boundaries - topology_tiered(tiers): Hierarchical fitness-stratified tiers (HFC architecture) - topology_custom(adjacency_matrix): User-defined directed graph

Available migration policies include: - policy_push_uniform(): Classic uniform broadcast to downstream neighbors - policy_gibbs_push(): Softmax-weighted push directing migrants toward lower-fitness islands - policy_gibbs_pull(): Demand-driven pull where stagnated islands pull elite solutions from high-performing neighbors - policy_tiered_admission(): Fitness-gated upward migration through hierarchical tiers

Caruana Post-Hoc Island Ensembling

Multi-island evolution naturally produces diverse, complementary feature recipes across islands. Rather than picking only the single best recipe, ensemble_islands() uses Caruana greedy forward selection with replacement to construct an optimal weighted ensemble across all island champions:

# 1. Evolve features across 3 islands
set.seed(42)
multi_recipe <- evolve_features(
  data = mtcars,
  target_col = "am",
  task = "classification",
  evaluator = "xgboost",
  islands = 3,
  generations = 5,
  pop_size = 6,
  verbose = FALSE
)

# 2. Greedily build the Caruana ensemble from island populations
ensemble <- ensemble_islands(
  multi_recipe,
  data = mtcars,
  max_rounds = 10,
  verbose = TRUE
)

# Inspect the ensemble weights and member recipes
summary(ensemble)

# 3. Predict directly with the ensemble
ensemble_preds <- predict_model(ensemble, mtcars[1:5, ])
ensemble_preds

predict_model.evo_ensemble() automatically routes the new observations through each member recipe’s transformation pipeline and computes the weighted blend of model predictions.

Custom Transformer Registration

evoFE makes it easy to register your own custom transformations. Use create_transformer() to define your transformer, and register_transformer() to make it available during evolution:

library(evoFE)

# 1. Define a transformer that adds a constant to a numeric variable
add_five_trans <- create_transformer(
  name = "add_five",
  type = "unary",
  input_type = "numeric",
  apply_func = function(data, gene, state = NULL) {
    data[[gene$input_cols[1]]] + 5
  },
  name_generator = function(gene) paste0("add5_", gene$input_cols[1])
)

# 2. Register it with the package registry
register_transformer("add_five", add_five_trans)

# Now "add_five" is part of the transformer pool for all future evolution runs.
# You can also restrict to only your custom transformer:
recipe <- evolve_features(
  data = df, target_col = "am", task = "classification",
  allowed_transformers = c("log", "sqrt", "add_five")
)

Stateful (fit-on-train) transformers

If your transformer needs to fit parameters on the training set (e.g., a custom normalisation), provide a fit_func:

z_score_trans <- create_transformer(
  name = "z_score",
  type = "unary",
  input_type = "numeric",
  fit_func = function(data, gene, target_col = NULL) {
    x <- data[[gene$input_cols[1]]]
    list(mean = mean(x, na.rm = TRUE), sd = sd(x, na.rm = TRUE))
  },
  apply_func = function(data, gene, state = NULL) {
    x <- data[[gene$input_cols[1]]]
    if (is.null(state) || state$sd == 0) return(rep(0, length(x)))
    (x - state$mean) / state$sd
  },
  name_generator = function(gene) paste0("zscore_", gene$input_cols[1])
)
register_transformer("z_score", z_score_trans)

Custom Evaluator Registration

evoFE is not limited to LightGBM and XGBoost. You can register any ML backend using register_evaluator():

# Register a simple linear model evaluator backed by glmnet
register_evaluator(
  "my_lm",
  train_func = function(x_train, y_train, x_val = NULL,
                        task = "regression", ...) {
    model <- lm(y ~ ., data = as.data.frame(cbind(y = y_train, x_train)))
    preds <- if (!is.null(x_val)) {
      predict(model, newdata = as.data.frame(x_val))
    } else {
      NULL
    }
    list(model = model, predictions = preds,
         importances = setNames(rep(1, ncol(x_train)), colnames(x_train)))
  },
  predict_func = function(model, x_new, task, ...) {
    predict(model, newdata = as.data.frame(x_new))
  }
)

# Use it in evolution
recipe <- evolve_features(
  data = df, target_col = "y", task = "regression",
  evaluator = "my_lm"
)

Bayesian Hyperparameter Tuning

make_tunable() wraps any registered evaluator in a mlr3mbo Bayesian Optimisation loop that automatically tunes hyperparameters during each fitness evaluation:

# Define parameter search space for XGBoost
param_ranges <- list(
  eta         = list(type = "numeric", lower = 0.01, upper = 0.3),
  max_depth   = list(type = "integer", lower = 3,    upper = 9),
  subsample   = list(type = "numeric", lower = 0.5,  upper = 1.0)
)

# Register a tunable version of xgboost
make_tunable("xgboost", param_ranges, tuner_name = "xgboost_tuned")

# Use it exactly like a regular evaluator
recipe <- evolve_features(
  data = df,
  target_col = "am",
  task = "classification",
  evaluator = "xgboost_tuned",
  generations = 5,
  pop_size = 6,
  mbo_iters = 5,          # Bayesian optimisation iterations per fitness eval
  mbo_init_design = 8     # initial LHS design points
)

evoFE ships two pre-configured tunable evaluators: lightgbm_mbo and xgboost_mbo (registered at package load).

Understanding the Output

evolve_features() returns an evo_recipe S3 object with:

Field Description
best_individual The winning recipe (list of genes, active column sets, fitness)
best_model The final LightGBM/XGBoost model trained on all data
history Full final-generation population (for inspection)
fitness_history Best fitness per generation (for plot())
task The task type used
evaluator The evaluator used
metric The optimization metric used
classes Class labels (multiclass only)

Inspecting the recipe

ind <- res$best_individual

# Human-readable recipe string
cat(individual_to_recipe_string(ind), "\n")
#> [truncated_svd1(carb, cyl, wt, ... + 1 more), truncated_svd2(carb, cyl, wt, ... + 1 more), random_projection(gear, drat, vs), umap_lumbermark_k2_nn13_d0.96(disp, qsec, vs)] (active: 5/10 raw, 4 genes)

# Number of evolved genes
cat("Evolved genes:", length(ind$genes), "\n")
#> Evolved genes: 4

# Original columns retained (active feature mask)
cat("Numeric cols: ", paste(ind$numeric_cols, collapse = ", "), "\n")
#> Numeric cols:  mpg, disp, hp, wt, gear
cat("Categorical cols:", paste(ind$categorical_cols, collapse = ", "), "\n")
#> Categorical cols:

# Individual gene details
for (g in ind$genes) {
  cat(sprintf("  %s(%s) -> %s\n",
    g$transformer_name,
    paste(g$input_cols, collapse = ", "),
    g$output_col))
}
#>   truncated_svd(carb, cyl, wt, qsec) -> svd_c46bbf
#>   truncated_svd(carb, cyl, wt, qsec) -> svd_570a2a
#>   random_projection(gear, drat, vs) -> rp_728f8e
#>   umap_lumbermark(disp, qsec, vs) -> ulm_2882e9

Evaluation Strategies

evoFE supports multiple evaluation strategies and cross-validation partition schemes designed to balance search speed, compute cost, and strict prevention of data leakage:

Cross-Validation vs. Train/Val Splits

  1. Cross-Validation (cv): The default strategy. Evaluates fitness across \(K\) folds (cv_folds parameter).
  2. Train/Validation/Holdout Split (split): Fast evaluation on large datasets using split_ratio = c(0.6, 0.2, 0.2) or explicit split_ids.
split_ids <- sample(
  c("train", "val", "holdout"),
  nrow(df),
  replace = TRUE,
  prob = c(0.6, 0.2, 0.2)
)

recipe <- evolve_features(
  data = df, target_col = "am", task = "classification",
  split_ids = split_ids   # overrides evaluation_strategy -> "split" automatically
)

Confirmation Holdout & Search-Gap Diagnostic

In iterative automated feature engineering, repeatedly evaluating hundreds of candidate pipelines on the same cross-validation folds can lead to search-level overfitting (selecting recipes that fit fold artifacts).

To safeguard against this, holdout_frac sets aside an untouched, stratified holdout sample before the search begins:

recipe <- evolve_features(
  data = df,
  target_col = "am",
  task = "classification",
  evaluator = "xgboost",
  holdout_frac = 0.20, # 20% untouched holdout
  generations = 5,
  pop_size = 8
)

# Inspect the holdout fitness and search gap
cat("Selection CV fitness:", recipe$best_individual$fitness, "\n")
cat("Confirmation holdout:", recipe$holdout_fitness, "\n")
cat("Search gap:          ", recipe$search_gap, "\n")

The search_gap (holdout score minus selection score) provides an unbiased check of generalization. A large negative search gap immediately alerts you to search-level overfitting.

Leakage-Safe Cross-Validation Strategies

Standard random K-fold CV can cause severe optimistic leakage when working with time-series or grouped observations. evoFE provides three fold construction strategies via cv_strategy:

Strategy When to use Behavior
"random" Independent i.i.d. observations Standard stratified random K-fold partitioning
"time" Chronological / time-series data Folds are contiguous chronological blocks ordered by time_col. The validation fold is strictly in the future of the training folds, preventing lookahead leakage.
"group" Grouped / clustered observations (e.g. subjects, patients, stores) All rows sharing a group_col entity are assigned to the same fold using greedy bin-packing, ensuring no subject appears in both train and validation. 
# Temporal data -- chronological forward validation
recipe_time <- evolve_features(
  data = financial_df,
  target_col = "returns",
  task = "regression",
  cv_strategy = "time",
  time_col = "date",
  cv_folds = 5
)

# Grouped data -- entity-isolated folds
recipe_group <- evolve_features(
  data = patient_df,
  target_col = "outcome",
  task = "classification",
  cv_strategy = "group",
  group_col = "patient_id",
  cv_folds = 5
)

Multi-Fidelity Evolutionary Screening

For large datasets, evaluating every candidate generation at full dataset size can be compute-intensive. Enabling multi_fidelity = TRUE uses a warm-up screening schedule:

recipe_fast <- evolve_features(
  data = big_df,
  target_col = "target",
  task = "classification",
  multi_fidelity = TRUE,
  mf_warmup_frac = 0.40,  # first 40% of generations screen on subsamples
  mf_sample_frac = 0.50,  # screen on 50% row subsamples
  generations = 10,
  pop_size = 12
)

During warm-up, candidates are screened rapidly on subsampled folds. Crucially, the top-performing half of candidates is re-evaluated at full fidelity before selection, ensuring that all fitness comparisons remain 100% apples-to-apples.

Alternative and Custom Metrics

By default, evoFE optimises LogLoss (classification) and RMSE (regression). Pass the metric parameter to change this:

Task Supported metrics
classification "default" (LogLoss), "auc", "f1", "ts-refinement"
multiclass "default" (Multiclass LogLoss), "auc", "ts-refinement"
regression "default" (neg RMSE), "mae"
Any Custom function function(y_true, y_pred) returning numeric (higher = better) 
# Optimize AUC
recipe_auc <- evolve_features(
  data = df, target_col = "am", task = "classification",
  metric = "auc", generations = 5, pop_size = 8
)

# Custom metric (MAPE, negated because evoFE maximises)
mape_metric <- function(y_true, y_pred) {
  -mean(abs((y_true - y_pred) / (y_true + 1e-8)))
}

recipe_mape <- evolve_features(
  data = iris[, 1:5], target_col = "Petal.Length", task = "regression",
  metric = mape_metric, generations = 5, pop_size = 8
)

TS-Refinement metric

TS-Refinement is a calibration-aware classification metric that finds the temperature \(T\) minimizing the log-loss of temperature-scaled prediction margins, then reports the unsmoothed log-loss at that temperature:

\[\mathcal{L}_\text{TS} = \min_{T > 0} \, \text{LogLoss}\!\left(\sigma\!\left(\frac{z}{T}\right),\, y\right)\]

It is more discriminating than raw log-loss when models differ mainly in calibration rather than ranking quality. Enable it with:

recipe <- evolve_features(
  data = df, target_col = "am", task = "classification",
  metric = "ts-refinement"
)

You can also call compute_ts_refinement() directly on your own predictions:

score <- compute_ts_refinement(
  y_true = c(0, 1, 1, 0),
  y_pred = c(0.2, 0.8, 0.7, 0.3),
  task   = "classification"
)
cat("TS-Refinement log-loss:", score, "\n")

Parameter Reference

evolve_features() – full parameter table

Parameter Default Description
data - Input data.frame or data.table
target_col - Name of the target column
task "classification" "classification", "multiclass", or "regression"
generations 10 Maximum evolutionary generations
pop_size 10 Individuals per generation
cv_folds 3 CV folds (only with evaluation_strategy = "cv")
evaluation_strategy "cv" "cv" or "split"
split_ratio c(0.6, 0.2, 0.2) Train/Val/Holdout proportions for "split"
split_ids NULL User-supplied split assignments ("train", "val", "holdout")
holdout_frac 0 Stratified holdout fraction frozen during search for unbiased confirmation
cv_strategy "random" CV fold construction: "random", "time", or "group"
time_col NULL Timestamp column used when cv_strategy = "time"
group_col NULL Group entity column used when cv_strategy = "group"
multi_fidelity FALSE Enable two-stage row-subsampling evaluation during warm-up
mf_sample_frac 0.5 Row fraction kept per fold during multi-fidelity screening
mf_warmup_frac 0.5 Fraction of total generations run in multi-fidelity screening mode
early_stopping_generations 3 Stop after N generations without improvement
evaluator "lightgbm" ML backend: "lightgbm", "xgboost", "catboost", "lm", "keras3", or vector of backends per island
seed NULL Integer RNG seed for fully reproducible evolutionary search (CRAN-safe)
dynamic_population TRUE Expand population dynamically during stagnation
dynamic_population_growth_rate 1.5 Growth multiplier during stagnation
dynamic_population_decay_rate 0.7 Decay multiplier when improvement resumes
crossover_type "both" "random", "union", or "both" (50/50 mix)
threads 2 Threads for model training and clustering
max_clustering_size 5000 Max unique rows passed to clustering transformers
verbose TRUE Print progress to console
metric "default" Optimisation metric ("default", "auc", "f1", "mae", "ts-refinement", "cal_rmse", "cal_mae", or custom function)
model_all_final_genes FALSE Train final model on union of all genes in final population
model_all_historical_genes FALSE Train final model on union of all genes across all generations
allowed_transformers "all" Transformer pool: "all", "basic", "robust", "clustering", or a character vector
complexity_penalty 0 Multiplier for parsimony penalty (e.g. 1.0 for standard BIC/PAC-Bayes)
complexity_mode "bic_dynamic" Penalty mode: "bic_dynamic", "bic", "pac_bayes_dynamic", "pac_bayes", or "none"
complexity_floor 0.20 Minimum safety floor (e.g. 20%) for dynamic penalty relaxation
complexity_target "all_features" Target to penalize: "all_features" (rewards raw feature pruning) or "genes"
migration NULL Optional evo_migration_config object from migration_config()
islands 1 Number of independent sub-populations
migration_interval 5 Generations between island migrations
migration_rate 1 Top-N individuals migrated from each island
gene_migration_prob 0.2 Probability of injecting a migrated gene during mutation
migration_topology "ring" Topology: "ring", "gibbs_stagnation", "gibbs_fitness", "dual_gibbs_pull", "random"
row_split_islands FALSE Split training rows across islands
per_island_validation FALSE Use per-island local validation split (requires row_split_islands = TRUE)
raw_toggle_prob 0.15 Probability of toggling raw feature(s) in active mask during mutation
recalculate_mask_prob 0.05 Probability of redrawing entire active mask from importances
mask_temp_factor 0.5 Temperature for importance-guided mask sampling
record FALSE Enable live evolution viewer
port NULL Port for the live viewer server
... - Extra arguments passed to the evaluator’s train_func

Transformer presets for allowed_transformers

Preset # Transformers When to use
"all" 42 Default – full search space
"basic" ~18 Fast runs; arithmetic + key encodings only
"robust" ~24 Outlier-resistant ops (rank, power, groupby median/quantile, WOE…) without heavy clustering
"clustering" ~10 Manifold & graph features only (UMAP, Genie, Lumbermark, MST…)

Advanced Options

Dynamic BIC & PAC-Bayes Complexity Penalties

Without regularization, complex feature engineering algorithms risk evolving bloated recipes that overfit cross-validation noise. evoFE provides theoretically grounded complexity penalties that scale with dataset size \(N\):

1. Bayesian Information Criterion (complexity_mode = "bic" / "bic_dynamic")

Uses asymptotic BIC scaling per feature: \[p = \lambda \cdot \frac{\ln(N)}{2N}\]

2. PAC-Bayes Generalization Bound (complexity_mode = "pac_bayes" / "pac_bayes_dynamic")

Uses PAC-Bayes generalization bound scaling per feature: \[p = \lambda \cdot \frac{1}{2\sqrt{N}}\]

Dynamic Relaxation & Safety Floor

Under dynamic modes ("bic_dynamic" and "pac_bayes_dynamic"), the penalty dynamically relaxes as evolution converges towards ideal fitness, preventing late-generation stagnation while enforcing a safety floor (complexity_floor = 0.20, or 20% of base penalty).

Feature Count Targets

  • complexity_target = "all_features" (default): Penalizes the total number of active features (retained raw inputs + derived genes). This actively incentivizes the genetic algorithm to prune uninformative raw columns from the active mask.
  • complexity_target = "genes": Penalizes only the number of newly derived transformation genes.
# Enable standard dynamic BIC parsimony with total feature regularization
recipe <- evolve_features(
  data = df,
  target_col = "am",
  task = "classification",
  complexity_penalty = 1.0,           # standard 1.0x BIC multiplier
  complexity_mode = "bic_dynamic",
  complexity_target = "all_features", # reward pruning redundant raw features
  complexity_floor = 0.20             # 20% minimum floor
)

Redundancy Pruning

During evolution, evoFE automatically rejects any new feature whose absolute Pearson correlation with an existing feature exceeds a threshold (default 0.95). This prevents wasting evaluations on near-duplicate columns.

The threshold is tunable:

# Tighten to 0.90 -- reject anything correlated above 90%
options(evoFE.redundancy_cor_threshold = 0.90)

# Disable entirely
options(evoFE.redundancy_cor_threshold = 1.0)

Gene Pooling for the Final Model

By default the final model is trained only on the best individual’s genes. Two flags expand this:

# Model trains on the union of all unique genes in the final population
recipe <- evolve_features(
  ..., model_all_final_genes = TRUE
)

# Model trains on the union of all unique genes across all generations
recipe <- evolve_features(
  ..., model_all_historical_genes = TRUE
)

This can help recover useful genes eliminated by stochastic noise.

Convergence Plot

plot(recipe) produces a generation-by-generation fitness curve:

plot(recipe, type = "fitness")    # fitness trajectory
plot(recipe, type = "importance") # top feature importances of winning model

Live Evolution Viewer

Enable the real-time browser dashboard:

recipe <- evolve_features(
  data = df, target_col = "am", task = "classification",
  record = TRUE  # opens a browser tab with live streaming updates
)

Reproducibility

Calling set.seed() before evolve_features() guarantees identical results across runs:

set.seed(42)
r1 <- evolve_features(iris[,1:5], "Petal.Length", task = "regression",
                       generations = 3, pop_size = 5, evaluator = "xgboost",
                       verbose = FALSE)
set.seed(42)
r2 <- evolve_features(iris[,1:5], "Petal.Length", task = "regression",
                       generations = 3, pop_size = 5, evaluator = "xgboost",
                       verbose = FALSE)

identical(r1$best_individual$fitness, r2$best_individual$fitness)
#> [1] TRUE
identical(
  individual_to_recipe_string(r1$best_individual),
  individual_to_recipe_string(r2$best_individual)
)
#> [1] TRUE

End-to-End Example: Train/Test Split

A realistic workflow with hold-out evaluation:

data(iris)
set.seed(1)
idx <- sample(nrow(iris), 0.7 * nrow(iris))
train <- iris[idx, ]
test  <- iris[-idx, ]

# Evolve on training data only
set.seed(7)
recipe <- evolve_features(
  data      = train[, 1:4],               # exclude Species
  target_col = "Petal.Length",
  task       = "regression",
  evaluator  = "xgboost",
  generations = 5,
  pop_size    = 8,
  verbose     = FALSE
)

# Predict on held-out test data
test_preds <- predict_model(recipe, test[, 1:4])

# Evaluate
rmse <- sqrt(mean((test$Petal.Length - test_preds)^2))
cat(sprintf("Test RMSE: %.4f\n", rmse))
#> Test RMSE: 0.3102
cat(sprintf("Recipe:    %s\n", individual_to_recipe_string(recipe$best_individual)))
#> Recipe:    [mst_score(Sepal.Width, Petal.Width), quantile_binning5(mst_7bcf02)] (active: 3/3 raw, 2 genes)

Session Info

sessionInfo()
#> R version 4.6.1 (2026-06-24)
#> Platform: aarch64-apple-darwin25.4.0
#> Running under: macOS Tahoe 26.5.2
#> 
#> Matrix products: default
#> BLAS:   /opt/homebrew/Cellar/openblas/0.3.34/lib/libopenblasp-r0.3.34.dylib 
#> LAPACK: /opt/homebrew/Cellar/r/4.6.1/lib/R/lib/libRlapack.dylib;  LAPACK version 3.12.1
#> 
#> locale:
#> [1] C/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
#> 
#> time zone: Europe/Stockholm
#> tzcode source: internal
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> other attached packages:
#> [1] evoFE_1.0.0
#> 
#> loaded via a namespace (and not attached):
#>  [1] Matrix_1.7-5        jsonlite_2.0.0      compiler_4.6.1     
#>  [4] crayon_1.5.3        Rcpp_1.1.1-1.1      quitefastmst_0.9.1 
#>  [7] moocore_0.3.1       parallel_4.6.1      jquerylib_0.1.4    
#> [10] RhpcBLASctl_0.23-42 uwot_0.2.4          yaml_2.3.12        
#> [13] fastmap_1.2.0       lattice_0.22-9      R6_2.6.1           
#> [16] knitr_1.51          rbibutils_2.4.1     backports_1.5.1    
#> [19] checkmate_2.3.4     lightgbm_4.6.0      lumbermark_0.9.0   
#> [22] deadwood_0.9.0-3    paradox_1.0.1       bslib_0.11.0       
#> [25] rlang_1.2.0         cachem_1.1.0        lgr_0.5.2          
#> [28] xfun_0.58           mlr3misc_0.22.0     sass_0.4.10        
#> [31] otel_0.2.0          cli_3.6.6           Rdpack_2.6.6       
#> [34] digest_0.6.39       grid_4.6.1          rstudioapi_0.19.0  
#> [37] bbotk_1.11.0        xgboost_3.2.1.1     genieclust_1.3.0   
#> [40] lifecycle_1.0.5     evaluate_1.0.5      data.table_1.18.4  
#> [43] codetools_0.2-20    lhs_1.3.0           RcppAnnoy_0.0.23   
#> [46] rmarkdown_2.31      matrixStats_1.5.0   tools_4.6.1        
#> [49] htmltools_0.5.9