Grid Search

Given a set of possible hyperparameter values, the function trains models with all the possible combinations of hyperparameters.

gridSearch(model, hypers, metric, test = NULL, env = NULL, save_models = TRUE)

Arguments

model	SDMmodel or SDMmodelCV object.
hypers	named list containing the values of the hyperparameters that should be tuned, see details.
metric	character. The metric used to evaluate the models, possible values are: "auc", "tss" and "aicc".
test	SWD object. Testing dataset used to evaluate the model, not used with aicc and SDMmodelCV objects, default is NULL.
env	stack containing the environmental variables, used only with "aicc", default is NULL.
save_models	logical, if FALSE the models are not saved and the output contains only a data frame with the metric values for each hyperparameter combination. Default is TRUE, set it to FALSE when there are many combinations to avoid R crashing for memory overload.

Value

SDMtune object.

Details

• To know which hyperparameters can be tuned you can use the output of the function getTunableArgs. Hyperparameters not included in the hypers argument take the value that they have in the passed model.

See also

randomSearch and optimizeModel.

Author

Sergio Vignali

Examples

# \donttest{
# Acquire environmental variables
files <- list.files(path = file.path(system.file(package = "dismo"), "ex"),
                    pattern = "grd", full.names = TRUE)
predictors <- raster::stack(files)

# Prepare presence and background locations
p_coords <- virtualSp$presence
bg_coords <- virtualSp$background

# Create SWD object
data <- prepareSWD(species = "Virtual species", p = p_coords, a = bg_coords,
                   env = predictors, categorical = "biome")
#> Extracting predictor information for presence locations...
#> Extracting predictor information for absence/background locations...

# Split presence locations in training (80%) and testing (20%) datasets
datasets <- trainValTest(data, test = 0.2, only_presence = TRUE)
train <- datasets[[1]]
test <- datasets[[2]]

# Train a model
model <- train(method = "Maxnet", data = train, fc = "l")

# Define the hyperparameters to test
h <- list(reg = 1:2, fc = c("lqp", "lqph"))

# Run the function using the AUC as metric
output <- gridSearch(model, hypers = h, metric = "auc", test = test)
output@results
#>     fc reg train_AUC  test_AUC    diff_AUC
#> 1  lqp   1 0.8620394 0.8550612 0.006978125
#> 2 lqph   1 0.8720638 0.8540037 0.018060000
#> 3  lqp   2 0.8588537 0.8488062 0.010047500
#> 4 lqph   2 0.8654625 0.8518813 0.013581250
output@models
#> [[1]]
#> Object of class SDMmodel 
#> Method: Maxnet 
#> 
#> Species: Virtual species 
#> Presence locations: 320 
#> Absence locations: 5000 
#> 
#> Model configurations:
#> --------------------
#> fc: lqp
#> reg: 1
#> 
#> Variables:
#> ---------
#> Continuous: bio1 bio12 bio16 bio17 bio5 bio6 bio7 bio8 
#> Categorical: biome
#> [[2]]
#> Object of class SDMmodel 
#> Method: Maxnet 
#> 
#> Species: Virtual species 
#> Presence locations: 320 
#> Absence locations: 5000 
#> 
#> Model configurations:
#> --------------------
#> fc: lqph
#> reg: 1
#> 
#> Variables:
#> ---------
#> Continuous: bio1 bio12 bio16 bio17 bio5 bio6 bio7 bio8 
#> Categorical: biome
#> [[3]]
#> Object of class SDMmodel 
#> Method: Maxnet 
#> 
#> Species: Virtual species 
#> Presence locations: 320 
#> Absence locations: 5000 
#> 
#> Model configurations:
#> --------------------
#> fc: lqp
#> reg: 2
#> 
#> Variables:
#> ---------
#> Continuous: bio1 bio12 bio16 bio17 bio5 bio6 bio7 bio8 
#> Categorical: biome
#> [[4]]
#> Object of class SDMmodel 
#> Method: Maxnet 
#> 
#> Species: Virtual species 
#> Presence locations: 320 
#> Absence locations: 5000 
#> 
#> Model configurations:
#> --------------------
#> fc: lqph
#> reg: 2
#> 
#> Variables:
#> ---------
#> Continuous: bio1 bio12 bio16 bio17 bio5 bio6 bio7 bio8 
#> Categorical: biome
# Order rusults by highest test AUC
head(output@results[order(-output@results$test_AUC), ])
#>     fc reg train_AUC  test_AUC    diff_AUC
#> 1  lqp   1 0.8620394 0.8550612 0.006978125
#> 2 lqph   1 0.8720638 0.8540037 0.018060000
#> 4 lqph   2 0.8654625 0.8518813 0.013581250
#> 3  lqp   2 0.8588537 0.8488062 0.010047500

# Run the function using the AICc as metric and without saving the trained
# models, helpful when numerous hyperparameters are tested to avoid memory
# problems
output <- gridSearch(model, hypers = h, metric = "aicc", env = predictors,
                     save_models = FALSE)
output@results
#>     fc reg     AICc delta_AICc
#> 1  lqp   1 5278.672   14.20438
#> 2 lqph   1 5283.754   19.28583
#> 3  lqp   2 5273.436    8.96854
#> 4 lqph   2 5264.468    0.00000
# }