The surrogate model is what the optimizer thinks the scoring function looks like. It is part of the mechanism used to choose optimal configurations.
You can generate predictions from the surrogate model (effectively asking OPTaaS to guess what the scoring function may be at a certain point) at any set of arbitrary configuration points.
library(optaas.client)
client <- OPTaaSClient$new("https://optaas.mindfoundry.ai", "Your OPTaaS API Key")
task <- client$create_task(
title="Basic 2D Example",
parameters=list(
FloatParameter('x', minimum = -3, maximum = 1),
FloatParameter('y', minimum = -6, maximum = 21)
),
goal="min"
)
Just a simple well with the minimum at (0, 0)
scoring_function <- function(x, y) {
x**2 + y**2
}
best_result <- task$run(scoring_function, 15)
[1] "Iteration: 1 Score: 57.25" [1] "Iteration: 2 Score: 207.0625" [1] "Iteration: 3 Score: 0.5625" [1] "Iteration: 4 Score: 19.265625" [1] "Iteration: 5 Score: 310.890625" [1] "Iteration: 6 Score: 124.515625" [1] "Iteration: 7 Score: 7.140625" [1] "Iteration: 8 Score: 3.94140625" [1] "Iteration: 9 Score: 157.87890625" [1] "Iteration: 10 Score: 380.53515625" [1] "Iteration: 11 Score: 1.0128225401033" [1] "Iteration: 12 Score: 0.00860669147744739" [1] "Iteration: 13 Score: 0.000837724442447517" [1] "Iteration: 14 Score: 0.00061925191746992" [1] "Iteration: 15 Score: 0.000603156957389812"
The surrogate model should predict a fairly low score with high confidence, since it has been exploring the vicinity of this point.
known_best <- list(x=0, y=0)
predictions <- task$get_surrogate_predictions(list(known_best))
print(paste("Score:", predictions[[1]]$mean, "Variance:", predictions[[1]]$variance))
[1] "Score: 0.0116939091971489 Variance: 0.000152567573711395"
The surrogate model should be significantly less confident, as there were no evaluations near this point.
far_away_points <- list(list(x=1, y=20), list(x=-3, y=-6))
predictions <- task$get_surrogate_predictions(far_away_points)
print(paste("Score:", predictions[[1]]$mean, "Variance:", predictions[[1]]$variance))
print(paste("Score:", predictions[[2]]$mean, "Variance:", predictions[[2]]$variance))
[1] "Score: 399.699123738722 Variance: 0.0407275300936555" [1] "Score: 42.0550946488813 Variance: 0.188253127450461"
Here's an article we wrote on how the surrogate works: https://towardsdatascience.com/the-intuitions-behind-bayesian-optimization-with-gaussian-processes-7e00fcc898a0