We use the same example as in the quickstart.
using MIPVerify
using HiGHS
mnist = MIPVerify.read_datasets("MNIST")
n1 = MIPVerify.get_example_network_params("MNIST.n1")
sample_image = MIPVerify.get_image(mnist.test.images, 1);
Our package logs to STDOUT at the debug
, info
, notice
and warn
levels.
The default output level is to show logs only at the notice
level and above, which is also the recommended level.
MIPVerify.find_adversarial_example(
n1,
sample_image,
10,
HiGHS.Optimizer,
Dict(),
pp = MIPVerify.LInfNormBoundedPerturbationFamily(0.05),
norm_order = Inf,
tightening_algorithm = lp,
)
[notice | MIPVerify]: Attempting to find adversarial example. Neural net predicted label is 8, target labels are [10] [notice | MIPVerify]: Determining upper and lower bounds for the input to each non-linear unit. Running HiGHS 1.4.2 [date: 1970-01-01, git hash: f797c1ab6] Copyright (c) 2022 ERGO-Code under MIT licence terms Presolving model 1779 rows, 1627 cols, 56716 nonzeros 1779 rows, 1627 cols, 56716 nonzeros Solving MIP model with: 1779 rows 1627 cols (29 binary, 0 integer, 0 implied int., 1598 continuous) 56716 nonzeros Nodes | B&B Tree | Objective Bounds | Dynamic Constraints | Work Proc. InQueue | Leaves Expl. | BestBound BestSol Gap | Cuts InLp Confl. | LpIters Time 0 0 0 0.00% 0 inf inf 0 0 0 0 0.0s 0 0 0 0.00% 0 inf inf 0 0 13 1616 0.1s 0 0 0 0.00% 0 inf inf 3942 12 99 30330 14.6s T 15 0 3 1.61% 0 0.0498354344 100.00% 3951 12 109 39558 16.3s T 27 1 10 3.42% 0 0.0474230913 100.00% 3974 12 131 45590 17.4s T 45 4 20 77.27% 0.0381825316 0.0460846816 17.15% 4563 15 162 58011 20.1s Solving report Status Optimal Primal bound 0.0460846815889 Dual bound 0.0460846815889 Gap 0% (tolerance: 0.01%) Solution status feasible 0.0460846815889 (objective) 0 (bound viol.) 0 (int. viol.) 0 (row viol.) Timing 21.37 (total) 0.02 (presolve) 0.00 (postsolve) Nodes 53 LP iterations 60163 (total) 19015 (strong br.) 707 (separation) 28049 (heuristics)
Dict{Any, Any} with 11 entries: :TargetIndexes => [10] :SolveTime => 21.369 :TotalTime => 39.5667 :Perturbation => [_[1] _[2] … _[27] _[28];;; _[29] _[30] … _[55] _[56];… :PerturbedInput => [_[785] _[786] … _[811] _[812];;; _[813] _[814] … _[83… :TighteningApproach => "lp" :PerturbationFamily => linf-norm-bounded-0.05 :SolveStatus => OPTIMAL :Model => A JuMP Model… :Output => JuMP.AffExpr[-0.012063867412507534 _[1601] + 0.6606525… :PredictedIndex => 8
More information is available if we set the log level to debug
.
MIPVerify.set_log_level!("debug")
MIPVerify.find_adversarial_example(
n1,
sample_image,
10,
HiGHS.Optimizer,
Dict(),
pp = MIPVerify.LInfNormBoundedPerturbationFamily(0.05),
norm_order = Inf,
tightening_algorithm = lp,
)
[debug | Memento]: Logger(root) is already registered and force=false. [notice | MIPVerify]: Attempting to find adversarial example. Neural net predicted label is 8, target labels are [10] [notice | MIPVerify]: Determining upper and lower bounds for the input to each non-linear unit. [info | MIPVerify]: Applying Flatten() ... [info | MIPVerify]: Applying Linear(784 -> 40) ... [info | MIPVerify]: Applying ReLU() ... [info | MIPVerify]: Applying Linear(40 -> 20) ... [info | MIPVerify]: Applying ReLU() ... [info | MIPVerify]: Applying Linear(20 -> 10) ... Running HiGHS 1.4.2 [date: 1970-01-01, git hash: f797c1ab6] Copyright (c) 2022 ERGO-Code under MIT licence terms Presolving model 1779 rows, 1627 cols, 56716 nonzeros 1779 rows, 1627 cols, 56716 nonzeros Solving MIP model with: 1779 rows 1627 cols (29 binary, 0 integer, 0 implied int., 1598 continuous) 56716 nonzeros Nodes | B&B Tree | Objective Bounds | Dynamic Constraints | Work Proc. InQueue | Leaves Expl. | BestBound BestSol Gap | Cuts InLp Confl. | LpIters Time 0 0 0 0.00% 0 inf inf 0 0 0 0 0.0s 0 0 0 0.00% 0 inf inf 0 0 13 1616 0.1s 0 0 0 0.00% 0 inf inf 3942 12 99 30330 14.3s T 15 0 3 1.61% 0 0.0498354344 100.00% 3951 12 109 39558 16.1s T 27 1 10 3.42% 0 0.0474230913 100.00% 3974 12 131 45590 17.1s T 45 4 20 77.27% 0.0381825316 0.0460846816 17.15% 4563 15 162 58011 20.2s Solving report Status Optimal Primal bound 0.0460846815889 Dual bound 0.0460846815889 Gap 0% (tolerance: 0.01%) Solution status feasible 0.0460846815889 (objective) 0 (bound viol.) 0 (int. viol.) 0 (row viol.) Timing 21.52 (total) 0.02 (presolve) 0.00 (postsolve) Nodes 53 LP iterations 60163 (total) 19015 (strong br.) 707 (separation) 28049 (heuristics)
Dict{Any, Any} with 11 entries: :TargetIndexes => [10] :SolveTime => 21.5182 :TotalTime => 22.2792 :Perturbation => [_[1] _[2] … _[27] _[28];;; _[29] _[30] … _[55] _[56];… :PerturbedInput => [_[785] _[786] … _[811] _[812];;; _[813] _[814] … _[83… :TighteningApproach => "lp" :PerturbationFamily => linf-norm-bounded-0.05 :SolveStatus => OPTIMAL :Model => A JuMP Model… :Output => JuMP.AffExpr[-0.012063867412507534 _[1601] + 0.6606525… :PredictedIndex => 8
We use the logger provided by the Memento
package. Rather than just having logs written to STDOUT, you can choose to capture the output (for example, to write logs to file) by adding your own handlers to MIPVerify.LOGGER
. See the Memento documentation for more.
typeof(MIPVerify.LOGGER)
Memento.Logger