We use the same example as in the quickstart.
using MIPVerify
using Gurobi
mnist = MIPVerify.read_datasets("MNIST")
n1 = MIPVerify.get_example_network_params("MNIST.WK17a_linf0.1_authors")
sample_image = MIPVerify.get_image(mnist.test.images, 1)
1×28×28×1 Array{Float64,4}: [:, :, 1, 1] = 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 … 0.0 0.0 0.0 0.0 0.0 0.0 0.0 [:, :, 2, 1] = 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 … 0.0 0.0 0.0 0.0 0.0 0.0 0.0 [:, :, 3, 1] = 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 … 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... [:, :, 26, 1] = 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 … 0.0 0.0 0.0 0.0 0.0 0.0 0.0 [:, :, 27, 1] = 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 … 0.0 0.0 0.0 0.0 0.0 0.0 0.0 [:, :, 28, 1] = 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 … 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Our package logs to STDOUT at the debug
, info
, notice
and warn
levels.
The default output level is to show logs only at the notice
level and above, which is also the recommended level.
MIPVerify.find_adversarial_example(
n1,
sample_image,
10,
Gurobi.Optimizer,
# OutputFlag=0 prevents any output from being printed out
Dict("OutputFlag" => 0),
pp = MIPVerify.LInfNormBoundedPerturbationFamily(0.05),
norm_order = Inf,
tightening_algorithm = lp,
)
Academic license - for non-commercial use only [notice | MIPVerify]: Attempting to find adversarial example. Neural net predicted label is 8, target labels are [10] [notice | MIPVerify]: Determining upper and lower bounds for the input to each non-linear unit.
Calculating upper bounds: 100%|███████████████████████| Time: 0:00:00
Academic license - for non-commercial use only
Calculating lower bounds: 100%|███████████████████████| Time: 0:00:00 Imposing relu constraint: 100%|███████████████████████| Time: 0:00:01 Calculating upper bounds: 1%|▏ | ETA: 0:47:34
Academic license - for non-commercial use only
Calculating upper bounds: 100%|███████████████████████| Time: 0:00:21 Calculating lower bounds: 100%|███████████████████████| Time: 0:00:00 Imposing relu constraint: 100%|███████████████████████| Time: 0:00:00 Calculating upper bounds: 100%|███████████████████████| Time: 0:00:00 Calculating lower bounds: 100%|███████████████████████| Time: 0:00:00 Imposing relu constraint: 100%|███████████████████████| Time: 0:00:00
Academic license - for non-commercial use only Academic license - for non-commercial use only
Dict{Any,Any} with 11 entries: :TargetIndexes => [10] :SolveTime => 0.00462294 :TotalTime => 50.902 :Perturbation => JuMP.VariableRef[noname noname … noname noname]… :PerturbedInput => JuMP.VariableRef[noname noname … noname noname]… :TighteningApproach => "lp" :PerturbationFamily => linf-norm-bounded-0.05 :SolveStatus => INFEASIBLE_OR_UNBOUNDED :Model => A JuMP Model… :Output => JuMP.GenericAffExpr{Float64,JuMP.VariableRef}[0.000383… :PredictedIndex => 8
More information is available if we set the log level to debug
.
MIPVerify.set_log_level!("debug")
MIPVerify.find_adversarial_example(
n1,
sample_image,
10,
Gurobi.Optimizer,
# OutputFlag=0 prevents any output from being printed out
Dict("OutputFlag" => 0),
pp = MIPVerify.LInfNormBoundedPerturbationFamily(0.05),
norm_order = Inf,
tightening_algorithm = lp,
)
[debug | Memento]: Logger(root) is already registered. Academic license - for non-commercial use only [notice | MIPVerify]: Attempting to find adversarial example. Neural net predicted label is 8, target labels are [10] [notice | MIPVerify]: Determining upper and lower bounds for the input to each non-linear unit. Academic license - for non-commercial use only [info | MIPVerify]: Applying Conv2d(1, 16, kernel_size=(4, 4), stride=(2, 2), padding=same) ... [info | MIPVerify]: Applying ReLU() ... [info | MIPVerify]: Applying Conv2d(16, 32, kernel_size=(4, 4), stride=(2, 2), padding=same) ... [info | MIPVerify]: Applying ReLU() ... Academic license - for non-commercial use only [info | MIPVerify]: Applying Flatten() ... [info | MIPVerify]: Applying Linear(1568 -> 100) ... [info | MIPVerify]: Applying ReLU() ... [info | MIPVerify]: Applying Linear(100 -> 10) ... Academic license - for non-commercial use only Academic license - for non-commercial use only
Dict{Any,Any} with 11 entries: :TargetIndexes => [10] :SolveTime => 0.00461507 :TotalTime => 4.68894 :Perturbation => JuMP.VariableRef[noname noname … noname noname]… :PerturbedInput => JuMP.VariableRef[noname noname … noname noname]… :TighteningApproach => "lp" :PerturbationFamily => linf-norm-bounded-0.05 :SolveStatus => INFEASIBLE_OR_UNBOUNDED :Model => A JuMP Model… :Output => JuMP.GenericAffExpr{Float64,JuMP.VariableRef}[0.000383… :PredictedIndex => 8
We use the logger provided by the Memento
package. Rather than just having logs written to STDOUT, you can choose to capture the output (for example, to write logs to file) by adding your own handlers to MIPVerify.LOGGER
. See the Memento documentation for more.
typeof(MIPVerify.LOGGER)
Memento.Logger