# rf606_nllerrorhandling¶

'LIKELIHOOD AND MINIMIZATION' RooFit tutorial macro #606

Understanding and customizing error handling in likelihood evaluations

Author: Clemens Lange, Wouter Verkerke (C version)
This notebook tutorial was automatically generated with ROOTBOOK-izer from the macro found in the ROOT repository on Sunday, November 27, 2022 at 11:08 AM.

In [1]:
import ROOT

Welcome to JupyROOT 6.27/01


## Create model and dataset¶

Observable

In [2]:
m = ROOT.RooRealVar("m", "m", 5.20, 5.30)


Parameters

In [3]:
m0 = ROOT.RooRealVar("m0", "m0", 5.291, 5.20, 5.30)
k = ROOT.RooRealVar("k", "k", -30, -50, -10)


Pdf

In [4]:
argus = ROOT.RooArgusBG("argus", "argus", m, m0, k)


Sample 1000 events in m from argus

In [5]:
data = argus.generate({m}, 1000)


## Plot model and data¶

In [6]:
frame1 = m.frame(Bins=40, Title="Argus model and data")
data.plotOn(frame1)
argus.plotOn(frame1)

Out[6]:
<cppyy.gbl.RooPlot object at 0x91f3780>

## Fit model to data¶

The ARGUS background shape has a sharp kinematic cutoff at m=m0 and is prone to evaluation errors if the cutoff parameter m0 is floated: when the pdf cutoff value is lower than that in data events with m>m0 will have zero probability

Perform unbinned ML fit. Print detailed error messages for up to 10 events per likelihood evaluation. The default error handling strategy is to return a very high value of the likelihood to MINUIT if errors occur, which will force MINUIT to retreat from the problematic area

In [7]:
argus.fitTo(data, PrintEvalErrors=10)

Out[7]:
<cppyy.gbl.RooFitResult object at 0x(nil)>
[#1] INFO:Minimization -- RooAbsMinimizerFcn::setOptimizeConst: activating const optimization
**********
**    1 **SET PRINT           1
**********
**********
**********
PARAMETER DEFINITIONS:
NO.   NAME         VALUE      STEP SIZE      LIMITS
1 k           -3.00000e+01  4.00000e+00   -5.00000e+01 -1.00000e+01
2 m0           5.29100e+00  4.50000e-03    5.20000e+00  5.30000e+00
**********
**    3 **SET ERR         0.5
**********
**********
**    4 **SET PRINT           1
**********
**********
**    5 **SET STR           1
**********
NOW USING STRATEGY  1: TRY TO BALANCE SPEED AGAINST RELIABILITY
**********
**********
FIRST CALL TO USER FUNCTION AT NEW START POINT, WITH IFLAG=4.
START MIGRAD MINIMIZATION.  STRATEGY  1.  CONVERGENCE WHEN EDM .LT. 1.00e-03
FCN=-2417.48 FROM MIGRAD    STATUS=INITIATE        6 CALLS           7 TOTAL
EDM= unknown      STRATEGY= 1      NO ERROR MATRIX
EXT PARAMETER               CURRENT GUESS       STEP         FIRST
NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE
1  k           -3.00000e+01   4.00000e+00   2.01358e-01   6.43007e+00
2  m0           5.29100e+00   4.50000e-03   1.62481e-01   6.64912e+01
ERR DEF= 0.5
RooAbsMinimizerFcn: Minimized function has error status.
Returning maximum FCN so far (-2417.21) to force MIGRAD to back out of this region. Error log follows.
Parameter values: 	k=-33.8074	m0=5.29014
RooArgusBG::argus[ m=m m0=m0 c=k p=0.5 ]
getLogVal() top-level p.d.f evaluates to zero @ m=m=5.29019, m0=m0=5.29014, c=k=-33.8074, p=0.5=0.5
RooNLLVar::nll_argus_argusData[ parameters=(k,m0) ]
function value is NAN @ parameters=(k = -33.8074,m0 = 5.29014)

RooAbsMinimizerFcn: Minimized function has error status.
Returning maximum FCN so far (-2417.21) to force MIGRAD to back out of this region. Error log follows.
Parameter values: 	k=-36.6878	m0=5.29012
RooArgusBG::argus[ m=m m0=m0 c=k p=0.5 ]
getLogVal() top-level p.d.f evaluates to zero @ m=m=5.29019, m0=m0=5.29012, c=k=-36.6878, p=0.5=0.5
RooNLLVar::nll_argus_argusData[ parameters=(k,m0) ]
function value is NAN @ parameters=(k = -36.6878,m0 = 5.29012)

MIGRAD WILL VERIFY CONVERGENCE AND ERROR MATRIX.
COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=-2419.31 FROM MIGRAD    STATUS=CONVERGED      44 CALLS          45 TOTAL
EDM=1.16119e-05    STRATEGY= 1      ERROR MATRIX ACCURATE
EXT PARAMETER                                   STEP         FIRST
NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE
1  k           -3.53724e+01   3.51965e+00   6.12703e-03  -1.66396e-02
2  m0           5.29040e+00   2.62256e-04   3.09448e-04   2.47503e-01
ERR DEF= 0.5
EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  2    ERR DEF=0.5
1.253e+01  1.785e-04
1.785e-04  6.878e-08
PARAMETER  CORRELATION COEFFICIENTS
NO.  GLOBAL      1      2
1  0.19224   1.000  0.192
2  0.19224   0.192  1.000
**********
**    7 **SET ERR         0.5
**********
**********
**    8 **SET PRINT           1
**********
**********
**    9 **HESSE        1000
**********
COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=-2419.31 FROM HESSE     STATUS=OK             10 CALLS          55 TOTAL
EDM=1.16219e-05    STRATEGY= 1      ERROR MATRIX ACCURATE
EXT PARAMETER                                INTERNAL      INTERNAL
NO.   NAME      VALUE            ERROR       STEP SIZE       VALUE
1  k           -3.53724e+01   3.51966e+00   1.22541e-03  -2.71960e-01
2  m0           5.29040e+00   2.62349e-04   1.23779e-05   9.40810e-01
ERR DEF= 0.5
EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  2    ERR DEF=0.5
1.253e+01  1.786e-04
1.786e-04  6.883e-08
PARAMETER  CORRELATION COEFFICIENTS
NO.  GLOBAL      1      2
1  0.19229   1.000  0.192
2  0.19229   0.192  1.000
[#1] INFO:Minimization -- RooAbsMinimizerFcn::setOptimizeConst: deactivating const optimization


Peform another fit. In self configuration only the number of errors per likelihood evaluation is shown, it is greater than zero. The EvalErrorWall(kFALSE) arguments disables the default error handling strategy and will cause the actual (problematic) value of the likelihood to be passed to MINUIT.

NB: Use of self option is NOT recommended as default strategt as broken -log(L) values can often be lower than 'good' ones because offending events are removed. ROOT.This may effectively create a False minimum in problem areas. ROOT.This is clearly illustrated in the second plot

In [8]:
m0.setError(0.1)
argus.fitTo(data, PrintEvalErrors=0, EvalErrorWall=False)

Out[8]:
<cppyy.gbl.RooFitResult object at 0x(nil)>
[#1] INFO:Minimization -- RooAbsMinimizerFcn::setOptimizeConst: activating const optimization
**********
**   10 **SET PRINT           1
**********
**********
**********
PARAMETER DEFINITIONS:
NO.   NAME         VALUE      STEP SIZE      LIMITS
1 k           -3.53724e+01  3.51966e+00   -5.00000e+01 -1.00000e+01
2 m0           5.29040e+00  1.00000e-01    5.20000e+00  5.30000e+00
MINUIT WARNING IN PARAMETR
============== VARIABLE2 BROUGHT BACK INSIDE LIMITS.
MINUIT WARNING IN PARAMETR
============== VARIABLE2 BROUGHT BACK INSIDE LIMITS.
**********
**   12 **SET ERR         0.5
**********
**********
**   13 **SET PRINT           1
**********
**********
**   14 **SET STR           1
**********
NOW USING STRATEGY  1: TRY TO BALANCE SPEED AGAINST RELIABILITY
**********
**********
FIRST CALL TO USER FUNCTION AT NEW START POINT, WITH IFLAG=4.
START MIGRAD MINIMIZATION.  STRATEGY  1.  CONVERGENCE WHEN EDM .LT. 1.00e-03
RooAbsMinimizerFcn: Minimized function has error status but is ignored.
Parameter values: 	k=-35.3724	m0=5.28878
RooArgusBG::argus[ m=m m0=m0 c=k p=0.5 ] has 4 errors
RooNLLVar::nll_argus_argusData[ parameters=(k,m0) ] has 1 errors

FCN=-2419.31 FROM MIGRAD    STATUS=INITIATE        8 CALLS           9 TOTAL
EDM= unknown      STRATEGY= 1      NO ERROR MATRIX
EXT PARAMETER               CURRENT GUESS       STEP         FIRST
NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE
1  k           -3.53724e+01   3.51966e+00   1.83981e-01  -1.64778e-02
2  m0           5.29040e+00   1.00000e-01   1.56884e+00   1.11846e-01
ERR DEF= 0.5
MIGRAD WILL VERIFY CONVERGENCE AND ERROR MATRIX.
COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=-2419.31 FROM MIGRAD    STATUS=CONVERGED      28 CALLS          29 TOTAL
EDM=4.57045e-07    STRATEGY= 1      ERROR MATRIX ACCURATE
EXT PARAMETER                                   STEP         FIRST
NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE
1  k           -3.53621e+01   3.51925e+00   6.12459e-03   8.76390e-04
2  m0           5.29040e+00   2.62011e-04   2.96587e-04   7.04254e-02
ERR DEF= 0.5
EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  2    ERR DEF=0.5
1.253e+01  1.781e-04
1.781e-04  6.865e-08
PARAMETER  CORRELATION COEFFICIENTS
NO.  GLOBAL      1      2
1  0.19204   1.000  0.192
2  0.19204   0.192  1.000
**********
**   16 **SET ERR         0.5
**********
**********
**   17 **SET PRINT           1
**********
**********
**   18 **HESSE        1000
**********
COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=-2419.31 FROM HESSE     STATUS=OK             10 CALLS          39 TOTAL
EDM=4.58038e-07    STRATEGY= 1      ERROR MATRIX ACCURATE
EXT PARAMETER                                INTERNAL      INTERNAL
NO.   NAME      VALUE            ERROR       STEP SIZE       VALUE
1  k           -3.53621e+01   3.51925e+00   2.44984e-04  -2.71425e-01
2  m0           5.29040e+00   2.62096e-04   1.18635e-05   9.40801e-01
ERR DEF= 0.5
EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  2    ERR DEF=0.5
1.253e+01  1.782e-04
1.782e-04  6.870e-08
PARAMETER  CORRELATION COEFFICIENTS
NO.  GLOBAL      1      2
1  0.19207   1.000  0.192
2  0.19207   0.192  1.000
[#1] INFO:Minimization -- RooAbsMinimizerFcn::setOptimizeConst: deactivating const optimization


## Plot likelihood as function of m0¶

Construct likelihood function of model and data

In [9]:
nll = argus.createNLL(data)


Plot likelihood in m0 in range that includes problematic values In self configuration no messages are printed for likelihood evaluation errors, but if an likelihood value evaluates with error, corresponding value on the curve will be set to the value given in EvalErrorValue().

In [10]:
frame2 = m0.frame(Range=(5.288, 5.293), Title="-log(L) scan vs m0, regions masked")
nll.plotOn(frame2, ShiftToZero=True, PrintEvalErrors=-1, EvalErrorValue=(nll.getVal() + 10), LineColor="r")
frame2.SetMaximum(15)
frame2.SetMinimum(0)

c = ROOT.TCanvas("rf606_nllerrorhandling", "rf606_nllerrorhandling", 1200, 400)
c.Divide(2)
c.cd(1)
frame1.GetYaxis().SetTitleOffset(1.4)
frame1.Draw()
c.cd(2)
frame2.GetYaxis().SetTitleOffset(1.4)
frame2.Draw()

c.SaveAs("rf606_nllerrorhandling.png")

Info in <TCanvas::Print>: png file rf606_nllerrorhandling.png has been created


Draw all canvases

In [11]:
from ROOT import gROOT
gROOT.GetListOfCanvases().Draw()