rf606_nllerrorhandling

'LIKELIHOOD AND MINIMIZATION' RooFit tutorial macro #606

Understanding and customizing error handling in likelihood evaluations

Author: Clemens Lange, Wouter Verkerke (C version)
This notebook tutorial was automatically generated with ROOTBOOK-izer from the macro found in the ROOT repository on Sunday, November 27, 2022 at 11:08 AM.

In [1]:
import ROOT
Welcome to JupyROOT 6.27/01

Create model and dataset

Observable

In [2]:
m = ROOT.RooRealVar("m", "m", 5.20, 5.30)

Parameters

In [3]:
m0 = ROOT.RooRealVar("m0", "m0", 5.291, 5.20, 5.30)
k = ROOT.RooRealVar("k", "k", -30, -50, -10)

Pdf

In [4]:
argus = ROOT.RooArgusBG("argus", "argus", m, m0, k)

Sample 1000 events in m from argus

In [5]:
data = argus.generate({m}, 1000)

Plot model and data

In [6]:
frame1 = m.frame(Bins=40, Title="Argus model and data")
data.plotOn(frame1)
argus.plotOn(frame1)
Out[6]:
<cppyy.gbl.RooPlot object at 0x91f3780>

Fit model to data

The ARGUS background shape has a sharp kinematic cutoff at m=m0 and is prone to evaluation errors if the cutoff parameter m0 is floated: when the pdf cutoff value is lower than that in data events with m>m0 will have zero probability

Perform unbinned ML fit. Print detailed error messages for up to 10 events per likelihood evaluation. The default error handling strategy is to return a very high value of the likelihood to MINUIT if errors occur, which will force MINUIT to retreat from the problematic area

In [7]:
argus.fitTo(data, PrintEvalErrors=10)
Out[7]:
<cppyy.gbl.RooFitResult object at 0x(nil)>
[#1] INFO:Minimization -- RooAbsMinimizerFcn::setOptimizeConst: activating const optimization
 **********
 **    1 **SET PRINT           1
 **********
 **********
 **    2 **SET NOGRAD
 **********
 PARAMETER DEFINITIONS:
    NO.   NAME         VALUE      STEP SIZE      LIMITS
     1 k           -3.00000e+01  4.00000e+00   -5.00000e+01 -1.00000e+01
     2 m0           5.29100e+00  4.50000e-03    5.20000e+00  5.30000e+00
 **********
 **    3 **SET ERR         0.5
 **********
 **********
 **    4 **SET PRINT           1
 **********
 **********
 **    5 **SET STR           1
 **********
 NOW USING STRATEGY  1: TRY TO BALANCE SPEED AGAINST RELIABILITY
 **********
 **    6 **MIGRAD        1000           1
 **********
 FIRST CALL TO USER FUNCTION AT NEW START POINT, WITH IFLAG=4.
 START MIGRAD MINIMIZATION.  STRATEGY  1.  CONVERGENCE WHEN EDM .LT. 1.00e-03
 FCN=-2417.48 FROM MIGRAD    STATUS=INITIATE        6 CALLS           7 TOTAL
                     EDM= unknown      STRATEGY= 1      NO ERROR MATRIX       
  EXT PARAMETER               CURRENT GUESS       STEP         FIRST   
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE 
   1  k           -3.00000e+01   4.00000e+00   2.01358e-01   6.43007e+00
   2  m0           5.29100e+00   4.50000e-03   1.62481e-01   6.64912e+01
                               ERR DEF= 0.5
RooAbsMinimizerFcn: Minimized function has error status.
Returning maximum FCN so far (-2417.21) to force MIGRAD to back out of this region. Error log follows.
Parameter values: 	k=-33.8074	m0=5.29014
RooArgusBG::argus[ m=m m0=m0 c=k p=0.5 ]
     getLogVal() top-level p.d.f evaluates to zero @ m=m=5.29019, m0=m0=5.29014, c=k=-33.8074, p=0.5=0.5
RooNLLVar::nll_argus_argusData[ parameters=(k,m0) ]
     function value is NAN @ parameters=(k = -33.8074,m0 = 5.29014)

RooAbsMinimizerFcn: Minimized function has error status.
Returning maximum FCN so far (-2417.21) to force MIGRAD to back out of this region. Error log follows.
Parameter values: 	k=-36.6878	m0=5.29012
RooArgusBG::argus[ m=m m0=m0 c=k p=0.5 ]
     getLogVal() top-level p.d.f evaluates to zero @ m=m=5.29019, m0=m0=5.29012, c=k=-36.6878, p=0.5=0.5
RooNLLVar::nll_argus_argusData[ parameters=(k,m0) ]
     function value is NAN @ parameters=(k = -36.6878,m0 = 5.29012)

 MIGRAD MINIMIZATION HAS CONVERGED.
 MIGRAD WILL VERIFY CONVERGENCE AND ERROR MATRIX.
 COVARIANCE MATRIX CALCULATED SUCCESSFULLY
 FCN=-2419.31 FROM MIGRAD    STATUS=CONVERGED      44 CALLS          45 TOTAL
                     EDM=1.16119e-05    STRATEGY= 1      ERROR MATRIX ACCURATE 
  EXT PARAMETER                                   STEP         FIRST   
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE 
   1  k           -3.53724e+01   3.51965e+00   6.12703e-03  -1.66396e-02
   2  m0           5.29040e+00   2.62256e-04   3.09448e-04   2.47503e-01
                               ERR DEF= 0.5
 EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  2    ERR DEF=0.5
  1.253e+01  1.785e-04 
  1.785e-04  6.878e-08 
 PARAMETER  CORRELATION COEFFICIENTS  
       NO.  GLOBAL      1      2
        1  0.19224   1.000  0.192
        2  0.19224   0.192  1.000
 **********
 **    7 **SET ERR         0.5
 **********
 **********
 **    8 **SET PRINT           1
 **********
 **********
 **    9 **HESSE        1000
 **********
 COVARIANCE MATRIX CALCULATED SUCCESSFULLY
 FCN=-2419.31 FROM HESSE     STATUS=OK             10 CALLS          55 TOTAL
                     EDM=1.16219e-05    STRATEGY= 1      ERROR MATRIX ACCURATE 
  EXT PARAMETER                                INTERNAL      INTERNAL  
  NO.   NAME      VALUE            ERROR       STEP SIZE       VALUE   
   1  k           -3.53724e+01   3.51966e+00   1.22541e-03  -2.71960e-01
   2  m0           5.29040e+00   2.62349e-04   1.23779e-05   9.40810e-01
                               ERR DEF= 0.5
 EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  2    ERR DEF=0.5
  1.253e+01  1.786e-04 
  1.786e-04  6.883e-08 
 PARAMETER  CORRELATION COEFFICIENTS  
       NO.  GLOBAL      1      2
        1  0.19229   1.000  0.192
        2  0.19229   0.192  1.000
[#1] INFO:Minimization -- RooAbsMinimizerFcn::setOptimizeConst: deactivating const optimization

Peform another fit. In self configuration only the number of errors per likelihood evaluation is shown, it is greater than zero. The EvalErrorWall(kFALSE) arguments disables the default error handling strategy and will cause the actual (problematic) value of the likelihood to be passed to MINUIT.

NB: Use of self option is NOT recommended as default strategt as broken -log(L) values can often be lower than 'good' ones because offending events are removed. ROOT.This may effectively create a False minimum in problem areas. ROOT.This is clearly illustrated in the second plot

In [8]:
m0.setError(0.1)
argus.fitTo(data, PrintEvalErrors=0, EvalErrorWall=False)
Out[8]:
<cppyy.gbl.RooFitResult object at 0x(nil)>
[#1] INFO:Minimization -- RooAbsMinimizerFcn::setOptimizeConst: activating const optimization
 **********
 **   10 **SET PRINT           1
 **********
 **********
 **   11 **SET NOGRAD
 **********
 PARAMETER DEFINITIONS:
    NO.   NAME         VALUE      STEP SIZE      LIMITS
     1 k           -3.53724e+01  3.51966e+00   -5.00000e+01 -1.00000e+01
     2 m0           5.29040e+00  1.00000e-01    5.20000e+00  5.30000e+00
 MINUIT WARNING IN PARAMETR
 ============== VARIABLE2 BROUGHT BACK INSIDE LIMITS.
 MINUIT WARNING IN PARAMETR
 ============== VARIABLE2 BROUGHT BACK INSIDE LIMITS.
 **********
 **   12 **SET ERR         0.5
 **********
 **********
 **   13 **SET PRINT           1
 **********
 **********
 **   14 **SET STR           1
 **********
 NOW USING STRATEGY  1: TRY TO BALANCE SPEED AGAINST RELIABILITY
 **********
 **   15 **MIGRAD        1000           1
 **********
 FIRST CALL TO USER FUNCTION AT NEW START POINT, WITH IFLAG=4.
 START MIGRAD MINIMIZATION.  STRATEGY  1.  CONVERGENCE WHEN EDM .LT. 1.00e-03
RooAbsMinimizerFcn: Minimized function has error status but is ignored.
Parameter values: 	k=-35.3724	m0=5.28878
RooArgusBG::argus[ m=m m0=m0 c=k p=0.5 ] has 4 errors
RooNLLVar::nll_argus_argusData[ parameters=(k,m0) ] has 1 errors

 FCN=-2419.31 FROM MIGRAD    STATUS=INITIATE        8 CALLS           9 TOTAL
                     EDM= unknown      STRATEGY= 1      NO ERROR MATRIX       
  EXT PARAMETER               CURRENT GUESS       STEP         FIRST   
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE 
   1  k           -3.53724e+01   3.51966e+00   1.83981e-01  -1.64778e-02
   2  m0           5.29040e+00   1.00000e-01   1.56884e+00   1.11846e-01
                               ERR DEF= 0.5
 MIGRAD MINIMIZATION HAS CONVERGED.
 MIGRAD WILL VERIFY CONVERGENCE AND ERROR MATRIX.
 COVARIANCE MATRIX CALCULATED SUCCESSFULLY
 FCN=-2419.31 FROM MIGRAD    STATUS=CONVERGED      28 CALLS          29 TOTAL
                     EDM=4.57045e-07    STRATEGY= 1      ERROR MATRIX ACCURATE 
  EXT PARAMETER                                   STEP         FIRST   
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE 
   1  k           -3.53621e+01   3.51925e+00   6.12459e-03   8.76390e-04
   2  m0           5.29040e+00   2.62011e-04   2.96587e-04   7.04254e-02
                               ERR DEF= 0.5
 EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  2    ERR DEF=0.5
  1.253e+01  1.781e-04 
  1.781e-04  6.865e-08 
 PARAMETER  CORRELATION COEFFICIENTS  
       NO.  GLOBAL      1      2
        1  0.19204   1.000  0.192
        2  0.19204   0.192  1.000
 **********
 **   16 **SET ERR         0.5
 **********
 **********
 **   17 **SET PRINT           1
 **********
 **********
 **   18 **HESSE        1000
 **********
 COVARIANCE MATRIX CALCULATED SUCCESSFULLY
 FCN=-2419.31 FROM HESSE     STATUS=OK             10 CALLS          39 TOTAL
                     EDM=4.58038e-07    STRATEGY= 1      ERROR MATRIX ACCURATE 
  EXT PARAMETER                                INTERNAL      INTERNAL  
  NO.   NAME      VALUE            ERROR       STEP SIZE       VALUE   
   1  k           -3.53621e+01   3.51925e+00   2.44984e-04  -2.71425e-01
   2  m0           5.29040e+00   2.62096e-04   1.18635e-05   9.40801e-01
                               ERR DEF= 0.5
 EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  2    ERR DEF=0.5
  1.253e+01  1.782e-04 
  1.782e-04  6.870e-08 
 PARAMETER  CORRELATION COEFFICIENTS  
       NO.  GLOBAL      1      2
        1  0.19207   1.000  0.192
        2  0.19207   0.192  1.000
[#1] INFO:Minimization -- RooAbsMinimizerFcn::setOptimizeConst: deactivating const optimization

Plot likelihood as function of m0

Construct likelihood function of model and data

In [9]:
nll = argus.createNLL(data)

Plot likelihood in m0 in range that includes problematic values In self configuration no messages are printed for likelihood evaluation errors, but if an likelihood value evaluates with error, corresponding value on the curve will be set to the value given in EvalErrorValue().

In [10]:
frame2 = m0.frame(Range=(5.288, 5.293), Title="-log(L) scan vs m0, regions masked")
nll.plotOn(frame2, ShiftToZero=True, PrintEvalErrors=-1, EvalErrorValue=(nll.getVal() + 10), LineColor="r")
frame2.SetMaximum(15)
frame2.SetMinimum(0)

c = ROOT.TCanvas("rf606_nllerrorhandling", "rf606_nllerrorhandling", 1200, 400)
c.Divide(2)
c.cd(1)
ROOT.gPad.SetLeftMargin(0.15)
frame1.GetYaxis().SetTitleOffset(1.4)
frame1.Draw()
c.cd(2)
ROOT.gPad.SetLeftMargin(0.15)
frame2.GetYaxis().SetTitleOffset(1.4)
frame2.Draw()

c.SaveAs("rf606_nllerrorhandling.png")
Info in <TCanvas::Print>: png file rf606_nllerrorhandling.png has been created

Draw all canvases

In [11]:
from ROOT import gROOT 
gROOT.GetListOfCanvases().Draw()