Using external minimizers

We show how to use an external minimizer to find the minimum of a function and then use iminuit to compute the parameter uncertainties.

We will demonstrate this with a maximum-likelihood fit of a normal distribution, which is carried out with scipy.optimize.minimize. iminuit is then used to compute the parameter uncertainties.

Note: iminuit can call the scipy minimizers directly with Minuit.scipy, so scipy.optimize.minimize is only used here to demonstrate the general approach.

[1]:
from iminuit import Minuit
import numpy as np
from scipy.stats import norm
from scipy.optimize import minimize
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xf
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Input In [1], in <cell line: 3>()
      1 from iminuit import Minuit
      2 import numpy as np
----> 3 from scipy.stats import norm
      4 from scipy.optimize import minimize

File /usr/lib/python3.10/site-packages/scipy/stats/__init__.py:467, in <module>
      1 """
      2 .. _statsrefmanual:
      3
   (...)
    462
    463 """
    465 from ._warnings_errors import (ConstantInputWarning, NearConstantInputWarning,
    466                                DegenerateDataWarning, FitError)
--> 467 from ._stats_py import *
    468 from ._variation import variation
    469 from .distributions import *

File /usr/lib/python3.10/site-packages/scipy/stats/_stats_py.py:39, in <module>
     36 from numpy.lib import NumpyVersion
     37 from numpy.testing import suppress_warnings
---> 39 from scipy.spatial.distance import cdist
     40 from scipy.ndimage import _measurements
     41 from scipy._lib._util import (check_random_state, MapWrapper,
     42                               rng_integers, _rename_parameter)

File /usr/lib/python3.10/site-packages/scipy/spatial/__init__.py:105, in <module>
      1 """
      2 =============================================================
      3 Spatial algorithms and data structures (:mod:`scipy.spatial`)
   (...)
    102    QhullError
    103 """
--> 105 from ._kdtree import *
    106 from ._ckdtree import *
    107 from ._qhull import *

File /usr/lib/python3.10/site-packages/scipy/spatial/_kdtree.py:5, in <module>
      3 import numpy as np
      4 import warnings
----> 5 from ._ckdtree import cKDTree, cKDTreeNode
      7 __all__ = ['minkowski_distance_p', 'minkowski_distance',
      8            'distance_matrix',
      9            'Rectangle', 'KDTree']
     12 def minkowski_distance_p(x, y, p=2):

File _ckdtree.pyx:10, in init scipy.spatial._ckdtree()

File /usr/lib/python3.10/site-packages/scipy/sparse/__init__.py:267, in <module>
    264 import warnings as _warnings
    266 from ._base import *
--> 267 from ._csr import *
    268 from ._csc import *
    269 from ._lil import *

File /usr/lib/python3.10/site-packages/scipy/sparse/_csr.py:10, in <module>
      7 import numpy as np
      9 from ._base import spmatrix
---> 10 from ._sparsetools import (csr_tocsc, csr_tobsr, csr_count_blocks,
     11                            get_csr_submatrix)
     12 from ._sputils import upcast, get_index_dtype
     14 from ._compressed import _cs_matrix

ImportError: numpy.core.multiarray failed to import
[2]:
# normally distributed data
rng = np.random.default_rng(1)
x = rng.normal(size=1000)

# negative log-likelihood for a normal distribution
def nll(par):
    return -np.sum(norm.logpdf(x, par[0], par[1]))

nll.errordef = Minuit.LIKELIHOOD

# minimize nll with scipy.optimize.minimize
result = minimize(nll, np.ones(2))
result
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Input In [2], in <cell line: 12>()
      9 nll.errordef = Minuit.LIKELIHOOD
     11 # minimize nll with scipy.optimize.minimize
---> 12 result = minimize(nll, np.ones(2))
     13 result

NameError: name 'minimize' is not defined
[3]:
# initialize Minuit with the fit result from scipy.optimize.minimize
m = Minuit(nll, result.x)
m.hesse()  # this also works without calling MIGRAD before
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Input In [3], in <cell line: 2>()
      1 # initialize Minuit with the fit result from scipy.optimize.minimize
----> 2 m = Minuit(nll, result.x)
      3 m.hesse()

NameError: name 'result' is not defined

We can also compute the “Hesse errors” at any other point than the minimum. These cannot be interpreted as parameter uncertainties, they are just some numbers related to the second derivative of the cost function at that point.

[4]:
m.values = (1.0, 0.5)
m.hesse()
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Input In [4], in <cell line: 1>()
----> 1 m.values = (1.0, 0.5)
      2 m.hesse()

NameError: name 'm' is not defined

Minuit now reports that the minimum is invalid, which is correct, but it does not matter for the Hesse errors, which are computed anyway.

Likewise, it one can also run MINOS to get MINOS estimates. Note that MINOS can fail if the starting point is not actually a minimum. So here we reset the values to the solution found by scipy.optimize.

[5]:
m.values = result.x
m.minos()
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Input In [5], in <cell line: 1>()
----> 1 m.values = result.x
      2 m.minos()

NameError: name 'result' is not defined

We can see that MINOS ran successfully. The Hesse Errors were also updated, because MINOS needs HESSE to run first. HESSE is called automatically in this case.