Since version 0.5, lmfit is also capable of calculating the confidence intervals directly. For most models, it is not necessary: the estimation of the standard error from the estimated covariance matrix is normally quite good.
But for some models, e.g. a sum of two exponentials, the approximation begins to fail. For this case, lmfit has the function conf_interval() to calculate confidence inverals directly. This is substantially slower than using the errors estimated from the covariance matrix, but the results are more robust.
The F-test is used to compare our null model, which is the best fit we have
found,with an alternate model, where one of the parameters is fixed to a
specific value. The value is changed until the differnce between
and
can't be explained by the loss of a degree of freedom
within a certain confidence.
N is the number of data-points, P the number of parameter of the null model.
is the number of fixed parameters (or to be more clear, the
difference of number of parameters betweeen our null model and the alternate
model).
A log-likelihood method will be added soon.
First we create a toy problem:
In [1]: import lmfit
In [2]: import numpy as np
In [3]: x=np.linspace(0.3,10,100)
In [4]: y=1/(0.1*x)+2+0.1*np.random.randn(x.size)
In [5]: p=lmfit.Parameters()
In [6]: p.add_many(('a',0.1),('b',1))
In [7]: def residual(p):
...: a=p['a'].value
...: b=p['b'].value
...: return 1/(a*x)+b-y
...:
We have to fit it, before we can generate the confidence intervals.
In [8]: mi=lmfit.minimize(residual, p)
In [9]: mi.leastsq()
Out[9]: True
In [10]: lmfit.printfuncs.report_errors(mi.params)
a: 0.099713 +/- 0.000193 (inital= 0.100000)
b: 1.988121 +/- 0.012165 (inital= 1.000000)
Correlations:
C(a, b) = 0.601
Now it just a simple function call to start the calculation:
In [11]: ci=lmfit.conf_interval(mi)
In [12]: lmfit.printfuncs.report_ci(ci)
99.70% 95.00% 67.40% 0.00% 67.40% 95.00% 99.70%
a 0.09894 0.09894 0.09894 0.09971 0.10049 0.10049 0.10049
99.70% 95.00% 67.40% 0.00% 67.40% 95.00% 99.70%
b 1.95151 1.96413 1.97643 1.98812 1.99981 2.01211 2.02473
As we can see, the estimated error is almost the same: it is not necessary to caclulate ci's for this problem.
Now we look at a problem, where calculating the error from approximated covariance can lead to wrong results:
In [14]: y=3*np.exp(-x/2.)-5*np.exp(-x/10.)+0.2*np.random.randn(x.size)
In [15]: p=lmfit.Parameters()
In [16]: p.add_many(('a1',5),('a2',-5),('t1',2),('t2',5))
In [17]: def residual(p):
....: a1,a2,t1,t2=[i.value for i in p.values()]
....: return a1*np.exp(-x/t1)+a2*np.exp(-x/t2)-y
....:
Now lets fit it:
In [18]: mi=lmfit.minimize(residual, p)
In [19]: mi.leastsq()
Out[19]: True
In [20]: lmfit.printfuncs.report_errors(mi.params, show_correl=False)
a1: 2.611014 +/- 0.327959 (inital= 5.000000)
a2: -4.512928 +/- 0.399194 (inital= -5.000000)
t1: 1.569477 +/- 0.334505 (inital= 2.000000)
t2: 10.961366 +/- 1.263868 (inital= 5.000000)
Again we call conf_interval(), this time with tracing and only for 1- and 2-sigma:
In [21]: ci, trace = lmfit.conf_interval(mi,sigmas=[0.68,0.95],trace=True, verbose=0)
In [22]: lmfit.printfuncs.report_ci(ci)
95.00% 68.00% 0.00% 68.00% 95.00%
a1 2.11696 2.33696 2.61101 3.06631 4.28728
95.00% 68.00% 0.00% 68.00% 95.00%
a2 -6.39492 -5.05982 -4.51293 -4.19528 -3.97850
95.00% 68.00% 0.00% 68.00% 95.00%
t2 8.00414 9.62688 10.96137 12.17947 13.34824
95.00% 68.00% 0.00% 68.00% 95.00%
t1 1.07036 1.28482 1.56948 1.97534 2.64341
If you compare the calculated error estimates, you will see that the regular estimate is too small. Now let's plot a confidence region:
In [23]: import matplotlib.pylab as plt
In [24]: x, y, grid=lmfit.conf_interval2d(mi,'a1','t2',30,30)
In [25]: plt.contourf(x,y,grid,np.linspace(0,1,11))
Out[25]: <matplotlib.contour.QuadContourSet instance at 0xa888d6c>
In [26]: plt.xlabel('a1');
In [27]: plt.colorbar();
In [28]: plt.ylabel('t2');
Remember the trace? It shows the dependence between two parameters.
In [33]: x,y,prob=trace['a1']['a1'], trace['a1']['t2'],trace['a1']['prob']
In [34]: x2,y2,prob2=trace['t2']['t2'], trace['t2']['a1'],trace['t2']['prob']
In [35]: plt.scatter(x,y,c=prob,s=30)
Out[35]: <matplotlib.collections.PathCollection at 0xab7cb6c>
In [36]: plt.scatter(x2,y2,c=prob2,s=30)
Out[36]: <matplotlib.collections.PathCollection at 0xab933ec>
Calculates the confidence interval for parameters from the given minimizer.
The parameter for which the ci is calculated will be varied, while the remaining parameters are reoptimized for minimizing chi-square. The resulting chi-square is used to calculate the probability with a given statistic e.g. F-statistic. This function uses a 1d-rootfinder from scipy to find the values resulting in the searched confidence region.
Parameters : | minimizer : Minimizer
p_names : list, optional
sigmas : list, optional
trace : bool, optional
|
---|---|
Returns : | output : dict
trace_dict : dict
|
Other Parameters: | |
maxiter : int
prob_func : None or callable
|
See also
Examples
>>> from lmfit.printfuncs import *
>>> mini=minimize(some_func, params)
>>> mini.leastsq()
True
>>> report_errors(params)
... #report
>>> ci=conf_interval(mini)
>>> report_ci(ci)
... #report
Now with quantils for the sigmas and using the trace.
>>> ci, trace=conf_interval(mini, sigmas=(0.25,0.5,0.75,0.999),trace=True)
>>> fixed=trace['para1']['para1']
>>> free=trace['para1']['not_para1']
>>> prob=trace['para1']['prob']
This makes it possible to plot the dependence between free and fixed.
Calculates confidence regions for two fixed parameters.
The method is explained in conf_interval: here we are fixing two parameters.
Parameters : | minimizer : minimizer
x_name : string
y_name : string
nx, ny : ints, optional
limits : tuple: optional
|
---|---|
Returns : | x : (nx)-array
y : (ny)-array
grid : (nx,ny)-array
|
Other Parameters: | |
prob_func : None or callable
|
Examples
>>> from lmfit.printfuncs import *
>>> mini=minimize(some_func, params)
>>> mini.leastsq()
True
>>> x,y,gr=conf_interval2d('para1','para2')
>>> plt.contour(x,y,gr)