arviz.loo#
- arviz.loo(data, pointwise=None, var_name=None, reff=None, scale=None)[source]#
Compute Pareto-smoothed importance sampling leave-one-out cross-validation (PSIS-LOO-CV).
Estimates the expected log pointwise predictive density (elpd) using Pareto-smoothed importance sampling leave-one-out cross-validation (PSIS-LOO-CV). Also calculates LOO’s standard error and the effective number of parameters. Read more theory here https://arxiv.org/abs/1507.04544 and here https://arxiv.org/abs/1507.02646
- Parameters
- data: obj
Any object that can be converted to an
arviz.InferenceData
object. Refer to documentation ofarviz.convert_to_dataset()
for details.- pointwise: bool, optional
If True the pointwise predictive accuracy will be returned. Defaults to
stats.ic_pointwise
rcParam.- var_namestr, optional
The name of the variable in log_likelihood groups storing the pointwise log likelihood data to use for loo computation.
- reff: float, optional
Relative MCMC efficiency,
ess / n
i.e. number of effective samples divided by the number of actual samples. Computed from trace by default.- scale: str
Output scale for loo. Available options are:
log
: (default) log-scorenegative_log
: -1 * log-scoredeviance
: -2 * log-score
A higher log-score (or a lower deviance or negative log_score) indicates a model with better predictive accuracy.
- Returns
- ELPDData object (inherits fromclass:pandas.Series) with the following row/attributes:
- loo: approximated expected log pointwise predictive density (elpd)
- loo_se: standard error of loo
- p_loo: effective number of parameters
- shape_warn: bool
True if the estimated shape parameter of Pareto distribution is greater than 0.7 for one or more samples
- loo_i: array of pointwise predictive accuracy, only if pointwise True
- pareto_k: array of Pareto shape values, only if pointwise True
- loo_scale: scale of the loo results
The returned object has a custom print method that overrides pd.Series method.
See also
compare
Compare models based on PSIS-LOO loo or WAIC waic cross-validation.
waic
Compute the widely applicable information criterion.
plot_compare
Summary plot for model comparison.
plot_elpd
Plot pointwise elpd differences between two or more models.
plot_khat
Plot Pareto tail indices for diagnosing convergence.
Examples
Calculate LOO of a model:
In [1]: import arviz as az ...: data = az.load_arviz_data("centered_eight") ...: az.loo(data) ...: Out[1]: Computed from 2000 posterior samples and 8 observations log-likelihood matrix. Estimate SE elpd_loo -30.81 1.43 p_loo 0.95 - ------ Pareto k diagnostic values: Count Pct. (-Inf, 0.5] (good) 6 75.0% (0.5, 0.7] (ok) 2 25.0% (0.7, 1] (bad) 0 0.0% (1, Inf) (very bad) 0 0.0%
Calculate LOO of a model and return the pointwise values:
In [2]: data_loo = az.loo(data, pointwise=True) ...: data_loo.loo_i ...: Out[2]: <xarray.DataArray 'loo_i' (school: 8)> array([-4.91696767, -3.41751961, -3.85571898, -3.46734224, -3.40488294, -3.47796516, -4.31633415, -3.95382766]) Coordinates: * school (school) object 'Choate' 'Deerfield' ... "St. Paul's" 'Mt. Hermon'