arviz_stats.summary

Contents

arviz_stats.summary#

arviz_stats.summary(data, var_names=None, filter_vars=None, group='posterior', coords=None, sample_dims=None, kind='all', fmt='wide', ci_prob=None, ci_kind=None, round_to='auto', skipna=False)[source]#

Create a data frame with summary statistics and or diagnostics.

Parameters:
dataxarray.DataTree, DataSet or InferenceData
var_nameslist of str, optional

Names of variables to include in summary. If None all variables are included.

filter_vars: {None, “like”, “regex”}, default None

Used for var_names only. If None (default), interpret var_names as the real variables names. If “like”, interpret var_names as substrings of the real variables names. If “regex”, interpret var_names as regular expressions on the real variables names.

group: str

Select a group for summary. Defaults to “posterior”.

coordsdict, optional

Coordinates defining a subset over the selected group.

sample_dimsstr or sequence of hashable, optional

Defaults to rcParams["data.sample_dims"]

kind: {‘all’, ‘stats’, ‘diagnostics’, ‘all_median’, ‘stats_median’,
‘diagnostics_median’, ‘mc_diagnostics’}, default ‘all’

  • all: mean, sd, ci, ess_bulk, ess_tail, r_hat, mcse_mean, mcse_sd.

  • stats: mean, sd, and ci.

  • diagnostics: ess_bulk, ess_tail, r_hat, mcse_mean, mcse_sd.

  • all_median: median, mad, ci, ess_median, ess_tail, r_hat, mcse_median.

  • stats_median: median, mad, and ci.

  • diagnostics_median: ess_median, ess_tail, r_hat, mcse_median.

  • mc_diagnostics: mcse_mean, ess_mean, and min_ss.

fmt: {‘wide’, ‘long’, ‘xarray’}

Return format is either pandas.DataFrame {‘wide’, ‘long’} or xarray.Dataset {‘xarray’}.

ci_probfloat, optional

Probability for the credible interval. Defaults to rcParams["stats.ci_prob"].

ci_kind{“hdi”, “eti”}, optional

Type of credible interval. Defaults to rcParams["stats.ci_kind"]. If kind is stats_median or all_median, ci_kind is forced to “eti”.

round_toint or {“auto”, “none”}, optional

Rounding specification. Defaults to “auto”. If integer, number of decimal places to round to. Use the string “None” or “none” to return raw numbers. If None use rcParams["stats.round_to"]. If “auto”, and fmt is “xarray” defaults to rcParams["stats.round_to"]. If “auto” and fmt is in {“wide”, “long”}, applies the following rounding rules:

  • ESS values (ess_bulk, ess_tail, ess_mean, ess_median, min_ss) are rounded down to int

  • R-hat always shows 2 digits after the decimal

  • If a column stat and mcse_stat are both present then the mcse is shown to 2 significant figures, and stat is shown with precision based on 2*mcse.

  • All other floating point numbers are shown following rcParams["stats.round_to"].

  • For all floating point numbers except R-hat, trailing zeros are removed and values are converted to string for consistent display.

skipna: bool

If true ignores nan values when computing the summary statistics. Defaults to false.

Returns:
SummaryDataFrame, pandas.DataFrame or xarray.Dataset

Return type determined by fmt argument.

See also

rhat

Compute estimate of rank normalized split R-hat for a set of traces.

ess

Calculate the effective sample size of a set of traces.

mcse

Calculate Markov Chain Standard Error statistic.

plot_ess

Plot quantile, local or evolution of effective sample sizes (ESS).

plot_mcse

Plot quantile, local or evolution of Markov Chain Standard Error (MCSE).

Notes

When round_to="auto", the returned object is a SummaryDataFrame (a subclass of pandas.DataFrame). It retains raw values under the hood but applies custom formatting intended for sensible DISPLAY of decimal places and significant figures.

Formatting is only guaranteed to be preserved for basic view-altering DataFrame operations, like row filtering or slicing (e.g., df.loc[["mu", "tau"]]) or Transposition.

Formatting will be lost if the structure is modified via renaming, concatenating, merging, or if the DataFrame is converted to other objects like a pandas.Series (e.g., df["r_hat"]), NumPy arrays, or Xarray datasets.

File export methods like df.to_csv() will write the unrounded,raw data to disk rather than the formatted strings.

Examples

In [1]: from arviz_base import load_arviz_data
   ...: from arviz_stats import summary
   ...: data = load_arviz_data("non_centered_eight")
   ...: summary(data, var_names=["mu", "tau"])
   ...: 
Out[1]: 
    mean   sd eti89_lb eti89_ub ess_bulk ess_tail r_hat mcse_mean mcse_sd
mu   4.3  3.3    -0.75      9.4     2114     1219  1.00     0.072   0.052
tau  3.5  3.2     0.22      9.4      833      712  1.00     0.091    0.13

You can use filter_vars to select variables without having to specify all the exact names. Use filter_vars="like" to select based on partial naming:

In [2]: summary(data, var_names=["the"], filter_vars="like")
Out[2]: 
                            mean    sd eti89_lb eti89_ub ess_bulk ess_tail r_hat mcse_mean mcse_sd
theta_t[Choate]             0.33     1     -1.3      1.9     2249     1467  1.00     0.021   0.015
theta_t[Deerfield]           0.1  0.95     -1.4      1.7     2638     1512  1.00     0.018   0.013
theta_t[Phillips Andover]  -0.08  0.97     -1.6      1.4     2689     1511  1.00     0.019   0.013
theta_t[Phillips Exeter]    0.05  0.93     -1.4      1.5     2431     1586  1.00     0.019   0.014
theta_t[Hotchkiss]         -0.15  0.91     -1.6      1.3     2190     1545  1.00      0.02   0.014
theta_t[Lawrenceville]     -0.03  0.94     -1.6      1.5     2372     1297  1.01     0.019   0.014
theta_t[St. Paul's]         0.35  0.99     -1.2      1.9     1980     1391  1.00     0.022   0.015
theta_t[Mt. Hermon]         0.05  0.99     -1.6      1.6     2420     1538  1.00      0.02   0.015
theta[Choate]                6.1   5.3     -1.1       15     2195     1443  1.00      0.12    0.12
theta[Deerfield]             4.9   4.7       -2       13     2321     1499  1.00     0.097     0.1
theta[Phillips Andover]      3.8   5.3     -4.6       11     2108     1504  1.00      0.12    0.15
theta[Phillips Exeter]       4.6   4.7     -2.7       12     2288     1434  1.00       0.1   0.091
theta[Hotchkiss]             3.6   4.6     -4.1       10     2103     1609  1.00       0.1   0.091
theta[Lawrenceville]         4.3   4.8     -2.9       12     2364     1662  1.00       0.1   0.097
theta[St. Paul's]            6.2   5.2    -0.64       15     1431     1629  1.00      0.14    0.16
theta[Mt. Hermon]            4.7   5.2     -2.8       13     2187     1569  1.00      0.12    0.12

Use filter_vars="regex" to select based on regular expressions, and prefix the variables you want to exclude by ~. Here, we exclude from the summary all the variables starting with the letter t:

In [3]: summary(data, var_names=["~^t"], filter_vars="regex")
Out[3]: 
   mean   sd eti89_lb eti89_ub ess_bulk ess_tail r_hat mcse_mean mcse_sd
mu  4.3  3.3    -0.75      9.4     2114     1219  1.00     0.072   0.052
Contents