modelsight.calibration.calib

This file deals with the implementation of the Hosmer-Lemeshow plot for the assessment of calibration of predicted probabilities.

Module Contents

Functions

ntile_name(→ str)

Returns the ntile name corresponding to an ntile integer.

make_recarray(→ numpy.recarray)

Combines arrays into a recarray.

hosmer_lemeshow_table(→ numpy.recarray)

Constructs a Hosmer–Lemeshow table.

hosmer_lemeshow_plot(, annotate_bars, title, ...)

Plot observed vs. predicted probabilities (can be risks in case of survival models), groups by
modelsight.calibration.calib.ntile_name(n: int) → str[source]

Returns the ntile name corresponding to an ntile integer.

Parameters:

n (int) – An ntile integer.

Returns:

ntile_name – The corresponding ntile name.

Return type:

str

modelsight.calibration.calib.make_recarray(y_true: modelsight._typing.ArrayLike, y_pred: modelsight._typing.ArrayLike) → numpy.recarray[source]

Combines arrays into a recarray.

Parameters:

  • y_true (array) – Observed labels, either 0 or 1.

  • y_pred (array) – Predicted probabilities, floats on [0, 1].

Returns:

table – A record array with observed label and predicted probability columns, sorted by predicted probability.

Return type:

recarray

modelsight.calibration.calib.hosmer_lemeshow_table(y_true: modelsight._typing.ArrayLike, y_pred: modelsight._typing.ArrayLike, n_bins: int = 10) → numpy.recarray[source]

Constructs a Hosmer–Lemeshow table.

Parameters:

  • y_true (array) – Observed labels, either 0 or 1.

  • y_pred (array) – Predicted probabilities, floats on [0, 1].

  • n_bins (int, optional) – The number of groups to create. The default value is 10, which corresponds to deciles of predicted probabilities.

Returns:

table – A record array with n_bins rows and four columns: Group Size, Observed Frequency, Predicted Frequency, and Mean Probability.

Return type:

recarray

modelsight.calibration.calib.hosmer_lemeshow_plot(y_true: modelsight._typing.ArrayLike, y_pred: modelsight._typing.ArrayLike, n_bins: int = 10, colors: Tuple[str, str] = ('blue', 'red'), annotate_bars: bool = True, title: str = '', brier_score_annot: str = '', ax: matplotlib.pyplot.Axes = None, **kwargs) → Tuple[matplotlib.pyplot.Figure, matplotlib.pyplot.Axes][source]

Plot observed vs. predicted probabilities (can be risks in case of survival models), groups by n-tiles of predicted probabilities.

Parameters:

  • y_true (ArrayLike) – (n_obs,) shaped array of ground-truth values

  • y_pred (ArrayLike) – (n_obs,) shaped array of predicted probabilities

  • n_bins (int) – Number of bins to group observed and predicted probabilities into

  • colors (Tuple[str, str]) – Pair of colors for observed (line) and predicted (vertical bars) probabilities.

  • annotate_bars (bool) – Whether bars should be annotated with the number of observed probabilities in each bin.

  • title (str) – Title to display on top of the calibration plot.

  • brier_score_annot (str) – Optional brier score (95% CI) annotation on the top-left corner.

  • ax (plt.Axes) – A matplotlib Axes object to draw the calibration plot into. If None, an Axes object is created by default.

Returns:

f, ax – f: pyplot figure ax: pyplot Axes

Return type:

Tuple[plt.Figure, plt.Axes]