modelsight.curves._delong

This file deals with the implementation of the DeLong test for the comparison of pairs of correlated areas under the receiver-operating characteristics curves.

Module Contents

Functions

compute_midrank(→ numpy.ndarray)

Computes midranks.

fastDeLong(→ Tuple[numpy.ndarray, numpy.ndarray])

The fast version of DeLong's method for computing the covariance of

calc_pvalue(→ float)

Computes log(10) of p-values.

compute_ground_truth_statistics(→ Tuple[numpy.ndarray, ...)

Compute statistics of ground-truth array.

delong_roc_test(→ float)

Compare areas-under-curve of two estimators using the DeLong test.
modelsight.curves._delong.compute_midrank(x: numpy.ndarray) → numpy.ndarray[source]

Computes midranks.

Parameters:

x (np.ndarray) – a 1-d array of predicted probabilities.

Returns:

T2 – array of midranks

Return type:

np.ndarray

modelsight.curves._delong.fastDeLong(predictions_sorted_transposed: numpy.ndarray, label_1_count: int) → Tuple[numpy.ndarray, numpy.ndarray][source]

The fast version of DeLong’s method for computing the covariance of unadjusted AUC.

Parameters:

predictions_sorted_transposed (a (n_classifiers, n_obs) numpy array containing) – the predicted probabilities by the two classifiers in the comparison. These probabilities are sorted such that the examples with label “1” come first.

Returns:

  • aucs, delongcov (Tuple[np.ndarray, np.ndarray]) – aucs: array of AUC values delongcov: array of DeLong covariance

  • Reference

  • ———

  • @article{sun2014fast, –

    title={Fast Implementation of DeLong’s Algorithm for

    Comparing the Areas Under Correlated Receiver Operating Characteristic Curves},

    author={Xu Sun and Weichao Xu}, journal={IEEE Signal Processing Letters}, volume={21}, number={11}, pages={1389–1393}, year={2014}, publisher={IEEE}

  • }

modelsight.curves._delong.calc_pvalue(aucs: numpy.ndarray, sigma: numpy.ndarray) → float[source]

Computes log(10) of p-values.

Parameters:

  • aucs (np.array) – a 1-d array of AUCs

  • sigma (np.array) – an array AUC DeLong covariances

Returns:

p – log10(pvalue)

Return type:

float

modelsight.curves._delong.compute_ground_truth_statistics(ground_truth: numpy.ndarray) → Tuple[numpy.ndarray, int][source]

Compute statistics of ground-truth array.

Parameters:

ground_truth (np.ndarray) – a (n_obs,) array of 0 and 1 values representing the ground-truth.

Returns:

order, label_1_count – order is a numpy array of sorted indexes label_1_count is the count of data points of the positive class.

Return type:

Tuple[np.ndarray, int]

modelsight.curves._delong.delong_roc_test(ground_truth: numpy.ndarray, predictions_one: numpy.ndarray, predictions_two: numpy.ndarray) → float[source]

Compare areas-under-curve of two estimators using the DeLong test. Concretely, it computes the pvalue for hypothesis that two ROC AUCs are different.

Parameters:

  • ground_truth (np.ndarray) – a (n_obs,) array of 0 and 1 representing ground-truths.

  • predictions_one (np.ndarray) – a (n_obs,) array of probabilities of class 1 predicted by the first model.

  • predictions_two (np.ndarray) – a (n_obs,) array of probabilities of class 1 predicted by the second model.

Returns:

p – the p-value for hypothesis that two ROC AUCs are different.

Return type:

float