ErrorByShiftDisplay#

class mlquantify.visualization.ErrorByShiftDisplay(true_prevalences, predicted_prevalences, *, train_prevalence=None, error_metric='ae', smoothing=None)[source]#

Estimation error as a function of prior-probability shift.

For each evaluation sample, the amount of shift is the absolute-error distance between the sample’s true prevalence and a reference (training) prevalence; the error is a quantification metric between the predicted and true prevalence. Samples are grouped into shift bins and the mean error per bin is drawn as a curve, optionally with a ±std band. This reveals how a quantifier degrades as the test distribution drifts away from training — a standard robustness diagnostic in the learning-to-quantify literature.

This is a multiple-sample display.

Parameters:

true_prevalencesndarray of shape (n_samples, n_classes): True prevalence of each evaluation sample.
predicted_prevalencesndarray of shape (n_samples, n_classes): Predicted prevalence of each evaluation sample.
train_prevalencendarray of shape (n_classes,), default=None: Reference prevalence against which shift is measured. When None, the mean true prevalence across samples is used.
error_metric{‘ae’, ‘se’, ‘rae’}, default=’ae’: Per-sample error measure plotted on the y-axis.
smoothingfloat, default=None: Additive (Forman) smoothing eps applied before computing RAE, to avoid division by zero on zero-prevalence classes. When None, a small default of 1e-3 is used (from_protocol overrides this with 1 / (2 * batch_size)). Ignored by non-relative metrics.

Attributes:

shift_ndarray of shape (n_samples,): Per-sample shift magnitude.
error_ndarray of shape (n_samples,): Per-sample error.
line_matplotlib Line2D: The mean-error curve.
fill_matplotlib PolyCollection or None: The ±std band (None when show_std=False).
ax_matplotlib Axes
figure_matplotlib Figure

Examples

>>> from mlquantify.visualization import ErrorByShiftDisplay
>>> from mlquantify.counting import CC
>>> from sklearn.linear_model import LogisticRegression
>>> from sklearn.datasets import make_classification
>>> X, y = make_classification(n_samples=400, random_state=0)
>>> disp = ErrorByShiftDisplay.from_protocol(   
...     CC(LogisticRegression()), X, y, protocol="upp", error_metric="ae")

classmethod from_predictions(true_prevalences, predicted_prevalences, *, train_prevalence=None, error_metric='ae', smoothing=None, ax=None, **kwargs)[source]#: Build an ErrorByShiftDisplay from precomputed arrays.

classmethod from_protocol(quantifier, X, y, *, protocol='upp', error_metric='ae', ax=None, name=None, n_bins=10, show_std=True, **protocol_kwargs)[source]#

Run a protocol and plot error against prior-probability shift.

The reference (training) prevalence is taken from the class proportions of y. **protocol_kwargs are forwarded to mlquantify.model_selection.apply_protocol. The uniform protocol ('upp') is the default because it spreads samples across the shift range.

plot(ax=None, *, n_bins=10, name=None, show_std=True, **kwargs)[source]#

Draw the error-by-shift curve.

Parameters:

axmatplotlib Axes, default=None: Axes to draw on.
n_binsint, default=10: Number of equal-width shift bins.
namestr, default=None: Legend label for the curve.
show_stdbool, default=True: Whether to draw a ±std shaded band.
**kwargs: Forwarded to ax.plot (the mean-error curve).

Returns:

displayErrorByShiftDisplay