APP#

class mlquantify.evaluation.protocol.APP(batch_size, n_prevalences, repeats=1, random_state=None)[source]#

Artificial Prevalence Protocol (APP) for evaluation. This protocol generates artificial prevalence distributions for the evaluation in an exhaustive manner, testing all possible combinations of prevalences.

Parameters:

batch_sizeint or list of int: The size of the batches to be used in the evaluation.
n_prevalencesint: The number of artificial prevalences to generate.
repeatsint, optional: The number of times to repeat the evaluation with different random seeds.
random_stateint, optional: The random seed for reproducibility.

Attributes:

n_prevalencesint: The number of artificial prevalences to generate.
repeatsint: The number of times to repeat the evaluation with different random seeds.
random_stateint: The random seed for reproducibility.

Notes

It is important to note that in case of multiclass problems, the time complexity of this protocol can be significantly higher due to the increased number of combinations to evaluate.

Examples

>>> protocol = APP(batch_size=[100, 200], n_prevalences=5, repeats=3, random_state=42)
>>> for train_idx, test_idx in protocol.split(X, y):
...     # Train and evaluate model
...     pass

get_n_combinations() → int[source]#: Get the number of combinations for the current protocol.

split(X: ndarray, y: ndarray)[source]#

Split the data into samples for evaluation.

Parameters:

Xnp.ndarray: The input features.
ynp.ndarray: The target labels.

Yields:

Generator[np.ndarray, np.ndarray]: A generator that yields the indices for each split.

APP#

This Page