Protocol

class mlquantify.evaluation.protocol.Protocol(batch_size, random_state=None, **kwargs)

Base class for evaluation protocols.

Parameters:

batch_sizeint or list of int

The size of the batches to be used in the evaluation.

random_stateint, optional

The random seed for reproducibility.

Attributes:

n_combinationsint

Raises:

ValueError

If the batch size is not a positive integer or list of positive integers.

Notes

This class serves as a base class for different evaluation protocols, each with its own strategy for splitting the data into batches.

Examples

>>> class MyCustomProtocol(Protocol):
...     def _iter_indices(self, X: np.ndarray, y: np.ndarray) -> Generator[np.ndarray]:
...         for batch_size in self.batch_size:
...             yield np.random.choice(X.shape[0], batch_size, replace=True)
...
>>> protocol = MyCustomProtocol(batch_size=100, random_state=42)
>>> for train_idx, test_idx in protocol.split(X, y):
...     # Train and evaluate model
...     pass

get_n_combinations() → int

Get the number of combinations for the current protocol.

split(X: ndarray, y: ndarray)

Split the data into samples for evaluation.

Parameters:

Xnp.ndarray

The input features.

ynp.ndarray

The target labels.

Yields:

Generator[np.ndarray, np.ndarray]

A generator that yields the indices for each split.