1.5. Nearest Neighbors#

1.5.1. PWK: Pair-wise Weighted K-Nearest Neighbors#

PWK and PWK\(\alpha\) are neighbor-based algorithms (k-Nearest Neighbors - NN) that apply class weighting techniques to estimate prevalence in binary problems [2]. They can be seen as aggregative quantification methods that adjust a k-NN classifier to account for class imbalance.

PWK methods focus on simplicity and stability, offering competitive quantification that balances effectiveness with interpretability. By exploring the topology of the data through proximity-based analysis, these algorithms are robust to distribution changes and retain local structure better than global classifiers [2]. The methods complement k-NN with class weighting policies designed to neutralize the bias toward majority classes inherent in imbalanced datasets [2].

1.5.1.1. PWK (Proportion-Weighted K-Nearest Neighbor)#

This is the simplest version of the algorithm.

  • Mechanism: The weight \(w_c\) is defined to be inversely proportional to the size of class \(c\) relative to the total training set size [2].

  • Simplicity: The weights are easily interpretable, and the method only requires the calibration of the number of neighbors \(k\).

Mathematical details - PWK Weights

The weight for a class \(\gamma_j\) is calculated as:

\[w_{\gamma_j} = \frac{1 - \frac{m_{\gamma_j}}{m}}{m_{\gamma_j}}\]

Where \(m\) is the total training size and \(m_{\gamma_j}\) is the count of samples in class \(\gamma_j\).

1.5.1.2. PWK\(\alpha\) (Alpha-Adjusted PWK)#

PWK\(\alpha\) is a general structure that encompasses both standard KNN and PWK as special cases [2].

  • Mechanism: It introduces a tunable parameter \(\alpha\) into the weighting formula.

  • Flexibility: \(\alpha\) is a free parameter that allows the model to adapt to specific datasets. As \(\alpha\) increases, the penalty for larger classes is progressively smoothed [2].

  • Relationship with KNN and PWK:

    • When \(\alpha \to \infty\), the class weights converge to 1, and the algorithm behaves like a traditional KNN.

    • When \(\alpha = 1\), the algorithm is equivalent to standard PWK.

  • Performance: Empirically, there is often no significant statistical difference in Absolute Error between PWK and PWK\(\alpha\). However, PWK\(\alpha\) tends to be more conservative and robust at lower prevalences, while PWK may be more competitive at higher prevalences [2].

Mathematical details - PWK-Alpha Weights

The weight for class \(\gamma_j\) is a ratio between the class cardinality and the minority class cardinality (\(M\)), raised to the power of \(1/\alpha\):

\[w_{\gamma_j}(\alpha) = \left( \frac{M}{m_{\gamma_j}} \right)^{-1/\alpha}, \quad \text{with } \alpha \ge 1\]

Example

from mlquantify.neighbors import PWK

# PWK operates directly on the feature space using k-NN
q = PWK(n_neighbors=10)
q.fit(X_train, y_train)
q.predict(X_test)
References