User Guide

• 1. Foundations
  • 1.1. What Is Quantification?
    • 1.1.1. When Is Quantification the Right Tool?
  • 1.2. Why Not Just Classify and Count?
    • 1.2.1. The CC Bias
  • 1.3. Dataset Shift
    • 1.3.1. Prior Probability Shift (Label Shift)
    • 1.3.2. Covariate Shift
    • 1.3.3. Concept Shift (Concept Drift)
  • 1.4. The Aggregative Quantification Framework
    • 1.4.1. The Role of Cross-Validation in fit
    • 1.4.2. The estimator_fitted parameter
  • 1.5. Method Families at a Glance
  • 1.6. Choosing the Right Evaluation Protocol
    • 1.6.1. Choosing the Right Evaluation Metric
  • 1.7. Multiclass Quantification
  • 1.8. Further Reading
• 2. Aggregative Quantification
  • 2.1. Using Aggregative Quantification Methods
  • 2.2. General Concept
    • 2.2.1. Examples
  • 2.3. Counting Methods
    • 2.3.1. CC — Classify and Count
    • 2.3.2. PCC — Probabilistic Classify and Count
    • 2.3.3. GACC — Generalised Adjusted Classify and Count
    • 2.3.4. GPACC — Generalised Probabilistic Adjusted Classify and Count
    • 2.3.5. Choosing Among CC, PCC, GACC, and GPACC
    • 2.3.6. References
  • 2.4. Adjusted Counting
    • 2.4.1. Problem formulation
    • 2.4.2. The Adjustment Formula
    • 2.4.3. ACC — Adjusted Classify and Count (hard predictions)
    • 2.4.4. ThresholdAdjustment — Base Class for ROC-Threshold Methods
    • 2.4.5. TAC — Threshold Adjusted Count (fixed threshold)
    • 2.4.6. TX — Threshold X (symmetric ROC point)
    • 2.4.7. TMAX — Maximum TPR−FPR Separation
    • 2.4.8. T50 — TPR ≈ 0.5 Threshold
    • 2.4.9. MS — Median Sweep
    • 2.4.10. MS2 — Median Sweep with Constraint
    • 2.4.11. Comparing Threshold-Adjustment Methods
    • 2.4.12. Assumptions and when to use
    • 2.4.13. References
  • 2.5. Likelihood Methods
    • 2.5.1. Prior Probability Shift — The Core Assumption
    • 2.5.2. MLPE — Maximum Likelihood Prevalence Estimation (trivial baseline)
    • 2.5.3. EMQ — Expectation-Maximization Quantifier (SLD)
    • 2.5.4. CDE — CDE-Iterate (threshold-adjustment via cost ratios)
    • 2.5.5. Method Comparison
    • 2.5.6. References
  • 2.6. Distribution Matching
    • 2.6.1. Histogram Methods
    • 2.6.2. Density Methods — KDEy
    • 2.6.3. Kernel Methods
    • 2.6.4. Score Methods
    • 2.6.5. Choosing a Distribution Matching Method
    • 2.6.6. Assumptions and when to use
    • 2.6.7. References
  • 2.7. Nearest Neighbours
    • 2.7.1. PWK — Probabilistic Weighted k-Nearest Neighbours
    • 2.7.2. References
• 3. Non Aggregative Quantification
  • 3.1. Mixture Models for Non-Aggregative Quantification
    • 3.1.1. HDx: Hellinger Distance x-Similarity
• 4. Meta Quantification
  • 4.1. Ensembles and Adaptation
    • 4.1.1. EnsembleQ — Ensemble of Quantifiers
    • 4.1.2. QuaDapt — Adaptive Score Simulation
    • 4.1.3. AggregativeBootstrap — Confidence Intervals via Bootstrap
    • 4.1.4. Choosing a Meta-Quantifier
    • 4.1.5. References
  • 4.2. Bootstrap in Quantification
  • 4.3. QuaDapt: Drift-Resilient Score Adaptation
• 5. Model Selection and Evaluation
  • 5.1. Evaluation Protocols
    • 5.1.1. Quick evaluation with apply_protocol
    • 5.1.2. APP — Artificial Prevalence Protocol
    • 5.1.3. NPP — Natural Prevalence Protocol
    • 5.1.4. UPP — Uniform Prevalence Protocol
    • 5.1.5. PPP — Personalized Prevalence Protocol
    • 5.1.6. Choosing a Protocol
    • 5.1.7. References
  • 5.2. Hyperparameter Tuning
    • 5.2.1. References
  • 5.3. Evaluation Metrics
    • 5.3.1. Absolute Error Metrics
    • 5.3.2. Relative Error Metrics
    • 5.3.3. Divergence Metrics
    • 5.3.4. Ordinal Metrics
    • 5.3.5. Distribution Distance Metrics
    • 5.3.6. Choosing a Metric
  • 5.4. Single Label Quantification (SLQ) Metrics
    • 5.4.1. AE (Absolute Error)
    • 5.4.2. SE (Squared Error)
    • 5.4.3. MAE (Mean Absolute Error)
    • 5.4.4. MSE (Mean Squared Error)
    • 5.4.5. KLD (Kullback-Leibler Divergence)
    • 5.4.6. RAE (Relative Absolute Error)
    • 5.4.7. NAE (Normalized Absolute Error)
    • 5.4.8. NRAE (Normalized Relative Absolute Error)
    • 5.4.9. NKLD (Normalized Kullback-Leibler Divergence)
  • 5.5. Regression-Based Quantification (RQ) Metrics
    • 5.5.1. VSE (Variance Shift Error)
    • 5.5.2. CvM_L1 (Cramér-von Mises L1 Distance)
  • 5.6. Ordinal Quantification (OQ) Metrics
    • 5.6.1. NMD (Normalized Match Distance)
    • 5.6.2. RNOD (Relative Normalized Overall Deviation)
• 6. Synthetic Datasets
  • 6.1. How it works
  • 6.2. Controlling the prevalence (prior shift)
  • 6.3. Shift types
  • 6.4. The decision boundary
  • 6.5. Stacking shifts
  • 6.6. Training data and difficulty
• 7. Real-World Datasets
  • 7.1. Shared configuration
  • 7.2. Per-dataset configuration
    • 7.2.1. Binary (tabular)
    • 7.2.2. Multiclass (tabular)
    • 7.2.3. Ordinal
    • 7.2.4. Text
    • 7.2.5. Temporal / time-shift
    • 7.2.6. Image, graph & synthetic (other shift)
  • 7.3. Optional download progress
  • 7.4. References
• 8. Confidence Intervals
  • 8.1. General Concept
  • 8.2. Percentile-Based Confidence Intervals
  • 8.3. Confidence Ellipse in Simplex
  • 8.4. Confidence Ellipse in CLR Space
    • 8.4.1. References
• 9. Calibration
  • 9.1. Classifier calibration
  • 9.2. Use with EMQ
  • 9.3. Quantifier calibration
  • 9.4. Custom calibrators
  • 9.5. References
• 10. Visualization
  • 10.1. Multiple-sample displays
    • 10.1.1. Diagonal plot
    • 10.1.2. Bias boxplots
    • 10.1.3. Error by prior-probability shift
  • 10.2. Single-sample displays
    • 10.2.1. Prevalence bars
    • 10.2.2. Confidence regions
  • 10.3. Combining plots on one figure
• 11. Building a Quantifier
  • 11.1. General Quantifiers
  • 11.2. Aggregative Quantifiers
  • 11.3. Aggregation Types
  • 11.4. Binary Quantifiers
  • 11.5. Validation Utilities
  • 11.6. Parameter Constraints
• 12. Mlquantify methods
• 13. Core Components
  • 13.1. Building blocks of a quantifier
    • 13.1.1. Representations
    • 13.1.2. Loss Functions
    • 13.1.3. Solvers
    • 13.1.4. Composable Quantifiers
  • 13.2. Multiclass and output handling
    • 13.2.1. Multiclass Quantification
    • 13.2.2. Prevalence Normalization
  • 13.3. Performance
    • 13.3.1. Cython Acceleration
  • 13.4. Calibration and neural quantifiers
    • 9. Calibration
    • 13.4.1. Neural Quantifiers