“Learning to Quantify: Methods and Applications” is a tutorial + workshop event co-located with the ECML/PKDD 2023 conference, and will take place on September 18, 2023, in Torino, Italy; the tutorial will take place in the morning, while the 3rd International Workshop on Learning to Quantify (LQ 2023) will take place in the afternoon.

Learning to Quantify (LQ - also known as “quantification“, or “supervised prevalence estimation“, or “class prior estimation“, or “unfolding”), is the task of training class prevalence estimators via supervised learning. In other words, the task of these trained models is to estimate, given an unlabelled sample of data items and a set of classes, the prevalence (i.e., relative frequency) of each such class in the sample.

LQ is interesting in all applications of classification in which the final goal is not determining which class (or classes) individual unlabelled data items belong to, but estimating the percentages of data items that belong to the classes of interest, i.e., estimating the distribution of the unlabelled data items across the classes. Example disciplines whose interest in labelling data items is at the aggregate level (rather than at the individual level), are the social sciences, political science, market research, ecological modelling, and epidemiology.

While LQ may in principle be solved by classifying each data item in the sample and counting how many such items have been labelled with a certain class, it has been shown that this “classify and count” method yields suboptimal quantification accuracy. As a result, quantification is now no longer considered a mere byproduct of classification, and has evolved as a task of its own.

The goal of the Learning to Quantify tutorial is to provide an introduction to Learning to Quantify that presents its rationale, applications, methods, and evaluation measures and methodologies, to researchers who might want to become up-to-speed on this topic; the tutorial will also feature a final hands-on session. A book on Learning to Quantify will also be distributed to participants, along with the slides used by the instructors.

The goal of the LQ 2023 workshop is to bring together all researchers interested in methods, algorithms, evaluation measures, evaluation protocols, and methodologies for LQ, as well as practitioners interested in the practical application of the above to managing large quantities of data, and will feature both presentations of submitted papers and a final open discussion on “what’s next in Learning to Quantify”.

LQ 2023 is supported by the SoBigData++ project, funded by the European Commission (Grant 871042) under the H2020 Programme INFRAIA-2019-1, and by the AI4Media project, funded by the European Commission (Grant 951911) under the H2020 Programme ICT-48-2020. The organizers’ opinions do not necessarily reflect those of the European Commission.

ai4media logo sobigdata logo

Call for papers for the LQ 2023 Workshop

We seek papers on any of the following topics, which will form the main themes of the LQ 2023 workshop:

  • Binary, multiclass, multilabel, and ordinal LQ
  • Supervised algorithms for LQ
  • Semi-supervised / transductive LQ
  • Deep learning for LQ
  • Representation learning for LQ
  • LQ and dataset shift
  • Evaluation measures for LQ
  • Experimental protocols for the evaluation of LQ
  • Quantification of streaming data
  • Cost-sensitive quantification
  • Improving classifier performance via LQ
  • New datasets for evaluating quantification systems
  • Novel applications of LQ

and other topics of relevance to LQ. Two categories of papers are of interest:

  • papers reporting original, unpublished research;
  • papers {published in 2023 / currently under submission / accepted in 2023} at other {workshops / conferences / journals}, provided this double submission does not violate the rules of these {workshops / conferences / journals}.

Papers should be submitted (specifying which of the two above categories they belong to) via EasyChair.

Papers should be formatted according to Springer’s LNCS template, and should be up to 16 pages (including references) in length; however, this is just the upper bound, and contributions of any length up to this bound will be considered.

Other information

At least one author of each accepted paper must register to present the work. The workshop will be a hybrid event, but it is strongly recommended that authors of accepted papers present the work in-presence. The proceedings of the workshop will not be formally published, so as to allow authors to resubmit their work to other conferences. Informal proceedings will be published on the workshop website; however, for each accepted paper, it will be left at the discretion of the authors to decide whether to contribute their paper or not to these proceedings.

Important dates (all 23:59 AoE)
  • Paper submission deadline: June 12, 2023
  • A/R notification deadline: July 17, 2023
  • Final copy submission deadline: August 30, 2023
  • Workshop: Afternoon of September 18, 2023


Mirko Bunse

Mirko Bunse (workshop organizer)

Artificial Intelligence Group, TU Dortmund University, Germany

Pablo González

Pablo González (workshop organizer)

Artificial Intelligence Center, University of Oviedo, Spain

Alejandro Moreo

Alejandro Moreo (tutorial speaker and workshop organizer)

Istituto di Scienza e Tecnologie dell’Informazione, Consiglio Nazionale delle Ricerche, Pisa, Italy

Fabrizio Sebastiani

Fabrizio Sebastiani (tutorial speaker and workshop organizer)

Istituto di Scienza e Tecnologie dell’Informazione, Consiglio Nazionale delle Ricerche, Pisa, Italy

Program Committee
  • Rocío Alaíz-Rodríguez, University of León, ES
  • Gustavo Batista, University of New South Wales, AU
  • Juan José del Coz, University of Oviedo, ES
  • Andrea Esuli, Consiglio Nazionale delle Ricerche, IT
  • Alessandro Fabris, Università di Padova, IT
  • Cèsar Ferri, Universitat Politècnica de València, ES
  • George Forman, Amazon Research, US
  • Wei Gao, Singapore Management University, SG
  • Rafael Izbicki, Federal University of São Carlos, BR
  • André G. Maletzke, Universidade Estadual do Oeste do Paraná, BR
  • Marco Saerens, Catholic University of Louvain, BE
  • Dirk Tasche, Swiss Financial Market Supervisory Authority, CH


The following is the final program of the LQ 2023 Tutorial+Workshop event.

When: Monday, September 18, 2023

Where: Aula 9I

09:0011:00Tutorial: Learning to Quantify, Part I, by Alejandro Moreo and Fabrizio Sebastiani (both National Council of Research, IT)
11:0011:30Coffee Break
11:3013:00Tutorial: Learning to Quantify, Part II, by Alejandro Moreo and Fabrizio Sebastiani (both National Council of Research, IT)
13:0014:30Lunch Break
Chair: Fabrizio Sebastiani14:3014:40Workshop Chairs' Introduction
Chair: Pablo Gonzalez14:4015:00Invariance Assumptions for Class Distribution Estimation, by Dirk Tasche (Independent researcher, CH)
15:0015:20MC-SQ and MC-MQ: Ensembles for Multi-class Quantification, by Zahra Donyavi (University of New South Wales, AU), Adriane Serapião (Saõ Paulo State University, BR), and Gustavo Batista (University of New South Wales, AU)
15:2015:40Measuring Fairness under Unawareness of Sensitive Attributes: A Quantification-Based Approach, by Alessandro Fabris (University of Padova, IT, and Max Planck Institute for Security and Privacy, DE), Andrea Esuli, Alejandro Moreo, Fabrizio Sebastiani (all National Council of Research, IT)
15:4016:00qunfold: Composable Quantification and Unfolding Methods in Python, by Mirko Bunse (University of Dortmund, DE)
16:0016:30Coffee Break
Chair: Mirko Bunse16:3016:50An Equivalence Analysis of Binary Quantification Methods, by Alberto Castaño, Jaime Alonso, Pablo González, and Juan José del Coz (all University of Oviedo at Gijón, ES)
16:5017:10Continuous Sweep: An Improved Binary Quantifier, by Kevin Kloos, Julian Karch (both Leiden University, NL), Quinten Meertens (University of Amsterdam, NL), and Mark de Rooij (Leiden University, NL)
17:1017:30Multi-Label Quantification, by Alejandro Moreo (National Council of Research, IT), Manuel Francisco (University of Granada, ES), and Fabrizio Sebastiani (National Council of Research, IT)
Chair: Alejandro Moreo17:3018:00Open discussion


The proceedings of LQ 2023 are available here. A videorecording of the tutorial is available here; a videorecording of the workshop is available here.