Konferenzbeitrag
FEERCI: A Package for Fast Non-Parametric Confidence Intervals for Equal Error Rates in Amortized O(m log n)
Lade...
Volltext URI
Dokumententyp
Text/Conference Paper
Zusatzinformation
Datum
2018
Zeitschriftentitel
ISSN der Zeitschrift
Bandtitel
Verlag
Köllen Druck+Verlag GmbH
Zusammenfassung
Equal Error Rates (EERs), or other weighted relations between False Match and Non-
Match Rates (FMR/FNMR), are often used as a performance metric for biometric systems. Confidence
Intervals (CIs) are used to denote the uncertainty underlying these EERs, with many methods
existing to estimate said CIs in both parametric and non-parametric ways. These confidence intervals
provide, foremost, a method of comparing scoring/ranking functions. Non-parametric methods
often suffer from high computational costs, but do not make assumptions as to the shape of the EERand
score distributions. For both EERs and CIs, contemporary open-source toolkits leave room for
improvement in terms of computational efficiency. In this paper, we introduce the Fast EER (FEER)
algorithm that calculates an EER in O(logn) on a sorted score list, we show how to adapt the FEER
algorithm to calculate non-parametric, bootstrapped EER CIs (FEERCI) in O(mlogn) given m resamplings,
and we introduce an opinionated open-source package named feerci that provides implementations
of the FEER and FEERCI algorithm.We provide speed and accuracy benchmarks for the
feerci package, comparing it against the most-used methods of calculating EERs in Python and show
how it is able to calculate EERs and CIs on very large score lists faster than contemporary toolkits
can calculate a single EER.