Humans Optional? Automatic Large-Scale Test Collections for Entity, Passage, and Entity-Passage Retrieval

dc.contributor.author	Dietz, Laura
dc.contributor.author	Dalton, Jeff
dc.date.accessioned	2021-05-04T09:36:34Z
dc.date.available	2021-05-04T09:36:34Z
dc.date.issued	2020
dc.description.abstract	Manually creating test collections is a time-, effort-, and cost-intensive process. This paper describes a fully automatic alternative for deriving large-scale test collections, where no human assessments are needed. The empirical experiments confirm that automatic test collection and manual assessments agree on the best performing systems. The collection includes relevance judgments for both text passages and knowledge base entities. Since test collections with relevance data for both entity and text passages are rare, this approach provides a cost-efficient way for training and evaluating ad hoc passage retrieval, entity retrieval, and entity-aware text retrieval methods.	de
dc.identifier.doi	10.1007/s13222-020-00334-y
dc.identifier.pissn	1610-1995
dc.identifier.uri	http://dx.doi.org/10.1007/s13222-020-00334-y
dc.identifier.uri	https://dl.gi.de/handle/20.500.12116/36388
dc.publisher	Springer
dc.relation.ispartof	Datenbank-Spektrum: Vol. 20, No. 1
dc.relation.ispartofseries	Datenbank-Spektrum
dc.subject	Automatic Evaluation
dc.subject	Complex Answer Retrieval
dc.subject	Entity and Passage Retrieval
dc.title	Humans Optional? Automatic Large-Scale Test Collections for Entity, Passage, and Entity-Passage Retrieval	de
dc.type	Text/Journal Article
gi.citation.endPage	28
gi.citation.startPage	17

Sammlungen