Tashkandi, AraekWiese, LenaBaum, MarcusMitschang, BernhardNicklas, DanielaLeymann, FrankSchöning, HaraldHerschel, MelanieTeubner, JensHärder, TheoKopp, OliverWieland, Matthias2017-06-212017-06-212017978-3-88579-660-2Recommender System (RS) technology is often used to overcome information overload. Recently, several open-source platforms have been available for the development of RSs. Thus, there is a need to estimate the predictive accuracy of such platforms to select a suitable framework. In this paper we perform an offline comparative evaluation of commonly used recommendation algorithms of collaborative filtering. They are implemented by three popular RS platforms (LensKit, Mahout, and MyMediaLite) using the BookCrossing data set containing 1,149,780 user ratings on books. Our main goal is to find out which of these RSs is the most applicable and has high performance and accuracy on these data. We consider performing a fair objective comparison by benchmarking the evaluation dimensions such as the data set and the evaluation metric. Our evaluation shows the disparity of evaluation results between the RS frameworks. This points to the need of standardizing evaluation methodologies for recommendation algorithms.enRecommender SystemLensKitMahoutMyMediaLiteBook recommendationsText/Conference Paper1617-5468