Konferenzbeitrag
AI said, She said - How Users Perceive Consumer Scoring in Practice
Vorschaubild nicht verfügbar
Volltext URI
Dokumententyp
Text/Conference Paper
Zusatzinformation
Datum
2023
Autor:innen
Zeitschriftentitel
ISSN der Zeitschrift
Bandtitel
Verlag
ACM
Zusammenfassung
As digitization continues, consumers are increasingly exposed to AI’s scoring decisions. However, we lack a thorough understanding of how users' misjudgments lead to a rejection of the system. Therefore, we must investigate the appropriation of such socio-technical systems in practice and how users describe their experience with algorithm-based scoring. To address this issue, we evaluated 1003 user reviews of an app of car insurance that calculates its premium based on the consumers' individual driving behavior. We find evidence that users develop their own folk theories to explain the algorithms with the help of situation-related experiences and that insufficient explanations lead to power asymmetries between consumers, the system, and the company. In particular, we uncover a fundamental conflict between computational risk assessment and the perceived agency to influence the score as a result of the different needs of the stakeholders involved.