KnuckleTouch: Enabling Knuckle Gestures on Capacitive Touchscreens using Deep Learning
| dc.contributor.author | Schweigert, Robin | |
| dc.contributor.author | Leusmann, Jan | |
| dc.contributor.author | Hagenmayer, Simon | |
| dc.contributor.author | Weiß, Maximilian | |
| dc.contributor.author | Le, Huy Viet | |
| dc.contributor.author | Mayer, Sven | |
| dc.contributor.author | Bulling, Andreas | |
| dc.contributor.editor | Alt, Florian | |
| dc.contributor.editor | Bulling, Andreas | |
| dc.contributor.editor | Döring, Tanja | |
| dc.date.accessioned | 2019-08-22T04:36:34Z | |
| dc.date.available | 2019-08-22T04:36:34Z | |
| dc.date.issued | 2019 | |
| dc.description.abstract | While mobile devices have become essential for social communication and have paved the way for work on the go, their interactive capabilities are still limited to simple touch input. A promising enhancement for touch interaction is knuckle input but recognizing knuckle gestures robustly and accurately remains challenging. We present a method to differentiate between 17 finger and knuckle gestures based on a long short-term memory (LSTM) machine learning model. Furthermore, we introduce an open source approach that is ready-to-deploy on commodity touch-based devices. The model was trained on a new dataset that we collected in a mobile interaction study with 18 participants. We show that our method can achieve an accuracy of 86.8% on recognizing one of the 17 gestures and an accuracy of 94.6% to differentiate between finger and knuckle. In our evaluation study, we validate our models and found that the LSTM gestures recognizing archived an accuracy of 88.6%. We show that KnuckleTouch can be used to improve the input expressiveness and to provide shortcuts to frequently used functions. | en |
| dc.description.uri | https://dl.acm.org/authorize?N681262 | |
| dc.identifier.doi | 10.1145/3340764.3340767 | |
| dc.identifier.uri | https://dl.gi.de/handle/20.500.12116/24596 | |
| dc.language.iso | en | |
| dc.publisher | ACM | |
| dc.relation.ispartof | Mensch und Computer 2019 - Tagungsband | |
| dc.relation.ispartofseries | Mensch und Computer | |
| dc.subject | KuckleTouch | |
| dc.subject | finger | |
| dc.subject | knuckle | |
| dc.subject | input | |
| dc.subject | data set | |
| dc.subject | deep neural networks | |
| dc.subject | convolutional neural network | |
| dc.subject | long short term memory | |
| dc.title | KnuckleTouch: Enabling Knuckle Gestures on Capacitive Touchscreens using Deep Learning | en |
| dc.type | Text/Conference Paper | |
| gi.citation.publisherPlace | New York | |
| gi.conference.date | 8.-11. September 2019 | |
| gi.conference.location | Hamburg | |
| gi.conference.sessiontitle | MCI: Full Paper | |
| gi.document.quality | digidoc |