Schliebitz, AndreasStolzenburg, Frieder2023-09-202023-09-202023https://dl.gi.de/handle/20.500.12116/42404This PhD proposal presents a concept for an AI-based computer vision system trained on image data streams for real-time classification of unlabeled objects. The required AI models are trained through an underexplored combination of self-supervised and incremental learning. Emphasis will be placed on contrastive learning using the SimCLR framework and its successors. The need for such a system is motivated by the observation that supervised learning approaches often require large labeled datasets. The labeling process, which is usually performed manually, is not only time-consuming but inherently prone to errors. For sufficiently large image data streams, timely labeling of samples becomes impossible leading to sporadic data annotation cycles and the capture of only temporarily representative features. Such an approach might also render the resulting classifier vulnerable to domain shift and concept drift. The image data stream used in this proposal consists of unlabeled color images of clean potatoes, which are to be sorted into several defect classes by a self-supervised classifier. Contrastive transfer learning is performed on this image data stream for the selection of a feature extractor. In this approach, different pre-trained backbone architectures are adapted and evaluated using the SimCLR framework. The classifiers are evaluated based on their generated feature vectors using cluster analysis. This involves searching for novel evaluation methods that do not require labels and are more suitable for judging model performance than existing methods. Furthermore, by clustering the feature vectors, an automatic and adaptive classification might be achievable without the use of labels. In a subsequent step, the self-supervised classifiers are continuously improved using incremental learning methods. For this the models are incrementally trained on the image data stream over a longer period of time. Potential adjustments to the data stream could increase the classifier’s accuracy as well as make it more robust to domain adaptation problems. A final validation of the incrementally self-learning classification system can be performed with smaller, manually annotated datasets.enFeature Learning, Self-Supervised Learning, Contrastive Learning, Incremental Learning, Image Classification, Data Stream Processing, Cluster Analysis, Computer VisionText10.18420/ki2023-dc-04