Experiments
Goal of this research strain is the development of machine learning methods that adapt to changes on the arriving data. The KMD team works on methods for prediction, active stream learning and active feature selection over timestamped data. Some of this work strain involves experiments, namely
- on how experiment participants learn: Experiments I
- on how humans and machines behave when they interact with each other: Experiments II
Experiments I: Analysis of participant behaviour in experimental settings
Experiments on human learning
Cooperation with Leibniz Institute of Neurobiology Magdeburg
Together with the Leibniz Institute of Neurobiology Magdeburg, we analyze experiments on human learning. We also cooperate in the context of the studies profile "Learning Systems" of the bachelor degree Informatik.
Publications:
-
Machine learning identifies the dynamics and influencing factors in an auditory category learning experiment. Scientific reports, (10)1:1--12, Nature Publishing Group, 2020.
Experiments II: Experiment design and analysis of human behaviour in human-machine interaction
(1) We develop learning algorithms for active and cost-aware feature/source selection on data streams. To account for the challenges of acquiring reliable labels from humans, (2) we investigate the challenges and potential of the 'pairwise comparisons' paradigm for the labeling of structured multidimensional objects. To understand the interplay of human and machine for label acquisition, (3) we design experiments where we trace human uncertainty during fully-specified tasks and underspecified tasks.
For (2) we cooperate with University Medicine Greifswald.
For (3) we additionally cooperate with TU Chemnitz and TU Ilmenau within in CHIM network, where we promote the paradigm of 'Productive Teaming' for human-machine cooperation.
Publications relating to (1,3):
-
Human uncertainty in interaction with a machine: establishing a reference dataset. 60th Ilmenau Scientific Colloquium, 2023.
-
Productive teaming under uncertainty: when a human and a machine classify objects together. 2023 IEEE International Conference on Advanced Robotics and Its Social Impacts (ARSO), 9-14, 2023.
-
Reducing Missingness in a Stream through Cost-Aware Active Feature Acquisition. 2022 IEEE 9th International Conference on Data Science and Advanced Analytics (DSAA), November 2022.
- Active feature acquisition on data streams under feature drift.. Ann. des Télécommunications, (75)9-10:597-611, 2020. URL
Publications relating to (2):
- Assessing the difficulty of annotating medical data in crowdworking with help of experiments. PLOS ONE, (16)7:1-26, Public Library of Science, July 2021.
-
Assessing the Difficulty of Labelling an Instance in Crowdworking.. 2nd Workshop on Evaluation and Experimental Design in Data Mining and Machine Learning@ ECML PKDD 2020, 2020.
-
Predicting worker disagreement for more effective crowd labeling. 2018 IEEE 5th International Conference on Data Science andAdvanced Analytics (DSAA), 179--188, 2018.
-
How do annotators label short texts? Toward understanding the temporal dynamics of tweet labeling. Information Sciences, (457–458):29-47, 2018.
- A framework for validating the merit of properties that predict the influence of a twitter user. Expert Systems with Applications, (42)5:2824-2834, 2015.