Labelling user data is a central part of the design and evaluation of pervasive systems that aim to support the user through situation-aware reasoning. It is essential both in designing and training the system to recognise and reason about the situation, either through the definition of a suitable situation model in knowledge-driven applications, or though the preparation of training data for learning tasks in data-driven models. Hence, the quality of annotations can have a significant impact on the performance of the derived systems. Labelling is also vital for validating and quantifying the performance of applications. With pervasive systems relying increasingly on large datasets for designing and testing models of users’ activities, the process of data labelling is becoming a major concern for the community. This also reflects the increasing need of (semi-)automated annotation tools and knowledge transfer methodologies, which can reduce the manual annotation effort and to improve the annotation performance in large datasets.
ARDUOUS started as a single workshop on Annotation of useR Data for UbiquitOUs Systems and evolved into a workshop series and various spin off collaboration works.

ARDUOUS Workshop
More details about the workshop can be found here.

Project-related publications:
2023

• T. Stoev, K. Yordanova and E. L. Tonkin, "Experiencing Annotation: Emotion, Motivation and Bias in Annotation Tasks," 2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), Atlanta, GA, USA, 2023, pp. 534-539, doi: 10.1109/PerComWorkshops56833.2023.10150364.

2022

• S. Suravee et al., "Annotation Scheme for Named Entity Recognition and Relation Extraction Tasks in the Domain of People with Dementia," 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), Pisa, Italy, 2022, pp. 236-241, doi: 10.1109/PerComWorkshops53856.2022.9767278.

2020

• Kristina Yordanova. Towards Automated Generation of Semantic Annotation for Activity Recognition Problems. In Proceedings of IEEE International Conference on Pervasive Computing and Communications (PerCom Workshops). Austin, Texas. 2020 (to appear)
• Kristina Yordanova, Albert Hein, Thomas Kirste. Kitchen Task Assessment Dataset for Measuring Errors due to Cognitive Impairments. In Proceedings of IEEE International Conference on Pervasive Computing and Communications (PerCom Workshops). Austin, Texas. 2020 (to appear)

2019

• Kristina Yordanova. Challenges Providing Ground Truth for Pervasive Healthcare Systems. In IEEE Pervasive Computing 18(2):100-104. 2019 [full text]
• David Schindler, Kristina Yordanova, Frank Krüger. An annotation scheme for references to research artifacts in scientific publications. Workshops Proceedings of the IEEE International Conference on Pervasive Computing and Communications (PerCom Workshops). Kyoto, Japan. 2019. [full text]

2018

• Kristina Yordanova and Frank Krüger. Creating and Exploring Semantic Annotation for Behaviour Analysis. In Sensors, 2018 [full text]
• Emma L. Tonkin,  Alison Burrows, Przemysław R. Woznowski, Pawel Laskowski, Kristina Y. Yordanova, Niall Twomey, Ian J Craddock. Talk, Text, Tag? Understanding Self-Annotation of Smart Home Data from a User’s Perspective. In Sensors, 2018 [full text]
• Kristina Yordanova, Frank Krüger, Thomas Kirste. Providing Semantic Annotation for the CMU Grand Challenge Dataset. Workshops Proceedings of the IEEE International Conference on Pervasive Computing and Communications (PerCom Workshops), Athens, Greece, 2018 [full text]
• Kristina Yordanova, Adeline Paiement, Max Schröder, Emma Tonkin, Przemyslaw Woznowski, Carl Magnus Olsson, Joseph Rafferty, Timo Sztyler. Challenges in Annotation of useR Data for UbiquitOUs Systems: Results from the 1st ARDUOUS Workshop. In Technical Report, arXiv preprint, arXiv:1803.05843, 2018 [full text]

2017

• Przemyslaw Woznowski, Emma Tonkin, Pawel Laskowski, Niall Twomey, Kristina Yordanova, Alison Burrow. Talk, text or tag? The development of a self-annotation app for activity recognition in smart environments.  In Proceedings of the 1st International Workshop on Annotation of useR Data for UbiquitOUs Systems (ARDUOUS), PerCom 2017, Hawaii, USA, 2017 [full text]

2016

• Max Schröder, Kristina Yordanova, Sebastian Bader, Thomas Kirste. Tool Support for the Online Annotation of Sensor Data.  In Proceedings of the 3rd International Workshop on Sensor-based Activity Recognition and Interaction (iWOAR 2016), Rostock, Germany, 2016 [full text]