Project Overview

This Master’s thesis investigates how artificial intelligence and robotics can be made tangible as a collaborative learning environment through the design of a physical human–machine interface. The central question is which forms of communication and interaction are suitable to ensure that learners do not merely consume the outputs of an AI system, but can actively help shape its behaviour. To this end, the collaborative learning robot KoLeRo has been designed: an AI-supported robotic arm that draws alongside pupils, responds to handwritten input and, through various interaction modes, can be used as a learning object, a tool and a creative partner. Unlike humanoid learning robots, KoLeRo deliberately avoids mimicking human appearance. Instead, human–machine communication takes place via a shared workspace, where visible movements, drawn inputs and gestures serve as forms of communication.

Methodologically, the work follows an iterative ‘research-through-design’ approach. In a four-stage prototyping series, collaborative drawing is first investigated between two people and subsequently with an AI system on a touchscreen, a drawing plotter and a robotic arm. The consistent turn-taking structure allows for a comparison of different interfaces and their effects on human response behaviour. The experiments show that the materiality and spatial presence of the system significantly influence the perception of the interaction. Whilst the touchscreen primarily encourages exploratory testing of the AI, the visible movements and mechanical noises of the plotter and the robotic arm create a stronger social presence for the machine.

Prototype interfaces

←

→

1 / 4

Analysis of human-robot drawings

The in-depth analysis of the recorded human–robot interaction is based on 349 documented intermediate stages from 65 collaborative drawings. Morphological and semantic analyses show that human drawing actions react directly to the position, complexity and orientation of the robot’s movements. At the same time, participants attempt to narratively integrate, reinterpret or deliberately contrast ambiguous robotic interventions. Of particular relevance is the recurring behaviour of testing the system’s capabilities and limitations through their own inputs. These initial findings suggest that the interface can thus support not only creative production, but also metacognitive reflection and an initial form of AI/system literacy.

Based on these findings, KoLeRo is being developed conceptually as a modular human–machine interface. Its interaction modes range from noughts and crosses and calligraphic exercises, through prompt-based and collaborative drawing, to handwritten and Python-based programming. In this context, the camera, image recognition, generative image models and robot control are integrated as distinct functions within a controllable system architecture.

The two galleries first show the morphological and semantic analysis of the human–robot interaction and then the documented drawing strokes from Algorithmic Cultural Vandalism. Red marks the newly added human stroke, blue the robot’s stroke. Black indicates the existing drawing context.

Teaching strategies

Learning with and about the robot

KoLeRo is designed to provide a basic education in computer science that is motivating, contemporary and creative. Learners actively help shape the system’s behaviour whilst at the same time engaging with its limitations. Drawing on Tulodziecki et al. (2019), the concept combines learning with media with learning about media. A key aspect of this is its integration into a specific learning task, as also described by Kerres (2018) for the design of digital learning resources.

Starting point: Computer science education in Germany

The project addresses the inconsistent framework conditions in schools: access to computer science education depends on the federal state, the type of school and the availability of teaching staff. The Informatik-Monitor 2022/23, cited in the Master’s thesis, highlights this need for action (Gesellschaft für Informatik [GI], 2022). KoLeRo is therefore also intended for interdisciplinary projects, without seeking to replace compulsory computer science lessons.

Update for the 2026/27 academic year: Ten federal states have now introduced computer science as a compulsory subject; six still lack a programme that is compulsory for all pupils. The scope and access to these programmes continue to vary significantly (GI et al., 2026).

A three-step introduction

• Primary level – exploratory play: Tic-Tac-Toe and calligraphy exercises provide a visual, non-verbal introduction. Through the movements and drawings, pupils can directly experience rules, repetition and the relationship between input and output. Questions such as ‘Why is the robot doing that?’ prompt initial reflection on the system.
• Lower secondary level – developing their own instructions: Through prompt-based drawing, collaborative drawing dialogue and handwritten commands, creative forms of expression are combined with algorithmic thinking. Learners compare their intentions with the machine’s output, adjust their inputs and discuss the possibilities and limitations of AI. This builds on the AI literacy skills outlined in Long und Magerko (2020).
• Upper secondary level – programming and expansion: Using the open, Python-based control system, pupils are encouraged to develop their own modes of interaction. Debugging, parameterisation and the modelling of the camera, AI and robot movement combine practical programming with questions of responsibility and machine decision-making.

Creating together, across age groups

I see the greatest potential in cross-age collaboration: the younger pupils formulate creative ideas, the middle-school pupils develop logical sequences, and the sixth-form pupils implement them in programmes. The new mode of interaction is then tested and refined together. This peer learning builds on the participatory design of learning environments as outlined in Mayrberger (2019). Writing workshops, drawn functions or joint art projects build bridges between German, Maths, Art and Computer Science. The concept is intended as a flexible, adaptable guide; it has yet to be tested in schools through pilot projects.

Supervision

Supervised by Prof. Albert Lang, Prof. Dr. Marc Pfaff, Prof. Dr. Dr. Daniel Hromada

References

Gesellschaft für Informatik. (2022). Informatik-Monitor 2022/2023. https://informatik-monitor.de/2022-23

Gesellschaft für Informatik, Stifterverband, & Heinz Nixdorf Stiftung. (2026). Informatik-Monitor 2026/27. https://informatik-monitor.de/2026-27

Kerres, M. (2018). Mediendidaktik: Konzeption und Entwicklung digitaler Lernangebote (5., erweiterte Aufl.). De Gruyter Oldenbourg. https://doi.org/10.1515/9783110456837

Long, D., & Magerko, B. (2020). What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (S. 1–16). Association for Computing Machinery. https://doi.org/10.1145/3313831.3376727

Mayrberger, K. (2019). Partizipative Mediendidaktik: Gestaltung der (Hochschul-)Bildung unter den Bedingungen der Digitalisierung. Beltz Juventa. https://doi.org/10.25656/01:29189

Tulodziecki, G., Herzig, B., & Grafe, S. (2019). Medienbildung in Schule und Unterricht: Grundlagen und Beispiele (2., vollständig überarbeitete und aktualisierte Aufl.). Julius Klinkhardt. Publikationsnachweis der Universität Paderborn