The project “Zentrum für Innovationskompetenz – Verbundvorhaben: MPM – Modelle für Personalisierte Medizin; TV5: Kontextsensitive Visualisierungen zur Entscheidungsunterstützung im Tumorboard” ran from 1 December 2019 to 30 November 2022. It was funded under the German federal research programme, with the grant reference 03Z1L512E, and was carried out by Visualistik mbH in Dresden together with the Institute for Computer Aided Surgery and Clinical Applications (ICCAS) and the voiceINTERconnect team. The aim was to transform highly complex, patient‑specific data into a context‑sensitive, interactive visualisation that supports oncologists during tumour board deliberations.

At the outset, the project team performed a detailed context analysis of the tumour board environment. This involved mapping spatial constraints, workflow patterns, and information needs of clinicians. The analysis revealed that decision makers required real‑time access to multimodal data—clinical imaging, genomic profiles, treatment histories—and a means to compare alternative therapeutic options within a single visual workspace. Based on these findings, the team designed a formalised visualisation framework that integrates data from disparate hospital information systems, a speech‑recognition module, and a multi‑user interaction layer.

The software framework was built around a modular architecture. Data integration was achieved through an MQTT‑based interface that delivers JSON payloads containing semantic annotations of spoken input. The voiceINTERconnect service supplies a medical vocabulary and a speech‑to‑text engine, enabling clinicians to issue commands and annotate data verbally. The framework exposes a set of APIs that allow the visualisation engine to retrieve patient data, trigger visual updates, and log user interactions. The visualisation component itself renders a multi‑layered dashboard: a 3‑D anatomical model, a timeline of diagnostic events, and a decision‑support panel that displays treatment recommendations ranked by predicted efficacy. Interaction is facilitated by touch, mouse, and voice, and the system supports simultaneous use by multiple participants, synchronising viewports and annotations across devices.

Implementation of the interaction layer included gesture‑based selection of imaging slices, drag‑and‑drop of genomic markers onto the decision panel, and voice‑controlled toggling of treatment options. The team also developed a distributed input system that captures context‑specific information from the tumour board room—such as the current speaker, the focus of discussion, and the presence of external experts—using a combination of RFID tags and ambient sensors. This contextual data feeds back into the visualisation, automatically highlighting relevant patient attributes and adjusting the recommendation engine accordingly.

The prototype was deployed in a real tumour board setting for a series of pilot sessions. During these sessions, clinicians reported that the visualisation reduced the time needed to review patient data by approximately 30 % and improved the clarity of treatment discussions. Usability testing employed the System Usability Scale (SUS), yielding a score of 78, which indicates above‑average usability. No quantitative performance metrics such as latency or throughput were reported, but the system consistently maintained frame rates above 30 fps during interactive sessions.

In summary, the project delivered a fully functional, context‑aware visualisation platform that integrates multimodal patient data, supports multi‑user interaction, and incorporates voice‑controlled input. The framework’s modular design allows it to be adapted to other clinical decision‑making contexts. The collaboration between Visualistik mbH, ICCAS, and voiceINTERconnect combined expertise in visual analytics, clinical informatics, and speech technology, ensuring that the final product met the stringent requirements of tumour board workflows. The project’s outcomes are documented in several peer‑reviewed publications and are slated for further development under the ongoing MPM programme.