publications

2024

1. ACL

   A Data-Driven Guided Decoding Mechanism for Diagnostic Captioning

   Panagiotis Kaliosis, John Pavlopoulos, Foivos Charalampakos, and 2 more authors

   In Findings of the Association for Computational Linguistics: ACL 2024, Bangkok, Thailand. Our code is available here , Aug 2024

   Abs DOI PDF

   

   Diagnostic Captioning (DC) automatically generates a diagnostic text from one or more medical images (e.g., X-rays, MRIs) of a patient. Treated as a draft, the generated text may assist clinicians, by providing an initial estimation of the patient’s condition, speeding up and helping safeguard the diagnostic process. The accuracy of a diagnostic text, however, strongly depends on how well the key medical conditions depicted in the images are expressed. We propose a new data-driven guided decoding method that incorporates medical information, in the form of existing tags capturing key conditions of the image(s), into the beam search of the diagnostic text generation process. We evaluate the proposed method on two medical datasets using four DC systems that range from generic image-to-text systems with CNN encoders and RNN decoders to pre-trained Large Language Models. The latter can also be used in few- and zero-shot learning scenarios. In most cases, the proposed mechanism improves performance with respect to all evaluation measures. We provide an open-source implementation of the proposed method at https://github.com/nlpaueb/dmmcs.

2. ICASSP

   A Self-Supervised Learning Approach for Detecting Non-Psychotic Relapses Using Wearable-Based Digital Phenotyping

   Panagiotis Kaliosis, Sofia Eleftheriou, Christos Nikou, and 1 more author

   In 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), Seoul, South Korea. Our code is available here , Apr 2024

   Abs DOI PDF

   

   We present MagCIL’s approach for the 1st track of the "2nd e-Prevention challenge: Psychotic and Non-Psychotic Relapse Detection using Wearable-Based Digital Phenotyping". First we present our approach for preprocessing and extracting features from the wearable’s raw data. We then propose a Transformer model for learning self-supervised representations from augmented features, trained on data from non-relapse days from each of the 9 patients of the challenge. We adopt two unsupervised methods for detecting relapse days as outliers. A separate unsupervised model is tuned for each patient using the validation data of the challenge. Our method ranked second with ROCAUC = 0.651 and PRAUC = 0.642 on the final test dataset of the challenge. The respective code is available at this repository.

3. SETN

   Greek2MathTex: A Greek Speech-to-Text Framework for LaTeX Equations Generation

   Evangelia Gkritzali, Panagiotis Kaliosis, Sofia Galanaki, and 2 more authors

   In Proceedings of the 13th Hellenic Conference on Artificial Intelligence, Piraeus, Greece. Our code is available here , Apr 2024

   Abs DOI

   

   In the vast majority of the academic and scientific domains, LaTeX has established itself as the de facto standard for typesetting complex mathematical equations and formulae. However, LaTeX ś complex syntax and code-like appearance present accessibility barriers for individuals with disabilities, as well as those unfamiliar with coding conventions. In this paper, we present a novel solution to this challenge through the development of a novel speech-to- LaTeX equations system specifically designed for the Greek language. We propose an end-to-end system that harnesses the power of Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) techniques to enable users to verbally dictate mathematical expressions and equations in natural language, which are subsequently converted into LaTeX format. We present the architecture and design principles of our system, highlighting key components such as the ASR engine, the LLM-based prompt-driven equations generation mechanism, as well as the application of a custom evaluation metric employed throughout the development process. We have made our system open source and available at https://github.com/magcil/greek-speech-to-math.

2023

1. CLEF 2023

   AUEB NLP Group at ImageCLEFmedical Caption 2023

   Panagiotis Kaliosis, Georgios Moschovis, Foivos Charalampakos, and 2 more authors

   In Conference and Labs of the Evaluation Forum (Working Notes), Thessaloniki, Greece, Apr 2023

   Abs

   

   This article describes the methods that the AUEB NLP Group experimented with during its participation in the 7th edition of the ImageCLEFmedical Caption sub-tasks, namely Concept Detection and Caption Prediction. The former intends to automatically classify biomedical images into a set of one or more tags based solely on the visual input, while the latter aims to generate a syntactically and semantically accurate diagnostic caption that addresses the medical conditions depicted on a given image. For the Concept Detection sub-task, extending our previous work, we utilized a wide range of Convolutional Neural Network encoders followed by a Feed-Forward Neural Network, both in a single-task and a multi-task fashion, as well as combined with a contrastive learning approach. Our methods concerning the Caption Prediction sub-task are influenced by both our previous work and recent progress in Natural Language Processing (NLP) methods. Our two base systems use CNN-RNN and Transformer-to-Transformer encoder-decoder architectures, respectively. Additionally, we experimented with a Transformer-based denoising component, which was trained to reformulate the generated captions in a more syntactically coherent and medically accurate way. Our group ranked 1st in Concept Detection and 3rd in Caption Prediction.