2021

1. Data Science Kitchen at GermEval 2021: A Fine Selection of Hand-Picked Features, Delivered Fresh from the Oven

   Niclas Hildebrandt, Benedikt Bönninghoff, Dennis Orth, and Christopher Schymura

   Konferenz zur Verarbeitung natürlicher Sprache/Conference on Natural Language Processing (KONVENS)

   Abs PDF

   This paper presents the contribution of the Data Science Kitchen at GermEval 2021 shared task on the identification of toxic, engaging, and fact-claiming comments. The task aims at extending the identification of offensive language, by including additional subtasks that identify comments which should be prioritized for fact-checking by moderators and community managers. Our contribution focuses on a feature-engineering approach with a conventional classification backend. We combine semantic and writing style embeddings derived from pre-trained deep neural networks with additional numerical features, specifically designed for this task. Ensembles of Logistic Regression classifiers and Support Vector Machines are used to derive predictions for each subtask via a majority voting scheme. Our best submission achieved macro-averaged F1-scores of 66.8%, 69.9% and 72.5% for the identification of toxic, engaging, and fact-claiming comments.

2. O2D2: Out-Of-Distribution Detector to Capture Undecidable Trials in Authorship Verification (1st place at PAN@CLEF 2021 Authorship Identification Challenge)

   Benedikt Bönninghoff, Robert M. Nickel, and Dorothea Kolossa

   CLEF 2021 Labs and Workshops, Notebook Papers

   Abs PDF

   The PAN 2021 authorship verification (AV) challenge is part of a three-year strategy, moving from a cross-topic/closed-set AV task to a cross-topic/open-set AV task over a collection of fanfiction texts. In this work, we present a novel hybrid neural-probabilistic framework that is designed to tackle the challenges of the 2021 task. Our system is based on our 2020 winning submission, with updates to significantly reduce sensitivities to topical variations and to further improve the system's calibration by means of an uncertainty-adaptation layer. Our framework additionally includes an out-of-distribution detector (O2D2) for defining non-responses. Our proposed system outperformed all other systems that participated in the PAN 2021 AV task.

3. Self-Calibrating Neural-Probabilistic Model for Authorship Verification Under Covariate Shift

   Benedikt Bönninghoff, Dorothea Kolossa, and Robert M. Nickel

   12th International Conference of the CLEF Association (CLEF 2021)

   Abs PDF

   We are addressing two fundamental problems in authorship verification (AV): Topic variability and miscalibration. Variations in the topic of two disputed texts are a major cause of error for most AV systems. In addition, it is observed that the underlying probability estimates produced by deep learning AV mechanisms oftentimes do not match the actual case counts in the respective training data. As such, probability estimates are poorly calibrated. We are expanding our framework from PAN 2020 to include Bayes factor scoring (BFS) and an uncertainty adaptation layer (UAL) to address both problems. Experiments with the 2020/21 PAN AV shared task data show that the proposed method significantly reduces sensitivities to topical variations and significantly improves the system's calibration.

4. PILOT: Introducing Transformers for Probabilistic Sound Event Localization

   Christopher Schymura, Benedikt Bönninghoff, Tsubasa Ochiai, Marc Delcroix, Keisuke Kinoshita, Tomohiro Nakatani, Shoko Araki, and Dorothea Kolossa

   Annual Conference of the International Speech Communication Association (INTERSPEECH)

   Abs arXiv

   Sound event localization aims at estimating the positions of sound sources in the environment with respect to an acoustic receiver (e.g. a microphone array). Recent advances in this domain most prominently focused on utilizing deep recurrent neural networks. Inspired by the success of transformer architectures as a suitable alternative to classical recurrent neural networks, this paper introduces a novel transformer-based sound event localization framework, where temporal dependencies in the received multi-channel audio signals are captured via self-attention mechanisms. Additionally, the estimated sound event positions are represented as multivariate Gaussian variables, yielding an additional notion of uncertainty, which many previously proposed deep learning-based systems designed for this application do not provide. The framework is evaluated on three publicly available multi-source sound event localization datasets and compared against state-of-the-art methods in terms of localization error and event detection accuracy. It outperforms all competing systems on all datasets with statistical significant differences in performance.

5. Data Fusion for Audiovisual Speaker Localization: Extending Dynamic Stream Weights to the Spatial Domain

   Julio Wissing, Benedikt Bönninghoff, Dorothea Kolossa, Tsubasa Ochiai, Marc Delcroix, Keisuke Kinoshita, Tomohiro Nakatani, Shoko Araki, and Christopher Schymura

   IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

   Abs arXiv PDF Code

   Estimating the positions of multiple speakers can be helpful for tasks like automatic speech recognition or speaker diarization. Both applications benefit from a known speaker position when, for instance, applying beamforming or assigning unique speaker identities. Recently, several approaches utilizing acoustic signals augmented with visual data have been proposed for this task. However, both the acoustic and the visual modality may be corrupted in specific spatial regions, for instance due to poor lighting conditions or to the presence of background noise. This paper proposes a novel audiovisual data fusion framework for speaker localization by assigning individual dynamic stream weights to specific regions in the localization space. This fusion is achieved via a neural network, which combines the predictions of individual audio and video trackers based on their time- and location-dependent reliability. A performance evaluation using audiovisual recordings yields promising results, with the proposed fusion approach outperforming all baseline models.

2020

1. Variational Autoencoder with Embedded Student-t Mixture Model for Authorship Attribution

   Benedikt Bönninghoff, Steffen Zeiler, Robert Nickel, and Dorothea Kolossa

   Proceedings of the 28th International Conference on Computational Linguistics (COLING)

   Abs PDF

   Traditional computational authorship attribution describes a classification task in a closed-set scenario. Given a finite set of candidate authors and corresponding labeled texts, the objective is to determine which of the authors has written another set of anonymous or disputed texts. In this work, we propose a probabilistic autoencoding framework to deal with this supervised classification task. Variational autoencoders (VAEs) have had tremendous success in learning latent representations. However, existing VAEs are currently still bound by limitations imposed by the assumed Gaussianity of the underlying probability distributions in the latent space. In this work, we are extending a VAE with an embedded Gaussian mixture model to a Student-t mixture model, which allows for an independent control of the {``}heaviness{''} of the respective tails of the implied probability densities. Experiments over an Amazon review dataset indicate superior performance of the proposed method.

2. Deep Bayes Factor Scoring for Authorship Verification (1st place at PAN@CLEF 2020 Authorship Identification Challenge)

   Benedikt Bönninghoff, Julian Rupp, Robert M. Nickel, and Dorothea Kolossa

   CLEF 2020 Labs and Workshops, Notebook Papers

   Abs PDF

   The PAN 2020 authorship verification (AV) challenge focuses on a cross-topic/closed-set AV task over a collection of fanfiction texts. Fanfiction is a fan-written extension of a storyline in which a so-called fandom topic describes the principal subject of the document. The data provided in the PAN 2020 AV task is quite challenging because authors of texts across multiple/different fandom topics are included. In this work, we present a hierarchical fusion of two well-known approaches into a single end-to-end learning procedure: A deep metric learning framework at the bottom aims to learn a pseudo-metric that maps a document of variable length onto a fixed-sized feature vector. At the top, we incorporate a probabilistic layer to perform Bayes factor scoring in the learned metric space. We also provide text preprocessing strategies to deal with the cross-topic issue.

2019

1. Autorschaftsanalyse- Verstellungsstrategien und Möglichkeiten der automatisierten Erkennung

   Benedikt Bönninghoff, Steffen Hessler, Dorothea Kolossa, Kerstin Kucharczik, Robert M. Nickel, and Karin Pittner

   Datenschutz und Datensicherheit - DuD

   Abs PDF

   Der Austausch von Informationen findet zunehmend über soziale Medien und Online-Nachrichtendienstestatt. Daraus kann ein hohes Schadenspotential erwachsen, wenn es gelingt, für kriminelle Absichteneine andere Identität vorzutäuschen oder im Schutze der Anonymität Fake News bzw. Hate Speechzu verbreiten. Zum Schutz vor solchen Angriffen untersucht die Forensische Linguistik Textsammlungen hinsichtlichder Urheberschaft und möglicher biographischer Aussagen. Doch der Umfang der über das Internetveröffentlichten Daten verlangt automatisierte Verfahren, die die Analyse unterstützen.

2. Explainable Authorship Verification in Social Media via Attention-based Similarity Learning

   Benedikt Bönninghoff, Steffen Hessler, Dorothea Kolossa, and Robert M. Nickel

   2019 IEEE International Conference on Big Data (Big Data)

   Abs PDF

   Authorship verification is the task of analyzing the linguistic patterns of two or more texts to determine whether they were written by the same author or not. The analysis is traditionally performed by experts who consider linguistic features, which include spelling mistakes, grammatical inconsistencies, and stylistics for example. Machine learning algorithms, on the other hand, can be trained to accomplish the same, but have traditionally relied on so-called stylometric features. The disadvantage of such features is that their reliability is greatly diminished for short and topically varied social media texts. In this interdisciplinary work, we propose a substantial extension of a recently published hierarchical Siamese neural network approach, with which it is feasible to learn neural features and to visualize the decision-making process. For this purpose, a new large-scale corpus of short Amazon reviews for text comparison research is compiled and we show that the Siamese network topologies outperform state-of-the-art approaches that were built up on stylometric features. Our linguistic analysis of the internal attention weights of the network shows that the proposed method is indeed able to latch on to some traditional linguistic categories.

3. Similarity Learning for Authorship Verification in Social Media

   Benedikt Bönninghoff, Robert M. Nickel, Steffen Zeiler, and Dorothea Kolossa

   ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

   Abs PDF

   Authorship verification tries to answer the question if two documents with unknown authors were written by the same author or not. A range of successful technical approaches has been proposed for this task, many of which are based on traditional linguistic features such as n-grams. These algorithms achieve good results for certain types of written documents like books and novels. Forensic authorship verification for social media, however, is a much more challenging task since messages tend to be relatively short, with a large variety of different genres and topics. At this point, traditional methods based on features like n-grams have had limited success. In this work, we propose a new neural network topology for similarity learning that significantly improves the performance on the author verification task with such challenging data sets.

2017

1. Leveraging Text Data for Word Segmentation for Underresourced Languages

   Thomas Glarner, Benedikt Bönninghoff, Oliver Walter, and Reinhold Haeb-Umbach

   Proc. Interspeech 2017

   Abs PDF

   In this contribution we show how to exploit text data to support word discovery from audio input in an underresourced target language. Given audio, of which a certain amount is transcribed at the word level, and additional unrelated text data, the approach is able to learn a probabilistic mapping from acoustic units to characters and utilize it to segment the audio data into words without the need of a pronunciation dictionary. This is achieved by three components: an unsupervised acoustic unit discovery system, a supervisedly trained acoustic unit-to-grapheme converter, and a word discovery system, which is initialized with a language model trained on the text data. Experiments for multiple setups show that the initialization of the language model with text data improves the word segmentation performance by a large margin.

2016

1. Unsupervised Classification of Voiced Speech and Pitch Tracking Using Forward-Backward Kalman Filtering

   Benedikt Bönninghoff, Robert M. Nickel, Steffen Zeiler, and Dorothea Kolossa

   Speech Communication; 12. ITG Symposium

   Abs PDF

   The detection of voiced speech, the estimation of the fundamental frequency, and the tracking of pitch values over time are crucial subtasks for a variety of speech processing techniques. Many different algorithms have been developed for each of the three subtasks. We present a new algorithm that integrates the three subtasks into a single procedure. The algorithm can be applied to pre-recorded speech utterances in the presence of considerable amounts of background noise. We combine a collection of standard metrics, such as the zero-crossing rate, for example, to formulate an unsupervised voicing classifier. The estimation of pitch values is accomplished with a hybrid autocorrelation-based technique. We propose a forward-backward Kalman filter to smooth the estimated pitch contour. In experiments, we are able to show that the proposed method compares favorably with current, state-of-the-art pitch detection algorithms.