Archive
IJMSTA - Vol. 7 - Issue 1 - January 2025
ISSN 2612-2146
Pages: 15
Contrastive and Transfer Learning for Effective Audio Fingerprinting through a Real-World Evaluation Protocol
Authors: Christos Nikou, Theodoros Giannakopoulos
Categories: Journal
Abstract - Recent advances in song identification leverage deep neural networks to learn compact audio fingerprints directly from raw waveforms. While these methods perform well under controlled conditions, their accuracy drops significantly in real-world scenarios where the audio is captured via mobile devices in noisy environments. In this paper, we introduce a novel evaluation protocol designed to better reflect such real-world conditions. We generate three recordings of the same audio, each with increasing levels of noise, captured using a mobile device's microphone. Our results reveal a substantial performance drop for two state-of-the-art CNN-based models under this protocol, compared to previously reported benchmarks. Additionally, we highlight the critical role of the augmentation pipeline during training with contrastive loss. By introducing low pass and high pass filters in the augmentation pipeline we significantly increase the performance of both systems in our proposed evaluation. Furthermore, we develop a transformer-based model with a tailored projection module and demonstrate that transferring knowledge from a semantically relevant domain yields a more robust solution. The transformer architecture outperforms CNN-based models across all noise levels, and query durations. In low noise conditions it achieves 47.9% for 1-sec queries, and 97% for 10-sec queries in finding the correct song, surpassing by 14%, and by 18,5% the second-best performing model, respectively. Under heavy noise levels, we achieve a detection rate 56,5% for 15-second query duration. All Experiments are conducted on a public large-scale dataset of over 100K songs, with queries matched against a database of 56 million vectors. We make our code publicly available on GitHub (https://github.com/magcil/deep-audio-fingerprinting-benchmark), supporting reproducibility and allowing future researchers to freely use our evaluation protocol to develop robust recognition systems for practical scenarios.
Keywords: Contrastive Learning, Transfer Learning, Song Identification, Audio Fingerprinting, Robustness
Download paper
