Skip to Main content Skip to Navigation
Conference papers

DARKGAN: EXPLOITING KNOWLEDGE DISTILLATION FOR COMPREHENSIBLE AUDIO SYNTHESIS WITH GANS

Abstract : Generative Adversarial Networks (GANs) have achieved excellent audio synthesis quality in the last years. However, making them operable with semantically meaningful controls remains an open challenge. An obvious approach is to control the GAN by conditioning it on metadata contained in audio datasets. Unfortunately, audio datasets often lack the desired annotations, especially in the musical domain. A way to circumvent this lack of annotations is to generate them, for example, with an automatic audio tagging system. The output probabilities of such systems (so-called "soft labels") carry rich information about the characteristics of the respective audios and can be used to distill the knowledge from a teacher model into a student model. In this work, we perform knowledge distillation from a large audio tagging system into an adversarial audio synthesizer that we call DarkGAN. Results show that DarkGAN can synthesize musical audio with acceptable quality and exhibits moderate attribute control even with out-of-distribution input conditioning. We release the code and provide audio examples on the accompanying website.
Document type :
Conference papers
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03349492
Contributor : Javier Nistal Connect in order to contact the contributor
Submitted on : Monday, September 20, 2021 - 3:23:33 PM
Last modification on : Wednesday, April 13, 2022 - 5:32:02 PM
Long-term archiving on: : Tuesday, December 21, 2021 - 7:00:49 PM

File

2108.01216 (1).pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03349492, version 1

Collections

Citation

Javier Nistal Hurlé, Stefan Lattner, Gael Richard. DARKGAN: EXPLOITING KNOWLEDGE DISTILLATION FOR COMPREHENSIBLE AUDIO SYNTHESIS WITH GANS. International Society for Music Information Retrieval, Nov 2021, Virtual, France. ⟨hal-03349492⟩

Share

Metrics

Record views

45

Files downloads

28