🐦 warbler v3 release notes

what changed

v3 fixes a calibration and generalization problem found while benchmarking the original (v1) model against held-out, real-world speech: v1 was trained almost entirely on studio-quality audio, so it learned "is this recording pristine?" as a shortcut for "is this speech?" - and fell apart on noisier, naturalistic recordings, especially Chinese, Korean, and Japanese speech.

• colored noise augmentation

  white / pink / brown noise mixed in at random 5-30dB SNR during training, replacing the old "light Gaussian noise" augmentation.

• label smoothing

  hard 0/1 targets softened to 0.05/0.95, preventing the sigmoid from saturating to the extremes and making the model's output probabilities meaningful again at the standard 0.5 threshold.

• more singing languages

  added Spanish, French, Thai, and Swedish singing data (~2 extra hours) on top of the existing Japanese/Chinese/English/Korean set - including some intentionally non-studio-quality Swedish clips.

• held-out-aware checkpointing

  training now tracks accuracy on a real held-out benchmark set every epoch and picks the "best" checkpoint by that score, instead of the manifest's own validation split - which turned out to look great right up until the model overfit.

benchmark

evaluated on 531 held-out clips (100 singing + 431 speech) across English, Japanese, Korean, and Chinese - none of which were used in training. v2 was an intermediate, unreleased checkpoint kept here for reference.

per-language accuracy (raw 0.5 threshold):

v3 beats both LAION CLAP variants on accuracy and F1, runs roughly 50x faster per clip, and is on the order of 1,800x smaller in parameter count - while CLAP is a general-purpose, zero-shot audio classifier built to handle 500+ arbitrary categories, not just this one binary task.

specs

• architectureconv blocks → global pooling → small FC head → sigmoid
• checkpoint size~410 KB
• inputlog-mel spectrogram, 80 mels, 4s clips @ 22.05kHz
• training languagesen, ja, zh, ko, es, fr, th, sv
• label smoothing0.1
• augmentationpitch shift, time stretch, white/pink/brown noise @ 5-30dB SNR
• checkpoint selectionbest held-out F1, not manifest val F1