Self-supervised learning enables deep neural networks to learn meaningful representations from large quantities of unlabeled data. The key idea behind the technique is to design an auxiliary task, also known as a pre-training task, which challenges the model to learn high-level properties of the data. Once the model has been pre-trained using the auxiliary task, the internal representations of the model can be used for downstream signal processing tasks through supervised fine-tuning. The internal representations of the model can also be used as an excellent basis for automatic clustering of animal vocalizations. We are working to develop self-supervised models to analyze the songs of canaries, juvenile zebra finches, and parrots. A link to a talk on this topic can be found here: https://www.interspecies.io/lectures/timgardner