Work in progress

SSL Stuff

1. If learning with teacher and student, make sure they are trained differently to avoid collapse, e.g. through different learing rates or momentum encoder. Also weight decay is very good. -> No direct gradient learning for the teacher

General stuf

SSL Stuff

1. If learning with teacher and student, make sure they are trained differently to avoid collapse, e.g. through different learing rates or momentum encoder. Also weight decay is very good. -> No direct gradient learning for the teacherf

2. Reduce learning Rate and Momentum

3. Use DropBlock as Regularization Method

4. Use Self-Adversarial Training

5. Mosaic data augmentation