Decoupled transfer learning cuts backbone backprop overhead
arXiv paper proposes a lightweight training strategy that avoids end-to-end backpropagation by freezing feature extraction and only updating normalization and classifier layers.
• Precompute features once, then optimize classifier independently
• Margin-based weighted loss reduces ambiguity without full backprop
• Tested on ResNet, MobileNet, DenseNet, ViT, Swin across medical imaging tasks
Trade-off: lower compute overhead and memory per step, likely at some cost to final accuracy — paper doesn't fully address the performance delta.