Sharpness-Feedback Classwise Allocation for Long-Tailed Visual Recognition

Class frequency is a stable but incomplete prior for optimization difficulty: two equally rare classes can sit in loss regions with very different local sensitivity. SFCA (Sharpness-Feedback Classwise Allocation) adds an online geometric correction to the frequency-driven CA-Focal-SAM base:

• Ascent-gap observation: for every class in the mini-batch, measure the change in class-conditioned loss along SAM’s own perturbation direction; both losses are already computed by SAM, so no extra forward pass is needed.
• Robust EMA state: per-class exponential moving average, median/IQR-centered and clipped, so a tail class seen in only a few difficult batches cannot dominate the allocation.
• Bounded allocation: scarcity prior + normalized feedback pass through a sigmoid and are clipped to [0.05, 0.95]; the controller keeps only class-level state and adds no inference overhead.

Key results (CIFAR-100-LT, pre-specified final-epoch checkpoints, 3 seeds): tail accuracy rises from 22.45% (Focal-SAM) to 24.49% under Logit Adjustment at imbalance ratio 200; destroying the class identity of the feedback signal (22.23%) or reversing its sign (22.89%) confirms the signal is class-specific; on CIFAR-100-C, SFCA attains the strongest corrupted tail accuracy among compared checkpoints. Code: github.com/Amo-Gideon/sfca-ltr.

Recommended citation: Appau, Gideon Kofi Amo, and Wang, Pengkun. (2026). "Sharpness-Feedback Classwise Allocation for Long-Tailed Visual Recognition." Manuscript in preparation.
Download Paper