Beyond Binary Erasure: Soft-Weighted Unlearning for Fairness and Robustnessedit

Beyond Binary Erasure: Soft-Weighted Unlearning for Fairness and Robustness is an AAAI 2026 conference paper by Xinbao Qiao, Ningning Ding, Yushi Cheng, and Meng Zhang. It reframes unlearning as a continuous data-influence correction problem rather than only a binary erase-or-keep operation. The paper asks how much influence each sample should retain when the goal is to improve fairness or robustness while limiting utility loss.

Overviewedit

The paper studies a mismatch between privacy-driven unlearning and correction-driven unlearning. In a right-to-be-forgotten setting, binary deletion is natural: a sample is either retained or removed. In fairness and robustness correction, however, the goal is often to reduce harmful influence without discarding useful signal.

The paper names the resulting failure mode over-unlearning: hard deletion can improve a target fairness or robustness metric while degrading utility, flipping bias in the opposite direction, or treating borderline samples as if they were highly detrimental.

Methodedit

The method replaces binary deletion weights with continuous sample weights. It first estimates each sample's influence on both the target metric and utility, then solves a convex quadratic program for a tailored weight vector. Influence-function-based unlearning or related correction methods then use these weights to adjust each sample's influence according to its estimated effects.

The three-stage workflow is:

1. estimate each sample's influence on fairness or robustness and on utility;
2. solve for continuous weights that improve the target metric while constraining utility loss;
3. apply a weighted model correction instead of deleting a fixed top-k set.

Key takeawaysedit

• The goal of correction determines how data influence should change. Fairness and robustness interventions may benefit from reducing a sample's harmful influence while retaining useful information.
• Hard deletion can remove too much. Samples that harm one metric can still contribute to predictive utility.
• Continuous weights allow finer control. Soft-weighted correction adjusts each sample's remaining influence while constraining utility loss.
• Unlearning can support model improvement. Influence-based updates provide a way to intervene in fairness and robustness as well as to remove data.

Resultsedit

The experiments evaluate fairness and robustness settings across tabular, image, and text datasets, including Adult, Bank, Jigsaw, CelebA, and CIFAR-based robustness evaluations. The paper reports that soft-weighted variants improve fairness or robustness metrics more consistently than hard-weighted schemes while reducing the loss in utility.

The diagnostic experiments also support the premise of the method: leave-one-out and influence-based analyses show that samples harmful to a target metric are not uniformly harmful to utility. This explains why the binary "remove or keep" rule is too coarse for correction-driven unlearning.

Placementedit

This work belongs to Machine Unlearning, Fairness and Robustness, Influence Functions, and Trustworthy AI. It complements Hessian-Free Online Certified Unlearning by shifting the problem from certified privacy deletion to fine-grained model correction.