研究介绍
出于隐私和安全考虑，如今变得越来越明显的是，需要从预训练的视觉模型中擦除不需要的信息。在现实世界场景中，用户和模型拥有者可以随时提出擦除请求。这些请求通常形成一个序列。因此，在这样的设置下，期望从预训练模型中连续移除选定信息，同时保留其余信息。我们将这个问题定义为持续遗忘，并确定了两个关键挑战。(i) 对于不需要的知识，有效且高效的删除至关重要。(ii) 对于剩余的知识，遗忘过程带来的影响应尽可能小。为了解决这些问题，我们提出了群稀疏LoRA（GS-LoRA）。具体来说，针对(i)，我们使用LoRA模块独立地对Transformer块中的FFN层进行微调，以应对每个遗忘任务，并针对(ii)，采用了简单的组稀疏正则化，实现了特定LoRA群组的自动选择并将其他群归零。GS-LoRA有效、参数高效、数据高效且易于实现。我们在人脸识别、目标检测和图像分类上进行了广泛实验，并展示了GS-LoRA能够在对其他类别影响最小的情况下忘记特定类别。
Abstract
For privacy and security concerns, the need to erase unwanted information from pre-trained vision models is becoming evident nowadays. In real-world scenarios, erasure requests originate at any time from both users and model owners. These requests usually form a sequence. Therefore, under such a setting, selective information is expected to be continuously removed from a pre-trained model while maintaining the rest. We define this problem as continual forgetting and identify two key challenges. (i) For unwanted knowledge, efficient and effective deleting is crucial. (ii) For remaining knowledge, the impact brought by the forgetting procedure should be minimal. To address them, we propose Group Sparse LoRA (GS-LoRA). Specifically, towards (i), we use LoRA modules to fine-tune the FFN layers in Transformer blocks for each forgetting task independently, and towards (ii), a simple group sparse regularization is adopted, enabling automatic selection of specific LoRA groups and zeroing out the others. GS-LoRA is effective, parameter-efficient, data-efficient, and easy to implement. We conduct extensive experiments on face recognition, object detection and image classification and demonstrate that GS-LoRA manages to forget specific classes with minimal impact on other classes. Codes will be released on https://github.com/bjzhb666/GS-LoRA.
