publications

2024

1. 

   Weighted Ensemble Models Are Strong Continual Learners

   Imad Eddine Marouf, Subhankar Roy, Enzo Tartaglione, and 1 more author

   In European Conference on Computer Vision (ECCV) , 2024

   Abs HTML PDF Code

   Continual learning (CL) aims to learn from a stream of tasks without forgetting previously acquired knowledge. While most CL methods focus on mitigating catastrophic forgetting, we argue that the key to strong CL performance lies in effectively leveraging knowledge from all tasks. In this work, we propose CoFiMA, a simple yet effective approach that combines fine-tuning and model averaging. CoFiMA maintains a separate model for each task and uses a weighted ensemble for inference. We introduce a novel weighting scheme based on Fisher Information, which effectively balances the contributions of different task models. Our extensive experiments on various CL benchmarks demonstrate that CoFiMA consistently outperforms state-of-the-art methods, often by a significant margin. We provide theoretical insights into why CoFiMA works well and empirically validate its effectiveness in different CL scenarios.

2. 

   Mini but Mighty: Finetuning ViTs With Mini Adapters

   Imad Eddine Marouf, Enzo Tartaglione, and Stéphane Lathuilière

   In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , Jan 2024

   Abs HTML PDF Code

   Vision Transformers (ViTs) have shown remarkable performance across various computer vision tasks. However, finetuning these large models on downstream tasks can be computationally expensive and prone to overfitting. In this paper, we introduce Mini Adapters (MiMi), a lightweight and efficient approach for adapting ViTs to new tasks. MiMi inserts small, task-specific adapters into the ViT architecture, allowing for efficient finetuning while maintaining the pre-trained model’s knowledge. Our method significantly reduces the number of trainable parameters and computational cost compared to full finetuning, while achieving comparable or better performance. We demonstrate the effectiveness of MiMi on various image classification tasks and provide insights into its workings through extensive ablation studies.

3. 

   Rethinking Class-incremental Learning in the Era of Large Pre-trained Models via Test-Time Adaptation

   Imad Eddine Marouf, Subhankar Roy, Enzo Tartaglione, and 1 more author

   Jan 2024

   Abs HTML PDF Code

   Class-incremental learning (CIL) aims to continually learn new classes while retaining knowledge of previously learned ones. Traditional CIL methods often struggle with catastrophic forgetting and are computationally expensive. In this paper, we propose a novel approach that leverages large pre-trained models and test-time adaptation for CIL. Our method, TTA-CIL, uses a frozen pre-trained backbone and only updates a small set of parameters during test time to adapt to new classes. This approach significantly reduces computational costs and memory requirements while maintaining high performance across all classes. We demonstrate the effectiveness of TTA-CIL on various benchmark datasets and compare it with state-of-the-art CIL methods.