RAFIC: Retrieval-Augmented Few-shot Image Classification

December 11, 2023 · Entered Twilight · 🏛 arXiv.org

Repo contents: .gitignore, LICENSE, README.md, clip-encoder.ipynb, data-aircrafts.ipynb, environment.yml, environment_cuda.yml, exp1.sh, exp2.sh, exp3.sh, maml_exp.ipynb, rafic, requirements.txt, run_maml.sh, search-and-eval-demo.ipynb

Authors Hangfei Lin, Li Miao, Amir Ziai arXiv ID 2312.06868 Category cs.CV: Computer Vision Cross-listed cs.LG Citations 0 Venue arXiv.org Repository https://github.com/amirziai/rafic ⭐ 7 Last Checked 3 months ago

Abstract

Few-shot image classification is the task of classifying unseen images to one of N mutually exclusive classes, using only a small number of training examples for each class. The limited availability of these examples (denoted as K) presents a significant challenge to classification accuracy in some cases. To address this, we have developed a method for augmenting the set of K with an addition set of A retrieved images. We call this system Retrieval-Augmented Few-shot Image Classification (RAFIC). Through a series of experiments, we demonstrate that RAFIC markedly improves performance of few-shot image classification across two challenging datasets. RAFIC consists of two main components: (a) a retrieval component which uses CLIP, LAION-5B, and faiss, in order to efficiently retrieve images similar to the supplied images, and (b) retrieval meta-learning, which learns to judiciously utilize the retrieved images. Code and data is available at github.com/amirziai/rafic.