Weight Factorization Based Incremental Learning in Generalized Few Shot Segmentation

Anuska Roy, Viswanath Gopalakrishnan

2025

Abstract

Generalized Few-shot Semantic Segmentation (GFSS) targets to segment novel object categories using a few annotated examples after learning the segmentation on a set of base classes. A typical GFSS training involves two stages - base class learning followed by novel class addition and learning. While existing methods have shown promise, they often struggle when novel classes are significant in number. Most current approaches freeze the encoder backbone to retain base class accuracy; however, freezing the encoder backbone can potentially impede the assimilation of novel information from the new classes. To address this challenge, we propose to use an incremental learning strategy in GFSS for learning both encoder backbone and novel class prototypes. Inspired by the recent success of Low Rank Adaptation techniques (LoRA), we introduce incremental learning to the GFSS encoder backbone with a novel weight factorization method. Our newly proposed rank adaptive weight merging strategy is sensitive to the varying degrees of novelty assimilated across various layers of the encoder backbone. In our work, we also introduce the incremental learning strategy to class prototype learning for novel categories. Our extensive experiments on Pascal-5i and COCO-20i databases showcase the effectiveness of incremental learning, especially when the novel classes outnumber base classes. With our proposed Weight Factorization based Incremental Learning (WFIL) method, a new set of state-of-the-art accuracy values is established in Generalized Few-shot Semantic Segmentation.

Paper Citation

in Harvard Style

Roy A. and Gopalakrishnan V. (2025). Weight Factorization Based Incremental Learning in Generalized Few Shot Segmentation. In Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP; ISBN 978-989-758-728-3, SciTePress, pages 556-562. DOI: 10.5220/0013187700003912

in Bibtex Style

@conference{visapp25,
author={Anuska Roy and Viswanath Gopalakrishnan},
title={Weight Factorization Based Incremental Learning in Generalized Few Shot Segmentation},
booktitle={Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP},
year={2025},
pages={556-562},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013187700003912},
isbn={978-989-758-728-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP
TI - Weight Factorization Based Incremental Learning in Generalized Few Shot Segmentation
SN - 978-989-758-728-3
AU - Roy A.
AU - Gopalakrishnan V.
PY - 2025
SP - 556
EP - 562
DO - 10.5220/0013187700003912
PB - SciTePress