文章概览
Parameter-efficient fine-tuning (PEFT)
NLP
- Adapters houlsby2019parameter
Parameter-efficient transfer learning for NLP插入块 - LoRA hu2021lora
Lora: Low-rank adaptation of large language models插入低秩块 - Bitfit zaken2021bitfit
Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models只调bias - Side-Tuning zhang2020side
Side-tuning: a baseline for network adaptation via additive side networks搭建side网络 - LST sung2022lst
Lst: Ladder side-tuning for parameter and memory efficient transfer learning搭建Ladder side网络
CV
- Adaptformer chen2022adaptformer
Adaptformer: Adapting vision transformers for scalable visual recognition - Convpass jie2022convolutional
Convolutional bypasses are better vision transformer adapters - ViT-adapter chen2022vision
Vision transformer adapter for dense predictions猫猫头
多想多做，发篇一作