Fast Randomized Low-Rank Adaptation of Pre-trained Language Models with PAC Regularization

Keyphrases

• Pre-trained Language Model 100%
• Low-rank Adaptation 100%
• Large Language Models 40%
• LoRa 40%
• Model Size 20%
• Parameter Tuning 20%
• Weight Update 20%
• Training Parameters 20%
• Adaptation Methods 20%
• Model Weight 20%
• Walsh-Hadamard Transform 20%
• GPT-2 20%
• Memory Overhead 20%
• PAC-Bayes 20%
• Low Memory Requirement 20%
• RoBERTa 20%
• Low-rank Projection Matrix 20%
• Few-shot Setting 20%
• Llama 20%

Computer Science

• Regularization 100%
• Large Language Model 100%
• Pre-Trained Language Models 100%
• Projection Matrix 50%
• Memory Requirement 50%
• Hadamard Transform 50%

Mathematics

• Regularization 100%
• Projection Matrix 100%
• Walsh-Hadamard Transform 100%