Tutorials
Mon 11 Sep 2023 08:30 - 10:00 at Room UK* - AI for SEMon 11 Sep 2023 10:30 - 12:00 at Room UK* - AI for SE
AI for SE: Neural Language Models, Few-Shot Learning, and Adapters for Source Code
Recently, many efforts have been made in the Software Engineering community to use language models (LM) such as CodeBERT or large language models (LLM) such as GPT3 to automate and improve the performance of SE tasks. Though fine-tuning the language models or providing a prompt to an LLM is a common way of using these models, the learned knowledge of these models can be transferred to a new language, task, or domain using other techniques such as few-shot learning and adapters. In this tutorial, we aim first to provide background knowledge about what language models are and how they work. Then, we will dive into different classes of few-shot learning and how they can be applied in the context of source code to the LMs when the training data samples are limited to only a few records. Finally, we will cover the theories required to use adapters and provide technical details about how they can transfer knowledge from other tasks and domains to new tasks and domains without fully fine-tuning or pre-training the models from scratch. The tutorial covers these concepts’ theories and practical aspects by providing interactive code on Google Colab. This tutorial provides the required background to use language models for knowledge transfer and better performance for low-resource languages.
