Project 12

Fast-adaptive foundation models

Project location:

Eveleigh (NSW)

Supervisory project team:

Qinghua Lu (Data61), Jing Jiang, Chengqi Zhang and Guodong Long (UTS)

Contact person:

Project description:

To train domain-specific foundation models usually takes two ways: one is to train from scratch which requires a large-scale dataset and large investment in AI engineering, and another solution is to adapt an pre-trained foundation model to a new domain with a relatively small-scale dataset. However, how to balance general knowledge and domain-specific knowledge is an open challenge of model adaptation. This project aims to develop various adaptation strategies on foundation models. We will collaborate with Associate Professor Jing Jiang and Distinguished Professor Chengqi Zhang who are the leading researcher in AI. Firstly, we need to choose a proper architecture of the foundation model that can easily be tuned to general knowledge and domain-specific knowledge respectively. Secondly, a new learning strategy will be designed to enable the foundation model to be fastly adapted to a new domain. The project’s outcome can be applied and evaluated on various tasks, such as science tasks in different domains. A benchmark setting and standard evaluation criteria will be developed for this fast-adaptive foundation model.