Ph.D. Preliminary Oral Exam: Sayem Mohammad Imtiaz

Sayem Mohammad Imtiaz
Tuesday, May 28, 2024 - 4:00pm
Atanasoff 223
Event Type: 

Semantic-aware Slicing of Language Models To Mitigate Data-Driven Errors

Language models (LMs) have demonstrated remarkable performance across a wide range of natural language processing (NLP) tasks. The effectiveness of the transformer architecture has allowed LMs to scale efficiently to large corpora, acquiring a broad range of concepts. However, despite their state-of-the-art (SOTA) capabilities, LMs remain susceptible to data-driven errors, such as hallucinations. These errors can manifest as factually incorrect, fabricated, offensive, unethical, or privacy-violating responses. To address these issues, we propose employing targeted mitigation strategies that leverage the concept of relevant slicing. Slicing, which involves identifying segments influenced by specific criteria, presents non-trivial challenges in the context of deep learning. Our initial work demonstrated the feasibility of identifying relevant model parts across the time dimension for NLP data, laying the foundation for identifying relevant parts in SOTA LM architectures. Building on this preliminary work, we propose three future directions: first, we aim to selectively repair data-driven errors while preserving the model's versatility; second, we propose identifying reusable slices to facilitate efficient transfer learning; third, we aim to identify and slice underperforming aspects of the model, replacing them with slices from better-performing models in those areas.

Committee: Hridesh Rajan (major professor), Myra Cohen, Hongyang Gao, Wei Le, Pavan Aduri