Reliable Named Entity Recognition in Low-Resource Specialized Domains: Distant Supervision, Discontinuous Structure, and LLM Faithfulness

Named entity recognition in specialized domains is often limited by expensive annotation, noisy weak supervision, and complex entity boundary structure. These challenges become more severe in low-resource settings, where manually labeled data are scarce and many domain entities are descriptive, structurally complex, and difficult to capture with standard contiguous formulations.

This dissertation studies how to build more reliable entity recognition systems for low-resource domain text from three complementary perspectives: robust distant supervision, discontinuous entity modeling, and reliable use of large language models (LLMs) in evidence-guided workflows.

First, this dissertation re-examines distantly supervised named entity recognition (DS-NER) under realistic annotation constraints. It introduces QTL, a real-life DS-NER benchmark in the animal science domain with dictionary-annotated training data and a very small manually labeled validation set, and shows that many existing DS-NER methods are less robust than previously suggested in such practical settings. To address this problem, it proposes CuPUL, a token-level curriculum-based positive-unlabeled learning framework that improves robustness to noisy supervision and false negatives.

Second, this dissertation extends the study of domain NER from contiguous spans to discontinuous structures. It proposes a POS-pattern-based, human-in-the-loop framework that mines reusable discontinuous patterns from existing resources, applies them to a contiguous target corpus for candidate generation, and combines LLM suggestions with human verification to construct a discontinuous benchmark in the QTL domain at reduced annotation cost. Experimental results further show that discontinuous entity recognition remains substantially more difficult than overall extraction for current LLMs, even when strong models and few-shot prompting are used.

Third, this dissertation investigates context faithfulness in LLMs to better understand their reliability in annotation and extraction workflows. It shows that LLM receptiveness to external evidence is strongly influenced by memory strength and that presenting paraphrased evidence can improve faithfulness more effectively than simple repetition or added detail. These findings provide practical insight into how LLMs should be selected and guided when used in evidence-aware information extraction settings.

Together, this dissertation argues that reliable entity recognition in low-resource specialized domains requires progress along three dimensions: noise-robust learning under distant supervision, structurally expressive modeling of discontinuous entities, and faithful use of LLMs in evidence-guided workflows. The proposed benchmarks, methods, and analyses provide a foundation for future work on more robust, structurally aware, and reliability-conscious domain information extraction.

Committee: Qi Li (major professor), Hongyang Gao, Zhu Zhang, Ying Cai and Wallapak Tavanapong