End-to-end Semantics-based Summary Quality Assessment for Single-document Summarization
Canonical automatic summary evaluation metrics, such as ROUGE, suffer from two drawbacks. First, semantic similarity and linguistic quality are not captured well. Second, a reference summary, which is expensive or impossible to obtain in many cases, is needed. Existing efforts to address the two drawbacks are done separately and have limitations. To holistically address them, we introduce an end-to-end approach for summary quality assessment by leveraging sentence or document embedding and introducing two negative sampling approaches to create training data for this supervised approach. The proposed approach exhibits promising results on several summarization datasets of various domains including news, legislative bills, scientific papers, and patents. We hope our approach can facilitate summarization research or applications when reference summaries are infeasible or costly to obtain, or when linguistic quality is a focus.
Ge Luo is a PhD student in Computer Science at Iowa State University working with Dr, Forrest Bao. He received his B.E. degree in Computer Science and Technology at Wuhan University, China, in 2019. His research interests include natural language processing, machine learning and its applications.
|Join from the meeting link|