Towards End-to-End Reference-Free Summarization Evaluation Via Negative Sampling
Evaluating machine-generated summaries without a human-written reference summary has been a need for a long time. In this paper, we present a proof-of-concept study to a summary evaluation approach without the presence of reference summaries. By negative sampling, massive data in existing summarization datasets are transformed for training by pairing documents with corrupted reference summaries. Two learning schemes are explored: weakly supervised learning with explicit number labels and preference learning with inexplicit labels. Extensive experiments on several datasets show that our approaches can produce scores highly correlated with human ratings.
Committee: Forrest Bao (major professor), Qi Li, and Wensheng Zhang
Join on Zoom: Please click this URL to start or join. https://iastate.zoom.us/j/97135461178 Or, go to https://iastate.zoom.us/join and enter meeting ID: 971 3546 1178