Hinglisheval Generation Challenge On Quality Estimation Of Synthetic Code-Mixed Text: Overview And Results
Vivek Srivastava, Mayank Singh
GenChal - Thursday 07/21 10:50 EST
Abstract:
We hosted a shared task to investigate the factors influencing the quality of the codemixed text generation systems. The teams experimented with two systems that generate synthetic code-mixed Hinglish sentences. They also experimented with human ratings that evaluate the generation quality of the two systems. The first-of-its-kind, proposed subtasks, (i) quality rating prediction and (ii) annotators’ disagreement prediction of the synthetic Hinglish dataset made the shared task quite popular among the multilingual research community. A total of 46 participants comprising 23 teams from 18 institutions registered for this shared task. The detailed description of the task and the leaderboard is available at https://codalab.lisn. upsaclay.fr/competitions/1688.