Dealing with Ellipses

[kevin3567]

Hi,

I have noticed that rouge cannot handle strings which contains only ellipses ("......", repeated periods) for either hypothesis or reference. I am wondering why this might be the case, and how do I resolve the problem without replacing ellipses with special characters/substrings?

Thanks

[pltrdy]

That's correct. In order to correctly calculate ROUGE-L we need to split the text in sentences. Periods are used to detect end of sentence so it's probably the source of the problem.

This could be solved by adding some logic to consider any repetitions of periods as a single end of sentence (i.e. a period after a period should not be considered as an empty sentence). However, I would think that one prefer to consider it as a single token: it does not make sens (to me) to calculate ROUGE over sequences of periods i.e. if both reference and hypothesis has an ellispse of 3 periods should it count as 3 token matches? (I think not).

Long story short: replacing it with special token would be cleaner I guess.