Hello :) Thank you for maintaining this repository.
I would like to ask about the issue caused by duplicate annotations. Since multiple annotators work for labeling narrations, there are multiple GT action instances corresponding to one real action instance. As shown by the image below, there are three GT action instances for one actual action instance. Some annotations are 99% overlapped while some are slightly overlapped which is caused by ambiguous temporal boundaries of action instances.
I checked that the official evaluation code in this repo does not handle such duplicate cases; this results in the underestimated performance of precise models. May I ask if is there any progress regarding improving the evaluation protocol?
Hello :) Thank you for maintaining this repository.
I would like to ask about the issue caused by duplicate annotations. Since multiple annotators work for labeling narrations, there are multiple GT action instances corresponding to one real action instance. As shown by the image below, there are three GT action instances for one actual action instance. Some annotations are 99% overlapped while some are slightly overlapped which is caused by ambiguous temporal boundaries of action instances.
I checked that the official evaluation code in this repo does not handle such duplicate cases; this results in the underestimated performance of precise models. May I ask if is there any progress regarding improving the evaluation protocol?