关于logits_path和slm_res

[klbian]

您好，请问logits_path和slm_res都是用的SLM生成的topk_results吗？

[Drasick]

Clarification on logits_path and slm_res

Our logits_path refers to the logits file produced by the trained SLM.
By reading this file, we obtain a List[dict]. Each dictionary corresponds to a document, and its position in the list matches the document’s index.

Inside each dictionary, every key is a tuple representing (h_idx, t_idx), and the value is the logits score for all relation labels plus the NA label.

The slm_res file refers to the final JSON file that stores the overall evaluation results of the SLM.
Each dictionary in this JSON file contains the following four fields:

• title
• h_idx
• t_idx
• r

The Refiner will augment this existing SLM evaluation JSON by adding additional entries {title, h_idx, t_idx, r} that satisfy the refinement conditions, thereby enhancing the final slm_res.

[klbian]

Thanks for your reply！
I understand what this slm_res file is now. However, I have some questions about the logits_path file.
The result obtained through SLM is the topk_results file, and there is no logits file. Can I directly use the score in the topk_results file as the logits?
This is the file format of logits file that I generated based on your code:

[Drasick]

Notes on Handling SLM Logits During Inference

You may want to review the forward function of the SLM. Before performing multi-label prediction, the raw logits output by the SLM are post-processed. Therefore, after training the SLM, it is recommended to save the logits locally during the inference stage.

As shown in the code:

logits_values = torch.tensor([logits[key] for key in keys])
softmax_values = F.softmax(logits_values, dim=0)

the logits are passed through a softmax operation before being used for our refiner method.

If you could be sure hat the current top-k logits_values you are using are already softmax-normalized, this may not significantly affect the results. However, the best practice is still to output and store the raw logits directly, and apply the softmax operation during post-processing as done in the original implementation.