Thank you for working open source, what are the several here? ? ?

[P-hyllis]

[model_name] [dataset_name] [batch_size] [device] ？[model_name] [dataset_name] [top_k] [device] ？

[SighingSnow]

For embedding.

• model_name is the name used in our paper. For example, you can use bm25, e5 or some other models. You can find all these model_names in https://github.com/SighingSnow/IFIR/blob/main/src/utils/constant.py#L19.
• dataset_name is the same. It can be found at
  https://huggingface.co/datasets/songtingyu/IFIR/tree/main.
• batch_size depends on you VRAM. You can use a large batch_size to accelerate embedding.
• device can be cuda:0 cuda:1. It depends on which GPU you wanna use.

For eval:
The top_k is a quite simple concept. If you choose 20, then it will calculate ndcg.20.

[P-hyllis]

Thank you for your answer, I always have this error, may I ask how to operate? Are there any detailed steps?

elastic_transport.ConnectionTimeout: Connection timed out

[SighingSnow]

Thank you for your answer, I always have this error, may I ask how to operate? Are there any detailed steps?

Maybe you can post your error or log file here ?

elastic_transport.ConnectionTimeout: Connection timed out

Could you please first check the es_utils folder? Maybe you should add some config to your elasticsearch config to avoid verification?

[P-hyllis]

I followed the "Experiments" steps you mentioned. After operating to "./bin/elasticsearch ", I didn't know what to do. The display was like this.

[P-hyllis]

When I disconnected the previous one, I was running "sh scripts/embed.sh [model_name] [dataset_name] [batch_size] [device]".
Will an error "elastic_transport. ConnectionTimeout: Connection timed out"

[SighingSnow]

I followed the "Experiments" steps you mentioned. After operating to "./bin/elasticsearch ", I didn't know what to do. The display was like this.

Hey, I think you should delete the elasticsearch folder. And re-download, unzip, update the config file to shut down the verification, then launch.

[P-hyllis]

Thank you for your answer. Is it necessary to operate like this every time? I have operated like this and run again, but still don't know how to proceed with the subsequent steps, such as "Embed the dataset and evaluate the model".

[SighingSnow]

Sorry for the late reply.

Is it necessary to operate like this every time?

Nope.

such as "Embed the dataset and evaluate the model".

like this,

sh scripts/embed.sh e5 fiqa 32 cuda:0
sh scripts/eval.sh e5 fiqa 20 cuda:0

I think you should get your hands dirty first — it may help you better understand the research.
Also, sorry for the lack of clarity in the README; we’ll update it later.
Thanks for the reminder!