Prompt construction with cross-file context

[IQ179]

Hi, I really enjoyed reading your paper. I have a question regarding the prompt construction using cross-file context. When looking at Figure 1 of the paper, it appears that all cross-file contexts are commented out and placed in front of the actual code. Is there a reason for commenting them out?

For another example, similar to the deepseek coder, repository-level code completion was performed. Only the path (or file name) of the file is commented out, but not the contents of each file.

I was wondering if there was any particular reason for doing prompt construction this way, e.g., due to better performance, etc.

Thanks!

[Leolty]

Hi, thanks for your interest and for raising this question!

irrc, there was a reverse engineering effort of Copilot here which mentioned that the cross-file contexts were commented out. This served as a reference point for our prompt construction approach in the paper.

We did conduct some ablation studies to compare the performance impact of commenting out the cross-file context versus including it uncommented. We did not observe a dramatic difference in performance between them. This led us to assume that the specific format of including the cross-file context (commented or uncommented) may not have a significant impact, at least for the Codex model we were using. However, Codex is no longer available for use, so I cannot be certain if this assumption holds true for all models.

I hope this solves your questions. If you have any further questions or insights, please feel free to let us know!