Skip to content

Commit 157a4ff

Browse files
fix: use shallow clone to prevent reward hacking via git history
Use --depth 1 when cloning repositories to prevent agents from accessing git history and exploiting it to retrieve original function implementations that were stripped out. This addresses a reward hacking vulnerability where agents can use git log/diff/show commands to find and copy original implementations instead of writing them from scratch. For Commit0Spec, also fetch the specific env_setup_commit with --depth 1 before resetting to it, since shallow clone only gets the default branch tip. Co-authored-by: openhands <openhands@all-hands.dev>
1 parent 4bbfa86 commit 157a4ff

File tree

1 file changed

+7
-2
lines changed

1 file changed

+7
-2
lines changed

commit0/harness/spec.py

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -114,9 +114,12 @@ def make_repo_script_list(self) -> list[str]:
114114
base_commit = self.instance["base_commit"]
115115

116116
setup_commands = [
117-
f"git clone -o origin https://github.com/{repo} {self.repo_directory}",
117+
# Use --depth 1 for shallow clone to prevent agents from accessing
118+
# git history and exploiting it to retrieve original implementations
119+
f"git clone --depth 1 -o origin https://github.com/{repo} {self.repo_directory}",
118120
f"chmod -R 777 {self.repo_directory}", # So nonroot user can run tests
119121
f"cd {self.repo_directory}",
122+
f"git fetch --depth 1 origin {env_setup_commit}",
120123
f"git reset --hard {env_setup_commit}",
121124
# Remove the remote so the agent won't see newer commits.
122125
"git remote remove origin",
@@ -218,7 +221,9 @@ def make_repo_script_list(self) -> list[str]:
218221
specs["python"] = 3.7
219222

220223
setup_commands = [
221-
f"git clone -o origin https://github.com/{repo} {self.repo_directory}",
224+
# Use --depth 1 for shallow clone to prevent agents from accessing
225+
# git history and exploiting it to retrieve original implementations
226+
f"git clone --depth 1 -o origin https://github.com/{repo} {self.repo_directory}",
222227
f"chmod -R 777 {self.repo_directory}", # So nonroot user can run tests
223228
f"cd {self.repo_directory}",
224229
# Remove the remote so the agent won't see newer commits.

0 commit comments

Comments
 (0)