Skip to content

Use dedicated temporary directory for all rsync pulls#4662

Open
happz wants to merge 3 commits intomainfrom
rsync-with-tempdir
Open

Use dedicated temporary directory for all rsync pulls#4662
happz wants to merge 3 commits intomainfrom
rsync-with-tempdir

Conversation

@happz
Copy link
Contributor

@happz happz commented Mar 9, 2026

This should prevent rsync process from creating temporary files in the destination directory tree. In case multiple rsync commands operate on the same directory structure, they may run into race condition:

  • rsync A is pull into the tree, creates a temporary file to hold partially transfered file;
  • rsync B is pushing the tree elsewhere, notices the file, and intends to push it;
  • rsync A completes the transfer, removes the temporary file;
  • rsync B sees the file is gone, and reports an error.

Isolating temporary files between rsync processes and from the manipulated tree should prevent this conflict.

The patch comes with new "tmpdir" attribute to provide run-specific temporary directories in a way compatible with tempfile.TemporaryDirectory.

Pull Request Checklist

  • implement the feature

@happz happz added this to planning Mar 9, 2026
@happz happz added bug Something isn't working step | provision Stuff related to the provision step labels Mar 9, 2026
@github-project-automation github-project-automation bot moved this to backlog in planning Mar 9, 2026
@happz happz added the ci | full test Pull request is ready for the full test execution label Mar 9, 2026
@happz happz moved this from backlog to implement in planning Mar 9, 2026
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds the --temp-dir option to rsync for pulls and local syncs. To fully resolve potential race conditions, also apply this to the push method in tmt/guest/__init__.py. Use the self.mkdtemp() context manager to create a temporary directory on the guest.

Note: Security Review did not run due to the size of the PR.

Copy link
Contributor

@LecrisUT LecrisUT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. Would be nice if this could be handled more centrally at the TransferOptions.to_rsync, but should be good for now

@happz happz moved this from implement to review in planning Mar 10, 2026
Copy link
Contributor

@thrix thrix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix correctly targets all 3 rsync invocations in the codebase (verified via ag). push() is correctly NOT modified since --temp-dir applies to the receiving side which is remote for push.

One concern: tempfile.TemporaryDirectory() creates in /tmp (tmpfs) while tmt workdirs are under /var/tmp/tmt/ (disk) - different filesystems. This makes rsync fall back to copy-in-place instead of atomic rename, introducing a brief window where destination files could have truncated data. See inline comment for details.

For small files (typical in tmt) this window is negligible and the fix is a clear net improvement over the original race condition.

@happz happz force-pushed the rsync-with-tempdir branch from 1bae78f to bce2f71 Compare March 10, 2026 20:29
Copy link
Contributor

@thrix thrix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cross-filesystem concern addressed in bce2f718. All 3 rsync invocations covered. Clean fix for the race condition.

@LecrisUT LecrisUT moved this from review to merge in planning Mar 12, 2026
happz added 3 commits March 13, 2026 10:09
This should prevent rsync process from creating temporary files in the
destination directory tree. In case multiple rsync commands operate on
the same directory structure, they may run into race condition:

* rsync A is pull into the tree, creates a temporary file to hold
  partially transfered file;
* rsync B is pushing the tree elsewhere, notices the file, and intends
  to push it;
* rsync A completes the transfer, removes the temporary file;
* rsync B sees the file is gone, and reports an error.

Isolating temporary files between rsync processes and from the
manipulated tree should prevent this conflict.
@happz happz force-pushed the rsync-with-tempdir branch from bce2f71 to 3ffb5b3 Compare March 13, 2026 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ci | full test Pull request is ready for the full test execution step | provision Stuff related to the provision step

Projects

Status: merge

Development

Successfully merging this pull request may close these issues.

3 participants