fix: add logging option for directory creation in get_full_path

[wakonig]

Description

get_full_path creates the directory by default. To be able to migrate to a less-permissive service account, we need to move the file access to the file writer service. For now, we should get a better awareness of methods that actually create them. To this end, I propose to add a log to the directory creation. We can monitor it later with grafana.
I would expect that they actually don't create the directory as the file writer should create it as soon as the scan is opened but there could be a timing issue.

Type of Change

• Add log to the get_full_path
• Add a wait_for_directory helper method

[codecov]

Codecov Report

❌ Patch coverage is 83.33333% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
bec_lib/bec_lib/file_utils.py	81.81%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[Copilot]

Pull request overview

This PR aims to improve observability around directory creation performed by bec_lib.file_utils.get_full_path, to help identify unexpected directory creation while migrating toward a less-permissive service account. It also introduces a small polling helper to wait for directory availability.

Changes:

• Added an optional warning log in get_full_path when it is about to create a missing directory.
• Added wait_for_directory(path, timeout, interval) helper for polling until a directory exists (or timing out).
• Added unit tests covering wait_for_directory success/timeout behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
bec_server/bec_server/file_writer/file_writer_manager.py	Passes a new flag to suppress directory-creation logging when calling get_full_path.
bec_lib/bec_lib/file_utils.py	Extends get_full_path with a logging option and adds wait_for_directory.
bec_lib/tests/test_file_utils.py	Adds tests for wait_for_directory.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.