Add repos[].path property

[un-def]

• If not set, the legacy /workflow path is used (only during the transitional period)
• Relative paths are resolved relative to working_dir
• ~[/path] is supported, but ~username[/path] is not, as with files
• Available inside the container as DSTACK_REPO_DIR env variable

working_dir

• Must be absolute
• If not set, the image default value is used for tasks and services without commands and the legacy /workflow path for other configurations (replicating pre-0.19.27 JobConfigurator logic in CLI only during the transitional period)
• ~[/path] is supported, but ~username[/path] is not, as with files

runner

• /workflow/.venv moved to /dstack/venv
• /tmp/dstack_profile moved to /dstack/profile
• --working-dir is deprecated and ignored

Closes: #2851

[un-def]

The documentation will be updated separetely

[peterschmidt85]

Very cool! Quick question: DSTACK_REPO_PATH or DSTACK_REPO_DIR? I see both are used in the code.

[un-def]

DSTACK_REPO_DIR

dstack/runner/internal/executor/executor.go

Line 396 in 6ecc396

"DSTACK_REPO_DIR": ex.repoDir,

DSTACK_REPO_PATH in Airflow examples is unrelated.

[r4victor]

Will [code]path[/code] be mandatory? I remember we discussed that it can default to /dstack/repos/{repo_name} or something like that.

[un-def]

I remember we discussed that it can default to /dstack/repos/{repo_name} or something like that.

Right. I've changed the wording of this warning to:

repos[0].path is not set, using legacy repo path /workflow

In a future version the default value will be changed. To keep using /workflow, explicitly set repos[0].path to /workflow

[r4victor]

Please remind what's the planned default value for working_dir?

[r4victor]

Or you're planning to make it mandatory?

[un-def]

The default value will be taken from the image: WORKDIR

The WORKDIR instruction sets the working directory for any RUN, CMD, ENTRYPOINT, COPY and ADD instructions that follow it in the Dockerfile.

If not specified, the default working directory is /.

Once we change the default value from LEGACY_REPO_DIR (/workflow) to None (the default value taken from the image), we'll update the docs.

For CLI users, there is a warning. For API users, I think we'll add this info to the ”Deprecation and removals” section of the release notes.

[jvstme]

Note that dstack-runner built from this branch seems to be incompatible with older dstack-server. Getting this error when staring a run with --no-repo:

Exited (none)
Failed to set up the repo (repo_dir is not set)

In production, this can occur during a dstack-server rolling update, although it's not very likely

[un-def]

@jvstme good catch. I intentionally didn't add backward compatibility, as in production the runner can't be newer than the server (unless you've messed up settings), but I didn't think about rolling update.

I think it's safe to ignore this possibility.

[jvstme]

@un-def, another compatibility question — I'm getting this error when running a newer server with an older runner:

time=2025-09-01T16:55:06.753832-04:00 level=error msg=Exec failed err=[executor.go:371 executor.(*RunExecutor).execJob] [exec.go:12 executor.joinRelPath] path must be relative
time=2025-09-01T16:55:06.753867-04:00 level=info msg=Job state changed new=failed

Simple service configuration, working_dir unset

[jvstme]

@un-def, I'm also still getting the same error when running a newer client with an older server.

My setup is:

• client - master (be36bde)
• server, shim, runner - 0.19.26

Here is an example of a configuration that works with client 0.19.26, but stops working after updating the client to master:

> cat test.dstack.yml
type: task
commands:
- echo hi
- sleep 5

> git checkout 0.19.26

> dstack apply -f test.dstack.yml --reuse -y

weak-panther-1 provisioning completed (running)
hi
Exited (0)

> git checkout master

> git log --oneline | head -1
be36bde4 Bind shim and exposed container ports to localhost (#3057)

> dstack apply -f test.dstack.yml --reuse -y

working_dir is not set. Using legacy working directory /workflow

In future versions the default value will be taken from the image
- To keep using legacy working directory, set working_dir to /workflow
- To prepare for the future default, set working_dir to the image default working directory (only until the new default takes effect)

 NAME            BACKEND          RESOURCES                        PRICE    STATUS   SUBMITTED 
 nasty-wombat-1  aws (eu-west-3)  cpu=2 mem=8GB disk=100GB (spot)  $0.0287  running  15:30     

nasty-wombat-1 provisioning completed (running)
Exited (none)
Check dstack logs -d nasty-wombat-1 for more details.

> dstack logs -d nasty-wombat-1
time=2025-09-03T09:31:28.065881-04:00 level=info msg=Run job log_level=trace
time=2025-09-03T09:31:28.065987-04:00 level=trace msg=checking if repo checkout is needed
time=2025-09-03T09:31:28.068047-04:00 level=trace msg=Extracting tar archive
time=2025-09-03T09:31:28.068097-04:00 level=trace msg=Extracting code archive dst=/workflow src=/tmp/runner/code
time=2025-09-03T09:31:28.069852-04:00 level=info msg=Job state changed new=running
time=2025-09-03T09:31:28.080899-04:00 level=info msg=Continuing without updating LD_LIBRARY_PATH
time=2025-09-03T09:31:28.080954-04:00 level=error msg=Exec failed err=[executor.go:371 executor.(*RunExecutor).execJob] [exec.go:12 executor.joinRelPath] path must be relative
time=2025-09-03T09:31:28.081019-04:00 level=info msg=Job state changed new=failed