Skip to content

Nothing, Deprecated #549

Closed
xiaojunxiang2023 wants to merge 1 commit into
TencentCloud:masterfrom
xiaojunxiang2023:fix/create-env-vars-injection
Closed

Nothing, Deprecated #549
xiaojunxiang2023 wants to merge 1 commit into
TencentCloud:masterfrom
xiaojunxiang2023:fix/create-env-vars-injection

Conversation

@xiaojunxiang2023

Copy link
Copy Markdown
Contributor

Summary

Make env_vars passed to Sandbox.create(...) visible to later
commands.run(...) / run_code(...), by treating them as sandbox runtime
metadata
and syncing them into the guest through envd's native
POST :49983/init at create/restore time.

This is an independent redesign of the previously rejected #484. It is based on
current master (single commit, no dependency on #484) and drops the
rootfs/profile/bind-mount injection entirely, following the direction requested
in that review.

Background

Previously env_vars from the SDK were dropped at the CubeAPI layer
(containers: vec![]), so they never reached the guest and were invisible to
commands.run. #484 tried to fix this by writing
/etc/profile.d/99-cube-create-env.sh into the guest rootfs, which was rejected
as too hacky and not a runtime-layer solution.

Approach

env_vars are carried as runtime metadata and injected into envd, the same way
the upstream E2B orchestrator does it (POST /init -> envd defaults.EnvVars).
envd then merges them when it spawns any process, so the precedence
create env < per-command env holds for every process model, not just login
shells.

  • CubeAPI (sandboxes.rs): serialize env_vars into the
    cube.master.sandbox.create_env_vars annotation (JSON object) instead of
    dropping them.
  • CubeMaster: the annotation is forwarded to Cubelet via the existing
    cube.master prefix passthrough, and stripped in
    normalizeStoredTemplateRequest on template commit so per-instance secrets
    are never persisted into template snapshots.
  • Cubelet (envd_init.go): after the readiness probe, POST /init to
    <sandbox-ip>:49983 with the create env_vars (bounded retry). envd stores
    them additively (defaults.EnvVars.Store), so env already present in the
    guest runtime (e.g. template env) is preserved.

No changes to envd (pinned e2b-dev/infra@2026.16); this only wires up the
client side that CubeSandbox was missing compared to the official platform.

How this addresses the previous review

Review concern (#484) This PR
Solve env sync at the sandbox/runtime layer, not via shell profile envd /init -> defaults.EnvVars, merged at the exec layer for every spawned process
Don't persist secrets into rootfs/profile files Nothing written to disk; annotation is in-memory metadata and stripped on template commit
Profile export overrides per-command env= (broken precedence) envd applies defaults.EnvVars first, per-command last -> per-command correctly overrides
Shell parsing / injection risk No shell script generated; env passed as a JSON map
Coupling across start/restore/netfile/hostdir/rootfs One new file + one hook after the probe + annotation passthrough + commit strip
Tied to /bin/bash -l + /etc/profile.d, not general Independent of shell type / entrypoint; applies to all envd-spawned processes
API/Master propagation is a useful part Kept (CubeAPI -> annotation -> CubeMaster passthrough)
Don't merge Cubelet profile/rootfs/bind-mount injection Removed entirely

Precedence

template env < create env < per-command env.
Template env already lives in the guest runtime today; /init only adds the
create-time keys on top (same-key create overrides template), and envd's
per-command env still overrides both. Docker image ENV is not surfaced to
commands.run, matching upstream envd behavior (see e2b-dev/infra#2268).

Tests

Unit tests were added at every layer (all green):

  • Cubelet envd_init_test.go: no-annotation no-op, invalid-JSON error,
    empty-IP error, success path asserting the envVars/timestamp sent to a
    stub /init, and retry-then-succeed.
  • CubeMaster store_test.go: normalizeStoredTemplateRequest strips the
    create-env annotation on commit while preserving unrelated annotations.
  • CubeAPI sandboxes.rs: annotation key uses the cube.master prefix,
    absent/empty maps are skipped, and a non-empty map serializes to a JSON
    object.

Build/format verified via the project toolchain: cargo build/cargo test,
go build/go vet, and gofmt + cargo fmt clean.

Notes

  • The /init body only sends envVars + timestamp; all other fields are
    omitted, so it never clears envd's access token, default user/workdir, etc.
    (CubeSandbox runs envd without an access token, so first-time /init is
    authorized.)
  • Covers create and restore-from-snapshot (the normal create-from-template
    path runs the same probe + sync). Across pause/resume of the same VM the env
    persists in guest memory, so no extra resume hook is required.

Treat Sandbox.create env_vars as sandbox runtime metadata and sync them
into the guest at create/restore time through envd's native POST /init,
so later commands.run / run_code can read them. No rootfs/profile writes,
and per-command env precedence is preserved (create < per-command).

- CubeAPI: serialize env_vars into the cube.master.sandbox.create_env_vars
  annotation instead of dropping them.
- Cubelet: after the readiness probe, POST /init to <sandbox-ip>:49983 with
  the create env_vars (bounded retry, additive Store in envd); no shell /
  profile / rootfs mutation.
- CubeMaster: forward the annotation (cube.master prefix) and strip it on
  template commit so per-instance secrets are not persisted into snapshots.

Signed-off-by: xiaojunxiang <xiaojunxiang@kingsoft.com>
@xiaojunxiang2023 xiaojunxiang2023 force-pushed the fix/create-env-vars-injection branch from 8678886 to 64d2f57 Compare June 13, 2026 03:33
@xiaojunxiang2023 xiaojunxiang2023 changed the title feat(env): inject create-time env_vars into the guest via envd /init (runtime-metadata approach) Nothing, Deprecated Jun 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant