diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a8117697..686bf788 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -88,6 +88,173 @@ jobs: shell: bash working-directory: backend/cli + # Task 7 gave macOS a seatbelt profile for network:"allowlist" — an SBPL + # profile plus an authenticated loopback proxy, built and unit-tested + # entirely from Linux with the platform injected, because no Mac exists on + # this project. `sandbox-exec` (macOS) and `bwrap --unshare-net` (Linux) + # are unrelated OS-level mechanisms underneath the same `Sandbox` API, so a + # green Linux run says nothing about whether seatbelt actually confines a + # real process the way the profile text claims — only this leg's macOS run + # does. See test/sandbox/egress-live-seatbelt.test.ts's doc comment for + # exactly what a red run here would mean. + sandbox: + name: Sandbox (${{ matrix.os }}) + runs-on: ${{ matrix.os }} + # Raised from 20: test/package/ now runs real pip installs through the + # sandbox against real pypi, which the sandbox suite alone never did. + timeout-minutes: 30 + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, macos-latest] + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - uses: ./.github/actions/setup-bun + # Same step the `test` job already has. The sandbox job never needed it + # until test/package/ joined it: those tests create a real project with + # `tmpdir({ git: true })`, and `git commit` exits 128 on a runner with no + # global identity configured. + - name: Configure git for tests + run: | + git config --global user.email "ci@openscience.dev" + git config --global user.name "OpenScience CI" + git config --global init.defaultBranch main + - name: Install and verify Linux sandbox + if: matrix.os == 'ubuntu-latest' + run: | + sudo apt-get update + sudo apt-get install --yes bubblewrap + # Ubuntu 24.04's host-wide AppArmor policy blocks unprivileged user + # namespaces on the hosted runner before bubblewrap can apply our + # stricter per-process profile. This runner is disposable; enable + # user namespaces for the job, then prove the sandbox can start. + if [[ -e /proc/sys/kernel/apparmor_restrict_unprivileged_userns ]]; then + echo 0 | sudo tee /proc/sys/kernel/apparmor_restrict_unprivileged_userns + fi + bwrap --ro-bind / / --dev /dev --proc /proc --unshare-pid --die-with-parent -- true + # R is the one backend with no verification anywhere: no runner has + # Rscript by default, and neither does any development machine on this + # project, so its two live tests skip everywhere and it ships on faith. + # r-base-core is the minimal package that provides Rscript. Linux only — + # `brew install r` on the macOS leg costs several minutes for a backend + # whose only platform-specific surface (the sandbox wrapper) is already + # covered there by the Python tests. + - name: Install R so the R installer tests actually run + if: matrix.os == 'ubuntu-latest' + run: | + sudo apt-get install --yes --no-install-recommends r-base-core + Rscript -e 'cat("Rscript", as.character(getRversion()), "\n")' + # test/package/ carries the merge gate: a governed install under + # network "allowlist", plus the assertion that the shell route to the + # same install is refused. Both legs run it, so the gate is a fact on + # Linux and macOS rather than a claim about one of them. + - run: bun test test/sandbox/ test/package/ + shell: bash + working-directory: backend/cli + + # Windows is deliberately NOT in the matrix above. That job's `test/package/` + # leg is the merge gate — a governed install under network "allowlist" — and + # allowlist egress does not exist on Windows yet: the container holds zero + # capabilities, so it has no network by construction, and nothing serves the + # broker pipe the spec carries. Adding windows-latest there would be red for a + # feature that was never built, which teaches a reader nothing. + # + # What this job DOES cover is the part that was only ever verified by hand: + # `test/sandbox/appcontainer-live.test.ts` runs a real CreateProcessW with real + # SECURITY_CAPABILITIES and asserts the child is confined. Everything else in + # test/sandbox/ exercises the Windows branch from Linux with the platform + # injected, which proves what we compose and nothing about what Windows does + # with it. That gap cost roughly ten manual round trips on a contributor's own + # machine, one command at a time, for bugs that were not exotic: `-c` where cmd + # wanted `/c`, `printf` in a shell with no printf, CommandLineToArgvW quoting + # handed to the one program that does not parse it that way. Each would have + # been red here within minutes. + # + # Widen this to test/package/ once the named-pipe broker lands. + sandbox-windows: + name: Sandbox (windows-latest) + runs-on: windows-latest + timeout-minutes: 20 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - uses: ./.github/actions/setup-bun + - run: >- + bun test + test/process/darwin-responsibility.test.ts + test/credentials/process-ledger.test.ts + test/project/authority-process-ledger.test.ts + shell: bash + working-directory: backend/cli + + - name: Configure git for tests + run: | + git config --global user.email "ci@openscience.dev" + git config --global user.name "OpenScience CI" + git config --global init.defaultBranch main + shell: bash + # OPENSCIENCE_SANDBOX_DEBUG makes the launcher dump what it hands the + # kernel — the SID, the attribute list, the whole STARTUPINFOEX with cb and + # lpAttributeList broken out. On a machine no one can log into, a failure + # that only says "expected true" is worth almost nothing. + # Just the live file. The rest of test/sandbox/ asserts POSIX composition + # -- seatbelt profile text, bubblewrap argv, `/tmp` paths that path.resolve + # turns into `C:\tmp` here -- and several tests read source through + # `new URL(...).pathname`, which yields `/D:/a/...` on Windows. Those are + # Linux/macOS concerns that happen to live in the same directory; running + # them here would produce 30-odd red results that say nothing about + # Windows. Widen deliberately, not by directory. + - run: bun test test/sandbox/appcontainer-live.test.ts test/sandbox/appcontainer-transport.test.ts + shell: bash + working-directory: backend/cli + env: + OPENSCIENCE_SANDBOX_DEBUG: "1" + + # A base interpreter the runner's own user owns. The whole Windows + # difficulty is that an AppContainer can only be granted paths its user + # owns, and every Python preinstalled on a GitHub runner is machine-wide — + # so without this step the install test would exercise the one + # configuration that is known not to work, and prove nothing about the one + # users are told to set up. + - name: Install uv and a user-owned Python + run: | + irm https://astral.sh/uv/install.ps1 | iex + $env:Path = "$env:USERPROFILE\.local\bin;$env:Path" + # 3.12.3 specifically, not "3.12". CPython 3.12.4 changed + # os.mkdir(mode=0o700) so tempfile.mkdtemp() creates a directory whose + # DACL does not inherit, and an AppContainer process cannot then write + # into the directory it just made (python/cpython#134587, fixed + # upstream but unreleased). 3.12.3 is the last release without it, and + # pinning here is what lets the install path stay on pip. + uv python install 3.12.3 + uv python list --only-installed --output-format json + shell: pwsh + + # The seven hops between "a process starts in a container" and "a package + # is importable": interpreter choice, venv creation, the base pin, the ACL + # grant, the launcher spawning its base, pip, and the pipe->broker->proxy + # chain. None of it was covered, so all of it was found one round trip at + # a time on a contributor's own machine. + # The sandbox re-enters this binary twice — as the launcher and as the + # egress shim — and in a source checkout neither is what ships. The shim + # especially: `bun ` dies inside the container with `error loading + # current directory`, while `bun --version` in the same container with the + # same working directory exits 0. That difference is a property of a dev + # artifact users never run, so the test drives the compiled binary. + - name: Compile the binary the sandbox re-enters + run: bun build --compile ./src/index.ts --outfile "$RUNNER_TEMP/openscience-self.exe" + shell: bash + working-directory: backend/cli + + - name: Package install, end to end + run: | + $env:Path = "$env:USERPROFILE\.local\bin;$env:Path" + bun test test/sandbox/appcontainer-install.test.ts + shell: pwsh + working-directory: backend/cli + env: + OPENSCIENCE_SANDBOX_DEBUG: "1" + OPENSCIENCE_SELF_BINARY: ${{ runner.temp }}\openscience-self.exe + test: name: Test runs-on: ubuntu-latest diff --git a/backend/cli/src/cli/cmd/sandbox.ts b/backend/cli/src/cli/cmd/sandbox.ts index f2c5da38..6d927ea0 100644 --- a/backend/cli/src/cli/cmd/sandbox.ts +++ b/backend/cli/src/cli/cmd/sandbox.ts @@ -17,6 +17,22 @@ function printStatus(config?: Config.Sandbox) { const enabled = config?.enabled === true UI.println(`${S.TEXT_NORMAL_BOLD}Execution sandbox${S.TEXT_NORMAL}`) + // Three states, not two. "enabled" describes the CONFIG; whether anything is + // actually confined depends on a backend existing. Keying the sentence off + // `enabled` alone told a Windows user "agent shell commands are confined to + // the workspace" on a machine where `Sandbox.backend()` is "none" and nothing + // confines anything — a false statement about a security property, which is + // the worst kind of wrong thing for this command to print. + // Then it printed "are confined to the workspace" on a Windows run whose + // `sandbox test` failed containment in the very next command. A backend being + // AVAILABLE is not the same as it working, and this command does not run the + // commands that would tell the difference — so it now reports what it actually + // knows (which backend is applied) and names the command that can prove it. + const effect = !enabled + ? "run with full user authority" + : d.available + ? `are launched through ${d.tool ?? d.backend} - run 'openscience sandbox test' to verify containment` + : "are NOT confined here: no backend on this platform" UI.println( ` status ${enabled ? `${S.TEXT_SUCCESS_BOLD}enabled` : `${S.TEXT_DIM}disabled`}${S.TEXT_NORMAL}` + `${S.TEXT_DIM} (agent shell commands${ @@ -28,21 +44,31 @@ function printStatus(config?: Config.Sandbox) { ` backend ${ d.available ? `${S.TEXT_SUCCESS}${d.backend}${S.TEXT_NORMAL} ${S.TEXT_DIM}(${d.tool})${S.TEXT_NORMAL}` - : `${S.TEXT_WARNING}unavailable${S.TEXT_NORMAL} ${S.TEXT_DIM}— ${d.reason}${S.TEXT_NORMAL}` + : `${S.TEXT_WARNING}unavailable${S.TEXT_NORMAL} ${S.TEXT_DIM}- ${d.reason}${S.TEXT_NORMAL}` }`, ) if (enabled) { - UI.println(` network ${config?.network ?? "deny"}`) + // "allow" is not what it says on bubblewrap or seatbelt: both deny every + // socket in every mode, because neither can grant outbound access without + // also exposing everything bound to 127.0.0.1. Printing the configured word + // alone made this command state a capability the machine does not have. + const net = config?.network ?? "deny" + const hollow = net === "allow" && (d.backend === "bubblewrap" || d.backend === "seatbelt") + UI.println( + ` network ${net}` + + (hollow ? `${S.TEXT_DIM} (this backend denies all sockets - use 'allowlist')${S.TEXT_NORMAL}` : ""), + ) UI.println( ` project trust ${config?.requireProjectTrust ? "required for all execution" : "routine sandboxed work allowed"}`, ) UI.println(` on missing backend ${config?.onUnavailable ?? "error"}`) if (config?.allowWrite?.length) UI.println(` extra writable ${config.allowWrite.join(", ")}`) + if (config?.allowHosts?.length) UI.println(` extra hosts ${config.allowHosts.join(", ")}`) } if (enabled && !d.available) { UI.println("") UI.println( - ` ${S.TEXT_WARNING_BOLD}Note:${S.TEXT_NORMAL} sandbox is on but no backend exists here — ` + + ` ${S.TEXT_WARNING_BOLD}Note:${S.TEXT_NORMAL} sandbox is on but no backend exists here - ` + `execution follows the "${config?.onUnavailable ?? "error"}" fallback policy. It takes effect on machines with a backend.`, ) } @@ -53,6 +79,21 @@ async function showStatus() { directory: process.cwd(), async fn() { printStatus(await effectiveSandbox()) + // A prerequisite the user cannot discover from anything else. On Windows + // an AppContainer can only be granted access to paths its user owns, so a + // machine-wide Python is unusable by a sandboxed process however healthy + // it is — and the only symptom otherwise is an install failing much later + // with an error about the interpreter rather than about ownership. + // + // Printed only when it applies, and worded so nobody reads it as + // "containment is broken": it is not, and a user who turns the sandbox + // off over this would lose confinement they still have. + const { Installer } = await import("../../package/installer") + const blocked = await Installer.blocked().catch(() => undefined) + if (blocked) { + UI.empty() + for (const line of blocked.split("\n")) UI.println(line ? ` ${S.TEXT_WARNING}${line}${S.TEXT_NORMAL}` : "") + } }, }) } @@ -72,14 +113,19 @@ const EnableCommand = cmd({ builder: (yargs: Argv) => yargs .option("network", { - choices: ["allow", "deny"] as const, - describe: "allow or deny network egress from sandboxed commands (default: deny)", + choices: ["deny", "allowlist", "allow"] as const, + describe: "network egress from sandboxed commands: deny (default), allowlist, or allow", }) .option("allow", { type: "string", array: true, describe: "extra absolute path the sandbox may write to (repeatable)", }) + .option("allow-host", { + type: "string", + array: true, + describe: "extra host the sandbox may reach when network is 'allowlist' (repeatable)", + }) .option("on-unavailable", { choices: ["warn", "error", "allow"] as const, describe: "what to do when no backend exists on a machine (default: error)", @@ -93,7 +139,7 @@ const EnableCommand = cmd({ directory: process.cwd(), async fn() { const patch: Partial = { enabled: true } - if (args.network) patch.network = args.network as "allow" | "deny" + if (args.network) patch.network = args.network if (args["on-unavailable"]) patch.onUnavailable = args["on-unavailable"] as "warn" | "error" | "allow" if (typeof args["require-project-trust"] === "boolean") { patch.requireProjectTrust = args["require-project-trust"] @@ -143,7 +189,7 @@ const TestCommand = cmd({ const result = await Sandbox.selfTest() if (!result.available) { const d = Sandbox.describe() - UI.println(`${S.TEXT_WARNING}No sandbox backend available${S.TEXT_NORMAL} — ${d.reason}.`) + UI.println(`${S.TEXT_WARNING}No sandbox backend available${S.TEXT_NORMAL} - ${d.reason}.`) UI.println(`${S.TEXT_DIM}Nothing to test here.${S.TEXT_NORMAL}`) return } @@ -151,6 +197,11 @@ const TestCommand = cmd({ `${S.TEXT_NORMAL_BOLD}Sandbox self-test${S.TEXT_NORMAL} ${S.TEXT_DIM}(${result.backend})${S.TEXT_NORMAL}`, ) for (const c of result.checks) { + // The glyphs below are only reachable when a backend EXISTS, so they + // cannot print on Windows today, where the command exits above. Anything + // printed on a backend-less machine must stay ASCII: a Windows console + // decodes our UTF-8 as its OEM code page, and an em dash arrived as + // "\u0393\u00c7\u00f6" in a real run. Keep that rule if a Windows backend lands. const mark = c.skipped ? `${S.TEXT_DIM}– skip` : c.pass ? `${S.TEXT_SUCCESS}✓ pass` : `${S.TEXT_DANGER}✗ FAIL` UI.println(` ${mark}${S.TEXT_NORMAL} ${c.name}${c.detail ? ` ${S.TEXT_DIM}(${c.detail})${S.TEXT_NORMAL}` : ""}`) } @@ -158,7 +209,7 @@ const TestCommand = cmd({ UI.println( result.ok ? `${S.TEXT_SUCCESS_BOLD}Containment verified.${S.TEXT_NORMAL}` - : `${S.TEXT_DANGER_BOLD}Containment FAILED — do not rely on the sandbox until this passes.${S.TEXT_NORMAL}`, + : `${S.TEXT_DANGER_BOLD}Containment FAILED - do not rely on the sandbox until this passes.${S.TEXT_NORMAL}`, ) }, }) diff --git a/backend/cli/src/compute/jobs.ts b/backend/cli/src/compute/jobs.ts index 4917734d..bf0e03c7 100644 --- a/backend/cli/src/compute/jobs.ts +++ b/backend/cli/src/compute/jobs.ts @@ -10,6 +10,7 @@ import { OpenScience } from "../openscience" import { Shell } from "../shell/shell" import { Instance } from "../project/instance" import { Sandbox } from "../sandbox/sandbox" +import { EgressRuntime } from "../sandbox/egress-runtime" import { Filesystem } from "../util/filesystem" import { FileLease } from "../util/file-lease" import { ProvenanceEnvelope } from "../science/provenance/envelope" @@ -237,6 +238,10 @@ export namespace ComputeJobs { recovery_attempts: z.number().int().nonnegative().optional(), recovery_retry_at: z.string().optional(), session_id: z.string().startsWith("ses_").optional(), + // Persists the whole Decision, including its own sandbox.network — a + // second copy of the persisted enum below `sandbox.network` carries; see + // the comment on ExecutionAuthority.Decision for the downgrade cost of + // widening either one. authority: ExecutionAuthority.Decision.optional(), scope: z .object({ @@ -248,8 +253,11 @@ export namespace ComputeJobs { .object({ requested: z.boolean(), enforced: z.boolean(), - backend: z.enum(["seatbelt", "bubblewrap", "none"]), - network: z.enum(["allow", "deny"]), + backend: z.enum(["seatbelt", "bubblewrap", "appcontainer", "none"]), + // Persisted — widening this costs an older binary its ability to + // read a newer record. `authority.sandbox.network` above is the same + // enum persisted a second time; see ExecutionAuthority.Decision. + network: z.enum(["deny", "allowlist", "allow"]), warning: z.string().optional(), }) .optional(), @@ -335,6 +343,8 @@ export namespace ComputeJobs { argv: string[] sandbox?: Job["sandbox"] temporary?: string + /** HTTP_PROXY-shaped route to the egress proxy, when the policy has one. */ + env?: Record } const active = new Map() @@ -1282,6 +1292,25 @@ export namespace ComputeJobs { } } + /** + * The network policy that actually applies to an SSH transport. + * + * "allowlist" is relaxed to "allow" because that policy is HTTP-only: it + * severs the network namespace and offers one HTTP proxy socket, which ssh + * cannot use at all — it does not read HTTP_PROXY and needs ProxyCommand or + * SOCKS. Applying it to ssh is not bounded egress, it is denial with a + * confusing error, and it buys nothing: the job's command runs on the remote + * machine, so the process being confined is a transport rather than the + * workload. + * + * An explicit "deny" is left alone. That is a user saying no network, not a + * default they never chose, and honouring it is the difference between + * relaxing a default and overriding an instruction. + */ + export function transportNetwork(requested: "deny" | "allowlist" | "allow") { + return requested === "allowlist" ? ("allow" as const) : requested + } + async function launch( job: Job, host: Host | undefined, @@ -1290,14 +1319,43 @@ export namespace ComputeJobs { ): Promise { const spec = command(job, host) if (host) { + // The ssh CLIENT is what gets wrapped here; the job's command runs on the + // remote machine. Two consequences. + // + // Network containment of this process buys nothing — the code being + // confined is a transport, not the workload — and under "allowlist" it + // actively breaks the feature: that policy severs the network namespace + // and offers one HTTP proxy socket as the only route out, which ssh + // cannot use. It reads HTTP_PROXY not at all and needs ProxyCommand or + // SOCKS. So an allowlist remote job did not fail closed with a useful + // message, it failed with an opaque connection error on the default + // policy. + // + // Filesystem containment still matters and is kept: ssh reads keys and + // can write locally. Only the network dimension is relaxed, and the value + // reported below is the one actually applied, not the one requested — + // reporting "allowlist" for a process running unconfined would be worse + // than the original bug. + const network = transportNetwork(authority.sandbox.network) + const relaxed = { ...authority.sandbox, network } + const egress = await EgressRuntime.egressFor(relaxed) const planned = Sandbox.wrapArgv({ file: spec.argv[0]!, args: spec.argv.slice(1), workspace: authority.writable, readable: authority.readable, unreadable: OpenScience.kernelSensitivePaths(), - options: authority.sandbox, + options: { ...relaxed, egress }, }) + const note = + network === authority.sandbox.network + ? planned.warning + : [ + planned.warning, + "network left unconfined for the ssh transport: the allowlist proxy is HTTP-only and ssh cannot use it. The job's own command runs on the remote host, outside this sandbox either way.", + ] + .filter(Boolean) + .join(" ") return { argv: [planned.file, ...planned.args], temporary: planned.temporary, @@ -1305,15 +1363,17 @@ export namespace ComputeJobs { requested: authority.sandbox.enabled, enforced: planned.sandboxed, backend: planned.backend, - network: authority.sandbox.network, - warning: planned.warning, + network, + warning: note, }, + env: planned.env, } } await fs.mkdir(logsOf(scope.root), { recursive: true }) await fs.writeFile(exitOf(scope.root, job.id), "", { mode: 0o600 }) const wrapped = `(${job.command}\n); code=$?; printf %s "$code" > ${quote(exitOf(scope.root, job.id))}; exit "$code"` + const egress = await EgressRuntime.egressFor(authority.sandbox) const planned = Sandbox.wrapArgv({ file: Shell.acceptable(), args: ["-lc", wrapped], @@ -1321,7 +1381,7 @@ export namespace ComputeJobs { readable: authority.readable, extraWritable: [exitOf(scope.root, job.id)], unreadable: OpenScience.kernelSensitivePaths(), - options: authority.sandbox, + options: { ...authority.sandbox, egress }, }) return { argv: [planned.file, ...planned.args], @@ -1333,6 +1393,7 @@ export namespace ComputeJobs { network: authority.sandbox.network, warning: planned.warning, }, + env: planned.env, } } @@ -1342,13 +1403,14 @@ export namespace ComputeJobs { authority: ExecutionAuthority.Decision, ): Promise { await currentAuthority(authority) + const egress = await EgressRuntime.egressFor(authority.sandbox) const planned = Sandbox.wrapArgv({ file: argv[0]!, args: argv.slice(1), workspace: authority.writable, readable: authority.readable, unreadable: OpenScience.kernelSensitivePaths(), - options: authority.sandbox, + options: { ...authority.sandbox, egress }, }) try { const proc = Bun.spawn([planned.file, ...planned.args], { diff --git a/backend/cli/src/config/config.ts b/backend/cli/src/config/config.ts index 8d9025e3..29ddb1ae 100644 --- a/backend/cli/src/config/config.ts +++ b/backend/cli/src/config/config.ts @@ -767,9 +767,15 @@ export namespace Config { "Run local terminals, kernels, and shell commands inside an OS sandbox (macOS Seatbelt / Linux bubblewrap) that confines writes to authorized project roots. Disabled by default; enable it from the composer or Sandbox settings.", ), network: z - .enum(["allow", "deny"]) + .enum(["deny", "allowlist", "allow"]) .optional() - .describe("Whether sandboxed commands may reach the network. Default: deny."), + .describe("Whether sandboxed commands may reach the network. Default: allowlist."), + allowHosts: z + .array(z.string()) + .optional() + .describe( + "Extra hosts sandboxed processes may reach when network is 'allowlist'. A leading dot matches subdomains, e.g. '.internal.example.com'.", + ), allowWrite: z .array(z.string()) .optional() @@ -1786,8 +1792,17 @@ export namespace Config { } const policy = { ...(base ?? {}), ...(managed ?? {}) } return { + // main's default, kept deliberately. This branch had flipped it to true; + // main's schema now documents "Disabled by default; enable it from the + // composer or Sandbox settings", which is a product decision a rebase has + // no business reversing quietly. enabled: policy.enabled ?? false, + // main's default too. This branch ADDS "allowlist" as a third state but does + // not make it the default: a rebase that changes what existing users get, + // on top of changing what the sandbox can do, is two changes wearing one + // commit. Callers that need it ask for it. network: policy.network ?? "deny", + allowHosts: policy.allowHosts ?? [], allowWrite: policy.allowWrite ?? [], onUnavailable: policy.onUnavailable ?? "error", requireProjectTrust: policy.requireProjectTrust ?? false, diff --git a/backend/cli/src/index.ts b/backend/cli/src/index.ts index 462e4c68..57178146 100644 --- a/backend/cli/src/index.ts +++ b/backend/cli/src/index.ts @@ -5,6 +5,13 @@ // import time with empty env (sync only catches up later in middleware). import "./openscience/preload-env" +// MUST be the next import, and must stay above every import below it. The three +// sandbox re-entry points live in there, and they have to be answered before +// this module's own import graph is evaluated: `./openscience` and friends +// bootstrap the user's data directories at module scope, which cannot happen +// inside an AppContainer. See the file header — it is the bug, not a caution. +import "./sandbox/fastpath" + import yargs from "yargs" import { hideBin } from "yargs/helpers" import { RunCommand } from "./cli/cmd/run" @@ -76,6 +83,14 @@ if (process.argv[2] === GROUP_LAUNCHER_ARG) { } } +// The sandbox re-entry points (`__egress-shim`, `__appcontainer-detached`, +// `__appcontainer-launch`) were here, guarded by a comment claiming they ran +// before any other CLI machinery. They did not: every static import above is +// evaluated first, and one of them bootstraps the user's data directories at +// module scope, which an AppContainer cannot do. They now live in +// `./sandbox/fastpath`, imported at the top of this file so the ordering is +// enforced by the module graph rather than asserted by a comment. + process.on("unhandledRejection", (e) => { Log.Default.error("rejection", { e: e instanceof Error ? e.message : e, diff --git a/backend/cli/src/openscience/index.ts b/backend/cli/src/openscience/index.ts index 5000267b..37ddb0a2 100644 --- a/backend/cli/src/openscience/index.ts +++ b/backend/cli/src/openscience/index.ts @@ -114,7 +114,16 @@ const KERNEL_RUNTIME_KEYS = new Set([ "WINDIR", "PATHEXT", "COMSPEC", + "HTTP_PROXY", + "HTTPS_PROXY", + "NO_PROXY", + "http_proxy", + "https_proxy", + "no_proxy", ]) +/** Derived rather than hand-written so the two can never drift. Every prefix in + * SAFE_ENV_PREFIXES is already uppercase, so only this set needs folding. */ +const KERNEL_RUNTIME_UPPER = new Set([...KERNEL_RUNTIME_KEYS].map((key) => key.toUpperCase())) const SAFE_SYNCED_KEYS = new Set([ ...BYOK_LLM_ENV_KEYS, ...SYNCED_SERVICE_ENV_KEYS, @@ -1171,9 +1180,23 @@ export namespace OpenScience { const result: Record = {} for (const [key, value] of Object.entries(env)) { if (!value) continue + // Case-insensitive, because Windows environment keys are. + // + // Windows presents these as `Path`, `SystemRoot`, `windir` and `ComSpec`, + // and every comparison here was exact — so the allowlist matched none of + // them and a kernel on Windows ran with no PATH and no SystemRoot at all. + // Measured downstream as `CreateProcess ... Win32 203` + // (ERROR_ENVVAR_NOT_FOUND) when launching the interpreter. + // + // This only ever widens on Windows, where the OS itself treats these + // names case-insensitively, so `path` and `PATH` are the same variable + // and matching one but not the other was never a security boundary. The + // original casing is preserved in the result: the child should see its + // environment exactly as we did. + const upper = key.toUpperCase() const runtime = - SAFE_ENV_PREFIXES.some((prefix) => (prefix.endsWith("_") ? key.startsWith(prefix) : key === prefix)) || - KERNEL_RUNTIME_KEYS.has(key) + SAFE_ENV_PREFIXES.some((prefix) => (prefix.endsWith("_") ? upper.startsWith(prefix) : upper === prefix)) || + KERNEL_RUNTIME_UPPER.has(upper) if (runtime) result[key] = value } return result diff --git a/backend/cli/src/package/environment.ts b/backend/cli/src/package/environment.ts new file mode 100644 index 00000000..14153922 --- /dev/null +++ b/backend/cli/src/package/environment.ts @@ -0,0 +1,228 @@ +import fs from "fs/promises" +import path from "path" +import z from "zod" +import { Global } from "../global" + +/** + * Environment state. No process spawning lives here — the installer owns that. + * + * The manifest is the source of truth and the directory is derived, therefore a + * cache. That is why they sit in different roots: `Global.Path.cache` may be + * cleared by the user or a cleaner at any time, and an environment must be + * rebuildable from its manifest afterwards. Putting the manifest inside the + * directory would make a cache clean an unrecoverable data loss. + */ +export namespace Environment { + export const Language = z.enum(["python", "r"]) + export type Language = z.infer + + export const Record = z.object({ + name: z.string(), + language: Language, + /** Only what was explicitly asked for, never the resolved closure. */ + requested: z.array(z.string()).default([]), + /** Resolved name → version, as the installer reported it after the fact. */ + installed: z.record(z.string(), z.string()).default({}), + /** Size of the resolved closure, reported as a number rather than listed. */ + total: z.number().int().nonnegative().default(0), + createdAt: z.number(), + updatedAt: z.number(), + }) + export type Record = z.infer + + export function manifest(projectID: string, name: string) { + return path.join(Global.Path.data, "envs", projectID, `${name}.json`) + } + + export function directory(projectID: string, name: string) { + return path.join(Global.Path.cache, "envs", projectID, name) + } + + export async function read(projectID: string, name: string) { + const file = Bun.file(manifest(projectID, name)) + if (!(await file.exists())) return undefined + const parsed = Record.safeParse(await file.json().catch(() => undefined)) + return parsed.success ? parsed.data : undefined + } + + /** + * Write the manifest, validating first. + * + * The parse is not ceremony. `JSON.stringify` drops keys whose value is + * `undefined`, so a caller that omits one — a tool invoked without zod + * having applied its defaults, say — writes a manifest that `read` then + * rejects. The result is an environment that exists on disk, holds installed + * packages, and is invisible to the inventory: silent, and indistinguishable + * from "never created" at every call site. Validating here turns that into a + * loud failure at the moment of the mistake. + */ + export async function write(projectID: string, value: Record) { + const parsed = Record.safeParse(value) + if (!parsed.success) { + throw new Error( + `Refusing to write an unreadable environment manifest for ${value.name}: ${parsed.error.issues + .map((i) => `${i.path.join(".") || "(root)"} ${i.message}`) + .join("; ")}`, + ) + } + const file = manifest(projectID, parsed.data.name) + await fs.mkdir(path.dirname(file), { recursive: true }) + await Bun.write(file, JSON.stringify(parsed.data, null, 2)) + } + + /** Every environment for a project. A manifest that fails to parse is skipped + * rather than thrown on: one hand-edited or half-written file must not make + * every other environment in the project invisible. */ + export async function list(projectID: string) { + const dir = path.join(Global.Path.data, "envs", projectID) + const names = await fs.readdir(dir).catch(() => [] as string[]) + const values = await Promise.all( + names.filter((n) => n.endsWith(".json")).map((n) => read(projectID, n.slice(0, -".json".length))), + ) + return values.filter((v): v is Record => Boolean(v)) + } + + /** + * Purely additive means every package present before is present after at the + * same version. + * + * Additive changes leave a live kernel correct: a new module imports on first + * use. Any removal, downgrade or version change does not — a module already + * loaded into the interpreter stays at the old version in memory while the + * files on disk say otherwise, which is worse than an obvious failure because + * it is silent. That asymmetry is the whole reason this function exists + * rather than restarting on every install. + */ + export function additive(before: Record["installed"], after: Record["installed"]) { + return Object.entries(before).every(([name, version]) => after[name] === version) + } + + /** + * A persisted record that an installer is working on this environment, with + * enough identity to tell "still running" from "died mid-install" after a CLI + * restart. pid alone is not enough — pids are reused — so the platform start + * token rides along, the same guard `science/kernel/process.ts` already + * applies to kernels. The token is optional because it does not exist on + * every platform. + * + * Under `Global.Path.state`, not `data`: this is per-machine liveness, not + * something to survive a restore onto another machine. + */ + const Claim = z.object({ + pid: z.number().int(), + token: z.string().optional(), + startedAt: z.number(), + /** Set when the install finished and FAILED. A claim carrying this is no + * longer about liveness — the process is gone and we know why. */ + error: z.string().optional(), + }) + + export function claimPath(projectID: string, name: string) { + return path.join(Global.Path.state, "envs", projectID, `${name}.claim.json`) + } + + export async function claim(projectID: string, name: string, pid: number, value?: string) { + const file = claimPath(projectID, name) + await fs.mkdir(path.dirname(file), { recursive: true }) + await Bun.write(file, JSON.stringify({ pid, token: value, startedAt: Date.now() })) + } + + export async function release(projectID: string, name: string) { + await fs.rm(claimPath(projectID, name), { force: true }) + } + + /** + * Record that a detached install failed. + * + * Without this a `wait: false` failure vanished: the error was caught and + * discarded, no manifest was written, the claim was released cleanly, and the + * agent had been told "started installing" with no way to ever learn + * otherwise. Replacing the claim rather than deleting it keeps one file as + * the single place an unfinished install is described, whatever became of it. + */ + export async function fail(projectID: string, name: string, message: string) { + const file = claimPath(projectID, name) + await fs.mkdir(path.dirname(file), { recursive: true }) + await Bun.write(file, JSON.stringify({ pid: process.pid, startedAt: Date.now(), error: message.slice(0, 2000) })) + } + + /** + * Resolve every outstanding claim for a project. + * + * An install that cannot be proven still running is `unknown`, never `fine`: + * pip has no transactions, so an interrupted one may have left a partial + * tree, and silently trusting it turns into a mystery ImportError several + * turns later. "Cannot prove" means the process is gone, or a token that was + * captured no longer matches — NOT merely that no token exists, which is the + * ordinary case on Windows. + * + * Resolved claims are deleted so a second call does not re-report them; a + * still-running one is left in place, because it is still true. + */ + export async function reconcile(projectID: string) { + const { KernelProcessIdentity } = await import("../science/kernel/process") + const dir = path.join(Global.Path.state, "envs", projectID) + const names = await fs.readdir(dir).catch(() => [] as string[]) + const out: { name: string; outcome: "running" | "unknown" | "failed"; message?: string }[] = [] + for (const file of names.filter((n) => n.endsWith(".claim.json"))) { + const name = file.slice(0, -".claim.json".length) + const parsed = Claim.safeParse( + await Bun.file(path.join(dir, file)) + .json() + .catch(() => undefined), + ) + if (!parsed.success) { + await fs.rm(path.join(dir, file), { force: true }) + out.push({ name, outcome: "unknown" }) + continue + } + // A recorded failure is not a liveness question — the process is gone and + // the reason is known, so report it and clear it. + if (parsed.data.error) { + await fs.rm(path.join(dir, file), { force: true }) + out.push({ name, outcome: "failed", message: parsed.data.error }) + continue + } + const alive = KernelProcessIdentity.running(parsed.data.pid, parsed.data.token) + if (!alive) await fs.rm(path.join(dir, file), { force: true }) + out.push({ name, outcome: alive ? "running" : "unknown" }) + } + return out + } + + const held = new Map>() + + const slot = (projectID: string, name: string) => `${projectID} ${name}` + + /** True while something holds this environment's lock. */ + export function busy(projectID: string, name: string) { + return held.has(slot(projectID, name)) + } + + /** + * Serialise work per environment. Other environments stay fully usable — the + * lock is per-env precisely so one long install does not stop every kernel in + * the project. + * + * The chain is built from the previous entry rather than awaited in place, so + * a caller arriving mid-install queues instead of racing. `previous.then(fn, + * fn)` runs the next body whether the one before it resolved or rejected: a + * failed install must not cancel the work queued behind it. The slot is + * cleared only if it is still ours, so a later waiter that replaced it is not + * evicted — and it is cleared in a `finally`, because a lock that survives a + * throw would brick the environment for the process lifetime, which is the + * latching bug the egress runtime shipped with. + */ + export async function lock(projectID: string, name: string, fn: () => Promise): Promise { + const id = slot(projectID, name) + const previous = held.get(id) ?? Promise.resolve() + const run = previous.then(fn, fn) + const tracked = run.catch(() => undefined) + held.set(id, tracked) + try { + return await run + } finally { + if (held.get(id) === tracked) held.delete(id) + } + } +} diff --git a/backend/cli/src/package/installer-r.ts b/backend/cli/src/package/installer-r.ts new file mode 100644 index 00000000..0b96fd8f --- /dev/null +++ b/backend/cli/src/package/installer-r.ts @@ -0,0 +1,152 @@ +import fs from "fs/promises" +import { Config } from "../config/config" +import { EgressRuntime } from "../sandbox/egress-runtime" +import { Sandbox } from "../sandbox/sandbox" +import { Installer } from "./installer" + +/** + * The R backend, and the simpler one by a wide margin. + * + * There is no ladder to probe and no pip to bootstrap: `install.packages` is + * part of base R, and the binding is a library path (`R_LIBS_USER`) rather than + * a per-environment interpreter. `cran.r-project.org` is already in + * `Egress.DEFAULT_RULES`, so the allowlist needs no change either. + * + * Runs under the same sandbox as the Python installer, for the same reason: the + * install is not more privileged than the kernel that will use it. + */ +export namespace InstallerR { + /** + * The package index. A named constant rather than a literal inside the + * generated R script: it is the one value that decides where packages come + * from, and a test can assert it by equality instead of grepping this file + * for a domain — which reads to a static analyser as an incomplete URL check. + * + * Already covered by `Egress.DEFAULT_RULES`, so changing it means changing + * the allowlist too. + */ + export const REPO = "https://cran.r-project.org" + + /** PEP 503-style normalisation is wrong for CRAN — R package names are + * case-sensitive and `.` is meaningful (`data.table`). Compared verbatim. */ + const key = (value: string) => value.trim() + + export async function create(directory: string) { + await fs.mkdir(Installer.rlibrary(directory), { recursive: true }) + } + + async function confined(directory: string, argv: string[]) { + const policy = await Config.trustedSandbox() + const egress = await EgressRuntime.egressFor(policy) + return Sandbox.wrapArgv({ + file: argv[0]!, + args: argv.slice(1), + workspace: [directory], + options: { ...policy, egress }, + }) + } + + /** `Rscript -e` with the library pinned to the environment. `lib` is passed + * explicitly as well as through `R_LIBS_USER`, because `install.packages` + * otherwise picks the first writable entry of `.libPaths()` — which on a + * machine with a user library already set would be the wrong directory and + * would leak this environment's packages into every other project. */ + export async function install(input: { directory: string; packages: string[]; signal?: AbortSignal }) { + const lib = Installer.rlibrary(input.directory) + await create(input.directory) + const names = input.packages.map((p) => JSON.stringify(key(p))).join(", ") + const script = [ + `lib <- ${JSON.stringify(lib)}`, + `.libPaths(c(lib, .libPaths()))`, + `install.packages(c(${names}), lib = lib, repos = ${JSON.stringify(REPO)}, quiet = TRUE)`, + // install.packages() signals failure with a warning, not a non-zero exit, + // so a missing package would otherwise look like success. + `missing <- setdiff(c(${names}), rownames(installed.packages(lib.loc = lib)))`, + `if (length(missing)) { cat("FAILED:", paste(missing, collapse = ", "), "\\n"); quit(status = 1) }`, + ].join("\n") + const spec = await confined(input.directory, ["Rscript", "-e", script]) + const proc = Bun.spawn([spec.file, ...(spec.args ?? [])], { + env: { ...process.env, ...spec.env, R_LIBS_USER: lib }, + stdout: "pipe", + stderr: "pipe", + signal: input.signal, + }) + const [out, err] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]) + await proc.exited + return { ok: proc.exitCode === 0, log: [out, err].filter(Boolean).join("\n") } + } + + /** name → version for everything in the environment's library. */ + export async function freeze(directory: string) { + const lib = Installer.rlibrary(directory) + const script = [ + `ip <- installed.packages(lib.loc = ${JSON.stringify(lib)})`, + `if (nrow(ip)) cat(paste(rownames(ip), ip[, "Version"], sep = "\\t", collapse = "\\n"))`, + ].join("\n") + const proc = Bun.spawn(["Rscript", "-e", script], { + env: { ...process.env, R_LIBS_USER: lib }, + stdout: "pipe", + stderr: "pipe", + }) + const text = await new Response(proc.stdout).text() + await proc.exited + const out: Record = {} + for (const line of text.split("\n")) { + const [name, version] = line.split("\t") + if (name && version) out[name.trim()] = version.trim() + } + return out + } + + /** Every package an R kernel bound to this environment can load, system + * libraries included — the same distinction `Installer.resolved` draws for + * Python, and needed for the same reason: the restart decision is about what + * the kernel sees, not what the environment owns. */ + export async function resolved(directory: string) { + const lib = Installer.rlibrary(directory) + const script = [ + `ip <- installed.packages()`, + `if (nrow(ip)) cat(paste(rownames(ip), ip[, "Version"], sep = "\t", collapse = "\n"))`, + ].join("\n") + const proc = Bun.spawn(["Rscript", "-e", script], { + env: { ...process.env, R_LIBS_USER: lib }, + stdout: "pipe", + stderr: "pipe", + }) + const text = await new Response(proc.stdout).text() + await proc.exited + const out: Record = {} + for (const line of text.split("\n")) { + const [name, version] = line.split("\t") + if (name && version) out[name.trim()] = version.trim() + } + return out + } + + export async function verify(directory: string, packages: string[]) { + const frozen = await freeze(directory) + const out: Record = {} + for (const name of packages) { + const version = frozen[key(name)] + if (version) out[name] = version + } + return out + } + + /** CRAN's failure text, reduced to the actionable line. Unlike pip there is + * no wheels-only concept, so the two surfaces are "no such package" and a + * compilation failure naming a system header. */ + export function explain(log: string) { + const missing = log.match(/^FAILED:\s*(.+)$/m) + const unavailable = log.match(/package ['‘]([^'’]+)['’] is not available/) + if (unavailable) { + return `CRAN has no package named ${unavailable[1]} for this R version. Check the spelling, or whether it lives on Bioconductor rather than CRAN.` + } + const fatal = log.match(/^\s*fatal error:\s*(.+)$/m) + if (fatal) { + return `An R package failed to compile: ${fatal[1]!.trim()} A sandboxed install cannot add system libraries — prefer a package that ships a binary, or install the system dependency outside OpenScience.` + } + if (missing) return `These packages did not install: ${missing[1]!.trim()}\n${log.trim()}` + return log.trim() + } +} diff --git a/backend/cli/src/package/installer.ts b/backend/cli/src/package/installer.ts new file mode 100644 index 00000000..5ebbbcc5 --- /dev/null +++ b/backend/cli/src/package/installer.ts @@ -0,0 +1,1026 @@ +import fs from "fs/promises" +import { realpathSync } from "fs" +import os from "os" +import path from "path" +import { Config } from "../config/config" +import { Global } from "../global" +import { EgressRuntime } from "../sandbox/egress-runtime" +import { Sandbox } from "../sandbox/sandbox" + +/** + * The installer ladder and the sandboxed run. + * + * The install runs in the SAME sandbox as the kernel, not a second more + * permissive one. Earlier drafts specified a separate network-enabled install + * sandbox because the kernel's was network-denied; the allowlist proxy removed + * that asymmetry, so the only differences are what is writable — the + * environment directory and a package cache inside it. + * + * uv is a fast path, never a requirement: `python3 -m venv` bootstraps pip + * offline from the interpreter's bundled `ensurepip` wheel, verified inside + * `--unshare-net` on a host whose `python3` has no pip at all. Never + * auto-download uv — probe, use if present, throw a remedy if not. House + * precedent: `compute/modal/volume.ts:112-116`. + */ +export namespace Installer { + export type Tool = { kind: "existing" | "uv" | "venv"; binary: string; report?: Report } + + const bindir = process.platform === "win32" ? "Scripts" : "bin" + const exe = process.platform === "win32" ? ".exe" : "" + + /** PEP 503 normalisation, matching `Requirement.parse`. Both sides of an + * additivity comparison have to agree or an upgrade looks like an addition. */ + const normalise = (value: string) => value.replace(/[-_.]+/g, "-").toLowerCase() + + /** The environment's own interpreter — what kernels bind to, and what every + * install and verification runs through. */ + export function interpreter(directory: string) { + return path.join(directory, bindir, `python${exe}`) + } + + /** + * The environment's R library directory — R's equivalent of the interpreter + * binding, since R has no per-environment binary to point at. Reached through + * `R_LIBS_USER`, which is already in the kernel env allowlist. + * + * Kept beside `interpreter` rather than in the R installer so both language + * backends derive their paths from one place; a kernel needs this before any + * R install has ever run. + */ + export function rlibrary(directory: string) { + return path.join(directory, "rlibs") + } + + /** + * The base interpreter a managed environment delegates to, per its own + * `pyvenv.cfg` — not a guess, the value venv itself wrote. + * + * A venv does not contain a complete Python. On Windows `Scripts\python.exe` + * is a REDIRECTOR that starts the interpreter named by `home`; on POSIX the + * binary is a symlink to it. Either way the base installation has to be + * reachable, and inside an AppContainer nothing is reachable unless its ACL + * says so. Without this the redirector reported `No Python at '...'` for an + * interpreter that was present and working the whole time. + */ + /** + * Everything about the base installation that must be readable, not just the + * directory `pyvenv.cfg` names. + * + * On POSIX `home` is `/bin` — the directory holding the interpreter — + * so granting it alone leaves `/lib` unreachable, and with it the + * entire standard library. A python-build-standalone interpreter then falls + * back to its baked-in build prefix and dies before it can report why: + * + * sys.path = ['/install/lib/python312.zip', '/install/lib/python3.12', ...] + * Fatal Python error: init_fs_encoding: failed to get the Python codec + * ModuleNotFoundError: No module named 'encodings' + * + * Invisible while bubblewrap mounted `--ro-bind / /`, because the whole tree + * was there whether or not anyone asked. On Windows `home` IS the prefix, so + * the parent is added only when the leaf is the POSIX `bin`. + */ + export async function baseRoots(directory: string) { + const home = await base(directory) + if (!home) return [] + return path.basename(home) === "bin" ? [home, path.dirname(home)] : [home] + } + + export async function base(directory: string) { + const cfg = Bun.file(path.join(directory, "pyvenv.cfg")) + if (!(await cfg.exists().catch(() => false))) return undefined + const text = await cfg.text().catch(() => "") + for (const line of text.split("\n")) { + const home = line.match(/^\s*home\s*=\s*(.+?)\s*$/)?.[1] + if (home) return home + } + return undefined + } + + /** + * A tool on PATH that actually runs. + * + * `Bun.which` alone is not enough, and Windows is where that bites. A default + * install has `python3.exe` and `python.exe` in `WindowsApps` as App + * Execution Aliases: zero-byte reparse points that open the Microsoft Store + * instead of an interpreter. `which` finds them, `python3 -m venv ` + * appears to do something, and the environment is then created without an + * interpreter inside it. Measured on a real Windows machine: every install + * failed with `Executable not found in $PATH` naming + * `...\envs\\default\Scripts\python.exe`, with nothing + * explaining why the environment was empty. + * + * `findPython` in the notebook tool has always verified with `--version`; + * this path had drifted from it. Same check, same reason. + */ + const which = (name: string) => { + const found = Bun.which(name) + if (!found) return undefined + try { + const proc = Bun.spawnSync([found, "--version"], { stdout: "ignore", stderr: "ignore" }) + return proc.exitCode === 0 ? found : undefined + } catch { + return undefined + } + } + + /** + * EVERY match for a bare name on PATH, in PATH order — not just the first. + * + * `Bun.which` answers once, so a single unusable early hit hides every valid + * interpreter behind it and the search ends there. Both Windows failures seen + * so far have this shape: a `WindowsApps` alias early on PATH, and an MSYS2 + * build ahead of a real python.org install. Rejecting a candidate has to mean + * "keep looking", not "give up". + */ + async function onPath(name: string) { + const out: string[] = [] + const seen = new Set() + for (const dir of (process.env["PATH"] ?? "").split(path.delimiter).filter(Boolean)) { + const full = path.join(dir, name) + const key = process.platform === "win32" ? full.toLowerCase() : full + if (seen.has(key)) continue + seen.add(key) + if (await Bun.file(full).exists()) out.push(full) + } + return out + } + + export type Report = { exe: string; version: number[]; platform: string; purelib: string; prefix: string } + + /** One round trip that answers everything worth knowing about a candidate. + * `sysconfig` is the authority on the layout it will produce, so we ask it + * rather than inferring the layout from `process.platform`. */ + const PROBE = + "import sys,sysconfig,json;print(json.dumps({" + + "'exe':sys.executable,'version':list(sys.version_info[:2])," + + "'platform':sysconfig.get_platform(),'purelib':sysconfig.get_paths()['purelib'],'prefix':sys.prefix}))" + + export async function inspect(binary: string): Promise { + // Every failure mode of a candidate must answer "not usable", never throw. + // spawn throws outright on a file that exists but is not executable, and + // PATH is full of those; JSON.parse throws on a candidate that runs but + // prints something else. Either would abort the whole search at the first + // bad entry rather than moving on to the next one. + try { + const proc = Bun.spawn([binary, "-c", PROBE], { stdout: "pipe", stderr: "ignore" }) + const out = await new Response(proc.stdout).text() + await proc.exited + if (proc.exitCode !== 0) return undefined + return JSON.parse(out.trim()) + } catch { + return undefined + } + } + + /** + * Why this interpreter cannot build an environment the rest of the module can + * use — or undefined if it can. + * + * The case this exists for, measured on a real Windows machine: PATH had no + * `python3.exe` until `C:\msys64\mingw64\bin`, so MSYS2's MinGW Python won, + * and MSYS2 is the one Windows-native build that patches `sysconfig` to the + * POSIX scheme. It created a perfectly valid environment at + * `/lib/python3.9/site-packages` with `/bin/python.exe`, while + * every other path in this module looks under `Scripts\`. `python -m venv` + * exited 0 and `ensurepip` genuinely ran, so nothing upstream could tell. + * + * Checking the scheme rather than only the vendor is what makes this general: + * it disqualifies Cygwin and any future cross-built oddity by the property + * that actually breaks us, before anything is written to disk. The vendor + * check runs first only because it produces the clearer sentence. + */ + /** + * Can the sandbox be given read access to this path? + * + * `icacls` can only change an ACL the caller owns, so on Windows an all-users + * install under `C:\` — owned by SYSTEM and Administrators — can never be made + * readable to an AppContainer without elevation, which this product does not + * ask for. A machine-wide Python is therefore unusable for a SANDBOXED run + * however healthy the interpreter itself is, and choosing it produces a + * grant failure several layers from the choice. Measured that way: + * `icacls C:\Python312` denied, then `No Python at ...` and exit 103. + * + * Ownership is not directly readable here, so this uses the proxy that + * actually decides it: paths under the user's profile are owned by the user. + */ + export function grantable(candidate: string) { + if (process.platform !== "win32") return true + const home = process.env["USERPROFILE"] ?? os.homedir() + return !!home && candidate.toLowerCase().startsWith(home.toLowerCase()) + } + + export function reject(report: Report): string | undefined { + if (process.platform !== "win32") return undefined + if (/[\\/](msys\d*|mingw\d*|clang\d*|cygwin\d*)[\\/]/i.test(report.exe)) + return `${report.exe} is an MSYS2/Cygwin build, which lays environments out with the POSIX scheme` + if (!report.platform.startsWith("win-")) + return `${report.exe} reports platform ${report.platform}, not a native win-* build` + if (!/[\\/]Lib[\\/]site-packages$/i.test(report.purelib)) + return `${report.exe} uses the POSIX layout (${report.purelib}) rather than Lib\\site-packages` + return undefined + } + + /** + * The first interpreter on PATH that passes `reject`, plus the reasons the + * ones before it did not — so a failure can say what it looked at. + * + * `python` before `python3` on Windows is deliberate and is the fix for the + * MSYS2 selection above. python.org ships `python.exe` and NO `python3.exe`, + * so on Windows `python3` resolves to either the Store alias or a POSIX + * flavoured distribution almost by definition. On every other platform the + * usual order holds, where `python` may still be Python 2. + * + * The `py` launcher is consulted first where it exists, being the authoritative + * registry of installed interpreters — but it is only a source of candidates, + * never a requirement: it was absent on the very machine this bug came from. + */ + export async function select() { + const rejected: string[] = [] + const ungrantable: string[] = [] + const names = process.platform === "win32" ? ["python.exe", "python3.exe"] : ["python3", "python"] + const candidates: string[] = [] + if (process.platform === "win32") candidates.push(...(await registered())) + for (const name of names) candidates.push(...(await onPath(name))) + const seen = new Set() + for (const candidate of candidates) { + const key = process.platform === "win32" ? candidate.toLowerCase() : candidate + if (seen.has(key)) continue + seen.add(key) + const report = await inspect(candidate) + if (!report) { + rejected.push(`${candidate} did not run (a Microsoft Store alias behaves this way)`) + continue + } + const why = reject(report) + if (why) { + rejected.push(why) + continue + } + // Prefer one the sandbox can actually be granted, but do not refuse the + // other outright: an unsandboxed run works fine with a machine-wide + // Python, and failing closed here would break users who never enable the + // sandbox at all. + if (!grantable(candidate)) { + ungrantable.push(candidate) + continue + } + return { binary: candidate, report, rejected } + } + // Nothing the sandbox could be granted. Fall back to one it cannot, so an + // unsandboxed run still works, and record why the sandbox will complain. + for (const candidate of ungrantable) { + const report = await inspect(candidate) + if (report) + return { + binary: candidate, + report, + rejected: [ + ...rejected, + `${candidate} is outside your user profile, so the sandbox cannot be granted read access to it`, + ], + } + } + return { binary: undefined, report: undefined, rejected } + } + + /** Interpreters the `py` launcher knows about. Absent launcher is normal. */ + async function registered() { + const py = Bun.which("py") + if (!py) return [] + const proc = Bun.spawn([py, "-0p"], { stdout: "pipe", stderr: "ignore" }) + const out = await new Response(proc.stdout).text() + await proc.exited + if (proc.exitCode !== 0) return [] + return out + .split("\n") + .map((line) => line.match(/(\S:\\.*python(?:w)?\.exe)/i)?.[1]) + .filter((found): found is string => Boolean(found)) + } + + /** + * Why Python environments cannot be provisioned here, or undefined if they can. + * + * Deliberately NOT called from `decide()` or `plan()`. Answering it means + * running candidate interpreters, which is far too expensive for a path every + * sandboxed command goes through. It is for the surfaces where a human is + * asking: `sandbox status`, the settings panel, and the agent's guidance. + * + * The case it exists for is Windows-specific and not obvious from any error + * the user would otherwise see. An AppContainer can only be granted access to + * paths its user OWNS, so a machine-wide Python -- `C:\Python312`, + * `C:\Program Files\Python` -- can never be read by a sandboxed process, no + * matter how healthy the interpreter is. Measured on a real machine: + * + * C:\Python312 SYSTEM:(F) Administrators:(F) Users:(RX) + * + * with no ALL APPLICATION PACKAGES entry to read it by, and only `RX` for the + * user, so no way to add one. `icacls` answers "Access is denied" and the run + * fails several layers later as `No Python at '...'`. + * + * Containment itself is unaffected, and saying otherwise would be worse than + * saying nothing: shell commands stay confined, writes stay blocked, egress + * stays bounded. Only Python environments and kernels are unavailable. + */ + /** Does uv have an interpreter under the user's profile? `uv python list` + * prints every interpreter it knows, managed and system alike, so the test is + * whether any line names a path we could actually be granted. */ + /** First line of a uv subcommand's stdout, or undefined if it failed. */ + async function ask(uv: string, ...argv: string[]) { + const proc = Bun.spawn([uv, ...argv], { stdout: "pipe", stderr: "ignore" }) + const out = await new Response(proc.stdout).text() + await proc.exited + if (proc.exitCode !== 0) return undefined + return out + } + + /** + * uv reports paths under the home directory RELATIVE to it — in `--output-format + * json` exactly as in the human table, which is the part that was assumed and + * is now measured. From a real Windows machine, uv 0.12.4: + * + * "path":"AppData\\Roaming\\uv\\python\\cpython-3.12.13-...\\python.exe" + * "path":"C:\\Python312\\python.exe" + * + * Absolute and home-relative in the same array. Resolving the relative ones + * against the process cwd is what made `grantable()` pass on a path that does + * not exist; rejecting them outright is what then hid every uv interpreter on + * the machine and told a user with working uv to go install uv. + */ + const absolute = (candidate: string) => + path.isAbsolute(candidate) ? candidate : path.resolve(process.env["USERPROFILE"] ?? os.homedir(), candidate) + + const under = (root: string, candidate: string) => { + const rel = path.relative(root, candidate) + return !!rel && !rel.startsWith("..") && !path.isAbsolute(rel) + } + + /** + * An interpreter uv MANAGES that we could actually be granted, as an absolute + * path — or undefined. + * + * Two filters, and both earned their place from a failed run. + * + * `uv python dir` gives the root uv installs into, and only candidates beneath + * it are accepted. `uv python list` reports discovered system interpreters and + * uv's own shims alongside the real installs, and both are traps here: a + * system Python is the ungrantable case this whole path exists to route + * around, and `~/.local/bin/python3.12.exe` is a uv TRAMPOLINE — a stub that + * re-execs the interpreter it points at. Pinning `--python` to the trampoline + * produced exactly "uv trampoline failed to spawn Python child process: + * permission denied (os error 5)". Nothing in the JSON marks either kind + * (`symlink` is null for the trampoline), so the install root is the only + * discriminator available. + * + * Then `grantable()`, which is the actual requirement: the sandbox can only be + * granted paths the user owns. + */ + /** Does this interpreter carry the AppContainer `mkdtemp` defect? Everything + * from CPython 3.12.4 onward does; see `managed()` for what that costs. */ + const affected = (entry: { version_parts?: { major: number; minor: number; patch: number } }) => { + const v = entry.version_parts + if (!v) return false + if (v.major !== 3) return v.major > 3 + if (v.minor !== 12) return v.minor > 12 + return v.patch >= 4 + } + + export async function managed(): Promise { + const uv = which("uv") + if (!uv) return undefined + try { + const dir = await ask(uv, "python", "dir") + const root = dir?.split("\n")[0]?.trim() + if (!root) return undefined + const out = await ask(uv, "python", "list", "--only-installed", "--output-format", "json") + if (!out) return undefined + const entries = JSON.parse(out) as Array<{ + path?: string + key?: string + symlink?: string | null + version_parts?: { major: number; minor: number; patch: number } + }> + const usable = entries + .filter((entry) => !!entry.path) + .map((entry) => ({ ...entry, path: absolute(entry.path!) })) + .filter((entry) => under(absolute(root), entry.path) && grantable(entry.path)) + // Unaffected interpreters first. CPython 3.12.4 changed + // `os.mkdir(mode=0o700)` so `tempfile.mkdtemp()` creates a directory whose + // DACL does not inherit, and an AppContainer process cannot then write into + // the directory it just made (python/cpython#134587, fixed upstream but + // unreleased). Every release from 3.12.4 onward carries it, so a sandboxed + // Windows environment must be built on something older or pip cannot unpack + // a wheel it has already downloaded — and neither can agent-authored code + // calling mkdtemp. + // + // Preference, not a filter: an affected interpreter is still returned when + // it is all there is, because an UNSANDBOXED run works fine on it. What + // must not happen is silently choosing one when a usable alternative is + // sitting right there. `blocked()` reports the remaining case. + const ordered = [...usable].sort((a, b) => Number(affected(a)) - Number(affected(b))) + for (const entry of ordered) if (await fs.stat(entry.path).catch(() => undefined)) return entry.path + return undefined + } catch { + return undefined + } + } + + /** The managed interpreter that would be chosen, and whether it carries the + * mkdtemp defect. Separate from `managed()` so the common path stays a plain + * string and only the prerequisite check pays for the extra detail. */ + export async function managedDetail(): Promise<{ path: string; affected: boolean } | undefined> { + const chosen = await managed() + if (!chosen) return undefined + const uv = which("uv") + if (!uv) return { path: chosen, affected: false } + try { + const out = await ask(uv, "python", "list", "--only-installed", "--output-format", "json") + if (!out) return { path: chosen, affected: false } + const entries = JSON.parse(out) as Array<{ + path?: string + version_parts?: { major: number; minor: number; patch: number } + }> + const match = entries.find((entry) => entry.path && absolute(entry.path) === chosen) + return { path: chosen, affected: match ? affected(match) : false } + } catch { + return { path: chosen, affected: false } + } + } + + export async function blocked(): Promise { + if (process.platform !== "win32") return undefined + if (!Sandbox.available()) return undefined + // uv being installed is not the same as uv having an interpreter we can + // use: `uv venv` builds from whatever uv DISCOVERS, which on a machine like + // this one can be the same machine-wide Python. Only a managed interpreter + // under the user's profile actually helps, so ask for one rather than + // treating uv's presence as an all-clear — which is precisely the false + // reassurance this check gave when uv was installed and nothing improved. + const detail = which("uv") ? await managedDetail() : undefined + if (detail && !detail.affected) return undefined + if (detail?.affected) { + // A usable interpreter exists, so nothing is "blocked" in the strict + // sense — but an install WILL fail on it, several layers from the cause, + // and the remedy is one command. Say it here rather than let pip report a + // permission error on a wheel it already downloaded. + return [ + `Python environments will fail to install under the sandbox: ${detail.path} is CPython 3.12.4 or newer.`, + "Those releases create temp directories an AppContainer cannot write into (python/cpython#134587),", + "so pip downloads a wheel and then cannot unpack it. Fixed upstream, not yet released.", + "", + "Fix:", + " uv python install 3.12.3", + "", + "Containment is unaffected - shell commands are still confined.", + ].join("\n") + } + const chosen = await select() + if (chosen.binary && grantable(chosen.binary)) return undefined + return [ + chosen.binary + ? `Python environments are unavailable: the sandbox cannot be granted access to ${chosen.binary}.` + : "Python environments are unavailable: no interpreter was found.", + "Windows only lets you grant access to paths you own, so a machine-wide Python cannot be used by a", + "sandboxed process. Containment is unaffected - shell commands are still confined.", + "", + "Fix either way:", + " winget install --id=astral-sh.uv then uv python install 3.12", + " or reinstall Python from python.org with 'Install for all users' left OFF", + ].join("\n") + } + + /** + * The ladder, in order: an existing environment wins over any tool, then uv, + * then venv, then a remedy. + * + * `available` exists so the uv/venv branches are testable on a machine that + * has only one of them; real callers omit it and get a live probe. + */ + export async function probe(directory: string, available?: { uv?: string; python?: string }): Promise { + const existing = await Bun.file(interpreter(directory)) + .exists() + .catch(() => false) + if (existing) { + // An environment pins its base interpreter in pyvenv.cfg at creation, and + // nothing re-selects afterwards. So one built when the only candidate was + // a machine-wide Python stays bound to it forever — and under the sandbox + // that is fatal, because the base has to be granted and an all-users + // install can never be. Installing uv afterwards changes nothing: the + // environment still names the old base. + // + // Measured exactly that way. After uv and a user-owned 3.12.13 were + // installed, every retry still failed with "could not grant sandbox + // access to C:\Python312" — the path in the existing pyvenv.cfg, not + // anything selection would choose now. + // + // Falling through rebuilds it: create() clears a directory that already + // has a pyvenv.cfg, so the next tool builds a fresh environment on a base + // that works. + const home = await base(directory) + if (!home || grantable(home) || !Sandbox.available()) return { kind: "existing", binary: interpreter(directory) } + } + + const uv = available ? available.uv : (which("uv") ?? undefined) + if (uv) return { kind: "uv", binary: uv } + + if (available?.python) return { kind: "venv", binary: available.python } + const chosen = available ? { binary: undefined, report: undefined, rejected: [] } : await select() + if (chosen.binary) return { kind: "venv", binary: chosen.binary, report: chosen.report } + + throw new Error( + [ + "No way to create a Python environment on this machine.", + // What was looked at and why each one lost. Without this the message is + // indistinguishable on a machine with no Python at all and on one whose + // only Python is disqualified — two problems with different remedies. + ...(chosen.rejected.length + ? [ + "Interpreters were found, but none can build a usable environment:", + ...chosen.rejected.map((r) => ` - ${r}`), + ] + : []), + "Install one of:", + " - the venv module: `apt install python3-venv` on Debian/Ubuntu (most other distributions ship it with python3)", + " - uv: https://docs.astral.sh/uv/getting-started/installation/", + "OpenScience never downloads either automatically.", + ].join("\n"), + ) + } + + /** + * Create the environment. A no-op when it already exists — rebuilding would + * silently discard everything installed into it. + * + * `--seed` on the uv branch is load-bearing, not a nicety. `python3 -m venv` + * bootstraps pip from `ensurepip`; `uv venv` deliberately does not, and + * `install()` shells out to `python -m pip` regardless of who created the + * environment. Without it the uv branch produces an environment the + * installer cannot use at all — measured as `No module named pip` from a + * venv that looked perfectly healthy from outside the sandbox. + * + * Seeding rather than adding a second `uv pip install` path keeps one + * install code path to test and maintain, and leaves the environment usable + * by hand. The cost is a few hundred milliseconds at creation only. + */ + export async function create(directory: string, tool: Tool) { + if (tool.kind === "existing") return + // A tree left behind by a failed creation poisons every retry after it. + // `probe` only calls an environment "existing" when the interpreter is where + // this module expects it, so a half-built tree falls through to here — and + // both `venv` and `uv` then short-circuit on the directory already being + // there, report success, and replace nothing. Measured on Windows: after the + // first bad creation, every retry printed "Requirement already satisfied" + // for pip and setuptools and failed identically, with no way out but + // deleting the directory by hand. Reaching this line at all means the tree + // is unusable, so clear it. + await fs.rm(directory, { recursive: true, force: true }).catch(() => {}) + await fs.mkdir(path.dirname(directory), { recursive: true }) + // `--system-site-packages` is not a convenience, it repairs a cliff. + // + // A kernel binds to the managed environment as soon as one exists, and + // falls back to the host interpreter while it does not. So without this, + // the FIRST install of anything silently removed every host package from + // every kernel in the project: install `tqdm`, lose `numpy`. Measured in + // real use — the notebook tool advertises numpy/pandas/scipy/matplotlib as + // pre-imported, and they vanished the moment an environment appeared. + // + // Inheriting is strictly a superset of the behaviour kernels had before + // managed environments existed, when they simply WERE the host + // interpreter, so it exposes nothing new: host site-packages was already + // readable under `--ro-bind / /`. The environment's own packages still take + // precedence, so installing a newer version shadows the host's. + // + // The cost is that the environment is not hermetic. A hermetic mode is a + // reasonable future flag; it is the wrong default for a tool whose users + // expect the scientific stack to be there. + // Name the interpreter when the default would not do. `uv venv` builds from + // whatever uv DISCOVERS, and on a machine whose PATH leads with a + // machine-wide Python that is the one it picks — so a rebuild triggered + // precisely BECAUSE the old base was ungrantable would land straight back on + // it, fail identically, and rebuild again on the next attempt. + // win32 ONLY. Grantability is a Windows question -- bubblewrap and seatbelt + // read any path the user can read -- so pinning elsewhere would override + // uv's own choice of interpreter for no reason. It did, briefly: four Linux + // tests changed which Python they built against, because grantable() is + // unconditionally true off Windows and every uv-listed path therefore + // "qualified". + const pinned = + process.platform === "win32" && tool.kind === "uv" && Sandbox.available() ? await managed() : undefined + const argv = + tool.kind === "uv" + ? [tool.binary, "venv", "--seed", "--system-site-packages", ...(pinned ? ["--python", pinned] : []), directory] + : [tool.binary, "-m", "venv", "--system-site-packages", directory] + const proc = Bun.spawn(argv, { stdout: "pipe", stderr: "pipe" }) + const [out, err] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]) + await proc.exited + if (proc.exitCode !== 0) throw new Error(`Could not create the environment at ${directory}.\n${err || out}`) + // Exit code 0 is not proof. A Windows App Execution Alias standing in for + // python exits cleanly having created nothing, and the failure then + // surfaced much later as "Executable not found in $PATH" from the install + // step, naming a path with no hint as to why it was missing. Assert the + // thing the rest of this module depends on, at the moment it should exist. + // + // Report what was MEASURED, never a guess at the cause. The previous + // version of this message asserted that Windows failures "usually mean" a + // Microsoft Store alias. On the machine that produced the next failure that + // claim was false — a real CPython had run and `ensurepip` had completed — + // and it cost a full debugging cycle, because the message read as a finding + // rather than as a hypothesis. Everything below is something we looked at. + const found = await locate(directory) + const check = found ? await inspect(found) : undefined + const rooted = check ? same(check.prefix, directory) : false + if (!found || !same(found, interpreter(directory)) || !rooted) { + const listing = await fs.readdir(directory).catch(() => [] as string[]) + // Leave nothing behind for the next run to short-circuit on. + await fs.rm(directory, { recursive: true, force: true }).catch(() => {}) + throw new Error( + [ + `Creating the environment at ${directory} reported success, but it has no usable interpreter at ${interpreter(directory)}.`, + ` created with: ${tool.binary} (exit code ${proc.exitCode})`, + tool.report ? ` which reports: platform ${tool.report.platform}, purelib ${tool.report.purelib}` : undefined, + found ? ` an interpreter was found instead at: ${found}` : " no interpreter was found anywhere in the tree", + found && check && !rooted ? ` and it reports sys.prefix ${check.prefix}, not ${directory}` : undefined, + found && !check ? " and it did not run" : undefined, + listing.length ? ` the tree contains: ${listing.join(", ")}` : " the tree is empty", + (err || out).trim(), + ] + .filter(Boolean) + .join("\n"), + ) + } + } + + /** + * Compare two paths as the host filesystem would — following symlinks. + * + * `path.resolve` alone is not enough, and macOS is where that bites. The + * system temp directory is `/var/folders/...`, and `/var` is a firmlink to + * `/private/var`, so Python reports `sys.prefix` under `/private/var` while + * the caller holds the `/var` spelling. The two are the same directory and + * compared unequal, so EVERY environment creation on macOS was judged to have + * produced no usable interpreter, deleted itself, and threw — taking the + * merge-gate tests with it. The sandbox already carries `withPrivateAliases` + * for this exact firmlink; this is the same hazard in a second place. + */ + const same = (a: string, b: string) => { + const real = (value: string) => { + const resolved = path.resolve(value) + try { + return realpathSync(resolved) + } catch { + return resolved + } + } + const [x, y] = [real(a), real(b)] + return process.platform === "win32" ? x.toLowerCase() === y.toLowerCase() : x === y + } + + /** + * The interpreter a creation actually produced, searched across both layouts + * rather than assumed at one path. + * + * `interpreter()` names where this module REQUIRES the interpreter to be; + * this finds where it IS. The two differing is the whole diagnosis in the + * POSIX-layout case, so the error can only say so if it looks in both places. + */ + export async function locate(directory: string) { + for (const dir of ["Scripts", "bin"]) + for (const name of ["python.exe", "python3.exe", "python", "python3"]) { + const full = path.join(directory, dir, name) + if (await Bun.file(full).exists()) return full + } + return undefined + } + + /** + * The wheel cache, shared by every environment on the machine. + * + * Deliberately NOT inside the environment directory, which is where it lived + * first. A per-environment cache means every new environment re-downloads + * everything: measured at 34 MB and a full download for scipy alone, in a + * second environment that had just been populated in the first — and the + * packages that make this hurt are the large ones, where it is hundreds of + * megabytes per environment. + * + * It is our own cache directory rather than user data, so sharing it across + * projects costs nothing in isolation terms. pip's cache is content-addressed + * and safe for concurrent readers and writers, which matters because the + * per-environment lock does not serialise installs into DIFFERENT + * environments. + */ + const shared = () => path.join(Global.Path.cache, "pip") + + /** Sandboxed argv for a command run against the environment: the same policy + * the kernel gets, plus write access to the environment directory and the + * shared wheel cache. */ + async function confined(directory: string, argv: string[], extraReadable: string[] = []) { + const policy = await Config.trustedSandbox() + const egress = await EgressRuntime.egressFor(policy) + // The base interpreter is READ-only on purpose: the install must be able to + // start Python, not to modify the Python installation it runs on. Stated + // unconditionally — which backend needs telling is the sandbox's business, + // not the installer's. + const roots = [...(await baseRoots(directory)), ...extraReadable] + return Sandbox.wrapArgv({ + file: argv[0]!, + args: argv.slice(1), + workspace: [directory, shared()], + ...(roots.length ? { readable: roots } : {}), + options: { ...policy, egress }, + }) + } + + /** + * The most recent line of pip output worth showing a human. + * + * pip reports phase and size continuously — "Collecting torch", "Downloading + * torch-…whl (906.4 MB)", "Installing collected packages: …" — and all of it + * used to be buffered and discarded unless the install failed. A pytorch + * install sat behind an unchanging ellipsis for 1m37s while that ran. + * + * Progress-bar redraws and continuation lines are skipped: they are noise at + * one line of visible status, and a bar rendered to a pipe is mostly control + * characters anyway. + */ + const progressLine = (chunk: string) => { + const lines = chunk + .split(/\r?\n/) + .map((l) => l.trim()) + .filter((l) => l && !/^[━╸\-=|/\\ ]*$/.test(l) && !l.startsWith("|")) + return lines.at(-1) + } + + export async function install(input: { + directory: string + packages: string[] + index: string + source: boolean + signal?: AbortSignal + /** Called with a short status as pip reports it. */ + onProgress?: (status: string) => void + }) { + // Two different directories with two different lifetimes. The wheel cache is + // shared across environments so a package is downloaded once per machine; + // the scratch directory pip unpacks into stays environment-local, because it + // is throwaway and sharing it would let concurrent installs collide. + const cache = shared() + + await fs.mkdir(cache, { recursive: true }) + // Wheels-only is a speed and reliability default, NOT a security boundary: + // if bwrap contains agent Python at import time it contains setup.py at + // install time. + const policy = input.source ? [] : ["--only-binary", ":all:"] + /** + * uv on Windows, pip everywhere else. + * + * Not a preference — a workaround for an upstream defect with a name. + * CPython 3.12.4 changed `os.mkdir(mode=0o700)` so `tempfile.mkdtemp()` + * creates a directory whose DACL does NOT inherit from its parent. Inside an + * AppContainer, access comes from the package SID; a DACL naming only the + * owner grants the process nothing. So pip downloads a wheel and then cannot + * write into the `pip-unpack-*` directory it just made: + * + * [Errno 13] Permission denied: '...\pip-unpack-xxxx\six-1.17.0-...whl.metadata' + * + * See python/cpython#134587. Two fixes are open upstream and unreleased. + * uv is Rust and never calls CPython's tempfile, so it is unaffected. + * + * Windows ONLY, deliberately. Linux and macOS install correctly today and + * are verified doing so on every push; switching their installer inside a + * fix for someone else's Windows bug would change working, security-relevant + * code for no reason. uv and pip can also resolve the same requirement to + * different closures, and the environment manifest records what landed, so + * the difference is captured rather than hidden. + * + * uv is not optional on Windows anyway: it is how a grantable interpreter + * gets there at all, which `blocked()` already tells the user. + */ + // pip on every platform, including Windows. + // + // uv was routed in here only to dodge python/cpython#134587 — + // `tempfile.mkdtemp()` creating a directory whose DACL does not inherit, + // which an AppContainer process cannot then write into. Pinning managed + // Windows environments to CPython 3.12.3 removes that bug at the source, so + // the reason for uv goes with it. uv still CREATES environments; it just no + // longer installs into them. + // + // It also fixes what uv never could: agent-authored notebook code calling + // `tempfile.mkdtemp()` inside the sandbox, which is broken on any + // interpreter from 3.12.4 onward. + // + // And it retires a wall five CI rounds could not get past — uv, running + // inside the container, fails to spawn the very interpreter that `cmd.exe` + // spawns successfully from inside the same container two tests earlier: + // + // Discovery(Query(Io(PermissionDenied), "...\\env\\Scripts\\python.exe", ProvidedPath)) + // + // That is still unexplained. It is no longer on the path. + const argv = [ + interpreter(input.directory), + "-m", + "pip", + "install", + "--disable-pip-version-check", + ...policy, + ...(input.index ? ["--index-url", input.index] : []), + ...input.packages, + ] + const spec = await confined(input.directory, argv) + const proc = Bun.spawn([spec.file, ...(spec.args ?? [])], { + // No TMPDIR of our own. It used to be `/.tmp`, pre-created here on the + // host — and on Windows the sandbox cannot make a pre-existing + // SUBDIRECTORY writable: `icacls /setintegritylevel (OI)(CI)L` labels the + // granted root and does not propagate to children that already exist, and + // Mandatory Integrity Control is evaluated before the DACL. Measured + // directly: a directory created before the launch is unwritable inside the + // container while one created inside it is fine. pip then failed unpacking + // a wheel it had already downloaded, which read as a transport fault. + // + // `spec.env` carries TMPDIR/TMP/TEMP for the sandbox's own per-spawn temp + // root, which IS granted and labelled as a root. Letting that through is + // both the fix and the reason the mechanism exists. + env: { + ...process.env, + ...spec.env, + PIP_CACHE_DIR: cache, + }, + // The environment, not wherever the server happens to be running. A + // sandboxed child inherits this working directory, and the sandbox is not + // granted the server's cwd — the same latent fault that killed the egress + // shim outright, where the process was created and then died on its first + // syscall. pip needs no particular cwd (its arguments are package names), + // so the only requirement is that the child can reach it. + cwd: input.directory, + stdout: "pipe", + stderr: "pipe", + signal: input.signal, + }) + // Drained as it arrives rather than awaited whole, so a caller can report + // progress. The full text is still accumulated: `explain()` needs the + // entire log to find the `fatal error:` line, which is rarely last. + const drain = async (stream: ReadableStream, report: boolean, into: { out: string; err: string }) => { + const reader = stream.getReader() + const decoder = new TextDecoder() + while (true) { + const { done, value } = await reader.read() + if (done) break + const piece = decoder.decode(value, { stream: true }) + // Accumulated where the caller can still read it if the race below + // gives up: a partial log explains a hang, a discarded one does not. + if (report) into.out += piece + else into.err += piece + if (!report || !input.onProgress) continue + const status = progressLine(piece) + if (status) input.onProgress(status) + } + } + // pip writes its progress to stdout and its diagnostics to stderr; only the + // former is worth surfacing as status. + const collected = { out: "", err: "" } + const finished = Promise.all([drain(proc.stdout, true, collected), drain(proc.stderr, false, collected)]) + // Wait on the PROCESS, then give the drains a bounded moment to flush. + // + // Awaiting the drains alone hangs forever on an abort. `signal` kills the + // launcher, but the sandboxed child inherits its stdio handles and is not a + // child of this process at all — on Windows it is created by CreateProcessW + // inside the container — so it keeps the write end of both pipes open and + // `reader.read()` never reports done. Measured: a 150s AbortSignal produced + // no return at all and the caller sat until its own 300s timeout, twice, + // each time discarding the output that would have explained the failure. + await proc.exited + await Promise.race([finished, Bun.sleep(2_000)]) + return { ok: proc.exitCode === 0, log: [collected.out, collected.err].filter(Boolean).join("\n") } + } + + /** name → version for everything resolved into the environment, names PEP 503 + * normalised so they compare against parsed requirements. */ + export async function freeze(directory: string) { + // `--local` matters now that environments inherit system site-packages: + // without it this reports every host package too, which would make `total` + // meaningless, bury the requested names in the agent's inventory, and turn + // `additive()` into a comparison against the machine rather than against + // the environment. What this environment OWNS is the question being asked. + const proc = Bun.spawn([interpreter(directory), "-m", "pip", "list", "--local", "--format=json"], { + stdout: "pipe", + stderr: "pipe", + }) + const text = await new Response(proc.stdout).text() + await proc.exited + const parsed = (() => { + try { + return JSON.parse(text) as { name: string; version: string }[] + } catch { + return [] + } + })() + return Object.fromEntries(parsed.map((p) => [normalise(p.name), p.version])) + } + + /** + * Every package the environment's interpreter can import, inherited ones + * included — what a KERNEL bound to this environment actually sees. + * + * Distinct from `freeze()` on purpose, and the distinction is load-bearing. + * `freeze()` answers "what does this environment own", which is the right + * question for the manifest. The restart decision asks something else: has + * what the kernel can import changed underneath it? Comparing owned-sets got + * that wrong the moment environments began inheriting system site-packages — + * requesting the version the host already provides installs nothing locally, + * so the package is absent from the "before" snapshot, and the next version + * then reads as an ADDITION rather than a change. Measured on CI: + * `six==1.16.0` then `six==1.17.0` reported additive, so kernels holding a + * stale `six` in memory were never restarted — exactly the silent staleness + * the rule exists to prevent. + */ + export async function resolved(directory: string) { + const proc = Bun.spawn([interpreter(directory), "-m", "pip", "list", "--format=json"], { + stdout: "pipe", + stderr: "pipe", + }) + const text = await new Response(proc.stdout).text() + await proc.exited + const parsed = (() => { + try { + return JSON.parse(text) as { name: string; version: string }[] + } catch { + return [] + } + })() + return Object.fromEntries(parsed.map((p) => [normalise(p.name), p.version])) + } + + /** + * Report the version of each requested name **as the environment's own + * interpreter resolves it**, whether it lives in the environment or is + * inherited from the host. + * + * Asked of the interpreter rather than of `freeze()`, which lists only what + * the environment owns. Since environments inherit system site-packages, pip + * treats a host-provided package as already satisfied and installs nothing — + * so a `freeze`-based answer reported "(nothing reported)" for a request that + * is, from the user's seat, perfectly satisfied. The question worth answering + * is "can the kernel use it, and at what version", and only the interpreter + * can answer that. + * + * `importlib.metadata` rather than a real import: it reads distribution + * metadata, so it needs no heavy import, triggers no import side effects, and + * handles name normalisation itself. It still catches an installer that + * exited 0 without producing anything usable, which is the point. + */ + export async function verify(directory: string, packages: string[]) { + const script = [ + "import json, sys", + "from importlib.metadata import version, PackageNotFoundError", + "out = {}", + "for name in json.loads(sys.argv[1]):", + " try:", + " out[name] = version(name)", + " except PackageNotFoundError:", + " pass", + "print(json.dumps(out))", + ].join("\n") + const proc = Bun.spawn([interpreter(directory), "-c", script, JSON.stringify(packages)], { + stdout: "pipe", + stderr: "pipe", + }) + const text = await new Response(proc.stdout).text() + await proc.exited + try { + return JSON.parse(text) as Record + } catch { + return {} + } + } + + /** + * Turn a pip log into something a reader can act on. + * + * Two surfaces matter. The wheels-only rejection reads as "no such package" + * and means "no wheel under this policy". A build failure's summary line + * names the package, but the `fatal error:` line above it names the missing + * system header — which usually means the install is unachievable in a + * sandbox and a pure-Python alternative is the real answer. + * + * An unrecognised log passes through untouched. Inventing a diagnosis for a + * failure mode nobody anticipated is worse than showing the log. + */ + export function explain(log: string) { + const wheels = log.match(/Could not find a version that satisfies the requirement (\S+)[^\n]*from versions: none/) + if (wheels) { + return [ + `No wheel is published for ${wheels[1]} under the current wheels-only policy.`, + `This is not "no such package" — it may exist only as a source distribution.`, + `Retry with source builds enabled if a compiler and headers are available.`, + ].join(" ") + } + const fatal = log.match(/^\s*fatal error:\s*(.+)$/m) + const failed = log.match(/Failed building wheel for (\S+)/) + if (fatal) { + return [ + failed ? `Building ${failed[1]} failed.` : "A wheel build failed.", + `The cause is a missing system dependency: ${fatal[1]!.trim()}`, + `A sandboxed install cannot add system packages — prefer a pure-Python alternative, or a package that publishes wheels.`, + ].join(" ") + } + return log.trim() + } +} diff --git a/backend/cli/src/package/prompt.ts b/backend/cli/src/package/prompt.ts new file mode 100644 index 00000000..9044d4e3 --- /dev/null +++ b/backend/cli/src/package/prompt.ts @@ -0,0 +1,157 @@ +import z from "zod" +// Aliased: this namespace exports its own `Environment` (the render schema), +// which would shadow the store inside every function body here. +import { Environment as Store } from "./environment" + +/** + * Capability contract for governed package installation. + * + * Modelled on `compute/prompt.ts`. The load-bearing mechanism there is not the + * skill override — it is `SystemPrompt.compute()`, injected unconditionally at + * `session/prompt.ts:863` on every request for every agent. That is what makes + * a contract hold across 293 skills, their reference files, and third-party + * skills cloned from GitHub that this repo cannot edit: it pre-empts rather + * than corrects, and it names the specific wrong commands rather than + * gesturing at a policy. + * + * Wire the same way — add `...(await SystemPrompt.packages())` to the system + * array. A skill-level override is deliberately NOT provided: it only reaches + * the front page of a skill, never its references, and the block below already + * covers what such an override would say. + */ +export namespace PackagePrompt { + export const Environment = z.object({ + name: z.string(), + language: z.enum(["python", "r"]), + /** + * Only what was explicitly asked for, never the resolved closure. + * + * A real environment listing is dominated by transitive and native + * dependencies — a reference implementation's shared env reports 168 + * entries, most of them `libgcc`, `harfbuzz`, `xorg-libx11`, `qt6-main`, + * with the importable Python packages a minority. Rendering that into every + * request would bury the contract in font libraries and teach the agent + * nothing it can act on. + */ + requested: z.array(z.string()).default([]), + /** Size of the resolved closure, reported as a number rather than listed. */ + total: z.number().int().nonnegative().optional(), + /** Set while an install holds this env's lock. */ + busy: z.boolean().default(false), + }) + export type Environment = z.infer + + const Stored = z + .object({ + environments: z.array(Environment).default([]), + /** Outcomes of installs that finished without anyone watching. */ + warnings: z.array(z.string()).default([]), + }) + .passthrough() + + const inventory = (values: Environment[]) => { + if (!values.length) { + return ["No environments exist yet. The first install creates one; you do not create it separately."] + } + return values.map((env) => { + const held = env.requested.length ? env.requested.toSorted().join(", ") : "(empty)" + const rest = env.total && env.total > env.requested.length ? ` (+${env.total - env.requested.length} deps)` : "" + const lock = env.busy ? " [INSTALL IN PROGRESS — do not execute in this environment until it finishes]" : "" + return `- ${env.name} (${env.language}): ${held}${rest}${lock}` + }) + } + + export function render(value: unknown) { + const parsed = Stored.safeParse(value) + const envs = parsed.success ? parsed.data.environments : [] + + const warnings = parsed.success ? parsed.data.warnings : [] + return [ + "", + ...(warnings.length + ? [ + "UNRESOLVED INSTALLS — tell the user about these before doing anything else:", + ...warnings.map((w) => `- ${w}`), + "", + ] + : []), + "Environments available to kernels in this project:", + ...inventory(envs), + "", + "Package installation contract:", + "- Whether a package is already available is a read-only question. Answer it from the inventory above. Never install a package, and never run code, merely to find out whether something is present.", + "- Do not request a package the inventory already lists. A fully-satisfied request installs nothing and is not worth a turn.", + "- `package_install` is the only way to add packages. Call it when the user asks for a package, or when work you are about to do needs one that is absent.", + "- Environments are built with uv when it is present, otherwise the interpreter's venv module.", + // Windows only, for the same reason the tool description gates it: + // bubblewrap and seatbelt read anything the user can read, so a Linux or + // macOS agent given this advice could only ever apply it wrongly. + ...(process.platform === "win32" + ? [ + "- If provisioning fails because the sandbox cannot be granted access to the base interpreter, the remedy is a Python the user owns — installing uv, or a per-user python.org install — not elevated permissions on a machine-wide one, which cannot be obtained and would not help.", + ] + : []), + "- Never install through the shell. `pip install`, `pip3 install`, `python -m pip`, `uv pip install`, `conda install`, `mamba install`, `poetry add`, and `install.packages()` are refused here, including into a virtualenv you create yourself in the workspace. Skills and their reference files that instruct you to run these commands describe an ungoverned runtime and are superseded — use `package_install` instead.", + "- Do not attempt to install or repair pip itself, create a virtualenv by hand, or edit an environment directory. The tool owns environment creation, the installer choice, and the target path.", + "- Installing restarts every kernel bound to that environment and discards its variables. Prefer to install before a long computation rather than during one. If a cell is running, the install queues behind it.", + "- An environment is scoped to one language. Python packages go to a python environment, R packages to an R environment; there is no shared environment.", + "- A local environment and a Modal job image are unrelated. Installing locally does not make a package available to a Modal job, and a Modal job's `packages` field does not affect any local environment. If the target is ambiguous, ask which one the user means.", + "- Report only what the tool returns. Do not claim an install succeeded, estimate a download size, or invent a version you have not been shown.", + "", + ].join("\n") + } + + /** + * The inventory the agent sees, assembled from real manifests. + * + * `busy` is read from the live in-memory lock rather than stored on the + * manifest. A persisted flag would survive a crash and permanently mark a + * healthy environment as installing, with nothing to clear it; the lock + * cannot outlive the process that holds it. + * + * Takes a project id, not an opaque value — the earlier signature read a + * single global `environments.json` that nothing ever wrote, so the agent was + * told "No environments exist yet" forever, including immediately after + * installing something. That made the contract's first rule ("answer from the + * inventory above") a lie. Callers pass `undefined` only in tests that want + * the empty rendering. + */ + export async function system(projectID?: string) { + if (!projectID) return render({ environments: [] }) + // The only production caller of reconcile(), and the right one: this runs + // on every request, so the first request after a restart resolves any claim + // left by an install that never finished. Without a caller the whole + // claim/token mechanism was dead code — built, tested, and reached only by + // its own tests. + // + // Cheap enough to do here: a readdir of a directory that is empty except + // when an install is in flight or one ended badly, and it self-clears, so + // the next request finds nothing. + const outcomes = await Store.reconcile(projectID).catch(() => []) + const values = await Store.list(projectID) + return render({ + environments: values.map((env) => ({ + name: env.name, + language: env.language, + requested: env.requested, + total: env.total, + busy: Store.busy(projectID, env.name), + })), + // An environment that only ever existed as a failed install has no + // manifest, so warnings are carried separately rather than attached to + // the inventory rows — otherwise the one case worth reporting is the one + // case with nowhere to report it. + warnings: outcomes.flatMap((o) => { + if (o.outcome === "failed") { + return [`Install into ${o.name} FAILED and nothing was landed: ${o.message ?? "no detail recorded"}`] + } + if (o.outcome === "unknown") { + return [ + `An install into ${o.name} was interrupted and its outcome is unknown. The environment may be incomplete — verify before relying on it, and re-install if in doubt.`, + ] + } + return [] + }), + }) + } +} diff --git a/backend/cli/src/package/refuse.ts b/backend/cli/src/package/refuse.ts new file mode 100644 index 00000000..d9a218ae --- /dev/null +++ b/backend/cli/src/package/refuse.ts @@ -0,0 +1,102 @@ +/** + * Shell-side refusal of package installers. + * + * A contract boundary, not a security boundary. The same allowlisted egress + * that lets `package_install` reach pypi lets a determined agent fetch a wheel + * by hand, and nothing here stops that. What this buys is that the *normal* + * path — every skill's `pip install` line, every reference file this repo + * cannot edit — arrives at the approval card instead of quietly succeeding. + * + * It became load-bearing only recently. Before the allowlist proxy, a shell + * `pip install` died at DNS, so the contract held by accident; measured on + * `feat/sandbox-network-policy`, `python3 -m venv /venv && + * /venv/bin/pip install tqdm` now succeeds, with no tool and no + * card. The proxy did not create the intent to bypass, it removed the accident + * that used to prevent it. + * + * Matching is over the tokenised argv `bash.ts` already builds from + * tree-sitter, not over the raw command line: `echo pip install numpy` is one + * command whose name is `echo`, and a regex over the line cannot tell the + * difference. + */ +export namespace Refuse { + /** Subcommands that mutate an environment. `list`, `show`, `--version` and + * friends are read-only questions and stay allowed — refusing them would + * break ordinary inspection and teach the agent the tool is unreliable. */ + const mutating = new Set(["install", "add", "uninstall", "remove"]) + + /** The last path segment, so `/work/venv/bin/pip` matches `pip`. */ + const leaf = (value: string) => value.split(/[/\\]/).pop() ?? value + + /** `python`, `python3`, `python3.14`, `/usr/bin/python3` — any of which can + * carry `-m pip`. */ + const python = (value: string) => /^python[0-9.]*$/.test(leaf(value)) + + const message = (packages: string[]) => + [ + "Refused: package installation goes through the `package_install` tool, not the shell.", + packages.length ? `Requested: ${packages.join(", ")}.` : "", + "Call `package_install` instead — it asks the user for approval, installs into a managed environment, and reports the versions it landed.", + "This applies to a virtualenv you create yourself in the workspace as well: the environment is the tool's to own.", + ] + .filter(Boolean) + .join(" ") + + /** Operands that look like package names, for the refusal message. Flags and + * their values are dropped; so is `-r requirements.txt`. */ + const operands = (rest: string[]) => { + const values: string[] = [] + for (let i = 0; i < rest.length; i++) { + const arg = rest[i]! + if (arg === "-r" || arg === "--requirement" || arg === "-c" || arg === "--constraint") { + i++ + continue + } + if (arg.startsWith("-")) continue + values.push(arg) + } + return values + } + + /** + * A refusal message when `command` is a package-installer invocation, + * `undefined` otherwise. `command` is the tokenised argv of one command node. + */ + export function installer(command: string[]): string | undefined { + const head = command[0] + if (!head) return undefined + const name = leaf(head) + + // `python -m pip install ...` / `./venv/bin/python -m pip install ...` + if (python(head) && command[1] === "-m" && command[2] === "pip") { + if (!mutating.has(command[3] ?? "")) return undefined + return message(operands(command.slice(4))) + } + + // `uv pip install ...` + if (name === "uv" && command[1] === "pip") { + if (!mutating.has(command[2] ?? "")) return undefined + return message(operands(command.slice(3))) + } + + // `pip install ...`, `pip3 install ...`, `/work/venv/bin/pip install ...` + if (/^pip[0-9.]*$/.test(name)) { + if (!mutating.has(command[1] ?? "")) return undefined + return message(operands(command.slice(2))) + } + + // `conda install ...`, `mamba install ...` + if (name === "conda" || name === "mamba") { + if (!mutating.has(command[1] ?? "")) return undefined + return message(operands(command.slice(2))) + } + + // `poetry add ...` + if (name === "poetry") { + if (!mutating.has(command[1] ?? "")) return undefined + return message(operands(command.slice(2))) + } + + return undefined + } +} diff --git a/backend/cli/src/package/requirement.ts b/backend/cli/src/package/requirement.ts new file mode 100644 index 00000000..bbd4fea2 --- /dev/null +++ b/backend/cli/src/package/requirement.ts @@ -0,0 +1,115 @@ +/** + * A deliberate PEP 508 subset: name, extras, version specifiers, environment + * markers, and the `name @ url` direct-reference form. Markers are captured but + * never evaluated — nothing here needs to. + * + * The spec's requirement is "parse with a real parser", meaning specifically: + * do not split on `==`. A split mishandles `numpy>=2.4`, `pandas[performance]` + * and `tqdm; python_version >= "3.9"`, and a mis-parsed name becomes a wrong + * permission pattern — an approval for something other than what runs. So + * anything outside this grammar throws rather than being guessed at. + */ +export namespace Requirement { + export type Parsed = { + name: string + extras: string[] + specifier: string + marker: string + url: string + } + + /** + * PEP 503 normalisation: runs of `-`, `_` and `.` collapse to one `-`, and + * comparison is lowercase. `Foo_Bar`, `Foo.Bar` and `foo-bar` are one + * package. Treating them as three would let an upgrade look additive to + * `Environment.additive`, which decides whether live kernels restart. + */ + const normalise = (value: string) => value.replace(/[-_.]+/g, "-").toLowerCase() + + const NAME = /^[A-Za-z0-9]([A-Za-z0-9._-]*[A-Za-z0-9])?$/ + + /** + * One comparison clause, anchored end to end: an operator followed by a + * version that actually starts with an alphanumeric. + * + * Anchoring matters more than it looks. A prefix test like + * `/^(===|==|…|>|<)\s*\S/` accepts `numpy >= ` — the alternation backtracks + * to the single-character `>` and then happily consumes the `=` as the + * version. A dangling operator would then reach pip as a literal + * requirement, having passed validation. + */ + const CLAUSE = /^(===|==|!=|~=|>=|<=|>|<)\s*[A-Za-z0-9][A-Za-z0-9.*+!_-]*$/ + + /** Every comma-separated clause must be well formed — `>=2.1,<3` is two. */ + const valid = (specifier: string) => + specifier + .split(",") + .map((clause) => clause.trim()) + .every((clause) => CLAUSE.test(clause)) + + export function parse(value: string): Parsed { + const text = value.trim() + if (!text) throw new Error(`Not a package requirement: ${JSON.stringify(value)}`) + + const [head, ...rest] = text.split(";") + const marker = rest.join(";").trim() + const body = head!.trim() + if (!body) throw new Error(`Not a package requirement: ${JSON.stringify(value)}`) + + const at = body.indexOf("@") + if (at !== -1) { + const name = body.slice(0, at).trim() + const url = body.slice(at + 1).trim() + if (!NAME.test(name) || !url) throw new Error(`Not a package requirement: ${JSON.stringify(value)}`) + return { name: normalise(name), extras: [], specifier: "", marker, url } + } + + const match = body.match(/^([A-Za-z0-9][A-Za-z0-9._-]*)\s*(\[[^\]]*\])?\s*(.*)$/) + if (!match) throw new Error(`Not a package requirement: ${JSON.stringify(value)}`) + const [, raw, bracket, tail] = match + if (!raw || !NAME.test(raw)) throw new Error(`Not a package requirement: ${JSON.stringify(value)}`) + + const extras = bracket + ? bracket + .slice(1, -1) + .split(",") + .map((e) => e.trim()) + .filter(Boolean) + : [] + + const specifier = (tail ?? "").trim() + if (specifier && !valid(specifier)) throw new Error(`Not a package requirement: ${JSON.stringify(value)}`) + + return { name: normalise(raw), extras, specifier: specifier.replace(/\s+/g, ""), marker, url: "" } + } + + /** + * Strip credentials and scheme from an index URL. + * + * Credentials are environment config, not part of the approved action: + * rotating a token must not invalidate a standing grant, and a secret must + * never be rendered on a card the user is about to screenshot. + */ + export function redact(index: string) { + const trimmed = index.trim() + const withoutScheme = trimmed.replace(/^[a-zA-Z][a-zA-Z0-9+.-]*:\/\//, "") + const at = withoutScheme.lastIndexOf("@") + return (at === -1 ? withoutScheme : withoutScheme.slice(at + 1)).replace(/\/+$/, "") + } + + /** + * The canonical command string — both what the approval card shows and what + * the permission system matches. Readable on purpose, unlike a digest: change + * the environment, the packages or the index and it is a different string, so + * the prompt reappears for free. + * + * Names only, sorted. Sorted so the same set in a different argument order + * matches an existing grant instead of prompting again; names only because + * resolution happens after approval — the card shows the request, so pinning + * a version must not fragment a grant the user already gave. + */ + export function pattern(input: { packages: string[]; environment: string; index: string }) { + const names = input.packages.map((p) => parse(p).name).toSorted() + return `install ${names.join(" ")} → ${input.environment} [${input.index}]` + } +} diff --git a/backend/cli/src/project/execution.ts b/backend/cli/src/project/execution.ts index d36a4f57..2330f61d 100644 --- a/backend/cli/src/project/execution.ts +++ b/backend/cli/src/project/execution.ts @@ -32,6 +32,20 @@ export namespace ExecutionAuthority { ]) export type Capability = z.infer + // Not pure in-memory state: `compute/jobs.ts`'s `Job.authority` field + // stores a `Decision` verbatim in the on-disk job history (`jobs.json`), so + // `sandbox.network` below is a *second* copy of the persisted enum that + // `Job.sandbox.network` also carries — widening it (e.g. adding + // "allowlist") has the same one-directional compatibility cost: a job + // record this binary writes with the new value is rejected by an older + // binary reading the same history. The stakes are higher than "that one + // record" though — `ComputeJobs`'s `read()` runs `Job.array().safeParse()` + // over the *whole file* and throws `ComputeJobsCorruptError` for all of it + // on any single unparseable record (`compute/jobs.ts`'s `read()`), moving + // the file aside as `.corrupt-` rather than skipping the bad one. So + // one job with `sandbox.network: "allowlist"` written by a newer binary + // makes an older binary reject its entire compute job history, not just + // fail to display that job. export const Decision = z.object({ allowed: z.boolean(), reason: z.enum(["allowed", "project_untrusted", "sandbox_unavailable"]), @@ -51,11 +65,11 @@ export namespace ExecutionAuthority { writable: z.array(z.string()), sandbox: z.object({ enabled: z.boolean(), - network: z.enum(["allow", "deny"]), + network: z.enum(["deny", "allowlist", "allow"]), allowWrite: z.array(z.string()), onUnavailable: z.enum(["warn", "error", "allow"]), requireProjectTrust: z.boolean().default(false), - backend: z.enum(["seatbelt", "bubblewrap", "none"]), + backend: z.enum(["seatbelt", "bubblewrap", "appcontainer", "none"]), available: z.boolean(), enforced: z.boolean(), }), diff --git a/backend/cli/src/pty/index.ts b/backend/cli/src/pty/index.ts index b0d5682c..6ed01ba1 100644 --- a/backend/cli/src/pty/index.ts +++ b/backend/cli/src/pty/index.ts @@ -12,6 +12,7 @@ import { ExecutionAuthority } from "@/project/execution" import { AuthoritySignal } from "@/project/authority-signal" import { AuthorityProcessLedger } from "@/project/authority-process" import { Sandbox } from "@/sandbox/sandbox" +import { EgressRuntime } from "@/sandbox/egress-runtime" import { OpenScience } from "@/openscience" import { terminalArgs, terminalEnv } from "./environment" import { WindowsJobLauncher } from "@/process/windows-job-launcher" @@ -121,15 +122,21 @@ export namespace Pty { // Interactive PTY output is not a redaction boundary. Keep provider/cloud // credentials on the host; terminals receive runtime discovery only. const source = OpenScience.kernelEnv(process.env) - const env = terminalEnv(source, Instance.project.id, input.sessionID, command) + const egress = await EgressRuntime.egressFor(authority.sandbox) const sandbox = Sandbox.wrapArgv({ file: command, args, workspace: authority.writable, readable: authority.readable, unreadable: OpenScience.kernelSensitivePaths(), - options: authority.sandbox, + options: { ...authority.sandbox, egress }, }) + // After wrapArgv, not before: sandbox.env carries the proxy variables the + // loopback shim needs, and they must survive terminalEnv's own shaping. + const env = { + ...terminalEnv(source, Instance.project.id, input.sessionID, command), + ...(sandbox.env ?? {}), + } const launch = WindowsJobLauncher.wrap({ file: sandbox.file, args: sandbox.args }) log.info("creating session", { id, cmd: command, args, cwd }) diff --git a/backend/cli/src/sandbox/appcontainer.ts b/backend/cli/src/sandbox/appcontainer.ts new file mode 100644 index 00000000..51a5f400 --- /dev/null +++ b/backend/cli/src/sandbox/appcontainer.ts @@ -0,0 +1,1190 @@ +/** + * Windows AppContainer launcher. + * + * Linux and macOS both have a wrapper executable — `bwrap`, `sandbox-exec` — + * so confinement is expressible as an argv. Windows has none: it is applied AT + * process creation, by passing `SECURITY_CAPABILITIES` through + * `UpdateProcThreadAttribute` to `CreateProcessW`. So the binary launches + * itself (`openscience __appcontainer-launch -- `, the pattern + * `__egress-shim` already established) and this module does the Win32 work. + * + * Every call is modelled on `docs/specs/windows-appcontainer-probe.ps1`, which + * ran this exact sequence on a real Windows 11 machine, unelevated, and + * measured it working: profile creation, a child launched with zero + * capabilities, that child unable to reach the network, and two children in the + * same container able to talk over loopback. + * + * The x64 struct offsets below are written out beside the fields they belong + * to rather than derived. Getting one wrong produces a `CreateProcess` failure + * that reads as a permissions problem rather than a marshalling one, and that + * is an expensive hour on a machine none of us can debug interactively. + * + * The FFI patterns used here — an out-parameter pointer read back with + * `read.ptr`, and bytes read at a returned pointer with `toArrayBuffer` — were + * verified against libc on Linux before being written, because the mechanism + * is the same and the platform is not available to test on. + */ + +import { realpathSync } from "fs" + +export namespace AppContainer { + /** What the launcher is told to do, carried as one base64 blob through the + * command line. Kept in step with `Sandbox.appContainerArgs`. */ + export type Spec = { + profile: string + writable: string[] + /** Paths the child must READ but not write — its interpreter, above all. */ + readable?: string[] + unreadable: string[] + network: "deny" | "allowlist" | "allow" + /** Well-known capability SIDs to grant. Empty for deny and allowlist. */ + capabilities?: string[] + /** Broker pipe name, when network is "allowlist". */ + pipe?: string + /** The host-side proxy the broker relays into. */ + proxy?: { port: number; secret: string } + /** Argv that starts the shim inside the container. Composed host-side, + * because only there is it known whether this is a release (which + * re-enters its own binary) or a source checkout (bun plus a bundle). */ + shim?: string[] + /** + * How to re-enter this program, as an argv prefix — the binary, plus an + * entry script when running from source. + * + * Carried rather than derived here for two reasons. `process.execPath` is + * `bun` in a checkout, and `bun __appcontainer-detached ...` is not a valid + * invocation. And importing the module that knows the difference pulled + * `Global`'s top-level await into this file's graph, which `bun build + * --compile` refuses outright — so deriving it here does not merely + * duplicate a rule, it breaks the release build. + */ + self?: string[] + } + + export function decode(blob: string): Spec { + const value = JSON.parse(Buffer.from(blob, "base64").toString("utf8")) as Spec + if (!value?.profile) throw new Error("appcontainer spec carries no profile name") + if (!Array.isArray(value.writable)) throw new Error("appcontainer spec carries no writable list") + return value + } + + /** A null-terminated UTF-16LE buffer, which every `...W` entry point expects. + * Bun's FFI has no wide-string type, so strings cross as pointers to buffers + * the caller keeps alive for the duration of the call. */ + export function wide(value: string) { + return Buffer.from(`${value}\0`, "utf16le") + } + + /** Reads a null-terminated UTF-16LE string out of a byte view. */ + export function readWide(bytes: Uint8Array) { + const chars: number[] = [] + for (let i = 0; i + 1 < bytes.length; i += 2) { + const code = bytes[i]! | (bytes[i + 1]! << 8) + if (code === 0) break + chars.push(code) + } + return String.fromCharCode(...chars) + } + + // ── x64 layouts ─────────────────────────────────────────────────────────── + /** SECURITY_CAPABILITIES { PSID AppContainerSid; PSID_AND_ATTRIBUTES* ; DWORD CapabilityCount; DWORD Reserved } */ + const SECURITY_CAPABILITIES_SIZE = 24 + /** STARTUPINFOW is 104 bytes on x64; STARTUPINFOEXW appends lpAttributeList at 104. */ + const STARTUPINFOEX_SIZE = 112 + const STARTUPINFO_CB_OFFSET = 0 + const STARTUPINFO_FLAGS_OFFSET = 60 + const STARTUPINFO_STDIN_OFFSET = 80 + const STARTUPINFO_STDOUT_OFFSET = 88 + const STARTUPINFO_STDERR_OFFSET = 96 + const STARTUPINFO_ATTRIBUTE_LIST_OFFSET = 104 + const STARTF_USESTDHANDLES = 0x00000100 + const HANDLE_FLAG_INHERIT = 0x00000001 + /** STD_INPUT_HANDLE, STD_OUTPUT_HANDLE, STD_ERROR_HANDLE as unsigned. */ + const STD_HANDLES = { input: 0xfffffff6, output: 0xfffffff5, error: 0xfffffff4 } + /** PROCESS_INFORMATION { HANDLE hProcess; HANDLE hThread; DWORD pid; DWORD tid } */ + const PROCESS_INFORMATION_SIZE = 24 + const PI_PROCESS_OFFSET = 0 + + /** SID_AND_ATTRIBUTES { PSID Sid; DWORD Attributes; } — 8 + 4, padded to 16. */ + const SID_AND_ATTRIBUTES_SIZE = 16 + const SE_GROUP_ENABLED = 0x00000004 + const TOKEN_QUERY = 0x0008 + /** TOKEN_INFORMATION_CLASS.TokenIsAppContainer — a DWORD, 1 inside a container. */ + const TOKEN_IS_APP_CONTAINER = 29 + const PROC_THREAD_ATTRIBUTE_SECURITY_CAPABILITIES = 0x00020009 + const EXTENDED_STARTUPINFO_PRESENT = 0x00080000 + /** HRESULT_FROM_WIN32(ERROR_ALREADY_EXISTS). The profile is per-user state + * that outlives a run by design, so this is the ordinary path. */ + const ALREADY_EXISTS = 0x800700b7 + const INFINITE = 0xffffffff + + type Bound = ReturnType + + /** + * Bound once per process, and kept. + * + * `dlopen` returns a library object that owns the handle; only `.symbols` was + * being kept, so the object was immediately garbage. Bun closes a library when + * that object is collected, which would unmap the very code a later call jumps + * into. `main` binds three times over one launch (ensureProfile, grant, launch) + * and `launch` opened advapi32 a fourth time, so there was ample opportunity. + * Caching removes the question entirely rather than reasoning about GC timing. + */ + let bound: Bound | undefined + function bind(): Bound { + bound ??= open() + return bound + } + + function open() { + if (process.platform !== "win32") throw new Error("the AppContainer launcher only runs on Windows") + // Required lazily and by name so `bun:ffi` never enters the module graph on + // platforms that cannot call this. These DLLs ship with Windows, so nothing + // additional is distributed. + const ffi = require("bun:ffi") as typeof import("bun:ffi") + const t = ffi.FFIType + const userenv = ffi.dlopen("userenv.dll", { + CreateAppContainerProfile: { args: [t.ptr, t.ptr, t.ptr, t.ptr, t.u32, t.ptr], returns: t.i32 }, + DeriveAppContainerSidFromAppContainerName: { args: [t.ptr, t.ptr], returns: t.i32 }, + DeleteAppContainerProfile: { args: [t.ptr], returns: t.i32 }, + }) + const advapi = ffi.dlopen("advapi32.dll", { + ConvertSidToStringSidW: { args: [t.ptr, t.ptr], returns: t.bool }, + ConvertStringSidToSidW: { args: [t.ptr, t.ptr], returns: t.bool }, + FreeSid: { args: [t.ptr], returns: t.ptr }, + ConvertStringSecurityDescriptorToSecurityDescriptorW: { args: [t.ptr, t.u32, t.ptr, t.ptr], returns: t.bool }, + OpenProcessToken: { args: [t.ptr, t.u32, t.ptr], returns: t.bool }, + GetTokenInformation: { args: [t.ptr, t.u32, t.ptr, t.u32, t.ptr], returns: t.bool }, + }) + const kernel = ffi.dlopen("kernel32.dll", { + LocalFree: { args: [t.ptr], returns: t.ptr }, + GetLastError: { args: [], returns: t.u32 }, + GetStdHandle: { args: [t.u32], returns: t.ptr }, + SetHandleInformation: { args: [t.ptr, t.u32, t.u32], returns: t.bool }, + InitializeProcThreadAttributeList: { args: [t.ptr, t.u32, t.u32, t.ptr], returns: t.bool }, + UpdateProcThreadAttribute: { args: [t.ptr, t.u32, t.u64, t.ptr, t.u64, t.ptr, t.ptr], returns: t.bool }, + DeleteProcThreadAttributeList: { args: [t.ptr], returns: t.void }, + CreateProcessW: { + args: [t.ptr, t.ptr, t.ptr, t.ptr, t.bool, t.u32, t.ptr, t.ptr, t.ptr, t.ptr], + returns: t.bool, + }, + WaitForSingleObject: { args: [t.ptr, t.u32], returns: t.u32 }, + GetExitCodeProcess: { args: [t.ptr, t.ptr], returns: t.bool }, + CloseHandle: { args: [t.ptr], returns: t.bool }, + CreateNamedPipeW: { args: [t.ptr, t.u32, t.u32, t.u32, t.u32, t.u32, t.u32, t.ptr], returns: t.ptr }, + ConnectNamedPipe: { args: [t.ptr, t.ptr], returns: t.bool }, + ReadFile: { args: [t.ptr, t.ptr, t.u32, t.ptr, t.ptr], returns: t.bool }, + WriteFile: { args: [t.ptr, t.ptr, t.u32, t.ptr, t.ptr], returns: t.bool }, + DisconnectNamedPipe: { args: [t.ptr], returns: t.bool }, + }) + // The library objects are returned, not just their symbols, so they stay + // reachable for the life of the process. + return { + ffi, + libs: [userenv, advapi, kernel], + userenv: userenv.symbols, + advapi: advapi.symbols, + kernel: kernel.symbols, + } + } + + /** + * Can this machine actually be confined by us? + * + * Loads the DLLs and derives a SID from a name. That is side-effect free — no + * profile is created — and it exercises the part most likely to be wrong: + * whether the FFI bindings resolve and the calling convention is right. A + * broken binding here is the difference between the sandbox being applied and + * silently not being. + * + * It does NOT prove the launch itself works. That is verified at first use, + * where `launch` throws with the Win32 error rather than degrading quietly. + * The alternative — assuming Windows can be confined because the platform + * says win32 — is how you ship a product that claims a sandbox it never + * applies. + */ + export function usable(): boolean { + if (process.platform !== "win32") return false + try { + const b = bind() + const out = new BigUint64Array(1) + const hr = b.userenv.DeriveAppContainerSidFromAppContainerName( + b.ffi.ptr(wide("openscience-capability")), + b.ffi.ptr(out), + ) + if (hr !== 0) return false + const sid = b.ffi.read.ptr(b.ffi.ptr(out), 0) + if (sid) b.advapi.FreeSid(sid as never) + return true + } catch { + return false + } + } + + /** + * Create the profile if absent, and return its package SID as a string. + * + * Idempotent by design. The profile is per-user state that outlives a run, + * and the SID derived from it is what filesystem ACEs and the broker pipe's + * DACL refer to — recreating it per launch would strand every grant the + * previous one made. That is why `Sandbox.appContainerProfile` derives a + * stable name from the workspace rather than generating one. + */ + export function ensureProfile(name: string, b: Bound = bind()): string { + const { ffi, userenv, advapi, kernel } = b + const wname = wide(name) + const display = wide(name) + const description = wide("OpenScience sandbox") + const sidOut = new BigUint64Array(1) + + let hr = userenv.CreateAppContainerProfile( + ffi.ptr(wname), + ffi.ptr(display), + ffi.ptr(description), + null, + 0, + ffi.ptr(sidOut), + ) + if (hr >>> 0 === ALREADY_EXISTS) { + hr = userenv.DeriveAppContainerSidFromAppContainerName(ffi.ptr(wname), ffi.ptr(sidOut)) + if (hr !== 0) throw new Error(`DeriveAppContainerSidFromAppContainerName failed: 0x${(hr >>> 0).toString(16)}`) + } else if (hr !== 0) { + throw new Error( + `CreateAppContainerProfile failed: 0x${(hr >>> 0).toString(16)}. Windows sandboxing rests on this call; ` + + `without it nothing can be confined. It is expected to succeed for a standard user, unelevated.`, + ) + } + + const sid = ffi.read.ptr(ffi.ptr(sidOut), 0) + const strOut = new BigUint64Array(1) + if (!advapi.ConvertSidToStringSidW(sid as never, ffi.ptr(strOut))) { + throw new Error(`ConvertSidToStringSid failed: Win32 ${kernel.GetLastError()}`) + } + const strPtr = ffi.read.ptr(ffi.ptr(strOut), 0) + // A package SID is well under 512 UTF-16 code units; readWide stops at the + // first null either way. + const text = readWide(new Uint8Array(ffi.toArrayBuffer(strPtr as never, 0, 1024))) + kernel.LocalFree(strPtr as never) + advapi.FreeSid(sid as never) + return text + } + + /** + * Grant the package SID access to paths the sandboxed process must write. + * + * An AppContainer reaches nothing outside its own package folders, so the + * workspace has to be granted explicitly. `icacls` rather than + * `SetNamedSecurityInfo` through FFI: it ships with Windows, takes a SID + * directly in the `*S-1-...` form, and a shelled command that fails is far + * easier to diagnose than a marshalled ACL that silently grants the wrong + * thing. The probe measured that the package's OWN temp is already writable + * with no grant, so only caller-supplied paths are touched. + * + * Returns the paths it could not grant rather than throwing: a workspace that + * is partly ungrantable should still launch and fail visibly at the write, + * not vanish behind a launcher error. + */ + export function grant(sid: string, writable: string[], readable: string[] = []) { + const failures: string[] = [] + const unreachable: string[] = [] + // Under OPENSCIENCE_SANDBOX_DEBUG, every ACL change is echoed with its exit + // code. An `icacls` that exits 0 is not proof the ACE landed where the child + // needed it, and a `readable` list that arrived here EMPTY is indistinguishable + // from one that was granted successfully — both are silent. That ambiguity is + // what left "the launcher cannot spawn its base" undiagnosed across a CI run + // that otherwise reported everything about the launch. + const debug = process.env["OPENSCIENCE_SANDBOX_DEBUG"] === "1" + const say = (line: string) => { + if (debug) process.stderr.write(`openscience[grant] ${line}\n`) + } + say(`writable(${writable.length}): ${writable.join(" | ") || ""}`) + say(`readable(${readable.length}): ${readable.join(" | ") || ""}`) + const icacls = (target: string, args: string[]) => { + const proc = Bun.spawnSync(["icacls.exe", target, ...args, "/Q"], { stdout: "pipe", stderr: "pipe" }) + say(`icacls ${target} ${args.join(" ")} -> exit ${proc.exitCode} ${proc.stderr.toString().trim()}`) + if (proc.exitCode !== 0) failures.push(`${target}: ${proc.stderr.toString().trim() || `exit ${proc.exitCode}`}`) + } + for (const target of writable) { + // Read-and-execute plus write, for the DACL. + icacls(target, ["/grant", `*${sid}:(OI)(CI)(F)`]) + // And the mandatory label, WITHOUT WHICH THE GRANT ABOVE DOES NOTHING. + // + // Every AppContainer runs at Low integrity; a file or directory created + // normally is Medium. Mandatory Integrity Control is evaluated BEFORE the + // DACL, and a Low-integrity principal cannot write to a Medium-integrity + // object even when the DACL explicitly grants it write access. So the + // grant above was necessary and never sufficient, and the self-test's + // "write inside the workspace succeeds" could not pass however the paths + // were spelled — which is what several rounds of shell-quoting fixes were + // actually chasing. + // + // The cost is real and worth stating: labelling the workspace Low means + // any OTHER low-integrity process on the machine can write there too — a + // sandboxed browser tab, say. That is the standard price of an + // AppContainer-writable directory and what Chromium's sandbox does for the + // same reason; there is no way to raise an AppContainer above Low. It is + // applied ONLY to paths already chosen as writable, never to the readable + // set: lowering the label on an interpreter installation would let any + // low-integrity process on the machine modify the Python we then execute. + icacls(target, ["/setintegritylevel", "(OI)(CI)L"]) + } + // Read AND execute: the interpreter must be runnable, so plain (R) is not + // enough; never (F), which would hand a sandboxed process write access to + // the Python installation it is confined away from. No label change — + // MIC's default policy is no-write-up only, so reading a Medium object + // from Low is already allowed. + // Each readable path AND whatever it resolves to. An ACE on a reparse point + // is not an ACE on its target, and Windows checks the target — so granting + // only the name we were handed produces a launch that is denied while every + // command involved reports success. + // + // Measured. uv keeps a managed interpreter under a patch-versioned + // directory and a stable one beside it, and the stable name is a link: + // + // managed interpreter: ...\uv\python\cpython-3.12.14-windows-x86_64-none\python.exe + // pyvenv.cfg home : ...\uv\python\cpython-3.12-windows-x86_64-none + // + // `pyvenv.cfg` names the stable one, so that is what we granted. icacls + // exited 0, and reading the ACL back showed the ACE sitting on it exactly + // as asked — all true, none of it any use to the container, which exited 53 + // with nothing on either stream. + const resolve = (target: string) => { + try { + const real = realpathSync(target) + if (real !== target) say(`${target} resolves to ${real}`) + return real === target ? [target] : [target, real] + } catch { + return [target] + } + } + const targets = [...new Set(readable.filter((p) => !writable.includes(p)).flatMap(resolve))] + for (const target of targets) { + const before = failures.length + icacls(target, ["/grant", `*${sid}:(OI)(CI)(RX)`]) + // A failed READ grant is fatal, unlike a failed write grant. It exists + // only because something in there must be readable — the interpreter, + // above all — so continuing produces a child that cannot start and an + // error several layers from its cause. Measured exactly that way: icacls + // on C:\Python312 denied, then `No Python at ...` and child exit 103, + // which reads as a broken Python rather than an ungrantable directory. + if (failures.length > before) unreachable.push(target) + // The resulting ACL, read back. `icacls /grant` exiting 0 says the command + // parsed, not that the container can reach anything through it. + if (debug) { + const read = Bun.spawnSync(["icacls.exe", target], { stdout: "pipe", stderr: "pipe" }) + for (const line of read.stdout.toString().trim().split("\n")) say(`acl ${line.trim()}`) + } + } + return { failures, unreachable } + } + + /** + * Delete a profile and the package folders under it. + * + * A profile is per-user state keyed to a name derived from the workspace, so a + * project reuses one forever and nothing accumulates — that is the design and + * it is right. What DOES accumulate is profiles for workspaces that were + * themselves ephemeral: every `sandbox test` run builds a fresh mkdtemp + * workspace, so every run has orphaned a profile and its + * `AppData\Local\Packages` folder. Observed as a different package SID on + * each run of the self-test. + * + * Callers that made a throwaway workspace should remove the profile it + * implied. Callers working in a real project should NOT: the profile is what + * makes grants stable across runs. + */ + export function removeProfile(name: string, b: Bound = bind()) { + const { ffi, userenv } = b + return userenv.DeleteAppContainerProfile(ffi.ptr(wide(name))) === 0 + } + + /** + * Put back what `grant` changed. + * + * `grant` lowers the workspace's mandatory label to Low, and that is the only + * way a Low-integrity AppContainer can write anywhere — but it is not a change + * to make and walk away from. A Low label means ANY low-integrity process on + * the machine can write there: a sandboxed browser tab, a document preview. + * Left behind, it outlives the run that needed it and applies to the user's + * own project directory. The DACL entry is narrower — only our package SID can + * use it — but there is no reason to leave that either. + * + * Medium rather than "no label": a user-created file has no explicit label and + * is treated as Medium, so this restores the effective behaviour rather than + * the exact bytes. Worth knowing when reading an ACL afterwards. + * + * Best effort, and two limits are worth stating rather than discovering: + * a killed process never reaches this, and two runs sharing one workspace will + * have the first to finish restore the label under the second. Both argue for + * making cleanup idempotent and cheap, which is why failures are collected + * rather than thrown. + */ + export function revoke(sid: string, writable: string[], readable: string[] = []) { + const failures: string[] = [] + const icacls = (target: string, args: string[]) => { + const proc = Bun.spawnSync(["icacls.exe", target, ...args, "/Q"], { stdout: "ignore", stderr: "pipe" }) + if (proc.exitCode !== 0) failures.push(`${target}: ${proc.stderr.toString().trim() || `exit ${proc.exitCode}`}`) + } + for (const target of writable) { + icacls(target, ["/setintegritylevel", "(OI)(CI)M"]) + icacls(target, ["/remove:g", `*${sid}`]) + } + for (const target of readable.filter((p) => !writable.includes(p))) icacls(target, ["/remove:g", `*${sid}`]) + return failures + } + + /** SECURITY_ATTRIBUTES { DWORD nLength; LPVOID lpSecurityDescriptor; BOOL bInheritHandle } */ + const SECURITY_ATTRIBUTES_SIZE = 24 + const PIPE_ACCESS_DUPLEX = 0x00000003 + const PIPE_TYPE_BYTE = 0x00000000 + /** Non-blocking mode. Deprecated by Microsoft in favour of overlapped I/O, + * and correct here for the same reason overlapped is correct in the broker: + * a synchronous blocking Win32 call in a JS process blocks the event loop, so + * the caller's own timeout can never fire. Measured the hard way — a bare + * ConnectNamedPipe hung a CI job until the 20-minute job limit, because the + * test's 120s timeout was waiting behind the call it was meant to bound. */ + const PIPE_NOWAIT = 0x00000001 + const ERROR_PIPE_CONNECTED = 535 + const ERROR_PIPE_LISTENING = 536 + const ERROR_NO_DATA = 232 + const ERROR_BROKEN_PIPE = 109 + const PIPE_UNLIMITED_INSTANCES = 255 + const SDDL_REVISION_1 = 1 + + /** + * A named pipe only this container can open. + * + * The DACL is the entire access decision here and deserves the scrutiny the + * seatbelt profile text gets. The probe measured both halves on a real + * machine: a pipe with the DEFAULT DACL is denied to the container outright + * ("Access to the path is denied"), and one granting the package SID connects + * and sustains 65540 bytes. So this grant is what makes the transport exist, + * and over-granting it is what would make the sandbox pointless. + * + * Written as SDDL rather than assembled from ACE structs: building an ACL + * through FFI is a lot of pointer arithmetic to express one sentence, and a + * mistake in it fails OPEN — a wider DACL than intended still works, so + * nothing would notice. The string says exactly who may connect. + * + * Only three principals are named. The package SID is the point. SYSTEM and + * the local Administrators group are listed because they can open any pipe on + * the machine whatever we write, so omitting them would imply a restriction + * that does not exist. + */ + export function pipeSecurity(sid: string) { + return `D:(A;;GA;;;${sid})(A;;GA;;;SY)(A;;GA;;;BA)` + } + + export function pipePath(name: string) { + return `\\\\.\\pipe\\${name}` + } + + /** One instance of the broker pipe. The caller owns closing the handle. */ + export function createPipe(name: string, sid: string, b: Bound = bind()) { + const { ffi, advapi, kernel } = b + const descriptor = new BigUint64Array(1) + if ( + !advapi.ConvertStringSecurityDescriptorToSecurityDescriptorW( + ffi.ptr(wide(pipeSecurity(sid))), + SDDL_REVISION_1, + ffi.ptr(descriptor), + null, + ) + ) + throw new Error(`Could not build the pipe security descriptor: Win32 ${kernel.GetLastError()}`) + const attributes = new Uint8Array(SECURITY_ATTRIBUTES_SIZE) + const view = new DataView(attributes.buffer) + view.setUint32(0, SECURITY_ATTRIBUTES_SIZE, true) + view.setBigUint64(8, BigInt(ffi.read.ptr(ffi.ptr(descriptor), 0) as number), true) + const handle = kernel.CreateNamedPipeW( + ffi.ptr(wide(pipePath(name))), + PIPE_ACCESS_DUPLEX, + PIPE_TYPE_BYTE | PIPE_NOWAIT, + PIPE_UNLIMITED_INSTANCES, + 65536, + 65536, + 0, + ffi.ptr(attributes), + ) as number + // INVALID_HANDLE_VALUE is -1, which arrives here as an unsafe integer. + if (!Number.isSafeInteger(handle) || handle <= 0) + throw new Error(`CreateNamedPipe failed for ${pipePath(name)}: Win32 ${kernel.GetLastError()}`) + return handle + } + + /** + * Block until a client connects, read one chunk, echo it back. + * + * Synchronous on purpose: this is the measurement harness that proves the DACL + * lets the container in, not the broker. The broker needs overlapped I/O and + * an instance per connection, and building that before knowing the grant works + * would be the expensive order to find out. + */ + export function pipeEchoOnce(handle: number, timeoutMs = 20_000, b: Bound = bind()) { + const { ffi, kernel } = b + const deadline = Date.now() + timeoutMs + const waiting = () => { + if (Date.now() < deadline) return true + throw new Error(`no client reached the pipe within ${timeoutMs}ms`) + } + // Poll, never block. ERROR_PIPE_CONNECTED means a client arrived between + // creation and this call, which is a success rather than a failure. + while (!kernel.ConnectNamedPipe(handle as never, null)) { + if (kernel.GetLastError() === ERROR_PIPE_CONNECTED) break + if (waiting()) Bun.sleepSync(25) + } + const buffer = new Uint8Array(4096) + const read = new Uint32Array(1) + while (!kernel.ReadFile(handle as never, ffi.ptr(buffer), buffer.length, ffi.ptr(read), null) || !read[0]) { + if (waiting()) Bun.sleepSync(25) + } + const got = Buffer.from(buffer.slice(0, read[0]!)).toString("utf8") + const reply = Buffer.from(`echo:${got}`, "utf8") + const wrote = new Uint32Array(1) + kernel.WriteFile(handle as never, ffi.ptr(reply), reply.length, ffi.ptr(wrote), null) + kernel.DisconnectNamedPipe(handle as never) + return got + } + + /** + * The broker: a named pipe only this container can open, relayed to the host + * proxy over ordinary TCP loopback. + * + * Why this shape, all of it measured rather than chosen: + * + * - The container cannot reach the host's loopback, and the host cannot reach + * a listener the container binds. Isolation runs both ways, so a pipe is the + * only transport across the boundary. + * - The HOST has no such restriction, so `Egress.serveProxy` runs unchanged on + * 127.0.0.1 exactly as it does for seatbelt, and this relays into it. Host + * allowlisting, Proxy-Authorization and the audit trail stay shared with the + * other two platforms rather than forked. + * - The in-container half needs no new code at all: `Bun.connect({unix})` maps + * a `\\.\pipe\` path through libuv, so `Egress.serveShim` works verbatim. + * + * Polled, not overlapped. Overlapped I/O needs an event object and a wait that + * a JS event loop cannot participate in; PIPE_NOWAIT plus a timer keeps every + * Win32 call non-blocking, which is the rule this file already had to learn + * the hard way — a bare ConnectNamedPipe once held a CI runner for twenty + * minutes because the timeout meant to bound it was queued behind it. + * + * The interval adapts: 1ms while bytes are moving, 15ms when idle. A wheel + * download is a burst, and paying a busy 1ms timer for the hours a session is + * idle would be worse than the latency it saves. + */ + export function serveBroker(input: { + name: string + sid: string + hostname: string + port: number + onError?: (message: string) => void + }) { + const b = bind() + const { ffi, kernel } = b + const report = input.onError ?? (() => {}) + + type Link = { + handle: number + buffer: Uint8Array + read: Uint32Array + wrote: Uint32Array + socket?: import("bun").Socket + /** Bytes from the proxy waiting for a WriteFile. */ + pending: Uint8Array[] + closing: boolean + } + const links = new Set() + let listening: Link | undefined + let stopped = false + + const openInstance = (): Link => ({ + handle: createPipe(input.name, input.sid, b), + // Allocated once per link and held: the kernel writes through this + // pointer, so a per-call buffer would be a fresh allocation the GC could + // move between the pointer being taken and ReadFile using it. + buffer: new Uint8Array(65536), + read: new Uint32Array(1), + wrote: new Uint32Array(1), + pending: [], + closing: false, + }) + + const drop = (link: Link) => { + links.delete(link) + try { + link.socket?.end() + } catch {} + try { + kernel.DisconnectNamedPipe(link.handle as never) + kernel.CloseHandle(link.handle as never) + } catch {} + } + + /** A client connected: dial the proxy for it, and open the next instance so + * the pipe is never momentarily unlistenable. */ + const accept = (link: Link) => { + links.add(link) + listening = openInstance() + Bun.connect({ + hostname: input.hostname, + port: input.port, + socket: { + data: (_socket, chunk) => { + link.pending.push(new Uint8Array(chunk)) + }, + close: () => { + link.closing = true + }, + error: (_socket, error) => { + report(`broker upstream error: ${error.message}`) + link.closing = true + }, + open: () => {}, + }, + }).then( + (socket) => { + if (stopped || link.closing) return socket.end() + link.socket = socket + }, + (error: Error) => { + report(`broker could not reach the proxy at ${input.hostname}:${input.port}: ${error.message}`) + drop(link) + }, + ) + } + + const pump = () => { + let moved = false + if (listening) { + const connected = kernel.ConnectNamedPipe(listening.handle as never, null) + const why = connected ? 0 : kernel.GetLastError() + if (connected || why === ERROR_PIPE_CONNECTED) { + accept(listening) + moved = true + } else if (why !== ERROR_PIPE_LISTENING && why !== ERROR_NO_DATA) { + report(`broker ConnectNamedPipe failed: Win32 ${why}`) + } + } + for (const link of [...links]) { + // Pipe -> proxy. + const ok = kernel.ReadFile( + link.handle as never, + ffi.ptr(link.buffer), + link.buffer.length, + ffi.ptr(link.read), + null, + ) + if (ok && link.read[0]) { + link.socket?.write(link.buffer.slice(0, link.read[0]!)) + moved = true + } else if (!ok && kernel.GetLastError() === ERROR_BROKEN_PIPE) { + drop(link) + continue + } + // Proxy -> pipe. Only once the socket exists, or bytes that arrive + // before the dial completes would be written to a handle whose peer has + // not been told anything yet. + while (link.pending.length && link.socket) { + const chunk = link.pending.shift()! + if (!kernel.WriteFile(link.handle as never, ffi.ptr(chunk), chunk.length, ffi.ptr(link.wrote), null)) { + const why = kernel.GetLastError() + if (why === ERROR_BROKEN_PIPE) { + drop(link) + break + } + report(`broker WriteFile failed: Win32 ${why}`) + break + } + moved = true + } + if (link.closing && !link.pending.length) drop(link) + } + timer = setTimeout(pump, moved ? 1 : 15) + } + + listening = openInstance() + let timer = setTimeout(pump, 1) + + return { + pipe: pipePath(input.name), + stop() { + stopped = true + clearTimeout(timer) + for (const link of [...links]) drop(link) + if (listening) { + try { + kernel.CloseHandle(listening.handle as never) + } catch {} + listening = undefined + } + }, + } + } + + /** + * Stand up the egress path for one launch, and return how to tear it down. + * + * Order matters and is not arbitrary. The pipe must exist before the shim + * dials it, the shim must be listening before the payload is told to use it, + * and `HTTP_PROXY` is set on THIS process rather than passed as an + * environment block because the payload inherits our environment — which + * saves marshalling a UTF-16 block through FFI for one variable. + */ + async function bridgeEgress(sid: string, spec: Spec) { + const broker = serveBroker({ + name: spec.pipe!, + sid, + hostname: "127.0.0.1", + port: spec.proxy!.port, + onError: (message) => process.stderr.write(`openscience: ${message}\n`), + }) + + // A port the shim binds INSIDE the container. An AppContainer shares the + // host's network stack rather than getting a namespace, so a port free here + // is free there — and a fixed one (as bubblewrap's SHIM_PORT is) could + // collide with whatever else is on the machine. + const probe = Bun.listen({ hostname: "127.0.0.1", port: 0, socket: { data() {} } }) + const port = probe.port + probe.stop(true) + + // No fallback to a locally composed argv. Guessing `process.execPath` here + // is what produced `Script not found "__egress-shim"` and a dead proxy port + // under `bun test`, and a silent wrong guess is worse than a loud absence. + if (!spec.shim) throw new Error("sandbox: allowlist egress requires a shim argv in the spec") + if (!spec.self) throw new Error("sandbox: allowlist egress requires a re-entry argv in the spec") + // Started inside the workspace, which is granted by construction. + const shim = launchDetached(spec.self, sid, [...spec.shim, String(port), broker.pipe], [], spec.writable[0]) + // The payload inherits this. `os:` is the same Proxy-Authorization + // seatbelt uses, and the shim relays bytes without interpreting them, so the + // credential travels end to end unchanged. + const url = `http://os:${spec.proxy!.secret}@127.0.0.1:${port}` + for (const key of ["HTTP_PROXY", "HTTPS_PROXY", "http_proxy", "https_proxy"]) process.env[key] = url + + return { + stop() { + try { + shim.kill() + } catch {} + broker.stop() + }, + } + } + + /** + * Quote one argument the way `CommandLineToArgvW` will parse it back. + * + * Windows has no argv: `CreateProcess` takes a single string and the child + * re-splits it. The rules are neither the shell's nor POSIX's — backslashes + * are literal except immediately before a quote, where they double. Getting + * this wrong on a path like `C:\Users\me\My Project\` silently changes what + * the child runs, which is the whole reason the sandbox spec travels as + * base64 rather than as flags. + */ + export function quote(value: string) { + if (value.length && !/[\s"]/.test(value)) return value + let out = '"' + let slashes = 0 + for (const ch of value) { + if (ch === "\\") { + slashes++ + continue + } + if (ch === '"') { + out += "\\".repeat(slashes * 2 + 1) + '"' + slashes = 0 + continue + } + out += "\\".repeat(slashes) + ch + slashes = 0 + } + return `${out}${"\\".repeat(slashes * 2)}"` + } + + export function commandLine(argv: string[]) { + // cmd.exe is the exception to `quote`, and it fails in a way that reads as + // a broken sandbox. It does NOT parse its `/c` tail with + // `CommandLineToArgvW` and does not recognise a backslash-escaped quote, so + // quoting the tail the normal way produced, on a real machine: + // echo hi>"C:\...\probe" -> "echo hi>\"C:\...\probe\"" + // and cmd read the backslashes literally, answering "The filename, + // directory name, or volume label syntax is incorrect." The same shape turned + // `dir C:\` into `dir C:\\`. + // + // With `/s` cmd strips exactly the first and last quote and takes the rest + // verbatim, so the tail is wrapped once and left alone. This is what Node + // does for every Windows spawn. + const at = argv.findIndex((value) => value.toLowerCase() === "/c") + if (at > 0 && /(^|[\\/])cmd(\.exe)?$/i.test(argv[0] ?? "")) { + return `${argv + .slice(0, at + 1) + .map(quote) + .join(" ")} "${argv.slice(at + 1).join(" ")}"` + } + return argv.map(quote).join(" ") + } + + /** + * Launch `argv` inside the AppContainer for `sid`, with NO capabilities, and + * return its exit code. + * + * Zero capabilities is the entire point: no `internetClient`, nothing. The + * probe measured that such a container reaches no external host, no host + * loopback listener, and resolves no DNS, while remaining able to talk to + * another process in the same container over loopback — which is what makes + * the shim model viable here. + */ + /** + * Start a child in the container WITHOUT waiting for it. + * + * The shim has to be running before the payload starts, and it never exits on + * its own — it serves until the container goes away. `launch` waits, which is + * right for the payload and would deadlock here. + */ + export function launchDetached( + self: string[], + sid: string, + argv: string[], + capabilities: string[] = [], + cwd?: string, + /** "inherit" for the payload, whose output is the point; "ignore" for the + * shim, which has none worth carrying. */ + stdout: "ignore" | "inherit" = "ignore", + ) { + // A helper process, not a thread: `launch` blocks on WaitForSingleObject and + // bun:ffi has no way to run that off the event loop. So the shim gets its + // own host process whose whole job is to hold that wait, and killing it is + // what tears the shim down. + // + // `cwd` is not cosmetic. CreateProcessW inherits the parent's working + // directory, and this helper's parent is the launcher, whose cwd is wherever + // the user happened to run from -- a path the container is not granted. The + // process is created (`CreateProcessW -> true`) and then dies on its first + // syscall, which for bun reads: + // + // error loading current directory + // + // with nothing else on either stream. So the shim must start somewhere the + // container can actually reach, which is what this passes down. + return Bun.spawn([...self, "__appcontainer-detached", sid, JSON.stringify(capabilities), cwd ?? "", ...argv], { + stdout, + stderr: "inherit", + ...(cwd ? { cwd } : {}), + }) + } + + export function launch( + sid: string, + argv: string[], + capabilities: string[] = [], + b: Bound = bind(), + cwd?: string, + ): number { + const { ffi, advapi, kernel } = b + // Set OPENSCIENCE_SANDBOX_DEBUG=1 to dump every intermediate value. + // + // `sandbox test` has now proved the child runs UNCONFINED: CreateProcess + // succeeds, the command executes, and the token carries no package SID. The + // probe ran this same sequence successfully in PowerShell on the same + // machine, so the difference is in what we hand the kernel, not in what the + // kernel supports. Guessing at that across a rebuild cycle each time has + // been the expensive part; this makes one run answer it. + const debug = process.env["OPENSCIENCE_SANDBOX_DEBUG"] === "1" + const say = (line: string) => { + if (debug) process.stderr.write(`openscience[appcontainer] ${line}\n`) + } + const bytes = (view: Uint8Array) => Buffer.from(view).toString("hex") + const keep: unknown[] = [] + + const sidBuf = new BigUint64Array(1) + // ConvertStringSidToSidW comes from the cached binding now. Opening + // advapi32 a second time here left a library object nothing referenced. + if (!advapi.ConvertStringSidToSidW(ffi.ptr(wide(sid)), ffi.ptr(sidBuf))) { + throw new Error(`ConvertStringSidToSid failed for ${sid}: Win32 ${kernel.GetLastError()}`) + } + const sidPtr = ffi.read.ptr(ffi.ptr(sidBuf), 0) + say(`sid ${sid} -> 0x${(sidPtr as number).toString(16)}`) + + // Size the attribute list, then allocate and initialise it. The first call + // is expected to fail with ERROR_INSUFFICIENT_BUFFER; only the size matters. + const sizeOut = new BigUint64Array(1) + kernel.InitializeProcThreadAttributeList(null, 1, 0, ffi.ptr(sizeOut)) + const listSize = Number(sizeOut[0]!) + say(`attribute list size ${listSize}`) + if (!listSize) throw new Error("InitializeProcThreadAttributeList reported a zero-length attribute list") + const attributes = new Uint8Array(listSize) + if (!kernel.InitializeProcThreadAttributeList(ffi.ptr(attributes), 1, 0, ffi.ptr(sizeOut))) { + throw new Error(`InitializeProcThreadAttributeList failed: Win32 ${kernel.GetLastError()}`) + } + + // The capability array, when the policy grants any. + // + // Zero capabilities is the containment for `deny` and `allowlist`; `allow` + // grants internetClient and privateNetworkClientServer so it means what it + // says. Each entry is a SID_AND_ATTRIBUTES, and the SIDs must outlive the + // call for the same reason the struct below does — the kernel reads through + // these pointers at CreateProcess, not at UpdateProcThreadAttribute. + const granted = new Uint8Array(Math.max(1, capabilities.length) * SID_AND_ATTRIBUTES_SIZE) + const grantedView = new DataView(granted.buffer) + capabilities.forEach((name, i) => { + const out = new BigUint64Array(1) + if (!advapi.ConvertStringSidToSidW(ffi.ptr(wide(name)), ffi.ptr(out))) { + throw new Error(`ConvertStringSidToSid failed for capability ${name}: Win32 ${kernel.GetLastError()}`) + } + keep.push(out) + grantedView.setBigUint64(i * SID_AND_ATTRIBUTES_SIZE, BigInt(ffi.read.ptr(ffi.ptr(out), 0) as number), true) + grantedView.setUint32(i * SID_AND_ATTRIBUTES_SIZE + 8, SE_GROUP_ENABLED, true) + }) + keep.push(granted) + say(`capabilities requested: ${capabilities.length ? capabilities.join(", ") : "none"}`) + + const security = new Uint8Array(SECURITY_CAPABILITIES_SIZE) + new DataView(security.buffer).setBigUint64(0, BigInt(sidPtr as number), true) + if (capabilities.length) { + new DataView(security.buffer).setBigUint64(8, BigInt(ffi.ptr(granted) as number), true) + new DataView(security.buffer).setUint32(16, capabilities.length, true) + } + // For deny and allowlist the Capabilities pointer stays null and the count + // stays 0 — that is the containment, and under allowlist it is what forces + // traffic through the broker instead of around it. + // + // These two buffers must outlive the call: UpdateProcThreadAttribute stores + // a POINTER to `capabilities` inside `attributes`, and does not copy it, so + // the value has to still be there when CreateProcess reads the list. C# uses + // AllocHGlobal for exactly this reason. Holding both in `keep` makes the + // lifetime explicit rather than relying on them merely still being in scope. + keep.push(attributes, security, sidBuf) + say(`security capabilities ${bytes(security)}`) + say(`attributes at 0x${(ffi.ptr(attributes) as number).toString(16)}`) + + if ( + !kernel.UpdateProcThreadAttribute( + ffi.ptr(attributes), + 0, + BigInt(PROC_THREAD_ATTRIBUTE_SECURITY_CAPABILITIES) as never, + ffi.ptr(security), + BigInt(SECURITY_CAPABILITIES_SIZE) as never, + null, + null, + ) + ) { + throw new Error(`UpdateProcThreadAttribute failed: Win32 ${kernel.GetLastError()}`) + } + + const startup = new Uint8Array(STARTUPINFOEX_SIZE) + const startupView = new DataView(startup.buffer) + startupView.setUint32(STARTUPINFO_CB_OFFSET, STARTUPINFOEX_SIZE, true) + startupView.setBigUint64(STARTUPINFO_ATTRIBUTE_LIST_OFFSET, BigInt(ffi.ptr(attributes)), true) + + // Hand the child our own std handles, and let it inherit them. + // + // `bInheritHandles: false` was silently fatal in a way that looked exactly + // like a containment failure. The launcher is spawned with its stdout on a + // PIPE, and a child inheriting nothing has nowhere to write, so every + // sandboxed command produced empty output. The first Windows self-test read + // that empty stdout, found no package SID in it, and reported the container + // as not applied — when the token may have been correct and merely + // unreadable. Two different bugs with one observable, which is precisely + // what the token check was added to prevent, so the check now reports the + // child's exit status and stderr as well. + // + // This is not a test-only concern. Every sandboxed command's output crosses + // this boundary: pip's progress, a bash tool's result, a kernel's stream. + const inherit = (id: number) => { + const h = kernel.GetStdHandle(id) as number + // GetStdHandle answers 0 for "none" and INVALID_HANDLE_VALUE for failure; + // the latter is -1, which arrives here as an unsafe integer. + if (!Number.isSafeInteger(h) || h <= 0) return 0n + // Inheritance is a property of the handle in THIS process, and the ones + // we were given are not necessarily marked for it. + kernel.SetHandleInformation(h as never, HANDLE_FLAG_INHERIT, HANDLE_FLAG_INHERIT) + return BigInt(h) + } + const stdout = inherit(STD_HANDLES.output) + const stderr = inherit(STD_HANDLES.error) + // Only claim the handles when there is something to claim: with the flag + // set and a null handle the child gets no stdout at all, which is the very + // failure this replaces. Without it the child attaches to our console, + // which is the right fallback when we have one. + if (stdout && stderr) { + startupView.setUint32(STARTUPINFO_FLAGS_OFFSET, STARTF_USESTDHANDLES, true) + startupView.setBigUint64(STARTUPINFO_STDIN_OFFSET, inherit(STD_HANDLES.input), true) + startupView.setBigUint64(STARTUPINFO_STDOUT_OFFSET, stdout, true) + startupView.setBigUint64(STARTUPINFO_STDERR_OFFSET, stderr, true) + } + + const info = new Uint8Array(PROCESS_INFORMATION_SIZE) + // Mutable: CreateProcessW may write into lpCommandLine. + const line = wide(commandLine(argv)) + // lpCurrentDirectory, explicitly. + // + // Passing null means "inherit the parent's", which is a path chosen by + // whoever launched us and is routinely one the container has no grant for. + // The child is then created successfully and dies on its first syscall — + // for bun, `error loading current directory` and nothing else. Setting the + // parent's cwd instead of this argument was not enough, so the value the + // kernel receives is now the value we chose, and it is logged. + const directory = cwd ? wide(cwd) : undefined + keep.push(startup, info, line) + if (directory) keep.push(directory) + say(`current directory ${cwd ?? ""}`) + // The whole STARTUPINFOEX as the kernel will read it. cb must be 0x70 (112) + // in the first four bytes, and the attribute-list pointer must be non-zero + // at offset 104 — if either is wrong, CreateProcess ignores the list and + // succeeds anyway, which is precisely the failure being chased. + say(`startupinfoex ${bytes(startup)}`) + say(` cb=${new DataView(startup.buffer).getUint32(STARTUPINFO_CB_OFFSET, true)} (expect ${STARTUPINFOEX_SIZE})`) + say( + ` lpAttributeList=0x${new DataView(startup.buffer).getBigUint64(STARTUPINFO_ATTRIBUTE_LIST_OFFSET, true).toString(16)}`, + ) + say(`commandline ${commandLine(argv)}`) + say(`creationflags 0x${EXTENDED_STARTUPINFO_PRESENT.toString(16)} (EXTENDED_STARTUPINFO_PRESENT)`) + + const ok = kernel.CreateProcessW( + null, + ffi.ptr(line), + null, + null, + true, + // No CREATE_UNICODE_ENVIRONMENT: lpEnvironment below is null, so the child + // inherits ours and the flag would describe a block never supplied. + EXTENDED_STARTUPINFO_PRESENT, + null, + directory ? ffi.ptr(directory) : null, + ffi.ptr(startup), + ffi.ptr(info), + ) + say(`CreateProcessW -> ${ok} (Win32 ${ok ? 0 : kernel.GetLastError()})`) + // Ask the KERNEL whether the child is contained, rather than asking the + // child to introspect itself. + // + // The self-test ran `whoami /groups` and pattern-matched its output, which + // made containment depend on a command succeeding INSIDE the container. On a + // CI runner it does not: `whoami /groups` resolves SIDs to display names + // through LSA, which an AppContainer with zero capabilities cannot reach, so + // it exits 66 with no output — while `exit 7` through the identical plan + // returns 7, proving the container hosts processes perfectly well. Two + // rounds were spent reading that as a containment failure. + // + // We hold the process handle, so TokenIsAppContainer answers directly and + // cannot be confounded by what the child can or cannot do. Queried before + // the wait: the handle keeps the process object alive either way, but a + // token query on a live process is the case Windows documents. + if (ok && process.env["OPENSCIENCE_APPCONTAINER_REPORT"] === "1") { + const child = ffi.read.ptr(ffi.ptr(info), PI_PROCESS_OFFSET) + const tokenOut = new BigUint64Array(1) + if (advapi.OpenProcessToken(child as never, TOKEN_QUERY, ffi.ptr(tokenOut))) { + const token = ffi.read.ptr(ffi.ptr(tokenOut), 0) + const valueOut = new Uint32Array(1) + const lenOut = new Uint32Array(1) + const read = advapi.GetTokenInformation( + token as never, + TOKEN_IS_APP_CONTAINER, + ffi.ptr(valueOut), + 4, + ffi.ptr(lenOut), + ) + process.stderr.write(`openscience[appcontainer] token appcontainer=${read ? valueOut[0] : "?"}\n`) + kernel.CloseHandle(token as never) + } else { + process.stderr.write( + `openscience[appcontainer] token appcontainer=? (OpenProcessToken Win32 ${kernel.GetLastError()})\n`, + ) + } + } + // Only now is the attribute list dead. Referenced here so nothing above can + // be considered unreachable while the kernel still holds pointers into it. + kernel.DeleteProcThreadAttributeList(ffi.ptr(attributes)) + keep.length = 0 + if (!ok) { + throw new Error( + `CreateProcess into the AppContainer failed: Win32 ${kernel.GetLastError()}. ` + + `Win32 5 is access denied; 2 means the executable was not found; ` + + // 203 was the one actually hit on a real machine, and it was not in + // this list, so the number carried no meaning at the point of failure. + // lpApplicationName is null, so Windows resolves argv[0] itself and + // needs an environment to do it in. + `203 is ERROR_ENVVAR_NOT_FOUND, which points at the environment this ` + + `process was given rather than at the command.`, + ) + } + + const handle = ffi.read.ptr(ffi.ptr(info), PI_PROCESS_OFFSET) + const waited = kernel.WaitForSingleObject(handle as never, INFINITE) + const codeOut = new Uint32Array(1) + const got = kernel.GetExitCodeProcess(handle as never, ffi.ptr(codeOut)) + // The debug trail stopped at CreateProcess, so a child that started and then + // died told us only "no output". On a CI runner the child exits 66 with + // nothing on either stream, where the same build on a developer machine + // produces a Low-integrity token — so what happens BETWEEN start and exit is + // the whole question. WaitForSingleObject answers 0 for a real exit and + // 0x102 for a timeout we never asked for; a false GetExitCodeProcess means + // the code below is not the child's at all. + say(`wait -> ${waited}, GetExitCodeProcess -> ${got}, child exit ${codeOut[0]}`) + kernel.CloseHandle(handle as never) + return codeOut[0]! + } + + /** + * The `__appcontainer-launch` entry point: decode the spec, ensure the + * profile, grant the workspace, run the real command, propagate its exit + * code. + * + * Grant failures are reported on stderr rather than thrown. The command + * should still run and fail visibly at the write it cannot make, rather than + * disappearing behind a launcher error that says nothing about what the user + * actually asked for. + */ + export async function main(blob: string, argv: string[]): Promise { + const spec = decode(blob) + const sid = ensureProfile(spec.profile) + const { failures, unreachable } = grant(sid, spec.writable, spec.readable ?? []) + for (const failure of failures) process.stderr.write(`openscience: could not grant sandbox access to ${failure}\n`) + if (unreachable.length) { + // Almost always ownership: `icacls` can only change an ACL the caller + // owns, and an all-users Python under C:\ belongs to SYSTEM and the + // Administrators group. So a machine-wide interpreter can never be made + // readable to an AppContainer without elevation, which this product does + // not ask for. Say that, with the remedy, rather than letting it surface + // as a Python that cannot start. + throw new Error( + [ + `The sandbox cannot be given read access to: ${unreachable.join(", ")}.`, + "icacls can only change an ACL you own, and an all-users install under C:\\ is owned by SYSTEM.", + "Use a Python you own instead - install it for your user (the python.org installer's", + "'Install for all users' left OFF puts it under %LOCALAPPDATA%\\Programs\\Python), or install uv", + "and let OpenScience provision one. Either lives under your profile, where the grant succeeds.", + ].join("\n"), + ) + } + // Bounded egress: a pipe only this container can open, a broker relaying it + // to the host proxy, and a shim inside the container giving the payload the + // host:port it expects. Each hop was measured before any of it was written; + // see docs/specs/windows-egress-design.md. + const bridge = spec.pipe && spec.proxy ? await bridgeEgress(sid, spec) : undefined + try { + // Without a broker, block here: one process, one wait, nothing to starve. + if (!bridge) return launch(sid, argv, spec.capabilities ?? []) + // With one, the payload needs its own process for the same reason the shim + // already had one. `launch` waits on WaitForSingleObject(INFINITE) through + // FFI, which bun cannot move off the event loop — and the broker's pump is + // a setTimeout in THIS process. So relaying stopped the instant the + // payload started, and pip saw a tunnel that connected and then delivered + // nothing: + // + // ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): + // Read timed out. (read timeout=15)") + // + // `launchDetached` was written for exactly this hazard and applied only to + // the shim. stdout is inherited rather than ignored here because this one + // is the payload, and its output is the point. + return await launchDetached(spec.self!, sid, argv, spec.capabilities ?? [], undefined, "inherit").exited + } finally { + bridge?.stop() + // Always, including when launch throws. The Low label is the part that + // must not outlive the run: it opens the workspace to every low-integrity + // process on the machine, not just to us. + for (const failure of revoke(sid, spec.writable, spec.readable ?? [])) + process.stderr.write(`openscience: could not restore ${failure}\n`) + } + } +} diff --git a/backend/cli/src/sandbox/egress-runtime.ts b/backend/cli/src/sandbox/egress-runtime.ts new file mode 100644 index 00000000..add693d5 --- /dev/null +++ b/backend/cli/src/sandbox/egress-runtime.ts @@ -0,0 +1,293 @@ +import crypto from "crypto" +import fs from "fs/promises" +import path from "path" +import { Config } from "@/config/config" +import { Global } from "@/global" +import { GlobalBus } from "@/bus/global" +import { Event } from "@/server/event" +import { Log } from "@/util/log" +import { Egress } from "./egress" +import { Sandbox } from "./sandbox" + +const log = Log.create({ service: "egress-runtime" }) + +/** + * Lifecycle for the host-side allowlist proxy — the listening end of + * `Egress.serveProxy` (see `egress.ts` for the proxy itself and + * `docs/adr/0002-sandbox-network-policy.md` for why it exists). + * + * One proxy per process, held lazily with a disposer, the same shape + * `science/kernel/registry.ts` uses for its kernel table: nothing runs until + * the first `ensure()`, and `stop()` tears down the server and unlinks the + * socket. Unlike that table, this is not `Instance.state` — the proxy must + * outlive any single project instance, since a global config write disposes + * every open instance (`Config`'s `patchConfigPath`) and the proxy must not + * go down with them, or every kernel bound to its socket would lose its only + * route out. + * + * `rules` is a live array, not a snapshot. `Egress.serveProxy` reads it by + * reference on every connection, so refreshing its *contents* in place — + * on every `ensure()`, and reactively whenever global config changes — is + * what lets an allowlist edit reach a kernel that is already running, + * without restarting the proxy or the kernel. Building a fresh array once + * at construction and handing it to `serveProxy` would silently defeat + * that: the proxy would keep the rules it was born with until the process + * itself restarted. This is also why `allowHosts` stays out of + * `ExecutionAuthority.generation` — that hash exists to decide when a + * kernel must be torn down and rebooted, and an allowlist edit is + * deliberately not that kind of change. + */ +export namespace EgressRuntime { + /** + * Bubblewrap (Linux) carries `socket` — the bind-mounted socket itself is + * the sandboxed process's only route in. Seatbelt (macOS) has no network + * namespace to bind a socket into, so `Egress.serveProxy` listens directly + * on a loopback TCP port instead (see its own doc comment and + * `sandbox.ts`'s `seatbeltProfile`), carried as `hostname`/`secret` — + * `secret` is the per-start `Proxy-Authorization` credential that port + * requires, since a loopback TCP port, unlike a unix socket, carries no + * filesystem permissions of its own. Optional fields rather than a + * discriminated union: every real caller narrows by checking `socket` + * (see `stop()`, `ensure()`, `egressFor()` below), and a union would force + * that same narrowing onto every *test* that reaches these fields too, + * including the bubblewrap-only ones this task must leave unchanged. + */ + type Running = { + socket?: string + hostname?: string + port: number + secret?: string + // Not `ReturnType`: TS resolves that utility + // against an overloaded function's LAST signature only (the TCP one + // here), not a union of all of them — this field needs both, since + // `startBubblewrap`'s `server` really is a `UnixSocketListener`. + server: Bun.UnixSocketListener | Bun.TCPSocketListener + rules: Egress.Rule[] + onGlobalChange: (event: { directory?: string; payload: unknown }) => void + } + + const state: { running?: Promise } = {} + + async function currentRules(): Promise { + const policy = await Config.trustedSandbox() + return [...Egress.DEFAULT_RULES, ...(policy.allowHosts ?? [])] + } + + /** Re-populate `rules` in place (same array reference) rather than + * replacing it, so `Egress.serveProxy`'s closure over that reference + * observes the update on its very next connection. A failed re-read + * (config file briefly unreadable mid-write, for example) keeps + * whatever rules were already live rather than clearing the allowlist. */ + async function refresh(rules: Egress.Rule[]) { + const next = await currentRules().catch((error) => { + log.warn("failed to refresh the sandbox allowlist, keeping the previous rules", { error }) + return undefined + }) + if (!next) return + rules.length = 0 + rules.push(...next) + } + + function isGlobalConfigChange(event: { directory?: string; payload: unknown }): boolean { + if (event.directory !== "global") return false + const payload = event.payload + if (typeof payload !== "object" || payload === null || !("type" in payload)) return false + return payload.type === Event.Disposed.type + } + + function listener(rules: Egress.Rule[]) { + const onGlobalChange = (event: { directory?: string; payload: unknown }) => { + if (!isGlobalConfigChange(event)) return + refresh(rules).catch(() => {}) + } + GlobalBus.on("event", onGlobalChange) + return onGlobalChange + } + + /** Bubblewrap (Linux): a bind-mountable unix socket under the state dir — + * unchanged from before Task 7 (macOS seatbelt support) added the + * loopback-TCP branch below. */ + async function startBubblewrap(): Promise { + const socket = path.join(Global.Path.state, `egress-${process.pid}.sock`) + // A stale socket file from a killed previous process (same pid, unlikely + // but possible after a pid wraparound) would make Bun.listen refuse to + // bind with EADDRINUSE. + await fs.rm(socket, { force: true }) + const rules = await currentRules() + // Bun.listen throws synchronously, with a message ("Failed to listen at + // ") that says nothing about what depends on it. Every sandboxed + // spawn does, so name that here rather than letting a bare bind error + // surface out of an unrelated-looking `bash`/kernel/job call. + const server = (() => { + try { + return Egress.serveProxy({ socket, rules, onEvent: (line) => log.info(line) }) + } catch (e) { + throw new Error( + `Could not start the sandbox allowlist proxy on ${socket}: ${e instanceof Error ? e.message : String(e)}. ` + + `Sandboxed commands need it to reach the network — retry once the path is writable, or set sandbox.network to "deny" or "allow".`, + ) + } + })() + const onGlobalChange = listener(rules) + log.info("egress proxy listening", { socket }) + return { socket, port: Sandbox.SHIM_PORT, server, rules, onGlobalChange } + } + + /** + * Seatbelt (macOS): no network namespace to bind a unix socket into, so + * `Egress.serveProxy` listens directly on a loopback TCP port instead — + * decision 1 of the Task 7 brief, deliberately *not* a host-side bridge + * from TCP to a unix socket (that would just be a second component doing + * what one listener already can). `port: 0` asks the OS for an ephemeral + * port: unlike bubblewrap's `SHIM_PORT`, nothing needs this value fixed in + * advance — no shim script embeds it as a literal, since seatbelt has no + * shim at all (see `sandbox.ts`'s `shimPlan` doc comment) — and a fixed + * port here, with no namespace to keep it private, would collide across + * every concurrently sandboxed process on the machine. + * + * `secret` is generated fresh per proxy start (decision 2): a loopback TCP + * port, unlike a unix socket, carries no filesystem permissions of its + * own, so every request to it must additionally prove it holds this — + * enforced inside `Egress.serveProxy` itself, not here. + */ + async function startSeatbelt(): Promise { + const rules = await currentRules() + const secret = crypto.randomUUID() + const server = (() => { + try { + return Egress.serveProxy({ hostname: "127.0.0.1", port: 0, secret, rules, onEvent: (line) => log.info(line) }) + } catch (e) { + throw new Error( + `Could not start the sandbox allowlist proxy on 127.0.0.1: ${e instanceof Error ? e.message : String(e)}. ` + + `Sandboxed commands need it to reach the network — retry, or set sandbox.network to "deny" or "allow".`, + ) + } + })() + const onGlobalChange = listener(rules) + log.info("egress proxy listening", { hostname: "127.0.0.1", port: server.port }) + return { hostname: "127.0.0.1", port: server.port, secret, server, rules, onGlobalChange } + } + + /** + * `platform` decides which of the two listeners above starts — defaulting + * to the real platform, like every other platform-injectable seam this + * branch added (`Sandbox.backend`, `plan`/`wrapArgv`), so the seatbelt + * branch is exercisable, deterministically, from a machine that has none. + */ + function start(platform: NodeJS.Platform = process.platform): Promise { + // Windows takes the seatbelt listener, not the bubblewrap one. Both of + // those are host-side proxies on TCP loopback with a shared secret, and the + // HOST has no loopback restriction on Windows — only the container does. + // The pipe that crosses the boundary is the broker's job, not this one's. + return platform === "darwin" || platform === "win32" ? startSeatbelt() : startBubblewrap() + } + + /** Start the proxy if it is not already running, and return where to + * reach it. Idempotent — a second call returns the same address without + * restarting anything. Also refreshes the live rules from the current + * config, so a caller composing a new sandboxed argv always gets the + * latest allowlist even between reactive updates. + * + * A failure is loud but never permanent. Caching the promise is what makes + * the success path idempotent, and it would just as happily cache a + * rejection: one transient failure — a state directory briefly unwritable, + * a socket path momentarily taken — would then be replayed to every later + * caller for the life of the process, and since every bash command, + * terminal, kernel and compute job routes through here under the + * "allowlist" default, that is the whole product failing until restart. + * So a rejected start un-caches itself and the next call genuinely + * retries. It still throws rather than degrading to no-proxy: `wrapArgv` + * would reject an "allowlist" policy with no egress socket anyway, and a + * silent downgrade is exactly the failure this feature keeps producing — + * a sandbox that looks like it has bounded egress and in fact has none. + * + * `platform` decides only which listener `start()` picks — see its doc + * comment — and is only ever non-default from a test; every real caller + * (`egressFor` below) leaves it at the real one. The proxy itself is not + * re-created per platform: `state.running` is one proxy for the process + * lifetime, same as before this parameter existed, so a caller that wants + * a differently-platformed proxy exercised must `stop()` first. */ + export async function ensure( + platform: NodeJS.Platform = process.platform, + ): Promise<{ socket?: string; hostname?: string; port: number; secret?: string }> { + const pending = (state.running ??= start(platform)) + const running = await pending.catch((error) => { + if (state.running === pending) state.running = undefined + throw error + }) + await refresh(running.rules) + return { socket: running.socket, hostname: running.hostname, port: running.port, secret: running.secret } + } + + /** Stop the proxy. The CLI process otherwise leaves this running for its + * own lifetime; tests use this to reset between cases. A no-op when + * nothing is running, and — because a caller reaching for the escape + * hatch after a failed start must not be handed that same failure again — + * when the last start rejected. Unlinks the unix socket on bubblewrap; + * seatbelt's loopback listener leaves nothing on disk to clean up. */ + export async function stop() { + const pending = state.running + state.running = undefined + if (!pending) return + const running = await pending.catch(() => undefined) + if (!running) return + GlobalBus.off("event", running.onGlobalChange) + running.server.stop(true) + if (running.socket) await fs.rm(running.socket, { force: true }) + } + + /** The value to pass as `Sandbox.Options.egress`, or `undefined` when the + * proxy would not actually be used: the sandbox is off, network isn't + * "allowlist", or the platform's backend is neither bubblewrap nor + * seatbelt. The shape differs by backend, matching `Options.egress`'s own + * doc comment: bubblewrap gets the bind-mountable unix socket path, since + * the bind-mounted socket itself is the sandboxed process's only route in; + * seatbelt gets `":"` — `buildPolicy` in sandbox.ts is what + * splits that back apart into `Policy.port`/`Policy.secret`, the same + * division of labour it already has for bubblewrap's `Policy.egress`. A + * disabled/deny/allow policy skips starting the proxy entirely — pure + * waste when nothing would ever connect to it. Every `wrapArgv` / + * `plan()` caller should route through this rather than calling `ensure()` + * directly, so a terminal or kernel with network "deny" never pays for a + * proxy it has no way to reach. + * + * `platform` defaults to the real one — the same injectable seam + * `Sandbox.backend`/`plan`/`wrapArgv` use — so the seatbelt branch is + * exercisable, deterministically, from a machine that has none. + * + * `ensure()` caches ONE proxy for the process lifetime (see its doc + * comment); `platform` only decides which listener `start()` picks when + * nothing is running yet. Asking for `"darwin"` after a differently- + * platformed proxy is already cached (a real caller never does this — + * `process.platform` is constant for the life of a process — but a test + * injecting platform explicitly can) silently reuses that cached + * listener instead of starting a seatbelt one. Interpolating a + * bubblewrap `Running`'s missing `secret` into the template literal + * below would then produce the *string* `"undefined"` — truthy, and + * therefore indistinguishable from a real secret to any check that only + * asks whether the value is present. Guarded explicitly rather than + * trusting the interpolation to fail loudly on its own, because it + * doesn't: confirmed by execution (Task 7 fix round 1 review) that it + * silently composes `":undefined"` instead. */ + export async function egressFor( + policy: Sandbox.Options, + platform: NodeJS.Platform = process.platform, + ): Promise { + const { enabled, network } = Sandbox.resolved(policy) + if (!enabled) return undefined + if (network !== "allowlist") return undefined + const b = Sandbox.backend(platform) + if (b === "bubblewrap") return (await ensure(platform)).socket + if (b === "seatbelt" || b === "appcontainer") { + const running = await ensure(platform) + if (!running.secret) { + throw new Error( + "sandbox egress proxy is already running as the bubblewrap (unix-socket) listener, not seatbelt's — " + + "call EgressRuntime.stop() first if a seatbelt proxy is genuinely needed here", + ) + } + return `${running.port}:${running.secret}` + } + return undefined + } +} diff --git a/backend/cli/src/sandbox/egress-shim-entry.ts b/backend/cli/src/sandbox/egress-shim-entry.ts new file mode 100644 index 00000000..1fc33795 --- /dev/null +++ b/backend/cli/src/sandbox/egress-shim-entry.ts @@ -0,0 +1,56 @@ +/** + * Minimal, dependency-light entry point for the sandboxed loopback shim. In + * development `Sandbox.shimPlan()` bundles this file and execs `bun` against + * the bundle — never against `src/index.ts` — because `src/index.ts`'s graph + * pulls in `Global` (an unguarded top-level `await Bun.file(...).write(...)` + * at `src/global/index.ts` — `EROFS` under a read-only source tree) and + * `ModelsDev` (a live fetch at module-eval time). Both run before any argv + * check could skip them and would kill the shim under exactly the + * read-only/no-network conditions this mechanism exists to survive. This + * file imports only `./egress` (nothing but a Bun type) and the marker + * constant below (a single string, no other exports), so evaluating it does + * no I/O beyond the two lines that matter. + * + * `shimPlan()` does not exec this file: it runs `bun build` over it and execs + * the resulting self-contained bundle, because only the bundle's own path has + * to be visible inside the sandbox, where `--tmpfs /tmp` masks whatever it + * covers. So an added import does not have to live anywhere in particular — + * a sibling module and an npm package are equally fine, and the npm case is + * specifically what a package-root bind got wrong before (in this bun + * workspace `node_modules/` is a symlink into the monorepo-root store, + * above the package root: the link was bound, its target was not). + * + * Four things are still not safe to add here, and none of them are about + * where files live. Import-time side effects (a top-level fetch, a top-level + * write) run in the bundle exactly as they would in the source, and would + * reintroduce the failure this file exists to avoid — the shim dies under the + * read-only, no-network conditions it is supposed to survive, with its output + * on /dev/null. Resolving anything from `import.meta.dir`/`url` points at + * `Global.Path.bin`, where the bundle runs, not at this directory. A runtime + * `import(expression)` cannot be inlined by the bundler, so it would resolve + * against a path nothing bound (a literal `import("./x")` is inlined and + * fine). And a dependency that loads a native binding bundles cleanly but + * still `dlopen`s a `.so` at run time, from a path nothing bound and nothing + * checks — see `shimPlan`'s residual list for the measurement. + * + * A compiled release has no separate entry to redirect to — `bun --compile` + * embeds a single one — so it still goes through `index.ts`'s + * `__egress-shim` argv check and therefore still evaluates that full graph. + * See `sandbox.ts`'s `shimPlan` doc comment and the Task 4 report for what + * that leaves reachable in a compiled binary. + * + * `Sandbox.shimScript` composes one call shape for both modes — + * ` __egress-shim ` — because the compiled path needs + * the "__egress-shim" token to dispatch inside `index.ts`'s single-entry + * argv check. This file has no dispatching to do, so it ignores that token + * and reads port/socket positionally from the end instead of assuming a + * fixed prefix, which also means it still works if `shimScript` ever calls + * it without the token. + */ +import { Egress } from "./egress" +import { SHIM_READY_MARKER } from "./egress-shim-marker" + +const [port, socket] = process.argv.slice(-2) +Egress.serveShim({ port: Number(port), socket: socket! }) +await Bun.write(SHIM_READY_MARKER, "").catch(() => {}) +await new Promise(() => {}) diff --git a/backend/cli/src/sandbox/egress-shim-marker.ts b/backend/cli/src/sandbox/egress-shim-marker.ts new file mode 100644 index 00000000..275023cf --- /dev/null +++ b/backend/cli/src/sandbox/egress-shim-marker.ts @@ -0,0 +1,17 @@ +/** + * Readiness marker `Sandbox.shimScript`'s composed wait loop polls for, and + * the `__egress-shim` handler (`index.ts`, `egress-shim-entry.ts`) touches + * once `Egress.serveShim`'s listener is bound. + * + * A single exported constant, not three independently hardcoded copies of + * the same string: the three call sites drifting apart is a silent 3s stall + * on every sandboxed command, not a loud failure, so nothing would catch it + * happening. This file has no other exports and does nothing at import + * time, so importing it (including from `egress-shim-entry.ts`, which must + * stay dependency-light) costs nothing. + * + * Lives under `/tmp` deliberately: `bubblewrapArgs` always mounts `/tmp` as + * a fresh, process-private tmpfs, so a fixed name here can't collide across + * sandboxed processes or persist from a previous run. + */ +export const SHIM_READY_MARKER = "/tmp/.openscience-egress-shim.ready" diff --git a/backend/cli/src/sandbox/egress.ts b/backend/cli/src/sandbox/egress.ts new file mode 100644 index 00000000..59b91d7d --- /dev/null +++ b/backend/cli/src/sandbox/egress.ts @@ -0,0 +1,572 @@ +import type { Socket, SocketHandler } from "bun" + +/** + * Allowlist egress proxy for sandboxed kernels. + * + * `sandbox.network` is otherwise binary: deny (`--unshare-net`) locks out + * NCBI, UniProt, PDB and PyPI, which is most of the product's purpose; allow + * is unrestricted egress. This gives a middle: an allowlist proxy, with no + * direct DNS inside the sandbox — name resolution happens at the proxy. + * + * A bind-mounted unix socket still crosses `--unshare-net`'s network + * namespace, so it is the only route out, and the proxy on the other end + * decides what is reachable. No pasta, no nftables, no root. + * + * Two roles: + * serveProxy — runs on the HOST, speaks HTTP proxy. Listens on a unix + * socket for bubblewrap (Linux), or directly on a loopback TCP + * port for seatbelt (macOS), which has no network namespace to + * bind a socket into — see sandbox.ts's seatbeltProfile. The + * TCP form additionally requires a Proxy-Authorization secret, + * since a loopback port (unlike a unix socket) carries no + * filesystem permissions of its own. + * serveShim — runs INSIDE the sandbox, TCP on loopback → the unix socket, + * because pip/requests/curl take a host:port proxy, not a + * unix path. Bubblewrap-only: seatbelt's serveProxy needs no + * bridge, since it already listens on TCP directly. + * + * Ported from the feasibility spike on `proto/sandbox-allowlist-proxy` + * (`src/sandbox/prototype/proxy.ts`); see that branch's README for the + * measurements behind the design. + */ +export namespace Egress { + export type Rule = string + + /** Exact host, or a leading dot for suffix match: ".ncbi.nlm.nih.gov". */ + export function allowed(host: string, rules: Rule[]): boolean { + const name = host.toLowerCase().split(":")[0] + return rules.some((rule) => { + const value = rule.toLowerCase() + if (value.startsWith(".")) return name === value.slice(1) || name.endsWith(value) + return name === value + }) + } + + export const DEFAULT_RULES: Rule[] = [ + // package registries + "pypi.org", + ".pypi.org", + "files.pythonhosted.org", + ".pythonhosted.org", + "cran.r-project.org", + ".bioconductor.org", + // scientific APIs + ".ncbi.nlm.nih.gov", + ".uniprot.org", + ".rcsb.org", + ".ebi.ac.uk", + ".ensembl.org", + "arxiv.org", + ".arxiv.org", + ] + + /** + * One direction of a bridged pair, with backpressure. + * + * `Socket.write` returns how many bytes the socket actually accepted, and + * that is fewer than the whole chunk the moment the kernel send buffer + * fills. Writing and discarding the count silently drops the remainder: + * measured on this proxy before this existed, a 40 MB transfer arrived as + * 2.6 MB through the proxy alone and 11.9 MB through shim + proxy, while + * the same origin read directly delivered all 40 MB. Small responses fit in + * one buffer and never show it, which is why every test that pushed + * `hello` through passed. + * + * So: queue whatever the destination refused, flush it from the + * destination's own `drain`, and pause the *source* while a backlog exists + * so the queue tracks the slower end's pace instead of growing to the size + * of the transfer. `end()` is deferred until the queue has actually gone + * out — an upstream that closes right after a large body must not truncate + * what is still in flight to the client. + */ + function pump(target: Socket) { + const queue: Buffer[] = [] + const hold = (chunk: Buffer) => { + // Copied, not retained: the buffer handed to a `data` callback belongs + // to the caller for the duration of that call, and this outlives it. + queue.push(Buffer.from(chunk)) + held.source?.pause() + } + const held = { + /** The socket feeding this direction; paused while a backlog exists. */ + source: undefined as Socket | undefined, + ending: false, + send(chunk: Buffer) { + if (queue.length > 0) return hold(chunk) + const wrote = target.write(chunk) + if (wrote >= chunk.length) return + hold(chunk.subarray(Math.max(wrote, 0))) + }, + /** Drive from the target socket's `drain` handler, nowhere else. */ + flush() { + while (queue.length > 0) { + const head = queue[0]! + const wrote = target.write(head) + if (wrote < head.length) { + if (wrote > 0) queue[0] = head.subarray(wrote) + return + } + queue.shift() + } + held.source?.resume() + if (held.ending) target.end() + }, + end() { + held.ending = true + if (queue.length === 0) target.end() + }, + } + return held + } + + type Pump = ReturnType + + /** + * A client connection's progress through the proxy, tracked explicitly + * because `data` is async and Bun does not serialize its handlers: a second + * chunk re-enters `data` while the first is parked on `await Bun.connect`. + * Without a state set *before* that await, the re-entrant call finds no + * link yet, re-parses the same still-buffered head, and dials the origin a + * second time — measured: one client POST whose body followed the head by + * 1/5/10ms produced 2 upstream connections to a local origin and 4 to a + * real remote one, each carrying a duplicate of a non-idempotent request, + * with `toUpstream` left pointing at whichever dial resolved last. + * + * head — still reading the request head + * dialing — head parsed and allowed, upstream connect in flight + * linked — bytes flow both ways + * closed — denied, unreachable, or the client went away + * + * `closed` is what a dial in flight checks when it resolves, so a client + * that aborts mid-dial cannot strand an upstream socket nobody will ever + * close. + */ + type Phase = "head" | "dialing" | "linked" | "closed" + + /** What both bridges track per client connection. The shim has no head to + * read, so it simply starts at `dialing`. */ + type Link = { phase: Phase; toClient: Pump; toUpstream?: Pump } + + type Pending = Link & { buffer: string } + + const state = new WeakMap, Pending>() + + /** Mark a client gone and release whatever it owns. Setting the phase is + * what a dial still in flight sees when it resolves; without it, the + * socket that dial produces is owned by nobody — `close` has already run + * and found no link to tear down. */ + function shut(held?: Link) { + if (!held) return + held.phase = "closed" + held.toUpstream?.end() + } + + /** Read the phase through a call, not a comparison in place: assigning + * `dialing` earlier in the same scope narrows the property to that literal, + * and the compiler has no way to know `shut` can change it while an + * `await` is parked — which is the entire point of asking. */ + const gone = (held: Link) => held.phase === "closed" + + const refuse = (status: string, reason: string) => + `HTTP/1.1 ${status}\r\nContent-Type: text/plain\r\nConnection: close\r\n\r\n${reason}\n` + + const deny = (reason: string) => refuse("403 Forbidden", reason) + + /** + * Sent only by the TCP/loopback listener (darwin's seatbelt path — see + * `egress-runtime.ts`). A unix socket's access control is the filesystem + * permissions on the path itself; a loopback TCP port has none — every + * process on the machine can dial it — so that listener additionally + * requires a `Proxy-Authorization` header carrying a secret generated once + * per proxy start, and refuses (without forwarding anything) a request + * missing it or carrying the wrong one. `Proxy-Authenticate` names the + * scheme per RFC 7235. + */ + const unauthorized = () => + `HTTP/1.1 407 Proxy Authentication Required\r\nProxy-Authenticate: Basic realm="os"\r\nContent-Type: text/plain\r\nConnection: close\r\n\r\nMissing or invalid Proxy-Authorization\n` + + const latin1 = (text: string) => Buffer.from(text, "latin1") + + /** + * How much of a request head to accept before refusing the connection. + * + * The head phase has no natural end other than `\r\n\r\n`, so a client that + * never sends one is buffered without limit — and because no dial is + * attempted on that path, nothing downstream bounds it either. Measured + * against this proxy before this cap existed: 93 MiB of never-terminated + * head took the *host* process from 36 MB to 1.34 GB of RSS in 8 seconds, + * and it was still climbing when the client stopped. + * + * 64 KiB is Squid's `request_header_max_size` default — the most generous of + * the conventional caps (nginx `large_client_header_buffers` 8k, Apache + * `LimitRequestFieldSize` 8190, Node `--max-http-header-size` 16 KiB) and the + * closest analogue, Squid being a forward proxy that speaks CONNECT. The + * clients here are pip, curl and requests, whose heads run 200-600 bytes, so + * this cannot plausibly refuse a real one. + */ + const HEAD_LIMIT = 64 * 1024 + + /** + * How long to wait for an upstream TCP connect before giving up. + * + * Linux retries a SYN for ~130 s by default, so an allowlisted host that + * black-holes packets — a firewall that drops rather than rejects — pins the + * client connection and its fd for over two minutes and then fails with no + * explanation. 30 s is far above any real handshake, which costs one RTT + * plus name resolution, and turns that wait into a legible 504. + */ + const DIAL_TIMEOUT = 30_000 + + type ServeProxyCommon = { rules: Rule[]; onEvent?: (line: string) => void; dialTimeout?: number } + + /** + * Host side. Proxies only allowlisted hosts. Listens on a unix socket + * (bubblewrap, Linux) or directly on a loopback TCP port with a required + * `secret` (seatbelt, macOS — see the module doc comment and + * `egress-runtime.ts`). Overloaded, not one union signature, so each call + * site gets back the concrete `UnixSocketListener`/`TCPSocketListener` its + * own input shape implies — `egress-runtime.ts`'s seatbelt path reads + * `.port` off the result, which only `TCPSocketListener` has. + * `dialTimeout` overrides `DIAL_TIMEOUT`; it exists so the timeout can be + * exercised in milliseconds rather than by making a test wait half a + * minute for the real one. + */ + export function serveProxy(input: ServeProxyCommon & { socket: string }): Bun.UnixSocketListener + export function serveProxy( + input: ServeProxyCommon & { hostname: string; port: number; secret: string }, + ): Bun.TCPSocketListener + export function serveProxy( + input: ServeProxyCommon & ({ socket: string } | { hostname: string; port: number; secret: string }), + ) { + const log = input.onEvent ?? (() => {}) + const budget = input.dialTimeout ?? DIAL_TIMEOUT + // Set only by the TCP/loopback listener — see `unauthorized` above for why. + const authorization = + "secret" in input ? `Basic ${Buffer.from(`os:${input.secret}`).toString("base64")}` : undefined + + // Branched, and the handlers built once and passed to whichever branch + // fires, rather than a spread of the two option shapes into one object: + // Bun.listen is overloaded on unix vs hostname/port, and a union spread + // matches neither overload (the same reason this file's own tests branch + // Bun.connect for the mirror image of this call). + const listen = (socket: SocketHandler) => + "socket" in input + ? Bun.listen({ unix: input.socket, socket }) + : Bun.listen({ hostname: input.hostname, port: input.port, socket }) + + return listen({ + open(client) { + state.set(client, { buffer: "", phase: "head", toClient: pump(client) }) + }, + async data(client, chunk) { + const held = state.get(client) + if (!held) return + if (held.phase === "linked") { + held.toUpstream?.send(chunk) + return + } + if (held.phase === "closed") return + + // Everything before the link is one buffer, so body bytes that land + // while the dial is in flight are simply still here when it + // resolves — `rest` is sliced after the await, not before it. + held.buffer += chunk.toString("latin1") + if (held.phase === "dialing") return + const end = held.buffer.indexOf("\r\n\r\n") + if (end === -1) { + if (held.buffer.length <= HEAD_LIMIT) return + // Fail closed. Unlike the dial window below there is no + // backpressure to apply here: the terminator is what the parse is + // waiting for, so refusing to read simply deadlocks the connection + // instead of ending it. A head this long is a protocol error. + log(`OVERSIZE ${held.buffer.length} bytes of head with no terminator`) + held.phase = "closed" + held.buffer = "" + held.toClient.send( + latin1(refuse("431 Request Header Fields Too Large", `Proxy request head exceeded ${HEAD_LIMIT} bytes`)), + ) + held.toClient.end() + return + } + + const head = held.buffer.slice(0, end) + const lines = head.split("\r\n") + const request = lines[0] ?? "" + const [method, target, version = "HTTP/1.1"] = request.split(" ") + + // Checked before anything about the request is even inspected for + // validity — an unauthenticated caller learns nothing about + // whether its target was well-formed, let alone allowlisted. + if (authorization) { + const header = lines.slice(1).find((line) => /^proxy-authorization:/i.test(line)) + const provided = header?.slice(header.indexOf(":") + 1).trim() + if (provided !== authorization) { + log(`AUTH missing or invalid Proxy-Authorization`) + held.phase = "closed" + held.buffer = "" + held.toClient.send(latin1(unauthorized())) + held.toClient.end() + return + } + } + + // CONNECT host:443 for TLS; absolute-form GET http://host/path for plain. + const url = + method === "CONNECT" + ? undefined + : (() => { + try { + return new URL(target) + } catch { + return undefined + } + })() + const authority = method === "CONNECT" ? target : url?.host + + if (!authority) { + log(`malformed ${request.slice(0, 60)}`) + held.phase = "closed" + held.toClient.send(latin1(deny("Malformed proxy request"))) + held.toClient.end() + return + } + + if (!allowed(authority, input.rules)) { + log(`DENY ${authority}`) + held.phase = "closed" + held.toClient.send(latin1(deny(`Host not on the sandbox allowlist: ${authority.split(":")[0]}`))) + held.toClient.end() + return + } + + const [hostname, port] = authority.split(":") + // Claim the dial before yielding. Everything above this line is + // synchronous, so no second chunk can be part-way through the same + // parse when it runs. + held.phase = "dialing" + // Backpressure, not a buffer limit. Everything the client sends + // while the dial is in flight would otherwise be held here, and a + // dial can be slow for as long as the OS retries a SYN: measured + // against a black-holed allowlisted origin, 8 seconds of blasting + // took the host process from 36 MB to 2.12 GB and then killed it + // outright with `RangeError: Out of memory` — and this proxy runs in + // the CLI's own process, so that is the supervisor dying at the hands + // of the thing the sandbox exists to contain. + // + // Pausing costs nothing and has no arbitrary limit: the bytes wait in + // the client's own socket buffer, and then in the client. Round 3 + // declined to do this on the grounds that "pausing the client is what + // would stop the FIN that tells us it left" — that is not so, and was + // never measured. A paused socket still reports its peer's departure: + // with delivery demonstrably stopped (0.21 MiB through a paused + // socket against 256 MiB through an unpaused one), the peer's `end()` + // still produced `close` while the pause was in force, for both FIN + // and RST. + client.pause() + // An allowlisted host that black-holes packets otherwise holds this + // connection for the kernel's whole SYN-retry budget. The phase is + // what makes this safe to fire late: `closed` is exactly what the + // dial below checks when it finally resolves, so the socket it + // produces is still ended by nobody-owns-it handling rather than + // stranded. + const timer = setTimeout(() => { + if (held.phase !== "dialing") return + log(`TIMEOUT ${authority}`) + held.phase = "closed" + held.buffer = "" + client.resume() + held.toClient.send(latin1(refuse("504 Gateway Timeout", `Timed out connecting to ${authority}`))) + held.toClient.end() + }, budget) + const upstream = await Bun.connect({ + hostname, + port: Number(port ?? (method === "CONNECT" ? 443 : 80)), + socket: { + data(_sock, payload) { + held.toClient.send(payload) + }, + drain() { + held.toUpstream?.flush() + }, + close() { + held.toClient.end() + }, + error() { + held.toClient.end() + }, + }, + }).catch(() => undefined) + clearTimeout(timer) + + // The client can have gone away while the dial was in flight — or the + // dial can have timed out above — in which case this is the only + // place that can release the socket it just produced. + if (gone(held)) { + upstream?.end() + return + } + + if (!upstream) { + log(`FAIL ${authority}`) + held.phase = "closed" + held.buffer = "" + client.resume() + held.toClient.send(latin1(deny(`Cannot reach ${authority}`))) + held.toClient.end() + return + } + + log(`ALLOW ${authority}`) + const toUpstream = pump(upstream) + toUpstream.source = client + held.toClient.source = upstream + held.toUpstream = toUpstream + held.phase = "linked" + // Sliced now rather than before the dial, so anything the client + // sent while it was in flight goes upstream in arrival order. + const rest = held.buffer.slice(end + 4) + held.buffer = "" + // Resumed before anything is forwarded, not after: `toUpstream` owns + // the client as its source from here, so a forward that has to queue + // re-pauses it through the pump. Resuming afterwards would undo that. + client.resume() + // CONNECT: acknowledge, then the client starts its TLS handshake. + // Plain HTTP: replay the request head we already consumed. + if (method === "CONNECT") { + held.toClient.send(latin1("HTTP/1.1 200 Connection Established\r\n\r\n")) + if (rest) toUpstream.send(latin1(rest)) + return + } + + // A proxy must rewrite absolute-form to origin-form. Forwarding + // `GET http://pypi.org/simple/ HTTP/1.1` verbatim is legal per RFC 7230 + // §5.3.2 but origin servers routinely reject it — measured: 403 from + // pypi.org on the plain-HTTP path while CONNECT to the same host + // returned 200. Also drop hop-by-hop `Proxy-*` headers, which are for + // us and must not travel upstream. + const origin = `${url!.pathname}${url!.search}` || "/" + const headers = lines + .slice(1) + .filter((line) => !/^proxy-/i.test(line)) + .filter((line) => !/^host:/i.test(line)) + const rewritten = [`${method} ${origin} ${version}`, `Host: ${url!.host}`, ...headers].join("\r\n") + toUpstream.send(latin1(`${rewritten}\r\n\r\n${rest}`)) + }, + drain(client) { + state.get(client)?.toClient.flush() + }, + close(client) { + shut(state.get(client)) + state.delete(client) + }, + error(client) { + shut(state.get(client)) + state.delete(client) + }, + }) + } + + /** + * Sandbox side. pip, requests and curl take `http://host:port` from + * HTTP_PROXY — none of them speak unix-socket proxies — so a loopback + * listener inside the namespace forwards raw bytes to the bind-mounted + * socket. + */ + export function serveShim(input: { port: number; socket: string }) { + // `pending` is for the window *before* the link exists — distinct from the + // backpressure queue inside `pump`, which is for after it does. `open` is + // async, so a client that writes immediately — curl sends CONNECT the + // moment the TCP handshake completes — arrives before the upstream link + // exists. Without this buffer those bytes are dropped and the connection + // hangs: the listener accepts, nothing is ever forwarded, and the client + // times out with the socket showing LISTEN the whole time. + // + // It is now a safety net rather than the main path: `open` pauses the + // client before it yields, so in practice nothing is delivered into + // `pending` at all. Keeping it costs nothing and is what stops a byte from + // being dropped should anything ever slip through ahead of the pause — + // dropping one here does not fail loudly, it hangs the connection. + // + // The `closed` phase covers the mirror image: a client that goes away + // *during* that same window. Its `close` runs while `toUpstream` is still + // undefined, so it has nothing to tear down, and the socket the dial then + // produces is owned by nobody. Measured on 300 connect-then-immediately- + // close connections, that stranded one fd per connection in this process + // and one in the host proxy on the other end of it — and a kernel or a + // terminal is a sandbox that lives for hours. + type Bridge = Link & { pending: Buffer[] } + const links = new WeakMap, Bridge>() + + return Bun.listen({ + hostname: "127.0.0.1", + port: input.port, + socket: { + async open(client) { + const held: Bridge = { pending: [], phase: "dialing", toClient: pump(client) } + links.set(client, held) + // The same backpressure the host proxy applies around its own dial, + // and for the same reason: without it a client that starts blasting + // before the link exists is buffered in `pending` without limit. The + // blast radius is smaller here — the shim lives inside the sandbox, + // so it is the sandbox's own memory — but it is the same defect, and + // an unbounded shim would in any case hand the whole blast to the + // host proxy the moment the link came up. + client.pause() + const upstream = await Bun.connect({ + unix: input.socket, + socket: { + data(_sock, payload) { + held.toClient.send(payload) + }, + drain() { + held.toUpstream?.flush() + }, + close() { + held.toClient.end() + }, + error() { + held.toClient.end() + }, + }, + }).catch(() => undefined) + if (gone(held)) { + upstream?.end() + held.pending.length = 0 + return + } + if (!upstream) { + client.resume() + held.toClient.end() + return + } + const toUpstream = pump(upstream) + toUpstream.source = client + held.toClient.source = upstream + held.toUpstream = toUpstream + held.phase = "linked" + // Before the replay, for the reason given in `serveProxy`: a queueing + // send re-pauses the client through the pump, and resuming after + // would undo it. + client.resume() + for (const chunk of held.pending) toUpstream.send(chunk) + held.pending.length = 0 + }, + data(client, chunk) { + const held = links.get(client) + if (!held || gone(held)) return + if (held.toUpstream) return void held.toUpstream.send(chunk) + held.pending.push(Buffer.from(chunk)) + }, + drain(client) { + links.get(client)?.toClient.flush() + }, + close(client) { + shut(links.get(client)) + }, + error(client) { + shut(links.get(client)) + }, + }, + }) + } +} diff --git a/backend/cli/src/sandbox/fastpath.ts b/backend/cli/src/sandbox/fastpath.ts new file mode 100644 index 00000000..ab13c021 --- /dev/null +++ b/backend/cli/src/sandbox/fastpath.ts @@ -0,0 +1,85 @@ +/** + * The three ways the binary re-enters itself for the sandbox, handled before + * anything else in the process exists. + * + * These used to sit at the top of `src/index.ts`, guarded by a comment saying + * they ran "before any other CLI machinery is reached". They did not. ESM + * hoists and evaluates every static import before the first statement of the + * importing module, so `import { OpenScience } from "./openscience"` and its + * neighbours had already pulled in `project/bootstrap` -> `plugin` -> `server` + * -> `global`, and `global` creates the user's data, config, state, log and bin + * directories in a top-level await. + * + * Inside an AppContainer none of those paths is reachable, so the egress SHIM — + * which is this binary, re-entered in the container — died during module + * evaluation, before a single line of its own code ran: + * + * EEXIST: file already exists, mkdir 'C:\Users\\.local\state\openscience' + * at async (src/global/index.ts:105:15) + * at async (src/server/server.ts:43:1) + * at async (src/plugin/index.ts:12:1) + * at async (src/project/bootstrap.ts:23:1) + * + * That is why the proxy inside the container was a dead port and DNS failed + * there: the thing serving it was never alive. A comment cannot enforce import + * order — a module can, so the checks live here and this is imported first. + * + * Everything reachable from here is loaded with `await import()` for the same + * reason. Keep it that way. + */ + +// The egress shim: opens a listener inside the sandbox and forwards bytes to +// the host's proxy. It must never touch the log file (the namespace bubblewrap +// builds is read-only — EROFS) or the network (unshared but for one socket, so +// a refresh check hangs). It also never returns: the hanging await below is +// what stops the rest of the import graph from evaluating behind it. +if (process.argv[2] === "__egress-shim") { + const { Egress } = await import("./egress") + const { SHIM_READY_MARKER } = await import("./egress-shim-marker") + Egress.serveShim({ port: Number(process.argv[3]), socket: process.argv[4] as string }) + await Bun.write(SHIM_READY_MARKER, "").catch(() => {}) + await new Promise(() => {}) +} + +// Windows containment is applied AT process creation rather than by a wrapper +// executable, so the binary launches itself into the AppContainer and execs the +// real command there. +// +// The shim has to run INSIDE the container and never exits on its own, while +// `AppContainer.launch` blocks on WaitForSingleObject. bun:ffi cannot move that +// wait off the event loop, so the shim gets a helper process whose only job is +// to hold it. +if (process.argv[2] === "__appcontainer-detached") { + const { AppContainer } = await import("./appcontainer") + const sid = process.argv[3] as string + const capabilities = JSON.parse(process.argv[4] as string) as string[] + // Empty means "inherit", which is only ever right when the caller knows the + // container can reach wherever this process happens to have started. + const cwd = process.argv[5] || undefined + const code = await Promise.resolve( + AppContainer.launch(sid, process.argv.slice(6), capabilities, undefined, cwd), + ).catch((error: Error) => { + process.stderr.write(`openscience: ${error.message}\n`) + return 1 + }) + process.exit(code) +} + +if (process.argv[2] === "__appcontainer-launch") { + const rest = process.argv.slice(3) + const split = rest.indexOf("--") + // Argv is checked before the import, so a malformed invocation answers the + // same way everywhere. `appcontainer` opens Windows system libraries. + if (split === -1) { + process.stderr.write("openscience: __appcontainer-launch requires -- \n") + process.exit(2) + } + const { AppContainer } = await import("./appcontainer") + const code = await AppContainer.main(rest[0] as string, rest.slice(split + 1)).catch((error: Error) => { + process.stderr.write(`openscience: ${error.message}\n`) + return 1 + }) + process.exit(code) +} + +export {} diff --git a/backend/cli/src/sandbox/sandbox.ts b/backend/cli/src/sandbox/sandbox.ts index 2b2c28ba..623d6faa 100644 --- a/backend/cli/src/sandbox/sandbox.ts +++ b/backend/cli/src/sandbox/sandbox.ts @@ -1,10 +1,14 @@ import path from "path" import os from "os" import fs from "fs" +import { createHash, randomBytes } from "crypto" import { spawn, spawnSync } from "child_process" import { lazy } from "@/util/lazy" import { Log } from "@/util/log" import { Shell } from "@/shell/shell" +import { Global } from "@/global" +import { Installation } from "@/installation" +import { SHIM_READY_MARKER } from "./egress-shim-marker" const log = Log.create({ service: "sandbox" }) @@ -28,7 +32,7 @@ const log = Log.create({ service: "sandbox" }) * root; mounting the host root, even read-only, would defeat read isolation. */ export namespace Sandbox { - export type Backend = "seatbelt" | "bubblewrap" | "none" + export type Backend = "seatbelt" | "bubblewrap" | "appcontainer" | "none" export interface Policy { /** Absolute paths the sandboxed process may write to. */ @@ -47,8 +51,59 @@ export namespace Sandbox { readableAliases?: MountAlias[] writableAliases?: MountAlias[] unreadableAliases?: MountAlias[] - /** Whether the sandboxed process may reach the network. */ - network: boolean + /** How the sandboxed process may reach the network. */ + network: "deny" | "allowlist" | "allow" + /** + * Unix socket that is the only egress route on Linux — bubblewrap's + * `--unshare-net` severs everything else. Required when network is + * "allowlist" and the backend is bubblewrap. + */ + egress?: string + /** + * TCP loopback port that is the only egress route on macOS. Seatbelt has + * no network namespace to sever, so there is no socket to bind-mount — + * `seatbeltProfile` instead narrows `network-outbound` to this one port + * (see its doc comment). Required when network is "allowlist" and the + * backend is seatbelt; carried on `Policy` rather than read from ambient + * state so profile generation stays a pure function of its input. + */ + port?: number + /** + * The `Proxy-Authorization` secret the loopback proxy requires on macOS + * — a TCP port, unlike `egress`'s unix socket, carries no filesystem + * permissions of its own, so `plan`/`wrapArgv` embed this in the proxy + * URL (`http://os:@127.0.0.1:`) rather than pointing the + * sandboxed process at an unauthenticated one. Not consumed by + * `seatbeltProfile` itself — the profile only narrows the network layer + * to `port`; the secret is enforced by `Egress.serveProxy` on the other + * end. Set together with `port` or not at all (see `buildPolicy`). + */ + secret?: string + /** + * AppContainer profile name on Windows. Containment there is anchored to a + * package SID rather than a namespace or a profile document: the SID is + * derived from this name, filesystem ACEs are granted to it, and the broker + * pipe's DACL names it. Required when the backend is "appcontainer". + * + * Derived from the workspace rather than passed in, so the same project + * gets the same SID across runs — ACLs granted once stay meaningful, and + * `CreateAppContainerProfile` is idempotent given a stable name. + */ + profile?: string + /** + * Read-only paths to bind into the namespace after `--tmpfs /tmp`, so + * they stay reachable regardless of where they happen to live on the + * host — including under `/tmp`, which `--tmpfs /tmp` otherwise masks + * unconditionally, `--ro-bind / /` notwithstanding. Used for the egress + * shim's executable — in dev, the generated launcher and the bundle it + * runs — and the interpreter that launcher execs. + */ + readBind?: string[] + /** Argv that starts the egress shim INSIDE an AppContainer, when one is + * needed. A release re-enters its own binary; from source it is bun plus + * the bundled shim entry, which is why it cannot be re-derived launcher-side + * from `process.execPath` alone. Its paths are added to `readable`. */ + shim?: string[] } export interface MountAlias { @@ -65,11 +120,51 @@ export namespace Sandbox { /** User-facing config knobs (mirrors Config.Sandbox, kept dependency-free). */ export interface Options { enabled?: boolean - network?: "allow" | "deny" + network?: "deny" | "allowlist" | "allow" + /** + * Address of the only egress route, in whatever shape the resolved + * backend needs: a bind-mountable unix socket path for bubblewrap, or + * `":"` for seatbelt (`EgressRuntime.egressFor` is the one + * producer, and it returns one string either way). `buildPolicy` is what + * interprets this per backend, into `Policy.egress` or + * `Policy.port`/`Policy.secret` respectively. + */ + egress?: string allowWrite?: string[] onUnavailable?: "warn" | "error" | "allow" } + /** + * The `enabled`/`network` an `Options` resolves to — the one place that + * answers both questions, so `decide()` and `buildPolicy()` below and + * `EgressRuntime.egressFor()` (which has to precompute the socket that will + * become `options.egress` *before* either of them runs) can't quietly + * disagree on what "unset" means. They did: a missing `enabled` used to + * read as off in `decide()` and on in `egressFor()`, and a missing + * `network` used to read as `"allowlist"` in `buildPolicy()` and not in + * `egressFor()` — each divergence invisible from the five production + * callers, all of which pass an already-fully-resolved policy, but real for + * any caller that doesn't. + * + * A wholly missing `Options` stays off: `enabled` requires an explicit + * `true`, matching `decide()`'s existing contract (see the "no options → + * runs the raw command unchanged" test in sandbox.test.ts) — this module is + * dependency-free and does not itself default a caller into being + * sandboxed. `network` unset defaults to `"allowlist"`, matching + * `buildPolicy()` and `Config.trustedSandbox()`. + */ + export function resolved(options?: Options): { enabled: boolean; network: "deny" | "allowlist" | "allow" } { + return { + enabled: options?.enabled === true, + // "deny", matching Config.trustedSandbox. These two defaults must agree: + // while they disagreed, every caller that omitted `network` resolved to + // "allowlist" here and then threw in bubblewrapArgs for want of an egress + // socket nobody had asked for. Failing closed is also the right default + // for a value that decides whether a sandboxed process reaches the network. + network: options?.network ?? "deny", + } + } + export interface Plan { /** Program to spawn. */ file: string @@ -84,6 +179,13 @@ export namespace Sandbox { temporary?: string /** One-time human-readable note (e.g. sandbox requested but unavailable). */ warning?: string + /** + * Proxy variables the caller must set on the child's env for the loopback + * shim to be used. Present only when the command was actually wrapped + * through the shim (bubblewrap, network "allowlist", a usable egress + * socket) — same condition as `Wrapped.env`. + */ + env?: Record } /** Result of wrapping a raw argv (used by the notebook/R kernels). */ @@ -97,6 +199,12 @@ export namespace Sandbox { /** Unique owner-only host temp directory granted only to this process. */ temporary?: string warning?: string + /** + * Proxy variables the caller must set on the child's env for the loopback + * shim to be used. Present only when the argv was actually wrapped through + * the shim (bubblewrap, network "allowlist", a usable egress socket). + */ + env?: Record } export class UnavailableError extends Error { @@ -114,7 +222,7 @@ export namespace Sandbox { // --unshare-pid needs a usable PID namespace. Probe with the same namespace // ops the real sandbox uses so detection matches enforcement. try { - const res = spawnSync(bin, [...bubblewrapArgs({ writable: [], network: false }), "--", "/usr/bin/true"], { + const res = spawnSync(bin, [...bubblewrapArgs({ writable: [], network: "deny" }), "--", "/usr/bin/true"], { stdio: "ignore", timeout: 5000, }) @@ -133,18 +241,123 @@ export namespace Sandbox { if (!bin) return "none" return probeBubblewrap(bin) ? "bubblewrap" : "none" } + if (process.platform === "win32") { + // Probed, never assumed. `AppContainer.usable()` loads the DLLs and + // derives a SID — side-effect free, and it catches the failure that + // matters: FFI bindings that do not resolve, which would mean composing a + // sandbox that is never actually applied. Anything it cannot prove falls + // back to "none", which is the behaviour Windows had before this existed. + const { AppContainer } = require("./appcontainer") as typeof import("./appcontainer") + return AppContainer.usable() ? "appcontainer" : "none" + } return "none" }) - /** The sandbox backend usable on this machine right now, or "none". */ - export function backend(): Backend { - return detected() + /** + * The sandbox backend for `platform`, defaulting to this machine's real + * one right now. + * + * For the default (or an explicitly-matching) `platform` this is exactly + * `detected()` — cached, and probed for real (`Bun.which`, + * `probeBubblewrap`) — so every existing zero-arg caller is unaffected. + * + * An explicitly *different* platform is the seam that lets the seatbelt + * code paths in `plan`/`wrapArgv`/`EgressRuntime` be exercised from Linux, + * where no Mac exists to install `sandbox-exec` on or probe for: probing a + * binary that cannot be present on the machine actually running the test + * would just report "none" and defeat the whole point. So a mismatched + * platform skips probing and assumes the backend that platform normally + * has — `sandbox-exec` ships with every macOS install, `bwrap` is what the + * real Linux branch above already probes for — trading "verified installed + * here" for "what plan()/wrapArgv() would compose for that platform", + * which is the property these tests actually need. + */ + export function backend(platform: NodeJS.Platform = process.platform): Backend { + if (platform === process.platform) return detected() + if (platform === "darwin") return "seatbelt" + if (platform === "linux") return "bubblewrap" + // An INJECTED win32 resolves to "appcontainer" so the Windows composition + // can be built and tested from a machine that is not Windows, exactly as + // the seatbelt paths were built from Linux. + // + // `detected()` above deliberately still answers "none" on a real Windows + // machine, and must keep doing so until the launcher exists. Flipping the + // live probe first would make `available()` true and have the product claim + // a sandbox it cannot actually apply — strictly worse than today's honest + // refusal to run kernels there. + if (platform === "win32") return "appcontainer" + return "none" } export function available(): boolean { return backend() !== "none" } + /** + * Can the sandbox reach this path at all — not "should it", but "is it + * possible on this machine"? + * + * On POSIX, yes: a namespace can bind anything. On Windows, access comes from + * an ACE naming the container's package SID, and `icacls` can only modify an + * ACL you OWN — so a path under `C:\Program Files` is unreachable no matter + * the policy, without elevation this product does not ask for. The one + * exception is the System32 subtree, which Windows already ships with an + * `ALL APPLICATION PACKAGES` ACE so that AppContainers can load system DLLs + * and run system binaries. + * + * Distinct from `Installer.grantable`, which asks whether WE can write a new + * ACE (ownership only). This asks whether the container can reach the path at + * all, which includes ACEs that are already there. + */ + export function reachable(target: string, platform: NodeJS.Platform = process.platform): boolean { + if (platform !== "win32") return true + const root = process.env["SystemRoot"] ?? "C:\\Windows" + const system = path.win32.join(root, "System32").toLowerCase() + const value = target.toLowerCase() + if (value.startsWith(system + path.win32.sep) || value === system) return true + const home = (process.env["USERPROFILE"] ?? os.homedir()).toLowerCase() + return !!home && value.startsWith(home + path.win32.sep) + } + + /** + * A shell the sandbox can actually execute. + * + * `Shell.acceptable()` answers a different question — the nicest shell on this + * machine — and on Windows with Git installed that is + * `C:\Program Files\Git\bin\bash.exe`, an MSYS2 program under a directory no + * ACE can be added to. The container cannot load `msys-2.0.dll`, so bash dies + * at `0xC0000142` (STATUS_DLL_INIT_FAILED) before running anything, and every + * sandboxed command fails for a reason that names neither the shell nor the + * sandbox. + * + * So the sandbox picks. Windows PowerShell 5.1 first — it lives in System32, + * which already carries the ACE, and `Shell.invocation` already knows to drive + * it with `-NoProfile -Command`. NOT `pwsh.exe`: PowerShell 7 installs under + * `C:\Program Files` and is exactly the same trap. + * + * When nothing is being confined, the host's own preference is returned + * unchanged — an unsandboxed run has no reason to lose Git Bash. + */ + export function shell(options?: Options, platform: NodeJS.Platform = process.platform): string { + const chosen = Shell.acceptable() + if (!resolved(options).enabled || backend(platform) === "none") return chosen + if (reachable(chosen, platform)) return chosen + if (platform === "win32") { + const root = process.env["SystemRoot"] ?? "C:\\Windows" + const candidates = [ + path.win32.join(root, "System32", "WindowsPowerShell", "v1.0", "powershell.exe"), + path.win32.join(root, "System32", "cmd.exe"), + ] + const usable = candidates.find((candidate) => fs.existsSync(candidate)) ?? candidates[1]! + if (!warned.shell) { + warned.shell = true + log.warn("sandbox cannot execute the preferred shell; using a system one", { chosen, usable }) + } + return usable + } + return chosen + } + /** Backend + platform summary for status output (CLI `doctor`, GUI panel). */ export function describe(): { platform: NodeJS.Platform @@ -176,6 +389,23 @@ export namespace Sandbox { tool: "bwrap", } } + // Widening `Backend` without widening this left `sandbox status` reporting + // "unavailable - no sandbox backend for platform win32" on a machine where + // `backend()` had already resolved to "appcontainer" and `sandbox test` was + // happily printing it. Two commands, same function, opposite answers. + if (b === "appcontainer") { + return { + platform: process.platform, + backend: b, + available: true, + // ACL-based rather than namespace-based, but the same contract: nothing + // is reachable unless the package SID was granted it, and zero + // capabilities means no network at all. + readIsolation: "grant_only", + networkIsolation: "deny_all", + tool: "AppContainer", + } + } const reason = process.platform === "darwin" ? "sandbox-exec not found on PATH" @@ -411,6 +641,16 @@ export namespace Sandbox { return [...roots] } + /** + * True when `p` is exactly `root` or lies inside it. Both arguments must + * already be `dedupe()`-normalized (`path.resolve()`d) — this does exact + * string comparison, the same convention `tooBroadToConfine` uses for the + * same reason. + */ + function isWithin(p: string, root: string): boolean { + return p === root || p.startsWith(root + path.sep) + } + /** * A path too broad to ever be a sandbox writable root: granting write here * would hand back most of the filesystem and defeat containment. Guards @@ -451,7 +691,18 @@ export namespace Sandbox { return canonical } - /** Assemble the writable allowlist for a policy, dropping over-broad roots. */ + /** + * Assemble the writable allowlist for a policy, dropping over-broad roots, + * and route `options.egress` to whichever of `Policy.egress`/`Policy.port` + * the resolved `backend` actually consumes. + * + * `backend` is required (not read from ambient state) for the same reason + * `plan`/`wrapArgv` take a `platform` parameter: it is what makes the + * seatbelt branch here exercisable from Linux, and it is also simply + * correct — the caller already resolved it before deciding whether to + * sandbox at all, and re-deriving it here from `process.platform` would + * silently disagree with that decision on an injected platform. + */ function buildPolicy(input: { workspace: string[] temporary: string @@ -460,6 +711,10 @@ export namespace Sandbox { unreadable?: string[] entrypoints?: string[] options: Options + backend: Backend + /** Defaults to the real platform; passed explicitly so the Windows policy + * is reachable from a Linux test, the way `plan`/`wrapArgv` already are. */ + platform?: NodeJS.Platform }): Policy { const writableInputs = [ ...input.workspace, @@ -483,7 +738,10 @@ export namespace Sandbox { ] const readable = dedupe(readableInputs).filter((value) => !tooBroadToConfine(value)) const unreadableInputs = input.unreadable ?? [] - return { + // Three-state, not main's boolean: "allowlist" is the whole point of this + // branch and it is neither "no network" nor "the host's network". + const network = resolved(input.options).network + const base = { writable, readable, readableExact: traversalRoots(readable), @@ -491,7 +749,159 @@ export namespace Sandbox { readableAliases: mountAliases(readableInputs), writableAliases: mountAliases(writableInputs), unreadableAliases: mountAliases(unreadableInputs), - network: (input.options.network ?? "allow") !== "deny", + network, + } + + // Seatbelt's egress route is a bare TCP loopback port plus the + // `Proxy-Authorization` secret that port requires (see seatbeltProfile + // and Options.egress's doc comment), not a filesystem path — + // options.egress here is ":", and none of the path + // machinery below (dedupe's path.resolve, tooBroadToConfine) applies to + // it: resolving "52341:abc" against cwd would silently turn it into an + // absolute path and corrupt it. Port and secret are validated and + // dropped together — a port with no secret would compose a proxy URL + // seatbelt's own proxy always rejects, which is a confusing way to fail + // compared to the same "allowlist requires an egress port" throw a wholly + // missing value already produces (seatbeltProfile is the fail-closed + // enforcement point, the same division of labour bubblewrapArgs already + // has with the path branch immediately below). + if (input.backend === "seatbelt") { + const raw = input.options.egress + const at = raw?.indexOf(":") ?? -1 + const port = at > 0 ? Number(raw!.slice(0, at)) : undefined + const secret = at > 0 ? raw!.slice(at + 1) : undefined + const valid = port !== undefined && Number.isInteger(port) && port > 0 && !!secret + if (raw !== undefined && !valid) { + // Only the port half, never the secret: this is a warning, not an + // error path guarded by anything that stops it reaching a log + // sink — logging the credential half here would defeat the whole + // point of requiring one. + log.warn("refusing to grant sandbox egress access to an invalid port/secret pair", { + port: at > 0 ? raw!.slice(0, at) : raw, + }) + } + return { ...base, ...(valid ? { port, secret } : {}) } + } + + // dedupe() applies the same path.resolve() normalization used for + // writable/unreadable above, so a trailing slash, a double slash, or an + // unresolved ".." can't slip an over-broad path past tooBroadToConfine's + // AppContainer's egress is a named pipe, identified by a NAME rather than a + // filesystem path (`\\.\pipe\` is a namespace of its own, not a + // directory). It must skip the path machinery below for exactly the reason + // seatbelt's port:secret does: `dedupe`'s `path.resolve` would silently + // rewrite `openscience-broker-abc` into an absolute path under the current + // directory, and the launcher would then ask for a pipe nobody serves. + if (input.backend === "appcontainer") { + // `egress` arrives as "port:secret", the same shape seatbelt gets, because + // the host-side proxy IS the same one: a TCP listener on loopback, which + // the host may reach freely. What differs is only how the CONTAINER + // reaches it, and that is the broker's pipe rather than a policy value. + const raw = input.options.egress?.trim() + const at = raw ? raw.lastIndexOf(":") : -1 + const port = at > 0 ? Number(raw!.slice(0, at)) : undefined + const secret = at > 0 ? raw!.slice(at + 1) : undefined + const proxyOk = !!port && Number.isInteger(port) && port > 0 && !!secret + if (raw && !proxyOk) { + log.warn("refusing to grant sandbox egress access to an invalid port/secret pair", { + port: at > 0 ? raw.slice(0, at) : raw, + }) + } + // A pipe name per run, unguessable, and never a path: `\\.\pipe\` is its + // own namespace, so `dedupe`'s path.resolve would rewrite it into a + // directory under the cwd and the launcher would serve a pipe nobody dials. + const pipe = proxyOk ? `openscience-broker-${randomBytes(16).toString("hex")}` : undefined + // `readable` matters HERE and nowhere else, which is why it was missed. + // + // bubblewrap mounts the whole filesystem read-only (`--ro-bind / /`) and + // seatbelt allows reads unless denied, so on both of those a path the + // process only needs to READ is already reachable and `readable` is a + // no-op. An AppContainer is the opposite: it reaches nothing whose ACL + // does not name its package SID. Dropping `readable` there left the + // kernel unable to read its own interpreter — measured as `dir` returning + // "Access is denied" and a venv redirector reporting `No Python at ...` + // for a base interpreter that was present the whole time. + // The binary itself joins the read set when a shim is needed: the shim IS + // this binary, re-entered inside the container, and an AppContainer can + // execute nothing whose ACL does not name its package SID. Without this + // the shim cannot start and the payload's proxy is a dead port. + const shim = pipe ? shimArgv() : undefined + // NOT dedupe(): it canonicalises, and canonicalising is precisely wrong + // here. uv keeps a patch-versioned interpreter directory with a stable + // name linked beside it, and `pyvenv.cfg` — and therefore `sys.base_prefix` + // — names the stable one. Resolving it away meant only the target got an + // ACE, and Python then failed to stat the path it actually uses: + // + // PermissionError: [WinError 5] Access is denied: + // '...\uv\python\cpython-3.12-windows-x86_64-none' + // + // On Linux the lexical spelling is restored by a mount alias. Windows has + // no such indirection: a name is reachable only if that exact name carries + // an ACE, so the caller's spelling is kept and `grant()` adds the resolved + // form alongside it. + const granted = [...new Set([...(input.readable ?? []), ...(shim?.paths ?? [])])].filter( + (value) => value && path.isAbsolute(value) && !tooBroadToConfine(value), + ) + return { + ...base, + profile: appContainerProfile(input.workspace), + // ALWAYS this list, never main's derived one, and unconditionally — + // including when it is empty. + // + // On POSIX `readable` means "what may be read" and is derived from the + // runtime roots, the workspace and the writable set; binding those into + // a namespace costs nothing. On Windows it is a GRANT list: every entry + // has an ACE written to it with `icacls`. Spreading it conditionally + // let main's derived set through whenever nothing explicit was named, + // and the launcher then tried to rewrite the ACLs of every directory on + // PATH: + // + // could not grant sandbox access to C:\Windows\System32: Access is denied. + // could not grant sandbox access to C:\Windows: Access is denied. + // + // Unelevated it merely failed, slowly — 117 seconds of icacls calls + // before a trivial `exit 7` gave up. Elevated it would have succeeded, + // and quietly granted an AppContainer standing access to the system + // directories. Windows grants exactly what the caller named, or nothing. + readable: granted, + ...(shim ? { shim: shim.argv } : {}), + ...(pipe && proxyOk ? { egress: pipe, port, secret } : {}), + } + } + + // string checks — the two normalization paths cannot drift apart because + // this is the exact same helper, not a parallel implementation of it. + const [egress] = dedupe(input.options.egress ? [input.options.egress] : []) + const egressOk = egress !== undefined && !tooBroadToConfine(egress) + if (egress !== undefined && !egressOk) { + log.warn("refusing to grant sandbox egress access to an over-broad path", { path: egress }) + } + // `readable` means "make sure the sandbox can read this", and each backend + // decides what that costs it. bubblewrap already binds the whole filesystem + // read-only, so the ONLY paths it must re-bind are the ones its own + // `--tmpfs /tmp` overlay hides. Binding anything else is not merely + // redundant: bwrap then has to create the mountpoint under a read-only + // root and fails — measured as + // `bwrap: Can't mkdir .../uv/python/cpython-3.12-linux-x86_64-gnu/bin` + // when the base interpreter was passed as readable, which broke three green + // Linux installs. + // + // Deciding that HERE rather than in the caller is what lets callers stay + // platform-agnostic: `Installer` and the kernels say what must be readable + // and never which backend needs telling. + // No readBind from `readable` at all. main binds every readable root itself, + // BEFORE the unreadable masks, so re-binding them here — after the masks — + // re-exposed the very files a mask had just covered: a `--ro-bind-try` of a + // directory shadows the /dev/null mount inside it. Measured as main's own + // symlink-escape test reading a masked file successfully. + // + // readBind now carries only what plan()/wrapArgv() add: the shim launcher, + // its bundle and the interpreter. Those are individual FILES, so nothing can + // be nested inside them and the shadowing has no trigger. + return { + ...base, + // The lexical spelling too, not just the canonical one. Global.Path.bin + ...(egressOk ? { egress } : {}), } } @@ -513,6 +923,66 @@ export namespace Sandbox { return [...out] } + /** + * `(deny network*)` then, for "allowlist" only, a narrow re-allow scoped + * to exactly one loopback port — the host-side proxy `EgressRuntime` + * starts for seatbelt (see egress-runtime.ts). Seatbelt has no network + * namespace to sever the way bubblewrap's `--unshare-net` does, so there + * is no unix socket to bind-mount either: the profile itself is the only + * boundary, which is why the deny must always precede the allow (an + * allow with no prior deny is the unfiltered, unrestricted-egress shape + * this function must never produce) and why a missing, non-positive, or + * out-of-range port throws rather than silently falling back to a bare + * deny — the same fail-closed rule `bubblewrapArgs` applies to a missing + * egress socket. Falling back to a plain deny instead of throwing would + * look identical to a user asking for `network: "deny"`, which is not + * what "allowlist" means and is exactly the kind of silent downgrade this + * branch exists to avoid. + * + * Three allow lines, not one: `docs/adr/0002-sandbox-network-policy.md` + * records the reference implementation + * (`anthropic-experimental/sandbox-runtime`) as permitting + * `network-bind`/`network-inbound`/`network-outbound`, all narrowed to the + * proxy's loopback port, filter spelled `tcp` — not the single + * `network-outbound` with `(remote ip ...)` this function emitted before + * Task 7's fix round 1. That original, narrower shape was never measured; + * it was this function's author's own guess at what a TCP `connect()` + * needs, and a Task 7 review flagged the failure mode a wrong guess + * produces here: if seatbelt classifies the implicit local port a + * `connect()` allocates under `network-bind` (this sandboxed process is + * never a listener, so `network-inbound` is included for the same + * uncertainty, not because a genuine inbound connection is expected), a + * profile missing that allow would make "allowlist" unreachable on every + * real Mac — silently, indistinguishable from the network simply being + * down, which is the one direction this task must not ship in. Matching a + * documented, cited-as-working reference is the safer default than an + * independently-derived narrower profile that has never been measured + * against a real `sandbox-exec`. `local`/`remote` for `network-bind`+ + * `network-inbound` vs `network-outbound` follows ordinary SBPL + * convention (bind/inbound describe the local endpoint, outbound the + * remote one) — the ADR does not itself quote a filter spelling for the + * first two, only for `network-outbound`, so that pairing is this + * function's own inference, not a documented fact. See the Task 7 + * report's unverified section: whether seatbelt needs `network-bind`/ + * `network-inbound` at all, and whether `local`/`remote` is the right + * pairing for them, are both open questions only a Mac can answer. + * + * A narrow, accepted consequence of matching that reference shape (Task 7 + * fix round 2): if the host proxy dies while the sandboxed child is still + * alive, `network-bind`+`network-inbound` on that same ephemeral port + * would let the child itself bind or listen there. That is still confined + * to the one port this profile names — not a broader network grant, and + * not a route to any host the child couldn't already reach through the + * (now-dead) proxy — so it is not treated as a defect. It is a real + * property of this design, not a hypothetical one, and belongs next to + * the other open questions above rather than being silently true. + * + * Never asserts enforcement — that a real `sandbox-exec` actually honours + * this text — only the text itself, its ordering, and this function's own + * refusal to emit an unfiltered allow. No Mac exists on this project to + * verify the former; see the Task 7 report for exactly what a Mac owner + * still needs to run. + */ export function seatbeltProfile(policy: Policy): string { const lines = [ "(version 1)", @@ -527,7 +997,19 @@ export namespace Sandbox { // SBPL's `remote ip` filter accepts only `*` and `localhost`, not literal // addresses or CIDR ranges. An allow-with-private-denies profile would // therefore expose LAN, link-local, and cloud-metadata endpoints. Keep the - // default deny in force for every socket operation in both policy modes. + // default deny in force for every socket operation in every policy mode — + // including "allow", which this branch does NOT reopen. The three lines + // below are the only sockets any mode gets, and they reach one loopback + // port that the host proxy answers. + if (policy.network === "allowlist") { + const port = policy.port + if (typeof port !== "number" || !Number.isInteger(port) || port <= 0 || port > 65535) { + throw new Error("sandbox network 'allowlist' requires an egress port") + } + lines.push(`(allow network-bind (local tcp "localhost:${port}"))`) + lines.push(`(allow network-inbound (local tcp "localhost:${port}"))`) + lines.push(`(allow network-outbound (remote tcp "localhost:${port}"))`) + } const readable = withPrivateAliases(dedupe(policy.readable ?? [])) if (readable.length) { lines.push( @@ -666,6 +1148,25 @@ export namespace Sandbox { // after every mount is in place. --remount-ro is non-recursive, so explicit // writable binds, the private /tmp tmpfs, /dev, and /proc keep their own // intended mount permissions. + // After the writable binds and before --remount-ro: these are the shim's + // launcher, its bundle and the interpreter, and under an empty root they + // exist only if named. Bound read-only, and after the writable loop so an + // overlap cannot silently turn part of a workspace read-only. + for (const value of dedupe(policy.readBind ?? [])) { + if (value === "/tmp") continue + args.push("--ro-bind-try", value, value) + } + if (policy.network === "allowlist") { + if (!policy.egress) throw new Error("sandbox network 'allowlist' requires an egress socket path") + // --unshare-net below is what makes this the ONLY route out: there is no + // other network device in the namespace. The bind merely makes the socket + // path reachable, and it is read-only on purpose — the bind shares the + // host inode, so a read-write one would let a sandboxed process find the + // path in /proc/self/mountinfo and `chmod 000` it, disabling egress for + // every other kernel, terminal and job sharing this socket. Verified: + // chmod fails EROFS while connect() still succeeds. + args.push("--ro-bind", policy.egress, policy.egress) + } args.push("--remount-ro", "/") // bubblewrap cannot express "internet but never host loopback" without a // separately configured network namespace. Sharing the host namespace in @@ -685,53 +1186,625 @@ export namespace Sandbox { return args } - /** Wrap an arbitrary argv under the active backend, or null when unavailable. */ - function specForArgv(argv: string[], policy: Policy): Spec | null { - switch (backend()) { + /** + * Wrap an arbitrary argv under `b`, or null when unavailable. `b` is + * passed in rather than read from `backend()` here — the caller already + * resolved it (via a possibly-injected `platform`), and re-deriving it + * from ambient state would silently disagree with that resolution. + */ + /** + * A stable AppContainer profile name for a workspace. + * + * Windows anchors containment to a package SID derived from this name, and + * the SID is what filesystem ACEs and the broker pipe's DACL refer to. So the + * name has to be stable across runs — a fresh name per launch would strand + * every ACE granted by the previous one — and distinct per project, so two + * projects cannot read each other's granted paths. + * + * Derived from the first workspace root rather than passed in, because the + * project id is not available this deep and the workspace already identifies + * the project uniquely. Hashed rather than embedded: a profile name is + * limited in length and character set, and a path contains separators, drive + * letters and spaces that are not valid in one. + */ + export function appContainerProfile(workspace: string[]): string { + const root = dedupe(workspace)[0] ?? "default" + return `openscience-${createHash("sha256").update(root).digest("hex").slice(0, 16)}` + } + + /** + * Argv that launches `argv` inside an AppContainer. + * + * Unlike bubblewrap and seatbelt there is no wrapper executable to exec: + * AppContainer confinement is applied AT process creation, through + * `SECURITY_CAPABILITIES` attributes passed to `CreateProcess`. That cannot be + * expressed as an argv, so the binary becomes its own launcher — exactly the + * pattern `__egress-shim` already uses at `index.ts:54`, and for the same + * reason: it needs no additional shipped artifact per architecture. + * + * The policy travels as one base64 blob rather than as flags. Windows command + * lines are re-parsed by `CommandLineToArgvW` with quoting rules that differ + * from every shell, and paths with spaces, quotes and backslashes are the norm + * there; a blob with no shell-significant characters cannot be mangled by + * them. The real argv still follows a `--` so the tail stays readable and + * matches the contract the other two backends keep. + */ + export function appContainerArgs(policy: Policy, argv: string[]): string[] { + if (!policy.profile) throw new Error("sandbox backend 'appcontainer' requires a profile name") + const spec = { + profile: policy.profile, + writable: policy.writable, + readable: policy.readable ?? [], + unreadable: policy.unreadable ?? [], + network: policy.network, + // Carried explicitly rather than re-derived in the launcher, so what a + // container was granted is readable straight off the spec blob when + // auditing a run. + capabilities: capabilitiesFor(policy.network), + // The host proxy the broker relays into. Carried so the launcher, which + // is the only process that knows the container's package SID, can build + // the pipe's DACL and dial the proxy on the container's behalf. + ...(policy.port && policy.secret ? { proxy: { port: policy.port, secret: policy.secret } } : {}), + ...(policy.egress ? { pipe: policy.egress } : {}), + // The launcher cannot work this out for itself: inside a source checkout + // `process.execPath` is bun, and bun needs an entry script before a + // subcommand. Composed where `Installation.isLocal()` is already known. + ...(policy.shim ? { shim: policy.shim } : {}), + // The same prefix `wrapArgv` uses to re-enter us as the launcher, carried + // so the launcher can re-enter us again for the detached shim helper. + self: [self() ?? process.execPath, ...launcherEntry()], + } + return ["__appcontainer-launch", Buffer.from(JSON.stringify(spec), "utf8").toString("base64"), "--", ...argv] + } + + /** + * What has to precede `__appcontainer-launch` for the binary to re-enter + * itself, which differs between a release and a source checkout. + * + * In a compiled release `process.execPath` IS the openscience binary, so + * `openscience __appcontainer-launch ...` runs directly. Under + * `bun run src/index.ts` — development, and every `bun test` — `process.execPath` + * is `bun`, and `bun __appcontainer-launch ...` is not a valid invocation: bun + * needs an entry script first. It exits 1 having printed nothing, which the + * self-test then reports as a child that produced no output, indistinguishable + * from a launcher that crashed. + * + * Found by CI on a real Windows runner, not by hand: every manual test ran the + * compiled binary, where this path is correct, so the dev-mode break was + * invisible from outside. `sandbox test` on a developer's checkout would have + * reported the sandbox as broken on a machine where it works. + * + * The egress shim has the same hazard and solves it by BUNDLING a separate + * entry, because `shimScript` interpolates its binary as one shell word and a + * two-word "bun " cannot be smuggled through. This launcher is a plain + * argv, so the two-word form is simply expressible and no artifact is needed. + */ + const launcherEntry = () => + self() ? [] : Installation.isLocal() ? [path.resolve(import.meta.dir, "..", "index.ts")] : [] + + /** + * A compiled binary to re-enter INSTEAD of this process, for testing the + * shipped path from a source checkout. + * + * Only honoured when `Installation.isLocal()`, so a release ignores it + * entirely and no environment variable can redirect what a shipped sandbox + * executes. + * + * It exists because the dev and release re-entry paths are different code and + * only one of them ships. Under `bun test` the shim is `bun `, which + * on Windows dies inside the container with `error loading current directory` + * — while `bun --version` in the same container, with the same working + * directory, exits 0. Two CI rounds went into that difference before the + * point registered: it is a property of an artifact users never run. + */ + const self = () => (Installation.isLocal() ? process.env["OPENSCIENCE_SELF_BINARY"] : undefined) + + function specForArgv(argv: string[], policy: Policy, b: Backend): Spec | null { + switch (b) { case "seatbelt": return { file: "sandbox-exec", args: ["-p", seatbeltProfile(policy), ...argv] } case "bubblewrap": return { file: "bwrap", args: [...bubblewrapArgs(policy), "--", ...argv] } + case "appcontainer": + // The binary launches itself into the container; see appContainerArgs. + return { file: self() ?? process.execPath, args: [...launcherEntry(), ...appContainerArgs(policy, argv)] } default: return null } } + // ── egress shim composition ───────────────────────────────────────────────── + + /** POSIX single-quote escaping: close, insert an escaped quote, reopen. */ + const quote = (value: string) => `'${value.replaceAll("'", `'"'"'`)}'` + + /** + * The sandboxed process needs a proxy at a host:port, but the only route out + * is a unix socket. This backgrounds a loopback bridge inside the namespace, + * waits (bounded) for it to signal readiness, and then execs the real + * command — so the sandbox still holds exactly one long-lived process, and + * the real command doesn't get a proxy env pointing at a port nothing is + * listening on yet. + * + * The wait is a marker-file poll, not a network probe: a POSIX `/bin/sh` + * (dash/busybox, not bash) has no built-in way to test a TCP connection — + * bash's `/dev/tcp` isn't portable here and `nc`/`curl` aren't guaranteed + * present. + * + * *Why the granularity is chosen at run time.* Fractional `sleep` is a + * GNU/BSD coreutils extension, not POSIX, and some busybox builds reject it + * outright (`sleep: invalid interval`) — which, in a loop, would print an + * error line per iteration to the real command's own stderr (this wait runs + * in the foreground, unlike the backgrounded shim) and, worse, skip the + * wait entirely, since a failing `sleep` doesn't slow a loop down at all. + * So the interval is settled once, before the loop, by attempting a single + * fractional `sleep` with its stderr discarded: it either works, and the + * loop polls at 0.02s, or it fails instantly and everything falls back to + * the whole seconds POSIX guarantees. That probe is the only place a + * fractional interval is ever attempted, its diagnostic can't reach the + * command's stderr, and its cost isn't waste — it is time the shim needs + * anyway. + * + * *Why not whole seconds throughout, as this did before.* Measured shim + * readiness (fork/exec, bundle load, listener bound) is ~12ms — the + * 600ms–1.1s in Task 4's report predates bundling the shim entry. At + * whole-second granularity the first check therefore always lost and every + * spawn paid a flat second: n=8, `network: "allowlist"` 1006-1007ms against + * `deny` 3-4ms, on `sh -c true`, i.e. 335x for a command that never touches + * the network. Every `ls` and every `git status` the agent ran paid it. At + * 0.02s the same measurement is 24-25ms. + * + * *Why there is a wall-clock deadline and not just an iteration count.* The + * count alone (150 * 0.02, 3 * 1) only equals 3s where forking `sleep` is + * nearly free. It isn't everywhere: a macOS CI runner measured 17.1s for + * the 150-iteration loop — ~114ms per iteration, of which ~94ms is + * fork/exec of `/bin/sleep`, a 5.7x overshoot of the documented cap. Any + * machine with expensive process creation (a CPU-throttled container, a + * loaded box) drifts the same way, so the loop carries an explicit deadline + * as well. `date +%s` is probed exactly like fractional `sleep` — a build + * without it leaves the deadline unset and the count is the only cap, which + * is the behaviour that shipped before — and `+ 4` rather than `+ 3` + * because `%s` truncates to whole seconds, which would otherwise cut a + * nominal 3s wait as short as 2.0s. + * + * The cap is therefore ~3s in both modes, and 3–4s when the deadline is the + * one that fires. If the shim never signals, the loop still exits at the cap + * and the real command runs anyway — against a closed proxy port, which + * fails fast and visibly (connection refused) rather than hanging forever. + */ + export function shimScript(input: { binary: string; port: number; socket: string; file: string; args: string[] }) { + const shim = [quote(input.binary), "__egress-shim", String(input.port), quote(input.socket)].join(" ") + const real = [quote(input.file), ...input.args.map(quote)].join(" ") + const marker = quote(SHIM_READY_MARKER) + // `s`/`n`/`i`/`d`/`t` are plain shell variables, never exported, and + // `exec` replaces this shell — so none of them reach the real command. + // `${t:-0}` keeps a `date` that starts failing mid-loop from breaking out + // early or printing to the real command's stderr: it degrades to the + // count-only cap, the same direction the probe failing does. + const wait = [ + `s=0.02; n=150`, + `sleep "$s" 2>/dev/null || { s=1; n=3; }`, + `d=$(date +%s 2>/dev/null); case "$d" in ''|*[!0-9]*) d= ;; *) d=$((d + 4)) ;; esac`, + `i=0; while [ ! -f ${marker} ] && [ "$i" -lt "$n" ] && { [ -z "$d" ] || { t=$(date +%s 2>/dev/null); [ "\${t:-0}" -lt "$d" ]; }; }; do sleep "$s"; i=$((i + 1)); done`, + ].join("; ") + return `${shim} >/dev/null 2>&1 & ${wait}; exec ${real}` + } + + /** + * Bubblewrap-only. Loopback port the shim binds inside the sandboxed + * network namespace. Fixed rather than negotiated: `--unshare-net` gives + * every sandboxed process its own private namespace, so this port can + * never collide across sandboxed processes or with anything on the host. + * Exported so `egress-runtime.ts` can hand it back to callers alongside + * the proxy's socket — one source of truth, rather than a second + * module-private 3128 that could drift from this one. + * + * Seatbelt has no such namespace — every process on the machine shares one + * loopback, so a fixed well-known port would collide across concurrent + * sandboxed processes the way it structurally cannot here. Its egress port + * (`Policy.port`) is instead assigned by the OS per proxy instance; see + * `egress-runtime.ts`. + */ + export const SHIM_PORT = 3128 + + /** + * Write one of the dev shim's generated artifacts, idempotently. Callers + * pass a content-addressed name, so a file already at that name already has + * this content and there is nothing to do; comparing anyway costs a few KB + * and repairs a truncated leftover. The write goes to a per-process + * temporary name and is renamed into place, which is atomic within a + * directory — two processes generating the same artifact concurrently write + * byte-identical content, and no third process can observe a half-written + * file at the real name. + * + * A missing file (first run, a fresh worktree, a test tmpdir) is the + * expected case, not a failure, so read errors of any kind just mean "write + * it" and are swallowed separately from the write's own errors. Those fail + * loud with an actionable message rather than a raw EACCES/EROFS out of + * `wrapArgv`: network "allowlist" without a working shim is a security- + * relevant misconfiguration (the caller explicitly asked for bounded + * egress), not something to silently downgrade. + */ + function place(file: string, content: Buffer, mode: number) { + const current = (() => { + try { + return fs.readFileSync(file) + } catch { + return undefined + } + })() + if (current?.equals(content)) return + const temp = `${file}.${process.pid}` + try { + fs.mkdirSync(path.dirname(file), { recursive: true }) + fs.writeFileSync(temp, content, { mode }) + fs.chmodSync(temp, mode) + fs.renameSync(temp, file) + } catch (e) { + fs.rmSync(temp, { force: true }) + throw new Error(`Could not write the dev egress shim to ${file}: ${e instanceof Error ? e.message : String(e)}`) + } + } + + /** + * The single executable `shimScript` execs as the loopback bridge, plus + * every read-only path that must be explicitly bound into the namespace + * (`Policy.readBind`, consumed by `bubblewrapArgs`) for that executable to + * actually be reachable from inside, regardless of where it lives on the + * host. + * + * In a compiled release `process.execPath` IS the openscience binary, so + * `openscience __egress-shim ...` runs directly — one self-contained file, + * no extra artifact ships. + * + * Under `bun run src/index.ts` in development no such file exists: + * `process.execPath` is `bun`, and `bun __egress-shim ...` is not a valid + * bun invocation (it needs an entry script too), while `shimScript`'s + * `binary` is a single shell word once quoted, so a two-word "bun " + * invocation cannot be smuggled through it. So dev *builds* the missing + * file: `bun build` bundles `egress-shim-entry.ts` (a sibling of this file) + * into one self-contained module, and a tiny `sh` launcher execs `bun` + * against it — the same trick `ensureAtlasBinDir` in + * `src/openscience/index.ts` uses to expose a package's JS entry as one + * executable path. The entry is that file and never `src/index.ts`, whose + * graph pulls in `Global` (an unguarded top-level file write) and a live + * models.dev fetch, both of which run before any argv check could skip + * them. A compiled binary has no separate entry to redirect to, so it + * still goes through `index.ts`'s `__egress-shim` check and still + * evaluates that graph — see the Task 4 report for what that leaves + * reachable there; restructuring `index.ts` so nothing runs before the + * check, for both modes, is a materially bigger change than this fix. + * + * *Why bundle instead of running the source.* `--tmpfs /tmp` masks the + * whole host `/tmp` subtree, `--ro-bind / /` notwithstanding, so every path + * the shim touches has to be bound back by name — and the set of paths + * *running a source file* touches is open-ended: the entry, its imports, + * their imports, and for an npm import both the `node_modules` symlink and + * its target, which in this bun workspace is the monorepo-root store, one + * level *above* the package root. Successive revisions of this function + * bound the paths their author thought of (the launcher, then this file's + * directory, then the interpreter and the whole package root) and each + * time missed one — the last of them that npm target, latent only because + * nothing in the graph resolves a package today. A + * bundle ends that class rather than extending the list: at run time bun + * opens the bundle and nothing else, so the bound set is closed by + * construction — launcher, bundle, interpreter — instead of having to keep + * pace with an import graph. + * + * *What that does not cover*, stated precisely because "no future edit can + * break this" is the claim that was false the last four times: `bun build` + * inlines statically resolvable imports only, so a runtime + * `import(expression)` reaching outside the bundle would still resolve + * against an unbound path (a literal `import("./x")` is inlined and fine); + * the bundle executes from `Global.Path.bin`, so anything resolving off its + * own `import.meta.dir`/`url` no longer lands in the source tree; and an + * import the bundler cannot inline fails the build here, loudly, at + * `wrapArgv` time instead of silently inside the sandbox. Import-time side + * effects stay forbidden for the separate reason in + * `egress-shim-entry.ts`'s own comment — bundling relocates that code, it + * does not stop it running. + * + * The fourth one escapes the framing rather than sitting inside it: a + * dependency that loads a *native binding* bundles cleanly and still + * resolves a path at run time. Measured with `bun-pty` as a probe — the + * build succeeds, the JS is inlined, and the output then carries + * `dlopen("….so")`, `import.meta.require` and `process.cwd`, with only + * `bun:ffi` left external. A `dlopen` argument is not an import specifier, + * so neither "bun opens the bundle and nothing else" nor the static test + * that enforces it covers this; such a dependency would need its shared + * library bound by name the way the artifacts are. + * + * *Where the artifacts live does not need to be "safe."* Earlier revisions + * tried to pick a location `--tmpfs /tmp` couldn't mask — `Global.Path.state`, + * then this file's own directory — and both were live-verified broken: + * `Global.Path.*` resolves under `os.tmpdir()` during `bun test` + * (`test/preload.ts` redirects every XDG dir there for isolation) and + * possibly for a real user with `$HOME` under `/tmp`; the repo checkout + * resolves under `/tmp` for a `git worktree add /tmp/...` (this repo's own + * workflow), a CI `mktemp -d` clone, or a container build. There is no + * location immune to both. The fix is the one `bubblewrapArgs` already uses + * for the egress socket: bind the exact path back in, explicitly, after + * `--tmpfs /tmp` — `--ro-bind-try`, not `--bind`, since these are executed + * and read, never written to, from inside. `process.execPath` is bound for + * the same reason and is a structurally separate input, not implied by + * binding the artifacts: a portable bun install, or `$HOME` under `/tmp`, + * puts the interpreter the launcher execs under the tmpfs too. + * + * Bubblewrap-only. Every one of the artifacts this produces exists to get + * a launcher into a severed network namespace and bind it back in by name + * — problems seatbelt does not have, since it has no namespace and the + * sandboxed process dials the loopback proxy directly (see + * `seatbeltProfile`). `plan()`/`wrapArgv()` only ever call this behind a + * `backend === "bubblewrap"` guard, so on darwin — real or + * platform-injected — this function, and everything it writes to + * `Global.Path.bin`, is never reached at all. + */ + const stamp = (value: Buffer) => createHash("sha256").update(value).digest("hex").slice(0, 16) + + /** + * The dev shim as one self-contained module, built once per process. + * + * Shared by both backends that need it. bubblewrap wraps it in an `sh` + * launcher because `shimScript` interpolates a single shell word; the + * AppContainer launcher spawns an argv and needs only the bundle itself. + * Neither may fall back to `src/index.ts` — see `egress-shim-entry.ts` for + * why that graph cannot survive inside the sandbox. + */ + const devBundle = lazy(() => { + const entry = path.resolve(import.meta.dir, "egress-shim-entry.ts") + const built = Bun.spawnSync([process.execPath, "build", "--target=bun", entry]) + if (!built.success) { + throw new Error(`Could not bundle the dev egress shim from ${entry}: ${built.stderr.toString().trim()}`) + } + // Content-addressed, not fixed names: a rebuilt bundle is a different + // file rather than an overwrite of the one another process may be + // executing, and a name that already exists already holds this exact + // content, by construction. The launcher's own digest covers the bundle's + // path, so a new bundle always produces a new launcher pointing at it — + // a stale pair cannot form. What makes the bytes differ is (source, bun, + // cwd): bun build writes cwd-relative module banners into the output, so + // the same source built from `backend/cli` and from anywhere else are + // different files. Only correctness is claimed here, not thrift — cwd is + // the dimension that varies per invocation, so it is also the one that + // drives how many of these accumulate. + // .mjs, not .js: nothing should make bun's module-type detection for this + // file depend on a package.json above Global.Path.bin. + const bundle = path.join(Global.Path.bin, `egress-shim-dev-${stamp(built.stdout)}.mjs`) + place(bundle, built.stdout, 0o644) + return bundle + }) + + const shimPlan = lazy((): { binary: string; bind: string[] } => { + if (!Installation.isLocal()) return { binary: process.execPath, bind: [process.execPath] } + const bundle = devBundle() + const script = Buffer.from(`#!/bin/sh\nexec ${quote(process.execPath)} ${quote(bundle)} "$@"\n`) + const launcher = path.join(Global.Path.bin, `egress-shim-dev-${stamp(script)}.sh`) + place(launcher, script, 0o755) + return { binary: launcher, bind: [launcher, bundle, process.execPath] } + }) + + /** + * The shim as an ARGV plus the paths that must be reachable for it to run — + * for the AppContainer launcher, which spawns a process rather than + * interpolating one shell word into a script. + * + * A release re-enters its own binary, so both are just `process.execPath`. + * From source, `bun __egress-shim ...` is not a valid invocation — bun needs + * an entry script — and CI caught exactly that: + * + * error: Script not found "__egress-shim" + * WARNING: Retrying ... 127.0.0.1:52939 ... actively refused it + * + * The shim never started and the payload's proxy port was dead. Handing bun + * `src/index.ts` would fix the invocation and break the containment: an ESM + * graph is linked before it is evaluated, so the container would need read + * access to the whole source tree and `node_modules` before the first argv + * check could run. The bundle keeps the reachable set to two files. + */ + const shimArgv = (): { argv: string[]; paths: string[] } => { + const binary = self() ?? (Installation.isLocal() ? undefined : process.execPath) + if (binary) return { argv: [binary, "__egress-shim"], paths: [binary] } + const bundle = devBundle() + return { argv: [process.execPath, bundle], paths: [process.execPath, bundle] } + } + // ── planning (consumed by the bash tool and the kernels) ──────────────────── + /** + * The `HTTP_PROXY`-shaped URL the sandboxed process should use, or + * `undefined` when nothing composed a route to the proxy at all. + * Bubblewrap: the shim's fixed `SHIM_PORT`, unauthenticated — the + * bind-mounted unix socket underneath it is already the sandboxed + * process's only route out, so the loopback hop inside the namespace + * needs no credential of its own. Seatbelt: no shim, so the sandboxed + * process dials `policy.port` directly, and — because that loopback port, + * unlike a unix socket, carries no filesystem permissions of its own — + * the URL embeds `policy.secret` as userinfo + * (`http://os:@127.0.0.1:`), which pip, curl and requests + * all parse into a `Proxy-Authorization` header. Both must be present, not + * just `port`: `buildPolicy` only ever sets them together, so a `port` + * with no `secret` means something upstream broke that invariant, and + * this fails closed to "no proxy configured" rather than emitting a URL + * seatbelt's own proxy would just reject with 407 anyway. + */ + function proxyUrl(shim: string | undefined, b: Backend, policy: Policy): string | undefined { + if (shim) return `http://127.0.0.1:${SHIM_PORT}` + if (b !== "seatbelt" || policy.network !== "allowlist" || !policy.port || !policy.secret) return undefined + return `http://os:${policy.secret}@127.0.0.1:${policy.port}` + } + // Warn only once per process so every command doesn't repeat the same notice. - const warned = { unavailable: false } + const warned = { unavailable: false, loopback: false, allow: false, shell: false } + + /** + * Forget which one-time warnings have been issued. + * + * A test seam, and the reason it exists is worth stating: warnings that fire + * once per process are, by construction, observable only by whichever test + * runs first. Without this, asserting on one means silently depending on + * source order — and the assertion stops holding the moment anything above it + * touches the same path, without failing in a way that says so. + */ + export function forgetWarnings() { + warned.unavailable = false + warned.loopback = false + warned.allow = false + warned.shell = false + } + + /** + * Well-known capability SIDs. Constants, not derived: these are fixed by + * Windows and `DeriveCapabilitySidsFromName` would be a second FFI surface to + * get the same two values. + */ + const CAPABILITY = { + /** internetClient — outbound to the internet, any protocol. */ + internet: "S-1-15-3-1", + /** privateNetworkClientServer — the local subnet. */ + privateNetwork: "S-1-15-3-3", + } + + /** + * What an AppContainer is allowed to reach, by policy. + * + * `allow` means unrestricted egress on the other two platforms, so it must + * mean that here too rather than quietly meaning less. Withholding these + * would make the knob claim more than it delivers, for no security anyone + * asked for. + * + * `allowlist` and `deny` keep ZERO capabilities, and that is load-bearing + * rather than incidental: under `allowlist` the broker is the enforcement + * point, so a container that could reach the internet directly would route + * around the allowlist while still reporting that a policy was applied. + */ + function capabilitiesFor(network: Policy["network"]): string[] { + return network === "allow" ? [CAPABILITY.internet, CAPABILITY.privateNetwork] : [] + } + + /** + * The one thing Windows cannot deliver, said once, with the remedy. + * + * AppContainer loopback is blocked at the firewall layer regardless of + * capability; the only exemption is `CheckNetIsolation LoopbackExempt`, which + * needs admin — out of scope for this product. So under `allow` a sandboxed + * process cannot reach a local Ollama, Jupyter, database or model server. + * + * `allowlist` CAN reach them, which inverts the usual intuition and is why + * this message carries a fix rather than an apology: the broker runs on the + * HOST, and a host process has no loopback restriction, so it dials 127.0.0.1 + * on the container's behalf. + * + * That inversion is a wart, and the design records the fix: once `allow` runs + * the broker too (a superset, matching bubblewrap and seatbelt where `allow` + * applies no restriction at all), THIS WORDING MUST NARROW to "non-HTTP + * connections to 127.0.0.1 are unavailable". A warning that overstates a + * limitation is the same defect as one that understates it, and right now it + * is accurate only because no broker exists yet. + * + * Note what is NOT affected: the probe measured loopback WITHIN the container + * working, so anything the sandboxed process starts itself is fine — + * torch.distributed rendezvous, multiprocessing, a Ray or Dask cluster it + * launches. Only a pre-existing service on the host is out of reach. + * + * Routed through the same one-time `warning` channel as "sandbox requested + * but unavailable" — same species of problem, and a second mechanism for it + * is how two commands end up disagreeing about the same state. + */ + function loopbackMessage() { + return ( + "Windows sandbox: network 'allow' cannot reach 127.0.0.1 - AppContainer loopback is blocked " + + "and the exemption requires admin. For a local service, use network 'allowlist' and add its " + + "host: the broker runs outside the container and dials it for you." + ) + } + + /** + * `network: "allow"` reaches nothing on the POSIX backends, and saying so is + * not optional. + * + * bubblewrap cannot express "the internet but never host loopback" without a + * separately configured network namespace, and SBPL's `remote ip` filter + * accepts only `*` and `localhost` — no literals, no CIDR — so an + * allow-with-private-denies profile would expose LAN, link-local and + * cloud-metadata endpoints. Both fail closed instead, which is right. + * + * What was wrong is that nothing told anyone. `sandbox status` printed + * "network allow", the settings panel showed it selected, the schema said + * "whether sandboxed commands may reach the network" — and every connection + * failed to resolve, with no path from the symptom back to the setting the + * user had deliberately turned on. A config value that reports one thing and + * does another is a bug even when the behaviour behind it is correct. + */ + function allowMessage(b: Backend): string { + return ( + `Sandbox network 'allow' reaches nothing on this backend (${b}): it denies every socket, because it ` + + "cannot grant outbound access without also exposing everything bound to 127.0.0.1 on this machine. " + + "Use 'allowlist' and add the hosts you need - that route goes through the proxy and is the only one " + + "with an audit trail." + ) + } function unavailableMessage(): string { return `Sandbox is enabled but unavailable on this machine (${describe().reason}). Running the command WITHOUT isolation. Install the backend, or set sandbox.onUnavailable to "error" to refuse instead.` } /** - * Resolve which backend a command should use given the config. Returns - * backend "none" (run unsandboxed) with an optional one-time warning, or the + * Resolve which backend a command should use given the config and + * `platform` (default the real one — see `backend()`). Returns backend + * "none" (run unsandboxed) with an optional one-time warning, or the * active backend. Throws UnavailableError only when `onUnavailable: "error"` * and no backend exists. */ - function decide(options?: Options): { backend: Backend; warning?: string } { - if (options?.enabled !== true) return { backend: "none" } - const b = backend() + function decide( + options: Options | undefined, + platform: NodeJS.Platform = process.platform, + ): { backend: Backend; warning?: string } { + if (!resolved(options).enabled) return { backend: "none" } + const b = backend(platform) + if (b === "appcontainer" && resolved(options).network === "allow" && !warned.loopback) { + warned.loopback = true + return { backend: b, warning: loopbackMessage() } + } + // Same channel, same one-time rule. Not an error: the run proceeds, confined, + // with no network — which is what the user gets either way. The warning is + // the part that was missing. + if ((b === "bubblewrap" || b === "seatbelt") && resolved(options).network === "allow" && !warned.allow) { + warned.allow = true + log.warn("sandbox network 'allow' denies all sockets on this backend", { backend: b }) + return { backend: b, warning: allowMessage(b) } + } if (b !== "none") return { backend: b } - const mode = options.onUnavailable ?? "warn" + const mode = options?.onUnavailable ?? "warn" if (mode === "error") throw new UnavailableError(unavailableMessage()) const warning = mode === "warn" && !warned.unavailable ? unavailableMessage() : undefined if (warning) { warned.unavailable = true - log.warn("sandbox enabled but unavailable", { platform: process.platform }) + log.warn("sandbox enabled but unavailable", { platform }) } return { backend: "none", warning } } /** * Decide how to run a shell command given the sandbox config and the - * workspace. Never throws unless `onUnavailable: "error"` and no backend - * exists. The `cwd` is *not* granted write access unless it lies within the - * workspace — an approved external working directory is a permission decision, - * not a reason to widen the write boundary to the escape target. + * workspace. Throws only in two cases: `onUnavailable: "error"` with no + * backend available, or `network: "allowlist"` with no `egress` socket path + * (directly, or because the supplied path was rejected as over-broad). The + * `cwd` is *not* granted write access unless it lies within the workspace — + * an approved external working directory is a permission decision, not a + * reason to widen the write boundary to the escape target. + * + * Composes the same loopback shim `wrapArgv` does, under the same + * condition (bubblewrap, network "allowlist", a usable egress socket) — + * `pip`/`curl`/`uv` run through here, not `wrapArgv`, so this is the path + * the feature's motivating case actually needs. `shimScript` already + * treats its `file`/`args` as an arbitrary argv to `exec`, so a shell + * invocation composes by feeding it `input.shell`/`["-c", input.command]` + * exactly as the no-shim branch below already passes to `specForArgv` — + * one shape, not a second implementation of "wrap a shell command". + * + * `platform` defaults to the real one — the only reason to pass a + * different value is to exercise the seatbelt/darwin branch from a + * machine that isn't one; see `backend()`. */ export function plan(input: { command: string @@ -744,8 +1817,10 @@ export namespace Sandbox { /** Exact host credential files to mask from the process. */ unreadable?: string[] options?: Options + platform?: NodeJS.Platform }): Plan { - const { backend: b, warning } = decide(input.options) + const platform = input.platform ?? process.platform + const { backend: b, warning } = decide(input.options, platform) if (b === "none") { return { file: input.command, useShell: input.shell, sandboxed: false, backend: "none", warning } } @@ -758,10 +1833,71 @@ export namespace Sandbox { unreadable: input.unreadable, entrypoints: [input.shell], options: input.options!, + backend: b, + platform, }) - const s = specForArgv(withTempEnvironment([input.shell, "-c", input.command], temporary), policy)! + // withTempEnvironment prefixes `/usr/bin/env TMPDIR=...`, which does not + // exist on Windows. The AppContainer path already carries variables to the + // child through `env`, so the private temp rides that instead — same + // guarantee, expressed the way the platform can express it. + const posix = b !== "appcontainer" + const tempEnv = posix ? undefined : { TMPDIR: temporary, TMP: temporary, TEMP: temporary } + // Bubblewrap's loopback shim bridges a bind-mounted unix socket that only + // exists inside its own network namespace. Seatbelt has no namespace, so + // there is nothing to bridge and no shim to compose — the sandboxed + // process instead dials the loopback proxy port seatbeltProfile allowed + // directly, which is why this guard stays bubblewrap-only rather than + // "any backend with allowlist + an egress value". + const shimmed = b === "bubblewrap" && policy.network === "allowlist" && policy.egress ? shimPlan() : undefined + const shim = shimmed + ? shimScript({ + binary: shimmed.binary, + port: SHIM_PORT, + socket: policy.egress!, + file: input.shell, + args: Shell.invocation(input.shell, input.command), + }) + : undefined + // Shell.invocation, not a hardcoded "-c": on Windows the shell is usually + // cmd.exe, which takes /c and reads -c as "start interactive". Given -c it + // printed its banner and a prompt, ran nothing, and exited 0 — so every + // sandboxed command silently did nothing, and the self-test read the banner + // as a process token. The shim branch is bubblewrap-only and so always + // POSIX, but it goes through the same helper rather than keeping a second + // copy of this knowledge, which is how the bug survived in the first place. + const argv = shim ? ["/bin/sh", "-c", shim] : [input.shell, ...Shell.invocation(input.shell, input.command)] + const s = specForArgv( + posix ? withTempEnvironment(argv, temporary) : argv, + shimmed + ? { + ...policy, + readBind: [...(policy.readBind ?? []), ...shimmed.bind], + // And their lexical spellings. shimPlan writes into Global.Path.bin, + // which reaches the data root through a symlink, so the path + // interpolated into the shim script is not the path that resolves + // on the host. buildPolicy cannot do this for us — these paths are + // added here, after it has already computed its aliases. + readableAliases: [...(policy.readableAliases ?? []), ...mountAliases(shimmed.bind)], + } + : policy, + b, + )! log.info("sandboxing command", { backend: b, network: policy.network, writable: policy.writable.length }) - return { file: s.file, args: s.args, useShell: false, sandboxed: true, backend: b, temporary, warning } + const proxy = proxyUrl(shim, b, policy) + const env = { + ...(proxy ? { HTTP_PROXY: proxy, HTTPS_PROXY: proxy, http_proxy: proxy, https_proxy: proxy } : {}), + ...(tempEnv ?? {}), + } + return { + file: s.file, + args: s.args, + useShell: false, + sandboxed: true, + backend: b, + temporary, + warning, + ...(Object.keys(env).length ? { env } : {}), + } } catch (error) { cleanup({ temporary }) throw error @@ -773,6 +1909,10 @@ export namespace Sandbox { * which spawn an interpreter directly. When the sandbox is off or unavailable * the original `file`/`args` are returned unchanged, so callers can spawn the * result verbatim. + * + * `platform` defaults to the real one — the only reason to pass a + * different value is to exercise the seatbelt/darwin branch from a + * machine that isn't one; see `backend()`. */ export function wrapArgv(input: { file: string @@ -786,8 +1926,10 @@ export namespace Sandbox { /** Exact host credential files to mask from the process. */ unreadable?: string[] options?: Options + platform?: NodeJS.Platform }): Wrapped { - const { backend: b, warning } = decide(input.options) + const platform = input.platform ?? process.platform + const { backend: b, warning } = decide(input.options, platform) if (b === "none") { return { file: input.file, args: input.args, sandboxed: false, backend: "none", warning } } @@ -801,10 +1943,53 @@ export namespace Sandbox { unreadable: input.unreadable, entrypoints: [input.file], options: input.options!, + backend: b, + platform, }) - const s = specForArgv(withTempEnvironment([input.file, ...input.args], temporary), policy)! + // See plan(): POSIX gets the temp through `/usr/bin/env`, Windows through + // the same `env` channel that carries the proxy variables. + const posix = b !== "appcontainer" + // Same shim composition plan() does, and it is not optional here: kernels + // reach the network through wrapArgv, so dropping it left every notebook + // and R kernel with no egress at all under "allowlist" — the exact route + // package installs from a kernel depend on. + const shimmed = b === "bubblewrap" && policy.network === "allowlist" && policy.egress ? shimPlan() : undefined + const shim = shimmed + ? shimScript({ + binary: shimmed.binary, + port: SHIM_PORT, + socket: policy.egress!, + file: input.file, + args: input.args, + }) + : undefined + const argv = shim ? ["/bin/sh", "-c", shim] : [input.file, ...input.args] + const s = specForArgv( + posix ? withTempEnvironment(argv, temporary) : argv, + shimmed + ? { + ...policy, + readBind: [...(policy.readBind ?? []), ...shimmed.bind], + readableAliases: [...(policy.readableAliases ?? []), ...mountAliases(shimmed.bind)], + } + : policy, + b, + )! log.info("sandboxing process", { backend: b, network: policy.network, writable: policy.writable.length }) - return { file: s.file, args: s.args, sandboxed: true, backend: b, temporary, warning } + const proxy = proxyUrl(shim, b, policy) + const env = { + ...(proxy ? { HTTP_PROXY: proxy, HTTPS_PROXY: proxy, http_proxy: proxy, https_proxy: proxy } : {}), + ...(posix ? {} : { TMPDIR: temporary, TMP: temporary, TEMP: temporary }), + } + return { + file: s.file, + args: s.args, + sandboxed: true, + backend: b, + temporary, + warning, + ...(Object.keys(env).length ? { env } : {}), + } } catch (error) { cleanup({ temporary }) throw error @@ -827,26 +2012,63 @@ export namespace Sandbox { ok: boolean } + /** + * Lines this process wrote, which are never the child's error. + * + * The launcher's debug dump was the first offender; the structured logger is + * the second, and it took a red CI job to notice because the check reported + * `INFO ... service=openscience api_base=...` as the reason a sandboxed curl + * failed. Anything on the same stderr that came from us has to be skipped, not + * just the one prefix that was noticed first. + */ + const ours = (line: string) => + line.startsWith("openscience[") || line.includes("service=openscience") || /^(INFO|WARN|ERROR|DEBUG)\s/.test(line) + function firstLine(s?: string): string | undefined { - const line = s?.trim().split("\n")[0] + // Skip our OWN diagnostic lines. The launcher's debug dump goes to the same + // stderr the checks read for the child's error, so with the dump on, every + // failure reported the first line of the dump instead of what went wrong — + // a diagnostic destroying the evidence it exists to surface, for the fourth + // time in this feature. + const line = s + ?.trim() + .split("\n") + .map((value) => value.trim()) + .find((value) => value && !ours(value)) return line || undefined } - function runAsync(file: string, args: string[], cwd: string): Promise<{ status: number; stderr: string }> { + /** stdout is captured as well as stderr: the AppContainer check reads the + * child's own token from `whoami /groups`, and a check that can only see + * exit codes cannot tell "not confined" from "confined but permissive". */ + function runAsync( + file: string, + args: string[], + cwd: string, + env?: Record, + ): Promise<{ status: number; stdout: string; stderr: string }> { return new Promise((resolve) => { - const proc = spawn(file, args, { cwd, stdio: ["ignore", "ignore", "pipe"] }) + const proc = spawn(file, args, { + cwd, + stdio: ["ignore", "pipe", "pipe"], + ...(env ? { env: { ...process.env, ...env } } : {}), + }) + let stdout = "" let stderr = "" + proc.stdout?.on("data", (d) => { + stdout += d.toString() + }) proc.stderr?.on("data", (d) => { stderr += d.toString() }) const timer = setTimeout(() => proc.kill("SIGKILL"), 15000) proc.once("exit", (code) => { clearTimeout(timer) - resolve({ status: code ?? 1, stderr }) + resolve({ status: code ?? 1, stdout, stderr }) }) proc.once("error", (err) => { clearTimeout(timer) - resolve({ status: 1, stderr: String(err) }) + resolve({ status: 1, stdout, stderr: String(err) }) }) }) } @@ -862,23 +2084,107 @@ export namespace Sandbox { const b = backend() if (b === "none") return { backend: b, available: false, checks: [], ok: false } - const shell = Shell.acceptable() + // The sandbox's own choice, not the machine's: this probe must exercise the + // shell a sandboxed command will actually get, or it verifies a path users + // never take. + const shell = Sandbox.shell({ enabled: true }) const work = fs.mkdtempSync(path.join(os.tmpdir(), "openscience-sbx-")) const outside = path.join(os.homedir(), `.openscience-sbx-escape-${process.pid}`) const outsideRead = path.join(os.tmpdir(), `.openscience-sbx-sibling-${process.pid}`) const checks: Check[] = [] - const run = async (command: string, network: "allow" | "deny") => { + const run = async (command: string, network: "allow" | "deny", env?: Record) => { const p = plan({ command, shell, cwd: work, workspace: [work], options: { enabled: true, network } }) try { - return await runAsync(p.file, p.args ?? [], work) + return await runAsync(p.file, p.args ?? [], work, env) } finally { cleanup(p) } } try { - const inside = await run(`printf hi > "${work}/probe" && cat "${work}/probe"`, "allow") + // Windows first, because every other check below is meaningless if this + // one fails. A child that never entered the container looks EXACTLY like + // a container with no policy applied: writes escape and the network works. + // Separating "the launcher did not confine it" from "it is confined and + // the policy is wrong" is the difference between debugging CreateProcess + // and debugging the spec, and they are indistinguishable from outside. + if (b === "appcontainer") { + // `exit 0`, not `whoami /groups`. The check used to run whoami and + // pattern-match its output, which made containment depend on a command + // succeeding INSIDE the container. On a CI runner it does not: whoami + // resolves SIDs to display names through LSA, which an AppContainer with + // zero capabilities cannot reach, so it exits 66 having printed nothing — + // while `exit 7` through the identical plan returns 7, proving the + // container hosts processes fine. Two rounds were spent reading that as a + // containment failure, after four spent on other diagnostics that + // reported conclusions rather than observations. + // + // The launcher holds the child's process handle, so it asks the kernel + // TokenIsAppContainer directly and reports the answer. Nothing here + // depends on what the child can do. + const query = await run("exit 0", "allow", { OPENSCIENCE_APPCONTAINER_REPORT: "1" }) + const reported = query.stderr.match(/openscience\[appcontainer\] token appcontainer=(\d|\?)/)?.[1] + const confined = reported === "1" + checks.push({ + name: "the child actually runs inside the AppContainer", + pass: confined, + detail: confined + ? "the kernel reports TokenIsAppContainer=1 for the child" + : [ + reported === undefined + ? `the launcher never reported a token (child exit ${query.status})` + : reported === "?" + ? "the child's token could not be read" + : "the kernel reports TokenIsAppContainer=0: SECURITY_CAPABILITIES did not take effect", + firstLine(query.stderr), + process.env["OPENSCIENCE_SANDBOX_DEBUG"] === "1" ? `\n${query.stderr.trim()}` : undefined, + ] + .filter(Boolean) + .join(": "), + }) + if (!confined) { + checks.push({ + name: "write outside the workspace is blocked", + pass: false, + skipped: true, + detail: "inconclusive - the child is not in a container", + }) + return { backend: b, available: true, checks, ok: false } + } + } + + // `printf` and `cat` do not exist in cmd.exe. The probe was POSIX-only, so + // on Windows this check could never pass and reported the sandbox as + // unable to write inside its own workspace — when the real fault was that + // the command did not exist. Compose per shell family instead. + const probe = path.join(work, "probe") + const write = (target: string, text: string) => { + switch (Shell.family(shell)) { + case "cmd": + // No space before ">", or cmd writes the space into the file. + return `echo ${text}>"${target}"` + case "powershell": + return `$ErrorActionPreference='Stop'; Set-Content -LiteralPath "${target}" -Value '${text}'` + default: + return `printf ${text} > "${target}"` + } + } + const read = (target: string) => { + switch (Shell.family(shell)) { + case "cmd": + return `type "${target}"` + case "powershell": + return `Get-Content -LiteralPath "${target}"` + default: + return `cat "${target}"` + } + } + // `;` for PowerShell because `&&` is PowerShell 7 only, and 5.1 is still + // what a default Windows box has; $ErrorActionPreference makes the first + // statement failing terminate the pipeline anyway. + const join = Shell.family(shell) === "powershell" ? "; " : " && " + const inside = await run(`${write(probe, "hi")}${join}${read(probe)}`, "allow") const insideOk = inside.status === 0 checks.push({ name: "write inside the workspace succeeds", @@ -907,7 +2213,7 @@ export namespace Sandbox { }) fs.rmSync(outside, { force: true }) - const escape = await run(`printf x > "${outside}"`, "allow") + const escape = await run(write(outside, "x"), "allow") const escaped = fs.existsSync(outside) checks.push({ name: "write outside the workspace is blocked", @@ -944,6 +2250,74 @@ export namespace Sandbox { detail: "curl not available — skipped", }) } + + // Windows only, and only because Windows is the one backend where "allow" + // can mean anything. seatbelt and bubblewrap deny every socket in every + // mode, so asserting that "allow" REACHES the network would demand a + // capability those platforms deliberately do not offer. AppContainer + // grants internetClient through a hand-marshalled SID_AND_ATTRIBUTES + // array, and this is the only check that can tell whether it took effect. + if (b === "appcontainer") { + const curl = Bun.which("curl") + // No output flag at all. Two earlier probes died on one: `-o /dev/null` + // made curl try to create C:\dev\null, and `-o NUL` failed INSIDE the + // container with exit 23 (CURLE_WRITE_ERROR) — curl had connected and + // received the response and could not write it, i.e. the network worked + // and the probe did not. Both were reported as network results. + const target = "https://example.com" + const curlCmd = `curl -m 5 -sf https://example.com` + if (!curl) { + checks.push({ + name: "network egress blocked in deny mode", + pass: true, + skipped: true, + detail: "curl not available - skipped", + }) + } else { + // The HOST first, unsandboxed. Without it, "the sandbox blocked this" + // and "this machine is offline" are indistinguishable and the check + // can only ever report an inconclusive skip. + const reachable = + Bun.spawnSync([curl, "-m", "5", "-sf", target], { stdout: "ignore", stderr: "ignore" }).exitCode === 0 + /** Only these mean the network itself was refused. curl reports its own + * problems with other codes, and blaming the sandbox for those is how + * the last two probes lied. */ + const refused = (status: number) => status === 6 || status === 7 || status === 28 + const allowed = await run(curlCmd, "allow") + if (!reachable) { + checks.push({ + name: "network egress blocked in deny mode", + pass: true, + skipped: true, + detail: "this machine has no outbound connectivity - inconclusive", + }) + } else if (allowed.status !== 0 && !refused(allowed.status)) { + checks.push({ + name: "network egress blocked in deny mode", + pass: true, + skipped: true, + detail: `the probe itself failed under allow (curl exit ${allowed.status}${firstLine(allowed.stderr) ? `, ${firstLine(allowed.stderr)}` : ", no stderr"}) - inconclusive`, + }) + } else if (allowed.status !== 0) { + checks.push({ + name: "network egress blocked in deny mode", + pass: false, + detail: `the host reached ${target} and the sandbox could not: the capability grant is not taking effect (exit ${allowed.status}${firstLine(allowed.stderr) ? `, ${firstLine(allowed.stderr)}` : ", no stderr"})`, + }) + } else { + // Both halves, because either alone is satisfiable by a sandbox that + // does nothing: "deny blocks" passes on a machine with no network, + // and "allow reaches" passes on one with no containment. + checks.push({ name: "network egress works in allow mode", pass: true }) + const denied = await run(curlCmd, "deny") + checks.push({ + name: "network egress blocked in deny mode", + pass: denied.status !== 0, + detail: denied.status === 0 ? "egress succeeded despite deny" : undefined, + }) + } + } + } } finally { try { fs.rmSync(outside, { force: true }) @@ -954,6 +2328,17 @@ export namespace Sandbox { try { fs.rmSync(work, { recursive: true, force: true }) } catch {} + // The workspace was a mkdtemp, so the profile derived from it is + // throwaway too. Without this every self-test run orphaned a profile and + // an AppData\Local\Packages folder — visible as a different package SID + // on each run. A real project reuses one profile forever and must not be + // cleaned up this way. + if (b === "appcontainer") { + try { + const { AppContainer } = await import("./appcontainer") + AppContainer.removeProfile(appContainerProfile([work])) + } catch {} + } } return { backend: b, available: true, checks, ok: checks.filter((c) => !c.skipped).every((c) => c.pass) } diff --git a/backend/cli/src/science/kernel/interpreter.ts b/backend/cli/src/science/kernel/interpreter.ts index 41724537..8ebbb861 100644 --- a/backend/cli/src/science/kernel/interpreter.ts +++ b/backend/cli/src/science/kernel/interpreter.ts @@ -3,6 +3,8 @@ import { constants } from "node:fs" import path from "node:path" import z from "zod" import type { KernelStartOptions } from "./types" +import { Environment } from "@/package/environment" +import { Instance } from "@/project/instance" export const KernelEnvironmentName = z .string() @@ -47,21 +49,41 @@ async function executable(file: string) { /** * Resolve a named project Python environment without accepting arbitrary paths. * - * Named venv or Conda-prefix environments live under `.venv/`. The - * conventional `.venv` layout and host interpreter remain fallbacks for the - * default `python` environment so existing projects work without setup. + * Three places are consulted, in order: the environment `package_install` + * manages for this project, then `.venv/`, then — for the default + * environment only — the conventional `.venv` and the host interpreter. + * + * The managed one comes first deliberately. It is the only one this product + * creates and installs into, so a name that `package_install` has provisioned + * must resolve to that environment rather than to a same-named directory a + * project happens to carry. Both remain project-scoped names; neither accepts + * an arbitrary path. */ export async function pythonEnvironment(projectRoot: string, input?: string): Promise { const environmentName = normalizeKernelEnvironmentName(input) - const roots = [path.join(projectRoot, ".venv", environmentName)] + // Instance is not always provided — `pythonEnvironment` is reachable from + // contexts with no project bound, and throwing there would turn "no managed + // environment" into "the resolver crashed". + const managed = (() => { + try { + return Environment.directory(Instance.project.id, environmentName) + } catch { + return undefined + } + })() + const roots = [...(managed ? [managed] : []), path.join(projectRoot, ".venv", environmentName)] if (environmentName === "python") roots.push(path.join(projectRoot, ".venv")) - const candidates = roots.map(layout) + const candidates = roots.map((root) => ({ root, ...layout(root) })) for (const candidate of candidates) { if (!(await executable(candidate.binary))) continue return { binary: candidate.binary, environmentName, + // Only for the managed one. `environment` is the DIRECTORY the sandbox has + // to be granted and the kernel is bound to; a project's own `.venv` needs + // neither, because it already lives inside the workspace. + ...(candidate.root === managed ? { environment: managed } : {}), env: { VIRTUAL_ENV: path.dirname(candidate.bin), PATH: [candidate.bin, process.env.PATH].filter(Boolean).join(path.delimiter), diff --git a/backend/cli/src/science/kernel/process.ts b/backend/cli/src/science/kernel/process.ts index 3dd5bc88..4079fb84 100644 --- a/backend/cli/src/science/kernel/process.ts +++ b/backend/cli/src/science/kernel/process.ts @@ -155,6 +155,32 @@ export namespace KernelProcessIdentity { throw new Error("Kernel manager returned a process without trusted durable containment registration") } + /** The start token for a pid, or undefined where the platform cannot supply + * one. Exported so callers holding a persisted pid — an installer claim, say + * — can capture the same value `capture()` stores for a kernel. */ + export function startToken(pid: number) { + return token(pid) + } + + /** + * Liveness for a bare pid + token pair, the shape a persisted record has + * after a restart when no `ChildProcess` survives. + * + * Applies the same fallback rule as `matches`: when no token was captured — + * Windows, or a read that failed — liveness alone is sufficient. Demanding a + * token match there would report every Windows process as dead, which for + * the installer claim would mark every environment permanently suspect. + */ + export function running(pid: number, value?: string) { + try { + process.kill(pid, 0) + } catch { + return false + } + if (!value) return true + return token(pid) === value + } + export function matches(proc: ChildProcess, identity?: KernelProcess) { if (!identity || proc.pid !== identity.pid || proc.exitCode !== null) return false try { diff --git a/backend/cli/src/science/kernel/registry.ts b/backend/cli/src/science/kernel/registry.ts index b1265aa1..6654ed4f 100644 --- a/backend/cli/src/science/kernel/registry.ts +++ b/backend/cli/src/science/kernel/registry.ts @@ -69,6 +69,17 @@ type Entry = { incarnation: number | null executionCount: number environment: KernelEnvironment | null + /** + * Directory of the managed package environment this kernel is bound to, or + * null for the host interpreter. + * + * Deliberately NOT the field above: `environment` is a `KernelEnvironment`, + * the kernel's runtime context (cwd, sandbox platform). Merging the two would + * bind kernels to the wrong thing. Deliberately not part of `KernelIdentity` + * either — that tuple is hashed into the storage key, so adding to it would + * orphan every persisted record. + */ + boundEnvironment: string | null startedAt: number | null lastActivityAt: number | null authority: ExecutionAuthority.Decision | null @@ -247,6 +258,7 @@ function restore(value: z.infer) { incarnation: value.incarnation, executionCount: value.execution_count, environment: null, + boundEnvironment: null, startedAt: null, lastActivityAt: value.last_activity_at, authority: null, @@ -292,6 +304,7 @@ const record = (identity: KernelIdentity) => { incarnation: null, executionCount: 0, environment: null, + boundEnvironment: null, startedAt: null, lastActivityAt: null, authority: null, @@ -785,6 +798,14 @@ const entry = async (identity: KernelIdentity, options?: KernelStartOptions, han async (kernel) => { if (stale()) return abort() value.environment = kernel.environment ?? null + // The managed environment DIRECTORY this kernel was started against, so + // `restartEnvironment` can find it after a non-additive install. Distinct + // from `environment` above, which is the kernel's runtime context (cwd, + // sandbox platform) and has nothing to do with packages — the two are + // carried separately precisely so they cannot be confused. Left null + // after the rebase, which meant the filter matched nothing and a version + // change silently kept a kernel holding stale imports. + value.boundEnvironment = options?.environment ?? null value.process = kernel.process ?? null value.ownershipID = kernel.process?.ownershipID ?? processOwnership.id value.authority = current @@ -1165,6 +1186,36 @@ export namespace KernelRuntime { return identity } + /** + * Restart every kernel bound to a package environment, leaving every other + * kernel untouched. Called only for a non-additive change: a module already + * loaded into a live interpreter stays at its old version in memory while the + * files on disk say otherwise, and a silently stale module is worse than an + * obvious restart. + * + * NOT to be confused with the entry's existing `environment` field, which is + * a `KernelEnvironment` — the kernel's runtime context (cwd, sandbox + * platform) and nothing to do with installed packages. The package binding is + * carried separately as `boundEnvironment` precisely to keep the two apart. + * + * `environment` is the environment *directory*, which is what a kernel binds + * to — the caller resolves the name through `Environment.directory` so this + * never has to know the project layout. + * + * Releasing rather than restarting in place is deliberate: kernels are lazy, + * so the next cell boots a fresh one against the new interpreter. Eagerly + * respawning here would pay the startup cost for kernels the session may + * never touch again. + */ + export async function restartEnvironment(projectID: string, environment: string) { + const values = Array.from(records().entries.values()).filter( + (value) => value.identity.projectID === projectID && value.boundEnvironment === environment, + ) + // Sequential, not Promise.all: `release` mutates the shared records map, + // and the set is small by construction — one project's live kernels. + for (const value of values) await release(value.identity) + } + export async function release(identity: KernelIdentity) { const value = records().entries.get(key(identity)) if (!value) return diff --git a/backend/cli/src/science/kernel/types.ts b/backend/cli/src/science/kernel/types.ts index b5fa20fd..495953b9 100644 --- a/backend/cli/src/science/kernel/types.ts +++ b/backend/cli/src/science/kernel/types.ts @@ -36,8 +36,8 @@ export const KernelEnvironment = z.object({ sandbox: z.object({ requested: z.boolean(), enforced: z.boolean(), - backend: z.enum(["seatbelt", "bubblewrap", "none"]), - network: z.enum(["allow", "deny"]), + backend: z.enum(["seatbelt", "bubblewrap", "appcontainer", "none"]), + network: z.enum(["deny", "allowlist", "allow"]), platform: z.string(), available: z.boolean(), tool: z.string().optional(), @@ -99,7 +99,7 @@ export interface ExecuteOptions { * Kernel managers consume this snapshot instead of re-reading mutable config. */ export interface KernelSandboxPolicy { readonly enabled: boolean - readonly network: "allow" | "deny" + readonly network: "deny" | "allowlist" | "allow" readonly allowWrite: readonly string[] readonly onUnavailable: "warn" | "error" | "allow" } @@ -136,6 +136,15 @@ export interface KernelStartOptions { /** Exact backend owner used by Linux's pre-exec subreaper gate. */ linuxOwner?: { pid: number; identity: string } } + /** + * Directory of the managed package environment this kernel binds to. + * + * A start option, never part of `KernelIdentity`: putting it in the identity + * tuple would rekey every persisted record and orphan them. Distinct from + * `KernelEnvironment`, which is the kernel's *runtime* context (cwd, sandbox + * platform) and has nothing to do with installed packages. + */ + environment?: string } export interface KernelProcess { diff --git a/backend/cli/src/server/routes/settings/sandbox.ts b/backend/cli/src/server/routes/settings/sandbox.ts index 843110d3..d767bfc5 100644 --- a/backend/cli/src/server/routes/settings/sandbox.ts +++ b/backend/cli/src/server/routes/settings/sandbox.ts @@ -4,13 +4,18 @@ import z from "zod" import { lazy } from "../../../util/lazy" import { Log } from "../../../util/log" import { Config } from "../../../config/config" +import { Installer } from "@/package/installer" import { Sandbox } from "../../../sandbox/sandbox" const log = Log.create({ service: "settings-sandbox" }) const PatchSchema = z.object({ enabled: z.boolean().optional(), - network: z.enum(["allow", "deny"]).optional(), + network: z.enum(["deny", "allowlist", "allow"]).optional(), + // Bounded the way main bounds allowWrite: this is attacker-reachable input + // that ends up in a proxy's host allowlist, so an unbounded array of + // unbounded strings is not something to accept just because it is new. + allowHosts: z.array(z.string().trim().min(1).max(256)).max(64).optional(), allowWrite: z.array(z.string().trim().min(1).max(4096)).max(64).optional(), onUnavailable: z.enum(["warn", "error", "allow"]).optional(), requireProjectTrust: z.boolean().optional(), @@ -32,7 +37,18 @@ async function currentConfig() { export const SandboxSettingsRoutes = lazy(() => new Hono() // Current config + backend availability. - .get("/", async (c) => c.json({ config: await currentConfig(), status: Sandbox.describe() })) + // `blocked` is a PREREQUISITE, not a status: on Windows an AppContainer can + // only be granted access to paths its user owns, so a machine-wide Python + // cannot be read by a sandboxed process however healthy it is. Nothing else + // in this payload can express that, and without it the settings panel shows + // a green sandbox on a machine where every install will fail. + .get("/", async (c) => + c.json({ + config: await currentConfig(), + status: Sandbox.describe(), + blocked: (await Installer.blocked().catch(() => undefined)) ?? null, + }), + ) // Persist a partial config patch (machine-wide / global). .put("/", validator("json", PatchSchema), async (c) => { @@ -45,8 +61,14 @@ export const SandboxSettingsRoutes = lazy(() => ...(roots ? { allowWrite: [...new Set(roots.map((value) => value.canonical!))] } : {}), } log.info("updating sandbox config", { keys: Object.keys(patch) }) + // `next`, not `patch`: main canonicalises the writable roots first, and + // persisting the raw patch would drop that. await Config.setSandbox(next) - return c.json({ config: await currentConfig(), status: Sandbox.describe() }) + return c.json({ + config: await currentConfig(), + status: Sandbox.describe(), + blocked: (await Installer.blocked().catch(() => undefined)) ?? null, + }) }) // Run the empirical containment self-test. diff --git a/backend/cli/src/session/prompt.ts b/backend/cli/src/session/prompt.ts index 027073db..410c01dd 100644 --- a/backend/cli/src/session/prompt.ts +++ b/backend/cli/src/session/prompt.ts @@ -867,6 +867,7 @@ export namespace SessionPrompt { const system = [ ...(await SystemPrompt.environment(model, sessionID)), ...(await SystemPrompt.compute()), + ...(await SystemPrompt.packages()), ...(await InstructionPrompt.system()), ...(SKILL_ROUTING_AGENTS.has(agent.name) ? [await SystemPrompt.availableSkills(agent.permission)] : []), ] diff --git a/backend/cli/src/session/system.ts b/backend/cli/src/session/system.ts index 1667e004..7ac2da9a 100644 --- a/backend/cli/src/session/system.ts +++ b/backend/cli/src/session/system.ts @@ -9,6 +9,7 @@ import { Config } from "../config/config" import { Skill } from "../skill" import { PermissionNext } from "../permission/next" import { ComputePrompt } from "../compute/prompt" +import { PackagePrompt } from "../package/prompt" export namespace SystemPrompt { export function instructions() { @@ -23,6 +24,22 @@ export namespace SystemPrompt { return [await ComputePrompt.system(value)] } + /** + * Governed package installation. Injected unconditionally, for the same + * reason `compute()` is: it has to pre-empt every skill, reference file and + * third-party document that says `pip install`. + * + * 199 of the 293 shipped `SKILL.md` files mention `pip install`. Editing them + * would be neither necessary nor sufficient — reference files are never + * intercepted by the skill tool, and skills cloned from GitHub are not this + * repo's to edit. A block on every request reaches all of them. + */ + export async function packages(projectID?: string) { + // Defaults to the live project so the injection site stays a bare call; + // tests pass an explicit id (or omit it for the empty rendering). + return [await PackagePrompt.system(projectID ?? Instance.project.id)] + } + /** When the user message begins with `/` matching an installed * skill, the model should invoke the skill tool immediately and * silently — zero text output before the tool call. */ diff --git a/backend/cli/src/shell/shell.ts b/backend/cli/src/shell/shell.ts index b84d57e3..2eb67dc4 100644 --- a/backend/cli/src/shell/shell.ts +++ b/backend/cli/src/shell/shell.ts @@ -202,6 +202,60 @@ export namespace Shell { } const BLACKLIST = new Set(["fish", "nu"]) + /** + * How to hand a shell exactly one command to run. + * + * `-c` is not universal, and assuming it was cost a full Windows debugging + * cycle. `cmd.exe` takes `/c`; given `-c` it does not error, it starts an + * INTERACTIVE shell — so on a real machine every sandboxed command printed the + * cmd banner and a prompt, ran nothing, and exited 0. Silent success is the + * worst possible failure here, because the sandbox self-test read that banner + * as a process token and reported a containment failure that had not happened. + * + * `session/prompt.ts` has always known this, but its table is declared inside + * a function and closes over the command, so it could not be reused and the + * sandbox path kept its own wrong copy. This is the single source of truth; + * that table stays only because it also sources rc files, which a sandboxed + * command must NOT do. + */ + export function invocation(shell: string, command: string): string[] { + switch (family(shell)) { + case "cmd": + // `/d /s /c`, the same shape Node uses for every Windows spawn. `/s` + // makes cmd strip exactly the first and last quote of the tail and take + // the rest verbatim, which is the only deterministic way to hand it a + // command containing quotes; `/d` skips AutoRun registry commands, so a + // sandboxed command cannot be prefixed by machine-local configuration. + return ["/d", "/s", "/c", command] + case "powershell": + return ["-NoProfile", "-Command", command] + default: + return ["-c", command] + } + } + + /** + * Which command language this shell speaks. + * + * Not just which FLAG it takes. `cmd.exe` has no `printf` and no `cat`, so a + * caller composing a command has to know the family too — the sandbox + * self-test wrote its probe file with `printf hi > f && cat f`, which on + * Windows failed for the plain reason that neither command exists, and read + * as the sandbox being unable to write inside its own workspace. + * + * Split on BOTH separators and drop `.exe` unconditionally, rather than + * branching on `process.platform`: that branch would be untestable from a + * Linux CI box, which is where this has to be verified since the machine that + * exposed the bug is not one the suite can run on. A POSIX file named + * literally `cmd.exe` would be read as cmd; that is not a real shell. + */ + export function family(shell: string): "cmd" | "powershell" | "posix" { + const name = (shell.split(/[\\/]/).pop() ?? shell).toLowerCase().replace(/\.exe$/, "") + if (name === "cmd" || name === "command") return "cmd" + if (name === "powershell" || name === "pwsh") return "powershell" + return "posix" + } + function exists(p: string) { try { return fs.existsSync(p) diff --git a/backend/cli/src/tool/bash.ts b/backend/cli/src/tool/bash.ts index 6679c3d3..447d51aa 100644 --- a/backend/cli/src/tool/bash.ts +++ b/backend/cli/src/tool/bash.ts @@ -16,7 +16,9 @@ import { Shell } from "@/shell/shell" import { BashArity } from "@/permission/arity" import { Truncate } from "./truncation" import { OpenScience } from "@/openscience" +import { Refuse } from "@/package/refuse" import { Sandbox } from "@/sandbox/sandbox" +import { EgressRuntime } from "@/sandbox/egress-runtime" import { SessionFilesystem } from "@/session/filesystem" import { Filesystem } from "@/util/filesystem" import { Provenance } from "@/science/provenance/store" @@ -134,6 +136,12 @@ const parser = lazy(async () => { // TODO: we may wanna rename this tool so it works better on other shells export const BashTool = Tool.define("bash", async () => { + // Chosen once, at tool definition, from the machine's preference. The + // per-call shell is re-resolved below against the authority's policy, because + // a shell the sandbox cannot execute is worse than a less pleasant one: on + // Windows `Shell.acceptable()` returns Git Bash under `C:\Program Files`, + // which no ACE can reach, and every sandboxed command dies at 0xC0000142 + // before running. const shell = Shell.acceptable() log.info("bash tool using shell", { shell }) @@ -198,6 +206,12 @@ export const BashTool = Tool.define("bash", async () => { command.push(child.text) } + // Before any ctx.ask, before the sandbox is composed, before anything + // runs: refusing after prompting would ask the user to approve a + // command that is then refused anyway. + const refusal = Refuse.installer(command) + if (refusal) throw new Error(refusal) + // not an exhaustive list, but covers most common cases if (["cd", "rm", "cp", "mv", "mkdir", "touch", "chmod", "chown", "cat"].includes(command[0])) { const operands = command @@ -304,16 +318,28 @@ export const BashTool = Tool.define("bash", async () => { } // Build the wrapper only after the final authority check, while trust // and filesystem mutations are excluded through durable registration. + // The egress route is resolved here for the same reason: it is part of + // the authority this launch runs with, not of the decision to launch. + const egress = await EgressRuntime.egressFor(current.sandbox) + // The sandbox picks the shell when it is going to confine one. Under no + // sandbox this is the machine's preference unchanged; under one on + // Windows it is a System32 shell, because the preferred one lives where + // no ACE can be added and the container cannot load it. + const usable = Sandbox.shell({ ...current.sandbox, egress }) const sandbox = Sandbox.plan({ command: params.command, - shell, + shell: usable, cwd, workspace: current.writable, readable: [...readable], unreadable: OpenScience.kernelSensitivePaths(), - options: current.sandbox, + options: { ...current.sandbox, egress }, }) - return OpenScience.withSubprocessEnv(process.env, async (env) => { + // sandbox.env carries the HTTP_PROXY-shaped route to the egress proxy. + // Merged over the subprocess env rather than into it, so a stray + // inherited proxy variable cannot outrank the one the sandbox minted. + return OpenScience.withSubprocessEnv(process.env, async (base) => { + const env = { ...base, ...(sandbox.env ?? {}) } let child: ReturnType const wrapped = await CommandRuntime.wrap({ file: sandbox.file, diff --git a/backend/cli/src/tool/biology/notebook.ts b/backend/cli/src/tool/biology/notebook.ts index 22e236c3..90029498 100644 --- a/backend/cli/src/tool/biology/notebook.ts +++ b/backend/cli/src/tool/biology/notebook.ts @@ -8,6 +8,7 @@ import { Shell } from "@/shell/shell" import { Instance } from "@/project/instance" import { OpenScience } from "@/openscience" import { Sandbox } from "@/sandbox/sandbox" +import { EgressRuntime } from "@/sandbox/egress-runtime" import { ExecutionAuthority } from "@/project/execution" import { AuthoritySignal } from "@/project/authority-signal" import { AuthorityProcessLedger } from "@/project/authority-process" @@ -187,6 +188,7 @@ async function getKernel(sessionID: string): Promise { }) // Confine the kernel to the workspace when execution sandboxing is on: it // runs arbitrary agent-authored code and shares Bash's threat model. + const egress = await EgressRuntime.egressFor(current.sandbox) const sandboxed = Sandbox.wrapArgv({ file: pythonBin, args: ["-u", scriptPath], @@ -194,7 +196,7 @@ async function getKernel(sessionID: string): Promise { readable: current.readable, extraWritable: [scriptPath, configPath, cachePath], unreadable: OpenScience.kernelSensitivePaths(), - options: current.sandbox, + options: { ...current.sandbox, egress }, }) const launch = WindowsJobLauncher.wrap({ file: sandboxed.file, args: sandboxed.args }) const proc = (() => { @@ -204,6 +206,7 @@ async function getKernel(sessionID: string): Promise { env: { ...OpenScience.kernelEnv(process.env), ...OpenScience.pythonThreadCapEnv(process.env), + ...(sandboxed.env ?? {}), ATLAS_CLI_CONFIG_PATH: configPath, MPLCONFIGDIR: path.join(cachePath, "matplotlib"), XDG_CACHE_HOME: path.join(cachePath, "xdg"), diff --git a/backend/cli/src/tool/notebook.ts b/backend/cli/src/tool/notebook.ts index fb400f34..8954bb94 100644 --- a/backend/cli/src/tool/notebook.ts +++ b/backend/cli/src/tool/notebook.ts @@ -8,7 +8,10 @@ import { Shell } from "@/shell/shell" import { Instance } from "@/project/instance" import { OpenScience } from "@/openscience" import { SessionFilesystem } from "@/session/filesystem" +import { Environment } from "@/package/environment" +import { Installer } from "@/package/installer" import { Sandbox } from "@/sandbox/sandbox" +import { EgressRuntime } from "@/sandbox/egress-runtime" import { KernelQueue } from "@/science/kernel/queue" import { KernelProcessIdentity } from "@/science/kernel/process" import { KernelRuntime } from "@/science/kernel/registry" @@ -223,7 +226,40 @@ interface RawPayload { execution_count: number } -async function findPython(override?: string): Promise<{ binary: string; version?: string }> { +/** + * The interpreter a kernel runs. A managed environment's own interpreter wins + * when it exists; otherwise the host's. + * + * The fallback is deliberate. Failing closed on a missing environment would + * make a typo'd name indistinguishable from a broken machine — the exact + * failure mode this design started from, where a missing pip, a severed + * network and a read-only site-packages all surfaced as one opaque error. + */ +export async function findPython( + override?: string, + environment?: string, +): Promise<{ binary: string; version?: string }> { + if (environment) { + // Run it, don't just stat it — the same check the PATH candidates below have + // always used. A venv's `Scripts\python.exe` on Windows is a REDIRECTOR that + // resolves its base interpreter from `pyvenv.cfg` at startup; when that + // resolution fails the file still exists, so an existence check hands the + // kernel a binary that cannot start. The observable was the redirector's own + // message, `No Python at '...'`, surfacing from a kernel-startup failure with + // nothing to connect it to the environment that produced it. + const bin = Installer.interpreter(environment) + if (await Bun.file(bin).exists()) { + // try/catch, not just an exit-code check: spawn THROWS on a file that + // exists but is not executable (EACCES), so a bare check would replace a + // broken environment with a crash. The candidate loop below has always + // been wrapped for the same reason. + try { + const proc = Bun.spawn([bin, "--version"], { stdout: "ignore", stderr: "ignore" }) + await proc.exited + if (proc.exitCode === 0) return { binary: bin } + } catch {} + } + } const candidates = override ? [override] : ["python3", "python"] for (const bin of candidates) { try { @@ -317,7 +353,10 @@ class PythonKernel implements Kernel { this.configPath = configPath this.cachePath = cachePath - const interpreter = await findPython(opts?.binary) + const interpreter = await findPython(opts?.binary, opts?.environment) + // baseRoots, not base: on POSIX pyvenv.cfg's `home` is /bin, so + // granting it alone leaves the standard library under /lib unreachable. + const base = opts?.environment ? await Installer.baseRoots(opts.environment) : [] const workspace = opts?.sessionID ? await SessionFilesystem.processWriteRoots(opts.sessionID) : [Instance.directory, Instance.worktree] @@ -327,11 +366,24 @@ class PythonKernel implements Kernel { // Confine the kernel to the workspace when the execution sandbox is on: the // runtime runs arbitrary agent-authored code — the same threat model as the // bash tool — so it must not be able to escape the boundary bash respects. + const egress = await EgressRuntime.egressFor({ + enabled: policy.enabled, + network: opts?.sandboxNetwork ?? policy.network, + allowWrite: [...policy.allowWrite], + onUnavailable: policy.onUnavailable, + }) const sandboxed = Sandbox.wrapArgv({ file: interpreter.binary, args: ["-u", scriptPath], workspace, - readable, + // The environment and the interpreter it delegates to are READ-only. A + // writable environment would let arbitrary kernel code pip-install into it + // over the same allowlisted egress, reopening through the notebook tool the + // bypass the bash-tool refusal closes. And a venv is not a complete Python: + // its interpreter delegates to the installation named in `pyvenv.cfg`, which + // an AppContainer reaches only if granted — without it the redirector reports + // `No Python at '...'` for an interpreter that is present and working. + readable: [...readable, ...(opts?.environment ? [opts.environment, ...base] : [])], extraWritable: [scriptPath, configPath, cachePath, ...(opts?.extraWritable ?? [])], unreadable: OpenScience.kernelSensitivePaths(), options: { @@ -339,6 +391,7 @@ class PythonKernel implements Kernel { network: opts?.sandboxNetwork ?? policy.network, allowWrite: [...policy.allowWrite], onUnavailable: policy.onUnavailable, + egress, }, }) const cwd = opts?.cwd ?? (opts?.sessionID ? await SessionFilesystem.workspace(opts.sessionID) : Instance.directory) diff --git a/backend/cli/src/tool/package.ts b/backend/cli/src/tool/package.ts new file mode 100644 index 00000000..1143b267 --- /dev/null +++ b/backend/cli/src/tool/package.ts @@ -0,0 +1,300 @@ +import z from "zod" +import { Environment } from "../package/environment" +import { Installer } from "../package/installer" +import { InstallerR } from "../package/installer-r" +import { Requirement } from "../package/requirement" +import { Instance } from "../project/instance" +import { KernelProcessIdentity } from "../science/kernel/process" +import { KernelRuntime } from "../science/kernel/registry" +import { Tool } from "./tool" + +/** The public index, shown on the card and matched by the permission system. + * Redacted through `Requirement.redact` so a credentialled mirror never puts + * a secret on the card and never fragments a standing grant. */ +const DEFAULT_INDEX = Requirement.redact("https://pypi.org/simple") + +export const PackageTool = Tool.define("package_install", { + description: [ + "Install packages into a managed, named environment that kernels can use.", + "This is the only way to add packages. Shell installers (pip, uv pip, conda, poetry) are refused.", + "An environment is scoped to one language: Python packages go to a python environment, R packages to an R environment.", + "A fully-satisfied request installs nothing — check the environment inventory in your context before calling.", + "Installing restarts kernels bound to that environment only when the change is not purely additive.", + "Environments are provisioned with uv when it is installed, otherwise with the interpreter's own venv module.", + // Windows only. bubblewrap and seatbelt read any path the user can read, so + // this whole consideration is meaningless there — and a tool description is + // sent on every request, so unconditional platform trivia is a cost every + // Linux and macOS user pays forever for advice they can never use. + ...(process.platform === "win32" + ? [ + "On Windows, prefer uv and suggest it if a user hits an environment problem: an AppContainer can only be granted read access to paths the user owns, so a machine-wide Python (C:\\Python312, C:\\Program Files\\Python) can never be used by a sandboxed run, while uv installs interpreters under the user's own profile.", + ] + : []), + ].join("\n"), + parameters: z.object({ + packages: z + .array(z.string().trim().min(1)) + .min(1) + .describe("Package requirements to install, e.g. ['numpy', 'pandas>=2.2']"), + environment: z + .string() + .trim() + .min(1) + .max(64) + .regex(/^[A-Za-z0-9][A-Za-z0-9._-]*$/) + .default("default") + .describe("Target environment. Created on first install."), + language: z + .enum(["python", "r"]) + .default("python") + .describe("Environment language. An environment is scoped to one."), + source: z + .boolean() + .default(false) + .describe("Allow source builds. Default is wheels-only, which is faster and more reliable."), + wait: z + .boolean() + .default(true) + .describe( + "Wait for the install to finish and report the versions it landed. Set false only for a long install; you then get no versions back and must not claim it succeeded.", + ), + }), + async execute(params, ctx) { + // Before anything else, and before the approval card: on Windows the + // sandbox may be unable to reach ANY interpreter, and no amount of asking + // the user to approve an install changes that. Checked here rather than in + // `Sandbox.plan` because answering it runs candidate interpreters — far too + // expensive for a path every sandboxed command takes, and entirely + // affordable for one deliberate install. + // + // This is the surface that matters. Nobody runs `sandbox status`; they hit + // it when the agent tries. Failing here means the agent is told the remedy + // at the moment it needs it, instead of inventing one — which it did, + // advising an admin grant that cannot be obtained and would not have + // helped. + // + // Skipped when THIS environment already exists on a base the sandbox can be + // granted, which `probe` answers and nothing else does: it is a statement + // about a machine, and an environment that already works is a counterexample + // to it. Without this a user whose environment was built correctly is + // refused an install into it and told to go set up the tool they already + // have — which is what happened when a parsing bug hid uv's interpreters. + const project = Instance.project.id + const name = params.environment + const directory = Environment.directory(project, name) + const usable = await Installer.probe(directory) + .then((tool) => tool.kind === "existing") + .catch(() => false) + if (!usable) { + const blocked = await Installer.blocked().catch(() => undefined) + if (blocked) throw new Error(blocked) + } + + const before = await Environment.read(project, name) + + // Parsed for its names only. Resolution happens after approval — the card + // shows the request, so approving two names must not silently approve the + // closure they pull in. + const language = params.language ?? "python" + // R package names are case-sensitive and `.` is meaningful (`data.table`), + // so the PEP 503 normalisation Requirement.parse applies is wrong for them: + // it would turn data.table into data-table and never match what CRAN + // installed. Python keeps the parser, which is what makes `numpy>=2.4` and + // `pandas[performance]` safe to accept. + const parsed = + language === "r" + ? params.packages.map((p) => ({ name: p.trim(), extras: [], specifier: "", marker: "", url: "" })) + : params.packages.map((p) => Requirement.parse(p)) + + // Already satisfied: skip outright — no card, no install, no restart. + // Nothing privileged happens, so nothing needs approving, and a + // fully-satisfied request is not worth a turn. + // A bare name is satisfied by any installed version. A requirement that + // constrains *which* version — a specifier, a direct URL, or extras that + // may not have been installed — is never assumed satisfied: `six==1.17.0` + // against an installed 1.16.0 is an upgrade, and skipping it would silently + // no-op the request and wrongly report the change as additive. Deciding + // that properly needs PEP 440 comparison; deferring to pip, which already + // implements it and no-ops when it is genuinely satisfied, is both correct + // and cheaper than reimplementing it here. + const constrained = parsed.some((p) => p.specifier || p.url || p.extras.length) + const satisfied = before && !constrained && parsed.every((p) => before.installed[p.name]) + if (satisfied) { + // The same metadata shape as the install branch below, deliberately. + // Two shapes would make every consumer — the UI, the session record, a + // test — handle a union whose arms differ only in which keys exist. + const versions = Object.fromEntries(parsed.map((p) => [p.name, before.installed[p.name]!])) + const listed = Object.entries(versions) + .map(([k, v]) => `${k} ${v}`) + .join(", ") + return { + title: `Already installed · ${name}`, + output: `Nothing to do. ${listed} already present in ${name}.`, + metadata: { + environment: name, + installed: false, + ok: true, + additive: true, + versions, + total: before.total, + }, + } + } + + const pattern = Requirement.pattern({ + packages: params.packages, + environment: name, + index: DEFAULT_INDEX, + }) + + ctx.metadata({ title: `Install · ${name}`, metadata: { environment: name, packages: params.packages } }) + + // The ordinary contract, not modal's. Installing a library must not be + // gated more strictly than running arbitrary code, because it costs + // nothing — hence no digest and no spendFilter entry. The command string is + // readable, and changes whenever the approved action changes, so the prompt + // reappears for free when it should. + await ctx.ask({ + permission: "package_install", + patterns: [pattern], + always: ["install*"], + metadata: { environment: name, packages: params.packages, index: DEFAULT_INDEX }, + }) + + // Dispatch without waiting. The lock is still taken, so a second install + // queues exactly as it would otherwise; what changes is that this turn does + // not hold open for it. The claim is written before returning so a CLI + // restart mid-install can tell "still running" from "died", and the output + // deliberately reports no versions — there are none yet, and inventing them + // is precisely what the contract forbids. + if (params.wait === false) { + const running = Environment.lock(project, name, async () => { + await Environment.claim(project, name, process.pid, KernelProcessIdentity.startToken(process.pid)) + try { + const value = await install() + await Environment.release(project, name) + return value + } catch (error) { + // Recorded, not swallowed. Nothing is awaiting this promise, so a + // discarded rejection meant the agent was told "started installing" + // and could never learn otherwise: no manifest written, claim cleared, + // no trace anywhere. The failure now replaces the claim and surfaces + // in the environment inventory on the next request. + await Environment.fail(project, name, error instanceof Error ? error.message : String(error)) + throw error + } + }) + // Already recorded above; this only stops an unobserved rejection + // surfacing as a process-level warning with no context. + running.catch(() => undefined) + return { + title: `Installing · ${name}`, + output: [ + `Started installing ${params.packages.join(", ")} into ${name}.`, + `It is still running. Do not execute in this environment, and do not report a version, until a later call confirms what landed.`, + ].join("\n"), + metadata: { environment: name, installed: false, ok: true, additive: true, versions: {}, total: 0 }, + } + } + + return await Environment.lock(project, name, install) + + async function install() { + // Only the backend differs by language. The card, the lock, the manifest + // write and the additivity check are identical, because they are + // properties of the contract rather than of pip or CRAN. + const r = language === "r" + if (r) await InstallerR.create(directory) + const tool = r ? undefined : await Installer.probe(directory) + if (tool) await Installer.create(directory, tool) + + // Two different questions, deliberately asked of two different sources. + // `owned` is what the environment itself holds and becomes the manifest. + // `seen` is everything the interpreter can import, inherited packages + // included, and is the only correct basis for the restart decision: + // requesting a version the host already provides installs nothing + // locally, so an owned-set comparison reads the NEXT version as an + // addition and leaves stale modules loaded in live kernels. + const owned = () => (r ? InstallerR.freeze(directory) : Installer.freeze(directory)) + const seen = () => (r ? InstallerR.resolved(directory) : Installer.resolved(directory)) + const snapshot = await seen() + + // Report what pip is doing while it does it. A tool with no dedicated + // renderer otherwise shows its name and an ellipsis for the whole call — + // measured at 1m37s for a pytorch install, with pip reporting phase and + // size the entire time. `metadata` is re-read as the call runs, so this + // reaches the running row; `input` is fixed at call time and cannot. + const progress = (status: string) => + ctx.metadata({ + title: `Install · ${name}`, + metadata: { environment: name, packages: params.packages, progress: status }, + }) + + const result = r + ? await InstallerR.install({ directory, packages: params.packages, signal: ctx.abort }) + : await Installer.install({ + directory, + packages: params.packages, + index: "", + source: params.source, + signal: ctx.abort, + onProgress: progress, + }) + + // Modern pip builds every wheel before the install phase, so a build + // failure aborts before anything is committed — verified during design, + // where a failing package's cleanly-resolving dependency was downloaded + // and still not installed. There is no subset to keep and nothing to + // retry, so this reports the cause and stops. R is checked explicitly by + // InstallerR, because install.packages() only warns and still exits 0. + if (!result.ok) throw new Error(r ? InstallerR.explain(result.log) : Installer.explain(result.log)) + + const after = await seen() + const held = await owned() + const names = parsed.map((p) => p.name) + const versions = r ? await InstallerR.verify(directory, names) : await Installer.verify(directory, names) + + const requested = Array.from(new Set([...(before?.requested ?? []), ...parsed.map((p) => p.name)])) + await Environment.write(project, { + name, + // Defaulted here rather than relied on from the schema: `execute` is + // reachable without zod having applied parameter defaults, and an + // undefined language used to produce a manifest that could never be + // read back. + language, + requested, + installed: held, + total: Object.keys(held).length, + createdAt: before?.createdAt ?? Date.now(), + updatedAt: Date.now(), + }) + + // Kernels bind to the environment *directory*, so that is what identifies + // them here — not the name, which the registry never sees. + const additive = Environment.additive(snapshot, after) + if (!additive) await KernelRuntime.restartEnvironment(project, directory) + + const landed = Object.entries(versions) + .map(([k, v]) => `${k} ${v}`) + .join(", ") + return { + title: `Installed · ${name}`, + output: [ + `Installed into ${name}: ${landed || "(nothing reported)"}.`, + `${Object.keys(held).length} packages total in the environment.`, + additive + ? "Purely additive — running kernels kept their state." + : "Not purely additive — kernels bound to this environment restarted and their variables were discarded.", + ].join("\n"), + metadata: { + environment: name, + installed: true, + ok: true, + additive, + versions, + total: Object.keys(held).length, + }, + } + } + }, +}) diff --git a/backend/cli/src/tool/registry.ts b/backend/cli/src/tool/registry.ts index 73937715..5b4a79a4 100644 --- a/backend/cli/src/tool/registry.ts +++ b/backend/cli/src/tool/registry.ts @@ -34,6 +34,7 @@ import { ScienceTools } from "./science" import { ProvenanceTools } from "./provenance" import { NotebookTool, PythonTool } from "./notebook" import { RKernelTool, RTool } from "./rkernel" +import { PackageTool } from "./package" import { AtlasTool } from "./atlas" import { AtlasRecordTool } from "./atlas-record" import { ArtifactSnapshotTool } from "./artifact-snapshot" @@ -171,6 +172,7 @@ export namespace ToolRegistry { AtlasRecordTool, PythonTool, RTool, + PackageTool, ArtifactTool, ComputeJobTool, ...custom.filter((tool) => !compatibility.has(tool.id) && tool.id !== PythonTool.id && tool.id !== RTool.id), diff --git a/backend/cli/src/tool/rkernel.ts b/backend/cli/src/tool/rkernel.ts index 9866cb87..d752f66d 100644 --- a/backend/cli/src/tool/rkernel.ts +++ b/backend/cli/src/tool/rkernel.ts @@ -8,11 +8,15 @@ import { Shell } from "@/shell/shell" import { Instance } from "@/project/instance" import { OpenScience } from "@/openscience" import { SessionFilesystem } from "@/session/filesystem" +import { Environment } from "@/package/environment" +import { Installer } from "@/package/installer" import { Sandbox } from "@/sandbox/sandbox" +import { EgressRuntime } from "@/sandbox/egress-runtime" import { KernelQueue } from "@/science/kernel/queue" import { KernelProcessIdentity } from "@/science/kernel/process" import { KernelRuntime } from "@/science/kernel/registry" import { KernelEnvironmentMutation } from "@/science/kernel/environment-mutation" +import { KernelEnvironmentName } from "@/science/kernel/interpreter" import { AtlasEnvironment } from "@/science/kernel/types" import type { Kernel, @@ -601,6 +605,9 @@ const RFields = { .max(1024) .optional() .describe("Script path associated with this execution, when applicable"), + environment: KernelEnvironmentName.optional().describe( + "Optional project R environment provisioned by package_install. Omit it or use 'default' for the host runtime.", + ), timeout: z.number().default(120_000).describe("Execution timeout in ms (default: 120s, max: 600s)"), } diff --git a/backend/cli/test/compute/jobs.test.ts b/backend/cli/test/compute/jobs.test.ts index bc4a400e..cc1d0aa7 100644 --- a/backend/cli/test/compute/jobs.test.ts +++ b/backend/cli/test/compute/jobs.test.ts @@ -1331,14 +1331,18 @@ describe("ComputeJobs Modal governance", () => { { root, workspace: tmp.path, modal, credentials, provider }, ), }) - const delivery = async (attempts = 100): Promise => { - const current = await ComputeJobs.get(job.id, { root, workspace: tmp.path }) - if (current?.lifecycle?.delivery === "failed") return current - if (!attempts) throw new Error("Timed out waiting for recoverable Modal Volume") - await Bun.sleep(20) - return delivery(attempts - 1) - } - const failed = await delivery() + // ComputeJobs.wait, not a hand-rolled poll on the persisted lifecycle. This + // is the only pair of tests in the file that rolled its own, and it is the + // pair that then calls retry() -- which refuses while a recovery is still + // active. `delivery: "failed"` is written by deferModal, but `active` is + // cleared in a .finally() AFTER that, so the state a poll can see becomes + // true strictly before retry() will accept it. On a loaded CI runner the + // gap opened and the run failed with + // "Compute job ... already has an active recovery" -- the same race, one test over + // + // wait() already encodes exactly retry()'s precondition: terminal, not + // pending, and not active. + const failed = await ComputeJobs.wait(job.id, { root, workspace: tmp.path, timeout: 10_000 }) expect(failed.status).toBe("succeeded") expect(failed.exit_code).toBe(0) @@ -2093,3 +2097,35 @@ describe("ComputeJobs project boundaries", () => { } }) }) + +describe("ComputeJobs ssh transport network policy", () => { + test("allowlist is relaxed, because the allowlist proxy is HTTP-only", () => { + // The bug: under "allowlist" the ssh CLIENT was wrapped in a severed + // network namespace whose only exit is an HTTP proxy socket. ssh does not + // read HTTP_PROXY and cannot use it, so remote jobs failed with an opaque + // connection error on the shipped default policy. + expect(ComputeJobs.transportNetwork("allowlist")).toBe("allow") + }) + + test("an explicit deny is honoured, not overridden", () => { + // The line between relaxing a default nobody chose and overriding an + // instruction somebody gave. A user who set "deny" means it. + expect(ComputeJobs.transportNetwork("deny")).toBe("deny") + }) + + test("allow is unchanged", () => { + expect(ComputeJobs.transportNetwork("allow")).toBe("allow") + }) + + test("the relaxation is reported, never silent", async () => { + // Reporting "allowlist" for a process actually running unconfined would be + // worse than the original bug, so the launch path reports the policy it + // applied and says why. + const source = await Bun.file(new URL("../../src/compute/jobs.ts", import.meta.url).pathname).text() + expect(source.includes("network left unconfined for the ssh transport")).toBe(true) + // The ssh branch reports the policy it APPLIED. The local-job branch below + // it still reports authority.sandbox.network, and correctly so — there the + // requested policy is the applied one, and nothing is relaxed. + expect(source.includes("const network = transportNetwork(authority.sandbox.network)")).toBe(true) + }) +}) diff --git a/backend/cli/test/installation/native-package-matrix.test.ts b/backend/cli/test/installation/native-package-matrix.test.ts index e1bff7d2..cbb38e79 100644 --- a/backend/cli/test/installation/native-package-matrix.test.ts +++ b/backend/cli/test/installation/native-package-matrix.test.ts @@ -19,8 +19,17 @@ async function pack(dir: string, output: string) { proc.exited, ]) if (code !== 0) throw new Error(stderr || stdout) - const result = JSON.parse(stdout) as { filename?: string }[] - const file = result[0]?.filename + // `npm pack --json` changed shape: npm 11 and earlier return an array of + // entries, npm 12 returns an object keyed by package name. Indexing [0] + // yields undefined on npm 12, so this failed with "did not return a tarball" + // on any machine with a current npm while still passing on CI's older one. + // Accept both rather than pinning a version — the test is about npm's + // package *selection*, not about its output format. + const parsed = JSON.parse(stdout) as unknown + const entries = (Array.isArray(parsed) ? parsed : Object.values(parsed as Record)) as { + filename?: string + }[] + const file = entries[0]?.filename if (!file) throw new Error(`npm pack did not return a tarball for ${dir}`) return path.join(output, file) } diff --git a/backend/cli/test/openscience-env.test.ts b/backend/cli/test/openscience-env.test.ts index fdf574a3..f969d799 100644 --- a/backend/cli/test/openscience-env.test.ts +++ b/backend/cli/test/openscience-env.test.ts @@ -141,3 +141,55 @@ test("mergeByokEnv supports the canonical direct-provider set and aliases", () = expect(merged.DEEPSEEK_API_KEY).toBe("deepseek-user") expect(merged.PERPLEXITY_API_KEY).toBe("perplexity-user") }) + +test("kernel env filtering matches Windows environment keys, which are case-insensitive", () => { + // Windows presents these as Path, SystemRoot, windir, ComSpec — never the + // uppercase spellings the allowlist was written in. Every comparison here was + // exact, so on Windows a kernel launched with NO PATH and NO SystemRoot. + // Measured downstream as `CreateProcess ... Win32 203` (ERROR_ENVVAR_NOT_FOUND) + // when the AppContainer launcher tried to start the interpreter. + const filtered = OpenScience.filterEnvForKernel({ + Path: "C:\\Python312;C:\\Windows\\system32", + SystemRoot: "C:\\Windows", + windir: "C:\\Windows", + ComSpec: "C:\\Windows\\system32\\cmd.exe", + PATHEXT: ".COM;.EXE;.BAT", + Tmp: "C:\\Users\\me\\AppData\\Local\\Temp", + }) + expect(filtered["Path"]).toBe("C:\\Python312;C:\\Windows\\system32") + expect(filtered["SystemRoot"]).toBe("C:\\Windows") + expect(filtered["windir"]).toBe("C:\\Windows") + expect(filtered["ComSpec"]).toBe("C:\\Windows\\system32\\cmd.exe") + expect(filtered["PATHEXT"]).toBe(".COM;.EXE;.BAT") + expect(filtered["Tmp"]).toBe("C:\\Users\\me\\AppData\\Local\\Temp") +}) + +test("kernel env filtering still withholds secrets, whatever their casing", () => { + // This is a security boundary, and the fix above widened the match. Folding + // case must not become a hole: a kernel runs arbitrary user code, so anything + // outside the runtime allowlist has to stay on the host regardless of how it + // is spelled. + const filtered = OpenScience.filterEnvForKernel({ + PATH: "/usr/bin", + OPENROUTER_API_KEY: "thk_managed_openrouter", + openrouter_api_key: "thk_lowercase", + ANTHROPIC_API_KEY: "sk-ant-secret", + AWS_SECRET_ACCESS_KEY: "aws-secret", + GITHUB_TOKEN: "ghp_secret", + OPENSCIENCE_API_BASE: "https://atlas.test", + Path_To_Secrets: "should not pass", + }) + expect(filtered["PATH"]).toBe("/usr/bin") + for (const key of [ + "OPENROUTER_API_KEY", + "openrouter_api_key", + "ANTHROPIC_API_KEY", + "AWS_SECRET_ACCESS_KEY", + "GITHUB_TOKEN", + "OPENSCIENCE_API_BASE", + // Prefix entries without a trailing underscore must stay EXACT matches, or + // folding case turns "PATH" into a prefix that swallows unrelated names. + "Path_To_Secrets", + ]) + expect(filtered[key]).toBeUndefined() +}) diff --git a/backend/cli/test/package/binding.test.ts b/backend/cli/test/package/binding.test.ts new file mode 100644 index 00000000..09b0ba1e --- /dev/null +++ b/backend/cli/test/package/binding.test.ts @@ -0,0 +1,212 @@ +import { expect, test } from "bun:test" +import fs from "fs" +import path from "path" +import { Environment } from "../../src/package/environment" +import { Installer } from "../../src/package/installer" +import { findPython } from "../../src/tool/notebook" +import { tmpdir } from "../fixture/fixture" + +const python = Bun.which("python3") + +const read = (relative: string) => Bun.file(new URL(relative, import.meta.url).pathname).text() + +test.skipIf(!python)("a kernel bound to an environment runs that environment's interpreter", async () => { + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, { kind: "venv", binary: python! }) + expect(fs.existsSync(Installer.interpreter(env))).toBe(true) + expect((await findPython(undefined, env)).binary).toBe(Installer.interpreter(env)) +}) + +test("a name with no environment behind it says so, and says where it looked", async () => { + // This branch used to fall back to the host interpreter, on the reasoning that + // failing closed makes a typo'd name look like a broken machine. main answers + // the same worry differently and better: it fails, and the error names every + // candidate path plus the way to ask for the default. Keep main's. + const { pythonEnvironment, KernelEnvironmentUnavailable } = await import("../../src/science/kernel/interpreter") + await using dir = await tmpdir() + const failed = await pythonEnvironment(dir.path, "nope-not-here").then( + () => undefined, + (error) => error, + ) + expect(failed).toBeInstanceOf(KernelEnvironmentUnavailable) + expect(String(failed)).toContain("nope-not-here") +}) + +test("no environment at all is the unchanged host lookup", async () => { + expect((await findPython()).binary).toBeString() +}) + +test("the managed environment is consulted before a project's own .venv", async () => { + // `package_install` only ever provisions the managed one, so a name it has + // provisioned must resolve there rather than to a same-named directory the + // project happens to carry. + const source = await read("../../src/science/kernel/interpreter.ts") + const roots = source.slice(source.indexOf("const managed ="), source.indexOf("const candidates =")) + expect(roots.indexOf("managed")).toBeLessThan(roots.indexOf(".venv")) +}) + +test("only the managed environment is handed to the sandbox as a grant", async () => { + // A project's own .venv lives inside the workspace and is already reachable; + // the managed one lives under the cache root and is not. + const source = await read("../../src/science/kernel/interpreter.ts") + expect(source).toContain("candidate.root === managed ? { environment: managed } : {}") +}) + +test("changing the environment gets a different kernel rather than reusing one", async () => { + const source = await read("../../src/science/kernel/registry.ts") + // main keys the identity by environmentName, so two names cannot share a + // kernel and no staleness comparison is needed. This branch carried a + // separate `boundEnvironment` for the same property; main's is load-bearing. + const identity = source.slice(source.indexOf("export type KernelIdentity"), source.indexOf("type KernelCell")) + expect(identity).toContain("environmentName") + expect(source).toContain("identity.environmentName ?") +}) + +test("both kernel tools accept an environment name, bounded", async () => { + for (const file of ["../../src/tool/notebook.ts", "../../src/tool/rkernel.ts"]) { + const source = await read(file) + // KernelEnvironmentName, not a bare string: it bounds length and refuses + // path separators, so a name can never address a directory outside the two + // places `pythonEnvironment` looks. + expect(source, file).toContain("KernelEnvironmentName") + } +}) + +test("the derived directory is stable for a project and name", () => { + expect(Environment.directory("p", "e")).toBe(Environment.directory("p", "e")) +}) + +// Everything above is structural. This runs a real kernel and asks it what it +// can import — the only assertion that can distinguish "the parameter is +// plumbed" from "the kernel actually runs in that environment". + +async function context() { + const { executionSession } = await import("../fixture/fixture") + const session = await executionSession() + return { + sessionID: session.id, + messageID: "", + callID: "", + agent: "research", + abort: AbortSignal.any([]), + messages: [], + metadata: () => {}, + ask: async () => {}, + } +} + +const live = (await import("../../src/sandbox/sandbox")).Sandbox.backend() !== "none" && Boolean(python) + +test.skipIf(!live)( + "a package installed into one environment is importable there and absent elsewhere", + async () => { + const { Instance } = await import("../../src/project/instance") + const { NotebookTool } = await import("../../src/tool/notebook") + const { PackageTool } = await import("../../src/tool/package") + await using tmp = await tmpdir({ git: true }) + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const install = await PackageTool.init() + await install.execute( + { packages: ["tqdm"], environment: "bound", language: "python", source: false, wait: true }, + await context(), + ) + + const notebook = await NotebookTool.init() + const inBound = await notebook.execute( + { + code: "import tqdm; print('BOUND', tqdm.__version__)", + kernel: "k-bound", + environment: "bound", + timeout: 120_000, + }, + await context(), + ) + expect(inBound.metadata.output).toContain("BOUND") + + // A second REAL environment, created by installing something else into + // it. Naming an environment that does not exist would not prove + // isolation: findPython deliberately falls back to the host + // interpreter, and the host may well have tqdm — measured, it does. + await install.execute( + { packages: ["six"], environment: "other", language: "python", source: false, wait: true }, + await context(), + ) + const elsewhere = await notebook.execute( + { + code: [ + "import importlib.util as u", + "print('TQDM', 'FOUND' if u.find_spec('tqdm') else 'ABSENT')", + "print('SIX', 'FOUND' if u.find_spec('six') else 'ABSENT')", + ].join("\n"), + kernel: "k-other", + environment: "other", + timeout: 120_000, + }, + await context(), + ) + // If binding were cosmetic both kernels would see the same site-packages + // and this would report TQDM FOUND — the exact false green a + // plumbing-only test cannot rule out. + expect(elsewhere.metadata.output).toContain("TQDM ABSENT") + expect(elsewhere.metadata.output).toContain("SIX FOUND") + }, + }) + }, + 600_000, +) + +test.skipIf(!live)( + "an additive install keeps kernel state, a version change discards it", + async () => { + const { Instance } = await import("../../src/project/instance") + const { NotebookTool } = await import("../../src/tool/notebook") + const { PackageTool } = await import("../../src/tool/package") + await using tmp = await tmpdir({ git: true }) + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const install = await PackageTool.init() + const notebook = await NotebookTool.init() + // ONE context for every cell. A fresh session id is a different + // KernelIdentity and therefore a different kernel, so re-deriving it + // per cell would look exactly like a restart and make this test pass + // for the wrong reason — measured: `marker` was undefined between two + // consecutive cells with no install between them. + const shared = await context() + const cell = async (code: string) => + (await notebook.execute({ code, kernel: "k-restart", environment: "restart", timeout: 120_000 }, shared)) + .metadata.output + + await install.execute( + { packages: ["six==1.16.0"], environment: "restart", language: "python", source: false, wait: true }, + shared, + ) + await cell("marker = 'alive'") + expect(await cell("print(marker)")).toContain("alive") + + // Additive: a package that was not there before. A live kernel stays + // correct, because a new module imports on first use. + const additive = await install.execute( + { packages: ["tqdm"], environment: "restart", language: "python", source: false, wait: true }, + shared, + ) + expect(additive.metadata.additive).toBe(true) + expect(await cell("print(marker)")).toContain("alive") + + // Not additive: six changes version. The module already loaded into the + // interpreter would stay at 1.16.0 in memory while the files on disk say + // 1.17.0 — silently wrong, which is why this restarts. + const changed = await install.execute( + { packages: ["six==1.17.0"], environment: "restart", language: "python", source: false, wait: true }, + shared, + ) + expect(changed.metadata.additive).toBe(false) + expect(await cell("print(marker)")).toContain("NameError") + }, + }) + }, + 900_000, +) diff --git a/backend/cli/test/package/dispatch.test.ts b/backend/cli/test/package/dispatch.test.ts new file mode 100644 index 00000000..17bb8d29 --- /dev/null +++ b/backend/cli/test/package/dispatch.test.ts @@ -0,0 +1,274 @@ +import { expect, test } from "bun:test" +import { Environment } from "../../src/package/environment" +import { KernelProcessIdentity } from "../../src/science/kernel/process" + +const seed = async (project: string, name: string) => + Environment.write(project, { + name, + language: "python", + requested: [], + installed: {}, + total: 0, + createdAt: 1, + updatedAt: 1, + }) + +test("a claim by a live process reconciles as still running", async () => { + const project = "proj_reconcile_live" + await seed(project, "live") + // This process is by definition alive, so it stands in for a live installer. + await Environment.claim(project, "live", process.pid, KernelProcessIdentity.startToken(process.pid)) + const outcomes = await Environment.reconcile(project) + expect(outcomes.find((o) => o.name === "live")?.outcome).toBe("running") +}) + +test("a live pid with no token available still reconciles as running", async () => { + // Windows has neither the /proc nor the `ps -o lstart=` branch, so the token + // is undefined there for every process. Treating that as unproven would mark + // every Windows install unknown forever and every environment permanently + // suspect. `matches()` already takes liveness alone as sufficient in that + // case; reconcile follows the same rule. + const project = "proj_reconcile_untokened" + await seed(project, "untokened") + await Environment.claim(project, "untokened", process.pid, undefined) + const outcomes = await Environment.reconcile(project) + expect(outcomes.find((o) => o.name === "untokened")?.outcome).toBe("running") +}) + +test("a claim by a dead pid reconciles as unknown, not as success", async () => { + const project = "proj_reconcile_dead" + await seed(project, "dead") + // process.execPath, not /bin/true: that path does not exist on macOS (it is + // /usr/bin/true there), and posix_spawn's ENOENT surfaced as this test + // failing for a reason unrelated to reconcile. Bun is by definition present. + const proc = Bun.spawn([process.execPath, "-e", ""], { stdout: "ignore", stderr: "ignore" }) + const pid = proc.pid + const captured = KernelProcessIdentity.startToken(pid) + await proc.exited + // `await proc.exited` is not the same as "the pid is gone": a just-reaped + // child can stay signalable briefly, and on a macOS runner it did — the + // claim then reconciled as "running" and this failed for a reason that had + // nothing to do with reconcile. Wait for the premise to actually hold, and + // fail loudly if it never does rather than asserting on a live pid. + for (let i = 0; i < 100; i++) { + try { + process.kill(pid, 0) + } catch { + break + } + await Bun.sleep(20) + } + expect(() => process.kill(pid, 0)).toThrow() + await Environment.claim(project, "dead", pid, captured) + const outcomes = await Environment.reconcile(project) + // Not "fine": pip has no transactions, so an interrupted install may have + // written a partial tree. Silently trusting it is how a half-installed + // environment becomes a mystery ImportError three turns later. + expect(outcomes.find((o) => o.name === "dead")?.outcome).toBe("unknown") +}) + +test("a claim whose token no longer matches reconciles as unknown", async () => { + // pid reuse: the number is alive but it is a different process. + const project = "proj_reconcile_reused" + await seed(project, "reused") + await Environment.claim(project, "reused", process.pid, "not-the-real-token") + const outcomes = await Environment.reconcile(project) + expect(outcomes.find((o) => o.name === "reused")?.outcome).toBe("unknown") +}) + +test("a corrupt claim file reconciles as unknown rather than throwing", async () => { + const project = "proj_reconcile_corrupt" + await seed(project, "corrupt") + await Environment.claim(project, "corrupt", process.pid, undefined) + await Bun.write(Environment.claimPath(project, "corrupt"), "{not json") + const outcomes = await Environment.reconcile(project) + expect(outcomes.find((o) => o.name === "corrupt")?.outcome).toBe("unknown") +}) + +test("reconcile clears a resolved claim so it is not reported twice", async () => { + const project = "proj_reconcile_once" + await seed(project, "once") + await Environment.claim(project, "once", 999_999, "gone") + expect(await Environment.reconcile(project)).toHaveLength(1) + expect(await Environment.reconcile(project)).toHaveLength(0) +}) + +test("reconcile keeps a claim that is still running, so a later check still sees it", async () => { + const project = "proj_reconcile_keep" + await seed(project, "keep") + await Environment.claim(project, "keep", process.pid, KernelProcessIdentity.startToken(process.pid)) + expect(await Environment.reconcile(project)).toHaveLength(1) + expect(await Environment.reconcile(project)).toHaveLength(1) +}) + +test("a project with no claims reconciles to nothing", async () => { + expect(await Environment.reconcile("proj_no_claims_at_all")).toEqual([]) +}) + +const python = Bun.which("python3") +const live = (await import("../../src/sandbox/sandbox")).Sandbox.backend() !== "none" && Boolean(python) + +test.skipIf(!live)( + "wait:false returns before the install finishes, then the package really lands", + async () => { + const { Instance } = await import("../../src/project/instance") + const { PackageTool } = await import("../../src/tool/package") + const { Installer } = await import("../../src/package/installer") + const { executionSession, tmpdir } = await import("../fixture/fixture") + await using tmp = await tmpdir({ git: true }) + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const session = await executionSession() + const ctx = { + sessionID: session.id, + messageID: "", + callID: "", + agent: "research", + abort: AbortSignal.any([]), + messages: [], + metadata: () => {}, + ask: async () => {}, + } + const tool = await PackageTool.init() + const result = await tool.execute( + { packages: ["tqdm"], environment: "async", language: "python" as const, source: false, wait: false }, + ctx as never, + ) + + // No versions, and it does not claim success — there is nothing to + // report yet, and inventing a version is what the contract forbids. + expect(result.metadata.installed).toBe(false) + expect(result.metadata.versions).toEqual({}) + expect(result.output).toContain("still running") + + // The work really is in flight: taking the lock waits it out, and the + // package is present afterwards. + const directory = Environment.directory(Instance.project.id, "async") + await Environment.lock(Instance.project.id, "async", async () => {}) + expect((await Installer.verify(directory, ["tqdm"]))["tqdm"]).toMatch(/^\d/) + + // And the claim is cleared once it finishes, so a later reconcile does + // not report a phantom install. + expect(await Environment.reconcile(Instance.project.id)).toEqual([]) + }, + }) + }, + 600_000, +) + +test("a claim survives a hard kill of its process and reconciles as unknown", async () => { + // The scenario the claim/token machinery exists for, which nothing exercised: + // the CLI is killed while an install runs, and on restart a claim file points + // at a pid that is gone. Every other test here uses a process that exited + // normally, or a synthetic pid. This one kills a live process outright and + // watches the SAME claim flip from running to unknown. + const project = "proj_reconcile_killed" + await seed(project, "killed") + + const proc = Bun.spawn([process.execPath, "-e", "setTimeout(() => {}, 60_000)"], { + stdout: "ignore", + stderr: "ignore", + }) + const pid = proc.pid + await Environment.claim(project, "killed", pid, KernelProcessIdentity.startToken(pid)) + + // Alive: the claim is true right now, so reconcile must leave it alone. + const before = await Environment.reconcile(project) + expect(before.find((o) => o.name === "killed")?.outcome).toBe("running") + + proc.kill("SIGKILL") + await proc.exited + for (let i = 0; i < 100; i++) { + try { + process.kill(pid, 0) + } catch { + break + } + await Bun.sleep(20) + } + expect(() => process.kill(pid, 0)).toThrow() + + // Dead: pip has no transactions, so an interrupted install may have left a + // partial tree. "unknown" is the only honest answer; "fine" would turn into a + // mystery ImportError several turns later. + const after = await Environment.reconcile(project) + expect(after.find((o) => o.name === "killed")?.outcome).toBe("unknown") + // And it is cleared, so a later boot does not re-report a resolved claim. + expect(await Environment.reconcile(project)).toEqual([]) +}, 60_000) + +test("a real install can be interrupted, and the environment is still usable after", async () => { + // The other half: after an abort, the environment must not be wedged. A + // half-written tree that no later install can repair would be worse than the + // interruption itself. + const { Sandbox } = await import("../../src/sandbox/sandbox") + const python = Bun.which("python3") + if (Sandbox.backend() === "none" || !python) return + const { Installer } = await import("../../src/package/installer") + const { tmpdir } = await import("../fixture/fixture") + + await using dir = await tmpdir() + const env = (await import("path")).join(dir.path, "env") + await Installer.create(env, await Installer.probe(env)) + + const control = new AbortController() + const running = Installer.install({ + directory: env, + packages: ["scipy"], + index: "", + source: false, + signal: control.signal, + }) + await Bun.sleep(600) + control.abort() + await running.catch(() => undefined) + + // The environment survives: a subsequent install into it works. + const after = await Installer.install({ directory: env, packages: ["tqdm"], index: "", source: false }) + expect(after.ok, after.log).toBe(true) + expect((await Installer.verify(env, ["tqdm"]))["tqdm"]).toMatch(/^\d/) +}, 600_000) + +test("a failed detached install is recorded, not swallowed", async () => { + // `wait: false` returns immediately and nothing awaits the promise, so a + // rejection used to be discarded outright: no manifest written, the claim + // released cleanly, no trace anywhere. The agent had been told "started + // installing" and could never learn otherwise. + const project = "proj_failed_detached" + await seed(project, "broken") + await Environment.fail(project, "broken", "No wheel is published for xyzzy under the current policy.") + const outcomes = await Environment.reconcile(project) + const found = outcomes.find((o) => o.name === "broken") + expect(found?.outcome).toBe("failed") + expect(found?.message).toContain("No wheel") + // Reported once, then cleared — a failure that repeated every request would + // be worse than one that vanished. + expect(await Environment.reconcile(project)).toEqual([]) +}) + +test("an unresolved install reaches the agent's contract, not just a log", async () => { + // reconcile() had no production caller at all: built, tested, and reached + // only by its own tests. It now runs where the result can act — the + // capability block injected on every request. + const { PackagePrompt } = await import("../../src/package/prompt") + const project = "proj_warning_surfaces" + await seed(project, "halfdone") + await Environment.claim(project, "halfdone", 999_998, "definitely-gone") + const block = await PackagePrompt.system(project) + expect(block).toContain("UNRESOLVED INSTALLS") + expect(block).toContain("halfdone") + expect(block).toContain("outcome is unknown") + // Self-clearing: the next request is clean. + expect(await PackagePrompt.system(project)).not.toContain("UNRESOLVED INSTALLS") +}) + +test("a recorded failure is reported to the agent with its cause", async () => { + const { PackagePrompt } = await import("../../src/package/prompt") + const project = "proj_failure_surfaces" + await seed(project, "nowheel") + await Environment.fail(project, "nowheel", "No wheel is published for xyzzy.") + const block = await PackagePrompt.system(project) + expect(block).toContain("FAILED and nothing was landed") + expect(block).toContain("No wheel is published") +}) diff --git a/backend/cli/test/package/environment.test.ts b/backend/cli/test/package/environment.test.ts new file mode 100644 index 00000000..567ba3d2 --- /dev/null +++ b/backend/cli/test/package/environment.test.ts @@ -0,0 +1,181 @@ +import { expect, test } from "bun:test" +import path from "path" +import { Global } from "../../src/global" +import { Environment } from "../../src/package/environment" + +const project = "proj_test" + +test("the manifest lives under data and the directory under cache", () => { + // Not interchangeable: the manifest is the source of truth and the directory + // is derived, so a cache cleaner must be able to remove one without + // destroying the record of what the environment is. + expect(Environment.manifest(project, "default")).toBe(path.join(Global.Path.data, "envs", project, "default.json")) + expect(Environment.directory(project, "default")).toBe(path.join(Global.Path.cache, "envs", project, "default")) +}) + +test("a written environment reads back", async () => { + const value = { + name: "e1", + language: "python" as const, + requested: ["numpy"], + installed: { numpy: "2.1.0" }, + total: 1, + createdAt: 1, + updatedAt: 1, + } + await Environment.write(project, value) + expect(await Environment.read(project, "e1")).toEqual(value) +}) + +test("writing a manifest that could not be read back throws instead", async () => { + // Regression. JSON.stringify drops undefined-valued keys, so a caller that + // omits one — a tool invoked without zod having applied its defaults — wrote + // a manifest that read() then rejected. The environment existed on disk, + // held installed packages, and was invisible to the inventory: silent, and + // indistinguishable from "never created" at every call site. + const bad = { name: "hole", requested: [], installed: {}, total: 0, createdAt: 1, updatedAt: 1 } + await expect(Environment.write(project, bad as never)).rejects.toThrow("unreadable") + expect(await Environment.read(project, "hole")).toBeUndefined() +}) + +test("a manifest that round-trips is exactly what write validated", async () => { + const value = { + name: "roundtrip", + language: "python" as const, + requested: ["numpy"], + installed: { numpy: "2.1.0" }, + total: 1, + createdAt: 1, + updatedAt: 2, + } + await Environment.write(project, value) + expect(await Environment.read(project, "roundtrip")).toEqual(value) +}) + +test("reading an environment that does not exist is undefined, not a throw", async () => { + expect(await Environment.read(project, "never-created")).toBeUndefined() +}) + +test("list returns every environment for the project and none from another", async () => { + await Environment.write(project, { + name: "e2", + language: "python", + requested: [], + installed: {}, + total: 0, + createdAt: 1, + updatedAt: 1, + }) + await Environment.write("other_project", { + name: "e3", + language: "python", + requested: [], + installed: {}, + total: 0, + createdAt: 1, + updatedAt: 1, + }) + const names = (await Environment.list(project)).map((e) => e.name) + expect(names).toContain("e2") + expect(names).not.toContain("e3") +}) + +test("a corrupt manifest is skipped, not fatal to the whole listing", async () => { + // One hand-edited or half-written file must not make every environment in + // the project invisible. + const fs = await import("fs/promises") + const file = Environment.manifest(project, "corrupt") + await fs.mkdir(path.dirname(file), { recursive: true }) + await Bun.write(file, "{not json") + const names = (await Environment.list(project)).map((e) => e.name) + expect(names).not.toContain("corrupt") + expect(names).toContain("e2") +}) + +// The additivity rule decides whether kernels restart, so each direction is +// asserted separately rather than as one truthiness check. +test("adding a package is additive", () => { + expect(Environment.additive({ numpy: "2.1.0" }, { numpy: "2.1.0", pandas: "2.2.0" })).toBe(true) +}) + +test("an unchanged set is additive", () => { + expect(Environment.additive({ numpy: "2.1.0" }, { numpy: "2.1.0" })).toBe(true) +}) + +test("an upgrade is NOT additive — a loaded module would stay stale", () => { + expect(Environment.additive({ numpy: "2.1.0" }, { numpy: "2.2.0" })).toBe(false) +}) + +test("a downgrade is NOT additive", () => { + expect(Environment.additive({ numpy: "2.2.0" }, { numpy: "2.1.0" })).toBe(false) +}) + +test("a removal is NOT additive", () => { + expect(Environment.additive({ numpy: "2.1.0", pandas: "2.2.0" }, { numpy: "2.1.0" })).toBe(false) +}) + +test("the lock serialises two installs into the same environment", async () => { + const order: string[] = [] + const first = Environment.lock(project, "locked", async () => { + order.push("first-start") + await Bun.sleep(50) + order.push("first-end") + }) + const second = Environment.lock(project, "locked", async () => { + order.push("second-start") + }) + await Promise.all([first, second]) + // Not interleaved: a cell that lazily imports a submodule mid-install can + // load a half-written file, so this is correctness, not scheduling. + expect(order).toEqual(["first-start", "first-end", "second-start"]) +}) + +test("a different environment is not blocked by a held lock", async () => { + const order: string[] = [] + const held = Environment.lock(project, "envA", async () => { + await Bun.sleep(80) + order.push("A") + }) + const free = Environment.lock(project, "envB", async () => { + order.push("B") + }) + await Promise.all([held, free]) + expect(order).toEqual(["B", "A"]) +}) + +test("busy() reports the lock while it is held and clears after", async () => { + let seen = false + await Environment.lock(project, "watched", async () => { + seen = Environment.busy(project, "watched") + }) + expect(seen).toBe(true) + expect(Environment.busy(project, "watched")).toBe(false) +}) + +test("the lock releases even when the body throws", async () => { + await Environment.lock(project, "boom", async () => { + throw new Error("install failed") + }).catch(() => {}) + // Otherwise one failed install bricks that environment for the process + // lifetime — the same latching bug the egress runtime shipped with. + expect(Environment.busy(project, "boom")).toBe(false) +}) + +test("a throw in the first holder does not cancel the one queued behind it", async () => { + const order: string[] = [] + const failing = Environment.lock(project, "chain", async () => { + order.push("first") + throw new Error("boom") + }) + const queued = Environment.lock(project, "chain", async () => { + order.push("second") + return "done" + }) + await failing.catch(() => {}) + expect(await queued).toBe("done") + expect(order).toEqual(["first", "second"]) +}) + +test("the lock returns the body's value to its own caller", async () => { + expect(await Environment.lock(project, "value", async () => 42)).toBe(42) +}) diff --git a/backend/cli/test/package/install-live.test.ts b/backend/cli/test/package/install-live.test.ts new file mode 100644 index 00000000..4a1ec857 --- /dev/null +++ b/backend/cli/test/package/install-live.test.ts @@ -0,0 +1,136 @@ +import { afterEach, describe, expect, test } from "bun:test" +import { Environment } from "../../src/package/environment" +import { Installer } from "../../src/package/installer" +import { Refuse } from "../../src/package/refuse" +import { Instance } from "../../src/project/instance" +import { Sandbox } from "../../src/sandbox/sandbox" +import { PackageTool } from "../../src/tool/package" +import type { PermissionNext } from "../../src/permission/next" +import { executionSession, tmpdir } from "../fixture/fixture" + +/** + * The merge gate, stated as an assertion. + * + * The condition this branch has to meet is that a package installs **with the + * user's approval**, under `network: "allowlist"`, on every platform we ship. + * Everything else in `test/package/` tests a component; this tests the claim. + * + * Both halves are asserted together on purpose. A green install with an open + * shell bypass is not the gate met — an agent that never calls the tool never + * shows a card, so the refusal is part of the same claim, not an adjacent + * feature. + * + * Gated on a real sandbox backend and a real interpreter, and skips rather than + * fails without them: a green run on a machine with no sandbox would assert + * nothing. On Linux and macOS CI both are present, so it runs unskipped there. + */ + +const python = Bun.which("python3") +const skip = Sandbox.backend() === "none" || !python + +async function approving() { + const session = await executionSession() + const asks: Array> = [] + return { + asks, + ctx: { + sessionID: session.id, + messageID: "", + callID: "", + agent: "research", + abort: AbortSignal.any([]), + messages: [], + metadata: () => {}, + ask: async (req: Omit) => { + asks.push(req) + }, + }, + } +} + +// A global config write is process-wide and outlives the test that made it, so +// it has to be undone or it leaks into whichever file bun runs next in this +// process — see the same note in test/sandbox/egress-runtime.test.ts. +afterEach(async () => { + const { Global } = await import("../../src/global") + const { Config } = await import("../../src/config/config") + const fs = await import("fs/promises") + const path = await import("path") + for (const name of ["openscience.jsonc", "openscience.json", "config.json"]) { + await fs.rm(path.join(Global.Path.config, name), { force: true }).catch(() => {}) + } + Config.global.reset() +}) + +describe.skipIf(skip)("merge gate: a governed install under network allowlist", () => { + test("the agent's only install route asks for approval and lands the package", async () => { + await using tmp = await tmpdir({ git: true }) + const { Config } = await import("../../src/config/config") + // Stated, not inherited. This used to read the ambient default and assert it + // was "allowlist" — which stopped being true the moment the default became + // main's "deny", and would have gone on passing vacuously if the default had + // drifted the other way. The gate is "an install works UNDER allowlist", so + // the policy is part of the test, not part of the environment. + await Config.setSandbox({ enabled: true, network: "allowlist" }) + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const tool = await PackageTool.init() + const { asks, ctx } = await approving() + const result = await tool.execute( + { packages: ["tqdm"], environment: "gate", language: "python", source: false, wait: true }, + ctx, + ) + + // Approval happened, and it named exactly what ran. + expect(asks).toHaveLength(1) + expect(asks[0]!.permission).toBe("package_install") + expect(asks[0]!.patterns).toEqual(["install tqdm → gate [pypi.org/simple]"]) + + // The package really landed — read back out of the environment, not + // taken from pip's exit code. + expect(result.metadata.versions["tqdm"]).toMatch(/^\d/) + const directory = Environment.directory(Instance.project.id, "gate") + expect((await Installer.verify(directory, ["tqdm"]))["tqdm"]).toMatch(/^\d/) + + // And it happened under that policy, with the sandbox on — read back + // from the trusted resolver rather than from what was written. + const policy = await Config.trustedSandbox() + expect(policy.enabled).toBe(true) + expect(policy.network).toBe("allowlist") + }, + }) + }, 600_000) + + test("the shell route to the same install is refused", () => { + // The exact line measured to succeed on feat/sandbox-network-policy before + // any of this existed: a venv in the writable workspace, pypi allowlisted, + // no tool and no card. If this ever returns undefined the gate is not met + // even when the test above is green. + expect(Refuse.installer(["/w/venv/bin/pip", "install", "tqdm"])).toBeString() + expect(Refuse.installer(["pip", "install", "tqdm"])).toBeString() + expect(Refuse.installer(["uv", "pip", "install", "tqdm"])).toBeString() + expect(Refuse.installer(["python3", "-m", "pip", "install", "tqdm"])).toBeString() + }) + + test("a package with no wheel under the default policy fails with a translated message", async () => { + await using tmp = await tmpdir({ git: true }) + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const directory = Environment.directory(Instance.project.id, "nowheel") + await Installer.create(directory, await Installer.probe(directory)) + const result = await Installer.install({ + directory, + packages: ["this-package-does-not-exist-anywhere-xyzzy"], + index: "", + source: false, + }) + expect(result.ok).toBe(false) + // The raw log reads as "no such package" regardless of which of the + // two things actually happened, which is why explain() exists. + expect(Installer.explain(result.log).length).toBeGreaterThan(0) + }, + }) + }, 600_000) +}) diff --git a/backend/cli/test/package/installer-r.test.ts b/backend/cli/test/package/installer-r.test.ts new file mode 100644 index 00000000..3a454418 --- /dev/null +++ b/backend/cli/test/package/installer-r.test.ts @@ -0,0 +1,136 @@ +import { expect, test } from "bun:test" +import fs from "fs" +import path from "path" +import { Installer } from "../../src/package/installer" +import { InstallerR } from "../../src/package/installer-r" +import { tmpdir } from "../fixture/fixture" + +const rscript = Bun.which("Rscript") +const read = (relative: string) => Bun.file(new URL(relative, import.meta.url).pathname).text() + +test("the library path is derived from the environment directory, beside the interpreter", () => { + // Both language backends derive their binding from one place, so a kernel can + // resolve it before any install has ever run. + expect(Installer.rlibrary("/envs/e")).toBe(path.join("/envs/e", "rlibs")) +}) + +test("the index is CRAN, asserted by value rather than by grepping for a domain", () => { + // Equality on an exported constant, not `source.includes("cran...")`. The + // substring form reads to CodeQL as incomplete URL sanitization — a false + // positive, but the constant is the better design anyway: one named value + // decides where packages come from, and it has to stay in step with + // Egress.DEFAULT_RULES. + expect(InstallerR.REPO).toBe("https://cran.r-project.org") +}) + +test("the index CRAN is allowlisted, or every R install fails closed", async () => { + const { Egress } = await import("../../src/sandbox/egress") + const host = new URL(InstallerR.REPO).hostname + const allowed = Egress.allowed(host, Egress.DEFAULT_RULES) + expect(allowed).toBe(true) +}) + +test("the install targets R_LIBS_USER, never the system library", async () => { + const source = await read("../../src/package/installer-r.ts") + // Writing to the system library would need root and would leak this + // environment's packages into every other project on the machine. + expect(source.includes("R_LIBS_USER")).toBe(true) + expect(source.includes("install.packages")).toBe(true) +}) + +test("lib is passed explicitly, not left to .libPaths() ordering", async () => { + const source = await read("../../src/package/installer-r.ts") + // install.packages() otherwise picks the first writable entry of .libPaths(), + // which on a machine with a user library already configured is the wrong + // directory. + expect(source.includes("lib = lib")).toBe(true) +}) + +test("a failed install is detected even though install.packages only warns", async () => { + const source = await read("../../src/package/installer-r.ts") + // install.packages() signals failure with a warning and still exits 0, so + // without the explicit check a missing package reads as success. + expect(source.includes("quit(status = 1)")).toBe(true) +}) + +test("explain names Bioconductor for a package CRAN does not have", () => { + const log = "Warning message:\npackage ‘DESeq2’ is not available for this version of R" + expect(InstallerR.explain(log)).toContain("Bioconductor") +}) + +test("explain surfaces a missing system header rather than the compile spew", () => { + const log = [" fatal error: libxml/parser.h: No such file or directory", " compilation terminated."].join("\n") + const message = InstallerR.explain(log) + expect(message).toContain("libxml/parser.h") + expect(message).toContain("system librar") +}) + +test("explain passes an unrecognised log through rather than inventing a diagnosis", () => { + expect(InstallerR.explain("something nobody anticipated")).toContain("something nobody anticipated") +}) + +test.skipIf(!rscript)("an empty library reports no packages", async () => { + await using dir = await tmpdir() + const env = path.join(dir.path, "renv") + await InstallerR.create(env) + expect(await InstallerR.freeze(env)).toEqual({}) +}) + +test.skipIf(!rscript)( + "a package CRAN does not have fails rather than reporting success", + async () => { + await using dir = await tmpdir() + const env = path.join(dir.path, "renv") + const result = await InstallerR.install({ directory: env, packages: ["definitelyNotARealCranPackage"] }) + expect(result.ok).toBe(false) + expect(await InstallerR.verify(env, ["definitelyNotARealCranPackage"])).toEqual({}) + }, + 600_000, +) + +test.skipIf(!rscript)( + "a real CRAN package installs into the environment library and reports its version", + async () => { + // The gap the existing live tests left: both of them assert FAILURE paths + // (an empty library, a package CRAN does not have), so nothing anywhere + // proved an R install can succeed at all. + // + // `praise` is pure R, a few kilobytes, and has no dependencies — CRAN + // serves Linux packages as source, so anything with compiled code would be + // testing a toolchain rather than this installer. + await using dir = await tmpdir() + const env = path.join(dir.path, "renv") + const result = await InstallerR.install({ directory: env, packages: ["praise"] }) + expect(result.ok, result.log).toBe(true) + + const versions = await InstallerR.verify(env, ["praise"]) + expect(versions["praise"]).toMatch(/^\d/) + + // It landed in the environment's own library, not a system or user one — + // the whole point of passing `lib` explicitly rather than trusting + // .libPaths() ordering. + expect(Object.keys(await InstallerR.freeze(env))).toContain("praise") + expect(fs.existsSync(path.join(Installer.rlibrary(env), "praise"))).toBe(true) + }, + 900_000, +) + +test.skipIf(!rscript)( + "a second package is additive alongside the first", + async () => { + // Mirrors the Python additivity check: the tool decides whether to restart + // kernels from freeze() before and after, so an R install has to report a + // growing set rather than replacing it. + await using dir = await tmpdir() + const env = path.join(dir.path, "renv") + await InstallerR.install({ directory: env, packages: ["praise"] }) + const before = await InstallerR.freeze(env) + await InstallerR.install({ directory: env, packages: ["R6"] }) + const after = await InstallerR.freeze(env) + expect(Object.keys(after)).toContain("praise") + expect(Object.keys(after)).toContain("R6") + const { Environment } = await import("../../src/package/environment") + expect(Environment.additive(before, after)).toBe(true) + }, + 900_000, +) diff --git a/backend/cli/test/package/installer.test.ts b/backend/cli/test/package/installer.test.ts new file mode 100644 index 00000000..9b73b100 --- /dev/null +++ b/backend/cli/test/package/installer.test.ts @@ -0,0 +1,455 @@ +import { expect, test } from "bun:test" +import fs from "fs" +import path from "path" +import { Installer } from "../../src/package/installer" +import { Sandbox } from "../../src/sandbox/sandbox" +import { tmpdir } from "../fixture/fixture" + +const python = Bun.which("python3") + +test("probe prefers an existing environment directory over any tool", async () => { + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + fs.mkdirSync(path.dirname(Installer.interpreter(env)), { recursive: true }) + fs.writeFileSync(Installer.interpreter(env), "") + expect((await Installer.probe(env)).kind).toBe("existing") +}) + +test("probe picks uv over venv when both are available", async () => { + await using dir = await tmpdir() + // uv is the fast path when present; venv is the guarantee that it is never + // required. + const tool = await Installer.probe(path.join(dir.path, "nothing"), { uv: "/fake/uv", python: "/fake/python3" }) + expect(tool).toEqual({ kind: "uv", binary: "/fake/uv" }) +}) + +test("probe falls back to venv when uv is absent", async () => { + await using dir = await tmpdir() + const tool = await Installer.probe(path.join(dir.path, "nothing"), { uv: undefined, python: "/fake/python3" }) + expect(tool).toEqual({ kind: "venv", binary: "/fake/python3" }) +}) + +test("the remedy names both routes, and never offers to download one", async () => { + await using dir = await tmpdir() + // An opaque failure here reads as a broken machine — the exact symptom this + // whole design started from, where a missing pip, a severed network and a + // read-only site-packages all surfaced as one unreadable error. + const message = await Installer.probe(path.join(dir.path, "nothing"), { uv: undefined, python: undefined }).then( + () => "", + (error: Error) => error.message, + ) + expect(message).toContain("python3-venv") + expect(message).toContain("uv") + expect(message).toContain("never downloads") +}) + +test.skipIf(!python)("creates a venv whose interpreter runs", async () => { + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, { kind: "venv", binary: python! }) + const proc = Bun.spawn([Installer.interpreter(env), "--version"], { stdout: "pipe" }) + expect(await new Response(proc.stdout).text()).toContain("Python 3") +}) + +test.skipIf(!python)("a fresh venv has pip even when the host python3 does not", async () => { + // Verified on Arch during design: python3 ships without pip there, and + // `python3 -m venv` still bootstraps pip from the bundled ensurepip wheel, + // offline. uv is a fast path, never a requirement. + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, { kind: "venv", binary: python! }) + const proc = Bun.spawn([Installer.interpreter(env), "-m", "pip", "--version"], { stdout: "pipe", stderr: "pipe" }) + await proc.exited + expect(proc.exitCode).toBe(0) +}) + +const uv = Bun.which("uv") + +test.skipIf(!uv)( + "an environment created by the uv branch has pip, because install() needs it", + async () => { + // Regression, and the reason `uv venv --seed` exists in create(). Plain + // `uv venv` does NOT bootstrap pip the way `python3 -m venv` does, while + // install() shells out to `python -m pip` regardless of who created the + // environment. Without the seed the uv branch produced an environment the + // installer could not use at all — "No module named pip" from a venv that + // looked perfectly healthy from outside the sandbox. + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, { kind: "uv", binary: uv! }) + const proc = Bun.spawn([Installer.interpreter(env), "-m", "pip", "--version"], { stdout: "pipe", stderr: "pipe" }) + await proc.exited + expect(proc.exitCode).toBe(0) + }, + 120_000, +) + +// The regression this pair exists for, measured in real use: a kernel binds to +// the managed environment as soon as one exists and falls back to the host +// interpreter while it does not, so the FIRST install of anything used to strip +// every host package from every kernel in the project. Install tqdm, lose numpy +// — while the notebook tool still advertised numpy as pre-imported. +const hostHas = (name: string) => { + const proc = Bun.spawnSync([python ?? "python3", "-c", `import ${name}`], { stdout: "ignore", stderr: "ignore" }) + return proc.exitCode === 0 +} + +test.skipIf(!python || !hostHas("numpy"))( + "a fresh environment can still import what the host interpreter had", + async () => { + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, { kind: "venv", binary: python! }) + const proc = Bun.spawn([Installer.interpreter(env), "-c", "import numpy"], { stdout: "ignore", stderr: "pipe" }) + const err = await new Response(proc.stderr).text() + await proc.exited + expect(proc.exitCode, err).toBe(0) + }, + 120_000, +) + +test.skipIf(!python || !hostHas("numpy"))( + "verify reports an inherited package, because the kernel can genuinely use it", + async () => { + // freeze() lists only what the environment OWNS. Since environments inherit + // system site-packages, pip treats a host-provided package as already + // satisfied and installs nothing — so a freeze-based verify answered + // "(nothing reported)" for a request that is, from the user's seat, + // perfectly satisfied. + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, { kind: "venv", binary: python! }) + expect(Object.keys(await Installer.freeze(env))).not.toContain("numpy") + expect((await Installer.verify(env, ["numpy"]))["numpy"]).toMatch(/^\d/) + }, + 120_000, +) + +test.skipIf(!python)( + "verify still reports nothing for a package that is genuinely absent", + async () => { + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, { kind: "venv", binary: python! }) + expect(await Installer.verify(env, ["definitely-not-a-real-distribution-xyzzy"])).toEqual({}) + }, + 120_000, +) + +test.skipIf(!python || !hostHas("numpy"))( + "freeze reports only what the environment owns, not the whole host", + async () => { + // Otherwise `total` is a fact about the machine, the agent's inventory is + // buried under host packages, and additive() compares against the wrong set. + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, { kind: "venv", binary: python! }) + expect(Object.keys(await Installer.freeze(env))).not.toContain("numpy") + }, + 120_000, +) + +test("whichever branch of the ladder creates it, the environment must expose pip", async () => { + // The invariant the bug violated, stated once so a future third branch has + // to satisfy it too rather than quietly repeating the same mistake. + const source = await Bun.file(new URL("../../src/package/installer.ts", import.meta.url).pathname).text() + expect(source.includes('"--seed"')).toBe(true) +}) + +test.skipIf(!python)("create on an existing environment is a no-op, not a rebuild", async () => { + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, { kind: "venv", binary: python! }) + const marker = path.join(env, "marker") + fs.writeFileSync(marker, "keep me") + await Installer.create(env, await Installer.probe(env)) + // Rebuilding would silently discard everything already installed. + expect(fs.existsSync(marker)).toBe(true) +}) + +test.skipIf(!python)("freeze reports name to version for what is installed", async () => { + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, { kind: "venv", binary: python! }) + const frozen = await Installer.freeze(env) + expect(Object.values(frozen).every((v) => /^\d/.test(v))).toBe(true) +}) + +test.skipIf(!python)("freeze normalises names so they compare against parsed requirements", async () => { + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, { kind: "venv", binary: python! }) + const frozen = await Installer.freeze(env) + // Environment.additive compares these keys against Requirement.parse output, + // so both sides must be PEP 503 normalised or an upgrade looks additive. + expect(Object.keys(frozen).every((k) => k === k.toLowerCase() && !k.includes("_"))).toBe(true) +}) + +test.skipIf(!python)("verify reports the version of a module that is present", async () => { + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, { kind: "venv", binary: python! }) + expect((await Installer.verify(env, ["pip"]))["pip"]).toMatch(/^\d/) +}) + +test.skipIf(!python)("verify reports nothing for a module that is absent", async () => { + // Catches an installer that exits 0 without producing a working module. + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, { kind: "venv", binary: python! }) + expect( + (await Installer.verify(env, ["definitely-not-a-real-module"]))["definitely-not-a-real-module"], + ).toBeUndefined() +}) + +// install() is the load-bearing function in this module and everything above +// only exercises what surrounds it. Task 11 proves the whole path end to end; +// these two prove the sandboxed argv composes and runs at all, here, where a +// break is cheap to localise. +const sandboxed = Sandbox.backend() !== "none" && Boolean(python) + +test.skipIf(!sandboxed)( + "a real install through the sandbox lands a real package", + async () => { + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, await Installer.probe(env)) + const result = await Installer.install({ directory: env, packages: ["tqdm"], index: "", source: false }) + expect(result.ok, result.log).toBe(true) + // Not "pip exited 0" — the version has to come back out of the environment. + expect((await Installer.verify(env, ["tqdm"]))["tqdm"]).toMatch(/^\d/) + }, + 300_000, +) + +test.skipIf(!sandboxed)( + "a failed install reports ok:false and a log, and lands nothing", + async () => { + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, await Installer.probe(env)) + const before = await Installer.freeze(env) + const result = await Installer.install({ + directory: env, + packages: ["this-package-does-not-exist-anywhere-xyzzy"], + index: "", + source: false, + }) + expect(result.ok).toBe(false) + expect(result.log.length).toBeGreaterThan(0) + // Modern pip builds every wheel before the install phase, so a failure + // aborts before anything is committed. There is no subset to keep. + expect(await Installer.freeze(env)).toEqual(before) + }, + 300_000, +) + +test("explain translates the wheels-only rejection into what it means", () => { + const log = "ERROR: Could not find a version that satisfies the requirement foo (from versions: none)" + const message = Installer.explain(log) + // Reads as "no such package" but means "no wheel under this policy". + expect(message).toContain("No wheel") + expect(message).toContain("source") + expect(message).toContain("foo") +}) + +test("explain surfaces the cause of a build failure, not pip's summary line", () => { + const log = [ + " #include ", + " ^~~~~~~~~~", + " fatal error: Python.h: No such file or directory", + " compilation terminated.", + " ERROR: Failed building wheel for cffi", + ].join("\n") + const message = Installer.explain(log) + // The summary names the package; the fatal error names the actual missing + // piece, which is what decides whether this is achievable in a sandbox. + expect(message).toContain("Python.h") + expect(message).toContain("cffi") +}) + +test("explain passes an unrecognised log through rather than inventing a diagnosis", () => { + expect(Installer.explain("ERROR: something nobody anticipated")).toContain("something nobody anticipated") +}) + +test.skipIf(!sandboxed)( + "install reports progress as pip works, not only at the end", + async () => { + // The defect this exists for: a pytorch install sat behind an unchanging + // ellipsis for 1m37s while pip reported phase and size the whole time, + // because the output was buffered and only read on completion. + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, await Installer.probe(env)) + const seen: string[] = [] + const result = await Installer.install({ + directory: env, + packages: ["tqdm"], + index: "", + source: false, + onProgress: (s) => seen.push(s), + }) + expect(result.ok, result.log).toBe(true) + expect(seen.length).toBeGreaterThan(0) + // Real pip phrasing, not a placeholder the tool invented. + expect(seen.join("\n")).toMatch(/Collecting|Downloading|Installing|Successfully/i) + // And the full log still survives for explain(), which needs lines that are + // rarely last. + expect(result.log.length).toBeGreaterThan(0) + }, + 300_000, +) + +test.skipIf(!sandboxed)( + "a second environment reuses the shared wheel cache instead of re-downloading", + async () => { + // The cache used to live inside the environment directory, so every new + // environment re-downloaded everything — measured at 34 MB and a full + // download for scipy alone, into an environment created seconds after one + // that already had it. The packages where this hurts are the large ones. + await using dir = await tmpdir() + const first = path.join(dir.path, "one") + const second = path.join(dir.path, "two") + await Installer.create(first, await Installer.probe(first)) + await Installer.create(second, await Installer.probe(second)) + + const a = await Installer.install({ directory: first, packages: ["tqdm"], index: "", source: false }) + expect(a.ok, a.log).toBe(true) + + const seen: string[] = [] + const b = await Installer.install({ + directory: second, + packages: ["tqdm"], + index: "", + source: false, + onProgress: (s) => seen.push(s), + }) + expect(b.ok, b.log).toBe(true) + // pip says so itself when it serves from cache rather than the network. + expect(b.log).toMatch(/cached|Using cached/i) + // And neither install put a cache inside the environment it populated. + expect(fs.existsSync(path.join(second, ".cache"))).toBe(false) + }, + 600_000, +) + +// `source: true` had never been exercised anywhere — not in tests, not in the +// product — while explain() actively tells users "Retry with source builds +// enabled if a compiler and headers are available". A user following our own +// error message would have been the first to run this path. +// +// sgmllib3k is published as an sdist with no wheel, so it is refused under the +// default wheels-only policy and installs only when source builds are allowed. +// That makes the flag's effect observable rather than asserted from argv. +test.skipIf(!sandboxed)( + "wheels-only refuses an sdist-only package, and says what it really means", + async () => { + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, await Installer.probe(env)) + const result = await Installer.install({ + directory: env, + packages: ["sgmllib3k"], + index: "", + source: false, + }) + expect(result.ok).toBe(false) + // The raw log reads as "no such package"; the translation has to say the + // truth, which is that a wheel is missing and source builds are the answer. + const message = Installer.explain(result.log) + expect(message).toContain("No wheel") + expect(message).toContain("source") + expect(await Installer.verify(env, ["sgmllib3k"])).toEqual({}) + }, + 600_000, +) + +test.skipIf(!sandboxed)( + "the same package installs when source builds are allowed", + async () => { + // The escalation explain() advertises, actually performed: a real sdist + // built inside the sandbox, through the allowlist proxy. + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, await Installer.probe(env)) + const result = await Installer.install({ + directory: env, + packages: ["sgmllib3k"], + index: "", + source: true, + }) + expect(result.ok, result.log).toBe(true) + expect((await Installer.verify(env, ["sgmllib3k"]))["sgmllib3k"]).toMatch(/^\d/) + }, + 600_000, +) + +// The version of a package the host interpreter already provides, or undefined. +// The bug below only appears when the requested version MATCHES the host's, so +// the test has to discover that version rather than hardcode one. +const hostVersion = (name: string) => { + const proc = Bun.spawnSync( + [python ?? "python3", "-c", `import importlib.metadata as m; print(m.version(${JSON.stringify(name)}))`], + { stdout: "pipe", stderr: "ignore" }, + ) + const text = proc.stdout.toString().trim() + return proc.exitCode === 0 && /^\d/.test(text) ? text : undefined +} + +test.skipIf(!sandboxed || !hostVersion("six"))( + "a version change is not additive even when the host already provided the old one", + async () => { + // The bug CI caught, and the reason `resolved()` exists apart from + // `freeze()`. Requesting the exact version the host provides installs + // nothing locally, so an owned-set comparison sees no `six` in the "before" + // snapshot and reads the next version as an ADDITION. Kernels holding a + // stale six in memory were then never restarted — the silent staleness the + // whole restart rule exists to prevent. + const host = hostVersion("six")! + const other = host === "1.17.0" ? "1.16.0" : "1.17.0" + + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, { kind: "venv", binary: python! }) + + const first = await Installer.install({ directory: env, packages: [`six==${host}`], index: "", source: false }) + expect(first.ok, first.log).toBe(true) + + const before = await Installer.resolved(env) + // The precondition that makes this test meaningful: the environment owns + // nothing, because the host already satisfied the request. + expect(Object.keys(await Installer.freeze(env))).not.toContain("six") + expect(before["six"]).toBe(host) + + const second = await Installer.install({ directory: env, packages: [`six==${other}`], index: "", source: false }) + expect(second.ok, second.log).toBe(true) + + const after = await Installer.resolved(env) + expect(after["six"]).toBe(other) + const { Environment } = await import("../../src/package/environment") + expect(Environment.additive(before, after)).toBe(false) + + // And the contrast that makes this a regression test rather than an + // assertion: comparing OWNED sets — what the code did before — calls the + // very same change additive, because the environment owned no six until the + // second install. Both lines have to stay true for the bug to be gone. + const ownedBefore = {} as Record + const ownedAfter = await Installer.freeze(env) + expect(ownedAfter["six"]).toBe(other) + expect(Environment.additive(ownedBefore, ownedAfter)).toBe(true) + }, + 600_000, +) + +test.skipIf(!python || !hostHas("numpy"))( + "resolved sees inherited packages, freeze does not", + async () => { + // The invariant behind the fix, stated once: two questions, two answers. + await using dir = await tmpdir() + const env = path.join(dir.path, "env") + await Installer.create(env, { kind: "venv", binary: python! }) + expect(Object.keys(await Installer.resolved(env))).toContain("numpy") + expect(Object.keys(await Installer.freeze(env))).not.toContain("numpy") + }, + 120_000, +) diff --git a/backend/cli/test/package/interpreter.test.ts b/backend/cli/test/package/interpreter.test.ts new file mode 100644 index 00000000..30ed76c9 --- /dev/null +++ b/backend/cli/test/package/interpreter.test.ts @@ -0,0 +1,415 @@ +import { expect, test } from "bun:test" +import path from "path" +import { Installer } from "../../src/package/installer" + +/** + * Interpreter selection, and the Windows failure that produced these tests. + * + * Measured on a real machine: `python -m venv` exited 0, `ensurepip` genuinely + * ran, and the environment came out at `/lib/python3.9/site-packages` with + * `/bin/python.exe` — while every path in `installer.ts` looks under + * `Scripts\`. `pyvenv.cfg` named the cause outright: + * + * home = C:\msys64\mingw64\bin + * version = 3.9.7 + * + * MSYS2's MinGW Python is a native Windows build that patches `sysconfig` to + * the POSIX scheme. It was selected because PATH had no `python3.exe` before + * `C:\msys64\mingw64\bin` — `C:\Python312` ships `python.exe` only — so the + * `python3 ?? python` preference walked straight past a valid 3.12. + */ + +const report = (over: Partial = {}): Installer.Report => ({ + exe: "C:\\Python312\\python.exe", + version: [3, 12], + platform: "win-amd64", + purelib: "C:\\Python312\\Lib\\site-packages", + prefix: "C:\\Python312", + ...over, +}) + +const win = process.platform === "win32" + +test.if(win)("a real python.org interpreter is accepted", () => { + expect(Installer.reject(report())).toBeUndefined() +}) + +test.if(win)("the MSYS2 interpreter that caused this is rejected", () => { + const why = Installer.reject( + report({ + exe: "C:\\msys64\\mingw64\\bin\\python3.exe", + version: [3, 9], + platform: "mingw_x86_64", + purelib: "C:\\msys64\\mingw64\\lib\\python3.9\\site-packages", + prefix: "C:\\msys64\\mingw64", + }), + ) + expect(why).toContain("MSYS2") +}) + +test.if(win)("a POSIX layout is rejected on the scheme alone, whatever the vendor", () => { + // The vendor check is a nicety for the error message; this is the property + // that actually breaks the module, so it must stand on its own — otherwise + // the next cross-built distribution walks through under a different name. + const why = Installer.reject(report({ purelib: "C:\\Weird\\lib\\python3.12\\site-packages" })) + expect(why).toContain("POSIX layout") +}) + +test.if(win)("a non-native platform tag is rejected", () => { + expect(Installer.reject(report({ platform: "cygwin_x86_64" }))).toContain("not a native win-* build") +}) + +test.if(!win)("nothing is rejected off Windows, where the POSIX layout is correct", () => { + expect( + Installer.reject(report({ platform: "linux-x86_64", purelib: "/usr/lib/python3.12/site-packages" })), + ).toBeUndefined() +}) + +test("select() prefers python over python3 on Windows, and the reverse elsewhere", async () => { + // The single line that chose MSYS2. python.org ships `python.exe` and NO + // `python3.exe`, so on Windows `python3` resolves to the Store alias or to a + // POSIX-flavoured distribution nearly by definition. This asserts the source, + // because the ordering cannot be observed from outside on a Linux CI box. + const source = await Bun.file(new URL("../../src/package/installer.ts", import.meta.url).pathname).text() + const body = source.slice(source.indexOf("export async function select()")) + const order = body.slice(body.indexOf("const names"), body.indexOf("\n", body.indexOf("const names"))) + expect(order.indexOf('"python.exe"')).toBeLessThan(order.indexOf('"python3.exe"')) + expect(order.indexOf('"python3"')).toBeLessThan(order.indexOf('"python"', order.indexOf('"python3"') + 1)) +}) + +test("select() finds a working interpreter on this machine", async () => { + const chosen = await Installer.select() + expect(chosen.binary).toBeTruthy() + expect(chosen.report?.prefix).toBeTruthy() +}) + +test("inspect() reports the real interpreter, and undefined for a non-interpreter", async () => { + const chosen = await Installer.select() + const found = await Installer.inspect(chosen.binary!) + expect(found?.version[0]).toBe(3) + // Debian and Ubuntu use dist-packages, not site-packages, for the system + // interpreter — the assertion is that a package directory was reported. + expect(found?.purelib).toMatch(/(site|dist)-packages/) + // A Store alias exits non-zero; stand in for it with something that exists + // and is not an interpreter, which is the same observable. + expect(await Installer.inspect(process.execPath).catch(() => undefined)).toBeUndefined() +}) + +test("a rejected candidate does not end the search", async () => { + // `Bun.which` answers once, so the old code stopped at the first hit. Both + // Windows failures had a bad candidate ahead of a good one — the alias, then + // MSYS2 — so "reject" has to mean "keep looking" across every PATH entry. + const source = await Bun.file(new URL("../../src/package/installer.ts", import.meta.url).pathname).text() + const body = source.slice( + source.indexOf("export async function select()"), + source.indexOf("async function registered"), + ) + expect(body).toContain("continue") + expect(body).not.toContain("Bun.which") + // Every rejection is recorded, so the failure can say what it looked at. + expect(body).toContain("rejected.push") +}) + +test("locate() searches both layouts, not just this platform's", async () => { + const source = await Bun.file(new URL("../../src/package/installer.ts", import.meta.url).pathname).text() + const body = source.slice(source.indexOf("export async function locate")) + expect(body).toContain('"Scripts", "bin"') +}) + +test("the error no longer asserts a cause it did not measure", async () => { + // The claim that a Windows failure "usually means" a Store alias was false on + // the machine that hit it next, and reading as a finding rather than a guess + // it sent the investigation to the Settings app for a full cycle. + const source = await Bun.file(new URL("../../src/package/installer.ts", import.meta.url).pathname).text() + const create = source.slice(source.indexOf("export async function create"), source.indexOf("const same =")) + expect(create).not.toContain("App execution aliases") + expect(create).not.toContain("usually means") + // What replaced it: the interpreter used, what it reports, where one was + // actually found, and what landed on disk. + for (const fact of ["created with:", "an interpreter was found instead at:", "the tree contains:"]) + expect(create).toContain(fact) +}) + +test("a half-built environment is cleared rather than retried into", async () => { + // `venv` and `uv` both short-circuit on an existing directory and report + // success without replacing what is missing, so the first bad creation + // repeats forever. Observed: "Requirement already satisfied" for pip and + // setuptools on every retry, and the identical failure after it. + const source = await Bun.file(new URL("../../src/package/installer.ts", import.meta.url).pathname).text() + const create = source.slice(source.indexOf("export async function create"), source.indexOf("const same =")) + const clear = create.indexOf("fs.rm(directory") + expect(clear).toBeGreaterThan(-1) + // Before the spawn, or the short-circuit still happens. + expect(clear).toBeLessThan(create.indexOf("Bun.spawn")) +}) + +test("interpreter() and locate() agree for an environment built here", async () => { + // The end-to-end property the Windows machine violated: what the module + // requires and what creation produces must be the same file. + const dir = path.join( + process.env["TMPDIR"] ?? "/tmp", + `openscience-interp-${process.pid}-${process.hrtime.bigint().toString(36)}`, + ) + const tool = await Installer.probe(dir) + await Installer.create(dir, tool) + try { + expect(await Installer.locate(dir)).toBe(Installer.interpreter(dir)) + // And it is genuinely rooted in the environment, not the host. + const check = await Installer.inspect(Installer.interpreter(dir)) + expect(check?.prefix).toBeTruthy() + // realpath, not just resolve — the same firmlink that broke `same()` in the + // installer breaks the assertion about it. macOS temp is /var/folders/..., + // /var is a symlink to /private/var, and Python reports the real path. + const { realpathSync } = await import("fs") + expect(realpathSync(check!.prefix)).toBe(realpathSync(dir)) + } finally { + await (await import("fs/promises")).rm(dir, { recursive: true, force: true }).catch(() => {}) + } +}, 180_000) + +test("the installer states what must be readable, never which backend needs telling", async () => { + // The seam that keeps package installation platform-agnostic. `Installer` + // knows a venv delegates to the interpreter named in pyvenv.cfg; it must not + // know that an AppContainer needs an explicit ACL while bubblewrap does not. + // + // It briefly did: baseReadable() returned [] unless win32, added as a quick + // fix when passing the base as readable made bwrap try to create a mountpoint + // under a read-only root. That put backend knowledge in the caller. The filter + // now lives in buildPolicy, where the backend is already known. + const installer = await Bun.file(new URL("../../src/package/installer.ts", import.meta.url).pathname).text() + const confined = installer.slice(installer.indexOf("async function confined"), installer.indexOf("progressLine")) + expect(confined).toContain("readable") + expect(confined).not.toContain("win32") + expect(installer).not.toContain("baseReadable") + + // And the backend still decides what "readable" costs it. The bubblewrap + // branch used to re-bind those paths itself, filtered to /tmp; main's model + // binds every readable root explicitly and before the unreadable masks, so + // re-binding them afterwards re-exposed masked files. The seam is unchanged — + // the installer names paths, the backend decides — only the branch that does + // the deciding moved. + const sandbox = await Bun.file(new URL("../../src/sandbox/sandbox.ts", import.meta.url).pathname).text() + const build = sandbox.slice( + sandbox.indexOf("function buildPolicy"), + sandbox.indexOf("export function seatbeltProfile"), + ) + expect(build).toContain("input.readable") + expect(build).toContain('input.backend === "appcontainer"') +}) + +test("base() still reports the interpreter a venv delegates to", async () => { + const dir = path.join( + process.env["TMPDIR"] ?? "/tmp", + `openscience-base-${process.pid}-${process.hrtime.bigint().toString(36)}`, + ) + const tool = await Installer.probe(dir) + await Installer.create(dir, tool) + try { + // venv writes `home` on every platform, so this is answerable everywhere and + // needs no platform branch to ask. + expect(await Installer.base(dir)).toBeTruthy() + } finally { + await (await import("fs/promises")).rm(dir, { recursive: true, force: true }).catch(() => {}) + } +}, 180_000) + +test("an interpreter the sandbox cannot be granted is a last resort, not a first choice", async () => { + // Measured on a real machine: icacls on C:\Python312 was DENIED, then the + // venv redirector said `No Python at '...'` and the child exited 103. The + // interpreter was perfectly healthy; it was simply owned by SYSTEM, and + // icacls can only change an ACL the caller owns. So a machine-wide Python is + // unusable for a SANDBOXED run however good it is, and picking it produces a + // failure several layers from the choice. + const source = await Bun.file(new URL("../../src/package/installer.ts", import.meta.url).pathname).text() + const body = source.slice(source.indexOf("export async function select()")) + expect(body).toContain("ungrantable.push(candidate)") + + // Not refused outright, though: an UNSANDBOXED run works fine with a + // machine-wide Python, and failing closed here would break everyone who never + // turns the sandbox on. It falls back, and says why the sandbox will object. + expect(body).toContain("so the sandbox cannot be granted read access to it") + + // Off Windows this whole question does not arise — bubblewrap and seatbelt + // need no ACL to read anything. + expect(Installer.grantable("/usr/bin/python3")).toBe(process.platform !== "win32") +}) + +test("the launcher refuses to continue when a read grant fails", async () => { + // A failed READ grant is fatal where a failed write grant is not: it exists + // only because something in there must be readable. Continuing produced a + // child that could not start and an error nobody could trace back. + const source = await Bun.file(new URL("../../src/sandbox/appcontainer.ts", import.meta.url).pathname).text() + const main = source.slice(source.indexOf("export async function main")) + expect(main).toContain("unreachable.length") + // The remedy, not just the diagnosis. + expect(main).toContain("install it for your user") + expect(main).toContain("uv") +}) + +test("the Windows prerequisite is one check, read by every surface", async () => { + // The user cannot discover this from anything else: an AppContainer can only + // be granted access to paths its user OWNS, so a machine-wide Python is + // unusable by a sandboxed process however healthy it is. Without a check, the + // only symptom is an install failing much later with an error about the + // interpreter rather than about ownership — which is exactly what happened, + // and the agent then advised asking an admin for a permission that cannot be + // granted. + // + // One source of truth, three surfaces. `describe()` and `selfTest()` once + // disagreed about whether a backend existed because each answered separately; + // a prerequisite with three implementations would drift the same way. + const installer = await Bun.file(new URL("../../src/package/installer.ts", import.meta.url).pathname).text() + expect(installer).toContain("export async function blocked()") + + const status = await Bun.file(new URL("../../src/cli/cmd/sandbox.ts", import.meta.url).pathname).text() + expect(status).toContain("Installer.blocked()") + const route = await Bun.file(new URL("../../src/server/routes/settings/sandbox.ts", import.meta.url).pathname).text() + expect(route).toContain("Installer.blocked()") + + // Not on the hot path. Answering it runs candidate interpreters, which every + // sandboxed command must not pay for. + const sandbox = await Bun.file(new URL("../../src/sandbox/sandbox.ts", import.meta.url).pathname).text() + expect(sandbox).not.toContain("Installer.blocked") +}) + +test("the prerequisite never claims containment is broken", async () => { + // Containment is unaffected by this: shell commands stay confined, writes stay + // blocked, egress stays bounded. Only Python environments are unavailable. A + // user who reads "the sandbox does not work" and turns it off would lose + // confinement they still had — a worse outcome than the problem. + const installer = await Bun.file(new URL("../../src/package/installer.ts", import.meta.url).pathname).text() + const body = installer.slice(installer.indexOf("export async function blocked()")) + const message = body.slice(0, body.indexOf("\n }")) + expect(message).toContain("Python environments are unavailable") + expect(message).toContain("Containment is unaffected") + // And it names the fix, with the command, on both routes out. + expect(message).toContain("winget install --id=astral-sh.uv") + expect(message).toContain("'Install for all users' left OFF") +}) + +test("blocked() answers only where the question applies", async () => { + // Off Windows there is no ACL to grant, so this must never fire — a POSIX + // machine with a system Python is entirely fine. + if (process.platform === "win32") return + expect(await Installer.blocked()).toBeUndefined() +}) + +test("the prerequisite is checked where users actually meet it", async () => { + // Nobody runs `sandbox status` voluntarily. A user meets this in one of two + // places: the settings panel, or the moment the agent tries to install + // something. Both now carry it. + // + // In package_install it runs BEFORE the approval card, because no amount of + // approving an install fixes an interpreter the sandbox cannot reach — and + // asking first would be a prompt whose only possible outcome is an error, the + // same defect the bash-tool refusal already had to fix. + const tool = await Bun.file(new URL("../../src/tool/package.ts", import.meta.url).pathname).text() + const execute = tool.slice(tool.indexOf("async execute(params, ctx)")) + expect(execute).toContain("Installer.blocked()") + expect(execute.indexOf("Installer.blocked()")).toBeLessThan(execute.indexOf("ctx.ask(")) + + const panel = await Bun.file( + new URL("../../../../frontend/workspace/src/components/settings/Sandbox.tsx", import.meta.url).pathname, + ).text() + expect(panel).toContain("data()?.blocked") + // Above the first policy control: a green backend sitting next to "installs + // will fail" reads as a contradiction, and this is the line the user has to + // act on before any of the switches below it matter. Anchored on the + // "Protection" section rather than a specific box, because the panel's own + // headings are main's and may be renamed without changing this property. + expect(panel.indexOf("data()?.blocked")).toBeLessThan(panel.indexOf('title="Protection"')) +}) + +test("an environment pinned to an ungrantable base is rebuilt, not reused forever", async () => { + // The failure that survived installing uv AND a user-owned Python. probe() + // returns "existing" the moment Scripts/python.exe is there, so it never + // re-selects — and an environment records its base in pyvenv.cfg at creation. + // One built when the only candidate was C:\Python312 stays bound to it, so + // every retry granted that path and was denied, no matter what interpreters + // appeared afterwards. + // + // Measured: after `uv python install 3.12` the error was still "could not + // grant sandbox access to C:\Python312" — the path in the existing pyvenv.cfg, + // not one selection would choose now. + const source = await Bun.file(new URL("../../src/package/installer.ts", import.meta.url).pathname).text() + const probe = source.slice( + source.indexOf("export async function probe"), + source.indexOf("export async function create"), + ) + expect(probe).toContain("const home = await base(directory)") + expect(probe).toContain("grantable(home)") + // Falling through is the rebuild: create() clears a directory that already has + // a pyvenv.cfg, so the next tool builds fresh on a base that works. + expect(probe).toContain("Sandbox.available()") +}) + +test("uv being installed is not treated as proof of a usable interpreter", async () => { + // The false all-clear. `uv venv` builds from whatever uv DISCOVERS, which on a + // machine with a system Python first is the same ungrantable one — so uv's + // mere presence said "fine" while nothing had improved, and the user was told + // to install uv, did, and hit the identical error. + const source = await Bun.file(new URL("../../src/package/installer.ts", import.meta.url).pathname).text() + const body = source.slice(source.indexOf("export async function blocked()")) + // managedDetail(), not managed(): the check now distinguishes "no usable + // interpreter" from "one that will fail to install under the sandbox", and + // only the second has a one-command remedy worth printing. + expect(body).toContain("await managedDetail()") + // And the same answer PINS the rebuild, or a rebuild triggered because the old + // base was ungrantable would let uv pick that same base again. + const create = source.slice(source.indexOf("export async function create")) + expect(create).toContain('"--python", pinned') + // And only on Windows: grantability is a Windows question, so pinning + // elsewhere would override uv's own choice for no reason. + expect(create).toContain('process.platform === "win32" && tool.kind === "uv"') + const helper = source.slice(source.indexOf("export async function managed()")) + // It asks uv what it HAS, and requires one we could actually be granted. + expect(helper).toContain('"--output-format", "json"') + expect(helper).toContain("grantable(entry.path)") + // And it prefers an interpreter without the AppContainer mkdtemp defect. + // Choosing 3.12.4+ when an older one is installed builds an environment that + // downloads wheels fine and then cannot unpack them. + expect(helper).toContain("Number(affected(a)) - Number(affected(b))") + // Only what uv MANAGES. `uv python list` also reports discovered system + // interpreters and uv's own trampolines, and both are traps: the system one is + // the ungrantable case this exists to route around, and pinning --python to a + // trampoline gives "uv trampoline failed to spawn Python child process". + expect(helper).toContain('"python", "dir"') + expect(helper).toContain("under(absolute(root), entry.path)") +}) + +test("uv paths are resolved against HOME, not the cwd", async () => { + // Measured on Windows, uv 0.12.4, from `--output-format json` -- so the + // shortening is not display-only, which is what the previous fix assumed: + // + // "path":"AppData\\Roaming\\uv\\python\\cpython-3.12.13-...\\python.exe" + // "path":"C:\\Python312\\python.exe" + // + // Absolute and home-relative in the same array. Resolving the relative ones + // against the cwd made grantable() pass on a path that does not exist; + // rejecting them outright then hid every uv interpreter on the machine and + // told a user with working uv to install uv. + const source = await Bun.file(new URL("../../src/package/installer.ts", import.meta.url).pathname).text() + const helper = source.slice(source.indexOf("const absolute ="), source.indexOf("const under =")) + expect(helper).toContain("os.homedir()") + expect(helper).not.toContain("process.cwd") +}) + +test("managed() returns an interpreter that exists, if uv has one here", async () => { + // The property the source assertions cannot reach: whatever comes back must be + // a real file. Both bugs so far returned a plausible path to nothing. + const found = await Installer.managed() + if (!found) return + expect(path.isAbsolute(found)).toBe(true) + expect(await Bun.file(found).exists()).toBe(true) +}) + +test("a working environment is not refused over a machine-wide prerequisite", async () => { + // blocked() is a statement about a MACHINE. An environment that already exists + // on a grantable base is a counterexample to it, and refusing an install into + // one -- while telling the user to set up the tool that built it -- is how the + // uv parsing bug surfaced. + const source = await Bun.file(new URL("../../src/tool/package.ts", import.meta.url).pathname).text() + const body = source.slice(source.indexOf("async execute(")) + expect(body).toContain("if (!usable) {") + expect(body.indexOf('tool.kind === "existing"')).toBeLessThan(body.indexOf("Installer.blocked()")) +}) diff --git a/backend/cli/test/package/prompt.test.ts b/backend/cli/test/package/prompt.test.ts new file mode 100644 index 00000000..a6e44479 --- /dev/null +++ b/backend/cli/test/package/prompt.test.ts @@ -0,0 +1,161 @@ +import { expect, test } from "bun:test" +import { PackagePrompt } from "../../src/package/prompt" +import { SystemPrompt } from "../../src/session/system" + +test("packages() returns the capability block, shaped like compute()", async () => { + const block = await SystemPrompt.packages("proj_empty_for_shape") + expect(block).toHaveLength(1) + expect(block[0]).toContain("") + expect(block[0]).toContain("") +}) + +test("an empty inventory tells the agent the first install creates one", () => { + const rendered = PackagePrompt.render({ environments: [] }) + expect(rendered).toContain("No environments exist yet") +}) + +test("an inventory lists requested packages only, with a dependency count", () => { + const rendered = PackagePrompt.render({ + environments: [{ name: "default", language: "python", requested: ["numpy", "pandas"], total: 168, busy: false }], + }) + expect(rendered).toContain("default (python): numpy, pandas (+166 deps)") + // The resolved closure is dominated by libgcc/harfbuzz/qt6-main and would + // bury the contract in font libraries. + expect(rendered).not.toContain("libgcc") +}) + +test("a busy environment is flagged so the agent does not execute into it", () => { + const rendered = PackagePrompt.render({ + environments: [{ name: "default", language: "python", requested: [], total: 0, busy: true }], + }) + expect(rendered).toContain("INSTALL IN PROGRESS") +}) + +test("the contract promises refusal, not a missing network", () => { + const rendered = PackagePrompt.render({ environments: [] }) + // The old wording said "the agent shell has no network", which the allowlist + // proxy made false — and it implied the venv-in-workspace route was + // impossible when it is exactly what works. + expect(rendered).not.toContain("no network") + expect(rendered).toContain("refused") + expect(rendered).toContain("virtualenv you create yourself") +}) + +test("the inventory reflects a real written environment", async () => { + const { Environment } = await import("../../src/package/environment") + const project = "proj_inventory" + await Environment.write(project, { + name: "torch", + language: "python", + requested: ["torch"], + installed: { torch: "2.4.0", filelock: "3.15.4" }, + total: 2, + createdAt: 1, + updatedAt: 1, + }) + const rendered = await PackagePrompt.system(project) + expect(rendered).toContain("torch (python): torch (+1 deps)") +}) + +test("a busy environment is reported from the live lock, not a stored flag", async () => { + const { Environment } = await import("../../src/package/environment") + const project = "proj_busy" + await Environment.write(project, { + name: "held", + language: "python", + requested: [], + installed: {}, + total: 0, + createdAt: 1, + updatedAt: 1, + }) + let rendered = "" + await Environment.lock(project, "held", async () => { + rendered = await PackagePrompt.system(project) + }) + // A stored flag would survive a crash and permanently mark a healthy + // environment busy. The lock lives in memory and is the truth. + expect(rendered).toContain("INSTALL IN PROGRESS") + expect(await PackagePrompt.system(project)).not.toContain("INSTALL IN PROGRESS") +}) + +test("an unknown project renders the empty inventory rather than throwing", async () => { + expect(await PackagePrompt.system("proj_never_seen")).toContain("No environments exist yet") +}) + +test("after a real install, the agent's contract lists what it installed", async () => { + // The whole point of this task. Before it, `system()` read a global + // environments.json that nothing wrote, so the agent was told "No + // environments exist yet" forever — including immediately after installing + // something — which makes the contract's first rule ("answer whether a + // package is available from the inventory above") actively misleading. + const { Sandbox } = await import("../../src/sandbox/sandbox") + if (Sandbox.backend() === "none" || !Bun.which("python3")) return + const { Instance } = await import("../../src/project/instance") + const { PackageTool } = await import("../../src/tool/package") + const { executionSession, tmpdir } = await import("../fixture/fixture") + await using tmp = await tmpdir({ git: true }) + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const session = await executionSession() + const tool = await PackageTool.init() + const before = await PackagePrompt.system(Instance.project.id) + expect(before).toContain("No environments exist yet") + + await tool.execute({ packages: ["tqdm"], environment: "seen", language: "python", source: false, wait: true }, { + sessionID: session.id, + messageID: "", + callID: "", + agent: "research", + abort: AbortSignal.any([]), + messages: [], + metadata: () => {}, + ask: async () => {}, + } as never) + + const after = await PackagePrompt.system(Instance.project.id) + expect(after).not.toContain("No environments exist yet") + expect(after).toContain("seen (python): tqdm") + }, + }) +}, 600_000) + +test("the injection is unconditional, beside compute()", async () => { + // The load-bearing mechanism is that this reaches EVERY request for EVERY + // agent — not a skill override, which only reaches a skill's front page and + // never its reference files or a third-party skill cloned from GitHub. + const source = await Bun.file(new URL("../../src/session/prompt.ts", import.meta.url).pathname).text() + // A boolean, not toContain(source): a failing toContain prints the whole + // 86KB file into the runner output and buries every other result. + expect(source.includes("await SystemPrompt.packages()")).toBe(true) +}) + +test("the agent is told uv is preferred, and what the Windows remedy actually is", async () => { + // The agent's advice was wrong in a real session. Hitting the ungrantable + // base interpreter, it told the user to get "an admin granting the + // OpenScience sandbox read/execute permission on C:\Python312" — which cannot + // work: icacls only changes an ACL you own, and an all-users install is owned + // by SYSTEM. The remedy is a Python the user owns, not more permissions on one + // they do not. + // + // probe() has always preferred uv over venv; the agent simply had no way to + // know, so it could not name the right fix. + const guidance = await Bun.file(new URL("../../src/package/prompt.ts", import.meta.url).pathname).text() + expect(guidance).toContain("Environments are built with uv when it is present") + expect(guidance).toContain("a Python the user owns") + expect(guidance).toContain("not elevated permissions") + + const tool = await Bun.file(new URL("../../src/tool/package.ts", import.meta.url).pathname).text() + expect(tool).toContain("can only be granted read access to paths the user owns") + + // And the Windows-specific half is GATED. bubblewrap and seatbelt read + // anything the user can read, so this consideration is meaningless there — and + // a tool description ships on every request, so unconditional platform trivia + // is a cost every Linux and macOS user pays forever for advice they can never + // act on. + for (const text of [guidance, tool]) { + const at = text.indexOf("a Python the user owns") >= 0 ? text.indexOf("a Python the user owns") : text.indexOf("C:") + expect(text.slice(0, at)).toContain('process.platform === "win32"') + } +}) diff --git a/backend/cli/test/package/refuse.test.ts b/backend/cli/test/package/refuse.test.ts new file mode 100644 index 00000000..690fa737 --- /dev/null +++ b/backend/cli/test/package/refuse.test.ts @@ -0,0 +1,47 @@ +import { expect, test } from "bun:test" +import { Refuse } from "../../src/package/refuse" + +test.each([ + ["pip install numpy"], + ["pip3 install numpy"], + ["pip install -r requirements.txt"], + ["uv pip install numpy"], + ["conda install numpy"], + ["mamba install numpy"], + ["poetry add numpy"], + // The venv-in-workspace route, which is what actually works today and what + // a bare "pip install" match misses entirely. + ["/work/project/venv/bin/pip install numpy"], + ["./venv/bin/pip install numpy"], + ["python -m pip install numpy"], + ["python3 -m pip install numpy"], + ["/usr/bin/python3.14 -m pip install numpy"], + ["./venv/bin/python -m pip install numpy"], +])("refuses %s", (line) => { + const message = Refuse.installer(line.split(" ")) + expect(message).toBeString() + expect(message).toContain("package_install") +}) + +test.each([ + // Read-only inspection stays allowed: refusing these would break ordinary + // work and teach the agent that the whole tool is unreliable. + ["pip list"], + ["pip show numpy"], + ["pip --version"], + ["python -m pip list"], + ["conda env list"], + // Not installers at all. + ["python analysis.py"], + ["npm install"], + ["git install-hooks"], + ["echo pip install numpy"], +])("allows %s", (line) => { + expect(Refuse.installer(line.split(" "))).toBeUndefined() +}) + +test("the message names the tool and the reason, not just a denial", () => { + const message = Refuse.installer(["pip", "install", "numpy"])! + expect(message).toContain("package_install") + expect(message).toContain("numpy") +}) diff --git a/backend/cli/test/package/requirement.test.ts b/backend/cli/test/package/requirement.test.ts new file mode 100644 index 00000000..f1063140 --- /dev/null +++ b/backend/cli/test/package/requirement.test.ts @@ -0,0 +1,146 @@ +import { expect, test } from "bun:test" +import { Requirement } from "../../src/package/requirement" + +test("a bare name", () => { + expect(Requirement.parse("numpy")).toEqual({ name: "numpy", extras: [], specifier: "", marker: "", url: "" }) +}) + +test("a version specifier is kept whole, not split on ==", () => { + // The exact case a naive `split("==")` gets wrong. + expect(Requirement.parse("numpy>=2.4")).toMatchObject({ name: "numpy", specifier: ">=2.4" }) +}) + +test.each([ + ["numpy==2.1.0", "==2.1.0"], + ["numpy!=2.0.0", "!=2.0.0"], + ["numpy~=2.1", "~=2.1"], + ["numpy<3", "<3"], + ["numpy<=3", "<=3"], + ["numpy>2", ">2"], + ["numpy===2.1.0", "===2.1.0"], + ["numpy>=2.1,<3", ">=2.1,<3"], +])("parses the specifier in %s", (input, specifier) => { + expect(Requirement.parse(input)).toMatchObject({ name: "numpy", specifier }) +}) + +test("extras are captured and not folded into the name", () => { + expect(Requirement.parse("pandas[performance,excel]")).toMatchObject({ + name: "pandas", + extras: ["performance", "excel"], + }) +}) + +test("extras combine with a specifier", () => { + expect(Requirement.parse("pandas[performance]>=2.2")).toMatchObject({ + name: "pandas", + extras: ["performance"], + specifier: ">=2.2", + }) +}) + +test("an environment marker is separated from the specifier", () => { + expect(Requirement.parse('tqdm>=4 ; python_version >= "3.9"')).toMatchObject({ + name: "tqdm", + specifier: ">=4", + marker: 'python_version >= "3.9"', + }) +}) + +test("a direct URL reference keeps the name and the url apart", () => { + expect(Requirement.parse("mypkg @ https://example.com/mypkg-1.0-py3-none-any.whl")).toMatchObject({ + name: "mypkg", + url: "https://example.com/mypkg-1.0-py3-none-any.whl", + }) +}) + +test("names normalise per PEP 503 so Foo_Bar and foo-bar are one package", () => { + // Treating them as different packages would let an upgrade look additive. + expect(Requirement.parse("Foo_Bar").name).toBe("foo-bar") + expect(Requirement.parse("Foo.Bar").name).toBe("foo-bar") + expect(Requirement.parse("FOO---BAR").name).toBe("foo-bar") +}) + +test.each([[""], [" "], ["=="], ["numpy=="], ["-rrequirements.txt"], ["numpy >= "], ["[extras]"], ["@ https://x"]])( + "rejects %p rather than guessing", + (input) => { + // A silently mis-parsed name becomes a wrong permission pattern, and a + // wrong pattern approves something other than what runs. + expect(() => Requirement.parse(input)).toThrow() + }, +) + +test.each([["numpy >= "], ["numpy>="], ["numpy<="], ["numpy=="], ["numpy~="], ["numpy>=2.1,"], ["numpy>=2.1,<"]])( + "rejects the dangling operator in %p", + (input) => { + // Regression: a prefix test like /^(===|==|…|>|<)\s*\S/ accepts these, + // because the alternation backtracks to the single-character `>` and + // consumes the `=` as the version. The clause regex is anchored end to end + // precisely to stop that — a dangling operator would otherwise reach pip + // as a literal requirement, having passed validation. + expect(() => Requirement.parse(input)).toThrow() + }, +) + +test("a multi-clause specifier is validated clause by clause", () => { + expect(Requirement.parse("numpy>=2.1,<3").specifier).toBe(">=2.1,<3") + expect(() => Requirement.parse("numpy>=2.1,<")).toThrow() +}) + +test("the canonical pattern is exactly the spec's string", () => { + expect(Requirement.pattern({ packages: ["numpy", "pandas"], environment: "default", index: "pypi.org/simple" })).toBe( + "install numpy pandas → default [pypi.org/simple]", + ) +}) + +test("the pattern is stable under argument order, so the same request matches the same grant", () => { + const a = Requirement.pattern({ packages: ["pandas", "numpy"], environment: "default", index: "pypi.org/simple" }) + const b = Requirement.pattern({ packages: ["numpy", "pandas"], environment: "default", index: "pypi.org/simple" }) + expect(a).toBe(b) +}) + +test("the pattern drops version specifiers, matching what the card shows", () => { + // Resolution happens after approval — the card shows the request, so pinning + // a version must not fragment an existing grant. + const a = Requirement.pattern({ packages: ["numpy>=2.4"], environment: "default", index: "pypi.org/simple" }) + const b = Requirement.pattern({ packages: ["numpy"], environment: "default", index: "pypi.org/simple" }) + expect(a).toBe(b) +}) + +test("changing the environment changes the pattern, so the prompt reappears", () => { + const a = Requirement.pattern({ packages: ["numpy"], environment: "default", index: "pypi.org/simple" }) + const b = Requirement.pattern({ packages: ["numpy"], environment: "torch", index: "pypi.org/simple" }) + expect(a).not.toBe(b) +}) + +test("changing the index changes the pattern", () => { + const a = Requirement.pattern({ packages: ["numpy"], environment: "default", index: "pypi.org/simple" }) + const b = Requirement.pattern({ packages: ["numpy"], environment: "default", index: "pypi.internal/simple" }) + expect(a).not.toBe(b) +}) + +test("index credentials are redacted, never shown on the card", () => { + const redacted = Requirement.redact("https://user:s3cret@pypi.internal/simple") + expect(redacted).toBe("pypi.internal/simple") + expect(redacted).not.toContain("s3cret") + expect(redacted).not.toContain("user") +}) + +test("a bare index is unchanged by redaction apart from its scheme", () => { + expect(Requirement.redact("https://pypi.org/simple/")).toBe("pypi.org/simple") +}) + +test("a credentialled index and a differently-credentialled one produce the same pattern", () => { + // Credentials are environment config, not part of the approved action — + // rotating a token must not invalidate a standing grant. + const a = Requirement.pattern({ + packages: ["numpy"], + environment: "default", + index: Requirement.redact("https://user:s3cret@pypi.internal/simple"), + }) + const b = Requirement.pattern({ + packages: ["numpy"], + environment: "default", + index: Requirement.redact("https://other:tok@pypi.internal/simple"), + }) + expect(a).toBe(b) +}) diff --git a/backend/cli/test/package/tool.test.ts b/backend/cli/test/package/tool.test.ts new file mode 100644 index 00000000..638e2d97 --- /dev/null +++ b/backend/cli/test/package/tool.test.ts @@ -0,0 +1,217 @@ +import { expect, test } from "bun:test" +import { Environment } from "../../src/package/environment" +import { Requirement } from "../../src/package/requirement" +import type { PermissionNext } from "../../src/permission/next" +import { Instance } from "../../src/project/instance" +import { Sandbox } from "../../src/sandbox/sandbox" +import { executionSession, tmpdir } from "../fixture/fixture" + +const read = (relative: string) => Bun.file(new URL(relative, import.meta.url).pathname).text() + +test("the approval pattern is the canonical command string the card shows", () => { + // The card and the permission matcher must use the ONE string. If they ever + // diverge, the user approves one thing and another runs. + expect(Requirement.pattern({ packages: ["numpy", "pandas"], environment: "default", index: "pypi.org/simple" })).toBe( + "install numpy pandas → default [pypi.org/simple]", + ) +}) + +test("the tool asks with the package_install capability and the install* grant", async () => { + const source = await read("../../src/tool/package.ts") + // Pinned as constants because the spec pins them: the capability was + // reserved in trust.ts and execution.ts with zero call sites, and the + // standing grant mirrors notebook.ts, which shows "python (notebook)" and + // stores the broad "python*". + expect(source.includes('permission: "package_install"')).toBe(true) + expect(source.includes('always: ["install*"]')).toBe(true) +}) + +test("the card is asked for before anything is installed", async () => { + const source = await read("../../src/tool/package.ts") + // Approval precedes the lock and the installer, not the other way round. + expect(source.indexOf("ctx.ask")).toBeGreaterThan(-1) + expect(source.indexOf("ctx.ask")).toBeLessThan(source.indexOf("Installer.install")) +}) + +test("resolution happens after approval, so the card shows the request", async () => { + const source = await read("../../src/tool/package.ts") + // Approving 2 names must not silently approve the 168-entry closure. + expect(source.indexOf("ctx.ask")).toBeLessThan(source.indexOf("Installer.freeze")) +}) + +test("the tool is registered", async () => { + const source = await read("../../src/tool/registry.ts") + expect(source.includes("PackageTool")).toBe(true) +}) + +test("installs are not a paid action, so no spendFilter entry exists", async () => { + const source = await read("../../src/permission/next.ts") + // Governing principle: nothing is gated more strictly than arbitrary code + // execution unless it costs money. An install costs nothing. + expect(source.includes("package_install")).toBe(false) +}) + +test("the tool exists and declares its parameters", async () => { + const { PackageTool } = await import("../../src/tool/package") + expect(PackageTool.id).toBe("package_install") +}) + +// Everything above reads the source. These run the tool. Source assertions +// cannot tell whether the card actually fires, and the card is the entire +// point of this task. + +async function context() { + const session = await executionSession() + const asks: Array> = [] + return { + asks, + ctx: { + sessionID: session.id, + messageID: "", + callID: "", + agent: "research", + abort: AbortSignal.any([]), + messages: [], + metadata: () => {}, + ask: async (req: Omit) => { + asks.push(req) + }, + }, + } +} + +const python = Bun.which("python3") +const live = Sandbox.backend() !== "none" && Boolean(python) + +test.skipIf(!live)( + "installing asks for approval with the canonical pattern, then lands the package", + async () => { + await using tmp = await tmpdir({ git: true }) + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const { PackageTool } = await import("../../src/tool/package") + const tool = await PackageTool.init() + const { asks, ctx } = await context() + const result = await tool.execute( + { packages: ["tqdm"], environment: "t1", language: "python" as const, source: false, wait: true }, + ctx, + ) + + expect(asks).toHaveLength(1) + expect(asks[0]!.permission).toBe("package_install") + expect(asks[0]!.patterns).toEqual(["install tqdm → t1 [pypi.org/simple]"]) + expect(asks[0]!.always).toEqual(["install*"]) + + expect(result.metadata.ok).toBe(true) + expect(result.metadata.versions["tqdm"]).toMatch(/^\d/) + // A first install into an empty environment is additive by definition. + expect(result.metadata.additive).toBe(true) + + // The manifest records the request, not the closure. + const stored = await Environment.read(Instance.project.id, "t1") + expect(stored?.requested).toEqual(["tqdm"]) + expect(stored!.total).toBeGreaterThan(0) + }, + }) + }, + 300_000, +) + +test.skipIf(!live)( + "a fully-satisfied request installs nothing and never shows a card", + async () => { + await using tmp = await tmpdir({ git: true }) + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const { PackageTool } = await import("../../src/tool/package") + const tool = await PackageTool.init() + const first = await context() + await tool.execute( + { packages: ["tqdm"], environment: "t2", language: "python" as const, source: false, wait: true }, + first.ctx, + ) + + const second = await context() + const result = await tool.execute( + { packages: ["tqdm"], environment: "t2", language: "python" as const, source: false, wait: true }, + second.ctx, + ) + // Nothing privileged happens, so nothing needs approving — and a + // fully-satisfied request is not worth a turn. + expect(second.asks).toHaveLength(0) + expect(result.metadata.installed).toBe(false) + }, + }) + }, + 300_000, +) + +test.skipIf(!live)( + "a pinned version is never treated as already satisfied by a different one", + async () => { + // Regression. The skip check compared package NAMES only, so + // `six==1.17.0` against an installed 1.16.0 returned "already installed", + // skipped the install, and reported the change as additive — leaving the + // environment on the old version while telling the agent it had the new + // one, and leaving bound kernels un-restarted. + await using tmp = await tmpdir({ git: true }) + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const { PackageTool } = await import("../../src/tool/package") + const tool = await PackageTool.init() + await tool.execute( + { packages: ["six==1.16.0"], environment: "pin", language: "python", source: false, wait: true }, + (await context()).ctx, + ) + const upgrade = await context() + const result = await tool.execute( + { packages: ["six==1.17.0"], environment: "pin", language: "python", source: false, wait: true }, + upgrade.ctx, + ) + // It really ran, it really asked, and it knows the change was not additive. + expect(upgrade.asks).toHaveLength(1) + expect(result.metadata.installed).toBe(true) + expect(result.metadata.additive).toBe(false) + expect(result.metadata.versions["six"]).toBe("1.17.0") + }, + }) + }, + 600_000, +) + +test.skipIf(!live)( + "a failed install throws the translated cause and writes no manifest", + async () => { + await using tmp = await tmpdir({ git: true }) + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const { PackageTool } = await import("../../src/tool/package") + const tool = await PackageTool.init() + const { ctx } = await context() + const failure = await tool + .execute( + { + packages: ["this-package-does-not-exist-anywhere-xyzzy"], + environment: "t3", + language: "python" as const, + source: false, + wait: true, + }, + ctx, + ) + .then( + () => undefined, + (error: Error) => error, + ) + expect(failure).toBeDefined() + // Nothing landed, so nothing is recorded as landed. + expect(await Environment.read(Instance.project.id, "t3")).toBeUndefined() + }, + }) + }, + 300_000, +) diff --git a/backend/cli/test/sandbox/appcontainer-install.test.ts b/backend/cli/test/sandbox/appcontainer-install.test.ts new file mode 100644 index 00000000..3b997825 --- /dev/null +++ b/backend/cli/test/sandbox/appcontainer-install.test.ts @@ -0,0 +1,436 @@ +import { afterAll, beforeAll, expect, test } from "bun:test" +import fs from "fs/promises" +import os from "os" +import path from "path" +import { Config } from "../../src/config/config" +import { Installer } from "../../src/package/installer" +import { EgressRuntime } from "../../src/sandbox/egress-runtime" +import { Sandbox } from "../../src/sandbox/sandbox" + +/** + * `package_install` end to end, on a real Windows kernel. + * + * This exists because the loop was wrong, not because a particular bug was + * hard. The chain a Windows install goes through is: + * + * select interpreter -> create venv -> pin base -> grant base ACL -> + * launcher spawns base -> pip runs -> shim -> broker -> network + * + * `appcontainer-live` covers containment and `appcontainer-transport` covers the + * pipe. NOTHING covered the seven hops between them, so each one was found by a + * human rebooting into Windows, running one command, and pasting the error — + * five rounds, each fix shipping a fresh unverified assumption that became the + * next round's failure. Two of them were assumptions about uv's output format + * that were checked on Linux, where they happen to be true. + * + * So the assertions below are deliberately staged rather than one big "it + * installed": a red run has to say WHICH hop broke, because the machine is not + * one anybody can log into. + * + * 1. "uv provisions a base under the user profile" — the prerequisite. Red + * here means the CI setup step did not install uv or a managed Python, and + * nothing after it means anything. + * 2. "an environment is created on a grantable base" — `create()` and the + * `--python` pin. Red means uv chose a base we cannot grant. + * 3. "the environment's launcher can spawn its base inside the container" — + * the hop that produced `uv trampoline failed to spawn Python child + * process: permission denied (os error 5)`. A venv is not a Python; on + * Windows `Scripts\python.exe` is a stub that spawns the real interpreter + * elsewhere, and an AppContainer reaches nothing whose ACL does not name + * its SID. Red means we granted a path that is not the one the stub uses. + * 4. "pip installs a package through the broker" — the merge gate itself, and + * the first thing that has ever moved a byte over the named pipe. + */ + +const windows = process.platform === "win32" + +// Stated, not inherited. `Config.trustedSandbox()` defaults to disabled/deny, so +// a test that reads the ambient policy runs UNSANDBOXED and passes while proving +// nothing — which is exactly what happened: "Successfully installed six-1.17.0" +// with `egress: `, on a job whose entire purpose is the sandboxed path. +beforeAll(async () => { + const { Config } = await import("../../src/config/config") + await Config.setSandbox({ enabled: true, network: "allowlist" }) +}) + +afterAll(async () => { + const { Global } = await import("../../src/global") + const { Config } = await import("../../src/config/config") + const fsp = await import("fs/promises") + for (const name of ["openscience.jsonc", "openscience.json", "config.json"]) { + await fsp.rm(path.join(Global.Path.config, name), { force: true }).catch(() => {}) + } + Config.global.reset() +}) + +/** Kept across the staged tests: each one builds on the last, and rebuilding an + * environment per test would triple an already slow job. */ +let workspace: string | undefined +let environment: string | undefined + +const scratch = async () => { + if (workspace) return workspace + workspace = await fs.mkdtemp(path.join(os.tmpdir(), "openscience-install-")) + environment = path.join(workspace, "env") + return workspace +} + +test.if(windows)( + "uv provisions a base interpreter under the user profile", + async () => { + const found = await Installer.managed() + console.log(` managed interpreter: ${found ?? ""}`) + // Not a soft skip. If CI has no managed Python the remaining tests would + // pass vacuously against a system interpreter, which is exactly the + // configuration that cannot work and exactly what we are here to catch. + expect(found).toBeTruthy() + expect(Installer.grantable(found!)).toBe(true) + expect(await Bun.file(found!).exists()).toBe(true) + }, + 120_000, +) + +test.if(windows)( + "an environment is created on a base the sandbox can be granted", + async () => { + await scratch() + const tool = await Installer.probe(environment!) + console.log(` tool: ${tool.kind} (${tool.binary})`) + await Installer.create(environment!, tool) + const home = await Installer.base(environment!) + console.log(` pyvenv.cfg home: ${home}`) + expect(home).toBeTruthy() + expect(Installer.grantable(home!)).toBe(true) + }, + 300_000, +) + +test.if(windows)( + "the granted base interpreter itself runs inside the container", + async () => { + // The discriminator. The first CI run reproduced `uv trampoline failed to + // spawn Python child process` and, in the same log, showed that the path we + // grant and the path `sys._base_executable` names are the SAME directory — + // so the leading theory (we grant the wrong hop) was wrong. + // + // That leaves two possibilities the failing test cannot tell apart: the + // grant is not taking effect at all, or it is and something about the + // trampoline's own spawn is refused. Running the base directly, with the + // identical grant and no venv in the picture, separates them. Green here + // plus red below means the grant works and the trampoline is the problem; + // red here means nothing downstream was ever going to work. + const home = await Installer.base(environment!) + expect(home).toBeTruthy() + // Printed unconditionally, because it is the difference between "the ACE + // did not land" and "it landed on a link". uv keeps a patch-versioned + // directory and a stable name beside it; `pyvenv.cfg` points at the stable + // one, and an ACE on a reparse point is not an ACE on its target. + const real = await fs.realpath(home!).catch(() => "") + console.log(` pyvenv home : ${home}`) + console.log(` resolves to : ${real}${real === home ? " (not a link)" : " (LINK)"}`) + const spec = await Sandbox.wrapArgv({ + file: path.join(home!, "python.exe"), + args: ["-c", "import sys; print(sys.version_info[:2])"], + workspace: [environment!], + readable: [home!], + options: { enabled: true, network: "deny", onUnavailable: "error", allowWrite: [] }, + }) + const proc = Bun.spawn([spec.file, ...(spec.args ?? [])], { + env: { ...process.env, ...spec.env }, + stdout: "pipe", + stderr: "pipe", + }) + const [out, err] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]) + await proc.exited + console.log(` exit ${proc.exitCode}\n stdout: ${out.trim()}\n stderr: ${err.trim()}`) + expect(proc.exitCode).toBe(0) + }, + 180_000, +) + +test.if(windows)( + "the environment's launcher can spawn its base inside the container", + async () => { + const home = await Installer.base(environment!) + const binary = Installer.interpreter(environment!) + + // Unsandboxed first, and printed. `sys._base_executable` is the binary the + // launcher actually exec'd — if the sandboxed run below fails, this line + // and `home` above are the whole diagnosis, because a mismatch between them + // IS the bug. Getting that comparison out of CI logs rather than out of a + // human's PowerShell window is most of the point of this file. + const code = + "import sys; print(sys.executable); print(getattr(sys,'_base_executable',None)); print(sys.base_prefix)" + const host = Bun.spawn([binary, "-c", code], { stdout: "pipe", stderr: "pipe" }) + const chain = await new Response(host.stdout).text() + await host.exited + console.log(` granted home : ${home}`) + for (const line of chain.trim().split("\n")) console.log(` resolves through : ${line.trim()}`) + + const spec = await Sandbox.wrapArgv({ + file: binary, + args: ["-c", "import sys; print(sys.version_info[:2])"], + workspace: [environment!], + ...(home ? { readable: [home] } : {}), + // No network: this isolates the spawn from everything the broker does. + options: { enabled: true, network: "deny", onUnavailable: "error", allowWrite: [] }, + }) + const proc = Bun.spawn([spec.file, ...(spec.args ?? [])], { + env: { ...process.env, ...spec.env }, + stdout: "pipe", + stderr: "pipe", + }) + const [out, err] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]) + await proc.exited + console.log(` exit ${proc.exitCode}\n stdout: ${out.trim()}\n stderr: ${err.trim()}`) + expect(proc.exitCode).toBe(0) + expect(out).toContain("(3,") + }, + 180_000, +) + +test.if(windows)( + "a process INSIDE the container can execute the environment's interpreter", + async () => { + // Not the same claim as the test above, which is why this exists. + // + // `CreateProcessW` opens the image file in the CALLER's context. Our + // launcher runs as the user and creates the process INTO the container, so + // that spawn proves the LAUNCHER can open the binary — not that the + // container can. uv is already inside when it spawns the interpreter to + // query it, and that is where it dies: + // + // DEBUG Checking for Python interpreter at path `Scripts\python.exe` + // error: Failed to query Python interpreter + // Caused by: Access is denied. (os error 5) + // + // So: launch a shell in the container and have IT spawn the interpreter. + // Red here means execute-from-inside is the missing right and every + // "runs inside the container" result above is weaker than it reads. + // cmd.exe explicitly, NOT Shell.acceptable(). On a GitHub runner that + // resolves to Git Bash under `C:\Program Files`, which the container cannot + // be granted — Windows only lets you grant paths you own. bash then fails + // its own DLL init with 0xC0000142 before reaching the interpreter, and the + // first version of this test read that as "the container cannot execute + // Python". It could not execute BASH. cmd.exe lives in System32 and starts + // in the container, as `a trivial command survives the container at all` + // already shows. + const home = await Installer.base(environment!) + const shell = `${process.env["SystemRoot"] ?? "C:\\Windows"}\\system32\\cmd.exe` + const plan = Sandbox.plan({ + command: `"${Installer.interpreter(environment!)}" -c "print(7)"`, + shell, + cwd: environment!, + workspace: [environment!], + ...(home ? { readable: [home] } : {}), + options: { enabled: true, network: "deny", onUnavailable: "error", allowWrite: [] }, + }) + const proc = Bun.spawn([plan.file, ...(plan.args ?? [])], { + env: { ...process.env, ...plan.env }, + // The spawn's cwd, not just the policy's. CreateProcessW inherits the + // launcher's working directory into the container, and this process runs + // from the checkout on D:\, which the container is not granted — so cmd + // exits 1 with "The current directory is invalid." before reaching Python. + // `Sandbox.plan({ cwd })` is policy input; the caller still has to spawn + // there, which is what tool/bash.ts does and what this test forgot. + cwd: environment!, + stdout: "pipe", + stderr: "pipe", + }) + const [out, err] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]) + await proc.exited + Sandbox.cleanup(plan) + // Assert on the child's OUTPUT before its exit code: three discriminators in + // a row have failed for reasons other than their hypothesis, and each time + // the child had said so in plain English on stderr while the exit code said + // nothing. + expect(err, `child stderr: ${err.trim()}`).not.toContain("current directory is invalid") + expect(err, `child stderr: ${err.trim()}`).not.toContain("Access is denied") + console.log(` exit ${proc.exitCode}\n stdout: ${out.trim()}\n stderr: ${err.trim()}`) + expect(out).toContain("7") + }, + 180_000, +) + +test.if(windows)( + "bun itself runs inside the container", + async () => { + // The shim is `bun ` in a source checkout, and it dies with + // `error loading current directory` even when lpCurrentDirectory names a + // directory the container holds (F) on and which is Low-labelled. So the + // question is no longer "which directory" but whether bun can run in an + // AppContainer at all. + // + // It matters beyond this test. If bun cannot, the dev shim path is not + // testable this way and the broker has to be exercised through a compiled + // binary — which is the code that ships, so that is the better test anyway + // and only the CI plumbing changes. If bun CAN, the fault is in the shim + // bundle or its arguments and stays where it is. + const spec = await Sandbox.wrapArgv({ + file: process.execPath, + args: ["--version"], + workspace: [environment!], + readable: [process.execPath], + options: { enabled: true, network: "deny", onUnavailable: "error", allowWrite: [] }, + }) + const proc = Bun.spawn([spec.file, ...(spec.args ?? [])], { + env: { ...process.env, ...spec.env }, + cwd: environment!, + stdout: "pipe", + stderr: "pipe", + }) + const [out, err] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]) + await proc.exited + console.log(` bun exit ${proc.exitCode}\n stdout: ${out.trim()}\n stderr: ${err.trim()}`) + expect(proc.exitCode).toBe(0) + }, + 180_000, +) + +test.if(windows)( + "a pre-existing subdirectory of the workspace is writable inside the container", + async () => { + // NOTE, corrected: this test now PASSES, and the conclusion drawn from its + // earlier failure was wrong. The Low label does reach a pre-existing + // subdirectory; what actually broke was an over-broad `readable` list + // leaking main's derived read roots into the Windows GRANT list, which sent + // icacls at every directory on PATH. The check is kept because it is cheap + // and it pins a property the sandbox genuinely depends on. + // + // Original note, left for the reasoning it records: + // TMPDIR pointed inside the granted environment and pip STILL could not + // write there: + // + // [Errno 13] Permission denied: + // '...\\openscience-install-K9UXyg\\env\\.tmp\\pip-unpack-...\\six-...whl.metadata' + // + // The self-test already proves a write in the workspace ROOT succeeds, and + // `grant()` labels that root Low because Mandatory Integrity Control is + // evaluated before the DACL and a Low process cannot write to a Medium + // object whatever the DACL says. The open question is whether that label + // reaches a subdirectory that already existed when the grant was applied — + // `icacls /setintegritylevel (OI)(CI)L` sets inheritance on the target, and + // inheritance is not the same as rewriting children. + // + // Both halves are measured here: a directory created BEFORE the launch and + // one created after. If only the second is writable, propagation is the + // fault and every pre-created scratch directory in the product has it. + const before = path.join(environment!, "pre-existing") + await fs.mkdir(before, { recursive: true }) + const code = [ + "import os, sys", + `open(os.path.join(r"${before}", "x.txt"), "w").write("ok")`, + `os.makedirs(os.path.join(r"${environment!}", "made-inside"), exist_ok=True)`, + `open(os.path.join(r"${environment!}", "made-inside", "y.txt"), "w").write("ok")`, + "print('both writes succeeded')", + ].join("\n") + const spec = await Sandbox.wrapArgv({ + file: Installer.interpreter(environment!), + args: ["-c", code], + workspace: [environment!], + options: { enabled: true, network: "deny", onUnavailable: "error", allowWrite: [] }, + }) + const proc = Bun.spawn([spec.file, ...(spec.args ?? [])], { + env: { ...process.env, ...spec.env }, + cwd: environment!, + stdout: "pipe", + stderr: "pipe", + }) + const [out, err] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]) + await proc.exited + console.log(` exit ${proc.exitCode}\n stdout: ${out.trim()}\n stderr: ${err.trim()}`) + expect(proc.exitCode).toBe(0) + }, + 180_000, +) + +test.if(windows)( + "pip reaches PyPI through the broker", + async () => { + // Spawned here rather than through `Installer.install` for one reason: + // stdio. `install` buffers stderr and prints it only when the install + // RETURNS, so the run that first got this far produced 600 seconds of + // silence and not one line about why. Inheriting stderr streams the + // launcher's debug into the CI log as it happens, which is the whole + // reason this job exists. + // + // The abort is deliberate and short. A refused connection fails in seconds; + // a hang means the shim accepted and the relay stalled, and those are + // different bugs. Ten minutes of waiting distinguishes them no better than + // three does. + const policy = await Config.trustedSandbox() + const egress = await EgressRuntime.egressFor(policy) + console.log(` egress: ${egress ?? ""}`) + expect(egress).toBeTruthy() + const home = await Installer.base(environment!) + const cache = path.join(workspace!, "pip-cache") + // No TMPDIR of its own. `spec.env` carries the sandbox's per-spawn temp + // root, which is granted and labelled as a ROOT; a directory pre-created + // here would be a child of the workspace and is not what the product uses + // any more. This test reconstructs `Installer.install`, and drifting from + // it is precisely how the reconstruction stops proving anything. + await fs.mkdir(cache, { recursive: true }) + const spec = await Sandbox.wrapArgv({ + file: Installer.interpreter(environment!), + // Back to `python -m pip`, matching install(). uv was only ever here to + // dodge the mkdtemp bug; pinning the managed interpreter to 3.12.3 removes + // that bug, so the reconstruction follows the product back. + args: ["-m", "pip", "install", "--disable-pip-version-check", "--only-binary", ":all:", "six"], + workspace: [environment!, cache], + ...(home ? { readable: [home] } : {}), + options: { ...policy, egress }, + }) + const proc = Bun.spawn([spec.file, ...(spec.args ?? [])], { + env: { + ...process.env, + ...spec.env, + PIP_CACHE_DIR: cache, + }, + cwd: environment!, + stdout: "pipe", + stderr: "inherit", + signal: AbortSignal.timeout(180_000), + }) + const out = await new Response(proc.stdout).text() + await proc.exited + for (const line of out.trim().split("\n")) console.log(` pip: ${line}`) + expect(proc.exitCode).toBe(0) + const found = await Installer.verify(environment!, ["six"]) + console.log(` verify: ${JSON.stringify(found)}`) + expect(found["six"]).toBeTruthy() + }, + 240_000, +) + +test.if(windows)( + "Installer.install works, not just a hand-composed equivalent", + async () => { + // The test above spawns pip itself so stderr can stream. That is a + // reconstruction of `install()`, and a reconstruction proves the transport + // rather than the product -- the TMPDIR omission above is precisely the + // class of difference that hides in one. So the real entry point runs too, + // once, against a package the previous test already cached. + const result = await Installer.install({ + directory: environment!, + packages: ["six"], + index: "", + source: false, + // Bounded so it RETURNS. install() buffers stderr and prints it only on + // return, so a run that outlives the test timeout produces five minutes + // of silence and not one line about why — which is exactly what the last + // run did. + signal: AbortSignal.timeout(150_000), + onProgress: (status) => console.log(` ${status}`), + }) + if (!result.ok) console.log(result.log) + expect(result.ok).toBe(true) + const found = await Installer.verify(environment!, ["six"]) + console.log(` verify: ${JSON.stringify(found)}`) + expect(found["six"]).toBeTruthy() + }, + 300_000, +) + +test.if(windows)("clean up", async () => { + if (workspace) await fs.rm(workspace, { recursive: true, force: true }).catch(() => {}) +}) diff --git a/backend/cli/test/sandbox/appcontainer-live.test.ts b/backend/cli/test/sandbox/appcontainer-live.test.ts new file mode 100644 index 00000000..3db9805f --- /dev/null +++ b/backend/cli/test/sandbox/appcontainer-live.test.ts @@ -0,0 +1,84 @@ +import { expect, test } from "bun:test" +import { Sandbox } from "../../src/sandbox/sandbox" + +/** + * The AppContainer counterpart to egress-live-seatbelt.test.ts: a real + * `CreateProcessW` with real `SECURITY_CAPABILITIES`, on a real Windows kernel. + * + * Everything else in `test/sandbox/` exercises the Windows branch from Linux + * with `platform: "win32"` injected, which proves what we COMPOSE and nothing + * about what Windows does with it. That gap cost roughly ten manual round trips + * on a human's machine, one command at a time, and the bugs it hid were not + * exotic: `-c` where cmd wanted `/c`, `printf` in a shell that has no `printf`, + * `CommandLineToArgvW` quoting handed to the one program that does not parse it + * that way, and a child with no inherited stdio. Every one of them would have + * been a red job here within minutes of being written. + * + * `platform` is deliberately never injected below. The whole point is to let + * `Sandbox.backend()` resolve for real, from a real `AppContainer.usable()` + * probe, on a machine where that probe can succeed. + * + * A red run means one of three things, and the check names distinguish them: + * 1. "the child actually runs inside the AppContainer" fails — the launch is + * not applying SECURITY_CAPABILITIES. Look at `appcontainer.ts`; run with + * OPENSCIENCE_SANDBOX_DEBUG=1 for the values handed to the kernel. + * 2. that passes but "write inside the workspace succeeds" fails — the + * container is live and the grants are wrong. Look at `grant()`. + * 3. that passes but "write outside" is not blocked — containment is real but + * leaky, which is the only genuinely alarming outcome. + */ + +const windows = process.platform === "win32" + +test.if(windows)( + "a trivial command survives the container at all", + async () => { + // Separates "the container cannot host a process here" from "this command + // failed inside it". A CI runner produced exit 66 with nothing on either + // stream, where a developer machine running the same build produced a token. + // `exit 7` needs no executable beyond the shell itself and no readable path, + // so it fails only if the container cannot host a process on this machine. + const { Shell } = await import("../../src/shell/shell") + const shell = Shell.acceptable() + const plan = Sandbox.plan({ + command: "exit 7", + shell, + cwd: process.cwd(), + workspace: [process.cwd()], + options: { enabled: true, network: "deny" }, + }) + const proc = Bun.spawn([plan.file, ...(plan.args ?? [])], { stdout: "pipe", stderr: "pipe" }) + const [out, err] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]) + await proc.exited + console.log( + ` shell ${shell}\n exit ${proc.exitCode} (expect 7)\n stdout: ${out.trim()}\n stderr: ${err.trim()}`, + ) + expect(proc.exitCode).toBe(7) + }, + 120_000, +) +test.if(windows)( + "the AppContainer confines a real child on a real Windows kernel", + async () => { + const result = await Sandbox.selfTest() + // Print every check before asserting: a bare "expected true, got false" from + // CI is worth almost nothing when the machine is not one we can log into. + for (const check of result.checks) + console.log( + ` ${check.skipped ? "skip" : check.pass ? "pass" : "FAIL"} ${check.name}${check.detail ? ` — ${check.detail}` : ""}`, + ) + expect(result.available).toBe(true) + expect(result.backend).toBe("appcontainer") + const containment = result.checks.find((c) => c.name.includes("runs inside the AppContainer")) + expect(containment?.pass).toBe(true) + expect(result.ok).toBe(true) + }, + 120_000, +) + +test.if(!windows)("this file's assertions are inert off Windows", () => { + // Guard against the file quietly becoming dead weight: if `selfTest` stops + // reporting an appcontainer backend name, the test above would skip forever + // on Windows too and nobody would notice. + expect(Sandbox.backend("win32")).toBe("appcontainer") +}) diff --git a/backend/cli/test/sandbox/appcontainer-transport.test.ts b/backend/cli/test/sandbox/appcontainer-transport.test.ts new file mode 100644 index 00000000..c1b7cabc --- /dev/null +++ b/backend/cli/test/sandbox/appcontainer-transport.test.ts @@ -0,0 +1,231 @@ +import { expect, test } from "bun:test" +import path from "path" +import { AppContainer } from "../../src/sandbox/appcontainer" +import { Installer } from "../../src/package/installer" +import { Sandbox } from "../../src/sandbox/sandbox" + +/** + * One measurement, on real hardware, that decides how Windows egress is built. + * + * The container cannot connect OUT to the host's loopback — the probe measured + * that (timeout). The design therefore assumes a named pipe as the only + * transport across the boundary, which in Bun means `CreateNamedPipeW` with a + * hand-built DACL plus overlapped I/O through FFI: heavy, and every Win32 step + * in this feature has cost several iterations. + * + * But the probe never tested the REVERSE direction. If the host can connect IN + * to a listener the container binds, the transport is plain TCP that Bun handles + * natively and the pipe disappears entirely. + * + * This file measures that and asserts nothing about the answer, deliberately. + * A red CI job should mean a defect, not an open question — so the assertion is + * only that the child bound a port (which container-internal loopback already + * proved possible), and the direction under test is reported for a human to read + * once. It becomes a real assertion as soon as we know which way it goes. + */ + +const windows = process.platform === "win32" + +test.if(windows)( + "can the host reach a listener bound inside the container?", + async () => { + const python = (await Installer.select()).binary + expect(python).toBeTruthy() + + // Bind, announce the port, accept one connection, answer. Nothing here needs + // network access: the container binds its own loopback, which is permitted. + const script = [ + "import socket,sys", + "s=socket.socket(); s.bind(('127.0.0.1',0)); s.listen(1)", + "print(s.getsockname()[1],flush=True)", + "c,_=s.accept(); c.sendall(b'reached'); c.close()", + ].join("\n") + + const wrapped = Sandbox.wrapArgv({ + file: python!, + args: ["-u", "-c", script], + workspace: [process.cwd()], + // The interpreter lives outside the workspace, so it must be granted read + // and execute or the container cannot start it. + readable: [path.dirname(python!)], + options: { enabled: true, network: "deny" }, + }) + expect(wrapped.sandboxed).toBe(true) + + const child = Bun.spawn([wrapped.file, ...wrapped.args], { stdout: "pipe", stderr: "pipe" }) + try { + const reader = child.stdout.getReader() + const deadline = Date.now() + 30_000 + let buffered = "" + let port = 0 + while (!port && Date.now() < deadline) { + const { value, done } = await reader.read() + if (done) break + buffered += new TextDecoder().decode(value) + port = Number(buffered.trim().split("\n")[0]) || 0 + } + console.log(` child bound 127.0.0.1:${port}`) + expect(port).toBeGreaterThan(0) + + // The question. A host process has no isolation restriction outbound, but + // whether Windows permits it to land on a socket owned by an AppContainer is + // exactly what nobody has measured. + const answer = await Bun.connect({ + hostname: "127.0.0.1", + port, + socket: { data() {}, open() {}, close() {}, error() {} }, + }) + .then(() => "REACHED") + .catch((error: Error) => `REFUSED (${error.message})`) + + console.log(`\n ===> host -> container listener: ${answer}\n`) + // Isolation runs both ways. If this ever reaches, Windows has relaxed + // AppContainer network isolation and the transport can lose the pipe. + expect(answer).toStartWith("REFUSED") + } finally { + child.kill() + } + }, + 120_000, +) + +test.if(windows)( + "only this container can open the broker pipe", + async () => { + // The measurement that decides whether the broker is buildable, taken before + // writing it. The probe measured these two halves with .NET; this takes them + // through OUR stack, because a grant that works in PowerShell but not through + // bun:ffi would surface at the worst possible moment. + // + // Two claims, and the second matters more than the first: + // 1. a pipe whose DACL names the package SID IS reachable from inside + // 2. the DACL is doing the work - a DEFAULT one is NOT + // Without (2) this proves nothing about confinement: a pipe anyone could + // open would also satisfy (1). So the negative case creates a REAL pipe with + // libuv's default security rather than pointing the child at a path that + // does not exist - which would fail for the wrong reason and look identical. + const python = (await Installer.select()).binary + // The SAME profile the sandbox will put the child in, not one of our own. + // Granting a pipe to a container nothing runs in denies exactly like having + // no grant at all -- which is how the first run of this read, with both + // halves reporting PermissionError and no way to tell them apart. + const sid = AppContainer.ensureProfile(Sandbox.appContainerProfile([process.cwd()])) + const stamp = `${process.pid}-${Date.now().toString(36)}` + + const script = (pipe: string) => + [ + `p = r'${AppContainer.pipePath(pipe)}'`, + "try:", + " f = open(p, 'r+b', buffering=0)", + " f.write(b'ping'); f.flush()", + " print('OPENED', f.read(9).decode(), flush=True)", + "except Exception as e:", + " print('DENIED', type(e).__name__, flush=True)", + ].join("\n") + + // Always read the child, even when the host side gives up. A harness that + // reports only its own timeout cannot tell "the DACL denied it" from "python + // never started" - which is the one distinction it exists to make, and the + // reason its first run taught us nothing. + const attempt = (pipe: string) => { + const wrapped = Sandbox.wrapArgv({ + file: python!, + args: ["-u", "-c", script(pipe)], + workspace: [process.cwd()], + readable: [path.dirname(python!)], + options: { enabled: true, network: "deny" }, + }) + const child = Bun.spawn([wrapped.file, ...wrapped.args], { stdout: "pipe", stderr: "pipe" }) + const said = Promise.all([new Response(child.stdout).text(), new Response(child.stderr).text()]) + return async () => { + const [out, err] = await said + return `${out.trim()}${err.trim() ? ` | stderr: ${err.trim().split("\n")[0]}` : ""}` + } + } + + // 1. Granted. + const grantedPipe = `openscience-broker-${stamp}` + const handle = AppContainer.createPipe(grantedPipe, sid) + const grantedSaid = attempt(grantedPipe) + let echoed: string + try { + echoed = AppContainer.pipeEchoOnce(handle, 30_000) + } catch (error) { + echoed = `` + } + const granted = await grantedSaid() + console.log(` granted DACL -> child said: ${granted}`) + console.log(` granted DACL -> host read: ${echoed}`) + + // 2. Default DACL: a REAL pipe, created by libuv with none of our security + // attributes, so the only difference from (1) is the descriptor. + const barePipe = `openscience-bare-${stamp}` + const net = await import("node:net") + const bare = net.createServer((socket) => socket.end("echo:ping")) + await new Promise((resolve) => bare.listen(AppContainer.pipePath(barePipe), () => resolve())) + const denied = await attempt(barePipe)() + bare.close() + console.log(` default DACL -> child said: ${denied}`) + + expect(granted).toContain("OPENED") + expect(echoed).toBe("ping") + // The grant is load-bearing, not decorative. + expect(denied).toContain("DENIED") + }, + 180_000, +) + +test.if(windows)( + "which client primitive speaks named pipes", + async () => { + // Decides whether the in-container shim needs writing at all. + // + // `Egress.serveShim` already does precisely what the Windows shim must do — + // accept on TCP loopback, relay to a socket path — and it connects with + // `Bun.connect({ unix })`. libuv treats a `\\.\pipe\...` path as a named pipe, + // so IF Bun's `unix:` option goes through that path, the shim is + // `serveShim({ port, socket: pipePath })` and there is no new code to write. + // + // If it does not, `node:net` certainly does, and the shim becomes a small + // variant that differs only in how it dials. Measured rather than assumed + // because the difference is "no work" versus "a new relay to test". + const net = await import("node:net") + const name = `openscience-probe-${process.pid}-${Date.now().toString(36)}` + const path_ = AppContainer.pipePath(name) + const server = net.createServer((socket) => socket.end("pong")) + await new Promise((resolve) => server.listen(path_, () => resolve())) + + const viaBun = await Bun.connect({ + unix: path_, + socket: { data() {}, open() {}, close() {}, error() {} }, + }) + .then((s) => { + s.end() + return "CONNECTED" + }) + .catch((error: Error) => `FAILED (${error.message})`) + + const viaNode = await new Promise((resolve) => { + const socket = net.connect(path_, () => { + socket.end() + resolve("CONNECTED") + }) + socket.on("error", (error) => resolve(`FAILED (${error.message})`)) + }) + + server.close() + console.log(`\n Bun.connect({unix}) -> ${viaBun}`) + console.log(` node:net.connect() -> ${viaNode}\n`) + console.log( + viaBun === "CONNECTED" + ? " serveShim works unchanged: the shim is serveShim({port, socket: pipePath})." + : " serveShim needs a Windows dial path; node:net is the primitive to use.", + ) + + // node:net is the floor — if even that cannot reach a pipe, the transport + // assumption underneath this whole design is wrong and everything else is + // moot, so that is the only hard assertion here. + expect(viaNode).toBe("CONNECTED") + }, + 60_000, +) diff --git a/backend/cli/test/sandbox/appcontainer.test.ts b/backend/cli/test/sandbox/appcontainer.test.ts new file mode 100644 index 00000000..4f1bde35 --- /dev/null +++ b/backend/cli/test/sandbox/appcontainer.test.ts @@ -0,0 +1,611 @@ +import { expect, test } from "bun:test" +import { AppContainer } from "../../src/sandbox/appcontainer" +import { Sandbox } from "../../src/sandbox/sandbox" + +/** + * What can be tested without Windows. + * + * The Win32 calls cannot run here, and pretending otherwise would be worse than + * admitting it — the probe already measured that sequence on a real machine. + * What IS testable is everything around them: the spec round trip, the + * UTF-16 encoding those `...W` entry points require, and the command-line + * quoting, which is where a silent mistake would hide. `CommandLineToArgvW` + * re-splits a single string with rules that are neither the shell's nor + * POSIX's, so a path like `C:\Users\me\My Project\` can quietly change what the + * child executes rather than failing loudly. + */ + +test("the spec survives the base64 round trip Sandbox composes", () => { + const policy = { + writable: ["C:\\work\\project"], + unreadable: ["C:\\Users\\me\\.ssh\\id_rsa"], + network: "allowlist" as const, + egress: "openscience-broker-abc", + profile: "openscience-deadbeef", + } + const args = Sandbox.appContainerArgs(policy, ["python.exe", "-u", "k.py"]) + const spec = AppContainer.decode(args[1]!) + expect(spec.profile).toBe("openscience-deadbeef") + expect(spec.writable).toEqual(["C:\\work\\project"]) + expect(spec.unreadable).toEqual(["C:\\Users\\me\\.ssh\\id_rsa"]) + expect(spec.network).toBe("allowlist") + expect(spec.pipe).toBe("openscience-broker-abc") +}) + +test("a spec with no profile is rejected rather than launched unconfined", () => { + const blob = Buffer.from(JSON.stringify({ writable: [], unreadable: [], network: "deny" })).toString("base64") + expect(() => AppContainer.decode(blob)).toThrow("profile") +}) + +test("wide() produces null-terminated UTF-16LE", () => { + // Every ...W entry point reads until a null. A missing terminator reads past + // the buffer; a UTF-8 buffer is silently misinterpreted as UTF-16 pairs. + const buf = AppContainer.wide("Hi") + expect([...buf]).toEqual([0x48, 0x00, 0x69, 0x00, 0x00, 0x00]) +}) + +test("readWide reverses wide(), and stops at the terminator", () => { + const sid = "S-1-15-2-3041870312-880516233" + const buf = AppContainer.wide(sid) + // Trailing garbage after the null must be ignored, the way a real SID buffer + // returned by ConvertSidToStringSid sits inside a larger allocation. + const padded = Buffer.concat([buf, Buffer.from([0x41, 0x00, 0x42, 0x00])]) + expect(AppContainer.readWide(new Uint8Array(padded))).toBe(sid) +}) + +test.each([ + ["plain", "python.exe", "python.exe"], + ["a space", "My Project", '"My Project"'], + ["a quote", 'say"hi', '"say\\"hi"'], + // A trailing backslash before the closing quote must be doubled, or it + // escapes the quote and swallows the next argument. + ["a trailing backslash with a space", "C:\\My Dir\\", '"C:\\My Dir\\\\"'], + ["backslashes before a quote", 'a\\\\"b', '"a\\\\\\\\\\"b"'], + ["backslashes with no quote", "C:\\a\\b", "C:\\a\\b"], +])("quoting %s survives CommandLineToArgvW", (_label, input, expected) => { + expect(AppContainer.quote(input)).toBe(expected) +}) + +test("a Windows path with spaces round-trips through the whole command line", () => { + // The case that matters in practice: the interpreter of a managed environment + // under a user profile whose name has a space in it. + const argv = ["C:\\Users\\A B\\.cache\\openscience\\envs\\p\\default\\Scripts\\python.exe", "-u", "C:\\w\\k.py"] + const line = AppContainer.commandLine(argv) + expect(line).toContain('"C:\\Users\\A B\\') + // Re-split the way CommandLineToArgvW would, to prove the quoting is not + // merely plausible. This mirrors the documented algorithm. + const parsed: string[] = [] + let current = "" + let quoted = false + let slashes = 0 + const flush = () => { + if (current || quoted) parsed.push(current) + current = "" + } + for (const ch of line) { + if (ch === "\\") { + slashes++ + continue + } + if (ch === '"') { + current += "\\".repeat(Math.floor(slashes / 2)) + if (slashes % 2) current += '"' + else quoted = !quoted + slashes = 0 + continue + } + current += "\\".repeat(slashes) + slashes = 0 + if (ch === " " && !quoted) { + flush() + continue + } + current += ch + } + current += "\\".repeat(slashes) + flush() + expect(parsed).toEqual(argv) +}) + +test("the launcher refuses to run anywhere but Windows", () => { + // Guards against a Linux caller reaching FFI that would dlopen kernel32. + if (process.platform === "win32") return + expect(() => AppContainer.launch("S-1-15-2-1", ["x"])).toThrow("only runs on Windows") +}) + +test("the entry point is wired before anything else the process does", async () => { + // This test used to check that the argv branch sat above `unhandledRejection` + // in `src/index.ts`, which measured nothing: ESM evaluates every static + // import before the first statement of the importing module, so "first in the + // file" and "first in the process" are not the same claim. The imports above + // it bootstrapped the user's directories, and the egress shim — this binary, + // re-entered INSIDE the sandbox — died there before running any of its own + // code. Position is now enforced by the module graph and asserted for real in + // test/sandbox/fastpath.test.ts, which makes those directories unwritable. + const source = await Bun.file(new URL("../../src/sandbox/fastpath.ts", import.meta.url).pathname).text() + expect(source).toContain('process.argv[2] === "__appcontainer-launch"') + // Nothing static: an import here is evaluated before the argv checks below it + // and puts the whole problem back. + expect(source.match(/^import .*/gm)).toBeNull() +}) + +test("the Windows grant list is exactly what the caller named", () => { + // `readable` means two different things per backend. On POSIX it is "what may + // be read", derived from runtime roots and the workspace, and binding those + // into a namespace costs nothing. On Windows every entry gets an ACE written + // to it with `icacls`, so inheriting the derived set made the launcher try to + // rewrite the ACLs of every directory on PATH — C:\Windows and + // C:\Windows\System32 among them. Unelevated that merely failed, slowly: + // 117 seconds of icacls calls before a trivial `exit 7` gave up. Elevated it + // would have SUCCEEDED, granting an AppContainer standing access to the + // system directories. + // + // So: nothing the caller did not name, and empty stays empty. + const spec = (wrapped: { args?: string[] }) => { + const args = wrapped.args ?? [] + const at = args.indexOf("__appcontainer-launch") + expect(at).toBeGreaterThan(-1) + return AppContainer.decode(args[at + 1]!) + } + + // POSIX-shaped paths even though the backend is win32: this runs on a Linux + // runner, where `dedupe`'s path.resolve rewrites a `C:\...` string into a + // cwd-relative one and the filters then drop it. The property under test is + // how many entries survive, which does not depend on their spelling. + const bare = Sandbox.wrapArgv({ + file: "cmd.exe", + args: ["/c", "exit 7"], + workspace: ["/work/project"], + options: { enabled: true, network: "deny" }, + platform: "win32", + }) + try { + expect(spec(bare).readable ?? []).toEqual([]) + } finally { + Sandbox.cleanup(bare) + } + + const base = "/opt/uv/python/cpython-3.12" + const named = Sandbox.wrapArgv({ + file: "python.exe", + args: [], + workspace: ["/work/project"], + readable: [base], + options: { enabled: true, network: "deny" }, + platform: "win32", + }) + try { + // Count and content, not the exact string: `dedupe` resolves paths, and a + // Windows path resolved on a Linux test runner comes back rewritten. The + // regression this guards against produced dozens of entries, so one is the + // assertion that matters. + const readable = spec(named).readable ?? [] + expect(readable).toHaveLength(1) + expect(readable[0]).toContain("cpython-3.12") + } finally { + Sandbox.cleanup(named) + } +}) + +test("describe() reports the appcontainer backend as available", () => { + // Two commands reading the same backend() disagreed on a real Windows + // machine: `sandbox status` printed "unavailable - no sandbox backend for + // platform win32" while `sandbox test` printed "Sandbox self-test + // (appcontainer)" and ran checks. describe() had a seatbelt/bubblewrap + // whitelist, so widening the Backend type without widening it here made the + // new backend fall through to the "none" branch. + const source = Bun.file(new URL("../../src/sandbox/sandbox.ts", import.meta.url).pathname) + return source.text().then((text) => { + const body = text.slice(text.indexOf("export function describe()"), text.indexOf("writable-path assembly")) + expect(body.includes('b === "appcontainer"')).toBe(true) + expect(body.includes('tool: "AppContainer"')).toBe(true) + }) +}) + +test("the child inherits the launcher's std handles", async () => { + // `bInheritHandles: false` with no STARTF_USESTDHANDLES was silently fatal: + // the launcher runs with its stdout on a pipe, so a child inheriting nothing + // had nowhere to write and EVERY sandboxed command came back empty. The first + // Windows self-test read that empty stdout, found no package SID, and + // reported the container as not applied — a launcher bug wearing a policy + // bug's clothes. Not test-only: pip progress and every tool result cross here. + const source = await Bun.file(new URL("../../src/sandbox/appcontainer.ts", import.meta.url).pathname).text() + const body = source.slice(source.indexOf("export function launch")) + // bInheritHandles is the last argument before the creation flags. Slicing to + // the first ")" would land inside `ffi.ptr(line)`, so bound it on the flags. + const create = body.slice(body.indexOf("kernel.CreateProcessW"), body.indexOf("EXTENDED_STARTUPINFO_PRESENT |")) + expect(create).toContain("true,") + expect(create).not.toContain("false,") + expect(body).toContain("STARTF_USESTDHANDLES") + // Handles we were given are not necessarily marked inheritable in us. + expect(body).toContain("SetHandleInformation") + // The flag must not be set without handles behind it, or the child gets no + // stdout at all — the same failure by another route. + expect(body.indexOf("if (stdout && stderr)")).toBeLessThan(body.indexOf("STARTF_USESTDHANDLES, true")) +}) + +test("the CreateProcess failure explains 203, the code a real machine returned", async () => { + // The hint listed Win32 5 and 2. The machine returned 203, so at the moment of + // failure the number carried no meaning at all. 203 is ERROR_ENVVAR_NOT_FOUND, + // which points at the environment rather than the command — lpApplicationName + // is null, so Windows resolves argv[0] itself and needs an environment to do + // it in. + const source = await Bun.file(new URL("../../src/sandbox/appcontainer.ts", import.meta.url).pathname).text() + expect(source).toContain("203 is ERROR_ENVVAR_NOT_FOUND") + // Comments are not code: this asserts the flag is not USED, and the comment + // explaining why it was removed must not trip it. + const code = source + .split("\n") + .filter((l) => !/^\s*(\/\/|\*|\/\*)/.test(l)) + .join("\n") + // And the flag that described an environment block we never supply is gone. + expect(code).not.toContain("CREATE_UNICODE_ENVIRONMENT") +}) + +test("readable paths reach the launcher and are granted read+execute, not full control", async () => { + // The gap that made Windows look like a broken machine. bubblewrap binds the + // whole filesystem read-only and seatbelt allows reads unless denied, so + // `readable` is a no-op on both and its absence here went unnoticed. An + // AppContainer reaches nothing whose ACL does not name its package SID, so + // dropping it left the kernel unable to read its own interpreter: `dir` + // returned "Access is denied" and the venv redirector reported + // `No Python at '...'` for a Python that was installed and working. + const args = Sandbox.appContainerArgs( + { + writable: ["C:\\work\\project"], + readable: ["C:\\Python312"], + unreadable: [], + network: "deny" as const, + profile: "openscience-deadbeef", + }, + ["python.exe"], + ) + expect(AppContainer.decode(args[1]!).readable).toEqual(["C:\\Python312"]) + + const source = await Bun.file(new URL("../../src/sandbox/appcontainer.ts", import.meta.url).pathname).text() + // Bounded at the next function, not at quote(): revoke() sits between them and + // legitimately sets the label back to Medium, which a wider slice misreads as + // grant() relabelling the READ set. + const body = source.slice(source.indexOf("export function grant"), source.indexOf("export function removeProfile")) + // Read AND execute: the interpreter must be runnable, so plain (R) is not + // enough. Never (F) for the read set — that would hand a sandboxed process + // write access to the Python installation it is confined away from. + expect(body).toContain("(OI)(CI)(RX)") + expect(body).toContain("(OI)(CI)(F)") + expect(body.indexOf("(OI)(CI)(F)")).toBeLessThan(body.indexOf("(OI)(CI)(RX)")) +}) + +test("a path that is already writable is not re-granted as read-only", () => { + // Two ACEs for one SID on one path is not wrong, but the weaker one is noise + // in `icacls` output and makes a real grant failure harder to spot. + const args = Sandbox.appContainerArgs( + { + writable: ["C:\\work\\project"], + readable: ["C:\\work\\project", "C:\\Python312"], + unreadable: [], + network: "deny" as const, + profile: "p", + }, + ["x.exe"], + ) + const spec = AppContainer.decode(args[1]!) + expect(spec.readable).toContain("C:\\work\\project") + // The de-duplication is in grant(), which is where both lists are known. + expect(Bun.file(new URL("../../src/sandbox/appcontainer.ts", import.meta.url).pathname).text()).resolves.toContain( + "readable.filter((p) => !writable.includes(p))", + ) +}) + +test("the launcher can dump every value CreateProcess is given", async () => { + // `sandbox test` proved the child runs unconfined: CreateProcess succeeds, the + // command executes, and the token carries no package SID. The probe ran this + // same sequence successfully in PowerShell on the same machine, so the fault + // is in what we hand the kernel. Each guess at that cost a full rebuild cycle, + // which is why the values are now dumpable in one run. + const source = await Bun.file(new URL("../../src/sandbox/appcontainer.ts", import.meta.url).pathname).text() + const body = source.slice(source.indexOf("export function launch")) + expect(body).toContain("OPENSCIENCE_SANDBOX_DEBUG") + // The four values that can each independently cause a silent no-op: the SID, + // the struct handed to UpdateProcThreadAttribute, cb, and the list pointer. + for (const value of ["capabilities ", "startupinfoex ", "cb=", "lpAttributeList=0x"]) expect(body).toContain(value) +}) + +test("the FFI bindings are opened once and held", async () => { + // dlopen returns a library object that owns the handle; keeping only .symbols + // left it garbage, and Bun closes a library when that object is collected — + // unmapping code a later call jumps into. main() bound three times per launch + // and launch() opened advapi32 a fourth. + const source = await Bun.file(new URL("../../src/sandbox/appcontainer.ts", import.meta.url).pathname).text() + expect(source).toContain("libs: [userenv, advapi, kernel]") + expect(source).toContain("bound ??= open()") + // advapi32 must not be reopened inside launch(). + const body = source.slice(source.indexOf("export function launch")) + expect(body).not.toContain("dlopen") +}) + +test("cmd.exe gets its tail verbatim, not CommandLineToArgvW quoting", () => { + // Measured on a real machine. cmd does NOT parse its /c tail with + // CommandLineToArgvW and does not recognise a backslash-escaped quote, so + // quoting the tail normally produced + // "echo hi>\"C:\\...\\probe\"" + // and cmd answered "The filename, directory name, or volume label syntax is + // incorrect." The same shape turned `dir C:\` into `dir C:\\`. + const command = 'echo hi>"C:\\Users\\naray\\AppData\\Local\\Temp\\openscience-sbx-ab12\\probe"' + const line = AppContainer.commandLine(["C:\\WINDOWS\\system32\\cmd.exe", "/d", "/s", "/c", command]) + // The tail is wrapped exactly once and its inner quotes are untouched: with + // /s cmd strips the first and last quote and takes the rest verbatim. + // The exe path has no spaces, so quote() correctly leaves it bare. + expect(line).toBe(`C:\\WINDOWS\\system32\\cmd.exe /d /s /c "${command}"`) + expect(line).not.toContain('\\"') +}) + +test("the trailing-backslash case that broke `dir C:\\`", () => { + const line = AppContainer.commandLine(["cmd.exe", "/d", "/s", "/c", "dir C:\\"]) + // Not `dir C:\\`, which is what doubling the backslash produced. + expect(line).toBe('cmd.exe /d /s /c "dir C:\\"') +}) + +test("everything that is not cmd still gets CommandLineToArgvW quoting", () => { + // The rule is a property of the TARGET's parser, so only cmd is special. A + // path with a space must still round-trip for python.exe. + const line = AppContainer.commandLine(["C:\\Py 3\\python.exe", "-c", 'print("hi")']) + expect(line).toContain('"C:\\Py 3\\python.exe"') + expect(line).toContain('\\"') + // And an executable merely named like cmd in an argument does not trigger it. + expect(AppContainer.commandLine(["python.exe", "/c", "x"])).toBe("python.exe /c x") +}) +test("containment is proved by the kernel, not by a command inside the container", async () => { + // The check used to run `whoami /groups` and pattern-match its output, which + // made containment depend on a command succeeding INSIDE the container. On a + // CI runner it does not: whoami resolves SIDs to display names through LSA, + // which an AppContainer with zero capabilities cannot reach, so it exits 66 + // having printed nothing — while `exit 7` through the identical plan returns + // 7, proving the container hosts processes perfectly well. Two rounds were + // spent reading that as a containment failure. + // + // The launcher holds the child's process handle, so it asks the kernel + // TokenIsAppContainer and reports the answer. Nothing depends on what the + // child can do. + const launcher = await Bun.file(new URL("../../src/sandbox/appcontainer.ts", import.meta.url).pathname).text() + expect(launcher).toContain("TOKEN_IS_APP_CONTAINER = 29") + expect(launcher).toContain("OpenProcessToken") + expect(launcher).toContain("token appcontainer=") + + const text = await Bun.file(new URL("../../src/sandbox/sandbox.ts", import.meta.url).pathname).text() + const body = text.slice(text.indexOf("export async function selfTest")) + expect(body).toContain("OPENSCIENCE_APPCONTAINER_REPORT") + expect(body).toContain("token appcontainer=") + // And it no longer asks a child to introspect itself. Comments are not code: + // the history of why whoami was wrong is worth keeping in the file. + const code = body + .split("\n") + .filter((l) => !/^\s*(\/\/|\*|\/\*)/.test(l)) + .join("\n") + expect(code).not.toContain("whoami") +}) + +test("a writable path gets a Low mandatory label, not only a DACL grant", async () => { + // The root cause of "write inside the workspace succeeds" failing, and it is + // not about paths or quoting, which is what several rounds chased. + // + // Every AppContainer runs at Low integrity; a directory created normally is + // Medium. Mandatory Integrity Control is evaluated BEFORE the DACL, and a + // Low-integrity principal cannot write to a Medium-integrity object even when + // the DACL grants it write access. So `/grant *SID:(OI)(CI)(F)` alone could + // never let the sandbox write anywhere. + const source = await Bun.file(new URL("../../src/sandbox/appcontainer.ts", import.meta.url).pathname).text() + // Bounded at the next function, not at quote(): revoke() sits between them and + // legitimately sets the label back to Medium, which a wider slice misreads as + // grant() relabelling the READ set. + const body = source.slice(source.indexOf("export function grant"), source.indexOf("export function removeProfile")) + expect(body).toContain("/setintegritylevel") + expect(body).toContain("(OI)(CI)L") + // Only writable paths are relabelled. Lowering the label on the READ set + // would let any low-integrity process on the machine modify the interpreter + // the sandbox then executes — the opposite of the point. + const relabel = body.indexOf("/setintegritylevel") + const readGrant = body.indexOf("(OI)(CI)(RX)") + expect(relabel).toBeLessThan(readGrant) + expect(body.slice(readGrant)).not.toContain("/setintegritylevel") +}) + +test("capabilities are granted only for allow, and travel in the spec", () => { + // `allow` means unrestricted egress on Linux and macOS, so it has to mean that + // here rather than quietly meaning less — withholding the capabilities would + // make the knob claim more than it delivers for no security anyone asked for. + const spec = (network: "deny" | "allowlist" | "allow") => + AppContainer.decode( + Sandbox.appContainerArgs( + { writable: [], unreadable: [], network, profile: "p", ...(network === "allowlist" ? { egress: "pipe" } : {}) }, + ["x.exe"], + )[1]!, + ) + expect(spec("allow").capabilities).toEqual(["S-1-15-3-1", "S-1-15-3-3"]) + // Zero for allowlist is load-bearing, not incidental: the broker is the + // enforcement point, so a container that could reach the internet directly + // would route around the allowlist while still reporting a policy was applied. + expect(spec("allowlist").capabilities).toEqual([]) + expect(spec("deny").capabilities).toEqual([]) +}) + +test("the launcher builds a SID_AND_ATTRIBUTES array, not just a count", async () => { + // UpdateProcThreadAttribute stores POINTERS; the kernel reads through them at + // CreateProcess. A capability array built and then dropped would be read after + // free — the same lifetime rule the SECURITY_CAPABILITIES struct follows. + const source = await Bun.file(new URL("../../src/sandbox/appcontainer.ts", import.meta.url).pathname).text() + const body = source.slice(source.indexOf("export function launch")) + expect(body).toContain("SID_AND_ATTRIBUTES_SIZE") + expect(body).toContain("SE_GROUP_ENABLED") + // Held alive alongside the attribute list until after CreateProcess. + expect(body).toContain("keep.push(granted)") +}) + +test("allow is not yet a superset of allowlist, and the code says so", async () => { + // A knob where LOOSENING the policy REMOVES a capability is a design smell. + // On the other two platforms `allow` applies no network restriction at all + // (`!== "allow"` guards both --unshare-net and `(deny network*)`), so it is + // already a superset there; Windows should match rather than invent an + // ordering of its own. + // + // Until the broker lands, `allow` grants capabilities and runs no broker, so + // localhost is genuinely unreachable and the warning is accurate. This test + // exists to make the follow-up visible rather than lost: when a broker starts + // under `allow`, the warning must narrow to non-HTTP. + const source = await Bun.file(new URL("../../src/sandbox/sandbox.ts", import.meta.url).pathname).text() + expect(source).toContain("THIS WORDING MUST NARROW") + // No longer a superset ANYWHERE, which changes what the follow-up is. main + // severs the network on bubblewrap and denies every socket on seatbelt in + // every mode, "allow" included, because neither can express "the internet but + // never host loopback". So Windows is now the only backend where "allow" + // reaches anything at all, and the three platforms disagree about what the + // word means. That is the thing to fix — by routing "allow" through the + // allowlist proxy with an unrestricted host list — and it is tracked here so + // it stays visible rather than becoming folklore. + expect(source).toContain('args.push("--unshare-net")') + expect(source).toContain("(deny default)") +}) + +test("the pipe never blocks the event loop waiting for a client", async () => { + // A synchronous blocking Win32 call in a JS process blocks the event loop, so + // the caller's own timeout can never fire. Measured the hard way: a bare + // ConnectNamedPipe hung a Windows CI job until the 20-minute job limit, while + // the test's 120s timeout sat waiting behind the very call it was meant to + // bound. Non-blocking mode plus a polled deadline is the fix here; the broker + // proper needs overlapped I/O for the same reason. + const source = await Bun.file(new URL("../../src/sandbox/appcontainer.ts", import.meta.url).pathname).text() + expect(source).toContain("PIPE_NOWAIT") + expect(source).toContain("PIPE_TYPE_BYTE | PIPE_NOWAIT") + const body = source.slice(source.indexOf("export function pipeEchoOnce")) + // Every wait is bounded and yields rather than blocking indefinitely. + expect(body).toContain("timeoutMs") + expect(body).toContain("Bun.sleepSync") + expect(body).toContain("no client reached the pipe within") +}) + +test("what grant() changes, revoke() puts back", async () => { + // The Low mandatory label is the only way a Low-integrity AppContainer can + // write anywhere, and it is not a change to make and walk away from: it means + // ANY low-integrity process on the machine can write there — a sandboxed + // browser tab, a document preview — and it was being left on the user's own + // project directory after the run that needed it had exited. + const source = await Bun.file(new URL("../../src/sandbox/appcontainer.ts", import.meta.url).pathname).text() + const body = source.slice(source.indexOf("export function revoke"), source.indexOf("SECURITY_ATTRIBUTES_SIZE")) + // Restored, not merely un-granted. + expect(body).toContain('"/setintegritylevel", "(OI)(CI)M"') + expect(body).toContain('"/remove:g"') + + // And it runs on every exit path, including a launch that throws. + const main = source.slice(source.indexOf("export async function main")) + expect(main).toContain("finally") + expect(main.indexOf("finally")).toBeLessThan(main.indexOf("revoke(")) +}) + +test("a throwaway workspace does not orphan an AppContainer profile", async () => { + // Every self-test run builds a fresh mkdtemp workspace, and the profile name + // is derived from the workspace — so every run left a profile and an + // AppData\Local\Packages folder behind. Visible on a real machine as a + // different package SID in each run's output. + // + // A real project must NOT be cleaned up this way: reusing one profile is what + // keeps its grants stable across runs. + const source = await Bun.file(new URL("../../src/sandbox/sandbox.ts", import.meta.url).pathname).text() + const body = source.slice(source.indexOf("export async function selfTest")) + expect(body).toContain("AppContainer.removeProfile") + const launcher = await Bun.file(new URL("../../src/sandbox/appcontainer.ts", import.meta.url).pathname).text() + expect(launcher).toContain("DeleteAppContainerProfile") +}) + +test("the network probe measures the host before blaming the sandbox", async () => { + // Two defects a real Windows run exposed, both mine, both the same shape as + // the ones that cost this feature days. + // + // `-o /dev/null` made curl try to create C:\dev\null, so the probe failed for + // a PATH reason and was reported as a network result — the same POSIX-only + // assumption that once put `printf` and `cat` into a cmd.exe probe. + // + // And the skip message asserted "the container holds no capabilities", which + // was true when written and became FALSE the moment `allow` started granting + // internetClient. A diagnostic that states a cause it never measured is + // exactly the pattern this feature keeps having to unlearn. + const source = await Bun.file(new URL("../../src/sandbox/sandbox.ts", import.meta.url).pathname).text() + const body = source.slice(source.indexOf("export async function selfTest")) + // No output flag at all now. Two probes died on that flag: `-o /dev/null` made + // curl try to create C:\dev\null, and `-o NUL` then failed INSIDE the + // container with curl exit 23 (CURLE_WRITE_ERROR) — which means curl connected + // and received the response and could not write it. The network worked; the + // probe did not. Both were reported as network results. + // The command itself, not the body: the comment explaining why `-o` was + // removed necessarily contains `-o`. + const command = body.slice(body.indexOf("const curlCmd"), body.indexOf("\n", body.indexOf("const curlCmd"))) + expect(command).toContain("curl -m 5 -sf https://example.com") + expect(command).not.toContain("-o ") + expect(body).not.toContain("holds no capabilities") + + // And an exit code that is not a network refusal must not be blamed on the + // sandbox. 6/7/28 are resolve/connect/timeout; everything else is curl + // reporting a problem of its own. + expect(body).toContain("const denied =") + expect(body).toContain("the probe itself failed under allow") + + // The measurement that makes this assertable rather than inconclusive: ask the + // HOST first. A sandbox that cannot reach a network the host can reach is a + // real failure, and on Windows it means the capability grant did not take + // effect — which is otherwise a silent no-op. + expect(body).toContain("const reachable") + expect(body).toContain("network egress works in allow mode") + expect(body).toContain("the capability grant is not taking effect") +}) + +test("no line this process wrote is ever reported as the child's error", async () => { + // Third variant of the same defect. The launcher's debug dump was caught + // first, so firstLine skipped that one prefix — and then a red CI job + // reported the reason a sandboxed curl failed as + // INFO 2026-08-14 service=openscience api_base=... + // which is the structured logger, on the same stderr, wearing the child's + // clothes. Skipping one known prefix is not the fix; skipping anything that + // came from us is. + const source = await Bun.file(new URL("../../src/sandbox/sandbox.ts", import.meta.url).pathname).text() + const body = source.slice(source.indexOf("const ours ="), source.indexOf("function runAsync")) + expect(body).toContain("openscience[") + expect(body).toContain("service=openscience") + expect(body).toContain("INFO|WARN|ERROR|DEBUG") + // And an empty stderr must still say something: the exit code. + const check = source.slice(source.indexOf("export async function selfTest")) + expect(check).toContain("no stderr") +}) + +test("the broker never blocks, and never leaves the pipe unlistenable", async () => { + // Every Win32 call the broker makes has to be non-blocking. A bare + // ConnectNamedPipe once held a CI runner for twenty minutes because the + // timeout meant to bound it was queued behind it, and the broker runs for the + // life of a session rather than one measurement. + const source = await Bun.file(new URL("../../src/sandbox/appcontainer.ts", import.meta.url).pathname).text() + const body = source.slice(source.indexOf("export function serveBroker"), source.indexOf("export function quote")) + expect(body).not.toContain("INFINITE") + expect(body).toContain("setTimeout(pump") + + // A new instance is opened the moment one is accepted. Without that there is + // a window where the pipe name exists but nothing is listening on it, and a + // client that arrives in it is refused rather than queued. + const accept = body.slice(body.indexOf("const accept =")) + expect(accept.indexOf("links.add(link)")).toBeLessThan(accept.indexOf("listening = openInstance()")) + + // Bytes from the proxy are only written once the socket exists. They can + // arrive before the dial resolves, and writing them to a handle whose peer has + // not been dialled yet loses them silently. + expect(body).toContain("while (link.pending.length && link.socket)") + + // Idle sessions must not pay for a 1ms timer they are not using. + expect(body).toContain("moved ? 1 : 15") +}) + +test("the broker's read buffer is allocated once per link and held", async () => { + // The kernel writes through this pointer. A per-call allocation could be moved + // by the GC between ffi.ptr() taking its address and ReadFile using it — the + // same lifetime rule the attribute list and capability array already follow. + const source = await Bun.file(new URL("../../src/sandbox/appcontainer.ts", import.meta.url).pathname).text() + const body = source.slice(source.indexOf("export function serveBroker"), source.indexOf("export function quote")) + const instance = body.slice(body.indexOf("const openInstance"), body.indexOf("const drop")) + expect(instance).toContain("buffer: new Uint8Array(65536)") + expect(body).toContain("ffi.ptr(link.buffer)") +}) diff --git a/backend/cli/test/sandbox/egress-live-seatbelt.test.ts b/backend/cli/test/sandbox/egress-live-seatbelt.test.ts new file mode 100644 index 00000000..36914c5d --- /dev/null +++ b/backend/cli/test/sandbox/egress-live-seatbelt.test.ts @@ -0,0 +1,246 @@ +import { describe, expect, test } from "bun:test" +import crypto from "crypto" +import { Egress } from "../../src/sandbox/egress" +import { Sandbox } from "../../src/sandbox/sandbox" +import { tmpdir } from "../fixture/fixture" + +/** + * The seatbelt counterpart to egress-live.test.ts: a real `sandbox-exec`, a + * real TCP-loopback `Egress.serveProxy`, and a real remote host, wired + * together exactly the way `Sandbox.plan` composes them in production. Task + * 7 (see `.superpowers/sdd/2026-08-09-sandbox-network-policy/task-7-report.md`) + * built the seatbelt profile and the authenticated loopback proxy entirely + * from Linux, with `platform: "darwin"` injected on every assertion — nobody + * on the project has a Mac, so none of it had ever reached a real + * `sandbox-exec`. This file is what runs when a Mac finally does. + * + * `platform` is deliberately never passed to `Sandbox.plan` below. Every + * seatbelt-specific test elsewhere in `test/sandbox/` injects `"darwin"` to + * exercise the branch from Linux; this file's whole reason to exist is to + * let `Sandbox.backend()`/`decide()` resolve for real, from a real + * `Bun.which("sandbox-exec")` probe, on a machine where that probe can + * actually succeed. + * + * Two open questions from the Task 7 report are what a red run here would + * mean, and this file is written so a reader can tell which: + * + * 1. Whether `network-bind`/`network-inbound` are needed at all for the + * implicit local bind a TCP `connect()` performs, or whether + * `(deny network*)` blocks it regardless of the three narrow allows — + * in which case the sandboxed process never reaches the proxy at all. + * A failure here shows up as the FIRST test below failing to reach + * "200" for the allowlisted host (the process can't dial the proxy + * port in the first place), typically with curl reporting a connection + * error in `stderr` rather than any HTTP status. + * 2. Whether the filter spelling seatbeltProfile emits — `(remote tcp + * "localhost:PORT")` — is what a real `sandbox-exec` expects, versus + * the `(remote ip ...)` this function used before Task 7's fix round 1. + * A failure here shows up the same way as (1) — a wrong filter keyword + * either fails `sandbox-exec -p` outright (a syntax/parse error in + * `stderr`, non-zero exit before the script's own commands ever run) + * or silently fails to match any traffic, which reads identically to + * (1) from this test's vantage point. Either way "the proxy is + * unreachable" is the shared symptom; telling the two apart needs a + * human reading `stderr` for a `sandbox-exec` parse error specifically + * — present means (2), absent means (1) or a genuine enforcement gap. + * + * Everything downstream of that — denied host refused, direct egress with + * the proxy env unset failing, DNS resolving nothing inside the sandbox, + * and volume surviving byte-for-byte through the seatbelt-side proxy path — + * is new coverage of its own kind, not a restatement of the Linux file: + * `Egress.serveProxy`'s TCP/authenticated branch (used only by seatbelt) has + * never taken a live client through a real OS network boundary before this. + * + * Gated on `Sandbox.backend() === "seatbelt"`, real and non-injected — this + * skips on Linux (where it stays exercised by the darwin-injected unit tests + * elsewhere in this directory) and runs, unskipped, on the one machine that + * can: a broken profile on that machine must fail this test, not quietly + * skip it. + */ + +const curl = Bun.which("curl") +const python = Bun.which("python3") + +/** + * A direct, un-sandboxed HEAD against a host the shipped default allowlist + * already permits, run once at collection time — same purpose as + * egress-live.test.ts's `reachable()`: without it, a macOS runner with no + * route to the internet would see the checks below fail exactly the way a + * broken profile would, which is not the defect this file exists to catch. + */ +function reachable() { + if (!curl) return false + const probe = Bun.spawnSync([ + curl, + "-sS", + "-I", + "-m", + "5", + "-o", + "/dev/null", + "-w", + "%{http_code}", + "https://pypi.org/simple/", + ]) + return probe.exitCode === 0 && probe.stdout.toString().trim() === "200" +} + +const skip = Sandbox.backend() !== "seatbelt" || !curl || !python || !reachable() + +/** A real host-side allowlist proxy on an OS-assigned loopback port, fed the + * real shipped `DEFAULT_RULES` — same shape as egress-live.test.ts's + * `proxy()`, but the seatbelt/TCP overload of `Egress.serveProxy` (a fresh + * `crypto.randomUUID()` secret per call, matching what `EgressRuntime`'s + * `startSeatbelt` does for a real proxy start) rather than a unix socket. */ +function proxy(rules: Egress.Rule[]) { + const secret = crypto.randomUUID() + const server = Egress.serveProxy({ hostname: "127.0.0.1", port: 0, secret, rules }) + return { + port: server.port, + secret, + stop: () => server.stop(true), + } +} + +/** Run `script` the way a sandboxed shell command actually runs in + * production — through `Sandbox.plan`, a real `sandbox-exec`-wrapped shell, + * and the real per-connection proxy allowlist check. `egress` is + * `":"`, the exact shape `EgressRuntime.egressFor` produces + * for seatbelt; `buildPolicy` splits it back into `Policy.port`/ + * `Policy.secret`. No `platform` override — see the file doc comment. */ +async function run(script: string, work: string, egress: string) { + const spec = Sandbox.plan({ + command: script, + shell: "/bin/sh", + cwd: work, + workspace: [work], + options: { enabled: true, network: "allowlist", egress }, + }) + const proc = Bun.spawn([spec.file, ...(spec.args ?? [])], { + cwd: work, + env: { ...process.env, ...spec.env }, + stdout: "pipe", + stderr: "pipe", + }) + const [stdout, stderr] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]) + await proc.exited + return { stdout, stderr } +} + +describe.skipIf(skip)("egress: a real seatbelt sandbox, a real proxy, a real remote host", () => { + test("an allowlisted host reaches 200, a denied one does not, and neither a direct connection nor DNS works outside the shim", async () => { + await using work = await tmpdir() + const host = proxy(Egress.DEFAULT_RULES) + try { + const egress = `${host.port}:${host.secret}` + const script = [ + `pypi=$(curl -sS -I -m 30 -o /dev/null -w '%{http_code}' https://pypi.org/simple/)`, + `eutils=$(curl -sS -m 30 -o /dev/null -w '%{http_code}' 'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/einfo.fcgi')`, + `example=$(curl -sS -I -m 15 -o /dev/null -w '%{http_code}' https://example.com/)`, + // Same load-bearing pair as egress-live.test.ts, and the same + // reasoning: `unset` inside a subshell strips every proxy var for + // this one curl only, so a 200 here would mean the loopback port is + // a convenience rather than the only way out. + `direct=$(unset HTTP_PROXY HTTPS_PROXY http_proxy https_proxy ALL_PROXY all_proxy NO_PROXY no_proxy; curl -sS -I -m 10 -o /dev/null -w '%{http_code}' https://pypi.org/)`, + // No `getent` on macOS. `python3 -c` resolves the same host and this + // captures only its exit status: 0 if `gethostbyname` returned an + // address (DNS worked, which it must not, inside the sandbox), a + // Python traceback's exit code (1, unhandled `socket.gaierror`) + // otherwise. Output is discarded either way — only the exit code + // is load-bearing, so nothing here depends on Python's traceback + // format. + `dns=$(python3 -c "import socket; socket.gethostbyname('pypi.org')" >/dev/null 2>&1; printf '%s' "$?")`, + `printf 'PYPI=%s\\nEUTILS=%s\\nEXAMPLE=%s\\nDIRECT=%s\\nDNS=%s\\n' "$pypi" "$eutils" "$example" "$direct" "$dns"`, + ].join("\n") + + const { stdout, stderr } = await run(script, work.path, egress) + const field = (name: string) => stdout.match(new RegExp(`^${name}=(.*)$`, "m"))?.[1] + const detail = `stdout=${stdout} stderr=${stderr}` + + // Guarded with a shape check first on every field, matching + // egress-live.test.ts's convention: without it, a parsing regression + // that made `field()` return undefined would still satisfy + // `not.toBe(...)` and pass with nothing actually verified. + expect(field("PYPI"), detail).toBe("200") + expect(field("EUTILS"), detail).toBe("200") + expect(field("EXAMPLE"), detail).toMatch(/^\d{3}$/) + expect(field("EXAMPLE"), detail).not.toBe("200") + expect(field("DIRECT"), detail).toMatch(/^\d{3}$/) + expect(field("DIRECT"), detail).not.toBe("200") + expect(field("DNS"), detail).toMatch(/^\d+$/) + expect(field("DNS"), detail).not.toBe("0") + } finally { + host.stop() + } + }, 120_000) + + // Same file, same size, same hash as egress-live.test.ts's wheel test — + // deliberately not re-derived, so a divergence between the two backends' + // handling of the exact same bytes would show up as one green and one red + // rather than two different payloads that happen to both pass. `pump` + // (egress.ts) is shared code between the unix-socket and TCP/loopback + // listeners; this is the first time its TCP branch has moved anything + // this large through a real OS network boundary rather than a stubbed one. + const WHEEL_URL = + "https://files.pythonhosted.org/packages/3a/d0/edc009c27b406c4f9cbc79274d6e46d634d139075492ad055e3d68445925/numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl" + const WHEEL_SIZE = 18_252_005 + const WHEEL_SHA256 = "666dbfb6ec68962c033a450943ded891bed2d54e6755e35e5835d63f4f6931d5" + + test("megabytes through a real proxy and a real host arrive byte-for-byte", async () => { + await using work = await tmpdir() + const host = proxy(Egress.DEFAULT_RULES) + try { + const egress = `${host.port}:${host.secret}` + const out = `${work.path}/numpy.whl` + // Same budget reasoning as egress-live.test.ts: curl's own timeout + // stays comfortably inside the outer 120_000ms so a genuinely slow + // download fails with a legible `curl: (28)` rather than racing the + // outer bun:test timeout and losing the diagnostic. + const script = `curl -sS -m 90 -o ${JSON.stringify(out)} -w '%{http_code} %{size_download}' ${JSON.stringify(WHEEL_URL)}` + + const { stdout, stderr } = await run(script, work.path, egress) + expect(stdout.trim(), `stderr=${stderr}`).toBe(`200 ${WHEEL_SIZE}`) + + const bytes = await Bun.file(out).arrayBuffer() + expect(bytes.byteLength).toBe(WHEEL_SIZE) + expect(new Bun.CryptoHasher("sha256").update(bytes).digest("hex")).toBe(WHEEL_SHA256) + } finally { + host.stop() + } + }, 120_000) + + // The seatbelt counterpart to egress-live.test.ts's pip test, and the + // macOS half of the merge gate: `pip install` under `network: "allowlist"` + // must work on every platform we ship, not just the one it was developed + // on. Deliberately the same package and the same flags as the Linux file, + // for the same reason the wheel test shares its URL and hash — a + // divergence between the two backends should surface as one green and one + // red, not as two different scenarios that happen to both pass. + // + // One thing this covers that the Linux file cannot: pip authenticating to + // the proxy. Seatbelt's loopback port is reachable by every process on the + // machine, so `Sandbox.plan` puts a per-start secret in the proxy URL + // (`http://os:@127.0.0.1:`) and the proxy 407s anything + // without it. curl and urllib are already covered; pip reaches the proxy + // through urllib3, whose own `Proxy-Authorization` handling on CONNECT is + // exercised here for the first time. + test("pip install reaches pypi through the authenticated proxy and the installed package imports", async () => { + await using work = await tmpdir() + const host = proxy(Egress.DEFAULT_RULES) + try { + const egress = `${host.port}:${host.secret}` + const venv = `${work.path}/venv` + const script = [ + `set -e`, + `python3 -m venv ${JSON.stringify(venv)}`, + `${JSON.stringify(`${venv}/bin/pip`)} install --only-binary :all: --disable-pip-version-check -q tqdm`, + `printf 'TQDM=%s\\n' "$(${JSON.stringify(`${venv}/bin/python`)} -c 'import tqdm; print(tqdm.__version__)')"`, + ].join("\n") + + const { stdout, stderr } = await run(script, work.path, egress) + expect(stdout.match(/^TQDM=(.+)$/m)?.[1], `stdout=${stdout} stderr=${stderr}`).toMatch(/^\d+\.\d+/) + } finally { + host.stop() + } + }, 240_000) +}) diff --git a/backend/cli/test/sandbox/egress-live.test.ts b/backend/cli/test/sandbox/egress-live.test.ts new file mode 100644 index 00000000..a8aabf67 --- /dev/null +++ b/backend/cli/test/sandbox/egress-live.test.ts @@ -0,0 +1,228 @@ +import { describe, expect, test } from "bun:test" +import fs from "fs" +import os from "os" +import path from "path" +import { Egress } from "../../src/sandbox/egress" +import { Sandbox } from "../../src/sandbox/sandbox" +import { tmpdir } from "../fixture/fixture" + +/** + * The composition nobody had committed: a real `bwrap --unshare-net`, a + * real `Egress.serveProxy` on the host, and a real remote host, wired + * together exactly the way `Sandbox.plan`/`wrapArgv` wire them in + * production. Every live test elsewhere in `test/sandbox/` stops short of + * this — `sandbox.test.ts`'s shim tests terminate at a stub `Bun.listen` + * standing in for the proxy, and `egress.test.ts`'s volume tests dial the + * proxy directly, never through a sandboxed process. Neither proves a + * sandboxed command can actually reach pypi.org, and that gap is exactly + * why the proxy shipped silently truncating every transfer above a few KB + * for four review rounds: every test that pushed a few bytes through + * passed. + * + * Two things this file asserts and nothing else does: + * - the socket is the ONLY route out (a denied host gets refused by the + * proxy, AND a direct connection with the proxy variables unset fails, + * AND DNS itself resolves nothing inside the namespace) — without that + * trio this would prove the proxy works, not that it is the only way + * out, which is the actual security claim + * - real volume survives byte-for-byte, not just "curl exited 0" + */ + +const curl = Bun.which("curl") +const python = Bun.which("python3") + +/** + * A direct, un-sandboxed HEAD against a host the shipped default allowlist + * already permits, run once at collection time. Without this, a machine + * with no route to the internet would see the PYPI/EUTILS checks below come + * back non-200 — which reads exactly like the policy defect this file + * exists to catch, when it is really just an unplugged network. Synchronous + * because bun:test needs the skip condition before any test body runs. + */ +function reachable() { + if (!curl) return false + const probe = Bun.spawnSync([ + curl, + "-sS", + "-I", + "-m", + "5", + "-o", + "/dev/null", + "-w", + "%{http_code}", + "https://pypi.org/simple/", + ]) + return probe.exitCode === 0 && probe.stdout.toString().trim() === "200" +} + +// bubblewrap, curl, getent and timeout are all load-bearing below — getent +// proves DNS resolves nothing inside the namespace, timeout bounds it in +// case that ever changes. Absent any of them, or with no network, this +// skips rather than fails: a red run here should mean the egress boundary +// broke, not that the host running the suite is a Mac or is offline. +const skip = + Sandbox.backend() !== "bubblewrap" || !curl || !Bun.which("getent") || !Bun.which("timeout") || !reachable() + +/** A real host-side allowlist proxy on a scratch unix socket — same shape + * as egress.test.ts's `proxy()`, but fed the real shipped `DEFAULT_RULES` + * rather than a synthetic rule, so pypi.org and the NCBI eutils subdomain + * are allowed and example.com is not, exactly as they are for a real user. */ +function proxy(rules: Egress.Rule[]) { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "egress-live-")) + const socket = path.join(dir, "e.sock") + const server = Egress.serveProxy({ socket, rules }) + return { + socket, + stop: () => { + server.stop(true) + fs.rmSync(dir, { recursive: true, force: true }) + }, + } +} + +/** Run `script` the way a sandboxed shell command actually runs in + * production — through `Sandbox.plan`, a real `bwrap --unshare-net`, the + * real composed shim script, and the real per-connection proxy allowlist + * check. Nothing here is stubbed. */ +async function run(script: string, work: string, socket: string) { + const spec = Sandbox.plan({ + command: script, + shell: "/bin/sh", + cwd: work, + workspace: [work], + options: { enabled: true, network: "allowlist", egress: socket }, + }) + const proc = Bun.spawn([spec.file, ...(spec.args ?? [])], { + cwd: work, + env: { ...process.env, ...spec.env }, + stdout: "pipe", + stderr: "pipe", + }) + const [stdout, stderr] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]) + await proc.exited + return { stdout, stderr } +} + +describe.skipIf(skip)("egress: a real sandbox, a real proxy, a real remote host", () => { + test("an allowlisted host reaches 200, a denied one does not, and neither a direct connection nor DNS works outside the shim", async () => { + await using work = await tmpdir() + const host = proxy(Egress.DEFAULT_RULES) + try { + const script = [ + `pypi=$(curl -sS -I -m 30 -o /dev/null -w '%{http_code}' https://pypi.org/simple/)`, + `eutils=$(curl -sS -m 30 -o /dev/null -w '%{http_code}' 'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/einfo.fcgi')`, + `example=$(curl -sS -I -m 15 -o /dev/null -w '%{http_code}' https://example.com/)`, + // The load-bearing pair. `unset` inside a subshell strips every proxy + // var — including `ALL_PROXY`, curl's protocol-agnostic fallback, + // which a host could export even with `HTTPS_PROXY` unset — for this + // one curl only; the checks above still route through the shim. So a + // 200 here would mean the socket is a convenience rather than the + // only way out, which is the entire claim this file exists to prove. + `direct=$(unset HTTP_PROXY HTTPS_PROXY http_proxy https_proxy ALL_PROXY all_proxy NO_PROXY no_proxy; curl -sS -I -m 10 -o /dev/null -w '%{http_code}' https://pypi.org/)`, + `resolved=$(timeout 10 getent hosts pypi.org)`, + `printf 'PYPI=%s\\nEUTILS=%s\\nEXAMPLE=%s\\nDIRECT=%s\\nGETENT=[%s]\\n' "$pypi" "$eutils" "$example" "$direct" "$resolved"`, + ].join("\n") + + const { stdout, stderr } = await run(script, work.path, host.socket) + const field = (name: string) => stdout.match(new RegExp(`^${name}=(.*)$`, "m"))?.[1] + const detail = `stdout=${stdout} stderr=${stderr}` + + expect(field("PYPI"), detail).toBe("200") + expect(field("EUTILS"), detail).toBe("200") + // Guarded with a shape check first: without it, a parsing regression + // that made `field()` return undefined would still satisfy + // `not.toBe("200")` and pass with nothing actually verified. + expect(field("EXAMPLE"), detail).toMatch(/^\d{3}$/) + expect(field("EXAMPLE"), detail).not.toBe("200") + expect(field("DIRECT"), detail).toMatch(/^\d{3}$/) + expect(field("DIRECT"), detail).not.toBe("200") + expect(field("GETENT"), detail).toBe("[]") + } finally { + host.stop() + } + }, 120_000) + + // Not pypi.org/simple/ itself: that index's byte length changes as + // packages are published, so only a content-addressed release file has a + // size and sha256 that stay true forever. This is numpy 1.26.4's + // manylinux cp311 wheel from files.pythonhosted.org — re-hashed directly + // against pypi.org while writing this test — chosen for size (18 MB, + // comfortably past the send-buffer boundary where the historical + // truncation bug was invisible) over the alternative of re-fetching + // pypi.org/simple/ (~45 MB) and asserting only its size, which drifts. + const WHEEL_URL = + "https://files.pythonhosted.org/packages/3a/d0/edc009c27b406c4f9cbc79274d6e46d634d139075492ad055e3d68445925/numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl" + const WHEEL_SIZE = 18_252_005 + const WHEEL_SHA256 = "666dbfb6ec68962c033a450943ded891bed2d54e6755e35e5835d63f4f6931d5" + + test("megabytes through a real proxy and a real host arrive byte-for-byte", async () => { + await using work = await tmpdir() + const host = proxy(Egress.DEFAULT_RULES) + try { + const out = path.join(work.path, "numpy.whl") + // curl's own budget stays comfortably inside the outer 120_000ms: the + // bwrap spawn and shim-readiness wait run before curl even starts, and + // `finally`'s cleanup runs after it ends. Matching the two would let a + // download that genuinely needs close to 120s race the outer bun:test + // timeout instead of curl's own — trading a diagnostic + // `curl: (28) Operation timed out` for a generic "test timed out" and + // deferring `host.stop()` until the abandoned promise chain resolves. + const script = `curl -sS -m 90 -o ${JSON.stringify(out)} -w '%{http_code} %{size_download}' ${JSON.stringify(WHEEL_URL)}` + + const { stdout, stderr } = await run(script, work.path, host.socket) + expect(stdout.trim(), `stderr=${stderr}`).toBe(`200 ${WHEEL_SIZE}`) + + // The status code and curl's own byte count are what a truncation + // bug can still get right — the response frame ends early but + // cleanly. The independent check is reading the file back and + // hashing what actually landed on disk. + const bytes = await Bun.file(out).arrayBuffer() + expect(bytes.byteLength).toBe(WHEEL_SIZE) + expect(new Bun.CryptoHasher("sha256").update(bytes).digest("hex")).toBe(WHEEL_SHA256) + } finally { + host.stop() + } + }, 120_000) + + // The case this whole branch exists to enable: a real `pip install` from + // pypi, inside the sandbox, with `network: "allowlist"` as the only route + // out. The tests above prove the boundary holds; this proves the boundary + // is *usable*, which is a different claim. pip does more than one curl + // does — it issues its own index request, follows a redirect from pypi.org + // to files.pythonhosted.org (a second allowlist entry, so this also covers + // a cross-host hop through the proxy), streams a wheel, and unpacks it. + // + // `--only-binary :all:` keeps this a network test rather than a toolchain + // test: a source build failing for want of a compiler would be a red run + // that says nothing about egress. tqdm is pure Python, small, and pulls no + // dependencies, so a version printed back means the index request, the + // download and the install all crossed the proxy. + // + // The venv itself needs no network — `python3 -m venv` bootstraps pip from + // the wheel bundled in the interpreter's own `ensurepip`. That is why this + // works on a machine with no pip on PATH at all, which was one of the + // three blockers that started this work. + test.skipIf(!python)( + "pip install reaches pypi through the proxy and the installed package imports", + async () => { + await using work = await tmpdir() + const host = proxy(Egress.DEFAULT_RULES) + try { + const venv = path.join(work.path, "venv") + const script = [ + `set -e`, + `python3 -m venv ${JSON.stringify(venv)}`, + `${JSON.stringify(path.join(venv, "bin/pip"))} install --only-binary :all: --disable-pip-version-check -q tqdm`, + `printf 'TQDM=%s\\n' "$(${JSON.stringify(path.join(venv, "bin/python"))} -c 'import tqdm; print(tqdm.__version__)')"`, + ].join("\n") + + const { stdout, stderr } = await run(script, work.path, host.socket) + expect(stdout.match(/^TQDM=(.+)$/m)?.[1], `stdout=${stdout} stderr=${stderr}`).toMatch(/^\d+\.\d+/) + } finally { + host.stop() + } + }, + 240_000, + ) +}) diff --git a/backend/cli/test/sandbox/egress-runtime.test.ts b/backend/cli/test/sandbox/egress-runtime.test.ts new file mode 100644 index 00000000..74164a79 --- /dev/null +++ b/backend/cli/test/sandbox/egress-runtime.test.ts @@ -0,0 +1,488 @@ +import { afterEach, expect, test } from "bun:test" +import fs from "fs/promises" +import path from "path" +import { Config } from "../../src/config/config" +import { Global } from "../../src/global" +import { EgressRuntime } from "../../src/sandbox/egress-runtime" +import { Sandbox } from "../../src/sandbox/sandbox" + +// A global config write is process-wide and outlives any one test, so every +// test that touches sandbox config must undo it — otherwise it leaks into +// whichever test file bun happens to run next in this process. +async function cleanGlobalSandboxConfig() { + for (const name of ["openscience.jsonc", "openscience.json", "config.json"]) { + await fs.rm(path.join(Global.Path.config, name), { force: true }).catch(() => {}) + } + Config.global.reset() +} + +afterEach(async () => { + await EgressRuntime.stop() + await cleanGlobalSandboxConfig() +}) + +test("ensure is idempotent and returns a stable address", async () => { + const first = await EgressRuntime.ensure() + const second = await EgressRuntime.ensure() + expect(second.socket).toBe(first.socket) + expect(second.port).toBe(first.port) + await EgressRuntime.stop() +}) + +test("a failed start does not latch — the next call really retries", async () => { + // Making the state directory unwritable is the cheapest real way to make + // the bind fail; every other route (a port already taken, a path too long) + // is either not applicable to a unix socket or harder to arrange + // deterministically. Global.Path.state is a per-test-process tmpdir (see + // test/preload.ts), so this cannot touch a developer's real state dir — + // but it is still restored in `finally`, because leaving it read-only + // would break every later test in this process rather than just this one. + const dir = Global.Path.state + await fs.mkdir(dir, { recursive: true }) + const mode = (await fs.stat(dir)).mode & 0o7777 + + const failure = await (async () => { + try { + await fs.chmod(dir, 0o500) + // platform "linux", not the ambient one: an unwritable state directory + // only fails the bubblewrap listener, which is the one that binds a + // unix socket there. The darwin listener binds a loopback port and + // would have started happily, leaving `failure` undefined. + return await EgressRuntime.ensure("linux").then( + () => undefined, + (error) => error as Error, + ) + } finally { + await fs.chmod(dir, mode) + } + })() + + // Loud: the message has to name the thing that broke and what depends on + // it, since the caller is an unrelated-looking bash/kernel/job spawn. + expect(failure?.message).toContain("sandbox allowlist proxy") + + // Recoverable: the rejected promise must not have been cached. Caching it + // would make one transient failure permanent for the process — and under + // the "allowlist" default that is every bash command, terminal, kernel and + // compute job failing until restart. + const recovered = await EgressRuntime.ensure("linux") + // Non-null: the bubblewrap listener always carries a socket — + // `egress-runtime.ts`'s `Running` type makes the field optional only + // because the darwin branch (added by Task 7) carries a TCP endpoint + // instead. + await expect(fs.stat(recovered.socket!)).resolves.toBeDefined() +}) + +test("stop is safe after a start that failed", async () => { + const dir = Global.Path.state + await fs.mkdir(dir, { recursive: true }) + const mode = (await fs.stat(dir)).mode & 0o7777 + try { + await fs.chmod(dir, 0o500) + // platform "linux": same reason as the test above — only the bubblewrap + // listener fails on an unwritable state directory, and a start that + // succeeded would not exercise the escape hatch this test is about. + await EgressRuntime.ensure("linux").catch(() => {}) + } finally { + await fs.chmod(dir, mode) + } + // The escape hatch must not hand back the same failure it exists to clear. + await expect(EgressRuntime.stop()).resolves.toBeUndefined() +}) + +test("the socket is created under the state directory, not the workspace", async () => { + // platform "linux": there is no socket to place at all on darwin, which + // listens on a loopback port instead. + const { socket } = await EgressRuntime.ensure("linux") + // Non-null: the bubblewrap listener always carries a socket. + // Not Bun.file(socket).exists(): Bun.file() only recognizes regular files, + // and a unix socket is a distinct inode type (S_IFSOCK) — verified with an + // isolated Bun.listen({ unix }) that Bun.file(...).exists() reports false + // for it while fs.stat sees it fine. fs.stat is the correct check here. + await expect(fs.stat(socket!)).resolves.toBeDefined() + expect(socket).not.toContain(process.cwd()) + await EgressRuntime.stop() +}) + +/** Speaks the proxy's wire format directly (see egress.ts) rather than going + * through the sandbox/shim, so this stays a test of EgressRuntime's rule + * freshness and not of the loopback bridge. CONNECT is used because its + * authority is the raw request target — no URL parsing to get right — and + * because both outcomes under test (denied vs. attempted-and-unreachable) + * answer with a 403 whose body text is the only thing distinguishing them. */ +function proxyRequest(socket: string, authority: string): Promise { + return new Promise((resolve, reject) => { + const timeout = setTimeout(() => reject(new Error(`no response from the proxy for ${authority}`)), 2_000) + let body = "" + Bun.connect({ + unix: socket, + socket: { + open(client) { + client.write(`CONNECT ${authority} HTTP/1.1\r\nHost: ${authority}\r\n\r\n`) + }, + data(_client, chunk) { + body += chunk.toString() + }, + close() { + clearTimeout(timeout) + resolve(body) + }, + error(_client, error) { + clearTimeout(timeout) + reject(error) + }, + }, + }).catch(reject) + }) +} + +test("an allowlist edit reaches a running proxy without restarting it", async () => { + // Nothing listens on loopback:1 (a privileged, essentially never-bound + // port), so a request that clears the allowlist check still gets a 403 — + // "cannot reach", not "not on the allowlist". That distinction is what + // proves the check ran, without needing a real upstream. + const authority = "127.0.0.1:1" + // platform "linux": `proxyRequest` speaks to a unix socket, which only the + // bubblewrap listener has. The freshness behaviour under test is the + // proxy's, not the listener's, so pinning the transport keeps this one + // test meaningful on either kind of machine. + const first = await EgressRuntime.ensure("linux") + + // Non-null: the bubblewrap listener always carries a socket. + const before = await proxyRequest(first.socket!, authority) + expect(before).toContain("not on the sandbox allowlist") + + await Config.setSandbox({ allowHosts: ["127.0.0.1"] }) + + // The proxy was never restarted — the same server, on the same socket, + // now answers differently because it re-reads the allowlist per + // connection rather than the snapshot it was born with. Retried rather + // than asserted on the first attempt: the update reaches the running + // proxy through a reactive config-change listener, not synchronously + // with Config.setSandbox's own return. + const deadline = Date.now() + 2_000 + let after = before + while (Date.now() < deadline && after.includes("not on the sandbox allowlist")) { + after = await proxyRequest(first.socket!, authority) + } + expect(after).not.toContain("not on the sandbox allowlist") + expect(after).toContain("Cannot reach") + + const second = await EgressRuntime.ensure("linux") + expect(second.socket).toBe(first.socket) // same proxy the whole time, not a restart +}) + +/** + * `egressFor` has to answer, ahead of time, the same "would this actually be + * sandboxed with an allowlist" question that `Sandbox.plan()`/`wrapArgv()` + * answer for real via `decide()` + `buildPolicy()` — its socket becomes their + * `options.egress`. The two used to default the unset cases in opposite + * directions from each other on both fields, invisibly, because production's + * five callers always pass an already-fully-resolved policy. These pin the + * shared default (`Sandbox.resolved`) so a future edit that reintroduces a + * hand-rolled check in just one of the two places fails here instead of + * shipping. + */ +test("egressFor treats a missing enabled the same way decide() does: off", async () => { + // Old behaviour: only an explicit `enabled: false` opted out, so this + // started a real proxy nothing could ever reach — decide() never wraps a + // command whose `options.enabled` isn't literally `true`. + const egress = await EgressRuntime.egressFor({ network: "allowlist" }) + expect(egress).toBeUndefined() +}) + +test.skipIf(Sandbox.backend() !== "bubblewrap")( + "egressFor and buildPolicy() agree on what a missing network means", + async () => { + // The property, not the value. These two read the same `Options` moments + // apart — egressFor decides whether to stand a proxy up, buildPolicy decides + // whether to demand one — so a disagreement between their defaults is a + // crash: "sandbox network 'allowlist' requires an egress socket path", from + // a caller that never mentioned the network at all. + // + // It has now been wrong in both directions. It read "not allowlist" here + // while buildPolicy defaulted to allowlist; then, after this branch aligned + // the config default to main's "deny", it read allowlist here while + // buildPolicy had moved to deny. The assertion is agreement, so it holds + // whichever value the default becomes next. + const egress = await EgressRuntime.egressFor({ enabled: true }) + expect(egress).toBeUndefined() + expect(() => + Sandbox.plan({ + command: "true", + shell: "/bin/sh", + cwd: "/tmp", + workspace: ["/tmp"], + options: { enabled: true, egress }, + }), + ).not.toThrow() + }, +) + +/** + * Same wire technique as `proxyRequest` above, but dialed over TCP loopback + * rather than the unix socket — what a seatbelt-sandboxed process reaches + * directly, since seatbelt has no namespace to bind a unix socket into and + * `Egress.serveProxy` listens on that loopback port itself (decision 1 of + * the Task 7 brief: no host-side bridge). `auth`, when given, is sent as the + * `Proxy-Authorization` secret the TCP listener requires (decision 2) — + * omitted or wrong, the request must never reach the allowlist check at all. + * + * This is the one seatbelt-specific piece of Task 7 that genuinely runs, + * without a Mac: everything here is a plain `Bun.connect`/`Bun.listen` pair + * with nothing namespace- or platform-specific about it, so starting + * `EgressRuntime` with `platform: "darwin"` injected and dialing it for real + * proves the proxy → auth → allowlist path actually runs. What it cannot + * prove is whether a real `sandbox-exec` restricts a sandboxed process to + * dialing only this one port in the first place — see the Task 7 report for + * exactly what a Mac owner still needs to run. + */ +function tcpProxyRequest(port: number, authority: string, auth?: string): Promise { + return new Promise((resolve, reject) => { + const timeout = setTimeout(() => reject(new Error(`no response from the proxy for ${authority}`)), 2_000) + let body = "" + const header = auth ? `\r\nProxy-Authorization: Basic ${Buffer.from(`os:${auth}`).toString("base64")}` : "" + Bun.connect({ + hostname: "127.0.0.1", + port, + socket: { + open(client) { + client.write(`CONNECT ${authority} HTTP/1.1\r\nHost: ${authority}${header}\r\n\r\n`) + }, + data(_client, chunk) { + body += chunk.toString() + }, + close() { + clearTimeout(timeout) + resolve(body) + }, + error(_client, error) { + clearTimeout(timeout) + reject(error) + }, + }, + }).catch(reject) + }) +} + +test("ensure with platform darwin listens on a loopback TCP port, not a unix socket", async () => { + const running = await EgressRuntime.ensure("darwin") + expect(running.hostname).toBe("127.0.0.1") + // Ephemeral, not the bwrap shim's fixed SHIM_PORT: seatbelt has no + // namespace to keep a fixed port private across concurrently sandboxed + // processes the way --unshare-net does for bubblewrap. + expect(running.port).not.toBe(Sandbox.SHIM_PORT) + expect(running.port).toBeGreaterThan(0) + // A secret was generated for this start — required to reach the proxy at + // all, since a loopback port (unlike a unix socket) carries no filesystem + // permissions of its own. + expect(running.secret).toBeTruthy() + // And no unix socket on this path at all — decision 1 of the Task 7 + // brief: serveProxy listens on TCP directly, no host-side bridge to one. + expect(running.socket).toBeUndefined() +}) + +test("the darwin proxy forwards a correctly-authenticated request past both auth and the allowlist check, live", async () => { + // 127.0.0.1 is not in Egress.DEFAULT_RULES, so it has to be added + // explicitly here — otherwise a request to 127.0.0.1:1 is denied at the + // allowlist check before the auth → dial chain this test exists to prove + // is ever reached at all. (Task 7 fix round 1, I2: the unfixed version of + // this test asserted "not on the sandbox allowlist" — the denial text — + // while its own comment claimed the opposite outcome. Both the comment + // and the assertion described a dial that never actually happened; + // measured by the reviewer, confirmed here by fixing it forward instead + // of just correcting the prose.) + await Config.setSandbox({ allowHosts: ["127.0.0.1"] }) + const running = await EgressRuntime.ensure("darwin") + // Nothing listens on loopback:1 (a privileged, essentially never-bound + // port), so a request that clears BOTH the auth check and the allowlist + // check still gets a 403 "Cannot reach" — not "not on the sandbox + // allowlist" and not 407 — which is what proves the whole + // auth → allowlist → dial chain actually ran end to end. + const body = await tcpProxyRequest(running.port, "127.0.0.1:1", running.secret) + expect(body).toContain("Cannot reach") + expect(body).not.toContain("not on the sandbox allowlist") + expect(body).not.toContain("407") +}) + +test("the darwin proxy refuses a request with no Proxy-Authorization, and never forwards it", async () => { + const running = await EgressRuntime.ensure("darwin") + const body = await tcpProxyRequest(running.port, "127.0.0.1:1") + expect(body).toContain("407 Proxy Authentication Required") + // Neither downstream outcome appears — the request was refused before the + // allowlist check or the dial ever ran. + expect(body).not.toContain("not on the sandbox allowlist") + expect(body).not.toContain("Cannot reach") +}) + +test("the darwin proxy refuses a request with the wrong secret", async () => { + const running = await EgressRuntime.ensure("darwin") + const body = await tcpProxyRequest(running.port, "127.0.0.1:1", `${running.secret}-wrong`) + expect(body).toContain("407 Proxy Authentication Required") +}) + +test('egressFor on darwin returns "port:secret", not a socket path', async () => { + const egress = await EgressRuntime.egressFor({ enabled: true, network: "allowlist" }, "darwin") + expect(egress).toBeDefined() + const [portPart, secretPart] = egress!.split(":") + expect(Number.isInteger(Number(portPart))).toBe(true) + // Shape-checked against crypto.randomUUID()'s actual format, not just + // toBeTruthy(): the string "undefined" — what a missing `secret` coerces + // to inside a template literal — is itself truthy, so a bare truthiness + // check structurally cannot catch the I1 defect the next test reproduces. + expect(secretPart).toMatch(/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/) + expect(egress).not.toContain("/") + expect(egress).not.toContain(".sock") +}) + +// Task 7 fix round 1, I1: egressFor's seatbelt branch used to interpolate +// `running.secret` with no guard. ensure()/start() cache ONE proxy for the +// process lifetime (see ensure()'s doc comment); `platform` only decides +// what starts when nothing is running yet. Asking for "darwin" after a +// bubblewrap proxy is already cached — impossible for a real caller, since +// process.platform never changes mid-process, but reachable here because +// platform is deliberately injectable for testing — used to silently reuse +// that cached listener and return the literal string "3128:undefined" +// (Buffer-safe, syntactically valid, and — per the test above — exactly +// what a bare `toBeTruthy()` on the secret half cannot distinguish from a +// real one). Confirmed by execution before the fix; asserts the fail-closed +// replacement here. +test("egressFor on darwin fails closed rather than composing an undefined secret when a differently-platformed proxy is already cached", async () => { + // Force the FIRST proxy to be the bubblewrap (unix-socket) shape, + // deterministically regardless of what machine actually runs this test — + // the same platform-injection seam every darwin test in this file uses, + // just pointed at the other platform. + await EgressRuntime.ensure("linux") + await expect(EgressRuntime.egressFor({ enabled: true, network: "allowlist" }, "darwin")).rejects.toThrow( + /already running as the bubblewrap/, + ) +}) + +test("egressFor with network deny or allow never starts a proxy on darwin", async () => { + const deny = await EgressRuntime.egressFor({ enabled: true, network: "deny" }, "darwin") + expect(deny).toBeUndefined() + const allow = await EgressRuntime.egressFor({ enabled: true, network: "allow" }, "darwin") + expect(allow).toBeUndefined() +}) + +test("egressFor on linux/bubblewrap keeps returning the unix socket path, unaffected by the darwin branch", async () => { + if (Sandbox.backend() !== "bubblewrap") return + const egress = await EgressRuntime.egressFor({ enabled: true, network: "allowlist" }) + expect(egress).toContain(".sock") +}) + +/** + * Task 7 fix round 1, I4: every auth test above hand-builds the + * `Proxy-Authorization` header itself, which leaves the seam joining + * `sandbox.ts`'s `proxyUrl()` (`http://os:@host:port`) to + * `egress.ts`'s own parser of that header unpinned in-suite — a rename of + * the userinfo user ("os") in one place only would still pass every other + * test here. These drive a real `curl`, an independent HTTP client + * implementation, at the *exact* URL `Sandbox.plan()` composes, covering + * both wire forms curl uses to talk to a proxy: an absolute-form GET (its + * default for a plain `http://` target) and a CONNECT tunnel (forced with + * `--proxytunnel`, and also what curl uses unprompted for an `https://` + * target — see `egress-live.test.ts` for that shape against a real host). + * A third, with Python's `urllib`, covers a second independent client + * library — the same proxy-auth mechanism pip itself relies on. + */ +function planProxyUrl(egress: string): string { + const plan = Sandbox.plan({ + command: "true", + shell: "/bin/sh", + cwd: "/tmp", + workspace: ["/tmp"], + options: { enabled: true, network: "allowlist", egress }, + platform: "darwin", + }) + const proxy = plan.env?.HTTP_PROXY + expect(proxy).toMatch(/^http:\/\/os:.+@127\.0\.0\.1:\d+$/) + return proxy! +} + +/** `Bun.spawn`, never `Bun.spawnSync`: the origin and the proxy both reply + * from `Bun.listen`/`Bun.serve` callbacks on this same event loop, so a + * *synchronous* spawn would block that loop for as long as the child runs + * — the child blocks on recv() waiting for a reply the loop can't yet + * deliver, deadlocking both sides until the test times out. Reproduced + * while writing these three tests (all three hung at 5s with empty stdout) + * — the same defect class `sandbox.test.ts`'s own "Bun.spawn, not + * spawnSync" comment documents for an identical reason. */ +async function runCapture(cmd: string[]) { + const proc = Bun.spawn(cmd, { stdout: "pipe", stderr: "pipe" }) + const [stdout, stderr] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]) + await proc.exited + return { stdout, stderr } +} + +test.skipIf(!Bun.which("curl"))( + "a real curl (absolute-form GET) using the exact URL Sandbox.plan() emits authenticates and is forwarded", + async () => { + const origin = Bun.serve({ port: 0, fetch: () => new Response("ok") }) + try { + await Config.setSandbox({ allowHosts: ["127.0.0.1"] }) + const running = await EgressRuntime.ensure("darwin") + const proxy = planProxyUrl(`${running.port}:${running.secret}`) + const { stdout, stderr } = await runCapture([ + "curl", + "-sS", + "-m", + "5", + "-x", + proxy, + `http://127.0.0.1:${origin.port}/`, + ]) + expect(stdout, stderr).toBe("ok") + } finally { + origin.stop(true) + } + }, +) + +test.skipIf(!Bun.which("curl"))( + "a real curl --proxytunnel (forced CONNECT) using the exact URL Sandbox.plan() emits authenticates and is forwarded", + async () => { + const origin = Bun.serve({ port: 0, fetch: () => new Response("ok") }) + try { + await Config.setSandbox({ allowHosts: ["127.0.0.1"] }) + const running = await EgressRuntime.ensure("darwin") + const proxy = planProxyUrl(`${running.port}:${running.secret}`) + const { stdout, stderr } = await runCapture([ + "curl", + "-sS", + "-m", + "5", + "--proxytunnel", + "-x", + proxy, + `http://127.0.0.1:${origin.port}/`, + ]) + expect(stdout, stderr).toBe("ok") + } finally { + origin.stop(true) + } + }, +) + +test.skipIf(!Bun.which("python3"))( + "a real Python urllib request using the exact URL Sandbox.plan() emits authenticates and is forwarded", + async () => { + const origin = Bun.serve({ port: 0, fetch: () => new Response("ok") }) + try { + await Config.setSandbox({ allowHosts: ["127.0.0.1"] }) + const running = await EgressRuntime.ensure("darwin") + const proxy = planProxyUrl(`${running.port}:${running.secret}`) + const target = `http://127.0.0.1:${origin.port}/` + const script = [ + "import urllib.request", + `handler = urllib.request.ProxyHandler({"http": ${JSON.stringify(proxy)}})`, + "opener = urllib.request.build_opener(handler)", + `print(opener.open(${JSON.stringify(target)}, timeout=5).read().decode(), end="")`, + ].join("\n") + const { stdout, stderr } = await runCapture(["python3", "-c", script]) + expect(stdout, stderr).toBe("ok") + } finally { + origin.stop(true) + } + }, +) diff --git a/backend/cli/test/sandbox/egress.test.ts b/backend/cli/test/sandbox/egress.test.ts new file mode 100644 index 00000000..979aaa7a --- /dev/null +++ b/backend/cli/test/sandbox/egress.test.ts @@ -0,0 +1,703 @@ +import { afterEach, expect, test } from "bun:test" +import fs from "fs" +import os from "os" +import path from "path" +import type { Socket } from "bun" +import { Egress } from "../../src/sandbox/egress" + +test("an exact rule matches only that host", () => { + expect(Egress.allowed("pypi.org", ["pypi.org"])).toBe(true) + expect(Egress.allowed("evil-pypi.org", ["pypi.org"])).toBe(false) +}) + +test("a leading dot matches the domain and its subdomains", () => { + expect(Egress.allowed("eutils.ncbi.nlm.nih.gov", [".ncbi.nlm.nih.gov"])).toBe(true) + expect(Egress.allowed("ncbi.nlm.nih.gov", [".ncbi.nlm.nih.gov"])).toBe(true) + expect(Egress.allowed("ncbi.nlm.nih.gov.evil.com", [".ncbi.nlm.nih.gov"])).toBe(false) +}) + +test("a port on the authority is ignored when matching", () => { + expect(Egress.allowed("pypi.org:443", ["pypi.org"])).toBe(true) +}) + +test("matching is case-insensitive in both directions", () => { + expect(Egress.allowed("PyPI.ORG", ["pypi.org"])).toBe(true) + expect(Egress.allowed("pypi.org", ["PyPI.ORG"])).toBe(true) +}) + +test("an empty ruleset allows nothing", () => { + expect(Egress.allowed("pypi.org", [])).toBe(false) +}) + +test("the shipped defaults cover the registries and scientific APIs the product needs", () => { + for (const host of [ + "pypi.org", + "files.pythonhosted.org", + "cran.r-project.org", + "eutils.ncbi.nlm.nih.gov", + "rest.uniprot.org", + ]) { + expect(Egress.allowed(host, Egress.DEFAULT_RULES)).toBe(true) + } +}) + +test("the shipped defaults do not permit general browsing", () => { + for (const host of ["example.com", "www.google.com", "raw.githubusercontent.com"]) { + expect(Egress.allowed(host, Egress.DEFAULT_RULES)).toBe(false) + } +}) + +// ── volume ────────────────────────────────────────────────────────────────── +// +// Everything above, and every earlier test of the bridge itself, moves a few +// bytes. That is exactly the size at which a dropped-backpressure bug is +// invisible: one small write fits the send buffer whole, so the byte count +// `Socket.write` returns equals what was asked of it and discarding that count +// costs nothing. Past a send buffer it does not — before `pump` existed, 8 MB +// through the proxy arrived as ~2.6 MB, and `pip download numpy` inside a real +// sandbox died with `SSL: RECORD_LAYER_FAILURE` while an 11 KB package +// installed fine. So these transfer real volume, in both directions, and +// compare the bytes rather than counting them. + +const VOLUME = 8 * 1024 * 1024 + +/** Not a constant fill: a repeated byte would pass even if the bridge + * duplicated or reordered a chunk, which is the other way a backpressure + * queue goes wrong. This makes position observable. */ +const sample = Buffer.from(Uint8Array.from({ length: VOLUME }, (_, i) => (i * 31 + (i >> 13)) % 251)) + +const opened: { stop: () => void }[] = [] + +afterEach(() => { + for (const it of opened.splice(0)) it.stop() +}) + +/** A raw TCP origin. Sends `sample` at whatever pace the peer accepts (so the + * test measures the bridge's backpressure, not the origin's), collects + * everything sent to it, and closes only once both halves are complete — + * which is also what makes an early `end()` on the bridge observable, since a + * close that jumps a queue truncates the tail rather than hanging. */ +type Talker = { sent: number; got: number; head: number; received: Buffer[] } + +function origin() { + const uploads: Buffer[][] = [] + const talkers = new WeakMap, Talker>() + + const push = (sock: Socket, held: Talker) => { + while (held.sent < VOLUME) { + const wrote = sock.write(sample.subarray(held.sent)) + if (wrote <= 0) return + held.sent += wrote + } + if (held.head >= 0 && held.got - held.head >= VOLUME) sock.end() + } + + const server = Bun.listen({ + hostname: "127.0.0.1", + port: 0, + socket: { + open(sock) { + const held: Talker = { sent: 0, got: 0, head: -1, received: [] } + uploads.push(held.received) + talkers.set(sock, held) + }, + data(sock, chunk) { + const held = talkers.get(sock) + if (!held) return + held.received.push(Buffer.from(chunk)) + held.got += chunk.length + // The request head is the cue to start sending, and its length is what + // makes "the whole upload arrived" a byte count rather than a guess. + if (held.head < 0) { + const end = Buffer.concat(held.received).indexOf("\r\n\r\n") + if (end < 0) return + held.head = end + 4 + } + push(sock, held) + }, + drain(sock) { + const held = talkers.get(sock) + if (held) push(sock, held) + }, + error() {}, + }, + }) + + opened.push({ stop: () => server.stop(true) }) + return { port: server.port, uploads } +} + +function proxy(rules: string[]) { + const socket = path.join(fs.mkdtempSync(path.join(os.tmpdir(), "egress-vol-")), "e.sock") + const server = Egress.serveProxy({ socket, rules }) + opened.push({ + stop: () => { + server.stop(true) + fs.rmSync(path.dirname(socket), { recursive: true, force: true }) + }, + }) + return socket +} + +function shim(socket: string) { + // Port 0 lets the OS pick, so concurrent test files cannot collide the way a + // fixed 3128 would. Inside a real sandbox the port is fixed instead, because + // --unshare-net makes collision impossible there. + const server = Egress.serveShim({ port: 0, socket }) + opened.push({ stop: () => server.stop(true) }) + return server.port +} + +/** Speak CONNECT to the proxy, upload `send`, then read until close. */ +function transfer(to: { unix: string } | { hostname: string; port: number }, authority: string, send: Buffer) { + return new Promise((resolve, reject) => { + const timeout = setTimeout(() => reject(new Error("the transfer never completed")), 60_000) + const chunks: Buffer[] = [] + const state = { established: false, at: 0 } + const upload = (sock: Socket) => { + while (state.at < send.length) { + const wrote = sock.write(send.subarray(state.at)) + if (wrote <= 0) return + state.at += wrote + } + } + const done = (result: Buffer) => { + clearTimeout(timeout) + resolve(result) + } + const handlers = { + open(sock: Socket) { + sock.write(`CONNECT ${authority} HTTP/1.1\r\nHost: ${authority}\r\n\r\n`) + }, + data(sock: Socket, chunk: Buffer) { + if (state.established) return void chunks.push(Buffer.from(chunk)) + const end = chunk.indexOf("\r\n\r\n") + if (end === -1) return + state.established = true + chunks.push(Buffer.from(chunk.subarray(end + 4))) + sock.write("GET / HTTP/1.0\r\n\r\n") + upload(sock) + }, + drain: upload, + close: () => done(Buffer.concat(chunks)), + error: () => done(Buffer.concat(chunks)), + } + // Branched rather than spread: Bun.connect is overloaded on unix vs + // hostname/port, and a union spread into one object literal matches + // neither overload. + const dial = + "unix" in to + ? Bun.connect({ unix: to.unix, socket: handlers }) + : Bun.connect({ hostname: to.hostname, port: to.port, socket: handlers }) + dial.catch(reject) + }) +} + +test("megabytes survive the proxy byte for byte, in both directions", async () => { + const upstream = origin() + const socket = proxy(["127.0.0.1"]) + + const down = await transfer({ unix: socket }, `127.0.0.1:${upstream.port}`, sample) + + expect(down.length).toBe(VOLUME) + expect(down.equals(sample)).toBe(true) + // The uploaded copy arrives behind the "GET / HTTP/1.0" that cued the + // download, so drop that prefix before comparing. + const up = Buffer.concat(upstream.uploads[0]!) + const body = up.subarray(up.indexOf("\r\n\r\n") + 4) + expect(body.length).toBe(VOLUME) + expect(body.equals(sample)).toBe(true) +}, 120_000) + +test("megabytes survive the shim and the proxy together, in both directions", async () => { + const upstream = origin() + const socket = proxy(["127.0.0.1"]) + const port = shim(socket) + + const down = await transfer({ hostname: "127.0.0.1", port }, `127.0.0.1:${upstream.port}`, sample) + + expect(down.length).toBe(VOLUME) + expect(down.equals(sample)).toBe(true) + const up = Buffer.concat(upstream.uploads[0]!) + const body = up.subarray(up.indexOf("\r\n\r\n") + 4) + expect(body.length).toBe(VOLUME) + expect(body.equals(sample)).toBe(true) +}, 120_000) + +// ── one client, one upstream ──────────────────────────────────────────────── +// +// Both bridges dial their upstream from inside an `async` handler, and Bun does +// not serialize those handlers — a second chunk, or a client's FIN, re-enters +// while the first call is parked on `await Bun.connect`. Two distinct defects +// live in that window, and neither is visible to a test that moves bytes +// through a connection that behaves politely from start to finish. + +/** Handlers for an upstream that counts connections and, crucially, whether + * each one was ever closed. Counting sockets rather than file descriptors + * keeps this honest on platforms without /proc, and measures the actual + * invariant: nothing a bridge dials may be left with no owner. Each + * connection's bytes accumulate in their own entry, so a concurrent second + * connection cannot have its bytes attributed to the first. */ +function counter() { + const seen: { text: string }[] = [] + const counts = { opened: 0, closed: 0 } + const entries = new WeakMap, { text: string }>() + const socket = { + open(sock: Socket) { + counts.opened++ + const entry = { text: "" } + seen.push(entry) + entries.set(sock, entry) + }, + data(sock: Socket, chunk: Buffer) { + const entry = entries.get(sock) + if (entry) entry.text += chunk.toString() + }, + close() { + counts.closed++ + }, + error() {}, + } + return { counts, seen, socket } +} + +function unixCounter(at: string) { + const held = counter() + const server = Bun.listen({ unix: at, socket: held.socket }) + opened.push({ stop: () => server.stop(true) }) + return held +} + +function tcpCounter() { + const held = counter() + const server = Bun.listen({ hostname: "127.0.0.1", port: 0, socket: held.socket }) + opened.push({ stop: () => server.stop(true) }) + return { ...held, port: server.port } +} + +function scratch(name: string) { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "egress-abort-")) + opened.push({ stop: () => fs.rmSync(dir, { recursive: true, force: true }) }) + return path.join(dir, name) +} + +/** Wait for the counts to stop moving rather than guessing at the race. */ +async function settle(counts: { opened: number; closed: number }) { + for (let i = 0; i < 40 && counts.closed < counts.opened; i++) await Bun.sleep(50) +} + +/** + * Repeat a batch until the race these tests exist to observe actually happens. + * + * Both abort tests need an inherently racy window: a client that leaves AFTER + * the proxy commits to dialling but BEFORE the dial completes. A fixed batch + * assumes at least one client wins it. On a loaded macOS CI runner none did, and + * the run failed as `Expected: > 0, Received: 0` — which reads like a stranded + * upstream and is the opposite: nothing was ever dialled, so there was nothing + * to strand. + * + * Repeating keeps the assertion honest rather than weakening it. `opened > 0` + * still has to hold, so the leak check can never pass trivially; only the + * dependence on how fast the machine is goes away. + */ +async function observe(counts: { opened: number; closed: number }, batch: () => Promise) { + const deadline = Date.now() + 20_000 + do { + await batch() + await settle(counts) + } while (counts.opened === 0 && Date.now() < deadline) +} + +/** connect(), then FIN in the same turn, with nothing sent. */ +function abort(port: number) { + return Bun.connect({ + hostname: "127.0.0.1", + port, + socket: { + open(sock) { + sock.end() + }, + data() {}, + close() {}, + error() {}, + }, + }) +} + +test("a client that aborts mid-dial does not strand the shim's upstream", async () => { + const socket = scratch("u.sock") + const upstream = unixCounter(socket) + const port = shim(socket) + + const N = 60 + await observe(upstream.counts, async () => { + for (let i = 0; i < N; i++) await abort(port) + }) + + // The dial really happened — otherwise this would pass trivially. + expect(upstream.counts.opened).toBeGreaterThan(0) + // And every one of them was closed. Before the shim tracked the client's + // departure, `close` ran while `toUpstream` was still undefined, so it had + // nothing to tear down: measured at 300 aborts across separate processes, + // 0.897 fd/conn stranded in the shim and the same in the host proxy, held + // for as long as the sandbox lives — hours, for a kernel or a terminal. + expect(upstream.counts.closed).toBe(upstream.counts.opened) +}, 60_000) + +test("a client that aborts mid-dial does not strand the proxy's upstream", async () => { + const socket = scratch("p.sock") + // "localhost" rather than 127.0.0.1: the dial then includes a name lookup, + // which is what holds the window open long enough to observe any of this. + // Resolved from /etc/hosts, so no network is involved. + const upstream = tcpCounter() + const server = Egress.serveProxy({ socket, rules: ["localhost"] }) + opened.push({ stop: () => server.stop(true) }) + + const N = 40 + await observe(upstream.counts, async () => { + for (let i = 0; i < N; i++) { + const client = await Bun.connect({ + unix: socket, + socket: { + open(sock) { + // A complete head, so the proxy commits to dialling, then leave. + sock.write(`CONNECT localhost:${upstream.port} HTTP/1.1\r\nHost: localhost\r\n\r\n`) + sock.end() + }, + data() {}, + close() {}, + error() {}, + }, + }) + client.end() + } + }) + + expect(upstream.counts.opened).toBeGreaterThan(0) + expect(upstream.counts.closed).toBe(upstream.counts.opened) +}, 60_000) + +test("a body that arrives after its head still produces exactly one upstream", async () => { + const pieces = ["id=1", "&id=2", "&x"] + const body = pieces.join("") + const socket = scratch("d.sock") + const upstream = tcpCounter() + const server = Egress.serveProxy({ socket, rules: ["localhost"] }) + opened.push({ stop: () => server.stop(true) }) + + // The shape NCBI E-utilities recommends for a large id list, and the shape + // `HTTP_PROXY` routes through this branch: a plain-http POST whose body + // follows the head across separate segments. Each of those segments used to + // re-enter `data`, find no link yet, re-parse the same buffered head and + // dial again — 2 upstream connections against a local origin, 4 against a + // real remote one, every one of them carrying a duplicate of a + // non-idempotent request. + // Run in parallel, and not as a nod to realism: the dial has to still be in + // flight when the next segment lands, and on loopback with a warm resolver a + // single dial finishes inside the 1ms gap. Concurrency is what holds the + // window open — enough simultaneous name lookups to queue behind the + // resolver — and it is the only trigger measured here that survives a warm + // cache. Against the unfixed proxy this produced 200 upstream connections + // for these 100 clients, three runs out of three; at 25 clients it was one + // run in three, and sequentially it needed a cold cache to reproduce at all. + const target = `localhost:${upstream.port}` + const clients = 100 + const socks = await Promise.all( + Array.from({ length: clients }, async () => { + const client = await Bun.connect({ + unix: socket, + socket: { data() {}, close() {}, error() {} }, + }) + client.write(`POST http://${target}/eutils HTTP/1.1\r\nHost: ${target}\r\nContent-Length: ${body.length}\r\n\r\n`) + for (const [i, gap] of [1, 5, 10].entries()) { + await Bun.sleep(gap) + client.write(pieces[i]!) + } + return client + }), + ) + await Bun.sleep(1_000) + + expect(upstream.counts.opened).toBe(clients) + // And each one carries a whole request, rather than the head being + // duplicated onto a second connection with the body split between them. + expect(upstream.seen.length).toBe(clients) + for (const entry of upstream.seen) { + expect(entry.text).toContain("POST /eutils") + expect(entry.text).toContain(body) + } + for (const s of socks) s.end() +}, 60_000) + +// ── what one client can make the host allocate ────────────────────────────── +// +// serveProxy runs on the HOST, outside the sandbox, and holds memory on behalf +// of a process the sandbox exists to contain — in the CLI's own process, so an +// OOM there is the supervisor dying, not a worker. Two sub-phases buffer +// without a link to push into, and neither was bounded: +// +// head never terminated by CRLFCRLF, so no dial is ever attempted and +// nothing downstream limits it. Measured: 93 MiB of head took the +// host process from 36.0 MB to 1344.9 MB of RSS in 8 s, climbing. +// dialing a complete head for an allowlisted host that black-holes SYNs. +// Measured: 2048.6 MiB blasted in 8 s took it from 36.0 MB to +// 2120.2 MB and then killed it with `RangeError: Out of memory`, +// with the dial still in flight and ~2 minutes of SYN retries left. +// +// Both assertions below are on bytes the proxy was willing to *accept*, which +// is what the growth was made of, and both are orders of magnitude clear of +// the fixed bounds so neither is timing-sensitive in the passing direction. + +/** Blast at a target until it stops accepting or `ms` elapses, and report how + * much it took. `stalled` distinguishes "the proxy stopped reading" — the + * backpressure this is looking for — from "we ran out of time". */ +function flood(to: { unix: string }, head: string, ms: number) { + return new Promise<{ sent: number; response: string; stalled: boolean }>((resolve, reject) => { + const CHUNK = Buffer.alloc(1 << 20, 0x41) // 'A' — cannot contain CRLFCRLF + const state = { sent: 0, response: "", done: false, stalled: false } + const finish = () => { + if (state.done) return + state.done = true + clearTimeout(timer) + resolve({ sent: state.sent, response: state.response, stalled: state.stalled }) + } + const timer = setTimeout(finish, ms) + const push = (sock: Socket) => { + state.stalled = false + while (!state.done) { + const wrote = sock.write(CHUNK) + if (wrote <= 0) return void (state.stalled = true) + state.sent += wrote + } + } + Bun.connect({ + unix: to.unix, + socket: { + open(sock) { + if (head) sock.write(head) + push(sock) + }, + drain: push, + data(_sock, chunk) { + state.response += chunk.toString("latin1") + }, + close: finish, + error: finish, + }, + }).catch(reject) + }) +} + +test("a head that never ends is refused rather than buffered", async () => { + const socket = proxy(["127.0.0.1"]) + + // No CRLFCRLF anywhere in the payload, so the parse never completes and the + // proxy never dials — this phase is not bounded by a connect() at all. + const flooded = await flood({ unix: socket }, "", 4_000) + + // Fail closed, and say why: 431 is the status RFC 6585 defines for exactly + // this. Against the unbounded version the client got no response whatsoever, + // because there is nothing in that code path that ever answers. + expect(flooded.response).toContain("431 Request Header Fields Too Large") + expect(flooded.response).toContain("Proxy request head exceeded") + + // And it was cut off early. The cap is 64 KiB; the client stops at whatever + // was already in flight when the proxy closed, which is a socket buffer or + // two. 16 MiB is a bound no correct implementation approaches and the + // unbounded one blows through in well under a second — it took 93 MiB in 8 s + // while growing the host by 1.3 GB. + expect(flooded.sent).toBeLessThan(16 * 1024 * 1024) +}, 60_000) + +test("a client cannot flood the host while its dial is in flight", async () => { + // 192.0.2.1 is TEST-NET-1 (RFC 5737): reserved for documentation, routed + // nowhere, so a SYN to it is dropped rather than refused and the dial stays + // in flight for the kernel's whole retry budget — ~130 s on Linux. That is + // the window under test, and allowlisting it is what gets the proxy to + // commit to dialling. Confirmed black-holed here by a bare connect that hung + // past 20 s with no RST and no ICMP. + const socket = proxy(["192.0.2.1"]) + const head = "CONNECT 192.0.2.1:443 HTTP/1.1\r\nHost: 192.0.2.1:443\r\n\r\n" + + const flooded = await flood({ unix: socket }, head, 4_000) + + // Precondition, asserted rather than assumed: the dial has to still be + // outstanding for this to be measuring anything. Any response at all — a 403 + // "Cannot reach", a 504 — means the environment answered TEST-NET-1 quickly + // and the window never opened, so the test would otherwise pass vacuously. + expect(flooded.response).toBe("") + + // The invariant: the proxy stopped reading, so the client cannot even + // generate the bytes — they stay in its socket buffer and then in it. This + // is backpressure rather than a cap, so there is no limit to tune; the bound + // asserted here is just the socket buffers on either side of the pause. + // Against the unbounded version this reached ~1 GiB inside 4 s. + expect(flooded.sent).toBeLessThan(16 * 1024 * 1024) + expect(flooded.stalled).toBe(true) +}, 60_000) + +test("a dial that never completes gives up and says so", async () => { + // Same black hole, but now waiting for the timeout rather than racing it. + // 150 ms stands in for the shipped 30 s so this costs the suite no real + // time; what it exercises is that the timer fires, answers, and closes, + // instead of the connection hanging for the kernel's ~130 s SYN budget. + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "egress-timeout-")) + const socket = path.join(dir, "e.sock") + const server = Egress.serveProxy({ socket, rules: ["192.0.2.1"], dialTimeout: 150 }) + opened.push({ + stop: () => { + server.stop(true) + fs.rmSync(dir, { recursive: true, force: true }) + }, + }) + + const started = Date.now() + const answer = await new Promise((resolve, reject) => { + const fail = setTimeout(() => reject(new Error("the proxy never gave up on the dial")), 20_000) + const body = { text: "" } + Bun.connect({ + unix: socket, + socket: { + open(sock) { + sock.write("CONNECT 192.0.2.1:443 HTTP/1.1\r\nHost: 192.0.2.1:443\r\n\r\n") + }, + data(_sock, chunk) { + body.text += chunk.toString("latin1") + }, + close() { + clearTimeout(fail) + resolve(body.text) + }, + error() { + clearTimeout(fail) + resolve(body.text) + }, + }, + }).catch(reject) + }) + + expect(answer).toContain("504 Gateway Timeout") + expect(answer).toContain("192.0.2.1:443") + // Bounded by the budget, not by the kernel. Generous upper bound so a loaded + // machine cannot flake it, but far below the ~130 s this used to take. + expect(Date.now() - started).toBeLessThan(10_000) +}, 60_000) + +// ── seatbelt: TCP loopback + Proxy-Authorization ──────────────────────────── +// +// Everything above dials a unix socket. macOS has no network namespace to +// bind one into, so serveProxy listens directly on a loopback TCP port +// instead (Task 7 brief, decision 1 — no host-side bridge between the two; +// see the module doc comment and sandbox.ts's seatbeltProfile). A loopback +// TCP port, unlike a unix socket, carries no filesystem permissions of its +// own — every process on the machine can dial it — so that listener +// additionally requires a `Proxy-Authorization` secret (decision 2). These +// test the listener and that requirement directly, with no EgressRuntime or +// sandbox in between; `test/sandbox/egress-runtime.test.ts` covers the same +// property one layer up, through the lifecycle that actually generates and +// threads the secret in production. + +function tcpProxy(rules: string[], secret: string) { + const server = Egress.serveProxy({ hostname: "127.0.0.1", port: 0, secret, rules }) + opened.push({ stop: () => server.stop(true) }) + return server +} + +/** Speaks CONNECT directly over TCP loopback, optionally with a + * Proxy-Authorization header — the wire shape pip/curl/requests produce + * from a `http://os:@host:port` proxy URL. */ +function tcpRequest(port: number, authority: string, auth?: string): Promise { + return new Promise((resolve, reject) => { + const timeout = setTimeout(() => reject(new Error(`no response from the proxy for ${authority}`)), 5_000) + let body = "" + const header = + auth !== undefined ? `\r\nProxy-Authorization: Basic ${Buffer.from(`os:${auth}`).toString("base64")}` : "" + Bun.connect({ + hostname: "127.0.0.1", + port, + socket: { + open(sock) { + sock.write(`CONNECT ${authority} HTTP/1.1\r\nHost: ${authority}${header}\r\n\r\n`) + }, + data(_sock, chunk) { + body += chunk.toString() + }, + close() { + clearTimeout(timeout) + resolve(body) + }, + error(_sock, error) { + clearTimeout(timeout) + reject(error) + }, + }, + }).catch(reject) + }) +} + +test("serveProxy on TCP binds 127.0.0.1, never 0.0.0.0", () => { + const server = tcpProxy([], "s") + expect(server.hostname).toBe("127.0.0.1") +}) + +test("a correctly-authenticated TCP request passes auth and reaches the dial", async () => { + const port = tcpProxy(["127.0.0.1"], "right-secret").port + // 127.0.0.1:1 is allowlisted but nothing listens there (a privileged, + // essentially never-bound port) — "Cannot reach" (not 407, not "not on + // the sandbox allowlist") is what proves the secret was accepted and the + // request reached the dial, the same distinction `proxyRequest`-based + // tests elsewhere in this suite use for the unix-socket listener. + const body = await tcpRequest(port, "127.0.0.1:1", "right-secret") + expect(body).toContain("Cannot reach") +}) + +test("a TCP request with no Proxy-Authorization is refused with 407 and never reaches the dial or the allowlist check", async () => { + const port = tcpProxy(["127.0.0.1"], "right-secret").port + const body = await tcpRequest(port, "127.0.0.1:1") + expect(body).toContain("407 Proxy Authentication Required") + expect(body).not.toContain("sandbox allowlist") + expect(body).not.toContain("Cannot reach") +}) + +test("a TCP request with the wrong secret is refused with 407", async () => { + const port = tcpProxy(["127.0.0.1"], "right-secret").port + const body = await tcpRequest(port, "127.0.0.1:1", "wrong-secret") + expect(body).toContain("407 Proxy Authentication Required") +}) + +test("the unix-socket listener requires no Proxy-Authorization — auth is TCP-only", async () => { + // Regression guard for the other direction: adding auth to the TCP branch + // must not leak onto bubblewrap's unix socket, which has no `secret` to + // check in the first place — filesystem permissions on the path are its + // access control (unchanged Linux behaviour this task must not regress). + const socket = proxy(["127.0.0.1"]) + const body = await proxyRequestNoAuth(socket, "127.0.0.1:1") + expect(body).not.toContain("407") + expect(body).toContain("Cannot reach") +}) + +function proxyRequestNoAuth(socket: string, authority: string): Promise { + return new Promise((resolve, reject) => { + const timeout = setTimeout(() => reject(new Error(`no response for ${authority}`)), 5_000) + let body = "" + Bun.connect({ + unix: socket, + socket: { + open(sock) { + sock.write(`CONNECT ${authority} HTTP/1.1\r\nHost: ${authority}\r\n\r\n`) + }, + data(_sock, chunk) { + body += chunk.toString() + }, + close() { + clearTimeout(timeout) + resolve(body) + }, + error(_sock, error) { + clearTimeout(timeout) + reject(error) + }, + }, + }).catch(reject) + }) +} diff --git a/backend/cli/test/sandbox/fastpath.test.ts b/backend/cli/test/sandbox/fastpath.test.ts new file mode 100644 index 00000000..cc668f91 --- /dev/null +++ b/backend/cli/test/sandbox/fastpath.test.ts @@ -0,0 +1,82 @@ +import { expect, test } from "bun:test" +import fs from "fs/promises" +import os from "os" +import path from "path" + +/** + * The sandbox re-entry points must be answered before the process builds + * anything of its own. + * + * This is not a style preference. The egress shim IS this binary, re-entered + * inside the sandbox, where the user's data, config, state, log and bin + * directories are not reachable — and `src/global/index.ts` creates all five in + * a top-level await. ESM evaluates every static import before the importing + * module's first statement, so the argv checks sitting at the top of + * `src/index.ts` ran far too late to help: + * + * EEXIST: file already exists, mkdir 'C:\Users\\.local\state\openscience' + * at async (src/global/index.ts:105:15) + * at async (src/server/server.ts:43:1) + * + * The shim died during module evaluation, so the proxy it was supposed to serve + * was a dead port and every network call inside the sandbox failed to resolve. + * + * The test makes the directories genuinely unwritable and asserts the entry + * point still answers, which is the property; asserting import ORDER in the + * source would pass just as happily on a graph that had grown a new heavyweight + * import somewhere below. + */ + +const root = process.getuid?.() === 0 + +test.if(!root)( + "a sandbox re-entry point answers without building the user's directories", + async () => { + const dir = await fs.mkdtemp(path.join(os.tmpdir(), "openscience-fastpath-")) + const ro = path.join(dir, "ro") + await fs.mkdir(ro) + await fs.chmod(ro, 0o500) + try { + const entry = new URL("../../src/index.ts", import.meta.url).pathname + const proc = Bun.spawn([process.execPath, entry, "__appcontainer-launch"], { + // Every directory the boot path would create now lives somewhere it + // cannot be created. If anything below the entry check runs, this exits + // on EACCES instead of on the usage message. + env: { + PATH: process.env["PATH"] ?? "", + HOME: ro, + OPENSCIENCE_TEST_HOME: ro, + XDG_DATA_HOME: path.join(ro, "data"), + XDG_CACHE_HOME: path.join(ro, "cache"), + XDG_CONFIG_HOME: path.join(ro, "config"), + XDG_STATE_HOME: path.join(ro, "state"), + }, + stdout: "pipe", + stderr: "pipe", + }) + const err = await new Response(proc.stderr).text() + await proc.exited + expect(err).toContain("__appcontainer-launch requires") + expect(err).not.toContain("EACCES") + expect(err).not.toContain("mkdir") + expect(proc.exitCode).toBe(2) + } finally { + await fs.chmod(ro, 0o700).catch(() => {}) + await fs.rm(dir, { recursive: true, force: true }).catch(() => {}) + } + }, + 60_000, +) + +test("the re-entry points live in one module, not scattered through the CLI entry", async () => { + // `src/index.ts` held them behind a comment asserting they ran first. A + // comment cannot order an import graph, so the guarantee now comes from the + // module being imported before anything heavy — and from the test above. + const index = await Bun.file(new URL("../../src/index.ts", import.meta.url).pathname).text() + expect(index).not.toContain('process.argv[2] === "__egress-shim"') + expect(index).not.toContain('process.argv[2] === "__appcontainer-launch"') + const before = index.slice(0, index.indexOf('import "./sandbox/fastpath"')) + // Only the synced-env preload, which is synchronous, fully guarded, and has + // to stay first for provider SDKs to see their keys at construction. + expect(before.match(/^import .*/gm)).toEqual(['import "./openscience/preload-env"']) +}) diff --git a/backend/cli/test/sandbox/sandbox.test.ts b/backend/cli/test/sandbox/sandbox.test.ts index 898b2541..1b20233c 100644 --- a/backend/cli/test/sandbox/sandbox.test.ts +++ b/backend/cli/test/sandbox/sandbox.test.ts @@ -1,9 +1,12 @@ import { describe, expect, test } from "bun:test" import fs from "fs" +import { builtinModules } from "module" import os from "os" import path from "path" import { ProcessIdentity } from "../../src/process/process-identity" import { Sandbox } from "../../src/sandbox/sandbox" +import { Shell } from "../../src/shell/shell" +import { SHIM_READY_MARKER } from "../../src/sandbox/egress-shim-marker" import { tmpdir } from "../fixture/fixture" const shell = "/bin/sh" @@ -28,7 +31,7 @@ async function executeWithoutCleanup(plan: Sandbox.Plan, cwd: string) { describe("Sandbox.seatbeltProfile", () => { test("denies writes by default and re-allows the workspace", () => { - const profile = Sandbox.seatbeltProfile({ writable: ["/work/project"], network: true }) + const profile = Sandbox.seatbeltProfile({ writable: ["/work/project"], network: "allow" }) expect(profile).toContain("(version 1)") expect(profile).toContain("(deny default)") expect(profile).toContain('(import "system.sb")') @@ -37,24 +40,24 @@ describe("Sandbox.seatbeltProfile", () => { }) test("both policy modes deny all sockets because SBPL cannot filter private CIDR ranges", () => { - expect(Sandbox.seatbeltProfile({ writable: ["/w"], network: false })).not.toContain("(allow network*)") - expect(Sandbox.seatbeltProfile({ writable: ["/w"], network: true })).not.toContain("(allow network*)") - expect(Sandbox.seatbeltProfile({ writable: ["/w"], network: true })).not.toContain("(system-network)") + expect(Sandbox.seatbeltProfile({ writable: ["/w"], network: "deny" })).not.toContain("(allow network*)") + expect(Sandbox.seatbeltProfile({ writable: ["/w"], network: "allow" })).not.toContain("(allow network*)") + expect(Sandbox.seatbeltProfile({ writable: ["/w"], network: "allow" })).not.toContain("(system-network)") }) test("a path outside the allowlist is not granted write access", () => { - const profile = Sandbox.seatbeltProfile({ writable: ["/work/project"], network: true }) + const profile = Sandbox.seatbeltProfile({ writable: ["/work/project"], network: "allow" }) expect(profile).not.toContain('(subpath "/etc/passwd")') expect(profile).not.toContain(process.env.HOME + "/.ssh") }) test("adds the macOS /private firmlink alias for /tmp", () => { - const profile = Sandbox.seatbeltProfile({ writable: ["/tmp"], network: true }) + const profile = Sandbox.seatbeltProfile({ writable: ["/tmp"], network: "allow" }) expect(profile).toContain(`(subpath "${fs.realpathSync.native("/tmp")}")`) }) test("escapes quotes in paths so the profile cannot be broken out of", () => { - const profile = Sandbox.seatbeltProfile({ writable: ['/weird/pa"th'], network: true }) + const profile = Sandbox.seatbeltProfile({ writable: ['/weird/pa"th'], network: "allow" }) expect(profile).toContain('/weird/pa\\"th') }) @@ -62,7 +65,7 @@ describe("Sandbox.seatbeltProfile", () => { const profile = Sandbox.seatbeltProfile({ writable: ["/work/project"], unreadable: ["/home/user/.config/atlas-cli/config.json"], - network: true, + network: "allow", }) expect(profile).toContain( `(deny file-read* (literal "${fs.realpathSync.native("/home")}/user/.config/atlas-cli/config.json"))`, @@ -77,7 +80,7 @@ describe("Sandbox.seatbeltProfile", () => { writable: ["/work/project"], readable: ["/work/project/packages/server"], readableExact: ["/work/project/packages", "/work/project"], - network: false, + network: "deny", }) expect(profile).toContain('(literal "/work/project/packages")') expect(profile).toContain('(literal "/work/project")') @@ -90,7 +93,7 @@ describe("Sandbox.bubblewrapArgs", () => { const args = Sandbox.bubblewrapArgs({ writable: ["/work/project"], readable: ["/work/reference"], - network: true, + network: "allow", }) const hostRoot = args.findIndex( (value, index) => value === "--ro-bind" && args[index + 1] === "/" && args[index + 2] === "/", @@ -119,7 +122,7 @@ describe("Sandbox.bubblewrapArgs", () => { const args = Sandbox.bubblewrapArgs({ writable: [source], writableAliases: [{ source, destination: path.join(destination, "nested", "..") }], - network: false, + network: "deny", }) const alias = args.findIndex( (value, index) => @@ -139,7 +142,7 @@ describe("Sandbox.bubblewrapArgs", () => { const args = Sandbox.bubblewrapArgs({ writable: [path.join(root, "workspace")], readableAliases: [{ source: alias, destination: alias }], - network: false, + network: "deny", }) expect( args.some((value, index) => value === "--ro-bind-try" && args[index + 1] === "/" && args[index + 2] === alias), @@ -160,7 +163,7 @@ describe("Sandbox.bubblewrapArgs", () => { writable: [path.join(root, "workspace")], unreadable: [source], unreadableAliases: [{ source, destination }], - network: false, + network: "deny", }) const masks = args.flatMap((value, index) => value === "--ro-bind-try" && args[index + 1] === "/dev/null" ? [args[index + 2]!] : [], @@ -173,12 +176,12 @@ describe("Sandbox.bubblewrapArgs", () => { }) test("fails closed to an isolated network namespace in both policy modes", () => { - expect(Sandbox.bubblewrapArgs({ writable: ["/w"], network: false })).toContain("--unshare-net") - expect(Sandbox.bubblewrapArgs({ writable: ["/w"], network: true })).toContain("--unshare-net") + expect(Sandbox.bubblewrapArgs({ writable: ["/w"], network: "deny" })).toContain("--unshare-net") + expect(Sandbox.bubblewrapArgs({ writable: ["/w"], network: "allow" })).toContain("--unshare-net") }) test("skips the /tmp tmpfs root but binds workspace paths under it", () => { - const args = Sandbox.bubblewrapArgs({ writable: ["/tmp", "/tmp/sub"], network: true }) + const args = Sandbox.bubblewrapArgs({ writable: ["/tmp", "/tmp/sub"], network: "allow" }) expect(args).toContain("--tmpfs") const binds = args.flatMap((a, n) => (a === "--bind-try" ? [args[n + 1]!] : [])) // the /tmp mount root itself is never bound from the host (the tmpfs provides it) @@ -190,11 +193,11 @@ describe("Sandbox.bubblewrapArgs", () => { }) test("unshares the PID namespace so /proc escape vectors are closed", () => { - expect(Sandbox.bubblewrapArgs({ writable: ["/w"], network: true })).toContain("--unshare-pid") + expect(Sandbox.bubblewrapArgs({ writable: ["/w"], network: "allow" })).toContain("--unshare-pid") }) test("does not implicitly expose Linux user-data roots", () => { - const args = Sandbox.bubblewrapArgs({ writable: ["/work/project"], network: false }) + const args = Sandbox.bubblewrapArgs({ writable: ["/work/project"], network: "deny" }) const sources = args.flatMap((value, index) => value === "--ro-bind" || value === "--ro-bind-try" || value === "--bind" || value === "--bind-try" ? [args[index + 1]!] @@ -210,7 +213,7 @@ describe("Sandbox.bubblewrapArgs", () => { const args = Sandbox.bubblewrapArgs({ writable: ["/work/project"], readable: ["/home/user/.bun"], - network: false, + network: "deny", }) const rootRemount = args.findIndex((value, index) => value === "--remount-ro" && args[index + 1] === "/") const filesystemOptions = new Set([ @@ -238,7 +241,7 @@ describe("Sandbox.bubblewrapArgs", () => { const inside = path.join(workspace, "inside") fs.rmSync(outside, { force: true }) try { - const args = Sandbox.bubblewrapArgs({ writable: [workspace], readable: [readable], network: false }) + const args = Sandbox.bubblewrapArgs({ writable: [workspace], readable: [readable], network: "deny" }) const script = [ `touch ${JSON.stringify(outside)} 2>/dev/null`, "outside_status=$?", @@ -270,7 +273,7 @@ describe("Sandbox.bubblewrapArgs", () => { const args = Sandbox.bubblewrapArgs({ writable: ["/work/project"], unreadable: [file], - network: true, + network: "allow", }) const mask = args.findIndex((value, index) => value === "--ro-bind-try" && args[index + 1] === "/dev/null") expect(args.slice(mask, mask + 3)).toEqual(["--ro-bind-try", "/dev/null", fs.realpathSync.native(file)]) @@ -285,7 +288,7 @@ describe("Sandbox.bubblewrapArgs", () => { const args = Sandbox.bubblewrapArgs({ writable: ["/work/project"], unreadable: [file], - network: true, + network: "allow", }) expect(args).not.toContain(file) }) @@ -293,7 +296,7 @@ describe("Sandbox.bubblewrapArgs", () => { test("covers an existing credential directory with an empty tmpfs", () => { const directory = fs.mkdtempSync(path.join(os.tmpdir(), `openscience-sandbox-credentials-${process.pid}-`)) try { - const args = Sandbox.bubblewrapArgs({ writable: ["/work/project"], unreadable: [directory], network: true }) + const args = Sandbox.bubblewrapArgs({ writable: ["/work/project"], unreadable: [directory], network: "allow" }) const mask = args.findIndex( (value, index) => value === "--tmpfs" && args[index + 1] === fs.realpathSync.native(directory), ) @@ -315,7 +318,7 @@ describe("Sandbox.bubblewrapArgs", () => { const args = Sandbox.bubblewrapArgs({ writable: [tmp.path], unreadable: [present, missing], - network: false, + network: "deny", }) const proc = Bun.spawn(["bwrap", ...args, "--", "/bin/echo", "ok"], { stdout: "pipe", stderr: "pipe" }) const [out, error, exit] = await Promise.all([ @@ -472,7 +475,9 @@ describe("Sandbox.plan", () => { }) test("enabled → sandboxed when a backend exists, else degrades", () => { - const p = Sandbox.plan({ ...base, options: { enabled: true } }) + // network is orthogonal to what this test checks; pin it to "allow" so the + // assertions below aren't coupled to the "allowlist" default's egress requirement + const p = Sandbox.plan({ ...base, options: { enabled: true, network: "allow" } }) if (Sandbox.available()) { expect(p.sandboxed).toBe(true) expect(["sandbox-exec", "bwrap"]).toContain(p.file) @@ -519,7 +524,7 @@ describe("Sandbox.plan", () => { shell, cwd: "/work/elsewhere", workspace: ["/work/project"], - options: { enabled: true }, + options: { enabled: true, network: "allow" }, }) const argv = (p.args ?? []).join(" ") expect(argv).toContain("/work/project") @@ -535,7 +540,7 @@ describe("Sandbox.plan", () => { shell, cwd: "/work/project", workspace: ["/work/project", "/"], - options: { enabled: true, allowWrite: [os.homedir()] }, + options: { enabled: true, network: "allow", allowWrite: [os.homedir()] }, }) const argv = (p.args ?? []).join(" ") expect(argv).toContain("/work/project") @@ -838,3 +843,1159 @@ describe("Sandbox native isolation", () => { } }) }) + +describe("Sandbox network policy", () => { + test("deny unshares the network and binds no socket", () => { + const args = Sandbox.bubblewrapArgs({ writable: ["/w"], network: "deny" }) + expect(args).toContain("--unshare-net") + expect(args.join(" ")).not.toContain(".sock") + }) + + test('"allow" is severed too, because bubblewrap cannot express anything narrower', () => { + // This branch let "allow" share the host network namespace. main does not, + // and its reasoning holds: bubblewrap cannot say "the internet but never + // host loopback" without a separately configured namespace, so sharing + // would hand arbitrary agent code every service bound to 127.0.0.1 — the + // user's databases, dev servers, agents, and the egress proxy of any other + // sandbox on the machine. It fails closed instead. + // + // The consequence is real and belongs in the open list, not hidden here: + // on this backend "allow" reaches nothing, so "allowlist" is the only mode + // with a route out, and that route is the audited one. + const args = Sandbox.bubblewrapArgs({ writable: ["/w"], network: "allow" }) + expect(args).toContain("--unshare-net") + expect(args).not.toContain("/run/os/e.sock") + }) + + // The namespace must stay severed — the socket is the ONLY route out. If + // --unshare-net were dropped here the proxy would become advisory. + test("allowlist unshares the network AND binds the socket read-only", () => { + const args = Sandbox.bubblewrapArgs({ writable: ["/w"], network: "allowlist", egress: "/run/os/e.sock" }) + expect(args).toContain("--unshare-net") + const at = args.indexOf("/run/os/e.sock") + expect(at).toBeGreaterThan(0) + // --ro-bind, not --bind: the bind shares the host inode, so a read-write + // bind would let a sandboxed process `chmod 000` the socket and disable + // egress host-wide (persists past this process, shared by every + // kernel/terminal/job). Read-only blocks chmod while still permitting + // connect() — verified live in the fix-round report. + expect(args[at - 1]).toBe("--ro-bind") + }) + + test("allowlist without a socket path is refused rather than silently opened", () => { + expect(() => Sandbox.bubblewrapArgs({ writable: ["/w"], network: "allowlist" })).toThrow() + }) + + // buildPolicy filters `egress` through the same tooBroadToConfine gate as + // `writable`/`unreadable`, normalized the same way (dedupe()'s path.resolve()) + // before the gate sees it — so a lexical variant of an over-broad path (a + // trailing slash, a double slash, an unresolved "..") can't slip past the + // gate's string checks the way the raw string comparison once did. An + // over-broad egress must never reach argv as a --bind at all — even + // read-only, that would expose the whole subtree's contents, not just + // widen network access. Proven previously by calling bubblewrapArgs + // directly with an unfiltered + // `egress: $HOME`, which emitted "--bind $HOME $HOME" and let a sandboxed + // write escape to the real $HOME — and, before normalization was added, the + // exact same escape via `egress: $HOME + "/"` (a trailing slash was enough + // to dodge the raw string check). Going through the public plan() (which + // runs buildPolicy) instead: the over-broad path is dropped, so "allowlist" + // is left without an egress socket and refuses to run — it fails closed + // rather than silently binding it, for the whole class of lexical variants, + // not just the one literal string. + test.each([ + ["exact", os.homedir()], + ["trailing slash", os.homedir() + "/"], + ["double slash", os.homedir() + "//"], + ["unresolved ..", os.homedir() + "/foo/.."], + ["root", "/"], + ])("an over-broad egress path (%s) is dropped, not bound as a read-write escape hatch", (_label, egress) => { + // platform "linux", not the ambient one: this asserts bubblewrap argv, + // and on a darwin runner the same call reaches `seatbeltProfile` and + // throws for an entirely different reason ("requires an egress port"), + // which a bare .toThrow() would have accepted as a pass. Asserting the + // message closes that hole for good. + expect(() => + Sandbox.plan({ + command: "true", + shell, + cwd: "/work/project", + workspace: ["/work/project"], + options: { enabled: true, network: "allowlist", egress }, + platform: "linux", + }), + ).toThrow("requires an egress socket path") + }) + + test("a legitimate, non-broad egress socket is still bound, read-only", () => { + const p = Sandbox.plan({ + command: "true", + shell, + cwd: "/work/project", + workspace: ["/work/project"], + options: { enabled: true, network: "allowlist", egress: "/run/os/e.sock" }, + platform: "linux", + }) + expect(p.args).toContain("/run/os/e.sock") + const at = (p.args ?? []).indexOf("/run/os/e.sock") + expect((p.args ?? [])[at - 1]).toBe("--ro-bind") + }) + + // Live regression for the read-only bind's actual purpose: the egress + // socket is one-per-CLI-process, shared by every kernel/terminal/job, so a + // sandboxed process that could `chmod 000` it would disable egress + // host-wide until restart (the bind shares the host inode, so the mode + // change persists on the host — even the host proxy can no longer + // connect()). Runs a real bwrap with the exact args bubblewrapArgs() + // produces (not the full shim/proxy plan() composes, which is exercised + // elsewhere) to prove both properties of --ro-bind at once: chmod fails + // closed inside the sandbox, and a plain client can still connect() + // through the same bind. + const python = Bun.which("python3") ?? Bun.which("python") + test.skipIf(Sandbox.backend() !== "bubblewrap" || !python)( + "the sandboxed process can connect through the egress bind but cannot chmod it", + async () => { + await using tmp = await tmpdir() + const sockPath = path.join(tmp.path, "e.sock") + const received: string[] = [] + const server = Bun.listen({ + unix: sockPath, + socket: { + data(sock, chunk) { + received.push(chunk.toString()) + sock.write(`ACK:${chunk.toString()}`) + }, + }, + }) + try { + const args = Sandbox.bubblewrapArgs({ writable: [tmp.path], network: "allowlist", egress: sockPath }) + + const chmod = Bun.spawnSync({ + cmd: ["bwrap", ...args, "--", "chmod", "000", sockPath], + stdout: "pipe", + stderr: "pipe", + }) + expect(chmod.exitCode).not.toBe(0) + expect(chmod.stderr.toString()).toContain("Read-only file system") + // Mode must be unchanged on the host — the whole point of --ro-bind. + expect(fs.statSync(sockPath).mode & 0o777).toBeGreaterThan(0) + + // Bun.spawn, not spawnSync: the server above replies from a Bun.listen + // "data" callback on this same event loop, so a *synchronous* spawn + // would block that loop for as long as the child runs — the child + // blocks on recv() waiting for a reply the loop can't yet deliver, + // deadlocking both sides until the test times out (reproduced while + // writing this test). + const clientSource = [ + "import socket", + "s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)", + `s.connect(${JSON.stringify(sockPath)})`, + "s.send(b'hi')", + "print(s.recv(1024).decode(), end='')", + ].join("\n") + const connectProc = Bun.spawn({ + cmd: ["bwrap", ...args, "--", python!, "-c", clientSource], + stdout: "pipe", + stderr: "pipe", + }) + const [connectOut, connectErr] = await Promise.all([ + new Response(connectProc.stdout).text(), + new Response(connectProc.stderr).text(), + ]) + await connectProc.exited + expect(connectOut, connectErr).toBe("ACK:hi") + expect(received).toContain("hi") + } finally { + server.stop(true) + } + }, + ) + + // Seatbelt has no namespace, so bwrap's --unshare-net has no equivalent + // here: the profile text itself is the only boundary. "allowlist" is + // therefore carried by Policy.port, not Policy.egress (that field stays + // bubblewrap's unix socket path — see Policy's doc comment). + // + // Task 7 fix round 1, I3: this used to emit only network-outbound, spelled + // (remote ip ...). docs/adr/0002-sandbox-network-policy.md:56-59 records + // the reference implementation as permitting network-bind/network-inbound/ + // network-outbound, all narrowed to the proxy's loopback port, spelled tcp + // — a narrower, unmeasured guess on the one platform this project cannot + // execute against is exactly the failure mode that makes "allowlist" + // silently unreachable on every real Mac. See seatbeltProfile's own doc + // comment for why network-bind/network-inbound are included even though + // this sandboxed process is only ever a TCP client, never a listener. + test("allowlist narrows to exactly one loopback port, under a deny-by-default profile", () => { + // main rewrote the profile from `(allow default)` plus targeted denies to + // `(deny default)` plus targeted allows, so there is no `(deny network*)` + // line to order against any more — the deny is the profile's first rule and + // covers every operation that is not explicitly allowed below. + const profile = Sandbox.seatbeltProfile({ writable: ["/w"], network: "allowlist", port: 54321 }) + const deny = profile.indexOf("(deny default)") + expect(deny).toBeGreaterThan(-1) + for (const line of [ + '(allow network-bind (local tcp "localhost:54321"))', + '(allow network-inbound (local tcp "localhost:54321"))', + '(allow network-outbound (remote tcp "localhost:54321"))', + ]) { + const at = profile.indexOf(line) + expect(at).toBeGreaterThan(deny) + } + }) + + // The safety rule from the task brief: a missing/invalid port must never + // silently downgrade to a plain deny (which would look identical to a user + // asking for network:"deny", not what "allowlist" means) or, worse, to an + // unfiltered allow. Same fail-closed contract bubblewrapArgs already + // applies to a missing egress socket. + test("allowlist with no port throws rather than silently degrading to deny", () => { + expect(() => Sandbox.seatbeltProfile({ writable: ["/w"], network: "allowlist" })).toThrow( + "sandbox network 'allowlist' requires an egress port", + ) + }) + + // Task 7 fix round 1, M5: 65536 and above are not valid TCP ports at all; + // an unbounded check let them through and would have composed a profile + // narrowing egress to a port number that can never exist. + test.each([ + ["zero", 0], + ["negative", -1], + ["fractional", 3.5], + ["one past the max valid port", 65536], + ["absurdly large", 1e21], + ])("allowlist with an invalid port (%s) throws", (_label, port) => { + expect(() => Sandbox.seatbeltProfile({ writable: ["/w"], network: "allowlist", port })).toThrow( + "sandbox network 'allowlist' requires an egress port", + ) + }) + + test("allowlist accepts the maximum valid port, 65535", () => { + expect(() => Sandbox.seatbeltProfile({ writable: ["/w"], network: "allowlist", port: 65535 })).not.toThrow() + }) + + // The dangerous direction named in the brief: a malformed or over-broad + // allow is the only failure mode that makes a macOS user worse off than + // today's plain deny. Pin the exact shapes seatbeltProfile can produce — + // never a bare, unfiltered allow of any of the three network operations. + test("never emits an unfiltered network-bind/network-inbound/network-outbound allow in any mode", () => { + for (const network of ["deny", "allow"] as const) { + const profile = Sandbox.seatbeltProfile({ writable: ["/w"], network }) + expect(profile).not.toContain("(allow network-bind") + expect(profile).not.toContain("(allow network-inbound") + expect(profile).not.toContain("(allow network-outbound") + } + const profile = Sandbox.seatbeltProfile({ writable: ["/w"], network: "allowlist", port: 4000 }) + const lines = profile.split("\n").filter((line) => /network-(bind|inbound|outbound)/.test(line)) + expect(lines).toEqual([ + '(allow network-bind (local tcp "localhost:4000"))', + '(allow network-inbound (local tcp "localhost:4000"))', + '(allow network-outbound (remote tcp "localhost:4000"))', + ]) + }) + + // Pins the deny/allow branches byte-for-byte: Policy.port only ever + // affects the "allowlist" branch, so these two must come out exactly as + // they did before this field existed. + test("neither deny nor allow gets the allowlist port machinery", () => { + // The exact line-by-line profile this used to pin belonged to the old + // `(allow default)` shape. main inverted it to `(deny default)` plus + // explicit allows, so pinning the whole document again would just re-encode + // whatever main happens to emit today. The property that matters is that + // the port lines appear for "allowlist" and for nothing else. + for (const network of ["deny", "allow"] as const) { + const profile = Sandbox.seatbeltProfile({ writable: ["/w"], network }) + expect(profile).toContain("(deny default)") + expect(profile).not.toContain("network-bind") + expect(profile).not.toContain("network-inbound") + expect(profile).not.toContain("network-outbound") + } + }) + + test("darwin resolves to seatbelt regardless of the machine actually running the test", () => { + expect(Sandbox.backend("darwin")).toBe("seatbelt") + }) + + test("linux resolves to bubblewrap regardless of the machine actually running the test", () => { + expect(Sandbox.backend("linux")).toBe("bubblewrap") + }) + + test("an unsupported platform resolves to none", () => { + // win32 is no longer one — it maps to appcontainer when injected. freebsd + // has no backend and is not planned to get one, so it still exercises the + // fallthrough this test exists for. + expect(Sandbox.backend("freebsd")).toBe("none") + }) + + test("an injected win32 resolves to appcontainer, so the Windows paths are reachable", () => { + // The same seam that let the seatbelt paths be built from Linux. It is + // deliberately NOT the live probe: `detected()` still answers "none" on a + // real Windows machine until a launcher exists, because claiming a sandbox + // the product cannot apply is worse than refusing to run kernels there. + expect(Sandbox.backend("win32")).toBe("appcontainer") + }) + + // The doc comment's exact claim: an explicit platform that matches the + // real one is the same code path as the zero-arg call, not a parallel + // implementation that could drift from the probed one. + test("an explicitly-matching platform is identical to the zero-arg call", () => { + expect(Sandbox.backend(process.platform)).toBe(Sandbox.backend()) + }) +}) + +describe("Sandbox.plan/wrapArgv on darwin (seatbelt)", () => { + // ":" — the shape EgressRuntime.egressFor() produces for + // seatbelt (see egress-runtime.ts); buildPolicy splits it back into + // Policy.port/Policy.secret. + const port = "54321" + const secret = "topsecret123" + const egress = `${port}:${secret}` + + test("plan composes no shim and dials the proxy port directly, with the secret in the proxy URL", () => { + const p = Sandbox.plan({ + command: "echo hi", + shell, + cwd: "/work/project", + workspace: ["/work/project"], + options: { enabled: true, network: "allowlist", egress }, + platform: "darwin", + }) + expect(p.sandboxed).toBe(true) + expect(p.backend).toBe("seatbelt") + expect(p.file).toBe("sandbox-exec") + const argv = (p.args ?? []).join(" ") + // No unix-socket shim exists on darwin: no launcher, no bundle, no + // __egress-shim marker — the real command runs directly under sandbox-exec. + expect(argv).not.toContain("__egress-shim") + expect(argv).toContain("echo hi") + // A loopback TCP port carries no filesystem permissions of its own (a + // unix socket does), so the URL embeds the per-start secret as userinfo + // — pip/curl/requests all parse this into Proxy-Authorization. + expect(p.env?.HTTP_PROXY).toBe(`http://os:${secret}@127.0.0.1:${port}`) + expect(p.env?.http_proxy).toBe(p.env?.HTTP_PROXY) + }) + + test("wrapArgv composes no shim and dials the proxy port directly, with the secret in the proxy URL", () => { + const w = Sandbox.wrapArgv({ + file: "python3", + args: ["-u", "/tmp/k.py"], + workspace: ["/work/project"], + options: { enabled: true, network: "allowlist", egress }, + platform: "darwin", + }) + expect(w.sandboxed).toBe(true) + expect(w.backend).toBe("seatbelt") + const argv = w.args.join(" ") + expect(argv).not.toContain("__egress-shim") + expect(argv).toContain("python3") + expect(w.env?.HTTP_PROXY).toBe(`http://os:${secret}@127.0.0.1:${port}`) + }) + + test("allowlist with no egress port throws rather than silently degrading to deny", () => { + expect(() => + Sandbox.wrapArgv({ + file: "python3", + args: ["-u", "/tmp/k.py"], + workspace: ["/work/project"], + options: { enabled: true, network: "allowlist" }, + platform: "darwin", + }), + ).toThrow() + }) + + // A port with no secret is exactly as fail-closed as no port at all — see + // buildPolicy's doc comment: the two are validated and dropped together, + // so a malformed "port with no secret" pairing can never compose a proxy + // URL missing the credential the darwin listener requires. + test("allowlist with a port but no secret (malformed egress) throws, not an unauthenticated proxy URL", () => { + expect(() => + Sandbox.wrapArgv({ + file: "python3", + args: ["-u", "/tmp/k.py"], + workspace: ["/work/project"], + options: { enabled: true, network: "allowlist", egress: `${port}:` }, + platform: "darwin", + }), + ).toThrow() + }) + + test("deny and allow never compose a shim or set a proxy env on darwin", () => { + const deny = Sandbox.wrapArgv({ + file: "python3", + args: ["-u", "/tmp/k.py"], + workspace: ["/work/project"], + options: { enabled: true, network: "deny" }, + platform: "darwin", + }) + expect(deny.args.join(" ")).not.toContain("__egress-shim") + expect(deny.env).toBeUndefined() + + const allow = Sandbox.wrapArgv({ + file: "python3", + args: ["-u", "/tmp/k.py"], + workspace: ["/work/project"], + options: { enabled: true, network: "allow" }, + platform: "darwin", + }) + expect(allow.args.join(" ")).not.toContain("__egress-shim") + expect(allow.env).toBeUndefined() + }) + + // Regression guard for the real-platform paths: an explicit platform that + // matches this machine's own must not diverge from the zero-arg call — + // "allow" keeps this cheap (no shim/proxy machinery) while still routing + // through decide()/buildPolicy() with a platform argument threaded in. + test("an explicitly-matching platform reproduces the zero-arg plan on this machine", () => { + const base = { + command: "echo hi", + shell, + cwd: "/work/project", + workspace: ["/work/project"], + options: { enabled: true, network: "allow" as const }, + } + // `temporary` excluded: every plan now allocates its own private temp root, + // so two plans are never byte-identical by construction. The claim here is + // that naming this platform explicitly changes nothing else about the plan. + const injected = Sandbox.plan({ ...base, platform: process.platform }) + const ambient = Sandbox.plan(base) + try { + const strip = ({ temporary, args, ...rest }: typeof injected) => ({ + ...rest, + // The private temp appears as a bind path AND inside three env + // assignments, so substitute it wherever it occurs rather than trying + // to recognise each shape. + args: args?.map((value) => (temporary ? value.replaceAll(temporary, "") : value)), + }) + expect(strip(injected)).toEqual(strip(ambient)) + } finally { + Sandbox.cleanup(injected) + Sandbox.cleanup(ambient) + } + }) +}) + +describe("Sandbox.shimScript", () => { + test("backgrounds the shim and execs the real command", () => { + const script = Sandbox.shimScript({ + binary: "/usr/bin/openscience", + port: 3128, + socket: "/run/os/e.sock", + file: "python3", + args: ["-u", "/tmp/k.py"], + }) + expect(script).toContain("__egress-shim") + expect(script).toContain("&") + expect(script).toContain("exec ") + }) + + test("quotes every interpolated value so a path with a space cannot split", () => { + const script = Sandbox.shimScript({ + binary: "/opt/my apps/openscience", + port: 3128, + socket: "/run/my dir/e.sock", + file: "python3", + args: ["-c", "print('hi there')"], + }) + expect(script).toContain("'/opt/my apps/openscience'") + expect(script).toContain("'/run/my dir/e.sock'") + expect(script).not.toMatch(/[^']\/opt\/my apps/) + }) + + test("a single quote in an argument cannot break out of the quoting", () => { + const script = Sandbox.shimScript({ + binary: "/usr/bin/openscience", + port: 3128, + socket: "/run/os/e.sock", + file: "python3", + args: ["-c", "x = 'a'; print(x)"], + }) + expect(script).toContain(`'"'"'`) + }) +}) + +// The readiness wait is the whole per-spawn cost of network "allowlist", and +// it is paid by every sandboxed command whether or not it touches the network. +// These run the composed script through a real /bin/sh — the only place its +// behaviour actually lives — with a stand-in for the shim binary, so they need +// no bubblewrap and no proxy. +describe("Sandbox.shimScript readiness wait", () => { + const posix = process.platform !== "win32" + + /** Runs the composed script and reports how long it took, plus whatever the + * wait leaked to the real command's stderr (it runs in the foreground, so + * anything it prints lands in the command's own output). */ + async function run(script: string, prefixPath?: string) { + const started = Date.now() + const proc = Bun.spawn(["/bin/sh", "-c", script], { + stdout: "pipe", + stderr: "pipe", + env: prefixPath ? { ...process.env, PATH: `${prefixPath}:${process.env["PATH"]}` } : process.env, + }) + const [stdout, stderr] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]) + await proc.exited + return { ms: Date.now() - started, stdout, stderr } + } + + /** A `sleep` that rejects fractional intervals the way some busybox builds + * do, so the fallback branch is exercised on a host whose real coreutils + * accepts them. */ + function busyboxishSleep(dir: string) { + const real = Bun.which("sleep") ?? "/bin/sleep" + const file = path.join(dir, "sleep") + fs.writeFileSync( + file, + `#!/bin/sh\ncase "$1" in\n *.*) echo "sleep: invalid number '$1'" >&2; exit 1 ;;\nesac\nexec ${real} "$@"\n`, + { mode: 0o755 }, + ) + return dir + } + + /** A `sleep` whose cost is dominated by process creation rather than by the + * interval asked for. A macOS CI runner measured ~114ms per iteration of + * the 0.02s poll — ~94ms of fork/exec — which stretched the nominal 3s cap + * to 17.1s and is what put the wall-clock deadline in `shimScript`. This + * reproduces that condition on any host. */ + function expensiveSleep(dir: string) { + const real = Bun.which("sleep") ?? "/bin/sleep" + const file = path.join(dir, "sleep") + fs.writeFileSync(file, `#!/bin/sh\nexec ${real} 0.12\n`, { mode: 0o755 }) + return dir + } + + test.skipIf(!posix)( + "waits for the shim, and only for as long as the shim takes", + async () => { + await using dir = await tmpdir() + // Stands in for the shim: ignores its arguments, becomes ready quickly. + const fake = path.join(dir.path, "shim") + fs.writeFileSync(fake, `#!/bin/sh\nsleep 0.15\n: > ${JSON.stringify(SHIM_READY_MARKER)}\n`, { mode: 0o755 }) + fs.rmSync(SHIM_READY_MARKER, { force: true }) + + try { + const { ms, stdout } = await run( + Sandbox.shimScript({ binary: fake, port: 3128, socket: "/run/os/e.sock", file: "/bin/echo", args: ["ran"] }), + ) + expect(stdout.trim()).toBe("ran") + // It really waited: the marker only lands at ~150ms. + expect(ms).toBeGreaterThanOrEqual(140) + // And it did not round that up to a whole second. Before the poll + // interval was chosen at run time this was a flat ~1.0s for a shim that + // is ready in ~12ms — measured 1006ms against 3ms for network "deny". + expect(ms).toBeLessThan(600) + } finally { + fs.rmSync(SHIM_READY_MARKER, { force: true }) + } + }, + 30_000, + ) + + test.skipIf(!posix)( + "a sleep that rejects fractions still waits, and says nothing about it", + async () => { + await using dir = await tmpdir() + fs.rmSync(SHIM_READY_MARKER, { force: true }) + // /bin/true ignores the shim arguments and never signals readiness, so the + // wait runs to its cap — which is the point: a fractional `sleep` that + // errors out returns instantly, so a loop that ignored the failure would + // spin through all its iterations in microseconds and skip the wait + // entirely, silently, while printing one error line per iteration. + const { ms, stderr } = await run( + Sandbox.shimScript({ + binary: "/bin/true", + port: 3128, + socket: "/run/os/e.sock", + file: "/bin/echo", + args: ["ran"], + }), + busyboxishSleep(dir.path), + ) + expect(stderr).toBe("") + expect(ms).toBeGreaterThanOrEqual(2_500) + expect(ms).toBeLessThan(6_000) + }, + 30_000, + ) + + test.skipIf(!posix)( + "the cap is the same 3s whichever granularity the shell supports", + async () => { + fs.rmSync(SHIM_READY_MARKER, { force: true }) + const { ms } = await run( + Sandbox.shimScript({ + binary: "/bin/true", + port: 3128, + socket: "/run/os/e.sock", + file: "/bin/echo", + args: ["ran"], + }), + ) + expect(ms).toBeGreaterThanOrEqual(2_500) + expect(ms).toBeLessThan(6_000) + }, + 30_000, + ) + + // The regression this file's macOS run found: the cap used to be an + // iteration count, so it only equalled 3s where forking `sleep` was nearly + // free. Without the deadline, 150 iterations at 0.12s each run for 18s. + test.skipIf(!posix)( + "a `sleep` whose real cost is fork/exec cannot stretch the cap", + async () => { + await using dir = await tmpdir() + fs.rmSync(SHIM_READY_MARKER, { force: true }) + const { ms, stderr } = await run( + Sandbox.shimScript({ + binary: "/bin/true", + port: 3128, + socket: "/run/os/e.sock", + file: "/bin/echo", + args: ["ran"], + }), + expensiveSleep(dir.path), + ) + // Still silent: the deadline probe's own diagnostics are discarded the + // same way the fractional-sleep probe's are. + expect(stderr).toBe("") + expect(ms).toBeGreaterThanOrEqual(2_500) + expect(ms).toBeLessThan(6_000) + }, + 30_000, + ) +}) + +describe("Sandbox.wrapArgv egress shim", () => { + test.skipIf(Sandbox.backend() !== "bubblewrap")( + "allowlist composes the shim into the argv and returns proxy env", + () => { + const wrapped = Sandbox.wrapArgv({ + file: "python3", + args: ["-u", "/tmp/k.py"], + workspace: ["/work/project"], + options: { enabled: true, network: "allowlist", egress: "/run/os/e.sock" }, + }) + expect(wrapped.sandboxed).toBe(true) + const argv = wrapped.args.join(" ") + expect(argv).toContain("__egress-shim") + expect(argv).toContain("/run/os/e.sock") + expect(argv).toContain("exec 'python3'") + expect(wrapped.env?.HTTP_PROXY).toMatch(/^http:\/\/127\.0\.0\.1:\d+$/) + expect(wrapped.env?.http_proxy).toBe(wrapped.env?.HTTP_PROXY) + }, + ) + + test.skipIf(Sandbox.backend() !== "bubblewrap")("deny and allow never compose the shim", () => { + const deny = Sandbox.wrapArgv({ + file: "python3", + args: ["-u", "/tmp/k.py"], + workspace: ["/work/project"], + options: { enabled: true, network: "deny" }, + }) + expect(deny.args.join(" ")).not.toContain("__egress-shim") + expect(deny.env).toBeUndefined() + + const allow = Sandbox.wrapArgv({ + file: "python3", + args: ["-u", "/tmp/k.py"], + workspace: ["/work/project"], + options: { enabled: true, network: "allow" }, + }) + expect(allow.args.join(" ")).not.toContain("__egress-shim") + expect(allow.env).toBeUndefined() + }) + + // Everything above asserts on the composed argv string without ever running + // it — which is exactly what let the shim silently die inside the real + // sandbox (EROFS from the CLI's logging middleware) go undetected. This + // spawns the actual `wrapArgv` output — real bwrap, the real composed + // `sh -c` script, and (in dev, which this test runs as) the real on-disk + // launcher `shimPlan()` writes — and proves a TCP client inside the + // namespace gets a connection accepted and bridged to the unix socket, not + // a refused one. Resolved once, from the same gate the skip condition uses + // — bash lives at /bin/bash on Alpine and non-usrmerge Debian, not + // /usr/bin/bash, and a hardcoded path there would fail instead of skip. + const bash = Bun.which("bash") + test.skipIf(Sandbox.backend() !== "bubblewrap" || !bash || !Bun.which("timeout") || !Bun.which("head"))( + "the composed script actually starts the shim inside a real sandbox and bridges a connection", + async () => { + await using tmp = await tmpdir() + const socket = path.join(tmp.path, "e.sock") + const received: string[] = [] + const server = Bun.listen({ + unix: socket, + socket: { + data(sock, chunk) { + received.push(chunk.toString()) + sock.write(`ACK:${chunk.toString()}`) + }, + }, + }) + try { + const wrapped = Sandbox.wrapArgv({ + file: bash!, + args: ["-c", "exec 3<>/dev/tcp/127.0.0.1/3128; printf hello >&3; timeout 3 head -c 9 <&3"], + workspace: [tmp.path], + options: { enabled: true, network: "allowlist", egress: socket }, + }) + const proc = Bun.spawn([wrapped.file, ...wrapped.args], { stdout: "pipe", stderr: "pipe" }) + const [out, err] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]) + await proc.exited + expect(out, `stdout=${out} stderr=${err}`).toContain("ACK:hello") + expect(received).toContain("hello") + } finally { + server.stop(true) + } + }, + 15000, + ) + + // Regression guard for the bug that survived two consecutive rounds: the + // interpreter the launcher execs (`process.execPath`) can itself live + // under /tmp — a portable bun install, `$HOME` under /tmp — independent of + // where the checkout or Global.Path.bin happen to be. `shimPlan()` reads + // `process.execPath` from the *current* process, so the only way to + // actually exercise this is to run under a /tmp-staged bun. Stages a real + // copy (a symlink wouldn't reproduce — Bun resolves it) and drives a small + // standalone script through it, since `bun:test` itself isn't the thing + // under test here. + test.skipIf(Sandbox.backend() !== "bubblewrap" || !bash || !Bun.which("timeout") || !Bun.which("head"))( + "the composed script still starts the shim when the interpreter itself is staged under /tmp", + async () => { + const stage = fs.mkdtempSync(path.join(os.tmpdir(), "staged-bun-")) + const stagedBun = path.join(stage, "bun") + const driver = path.join(stage, "driver.ts") + try { + fs.copyFileSync(process.execPath, stagedBun) + fs.chmodSync(stagedBun, 0o755) + const sandboxPath = path.resolve(import.meta.dir, "..", "..", "src", "sandbox", "sandbox.ts") + fs.writeFileSync( + driver, + [ + `import path from "path"`, + `import fs from "fs"`, + `import os from "os"`, + `import { Sandbox } from ${JSON.stringify(sandboxPath)}`, + `const work = fs.mkdtempSync(path.join(os.tmpdir(), "staged-bun-driver-"))`, + `const socket = path.join(work, "e.sock")`, + `const server = Bun.listen({ unix: socket, socket: { data(sock, chunk) { sock.write("ACK:" + chunk) } } })`, + `try {`, + ` const wrapped = Sandbox.wrapArgv({`, + ` file: ${JSON.stringify(bash)},`, + ` args: ["-c", "exec 3<>/dev/tcp/127.0.0.1/3128; printf hello >&3; timeout 3 head -c 9 <&3"],`, + ` workspace: [work],`, + ` options: { enabled: true, network: "allowlist", egress: socket },`, + ` })`, + ` const proc = Bun.spawn([wrapped.file, ...wrapped.args], { stdout: "pipe", stderr: "pipe" })`, + ` const [out, err] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()])`, + ` await proc.exited`, + ` console.log(out.includes("ACK:hello") ? "STAGED_BUN_PASS" : "STAGED_BUN_FAIL " + JSON.stringify({ out, err }))`, + `} finally {`, + ` server.stop(true)`, + ` fs.rmSync(work, { recursive: true, force: true })`, + `}`, + ].join("\n"), + ) + const proc = Bun.spawn([stagedBun, "run", driver], { stdout: "pipe", stderr: "pipe" }) + const [out, err] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]) + await proc.exited + expect(out, `stdout=${out} stderr=${err}`).toContain("STAGED_BUN_PASS") + } finally { + fs.rmSync(stage, { recursive: true, force: true }) + } + }, + 20000, + ) + + // Regression guard for the fifth variant of "a path the shim needs is + // masked by --tmpfs /tmp": an npm import in the shim's graph resolves + // through a node_modules symlink whose target is the monorepo-root store, + // above the package root, so binding the package root left the target + // unbound. Reproducing that directly needs a /tmp-relocated checkout with a + // real hoisted store — a fixture too elaborate to keep honest here. These + // two tests assert the property that makes the whole class impossible + // instead: the shim resolves nothing from disk at run time. + // + // First, statically, on the artifact `shimPlan()` actually generated: a + // bundle with no import specifiers left in it cannot resolve anything, + // whether the import was a sibling file or a package. Builtins are allowed + // through — they come from inside bun, not the filesystem — so this keeps + // passing if the shim ever imports node:net, and fails if a future import + // is left external or the plan goes back to executing source. It does not + // see a native binding: `dlopen("…so")` in a bundled dependency is not an + // import specifier (see shimPlan's residual list). + // + // Classifying a specifier is the whole guard, so it is done against the + // real builtin list rather than by prefix. `bun build` *strips* the node: + // prefix — a bundled `import "node:net"` comes out as `from "net"` — so a + // prefix test flags a legitimate builtin, and the natural reaction to that + // false alarm is to loosen the one check standing between here and variant + // six. In the other direction a bare /^bun/ would quietly excuse any + // package named bun-something. builtinModules already carries bun's own + // entries (bun, bun:ffi, …), so stripping node: and asking it is both + // directions at once. + test.skipIf(Sandbox.backend() !== "bubblewrap")("the generated dev shim bundle resolves nothing from disk", () => { + const wrapped = Sandbox.wrapArgv({ + file: "python3", + args: ["-u", "/tmp/k.py"], + workspace: ["/work/project"], + options: { enabled: true, network: "allowlist", egress: "/run/os/e.sock" }, + }) + const bundle = wrapped.args.find((value) => value.endsWith(".mjs")) + expect(bundle, `argv=${wrapped.args.join(" ")}`).toBeDefined() + const source = fs.readFileSync(bundle!, "utf8") + // The three shapes an unbundled dependency can survive as: `from "x"` + // (covers `import x from` and `export … from`), a call — `require("x")`, + // `import("x")` — and a bare side-effect `import "x"`. + const found = [ + ...source.matchAll(/\bfrom\s*"([^"]+)"|\b(?:require|import)\(\s*"([^"]+)"\s*\)|\bimport\s*"([^"]+)"/g), + ] + const builtin = new Set(builtinModules) + const external = found + .map((match) => match[1] ?? match[2] ?? match[3]!) + .filter((spec) => !builtin.has(spec.replace(/^node:/, ""))) + expect(external).toEqual([]) + }) + + // Second, live: mask the shim's own source entry with /dev/null (the + // sandbox's existing `unreadable` mechanism) and run the real composed + // script anyway. If the shim still bridges, nothing it ran came from the + // source tree — which is what makes where that tree lives, and what it + // imports, irrelevant. Executing the source instead reads an empty file, + // starts no listener, and the connection is refused (verified: pointing the + // launcher back at the entry fails this test in 3s, the readiness cap). + // The mask only holds because readBind is files: a readBind *directory* + // over the entry would re-expose it and this test would pass regardless, + // which is why the static check above is the one that pins the bundle. + test.skipIf(Sandbox.backend() !== "bubblewrap" || !bash || !Bun.which("timeout") || !Bun.which("head"))( + "the composed script still starts the shim when its own source entry is masked", + async () => { + await using tmp = await tmpdir() + const socket = path.join(tmp.path, "e.sock") + const server = Bun.listen({ + unix: socket, + socket: { + data(sock, chunk) { + sock.write(`ACK:${chunk.toString()}`) + }, + }, + }) + try { + const wrapped = Sandbox.wrapArgv({ + file: bash!, + args: ["-c", "exec 3<>/dev/tcp/127.0.0.1/3128; printf hello >&3; timeout 3 head -c 9 <&3"], + workspace: [tmp.path], + unreadable: [path.resolve(import.meta.dir, "..", "..", "src", "sandbox", "egress-shim-entry.ts")], + options: { enabled: true, network: "allowlist", egress: socket }, + }) + const proc = Bun.spawn([wrapped.file, ...wrapped.args], { stdout: "pipe", stderr: "pipe" }) + const [out, err] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]) + await proc.exited + expect(out, `stdout=${out} stderr=${err}`).toContain("ACK:hello") + } finally { + server.stop(true) + } + }, + 15000, + ) + + // End-to-end guard on the property Important B was about: self-hosting — + // opening OpenScience on its own checkout, the case Task 5 will dogfood — + // must not lose write access to part of the workspace just because the + // sandbox also needs some path bound read-only. It caught a real bug when + // shimPlan's readBind held the package root and bubblewrapArgs emitted it + // after the writable --bind-try loop. That trigger is gone (readBind is now + // two generated files plus the interpreter, none inside a checkout), so the + // test no longer has a failing negative control — it passes by the bind set + // being small rather than by the overlap exclusion doing anything. Kept as + // the assertion that the property still holds however the set changes. + test.skipIf(Sandbox.backend() !== "bubblewrap" || !bash)( + "a workspace path under the package root stays writable under allowlist", + async () => { + await using tmp = await tmpdir() + const socket = path.join(tmp.path, "e.sock") + const server = Bun.listen({ unix: socket, socket: { data() {} } }) + const packageRoot = path.resolve(import.meta.dir, "..", "..") + const probe = path.join(packageRoot, "src", "sandbox", `.regression-write-probe-${process.pid}`) + try { + const wrapped = Sandbox.wrapArgv({ + file: bash!, + args: ["-c", `echo probe > '${probe}' && cat '${probe}' && rm '${probe}' && echo WRITE_OK`], + workspace: [packageRoot], + options: { enabled: true, network: "allowlist", egress: socket }, + }) + const proc = Bun.spawn([wrapped.file, ...wrapped.args], { stdout: "pipe", stderr: "pipe" }) + const [out, err] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]) + await proc.exited + expect(out, `stdout=${out} stderr=${err}`).toContain("WRITE_OK") + } finally { + server.stop(true) + fs.rmSync(probe, { force: true }) + } + }, + 15000, + ) +}) + +describe("Sandbox on win32 (AppContainer composition)", () => { + const base = { + file: "python3", + args: ["-u", "/w/k.py"], + workspace: ["/w/project"], + platform: "win32" as const, + } + const decode = (args: string[]) => + JSON.parse(Buffer.from(args[args.indexOf("__appcontainer-launch") + 1]!, "base64").toString("utf8")) + + test("the profile name is stable for a workspace and distinct between workspaces", () => { + // The package SID is derived from this name, and filesystem ACEs plus the + // broker pipe's DACL refer to that SID. A fresh name per launch would + // strand every ACE the previous one granted; a shared name across projects + // would let one read another's granted paths. + expect(Sandbox.appContainerProfile(["/w/project"])).toBe(Sandbox.appContainerProfile(["/w/project"])) + expect(Sandbox.appContainerProfile(["/w/a"])).not.toBe(Sandbox.appContainerProfile(["/w/b"])) + }) + + test("the profile name is a legal AppContainer name", () => { + // Windows limits both length and character set, and a workspace path + // carries separators, drive letters and spaces that are not valid in one. + const name = Sandbox.appContainerProfile(["C:\\Users\\me\\My Project (v2)"]) + expect(name).toMatch(/^[A-Za-z0-9.-]{1,64}$/) + }) + + test("wrapArgv launches the binary as its own container launcher", () => { + // There is no wrapper executable on Windows: confinement is applied AT + // CreateProcess through SECURITY_CAPABILITIES, which cannot be expressed as + // an argv. The binary becomes the launcher, exactly as __egress-shim does. + const w = Sandbox.wrapArgv({ ...base, options: { enabled: true, network: "allow" } }) + expect(w.sandboxed).toBe(true) + expect(w.backend).toBe("appcontainer") + expect(w.file).toBe(process.execPath) + // Located, not indexed at 0. Running from source, `process.execPath` is bun + // and an entry script has to precede the flag — `bun __appcontainer-launch` + // is not a valid invocation and exits 1 silently, which the self-test reads + // as a child that produced no output. A release binary needs no entry, so + // the flag's position differs between the two modes by design. + expect(w.args).toContain("__appcontainer-launch") + }) + + test("the real argv survives at the tail, after a --", () => { + const w = Sandbox.wrapArgv({ ...base, options: { enabled: true, network: "allow" } }) + expect(w.args.slice(-3)).toEqual(["python3", "-u", "/w/k.py"]) + expect(w.args[w.args.length - 4]).toBe("--") + }) + + test("the policy travels as one base64 blob, not as flags", () => { + // Windows re-parses command lines with CommandLineToArgvW rules that differ + // from every shell, and paths there routinely carry spaces, quotes and + // backslashes. A blob with no shell-significant characters cannot be mangled. + const w = Sandbox.wrapArgv({ ...base, options: { enabled: true, network: "allow" } }) + const blob = w.args[w.args.indexOf("__appcontainer-launch") + 1]! + expect(blob).toMatch(/^[A-Za-z0-9+/=]+$/) + const spec = decode(w.args) + expect(spec.profile).toBe(Sandbox.appContainerProfile(["/w/project"])) + expect(spec.writable).toContain("/w/project") + expect(spec.network).toBe("allow") + }) + + test("allowlist mints a pipe and carries the host proxy; deny and allow carry neither", () => { + // `egress` arrives as "port:secret" — the same shape seatbelt gets, because + // the host-side proxy IS the same TCP listener on loopback. What differs is + // only how the CONTAINER reaches it, and that is the broker's pipe. + // + // The pipe NAME is minted here rather than supplied by the caller: it must + // be unguessable, since the DACL is the whole access decision and a + // predictable name is one an unrelated AppContainer could try to open. + const withPipe = Sandbox.wrapArgv({ + ...base, + options: { enabled: true, network: "allowlist", egress: "49876:s3cret" }, + }) + const spec = decode(withPipe.args) + expect(spec.pipe).toMatch(/^openscience-broker-[0-9a-f]{32}$/) + expect(spec.proxy).toEqual({ port: 49876, secret: "s3cret" }) + // Two runs never share a name. + const again = Sandbox.wrapArgv({ + ...base, + options: { enabled: true, network: "allowlist", egress: "49876:s3cret" }, + }) + expect(decode(again.args).pipe).not.toBe(spec.pipe) + + for (const network of ["deny", "allow"] as const) { + const w = Sandbox.wrapArgv({ ...base, options: { enabled: true, network } }) + expect(decode(w.args).pipe).toBeUndefined() + expect(decode(w.args).proxy).toBeUndefined() + } + }) + + test("the shim's own binary is readable, or the proxy is a dead port", () => { + // The shim IS this binary, re-entered inside the container, and an + // AppContainer executes nothing whose ACL does not name its package SID. + // Without this grant the shim never starts and the payload is handed a + // 127.0.0.1 port with nothing behind it — which presents as a network + // failure rather than as a missing grant. + const w = Sandbox.wrapArgv({ + ...base, + options: { enabled: true, network: "allowlist", egress: "49876:s3cret" }, + }) + expect(decode(w.args).readable).toContain(process.execPath) + // Only when a shim is actually needed. + const denied = Sandbox.wrapArgv({ ...base, options: { enabled: true, network: "deny" } }) + expect(decode(denied.args).readable ?? []).not.toContain(process.execPath) + }) + + test("composing without a profile throws rather than launching unconfined", () => { + // Fail closed, the same rule bubblewrapArgs applies to a missing egress + // socket: a launch with no profile has no package SID, so nothing is + // contained and every ACE and DACL downstream refers to nothing. + expect(() => Sandbox.appContainerArgs({ writable: ["/w"], network: "allow" }, ["cmd"])).toThrow("profile") + }) + + test("the real Windows backend is probed, never assumed", async () => { + // This test previously asserted the probe never said "appcontainer" at all, + // which was correct while no launcher existed. Now one does, so the + // invariant moves rather than disappears: win32 must resolve through + // AppContainer.usable(), which loads the DLLs and derives a SID, and must + // fall back to "none" when that fails. Returning "appcontainer" because the + // platform says win32 is how a product ends up claiming a sandbox it never + // applies. + const source = await Bun.file(new URL("../../src/sandbox/sandbox.ts", import.meta.url).pathname).text() + const detected = source.slice(source.indexOf("const detected = lazy"), source.indexOf("export function backend")) + expect(detected.includes("AppContainer.usable()")).toBe(true) + expect(detected.includes('AppContainer.usable() ? "appcontainer" : "none"')).toBe(true) + }) + + test("the capability probe is side-effect free and fails closed", async () => { + // It derives a SID rather than creating a profile, so a probe on a machine + // we end up not sandboxing leaves nothing behind; and every failure path + // returns false rather than throwing, because a probe that throws would + // take down callers that only wanted to know whether a backend exists. + const source = await Bun.file(new URL("../../src/sandbox/appcontainer.ts", import.meta.url).pathname).text() + const usable = source.slice( + source.indexOf("export function usable"), + source.indexOf("export function ensureProfile"), + ) + expect(usable.includes("CreateAppContainerProfile")).toBe(false) + expect(usable.includes("catch")).toBe(true) + expect(usable.includes("return false")).toBe(true) + }) +}) + +describe("Sandbox.plan on win32", () => { + // The win32 describe above only ever exercised wrapArgv, so nothing asserted + // what plan() composes — which is where the shell flag lives, and where the + // bug that made every Windows command a no-op survived. + const base = { + cwd: "C:\\work\\project", + workspace: ["C:\\work\\project"], + options: { enabled: true, network: "deny" as const }, + platform: "win32" as const, + } + + test("a cmd.exe shell is invoked with /c, never -c", () => { + const p = Sandbox.plan({ ...base, command: "whoami /groups", shell: "C:\\Windows\\system32\\cmd.exe" }) + // The launcher argv is: openscience __appcontainer-launch -- /c + const tail = p.args!.slice(p.args!.indexOf("--") + 1) + // /d /s /c, the shape Node uses: /s makes cmd take the tail verbatim after + // stripping one quote pair, /d skips AutoRun registry commands so a + // sandboxed command cannot be prefixed by machine-local configuration. + expect(tail.slice(1, 4)).toEqual(["/d", "/s", "/c"]) + expect(tail).not.toContain("-c") + expect(tail[4]).toBe("whoami /groups") + }) + + test("a Git Bash shell still gets -c", () => { + // Shell.fallback() prefers Git Bash over cmd on Windows, so both shapes are + // reachable on the same platform and the flag cannot key off the platform. + const p = Sandbox.plan({ ...base, command: "echo hi", shell: "C:\\Program Files\\Git\\bin\\bash.exe" }) + const tail = p.args!.slice(p.args!.indexOf("--") + 1) + expect(tail[1]).toBe("-c") + }) + + test("no POSIX temp path reaches the writable spec", () => { + // "/tmp" resolves to "C:\tmp" on Windows, which does not exist, so icacls + // failed on it and every sandboxed command carried a grant warning on its + // stderr — burying the real errors under it. + const p = Sandbox.plan({ ...base, command: "echo hi", shell: "cmd.exe" }) + const spec = JSON.parse( + Buffer.from(p.args![p.args!.indexOf("__appcontainer-launch") + 1]!, "base64").toString("utf8"), + ) + // On a Windows host "/tmp" resolves to "C:\tmp"; asserting the absence of + // "/tmp" itself would only pass here by accident, because this Linux box's + // own os.tmpdir() IS "/tmp" and legitimately belongs in the list. + expect(spec.writable.some((w: string) => /^[A-Za-z]:\\tmp$/i.test(w))).toBe(false) + }) +}) + +// `tempDirs` is gone. It granted the sandbox the SHARED host temp, which let +// mutually untrusted sessions read and overwrite each other's scratch files, and +// on Windows it also labelled the user's whole %TEMP% Low integrity for the +// duration of every run. main replaced it with `privateTemp()`: one 0700 +// directory per spawn, carried on the Plan and reclaimed by the caller. The +// property that mattered here — no POSIX literals on win32 — is now structural, +// because no shared temp is granted on any platform. + +test("the one thing Windows cannot deliver is said once, with the remedy", () => { + // AppContainer loopback is blocked at the firewall layer regardless of + // capability, and the exemption needs admin — out of scope for this product. + // So `allow` cannot reach a local Ollama, Jupyter or model server. A doc line + // is where that goes to die: the user would see `connection refused` and + // nothing else, which is the shape of every expensive bug in this feature. + // + // It rides the SAME one-time `warning` channel as "sandbox requested but + // unavailable" — the same species of problem, and a second mechanism for it is + // how two commands end up disagreeing about one state. + const base = { + command: "curl https://example.com", + shell: "cmd.exe", + cwd: "C:\\w", + workspace: ["C:\\w"], + platform: "win32" as const, + } + // The win32 block above already planned with network "allow", consuming the + // one-time flag. Reset rather than depend on source order. + Sandbox.forgetWarnings() + const first = Sandbox.plan({ ...base, options: { enabled: true, network: "allow" } }) + expect(first.warning).toContain("127.0.0.1") + // The remedy, not an apology: allowlist CAN reach localhost, because the + // broker runs on the host and a host process has no loopback restriction. + expect(first.warning).toContain("allowlist") + // Once per process, like warned.unavailable. + const second = Sandbox.plan({ ...base, options: { enabled: true, network: "allow" } }) + expect(second.warning).toBeUndefined() +}) + +describe("the sandbox chooses a shell it can execute", () => { + // CI has never caught this. That job step runs under `shell: bash`, which sets + // $SHELL, so `Shell.acceptable()` returns it and never reaches the fallback + // that finds Git Bash. On a developer's Windows machine with Git installed and + // no $SHELL, the fallback wins and every sandboxed command dies at 0xC0000142 + // — bash failing to load msys-2.0.dll from a directory no ACE can reach. + // These assert against an injected win32 so the property holds from any host. + + test("Program Files is unreachable, System32 and the profile are not", () => { + const root = process.env["SystemRoot"] ?? "C:\\Windows" + expect(Sandbox.reachable("C:\\Program Files\\Git\\bin\\bash.exe", "win32")).toBe(false) + // pwsh 7 installs here too. Same trap, newer binary. + expect(Sandbox.reachable("C:\\Program Files\\PowerShell\\7\\pwsh.exe", "win32")).toBe(false) + // System32 already ships an ALL APPLICATION PACKAGES ACE so AppContainers + // can load system DLLs — nothing for us to grant. + expect(Sandbox.reachable(`${root}\\System32\\cmd.exe`, "win32")).toBe(true) + // Everything is reachable in a namespace; only Windows has this problem. + expect(Sandbox.reachable("C:\\Program Files\\Git\\bin\\bash.exe", "linux")).toBe(true) + }) + + test("a confined run gets a System32 shell, never one under Program Files", () => { + const chosen = Sandbox.shell({ enabled: true }, "win32") + expect(chosen.toLowerCase()).toContain("system32") + expect(chosen.toLowerCase()).not.toContain("program files") + // And whatever it picks, the invocation table knows how to drive it — + // cmd takes /d /s /c and PowerShell takes -NoProfile -Command, and handing + // either the other's flags runs nothing while exiting 0. + const args = Shell.invocation(chosen, "echo hi") + expect(args.length).toBeGreaterThan(0) + expect(args).not.toContain("-c") + }) + + test("an unconfined run keeps the machine's own preference", () => { + // Losing Git Bash when nothing is being confined would be a regression for + // every Windows user who never turns the sandbox on. + expect(Sandbox.shell({ enabled: false }, "win32")).toBe(Shell.acceptable()) + }) +}) diff --git a/backend/cli/test/science/fixtures/fetch/alphafold.json b/backend/cli/test/science/fixtures/fetch/alphafold.json index 099134f9..26d87388 100644 --- a/backend/cli/test/science/fixtures/fetch/alphafold.json +++ b/backend/cli/test/science/fixtures/fetch/alphafold.json @@ -15,14 +15,7 @@ "fractionPlddtConfident": 0.071, "fractionPlddtVeryHigh": 0.527, "latestVersion": 6, - "allVersions": [ - 1, - 2, - 3, - 4, - 5, - 6 - ], + "allVersions": [1, 2, 3, 4, 5, 6], "sequence": "MEEPQSDPSVEPPLSQETFSDLWKLLPENNVLSPLPSQAMDDLMLSPDDIEQWFTEDPGPDEAPRMPEAAPPVAPAPAAPTPAAPAPAPSWPLSSSVPSQKTYQGSYGFRLGFLHSGTAKSVTCTYSPALNKMFCQLAKTCPVQLWVDSTPPPGTRVRAMAIYKQSQHMTEVVRRCPHHERCSDSDGLAPPQHLIRVEGNLRVEYLDDRNTFRHSVVVPYEPPEVGSDCTTIHYNYMCNSSCMGGMNRRPILTIITLEDSSGNLLGRNSFEVRVCACPGRDRRTEEENLRKKGEPHHELPPGSTKRALPNNTSSSPQPKKKPLDGEYFTLQIRGRERFEMFRELNEALELKDAQAGKEPGGSRAHSSHLKSKKGQSTSRHKKLMFKTEGPDSD", "sequenceStart": 1, "sequenceEnd": 393, @@ -68,10 +61,7 @@ "fractionPlddtConfident": 0.033, "fractionPlddtVeryHigh": 0.715, "latestVersion": 6, - "allVersions": [ - 5, - 6 - ], + "allVersions": [5, 6], "sequence": "MFCQLAKTCPVQLWVDSTPPPGTRVRAMAIYKQSQHMTEVVRRCPHHERCSDSDGLAPPQHLIRVEGNLRVEYLDDRNTFRHSVVVPYEPPEVGSDCTTIHYNYMCNSSCMGGMNRRPILTIITLEDSSGNLLGRNSFEVRVCACPGRDRRTEEENLRKKGEPHHELPPGSTKRALPNNTSSSPQPKKKPLDGEYFTLQMLLDLRWCYFLINSS", "sequenceStart": 1, "sequenceEnd": 214, @@ -114,10 +104,7 @@ "fractionPlddtConfident": 0.024, "fractionPlddtVeryHigh": 0.746, "latestVersion": 6, - "allVersions": [ - 5, - 6 - ], + "allVersions": [5, 6], "sequence": "MFCQLAKTCPVQLWVDSTPPPGTRVRAMAIYKQSQHMTEVVRRCPHHERCSDSDGLAPPQHLIRVEGNLRVEYLDDRNTFRHSVVVPYEPPEVGSDCTTIHYNYMCNSSCMGGMNRRPILTIITLEDSSGNLLGRNSFEVRVCACPGRDRRTEEENLRKKGEPHHELPPGSTKRALPNNTSSSPQPKKKPLDGEYFTLQDQTSFQKENC", "sequenceStart": 1, "sequenceEnd": 209, @@ -160,10 +147,7 @@ "fractionPlddtConfident": 0.038, "fractionPlddtVeryHigh": 0.678, "latestVersion": 6, - "allVersions": [ - 5, - 6 - ], + "allVersions": [5, 6], "sequence": "MFCQLAKTCPVQLWVDSTPPPGTRVRAMAIYKQSQHMTEVVRRCPHHERCSDSDGLAPPQHLIRVEGNLRVEYLDDRNTFRHSVVVPYEPPEVGSDCTTIHYNYMCNSSCMGGMNRRPILTIITLEDSSGNLLGRNSFEVRVCACPGRDRRTEEENLRKKGEPHHELPPGSTKRALPNNTSSSPQPKKKPLDGEYFTLQIRGRERFEMFRELNEALELKDAQAGKEPGGSRAHSSHLKSKKGQSTSRHKKLMFKTEGPDSD", "sequenceStart": 1, "sequenceEnd": 261, @@ -206,10 +190,7 @@ "fractionPlddtConfident": 0.094, "fractionPlddtVeryHigh": 0.629, "latestVersion": 6, - "allVersions": [ - 5, - 6 - ], + "allVersions": [5, 6], "sequence": "MDDLMLSPDDIEQWFTEDPGPDEAPRMPEAAPPVAPAPAAPTPAAPAPAPSWPLSSSVPSQKTYQGSYGFRLGFLHSGTAKSVTCTYSPALNKMFCQLAKTCPVQLWVDSTPPPGTRVRAMAIYKQSQHMTEVVRRCPHHERCSDSDGLAPPQHLIRVEGNLRVEYLDDRNTFRHSVVVPYEPPEVGSDCTTIHYNYMCNSSCMGGMNRRPILTIITLEDSSGNLLGRNSFEVRVCACPGRDRRTEEENLRKKGEPHHELPPGSTKRALPNNTSSSPQPKKKPLDGEYFTLQMLLDLRWCYFLINSS", "sequenceStart": 1, "sequenceEnd": 307, @@ -252,10 +233,7 @@ "fractionPlddtConfident": 0.023, "fractionPlddtVeryHigh": 0.652, "latestVersion": 6, - "allVersions": [ - 5, - 6 - ], + "allVersions": [5, 6], "sequence": "MDDLMLSPDDIEQWFTEDPGPDEAPRMPEAAPPVAPAPAAPTPAAPAPAPSWPLSSSVPSQKTYQGSYGFRLGFLHSGTAKSVTCTYSPALNKMFCQLAKTCPVQLWVDSTPPPGTRVRAMAIYKQSQHMTEVVRRCPHHERCSDSDGLAPPQHLIRVEGNLRVEYLDDRNTFRHSVVVPYEPPEVGSDCTTIHYNYMCNSSCMGGMNRRPILTIITLEDSSGNLLGRNSFEVRVCACPGRDRRTEEENLRKKGEPHHELPPGSTKRALPNNTSSSPQPKKKPLDGEYFTLQDQTSFQKENC", "sequenceStart": 1, "sequenceEnd": 302, @@ -298,10 +276,7 @@ "fractionPlddtConfident": 0.056, "fractionPlddtVeryHigh": 0.576, "latestVersion": 6, - "allVersions": [ - 5, - 6 - ], + "allVersions": [5, 6], "sequence": "MDDLMLSPDDIEQWFTEDPGPDEAPRMPEAAPPVAPAPAAPTPAAPAPAPSWPLSSSVPSQKTYQGSYGFRLGFLHSGTAKSVTCTYSPALNKMFCQLAKTCPVQLWVDSTPPPGTRVRAMAIYKQSQHMTEVVRRCPHHERCSDSDGLAPPQHLIRVEGNLRVEYLDDRNTFRHSVVVPYEPPEVGSDCTTIHYNYMCNSSCMGGMNRRPILTIITLEDSSGNLLGRNSFEVRVCACPGRDRRTEEENLRKKGEPHHELPPGSTKRALPNNTSSSPQPKKKPLDGEYFTLQIRGRERFEMFRELNEALELKDAQAGKEPGGSRAHSSHLKSKKGQSTSRHKKLMFKTEGPDSD", "sequenceStart": 1, "sequenceEnd": 354, @@ -344,10 +319,7 @@ "fractionPlddtConfident": 0.087, "fractionPlddtVeryHigh": 0.538, "latestVersion": 6, - "allVersions": [ - 5, - 6 - ], + "allVersions": [5, 6], "sequence": "MEEPQSDPSVEPPLSQETFSDLWKLLPENNVLSPLPSQAMDDLMLSPDDIEQWFTEDPGPDEAPRMPEAAPPVAPAPAAPTPAAPAPAPSWPLSSSVPSQKTYQGSYGFRLGFLHSGTAKSVTCTYSPALNKMFCQLAKTCPVQLWVDSTPPPGTRVRAMAIYKQSQHMTEVVRRCPHHERCSDSDGLAPPQHLIRVEGNLRVEYLDDRNTFRHSVVVPYEPPEVGSDCTTIHYNYMCNSSCMGGMNRRPILTIITLEDSSGNLLGRNSFEVRVCACPGRDRRTEEENLRKKGEPHHELPPGSTKRALPNNTSSSPQPKKKPLDGEYFTLQMLLDLRWCYFLINSS", "sequenceStart": 1, "sequenceEnd": 346, @@ -390,10 +362,7 @@ "fractionPlddtConfident": 0.047, "fractionPlddtVeryHigh": 0.569, "latestVersion": 6, - "allVersions": [ - 5, - 6 - ], + "allVersions": [5, 6], "sequence": "MEEPQSDPSVEPPLSQETFSDLWKLLPENNVLSPLPSQAMDDLMLSPDDIEQWFTEDPGPDEAPRMPEAAPPVAPAPAAPTPAAPAPAPSWPLSSSVPSQKTYQGSYGFRLGFLHSGTAKSVTCTYSPALNKMFCQLAKTCPVQLWVDSTPPPGTRVRAMAIYKQSQHMTEVVRRCPHHERCSDSDGLAPPQHLIRVEGNLRVEYLDDRNTFRHSVVVPYEPPEVGSDCTTIHYNYMCNSSCMGGMNRRPILTIITLEDSSGNLLGRNSFEVRVCACPGRDRRTEEENLRKKGEPHHELPPGSTKRALPNNTSSSPQPKKKPLDGEYFTLQDQTSFQKENC", "sequenceStart": 1, "sequenceEnd": 341, @@ -423,4 +392,4 @@ "isReviewed": true } ] -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/arrayexpress.json b/backend/cli/test/science/fixtures/fetch/arrayexpress.json index d9d5f49a..a133a00d 100644 --- a/backend/cli/test/science/fixtures/fetch/arrayexpress.json +++ b/backend/cli/test/science/fixtures/fetch/arrayexpress.json @@ -540,4 +540,4 @@ }, "type": "submission" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/arxiv.json b/backend/cli/test/science/fixtures/fetch/arxiv.json index babc04fb..bdab619b 100644 --- a/backend/cli/test/science/fixtures/fetch/arxiv.json +++ b/backend/cli/test/science/fixtures/fetch/arxiv.json @@ -20,4 +20,4 @@ "pdf": "https://arxiv.org/pdf/1706.03762v7", "raw": "\n http://arxiv.org/abs/1706.03762v7\n Attention Is All You Need\n 2023-08-02T00:41:18Z\n \n \n The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.\n \n \n 2017-06-12T17:57:34Z\n 15 pages, 5 figures\n \n \n Ashish Vaswani\n \n \n Noam Shazeer\n \n \n Niki Parmar\n \n \n Jakob Uszkoreit\n \n \n Llion Jones\n \n \n Aidan N. Gomez\n \n \n Lukasz Kaiser\n \n \n Illia Polosukhin\n \n " } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/bindingdb.json b/backend/cli/test/science/fixtures/fetch/bindingdb.json index fbbce190..dbf0da96 100644 --- a/backend/cli/test/science/fixtures/fetch/bindingdb.json +++ b/backend/cli/test/science/fixtures/fetch/bindingdb.json @@ -8853,4 +8853,4 @@ ] } } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/biogrid.json b/backend/cli/test/science/fixtures/fetch/biogrid.json index efb6a16f..c092b86d 100644 --- a/backend/cli/test/science/fixtures/fetch/biogrid.json +++ b/backend/cli/test/science/fixtures/fetch/biogrid.json @@ -4,4 +4,4 @@ "id": "7157", "error": "BioGRID access key required (set BIOGRID_ACCESS_KEY or pass params.accessKey)" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/biorxiv.json b/backend/cli/test/science/fixtures/fetch/biorxiv.json index 27d55175..e717a5e5 100644 --- a/backend/cli/test/science/fixtures/fetch/biorxiv.json +++ b/backend/cli/test/science/fixtures/fetch/biorxiv.json @@ -17,4 +17,4 @@ "published": "NA", "server": "bioRxiv" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/chebi.json b/backend/cli/test/science/fixtures/fetch/chebi.json index db7cc5c6..3b013bd1 100644 --- a/backend/cli/test/science/fixtures/fetch/chebi.json +++ b/backend/cli/test/science/fixtures/fetch/chebi.json @@ -3,9 +3,7 @@ "payload": { "iri": "http://purl.obolibrary.org/obo/CHEBI_15377", "lang": "en", - "description": [ - "An oxygen hydride consisting of an oxygen atom that is covalently bonded to two hydrogen atoms" - ], + "description": ["An oxygen hydride consisting of an oxygen atom that is covalently bonded to two hydrogen atoms"], "synonyms": [ "BOUND WATER", "H(2)O", @@ -30,9 +28,7 @@ "water" ], "annotation": { - "charge": [ - 0 - ], + "charge": [0], "database_cross_reference": [ "cas:7732-18-5", "gmelin:117", @@ -45,9 +41,7 @@ "reaxys:3587155", "wikipedia.en:Water" ], - "generalized_empirical_formula": [ - "H2O" - ], + "generalized_empirical_formula": ["H2O"], "has_alternative_id": [ "CHEBI:10743", "CHEBI:13352", @@ -60,27 +54,13 @@ "CHEBI:44819", "CHEBI:5585" ], - "has_obo_namespace": [ - "chebi_ontology" - ], - "id": [ - "CHEBI:15377" - ], - "inchi_key_string": [ - "XLYOFNOQVPJJNP-UHFFFAOYSA-N" - ], - "inchi_string": [ - "InChI=1S/H2O/h1H2" - ], - "mass": [ - "18.015" - ], - "monoisotopic_mass": [ - "18.01056" - ], - "smiles_string": [ - "[H]O[H]" - ] + "has_obo_namespace": ["chebi_ontology"], + "id": ["CHEBI:15377"], + "inchi_key_string": ["XLYOFNOQVPJJNP-UHFFFAOYSA-N"], + "inchi_string": ["InChI=1S/H2O/h1H2"], + "mass": ["18.015"], + "monoisotopic_mass": ["18.01056"], + "smiles_string": ["[H]O[H]"] }, "label": "water", "ontology_name": "chebi", @@ -93,9 +73,7 @@ "is_root": false, "short_form": "CHEBI_15377", "obo_id": "CHEBI:15377", - "in_subset": [ - "3_STAR" - ], + "in_subset": ["3_STAR"], "obo_definition_citation": null, "obo_xref": [ { @@ -480,4 +458,4 @@ } } } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/chembl.json b/backend/cli/test/science/fixtures/fetch/chembl.json index f1096284..4612a2eb 100644 --- a/backend/cli/test/science/fixtures/fetch/chembl.json +++ b/backend/cli/test/science/fixtures/fetch/chembl.json @@ -1,13 +1,7 @@ { "id": "CHEMBL25", "payload": { - "atc_classifications": [ - "B01AC06", - "N02BA01", - "N02BA51", - "A01AD05", - "N02BA71" - ], + "atc_classifications": ["B01AC06", "N02BA01", "N02BA51", "A01AD05", "N02BA71"], "availability_type": 2, "biotherapeutic": null, "black_box_warning": 0, @@ -474,4 +468,4 @@ "veterinary": 0, "withdrawn_flag": false } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/clinvar.json b/backend/cli/test/science/fixtures/fetch/clinvar.json index b48326f8..4e931c34 100644 --- a/backend/cli/test/science/fixtures/fetch/clinvar.json +++ b/backend/cli/test/science/fixtures/fetch/clinvar.json @@ -29,9 +29,7 @@ ], "variation_name": "NM_001065.4(TNFRSF1A):c.295T>A (p.Cys99Ser)", "cdna_change": "c.295T>A", - "aliases": [ - "C70S" - ], + "aliases": ["C70S"], "variation_loc": [ { "status": "current", @@ -77,13 +75,8 @@ } ], "supporting_submissions": { - "scv": [ - "SCV000116051", - "SCV000033385" - ], - "rcv": [ - "RCV000013138" - ] + "scv": ["SCV000116051", "SCV000033385"], + "rcv": ["RCV000013138"] }, "germline_classification": { "description": "Pathogenic", @@ -142,12 +135,8 @@ "source": "submitted" } ], - "molecular_consequence_list": [ - "missense variant", - "5 prime UTR variant", - "non-coding transcript variant" - ], + "molecular_consequence_list": ["missense variant", "5 prime UTR variant", "non-coding transcript variant"], "protein_change": "C99S", "fda_recognized_database": "" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/crossref.json b/backend/cli/test/science/fixtures/fetch/crossref.json index 10046735..44c47e39 100644 --- a/backend/cli/test/science/fixtures/fetch/crossref.json +++ b/backend/cli/test/science/fixtures/fetch/crossref.json @@ -2,13 +2,7 @@ "id": "10.1038/nature12373", "payload": { "indexed": { - "date-parts": [ - [ - 2026, - 7, - 25 - ] - ], + "date-parts": [[2026, 7, 25]], "date-time": "2026-07-25T06:56:20Z", "timestamp": 1784962580403, "version": "3.55.0" @@ -19,13 +13,7 @@ "license": [ { "start": { - "date-parts": [ - [ - 2013, - 7, - 31 - ] - ], + "date-parts": [[2013, 7, 31]], "date-time": "2013-07-31T00:00:00Z", "timestamp": 1375228800000 }, @@ -38,36 +26,21 @@ "domain": [], "crossmark-restriction": false }, - "short-container-title": [ - "Nature" - ], + "short-container-title": ["Nature"], "published-print": { - "date-parts": [ - [ - 2013, - 8 - ] - ] + "date-parts": [[2013, 8]] }, "DOI": "10.1038/nature12373", "type": "journal-article", "created": { - "date-parts": [ - [ - 2013, - 7, - 30 - ] - ], + "date-parts": [[2013, 7, 30]], "date-time": "2013-07-30T12:59:50Z", "timestamp": 1375189190000 }, "page": "54-58", "source": "Crossref", "is-referenced-by-count": 1792, - "title": [ - "Nanometre-scale thermometry in a living cell" - ], + "title": ["Nanometre-scale thermometry in a living cell"], "prefix": "10.1038", "volume": "500", "author": [ @@ -170,13 +143,7 @@ ], "member": "297", "published-online": { - "date-parts": [ - [ - 2013, - 7, - 31 - ] - ] + "date-parts": [[2013, 7, 31]] }, "reference": [ { @@ -503,9 +470,7 @@ "journal-title": "Cancer Res." } ], - "container-title": [ - "Nature" - ], + "container-title": ["Nature"], "original-title": [], "language": "en", "link": [ @@ -529,13 +494,7 @@ } ], "deposited": { - "date-parts": [ - [ - 2023, - 5, - 18 - ] - ], + "date-parts": [[2023, 5, 18]], "date-time": "2023-05-18T18:13:47Z", "timestamp": 1684433627000 }, @@ -548,35 +507,19 @@ "subtitle": [], "short-title": [], "issued": { - "date-parts": [ - [ - 2013, - 7, - 31 - ] - ] + "date-parts": [[2013, 7, 31]] }, "references-count": 30, "journal-issue": { "issue": "7460", "published-print": { - "date-parts": [ - [ - 2013, - 8 - ] - ] + "date-parts": [[2013, 8]] } }, - "alternative-id": [ - "BFnature12373" - ], + "alternative-id": ["BFnature12373"], "URL": "https://doi.org/10.1038/nature12373", "relation": {}, - "ISSN": [ - "0028-0836", - "1476-4687" - ], + "ISSN": ["0028-0836", "1476-4687"], "issn-type": [ { "value": "0028-0836", @@ -589,13 +532,7 @@ ], "subject": [], "published": { - "date-parts": [ - [ - 2013, - 7, - 31 - ] - ] + "date-parts": [[2013, 7, 31]] } } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/dbsnp.json b/backend/cli/test/science/fixtures/fetch/dbsnp.json index 97ace29c..6963a6a2 100644 --- a/backend/cli/test/science/fixtures/fetch/dbsnp.json +++ b/backend/cli/test/science/fixtures/fetch/dbsnp.json @@ -84,4 +84,4 @@ "chrpos_sort": "0005227002", "merged_sort": "0" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/depmap.json b/backend/cli/test/science/fixtures/fetch/depmap.json index 99b9fb3c..8733416c 100644 --- a/backend/cli/test/science/fixtures/fetch/depmap.json +++ b/backend/cli/test/science/fixtures/fetch/depmap.json @@ -4,4 +4,4 @@ "id": "CRISPR", "found": false } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/ensembl.json b/backend/cli/test/science/fixtures/fetch/ensembl.json index 382f6c29..6b13f180 100644 --- a/backend/cli/test/science/fixtures/fetch/ensembl.json +++ b/backend/cli/test/science/fixtures/fetch/ensembl.json @@ -5979,4 +5979,4 @@ "strand": -1, "seq_region_name": "17" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/europepmc.json b/backend/cli/test/science/fixtures/fetch/europepmc.json index ef77169a..1be341f1 100644 --- a/backend/cli/test/science/fixtures/fetch/europepmc.json +++ b/backend/cli/test/science/fixtures/fetch/europepmc.json @@ -6,9 +6,7 @@ "pmid": "37466043", "pmcid": "PMC10508479", "fullTextIdList": { - "fullTextId": [ - "PMC10508479" - ] + "fullTextId": ["PMC10508479"] }, "doi": "10.1002/vms3.1202", "title": "Interactions of dietary wheat cultivars and NSP-degrading enzyme on productive performance and egg quality traits.", @@ -82,9 +80,7 @@ ] }, "dataLinksTagsList": { - "dataLinkstag": [ - "altmetrics" - ] + "dataLinkstag": ["altmetrics"] }, "journalInfo": { "issue": "5", @@ -111,11 +107,7 @@ "language": "eng", "pubModel": "Print-Electronic", "pubTypeList": { - "pubType": [ - "Randomized Controlled Trial, Veterinary", - "research-article", - "Journal Article" - ] + "pubType": ["Randomized Controlled Trial, Veterinary", "research-article", "Journal Article"] }, "grantsList": { "grant": [ @@ -167,12 +159,7 @@ ] }, "keywordList": { - "keyword": [ - "Laying hens", - "Productive performance", - "Wheat cultivars", - "Egg Quality Indices" - ] + "keyword": ["Laying hens", "Productive performance", "Wheat cultivars", "Egg Quality Indices"] }, "subsetList": { "subset": [ @@ -254,4 +241,4 @@ "electronicPublicationDate": "2023-07-19", "firstPublicationDate": "2023-07-19" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/expression-atlas.json b/backend/cli/test/science/fixtures/fetch/expression-atlas.json index 1d4b27cb..95901001 100644 --- a/backend/cli/test/science/fixtures/fetch/expression-atlas.json +++ b/backend/cli/test/science/fixtures/fetch/expression-atlas.json @@ -4,4 +4,4 @@ "id": "E-MTAB-5214", "found": false } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/geo.json b/backend/cli/test/science/fixtures/fetch/geo.json index 90b8b6db..2e73079f 100644 --- a/backend/cli/test/science/fixtures/fetch/geo.json +++ b/backend/cli/test/science/fixtures/fetch/geo.json @@ -72,4 +72,4 @@ "geo2r": "yes", "bioproject": "PRJNA87053" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/gnomad.json b/backend/cli/test/science/fixtures/fetch/gnomad.json index 76295b7f..e018a2e7 100644 --- a/backend/cli/test/science/fixtures/fetch/gnomad.json +++ b/backend/cli/test/science/fixtures/fetch/gnomad.json @@ -9,4 +9,4 @@ "stop": 7687538, "canonical_transcript_id": "ENST00000269305" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/gtex.json b/backend/cli/test/science/fixtures/fetch/gtex.json index 98000a78..4857b97e 100644 --- a/backend/cli/test/science/fixtures/fetch/gtex.json +++ b/backend/cli/test/science/fixtures/fetch/gtex.json @@ -20,4 +20,4 @@ }, "medianExpression": [] } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/gtopdb.json b/backend/cli/test/science/fixtures/fetch/gtopdb.json index c0f1a8c7..f9cbd2d7 100644 --- a/backend/cli/test/science/fixtures/fetch/gtopdb.json +++ b/backend/cli/test/science/fixtures/fetch/gtopdb.json @@ -25,4 +25,4 @@ "inchi": "InChI=1S/C9H8O4/c1-6(10)13-8-5-3-2-4-7(8)9(11)12/h2-5H,1H3,(H,11,12)", "inchiKey": "BSYNRYMUTXBXSQ-UHFFFAOYSA-N" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/hpa.json b/backend/cli/test/science/fixtures/fetch/hpa.json index 08a01820..53e56fd3 100644 --- a/backend/cli/test/science/fixtures/fetch/hpa.json +++ b/backend/cli/test/science/fixtures/fetch/hpa.json @@ -2,15 +2,10 @@ "id": "ENSG00000141510", "payload": { "Gene": "TP53", - "Gene synonym": [ - "LFS1", - "p53" - ], + "Gene synonym": ["LFS1", "p53"], "Ensembl": "ENSG00000141510", "Gene description": "Tumor protein p53", - "Uniprot": [ - "P04637" - ], + "Uniprot": ["P04637"], "Chromosome": "17", "Position": "7661779-7687538", "Protein class": [ @@ -32,17 +27,8 @@ "Transcription", "Transcription regulation" ], - "Molecular function": [ - "Activator", - "DNA-binding", - "Repressor" - ], - "Disease involvement": [ - "Cancer-related genes", - "Disease variant", - "Li-Fraumeni syndrome", - "Tumor suppressor" - ], + "Molecular function": ["Activator", "DNA-binding", "Repressor"], + "Disease involvement": ["Cancer-related genes", "Disease variant", "Li-Fraumeni syndrome", "Tumor suppressor"], "Evidence": "Evidence at protein level", "HPA evidence": "Evidence at protein level", "UniProt evidence": "Evidence at protein level", @@ -85,9 +71,7 @@ "RNA cell line distribution": "Detected in all", "RNA cell line specificity score": null, "RNA cell line specific nTPM": null, - "RNA tissue cell type enrichment": [ - "Stomach - Mitotic cells (Stomach)" - ], + "RNA tissue cell type enrichment": ["Stomach - Mitotic cells (Stomach)"], "RNA mouse brain regional specificity": "Low region specificity", "RNA mouse brain regional distribution": "Detected in all", "RNA mouse brain regional specificity score": null, @@ -104,22 +88,11 @@ "Protein tissue distribution": "Not detected", "Protein tissue specificity score": null, "Protein tissue specific Intensity": null, - "Antibody": [ - "CAB002973", - "CAB039238", - "CAB039239", - "HPA051244", - "HPA063532", - "CAB072876" - ], + "Antibody": ["CAB002973", "CAB039238", "CAB039239", "HPA051244", "HPA063532", "CAB072876"], "Reliability (IH)": "Enhanced", "Reliability (Mouse Brain)": null, "Reliability (IF)": "Enhanced", - "Subcellular location": [ - "Nucleoplasm", - "Vesicles", - "Cytosol" - ], + "Subcellular location": ["Nucleoplasm", "Vesicles", "Cytosol"], "Secretome location": null, "Secretome function": null, "CCD Protein": "Yes", @@ -132,13 +105,8 @@ "Cell line expression cluster": "Cluster 42: Non-specific - Unknown function", "Single cell expression cluster": "Cluster 65: Non-specific - Cell growth & division", "Interactions": 998, - "Subcellular main location": [ - "Nucleoplasm" - ], - "Subcellular additional location": [ - "Vesicles", - "Cytosol" - ], + "Subcellular main location": ["Nucleoplasm"], + "Subcellular additional location": ["Vesicles", "Cytosol"], "Antibody RRID": { "CAB002973": null, "CAB039238": null, @@ -334,4 +302,4 @@ "p_val": "2.07e-2" } } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/intact.json b/backend/cli/test/science/fixtures/fetch/intact.json index ab08fa28..7857edd6 100644 --- a/backend/cli/test/science/fixtures/fetch/intact.json +++ b/backend/cli/test/science/fixtures/fetch/intact.json @@ -16,14 +16,8 @@ "intactNameB": "p05067-pro_0000000092", "mutationA": false, "mutationB": false, - "altIdsA": [ - "EBI-821758 (intact)", - "P05067-PRO_0000000092 (uniprotkb)" - ], - "altIdsB": [ - "EBI-821758 (intact)", - "P05067-PRO_0000000092 (uniprotkb)" - ], + "altIdsA": ["EBI-821758 (intact)", "P05067-PRO_0000000092 (uniprotkb)"], + "altIdsB": ["EBI-821758 (intact)", "P05067-PRO_0000000092 (uniprotkb)"], "aliasesA": [ "A4 (MI:0302 (gene name synonym))", "AD1 (MI:0302 (gene name synonym))", @@ -63,26 +57,12 @@ "typeB": "protein", "typeMIA": "MI:0326", "typeMIB": "MI:0326", - "xrefsA": [ - "EBI-77613 (intact)" - ], - "xrefsB": [ - "EBI-77613 (intact)" - ], - "annotationsA": [ - "chain-seq-start (672)", - "chain-seq-end (713)" - ], - "annotationsB": [ - "chain-seq-start (672)", - "chain-seq-end (713)" - ], - "checksumsA": [ - "crc64 (3AC85563D7858C37)" - ], - "checksumsB": [ - "crc64 (3AC85563D7858C37)" - ], + "xrefsA": ["EBI-77613 (intact)"], + "xrefsB": ["EBI-77613 (intact)"], + "annotationsA": ["chain-seq-start (672)", "chain-seq-end (713)"], + "annotationsB": ["chain-seq-start (672)", "chain-seq-end (713)"], + "checksumsA": ["crc64 (3AC85563D7858C37)"], + "checksumsB": ["crc64 (3AC85563D7858C37)"], "speciesA": "Homo sapiens", "speciesB": "Homo sapiens", "intraSpecies": "Homo sapiens", @@ -105,36 +85,18 @@ "featureCount": 0, "stoichiometryA": "0-0", "stoichiometryB": "0-0", - "identificationMethodsA": [ - "predetermined" - ], - "identificationMethodsB": [ - "predetermined" - ], - "identificationMethodMIIdentifiersA": [ - "MI:0396" - ], - "identificationMethodMIIdentifiersB": [ - "MI:0396" - ], + "identificationMethodsA": ["predetermined"], + "identificationMethodsB": ["predetermined"], + "identificationMethodMIIdentifiersA": ["MI:0396"], + "identificationMethodMIIdentifiersB": ["MI:0396"], "experimentalPreparationsA": null, "experimentalPreparationsB": null, "detectionMethod": "tem", "detectionMethodMIIdentifier": "MI:0020", - "authors": [ - "Stroud JC.", - " Liu C.", - " Teng PK.", - " Eisenberg D." - ], + "authors": ["Stroud JC.", " Liu C.", " Teng PK.", " Eisenberg D."], "sourceDatabase": "DIP", - "identifiers": [ - "DIP-87859E (dip)", - "EBI-15982574 (intact)" - ], - "confidenceValues": [ - "intact-miscore:0.99" - ], + "identifiers": ["DIP-87859E (dip)", "EBI-15982574 (intact)"], + "confidenceValues": ["intact-miscore:0.99"], "expansionMethod": null, "xrefs": [ "CPX-1062 (complex portal)", @@ -166,11 +128,7 @@ "hostOrganism": "In vitro", "hostOrganismTaxId": -1, "intactMiscore": 0.99, - "publicationIdentifiers": [ - "EBI-15982555 (intact)", - "10.1073/pnas.1203193109 (doi)", - "22547798 (pubmed)" - ], + "publicationIdentifiers": ["EBI-15982555 (intact)", "10.1073/pnas.1203193109 (doi)", "22547798 (pubmed)"], "publicationAnnotations": [ "Only protein-protein interactions", "Alzheimers - Interactions investigated in the context of Alzheimers disease" @@ -186,12 +144,8 @@ "taxIdBStyled": "9606__Homo sapiens__#335e94", "typeMIIdentifierStyled": "MI:0407__direct interaction__#43a2ca", "detectionMethodMIIdentifierStyled": "MI:0020__tem", - "identificationMethodMIAStyled": [ - "MI:0396__predetermined" - ], - "identificationMethodMIBStyled": [ - "MI:0396__predetermined" - ], + "identificationMethodMIAStyled": ["MI:0396__predetermined"], + "identificationMethodMIBStyled": ["MI:0396__predetermined"], "typeMIAStyled": "MI:0326__protein__ELLIPSE", "typeMIBStyled": "MI:0326__protein__ELLIPSE", "hostOrganismTaxIdStyled": "-1__In vitro__#8d6666", @@ -204,4 +158,4 @@ "tab27Format": "uniprotkb:P05067-PRO_0000000092\tuniprotkb:P05067-PRO_0000000092\tintact:EBI-821758|uniprotkb:P05067-PRO_0000000092\tintact:EBI-821758|uniprotkb:P05067-PRO_0000000092\tpsi-mi:p05067-pro_0000000092(display_short)|psi-mi:Amyloid-beta protein 42(display_long)|uniprotkb:Alzheimer disease amyloid protein(gene name synonym)|uniprotkb:Cerebral vascular amyloid peptide(gene name synonym)|uniprotkb:A4(gene name synonym)|uniprotkb:AD1(gene name synonym)|uniprotkb:ABPP(gene name synonym)|uniprotkb:PreA4(gene name synonym)|uniprotkb:Protease nexin-II(gene name synonym)|uniprotkb:APPI(gene name synonym)|uniprotkb:Amyloid precursor protein(gene name synonym)|uniprotkb:Amyloid-beta A4 protein(gene name synonym)|uniprotkb:Alzheimer disease amyloid A4 protein homolog(gene name synonym)|uniprotkb:\"Amyloid-beta (A4) precursor protein\"(gene name synonym)|uniprotkb:APP(gene name)\tpsi-mi:p05067-pro_0000000092(display_short)|psi-mi:Amyloid-beta protein 42(display_long)|uniprotkb:Alzheimer disease amyloid protein(gene name synonym)|uniprotkb:Cerebral vascular amyloid peptide(gene name synonym)|uniprotkb:A4(gene name synonym)|uniprotkb:AD1(gene name synonym)|uniprotkb:ABPP(gene name synonym)|uniprotkb:PreA4(gene name synonym)|uniprotkb:Protease nexin-II(gene name synonym)|uniprotkb:APPI(gene name synonym)|uniprotkb:Amyloid precursor protein(gene name synonym)|uniprotkb:Amyloid-beta A4 protein(gene name synonym)|uniprotkb:Alzheimer disease amyloid A4 protein homolog(gene name synonym)|uniprotkb:\"Amyloid-beta (A4) precursor protein\"(gene name synonym)|uniprotkb:APP(gene name)\tpsi-mi:\"MI:0020\"(transmission electron microscopy)\tStroud JC. et al.(2012)\tintact:EBI-15982555|doi:10.1073/pnas.1203193109|pubmed:22547798|imex:IM-21274\ttaxid:9606(human)|taxid:9606(Homo sapiens)\ttaxid:9606(human)|taxid:9606(Homo sapiens)\tpsi-mi:\"MI:0407\"(direct interaction)\tpsi-mi:\"MI:0465\"(Database of Interacting Proteins)\tintact:EBI-15982574|dip:DIP-87859E|imex:IM-21274-1\tintact-miscore:0.99\t-\tpsi-mi:\"MI:0499\"(unspecified role)\tpsi-mi:\"MI:0499\"(unspecified role)\tpsi-mi:\"MI:0499\"(unspecified role)\tpsi-mi:\"MI:0499\"(unspecified role)\tpsi-mi:\"MI:0326\"(protein)\tpsi-mi:\"MI:0326\"(protein)\tintact:EBI-77613(chain-parent)\tintact:EBI-77613(chain-parent)\tcomplex portal:CPX-1134(see-also)|complex portal:CPX-1120(see-also)|complex portal:CPX-1070(see-also)|complex portal:CPX-1062(see-also)|psi-mi:\"MI:0465\"(imex source)\tStart position of the feature chain:672|End position of the feature chain:713\tStart position of the feature chain:672|End position of the feature chain:713\tfull coverage:Only protein-protein interactions\ttaxid:-1(in vitro)|taxid:-1(In vitro)\t-\t2017/08/07\t2025/07/19\tcrc64:3AC85563D7858C37\tcrc64:3AC85563D7858C37\t-\t-\t-\t-\t0\t0\tpsi-mi:\"MI:0396\"(predetermined participant)\tpsi-mi:\"MI:0396\"(predetermined participant)", "searchChildInteractors": null } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/interpro.json b/backend/cli/test/science/fixtures/fetch/interpro.json index 41661337..dd9f1536 100644 --- a/backend/cli/test/science/fixtures/fetch/interpro.json +++ b/backend/cli/test/science/fixtures/fetch/interpro.json @@ -53,10 +53,7 @@ "raw_pages": "5328-41", "medline_journal": "J Biol Chem", "ISO_journal": "J. Biol. Chem.", - "authors": [ - "McMullen BA", - "Fujikawa K." - ], + "authors": ["McMullen BA", "Fujikawa K."], "DOI_URL": "http://intl.jbc.org/cgi/content/abstract/260/9/5328" }, "PUB00003400": { @@ -70,10 +67,7 @@ "raw_pages": "358-69", "medline_journal": "J Mol Evol", "ISO_journal": "J. Mol. Evol.", - "authors": [ - "Castellino FJ", - "Beals JM." - ], + "authors": ["Castellino FJ", "Beals JM."], "DOI_URL": "http://dx.doi.org/10.1007/BF02101155" }, "PUB00001541": { @@ -87,13 +81,7 @@ "raw_pages": "131-6", "medline_journal": "FEBS Lett", "ISO_journal": "FEBS Lett.", - "authors": [ - "Patthy L", - "Trexler M", - "Vali Z", - "Banyai L", - "Varadi A." - ], + "authors": ["Patthy L", "Trexler M", "Vali Z", "Banyai L", "Varadi A."], "DOI_URL": "http://dx.doi.org/10.1016/0014-5793(84)80473-1" }, "PUB00001620": { @@ -107,11 +95,7 @@ "raw_pages": "146-8", "medline_journal": "FEBS Lett", "ISO_journal": "FEBS Lett.", - "authors": [ - "Ikeo K", - "Takahashi K", - "Gojobori T." - ], + "authors": ["Ikeo K", "Takahashi K", "Gojobori T."], "DOI_URL": "http://dx.doi.org/10.1016/0014-5793(91)80036-3" }, "PUB00000803": { @@ -125,9 +109,7 @@ "raw_pages": "657-63", "medline_journal": "Cell", "ISO_journal": "Cell", - "authors": [ - "Patthy L." - ], + "authors": ["Patthy L."], "DOI_URL": "http://dx.doi.org/10.1016/S0092-8674(85)80046-5" }, "PUB00003257": { @@ -141,10 +123,7 @@ "raw_pages": "541-52", "medline_journal": "J Mol Biol", "ISO_journal": "J. Mol. Biol.", - "authors": [ - "Atkinson RA", - "Williams RJ." - ], + "authors": ["Atkinson RA", "Williams RJ."], "DOI_URL": "http://dx.doi.org/10.1016/0022-2836(90)90330-O" } }, @@ -213,4 +192,4 @@ } } } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/kegg.json b/backend/cli/test/science/fixtures/fetch/kegg.json index 3595a919..e425d874 100644 --- a/backend/cli/test/science/fixtures/fetch/kegg.json +++ b/backend/cli/test/science/fixtures/fetch/kegg.json @@ -5,4 +5,4 @@ "format": "kegg-flat", "text": "ENTRY 7157 CDS T01001\nSYMBOL TP53, BCC7, BMFS5, LFS1, P53, TRP53\nNAME (RefSeq) cellular tumor antigen p53 isoform a\nORTHOLOGY K04451 tumor protein p53\nORGANISM hsa Homo sapiens (human)\nPATHWAY hsa01522 Endocrine resistance\n hsa01524 Platinum drug resistance\n hsa04010 MAPK signaling pathway\n hsa04071 Sphingolipid signaling pathway\n hsa04110 Cell cycle\n hsa04115 p53 signaling pathway\n hsa04137 Mitophagy - animal\n hsa04151 PI3K-Akt signaling pathway\n hsa04210 Apoptosis\n hsa04211 Longevity regulating pathway\n hsa04216 Ferroptosis\n hsa04218 Cellular senescence\n hsa04310 Wnt signaling pathway\n hsa04722 Neurotrophin signaling pathway\n hsa04919 Thyroid hormone signaling pathway\n hsa05012 Parkinson disease\n hsa05014 Amyotrophic lateral sclerosis\n hsa05016 Huntington disease\n hsa05131 Shigellosis\n hsa05160 Hepatitis C\n hsa05161 Hepatitis B\n hsa05162 Measles\n hsa05163 Human cytomegalovirus infection\n hsa05165 Human papillomavirus infection\n hsa05166 Human T-cell leukemia virus 1 infection\n hsa05167 Kaposi sarcoma-associated herpesvirus infection\n hsa05168 Herpes simplex virus 1 infection\n hsa05169 Epstein-Barr virus infection\n hsa05200 Pathways in cancer\n hsa05202 Transcriptional misregulation in cancer\n hsa05203 Viral carcinogenesis\n hsa05205 Proteoglycans in cancer\n hsa05206 MicroRNAs in cancer\n hsa05210 Colorectal cancer\n hsa05212 Pancreatic cancer\n hsa05213 Endometrial cancer\n hsa05214 Glioma\n hsa05215 Prostate cancer\n hsa05216 Thyroid cancer\n hsa05217 Basal cell carcinoma\n hsa05218 Melanoma\n hsa05219 Bladder cancer\n hsa05220 Chronic myeloid leukemia\n hsa05222 Small cell lung cancer\n hsa05223 Non-small cell lung cancer\n hsa05224 Breast cancer\n hsa05225 Hepatocellular carcinoma\n hsa05226 Gastric cancer\n hsa05230 Central carbon metabolism in cancer\n hsa05417 Lipid and atherosclerosis\n hsa05418 Fluid shear stress and atherosclerosis\nNETWORK nt06160 Human T-cell leukemia virus 1 (HTLV-1)\n nt06162 Hepatitis B virus (HBV)\n nt06163 Hepatitis C virus (HCV)\n nt06164 Kaposi sarcoma-associated herpesvirus (KSHV)\n nt06165 Epstein-Barr virus (EBV)\n nt06166 Human papillomavirus (HPV)\n nt06167 Human cytomegalovirus (HCMV)\n nt06168 Herpes simplex virus 1 (HSV-1)\n nt06169 Measles virus (MV)\n nt06170 Influenza A virus (IAV)\n nt06230 Cell cycle (cancer)\n nt06240 Transcription (cancer)\n nt06260 Colorectal cancer\n nt06261 Gastric cancer\n nt06262 Pancreatic cancer\n nt06263 Hepatocellular carcinoma\n nt06265 Bladder cancer\n nt06266 Non-small cell lung cancer\n nt06267 Small cell lung cancer\n nt06268 Melanoma\n nt06269 Basal cell carcinoma\n nt06270 Breast cancer\n nt06271 Endometrial cancer\n nt06273 Glioma\n nt06274 Thyroid cancer\n nt06276 Chronic myeloid leukemia\n nt06461 Huntington disease\n nt06463 Parkinson disease\n ELEMENT N00066 MDM2-p21-Cell cycle G1/S\n N00067 Deleted p14(ARF) to p21-cell cycle G1/S\n N00068 Amplified MDM2 to p21-cell cycle G1/S\n N00076 Mutation-inactivated p14(ARF) to p21-cell cycle G1/S\n N00115 Mutation-inactivated TP53 to transcription\n N00131 Amplified MYCN to transcriptional activation\n N00167 KSHV vIRF1/3 to p21-cell cycle G1/S\n N00169 KSHV LANA to p21-cell cycle G1/S\n N00223 EBV EBNA1 to p53-mediated transcription\n N00263 EBV EBNA3C to p53-mediated transcription\n N00347 p300-p21-Cell cycle G1/S\n N00358 HPV E6 to p21-cell cycle G1/S\n N00420 HCMV IE2-86 to p21-cell cycle G1/S\n N00481 EBV BZLF1 to p53-mediated transcription\n N00497 HTLV-1 Tax to p21-cell cycle G1/S\n N00499 ATR-p21-Cell cycle G2/M\n N00520 HCV NS5A to p21-cell cycle G1/S\n N00521 HCV Core to p21-cell cycle G1/S\n N00522 HCV NS3 to p21-cell cycle G1/S\n N00535 HBV HBx to p53-mediated transcription\n N00536 MDM2-p21-Cell cycle G1/S\n N00592 HSV ICP0 to p53-mediated transcription\n N00697 HV P to p53-mediated transcription\n N00982 Mutation-caused aberrant Htt to p53-mediated transcription\n N01058 Mutation-inactivated DJ1 to to p53-mediated transcription\nDISEASE H00004 Chronic myeloid leukemia\n H00005 Chronic lymphocytic leukemia\n H00006 Hairy cell leukemia\n H00008 Burkitt lymphoma\n H00009 Adult T-cell leukemia\n H00010 Multiple myeloma\n H00013 Small cell lung cancer\n H00014 Non-small cell lung cancer\n H00015 Malignant pleural mesothelioma\n H00016 Oral cancer\n H00017 Esophageal cancer\n H00018 Gastric cancer\n H00019 Pancreatic cancer\n H00020 Colorectal cancer\n H00022 Bladder cancer\n H00025 Penile cancer\n H00026 Endometrial cancer\n H00027 Ovarian cancer\n H00028 Choriocarcinoma\n H00029 Vulvar cancer\n H00031 Breast cancer\n H00032 Thyroid cancer\n H00033 Adrenal carcinoma\n H00036 Osteosarcoma\n H00038 Melanoma\n H00039 Basal cell carcinoma\n H00040 Squamous cell carcinoma\n H00041 Kaposi sarcoma\n H00042 Glioma\n H00044 Cancer of the anal canal\n H00046 Cholangiocarcinoma\n H00047 Gallbladder cancer\n H00048 Hepatocellular carcinoma\n H00055 Laryngeal cancer\n H00881 Li-Fraumeni syndrome\n H01007 Choroid plexus papilloma\n H01463 Mycosis fungoides\n H01464 Mantle cell lymphoma\n H01470 Giant cell tumor of bone\n H01554 Fallopian tube cancer\n H01555 Merkel cell carcinoma\n H01557 Hepatic angiosarcoma\n H01559 Oropharyngeal cancer\n H01667 Medulloblastoma\n H02301 Nephroblastoma\n H02411 Chronic myelomonocytic leukemia\n H02434 Diffuse large B-cell lymphoma, not otherwise specified\n H02529 Bone marrow failure syndrome\nDRUG_TARGET Cenersen sodium: D08887\n Rezatapopt: D12982\nBRITE KEGG Orthology (KO) [BR:hsa00001]\n 09130 Environmental Information Processing\n 09132 Signal transduction\n 04010 MAPK signaling pathway\n 7157 (TP53)\n 04310 Wnt signaling pathway\n 7157 (TP53)\n 04071 Sphingolipid signaling pathway\n 7157 (TP53)\n 04151 PI3K-Akt signaling pathway\n 7157 (TP53)\n 09140 Cellular Processes\n 09141 Transport and catabolism\n 04137 Mitophagy - animal\n 7157 (TP53)\n 09143 Cell growth and death\n 04110 Cell cycle\n 7157 (TP53)\n 04210 Apoptosis\n 7157 (TP53)\n 04216 Ferroptosis\n 7157 (TP53)\n 04115 p53 signaling pathway\n 7157 (TP53)\n 04218 Cellular senescence\n 7157 (TP53)\n 09150 Organismal Systems\n 09152 Endocrine system\n 04919 Thyroid hormone signaling pathway\n 7157 (TP53)\n 09156 Nervous system\n 04722 Neurotrophin signaling pathway\n 7157 (TP53)\n 09149 Aging\n 04211 Longevity regulating pathway\n 7157 (TP53)\n 09160 Human Diseases\n 09161 Cancer: overview\n 05200 Pathways in cancer\n 7157 (TP53)\n 05202 Transcriptional misregulation in cancer\n 7157 (TP53)\n 05206 MicroRNAs in cancer\n 7157 (TP53)\n 05205 Proteoglycans in cancer\n 7157 (TP53)\n 05203 Viral carcinogenesis\n 7157 (TP53)\n 05230 Central carbon metabolism in cancer\n 7157 (TP53)\n 09162 Cancer: specific types\n 05210 Colorectal cancer\n 7157 (TP53)\n 05212 Pancreatic cancer\n 7157 (TP53)\n 05225 Hepatocellular carcinoma\n 7157 (TP53)\n 05226 Gastric cancer\n 7157 (TP53)\n 05214 Glioma\n 7157 (TP53)\n 05216 Thyroid cancer\n 7157 (TP53)\n 05220 Chronic myeloid leukemia\n 7157 (TP53)\n 05217 Basal cell carcinoma\n 7157 (TP53)\n 05218 Melanoma\n 7157 (TP53)\n 05219 Bladder cancer\n 7157 (TP53)\n 05215 Prostate cancer\n 7157 (TP53)\n 05213 Endometrial cancer\n 7157 (TP53)\n 05224 Breast cancer\n 7157 (TP53)\n 05222 Small cell lung cancer\n 7157 (TP53)\n 05223 Non-small cell lung cancer\n 7157 (TP53)\n 09172 Infectious disease: viral\n 05166 Human T-cell leukemia virus 1 infection\n 7157 (TP53)\n 05161 Hepatitis B\n 7157 (TP53)\n 05160 Hepatitis C\n 7157 (TP53)\n 05162 Measles\n 7157 (TP53)\n 05168 Herpes simplex virus 1 infection\n 7157 (TP53)\n 05163 Human cytomegalovirus infection\n 7157 (TP53)\n 05167 Kaposi sarcoma-associated herpesvirus infection\n 7157 (TP53)\n 05169 Epstein-Barr virus infection\n 7157 (TP53)\n 05165 Human papillomavirus infection\n 7157 (TP53)\n 09171 Infectious disease: bacterial\n 05131 Shigellosis\n 7157 (TP53)\n 09164 Neurodegenerative disease\n 05012 Parkinson disease\n 7157 (TP53)\n 05014 Amyotrophic lateral sclerosis\n 7157 (TP53)\n 05016 Huntington disease\n 7157 (TP53)\n 09166 Cardiovascular disease\n 05417 Lipid and atherosclerosis\n 7157 (TP53)\n 05418 Fluid shear stress and atherosclerosis\n 7157 (TP53)\n 09176 Drug resistance: antineoplastic\n 01524 Platinum drug resistance\n 7157 (TP53)\n 01522 Endocrine resistance\n 7157 (TP53)\n 09180 Brite Hierarchies\n 09182 Protein families: genetic information processing\n 03000 Transcription factors [BR:hsa03000]\n 7157 (TP53)\n 03036 Chromosome and associated proteins [BR:hsa03036]\n 7157 (TP53)\n 03400 DNA repair and recombination proteins [BR:hsa03400]\n 7157 (TP53)\n Transcription factors [BR:hsa03000]\n Eukaryotic type\n beta-Scaffold factors with minor groove contacts\n p53\n 7157 (TP53)\n Chromosome and associated proteins [BR:hsa03036]\n Eukaryotic type\n Sister chromatid separation proteins\n Aurora kinases\n Regulators of Aurora kinases\n 7157 (TP53)\n DNA repair and recombination proteins [BR:hsa03400]\n Eukaryotic type\n Check point factors\n Other check point factors\n 7157 (TP53)\nPOSITION 17:complement(7668421..7687490)\nMOTIF Pfam: P53 TAD2 P53_tetramer P53_TAD\nDBLINKS NCBI-GeneID: 7157\n NCBI-ProteinID: NP_000537\n OMIM: 191170\n HGNC: 11998\n Ensembl: ENSP00000269305.4\n UniProt: P04637 K7PPA8 Q53GA5\nSTRUCTURE PDB\nAASEQ 393\n MEEPQSDPSVEPPLSQETFSDLWKLLPENNVLSPLPSQAMDDLMLSPDDIEQWFTEDPGP\n DEAPRMPEAAPPVAPAPAAPTPAAPAPAPSWPLSSSVPSQKTYQGSYGFRLGFLHSGTAK\n SVTCTYSPALNKMFCQLAKTCPVQLWVDSTPPPGTRVRAMAIYKQSQHMTEVVRRCPHHE\n RCSDSDGLAPPQHLIRVEGNLRVEYLDDRNTFRHSVVVPYEPPEVGSDCTTIHYNYMCNS\n SCMGGMNRRPILTIITLEDSSGNLLGRNSFEVRVCACPGRDRRTEEENLRKKGEPHHELP\n PGSTKRALPNNTSSSPQPKKKPLDGEYFTLQIRGRERFEMFRELNEALELKDAQAGKEPG\n GSRAHSSHLKSKKGQSTSRHKKLMFKTEGPDSD\nNTSEQ 1182\n atggaggagccgcagtcagatcctagcgtcgagccccctctgagtcaggaaacattttca\n gacctatggaaactacttcctgaaaacaacgttctgtcccccttgccgtcccaagcaatg\n gatgatttgatgctgtccccggacgatattgaacaatggttcactgaagacccaggtcca\n gatgaagctcccagaatgccagaggctgctccccccgtggcccctgcaccagcagctcct\n acaccggcggcccctgcaccagccccctcctggcccctgtcatcttctgtcccttcccag\n aaaacctaccagggcagctacggtttccgtctgggcttcttgcattctgggacagccaag\n tctgtgacttgcacgtactcccctgccctcaacaagatgttttgccaactggccaagacc\n tgccctgtgcagctgtgggttgattccacacccccgcccggcacccgcgtccgcgccatg\n gccatctacaagcagtcacagcacatgacggaggttgtgaggcgctgcccccaccatgag\n cgctgctcagatagcgatggtctggcccctcctcagcatcttatccgagtggaaggaaat\n ttgcgtgtggagtatttggatgacagaaacacttttcgacatagtgtggtggtgccctat\n gagccgcctgaggttggctctgactgtaccaccatccactacaactacatgtgtaacagt\n tcctgcatgggcggcatgaaccggaggcccatcctcaccatcatcacactggaagactcc\n agtggtaatctactgggacggaacagctttgaggtgcgtgtttgtgcctgtcctgggaga\n gaccggcgcacagaggaagagaatctccgcaagaaaggggagcctcaccacgagctgccc\n ccagggagcactaagcgagcactgcccaacaacaccagctcctctccccagccaaagaag\n aaaccactggatggagaatatttcacccttcagatccgtgggcgtgagcgcttcgagatg\n ttccgagagctgaatgaggccttggaactcaaggatgcccaggctgggaaggagccaggg\n gggagcagggctcactccagccacctgaagtccaaaaagggtcagtctacctcccgccat\n aaaaaactcatgttcaagacagaagggcctgactcagactga\n///\n" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/mygene.json b/backend/cli/test/science/fixtures/fetch/mygene.json index 3d574d4f..94328d9b 100644 --- a/backend/cli/test/science/fixtures/fetch/mygene.json +++ b/backend/cli/test/science/fixtures/fetch/mygene.json @@ -3851,13 +3851,7 @@ } ] }, - "alias": [ - "BCC7", - "BMFS5", - "LFS1", - "P53", - "TRP53" - ], + "alias": ["BCC7", "BMFS5", "LFS1", "P53", "TRP53"], "chembl": { "chembl_target": [ "CHEMBL1907611", @@ -4186,50 +4180,17 @@ "cdsstart": 7669608, "chr": "17", "position": [ - [ - 7668420, - 7669690 - ], - [ - 7670608, - 7670715 - ], - [ - 7673534, - 7673608 - ], - [ - 7673700, - 7673837 - ], - [ - 7674180, - 7674290 - ], - [ - 7674858, - 7674971 - ], - [ - 7675052, - 7675236 - ], - [ - 7675993, - 7676272 - ], - [ - 7676381, - 7676403 - ], - [ - 7676520, - 7676622 - ], - [ - 7687376, - 7687490 - ] + [7668420, 7669690], + [7670608, 7670715], + [7673534, 7673608], + [7673700, 7673837], + [7674180, 7674290], + [7674858, 7674971], + [7675052, 7675236], + [7675993, 7676272], + [7676381, 7676403], + [7676520, 7676622], + [7687376, 7687490] ], "strand": -1, "transcript": "NM_000546", @@ -4241,50 +4202,17 @@ "cdsstart": 7669608, "chr": "17", "position": [ - [ - 7668401, - 7669690 - ], - [ - 7670608, - 7670715 - ], - [ - 7673534, - 7673608 - ], - [ - 7673700, - 7673837 - ], - [ - 7674180, - 7674290 - ], - [ - 7674858, - 7674971 - ], - [ - 7675052, - 7675236 - ], - [ - 7675993, - 7676272 - ], - [ - 7676381, - 7676403 - ], - [ - 7676520, - 7676619 - ], - [ - 7687376, - 7687550 - ] + [7668401, 7669690], + [7670608, 7670715], + [7673534, 7673608], + [7673700, 7673837], + [7674180, 7674290], + [7674858, 7674971], + [7675052, 7675236], + [7675993, 7676272], + [7676381, 7676403], + [7676520, 7676619], + [7687376, 7687550] ], "strand": -1, "transcript": "NM_001126112", @@ -4296,54 +4224,18 @@ "cdsstart": 7673218, "chr": "17", "position": [ - [ - 7668401, - 7669690 - ], - [ - 7670608, - 7670715 - ], - [ - 7673206, - 7673266 - ], - [ - 7673534, - 7673608 - ], - [ - 7673700, - 7673837 - ], - [ - 7674180, - 7674290 - ], - [ - 7674858, - 7674971 - ], - [ - 7675052, - 7675236 - ], - [ - 7675993, - 7676272 - ], - [ - 7676381, - 7676403 - ], - [ - 7676520, - 7676622 - ], - [ - 7687376, - 7687550 - ] + [7668401, 7669690], + [7670608, 7670715], + [7673206, 7673266], + [7673534, 7673608], + [7673700, 7673837], + [7674180, 7674290], + [7674858, 7674971], + [7675052, 7675236], + [7675993, 7676272], + [7676381, 7676403], + [7676520, 7676622], + [7687376, 7687550] ], "strand": -1, "transcript": "NM_001126113", @@ -4355,54 +4247,18 @@ "cdsstart": 7673306, "chr": "17", "position": [ - [ - 7668401, - 7669690 - ], - [ - 7670608, - 7670715 - ], - [ - 7673206, - 7673339 - ], - [ - 7673534, - 7673608 - ], - [ - 7673700, - 7673837 - ], - [ - 7674180, - 7674290 - ], - [ - 7674858, - 7674971 - ], - [ - 7675052, - 7675236 - ], - [ - 7675993, - 7676272 - ], - [ - 7676381, - 7676403 - ], - [ - 7676520, - 7676622 - ], - [ - 7687376, - 7687550 - ] + [7668401, 7669690], + [7670608, 7670715], + [7673206, 7673339], + [7673534, 7673608], + [7673700, 7673837], + [7674180, 7674290], + [7674858, 7674971], + [7675052, 7675236], + [7675993, 7676272], + [7676381, 7676403], + [7676520, 7676622], + [7687376, 7687550] ], "strand": -1, "transcript": "NM_001126114", @@ -4414,34 +4270,13 @@ "cdsstart": 7669608, "chr": "17", "position": [ - [ - 7668401, - 7669690 - ], - [ - 7670608, - 7670715 - ], - [ - 7673534, - 7673608 - ], - [ - 7673700, - 7673837 - ], - [ - 7674180, - 7674290 - ], - [ - 7674858, - 7674971 - ], - [ - 7675052, - 7675493 - ] + [7668401, 7669690], + [7670608, 7670715], + [7673534, 7673608], + [7673700, 7673837], + [7674180, 7674290], + [7674858, 7674971], + [7675052, 7675493] ], "strand": -1, "transcript": "NM_001126115", @@ -4453,38 +4288,14 @@ "cdsstart": 7673306, "chr": "17", "position": [ - [ - 7668401, - 7669690 - ], - [ - 7670608, - 7670715 - ], - [ - 7673206, - 7673339 - ], - [ - 7673534, - 7673608 - ], - [ - 7673700, - 7673837 - ], - [ - 7674180, - 7674290 - ], - [ - 7674858, - 7674971 - ], - [ - 7675052, - 7675493 - ] + [7668401, 7669690], + [7670608, 7670715], + [7673206, 7673339], + [7673534, 7673608], + [7673700, 7673837], + [7674180, 7674290], + [7674858, 7674971], + [7675052, 7675493] ], "strand": -1, "transcript": "NM_001126116", @@ -4496,38 +4307,14 @@ "cdsstart": 7673218, "chr": "17", "position": [ - [ - 7668401, - 7669690 - ], - [ - 7670608, - 7670715 - ], - [ - 7673206, - 7673266 - ], - [ - 7673534, - 7673608 - ], - [ - 7673700, - 7673837 - ], - [ - 7674180, - 7674290 - ], - [ - 7674858, - 7674971 - ], - [ - 7675052, - 7675493 - ] + [7668401, 7669690], + [7670608, 7670715], + [7673206, 7673266], + [7673534, 7673608], + [7673700, 7673837], + [7674180, 7674290], + [7674858, 7674971], + [7675052, 7675493] ], "strand": -1, "transcript": "NM_001126117", @@ -4539,46 +4326,16 @@ "cdsstart": 7669608, "chr": "17", "position": [ - [ - 7668401, - 7669690 - ], - [ - 7670608, - 7670715 - ], - [ - 7673534, - 7673608 - ], - [ - 7673700, - 7673837 - ], - [ - 7674180, - 7674290 - ], - [ - 7674858, - 7674971 - ], - [ - 7675052, - 7675236 - ], - [ - 7675993, - 7676272 - ], - [ - 7676381, - 7676622 - ], - [ - 7687376, - 7687550 - ] + [7668401, 7669690], + [7670608, 7670715], + [7673534, 7673608], + [7673700, 7673837], + [7674180, 7674290], + [7674858, 7674971], + [7675052, 7675236], + [7675993, 7676272], + [7676381, 7676622], + [7687376, 7687550] ], "strand": -1, "transcript": "NM_001126118", @@ -4590,54 +4347,18 @@ "cdsstart": 7673218, "chr": "17", "position": [ - [ - 7668401, - 7669690 - ], - [ - 7670608, - 7670715 - ], - [ - 7673206, - 7673266 - ], - [ - 7673534, - 7673608 - ], - [ - 7673700, - 7673837 - ], - [ - 7674180, - 7674290 - ], - [ - 7674858, - 7674971 - ], - [ - 7675052, - 7675236 - ], - [ - 7675993, - 7676272 - ], - [ - 7676381, - 7676403 - ], - [ - 7676520, - 7676622 - ], - [ - 7687376, - 7687490 - ] + [7668401, 7669690], + [7670608, 7670715], + [7673206, 7673266], + [7673534, 7673608], + [7673700, 7673837], + [7674180, 7674290], + [7674858, 7674971], + [7675052, 7675236], + [7675993, 7676272], + [7676381, 7676403], + [7676520, 7676622], + [7687376, 7687490] ], "strand": -1, "transcript": "NM_001276695", @@ -4649,54 +4370,18 @@ "cdsstart": 7673306, "chr": "17", "position": [ - [ - 7668401, - 7669690 - ], - [ - 7670608, - 7670715 - ], - [ - 7673206, - 7673339 - ], - [ - 7673534, - 7673608 - ], - [ - 7673700, - 7673837 - ], - [ - 7674180, - 7674290 - ], - [ - 7674858, - 7674971 - ], - [ - 7675052, - 7675236 - ], - [ - 7675993, - 7676272 - ], - [ - 7676381, - 7676403 - ], - [ - 7676520, - 7676622 - ], - [ - 7687376, - 7687490 - ] + [7668401, 7669690], + [7670608, 7670715], + [7673206, 7673339], + [7673534, 7673608], + [7673700, 7673837], + [7674180, 7674290], + [7674858, 7674971], + [7675052, 7675236], + [7675993, 7676272], + [7676381, 7676403], + [7676520, 7676622], + [7687376, 7687490] ], "strand": -1, "transcript": "NM_001276696", @@ -4708,34 +4393,13 @@ "cdsstart": 7669608, "chr": "17", "position": [ - [ - 7668401, - 7669690 - ], - [ - 7670608, - 7670715 - ], - [ - 7673534, - 7673608 - ], - [ - 7673700, - 7673837 - ], - [ - 7674180, - 7674290 - ], - [ - 7674858, - 7674971 - ], - [ - 7675052, - 7675244 - ] + [7668401, 7669690], + [7670608, 7670715], + [7673534, 7673608], + [7673700, 7673837], + [7674180, 7674290], + [7674858, 7674971], + [7675052, 7675244] ], "strand": -1, "transcript": "NM_001276697", @@ -4747,38 +4411,14 @@ "cdsstart": 7673306, "chr": "17", "position": [ - [ - 7668401, - 7669690 - ], - [ - 7670608, - 7670715 - ], - [ - 7673206, - 7673339 - ], - [ - 7673534, - 7673608 - ], - [ - 7673700, - 7673837 - ], - [ - 7674180, - 7674290 - ], - [ - 7674858, - 7674971 - ], - [ - 7675052, - 7675244 - ] + [7668401, 7669690], + [7670608, 7670715], + [7673206, 7673339], + [7673534, 7673608], + [7673700, 7673837], + [7674180, 7674290], + [7674858, 7674971], + [7675052, 7675244] ], "strand": -1, "transcript": "NM_001276698", @@ -4790,38 +4430,14 @@ "cdsstart": 7673218, "chr": "17", "position": [ - [ - 7668401, - 7669690 - ], - [ - 7670608, - 7670715 - ], - [ - 7673206, - 7673266 - ], - [ - 7673534, - 7673608 - ], - [ - 7673700, - 7673837 - ], - [ - 7674180, - 7674290 - ], - [ - 7674858, - 7674971 - ], - [ - 7675052, - 7675244 - ] + [7668401, 7669690], + [7670608, 7670715], + [7673206, 7673266], + [7673534, 7673608], + [7673700, 7673837], + [7674180, 7674290], + [7674858, 7674971], + [7675052, 7675244] ], "strand": -1, "transcript": "NM_001276699", @@ -4833,50 +4449,17 @@ "cdsstart": 7669608, "chr": "17", "position": [ - [ - 7668401, - 7669690 - ], - [ - 7670608, - 7670715 - ], - [ - 7673534, - 7673608 - ], - [ - 7673700, - 7673837 - ], - [ - 7674180, - 7674290 - ], - [ - 7674858, - 7674971 - ], - [ - 7675052, - 7675236 - ], - [ - 7675993, - 7676272 - ], - [ - 7676381, - 7676403 - ], - [ - 7676520, - 7676622 - ], - [ - 7687376, - 7687490 - ] + [7668401, 7669690], + [7670608, 7670715], + [7673534, 7673608], + [7673700, 7673837], + [7674180, 7674290], + [7674858, 7674971], + [7675052, 7675236], + [7675993, 7676272], + [7676381, 7676403], + [7676520, 7676622], + [7687376, 7687490] ], "strand": -1, "transcript": "NM_001276760", @@ -4888,50 +4471,17 @@ "cdsstart": 7669608, "chr": "17", "position": [ - [ - 7668401, - 7669690 - ], - [ - 7670608, - 7670715 - ], - [ - 7673534, - 7673608 - ], - [ - 7673700, - 7673837 - ], - [ - 7674180, - 7674290 - ], - [ - 7674858, - 7674971 - ], - [ - 7675052, - 7675236 - ], - [ - 7675993, - 7676272 - ], - [ - 7676381, - 7676403 - ], - [ - 7676520, - 7676619 - ], - [ - 7687376, - 7687490 - ] + [7668401, 7669690], + [7670608, 7670715], + [7673534, 7673608], + [7673700, 7673837], + [7674180, 7674290], + [7674858, 7674971], + [7675052, 7675236], + [7675993, 7676272], + [7676381, 7676403], + [7676520, 7676619], + [7687376, 7687490] ], "strand": -1, "transcript": "NM_001276761", @@ -4945,50 +4495,17 @@ "cdsstart": 7572926, "chr": "17", "position": [ - [ - 7571738, - 7573008 - ], - [ - 7573926, - 7574033 - ], - [ - 7576852, - 7576926 - ], - [ - 7577018, - 7577155 - ], - [ - 7577498, - 7577608 - ], - [ - 7578176, - 7578289 - ], - [ - 7578370, - 7578554 - ], - [ - 7579311, - 7579590 - ], - [ - 7579699, - 7579721 - ], - [ - 7579838, - 7579940 - ], - [ - 7590694, - 7590808 - ] + [7571738, 7573008], + [7573926, 7574033], + [7576852, 7576926], + [7577018, 7577155], + [7577498, 7577608], + [7578176, 7578289], + [7578370, 7578554], + [7579311, 7579590], + [7579699, 7579721], + [7579838, 7579940], + [7590694, 7590808] ], "strand": -1, "transcript": "NM_000546", @@ -5000,50 +4517,17 @@ "cdsstart": 7572926, "chr": "17", "position": [ - [ - 7571719, - 7573008 - ], - [ - 7573926, - 7574033 - ], - [ - 7576852, - 7576926 - ], - [ - 7577018, - 7577155 - ], - [ - 7577498, - 7577608 - ], - [ - 7578176, - 7578289 - ], - [ - 7578370, - 7578554 - ], - [ - 7579311, - 7579590 - ], - [ - 7579699, - 7579721 - ], - [ - 7579838, - 7579937 - ], - [ - 7590694, - 7590868 - ] + [7571719, 7573008], + [7573926, 7574033], + [7576852, 7576926], + [7577018, 7577155], + [7577498, 7577608], + [7578176, 7578289], + [7578370, 7578554], + [7579311, 7579590], + [7579699, 7579721], + [7579838, 7579937], + [7590694, 7590868] ], "strand": -1, "transcript": "NM_001126112", @@ -5055,54 +4539,18 @@ "cdsstart": 7576536, "chr": "17", "position": [ - [ - 7571719, - 7573008 - ], - [ - 7573926, - 7574033 - ], - [ - 7576524, - 7576584 - ], - [ - 7576852, - 7576926 - ], - [ - 7577018, - 7577155 - ], - [ - 7577498, - 7577608 - ], - [ - 7578176, - 7578289 - ], - [ - 7578370, - 7578554 - ], - [ - 7579311, - 7579590 - ], - [ - 7579699, - 7579721 - ], - [ - 7579838, - 7579940 - ], - [ - 7590694, - 7590868 - ] + [7571719, 7573008], + [7573926, 7574033], + [7576524, 7576584], + [7576852, 7576926], + [7577018, 7577155], + [7577498, 7577608], + [7578176, 7578289], + [7578370, 7578554], + [7579311, 7579590], + [7579699, 7579721], + [7579838, 7579940], + [7590694, 7590868] ], "strand": -1, "transcript": "NM_001126113", @@ -5114,54 +4562,18 @@ "cdsstart": 7576624, "chr": "17", "position": [ - [ - 7571719, - 7573008 - ], - [ - 7573926, - 7574033 - ], - [ - 7576524, - 7576657 - ], - [ - 7576852, - 7576926 - ], - [ - 7577018, - 7577155 - ], - [ - 7577498, - 7577608 - ], - [ - 7578176, - 7578289 - ], - [ - 7578370, - 7578554 - ], - [ - 7579311, - 7579590 - ], - [ - 7579699, - 7579721 - ], - [ - 7579838, - 7579940 - ], - [ - 7590694, - 7590868 - ] + [7571719, 7573008], + [7573926, 7574033], + [7576524, 7576657], + [7576852, 7576926], + [7577018, 7577155], + [7577498, 7577608], + [7578176, 7578289], + [7578370, 7578554], + [7579311, 7579590], + [7579699, 7579721], + [7579838, 7579940], + [7590694, 7590868] ], "strand": -1, "transcript": "NM_001126114", @@ -5173,34 +4585,13 @@ "cdsstart": 7572926, "chr": "17", "position": [ - [ - 7571719, - 7573008 - ], - [ - 7573926, - 7574033 - ], - [ - 7576852, - 7576926 - ], - [ - 7577018, - 7577155 - ], - [ - 7577498, - 7577608 - ], - [ - 7578176, - 7578289 - ], - [ - 7578370, - 7578811 - ] + [7571719, 7573008], + [7573926, 7574033], + [7576852, 7576926], + [7577018, 7577155], + [7577498, 7577608], + [7578176, 7578289], + [7578370, 7578811] ], "strand": -1, "transcript": "NM_001126115", @@ -5212,38 +4603,14 @@ "cdsstart": 7576624, "chr": "17", "position": [ - [ - 7571719, - 7573008 - ], - [ - 7573926, - 7574033 - ], - [ - 7576524, - 7576657 - ], - [ - 7576852, - 7576926 - ], - [ - 7577018, - 7577155 - ], - [ - 7577498, - 7577608 - ], - [ - 7578176, - 7578289 - ], - [ - 7578370, - 7578811 - ] + [7571719, 7573008], + [7573926, 7574033], + [7576524, 7576657], + [7576852, 7576926], + [7577018, 7577155], + [7577498, 7577608], + [7578176, 7578289], + [7578370, 7578811] ], "strand": -1, "transcript": "NM_001126116", @@ -5255,38 +4622,14 @@ "cdsstart": 7576536, "chr": "17", "position": [ - [ - 7571719, - 7573008 - ], - [ - 7573926, - 7574033 - ], - [ - 7576524, - 7576584 - ], - [ - 7576852, - 7576926 - ], - [ - 7577018, - 7577155 - ], - [ - 7577498, - 7577608 - ], - [ - 7578176, - 7578289 - ], - [ - 7578370, - 7578811 - ] + [7571719, 7573008], + [7573926, 7574033], + [7576524, 7576584], + [7576852, 7576926], + [7577018, 7577155], + [7577498, 7577608], + [7578176, 7578289], + [7578370, 7578811] ], "strand": -1, "transcript": "NM_001126117", @@ -5298,46 +4641,16 @@ "cdsstart": 7572926, "chr": "17", "position": [ - [ - 7571719, - 7573008 - ], - [ - 7573926, - 7574033 - ], - [ - 7576852, - 7576926 - ], - [ - 7577018, - 7577155 - ], - [ - 7577498, - 7577608 - ], - [ - 7578176, - 7578289 - ], - [ - 7578370, - 7578554 - ], - [ - 7579311, - 7579590 - ], - [ - 7579699, - 7579940 - ], - [ - 7590694, - 7590868 - ] + [7571719, 7573008], + [7573926, 7574033], + [7576852, 7576926], + [7577018, 7577155], + [7577498, 7577608], + [7578176, 7578289], + [7578370, 7578554], + [7579311, 7579590], + [7579699, 7579940], + [7590694, 7590868] ], "strand": -1, "transcript": "NM_001126118", @@ -5349,54 +4662,18 @@ "cdsstart": 7576536, "chr": "17", "position": [ - [ - 7571719, - 7573008 - ], - [ - 7573926, - 7574033 - ], - [ - 7576524, - 7576584 - ], - [ - 7576852, - 7576926 - ], - [ - 7577018, - 7577155 - ], - [ - 7577498, - 7577608 - ], - [ - 7578176, - 7578289 - ], - [ - 7578370, - 7578554 - ], - [ - 7579311, - 7579590 - ], - [ - 7579699, - 7579721 - ], - [ - 7579838, - 7579940 - ], - [ - 7590694, - 7590808 - ] + [7571719, 7573008], + [7573926, 7574033], + [7576524, 7576584], + [7576852, 7576926], + [7577018, 7577155], + [7577498, 7577608], + [7578176, 7578289], + [7578370, 7578554], + [7579311, 7579590], + [7579699, 7579721], + [7579838, 7579940], + [7590694, 7590808] ], "strand": -1, "transcript": "NM_001276695", @@ -5408,54 +4685,18 @@ "cdsstart": 7576624, "chr": "17", "position": [ - [ - 7571719, - 7573008 - ], - [ - 7573926, - 7574033 - ], - [ - 7576524, - 7576657 - ], - [ - 7576852, - 7576926 - ], - [ - 7577018, - 7577155 - ], - [ - 7577498, - 7577608 - ], - [ - 7578176, - 7578289 - ], - [ - 7578370, - 7578554 - ], - [ - 7579311, - 7579590 - ], - [ - 7579699, - 7579721 - ], - [ - 7579838, - 7579940 - ], - [ - 7590694, - 7590808 - ] + [7571719, 7573008], + [7573926, 7574033], + [7576524, 7576657], + [7576852, 7576926], + [7577018, 7577155], + [7577498, 7577608], + [7578176, 7578289], + [7578370, 7578554], + [7579311, 7579590], + [7579699, 7579721], + [7579838, 7579940], + [7590694, 7590808] ], "strand": -1, "transcript": "NM_001276696", @@ -5467,34 +4708,13 @@ "cdsstart": 7572926, "chr": "17", "position": [ - [ - 7571719, - 7573008 - ], - [ - 7573926, - 7574033 - ], - [ - 7576852, - 7576926 - ], - [ - 7577018, - 7577155 - ], - [ - 7577498, - 7577608 - ], - [ - 7578176, - 7578289 - ], - [ - 7578370, - 7578562 - ] + [7571719, 7573008], + [7573926, 7574033], + [7576852, 7576926], + [7577018, 7577155], + [7577498, 7577608], + [7578176, 7578289], + [7578370, 7578562] ], "strand": -1, "transcript": "NM_001276697", @@ -5506,38 +4726,14 @@ "cdsstart": 7576624, "chr": "17", "position": [ - [ - 7571719, - 7573008 - ], - [ - 7573926, - 7574033 - ], - [ - 7576524, - 7576657 - ], - [ - 7576852, - 7576926 - ], - [ - 7577018, - 7577155 - ], - [ - 7577498, - 7577608 - ], - [ - 7578176, - 7578289 - ], - [ - 7578370, - 7578562 - ] + [7571719, 7573008], + [7573926, 7574033], + [7576524, 7576657], + [7576852, 7576926], + [7577018, 7577155], + [7577498, 7577608], + [7578176, 7578289], + [7578370, 7578562] ], "strand": -1, "transcript": "NM_001276698", @@ -5549,38 +4745,14 @@ "cdsstart": 7576536, "chr": "17", "position": [ - [ - 7571719, - 7573008 - ], - [ - 7573926, - 7574033 - ], - [ - 7576524, - 7576584 - ], - [ - 7576852, - 7576926 - ], - [ - 7577018, - 7577155 - ], - [ - 7577498, - 7577608 - ], - [ - 7578176, - 7578289 - ], - [ - 7578370, - 7578562 - ] + [7571719, 7573008], + [7573926, 7574033], + [7576524, 7576584], + [7576852, 7576926], + [7577018, 7577155], + [7577498, 7577608], + [7578176, 7578289], + [7578370, 7578562] ], "strand": -1, "transcript": "NM_001276699", @@ -5592,50 +4764,17 @@ "cdsstart": 7572926, "chr": "17", "position": [ - [ - 7571719, - 7573008 - ], - [ - 7573926, - 7574033 - ], - [ - 7576852, - 7576926 - ], - [ - 7577018, - 7577155 - ], - [ - 7577498, - 7577608 - ], - [ - 7578176, - 7578289 - ], - [ - 7578370, - 7578554 - ], - [ - 7579311, - 7579590 - ], - [ - 7579699, - 7579721 - ], - [ - 7579838, - 7579940 - ], - [ - 7590694, - 7590808 - ] + [7571719, 7573008], + [7573926, 7574033], + [7576852, 7576926], + [7577018, 7577155], + [7577498, 7577608], + [7578176, 7578289], + [7578370, 7578554], + [7579311, 7579590], + [7579699, 7579721], + [7579838, 7579940], + [7590694, 7590808] ], "strand": -1, "transcript": "NM_001276760", @@ -5647,50 +4786,17 @@ "cdsstart": 7572926, "chr": "17", "position": [ - [ - 7571719, - 7573008 - ], - [ - 7573926, - 7574033 - ], - [ - 7576852, - 7576926 - ], - [ - 7577018, - 7577155 - ], - [ - 7577498, - 7577608 - ], - [ - 7578176, - 7578289 - ], - [ - 7578370, - 7578554 - ], - [ - 7579311, - 7579590 - ], - [ - 7579699, - 7579721 - ], - [ - 7579838, - 7579937 - ], - [ - 7590694, - 7590808 - ] + [7571719, 7573008], + [7573926, 7574033], + [7576852, 7576926], + [7577018, 7577155], + [7577498, 7577608], + [7578176, 7578289], + [7578370, 7578554], + [7579311, 7579590], + [7579699, 7579721], + [7579838, 7579937], + [7590694, 7590808] ], "strand": -1, "transcript": "NM_001276761", @@ -47287,10 +46393,7 @@ "evidence": "IDA", "gocategory": "BP", "id": "GO:0000122", - "pubmed": [ - 10329733, - 15340061 - ], + "pubmed": [10329733, 15340061], "qualifier": "involved_in", "term": "negative regulation of transcription by RNA polymerase II" }, @@ -47376,11 +46479,7 @@ "evidence": "IDA", "gocategory": "BP", "id": "GO:0006974", - "pubmed": [ - 15710329, - 17938203, - 30089260 - ], + "pubmed": [15710329, 17938203, 30089260], "qualifier": "involved_in", "term": "DNA damage response" }, @@ -47419,10 +46518,7 @@ "evidence": "IDA", "gocategory": "BP", "id": "GO:0008104", - "pubmed": [ - 15340061, - 16507995 - ], + "pubmed": [15340061, 16507995], "qualifier": "involved_in", "term": "intracellular protein localization" }, @@ -47512,11 +46608,7 @@ "evidence": "IDA", "gocategory": "BP", "id": "GO:0010628", - "pubmed": [ - 15314173, - 20332243, - 26100857 - ], + "pubmed": [15314173, 20332243, 26100857], "qualifier": "involved_in", "term": "positive regulation of gene expression" }, @@ -47555,11 +46647,7 @@ "evidence": "IDA", "gocategory": "BP", "id": "GO:0030330", - "pubmed": [ - 15149599, - 16213212, - 29681526 - ], + "pubmed": [15149599, 16213212, 29681526], "qualifier": "involved_in", "term": "DNA damage response, signal transduction by p53 class mediator" }, @@ -47567,12 +46655,7 @@ "evidence": "IMP", "gocategory": "BP", "id": "GO:0030330", - "pubmed": [ - 7958916, - 16213212, - 16479015, - 20160708 - ], + "pubmed": [7958916, 16213212, 16479015, 20160708], "qualifier": "involved_in", "term": "DNA damage response, signal transduction by p53 class mediator" }, @@ -47635,10 +46718,7 @@ "evidence": "IDA", "gocategory": "BP", "id": "GO:0042771", - "pubmed": [ - 14654789, - 14744935 - ], + "pubmed": [14654789, 14744935], "qualifier": "acts_upstream_of_or_within", "term": "intrinsic apoptotic signaling pathway in response to DNA damage by p53 class mediator" }, @@ -47684,11 +46764,7 @@ "evidence": "IDA", "gocategory": "BP", "id": "GO:0043065", - "pubmed": [ - 12667443, - 15565177, - 20959462 - ], + "pubmed": [12667443, 15565177, 20959462], "qualifier": "involved_in", "term": "positive regulation of apoptotic process" }, @@ -47704,10 +46780,7 @@ "evidence": "IGI", "gocategory": "BP", "id": "GO:0043066", - "pubmed": [ - 12203124, - 12433990 - ], + "pubmed": [12203124, 12433990], "qualifier": "involved_in", "term": "negative regulation of apoptotic process" }, @@ -47745,10 +46818,7 @@ "evidence": "IDA", "gocategory": "BP", "id": "GO:0045892", - "pubmed": [ - 9271120, - 24051492 - ], + "pubmed": [9271120, 24051492], "qualifier": "involved_in", "term": "negative regulation of DNA-templated transcription" }, @@ -47779,11 +46849,7 @@ "evidence": "IDA", "gocategory": "BP", "id": "GO:0045893", - "pubmed": [ - 16322561, - 17403783, - 20378837 - ], + "pubmed": [16322561, 17403783, 20378837], "qualifier": "involved_in", "term": "positive regulation of DNA-templated transcription" }, @@ -47829,16 +46895,7 @@ "evidence": "IDA", "gocategory": "BP", "id": "GO:0045944", - "pubmed": [ - 11672523, - 16131611, - 17145718, - 17310983, - 18549481, - 20959462, - 24652652, - 30089260 - ], + "pubmed": [11672523, 16131611, 17145718, 17310983, 18549481, 20959462, 24652652, 30089260], "qualifier": "involved_in", "term": "positive regulation of transcription by RNA polymerase II" }, @@ -47854,11 +46911,7 @@ "evidence": "IMP", "gocategory": "BP", "id": "GO:0045944", - "pubmed": [ - 7958916, - 17146433, - 24356969 - ], + "pubmed": [7958916, 17146433, 24356969], "qualifier": "involved_in", "term": "positive regulation of transcription by RNA polymerase II" }, @@ -47934,10 +46987,7 @@ "evidence": "IGI", "gocategory": "BP", "id": "GO:0051726", - "pubmed": [ - 12203124, - 12433990 - ], + "pubmed": [12203124, 12433990], "qualifier": "involved_in", "term": "regulation of cell cycle" }, @@ -48051,12 +47101,7 @@ "evidence": "IDA", "gocategory": "BP", "id": "GO:0072331", - "pubmed": [ - 15314173, - 18549481, - 25384516, - 29681526 - ], + "pubmed": [15314173, 18549481, 25384516, 29681526], "qualifier": "involved_in", "term": "signal transduction by p53 class mediator" }, @@ -48064,11 +47109,7 @@ "evidence": "IMP", "gocategory": "BP", "id": "GO:0072332", - "pubmed": [ - 12172011, - 16322561, - 17310983 - ], + "pubmed": [12172011, 16322561, 17310983], "qualifier": "involved_in", "term": "intrinsic apoptotic signaling pathway by p53 class mediator" }, @@ -48241,10 +47282,7 @@ "evidence": "IMP", "gocategory": "BP", "id": "GO:2001244", - "pubmed": [ - 14963330, - 27031958 - ], + "pubmed": [14963330, 27031958], "qualifier": "involved_in", "term": "positive regulation of intrinsic apoptotic signaling pathway" }, @@ -48268,13 +47306,7 @@ "evidence": "IDA", "gocategory": "CC", "id": "GO:0000785", - "pubmed": [ - 15710329, - 17805299, - 22521434, - 23629966, - 24289924 - ], + "pubmed": [15710329, 17805299, 22521434, 23629966, 24289924], "qualifier": "located_in", "term": "chromatin" }, @@ -48297,13 +47329,7 @@ "evidence": "EXP", "gocategory": "CC", "id": "GO:0005634", - "pubmed": [ - 15340061, - 17170702, - 17591690, - 18206965, - 23752197 - ], + "pubmed": [15340061, 17170702, 17591690, 18206965, 23752197], "qualifier": "located_in", "term": "nucleus" }, @@ -48327,25 +47353,8 @@ "gocategory": "CC", "id": "GO:0005634", "pubmed": [ - 7720704, - 14744935, - 14963330, - 15340061, - 16131611, - 16322561, - 16507995, - 17403783, - 18756595, - 19011621, - 19234109, - 20096447, - 20810912, - 21597459, - 22914926, - 24101517, - 24289924, - 24625977, - 26634371 + 7720704, 14744935, 14963330, 15340061, 16131611, 16322561, 16507995, 17403783, 18756595, 19011621, 19234109, + 20096447, 20810912, 21597459, 22914926, 24101517, 24289924, 24625977, 26634371 ], "qualifier": "located_in", "term": "nucleus" @@ -48369,10 +47378,7 @@ "evidence": "IDA", "gocategory": "CC", "id": "GO:0005654", - "pubmed": [ - 11080164, - 12915590 - ], + "pubmed": [11080164, 12915590], "qualifier": "located_in", "term": "nucleoplasm" }, @@ -48410,12 +47416,7 @@ "evidence": "EXP", "gocategory": "CC", "id": "GO:0005737", - "pubmed": [ - 15340061, - 17170702, - 19033443, - 22726440 - ], + "pubmed": [15340061, 17170702, 19033443, 22726440], "qualifier": "located_in", "term": "cytoplasm" }, @@ -48424,17 +47425,7 @@ "gocategory": "CC", "id": "GO:0005737", "pubmed": [ - 7720704, - 14744935, - 15340061, - 16131611, - 19011621, - 20096447, - 20810912, - 21597459, - 23629966, - 24625977, - 26634371 + 7720704, 14744935, 15340061, 16131611, 19011621, 20096447, 20810912, 21597459, 23629966, 24625977, 26634371 ], "qualifier": "located_in", "term": "cytoplasm" @@ -48458,11 +47449,7 @@ "evidence": "IDA", "gocategory": "CC", "id": "GO:0005739", - "pubmed": [ - 12667443, - 24101517, - 25168243 - ], + "pubmed": [12667443, 24101517, 25168243], "qualifier": "located_in", "term": "mitochondrion" }, @@ -48477,11 +47464,7 @@ "evidence": "EXP", "gocategory": "CC", "id": "GO:0005759", - "pubmed": [ - 22726440, - 25168243, - 27323408 - ], + "pubmed": [22726440, 25168243, 27323408], "qualifier": "located_in", "term": "mitochondrial matrix" }, @@ -48526,10 +47509,7 @@ "evidence": "IDA", "gocategory": "CC", "id": "GO:0005829", - "pubmed": [ - 14963330, - 24101517 - ], + "pubmed": [14963330, 24101517], "qualifier": "located_in", "term": "cytosol" }, @@ -48567,10 +47547,7 @@ "evidence": "IDA", "gocategory": "CC", "id": "GO:0016605", - "pubmed": [ - 12006491, - 22869143 - ], + "pubmed": [12006491, 22869143], "qualifier": "located_in", "term": "PML body" }, @@ -48585,10 +47562,7 @@ "evidence": "IPI", "gocategory": "CC", "id": "GO:0017053", - "pubmed": [ - 8875929, - 18677113 - ], + "pubmed": [8875929, 18677113], "qualifier": "part_of", "term": "transcription repressor complex" }, @@ -48596,10 +47570,7 @@ "evidence": "IDA", "gocategory": "CC", "id": "GO:0032991", - "pubmed": [ - 9529249, - 17310983 - ], + "pubmed": [9529249, 17310983], "qualifier": "part_of", "term": "protein-containing complex" }, @@ -48624,11 +47595,7 @@ "category": "MF", "evidence": "IDA", "id": "GO:0000976", - "pubmed": [ - 15710329, - 16131611, - 17996705 - ], + "pubmed": [15710329, 16131611, 17996705], "qualifier": "enables", "term": "transcription cis-regulatory region binding" }, @@ -48657,10 +47624,7 @@ "category": "MF", "evidence": "IDA", "id": "GO:0000978", - "pubmed": [ - 22578566, - 24289924 - ], + "pubmed": [22578566, 24289924], "qualifier": "enables", "term": "RNA polymerase II cis-regulatory region sequence-specific DNA binding" }, @@ -48689,15 +47653,7 @@ "category": "MF", "evidence": "IDA", "id": "GO:0000981", - "pubmed": [ - 17310983, - 24289924, - 24652652, - 34381247, - 35618207, - 36634798, - 38653238 - ], + "pubmed": [17310983, 24289924, 24652652, 34381247, 35618207, 36634798, 38653238], "qualifier": "enables", "term": "DNA-binding transcription factor activity, RNA polymerase II-specific" }, @@ -48752,12 +47708,7 @@ "category": "MF", "evidence": "IDA", "id": "GO:0001228", - "pubmed": [ - 12609999, - 17145718, - 17146433, - 22578566 - ], + "pubmed": [12609999, 17145718, 17146433, 22578566], "qualifier": "enables", "term": "DNA-binding transcription activator activity, RNA polymerase II-specific" }, @@ -48803,10 +47754,7 @@ "category": "MF", "evidence": "IMP", "id": "GO:0003677", - "pubmed": [ - 2144364, - 15629713 - ], + "pubmed": [2144364, 15629713], "qualifier": "enables", "term": "DNA binding" }, @@ -48814,11 +47762,7 @@ "category": "MF", "evidence": "IDA", "id": "GO:0003682", - "pubmed": [ - 16322561, - 17599062, - 26334721 - ], + "pubmed": [16322561, 17599062, 26334721], "qualifier": "enables", "term": "chromatin binding" }, @@ -48833,11 +47777,7 @@ "category": "MF", "evidence": "IDA", "id": "GO:0003700", - "pubmed": [ - 7587074, - 18549481, - 26334721 - ], + "pubmed": [7587074, 18549481, 26334721], "qualifier": "enables", "term": "DNA-binding transcription factor activity" }, @@ -48877,341 +47817,40 @@ "evidence": "IPI", "id": "GO:0005515", "pubmed": [ - 1465435, - 7663514, - 7799929, - 8207801, - 8344494, - 8675009, - 8875926, - 8875929, - 9050995, - 9188558, - 9194564, - 9194565, - 9380510, - 9472015, - 9529249, - 9733515, - 9807817, - 9827557, - 10196247, - 10226625, - 10415337, - 10518217, - 10597287, - 10608892, - 10666337, - 10823891, - 11080164, - 11146555, - 11178989, - 11359905, - 11388671, - 11427532, - 11546806, - 11672523, - 11684014, - 11706030, - 11781842, - 11861836, - 11877378, - 12006491, - 12080348, - 12620801, - 12667443, - 12692135, - 12702766, - 12730672, - 12750254, - 12915590, - 14557665, - 14627987, - 14744935, - 14759370, - 14963330, - 14985081, - 15044383, - 15068796, - 15133049, - 15144954, - 15205477, - 15310821, - 15364927, - 15525938, - 15542844, - 15577914, - 15580310, - 15604276, - 15660129, - 15687255, - 15710329, - 15735003, - 15735006, - 15782130, - 15855171, - 15916963, - 15960975, - 15989956, - 16003391, - 16151013, - 16169070, - 16189514, - 16227626, - 16319068, - 16322561, - 16376338, - 16376884, - 16377624, - 16402859, - 16415881, - 16432196, - 16442532, - 16443602, - 16474402, - 16493710, - 16511572, - 16601686, - 16611888, - 16713569, - 16732283, - 16753148, - 16793543, - 16845383, - 16847267, - 16951253, - 16959611, - 17080083, - 17098746, - 17108971, - 17121812, - 17139261, - 17145718, - 17159902, - 17170702, - 17184779, - 17237821, - 17245430, - 17254968, - 17268548, - 17274640, - 17290220, - 17298945, - 17310983, - 17317671, - 17347673, - 17438265, - 17470788, - 17482142, - 17568776, - 17612295, - 17662718, - 17707234, - 17719541, - 17719542, - 17805299, - 17875722, - 17904127, - 17906639, - 17964266, - 18087040, - 18172499, - 18230339, - 18235501, - 18235502, - 18275817, - 18309296, - 18316739, - 18354501, - 18382127, - 18388957, - 18391200, - 18485870, - 18504427, - 18510931, - 18566590, - 18624398, - 18656471, - 18690848, - 18695251, - 18812399, - 18952844, - 18977328, - 19008854, - 19011621, - 19043414, - 19098711, - 19151705, - 19166840, - 19196987, - 19217391, - 19234109, - 19255450, - 19339993, - 19345189, - 19411066, - 19433796, - 19483087, - 19508870, - 19521340, - 19536131, - 19556538, - 19619542, - 19626115, - 19651603, - 19656744, - 19680552, - 19684601, - 19740107, - 19798103, - 19805293, - 19833129, - 19857493, - 19933256, - 20075864, - 20096447, - 20118233, - 20123734, - 20124405, - 20134482, - 20153329, - 20159018, - 20167603, - 20173098, - 20206173, - 20227041, - 20228809, - 20385133, - 20421506, - 20452352, - 20515689, - 20534433, - 20562916, - 20591429, - 20622899, - 20660729, - 20705607, - 20708156, - 20713054, - 20818423, - 20864041, - 20959462, - 21057547, - 21078964, - 21081126, - 21130767, - 21132010, - 21170034, - 21170087, - 21245319, - 21317932, - 21390126, - 21397192, - 21423215, - 21460856, - 21471221, - 21513714, - 21597459, - 21625211, - 21653829, - 21670263, - 21726810, - 21741598, - 21782458, - 21821029, - 21831840, - 21857681, - 21892170, - 21900206, - 21952639, - 21988832, - 22056774, - 22085928, - 22103682, - 22124327, - 22265415, - 22340593, - 22451927, - 22499945, - 22510990, - 22522597, - 22575647, - 22653443, - 22659184, - 22723347, - 22726440, - 22810585, - 22810586, - 22819825, - 22945289, - 22975381, - 23010591, - 23063560, - 23092970, - 23320542, - 23431171, - 23576507, - 23623661, - 23734815, - 23752197, - 23776060, - 23870121, - 24207125, - 24219989, - 24380853, - 24449765, - 24492002, - 24625977, - 24667498, - 24722188, - 24814347, - 25168243, - 25241761, - 25314079, - 25402006, - 25422469, - 25502805, - 25579814, - 25591766, - 25609649, - 25651062, - 25670079, - 25837623, - 25857266, - 26302407, - 26331536, - 26334721, - 26789255, - 27107012, - 27323408, - 27519799, - 27605672, - 28842590, - 29187402, - 29340707, - 29628311, - 29997244, - 31467278, - 31511497, - 31515488, - 31837246, - 32606738, - 32814053, - 33591310, - 33961781, - 34316702, - 34404770, - 34591612, - 34591642, - 35044719, - 35122041, - 35140242, - 35271311, - 35512704, - 36897777, - 36931259, - 39009827, - 40205054 + 1465435, 7663514, 7799929, 8207801, 8344494, 8675009, 8875926, 8875929, 9050995, 9188558, 9194564, 9194565, + 9380510, 9472015, 9529249, 9733515, 9807817, 9827557, 10196247, 10226625, 10415337, 10518217, 10597287, + 10608892, 10666337, 10823891, 11080164, 11146555, 11178989, 11359905, 11388671, 11427532, 11546806, + 11672523, 11684014, 11706030, 11781842, 11861836, 11877378, 12006491, 12080348, 12620801, 12667443, + 12692135, 12702766, 12730672, 12750254, 12915590, 14557665, 14627987, 14744935, 14759370, 14963330, + 14985081, 15044383, 15068796, 15133049, 15144954, 15205477, 15310821, 15364927, 15525938, 15542844, + 15577914, 15580310, 15604276, 15660129, 15687255, 15710329, 15735003, 15735006, 15782130, 15855171, + 15916963, 15960975, 15989956, 16003391, 16151013, 16169070, 16189514, 16227626, 16319068, 16322561, + 16376338, 16376884, 16377624, 16402859, 16415881, 16432196, 16442532, 16443602, 16474402, 16493710, + 16511572, 16601686, 16611888, 16713569, 16732283, 16753148, 16793543, 16845383, 16847267, 16951253, + 16959611, 17080083, 17098746, 17108971, 17121812, 17139261, 17145718, 17159902, 17170702, 17184779, + 17237821, 17245430, 17254968, 17268548, 17274640, 17290220, 17298945, 17310983, 17317671, 17347673, + 17438265, 17470788, 17482142, 17568776, 17612295, 17662718, 17707234, 17719541, 17719542, 17805299, + 17875722, 17904127, 17906639, 17964266, 18087040, 18172499, 18230339, 18235501, 18235502, 18275817, + 18309296, 18316739, 18354501, 18382127, 18388957, 18391200, 18485870, 18504427, 18510931, 18566590, + 18624398, 18656471, 18690848, 18695251, 18812399, 18952844, 18977328, 19008854, 19011621, 19043414, + 19098711, 19151705, 19166840, 19196987, 19217391, 19234109, 19255450, 19339993, 19345189, 19411066, + 19433796, 19483087, 19508870, 19521340, 19536131, 19556538, 19619542, 19626115, 19651603, 19656744, + 19680552, 19684601, 19740107, 19798103, 19805293, 19833129, 19857493, 19933256, 20075864, 20096447, + 20118233, 20123734, 20124405, 20134482, 20153329, 20159018, 20167603, 20173098, 20206173, 20227041, + 20228809, 20385133, 20421506, 20452352, 20515689, 20534433, 20562916, 20591429, 20622899, 20660729, + 20705607, 20708156, 20713054, 20818423, 20864041, 20959462, 21057547, 21078964, 21081126, 21130767, + 21132010, 21170034, 21170087, 21245319, 21317932, 21390126, 21397192, 21423215, 21460856, 21471221, + 21513714, 21597459, 21625211, 21653829, 21670263, 21726810, 21741598, 21782458, 21821029, 21831840, + 21857681, 21892170, 21900206, 21952639, 21988832, 22056774, 22085928, 22103682, 22124327, 22265415, + 22340593, 22451927, 22499945, 22510990, 22522597, 22575647, 22653443, 22659184, 22723347, 22726440, + 22810585, 22810586, 22819825, 22945289, 22975381, 23010591, 23063560, 23092970, 23320542, 23431171, + 23576507, 23623661, 23734815, 23752197, 23776060, 23870121, 24207125, 24219989, 24380853, 24449765, + 24492002, 24625977, 24667498, 24722188, 24814347, 25168243, 25241761, 25314079, 25402006, 25422469, + 25502805, 25579814, 25591766, 25609649, 25651062, 25670079, 25837623, 25857266, 26302407, 26331536, + 26334721, 26789255, 27107012, 27323408, 27519799, 27605672, 28842590, 29187402, 29340707, 29628311, + 29997244, 31467278, 31511497, 31515488, 31837246, 32606738, 32814053, 33591310, 33961781, 34316702, + 34404770, 34591612, 34591642, 35044719, 35122041, 35140242, 35271311, 35512704, 36897777, 36931259, + 39009827, 40205054 ], "qualifier": "enables", "term": "protein binding" @@ -49283,30 +47922,9 @@ "evidence": "IPI", "id": "GO:0042802", "pubmed": [ - 10876243, - 14759370, - 14985081, - 15629713, - 16291740, - 16461914, - 17612295, - 17620598, - 18087040, - 19011621, - 19339993, - 19667193, - 20004160, - 20159469, - 20364130, - 21178074, - 21522129, - 21988832, - 22653443, - 22972749, - 25402006, - 25609649, - 31837246, - 35512704 + 10876243, 14759370, 14985081, 15629713, 16291740, 16461914, 17612295, 17620598, 18087040, 19011621, + 19339993, 19667193, 20004160, 20159469, 20364130, 21178074, 21522129, 21988832, 22653443, 22972749, + 25402006, 25609649, 31837246, 35512704 ], "qualifier": "enables", "term": "identical protein binding" @@ -49331,10 +47949,7 @@ "category": "MF", "evidence": "IPI", "id": "GO:0051087", - "pubmed": [ - 15358771, - 18086682 - ], + "pubmed": [15358771, 18086682], "qualifier": "enables", "term": "protein-folding chaperone binding" }, @@ -49350,12 +47965,7 @@ "category": "MF", "evidence": "IPI", "id": "GO:0061629", - "pubmed": [ - 15705871, - 18549481, - 19505873, - 23329847 - ], + "pubmed": [15705871, 18549481, 19505873, 23329847], "qualifier": "enables", "term": "RNA polymerase II-specific DNA-binding transcription factor binding" }, @@ -49394,10 +48004,7 @@ "category": "MF", "evidence": "EXP", "id": "GO:0140677", - "pubmed": [ - 14759370, - 16793543 - ], + "pubmed": [14759370, 16793543], "qualifier": "enables", "term": "molecular function activator activity" }, @@ -49413,13 +48020,7 @@ "category": "MF", "evidence": "IDA", "id": "GO:0140693", - "pubmed": [ - 31953488, - 35618207, - 36108750, - 36634798, - 38653238 - ], + "pubmed": [31953488, 35618207, 36108750, 36634798, 38653238], "qualifier": "enables", "term": "molecular condensate scaffold activity" }, @@ -49434,11 +48035,7 @@ "category": "MF", "evidence": "IDA", "id": "GO:1990841", - "pubmed": [ - 20725088, - 24356969, - 24652652 - ], + "pubmed": [20725088, 24356969, 24652652], "qualifier": "enables", "term": "promoter-specific chromatin binding" } @@ -49446,42 +48043,15 @@ }, "homologene": { "genes": [ - [ - 7955, - 30590 - ], - [ - 8364, - 431679 - ], - [ - 9544, - 716170 - ], - [ - 9598, - 455214 - ], - [ - 9606, - 7157 - ], - [ - 9615, - 403869 - ], - [ - 9913, - 281542 - ], - [ - 10090, - 22059 - ], - [ - 10116, - 24842 - ] + [7955, 30590], + [8364, 431679], + [9544, 716170], + [9598, 455214], + [9606, 7157], + [9615, 403869], + [9913, 281542], + [10090, 22059], + [10116, 24842] ], "id": 460 }, @@ -51016,12 +49586,7 @@ "9R2P", "9R2Q" ], - "pfam": [ - "PF00870", - "PF07710", - "PF08563", - "PF18521" - ], + "pfam": ["PF00870", "PF07710", "PF08563", "PF18521"], "pharmgkb": "PA36679", "pharos": [ { @@ -51253,11 +49818,7 @@ ] }, "refseq": { - "genomic": [ - "NC_000017.11", - "NC_060941.1", - "NG_017013.2" - ], + "genomic": ["NC_000017.11", "NC_060941.1", "NG_017013.2"], "protein": [ "NP_000537.3", "NP_001119584.1", @@ -51417,28 +49978,14 @@ ] }, "reporter": { - "HG-U133_Plus_2": [ - "201746_at", - "211300_s_at" - ], - "HG-U95Av2": [ - "1939_at", - "1974_s_at", - "31618_at" - ], - "HTA-2_0": [ - "TC17001094.hg.1", - "TC17002468.hg.1" - ], + "HG-U133_Plus_2": ["201746_at", "211300_s_at"], + "HG-U95Av2": ["1939_at", "1974_s_at", "31618_at"], + "HTA-2_0": ["TC17001094.hg.1", "TC17002468.hg.1"], "HuEx-1_0": "3743906", "HuGene-1_1": "8012257", "HuGene-2_1": "16840732" }, - "retired": [ - 146749, - 201237, - 553989 - ], + "retired": [146749, 201237, 553989], "summary": "This gene encodes a tumor suppressor protein containing transcriptional activation, DNA binding, and oligomerization domains. The encoded protein responds to diverse cellular stresses to regulate expression of target genes, thereby inducing cell cycle arrest, apoptosis, senescence, DNA repair, or changes in metabolism. Mutations in this gene are associated with a variety of human cancers, including hereditary cancers such as Li-Fraumeni syndrome. Alternative splicing of this gene and the use of alternate promoters result in multiple transcript variants and isoforms. Additional isoforms have also been shown to result from the use of alternate translation initiation codons from identical transcript variants (PMIDs: 12032546, 20937277). [provided by RefSeq, Dec 2016].", "symbol": "TP53", "taxid": 9606, @@ -51446,13 +49993,8 @@ "umls": { "cui": "C0079419" }, - "unigene": [ - "Hs.740601", - "Hs.437460" - ], - "unii": [ - "P0MIG0N03Z" - ], + "unigene": ["Hs.740601", "Hs.437460"], + "unii": ["P0MIG0N03Z"], "uniprot": { "Swiss-Prot": "P04637", "TrEMBL": [ @@ -51479,4 +50021,4 @@ "url_stub": "P53" } } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/ncbi-gene.json b/backend/cli/test/science/fixtures/fetch/ncbi-gene.json index 75d1d4f0..3e7de495 100644 --- a/backend/cli/test/science/fixtures/fetch/ncbi-gene.json +++ b/backend/cli/test/science/fixtures/fetch/ncbi-gene.json @@ -14,9 +14,7 @@ "nomenclaturesymbol": "TP53", "nomenclaturename": "tumor protein p53", "nomenclaturestatus": "Official", - "mim": [ - "191170" - ], + "mim": ["191170"], "genomicinfo": [ { "chrloc": "17", @@ -150,4 +148,4 @@ } ] } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/openalex.json b/backend/cli/test/science/fixtures/fetch/openalex.json index 994e2fc0..9b1a4f6e 100644 --- a/backend/cli/test/science/fixtures/fetch/openalex.json +++ b/backend/cli/test/science/fixtures/fetch/openalex.json @@ -23,20 +23,14 @@ "id": "https://openalex.org/S1983995261", "display_name": "PeerJ", "issn_l": "2167-8359", - "issn": [ - "2167-8359" - ], + "issn": ["2167-8359"], "is_oa": true, "is_in_doaj": true, "is_core": true, "host_organization": "https://openalex.org/P4310320104", "host_organization_name": "PeerJ, Inc.", - "host_organization_lineage": [ - "https://openalex.org/P4310320104" - ], - "host_organization_lineage_names": [ - "PeerJ, Inc." - ], + "host_organization_lineage": ["https://openalex.org/P4310320104"], + "host_organization_lineage_names": ["PeerJ, Inc."], "type": "journal" }, "license": "cc-by", @@ -48,12 +42,7 @@ "raw_type": "journal-article" }, "type": "book-chapter", - "indexed_in": [ - "crossref", - "datacite", - "doaj", - "pubmed" - ], + "indexed_in": ["crossref", "datacite", "doaj", "pubmed"], "open_access": { "is_oa": true, "oa_status": "gold", @@ -75,9 +64,7 @@ "ror": "https://ror.org/02nr0ka47", "country_code": "CA", "type": "nonprofit", - "lineage": [ - "https://openalex.org/I4200000001" - ] + "lineage": ["https://openalex.org/I4200000001"] }, { "id": "https://openalex.org/I4210166736", @@ -85,28 +72,18 @@ "ror": "https://ror.org/05ppvf150", "country_code": "US", "type": "company", - "lineage": [ - "https://openalex.org/I4210166736" - ] + "lineage": ["https://openalex.org/I4210166736"] } ], - "countries": [ - "CA", - "US" - ], + "countries": ["CA", "US"], "is_corresponding": false, "raw_author_name": "Heather Piwowar", - "raw_affiliation_strings": [ - "Impactstory, Sanford, NC, USA" - ], + "raw_affiliation_strings": ["Impactstory, Sanford, NC, USA"], "raw_orcid": null, "affiliations": [ { "raw_affiliation_string": "Impactstory, Sanford, NC, USA", - "institution_ids": [ - "https://openalex.org/I4200000001", - "https://openalex.org/I4210166736" - ] + "institution_ids": ["https://openalex.org/I4200000001", "https://openalex.org/I4210166736"] } ] }, @@ -124,9 +101,7 @@ "ror": "https://ror.org/02nr0ka47", "country_code": "CA", "type": "nonprofit", - "lineage": [ - "https://openalex.org/I4200000001" - ] + "lineage": ["https://openalex.org/I4200000001"] }, { "id": "https://openalex.org/I4210166736", @@ -134,28 +109,18 @@ "ror": "https://ror.org/05ppvf150", "country_code": "US", "type": "company", - "lineage": [ - "https://openalex.org/I4210166736" - ] + "lineage": ["https://openalex.org/I4210166736"] } ], - "countries": [ - "CA", - "US" - ], + "countries": ["CA", "US"], "is_corresponding": false, "raw_author_name": "Jason Priem", - "raw_affiliation_strings": [ - "Impactstory, Sanford, NC, USA" - ], + "raw_affiliation_strings": ["Impactstory, Sanford, NC, USA"], "raw_orcid": null, "affiliations": [ { "raw_affiliation_string": "Impactstory, Sanford, NC, USA", - "institution_ids": [ - "https://openalex.org/I4200000001", - "https://openalex.org/I4210166736" - ] + "institution_ids": ["https://openalex.org/I4200000001", "https://openalex.org/I4210166736"] } ] }, @@ -173,10 +138,7 @@ "ror": "https://ror.org/002rjbv21", "country_code": "CA", "type": "education", - "lineage": [ - "https://openalex.org/I159129438", - "https://openalex.org/I49663120" - ] + "lineage": ["https://openalex.org/I159129438", "https://openalex.org/I49663120"] }, { "id": "https://openalex.org/I70931966", @@ -184,14 +146,10 @@ "ror": "https://ror.org/0161xgx34", "country_code": "CA", "type": "education", - "lineage": [ - "https://openalex.org/I70931966" - ] + "lineage": ["https://openalex.org/I70931966"] } ], - "countries": [ - "CA" - ], + "countries": ["CA"], "is_corresponding": false, "raw_author_name": "Vincent Larivière", "raw_affiliation_strings": [ @@ -202,15 +160,11 @@ "affiliations": [ { "raw_affiliation_string": "Observatoire des Sciences et des Technologies (OST), Centre Interuniversitaire de Recherche sur la Science et la Technologie (CIRST), Université du Québec à Montréal, Montréal, QC, Canada", - "institution_ids": [ - "https://openalex.org/I159129438" - ] + "institution_ids": ["https://openalex.org/I159129438"] }, { "raw_affiliation_string": "École de bibliothéconomie et des sciences de l’information, Université de Montréal, Montréal, QC, Canada", - "institution_ids": [ - "https://openalex.org/I70931966" - ] + "institution_ids": ["https://openalex.org/I70931966"] } ] }, @@ -228,9 +182,7 @@ "ror": "https://ror.org/0213rcc28", "country_code": "CA", "type": "education", - "lineage": [ - "https://openalex.org/I18014758" - ] + "lineage": ["https://openalex.org/I18014758"] }, { "id": "https://openalex.org/I4387153203", @@ -238,15 +190,10 @@ "ror": "https://ror.org/05ek4tb53", "country_code": null, "type": "other", - "lineage": [ - "https://openalex.org/I18014758", - "https://openalex.org/I4387153203" - ] + "lineage": ["https://openalex.org/I18014758", "https://openalex.org/I4387153203"] } ], - "countries": [ - "CA" - ], + "countries": ["CA"], "is_corresponding": false, "raw_author_name": "Juan Pablo Alperin", "raw_affiliation_strings": [ @@ -257,15 +204,11 @@ "affiliations": [ { "raw_affiliation_string": "Canadian Institute for Studies in Publishing, Simon Fraser University, Vancouver, BC, Canada", - "institution_ids": [ - "https://openalex.org/I18014758" - ] + "institution_ids": ["https://openalex.org/I18014758"] }, { "raw_affiliation_string": "Public Knowledge Project, Canada", - "institution_ids": [ - "https://openalex.org/I4387153203" - ] + "institution_ids": ["https://openalex.org/I4387153203"] } ] }, @@ -283,26 +226,18 @@ "ror": "https://ror.org/0213rcc28", "country_code": "CA", "type": "education", - "lineage": [ - "https://openalex.org/I18014758" - ] + "lineage": ["https://openalex.org/I18014758"] } ], - "countries": [ - "CA" - ], + "countries": ["CA"], "is_corresponding": false, "raw_author_name": "Lisa Matthias", - "raw_affiliation_strings": [ - "Scholarly Communications Lab, Simon Fraser University, Vancouver, Canada" - ], + "raw_affiliation_strings": ["Scholarly Communications Lab, Simon Fraser University, Vancouver, Canada"], "raw_orcid": null, "affiliations": [ { "raw_affiliation_string": "Scholarly Communications Lab, Simon Fraser University, Vancouver, Canada", - "institution_ids": [ - "https://openalex.org/I18014758" - ] + "institution_ids": ["https://openalex.org/I18014758"] } ] }, @@ -320,9 +255,7 @@ "ror": "https://ror.org/00cvxb145", "country_code": "US", "type": "education", - "lineage": [ - "https://openalex.org/I201448701" - ] + "lineage": ["https://openalex.org/I201448701"] }, { "id": "https://openalex.org/I58610484", @@ -330,20 +263,13 @@ "ror": "https://ror.org/02jqc0m91", "country_code": "US", "type": "education", - "lineage": [ - "https://openalex.org/I58610484" - ] + "lineage": ["https://openalex.org/I58610484"] } ], - "countries": [ - "US" - ], + "countries": ["US"], "is_corresponding": false, "raw_author_name": "Bree Norlander", - "raw_affiliation_strings": [ - "FlourishOA, USA", - "Information School, University of Washington, Seattle, USA" - ], + "raw_affiliation_strings": ["FlourishOA, USA", "Information School, University of Washington, Seattle, USA"], "raw_orcid": null, "affiliations": [ { @@ -352,10 +278,7 @@ }, { "raw_affiliation_string": "Information School, University of Washington, Seattle, USA", - "institution_ids": [ - "https://openalex.org/I201448701", - "https://openalex.org/I58610484" - ] + "institution_ids": ["https://openalex.org/I201448701", "https://openalex.org/I58610484"] } ] }, @@ -373,9 +296,7 @@ "ror": "https://ror.org/00cvxb145", "country_code": "US", "type": "education", - "lineage": [ - "https://openalex.org/I201448701" - ] + "lineage": ["https://openalex.org/I201448701"] }, { "id": "https://openalex.org/I58610484", @@ -383,20 +304,13 @@ "ror": "https://ror.org/02jqc0m91", "country_code": "US", "type": "education", - "lineage": [ - "https://openalex.org/I58610484" - ] + "lineage": ["https://openalex.org/I58610484"] } ], - "countries": [ - "US" - ], + "countries": ["US"], "is_corresponding": false, "raw_author_name": "Ashley Farley", - "raw_affiliation_strings": [ - "FlourishOA, USA", - "Information School, University of Washington, Seattle, USA" - ], + "raw_affiliation_strings": ["FlourishOA, USA", "Information School, University of Washington, Seattle, USA"], "raw_orcid": null, "affiliations": [ { @@ -405,10 +319,7 @@ }, { "raw_affiliation_string": "Information School, University of Washington, Seattle, USA", - "institution_ids": [ - "https://openalex.org/I201448701", - "https://openalex.org/I58610484" - ] + "institution_ids": ["https://openalex.org/I201448701", "https://openalex.org/I58610484"] } ] }, @@ -426,9 +337,7 @@ "ror": "https://ror.org/00cvxb145", "country_code": "US", "type": "education", - "lineage": [ - "https://openalex.org/I201448701" - ] + "lineage": ["https://openalex.org/I201448701"] }, { "id": "https://openalex.org/I58610484", @@ -436,27 +345,18 @@ "ror": "https://ror.org/02jqc0m91", "country_code": "US", "type": "education", - "lineage": [ - "https://openalex.org/I58610484" - ] + "lineage": ["https://openalex.org/I58610484"] } ], - "countries": [ - "US" - ], + "countries": ["US"], "is_corresponding": false, "raw_author_name": "Jevin West", - "raw_affiliation_strings": [ - "Information School, University of Washington, Seattle, USA" - ], + "raw_affiliation_strings": ["Information School, University of Washington, Seattle, USA"], "raw_orcid": null, "affiliations": [ { "raw_affiliation_string": "Information School, University of Washington, Seattle, USA", - "institution_ids": [ - "https://openalex.org/I201448701", - "https://openalex.org/I58610484" - ] + "institution_ids": ["https://openalex.org/I201448701", "https://openalex.org/I58610484"] } ] }, @@ -474,9 +374,7 @@ "ror": "https://ror.org/03c4mmv16", "country_code": "CA", "type": "education", - "lineage": [ - "https://openalex.org/I153718931" - ] + "lineage": ["https://openalex.org/I153718931"] }, { "id": "https://openalex.org/I159129438", @@ -484,15 +382,10 @@ "ror": "https://ror.org/002rjbv21", "country_code": "CA", "type": "education", - "lineage": [ - "https://openalex.org/I159129438", - "https://openalex.org/I49663120" - ] + "lineage": ["https://openalex.org/I159129438", "https://openalex.org/I49663120"] } ], - "countries": [ - "CA" - ], + "countries": ["CA"], "is_corresponding": false, "raw_author_name": "Stefanie Haustein", "raw_affiliation_strings": [ @@ -503,15 +396,11 @@ "affiliations": [ { "raw_affiliation_string": "Observatoire des Sciences et des Technologies (OST), Centre Interuniversitaire de Recherche sur la Science et la Technologie (CIRST), Université du Québec à Montréal, Montréal, QC, Canada", - "institution_ids": [ - "https://openalex.org/I159129438" - ] + "institution_ids": ["https://openalex.org/I159129438"] }, { "raw_affiliation_string": "School of Information Studies, University of Ottawa, Ottawa, ON, Canada", - "institution_ids": [ - "https://openalex.org/I153718931" - ] + "institution_ids": ["https://openalex.org/I153718931"] } ] } @@ -851,20 +740,14 @@ "id": "https://openalex.org/S1983995261", "display_name": "PeerJ", "issn_l": "2167-8359", - "issn": [ - "2167-8359" - ], + "issn": ["2167-8359"], "is_oa": true, "is_in_doaj": true, "is_core": true, "host_organization": "https://openalex.org/P4310320104", "host_organization_name": "PeerJ, Inc.", - "host_organization_lineage": [ - "https://openalex.org/P4310320104" - ], - "host_organization_lineage_names": [ - "PeerJ, Inc." - ], + "host_organization_lineage": ["https://openalex.org/P4310320104"], + "host_organization_lineage_names": ["PeerJ, Inc."], "type": "journal" }, "license": "cc-by", @@ -890,9 +773,7 @@ "is_core": false, "host_organization": "https://openalex.org/I1299303238", "host_organization_name": "National Institutes of Health", - "host_organization_lineage": [ - "https://openalex.org/I1299303238" - ], + "host_organization_lineage": ["https://openalex.org/I1299303238"], "host_organization_lineage_names": [], "type": "repository" }, @@ -919,9 +800,7 @@ "is_core": false, "host_organization": "https://openalex.org/I114395901", "host_organization_name": "University of Nebraska–Lincoln", - "host_organization_lineage": [ - "https://openalex.org/I114395901" - ], + "host_organization_lineage": ["https://openalex.org/I114395901"], "host_organization_lineage_names": [], "type": "repository" }, @@ -948,9 +827,7 @@ "is_core": false, "host_organization": "https://openalex.org/I1294671590", "host_organization_name": "Centre National de la Recherche Scientifique", - "host_organization_lineage": [ - "https://openalex.org/I1294671590" - ], + "host_organization_lineage": ["https://openalex.org/I1294671590"], "host_organization_lineage_names": [], "type": "repository" }, @@ -1004,9 +881,7 @@ "is_core": false, "host_organization": "https://openalex.org/I1303153112", "host_organization_name": "European Bioinformatics Institute", - "host_organization_lineage": [ - "https://openalex.org/I1303153112" - ], + "host_organization_lineage": ["https://openalex.org/I1303153112"], "host_organization_lineage_names": [], "type": "repository" }, @@ -1033,9 +908,7 @@ "is_core": false, "host_organization": "https://openalex.org/I70931966", "host_organization_name": "Université de Montréal", - "host_organization_lineage": [ - "https://openalex.org/I70931966" - ], + "host_organization_lineage": ["https://openalex.org/I70931966"], "host_organization_lineage_names": [], "type": "repository" }, @@ -1062,9 +935,7 @@ "is_core": false, "host_organization": "https://openalex.org/I18014758", "host_organization_name": "Simon Fraser University", - "host_organization_lineage": [ - "https://openalex.org/I18014758" - ], + "host_organization_lineage": ["https://openalex.org/I18014758"], "host_organization_lineage_names": [], "type": "repository" }, @@ -1091,9 +962,7 @@ "is_core": false, "host_organization": "https://openalex.org/I67311998", "host_organization_name": "European Organization for Nuclear Research", - "host_organization_lineage": [ - "https://openalex.org/I67311998" - ], + "host_organization_lineage": ["https://openalex.org/I67311998"], "host_organization_lineage_names": [], "type": "repository" }, @@ -1115,20 +984,14 @@ "id": "https://openalex.org/S1983995261", "display_name": "PeerJ", "issn_l": "2167-8359", - "issn": [ - "2167-8359" - ], + "issn": ["2167-8359"], "is_oa": true, "is_in_doaj": true, "is_core": true, "host_organization": "https://openalex.org/P4310320104", "host_organization_name": "PeerJ, Inc.", - "host_organization_lineage": [ - "https://openalex.org/P4310320104" - ], - "host_organization_lineage_names": [ - "PeerJ, Inc." - ], + "host_organization_lineage": ["https://openalex.org/P4310320104"], + "host_organization_lineage_names": ["PeerJ, Inc."], "type": "journal" }, "license": "cc-by", @@ -1217,598 +1080,173 @@ "https://openalex.org/W2608652318" ], "abstract_inverted_index": { - "67": [ - 43 - ], - "Despite": [ - 0 - ], - "growing": [ - 1 - ], - "interest": [ - 2 - ], - "in": [ - 3, - 57, - 73, - 110, - 122 - ], - "Open": [ - 4, - 201 - ], - "Access": [ - 5 - ], - "(OA)": [ - 6 - ], - "to": [ - 7, - 54, - 252 - ], - "scholarly": [ - 8, - 105 - ], - "literature,": [ - 9 - ], - "there": [ - 10 - ], - "is": [ - 11, - 107, - 116, - 176 - ], - "an": [ - 12, - 34, - 85, - 185, - 199, - 231 - ], - "unmet": [ - 13 - ], - "need": [ - 14, - 31 - ], - "for": [ - 15, - 42, - 174, - 219 - ], - "large-scale,": [ - 16 - ], - "up-to-date,": [ - 17 - ], - "and": [ - 18, - 24, - 77, - 112, - 124, - 144, - 221, - 237, - 256 - ], - "reproducible": [ - 19 - ], - "studies": [ - 20 - ], - "assessing": [ - 21 - ], - "the": [ - 22, - 104, - 134, - 145, - 170, - 195, - 206, - 213, - 245 - ], - "prevalence": [ - 23 - ], - "characteristics": [ - 25 - ], - "of": [ - 26, - 51, - 75, - 83, - 103, - 137, - 141, - 163, - 209 - ], - "OA.": [ - 27, - 168, - 239 - ], - "We": [ - 28, - 46, - 97, - 203, - 240 - ], - "address": [ - 29 - ], - "this": [ - 30, - 114, - 142 - ], - "using": [ - 32, - 95, - 244 - ], - "oaDOI,": [ - 33 - ], - "open": [ - 35 - ], - "online": [ - 36 - ], - "service": [ - 37 - ], - "that": [ - 38, - 89, - 99, - 113, - 147, - 155 - ], - "determines": [ - 39 - ], - "OA": [ - 40, - 56, - 93, - 108, - 138, - 159, - 175, - 210, - 223, - 254 - ], - "status": [ - 41 - ], - "million": [ - 44 - ], - "articles.": [ - 45 - ], - "use": [ - 47 - ], - "three": [ - 48, - 58 - ], - "samples,": [ - 49 - ], - "each": [ - 50 - ], - "100,000": [ - 52 - ], - "articles,": [ - 53, - 152, - 211 - ], - "investigate": [ - 55 - ], - "populations:": [ - 59 - ], - "(1)": [ - 60 - ], - "all": [ - 61 - ], - "journal": [ - 62, - 70 - ], - "articles": [ - 63, - 71, - 79, - 94, - 164, - 191, - 224 - ], - "assigned": [ - 64 - ], - "a": [ - 65, - 250 - ], - "Crossref": [ - 66 - ], - "DOI,": [ - 67 - ], - "(2)": [ - 68 - ], - "recent": [ - 69, - 128 - ], - "indexed": [ - 72 - ], - "Web": [ - 74 - ], - "Science,": [ - 76 - ], - "(3)": [ - 78 - ], - "viewed": [ - 80 - ], - "by": [ - 81, - 120, - 235 - ], - "users": [ - 82, - 91, - 157 - ], - "Unpaywall,": [ - 84 - ], - "open-source": [ - 86 - ], - "browser": [ - 87 - ], - "extension": [ - 88 - ], - "lets": [ - 90 - ], - "find": [ - 92, - 154 - ], - "oaDOI.": [ - 96 - ], - "estimate": [ - 98 - ], - "at": [ - 100 - ], - "least": [ - 101 - ], - "28%": [ - 102 - ], - "literature": [ - 106 - ], - "(19M": [ - 109 - ], - "total)": [ - 111 - ], - "proportion": [ - 115 - ], - "growing,": [ - 117 - ], - "driven": [ - 118, - 233 - ], - "particularly": [ - 119 - ], - "growth": [ - 121 - ], - "Gold": [ - 123 - ], - "Hybrid.": [ - 125 - ], - "The": [ - 126 - ], - "most": [ - 127, - 171 - ], - "year": [ - 129 - ], - "analyzed": [ - 130 - ], - "(2015)": [ - 131 - ], - "also": [ - 132, - 204 - ], - "has": [ - 133 - ], - "highest": [ - 135 - ], - "percentage": [ - 136 - ], - "(45%).": [ - 139 - ], - "Because": [ - 140 - ], - "growth,": [ - 143 - ], - "fact": [ - 146 - ], - "readers": [ - 148 - ], - "disproportionately": [ - 149 - ], - "access": [ - 150 - ], - "newer": [ - 151 - ], - "we": [ - 153, - 188 - ], - "Unpaywall": [ - 156 - ], - "encounter": [ - 158 - ], - "quite": [ - 160 - ], - "frequently:": [ - 161 - ], - "47%": [ - 162 - ], - "they": [ - 165 - ], - "view": [ - 166 - ], - "are": [ - 167 - ], - "Notably,": [ - 169 - ], - "common": [ - 172 - ], - "mechanism": [ - 173 - ], - "not": [ - 177 - ], - "Gold,": [ - 178 - ], - "Green,": [ - 179 - ], - "or": [ - 180 - ], - "Hybrid": [ - 181, - 238 - ], - "OA,": [ - 182 - ], - "but": [ - 183 - ], - "rather": [ - 184 - ], - "under-discussed": [ - 186 - ], - "category": [ - 187 - ], - "dub": [ - 189 - ], - "Bronze:": [ - 190 - ], - "made": [ - 192 - ], - "free-to-read": [ - 193 - ], - "on": [ - 194 - ], - "publisher": [ - 196 - ], - "website,": [ - 197 - ], - "without": [ - 198 - ], - "explicit": [ - 200 - ], - "license.": [ - 202 - ], - "examine": [ - 205 - ], - "citation": [ - 207, - 216 - ], - "impact": [ - 208 - ], - "corroborating": [ - 212 - ], - "so-called": [ - 214 - ], - "open-access": [ - 215 - ], - "advantage:": [ - 217 - ], - "accounting": [ - 218 - ], - "age": [ - 220 - ], - "discipline,": [ - 222 - ], - "receive": [ - 225 - ], - "18%": [ - 226 - ], - "more": [ - 227 - ], - "citations": [ - 228 - ], - "than": [ - 229 - ], - "average,": [ - 230 - ], - "effect": [ - 232 - ], - "primarily": [ - 234 - ], - "Green": [ - 236 - ], - "encourage": [ - 241 - ], - "further": [ - 242 - ], - "research": [ - 243 - ], - "free": [ - 246 - ], - "oaDOI": [ - 247 - ], - "service,": [ - 248 - ], - "as": [ - 249 - ], - "way": [ - 251 - ], - "inform": [ - 253 - ], - "policy": [ - 255 - ], - "practice.": [ - 257 - ] + "67": [43], + "Despite": [0], + "growing": [1], + "interest": [2], + "in": [3, 57, 73, 110, 122], + "Open": [4, 201], + "Access": [5], + "(OA)": [6], + "to": [7, 54, 252], + "scholarly": [8, 105], + "literature,": [9], + "there": [10], + "is": [11, 107, 116, 176], + "an": [12, 34, 85, 185, 199, 231], + "unmet": [13], + "need": [14, 31], + "for": [15, 42, 174, 219], + "large-scale,": [16], + "up-to-date,": [17], + "and": [18, 24, 77, 112, 124, 144, 221, 237, 256], + "reproducible": [19], + "studies": [20], + "assessing": [21], + "the": [22, 104, 134, 145, 170, 195, 206, 213, 245], + "prevalence": [23], + "characteristics": [25], + "of": [26, 51, 75, 83, 103, 137, 141, 163, 209], + "OA.": [27, 168, 239], + "We": [28, 46, 97, 203, 240], + "address": [29], + "this": [30, 114, 142], + "using": [32, 95, 244], + "oaDOI,": [33], + "open": [35], + "online": [36], + "service": [37], + "that": [38, 89, 99, 113, 147, 155], + "determines": [39], + "OA": [40, 56, 93, 108, 138, 159, 175, 210, 223, 254], + "status": [41], + "million": [44], + "articles.": [45], + "use": [47], + "three": [48, 58], + "samples,": [49], + "each": [50], + "100,000": [52], + "articles,": [53, 152, 211], + "investigate": [55], + "populations:": [59], + "(1)": [60], + "all": [61], + "journal": [62, 70], + "articles": [63, 71, 79, 94, 164, 191, 224], + "assigned": [64], + "a": [65, 250], + "Crossref": [66], + "DOI,": [67], + "(2)": [68], + "recent": [69, 128], + "indexed": [72], + "Web": [74], + "Science,": [76], + "(3)": [78], + "viewed": [80], + "by": [81, 120, 235], + "users": [82, 91, 157], + "Unpaywall,": [84], + "open-source": [86], + "browser": [87], + "extension": [88], + "lets": [90], + "find": [92, 154], + "oaDOI.": [96], + "estimate": [98], + "at": [100], + "least": [101], + "28%": [102], + "literature": [106], + "(19M": [109], + "total)": [111], + "proportion": [115], + "growing,": [117], + "driven": [118, 233], + "particularly": [119], + "growth": [121], + "Gold": [123], + "Hybrid.": [125], + "The": [126], + "most": [127, 171], + "year": [129], + "analyzed": [130], + "(2015)": [131], + "also": [132, 204], + "has": [133], + "highest": [135], + "percentage": [136], + "(45%).": [139], + "Because": [140], + "growth,": [143], + "fact": [146], + "readers": [148], + "disproportionately": [149], + "access": [150], + "newer": [151], + "we": [153, 188], + "Unpaywall": [156], + "encounter": [158], + "quite": [160], + "frequently:": [161], + "47%": [162], + "they": [165], + "view": [166], + "are": [167], + "Notably,": [169], + "common": [172], + "mechanism": [173], + "not": [177], + "Gold,": [178], + "Green,": [179], + "or": [180], + "Hybrid": [181, 238], + "OA,": [182], + "but": [183], + "rather": [184], + "under-discussed": [186], + "category": [187], + "dub": [189], + "Bronze:": [190], + "made": [192], + "free-to-read": [193], + "on": [194], + "publisher": [196], + "website,": [197], + "without": [198], + "explicit": [200], + "license.": [202], + "examine": [205], + "citation": [207, 216], + "impact": [208], + "corroborating": [212], + "so-called": [214], + "open-access": [215], + "advantage:": [217], + "accounting": [218], + "age": [220], + "discipline,": [222], + "receive": [225], + "18%": [226], + "more": [227], + "citations": [228], + "than": [229], + "average,": [230], + "effect": [232], + "primarily": [234], + "Green": [236], + "encourage": [241], + "further": [242], + "research": [243], + "free": [246], + "oaDOI": [247], + "service,": [248], + "as": [249], + "way": [251], + "inform": [253], + "policy": [255], + "practice.": [257] }, "counts_by_year": [ { @@ -1855,4 +1293,4 @@ "updated_date": "2026-07-25T09:21:30.201066", "created_date": "2025-10-10T00:00:00" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/opentargets.json b/backend/cli/test/science/fixtures/fetch/opentargets.json index 37578a27..05e59ccd 100644 --- a/backend/cli/test/science/fixtures/fetch/opentargets.json +++ b/backend/cli/test/science/fixtures/fetch/opentargets.json @@ -6,4 +6,4 @@ "approvedName": "tumor protein p53", "biotype": "protein_coding" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/pdbe.json b/backend/cli/test/science/fixtures/fetch/pdbe.json index 8c33a673..fc037c48 100644 --- a/backend/cli/test/science/fixtures/fetch/pdbe.json +++ b/backend/cli/test/science/fixtures/fetch/pdbe.json @@ -7,21 +7,11 @@ "deposition_date": "20200126", "release_date": "20200205", "revision_date": "20241120", - "experimental_method_class": [ - "x-ray" - ], - "experimental_method": [ - "X-ray diffraction" - ], + "experimental_method_class": ["x-ray"], + "experimental_method": ["X-ray diffraction"], "split_entry": [], "related_structures": [], - "entry_authors": [ - "Liu, X.", - "Zhang, B.", - "Jin, Z.", - "Yang, H.", - "Rao, Z." - ], + "entry_authors": ["Liu, X.", "Zhang, B.", "Jin, Z.", "Yang, H.", "Rao, Z."], "number_of_entities": { "water": 1, "polypeptide": 2, @@ -44,4 +34,4 @@ } ] } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/pfam.json b/backend/cli/test/science/fixtures/fetch/pfam.json index 97963796..056f9486 100644 --- a/backend/cli/test/science/fixtures/fetch/pfam.json +++ b/backend/cli/test/science/fixtures/fetch/pfam.json @@ -46,9 +46,7 @@ "raw_pages": "213", "medline_journal": "Genome Biol", "ISO_journal": "Genome Biol.", - "authors": [ - "Terakita A." - ], + "authors": ["Terakita A."], "DOI_URL": "http://dx.doi.org/10.1186/gb-2005-6-3-213" } }, @@ -88,4 +86,4 @@ } } } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/pubchem.json b/backend/cli/test/science/fixtures/fetch/pubchem.json index c628b121..a7cc0bb4 100644 --- a/backend/cli/test/science/fixtures/fetch/pubchem.json +++ b/backend/cli/test/science/fixtures/fetch/pubchem.json @@ -9,227 +9,32 @@ } }, "atoms": { - "aid": [ - 1, - 2, - 3, - 4, - 5, - 6, - 7, - 8, - 9, - 10, - 11, - 12, - 13, - 14, - 15, - 16, - 17, - 18, - 19, - 20, - 21 - ], - "element": [ - 8, - 8, - 8, - 8, - 6, - 6, - 6, - 6, - 6, - 6, - 6, - 6, - 6, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1 - ] + "aid": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21], + "element": [8, 8, 8, 8, 6, 6, 6, 6, 6, 6, 6, 6, 6, 1, 1, 1, 1, 1, 1, 1, 1] }, "bonds": { - "aid1": [ - 1, - 1, - 2, - 2, - 3, - 4, - 5, - 5, - 6, - 6, - 7, - 7, - 8, - 8, - 9, - 9, - 10, - 12, - 13, - 13, - 13 - ], - "aid2": [ - 5, - 12, - 11, - 21, - 11, - 12, - 6, - 7, - 8, - 11, - 9, - 14, - 10, - 15, - 10, - 16, - 17, - 13, - 18, - 19, - 20 - ], - "order": [ - 1, - 1, - 1, - 1, - 2, - 2, - 1, - 2, - 2, - 1, - 1, - 1, - 1, - 1, - 2, - 1, - 1, - 1, - 1, - 1, - 1 - ] + "aid1": [1, 1, 2, 2, 3, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 12, 13, 13, 13], + "aid2": [5, 12, 11, 21, 11, 12, 6, 7, 8, 11, 9, 14, 10, 15, 10, 16, 17, 13, 18, 19, 20], + "order": [1, 1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1] }, "coords": [ { - "type": [ - 1, - 5, - 255 - ], - "aid": [ - 1, - 2, - 3, - 4, - 5, - 6, - 7, - 8, - 9, - 10, - 11, - 12, - 13, - 14, - 15, - 16, - 17, - 18, - 19, - 20, - 21 - ], + "type": [1, 5, 255], + "aid": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21], "conformers": [ { "x": [ - 3.7321, - 6.3301, - 4.5981, - 2.866, - 4.5981, - 5.4641, - 4.5981, - 6.3301, - 5.4641, - 6.3301, - 5.4641, - 2.866, - 2, - 4.0611, - 6.8671, - 5.4641, - 6.8671, - 2.31, - 1.4631, - 1.69, - 6.3301 + 3.7321, 6.3301, 4.5981, 2.866, 4.5981, 5.4641, 4.5981, 6.3301, 5.4641, 6.3301, 5.4641, 2.866, 2, + 4.0611, 6.8671, 5.4641, 6.8671, 2.31, 1.4631, 1.69, 6.3301 ], "y": [ - -0.06, - 1.44, - 1.44, - -1.56, - -0.56, - -0.06, - -1.56, - -0.56, - -2.06, - -1.56, - 0.94, - -0.56, - -0.06, - -1.87, - -0.25, - -2.68, - -1.87, - 0.4769, - 0.25, - -0.5969, - 2.06 + -0.06, 1.44, 1.44, -1.56, -0.56, -0.06, -1.56, -0.56, -2.06, -1.56, 0.94, -0.56, -0.06, -1.87, -0.25, + -2.68, -1.87, 0.4769, 0.25, -0.5969, 2.06 ], "style": { - "annotation": [ - 8, - 8, - 8, - 8, - 8, - 8 - ], - "aid1": [ - 5, - 5, - 6, - 7, - 8, - 9 - ], - "aid2": [ - 6, - 7, - 8, - 9, - 10, - 10 - ] + "annotation": [8, 8, 8, 8, 8, 8], + "aid1": [5, 5, 6, 7, 8, 9], + "aid2": [6, 7, 8, 9, 10, 10] } } ] @@ -561,4 +366,4 @@ } ] } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/pubmed.json b/backend/cli/test/science/fixtures/fetch/pubmed.json index 8530670e..8f8a8855 100644 --- a/backend/cli/test/science/fixtures/fetch/pubmed.json +++ b/backend/cli/test/science/fixtures/fetch/pubmed.json @@ -76,15 +76,11 @@ "volume": "83", "issue": "4", "pages": "456-64", - "lang": [ - "eng" - ], + "lang": ["eng"], "nlmuniqueid": "0042124", "issn": "0020-7136", "essn": "", - "pubtype": [ - "Journal Article" - ], + "pubtype": ["Journal Article"], "recordstatus": "PubMed - indexed for MEDLINE", "pubstatus": "4", "articleids": [ @@ -119,9 +115,7 @@ } ], "references": [], - "attributes": [ - "Has Abstract" - ], + "attributes": ["Has Abstract"], "pmcrefcount": "", "fulljournalname": "International journal of cancer", "elocationid": "", @@ -146,4 +140,4 @@ }, "abstract": "1. Int J Cancer. 1999 Nov 12;83(4):456-64. doi: \n10.1002/(sici)1097-0215(19991112)83:4<456::aid-ijc4>3.0.co;2-5.\n\nAntigens recognized by autologous antibody in patients with renal-cell \ncarcinoma.\n\nScanlan MJ(1), Gordan JD, Williamson B, Stockert E, Bander NH, Jongeneel V, Gure \nAO, Jäger D, Jäger E, Knuth A, Chen YT, Old LJ.\n\nAuthor information:\n(1)Ludwig Institute for Cancer Research, New York Branch at Memorial \nSloan-Kettering Cancer Center, New York, New York 10021, USA. scanlanm@mskcc.org\n\nThe screening of cDNA expression libraries derived from human tumors with \nautologous antibody (SEREX) is a powerful method for defining the structure of \ntumor antigens recognized by the humoral immune system. Sixty-five distinct \nantigens (NY-REN-1 to NY-REN-65) reactive with autologous IgG were identified by \nSEREX analysis of 4 renal cancer patients and were characterized in terms of \ncDNA sequence, mRNA expression pattern, and reactivity with allogeneic sera. \nREN-9, -10, -19, and -26 have a known association with human cancer. REN-9 \n(LUCA-15) and REN-10 (gene 21) map to the small cell lung cancer tumor \nsuppressor gene locus on chromosome 3p21.3. REN-19 is equivalent to LKB1/STK11, \na gene that is defective in Peutz-Jeghers syndrome and cancer. REN-26 is encoded \nby the bcr gene involved in the [t(9:22)] bcr/abl translocation. Genes encoding \n3 of the antigens in the series showed differential mRNA expression; REN-3 \ndisplays a pattern of tissue-specific isoforms, and REN-21 and REN-43 are \nexpressed at a high level in testis in comparison to 15 other normal tissues. \nThe other 62 antigens were broadly expressed in normal tissues. With regard to \nimmunogenicity, 20 of the 65 antigens reacted only with autologous sera. \nThirty-three antigens reacted with sera from normal donors, indicating that \ntheir immunogenicity is not restricted to cancer. The remaining 12 antigens \nreacted with sera from 5-25% of the cancer patients but not with sera from \nnormal donors. Seventy percent of the renal cancer patients had antibodies \ndirected against one or more of these 12 antigens. Our results demonstrate the \npotential of the SEREX approach for the analysis of the humoral immune response \nagainst human cancer.\n\nCopyright 1999 Wiley-Liss, Inc.\n\nDOI: 10.1002/(sici)1097-0215(19991112)83:4<456::aid-ijc4>3.0.co;2-5\nPMID: 10508479 [Indexed for MEDLINE]" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/rcsb-pdb.json b/backend/cli/test/science/fixtures/fetch/rcsb-pdb.json index 5b0b3c73..ef478cec 100644 --- a/backend/cli/test/science/fixtures/fetch/rcsb-pdb.json +++ b/backend/cli/test/science/fixtures/fetch/rcsb-pdb.json @@ -1627,22 +1627,11 @@ "status_code": "REL" }, "rcsb_entry_container_identifiers": { - "assembly_ids": [ - "1" - ], - "entity_ids": [ - "1", - "2", - "3" - ], + "assembly_ids": ["1"], + "entity_ids": ["1", "2", "3"], "entry_id": "6LU7", - "model_ids": [ - 1 - ], - "polymer_entity_ids": [ - "1", - "2" - ], + "model_ids": [1], + "polymer_entity_ids": ["1", "2"], "pubmed_id": 32272481, "rcsb_id": "6LU7" }, @@ -1683,16 +1672,9 @@ "polymer_molecular_weight_minimum": 0.68, "polymer_monomer_count_maximum": 306, "polymer_monomer_count_minimum": 6, - "resolution_combined": [ - 2.16 - ], + "resolution_combined": [2.16], "selected_polymer_entity_types": "Protein (only)", - "software_programs_combined": [ - "PDB_EXTRACT", - "PHASER", - "PHENIX", - "XIA2" - ], + "software_programs_combined": ["PDB_EXTRACT", "PHASER", "PHENIX", "XIA2"], "solvent_entity_count": 1, "structure_determination_methodology": "experimental", "structure_determination_methodology_priority": 10, @@ -1799,9 +1781,7 @@ "ls_percent_reflns_R_free": 5.13, "ls_percent_reflns_obs": 99.5, "overall_SU_ML": 0.21, - "pdbx_diffrn_id": [ - "1" - ], + "pdbx_diffrn_id": ["1"], "pdbx_ls_cross_valid_method": "THROUGHOUT", "pdbx_ls_sigma_F": 1.34, "pdbx_method_to_determine_struct": "MOLECULAR REPLACEMENT", @@ -1836,9 +1816,7 @@ "d_resolution_low": 42.29, "number_obs": 19455, "pdbx_Rmerge_I_obs": 0.189, - "pdbx_diffrn_id": [ - "1" - ], + "pdbx_diffrn_id": ["1"], "pdbx_netI_over_sigmaI": 6.3, "pdbx_ordinal": 1, "pdbx_redundancy": 6.6, @@ -1851,9 +1829,7 @@ "d_res_high": 2.16, "d_res_low": 2.22, "number_unique_obs": 1431, - "pdbx_diffrn_id": [ - "1" - ], + "pdbx_diffrn_id": ["1"], "pdbx_ordinal": 1, "pdbx_redundancy": 6.1, "percent_possible_all": 100 @@ -1902,4 +1878,4 @@ }, "rcsb_id": "6LU7" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/reactome.json b/backend/cli/test/science/fixtures/fetch/reactome.json index 868a8bc5..31734057 100644 --- a/backend/cli/test/science/fixtures/fetch/reactome.json +++ b/backend/cli/test/science/fixtures/fetch/reactome.json @@ -9,9 +9,7 @@ "isInDisease": false, "isInferred": false, "maxDepth": 6, - "name": [ - "M Phase" - ], + "name": ["M Phase"], "releaseDate": "2004-07-06", "speciesName": "Homo sapiens", "figure": [ @@ -33,14 +31,10 @@ "isInDisease": false, "isInferred": false, "maxDepth": 5, - "name": [ - "G2/M Transition" - ], + "name": ["G2/M Transition"], "releaseDate": "2004-07-06", "speciesName": "Homo sapiens", - "followingEvent": [ - 68886 - ], + "followingEvent": [68886], "doi": "10.3180/REACT_2203.2", "hasDiagram": false, "hasEHLD": false, @@ -69,14 +63,10 @@ "isInDisease": false, "isInferred": true, "maxDepth": 6, - "name": [ - "M Phase" - ], + "name": ["M Phase"], "releaseDate": "2026-06-17", "speciesName": "Plasmodium falciparum", - "inferredFrom": [ - 68886 - ], + "inferredFrom": [68886], "hasDiagram": true, "hasEHLD": false, "schemaClass": "Pathway", @@ -90,14 +80,10 @@ "isInDisease": false, "isInferred": true, "maxDepth": 6, - "name": [ - "M Phase" - ], + "name": ["M Phase"], "releaseDate": "2026-06-17", "speciesName": "Saccharomyces cerevisiae", - "inferredFrom": [ - 68886 - ], + "inferredFrom": [68886], "hasDiagram": true, "hasEHLD": false, "schemaClass": "Pathway", @@ -111,14 +97,10 @@ "isInDisease": false, "isInferred": true, "maxDepth": 6, - "name": [ - "M Phase" - ], + "name": ["M Phase"], "releaseDate": "2026-06-17", "speciesName": "Schizosaccharomyces pombe", - "inferredFrom": [ - 68886 - ], + "inferredFrom": [68886], "hasDiagram": true, "hasEHLD": false, "schemaClass": "Pathway", @@ -132,14 +114,10 @@ "isInDisease": false, "isInferred": true, "maxDepth": 6, - "name": [ - "M Phase" - ], + "name": ["M Phase"], "releaseDate": "2026-06-17", "speciesName": "Dictyostelium discoideum", - "inferredFrom": [ - 68886 - ], + "inferredFrom": [68886], "hasDiagram": true, "hasEHLD": false, "schemaClass": "Pathway", @@ -153,14 +131,10 @@ "isInDisease": false, "isInferred": true, "maxDepth": 6, - "name": [ - "M Phase" - ], + "name": ["M Phase"], "releaseDate": "2026-06-17", "speciesName": "Caenorhabditis elegans", - "inferredFrom": [ - 68886 - ], + "inferredFrom": [68886], "hasDiagram": true, "hasEHLD": false, "schemaClass": "Pathway", @@ -174,14 +148,10 @@ "isInDisease": false, "isInferred": true, "maxDepth": 6, - "name": [ - "M Phase" - ], + "name": ["M Phase"], "releaseDate": "2026-06-17", "speciesName": "Drosophila melanogaster", - "inferredFrom": [ - 68886 - ], + "inferredFrom": [68886], "hasDiagram": true, "hasEHLD": false, "schemaClass": "Pathway", @@ -195,14 +165,10 @@ "isInDisease": false, "isInferred": true, "maxDepth": 6, - "name": [ - "M Phase" - ], + "name": ["M Phase"], "releaseDate": "2026-06-17", "speciesName": "Gallus gallus", - "inferredFrom": [ - 68886 - ], + "inferredFrom": [68886], "hasDiagram": true, "hasEHLD": false, "schemaClass": "Pathway", @@ -216,14 +182,10 @@ "isInDisease": false, "isInferred": true, "maxDepth": 6, - "name": [ - "M Phase" - ], + "name": ["M Phase"], "releaseDate": "2026-06-17", "speciesName": "Xenopus tropicalis", - "inferredFrom": [ - 68886 - ], + "inferredFrom": [68886], "hasDiagram": true, "hasEHLD": false, "schemaClass": "Pathway", @@ -237,14 +199,10 @@ "isInDisease": false, "isInferred": true, "maxDepth": 6, - "name": [ - "M Phase" - ], + "name": ["M Phase"], "releaseDate": "2026-06-17", "speciesName": "Danio rerio", - "inferredFrom": [ - 68886 - ], + "inferredFrom": [68886], "hasDiagram": true, "hasEHLD": false, "schemaClass": "Pathway", @@ -258,14 +216,10 @@ "isInDisease": false, "isInferred": true, "maxDepth": 6, - "name": [ - "M Phase" - ], + "name": ["M Phase"], "releaseDate": "2026-06-17", "speciesName": "Sus scrofa", - "inferredFrom": [ - 68886 - ], + "inferredFrom": [68886], "hasDiagram": true, "hasEHLD": false, "schemaClass": "Pathway", @@ -279,14 +233,10 @@ "isInDisease": false, "isInferred": true, "maxDepth": 6, - "name": [ - "M Phase" - ], + "name": ["M Phase"], "releaseDate": "2026-06-17", "speciesName": "Bos taurus", - "inferredFrom": [ - 68886 - ], + "inferredFrom": [68886], "hasDiagram": true, "hasEHLD": false, "schemaClass": "Pathway", @@ -300,14 +250,10 @@ "isInDisease": false, "isInferred": true, "maxDepth": 6, - "name": [ - "M Phase" - ], + "name": ["M Phase"], "releaseDate": "2026-06-17", "speciesName": "Canis familiaris", - "inferredFrom": [ - 68886 - ], + "inferredFrom": [68886], "hasDiagram": true, "hasEHLD": false, "schemaClass": "Pathway", @@ -321,14 +267,10 @@ "isInDisease": false, "isInferred": true, "maxDepth": 6, - "name": [ - "M Phase" - ], + "name": ["M Phase"], "releaseDate": "2026-06-17", "speciesName": "Rattus norvegicus", - "inferredFrom": [ - 68886 - ], + "inferredFrom": [68886], "hasDiagram": true, "hasEHLD": false, "schemaClass": "Pathway", @@ -342,14 +284,10 @@ "isInDisease": false, "isInferred": true, "maxDepth": 6, - "name": [ - "M Phase" - ], + "name": ["M Phase"], "releaseDate": "2026-06-17", "speciesName": "Mus musculus", - "inferredFrom": [ - 68886 - ], + "inferredFrom": [68886], "hasDiagram": true, "hasEHLD": false, "schemaClass": "Pathway", @@ -360,13 +298,7 @@ { "dbId": 48887, "displayName": "Homo sapiens", - "name": [ - "Homo sapiens", - "H. sapiens", - "Hs", - "human", - "man" - ], + "name": ["Homo sapiens", "H. sapiens", "Hs", "human", "man"], "taxId": "9606", "abbreviation": "HSA", "className": "Species", @@ -386,9 +318,7 @@ "dbId": 9821382, "displayName": "five stars", "definition": "externally reviewed", - "name": [ - "five stars" - ], + "name": ["five stars"], "className": "ReviewStatus", "schemaClass": "ReviewStatus" }, @@ -406,14 +336,10 @@ "isInDisease": false, "isInferred": false, "maxDepth": 4, - "name": [ - "Mitotic Prophase" - ], + "name": ["Mitotic Prophase"], "releaseDate": "2004-07-06", "speciesName": "Homo sapiens", - "eventOf": [ - 68886 - ], + "eventOf": [68886], "hasDiagram": true, "hasEHLD": false, "lastUpdatedDate": "2023-03-29", @@ -429,14 +355,10 @@ "isInDisease": false, "isInferred": false, "maxDepth": 3, - "name": [ - "Mitotic Prometaphase" - ], + "name": ["Mitotic Prometaphase"], "releaseDate": "2004-07-06", "speciesName": "Homo sapiens", - "eventOf": [ - 68886 - ], + "eventOf": [68886], "hasDiagram": true, "hasEHLD": false, "lastUpdatedDate": "2019-12-10", @@ -451,14 +373,10 @@ "isInDisease": false, "isInferred": false, "maxDepth": 5, - "name": [ - "Mitotic Metaphase and Anaphase" - ], + "name": ["Mitotic Metaphase and Anaphase"], "releaseDate": "2012-09-18", "speciesName": "Homo sapiens", - "eventOf": [ - 68886 - ], + "eventOf": [68886], "hasDiagram": true, "hasEHLD": false, "lastUpdatedDate": "2020-03-16", @@ -474,15 +392,10 @@ "isInDisease": false, "isInferred": false, "maxDepth": 3, - "name": [ - "Mitotic Telophase/Cytokinesis", - "cell division" - ], + "name": ["Mitotic Telophase/Cytokinesis", "cell division"], "releaseDate": "2004-07-06", "speciesName": "Homo sapiens", - "eventOf": [ - 68886 - ], + "eventOf": [68886], "hasDiagram": true, "hasEHLD": false, "lastUpdatedDate": "2023-03-29", @@ -493,4 +406,4 @@ "schemaClass": "Pathway", "className": "Pathway" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/sifts.json b/backend/cli/test/science/fixtures/fetch/sifts.json index 8321885a..1b6f9e82 100644 --- a/backend/cli/test/science/fixtures/fetch/sifts.json +++ b/backend/cli/test/science/fixtures/fetch/sifts.json @@ -12081,4 +12081,4 @@ } } } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/single-cell-atlas.json b/backend/cli/test/science/fixtures/fetch/single-cell-atlas.json index 2dca41d7..200a5262 100644 --- a/backend/cli/test/science/fixtures/fetch/single-cell-atlas.json +++ b/backend/cli/test/science/fixtures/fetch/single-cell-atlas.json @@ -8,9 +8,7 @@ "loadDate": "26-05-2018", "lastUpdate": "26-05-2018", "rawExperimentType": "SINGLE_CELL_RNASEQ_MRNA_BASELINE", - "technologyType": [ - "smart-seq2" - ], + "technologyType": ["smart-seq2"], "numberOfAssays": 2942, "experimentalFactors": [ "single cell identifier", @@ -18,10 +16,7 @@ "inferred cell type - ontology labels", "inferred cell type - authors labels" ], - "experimentProjects": [ - "Human Cell Atlas - Data Portal", - "Human Cell Atlas" - ], + "experimentProjects": ["Human Cell Atlas - Data Portal", "Human Cell Atlas"], "experimentType": "Baseline" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/string-db.json b/backend/cli/test/science/fixtures/fetch/string-db.json index 4527e5ee..1cbf48d7 100644 --- a/backend/cli/test/science/fixtures/fetch/string-db.json +++ b/backend/cli/test/science/fixtures/fetch/string-db.json @@ -380,4 +380,4 @@ } ] } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/surechembl.json b/backend/cli/test/science/fixtures/fetch/surechembl.json index 3bd2f859..10d3130c 100644 --- a/backend/cli/test/science/fixtures/fetch/surechembl.json +++ b/backend/cli/test/science/fixtures/fetch/surechembl.json @@ -25,4 +25,4 @@ "rtb": 0, "similarity": "" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/ucsc.json b/backend/cli/test/science/fixtures/fetch/ucsc.json index de9224dc..cb70d0e1 100644 --- a/backend/cli/test/science/fixtures/fetch/ucsc.json +++ b/backend/cli/test/science/fixtures/fetch/ucsc.json @@ -9,4 +9,4 @@ "end": 7687550, "dna": "ACCCCTCAGACACACAGGTGGCAGCAAAGTTTTATTGTAAAATAAGAGATCGATATAAAAATGGGATATAAAAAGGGAGAAGGAGGGGAAGGGTGGGGTGAAAATGCAGATGTGCTTGCAGAATGTAAAAGATGTTGACCCTTCCAgctggacgtggtggctcacaattgtaatcccagcactctgggaggctgagacaggtggatcgcctgagcccaggagtttgagaccagcctgggcaacactgtgagaccccatctctacaaaacatgcaaaagttggctggccatggtggcatgaacctgtggtcccagctactccggaggctgaggcaggactgctcgagccggggaggcaaaggctgcagtaagccaagatcacgccactccactccagcctgggcaacaaagcgagacccagtctcaaagaaaaagaaaaaaaaaaaaaaaaaagaaaaaagaaaTTGACCCTGAGCATAAAACAAGTCTTGGTGGATCCAGATCATCATATACAAGAGATGAAATCCTCCAGGGTGTGGGATGGGGTGAGATTTCCTTTTAGGTACTAAGGTTCACCAAGAGGTTGTCAGACAGGGTTTGGCTGGGCCAGCAGAGACTTGACAACTCCCTCTACCTAACCAGCTGCCCAACTGTAGAAACTACCAACCCACCGACCAACAGGGAGAGGGAACAAGCACCCTCAAGGGGGTCAAGTTCTAGACCCCATGTAATAAAAGGTGgtttcaaggccagatgtacattatttcattaaccctcacaatgcactctgtgaggtaggtgcaaatgccagcatttcacagatatgggccttgaagttagagaAAATTCAACAGTGAGGGACAGCTTCCCTGGTTAGTACGGTGAAGTGGGCCCCTACCTAGAATGTGGCTGATTGTAAACTAACCCTTAACTGCAAGAACATTTCTTACATCTCCCAAACATCCCTCACAGTAAAAACCTTAAAATCTAAGCTGGTATGTCCTACTCCCCATCCTCCTCCCCACAACAAAACACCAGTGCAGGCCAACTTGTTCAGTGGAGCCCCGGGACAAAGCAAATGGAAGTCCTGGGTGCTTCTGACGCACACCTATTGCAAGCAAGGGTTCAAAGACCCAAAACCCAAAATGGCAGGGGAGGGAGAGATGGGGGTGGGAGGCTGTCAGTGGGGAACAAGAAGTGGAGAATGTCAGTCTGAGTCAGGCCCTTCTGTCTTGAACATGAGTTTTTTATGGCGGGAGGTAGACTGACCCTTTTTGGACTTCAGGTGGCTGTAGGAGACAGAAGCAGGGAGGAGAGATGACATCACATGAGTGAGAGGGTCTGTGCCCCTTTTCCCTGACCAATGCTTTGAAGGGCCTAAGGCTGGGACAACGGGAATTCAAATCAAGATGGTGGCCACACCCCATGCAAATATGTTTACTGAGCACCTCAGAGTATTAGTGTGTATTAGTCTCGTAATCTTCCCTTACCCCAttttactttatttatcttttttgagacggagtttcactcttgttgcccaggctggagtgtaatggtgagatctcagctcaccgcaacctctgcctcccgggttcaagcgattctcctgcctcagcctcccgagtaggtagctgggattacaggcatgcatcaccacgcccggctacttttgtatttttagtagagatggggtttctccatgttggtcaggctgggctcaaactcccgacctcaggtgatccactcgccttggcctcccagagtgtgggattcgtgagccactgcgcccggccCCCTTAccccattttatatataaggaaactgagtttgacgggggtcacctaggacctgccggtgcatggcagggctgagtatatgacctgaaactcTGGCTGTATTCAGTATTACACAATTATTAGGCCCCTCCTTGAGACCCTCCAGCTCTGGGCTGGGAGTTGCGGAGAATGGCAAAGAAGTATCCACACTCGTCCCTGGGTTTGGATGTTCTGTGGATACACTGAGGCAAGAATGTGGTTATAGGATTCAACCGGAGGAAGACTAAAAAAATGTCTGTGCAGGGCTGGGACCCAATGAGATGGGGTCAGCTGCCTTTGACCATGAAGGCAGGATGAGAATGGAATCCTATGGCTTTCCAACCTAGGAAGGCAGGGGAGTAGGGCCAGGAAGGGGCTGAGGTCACTCACCTGGAGTGAGCCCTGCTCCCCCCTGGCTCCTTCCCAGCCTGGGCATCCTTGAGTTCCAAGGCCTCATTCAGCTCTCGGAACATCTCGAAGCGCTCACGCCCACGGATCTGCAGCAACAGAGGAGGGGGAGAAGTAAGTATATACacagtacctgagttaaaagatggttcaagttacaattgtttgactttatgacggtacaaaagcaacatgcatttagtagaaactgcacttcaagtacctatacagcTGACTTTTAAAAATAtttatttatttattttgagatggggtctcactctgttgcccaggcgggagtgcaatggtgcaatcttggctgattgcaatctccgcctctggggttcaagtgattcttgtgcctcagcctcccgagtagctgggactacaggcgtgtgctaccacacctggctaatttttgtgtttttagtagagatggggcttcaccatgttagccaggctggtttccaactcctgacgtcaggtgatctacccacctccacctcccaaagtgctgggattacaggtgtgagccactgtgcccggccCttttttaaattttagagatgatgtcttgctatgttgttcaggctggactcaaactcttgggctcaagagatcctcctgccttagcctctcaagtaactgggactacatgtgcatgcgactgtgcctcgtttcttttcttttttttctgagacggagtctcactctatcgcccaggctggagtgcagtggcgccatcttggctccctgcaacctccgcctcctggttcaagcgattctcctgcctcagcctcccaagtagctgggattacaggcacctgccatcacgcccggttaatttttgtattttagtagagacggggtttcaccatgttggctaggctggtcttgaactcctgacctcaggtgatccacccgcctcagcctcccgaaatgctgggattacaggcgtgagccagtgcgcctggccttttctttttttgagtctcgctctgcgcccaggctgTGCCTGGCTCGACTGTGCCTCCTTTcatgcaaccatgctgtttctcactttcagtaacaatattcaataaatcacatgagatatacaacattttattactataaaaagggctttgtgttagatgactttgcccaactgtagggtaacttaaatgctctgaacacgtttcaagtaggctagggctgagtgtggtagctcatgcctgtaaccccaatacttggggaggctgaggtggaaggattgattgagcccaggggtttgataccagcatgggcaacgtagcaagaccttgacttcacagaaaataaaaaattagctgggtgtcgtggcatgtgcctgtagtcctagctacttgggagggtgaaatcaccggagcccagggaggtcaaggctgcagtgagctgagatggtgccactgcactctagcctgagtgacagagtgagactctgtctttaaataaataaataaaaaTTAgccgggcgtggtggctcacacctgtaatcccagcactttgggaggccgaggcgggcggatcacatggtcagaagttcgagaccagcctggccaacatggtgaaaccctgtctctactaaaaatacaaaaattagctgggcgtggtagcaggcgcttgtagtcctagctattcgggaggctgaggcaggagaatcacttgaacccaggaggcagaggttgcagtgagccgagatcatgccactgcactccagcctgggcgacagagtgagactgagtctcaaaaaaataaaataaaataaaataaaaataaataaataaaaattagccaggcatggtggtgcaggcctgtagttgaagcaacttgggaggctgagctgggaggatggatggagcctgggaggtggaggctgcagtgagctgtgactgcactactgcactctatccagcctgggtgacagagcaagaccttgtctcaaaaaaGTAGGCTAgagaccagcctgggcaacatagtgagactctatctatctacaaaaaattttaaaaattagctgggtatggtggtgtatgcctgtggtcctagctactggggaggcagagttagggggattgcttgagcccaggagggtataatgagctatgatcacatcactgtaatccagcctgggcaacagagcaagatgctgtctccattaaaaataaaataaaaGTAGGCTAGGCAggccgggtgcggtggctcacgcctgtaatcccagcactttgggaggccaaggcaggcagatcacaaggtcaggagttcgagactagcctggccaacatggtgaaacctcatctctactaaaaaaaaaaataaataaataacaaaaaattagctgggcgtcggggcaggtgcctgtaatcccagctactcagtgggctgaggcaggagaatcgcttgaacccagaaggcggaggttgcagtgagccgagatcccgccactgcactccagcctgggtgacagagtgagactctgtctccaaaaaaaaaaaaaaaaaaagcaggctaggctaagctatgatgttccttagattaggtgtattaaatccattttcaacttacaatattttcaacttacgacgagtttatcaggaagtaacaccatcgtaagtcaagtagcatctgTATCAGGCAAAGTCATAGAACCATTTTCATGCTCTCTTTAACAATTTTCTTTTTGAAAGCTGGTCTGGTCCTTTAAAATATATATTATGGTATAAGTTGGTGTTCTGAAGTTAGTTAGCTACAACCAGGAGCCATTGTCTTTGAGGCATCACTGCCCCCTGATGGCAAATGCCCCAATTGCAGGTAAAACAGTCAAGAAGAAAACGGCATTTTGAGTGTTAGACTGGAAACTTTCCACTTGATAAGAGGTCCCAAGACTTAGTACCTGAAGGGTGAAATATTCTCCATCCAGTGGTTTCTTCTTTGGCTGGGGAGAGGAGCTGGTGTTGTTGGGCAGTGCTAGGAAAGAGGCAAGGAAAGGTGATAAAAGTGAATCTGAGGCATAACTGCACCCTTGGTCTCCTCCACCGCTTCTTGTCCTGCTTGCTTACCTCGCTTAGTGCTCCCTGGGGGCAGCTCGTGGTGAGGCTCCCCTTTCTTGCGGAGATTCTCTTCCTCTGTGCGCCGGTCTCTCCCAGGACAGGCACAAACACGCACCTCAAAGCTGTTCCGTCCCAGTAGATTACCACTACTCAGGATAGGAAAAGAGAAGCAAGAGGCAGTAAGGAAATCAGGTCCTACCTGTCCCATTTAAAAAACCAGGCTCCATCTACTCCCAACCACCCTTGTCCTTTCTGGAGCCTAAGCTCCAGCTCCAGGTAGGTGGAGGAGAAGCCACAGGTTAAGAGGTCCCAAAGCCAGAGAAAAGAAAACTGAGTGGGAGCAGTAAGGAGATTCCCCGCCGGGGATGTGATGAGAGGTGGATGGGTAGTAGTATGGAAGAAATCGGTAAGAGGTGGGCCCAGGGGTCAGAGGCAAGCAGAGGCTGGGGCACAGCAGGCCAGTGTGCAGGGTGGCAAGTGGCTCCTGACCTGGAGTCTTCCAGTGTGATGATGGTGAGGATGGGCCTCCGGTTCATGCCGCCCATGCAGGAACTGTTACACATGTAGTTGTAGTGGATGGTGGTACAGTCAGAGCCAACCTAGGAGATAACACAGGCCCAAGATGAGGCCAGTGCGCCTTGGGGAGACCTGTGGCAAGCAGGGGAGGCCttttttttttttttttgagatggaatctcgctctgtcgcccaggctggagtgcagtggcgtgatctcagctcactgcaagctccaccgcccaggttcacgccattctccttcctcagcctcccgagtagctgggactacaggtgcccagcaccacgcccggctaatttttttttgtatttttcagtagagacggggtttcaccgttagccaggatggtctcgatctcccaacctcgtgatccgcctgccttggcctcccaaagtgctgggattacaggcatgagccactgcgcccagccAAGCAGGGGaggcccttagcctctgtaagcttcagttttttcaactgtgcaatagttaaacccatttactttgcacatctcatggggttatagggaggtcaaataagCAGCAGGAGAAAGCCCCCCTACTGCTCACCTGGAGGGCCACTGACAACCACCCTTAACCCCTCCTCCCAGAGACCCCAGTTGCAAACCAGACCTCAGGCGGCTCATAGGGCACCACCACACTATGTCGAAAAGTGTTTCTGTCATCCAAATACTCCACACGCAAATTTCCTTCCACTCGGATAAGATGCTGAGGAGGGGCCAGACCTAAGAGCAATCAGTGAGGAATCAGAGGCCTGGGGACCCTGGGCAACCAGCCCTGTCGTCTCTCCAGCCCCAGCTGCTCACCATCGCTATCTGAGCAGCGCTCATGGTGGGGGCAGCGCCTCACAACCTCCGTCATGTGCTGTGACTGCTTGTAGATGGCCATGGCGCGGACGCGGGTGCCGGGCGGGGGTGTGGAATCAACCCACAGCTGCACAGGGCAGGTCTTGGCCAGTTGGCAAAACATCTTGTTGAGGGCAGGGGAGTACTGTAGGAAGAGGAAGGAGACAGAGTTGAAAGTCAGGGCACAAGTGAACAGATAAAGCAACTGGAAGACGGCAGCAAAGAAACAAACATGCGTAAGCACCTCCTGCAACCCACTAGCGAGCTAGAGAGAGTTGGCGTCTACACCTCAGGAGCttttcttttttttttttttttttgagatagggtcttgctctgtcactcaggctggagcacagtggtgtgatcacagctcactgcagcctccatctcctggcctcaagtgatcttcccacctcagcctcctaagtggctgggactataggtgtgcaccaccatgcctggctaattttttgtatttttttgtagagacgaggtttcatcatgttacccaggctggtcttgaactcctgggctcaggtgatctgcctgccttggcctctttgagagtgctgggattgcaggtgtgagccaccaagcctggtcAGGAGCTTATTTTCAAAAGCCAAGGAATACACGTGGATGAAGAAAAAGAAAAGTTCTGCATCCCCAGGAGAGATGCTGAGGGTGTGATGGGATGGATAAAAGCCCAAATTCAAGGGGGGAATATTCAACTTTGGGACAGGAGTCAGAGATCACACATTAAGTGGGTAAACTATAAAAAAACACTGACAGGAAGCCAAAGGGTGAAGAGGAATCCCAAAGTTCCAAACAAAAGAAATGCAGGGGGATACGGCCAGGCATTGAAGTCTCATGGAAGCCAGCCCCTCAGGGCAACTGACCGTGCAAGTCACAGACTTGGCTGTCCCAGAATGCAAGAAGCCCAGACGGAAACCGTAGCTGCCCTGGTAGGTTTTCTGGGAAGGGACAGAAGATGACAGGGGCCAGGAGGGGGCTGGTGCAGGGGCCGCCGGTGTAGGAGCTGCTGGTGCAGGGGCCACGGGGGGAGCAGCCTCTGGCATTCTGGGAGCTTCATCTGGACCTGGGTCTTCAGTGAACCATTGTTCAATATCGTCCGGGGACAGCATCAAATCATCCATTGCTTGGGACGGCAAGGGGGACTGTAGATGGGTGAAAAGAGCAGTCAGAGGACCAGGTCCTcagccccccagccccccagccctccaggtccccagccctccaggtccccagcccAACCCTTGTCCTTACCAGAACGTTGTTTTCAGGAAGTCTGAAAGACAAGAGCAGAAAGTCAGTCCCATGGAATTTTCGCTTCCCACAGGTCTCTGCTAGGgggctggggttggggtgggggtggtgggCCTGCCCTTCCAATGGATCCACTCACAGTTTCCATAGGTCTGAAAATGTTTCCTGACTCAGAGGGGGCTCGACGCTAGGATCTGACTGCGGCTCCTCCATGGCAGTGACCCGGAAGGCAGTCTGGCTGCTGCAAGAGGAAAAGTGGGGATCCAGCATGAGACACTTCCAACCCTGGGTCACCTGGGCCTGCAGAGAAGGAACCCCCTCCCCCAACACCATGCCAGTGTCTGAGACAGCTCGGCTTCCTGTGGAGCAGGAAAAGAatggctgcttcacattctctcttccaatgtttcaccacaacccaagcactcctgccccacccctcaccagccatgcacttctttgaggaaaagacaatcagagagggacttccaaccttcccaccactaaatccccaagacttcctaaatgtgcaccctattcccaactcccttcctgtAttttttttttttttttgagatggagtctctctctgtcacctaggctggagcacagtggcatgatctcagctcactgcaacctctaccttccgggttcaagccattctcctgcctcagtctcccgagtagctgggattacaggcgagtaccaccacacccagctaatttttgtatttttagtagagacagggctttgcatgttggccaggctggtctcgaactccttacttcaggtgatcggcccgcctcagcctcctaaagtgccaagattacaggtgtgagctaccgtgccctgcTCCCACCTCCTGTTAACAAGGATATAGTCATTCTCAGCCTGCAATCTCTGTATGGGGAAGGACACCCCCTTGGCCCCCACCCTTCCCCACCTGATACACGGCTCCATTTCTTTGATTCCTTTCACTGCAAAGCTTCTGGAAGAACAACTGTCTCACCGCTCACCTGCCCATTCTCTTCGGACACTCCTCAGCCCTGCATTACAAACCCCTCACGAATGGCCCGTCTCGGCTTCTTTAATCTCATCTCTTAACAACCACTCCCTCTTCCCCAAAAGCTCTAGCTAGACTGGCTGCCCTTCTCTGCTAATCAACTGGTGGTTCCTTGGCTAGCCAGGAACATGGGGGTAGGCTCCTTCCCGTGCAGACTTTAAGTCATCCTATTTTAATTCACATCAcctcatttgcattctcatagcacttacattgtctgatacttttccttgtttattttatctgtttcctctaatagcatatacacttcctaagggcagggcagtgatctatcttgttgtcttgctgaccaaagtattagatcacaatgccttgcacctgcttgggctcaataaatgtGAATAACACACAAGCCTGTTATATGAGAGGTTAAGAGAGCGAGAAAGAGCAAGGGGCAGCCCCTGTGTGGACCAGCATCTTGCACGAAGTTATGCAACTATCATCGCACCTTCTCCCAGACAAGCTTTCAAAGGCTTTGCCATGTTTtcttttgttttgtttttttgtttgttttttgagatggagtttggctcttttcgcccaggctggagtgcagtggtgcagtctaggctcactgcaacctctgccttctggtttcaagcaattctcctgcctcagcctcccgagtagctgggattacaggcccctgccaccatgcctggctgattttttgtatttttagtagagacagggtttcaccatgttggccaggctggtgtggaactcctgaccttgtgatccacctgcctcggcctctcaaagcgctgggattacaggtgtgagccactgtgcctggccCGCCATGttctttctttctttcttttttcttttgaggcagggtcttgctttgttgcccaagctagggtacagtggtgcaatcatggctcactacagcctcggactcctgggctcagtgattctcctgcctcagcctcccaagtagctgggaccacagaggcctgcctggctaattttttagtctttttctttttctttttttttttggagacggagtctcgccctgtcacccaggctagagtgcagtggtgtgatctcgactcactgtaacctccacctcccagattcaagcgatcctcttgcctcaacttcctgagtagctgggattacaggcgcccaccaatgcgcctgattaattttttgtatttttagtagagatggggttttgccatgttggccaggctggtttcgaactcctgacctcaggtgatcctcccgcctccgactcccaaagtgctgggattacaggtgtgagccactgcacccggccaatttttgagtttttttgtagaggcagggtttcactatgttgcccaggcAGGATGCTCTCtttctttttctttttttttttttCAGGGATGCTCTCTTTCTTTATGCCAAATTTGTCATCAGATTTGCTAAGAAACATGCCTACTGTAAGTGTTTGTTACACttttctgtttttttttttttttgagacagagttttgctctcgtccaggctggagtgcaatggtgcgatctcggctcaccgcaacctctgcctcccaggttcaagcgattctcctgcctcagcctcccgagtagctgggattacaggcatgcgccactgcgcccggctaattttgtatttttagtagagacggggtttctccatattggtcaggctggtcttgaactcccaacttcaggtgatccgcccgccttggcctcccaaagtgctgggattacaggcatgaaccactgtgcccagccCACTTTTCTGTTGTTTGCACTGACAAAACATCCCCTACCAAACAGCTCCTTTAATGGCAGGCTCTTTTCTTTttttattttattttattttattttattttattttattttattttgagacggagtctcagctcttattgcccaggctggagtacagtggcactatctccgcttactgcaacctcctcccggattcaaatgattctcctgcctcagcctgctgagtagctaggactacaagcatgtgccaccacacctggctaattttgtacttctagtagagacggggtttcaccatgttggtcaggctggtcttgaactcccgacctcaggtgatccaccccccttggcctcctaaagtgctgggattacaggcgtgagccaccaagcctggccTACCTAGTACTCTGTGTATTATGGGAAatgtagagttgaggaaagtgctgggcacacagtaagagctcaacaaaggttagctctTTCTGCAATTGTTCTATTTCACTTGTTCTATATTATTATTCTAGAGAGAACTGTGTGATTGTTAGTGCGGATCTGTGGTACTGCTCCCACCCCCACTCCATTAATGCAAGTACACCTCCTTCAGGGATCTATTCAGTCAACAGGCCAGGAGGTGCTGTCCTGAAATGGGGGGCCCAAAGTCTCAATCCCACTTGGAGGGACACAGGTCTACAGACAGGTCTCCCTGTCTTTATCTCTCAAATCTTCAGTAGCAACTAAAATCTCCGTGTTTTTCAGAGCAGGACCTTCCCAGGGGTACCAGCATCAGTGGGCCAGGATACAAATGTGCCAGGCTGAACTAGGCCTTCCAAATGGCCAGGGAGCCAAGAGAAATGCAGGTGCCCTTGGCTGGGTGGGAAGGCAATGAGATCAACTGAGACCCCAAACAGGGGCAGGCCTGACCAGAATCTTAACAGTGGCTGCTGGTATCAGTCTTGAAGGCCTATATGTCCAGTGattccctaaacaatatagtacaagtacttacatacatacttgcatgtacattagcattttcattgtatgggggtaatgtaagtaatctagagataatttaaactatatgggaggatatgtgtaggttaaatccaaatactataccgtcttatatatgggacttagacatctgtgggtttggtgtgtgaggagtcccagaaccaagcccctacagatagagggataactatattgccctgtaacctgcaaccctgctatatttatttattagttttggtagctttttatggattttccattaggacttttttttttttttgagatagagtttcactctttttttttttttttgagacggaatctcgctctgttgcccggcgtggtgtgcaatggcatgatctcagctcactgcaacctccacctcctgggttcaagcaattcttctgtctcagcctcccaagtagctgggattacaggcgcccaccactacacccagctaattcttgtatttttagtagagacggggtttcaccatgttaggttggtctcaaactcctgacctcaggtgatcggcctgcctcagcctcccgaagtgctgggattacaggcgtaagccaccacaccctgccggagtttcactcttgttgcccaggctggagtgcaatagcgcgatctcggctcactgcaacctctgcctcccaggtccaagcaattctcctgcctcagcctcctgagtagctgggattagaggtgcccgtcaccacgcctggctgattttttgtatttttattagagttggggtttcaccatgttggccaggctggtcccagggaagccacctgcctcagcctcctaaaagtgctaggattacaggcatgagccaccacgcctggccccattaggacatgtatgtatagaatcatactggctgtgaatgtgttttatttcttcctttctaatcgttattttttttctttccttttttttttttttttttgacatggaattttgctcttgtcgcctaggctggagtgcaatgggacaatctcggctcactgcaacctctgcctcctgggttcaagtgattctcctgcctctgcctcctgagtagctgggactacaggcgttcaccactacccctggctaattcttttttttttgagacggagtttttgcttttgtcacccaggctggagtgcaatggtgcaatctcggctcactacaacctccgcctcccaggttcaagcgattctcctgcctaagcctcccaagtagctgggattacaggcgcccgccactacgcccggctaatttttgtatttttagtagagatggggtttcaccatgttggccaggctggtgttgaactcctgacctcaggtgatccacccacctcggcctcccaaagtgctgggattacaggcatgagccactgtacccggccaacgcctggctatttttttaatattttaatagagacgaggtttcaccatctttgtcaggctggtctccaactccagacctcaggtgatctgcccacctcggcctcccaaagtgctgggattataggcgtgagacatcgggccACTAATCAttatttctttttctttttttttttttgagacacagtcttgctctgtcgcccaggctggagtgcagtggctcgatctcagctcactgcaagctccgccccctgagttcacgccattctcctgcctcagcctcccgagtagctgggactacaggcgcccgccactacgcccggctaattttttgtatatttagtagagacagggtttcaccgtgttagccaggatggtctcgatctcctgacctcgtgatccacccgtctcggcttcccaaagtgctgggattacaggcctgagccaccgcacccggccctcattatttctttttcttgcctggttaaaacctccagtatggtatcaacgttgtgagagtcaaatccttttctagttcctgatcttagaggaaaaagcgttgagttttcttttcttttttttttttttttttttttgagacgaagtctcactctgtcacccaggctggagtgcagtggcacgatctaggctctgcaagctccgcctcccgggttcacgccattctctcgcctcagcctcccgagtagctgggactataggcgcccgccaccatgcccggctaattttttgtttttgtatttttagtagagacggggtttcagcatgctagtcaggacagtctcgatctcctgacctcgtgatccgcccgcctaggcctcccaaagtgctggcattacaggcgtgagccaccgcgcccggcagcactgagttttctaccattatgtatgctgctagtggaactccgactgtggacgccctgttatcaaactaggttaagtttcctttccctagtttgctaggaggttggttggtttgtaatcatgcatatgtgttgaatatcattaactgcttttgctacatctgttgaaatgatcatagggtttttatgtttccctttgttaatgtggtgaattacacagactgatttttttccccccagtaaagaccagtctgactatgttgcccaggctggtcttgaaatcctgggctcaagagatcttcctgcctcagcctcctaaaatgttgggattacaggcctgagctactgcaccaggccaatttttgaatgttgaatcagctacaatcatgagataaacattatttggttagaatgtatttatcctttttctttttctttttttgagatggagtctcactctgttgcccaggctggagtgcaatggtgtgatcccagctcagtgcaacttctgcctcctgggttcaagcgattctcctgcctcagactcccgagtagctgggatttcaggtgcccaccaccatgcccagctaatttttttttttttttgagatgaagtcttgctctgtcgcccaggctggagtgcagtggcacgatcttggctcactgcaacctctgcctcccgggttcaagcaattctgcctcagcctcctgagtagctgggattacaggcaggtgccaccacaccggctgatttttgtatttttagtagagatggcgtttcaccacattggtcaggctggtcttgaactcctgacctcgtgatctgcccacctcggcctcccaaagtgctgggattacaggcgtgagccaccgtgcccagcctgatttttgtatttttattagaaacggggtttcaccatgttggtcaggctggtctcaaactcctgacctcaagtgatctgcctgcctcagcctcccaaagtgctgggattacaggcgtgagccaccgcgactggcctatttatccttttttctatattaccaggtttggtttgctaaaattggttagctgttgcatgtctatgctaacaggaatattggtctatattttcttttcttataatgtccttgtttggttttggtaccaggattatgctggcttcgaaaacaagttgggaaatattcctctattttttctttctttctttttttttctgagacagggtctcactctgttgcccaggctggagtgcagtggcgcaatctcggctcactgcaacctccgcttcccaggttcaagggattcttgtgcctcagcctcctgagtaactggcattacaggtatgtgtcaccacgcctagctactttttgtatttttagtagagatggggtttcgccgtgttggccaggctggtctcgaactcccgacctcaaatgatcccctgcctcagcgtaccaaagtgctgagattacaggtatgagccaccgcgcccagtctgttcctctgttttctgaagagtttgtgtaagatgggtactgtttcttcctccaacgtttAAAGAgagcagagaacagaggagataaatagaaaatagcactaagaggtcaggcatggtggctcacacctgtaatcccagcactttgggaggccgagatgggatgaaagcttgaggccagcagtttgagaccagcctgggcaacatagtgagatcctgtcaatacaaaaaaataaaatagttagctgggtgtggtggagtaatcccagctactcgtgaggctgctagaggactgcttgagcccaggggttcgaggctgcagtaagccttgattgtgccattgtactctagcctgggcaacagtgtgagtccctgtctcaaaaattaacaaagaaaaaaagaaaatagcactaaaatggtagccctatactccaactgtaaaataattaaaattaaaAGCattcagtagagaaaggaagcctatttcaacaagtggagacagaataactggatttccatacaggaaagataccagagactgactcctacacctcacaccataaacaattaattttaagaattaattaatggctcaaggaccttactgtaaaacttacaaccataaaggtcctaaaagaaaatgtaagataatatcttcatgaccctggggttaaaaaaaaaaaaaaagatgtcctaaacaggacaaggcaaatactgaacataaaaaagataaatccactcctcttaagatactgtaaactctgtaaagcaaacaaataggcaagcaacagatcagaagaaaacattcacgacacatggatctgataaaggacttgtatccagaatgtataaagcagtcccacaactgaacaataaaaacaaacaaaaaaccaaaataacaggtaaaagactcgaagagctactttacaaacaaaatacgaatggccaataggcacatgaaaaaatgctgaacatccttagtcaatagagaactgtaaattacaaccacaaggatataccacattagaaagactgacaatacctaatgtccggaaggctgtggcacaaccataataactcccataccttgctagttggagtgtaaaatggtacaaccgctctggaaaactcagagcttctgaaaaagttcaaaatacagctactttttacttccaaactcgcaattcccctcctaagtatttctccaagaaacacgaaaacatatgatcacaaaaagaattgtacaagaatgtttatagcagctttatttcataaccgcaaatgggaaacaactcaaaaggccatcaaaaggacggatatacaatcgatggactatactaaatgaaaaggagcaaaatactgatatatacaacatgaacgaatgtcagacagtacattgaaggacagaagcccgacaaaaatgagcacataatgtatgattccccccttttttttgagacggagtttcgttcttgtcgcccaggctggagtgtagtggcacgatcttggctcactgcaacctctgcctcccgggttcaagcgattctcctgcctcaccctcccgaatagctgggattacaggcacccaccacgcccagctaatttttgtattttttagtagagacggggattcaccacgttggccacgctggtctggaactcctatcctcaagtaatccgcccgcctcggcctcccaaagtgcaggcgtgagccacagcgcccagcctgattccattctatatgaagttctccaacaggcaaaatggttatggagatcaaaataaaggtggggtcgggaatcgactgggaagagacgtgatgaaacgtttctgggacgatgaaaagggtctgtgacttggtaggcatcacggagcggttaggggccaaaactcatcttcctgtgcacttgctgtgtgcactggcgctgtgtgtaaatgccacctcgaTTTAGGAAAAAGATGACGTAAGTACGGCACAAAGTGGCCGGTACGCGGCAGGTGCATGGGAAGAAACTGCGGAATGAAACAACCGCGAGCTAAGAGATGGGGCAGCGGGAGAAATGAATTCGAGTTCCGCCTCCTACCAGGAAGAACCGGCTCGGGCCGGAGGGCTGCACGGAGGACCACACGGACGCCTGCGGGCCCGCCCCTTCCGCTTCACGACGTTCAGCCTGCGTCTGGAACTGGAATGGCCTAGCCCAAAGCTAGATAACAGGTAGATTGTTTTTCCGACAAATTATCAAACGACCCATCATTGCACTCTTTCAAAATTTGATTCTCAGACGTACCCATTCTTTTTTTTTTTCCTCCGGGAAGATGAGATATACTCATTCTTGAAAATACCTCCGGGCTTGCCTTCTGCACACTTCTTTCCCTCCCTGTCTCACGCCATGGTAGCGTCCGCCTAGGTTGCAGGCGACCCGCGGGGTGGGGCACACCATTCAAAGAAGGGGAGGGATTGAGGTTTGCATCAAAACAAATACCCCTGCCTTTGCAAAGGCCATAACTAAGTAATCCAGAAAAAGAAATGCAGGCGGAGAATAGCAGCCTCCCTCTGCCAAGTAAGAGGAACCGGCCTAAAGGACATTTTCTCTCTCTCTCCTCCCCTCTCATCGGGTGAATAGTGAGCTGCTCCGGCAAAAAGAAACCGGAAATGCTGCTGCAAGAGGCAGAAATGTAAATGTGGAGCCAAACAATAACAGGGCTGCCGGGCCTCTCAGATTGCGACGGTCCTCCTCGGCCTGGCGGGCAAACCCCTGGTTTAGCACTTCTCACTTCCACGACTGACAGCCTTCAATTGGATTTTCTCCATCTAGCGGAGCCGGGGGCTGCCTGGAAAGATCGCTCCAGGAAGGACAAAGGTCCGGAAGTTGTGGGACCTTAGCAGCTTGGGCTCCCCGGATCACCCCCAAATGATCATTTCGGAATGGAGCCCCAGTTTTCACTAGGATGCCATGGGCTCTAAAATATACAGCTATGAGTTCTCAATGTTTCGAGATCCAAAAGTCTCAGACCTCAATGCTTTGTGCATCTTTTATTTCAGGGATTCCCTACGCCCAGCACCGGGTGGATGTGCAAAGAAGTACGCTTTAGGCCGGCTCAAGGTTCCCCAAAGCTCCACTCCTCTGCCTAGGCGTTCAACTTTGAGTTCGGATGGTCCTAACATCCCCATCATCTACACCCAGGTCTCCCAACAATGCAACTCCTATGATGATCCCTCTAGCCAAGCTTCCATCCCACTCACCCCCAAACTCGCTAAGTCCCCACTGCCCCACCCCCAGCCCCAGCGATTTTCCCGAGCTGAAAATACACGGAGCCGAGAGCCCGTGACTCAGAGAGGACTCATCAAGTTCAGTCAGGAGCTTACCCAATCCAGGGAAGCGTGTCACCGTCGTGGAAAGCACGCTCCCAGCCCGAACGCAAAGTGTCCCCGGAGCCCAGCAGCTACCTGCTCCCTGGACGGTGGCTCTAGACTTTTGAGAAGCTCAAAACTTTTAGCGCCAGTCTTGAGCACATGGGAGGGGAAAACCCCAATCCCATC" } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/uniprot.json b/backend/cli/test/science/fixtures/fetch/uniprot.json index bbcad519..454cab83 100644 --- a/backend/cli/test/science/fixtures/fetch/uniprot.json +++ b/backend/cli/test/science/fixtures/fetch/uniprot.json @@ -3573,10 +3573,7 @@ }, { "commentType": "ALTERNATIVE PRODUCTS", - "events": [ - "Alternative promoter usage", - "Alternative splicing" - ], + "events": ["Alternative promoter usage", "Alternative splicing"], "isoforms": [ { "name": { @@ -3590,9 +3587,7 @@ "value": "p53alpha" } ], - "isoformIds": [ - "P04637-1" - ], + "isoformIds": ["P04637-1"], "isoformSequenceStatus": "Displayed" }, { @@ -3607,13 +3602,8 @@ "value": "p53beta" } ], - "isoformIds": [ - "P04637-2" - ], - "sequenceIds": [ - "VSP_006535", - "VSP_006536" - ], + "isoformIds": ["P04637-2"], + "sequenceIds": ["VSP_006535", "VSP_006536"], "isoformSequenceStatus": "Described" }, { @@ -3625,13 +3615,8 @@ "value": "p53gamma" } ], - "isoformIds": [ - "P04637-3" - ], - "sequenceIds": [ - "VSP_040560", - "VSP_040561" - ], + "isoformIds": ["P04637-3"], + "sequenceIds": ["VSP_040560", "VSP_040561"], "isoformSequenceStatus": "Described" }, { @@ -3649,12 +3634,8 @@ "value": "p47" } ], - "isoformIds": [ - "P04637-4" - ], - "sequenceIds": [ - "VSP_040832" - ], + "isoformIds": ["P04637-4"], + "sequenceIds": ["VSP_040832"], "isoformSequenceStatus": "Described" }, { @@ -3666,14 +3647,8 @@ "value": "Del40-p53beta" } ], - "isoformIds": [ - "P04637-5" - ], - "sequenceIds": [ - "VSP_040832", - "VSP_006535", - "VSP_006536" - ], + "isoformIds": ["P04637-5"], + "sequenceIds": ["VSP_040832", "VSP_006535", "VSP_006536"], "isoformSequenceStatus": "Described" }, { @@ -3685,14 +3660,8 @@ "value": "Del40-p53gamma" } ], - "isoformIds": [ - "P04637-6" - ], - "sequenceIds": [ - "VSP_040832", - "VSP_040560", - "VSP_040561" - ], + "isoformIds": ["P04637-6"], + "sequenceIds": ["VSP_040832", "VSP_040560", "VSP_040561"], "isoformSequenceStatus": "Described" }, { @@ -3707,12 +3676,8 @@ "value": "Del133-p53alpha" } ], - "isoformIds": [ - "P04637-7" - ], - "sequenceIds": [ - "VSP_040833" - ], + "isoformIds": ["P04637-7"], + "sequenceIds": ["VSP_040833"], "isoformSequenceStatus": "Described" }, { @@ -3724,14 +3689,8 @@ "value": "Del133-p53beta" } ], - "isoformIds": [ - "P04637-8" - ], - "sequenceIds": [ - "VSP_040833", - "VSP_006535", - "VSP_006536" - ], + "isoformIds": ["P04637-8"], + "sequenceIds": ["VSP_040833", "VSP_006535", "VSP_006536"], "isoformSequenceStatus": "Described" }, { @@ -3743,14 +3702,8 @@ "value": "Del133-p53gamma" } ], - "isoformIds": [ - "P04637-9" - ], - "sequenceIds": [ - "VSP_040833", - "VSP_040560", - "VSP_040561" - ], + "isoformIds": ["P04637-9"], + "sequenceIds": ["VSP_040833", "VSP_040560", "VSP_040561"], "isoformSequenceStatus": "Described" } ] @@ -6614,9 +6567,7 @@ "featureId": "VSP_040560", "alternativeSequence": { "originalSequence": "IRGRERFEMFRELNE", - "alternativeSequences": [ - "MLLDLRWCYFLINSS" - ] + "alternativeSequences": ["MLLDLRWCYFLINSS"] } }, { @@ -6642,9 +6593,7 @@ "featureId": "VSP_006535", "alternativeSequence": { "originalSequence": "IRGRERFEMF", - "alternativeSequences": [ - "DQTSFQKENC" - ] + "alternativeSequences": ["DQTSFQKENC"] } }, { @@ -6709,9 +6658,7 @@ "featureId": "VAR_044543", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -6730,9 +6677,7 @@ "featureId": "VAR_044544", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -6757,9 +6702,7 @@ "featureId": "VAR_005851", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -6778,9 +6721,7 @@ "featureId": "VAR_044545", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -6805,9 +6746,7 @@ "featureId": "VAR_044546", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -6832,9 +6771,7 @@ "featureId": "VAR_044547", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -6859,9 +6796,7 @@ "featureId": "VAR_044548", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -6880,9 +6815,7 @@ "featureId": "VAR_044549", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -6901,9 +6834,7 @@ "featureId": "VAR_044550", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -6922,9 +6853,7 @@ "featureId": "VAR_044551", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -6943,9 +6872,7 @@ "featureId": "VAR_044552", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -6964,9 +6891,7 @@ "featureId": "VAR_044553", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -6985,9 +6910,7 @@ "featureId": "VAR_047158", "alternativeSequence": { "originalSequence": "NN", - "alternativeSequences": [ - "KD" - ] + "alternativeSequences": ["KD"] } }, { @@ -7012,9 +6935,7 @@ "featureId": "VAR_044554", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -7033,9 +6954,7 @@ "featureId": "VAR_044555", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -7054,9 +6973,7 @@ "featureId": "VAR_044556", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -7081,9 +6998,7 @@ "featureId": "VAR_005852", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -7108,9 +7023,7 @@ "featureId": "VAR_044557", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -7129,9 +7042,7 @@ "featureId": "VAR_044558", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -7150,9 +7061,7 @@ "featureId": "VAR_044559", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -7171,9 +7080,7 @@ "featureId": "VAR_044560", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -7198,9 +7105,7 @@ "featureId": "VAR_044561", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -7219,9 +7124,7 @@ "featureId": "VAR_044562", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -7240,9 +7143,7 @@ "featureId": "VAR_005853", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -7267,9 +7168,7 @@ "featureId": "VAR_044563", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -7288,9 +7187,7 @@ "featureId": "VAR_044564", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -7309,9 +7206,7 @@ "featureId": "VAR_044565", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -7330,9 +7225,7 @@ "featureId": "VAR_044566", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -7351,9 +7244,7 @@ "featureId": "VAR_044567", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -7378,9 +7269,7 @@ "featureId": "VAR_044568", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -7399,9 +7288,7 @@ "featureId": "VAR_044569", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -7433,9 +7320,7 @@ "featureId": "VAR_014632", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -7454,9 +7339,7 @@ "featureId": "VAR_044570", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -7481,9 +7364,7 @@ "featureId": "VAR_044571", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -7508,9 +7389,7 @@ "featureId": "VAR_044572", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -7529,9 +7408,7 @@ "featureId": "VAR_044573", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -7550,9 +7427,7 @@ "featureId": "VAR_044574", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -7571,9 +7446,7 @@ "featureId": "VAR_005854", "alternativeSequence": { "originalSequence": "W", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -7592,9 +7465,7 @@ "featureId": "VAR_044575", "alternativeSequence": { "originalSequence": "W", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -7619,9 +7490,7 @@ "featureId": "VAR_044576", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -7640,9 +7509,7 @@ "featureId": "VAR_044577", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -7661,9 +7528,7 @@ "featureId": "VAR_044578", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -7682,9 +7547,7 @@ "featureId": "VAR_044579", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -7703,9 +7566,7 @@ "featureId": "VAR_044580", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -7724,9 +7585,7 @@ "featureId": "VAR_044581", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -7751,9 +7610,7 @@ "featureId": "VAR_044582", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -7772,9 +7629,7 @@ "featureId": "VAR_044583", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -7793,9 +7648,7 @@ "featureId": "VAR_045783", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -7814,9 +7667,7 @@ "featureId": "VAR_044584", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -7835,9 +7686,7 @@ "featureId": "VAR_044585", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -7856,9 +7705,7 @@ "featureId": "VAR_005855", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -7883,9 +7730,7 @@ "featureId": "VAR_044586", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -7904,9 +7749,7 @@ "featureId": "VAR_044587", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -7925,9 +7768,7 @@ "featureId": "VAR_044588", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -7952,9 +7793,7 @@ "featureId": "VAR_044589", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -7979,9 +7818,7 @@ "featureId": "VAR_044590", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -8006,9 +7843,7 @@ "featureId": "VAR_044591", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -8033,9 +7868,7 @@ "featureId": "VAR_044592", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -8054,9 +7887,7 @@ "featureId": "VAR_044593", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -8075,9 +7906,7 @@ "featureId": "VAR_044594", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -8096,9 +7925,7 @@ "featureId": "VAR_044595", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -8117,9 +7944,7 @@ "featureId": "VAR_044596", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -8138,9 +7963,7 @@ "featureId": "VAR_044597", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -8159,9 +7982,7 @@ "featureId": "VAR_044598", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -8180,9 +8001,7 @@ "featureId": "VAR_044599", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -8207,9 +8026,7 @@ "featureId": "VAR_044600", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -8228,9 +8045,7 @@ "featureId": "VAR_044601", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -8255,9 +8070,7 @@ "featureId": "VAR_044602", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -8276,9 +8089,7 @@ "featureId": "VAR_044603", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -8297,9 +8108,7 @@ "featureId": "VAR_044604", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -8324,9 +8133,7 @@ "featureId": "VAR_045784", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -8345,9 +8152,7 @@ "featureId": "VAR_045785", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -8372,9 +8177,7 @@ "featureId": "VAR_045786", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -8393,9 +8196,7 @@ "featureId": "VAR_045787", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -8442,9 +8243,7 @@ "featureId": "VAR_005856", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -8463,9 +8262,7 @@ "featureId": "VAR_044605", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -8484,9 +8281,7 @@ "featureId": "VAR_044606", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -8511,9 +8306,7 @@ "featureId": "VAR_044607", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -8532,9 +8325,7 @@ "featureId": "VAR_044608", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -8553,9 +8344,7 @@ "featureId": "VAR_044609", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -8574,9 +8363,7 @@ "featureId": "VAR_044610", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -8595,9 +8382,7 @@ "featureId": "VAR_044611", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -8623,9 +8408,7 @@ "featureId": "VAR_044612", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -8644,9 +8427,7 @@ "featureId": "VAR_044613", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -8665,9 +8446,7 @@ "featureId": "VAR_044614", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -8686,9 +8465,7 @@ "featureId": "VAR_044615", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -8707,9 +8484,7 @@ "featureId": "VAR_044616", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -8728,9 +8503,7 @@ "featureId": "VAR_005857", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -8749,9 +8522,7 @@ "featureId": "VAR_044617", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -8770,9 +8541,7 @@ "featureId": "VAR_044618", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -8797,9 +8566,7 @@ "featureId": "VAR_044619", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -8818,9 +8585,7 @@ "featureId": "VAR_044620", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -8845,9 +8610,7 @@ "featureId": "VAR_044621", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -8866,9 +8629,7 @@ "featureId": "VAR_044622", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -8893,9 +8654,7 @@ "featureId": "VAR_044623", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -8920,9 +8679,7 @@ "featureId": "VAR_044624", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -8941,9 +8698,7 @@ "featureId": "VAR_044625", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -8962,9 +8717,7 @@ "featureId": "VAR_044626", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -8983,9 +8736,7 @@ "featureId": "VAR_044627", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -9004,9 +8755,7 @@ "featureId": "VAR_044628", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -9025,9 +8774,7 @@ "featureId": "VAR_044629", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -9046,9 +8793,7 @@ "featureId": "VAR_005858", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -9067,9 +8812,7 @@ "featureId": "VAR_044630", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -9094,9 +8837,7 @@ "featureId": "VAR_044631", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -9121,9 +8862,7 @@ "featureId": "VAR_044632", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -9142,9 +8881,7 @@ "featureId": "VAR_044633", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -9169,9 +8906,7 @@ "featureId": "VAR_044634", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -9190,9 +8925,7 @@ "featureId": "VAR_044635", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -9211,9 +8944,7 @@ "featureId": "VAR_044636", "alternativeSequence": { "originalSequence": "W", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -9232,9 +8963,7 @@ "featureId": "VAR_044637", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -9259,9 +8988,7 @@ "featureId": "VAR_044638", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -9280,9 +9007,7 @@ "featureId": "VAR_044639", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -9301,9 +9026,7 @@ "featureId": "VAR_044640", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -9322,9 +9045,7 @@ "featureId": "VAR_044641", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -9343,9 +9064,7 @@ "featureId": "VAR_044642", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -9364,9 +9083,7 @@ "featureId": "VAR_005859", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -9385,9 +9102,7 @@ "featureId": "VAR_044643", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -9406,9 +9121,7 @@ "featureId": "VAR_044644", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -9427,9 +9140,7 @@ "featureId": "VAR_044645", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -9448,9 +9159,7 @@ "featureId": "VAR_044646", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -9469,9 +9178,7 @@ "featureId": "VAR_044647", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -9490,9 +9197,7 @@ "featureId": "VAR_044648", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -9511,9 +9216,7 @@ "featureId": "VAR_044649", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -9538,9 +9241,7 @@ "featureId": "VAR_044650", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -9565,9 +9266,7 @@ "featureId": "VAR_044651", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -9586,9 +9285,7 @@ "featureId": "VAR_044652", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -9607,9 +9304,7 @@ "featureId": "VAR_044653", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -9628,9 +9323,7 @@ "featureId": "VAR_044654", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -9649,9 +9342,7 @@ "featureId": "VAR_044655", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -9676,9 +9367,7 @@ "featureId": "VAR_044656", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -9697,9 +9386,7 @@ "featureId": "VAR_044657", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -9724,9 +9411,7 @@ "featureId": "VAR_044658", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -9745,9 +9430,7 @@ "featureId": "VAR_044659", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -9766,9 +9449,7 @@ "featureId": "VAR_044660", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -9787,9 +9468,7 @@ "featureId": "VAR_044661", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -9814,9 +9493,7 @@ "featureId": "VAR_044662", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -9841,9 +9518,7 @@ "featureId": "VAR_044663", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -9868,9 +9543,7 @@ "featureId": "VAR_044664", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -9889,9 +9562,7 @@ "featureId": "VAR_044665", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -9910,9 +9581,7 @@ "featureId": "VAR_044666", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -9937,9 +9606,7 @@ "featureId": "VAR_044667", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -9964,9 +9631,7 @@ "featureId": "VAR_044668", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -9985,9 +9650,7 @@ "featureId": "VAR_044669", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -10012,9 +9675,7 @@ "featureId": "VAR_044670", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -10033,9 +9694,7 @@ "featureId": "VAR_044671", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -10060,9 +9719,7 @@ "featureId": "VAR_044672", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -10081,9 +9738,7 @@ "featureId": "VAR_044673", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -10102,9 +9757,7 @@ "featureId": "VAR_044674", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -10129,9 +9782,7 @@ "featureId": "VAR_044675", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -10156,9 +9807,7 @@ "featureId": "VAR_005860", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -10177,9 +9826,7 @@ "featureId": "VAR_044676", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -10204,9 +9851,7 @@ "featureId": "VAR_044677", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -10238,9 +9883,7 @@ "featureId": "VAR_005861", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -10272,9 +9915,7 @@ "featureId": "VAR_005862", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -10299,9 +9940,7 @@ "featureId": "VAR_044678", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -10320,9 +9959,7 @@ "featureId": "VAR_044679", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -10347,9 +9984,7 @@ "featureId": "VAR_044680", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -10374,9 +10009,7 @@ "featureId": "VAR_044681", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -10401,9 +10034,7 @@ "featureId": "VAR_044682", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -10422,9 +10053,7 @@ "featureId": "VAR_044683", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -10449,9 +10078,7 @@ "featureId": "VAR_044684", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -10470,9 +10097,7 @@ "featureId": "VAR_005863", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -10491,9 +10116,7 @@ "featureId": "VAR_045788", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -10512,9 +10135,7 @@ "featureId": "VAR_044685", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -10533,9 +10154,7 @@ "featureId": "VAR_044686", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -10554,9 +10173,7 @@ "featureId": "VAR_044687", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -10588,9 +10205,7 @@ "featureId": "VAR_033033", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -10609,9 +10224,7 @@ "featureId": "VAR_044688", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -10630,9 +10243,7 @@ "featureId": "VAR_044689", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -10651,9 +10262,7 @@ "featureId": "VAR_044690", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -10672,9 +10281,7 @@ "featureId": "VAR_044691", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -10699,9 +10306,7 @@ "featureId": "VAR_044692", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -10726,9 +10331,7 @@ "featureId": "VAR_044693", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -10747,9 +10350,7 @@ "featureId": "VAR_044694", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -10774,9 +10375,7 @@ "featureId": "VAR_044695", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -10795,9 +10394,7 @@ "featureId": "VAR_044696", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -10816,9 +10413,7 @@ "featureId": "VAR_044697", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -10837,9 +10432,7 @@ "featureId": "VAR_044698", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -10876,9 +10469,7 @@ "featureId": "VAR_044699", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -10904,9 +10495,7 @@ "featureId": "VAR_044700", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -10938,9 +10527,7 @@ "featureId": "VAR_044701", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -10966,9 +10553,7 @@ "featureId": "VAR_044702", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -10987,9 +10572,7 @@ "featureId": "VAR_044703", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -11008,9 +10591,7 @@ "featureId": "VAR_044704", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -11035,9 +10616,7 @@ "featureId": "VAR_044705", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -11056,9 +10635,7 @@ "featureId": "VAR_044706", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -11083,9 +10660,7 @@ "featureId": "VAR_044707", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -11104,9 +10679,7 @@ "featureId": "VAR_044708", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -11131,9 +10704,7 @@ "featureId": "VAR_044709", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -11152,9 +10723,7 @@ "featureId": "VAR_044710", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -11173,9 +10742,7 @@ "featureId": "VAR_044711", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -11194,9 +10761,7 @@ "featureId": "VAR_044712", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -11221,9 +10786,7 @@ "featureId": "VAR_044713", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -11248,9 +10811,7 @@ "featureId": "VAR_005864", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -11275,9 +10836,7 @@ "featureId": "VAR_044714", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -11302,9 +10861,7 @@ "featureId": "VAR_044715", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -11329,9 +10886,7 @@ "featureId": "VAR_044716", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -11356,9 +10911,7 @@ "featureId": "VAR_005865", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -11377,9 +10930,7 @@ "featureId": "VAR_044717", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -11398,9 +10949,7 @@ "featureId": "VAR_045789", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -11419,9 +10968,7 @@ "featureId": "VAR_044718", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -11446,9 +10993,7 @@ "featureId": "VAR_005866", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -11467,9 +11012,7 @@ "featureId": "VAR_044719", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -11494,9 +11037,7 @@ "featureId": "VAR_044720", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -11521,9 +11062,7 @@ "featureId": "VAR_005867", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -11542,9 +11081,7 @@ "featureId": "VAR_044721", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -11563,9 +11100,7 @@ "featureId": "VAR_044722", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -11584,9 +11119,7 @@ "featureId": "VAR_044723", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -11605,9 +11138,7 @@ "featureId": "VAR_044724", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -11626,9 +11157,7 @@ "featureId": "VAR_044725", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -11647,9 +11176,7 @@ "featureId": "VAR_044726", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -11668,9 +11195,7 @@ "featureId": "VAR_005868", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -11689,9 +11214,7 @@ "featureId": "VAR_005869", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -11710,9 +11233,7 @@ "featureId": "VAR_044727", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -11737,9 +11258,7 @@ "featureId": "VAR_044728", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -11764,9 +11283,7 @@ "featureId": "VAR_044729", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -11791,9 +11308,7 @@ "featureId": "VAR_044730", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -11812,9 +11327,7 @@ "featureId": "VAR_044731", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -11833,9 +11346,7 @@ "featureId": "VAR_044732", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -11860,9 +11371,7 @@ "featureId": "VAR_044733", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -11881,9 +11390,7 @@ "featureId": "VAR_005870", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -11908,9 +11415,7 @@ "featureId": "VAR_044734", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -11929,9 +11434,7 @@ "featureId": "VAR_044735", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -11950,9 +11453,7 @@ "featureId": "VAR_044736", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -11977,9 +11478,7 @@ "featureId": "VAR_044737", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -12004,9 +11503,7 @@ "featureId": "VAR_005872", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -12025,9 +11522,7 @@ "featureId": "VAR_005871", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -12046,9 +11541,7 @@ "featureId": "VAR_044738", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -12073,9 +11566,7 @@ "featureId": "VAR_044739", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -12094,9 +11585,7 @@ "featureId": "VAR_047159", "alternativeSequence": { "originalSequence": "KM", - "alternativeSequences": [ - "NL" - ] + "alternativeSequences": ["NL"] } }, { @@ -12121,9 +11610,7 @@ "featureId": "VAR_044740", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -12142,9 +11629,7 @@ "featureId": "VAR_045790", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -12169,9 +11654,7 @@ "featureId": "VAR_005873", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -12196,9 +11679,7 @@ "featureId": "VAR_044741", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -12230,9 +11711,7 @@ "featureId": "VAR_005874", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -12257,9 +11736,7 @@ "featureId": "VAR_044742", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -12284,9 +11761,7 @@ "featureId": "VAR_044743", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -12305,9 +11780,7 @@ "featureId": "VAR_045791", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -12332,9 +11805,7 @@ "featureId": "VAR_044744", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -12353,9 +11824,7 @@ "featureId": "VAR_044745", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -12374,9 +11843,7 @@ "featureId": "VAR_044746", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -12395,9 +11862,7 @@ "featureId": "VAR_044747", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -12429,9 +11894,7 @@ "featureId": "VAR_005875", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -12456,9 +11919,7 @@ "featureId": "VAR_044748", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -12483,9 +11944,7 @@ "featureId": "VAR_044749", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -12504,9 +11963,7 @@ "featureId": "VAR_044750", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -12538,9 +11995,7 @@ "featureId": "VAR_036504", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -12559,9 +12014,7 @@ "featureId": "VAR_044751", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -12580,9 +12033,7 @@ "featureId": "VAR_044752", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -12607,9 +12058,7 @@ "featureId": "VAR_005877", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -12634,9 +12083,7 @@ "featureId": "VAR_044753", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -12661,9 +12108,7 @@ "featureId": "VAR_044754", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -12688,9 +12133,7 @@ "featureId": "VAR_005876", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -12709,9 +12152,7 @@ "featureId": "VAR_045792", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -12736,9 +12177,7 @@ "featureId": "VAR_044755", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -12763,9 +12202,7 @@ "featureId": "VAR_044756", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -12790,9 +12227,7 @@ "featureId": "VAR_005878", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -12817,9 +12252,7 @@ "featureId": "VAR_044757", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -12838,9 +12271,7 @@ "featureId": "VAR_005879", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -12859,9 +12290,7 @@ "featureId": "VAR_044758", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -12880,9 +12309,7 @@ "featureId": "VAR_044759", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -12901,9 +12328,7 @@ "featureId": "VAR_044760", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -12922,9 +12347,7 @@ "featureId": "VAR_044761", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -12943,9 +12366,7 @@ "featureId": "VAR_005880", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -12964,9 +12385,7 @@ "featureId": "VAR_044762", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -12985,9 +12404,7 @@ "featureId": "VAR_044763", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -13012,9 +12429,7 @@ "featureId": "VAR_005881", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -13033,9 +12448,7 @@ "featureId": "VAR_044764", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -13054,9 +12467,7 @@ "featureId": "VAR_044765", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -13093,9 +12504,7 @@ "featureId": "VAR_033034", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -13127,9 +12536,7 @@ "featureId": "VAR_044766", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -13160,9 +12567,7 @@ "featureId": "VAR_005882", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -13188,9 +12593,7 @@ "featureId": "VAR_044767", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -13209,9 +12612,7 @@ "featureId": "VAR_044768", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -13237,9 +12638,7 @@ "featureId": "VAR_044769", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -13258,9 +12657,7 @@ "featureId": "VAR_044770", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -13279,9 +12676,7 @@ "featureId": "VAR_044771", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -13306,9 +12701,7 @@ "featureId": "VAR_044772", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -13327,9 +12720,7 @@ "featureId": "VAR_044773", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -13348,9 +12739,7 @@ "featureId": "VAR_044774", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -13369,9 +12758,7 @@ "featureId": "VAR_045793", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -13396,9 +12783,7 @@ "featureId": "VAR_005885", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -13423,9 +12808,7 @@ "featureId": "VAR_005884", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -13450,9 +12833,7 @@ "featureId": "VAR_044775", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -13477,9 +12858,7 @@ "featureId": "VAR_044776", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -13504,9 +12883,7 @@ "featureId": "VAR_044777", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -13531,9 +12908,7 @@ "featureId": "VAR_005886", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -13552,9 +12927,7 @@ "featureId": "VAR_044778", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -13573,9 +12946,7 @@ "featureId": "VAR_045794", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -13594,9 +12965,7 @@ "featureId": "VAR_044779", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -13621,9 +12990,7 @@ "featureId": "VAR_044780", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -13642,9 +13009,7 @@ "featureId": "VAR_044781", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -13663,9 +13028,7 @@ "featureId": "VAR_044782", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -13684,9 +13047,7 @@ "featureId": "VAR_044783", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -13717,9 +13078,7 @@ "featureId": "VAR_005887", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -13738,9 +13097,7 @@ "featureId": "VAR_044784", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -13765,9 +13122,7 @@ "featureId": "VAR_044785", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -13792,9 +13147,7 @@ "featureId": "VAR_044786", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -13819,9 +13172,7 @@ "featureId": "VAR_044787", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -13846,9 +13197,7 @@ "featureId": "VAR_044788", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -13867,9 +13216,7 @@ "featureId": "VAR_044789", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -13894,9 +13241,7 @@ "featureId": "VAR_044790", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -13921,9 +13266,7 @@ "featureId": "VAR_005888", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -13942,9 +13285,7 @@ "featureId": "VAR_044791", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -13963,9 +13304,7 @@ "featureId": "VAR_044792", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -13990,9 +13329,7 @@ "featureId": "VAR_005889", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -14011,9 +13348,7 @@ "featureId": "VAR_005890", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -14032,9 +13367,7 @@ "featureId": "VAR_044793", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -14053,9 +13386,7 @@ "featureId": "VAR_044794", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -14074,9 +13405,7 @@ "featureId": "VAR_044795", "alternativeSequence": { "originalSequence": "W", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -14101,9 +13430,7 @@ "featureId": "VAR_044796", "alternativeSequence": { "originalSequence": "W", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -14122,9 +13449,7 @@ "featureId": "VAR_044797", "alternativeSequence": { "originalSequence": "W", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -14143,9 +13468,7 @@ "featureId": "VAR_044798", "alternativeSequence": { "originalSequence": "W", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -14164,9 +13487,7 @@ "featureId": "VAR_044799", "alternativeSequence": { "originalSequence": "W", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -14185,9 +13506,7 @@ "featureId": "VAR_044800", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -14206,9 +13525,7 @@ "featureId": "VAR_005891", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -14227,9 +13544,7 @@ "featureId": "VAR_044801", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -14248,9 +13563,7 @@ "featureId": "VAR_044802", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -14275,9 +13588,7 @@ "featureId": "VAR_005892", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -14302,9 +13613,7 @@ "featureId": "VAR_044803", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -14329,9 +13638,7 @@ "featureId": "VAR_044804", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -14350,9 +13657,7 @@ "featureId": "VAR_044805", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -14371,9 +13676,7 @@ "featureId": "VAR_044806", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -14392,9 +13695,7 @@ "featureId": "VAR_044807", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -14413,9 +13714,7 @@ "featureId": "VAR_044808", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -14440,9 +13739,7 @@ "featureId": "VAR_044809", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -14467,9 +13764,7 @@ "featureId": "VAR_044810", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -14488,9 +13783,7 @@ "featureId": "VAR_005893", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -14509,9 +13802,7 @@ "featureId": "VAR_044811", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -14530,9 +13821,7 @@ "featureId": "VAR_044812", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -14551,9 +13840,7 @@ "featureId": "VAR_044813", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -14572,9 +13859,7 @@ "featureId": "VAR_044814", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -14593,9 +13878,7 @@ "featureId": "VAR_044815", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -14614,9 +13897,7 @@ "featureId": "VAR_044816", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -14635,9 +13916,7 @@ "featureId": "VAR_044817", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -14662,9 +13941,7 @@ "featureId": "VAR_005894", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -14689,9 +13966,7 @@ "featureId": "VAR_044818", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -14710,9 +13985,7 @@ "featureId": "VAR_044819", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -14737,9 +14010,7 @@ "featureId": "VAR_044820", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -14771,9 +14042,7 @@ "featureId": "VAR_005895", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -14798,9 +14067,7 @@ "featureId": "VAR_005896", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -14819,9 +14086,7 @@ "featureId": "VAR_044821", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -14853,9 +14118,7 @@ "featureId": "VAR_005897", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -14874,9 +14137,7 @@ "featureId": "VAR_044822", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -14895,9 +14156,7 @@ "featureId": "VAR_044823", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -14929,9 +14188,7 @@ "featureId": "VAR_005898", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -14950,9 +14207,7 @@ "featureId": "VAR_044824", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -14971,9 +14226,7 @@ "featureId": "VAR_044825", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -14992,9 +14245,7 @@ "featureId": "VAR_045795", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -15013,9 +14264,7 @@ "featureId": "VAR_044826", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -15034,9 +14283,7 @@ "featureId": "VAR_044827", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -15055,9 +14302,7 @@ "featureId": "VAR_044828", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -15082,9 +14327,7 @@ "featureId": "VAR_044829", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -15103,9 +14346,7 @@ "featureId": "VAR_005899", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -15124,9 +14365,7 @@ "featureId": "VAR_044830", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -15145,9 +14384,7 @@ "featureId": "VAR_044831", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -15172,9 +14409,7 @@ "featureId": "VAR_044832", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -15193,9 +14428,7 @@ "featureId": "VAR_045796", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -15220,9 +14453,7 @@ "featureId": "VAR_044833", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -15254,9 +14485,7 @@ "featureId": "VAR_005900", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -15288,9 +14517,7 @@ "featureId": "VAR_005901", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -15309,9 +14536,7 @@ "featureId": "VAR_044834", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -15330,9 +14555,7 @@ "featureId": "VAR_044835", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -15357,9 +14580,7 @@ "featureId": "VAR_044836", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -15390,9 +14611,7 @@ "featureId": "VAR_044837", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -15417,9 +14636,7 @@ "featureId": "VAR_044838", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -15444,9 +14661,7 @@ "featureId": "VAR_044839", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -15465,9 +14680,7 @@ "featureId": "VAR_044840", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -15492,9 +14705,7 @@ "featureId": "VAR_044841", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -15513,9 +14724,7 @@ "featureId": "VAR_044842", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -15541,9 +14750,7 @@ "featureId": "VAR_005902", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -15562,9 +14769,7 @@ "featureId": "VAR_044843", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -15589,9 +14794,7 @@ "featureId": "VAR_044844", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -15610,9 +14813,7 @@ "featureId": "VAR_005903", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -15644,9 +14845,7 @@ "featureId": "VAR_005904", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -15665,9 +14864,7 @@ "featureId": "VAR_044845", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -15699,9 +14896,7 @@ "featureId": "VAR_012977", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -15720,9 +14915,7 @@ "featureId": "VAR_044846", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -15747,9 +14940,7 @@ "featureId": "VAR_005905", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -15768,9 +14959,7 @@ "featureId": "VAR_045797", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -15789,9 +14978,7 @@ "featureId": "VAR_005906", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -15816,9 +15003,7 @@ "featureId": "VAR_005907", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -15837,9 +15022,7 @@ "featureId": "VAR_044847", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -15864,9 +15047,7 @@ "featureId": "VAR_044848", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -15885,9 +15066,7 @@ "featureId": "VAR_044849", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -15906,9 +15085,7 @@ "featureId": "VAR_044850", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -15927,9 +15104,7 @@ "featureId": "VAR_045798", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -15948,9 +15123,7 @@ "featureId": "VAR_044851", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -15975,9 +15148,7 @@ "featureId": "VAR_045799", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -15996,9 +15167,7 @@ "featureId": "VAR_044852", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -16023,9 +15192,7 @@ "featureId": "VAR_044853", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -16044,9 +15211,7 @@ "featureId": "VAR_044854", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -16071,9 +15236,7 @@ "featureId": "VAR_044855", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -16098,9 +15261,7 @@ "featureId": "VAR_044856", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -16119,9 +15280,7 @@ "featureId": "VAR_047160", "alternativeSequence": { "originalSequence": "MA", - "alternativeSequences": [ - "IP" - ] + "alternativeSequences": ["IP"] } }, { @@ -16140,9 +15299,7 @@ "featureId": "VAR_047161", "alternativeSequence": { "originalSequence": "MA", - "alternativeSequences": [ - "IS" - ] + "alternativeSequences": ["IS"] } }, { @@ -16161,9 +15318,7 @@ "featureId": "VAR_047162", "alternativeSequence": { "originalSequence": "MA", - "alternativeSequences": [ - "IT" - ] + "alternativeSequences": ["IT"] } }, { @@ -16188,9 +15343,7 @@ "featureId": "VAR_005908", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -16209,9 +15362,7 @@ "featureId": "VAR_044857", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -16230,9 +15381,7 @@ "featureId": "VAR_044858", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -16257,9 +15406,7 @@ "featureId": "VAR_044859", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -16284,9 +15431,7 @@ "featureId": "VAR_044860", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -16305,9 +15450,7 @@ "featureId": "VAR_045800", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -16326,9 +15469,7 @@ "featureId": "VAR_044861", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -16347,9 +15488,7 @@ "featureId": "VAR_044862", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -16368,9 +15507,7 @@ "featureId": "VAR_005909", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -16395,9 +15532,7 @@ "featureId": "VAR_044863", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -16416,9 +15551,7 @@ "featureId": "VAR_044864", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -16437,9 +15570,7 @@ "featureId": "VAR_044865", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -16458,9 +15589,7 @@ "featureId": "VAR_044866", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -16479,9 +15608,7 @@ "featureId": "VAR_044867", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -16506,9 +15633,7 @@ "featureId": "VAR_005910", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -16527,9 +15652,7 @@ "featureId": "VAR_044868", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -16548,9 +15671,7 @@ "featureId": "VAR_005911", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -16587,9 +15708,7 @@ "featureId": "VAR_033035", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -16614,9 +15733,7 @@ "featureId": "VAR_044869", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -16635,9 +15752,7 @@ "featureId": "VAR_044870", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -16669,9 +15784,7 @@ "featureId": "VAR_005912", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -16696,9 +15809,7 @@ "featureId": "VAR_044871", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -16717,9 +15828,7 @@ "featureId": "VAR_044872", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -16744,9 +15853,7 @@ "featureId": "VAR_044873", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -16765,9 +15872,7 @@ "featureId": "VAR_044874", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -16792,9 +15897,7 @@ "featureId": "VAR_005913", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -16813,9 +15916,7 @@ "featureId": "VAR_005914", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -16834,9 +15935,7 @@ "featureId": "VAR_044875", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -16855,9 +15954,7 @@ "featureId": "VAR_044876", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -16876,9 +15973,7 @@ "featureId": "VAR_044877", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -16897,9 +15992,7 @@ "featureId": "VAR_044878", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -16918,9 +16011,7 @@ "featureId": "VAR_005915", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -16939,9 +16030,7 @@ "featureId": "VAR_044879", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -16960,9 +16049,7 @@ "featureId": "VAR_005916", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -16981,9 +16068,7 @@ "featureId": "VAR_044880", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -17002,9 +16087,7 @@ "featureId": "VAR_044881", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -17029,9 +16112,7 @@ "featureId": "VAR_005917", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -17050,9 +16131,7 @@ "featureId": "VAR_044882", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -17071,9 +16150,7 @@ "featureId": "VAR_044883", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -17092,9 +16169,7 @@ "featureId": "VAR_047163", "alternativeSequence": { "originalSequence": "QH", - "alternativeSequences": [ - "HD" - ] + "alternativeSequences": ["HD"] } }, { @@ -17113,9 +16188,7 @@ "featureId": "VAR_047164", "alternativeSequence": { "originalSequence": "QH", - "alternativeSequences": [ - "YL" - ] + "alternativeSequences": ["YL"] } }, { @@ -17134,9 +16207,7 @@ "featureId": "VAR_044884", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -17155,9 +16226,7 @@ "featureId": "VAR_044885", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -17176,9 +16245,7 @@ "featureId": "VAR_044886", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -17197,9 +16264,7 @@ "featureId": "VAR_044887", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -17218,9 +16283,7 @@ "featureId": "VAR_047165", "alternativeSequence": { "originalSequence": "HM", - "alternativeSequences": [ - "LI" - ] + "alternativeSequences": ["LI"] } }, { @@ -17239,9 +16302,7 @@ "featureId": "VAR_044888", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -17260,9 +16321,7 @@ "featureId": "VAR_044889", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -17281,9 +16340,7 @@ "featureId": "VAR_044890", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -17302,9 +16359,7 @@ "featureId": "VAR_044891", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -17323,9 +16378,7 @@ "featureId": "VAR_044892", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -17350,9 +16403,7 @@ "featureId": "VAR_005918", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -17371,9 +16422,7 @@ "featureId": "VAR_045801", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -17392,9 +16441,7 @@ "featureId": "VAR_044893", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -17413,9 +16460,7 @@ "featureId": "VAR_047166", "alternativeSequence": { "originalSequence": "MT", - "alternativeSequences": [ - "IS" - ] + "alternativeSequences": ["IS"] } }, { @@ -17441,9 +16486,7 @@ "featureId": "VAR_005919", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -17462,9 +16505,7 @@ "featureId": "VAR_044894", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -17483,9 +16524,7 @@ "featureId": "VAR_005920", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -17504,9 +16543,7 @@ "featureId": "VAR_044895", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -17531,9 +16568,7 @@ "featureId": "VAR_044896", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -17552,9 +16587,7 @@ "featureId": "VAR_044897", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -17579,9 +16612,7 @@ "featureId": "VAR_005921", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -17600,9 +16631,7 @@ "featureId": "VAR_044898", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -17621,9 +16650,7 @@ "featureId": "VAR_005922", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -17642,9 +16669,7 @@ "featureId": "VAR_044899", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -17663,9 +16688,7 @@ "featureId": "VAR_044900", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -17684,9 +16707,7 @@ "featureId": "VAR_044901", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -17711,9 +16732,7 @@ "featureId": "VAR_044902", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -17732,9 +16751,7 @@ "featureId": "VAR_044903", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -17753,9 +16770,7 @@ "featureId": "VAR_044904", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -17774,9 +16789,7 @@ "featureId": "VAR_005923", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -17795,9 +16808,7 @@ "featureId": "VAR_044905", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -17822,9 +16833,7 @@ "featureId": "VAR_044906", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -17849,9 +16858,7 @@ "featureId": "VAR_044907", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -17870,9 +16877,7 @@ "featureId": "VAR_044908", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -17897,9 +16902,7 @@ "featureId": "VAR_044909", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -17924,9 +16927,7 @@ "featureId": "VAR_005924", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -17951,9 +16952,7 @@ "featureId": "VAR_044910", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -17978,9 +16977,7 @@ "featureId": "VAR_005925", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -18005,9 +17002,7 @@ "featureId": "VAR_005926", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -18026,9 +17021,7 @@ "featureId": "VAR_045802", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -18053,9 +17046,7 @@ "featureId": "VAR_044911", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -18087,9 +17078,7 @@ "featureId": "VAR_005927", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -18108,9 +17097,7 @@ "featureId": "VAR_044912", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -18129,9 +17116,7 @@ "featureId": "VAR_044913", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -18150,9 +17135,7 @@ "featureId": "VAR_044914", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -18171,9 +17154,7 @@ "featureId": "VAR_044915", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -18198,9 +17179,7 @@ "featureId": "VAR_005928", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -18225,9 +17204,7 @@ "featureId": "VAR_005929", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -18289,9 +17266,7 @@ "featureId": "VAR_005932", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -18316,9 +17291,7 @@ "featureId": "VAR_005930", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -18344,9 +17317,7 @@ "featureId": "VAR_005931", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -18365,9 +17336,7 @@ "featureId": "VAR_044916", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -18386,9 +17355,7 @@ "featureId": "VAR_044917", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -18407,9 +17374,7 @@ "featureId": "VAR_047167", "alternativeSequence": { "originalSequence": "CP", - "alternativeSequences": [ - "FS" - ] + "alternativeSequences": ["FS"] } }, { @@ -18456,9 +17421,7 @@ "featureId": "VAR_005933", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -18477,9 +17440,7 @@ "featureId": "VAR_044918", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -18498,9 +17459,7 @@ "featureId": "VAR_044919", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -18525,9 +17484,7 @@ "featureId": "VAR_044920", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -18552,9 +17509,7 @@ "featureId": "VAR_005934", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -18579,9 +17534,7 @@ "featureId": "VAR_044921", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -18600,9 +17553,7 @@ "featureId": "VAR_044922", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -18621,9 +17572,7 @@ "featureId": "VAR_045803", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -18642,9 +17591,7 @@ "featureId": "VAR_044923", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -18663,9 +17610,7 @@ "featureId": "VAR_045804", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -18690,9 +17635,7 @@ "featureId": "VAR_005935", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -18724,9 +17667,7 @@ "featureId": "VAR_036505", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -18751,9 +17692,7 @@ "featureId": "VAR_044924", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -18772,9 +17711,7 @@ "featureId": "VAR_044925", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -18793,9 +17730,7 @@ "featureId": "VAR_047168", "alternativeSequence": { "originalSequence": "HH", - "alternativeSequences": [ - "QS" - ] + "alternativeSequences": ["QS"] } }, { @@ -18814,9 +17749,7 @@ "featureId": "VAR_044926", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -18842,9 +17775,7 @@ "featureId": "VAR_005936", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "HPHP" - ] + "alternativeSequences": ["HPHP"] } }, { @@ -18863,9 +17794,7 @@ "featureId": "VAR_044927", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -18890,9 +17819,7 @@ "featureId": "VAR_044928", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -18917,9 +17844,7 @@ "featureId": "VAR_044929", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -18938,9 +17863,7 @@ "featureId": "VAR_044930", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -18959,9 +17882,7 @@ "featureId": "VAR_044931", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -18980,9 +17901,7 @@ "featureId": "VAR_044932", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -19007,9 +17926,7 @@ "featureId": "VAR_044933", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -19034,9 +17951,7 @@ "featureId": "VAR_044934", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -19061,9 +17976,7 @@ "featureId": "VAR_044935", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -19088,9 +18001,7 @@ "featureId": "VAR_044936", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -19115,9 +18026,7 @@ "featureId": "VAR_044937", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -19142,9 +18051,7 @@ "featureId": "VAR_044938", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -19169,9 +18076,7 @@ "featureId": "VAR_044939", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -19190,9 +18095,7 @@ "featureId": "VAR_044940", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -19211,9 +18114,7 @@ "featureId": "VAR_044941", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -19232,9 +18133,7 @@ "featureId": "VAR_044942", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -19259,9 +18158,7 @@ "featureId": "VAR_044943", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -19280,9 +18177,7 @@ "featureId": "VAR_044944", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -19301,9 +18196,7 @@ "featureId": "VAR_044945", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -19328,9 +18221,7 @@ "featureId": "VAR_044946", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -19349,9 +18240,7 @@ "featureId": "VAR_044947", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -19376,9 +18265,7 @@ "featureId": "VAR_044948", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -19403,9 +18290,7 @@ "featureId": "VAR_005937", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -19424,9 +18309,7 @@ "featureId": "VAR_044949", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -19451,9 +18334,7 @@ "featureId": "VAR_044950", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -19472,9 +18353,7 @@ "featureId": "VAR_044951", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -19493,9 +18372,7 @@ "featureId": "VAR_005938", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -19514,9 +18391,7 @@ "featureId": "VAR_044952", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -19541,9 +18416,7 @@ "featureId": "VAR_044953", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -19562,9 +18435,7 @@ "featureId": "VAR_044954", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -19589,9 +18460,7 @@ "featureId": "VAR_044955", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -19610,9 +18479,7 @@ "featureId": "VAR_044956", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -19637,9 +18504,7 @@ "featureId": "VAR_047169", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -19658,9 +18523,7 @@ "featureId": "VAR_044957", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -19679,9 +18542,7 @@ "featureId": "VAR_005939", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -19700,9 +18561,7 @@ "featureId": "VAR_044958", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -19721,9 +18580,7 @@ "featureId": "VAR_044959", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -19742,9 +18599,7 @@ "featureId": "VAR_044960", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -19769,9 +18624,7 @@ "featureId": "VAR_044961", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -19790,9 +18643,7 @@ "featureId": "VAR_044962", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -19811,9 +18662,7 @@ "featureId": "VAR_044963", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -19838,9 +18687,7 @@ "featureId": "VAR_044964", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -19859,9 +18706,7 @@ "featureId": "VAR_044965", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -19880,9 +18725,7 @@ "featureId": "VAR_044966", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -19907,9 +18750,7 @@ "featureId": "VAR_044967", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -19928,9 +18769,7 @@ "featureId": "VAR_044968", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -19949,9 +18788,7 @@ "featureId": "VAR_005940", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -19970,9 +18807,7 @@ "featureId": "VAR_005941", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -19991,9 +18826,7 @@ "featureId": "VAR_044969", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -20012,9 +18845,7 @@ "featureId": "VAR_045805", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -20033,9 +18864,7 @@ "featureId": "VAR_044970", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -20060,9 +18889,7 @@ "featureId": "VAR_005942", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -20081,9 +18908,7 @@ "featureId": "VAR_044971", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -20108,9 +18933,7 @@ "featureId": "VAR_044972", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -20129,9 +18952,7 @@ "featureId": "VAR_044973", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -20150,9 +18971,7 @@ "featureId": "VAR_044974", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -20171,9 +18990,7 @@ "featureId": "VAR_044975", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -20192,9 +19009,7 @@ "featureId": "VAR_005943", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -20219,9 +19034,7 @@ "featureId": "VAR_044976", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -20240,9 +19053,7 @@ "featureId": "VAR_044977", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -20267,9 +19078,7 @@ "featureId": "VAR_044978", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -20288,9 +19097,7 @@ "featureId": "VAR_044979", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -20309,9 +19116,7 @@ "featureId": "VAR_044980", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -20336,9 +19141,7 @@ "featureId": "VAR_005944", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -20363,9 +19166,7 @@ "featureId": "VAR_044981", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -20384,9 +19185,7 @@ "featureId": "VAR_044982", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -20411,9 +19210,7 @@ "featureId": "VAR_044983", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -20438,9 +19235,7 @@ "featureId": "VAR_044984", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -20465,9 +19260,7 @@ "featureId": "VAR_044985", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -20492,9 +19285,7 @@ "featureId": "VAR_044986", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -20519,9 +19310,7 @@ "featureId": "VAR_044987", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -20540,9 +19329,7 @@ "featureId": "VAR_005945", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -20561,9 +19348,7 @@ "featureId": "VAR_047170", "alternativeSequence": { "originalSequence": "QH", - "alternativeSequences": [ - "HN" - ] + "alternativeSequences": ["HN"] } }, { @@ -20582,9 +19367,7 @@ "featureId": "VAR_047171", "alternativeSequence": { "originalSequence": "QH", - "alternativeSequences": [ - "HY" - ] + "alternativeSequences": ["HY"] } }, { @@ -20603,9 +19386,7 @@ "featureId": "VAR_044988", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -20624,9 +19405,7 @@ "featureId": "VAR_044989", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -20645,9 +19424,7 @@ "featureId": "VAR_044990", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -20666,9 +19443,7 @@ "featureId": "VAR_044991", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -20693,9 +19468,7 @@ "featureId": "VAR_005946", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -20720,9 +19493,7 @@ "featureId": "VAR_005947", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -20747,9 +19518,7 @@ "featureId": "VAR_044992", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -20774,9 +19543,7 @@ "featureId": "VAR_044993", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -20801,9 +19568,7 @@ "featureId": "VAR_044994", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -20822,9 +19587,7 @@ "featureId": "VAR_044995", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -20861,9 +19624,7 @@ "featureId": "VAR_005948", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -20888,9 +19649,7 @@ "featureId": "VAR_044996", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -20915,9 +19674,7 @@ "featureId": "VAR_044997", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -20942,9 +19699,7 @@ "featureId": "VAR_044998", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -20963,9 +19718,7 @@ "featureId": "VAR_044999", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -20990,9 +19743,7 @@ "featureId": "VAR_005949", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -21024,9 +19775,7 @@ "featureId": "VAR_005950", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -21045,9 +19794,7 @@ "featureId": "VAR_045000", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -21072,9 +19819,7 @@ "featureId": "VAR_045001", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -21093,9 +19838,7 @@ "featureId": "VAR_047172", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -21120,9 +19863,7 @@ "featureId": "VAR_045002", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -21147,9 +19888,7 @@ "featureId": "VAR_045003", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -21186,9 +19925,7 @@ "featureId": "VAR_005951", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -21207,9 +19944,7 @@ "featureId": "VAR_045004", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -21228,9 +19963,7 @@ "featureId": "VAR_045806", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -21255,9 +19988,7 @@ "featureId": "VAR_045005", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -21282,9 +20013,7 @@ "featureId": "VAR_045006", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -21309,9 +20038,7 @@ "featureId": "VAR_045007", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -21336,9 +20063,7 @@ "featureId": "VAR_045008", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -21357,9 +20082,7 @@ "featureId": "VAR_045009", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -21378,9 +20101,7 @@ "featureId": "VAR_045010", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -21399,9 +20120,7 @@ "featureId": "VAR_045011", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -21426,9 +20145,7 @@ "featureId": "VAR_045012", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -21453,9 +20170,7 @@ "featureId": "VAR_045013", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -21474,9 +20189,7 @@ "featureId": "VAR_045014", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -21495,9 +20208,7 @@ "featureId": "VAR_045015", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -21516,9 +20227,7 @@ "featureId": "VAR_005952", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -21537,9 +20246,7 @@ "featureId": "VAR_045016", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -21558,9 +20265,7 @@ "featureId": "VAR_045017", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -21579,9 +20284,7 @@ "featureId": "VAR_045018", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -21600,9 +20303,7 @@ "featureId": "VAR_045019", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -21621,9 +20322,7 @@ "featureId": "VAR_045020", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -21648,9 +20347,7 @@ "featureId": "VAR_045021", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -21669,9 +20366,7 @@ "featureId": "VAR_045022", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -21690,9 +20385,7 @@ "featureId": "VAR_045023", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -21711,9 +20404,7 @@ "featureId": "VAR_045024", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -21732,9 +20423,7 @@ "featureId": "VAR_045807", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -21753,9 +20442,7 @@ "featureId": "VAR_045025", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -21774,9 +20461,7 @@ "featureId": "VAR_045026", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -21795,9 +20480,7 @@ "featureId": "VAR_047173", "alternativeSequence": { "originalSequence": "LR", - "alternativeSequences": [ - "FC" - ] + "alternativeSequences": ["FC"] } }, { @@ -21822,9 +20505,7 @@ "featureId": "VAR_045027", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -21843,9 +20524,7 @@ "featureId": "VAR_045028", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -21864,9 +20543,7 @@ "featureId": "VAR_045029", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -21891,9 +20568,7 @@ "featureId": "VAR_045030", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -21918,9 +20593,7 @@ "featureId": "VAR_045031", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -21945,9 +20618,7 @@ "featureId": "VAR_045032", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -21972,9 +20643,7 @@ "featureId": "VAR_045033", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -21993,9 +20662,7 @@ "featureId": "VAR_045034", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -22014,9 +20681,7 @@ "featureId": "VAR_045035", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -22035,9 +20700,7 @@ "featureId": "VAR_047174", "alternativeSequence": { "originalSequence": "VE", - "alternativeSequences": [ - "LV" - ] + "alternativeSequences": ["LV"] } }, { @@ -22056,9 +20719,7 @@ "featureId": "VAR_045036", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -22077,9 +20738,7 @@ "featureId": "VAR_045037", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -22098,9 +20757,7 @@ "featureId": "VAR_045038", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -22125,9 +20782,7 @@ "featureId": "VAR_045039", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -22146,9 +20801,7 @@ "featureId": "VAR_045808", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -22167,9 +20820,7 @@ "featureId": "VAR_045040", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -22188,9 +20839,7 @@ "featureId": "VAR_045041", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -22215,9 +20864,7 @@ "featureId": "VAR_045042", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -22236,9 +20883,7 @@ "featureId": "VAR_045043", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -22257,9 +20902,7 @@ "featureId": "VAR_045044", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -22278,9 +20921,7 @@ "featureId": "VAR_045045", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -22312,9 +20953,7 @@ "featureId": "VAR_005953", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -22339,9 +20978,7 @@ "featureId": "VAR_005954", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -22366,9 +21003,7 @@ "featureId": "VAR_047175", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -22393,9 +21028,7 @@ "featureId": "VAR_045046", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -22420,9 +21053,7 @@ "featureId": "VAR_045047", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -22447,9 +21078,7 @@ "featureId": "VAR_045048", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -22468,9 +21097,7 @@ "featureId": "VAR_045049", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -22489,9 +21116,7 @@ "featureId": "VAR_045050", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -22510,9 +21135,7 @@ "featureId": "VAR_047176", "alternativeSequence": { "originalSequence": "DD", - "alternativeSequences": [ - "EY" - ] + "alternativeSequences": ["EY"] } }, { @@ -22531,9 +21154,7 @@ "featureId": "VAR_045051", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -22552,9 +21173,7 @@ "featureId": "VAR_045052", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -22573,9 +21192,7 @@ "featureId": "VAR_045053", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -22600,9 +21217,7 @@ "featureId": "VAR_045054", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -22621,9 +21236,7 @@ "featureId": "VAR_045055", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -22642,9 +21255,7 @@ "featureId": "VAR_045056", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -22663,9 +21274,7 @@ "featureId": "VAR_045057", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -22684,9 +21293,7 @@ "featureId": "VAR_045058", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -22705,9 +21312,7 @@ "featureId": "VAR_045059", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -22726,9 +21331,7 @@ "featureId": "VAR_045809", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -22747,9 +21350,7 @@ "featureId": "VAR_045060", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -22774,9 +21375,7 @@ "featureId": "VAR_045061", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -22795,9 +21394,7 @@ "featureId": "VAR_045062", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -22816,9 +21413,7 @@ "featureId": "VAR_045063", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -22837,9 +21432,7 @@ "featureId": "VAR_045064", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -22858,9 +21451,7 @@ "featureId": "VAR_045065", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -22879,9 +21470,7 @@ "featureId": "VAR_045066", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -22900,9 +21489,7 @@ "featureId": "VAR_045067", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -22921,9 +21508,7 @@ "featureId": "VAR_045068", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -22942,9 +21527,7 @@ "featureId": "VAR_045069", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -22963,9 +21546,7 @@ "featureId": "VAR_045070", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -22984,9 +21565,7 @@ "featureId": "VAR_045071", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -23005,9 +21584,7 @@ "featureId": "VAR_045072", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -23032,9 +21609,7 @@ "featureId": "VAR_045073", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -23059,9 +21634,7 @@ "featureId": "VAR_045074", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -23080,9 +21653,7 @@ "featureId": "VAR_045075", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -23101,9 +21672,7 @@ "featureId": "VAR_045076", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -23122,9 +21691,7 @@ "featureId": "VAR_045077", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -23143,9 +21710,7 @@ "featureId": "VAR_045078", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -23170,9 +21735,7 @@ "featureId": "VAR_045079", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -23191,9 +21754,7 @@ "featureId": "VAR_045080", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -23212,9 +21773,7 @@ "featureId": "VAR_045081", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -23233,9 +21792,7 @@ "featureId": "VAR_045082", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -23254,9 +21811,7 @@ "featureId": "VAR_045083", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -23281,9 +21836,7 @@ "featureId": "VAR_045084", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -23308,9 +21861,7 @@ "featureId": "VAR_045085", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -23342,9 +21893,7 @@ "featureId": "VAR_036506", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -23369,9 +21918,7 @@ "featureId": "VAR_005955", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -23390,9 +21937,7 @@ "featureId": "VAR_045086", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -23411,9 +21956,7 @@ "featureId": "VAR_045087", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -23438,9 +21981,7 @@ "featureId": "VAR_045088", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -23465,9 +22006,7 @@ "featureId": "VAR_047177", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -23492,9 +22031,7 @@ "featureId": "VAR_045089", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -23513,9 +22050,7 @@ "featureId": "VAR_045090", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -23534,9 +22069,7 @@ "featureId": "VAR_045091", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -23561,9 +22094,7 @@ "featureId": "VAR_045092", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -23588,9 +22119,7 @@ "featureId": "VAR_045093", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -23609,9 +22138,7 @@ "featureId": "VAR_045810", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -23636,9 +22163,7 @@ "featureId": "VAR_045094", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -23663,9 +22188,7 @@ "featureId": "VAR_045095", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -23690,9 +22213,7 @@ "featureId": "VAR_045096", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -23711,9 +22232,7 @@ "featureId": "VAR_045097", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -23738,9 +22257,7 @@ "featureId": "VAR_045098", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -23765,9 +22282,7 @@ "featureId": "VAR_045099", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -23792,9 +22307,7 @@ "featureId": "VAR_045100", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -23826,9 +22339,7 @@ "featureId": "VAR_005956", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -23847,9 +22358,7 @@ "featureId": "VAR_045811", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -23868,9 +22377,7 @@ "featureId": "VAR_045101", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -23889,9 +22396,7 @@ "featureId": "VAR_045102", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -23910,9 +22415,7 @@ "featureId": "VAR_045103", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -23931,9 +22434,7 @@ "featureId": "VAR_045104", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -23952,9 +22453,7 @@ "featureId": "VAR_045105", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -23979,9 +22478,7 @@ "featureId": "VAR_047178", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -24000,9 +22497,7 @@ "featureId": "VAR_045106", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -24021,9 +22516,7 @@ "featureId": "VAR_045107", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -24048,9 +22541,7 @@ "featureId": "VAR_045108", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -24069,9 +22560,7 @@ "featureId": "VAR_045109", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -24096,9 +22585,7 @@ "featureId": "VAR_045110", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -24117,9 +22604,7 @@ "featureId": "VAR_045812", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -24138,9 +22623,7 @@ "featureId": "VAR_045111", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -24165,9 +22648,7 @@ "featureId": "VAR_045112", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -24186,9 +22667,7 @@ "featureId": "VAR_045113", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -24213,9 +22692,7 @@ "featureId": "VAR_045114", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -24234,9 +22711,7 @@ "featureId": "VAR_045115", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -24273,9 +22748,7 @@ "featureId": "VAR_005957", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -24300,9 +22773,7 @@ "featureId": "VAR_045116", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -24321,9 +22792,7 @@ "featureId": "VAR_045117", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -24348,9 +22817,7 @@ "featureId": "VAR_005958", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -24375,9 +22842,7 @@ "featureId": "VAR_045118", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -24402,9 +22867,7 @@ "featureId": "VAR_005959", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -24423,9 +22886,7 @@ "featureId": "VAR_045119", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -24444,9 +22905,7 @@ "featureId": "VAR_045120", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -24465,9 +22924,7 @@ "featureId": "VAR_045121", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -24492,9 +22949,7 @@ "featureId": "VAR_045122", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -24513,9 +22968,7 @@ "featureId": "VAR_045123", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -24534,9 +22987,7 @@ "featureId": "VAR_045124", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -24561,9 +23012,7 @@ "featureId": "VAR_045125", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -24582,9 +23031,7 @@ "featureId": "VAR_045126", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -24603,9 +23050,7 @@ "featureId": "VAR_045127", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -24630,9 +23075,7 @@ "featureId": "VAR_045128", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -24651,9 +23094,7 @@ "featureId": "VAR_045129", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -24672,9 +23113,7 @@ "featureId": "VAR_047179", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -24699,9 +23138,7 @@ "featureId": "VAR_045130", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -24726,9 +23163,7 @@ "featureId": "VAR_045131", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -24747,9 +23182,7 @@ "featureId": "VAR_045132", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -24768,9 +23201,7 @@ "featureId": "VAR_045133", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -24789,9 +23220,7 @@ "featureId": "VAR_045134", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -24816,9 +23245,7 @@ "featureId": "VAR_045135", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -24837,9 +23264,7 @@ "featureId": "VAR_045136", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -24864,9 +23289,7 @@ "featureId": "VAR_045137", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -24885,9 +23308,7 @@ "featureId": "VAR_045138", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -24906,9 +23327,7 @@ "featureId": "VAR_045139", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -24927,9 +23346,7 @@ "featureId": "VAR_045140", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -24948,9 +23365,7 @@ "featureId": "VAR_045141", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -24969,9 +23384,7 @@ "featureId": "VAR_045142", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -24990,9 +23403,7 @@ "featureId": "VAR_045143", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -25017,9 +23428,7 @@ "featureId": "VAR_045144", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -25038,9 +23447,7 @@ "featureId": "VAR_045145", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -25059,9 +23466,7 @@ "featureId": "VAR_047180", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -25080,9 +23485,7 @@ "featureId": "VAR_045844", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -25101,9 +23504,7 @@ "featureId": "VAR_045146", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -25128,9 +23529,7 @@ "featureId": "VAR_045147", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -25149,9 +23548,7 @@ "featureId": "VAR_045148", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -25170,9 +23567,7 @@ "featureId": "VAR_045149", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -25191,9 +23586,7 @@ "featureId": "VAR_045150", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -25212,9 +23605,7 @@ "featureId": "VAR_045151", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -25233,9 +23624,7 @@ "featureId": "VAR_045152", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -25254,9 +23643,7 @@ "featureId": "VAR_005960", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -25275,9 +23662,7 @@ "featureId": "VAR_045153", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -25296,9 +23681,7 @@ "featureId": "VAR_045154", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -25317,9 +23700,7 @@ "featureId": "VAR_045155", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -25338,9 +23719,7 @@ "featureId": "VAR_045845", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -25359,9 +23738,7 @@ "featureId": "VAR_045156", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -25380,9 +23757,7 @@ "featureId": "VAR_045157", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -25401,9 +23776,7 @@ "featureId": "VAR_045158", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -25422,9 +23795,7 @@ "featureId": "VAR_045846", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -25449,9 +23820,7 @@ "featureId": "VAR_045159", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -25470,9 +23839,7 @@ "featureId": "VAR_045160", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -25497,9 +23864,7 @@ "featureId": "VAR_045161", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -25518,9 +23883,7 @@ "featureId": "VAR_045162", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -25546,9 +23909,7 @@ "featureId": "VAR_005961", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -25567,9 +23928,7 @@ "featureId": "VAR_045163", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -25588,9 +23947,7 @@ "featureId": "VAR_045164", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -25609,9 +23966,7 @@ "featureId": "VAR_045165", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -25630,9 +23985,7 @@ "featureId": "VAR_045166", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -25657,9 +24010,7 @@ "featureId": "VAR_045167", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -25678,9 +24029,7 @@ "featureId": "VAR_045168", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -25699,9 +24048,7 @@ "featureId": "VAR_045169", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -25720,9 +24067,7 @@ "featureId": "VAR_045170", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -25747,9 +24092,7 @@ "featureId": "VAR_045171", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -25768,9 +24111,7 @@ "featureId": "VAR_045172", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -25796,9 +24137,7 @@ "featureId": "VAR_045173", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -25823,9 +24162,7 @@ "featureId": "VAR_005962", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -25844,9 +24181,7 @@ "featureId": "VAR_045174", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -25865,9 +24200,7 @@ "featureId": "VAR_045175", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -25886,9 +24219,7 @@ "featureId": "VAR_045176", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -25907,9 +24238,7 @@ "featureId": "VAR_045177", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -25928,9 +24257,7 @@ "featureId": "VAR_045178", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -25955,9 +24282,7 @@ "featureId": "VAR_047181", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -25976,9 +24301,7 @@ "featureId": "VAR_045179", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -26003,9 +24326,7 @@ "featureId": "VAR_005963", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -26030,9 +24351,7 @@ "featureId": "VAR_045180", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -26051,9 +24370,7 @@ "featureId": "VAR_045181", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -26078,9 +24395,7 @@ "featureId": "VAR_005964", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -26099,9 +24414,7 @@ "featureId": "VAR_045847", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -26126,9 +24439,7 @@ "featureId": "VAR_045182", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -26147,9 +24458,7 @@ "featureId": "VAR_045848", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -26174,9 +24483,7 @@ "featureId": "VAR_045183", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -26195,9 +24502,7 @@ "featureId": "VAR_047182", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -26216,9 +24521,7 @@ "featureId": "VAR_045184", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -26243,9 +24546,7 @@ "featureId": "VAR_045185", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -26264,9 +24565,7 @@ "featureId": "VAR_045849", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -26291,9 +24590,7 @@ "featureId": "VAR_045186", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -26312,9 +24609,7 @@ "featureId": "VAR_045187", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -26339,9 +24634,7 @@ "featureId": "VAR_045188", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -26366,9 +24659,7 @@ "featureId": "VAR_045189", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -26393,9 +24684,7 @@ "featureId": "VAR_045190", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -26414,9 +24703,7 @@ "featureId": "VAR_045191", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -26441,9 +24728,7 @@ "featureId": "VAR_045192", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -26468,9 +24753,7 @@ "featureId": "VAR_045193", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -26495,9 +24778,7 @@ "featureId": "VAR_045194", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -26529,9 +24810,7 @@ "featureId": "VAR_005965", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -26556,9 +24835,7 @@ "featureId": "VAR_045195", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -26577,9 +24854,7 @@ "featureId": "VAR_045196", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -26604,9 +24879,7 @@ "featureId": "VAR_045197", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -26625,9 +24898,7 @@ "featureId": "VAR_045198", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -26652,9 +24923,7 @@ "featureId": "VAR_045199", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -26679,9 +24948,7 @@ "featureId": "VAR_005966", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -26706,9 +24973,7 @@ "featureId": "VAR_045200", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -26727,9 +24992,7 @@ "featureId": "VAR_045850", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -26754,9 +25017,7 @@ "featureId": "VAR_045201", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -26781,9 +25042,7 @@ "featureId": "VAR_045202", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -26808,9 +25067,7 @@ "featureId": "VAR_045203", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -26835,9 +25092,7 @@ "featureId": "VAR_005967", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -26862,9 +25117,7 @@ "featureId": "VAR_045204", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -26883,9 +25136,7 @@ "featureId": "VAR_045205", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -26904,9 +25155,7 @@ "featureId": "VAR_045206", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -26931,9 +25180,7 @@ "featureId": "VAR_045207", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -26958,9 +25205,7 @@ "featureId": "VAR_045208", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -26985,9 +25230,7 @@ "featureId": "VAR_045209", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -27006,9 +25249,7 @@ "featureId": "VAR_045210", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -27027,9 +25268,7 @@ "featureId": "VAR_045211", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -27048,9 +25287,7 @@ "featureId": "VAR_045212", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -27069,9 +25306,7 @@ "featureId": "VAR_005968", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -27090,9 +25325,7 @@ "featureId": "VAR_045213", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -27111,9 +25344,7 @@ "featureId": "VAR_045214", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -27132,9 +25363,7 @@ "featureId": "VAR_045215", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -27153,9 +25382,7 @@ "featureId": "VAR_045216", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -27187,9 +25414,7 @@ "featureId": "VAR_033036", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -27214,9 +25439,7 @@ "featureId": "VAR_045217", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -27253,9 +25476,7 @@ "featureId": "VAR_005969", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -27280,9 +25501,7 @@ "featureId": "VAR_045218", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -27301,9 +25520,7 @@ "featureId": "VAR_047183", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -27328,9 +25545,7 @@ "featureId": "VAR_045219", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -27367,9 +25582,7 @@ "featureId": "VAR_005970", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -27394,9 +25607,7 @@ "featureId": "VAR_045220", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -27415,9 +25626,7 @@ "featureId": "VAR_045221", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -27442,9 +25651,7 @@ "featureId": "VAR_045222", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -27469,9 +25676,7 @@ "featureId": "VAR_045223", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -27496,9 +25701,7 @@ "featureId": "VAR_045224", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -27517,9 +25720,7 @@ "featureId": "VAR_047184", "alternativeSequence": { "originalSequence": "MG", - "alternativeSequences": [ - "IC" - ] + "alternativeSequences": ["IC"] } }, { @@ -27538,9 +25739,7 @@ "featureId": "VAR_047185", "alternativeSequence": { "originalSequence": "MG", - "alternativeSequences": [ - "IS" - ] + "alternativeSequences": ["IS"] } }, { @@ -27559,9 +25758,7 @@ "featureId": "VAR_045225", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -27580,9 +25777,7 @@ "featureId": "VAR_045226", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -27607,9 +25802,7 @@ "featureId": "VAR_045227", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -27628,9 +25821,7 @@ "featureId": "VAR_045228", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -27655,9 +25846,7 @@ "featureId": "VAR_045229", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -27682,9 +25871,7 @@ "featureId": "VAR_045230", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -27709,9 +25896,7 @@ "featureId": "VAR_047186", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -27736,9 +25921,7 @@ "featureId": "VAR_045231", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -27763,9 +25946,7 @@ "featureId": "VAR_045232", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -27784,9 +25965,7 @@ "featureId": "VAR_045233", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -27811,9 +25990,7 @@ "featureId": "VAR_045234", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -27838,9 +26015,7 @@ "featureId": "VAR_045235", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -27865,9 +26040,7 @@ "featureId": "VAR_045236", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -27892,9 +26065,7 @@ "featureId": "VAR_005971", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -27931,9 +26102,7 @@ "featureId": "VAR_005972", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -27965,9 +26134,7 @@ "featureId": "VAR_005973", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -27986,9 +26153,7 @@ "featureId": "VAR_045237", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -28007,9 +26172,7 @@ "featureId": "VAR_045851", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -28028,9 +26191,7 @@ "featureId": "VAR_045852", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -28049,9 +26210,7 @@ "featureId": "VAR_045853", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -28070,9 +26229,7 @@ "featureId": "VAR_045854", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -28097,9 +26254,7 @@ "featureId": "VAR_045238", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -28131,9 +26286,7 @@ "featureId": "VAR_005974", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -28165,9 +26318,7 @@ "featureId": "VAR_005975", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -28192,9 +26343,7 @@ "featureId": "VAR_045239", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -28213,9 +26362,7 @@ "featureId": "VAR_045240", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -28240,9 +26387,7 @@ "featureId": "VAR_044020", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -28267,9 +26412,7 @@ "featureId": "VAR_005976", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -28294,9 +26437,7 @@ "featureId": "VAR_005977", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -28321,9 +26462,7 @@ "featureId": "VAR_005978", "alternativeSequence": { "originalSequence": "M", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -28342,9 +26481,7 @@ "featureId": "VAR_047187", "alternativeSequence": { "originalSequence": "NR", - "alternativeSequences": [ - "IP" - ] + "alternativeSequences": ["IP"] } }, { @@ -28363,9 +26500,7 @@ "featureId": "VAR_047188", "alternativeSequence": { "originalSequence": "NR", - "alternativeSequences": [ - "KW" - ] + "alternativeSequences": ["KW"] } }, { @@ -28390,9 +26525,7 @@ "featureId": "VAR_045241", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -28411,9 +26544,7 @@ "featureId": "VAR_045855", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -28438,9 +26569,7 @@ "featureId": "VAR_005980", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -28459,9 +26588,7 @@ "featureId": "VAR_045242", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -28486,9 +26613,7 @@ "featureId": "VAR_045243", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -28507,9 +26632,7 @@ "featureId": "VAR_047189", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -28528,9 +26651,7 @@ "featureId": "VAR_045244", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -28549,9 +26670,7 @@ "featureId": "VAR_045245", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -28576,9 +26695,7 @@ "featureId": "VAR_005981", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -28615,9 +26732,7 @@ "featureId": "VAR_005982", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -28642,9 +26757,7 @@ "featureId": "VAR_045246", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -28691,9 +26804,7 @@ "featureId": "VAR_005983", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -28740,9 +26851,7 @@ "featureId": "VAR_005984", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -28761,9 +26870,7 @@ "featureId": "VAR_047190", "alternativeSequence": { "originalSequence": "RP", - "alternativeSequences": [ - "SA" - ] + "alternativeSequences": ["SA"] } }, { @@ -28782,9 +26889,7 @@ "featureId": "VAR_047191", "alternativeSequence": { "originalSequence": "RP", - "alternativeSequences": [ - "SS" - ] + "alternativeSequences": ["SS"] } }, { @@ -28809,9 +26914,7 @@ "featureId": "VAR_005985", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -28830,9 +26933,7 @@ "featureId": "VAR_045247", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -28857,9 +26958,7 @@ "featureId": "VAR_045248", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -28891,9 +26990,7 @@ "featureId": "VAR_033037", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -28912,9 +27009,7 @@ "featureId": "VAR_045856", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -28956,9 +27051,7 @@ "featureId": "VAR_005986", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -28983,9 +27076,7 @@ "featureId": "VAR_045249", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -29010,9 +27101,7 @@ "featureId": "VAR_045250", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -29031,9 +27120,7 @@ "featureId": "VAR_045251", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -29052,9 +27139,7 @@ "featureId": "VAR_045857", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -29073,9 +27158,7 @@ "featureId": "VAR_045252", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -29100,9 +27183,7 @@ "featureId": "VAR_047192", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -29121,9 +27202,7 @@ "featureId": "VAR_045858", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -29142,9 +27221,7 @@ "featureId": "VAR_045253", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -29163,9 +27240,7 @@ "featureId": "VAR_045254", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -29184,9 +27259,7 @@ "featureId": "VAR_045255", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -29205,9 +27278,7 @@ "featureId": "VAR_045256", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -29232,9 +27303,7 @@ "featureId": "VAR_045257", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -29259,9 +27328,7 @@ "featureId": "VAR_045258", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -29280,9 +27347,7 @@ "featureId": "VAR_005987", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -29314,9 +27379,7 @@ "featureId": "VAR_033038", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -29335,9 +27398,7 @@ "featureId": "VAR_045259", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -29356,9 +27417,7 @@ "featureId": "VAR_045260", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -29377,9 +27436,7 @@ "featureId": "VAR_045261", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -29398,9 +27455,7 @@ "featureId": "VAR_045262", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -29419,9 +27474,7 @@ "featureId": "VAR_045263", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -29453,9 +27506,7 @@ "featureId": "VAR_005988", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -29474,9 +27525,7 @@ "featureId": "VAR_045264", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -29495,9 +27544,7 @@ "featureId": "VAR_045265", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -29516,9 +27563,7 @@ "featureId": "VAR_045266", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -29543,9 +27588,7 @@ "featureId": "VAR_045267", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -29564,9 +27607,7 @@ "featureId": "VAR_047193", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -29585,9 +27626,7 @@ "featureId": "VAR_045268", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -29606,9 +27645,7 @@ "featureId": "VAR_045859", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -29627,9 +27664,7 @@ "featureId": "VAR_045269", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -29648,9 +27683,7 @@ "featureId": "VAR_045270", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -29669,9 +27702,7 @@ "featureId": "VAR_045271", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -29690,9 +27721,7 @@ "featureId": "VAR_017908", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -29717,9 +27746,7 @@ "featureId": "VAR_045272", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -29738,9 +27765,7 @@ "featureId": "VAR_017909", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -29765,9 +27790,7 @@ "featureId": "VAR_045273", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -29792,9 +27815,7 @@ "featureId": "VAR_045274", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -29813,9 +27834,7 @@ "featureId": "VAR_045275", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -29840,9 +27859,7 @@ "featureId": "VAR_045276", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -29867,9 +27884,7 @@ "featureId": "VAR_045277", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -29894,9 +27909,7 @@ "featureId": "VAR_045278", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -29915,9 +27928,7 @@ "featureId": "VAR_045279", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -29936,9 +27947,7 @@ "featureId": "VAR_045280", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -29957,9 +27966,7 @@ "featureId": "VAR_045281", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -29978,9 +27985,7 @@ "featureId": "VAR_045282", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -29999,9 +28004,7 @@ "featureId": "VAR_045283", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -30020,9 +28023,7 @@ "featureId": "VAR_005989", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -30047,9 +28048,7 @@ "featureId": "VAR_045284", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -30074,9 +28073,7 @@ "featureId": "VAR_045285", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -30095,9 +28092,7 @@ "featureId": "VAR_045286", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -30116,9 +28111,7 @@ "featureId": "VAR_045287", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -30137,9 +28130,7 @@ "featureId": "VAR_005990", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -30164,9 +28155,7 @@ "featureId": "VAR_045288", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -30198,9 +28187,7 @@ "featureId": "VAR_005991", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -30219,9 +28206,7 @@ "featureId": "VAR_045860", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -30240,9 +28225,7 @@ "featureId": "VAR_045289", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -30261,9 +28244,7 @@ "featureId": "VAR_045290", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -30282,9 +28263,7 @@ "featureId": "VAR_047194", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -30303,9 +28282,7 @@ "featureId": "VAR_045291", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -30330,9 +28307,7 @@ "featureId": "VAR_045292", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -30351,9 +28326,7 @@ "featureId": "VAR_045293", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -30372,9 +28345,7 @@ "featureId": "VAR_045294", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -30393,9 +28364,7 @@ "featureId": "VAR_045861", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -30414,9 +28383,7 @@ "featureId": "VAR_045862", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -30435,9 +28402,7 @@ "featureId": "VAR_045295", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -30463,9 +28428,7 @@ "featureId": "VAR_033039", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -30484,9 +28447,7 @@ "featureId": "VAR_045296", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -30505,9 +28466,7 @@ "featureId": "VAR_045297", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -30526,9 +28485,7 @@ "featureId": "VAR_045298", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -30547,9 +28504,7 @@ "featureId": "VAR_045299", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -30568,9 +28523,7 @@ "featureId": "VAR_045300", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -30595,9 +28548,7 @@ "featureId": "VAR_045301", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -30616,9 +28567,7 @@ "featureId": "VAR_045302", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -30637,9 +28586,7 @@ "featureId": "VAR_045303", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -30658,9 +28605,7 @@ "featureId": "VAR_045304", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -30679,9 +28624,7 @@ "featureId": "VAR_045305", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -30700,9 +28643,7 @@ "featureId": "VAR_045306", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -30721,9 +28662,7 @@ "featureId": "VAR_047195", "alternativeSequence": { "originalSequence": "GN", - "alternativeSequences": [ - "PD" - ] + "alternativeSequences": ["PD"] } }, { @@ -30748,9 +28687,7 @@ "featureId": "VAR_045307", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -30769,9 +28706,7 @@ "featureId": "VAR_047196", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -30790,9 +28725,7 @@ "featureId": "VAR_045863", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -30817,9 +28750,7 @@ "featureId": "VAR_045308", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -30851,9 +28782,7 @@ "featureId": "VAR_045309", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -30878,9 +28807,7 @@ "featureId": "VAR_045310", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -30899,9 +28826,7 @@ "featureId": "VAR_045311", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -30920,9 +28845,7 @@ "featureId": "VAR_045312", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -30941,9 +28864,7 @@ "featureId": "VAR_045313", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -30962,9 +28883,7 @@ "featureId": "VAR_045314", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -30983,9 +28902,7 @@ "featureId": "VAR_045315", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -31010,9 +28927,7 @@ "featureId": "VAR_045316", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -31031,9 +28946,7 @@ "featureId": "VAR_045317", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -31052,9 +28965,7 @@ "featureId": "VAR_045318", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -31073,9 +28984,7 @@ "featureId": "VAR_045319", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -31094,9 +29003,7 @@ "featureId": "VAR_045320", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -31121,9 +29028,7 @@ "featureId": "VAR_045321", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -31142,9 +29047,7 @@ "featureId": "VAR_045322", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -31163,9 +29066,7 @@ "featureId": "VAR_047197", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -31184,9 +29085,7 @@ "featureId": "VAR_045323", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -31211,9 +29110,7 @@ "featureId": "VAR_045324", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -31238,9 +29135,7 @@ "featureId": "VAR_045325", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -31265,9 +29160,7 @@ "featureId": "VAR_045326", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -31286,9 +29179,7 @@ "featureId": "VAR_045327", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -31307,9 +29198,7 @@ "featureId": "VAR_045328", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -31334,9 +29223,7 @@ "featureId": "VAR_045329", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -31361,9 +29248,7 @@ "featureId": "VAR_045330", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -31395,9 +29280,7 @@ "featureId": "VAR_036507", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -31416,9 +29299,7 @@ "featureId": "VAR_045864", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -31437,9 +29318,7 @@ "featureId": "VAR_045331", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -31458,9 +29337,7 @@ "featureId": "VAR_045332", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -31479,9 +29356,7 @@ "featureId": "VAR_045333", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -31500,9 +29375,7 @@ "featureId": "VAR_045334", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -31521,9 +29394,7 @@ "featureId": "VAR_045335", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -31542,9 +29413,7 @@ "featureId": "VAR_045336", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -31563,9 +29432,7 @@ "featureId": "VAR_045337", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -31584,9 +29451,7 @@ "featureId": "VAR_047198", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -31605,9 +29470,7 @@ "featureId": "VAR_045338", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -31626,9 +29489,7 @@ "featureId": "VAR_045339", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -31647,9 +29508,7 @@ "featureId": "VAR_045340", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -31674,9 +29533,7 @@ "featureId": "VAR_045341", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -31701,9 +29558,7 @@ "featureId": "VAR_045342", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -31728,9 +29583,7 @@ "featureId": "VAR_045343", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -31755,9 +29608,7 @@ "featureId": "VAR_045344", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -31782,9 +29633,7 @@ "featureId": "VAR_045345", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -31803,9 +29652,7 @@ "featureId": "VAR_045346", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -31824,9 +29671,7 @@ "featureId": "VAR_045347", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -31845,9 +29690,7 @@ "featureId": "VAR_045348", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -31866,9 +29709,7 @@ "featureId": "VAR_045349", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -31900,9 +29741,7 @@ "featureId": "VAR_036508", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -31921,9 +29760,7 @@ "featureId": "VAR_045865", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -31942,9 +29779,7 @@ "featureId": "VAR_045350", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -31963,9 +29798,7 @@ "featureId": "VAR_045866", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -31984,9 +29817,7 @@ "featureId": "VAR_047199", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -32005,9 +29836,7 @@ "featureId": "VAR_045351", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -32032,9 +29861,7 @@ "featureId": "VAR_045352", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -32059,9 +29886,7 @@ "featureId": "VAR_045353", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -32093,9 +29918,7 @@ "featureId": "VAR_005992", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -32120,9 +29943,7 @@ "featureId": "VAR_045354", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -32169,9 +29990,7 @@ "featureId": "VAR_005993", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -32190,9 +30009,7 @@ "featureId": "VAR_005994", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -32264,9 +30081,7 @@ "featureId": "VAR_005995", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -32298,9 +30113,7 @@ "featureId": "VAR_036509", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -32319,9 +30132,7 @@ "featureId": "VAR_045867", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -32353,9 +30164,7 @@ "featureId": "VAR_045355", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -32374,9 +30183,7 @@ "featureId": "VAR_045356", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -32401,9 +30208,7 @@ "featureId": "VAR_045357", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -32422,9 +30227,7 @@ "featureId": "VAR_045868", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -32449,9 +30252,7 @@ "featureId": "VAR_045358", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -32476,9 +30277,7 @@ "featureId": "VAR_045359", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -32503,9 +30302,7 @@ "featureId": "VAR_005997", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -32530,9 +30327,7 @@ "featureId": "VAR_047200", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -32551,9 +30346,7 @@ "featureId": "VAR_045360", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -32578,9 +30371,7 @@ "featureId": "VAR_045361", "alternativeSequence": { "originalSequence": "V", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -32605,9 +30396,7 @@ "featureId": "VAR_045362", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -32626,9 +30415,7 @@ "featureId": "VAR_045363", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -32653,9 +30440,7 @@ "featureId": "VAR_045364", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -32680,9 +30465,7 @@ "featureId": "VAR_045365", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -32707,9 +30490,7 @@ "featureId": "VAR_005999", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -32741,9 +30522,7 @@ "featureId": "VAR_005998", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -32768,9 +30547,7 @@ "featureId": "VAR_045366", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -32795,9 +30572,7 @@ "featureId": "VAR_045367", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -32822,9 +30597,7 @@ "featureId": "VAR_045368", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -32843,9 +30616,7 @@ "featureId": "VAR_045369", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -32864,9 +30635,7 @@ "featureId": "VAR_045370", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -32885,9 +30654,7 @@ "featureId": "VAR_045371", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -32912,9 +30679,7 @@ "featureId": "VAR_045372", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -32939,9 +30704,7 @@ "featureId": "VAR_006000", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -32966,9 +30729,7 @@ "featureId": "VAR_045373", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -32987,9 +30748,7 @@ "featureId": "VAR_045374", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -33008,9 +30767,7 @@ "featureId": "VAR_047201", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -33035,9 +30792,7 @@ "featureId": "VAR_045375", "alternativeSequence": { "originalSequence": "C", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -33069,9 +30824,7 @@ "featureId": "VAR_006001", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -33090,9 +30843,7 @@ "featureId": "VAR_045869", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -33117,9 +30868,7 @@ "featureId": "VAR_006002", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -33151,9 +30900,7 @@ "featureId": "VAR_006003", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -33178,9 +30925,7 @@ "featureId": "VAR_045376", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -33222,9 +30967,7 @@ "featureId": "VAR_006004", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -33249,9 +30992,7 @@ "featureId": "VAR_006005", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -33276,9 +31017,7 @@ "featureId": "VAR_006006", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -33303,9 +31042,7 @@ "featureId": "VAR_045377", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -33324,9 +31061,7 @@ "featureId": "VAR_045378", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -33345,9 +31080,7 @@ "featureId": "VAR_045379", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -33372,9 +31105,7 @@ "featureId": "VAR_045380", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -33406,9 +31137,7 @@ "featureId": "VAR_006008", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -33445,9 +31174,7 @@ "featureId": "VAR_006007", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -33466,9 +31193,7 @@ "featureId": "VAR_045381", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -33487,9 +31212,7 @@ "featureId": "VAR_045382", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -33521,9 +31244,7 @@ "featureId": "VAR_006009", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -33542,9 +31263,7 @@ "featureId": "VAR_047203", "alternativeSequence": { "originalSequence": "DR", - "alternativeSequences": [ - "EW" - ] + "alternativeSequences": ["EW"] } }, { @@ -33569,9 +31288,7 @@ "featureId": "VAR_006010", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -33603,9 +31320,7 @@ "featureId": "VAR_006011", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -33630,9 +31345,7 @@ "featureId": "VAR_006012", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -33664,9 +31377,7 @@ "featureId": "VAR_006013", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -33691,9 +31402,7 @@ "featureId": "VAR_047202", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -33712,9 +31421,7 @@ "featureId": "VAR_045870", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -33739,9 +31446,7 @@ "featureId": "VAR_006014", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -33766,9 +31471,7 @@ "featureId": "VAR_045383", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -33793,9 +31496,7 @@ "featureId": "VAR_045384", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -33814,9 +31515,7 @@ "featureId": "VAR_045385", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -33841,9 +31540,7 @@ "featureId": "VAR_006015", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -33868,9 +31565,7 @@ "featureId": "VAR_045386", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -33907,9 +31602,7 @@ "featureId": "VAR_045387", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -33946,9 +31639,7 @@ "featureId": "VAR_006016", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -33973,9 +31664,7 @@ "featureId": "VAR_006017", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -33994,9 +31683,7 @@ "featureId": "VAR_006018", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -34021,9 +31708,7 @@ "featureId": "VAR_006019", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -34042,9 +31727,7 @@ "featureId": "VAR_045388", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -34063,9 +31746,7 @@ "featureId": "VAR_006020", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -34090,9 +31771,7 @@ "featureId": "VAR_045389", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -34118,9 +31797,7 @@ "featureId": "VAR_006021", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -34145,9 +31822,7 @@ "featureId": "VAR_045390", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -34166,9 +31841,7 @@ "featureId": "VAR_045391", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -34193,9 +31866,7 @@ "featureId": "VAR_006022", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -34214,9 +31885,7 @@ "featureId": "VAR_045392", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -34235,9 +31904,7 @@ "featureId": "VAR_045393", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -34256,9 +31923,7 @@ "featureId": "VAR_045394", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -34295,9 +31960,7 @@ "featureId": "VAR_006023", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -34316,9 +31979,7 @@ "featureId": "VAR_006024", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -34343,9 +32004,7 @@ "featureId": "VAR_006025", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -34370,9 +32029,7 @@ "featureId": "VAR_006026", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -34391,9 +32048,7 @@ "featureId": "VAR_006027", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -34418,9 +32073,7 @@ "featureId": "VAR_006028", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -34457,9 +32110,7 @@ "featureId": "VAR_006029", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -34478,9 +32129,7 @@ "featureId": "VAR_045871", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -34512,9 +32161,7 @@ "featureId": "VAR_006030", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -34539,9 +32186,7 @@ "featureId": "VAR_045395", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -34560,9 +32205,7 @@ "featureId": "VAR_047204", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -34587,9 +32230,7 @@ "featureId": "VAR_045396", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -34608,9 +32249,7 @@ "featureId": "VAR_045397", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -34635,9 +32274,7 @@ "featureId": "VAR_045398", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -34656,9 +32293,7 @@ "featureId": "VAR_045399", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -34677,9 +32312,7 @@ "featureId": "VAR_045400", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -34698,9 +32331,7 @@ "featureId": "VAR_045401", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -34719,9 +32350,7 @@ "featureId": "VAR_045402", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -34740,9 +32369,7 @@ "featureId": "VAR_045403", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -34761,9 +32388,7 @@ "featureId": "VAR_045404", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -34782,9 +32407,7 @@ "featureId": "VAR_045405", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -34803,9 +32426,7 @@ "featureId": "VAR_045406", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -34824,9 +32445,7 @@ "featureId": "VAR_045407", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -34845,9 +32464,7 @@ "featureId": "VAR_045408", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -34872,9 +32489,7 @@ "featureId": "VAR_045409", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -34899,9 +32514,7 @@ "featureId": "VAR_045410", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -34926,9 +32539,7 @@ "featureId": "VAR_045411", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -34947,9 +32558,7 @@ "featureId": "VAR_045412", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -34974,9 +32583,7 @@ "featureId": "VAR_045413", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -34995,9 +32602,7 @@ "featureId": "VAR_045414", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -35022,9 +32627,7 @@ "featureId": "VAR_045415", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -35043,9 +32646,7 @@ "featureId": "VAR_047205", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -35070,9 +32671,7 @@ "featureId": "VAR_045416", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -35091,9 +32690,7 @@ "featureId": "VAR_045417", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -35112,9 +32709,7 @@ "featureId": "VAR_045418", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -35133,9 +32728,7 @@ "featureId": "VAR_045872", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -35167,9 +32760,7 @@ "featureId": "VAR_015819", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -35188,9 +32779,7 @@ "featureId": "VAR_045419", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -35209,9 +32798,7 @@ "featureId": "VAR_045420", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -35236,9 +32823,7 @@ "featureId": "VAR_045421", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -35257,9 +32842,7 @@ "featureId": "VAR_045422", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -35278,9 +32861,7 @@ "featureId": "VAR_045423", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -35305,9 +32886,7 @@ "featureId": "VAR_045424", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -35326,9 +32905,7 @@ "featureId": "VAR_045425", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -35353,9 +32930,7 @@ "featureId": "VAR_045426", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -35374,9 +32949,7 @@ "featureId": "VAR_045427", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -35401,9 +32974,7 @@ "featureId": "VAR_045428", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -35422,9 +32993,7 @@ "featureId": "VAR_045429", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -35443,9 +33012,7 @@ "featureId": "VAR_047206", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -35464,9 +33031,7 @@ "featureId": "VAR_045430", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -35485,9 +33050,7 @@ "featureId": "VAR_045431", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -35506,9 +33069,7 @@ "featureId": "VAR_045432", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -35533,9 +33094,7 @@ "featureId": "VAR_045433", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -35560,9 +33119,7 @@ "featureId": "VAR_045434", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -35587,9 +33144,7 @@ "featureId": "VAR_045435", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -35608,9 +33163,7 @@ "featureId": "VAR_045873", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -35629,9 +33182,7 @@ "featureId": "VAR_045436", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -35650,9 +33201,7 @@ "featureId": "VAR_047207", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -35671,9 +33220,7 @@ "featureId": "VAR_045437", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -35692,9 +33239,7 @@ "featureId": "VAR_006031", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -35713,9 +33258,7 @@ "featureId": "VAR_045438", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -35740,9 +33283,7 @@ "featureId": "VAR_045439", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -35767,9 +33308,7 @@ "featureId": "VAR_045440", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -35788,9 +33327,7 @@ "featureId": "VAR_045441", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -35809,9 +33346,7 @@ "featureId": "VAR_045442", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -35830,9 +33365,7 @@ "featureId": "VAR_045443", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -35857,9 +33390,7 @@ "featureId": "VAR_045444", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -35878,9 +33409,7 @@ "featureId": "VAR_045445", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -35899,9 +33428,7 @@ "featureId": "VAR_045446", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -35920,9 +33447,7 @@ "featureId": "VAR_045447", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -35947,9 +33472,7 @@ "featureId": "VAR_045448", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -35974,9 +33497,7 @@ "featureId": "VAR_045449", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -35995,9 +33516,7 @@ "featureId": "VAR_045450", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -36016,9 +33535,7 @@ "featureId": "VAR_045451", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -36037,9 +33554,7 @@ "featureId": "VAR_045452", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -36058,9 +33573,7 @@ "featureId": "VAR_045453", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -36079,9 +33592,7 @@ "featureId": "VAR_045454", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -36100,9 +33611,7 @@ "featureId": "VAR_045455", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -36127,9 +33636,7 @@ "featureId": "VAR_045456", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -36148,9 +33655,7 @@ "featureId": "VAR_006032", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -36169,9 +33674,7 @@ "featureId": "VAR_045457", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -36190,9 +33693,7 @@ "featureId": "VAR_045458", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -36217,9 +33718,7 @@ "featureId": "VAR_006033", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -36238,9 +33737,7 @@ "featureId": "VAR_045459", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -36259,9 +33756,7 @@ "featureId": "VAR_045460", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -36280,9 +33775,7 @@ "featureId": "VAR_047208", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -36301,9 +33794,7 @@ "featureId": "VAR_045461", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -36328,9 +33819,7 @@ "featureId": "VAR_006034", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -36355,9 +33844,7 @@ "featureId": "VAR_045462", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -36376,9 +33863,7 @@ "featureId": "VAR_006035", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -36397,9 +33882,7 @@ "featureId": "VAR_045463", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -36418,9 +33901,7 @@ "featureId": "VAR_045464", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -36445,9 +33926,7 @@ "featureId": "VAR_045465", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -36466,9 +33945,7 @@ "featureId": "VAR_045466", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -36493,9 +33970,7 @@ "featureId": "VAR_045467", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -36514,9 +33989,7 @@ "featureId": "VAR_045468", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -36535,9 +34008,7 @@ "featureId": "VAR_045469", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -36556,9 +34027,7 @@ "featureId": "VAR_047209", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -36577,9 +34046,7 @@ "featureId": "VAR_045470", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -36598,9 +34065,7 @@ "featureId": "VAR_045471", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -36626,9 +34091,7 @@ "featureId": "VAR_045472", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -36647,9 +34110,7 @@ "featureId": "VAR_045473", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -36668,9 +34129,7 @@ "featureId": "VAR_045474", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -36689,9 +34148,7 @@ "featureId": "VAR_045475", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -36716,9 +34173,7 @@ "featureId": "VAR_006036", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -36737,9 +34192,7 @@ "featureId": "VAR_045476", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -36758,9 +34211,7 @@ "featureId": "VAR_045477", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -36779,9 +34230,7 @@ "featureId": "VAR_006037", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -36800,9 +34249,7 @@ "featureId": "VAR_045478", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -36821,9 +34268,7 @@ "featureId": "VAR_045479", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -36842,9 +34287,7 @@ "featureId": "VAR_045480", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -36876,9 +34319,7 @@ "featureId": "VAR_006038", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -36897,9 +34338,7 @@ "featureId": "VAR_045481", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -36918,9 +34357,7 @@ "featureId": "VAR_045482", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -36945,9 +34382,7 @@ "featureId": "VAR_045483", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -36966,9 +34401,7 @@ "featureId": "VAR_045484", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -36993,9 +34426,7 @@ "featureId": "VAR_045485", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -37020,9 +34451,7 @@ "featureId": "VAR_045486", "alternativeSequence": { "originalSequence": "N", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -37041,9 +34470,7 @@ "featureId": "VAR_045487", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -37068,9 +34495,7 @@ "featureId": "VAR_045488", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -37089,9 +34514,7 @@ "featureId": "VAR_045489", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -37110,9 +34533,7 @@ "featureId": "VAR_045490", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -37131,9 +34552,7 @@ "featureId": "VAR_045491", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -37158,9 +34577,7 @@ "featureId": "VAR_045492", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -37185,9 +34602,7 @@ "featureId": "VAR_045493", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -37206,9 +34621,7 @@ "featureId": "VAR_045494", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -37227,9 +34640,7 @@ "featureId": "VAR_045495", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -37248,9 +34659,7 @@ "featureId": "VAR_045496", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -37275,9 +34684,7 @@ "featureId": "VAR_045497", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -37302,9 +34709,7 @@ "featureId": "VAR_045498", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -37329,9 +34734,7 @@ "featureId": "VAR_045499", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -37356,9 +34759,7 @@ "featureId": "VAR_045500", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -37377,9 +34778,7 @@ "featureId": "VAR_047210", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -37398,9 +34797,7 @@ "featureId": "VAR_045501", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -37425,9 +34822,7 @@ "featureId": "VAR_045502", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -37452,9 +34847,7 @@ "featureId": "VAR_045503", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -37473,9 +34866,7 @@ "featureId": "VAR_045504", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -37494,9 +34885,7 @@ "featureId": "VAR_045505", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -37515,9 +34904,7 @@ "featureId": "VAR_045506", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -37536,9 +34923,7 @@ "featureId": "VAR_045507", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "N" - ] + "alternativeSequences": ["N"] } }, { @@ -37557,9 +34942,7 @@ "featureId": "VAR_045508", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -37578,9 +34961,7 @@ "featureId": "VAR_045509", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -37599,9 +34980,7 @@ "featureId": "VAR_045510", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -37620,9 +34999,7 @@ "featureId": "VAR_045511", "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -37641,9 +35018,7 @@ "featureId": "VAR_045874", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -37662,9 +35037,7 @@ "featureId": "VAR_045512", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "M" - ] + "alternativeSequences": ["M"] } }, { @@ -37683,9 +35056,7 @@ "featureId": "VAR_045513", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -37704,9 +35075,7 @@ "featureId": "VAR_045514", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -37731,9 +35100,7 @@ "featureId": "VAR_047211", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -37752,9 +35119,7 @@ "featureId": "VAR_045515", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -37773,9 +35138,7 @@ "featureId": "VAR_045875", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -37794,9 +35157,7 @@ "featureId": "VAR_045516", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -37815,9 +35176,7 @@ "featureId": "VAR_045517", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -37836,9 +35195,7 @@ "featureId": "VAR_045518", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -37870,9 +35227,7 @@ "featureId": "VAR_006039", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -37891,9 +35246,7 @@ "featureId": "VAR_045519", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -37912,9 +35265,7 @@ "featureId": "VAR_045520", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -37933,9 +35284,7 @@ "featureId": "VAR_045521", "alternativeSequence": { "originalSequence": "Y", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -37954,9 +35303,7 @@ "featureId": "VAR_045522", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -37975,9 +35322,7 @@ "featureId": "VAR_045523", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -37996,9 +35341,7 @@ "featureId": "VAR_045524", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -38023,9 +35366,7 @@ "featureId": "VAR_045525", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -38044,9 +35385,7 @@ "featureId": "VAR_045526", "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -38065,9 +35404,7 @@ "featureId": "VAR_045527", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -38086,9 +35423,7 @@ "featureId": "VAR_047212", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -38107,9 +35442,7 @@ "featureId": "VAR_045528", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -38134,9 +35467,7 @@ "featureId": "VAR_045529", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -38155,9 +35486,7 @@ "featureId": "VAR_045530", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -38182,9 +35511,7 @@ "featureId": "VAR_045531", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -38203,9 +35530,7 @@ "featureId": "VAR_045532", "alternativeSequence": { "originalSequence": "I", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -38224,9 +35549,7 @@ "featureId": "VAR_006040", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -38251,9 +35574,7 @@ "featureId": "VAR_045533", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -38272,9 +35593,7 @@ "featureId": "VAR_045534", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -38299,9 +35618,7 @@ "featureId": "VAR_045535", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -38320,9 +35637,7 @@ "featureId": "VAR_045536", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -38359,9 +35674,7 @@ "featureId": "VAR_006041", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -38398,9 +35711,7 @@ "featureId": "VAR_035016", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -38425,9 +35736,7 @@ "featureId": "VAR_045537", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -38459,9 +35768,7 @@ "featureId": "VAR_045538", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -38480,9 +35787,7 @@ "featureId": "VAR_045539", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "I" - ] + "alternativeSequences": ["I"] } }, { @@ -38507,9 +35812,7 @@ "featureId": "VAR_045540", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -38541,9 +35844,7 @@ "featureId": "VAR_022316", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -38568,9 +35869,7 @@ "featureId": "VAR_045541", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -38589,9 +35888,7 @@ "featureId": "VAR_045542", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "C" - ] + "alternativeSequences": ["C"] } }, { @@ -38610,9 +35907,7 @@ "featureId": "VAR_045543", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -38637,9 +35932,7 @@ "featureId": "VAR_045544", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -38664,9 +35957,7 @@ "featureId": "VAR_047213", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -38685,9 +35976,7 @@ "featureId": "VAR_045545", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -38719,9 +36008,7 @@ "featureId": "VAR_045546", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -38740,9 +36027,7 @@ "featureId": "VAR_045547", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -38761,9 +36046,7 @@ "featureId": "VAR_045548", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -38782,9 +36065,7 @@ "featureId": "VAR_045549", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "G" - ] + "alternativeSequences": ["G"] } }, { @@ -38803,9 +36084,7 @@ "featureId": "VAR_045550", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -38824,9 +36103,7 @@ "featureId": "VAR_045551", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "F" - ] + "alternativeSequences": ["F"] } }, { @@ -38845,9 +36122,7 @@ "featureId": "VAR_045552", "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -38866,9 +36141,7 @@ "featureId": "VAR_045553", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -38887,9 +36160,7 @@ "featureId": "VAR_045554", "alternativeSequence": { "originalSequence": "D", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -38908,9 +36179,7 @@ "featureId": "VAR_045555", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -38929,9 +36198,7 @@ "featureId": "VAR_045556", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -38956,9 +36223,7 @@ "featureId": "VAR_045557", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -38983,9 +36248,7 @@ "featureId": "VAR_047214", "alternativeSequence": { "originalSequence": "Q", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -39004,9 +36267,7 @@ "featureId": "VAR_045558", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -39025,9 +36286,7 @@ "featureId": "VAR_045559", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -39046,9 +36305,7 @@ "featureId": "VAR_045560", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -39073,9 +36330,7 @@ "featureId": "VAR_045561", "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -39100,9 +36355,7 @@ "featureId": "VAR_045562", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -39127,9 +36380,7 @@ "featureId": "VAR_045563", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -39154,9 +36405,7 @@ "featureId": "VAR_045564", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "K" - ] + "alternativeSequences": ["K"] } }, { @@ -39175,9 +36424,7 @@ "featureId": "VAR_045565", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "P" - ] + "alternativeSequences": ["P"] } }, { @@ -39196,9 +36443,7 @@ "featureId": "VAR_045566", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -39217,9 +36462,7 @@ "featureId": "VAR_045567", "alternativeSequence": { "originalSequence": "A", - "alternativeSequences": [ - "V" - ] + "alternativeSequences": ["V"] } }, { @@ -39238,9 +36481,7 @@ "featureId": "VAR_047215", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -39265,9 +36506,7 @@ "featureId": "VAR_045568", "alternativeSequence": { "originalSequence": "H", - "alternativeSequences": [ - "Y" - ] + "alternativeSequences": ["Y"] } }, { @@ -39299,9 +36538,7 @@ "featureId": "VAR_022317", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -39320,9 +36557,7 @@ "featureId": "VAR_045569", "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -39341,9 +36576,7 @@ "featureId": "VAR_045570", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -39362,9 +36595,7 @@ "featureId": "VAR_045571", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "T" - ] + "alternativeSequences": ["T"] } }, { @@ -39389,9 +36620,7 @@ "featureId": "VAR_045572", "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "H" - ] + "alternativeSequences": ["H"] } }, { @@ -39416,9 +36645,7 @@ "featureId": "VAR_045573", "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -39443,9 +36670,7 @@ "featureId": "VAR_045574", "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "W" - ] + "alternativeSequences": ["W"] } }, { @@ -39464,9 +36689,7 @@ "featureId": "VAR_045575", "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "L" - ] + "alternativeSequences": ["L"] } }, { @@ -39491,9 +36714,7 @@ ], "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -39518,9 +36739,7 @@ ], "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -39545,9 +36764,7 @@ ], "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -39572,9 +36789,7 @@ ], "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -39599,9 +36814,7 @@ ], "alternativeSequence": { "originalSequence": "LW", - "alternativeSequences": [ - "QS" - ] + "alternativeSequences": ["QS"] } }, { @@ -39626,9 +36839,7 @@ ], "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -39653,9 +36864,7 @@ ], "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -39690,9 +36899,7 @@ ], "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -39749,9 +36956,7 @@ ], "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -39776,9 +36981,7 @@ ], "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -39803,9 +37006,7 @@ ], "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -39830,9 +37031,7 @@ ], "alternativeSequence": { "originalSequence": "R", - "alternativeSequences": [ - "S" - ] + "alternativeSequences": ["S"] } }, { @@ -39857,9 +37056,7 @@ ], "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -39884,9 +37081,7 @@ ], "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -39911,9 +37106,7 @@ ], "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -39938,9 +37131,7 @@ ], "alternativeSequence": { "originalSequence": "KK", - "alternativeSequences": [ - "RR" - ] + "alternativeSequences": ["RR"] } }, { @@ -39965,9 +37156,7 @@ ], "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -39992,9 +37181,7 @@ ], "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -40019,9 +37206,7 @@ ], "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -40046,9 +37231,7 @@ ], "alternativeSequence": { "originalSequence": "RGRER", - "alternativeSequences": [ - "KGKEK" - ] + "alternativeSequences": ["KGKEK"] } }, { @@ -40073,9 +37256,7 @@ ], "alternativeSequence": { "originalSequence": "P", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -40100,9 +37281,7 @@ ], "alternativeSequence": { "originalSequence": "G", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -40127,9 +37306,7 @@ ], "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -40154,9 +37331,7 @@ ], "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -40181,9 +37356,7 @@ ], "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -40208,9 +37381,7 @@ ], "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -40235,9 +37406,7 @@ ], "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "Q" - ] + "alternativeSequences": ["Q"] } }, { @@ -40262,9 +37431,7 @@ ], "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -40304,9 +37471,7 @@ ], "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -40346,9 +37511,7 @@ ], "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "R" - ] + "alternativeSequences": ["R"] } }, { @@ -40373,9 +37536,7 @@ ], "alternativeSequence": { "originalSequence": "L", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -40405,9 +37566,7 @@ ], "alternativeSequence": { "originalSequence": "F", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -40437,9 +37596,7 @@ ], "alternativeSequence": { "originalSequence": "K", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -40464,9 +37621,7 @@ ], "alternativeSequence": { "originalSequence": "T", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -40491,9 +37646,7 @@ ], "alternativeSequence": { "originalSequence": "E", - "alternativeSequences": [ - "A" - ] + "alternativeSequences": ["A"] } }, { @@ -40518,9 +37671,7 @@ ], "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "D" - ] + "alternativeSequences": ["D"] } }, { @@ -40545,9 +37696,7 @@ ], "alternativeSequence": { "originalSequence": "S", - "alternativeSequences": [ - "E" - ] + "alternativeSequences": ["E"] } }, { @@ -41664,12 +38813,7 @@ "citation": { "id": "4006916", "citationType": "journal article", - "authors": [ - "Zakut-Houri R.", - "Bienz-Tadmor B.", - "Givol D.", - "Oren M." - ], + "authors": ["Zakut-Houri R.", "Bienz-Tadmor B.", "Givol D.", "Oren M."], "citationCrossReferences": [ { "database": "PubMed", @@ -41687,19 +38831,14 @@ "lastPage": "1255", "volume": "4" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [MRNA] (ISOFORM 1)" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [MRNA] (ISOFORM 1)"] }, { "referenceNumber": 2, "citation": { "id": "2946935", "citationType": "journal article", - "authors": [ - "Lamb P.", - "Crawford L." - ], + "authors": ["Lamb P.", "Crawford L."], "citationCrossReferences": [ { "database": "PubMed", @@ -41717,23 +38856,14 @@ "lastPage": "1385", "volume": "6" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [GENOMIC DNA / MRNA] (ISOFORM 1)", - "VARIANT GLY-76" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [GENOMIC DNA / MRNA] (ISOFORM 1)", "VARIANT GLY-76"] }, { "referenceNumber": 3, "citation": { "id": "3894933", "citationType": "journal article", - "authors": [ - "Harlow E.", - "Williamson N.M.", - "Ralston R.", - "Helfman D.M.", - "Adams T.E." - ], + "authors": ["Harlow E.", "Williamson N.M.", "Ralston R.", "Helfman D.M.", "Adams T.E."], "citationCrossReferences": [ { "database": "PubMed", @@ -41751,24 +38881,14 @@ "lastPage": "1610", "volume": "5" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [MRNA] (ISOFORM 1)" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [MRNA] (ISOFORM 1)"] }, { "referenceNumber": 4, "citation": { "id": "3025664", "citationType": "journal article", - "authors": [ - "Harris N.", - "Brill E.", - "Shohat O.", - "Prokocimer M.", - "Wolf D.", - "Arai N.", - "Rotter V." - ], + "authors": ["Harris N.", "Brill E.", "Shohat O.", "Prokocimer M.", "Wolf D.", "Arai N.", "Rotter V."], "citationCrossReferences": [ { "database": "PubMed", @@ -41786,22 +38906,14 @@ "lastPage": "4656", "volume": "6" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [MRNA] (ISOFORM 1)" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [MRNA] (ISOFORM 1)"] }, { "referenceNumber": 5, "citation": { "id": "2905688", "citationType": "journal article", - "authors": [ - "Buchman V.L.", - "Chumakov P.M.", - "Ninkina N.N.", - "Samarina O.P.", - "Georgiev G.P." - ], + "authors": ["Buchman V.L.", "Chumakov P.M.", "Ninkina N.N.", "Samarina O.P.", "Georgiev G.P."], "citationCrossReferences": [ { "database": "PubMed", @@ -41819,22 +38931,14 @@ "lastPage": "252", "volume": "70" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [GENOMIC DNA]" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [GENOMIC DNA]"] }, { "referenceNumber": 6, "citation": { "id": "1915267", "citationType": "journal article", - "authors": [ - "Farrell P.J.", - "Allan G.", - "Shanahan F.", - "Vousden K.H.", - "Crook T." - ], + "authors": ["Farrell P.J.", "Allan G.", "Shanahan F.", "Vousden K.H.", "Crook T."], "citationCrossReferences": [ { "database": "PubMed", @@ -41852,23 +38956,14 @@ "lastPage": "2887", "volume": "10" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [MRNA] (ISOFORM 1)", - "VARIANTS SPORADIC CANCERS" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [MRNA] (ISOFORM 1)", "VARIANTS SPORADIC CANCERS"] }, { "referenceNumber": 7, "citation": { "id": "8316628", "citationType": "journal article", - "authors": [ - "Allalunis-Turner M.J.", - "Barron G.M.", - "Day R.S. III", - "Dobler K.D.", - "Mirzayans R." - ], + "authors": ["Allalunis-Turner M.J.", "Barron G.M.", "Day R.S. III", "Dobler K.D.", "Mirzayans R."], "citationCrossReferences": [ { "database": "PubMed", @@ -41886,25 +38981,14 @@ "lastPage": "354", "volume": "134" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [GENOMIC DNA]", - "VARIANT LYS-286" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [GENOMIC DNA]", "VARIANT LYS-286"] }, { "referenceNumber": 8, "citation": { "id": "11058590", "citationType": "journal article", - "authors": [ - "Chang N.-S.", - "Pratt N.", - "Heath J.", - "Schultz L.", - "Sleve D.", - "Carey G.B.", - "Zevotek N." - ], + "authors": ["Chang N.-S.", "Pratt N.", "Heath J.", "Schultz L.", "Sleve D.", "Carey G.B.", "Zevotek N."], "citationCrossReferences": [ { "database": "PubMed", @@ -41922,10 +39006,7 @@ "lastPage": "3370", "volume": "276" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [MRNA] (ISOFORM 1)", - "INTERACTION WITH WWOX" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [MRNA] (ISOFORM 1)", "INTERACTION WITH WWOX"] }, { "referenceNumber": 9, @@ -41980,60 +39061,41 @@ "citation": { "id": "CI-8C1T8BUL7KG2A", "citationType": "submission", - "authors": [ - "Chumakov P.M.", - "Almazov V.P.", - "Jenkins J.R." - ], + "authors": ["Chumakov P.M.", "Almazov V.P.", "Jenkins J.R."], "publicationDate": "JUN-1991", "submissionDatabase": "EMBL/GenBank/DDBJ databases" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [GENOMIC DNA]" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [GENOMIC DNA]"] }, { "referenceNumber": 11, "citation": { "id": "CI-9M3D9J7U2MDR4", "citationType": "submission", - "authors": [ - "Rozemuller E.H.", - "Tilanus M.G.J." - ], + "authors": ["Rozemuller E.H.", "Tilanus M.G.J."], "title": "P53 genomic sequence. Corrections and polymorphism.", "publicationDate": "MAR-1997", "submissionDatabase": "EMBL/GenBank/DDBJ databases" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [GENOMIC DNA]" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [GENOMIC DNA]"] }, { "referenceNumber": 12, "citation": { "id": "CI-86ALM2TFFQ300", "citationType": "submission", - "authoringGroup": [ - "NIEHS SNPs program" - ], + "authoringGroup": ["NIEHS SNPs program"], "publicationDate": "NOV-2004", "submissionDatabase": "EMBL/GenBank/DDBJ databases" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [GENOMIC DNA]", - "VARIANTS SER-47; LYS-339 AND ALA-366" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [GENOMIC DNA]", "VARIANTS SER-47; LYS-339 AND ALA-366"] }, { "referenceNumber": 13, "citation": { "id": "11023613", "citationType": "journal article", - "authors": [ - "Anderson C.W.", - "Allalunis-Turner M.J." - ], + "authors": ["Anderson C.W.", "Allalunis-Turner M.J."], "citationCrossReferences": [ { "database": "PubMed", @@ -42051,29 +39113,19 @@ "lastPage": "476", "volume": "154" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [GENOMIC DNA]", - "VARIANT LYS-286" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [GENOMIC DNA]", "VARIANT LYS-286"] }, { "referenceNumber": 14, "citation": { "id": "CI-AMPIBVI7IO3G4", "citationType": "submission", - "authors": [ - "Azuma K.", - "Shichijo S.", - "Itoh K." - ], + "authors": ["Azuma K.", "Shichijo S.", "Itoh K."], "title": "Identification of a tumor-rejection antigen recognized by HLA-B46 restricted CTL.", "publicationDate": "MAR-2002", "submissionDatabase": "EMBL/GenBank/DDBJ databases" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [MRNA] (ISOFORM 1)", - "VARIANTS HIS-273 AND SER-309" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [MRNA] (ISOFORM 1)", "VARIANTS HIS-273 AND SER-309"] }, { "referenceNumber": 15, @@ -42256,10 +39308,7 @@ "lastPage": "45", "volume": "36" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [LARGE SCALE MRNA] (ISOFORM 1)", - "VARIANT ARG-72" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [LARGE SCALE MRNA] (ISOFORM 1)", "VARIANT ARG-72"] }, { "referenceNumber": 16, @@ -42359,9 +39408,7 @@ "lastPage": "1049", "volume": "440" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [LARGE SCALE GENOMIC DNA]" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [LARGE SCALE GENOMIC DNA]"] }, { "referenceNumber": 17, @@ -42410,19 +39457,14 @@ "publicationDate": "SEP-2005", "submissionDatabase": "EMBL/GenBank/DDBJ databases" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [LARGE SCALE GENOMIC DNA]", - "VARIANT ARG-72" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [LARGE SCALE GENOMIC DNA]", "VARIANT ARG-72"] }, { "referenceNumber": 18, "citation": { "id": "15489334", "citationType": "journal article", - "authoringGroup": [ - "The MGC Project Team" - ], + "authoringGroup": ["The MGC Project Team"], "citationCrossReferences": [ { "database": "PubMed", @@ -42440,10 +39482,7 @@ "lastPage": "2127", "volume": "14" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [LARGE SCALE MRNA] (ISOFORM 1)", - "VARIANT ALA-278" - ], + "referencePositions": ["NUCLEOTIDE SEQUENCE [LARGE SCALE MRNA] (ISOFORM 1)", "VARIANT ALA-278"], "referenceComments": [ { "value": "Kidney", @@ -42456,14 +39495,7 @@ "citation": { "id": "14660794", "citationType": "journal article", - "authors": [ - "Kanashiro C.A.", - "Schally A.V.", - "Groot K.", - "Armatis P.", - "Bernardino A.L.", - "Varga J.L." - ], + "authors": ["Kanashiro C.A.", "Schally A.V.", "Groot K.", "Armatis P.", "Bernardino A.L.", "Varga J.L."], "citationCrossReferences": [ { "database": "PubMed", @@ -42481,10 +39513,7 @@ "lastPage": "15841", "volume": "100" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [MRNA] OF 1-379 (ISOFORM 1)", - "VARIANTS ASN-139 AND PRO-155" - ], + "referencePositions": ["NUCLEOTIDE SEQUENCE [MRNA] OF 1-379 (ISOFORM 1)", "VARIANTS ASN-139 AND PRO-155"], "referenceComments": [ { "value": "Lung carcinoma", @@ -42497,14 +39526,7 @@ "citation": { "id": "6396087", "citationType": "journal article", - "authors": [ - "Matlashewski G.", - "Lamb P.", - "Pim D.", - "Peacock J.", - "Crawford L.", - "Benchimol S." - ], + "authors": ["Matlashewski G.", "Lamb P.", "Pim D.", "Peacock J.", "Crawford L.", "Benchimol S."], "citationCrossReferences": [ { "database": "PubMed", @@ -42522,45 +39544,31 @@ "lastPage": "3262", "volume": "3" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [GENOMIC DNA / MRNA] OF 101-393" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [GENOMIC DNA / MRNA] OF 101-393"] }, { "referenceNumber": 21, "citation": { "id": "CI-668DAA267SUPO", "citationType": "submission", - "authors": [ - "Pan X.L.", - "Zhang A.H." - ], + "authors": ["Pan X.L.", "Zhang A.H."], "title": "Study on the effect of tumor suppressor gene p53 in arsenism patients.", "publicationDate": "SEP-2003", "submissionDatabase": "EMBL/GenBank/DDBJ databases" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [GENOMIC DNA] OF 126-185" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [GENOMIC DNA] OF 126-185"] }, { "referenceNumber": 22, "citation": { "id": "CI-D45UV8P9LEPU6", "citationType": "submission", - "authors": [ - "Nimri L.F.", - "Owais W.", - "Momani E." - ], + "authors": ["Nimri L.F.", "Owais W.", "Momani E."], "title": "Detection of P53 gene mutations and serum p53 antibodies associated with cigarette smoking.", "publicationDate": "AUG-2003", "submissionDatabase": "EMBL/GenBank/DDBJ databases" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [GENOMIC DNA] OF 261-298", - "VARIANT GLN-282" - ], + "referencePositions": ["NUCLEOTIDE SEQUENCE [GENOMIC DNA] OF 261-298", "VARIANT GLN-282"], "referenceComments": [ { "value": "Blood", @@ -42573,17 +39581,11 @@ "citation": { "id": "CI-ADDS09TRP9ONS", "citationType": "submission", - "authors": [ - "Filippini G.", - "Soldati G." - ], + "authors": ["Filippini G.", "Soldati G."], "publicationDate": "JUL-1996", "submissionDatabase": "EMBL/GenBank/DDBJ databases" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [GENOMIC DNA] OF 262-306", - "VARIANT VAL-262" - ], + "referencePositions": ["NUCLEOTIDE SEQUENCE [GENOMIC DNA] OF 262-306", "VARIANT VAL-262"], "referenceComments": [ { "value": "Ovarian adenocarcinoma", @@ -42611,9 +39613,7 @@ "publicationDate": "DEC-1999", "submissionDatabase": "EMBL/GenBank/DDBJ databases" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [GENOMIC DNA] OF 225-260" - ], + "referencePositions": ["NUCLEOTIDE SEQUENCE [GENOMIC DNA] OF 225-260"], "referenceComments": [ { "value": "Glial cell", @@ -42630,21 +39630,12 @@ "citation": { "id": "CI-33DBJ1IU1IEBG", "citationType": "submission", - "authors": [ - "Yavuz A.S.", - "Farner N.L.", - "Yavuz S.", - "Grammer A.C.", - "Girschick H.J.", - "Lipsky P.E." - ], + "authors": ["Yavuz A.S.", "Farner N.L.", "Yavuz S.", "Grammer A.C.", "Girschick H.J.", "Lipsky P.E."], "title": "Bcl6 and P53 gene mutations in tonsillar B cells.", "publicationDate": "MAR-2000", "submissionDatabase": "EMBL/GenBank/DDBJ databases" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [GENOMIC DNA] OF 225-260" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [GENOMIC DNA] OF 225-260"] }, { "referenceNumber": 26, @@ -42692,31 +39683,19 @@ "citation": { "id": "CI-B1KJF44ND99KF", "citationType": "submission", - "authors": [ - "Pinto E.M.", - "Mendonca B.B.", - "Latronico A.C." - ], + "authors": ["Pinto E.M.", "Mendonca B.B.", "Latronico A.C."], "title": "Allelic variant in intron 9 of TP53 gene.", "publicationDate": "APR-2003", "submissionDatabase": "EMBL/GenBank/DDBJ databases" }, - "referencePositions": [ - "NUCLEOTIDE SEQUENCE [GENOMIC DNA] OF 332-366" - ] + "referencePositions": ["NUCLEOTIDE SEQUENCE [GENOMIC DNA] OF 332-366"] }, { "referenceNumber": 28, "citation": { "id": "2175676", "citationType": "journal article", - "authors": [ - "Scheffner M.", - "Werness B.A.", - "Huibregtse J.M.", - "Levine A.J.", - "Howley P.M." - ], + "authors": ["Scheffner M.", "Werness B.A.", "Huibregtse J.M.", "Levine A.J.", "Howley P.M."], "citationCrossReferences": [ { "database": "PubMed", @@ -42734,20 +39713,14 @@ "lastPage": "1136", "volume": "63" }, - "referencePositions": [ - "INTERACTION WITH HUMAN PAPILLOMAVIRUS TYPE 6 AND 11 PROTEIN E6 (MICROBIAL INFECTION)" - ] + "referencePositions": ["INTERACTION WITH HUMAN PAPILLOMAVIRUS TYPE 6 AND 11 PROTEIN E6 (MICROBIAL INFECTION)"] }, { "referenceNumber": 29, "citation": { "id": "2156209", "citationType": "journal article", - "authors": [ - "Addison C.", - "Jenkins J.R.", - "Sturzbecher H.-W." - ], + "authors": ["Addison C.", "Jenkins J.R.", "Sturzbecher H.-W."], "citationCrossReferences": [ { "database": "PubMed", @@ -42761,23 +39734,14 @@ "lastPage": "426", "volume": "5" }, - "referencePositions": [ - "NUCLEAR LOCALIZATION SIGNAL", - "MUTAGENESIS OF LYS-319; LYS-320 AND LYS-321" - ] + "referencePositions": ["NUCLEAR LOCALIZATION SIGNAL", "MUTAGENESIS OF LYS-319; LYS-320 AND LYS-321"] }, { "referenceNumber": 30, "citation": { "id": "2141171", "citationType": "journal article", - "authors": [ - "Bischoff J.R.", - "Friedman P.N.", - "Marshak D.R.", - "Prives C.", - "Beach D." - ], + "authors": ["Bischoff J.R.", "Friedman P.N.", "Marshak D.R.", "Prives C.", "Beach D."], "citationCrossReferences": [ { "database": "PubMed", @@ -42795,19 +39759,14 @@ "lastPage": "4770", "volume": "87" }, - "referencePositions": [ - "PHOSPHORYLATION BY P60/CDC2 AND CYCLIN B/CDC2" - ] + "referencePositions": ["PHOSPHORYLATION BY P60/CDC2 AND CYCLIN B/CDC2"] }, { "referenceNumber": 31, "citation": { "id": "1705009", "citationType": "journal article", - "authors": [ - "Samad A.", - "Carroll R.B." - ], + "authors": ["Samad A.", "Carroll R.B."], "citationCrossReferences": [ { "database": "PubMed", @@ -42825,21 +39784,14 @@ "lastPage": "1606", "volume": "11" }, - "referencePositions": [ - "PHOSPHORYLATION" - ] + "referencePositions": ["PHOSPHORYLATION"] }, { "referenceNumber": 32, "citation": { "id": "1848668", "citationType": "journal article", - "authors": [ - "Scheidtmann K.H.", - "Mumby M.C.", - "Rundell K.", - "Walter G." - ], + "authors": ["Scheidtmann K.H.", "Mumby M.C.", "Rundell K.", "Walter G."], "citationCrossReferences": [ { "database": "PubMed", @@ -42857,9 +39809,7 @@ "lastPage": "2003", "volume": "11" }, - "referencePositions": [ - "DEPHOSPHORYLATION BY PP2A" - ] + "referencePositions": ["DEPHOSPHORYLATION BY PP2A"] }, { "referenceNumber": 33, @@ -42889,21 +39839,14 @@ "lastPage": "818", "volume": "12" }, - "referencePositions": [ - "ALTERNATIVE SPLICING" - ] + "referencePositions": ["ALTERNATIVE SPLICING"] }, { "referenceNumber": 34, "citation": { "id": "8632915", "citationType": "journal article", - "authors": [ - "Shaw P.", - "Freeman J.", - "Bovey R.", - "Iggo R." - ], + "authors": ["Shaw P.", "Freeman J.", "Bovey R.", "Iggo R."], "citationCrossReferences": [ { "database": "PubMed", @@ -42917,9 +39860,7 @@ "lastPage": "930", "volume": "12" }, - "referencePositions": [ - "GLYCOSYLATION" - ] + "referencePositions": ["GLYCOSYLATION"] }, { "referenceNumber": 35, @@ -42953,20 +39894,14 @@ "lastPage": "7229", "volume": "17" }, - "referencePositions": [ - "PHOSPHORYLATION AT SER-33" - ] + "referencePositions": ["PHOSPHORYLATION AT SER-33"] }, { "referenceNumber": 36, "citation": { "id": "9840937", "citationType": "journal article", - "authors": [ - "Schneider E.", - "Montenarh M.", - "Wagner P." - ], + "authors": ["Schneider E.", "Montenarh M.", "Wagner P."], "citationCrossReferences": [ { "database": "PubMed", @@ -42984,10 +39919,7 @@ "lastPage": "2741", "volume": "17" }, - "referencePositions": [ - "FUNCTION", - "IDENTIFICATION IN COMPLEX WITH CAK" - ] + "referencePositions": ["FUNCTION", "IDENTIFICATION IN COMPLEX WITH CAK"] }, { "referenceNumber": 37, @@ -43010,21 +39942,14 @@ "lastPage": "6471", "volume": "18" }, - "referencePositions": [ - "SUMOYLATION AT LYS-386", - "MUTAGENESIS OF LYS-386" - ] + "referencePositions": ["SUMOYLATION AT LYS-386", "MUTAGENESIS OF LYS-386"] }, { "referenceNumber": 38, "citation": { "id": "10606744", "citationType": "journal article", - "authors": [ - "Dumaz N.", - "Milne D.M.", - "Meek D.W." - ], + "authors": ["Dumaz N.", "Milne D.M.", "Meek D.W."], "citationCrossReferences": [ { "database": "PubMed", @@ -43042,19 +39967,14 @@ "lastPage": "316", "volume": "463" }, - "referencePositions": [ - "PHOSPHORYLATION AT THR-18 BY CSNK1D/CK1" - ] + "referencePositions": ["PHOSPHORYLATION AT THR-18 BY CSNK1D/CK1"] }, { "referenceNumber": 39, "citation": { "id": "10551826", "citationType": "journal article", - "authors": [ - "Liang S.H.", - "Clarke M.F." - ], + "authors": ["Liang S.H.", "Clarke M.F."], "citationCrossReferences": [ { "database": "PubMed", @@ -43072,22 +39992,14 @@ "lastPage": "32703", "volume": "274" }, - "referencePositions": [ - "BIPARTITE NUCLEAR LOCALIZATION SIGNAL", - "CHARACTERIZATION OF VARIANT ASN-305" - ] + "referencePositions": ["BIPARTITE NUCLEAR LOCALIZATION SIGNAL", "CHARACTERIZATION OF VARIANT ASN-305"] }, { "referenceNumber": 40, "citation": { "id": "10570149", "citationType": "journal article", - "authors": [ - "Chehab N.H.", - "Malikzay A.", - "Stavridi E.S.", - "Halazonetis T.D." - ], + "authors": ["Chehab N.H.", "Malikzay A.", "Stavridi E.S.", "Halazonetis T.D."], "citationCrossReferences": [ { "database": "PubMed", @@ -43119,13 +40031,7 @@ "citation": { "id": "10722742", "citationType": "journal article", - "authors": [ - "Fang S.", - "Jensen J.P.", - "Ludwig R.L.", - "Vousden K.H.", - "Weissman A.M." - ], + "authors": ["Fang S.", "Jensen J.P.", "Ludwig R.L.", "Vousden K.H.", "Weissman A.M."], "citationCrossReferences": [ { "database": "PubMed", @@ -43143,21 +40049,14 @@ "lastPage": "8951", "volume": "275" }, - "referencePositions": [ - "UBIQUITINATION" - ] + "referencePositions": ["UBIQUITINATION"] }, { "referenceNumber": 42, "citation": { "id": "10656795", "citationType": "journal article", - "authors": [ - "Abraham J.", - "Kelly J.", - "Thibault P.", - "Benchimol S." - ], + "authors": ["Abraham J.", "Kelly J.", "Thibault P.", "Benchimol S."], "citationCrossReferences": [ { "database": "PubMed", @@ -43175,21 +40074,14 @@ "lastPage": "864", "volume": "295" }, - "referencePositions": [ - "ACETYLATION AT LYS-373 AND LYS-382" - ] + "referencePositions": ["ACETYLATION AT LYS-373 AND LYS-382"] }, { "referenceNumber": 43, "citation": { "id": "10884347", "citationType": "journal article", - "authors": [ - "Luciani M.G.", - "Hutchins J.R.A.", - "Zheleva D.", - "Hupp T.R." - ], + "authors": ["Luciani M.G.", "Hutchins J.R.A.", "Zheleva D.", "Hupp T.R."], "citationCrossReferences": [ { "database": "PubMed", @@ -43217,15 +40109,7 @@ "citation": { "id": "11025664", "citationType": "journal article", - "authors": [ - "Guo A.", - "Salomoni P.", - "Luo J.", - "Shih A.", - "Zhong S.", - "Gu W.", - "Pandolfi P.P." - ], + "authors": ["Guo A.", "Salomoni P.", "Luo J.", "Shih A.", "Zhong S.", "Gu W.", "Pandolfi P.P."], "citationCrossReferences": [ { "database": "PubMed", @@ -43243,11 +40127,7 @@ "lastPage": "736", "volume": "2" }, - "referencePositions": [ - "FUNCTION", - "INTERACTION WITH PML", - "SUBCELLULAR LOCATION" - ] + "referencePositions": ["FUNCTION", "INTERACTION WITH PML", "SUBCELLULAR LOCATION"] }, { "referenceNumber": 45, @@ -43281,19 +40161,14 @@ "lastPage": "199", "volume": "19" }, - "referencePositions": [ - "INTERACTION WITH E4F1" - ] + "referencePositions": ["INTERACTION WITH E4F1"] }, { "referenceNumber": 46, "citation": { "id": "10951572", "citationType": "journal article", - "authors": [ - "Lopez-Borges S.", - "Lazo P.A." - ], + "authors": ["Lopez-Borges S.", "Lazo P.A."], "citationCrossReferences": [ { "database": "PubMed", @@ -43311,19 +40186,14 @@ "lastPage": "3664", "volume": "19" }, - "referencePositions": [ - "PHOSPHORYLATION AT THR-18" - ] + "referencePositions": ["PHOSPHORYLATION AT THR-18"] }, { "referenceNumber": 47, "citation": { "id": "11554448", "citationType": "journal article", - "authors": [ - "Hainaut P.", - "Mann K." - ], + "authors": ["Hainaut P.", "Mann K."], "citationCrossReferences": [ { "database": "PubMed", @@ -43341,22 +40211,14 @@ "lastPage": "623", "volume": "3" }, - "referencePositions": [ - "REVIEW ON ZINC-BINDING PROPERTIES" - ] + "referencePositions": ["REVIEW ON ZINC-BINDING PROPERTIES"] }, { "referenceNumber": 48, "citation": { "id": "11554766", "citationType": "journal article", - "authors": [ - "Imamura K.", - "Ogura T.", - "Kishimoto A.", - "Kaminishi M.", - "Esumi H." - ], + "authors": ["Imamura K.", "Ogura T.", "Kishimoto A.", "Kaminishi M.", "Esumi H."], "citationCrossReferences": [ { "database": "PubMed", @@ -43374,9 +40236,7 @@ "lastPage": "567", "volume": "287" }, - "referencePositions": [ - "PHOSPHORYLATION AT SER-15" - ] + "referencePositions": ["PHOSPHORYLATION AT SER-15"] }, { "referenceNumber": 49, @@ -43410,22 +40270,14 @@ "lastPage": "159", "volume": "107" }, - "referencePositions": [ - "DEACETYLATION AT LYS-382 BY SIRT1" - ] + "referencePositions": ["DEACETYLATION AT LYS-382 BY SIRT1"] }, { "referenceNumber": 50, "citation": { "id": "11007800", "citationType": "journal article", - "authors": [ - "Hong T.M.", - "Chen J.J.", - "Peck K.", - "Yang P.C.", - "Wu C.W." - ], + "authors": ["Hong T.M.", "Chen J.J.", "Peck K.", "Yang P.C.", "Wu C.W."], "citationCrossReferences": [ { "database": "PubMed", @@ -43443,24 +40295,14 @@ "lastPage": "1515", "volume": "276" }, - "referencePositions": [ - "MINIMAL REPRESSION DOMAIN" - ] + "referencePositions": ["MINIMAL REPRESSION DOMAIN"] }, { "referenceNumber": 51, "citation": { "id": "11447225", "citationType": "journal article", - "authors": [ - "Xie S.", - "Wang Q.", - "Wu H.", - "Cogswell J.", - "Lu L.", - "Jhanwar-Uniyal M.", - "Dai W." - ], + "authors": ["Xie S.", "Wang Q.", "Wu H.", "Cogswell J.", "Lu L.", "Jhanwar-Uniyal M.", "Dai W."], "citationCrossReferences": [ { "database": "PubMed", @@ -43478,9 +40320,7 @@ "lastPage": "36199", "volume": "276" }, - "referencePositions": [ - "PHOSPHORYLATION AT SER-20 BY PLK3" - ] + "referencePositions": ["PHOSPHORYLATION AT SER-20 BY PLK3"] }, { "referenceNumber": 52, @@ -43515,20 +40355,14 @@ "lastPage": "43312", "volume": "276" }, - "referencePositions": [ - "PHOSPHORYLATION AT SER-20 BY PLK3" - ] + "referencePositions": ["PHOSPHORYLATION AT SER-20 BY PLK3"] }, { "referenceNumber": 53, "citation": { "id": "11124955", "citationType": "journal article", - "authors": [ - "Rodriguez M.S.", - "Dargemont C.", - "Hay R.T." - ], + "authors": ["Rodriguez M.S.", "Dargemont C.", "Hay R.T."], "citationCrossReferences": [ { "database": "PubMed", @@ -43587,9 +40421,7 @@ "lastPage": "44011", "volume": "276" }, - "referencePositions": [ - "PHOSPHORYLATION BY PRPK" - ] + "referencePositions": ["PHOSPHORYLATION BY PRPK"] }, { "referenceNumber": 55, @@ -43625,26 +40457,14 @@ "lastPage": "292", "volume": "7" }, - "referencePositions": [ - "PHOSPHORYLATION AT SER-392" - ] + "referencePositions": ["PHOSPHORYLATION AT SER-392"] }, { "referenceNumber": 56, "citation": { "id": "12507430", "citationType": "journal article", - "authors": [ - "Hu M.", - "Li P.", - "Li M.", - "Li W.", - "Yao T.", - "Wu J.-W.", - "Gu W.", - "Cohen R.E.", - "Shi Y." - ], + "authors": ["Hu M.", "Li P.", "Li M.", "Li W.", "Yao T.", "Wu J.-W.", "Gu W.", "Cohen R.E.", "Shi Y."], "citationCrossReferences": [ { "database": "PubMed", @@ -43662,22 +40482,14 @@ "lastPage": "1054", "volume": "111" }, - "referencePositions": [ - "INTERACTION WITH USP7" - ] + "referencePositions": ["INTERACTION WITH USP7"] }, { "referenceNumber": 57, "citation": { "id": "11706030", "citationType": "journal article", - "authors": [ - "Tsuji K.", - "Mizumoto K.", - "Yamochi T.", - "Nishimoto I.", - "Matsuoka M." - ], + "authors": ["Tsuji K.", "Mizumoto K.", "Yamochi T.", "Nishimoto I.", "Matsuoka M."], "citationCrossReferences": [ { "database": "PubMed", @@ -43695,20 +40507,14 @@ "lastPage": "2957", "volume": "277" }, - "referencePositions": [ - "IDENTIFICATION IN A COMPLEX WITH CABLES1 AND TP73" - ] + "referencePositions": ["IDENTIFICATION IN A COMPLEX WITH CABLES1 AND TP73"] }, { "referenceNumber": 58, "citation": { "id": "11925430", "citationType": "journal article", - "authors": [ - "Kim E.-J.", - "Park J.-S.", - "Um S.-J." - ], + "authors": ["Kim E.-J.", "Park J.-S.", "Um S.-J."], "citationCrossReferences": [ { "database": "PubMed", @@ -43726,9 +40532,7 @@ "lastPage": "32028", "volume": "277" }, - "referencePositions": [ - "INTERACTION WITH HIPK2" - ] + "referencePositions": ["INTERACTION WITH HIPK2"] }, { "referenceNumber": 59, @@ -43806,10 +40610,7 @@ "lastPage": "19", "volume": "4" }, - "referencePositions": [ - "INTERACTION WITH HIPK2", - "PHOSPHORYLATION AT SER-46" - ] + "referencePositions": ["INTERACTION WITH HIPK2", "PHOSPHORYLATION AT SER-46"] }, { "referenceNumber": 61, @@ -43842,22 +40643,14 @@ "lastPage": "2378", "volume": "63" }, - "referencePositions": [ - "INTERACTION WITH ING4" - ] + "referencePositions": ["INTERACTION WITH ING4"] }, { "referenceNumber": 62, "citation": { "id": "12724314", "citationType": "journal article", - "authors": [ - "Wang Y.H.", - "Tsay Y.G.", - "Tan B.C.", - "Lo W.Y.", - "Lee S.C." - ], + "authors": ["Wang Y.H.", "Tsay Y.G.", "Tan B.C.", "Lo W.Y.", "Lee S.C."], "citationCrossReferences": [ { "database": "PubMed", @@ -43875,23 +40668,14 @@ "lastPage": "25576", "volume": "278" }, - "referencePositions": [ - "ACETYLATION AT LYS-305" - ] + "referencePositions": ["ACETYLATION AT LYS-305"] }, { "referenceNumber": 63, "citation": { "id": "12810724", "citationType": "journal article", - "authors": [ - "Louria-Hayon I.", - "Grossman T.", - "Sionov R.V.", - "Alsheich O.", - "Pandolfi P.P.", - "Haupt Y." - ], + "authors": ["Louria-Hayon I.", "Grossman T.", "Sionov R.V.", "Alsheich O.", "Pandolfi P.P.", "Haupt Y."], "citationCrossReferences": [ { "database": "PubMed", @@ -43951,20 +40735,14 @@ "lastPage": "37729", "volume": "278" }, - "referencePositions": [ - "INTERACTION WITH TP53INP1" - ] + "referencePositions": ["INTERACTION WITH TP53INP1"] }, { "referenceNumber": 65, "citation": { "id": "12944468", "citationType": "journal article", - "authors": [ - "O'Keefe K.", - "Li H.", - "Zhang Y." - ], + "authors": ["O'Keefe K.", "Li H.", "Zhang Y."], "citationCrossReferences": [ { "database": "PubMed", @@ -43982,10 +40760,7 @@ "lastPage": "6405", "volume": "23" }, - "referencePositions": [ - "NUCLEOCYTOPLASMIC SHUTTLING", - "NUCLEAR EXPORT SIGNAL" - ] + "referencePositions": ["NUCLEOCYTOPLASMIC SHUTTLING", "NUCLEAR EXPORT SIGNAL"] }, { "referenceNumber": 66, @@ -44023,10 +40798,7 @@ "lastPage": "167", "volume": "33" }, - "referencePositions": [ - "FUNCTION", - "INTERACTION WITH PPP1R13L; PPP1R13B AND TP53BP2" - ] + "referencePositions": ["FUNCTION", "INTERACTION WITH PPP1R13L; PPP1R13B AND TP53BP2"] }, { "referenceNumber": 67, @@ -44067,24 +40839,14 @@ "lastPage": "5436", "volume": "100" }, - "referencePositions": [ - "INTERACTION WITH HIPK1" - ] + "referencePositions": ["INTERACTION WITH HIPK1"] }, { "referenceNumber": 68, "citation": { "id": "15109303", "citationType": "journal article", - "authors": [ - "Hasan M.K.", - "Yaguchi T.", - "Minoda Y.", - "Hirano T.", - "Taira K.", - "Wadhwa R.", - "Kaul S.C." - ], + "authors": ["Hasan M.K.", "Yaguchi T.", "Minoda Y.", "Hirano T.", "Taira K.", "Wadhwa R.", "Kaul S.C."], "citationCrossReferences": [ { "database": "PubMed", @@ -44102,20 +40864,14 @@ "lastPage": "610", "volume": "380" }, - "referencePositions": [ - "INTERACTION WITH CDKN2AIP" - ] + "referencePositions": ["INTERACTION WITH CDKN2AIP"] }, { "referenceNumber": 69, "citation": { "id": "15186775", "citationType": "journal article", - "authors": [ - "An W.", - "Kim J.", - "Roeder R.G." - ], + "authors": ["An W.", "Kim J.", "Roeder R.G."], "citationCrossReferences": [ { "database": "PubMed", @@ -44133,10 +40889,7 @@ "lastPage": "748", "volume": "117" }, - "referencePositions": [ - "INTERACTION WITH HRMT1L2; EP300 AND CARM1", - "FUNCTION" - ] + "referencePositions": ["INTERACTION WITH HRMT1L2; EP300 AND CARM1", "FUNCTION"] }, { "referenceNumber": 70, @@ -44170,21 +40923,14 @@ "lastPage": "325", "volume": "339" }, - "referencePositions": [ - "INTERACTION WITH ANKRD2" - ] + "referencePositions": ["INTERACTION WITH ANKRD2"] }, { "referenceNumber": 71, "citation": { "id": "15053879", "citationType": "journal article", - "authors": [ - "Li H.-H.", - "Li A.G.", - "Sheppard H.M.", - "Liu X." - ], + "authors": ["Li H.-H.", "Li A.G.", "Sheppard H.M.", "Liu X."], "citationCrossReferences": [ { "database": "PubMed", @@ -44202,23 +40948,14 @@ "lastPage": "878", "volume": "13" }, - "referencePositions": [ - "PHOSPHORYLATION AT THR-55", - "MUTAGENESIS OF THR-55", - "INTERACTION WITH TAF1" - ] + "referencePositions": ["PHOSPHORYLATION AT THR-55", "MUTAGENESIS OF THR-55", "INTERACTION WITH TAF1"] }, { "referenceNumber": 72, "citation": { "id": "15053880", "citationType": "journal article", - "authors": [ - "Li M.", - "Brooks C.L.", - "Kon N.", - "Gu W." - ], + "authors": ["Li M.", "Brooks C.L.", "Kon N.", "Gu W."], "citationCrossReferences": [ { "database": "PubMed", @@ -44236,20 +40973,14 @@ "lastPage": "886", "volume": "13" }, - "referencePositions": [ - "DEUBIQUITINATION BY USP7" - ] + "referencePositions": ["DEUBIQUITINATION BY USP7"] }, { "referenceNumber": 73, "citation": { "id": "15340061", "citationType": "journal article", - "authors": [ - "Ghosh A.", - "Stewart D.", - "Matlashewski G." - ], + "authors": ["Ghosh A.", "Stewart D.", "Matlashewski G."], "citationCrossReferences": [ { "database": "PubMed", @@ -44267,12 +40998,7 @@ "lastPage": "7997", "volume": "24" }, - "referencePositions": [ - "ALTERNATIVE SPLICING (ISOFORM 4)", - "FUNCTION", - "SUBCELLULAR LOCATION", - "UBIQUITINATION" - ] + "referencePositions": ["ALTERNATIVE SPLICING (ISOFORM 4)", "FUNCTION", "SUBCELLULAR LOCATION", "UBIQUITINATION"] }, { "referenceNumber": 74, @@ -44310,10 +41036,7 @@ "lastPage": "360", "volume": "432" }, - "referencePositions": [ - "METHYLATION AT LYS-372", - "MUTAGENESIS OF LYS-372" - ] + "referencePositions": ["METHYLATION AT LYS-372", "MUTAGENESIS OF LYS-372"] }, { "referenceNumber": 75, @@ -44346,9 +41069,7 @@ "lastPage": "976", "volume": "6" }, - "referencePositions": [ - "ACETYLATION AT LYS-382" - ] + "referencePositions": ["ACETYLATION AT LYS-382"] }, { "referenceNumber": 76, @@ -44384,22 +41105,14 @@ "lastPage": "62", "volume": "36" }, - "referencePositions": [ - "INTERACTION WITH AURKA", - "PHOSPHORYLATION AT SER-315" - ] + "referencePositions": ["INTERACTION WITH AURKA", "PHOSPHORYLATION AT SER-315"] }, { "referenceNumber": 77, "citation": { "id": "15687255", "citationType": "journal article", - "authors": [ - "Asher G.", - "Tsvetkov P.", - "Kahana C.", - "Shaul Y." - ], + "authors": ["Asher G.", "Tsvetkov P.", "Kahana C.", "Shaul Y."], "citationCrossReferences": [ { "database": "PubMed", @@ -44417,9 +41130,7 @@ "lastPage": "321", "volume": "19" }, - "referencePositions": [ - "INTERACTION WITH NQO1" - ] + "referencePositions": ["INTERACTION WITH NQO1"] }, { "referenceNumber": 78, @@ -44461,9 +41172,7 @@ "lastPage": "2924", "volume": "19" }, - "referencePositions": [ - "INTERACTION WITH NOC2L" - ] + "referencePositions": ["INTERACTION WITH NOC2L"] }, { "referenceNumber": 79, @@ -44495,23 +41204,14 @@ "lastPage": "16029", "volume": "280" }, - "referencePositions": [ - "RETRACTED PAPER" - ] + "referencePositions": ["RETRACTED PAPER"] }, { "referenceNumber": 80, "citation": { "id": "32144153", "citationType": "journal article", - "authors": [ - "Jalota A.", - "Singh K.", - "Pavithra L.", - "Kaul-Ghanekar R.", - "Jameel S.", - "Chattopadhyay S." - ], + "authors": ["Jalota A.", "Singh K.", "Pavithra L.", "Kaul-Ghanekar R.", "Jameel S.", "Chattopadhyay S."], "citationCrossReferences": [ { "database": "PubMed", @@ -44528,20 +41228,14 @@ "lastPage": "3390", "volume": "295" }, - "referencePositions": [ - "RETRACTION NOTICE OF PUBMED:15701641" - ] + "referencePositions": ["RETRACTION NOTICE OF PUBMED:15701641"] }, { "referenceNumber": 81, "citation": { "id": "15855171", "citationType": "journal article", - "authors": [ - "Golubovskaya V.M.", - "Finch R.", - "Cance W.G." - ], + "authors": ["Golubovskaya V.M.", "Finch R.", "Cance W.G."], "citationCrossReferences": [ { "database": "PubMed", @@ -44559,23 +41253,14 @@ "lastPage": "25021", "volume": "280" }, - "referencePositions": [ - "INTERACTION WITH PTK2/FAK1" - ] + "referencePositions": ["INTERACTION WITH PTK2/FAK1"] }, { "referenceNumber": 82, "citation": { "id": "16219768", "citationType": "journal article", - "authors": [ - "Chang N.-S.", - "Doherty J.", - "Ensign A.", - "Schultz L.", - "Hsu L.-J.", - "Hong Q." - ], + "authors": ["Chang N.-S.", "Doherty J.", "Ensign A.", "Schultz L.", "Hsu L.-J.", "Hong Q."], "citationCrossReferences": [ { "database": "PubMed", @@ -44593,10 +41278,7 @@ "lastPage": "43108", "volume": "280" }, - "referencePositions": [ - "INTERACTION WITH WWOX", - "MUTAGENESIS OF SER-46" - ] + "referencePositions": ["INTERACTION WITH WWOX", "MUTAGENESIS OF SER-46"] }, { "referenceNumber": 83, @@ -44630,19 +41312,14 @@ "lastPage": "293", "volume": "18" }, - "referencePositions": [ - "PHOSPHORYLATION AT SER-15" - ] + "referencePositions": ["PHOSPHORYLATION AT SER-15"] }, { "referenceNumber": 84, "citation": { "id": "17108107", "citationType": "journal article", - "authors": [ - "Zeng P.Y.", - "Berger S.L." - ], + "authors": ["Zeng P.Y.", "Berger S.L."], "citationCrossReferences": [ { "database": "PubMed", @@ -44660,22 +41337,14 @@ "lastPage": "10708", "volume": "66" }, - "referencePositions": [ - "INTERACTION WITH STK11/LKB1", - "PHOSPHORYLATION AT SER-15 AND SER-392" - ] + "referencePositions": ["INTERACTION WITH STK11/LKB1", "PHOSPHORYLATION AT SER-15 AND SER-392"] }, { "referenceNumber": 85, "citation": { "id": "16704422", "citationType": "journal article", - "authors": [ - "Blanco S.", - "Klimcakova L.", - "Vega F.M.", - "Lazo P.A." - ], + "authors": ["Blanco S.", "Klimcakova L.", "Vega F.M.", "Lazo P.A."], "citationCrossReferences": [ { "database": "PubMed", @@ -44693,23 +41362,14 @@ "lastPage": "2504", "volume": "273" }, - "referencePositions": [ - "PHOSPHORYLATION AT THR-18" - ] + "referencePositions": ["PHOSPHORYLATION AT THR-18"] }, { "referenceNumber": 86, "citation": { "id": "16376338", "citationType": "journal article", - "authors": [ - "Gu Y.-M.", - "Jin Y.-H.", - "Choi J.-K.", - "Baek K.-H.", - "Yeo C.-Y.", - "Lee K.-Y." - ], + "authors": ["Gu Y.-M.", "Jin Y.-H.", "Choi J.-K.", "Baek K.-H.", "Yeo C.-Y.", "Lee K.-Y."], "citationCrossReferences": [ { "database": "PubMed", @@ -44727,20 +41387,14 @@ "lastPage": "310", "volume": "580" }, - "referencePositions": [ - "INTERACTION WITH YWHAZ" - ] + "referencePositions": ["INTERACTION WITH YWHAZ"] }, { "referenceNumber": 87, "citation": { "id": "16377624", "citationType": "journal article", - "authors": [ - "Yoshida K.", - "Liu H.", - "Miki Y." - ], + "authors": ["Yoshida K.", "Liu H.", "Miki Y."], "citationCrossReferences": [ { "database": "PubMed", @@ -44758,10 +41412,7 @@ "lastPage": "5740", "volume": "281" }, - "referencePositions": [ - "PHOSPHORYLATION AT SER-46", - "INTERACTION WITH PRKCG" - ] + "referencePositions": ["PHOSPHORYLATION AT SER-46", "INTERACTION WITH PRKCG"] }, { "referenceNumber": 88, @@ -44835,10 +41486,7 @@ "lastPage": "632", "volume": "444" }, - "referencePositions": [ - "METHYLATION AT LYS-370", - "MUTAGENESIS OF LYS-370" - ] + "referencePositions": ["METHYLATION AT LYS-370", "MUTAGENESIS OF LYS-370"] }, { "referenceNumber": 90, @@ -44872,21 +41520,14 @@ "lastPage": "862", "volume": "8" }, - "referencePositions": [ - "INTERACTION WITH DAXX" - ] + "referencePositions": ["INTERACTION WITH DAXX"] }, { "referenceNumber": 91, "citation": { "id": "16415881", "citationType": "journal article", - "authors": [ - "Couture J.-F.", - "Collazo E.", - "Hauk G.", - "Trievel R.C." - ], + "authors": ["Couture J.-F.", "Collazo E.", "Hauk G.", "Trievel R.C."], "citationCrossReferences": [ { "database": "PubMed", @@ -44904,10 +41545,7 @@ "lastPage": "146", "volume": "13" }, - "referencePositions": [ - "MOTIF", - "METHYLATION AT LYS-372" - ] + "referencePositions": ["MOTIF", "METHYLATION AT LYS-372"] }, { "referenceNumber": 92, @@ -44940,9 +41578,7 @@ "lastPage": "6652", "volume": "34" }, - "referencePositions": [ - "INTERACTION WITH POU4F2" - ] + "referencePositions": ["INTERACTION WITH POU4F2"] }, { "referenceNumber": 93, @@ -44983,25 +41619,14 @@ "lastPage": "308", "volume": "128" }, - "referencePositions": [ - "PHOSPHORYLATION AT SER-37", - "MUTAGENESIS OF SER-37" - ] + "referencePositions": ["PHOSPHORYLATION AT SER-37", "MUTAGENESIS OF SER-37"] }, { "referenceNumber": 94, "citation": { "id": "17719541", "citationType": "journal article", - "authors": [ - "Das S.", - "Raj L.", - "Zhao B.", - "Kimura Y.", - "Bernstein A.", - "Aaronson S.A.", - "Lee S.W." - ], + "authors": ["Das S.", "Raj L.", "Zhao B.", "Kimura Y.", "Bernstein A.", "Aaronson S.A.", "Lee S.W."], "citationCrossReferences": [ { "database": "PubMed", @@ -45019,10 +41644,7 @@ "lastPage": "637", "volume": "130" }, - "referencePositions": [ - "INTERACTION WITH ZNF385A", - "CHARACTERIZATION OF VARIANTS ALA-143; HIS-175 AND PRO-175" - ] + "referencePositions": ["INTERACTION WITH ZNF385A", "CHARACTERIZATION OF VARIANTS ALA-143; HIS-175 AND PRO-175"] }, { "referenceNumber": 95, @@ -45080,24 +41702,14 @@ "lastPage": "122", "volume": "26" }, - "referencePositions": [ - "UBIQUITINATION", - "INTERACTION WITH SYVN1", - "SUBCELLULAR LOCATION" - ] + "referencePositions": ["UBIQUITINATION", "INTERACTION WITH SYVN1", "SUBCELLULAR LOCATION"] }, { "referenceNumber": 96, "citation": { "id": "17245430", "citationType": "journal article", - "authors": [ - "Li H.H.", - "Cai X.", - "Shouse G.P.", - "Piluso L.G.", - "Liu X." - ], + "authors": ["Li H.H.", "Cai X.", "Shouse G.P.", "Piluso L.G.", "Liu X."], "citationCrossReferences": [ { "database": "PubMed", @@ -45115,22 +41727,14 @@ "lastPage": "411", "volume": "26" }, - "referencePositions": [ - "INTERACTION WITH PPP2CA; PPP2R1A; PPP2R2A AND PPP2R5C" - ] + "referencePositions": ["INTERACTION WITH PPP2CA; PPP2R1A; PPP2R2A AND PPP2R5C"] }, { "referenceNumber": 97, "citation": { "id": "17904127", "citationType": "journal article", - "authors": [ - "Zhou X.", - "Yang G.", - "Huang R.", - "Chen X.", - "Hu G." - ], + "authors": ["Zhou X.", "Yang G.", "Huang R.", "Chen X.", "Hu G."], "citationCrossReferences": [ { "database": "PubMed", @@ -45148,23 +41752,14 @@ "lastPage": "4948", "volume": "581" }, - "referencePositions": [ - "INTERACTION WITH ARMC10" - ] + "referencePositions": ["INTERACTION WITH ARMC10"] }, { "referenceNumber": 98, "citation": { "id": "18022393", "citationType": "journal article", - "authors": [ - "Arai S.", - "Matsushita A.", - "Du K.", - "Yagi K.", - "Okazaki Y.", - "Kurokawa R." - ], + "authors": ["Arai S.", "Matsushita A.", "Du K.", "Yagi K.", "Okazaki Y.", "Kurokawa R."], "citationCrossReferences": [ { "database": "PubMed", @@ -45182,23 +41777,14 @@ "lastPage": "5657", "volume": "581" }, - "referencePositions": [ - "PHOSPHORYLATION AT SER-9" - ] + "referencePositions": ["PHOSPHORYLATION AT SER-9"] }, { "referenceNumber": 99, "citation": { "id": "17467953", "citationType": "journal article", - "authors": [ - "Piskacek S.", - "Gregor M.", - "Nemethova M.", - "Grabner M.", - "Kovarik P.", - "Piskacek M." - ], + "authors": ["Piskacek S.", "Gregor M.", "Nemethova M.", "Grabner M.", "Kovarik P.", "Piskacek M."], "citationCrossReferences": [ { "database": "PubMed", @@ -45216,9 +41802,7 @@ "lastPage": "768", "volume": "89" }, - "referencePositions": [ - "DOMAIN" - ] + "referencePositions": ["DOMAIN"] }, { "referenceNumber": 100, @@ -45253,9 +41837,7 @@ "lastPage": "3281", "volume": "282" }, - "referencePositions": [ - "INTERACTION WITH RFFL AND RNF34" - ] + "referencePositions": ["INTERACTION WITH RFFL AND RNF34"] }, { "referenceNumber": 101, @@ -45302,22 +41884,14 @@ "lastPage": "11981", "volume": "282" }, - "referencePositions": [ - "FUNCTION", - "INTERACTION WITH MAML1" - ] + "referencePositions": ["FUNCTION", "INTERACTION WITH MAML1"] }, { "referenceNumber": 102, "citation": { "id": "17591690", "citationType": "journal article", - "authors": [ - "Lee J.-H.", - "Kim H.-S.", - "Lee S.-J.", - "Kim K.-T." - ], + "authors": ["Lee J.-H.", "Kim H.-S.", "Lee S.-J.", "Kim K.-T."], "citationCrossReferences": [ { "database": "PubMed", @@ -45375,22 +41949,14 @@ "lastPage": "1709", "volume": "18" }, - "referencePositions": [ - "INTERACTION WITH MORC3" - ] + "referencePositions": ["INTERACTION WITH MORC3"] }, { "referenceNumber": 104, "citation": { "id": "17349958", "citationType": "journal article", - "authors": [ - "Taira N.", - "Nihira K.", - "Yamaguchi T.", - "Miki Y.", - "Yoshida K." - ], + "authors": ["Taira N.", "Nihira K.", "Yamaguchi T.", "Miki Y.", "Yoshida K."], "citationCrossReferences": [ { "database": "PubMed", @@ -45408,11 +41974,7 @@ "lastPage": "738", "volume": "25" }, - "referencePositions": [ - "FUNCTION", - "PHOSPHORYLATION AT SER-46", - "MUTAGENESIS OF SER-46" - ] + "referencePositions": ["FUNCTION", "PHOSPHORYLATION AT SER-46", "MUTAGENESIS OF SER-46"] }, { "referenceNumber": 105, @@ -45449,9 +42011,7 @@ "lastPage": "108", "volume": "449" }, - "referencePositions": [ - "DEMETHYLATION BY KDM1A" - ] + "referencePositions": ["DEMETHYLATION BY KDM1A"] }, { "referenceNumber": 106, @@ -45486,10 +42046,7 @@ "lastPage": "646", "volume": "27" }, - "referencePositions": [ - "METHYLATION AT LYS-382", - "MUTAGENESIS OF LYS-382" - ] + "referencePositions": ["METHYLATION AT LYS-382", "MUTAGENESIS OF LYS-382"] }, { "referenceNumber": 107, @@ -45528,9 +42085,7 @@ "lastPage": "1166", "volume": "316" }, - "referencePositions": [ - "IDENTIFICATION BY MASS SPECTROMETRY [LARGE SCALE ANALYSIS]" - ], + "referencePositions": ["IDENTIFICATION BY MASS SPECTROMETRY [LARGE SCALE ANALYSIS]"], "referenceComments": [ { "value": "Embryonic kidney", @@ -45543,15 +42098,7 @@ "citation": { "id": "18249187", "citationType": "journal article", - "authors": [ - "Jin Y.H.", - "Kim Y.J.", - "Kim D.W.", - "Baek K.H.", - "Kang B.Y.", - "Yeo C.Y.", - "Lee K.Y." - ], + "authors": ["Jin Y.H.", "Kim Y.J.", "Kim D.W.", "Baek K.H.", "Kang B.Y.", "Yeo C.Y.", "Lee K.Y."], "citationCrossReferences": [ { "database": "PubMed", @@ -45569,26 +42116,14 @@ "lastPage": "695", "volume": "368" }, - "referencePositions": [ - "ACETYLATION", - "DEACETYLATION BY SIRT2" - ] + "referencePositions": ["ACETYLATION", "DEACETYLATION BY SIRT2"] }, { "referenceNumber": 109, "citation": { "id": "18585004", "citationType": "journal article", - "authors": [ - "Xie P.", - "Tian C.", - "An L.", - "Nie J.", - "Lu K.", - "Xing G.", - "Zhang L.", - "He F." - ], + "authors": ["Xie P.", "Tian C.", "An L.", "Nie J.", "Lu K.", "Xing G.", "Zhang L.", "He F."], "citationCrossReferences": [ { "database": "PubMed", @@ -45606,9 +42141,7 @@ "lastPage": "1678", "volume": "20" }, - "referencePositions": [ - "INTERACTION WITH SETD2" - ] + "referencePositions": ["INTERACTION WITH SETD2"] }, { "referenceNumber": 110, @@ -45641,9 +42174,7 @@ "lastPage": "430", "volume": "8" }, - "referencePositions": [ - "INTERACTION WITH NUPR1" - ] + "referencePositions": ["INTERACTION WITH NUPR1"] }, { "referenceNumber": 111, @@ -45679,24 +42210,14 @@ "lastPage": "22", "volume": "29" }, - "referencePositions": [ - "UBIQUITINATION", - "SUBCELLULAR LOCATION" - ] + "referencePositions": ["UBIQUITINATION", "SUBCELLULAR LOCATION"] }, { "referenceNumber": 112, "citation": { "id": "17954561", "citationType": "journal article", - "authors": [ - "Iizuka M.", - "Sarmento O.F.", - "Sekiya T.", - "Scrable H.", - "Allis C.D.", - "Smith M.M." - ], + "authors": ["Iizuka M.", "Sarmento O.F.", "Sekiya T.", "Scrable H.", "Allis C.D.", "Smith M.M."], "citationCrossReferences": [ { "database": "PubMed", @@ -45714,20 +42235,14 @@ "lastPage": "153", "volume": "28" }, - "referencePositions": [ - "INTERACTION WITH KAT7" - ] + "referencePositions": ["INTERACTION WITH KAT7"] }, { "referenceNumber": 113, "citation": { "id": "17967874", "citationType": "journal article", - "authors": [ - "Shouse G.P.", - "Cai X.", - "Liu X." - ], + "authors": ["Shouse G.P.", "Cai X.", "Liu X."], "citationCrossReferences": [ { "database": "PubMed", @@ -45756,14 +42271,7 @@ "citation": { "id": "19413330", "citationType": "journal article", - "authors": [ - "Gauci S.", - "Helbig A.O.", - "Slijper M.", - "Krijgsveld J.", - "Heck A.J.", - "Mohammed S." - ], + "authors": ["Gauci S.", "Helbig A.O.", "Slijper M.", "Krijgsveld J.", "Heck A.J.", "Mohammed S."], "citationCrossReferences": [ { "database": "PubMed", @@ -45781,9 +42289,7 @@ "lastPage": "4501", "volume": "81" }, - "referencePositions": [ - "IDENTIFICATION BY MASS SPECTROMETRY [LARGE SCALE ANALYSIS]" - ] + "referencePositions": ["IDENTIFICATION BY MASS SPECTROMETRY [LARGE SCALE ANALYSIS]"] }, { "referenceNumber": 115, @@ -45829,15 +42335,7 @@ "citation": { "id": "19473992", "citationType": "journal article", - "authors": [ - "Yang X.", - "Li H.", - "Zhou Z.", - "Wang W.H.", - "Deng A.", - "Andrisani O.", - "Liu X." - ], + "authors": ["Yang X.", "Li H.", "Zhou Z.", "Wang W.H.", "Deng A.", "Andrisani O.", "Liu X."], "citationCrossReferences": [ { "database": "PubMed", @@ -45855,19 +42353,14 @@ "lastPage": "18592", "volume": "284" }, - "referencePositions": [ - "UBIQUITINATION BY TOPORS" - ] + "referencePositions": ["UBIQUITINATION BY TOPORS"] }, { "referenceNumber": 117, "citation": { "id": "19033443", "citationType": "journal article", - "authors": [ - "Kruse J.P.", - "Gu W." - ], + "authors": ["Kruse J.P.", "Gu W."], "citationCrossReferences": [ { "database": "PubMed", @@ -45885,10 +42378,7 @@ "lastPage": "3263", "volume": "284" }, - "referencePositions": [ - "UBIQUITINATION AT LYS-351 AND LYS-357", - "SUBCELLULAR LOCATION" - ] + "referencePositions": ["UBIQUITINATION AT LYS-351 AND LYS-357", "SUBCELLULAR LOCATION"] }, { "referenceNumber": 118, @@ -45921,25 +42411,14 @@ "lastPage": "34552", "volume": "284" }, - "referencePositions": [ - "INTERACTION WITH MTA1 AND COP1", - "UBIQUITINATION" - ] + "referencePositions": ["INTERACTION WITH MTA1 AND COP1", "UBIQUITINATION"] }, { "referenceNumber": 119, "citation": { "id": "19776115", "citationType": "journal article", - "authors": [ - "Hwang E.S.", - "Zhang Z.", - "Cai H.", - "Huang D.Y.", - "Huong S.M.", - "Cha C.Y.", - "Huang E.S." - ], + "authors": ["Hwang E.S.", "Zhang Z.", "Cai H.", "Huang D.Y.", "Huong S.M.", "Cha C.Y.", "Huang E.S."], "citationCrossReferences": [ { "database": "PubMed", @@ -45957,22 +42436,14 @@ "lastPage": "12398", "volume": "83" }, - "referencePositions": [ - "INTERACTION WITH HHV-5 PROTEIN UL123 (MICROBIAL INFECTION)" - ] + "referencePositions": ["INTERACTION WITH HHV-5 PROTEIN UL123 (MICROBIAL INFECTION)"] }, { "referenceNumber": 120, "citation": { "id": "19854137", "citationType": "journal article", - "authors": [ - "Li X.", - "Wu L.", - "Corsa C.A.", - "Kunkel S.", - "Dou Y." - ], + "authors": ["Li X.", "Wu L.", "Corsa C.A.", "Kunkel S.", "Dou Y."], "citationCrossReferences": [ { "database": "PubMed", @@ -45990,9 +42461,7 @@ "lastPage": "301", "volume": "36" }, - "referencePositions": [ - "ACETYLATION AT LYS-120" - ] + "referencePositions": ["ACETYLATION AT LYS-120"] }, { "referenceNumber": 121, @@ -46030,9 +42499,7 @@ "lastPage": "10200", "volume": "106" }, - "referencePositions": [ - "INTERACTION WITH FBXO42" - ] + "referencePositions": ["INTERACTION WITH FBXO42"] }, { "referenceNumber": 122, @@ -46067,11 +42534,7 @@ "lastPage": "11616", "volume": "106" }, - "referencePositions": [ - "FUNCTION", - "UBIQUITINATION", - "INTERACTION WITH TRIM24" - ] + "referencePositions": ["FUNCTION", "UBIQUITINATION", "INTERACTION WITH TRIM24"] }, { "referenceNumber": 123, @@ -46142,22 +42605,14 @@ "lastPage": "10", "volume": "11" }, - "referencePositions": [ - "INTERACTION WITH TAF6 ISOFORMS 1 AND 4" - ] + "referencePositions": ["INTERACTION WITH TAF6 ISOFORMS 1 AND 4"] }, { "referenceNumber": 125, "citation": { "id": "20096447", "citationType": "journal article", - "authors": [ - "Yuan J.", - "Luo K.", - "Zhang L.", - "Cheville J.C.", - "Lou Z." - ], + "authors": ["Yuan J.", "Luo K.", "Zhang L.", "Cheville J.C.", "Lou Z."], "citationCrossReferences": [ { "database": "PubMed", @@ -46175,10 +42630,7 @@ "lastPage": "396", "volume": "140" }, - "referencePositions": [ - "UBIQUITINATION", - "DEUBIQUITINATION BY USP10" - ] + "referencePositions": ["UBIQUITINATION", "DEUBIQUITINATION BY USP10"] }, { "referenceNumber": 126, @@ -46219,23 +42671,14 @@ "lastPage": "419", "volume": "142" }, - "referencePositions": [ - "FUNCTION" - ] + "referencePositions": ["FUNCTION"] }, { "referenceNumber": 127, "citation": { "id": "20041275", "citationType": "journal article", - "authors": [ - "Venerando A.", - "Marin O.", - "Cozza G.", - "Bustos V.H.", - "Sarno S.", - "Pinna L.A." - ], + "authors": ["Venerando A.", "Marin O.", "Cozza G.", "Bustos V.H.", "Sarno S.", "Pinna L.A."], "citationCrossReferences": [ { "database": "PubMed", @@ -46253,20 +42696,14 @@ "lastPage": "1118", "volume": "67" }, - "referencePositions": [ - "PHOSPHORYLATION AT SER-20 BY CSNK1D/CK1" - ] + "referencePositions": ["PHOSPHORYLATION AT SER-20 BY CSNK1D/CK1"] }, { "referenceNumber": 128, "citation": { "id": "20385133", "citationType": "journal article", - "authors": [ - "Lim S.O.", - "Kim H.", - "Jung G." - ], + "authors": ["Lim S.O.", "Kim H.", "Jung G."], "citationCrossReferences": [ { "database": "PubMed", @@ -46295,14 +42732,7 @@ "citation": { "id": "19880522", "citationType": "journal article", - "authors": [ - "Lim S.T.", - "Miller N.L.", - "Nam J.O.", - "Chen X.L.", - "Lim Y.", - "Schlaepfer D.D." - ], + "authors": ["Lim S.T.", "Miller N.L.", "Nam J.O.", "Chen X.L.", "Lim Y.", "Schlaepfer D.D."], "citationCrossReferences": [ { "database": "PubMed", @@ -46320,11 +42750,7 @@ "lastPage": "1753", "volume": "285" }, - "referencePositions": [ - "INTERACTION WITH PTK2B/PYK2 AND MDM2", - "UBIQUITINATION", - "SUBCELLULAR LOCATION" - ] + "referencePositions": ["INTERACTION WITH PTK2B/PYK2 AND MDM2", "UBIQUITINATION", "SUBCELLULAR LOCATION"] }, { "referenceNumber": 130, @@ -46358,10 +42784,7 @@ "lastPage": "9641", "volume": "285" }, - "referencePositions": [ - "METHYLATION AT LYS-373", - "MUTAGENESIS OF LYS-373" - ] + "referencePositions": ["METHYLATION AT LYS-373", "MUTAGENESIS OF LYS-373"] }, { "referenceNumber": 131, @@ -46384,23 +42807,14 @@ "lastPage": "18122", "volume": "285" }, - "referencePositions": [ - "ERRATUM OF PUBMED:20118233" - ] + "referencePositions": ["ERRATUM OF PUBMED:20118233"] }, { "referenceNumber": 132, "citation": { "id": "20124405", "citationType": "journal article", - "authors": [ - "Chen X.", - "Zhu H.", - "Yuan M.", - "Fu J.", - "Zhou Y.", - "Ma L." - ], + "authors": ["Chen X.", "Zhu H.", "Yuan M.", "Fu J.", "Zhou Y.", "Ma L."], "citationCrossReferences": [ { "database": "PubMed", @@ -46418,10 +42832,7 @@ "lastPage": "12830", "volume": "285" }, - "referencePositions": [ - "PHOSPHORYLATION AT THR-55", - "INTERACTION WITH GRK5" - ] + "referencePositions": ["PHOSPHORYLATION AT THR-55", "INTERACTION WITH GRK5"] }, { "referenceNumber": 133, @@ -46458,10 +42869,7 @@ "lastPage": "389", "volume": "12" }, - "referencePositions": [ - "INTERACTION WITH BRD7", - "ACETYLATION AT LYS-382" - ] + "referencePositions": ["INTERACTION WITH BRD7", "ACETYLATION AT LYS-382"] }, { "referenceNumber": 134, @@ -46500,20 +42908,14 @@ "lastPage": "4584", "volume": "107" }, - "referencePositions": [ - "UBIQUITINATION BY RFWD3" - ] + "referencePositions": ["UBIQUITINATION BY RFWD3"] }, { "referenceNumber": 135, "citation": { "id": "20660729", "citationType": "journal article", - "authors": [ - "Burrows A.E.", - "Smogorzewska A.", - "Elledge S.J." - ], + "authors": ["Burrows A.E.", "Smogorzewska A.", "Elledge S.J."], "citationCrossReferences": [ { "database": "PubMed", @@ -46531,9 +42933,7 @@ "lastPage": "14285", "volume": "107" }, - "referencePositions": [ - "INTERACTION WITH BRD7" - ] + "referencePositions": ["INTERACTION WITH BRD7"] }, { "referenceNumber": 136, @@ -46567,22 +42967,14 @@ "lastPage": "17", "volume": "5" }, - "referencePositions": [ - "IDENTIFICATION BY MASS SPECTROMETRY [LARGE SCALE ANALYSIS]" - ] + "referencePositions": ["IDENTIFICATION BY MASS SPECTROMETRY [LARGE SCALE ANALYSIS]"] }, { "referenceNumber": 137, "citation": { "id": "21952639", "citationType": "journal article", - "authors": [ - "Mori T.", - "Ikeda D.D.", - "Fukushima T.", - "Takenoshita S.", - "Kochi H." - ], + "authors": ["Mori T.", "Ikeda D.D.", "Fukushima T.", "Takenoshita S.", "Kochi H."], "citationCrossReferences": [ { "database": "PubMed", @@ -46600,9 +42992,7 @@ "lastPage": "3299", "volume": "10" }, - "referencePositions": [ - "INTERACTION WITH UHRF2" - ] + "referencePositions": ["INTERACTION WITH UHRF2"] }, { "referenceNumber": 138, @@ -46653,12 +43043,7 @@ "citation": { "id": "20959462", "citationType": "journal article", - "authors": [ - "Wu L.", - "Ma C.A.", - "Zhao Y.", - "Jain A." - ], + "authors": ["Wu L.", "Ma C.A.", "Zhao Y.", "Jain A."], "citationCrossReferences": [ { "database": "PubMed", @@ -46690,16 +43075,7 @@ "citation": { "id": "22914926", "citationType": "journal article", - "authors": [ - "Xu L.", - "Hu J.", - "Zhao Y.", - "Hu J.", - "Xiao J.", - "Wang Y.", - "Ma D.", - "Chen Y." - ], + "authors": ["Xu L.", "Hu J.", "Zhao Y.", "Hu J.", "Xiao J.", "Wang Y.", "Ma D.", "Chen Y."], "citationCrossReferences": [ { "database": "PubMed", @@ -46717,10 +43093,7 @@ "lastPage": "1245", "volume": "17" }, - "referencePositions": [ - "INTERACTION WITH MDM2 AND PDCD5", - "SUBCELLULAR LOCATION" - ] + "referencePositions": ["INTERACTION WITH MDM2 AND PDCD5", "SUBCELLULAR LOCATION"] }, { "referenceNumber": 141, @@ -46776,9 +43149,7 @@ "lastPage": "747", "volume": "1" }, - "referencePositions": [ - "INTERACTION WITH ALDOB AND G6PD" - ] + "referencePositions": ["INTERACTION WITH ALDOB AND G6PD"] }, { "referenceNumber": 142, @@ -46907,23 +43278,14 @@ "lastPage": "1103", "volume": "43" }, - "referencePositions": [ - "INVOLVEMENT IN BCC7" - ] + "referencePositions": ["INVOLVEMENT IN BCC7"] }, { "referenceNumber": 143, "citation": { "id": "21317932", "citationType": "journal article", - "authors": [ - "Hou X.", - "Liu J.E.", - "Liu W.", - "Liu C.Y.", - "Liu Z.Y.", - "Sun Z.Y." - ], + "authors": ["Hou X.", "Liu J.E.", "Liu W.", "Liu C.Y.", "Liu Z.Y.", "Sun Z.Y."], "citationCrossReferences": [ { "database": "PubMed", @@ -46941,24 +43303,14 @@ "lastPage": "2942", "volume": "30" }, - "referencePositions": [ - "INTERACTION WITH NUAK1", - "PHOSPHORYLATION AT SER-15 AND SER-392" - ] + "referencePositions": ["INTERACTION WITH NUAK1", "PHOSPHORYLATION AT SER-15 AND SER-392"] }, { "referenceNumber": 144, "citation": { "id": "22726440", "citationType": "journal article", - "authors": [ - "Vaseva A.V.", - "Marchenko N.D.", - "Ji K.", - "Tsirka S.E.", - "Holzmann S.", - "Moll U.M." - ], + "authors": ["Vaseva A.V.", "Marchenko N.D.", "Ji K.", "Tsirka S.E.", "Holzmann S.", "Moll U.M."], "citationCrossReferences": [ { "database": "PubMed", @@ -46976,24 +43328,14 @@ "lastPage": "1548", "volume": "149" }, - "referencePositions": [ - "FUNCTION", - "SUBCELLULAR LOCATION", - "INTERACTION WITH PPIF" - ] + "referencePositions": ["FUNCTION", "SUBCELLULAR LOCATION", "INTERACTION WITH PPIF"] }, { "referenceNumber": 145, "citation": { "id": "22214662", "citationType": "journal article", - "authors": [ - "Bennett R.L.", - "Pan Y.", - "Christian J.", - "Hui T.", - "May W.S. Jr." - ], + "authors": ["Bennett R.L.", "Pan Y.", "Christian J.", "Hui T.", "May W.S. Jr."], "citationCrossReferences": [ { "database": "PubMed", @@ -47011,11 +43353,7 @@ "lastPage": "417", "volume": "11" }, - "referencePositions": [ - "INTERACTION WITH UBC9", - "PHOSPHORYLATION AT SER-392", - "SUMOYLATION AT LYS-386" - ] + "referencePositions": ["INTERACTION WITH UBC9", "PHOSPHORYLATION AT SER-392", "SUMOYLATION AT LYS-386"] }, { "referenceNumber": 146, @@ -47049,9 +43387,7 @@ "lastPage": "1622", "volume": "19" }, - "referencePositions": [ - "INTERACTION WITH NOP53" - ] + "referencePositions": ["INTERACTION WITH NOP53"] }, { "referenceNumber": 147, @@ -47089,9 +43425,7 @@ "lastPage": "3415", "volume": "42" }, - "referencePositions": [ - "INTERACTION WITH ZNF385B" - ] + "referencePositions": ["INTERACTION WITH ZNF385B"] }, { "referenceNumber": 148, @@ -47130,21 +43464,14 @@ "lastPage": "924", "volume": "19" }, - "referencePositions": [ - "METHYLATION AT LYS-370 AND LYS-382" - ] + "referencePositions": ["METHYLATION AT LYS-370 AND LYS-382"] }, { "referenceNumber": 149, "citation": { "id": "24051492", "citationType": "journal article", - "authors": [ - "Miki T.", - "Matsumoto T.", - "Zhao Z.", - "Lee C.C." - ], + "authors": ["Miki T.", "Matsumoto T.", "Zhao Z.", "Lee C.C."], "citationCrossReferences": [ { "database": "PubMed", @@ -47162,23 +43489,14 @@ "lastPage": "2444", "volume": "4" }, - "referencePositions": [ - "FUNCTION" - ] + "referencePositions": ["FUNCTION"] }, { "referenceNumber": 150, "citation": { "id": "23431171", "citationType": "journal article", - "authors": [ - "Rokudai S.", - "Laptenko O.", - "Arnal S.M.", - "Taya Y.", - "Kitabayashi I.", - "Prives C." - ], + "authors": ["Rokudai S.", "Laptenko O.", "Arnal S.M.", "Taya Y.", "Kitabayashi I.", "Prives C."], "citationCrossReferences": [ { "database": "PubMed", @@ -47196,9 +43514,7 @@ "lastPage": "3900", "volume": "110" }, - "referencePositions": [ - "ACETYLATION AT LYS-120 AND LYS-382" - ] + "referencePositions": ["ACETYLATION AT LYS-120 AND LYS-382"] }, { "referenceNumber": 151, @@ -47237,10 +43553,7 @@ "lastPage": "e1118", "volume": "5" }, - "referencePositions": [ - "INTERACTION WITH HSPA9", - "SUBCELLULAR LOCATION" - ] + "referencePositions": ["INTERACTION WITH HSPA9", "SUBCELLULAR LOCATION"] }, { "referenceNumber": 152, @@ -47273,10 +43586,7 @@ "lastPage": "5540", "volume": "32" }, - "referencePositions": [ - "INTERACTION WITH S100A4", - "SUBCELLULAR LOCATION" - ] + "referencePositions": ["INTERACTION WITH S100A4", "SUBCELLULAR LOCATION"] }, { "referenceNumber": 153, @@ -47313,9 +43623,7 @@ "lastPage": "10", "volume": "106" }, - "referencePositions": [ - "FUNCTION" - ] + "referencePositions": ["FUNCTION"] }, { "referenceNumber": 154, @@ -47352,23 +43660,14 @@ "lastPage": "E5291", "volume": "111" }, - "referencePositions": [ - "INTERACTION WITH UBD" - ] + "referencePositions": ["INTERACTION WITH UBD"] }, { "referenceNumber": 155, "citation": { "id": "25168243", "citationType": "journal article", - "authors": [ - "Maniam S.", - "Coutts A.S.", - "Stratford M.R.", - "McGouran J.", - "Kessler B.", - "La Thangue N.B." - ], + "authors": ["Maniam S.", "Coutts A.S.", "Stratford M.R.", "McGouran J.", "Kessler B.", "La Thangue N.B."], "citationCrossReferences": [ { "database": "PubMed", @@ -47386,25 +43685,14 @@ "lastPage": "163", "volume": "22" }, - "referencePositions": [ - "INTERACTION WITH TTC5", - "SUBCELLULAR LOCATION" - ] + "referencePositions": ["INTERACTION WITH TTC5", "SUBCELLULAR LOCATION"] }, { "referenceNumber": 156, "citation": { "id": "26634371", "citationType": "journal article", - "authors": [ - "Sane S.", - "Abdullah A.", - "Nelson M.E.", - "Wang H.", - "Chauhan S.C.", - "Newton S.S.", - "Rezvani K." - ], + "authors": ["Sane S.", "Abdullah A.", "Nelson M.E.", "Wang H.", "Chauhan S.C.", "Newton S.S.", "Rezvani K."], "citationCrossReferences": [ { "database": "PubMed", @@ -47422,24 +43710,14 @@ "lastPage": "326", "volume": "21" }, - "referencePositions": [ - "SUBCELLULAR LOCATION" - ] + "referencePositions": ["SUBCELLULAR LOCATION"] }, { "referenceNumber": 157, "citation": { "id": "25591766", "citationType": "journal article", - "authors": [ - "Yang L.", - "Zhou B.", - "Li X.", - "Lu Z.", - "Li W.", - "Huo X.", - "Miao Z." - ], + "authors": ["Yang L.", "Zhou B.", "Li X.", "Lu Z.", "Li W.", "Huo X.", "Miao Z."], "citationCrossReferences": [ { "database": "PubMed", @@ -47457,9 +43735,7 @@ "lastPage": "245", "volume": "35" }, - "referencePositions": [ - "UBIQUITINATION" - ] + "referencePositions": ["UBIQUITINATION"] }, { "referenceNumber": 158, @@ -47493,9 +43769,7 @@ "lastPage": "82", "volume": "35" }, - "referencePositions": [ - "INTERACTION WITH HADV5 E1B-55K (MICROBIAL INFECTION)" - ] + "referencePositions": ["INTERACTION WITH HADV5 E1B-55K (MICROBIAL INFECTION)"] }, { "referenceNumber": 159, @@ -47536,24 +43810,14 @@ "lastPage": "10574", "volume": "7" }, - "referencePositions": [ - "INTERACTION WITH FBXO22", - "UBIQUITINATION" - ] + "referencePositions": ["INTERACTION WITH FBXO22", "UBIQUITINATION"] }, { "referenceNumber": 160, "citation": { "id": "27323408", "citationType": "journal article", - "authors": [ - "Cesnekova J.", - "Spacilova J.", - "Hansikova H.", - "Houstek J.", - "Zeman J.", - "Stiburek L." - ], + "authors": ["Cesnekova J.", "Spacilova J.", "Hansikova H.", "Houstek J.", "Zeman J.", "Stiburek L."], "citationCrossReferences": [ { "database": "PubMed", @@ -47571,23 +43835,14 @@ "lastPage": "47698", "volume": "7" }, - "referencePositions": [ - "INTERACTION WITH AFG1L", - "SUBCELLULAR LOCATION" - ] + "referencePositions": ["INTERACTION WITH AFG1L", "SUBCELLULAR LOCATION"] }, { "referenceNumber": 161, "citation": { "id": "28842590", "citationType": "journal article", - "authors": [ - "Chen W.J.", - "Wang W.T.", - "Tsai T.Y.", - "Li H.K.", - "Lee Y.W." - ], + "authors": ["Chen W.J.", "Wang W.T.", "Tsai T.Y.", "Li H.K.", "Lee Y.W."], "citationCrossReferences": [ { "database": "PubMed", @@ -47616,17 +43871,7 @@ "citation": { "id": "28807825", "citationType": "journal article", - "authors": [ - "Fu S.", - "Shao S.", - "Wang L.", - "Liu H.", - "Hou H.", - "Wang Y.", - "Wang H.", - "Huang X.", - "Lv R." - ], + "authors": ["Fu S.", "Shao S.", "Wang L.", "Liu H.", "Hou H.", "Wang Y.", "Wang H.", "Huang X.", "Lv R."], "citationCrossReferences": [ { "database": "PubMed", @@ -47644,9 +43889,7 @@ "lastPage": "183", "volume": "492" }, - "referencePositions": [ - "DEUBIQUITINATION BY USP3" - ] + "referencePositions": ["DEUBIQUITINATION BY USP3"] }, { "referenceNumber": 163, @@ -47680,9 +43923,7 @@ "lastPage": "2776", "volume": "49" }, - "referencePositions": [ - "INTERACTION WITH DAZAP2" - ] + "referencePositions": ["INTERACTION WITH DAZAP2"] }, { "referenceNumber": 164, @@ -47747,9 +43988,7 @@ "lastPage": "447", "volume": "103" }, - "referencePositions": [ - "INVOLVEMENT IN BMFS5" - ] + "referencePositions": ["INVOLVEMENT IN BMFS5"] }, { "referenceNumber": 165, @@ -47796,11 +44035,7 @@ "lastPage": "774.e5", "volume": "25" }, - "referencePositions": [ - "INTERACTION WITH MORN3", - "DEACETYLATION AT LYS-382", - "UBQIQUITINATION" - ] + "referencePositions": ["INTERACTION WITH MORN3", "DEACETYLATION AT LYS-382", "UBQIQUITINATION"] }, { "referenceNumber": 166, @@ -47835,23 +44070,14 @@ "lastPage": "0", "volume": "7" }, - "referencePositions": [ - "ACETYLATION AT LYS-381", - "DEACETYLATION BY SIRT6", - "MUTAGENESIS OF LYS-381" - ] + "referencePositions": ["ACETYLATION AT LYS-381", "DEACETYLATION BY SIRT6", "MUTAGENESIS OF LYS-381"] }, { "referenceNumber": 167, "citation": { "id": "31527692", "citationType": "journal article", - "authors": [ - "Martin-Doncel E.", - "Rojas A.M.", - "Cantarero L.", - "Lazo P.A." - ], + "authors": ["Martin-Doncel E.", "Rojas A.M.", "Cantarero L.", "Lazo P.A."], "citationCrossReferences": [ { "database": "PubMed", @@ -47869,9 +44095,7 @@ "lastPage": "13381", "volume": "9" }, - "referencePositions": [ - "PHOSPHORYLATION AT THR-18 BY VRK1" - ] + "referencePositions": ["PHOSPHORYLATION AT THR-18 BY VRK1"] }, { "referenceNumber": 168, @@ -47905,22 +44129,14 @@ "lastPage": "580", "volume": "10" }, - "referencePositions": [ - "DOMAIN", - "MUTAGENESIS OF SER-392" - ] + "referencePositions": ["DOMAIN", "MUTAGENESIS OF SER-392"] }, { "referenceNumber": 169, "citation": { "id": "34523970", "citationType": "journal article", - "authors": [ - "Alzhanova D.", - "Meyo J.O.", - "Juarez A.", - "Dittmer D.P." - ], + "authors": ["Alzhanova D.", "Meyo J.O.", "Juarez A.", "Dittmer D.P."], "citationCrossReferences": [ { "database": "PubMed", @@ -47991,21 +44207,14 @@ "lastPage": "4841", "volume": "12" }, - "referencePositions": [ - "INTERACTION WITH ZNF768" - ] + "referencePositions": ["INTERACTION WITH ZNF768"] }, { "referenceNumber": 171, "citation": { "id": "35618207", "citationType": "journal article", - "authors": [ - "Dai Z.", - "Li G.", - "Chen Q.", - "Yang X." - ], + "authors": ["Dai Z.", "Li G.", "Chen Q.", "Yang X."], "citationCrossReferences": [ { "database": "PubMed", @@ -48023,25 +44232,14 @@ "lastPage": "194827", "volume": "1865" }, - "referencePositions": [ - "FUNCTION", - "DOMAIN", - "PHOSPHORYLATION AT SER-392", - "MUTAGENESIS OF SER-392" - ] + "referencePositions": ["FUNCTION", "DOMAIN", "PHOSPHORYLATION AT SER-392", "MUTAGENESIS OF SER-392"] }, { "referenceNumber": 172, "citation": { "id": "36108750", "citationType": "journal article", - "authors": [ - "Chen C.", - "Fu G.", - "Guo Q.", - "Xue S.", - "Luo S.Z." - ], + "authors": ["Chen C.", "Fu G.", "Guo Q.", "Xue S.", "Luo S.Z."], "citationCrossReferences": [ { "database": "PubMed", @@ -48071,13 +44269,7 @@ "citation": { "id": "36634798", "citationType": "journal article", - "authors": [ - "Chen Q.", - "Wu Y.", - "Dai Z.", - "Zhang Z.", - "Yang X." - ], + "authors": ["Chen Q.", "Wu Y.", "Dai Z.", "Zhang Z.", "Yang X."], "citationCrossReferences": [ { "database": "PubMed", @@ -48095,25 +44287,14 @@ "lastPage": "123221", "volume": "230" }, - "referencePositions": [ - "FUNCTION", - "DOMAIN" - ] + "referencePositions": ["FUNCTION", "DOMAIN"] }, { "referenceNumber": 174, "citation": { "id": "38653238", "citationType": "journal article", - "authors": [ - "Zong Z.", - "Xie F.", - "Wang S.", - "Wu X.", - "Zhang Z.", - "Yang B.", - "Zhou F." - ], + "authors": ["Zong Z.", "Xie F.", "Wang S.", "Wu X.", "Zhang Z.", "Yang B.", "Zhou F."], "citationCrossReferences": [ { "database": "PubMed", @@ -48169,23 +44350,14 @@ "lastPage": "391", "volume": "265" }, - "referencePositions": [ - "STRUCTURE BY NMR OF 319-360" - ] + "referencePositions": ["STRUCTURE BY NMR OF 319-360"] }, { "referenceNumber": 176, "citation": { "id": "7773777", "citationType": "journal article", - "authors": [ - "Lee W.", - "Harvey T.S.", - "Yin Y.", - "Yau P.", - "Litchfield D.", - "Arrowsmith C.H." - ], + "authors": ["Lee W.", "Harvey T.S.", "Yin Y.", "Yau P.", "Litchfield D.", "Arrowsmith C.H."], "citationCrossReferences": [ { "database": "PubMed", @@ -48203,9 +44375,7 @@ "lastPage": "890", "volume": "1" }, - "referencePositions": [ - "STRUCTURE BY NMR OF 325-355" - ] + "referencePositions": ["STRUCTURE BY NMR OF 325-355"] }, { "referenceNumber": 177, @@ -48237,21 +44407,14 @@ "lastPage": "6236", "volume": "16" }, - "referencePositions": [ - "STRUCTURE BY NMR OF 326-354" - ] + "referencePositions": ["STRUCTURE BY NMR OF 326-354"] }, { "referenceNumber": 178, "citation": { "id": "8023157", "citationType": "journal article", - "authors": [ - "Cho Y.", - "Gorina S.", - "Jeffrey P.D.", - "Pavletich N.P." - ], + "authors": ["Cho Y.", "Gorina S.", "Jeffrey P.D.", "Pavletich N.P."], "citationCrossReferences": [ { "database": "PubMed", @@ -48269,20 +44432,14 @@ "lastPage": "355", "volume": "265" }, - "referencePositions": [ - "X-RAY CRYSTALLOGRAPHY (2.2 ANGSTROMS) OF 94-289" - ] + "referencePositions": ["X-RAY CRYSTALLOGRAPHY (2.2 ANGSTROMS) OF 94-289"] }, { "referenceNumber": 179, "citation": { "id": "7878469", "citationType": "journal article", - "authors": [ - "Jeffrey P.D.", - "Gorina S.", - "Pavletich N.P." - ], + "authors": ["Jeffrey P.D.", "Gorina S.", "Pavletich N.P."], "citationCrossReferences": [ { "database": "PubMed", @@ -48300,9 +44457,7 @@ "lastPage": "1502", "volume": "267" }, - "referencePositions": [ - "X-RAY CRYSTALLOGRAPHY (1.7 ANGSTROMS) OF 325-356" - ] + "referencePositions": ["X-RAY CRYSTALLOGRAPHY (1.7 ANGSTROMS) OF 325-356"] }, { "referenceNumber": 180, @@ -48335,19 +44490,14 @@ "lastPage": "953", "volume": "274" }, - "referencePositions": [ - "X-RAY CRYSTALLOGRAPHY (2.3 ANGSTROMS) OF 13-29 IN COMPLEX WITH MDM2" - ] + "referencePositions": ["X-RAY CRYSTALLOGRAPHY (2.3 ANGSTROMS) OF 13-29 IN COMPLEX WITH MDM2"] }, { "referenceNumber": 181, "citation": { "id": "8875926", "citationType": "journal article", - "authors": [ - "Gorina S.", - "Pavletich N.P." - ], + "authors": ["Gorina S.", "Pavletich N.P."], "citationCrossReferences": [ { "database": "PubMed", @@ -48365,20 +44515,14 @@ "lastPage": "1005", "volume": "274" }, - "referencePositions": [ - "X-RAY CRYSTALLOGRAPHY (2.2 ANGSTROMS) OF 97-287 IN COMPLEX WITH 53BP2" - ] + "referencePositions": ["X-RAY CRYSTALLOGRAPHY (2.2 ANGSTROMS) OF 97-287 IN COMPLEX WITH 53BP2"] }, { "referenceNumber": 182, "citation": { "id": "14534297", "citationType": "journal article", - "authors": [ - "Joerger A.C.", - "Allen M.D.", - "Fersht A.R." - ], + "authors": ["Joerger A.C.", "Allen M.D.", "Fersht A.R."], "citationCrossReferences": [ { "database": "PubMed", @@ -48396,10 +44540,7 @@ "lastPage": "1296", "volume": "279" }, - "referencePositions": [ - "X-RAY CRYSTALLOGRAPHY (1.9 ANGSTROMS) OF 94-312 IN COMPLEX WITH ZINC IONS", - "SUBUNIT" - ] + "referencePositions": ["X-RAY CRYSTALLOGRAPHY (1.9 ANGSTROMS) OF 94-312 IN COMPLEX WITH ZINC IONS", "SUBUNIT"] }, { "referenceNumber": 183, @@ -48442,15 +44583,7 @@ "citation": { "id": "16474402", "citationType": "journal article", - "authors": [ - "Sheng Y.", - "Saridakis V.", - "Sarkari F.", - "Duan S.", - "Wu T.", - "Arrowsmith C.H.", - "Frappier L." - ], + "authors": ["Sheng Y.", "Saridakis V.", "Sarkari F.", "Duan S.", "Wu T.", "Arrowsmith C.H.", "Frappier L."], "citationCrossReferences": [ { "database": "PubMed", @@ -48478,14 +44611,7 @@ "citation": { "id": "16402859", "citationType": "journal article", - "authors": [ - "Hu M.", - "Gu L.", - "Li M.", - "Jeffrey P.D.", - "Gu W.", - "Shi Y." - ], + "authors": ["Hu M.", "Gu L.", "Li M.", "Jeffrey P.D.", "Gu W.", "Shi Y."], "citationCrossReferences": [ { "database": "PubMed", @@ -48553,9 +44679,7 @@ "citation": { "id": "8266092", "citationType": "journal article", - "authors": [ - "Harris C.C." - ], + "authors": ["Harris C.C."], "citationCrossReferences": [ { "database": "PubMed", @@ -48573,21 +44697,14 @@ "lastPage": "1981", "volume": "262" }, - "referencePositions": [ - "REVIEW" - ] + "referencePositions": ["REVIEW"] }, { "referenceNumber": 188, "citation": { "id": "1905840", "citationType": "journal article", - "authors": [ - "Hoolstein M.", - "Sidransky D.", - "Vogelstein B.", - "Harris C.C." - ], + "authors": ["Hoolstein M.", "Sidransky D.", "Vogelstein B.", "Harris C.C."], "citationCrossReferences": [ { "database": "PubMed", @@ -48605,9 +44722,7 @@ "lastPage": "53", "volume": "253" }, - "referencePositions": [ - "REVIEW ON VARIANTS" - ] + "referencePositions": ["REVIEW ON VARIANTS"] }, { "referenceNumber": 189, @@ -48642,20 +44757,14 @@ "lastPage": "213", "volume": "7" }, - "referencePositions": [ - "REVIEW ON VARIANTS" - ] + "referencePositions": ["REVIEW ON VARIANTS"] }, { "referenceNumber": 190, "citation": { "id": "17015838", "citationType": "journal article", - "authors": [ - "Joerger A.C.", - "Ang H.C.", - "Fersht A.R." - ], + "authors": ["Joerger A.C.", "Ang H.C.", "Fersht A.R."], "citationCrossReferences": [ { "database": "PubMed", @@ -48673,24 +44782,14 @@ "lastPage": "15061", "volume": "103" }, - "referencePositions": [ - "X-RAY CRYSTALLOGRAPHY (1.65 ANGSTROMS) OF 94-312 IN COMPLEX WITH ZINC IONS" - ] + "referencePositions": ["X-RAY CRYSTALLOGRAPHY (1.65 ANGSTROMS) OF 94-312 IN COMPLEX WITH ZINC IONS"] }, { "referenceNumber": 191, "citation": { "id": "18453682", "citationType": "journal article", - "authors": [ - "Tu C.", - "Tan Y.H.", - "Shaw G.", - "Zhou Z.", - "Bai Y.", - "Luo R.", - "Ji X." - ], + "authors": ["Tu C.", "Tan Y.H.", "Shaw G.", "Zhou Z.", "Bai Y.", "Luo R.", "Ji X."], "citationCrossReferences": [ { "database": "PubMed", @@ -48708,23 +44807,14 @@ "lastPage": "477", "volume": "64" }, - "referencePositions": [ - "X-RAY CRYSTALLOGRAPHY (1.54 ANGSTROMS) OF 94-292 OF VARIANT GLN-282" - ] + "referencePositions": ["X-RAY CRYSTALLOGRAPHY (1.54 ANGSTROMS) OF 94-292 OF VARIANT GLN-282"] }, { "referenceNumber": 192, "citation": { "id": "18650397", "citationType": "journal article", - "authors": [ - "Boeckler F.M.", - "Joerger A.C.", - "Jaggi G.", - "Rutherford T.J.", - "Veprintsev D.B.", - "Fersht A.R." - ], + "authors": ["Boeckler F.M.", "Joerger A.C.", "Jaggi G.", "Rutherford T.J.", "Veprintsev D.B.", "Fersht A.R."], "citationCrossReferences": [ { "database": "PubMed", @@ -48780,21 +44870,14 @@ "lastPage": "265", "volume": "385" }, - "referencePositions": [ - "X-RAY CRYSTALLOGRAPHY (1.2 ANGSTROMS) OF 94-293 OF VARIANT SER-249 IN COMPLEX WITH DNA" - ] + "referencePositions": ["X-RAY CRYSTALLOGRAPHY (1.2 ANGSTROMS) OF 94-293 OF VARIANT SER-249 IN COMPLEX WITH DNA"] }, { "referenceNumber": 194, "citation": { "id": "19515728", "citationType": "journal article", - "authors": [ - "Khoo K.H.", - "Joerger A.C.", - "Freund S.M.", - "Fersht A.R." - ], + "authors": ["Khoo K.H.", "Joerger A.C.", "Freund S.M.", "Fersht A.R."], "citationCrossReferences": [ { "database": "PubMed", @@ -48812,23 +44895,14 @@ "lastPage": "430", "volume": "22" }, - "referencePositions": [ - "X-RAY CRYSTALLOGRAPHY (1.75 ANGSTROMS) OF 94-310 IN COMPLEX WITH ZINC IONS" - ] + "referencePositions": ["X-RAY CRYSTALLOGRAPHY (1.75 ANGSTROMS) OF 94-310 IN COMPLEX WITH ZINC IONS"] }, { "referenceNumber": 195, "citation": { "id": "20142040", "citationType": "journal article", - "authors": [ - "Basse N.", - "Kaar J.L.", - "Settanni G.", - "Joerger A.C.", - "Rutherford T.J.", - "Fersht A.R." - ], + "authors": ["Basse N.", "Kaar J.L.", "Settanni G.", "Joerger A.C.", "Rutherford T.J.", "Fersht A.R."], "citationCrossReferences": [ { "database": "PubMed", @@ -48855,15 +44929,7 @@ "citation": { "id": "20364130", "citationType": "journal article", - "authors": [ - "Kitayner M.", - "Rozenberg H.", - "Rohs R.", - "Suad O.", - "Rabinovich D.", - "Honig B.", - "Shakked Z." - ], + "authors": ["Kitayner M.", "Rozenberg H.", "Rohs R.", "Suad O.", "Rabinovich D.", "Honig B.", "Shakked Z."], "citationCrossReferences": [ { "database": "PubMed", @@ -48891,14 +44957,7 @@ "citation": { "id": "32442400", "citationType": "journal article", - "authors": [ - "Ecsedi P.", - "Gogl G.", - "Hof H.", - "Kiss B.", - "Harmat V.", - "Nyitray L." - ], + "authors": ["Ecsedi P.", "Gogl G.", "Hof H.", "Kiss B.", "Harmat V.", "Nyitray L."], "citationCrossReferences": [ { "database": "PubMed", @@ -48916,10 +44975,7 @@ "lastPage": "953.e4", "volume": "28" }, - "referencePositions": [ - "X-RAY CRYSTALLOGRAPHY (3.10 ANGSTROMS) OF 17-56", - "INTERACTION WITH S100A4" - ], + "referencePositions": ["X-RAY CRYSTALLOGRAPHY (3.10 ANGSTROMS) OF 17-56", "INTERACTION WITH S100A4"], "evidences": [ { "evidenceCode": "ECO:0007744", @@ -48933,13 +44989,7 @@ "citation": { "id": "1999338", "citationType": "journal article", - "authors": [ - "Olschwang S.", - "Laurent-Puig P.", - "Vassal A.", - "Salmon R.-J.", - "Thomas G." - ], + "authors": ["Olschwang S.", "Laurent-Puig P.", "Vassal A.", "Salmon R.-J.", "Thomas G."], "citationCrossReferences": [ { "database": "PubMed", @@ -48957,21 +45007,14 @@ "lastPage": "370", "volume": "86" }, - "referencePositions": [ - "VARIANT ARG-72" - ] + "referencePositions": ["VARIANT ARG-72"] }, { "referenceNumber": 199, "citation": { "id": "1933902", "citationType": "journal article", - "authors": [ - "Law J.C.", - "Strong L.C.", - "Chidambaram A.", - "Ferrell R.E." - ], + "authors": ["Law J.C.", "Strong L.C.", "Chidambaram A.", "Ferrell R.E."], "citationCrossReferences": [ { "database": "PubMed", @@ -48985,9 +45028,7 @@ "lastPage": "6387", "volume": "51" }, - "referencePositions": [ - "VARIANT LFS THR-133" - ] + "referencePositions": ["VARIANT LFS THR-133"] }, { "referenceNumber": 200, @@ -49024,22 +45065,14 @@ "lastPage": "1238", "volume": "250" }, - "referencePositions": [ - "VARIANTS LFS CYS-245; TRP-248; PRO-252 AND LYS-258" - ] + "referencePositions": ["VARIANTS LFS CYS-245; TRP-248; PRO-252 AND LYS-258"] }, { "referenceNumber": 201, "citation": { "id": "2259385", "citationType": "journal article", - "authors": [ - "Srivastava S.", - "Zou Z.", - "Pirollo K.", - "Blattner W.", - "Chang E.H." - ], + "authors": ["Srivastava S.", "Zou Z.", "Pirollo K.", "Blattner W.", "Chang E.H."], "citationCrossReferences": [ { "database": "PubMed", @@ -49057,9 +45090,7 @@ "lastPage": "749", "volume": "348" }, - "referencePositions": [ - "VARIANT LFS ASP-245" - ] + "referencePositions": ["VARIANT LFS ASP-245"] }, { "referenceNumber": 202, @@ -49097,9 +45128,7 @@ "lastPage": "647", "volume": "89" }, - "referencePositions": [ - "VARIANT LFS LEU-272" - ] + "referencePositions": ["VARIANT LFS LEU-272"] }, { "referenceNumber": 203, @@ -49136,21 +45165,14 @@ "lastPage": "1315", "volume": "326" }, - "referencePositions": [ - "VARIANTS LFS HIS-273 AND VAL-325" - ] + "referencePositions": ["VARIANTS LFS HIS-273 AND VAL-325"] }, { "referenceNumber": 204, "citation": { "id": "1694291", "citationType": "journal article", - "authors": [ - "Bartek J.", - "Iggo R.", - "Gannon J.", - "Lane D.P." - ], + "authors": ["Bartek J.", "Iggo R.", "Gannon J.", "Lane D.P."], "citationCrossReferences": [ { "database": "PubMed", @@ -49164,9 +45186,7 @@ "lastPage": "899", "volume": "5" }, - "referencePositions": [ - "VARIANTS SPORADIC CANCERS GLN-132; SER-249; LYS-280 AND LYS-285" - ] + "referencePositions": ["VARIANTS SPORADIC CANCERS GLN-132; SER-249; LYS-280 AND LYS-285"] }, { "referenceNumber": 205, @@ -49199,22 +45219,14 @@ "lastPage": "7559", "volume": "87" }, - "referencePositions": [ - "VARIANTS SPORADIC CANCERS PHE-241 AND HIS-273" - ] + "referencePositions": ["VARIANTS SPORADIC CANCERS PHE-241 AND HIS-273"] }, { "referenceNumber": 206, "citation": { "id": "2263646", "citationType": "journal article", - "authors": [ - "Hollstein M.C.", - "Metcalf R.A.", - "Welsh J.A.", - "Montesano R.", - "Harris C.C." - ], + "authors": ["Hollstein M.C.", "Metcalf R.A.", "Welsh J.A.", "Montesano R.", "Harris C.C."], "citationCrossReferences": [ { "database": "PubMed", @@ -49232,9 +45244,7 @@ "lastPage": "9961", "volume": "87" }, - "referencePositions": [ - "VARIANTS SPORADIC CANCER VAL-154; VAL-245; GLN-248; LEU-278 AND SER-278" - ] + "referencePositions": ["VARIANTS SPORADIC CANCER VAL-154; VAL-245; GLN-248; LEU-278 AND SER-278"] }, { "referenceNumber": 207, @@ -49268,23 +45278,14 @@ "lastPage": "906", "volume": "177" }, - "referencePositions": [ - "VARIANTS SPORADIC CANCERS" - ] + "referencePositions": ["VARIANTS SPORADIC CANCERS"] }, { "referenceNumber": 208, "citation": { "id": "1868473", "citationType": "journal article", - "authors": [ - "Casson A.G.", - "Mukhopadhyay T.", - "Cleary K.R.", - "Ro J.Y.", - "Levin B.", - "Roth J.A." - ], + "authors": ["Casson A.G.", "Mukhopadhyay T.", "Cleary K.R.", "Ro J.Y.", "Levin B.", "Roth J.A."], "citationCrossReferences": [ { "database": "PubMed", @@ -49298,23 +45299,14 @@ "lastPage": "4499", "volume": "51" }, - "referencePositions": [ - "VARIANTS SPORADIC CANCERS LEU-152; ALA-155; HIS-175; PHE-176 AND HIS-273" - ] + "referencePositions": ["VARIANTS SPORADIC CANCERS LEU-152; ALA-155; HIS-175; PHE-176 AND HIS-273"] }, { "referenceNumber": 209, "citation": { "id": "1849234", "citationType": "journal article", - "authors": [ - "Hsu I.C.", - "Metcalf R.A.", - "Sun T.", - "Welsh J.A.", - "Wang N.J.", - "Harris C.C." - ], + "authors": ["Hsu I.C.", "Metcalf R.A.", "Sun T.", "Welsh J.A.", "Wang N.J.", "Harris C.C."], "citationCrossReferences": [ { "database": "PubMed", @@ -49332,21 +45324,14 @@ "lastPage": "428", "volume": "350" }, - "referencePositions": [ - "VARIANTS SPORADIC CANCERS IN CHINA" - ] + "referencePositions": ["VARIANTS SPORADIC CANCERS IN CHINA"] }, { "referenceNumber": 210, "citation": { "id": "1672732", "citationType": "journal article", - "authors": [ - "Bressac B.", - "Kew M.", - "Wands J.", - "Ozturk M." - ], + "authors": ["Bressac B.", "Kew M.", "Wands J.", "Ozturk M."], "citationCrossReferences": [ { "database": "PubMed", @@ -49364,23 +45349,14 @@ "lastPage": "431", "volume": "350" }, - "referencePositions": [ - "VARIANTS SPORADIC CANCERS IN SOUTH AFRICA" - ] + "referencePositions": ["VARIANTS SPORADIC CANCERS IN SOUTH AFRICA"] }, { "referenceNumber": 211, "citation": { "id": "1394225", "citationType": "journal article", - "authors": [ - "Somers K.D.", - "Merrick M.A.", - "Lopez M.E.", - "Incognito L.S.", - "Schechter G.L.", - "Casey G." - ], + "authors": ["Somers K.D.", "Merrick M.A.", "Lopez M.E.", "Incognito L.S.", "Schechter G.L.", "Casey G."], "citationCrossReferences": [ { "database": "PubMed", @@ -49394,19 +45370,14 @@ "lastPage": "6000", "volume": "52" }, - "referencePositions": [ - "VARIANTS SPORADIC CANCERS PHE-176; PHE-242; CYS-245; LEU-248 AND HIS-273" - ] + "referencePositions": ["VARIANTS SPORADIC CANCERS PHE-176; PHE-242; CYS-245; LEU-248 AND HIS-273"] }, { "referenceNumber": 212, "citation": { "id": "1327751", "citationType": "journal article", - "authors": [ - "Crook T.", - "Vousden K.H." - ], + "authors": ["Crook T.", "Vousden K.H."], "citationCrossReferences": [ { "database": "PubMed", @@ -49424,9 +45395,7 @@ "lastPage": "3940", "volume": "11" }, - "referencePositions": [ - "VARIANTS SPORADIC CANCERS" - ] + "referencePositions": ["VARIANTS SPORADIC CANCERS"] }, { "referenceNumber": 213, @@ -49460,20 +45429,14 @@ "lastPage": "872", "volume": "52" }, - "referencePositions": [ - "VARIANTS SPORADIC CANCERS CYS-205; GLU-281 AND LYS-285" - ] + "referencePositions": ["VARIANTS SPORADIC CANCERS CYS-205; GLU-281 AND LYS-285"] }, { "referenceNumber": 214, "citation": { "id": "1303181", "citationType": "journal article", - "authors": [ - "Bhatia K.", - "Guiterrez M.I.", - "Magrath I.T." - ], + "authors": ["Bhatia K.", "Guiterrez M.I.", "Magrath I.T."], "citationCrossReferences": [ { "database": "PubMed", @@ -49491,9 +45454,7 @@ "lastPage": "208", "volume": "1" }, - "referencePositions": [ - "VARIANT PRO-HIS-PRO-178 INS" - ] + "referencePositions": ["VARIANT PRO-HIS-PRO-178 INS"] }, { "referenceNumber": 215, @@ -49523,9 +45484,7 @@ "lastPage": "2167", "volume": "7" }, - "referencePositions": [ - "VARIANTS SPORADIC CANCERS" - ] + "referencePositions": ["VARIANTS SPORADIC CANCERS"] }, { "referenceNumber": 216, @@ -49560,22 +45519,14 @@ "lastPage": "6520", "volume": "89" }, - "referencePositions": [ - "VARIANT SPORADIC CANCER THR-280" - ] + "referencePositions": ["VARIANT SPORADIC CANCER THR-280"] }, { "referenceNumber": 217, "citation": { "id": "7682763", "citationType": "journal article", - "authors": [ - "Caamano J.", - "Zhang S.Y.", - "Rosvold E.A.", - "Bauer B.", - "Klein-Szanto A.J.P." - ], + "authors": ["Caamano J.", "Zhang S.Y.", "Rosvold E.A.", "Bauer B.", "Klein-Szanto A.J.P."], "citationCrossReferences": [ { "database": "PubMed", @@ -49623,22 +45574,14 @@ "lastPage": "4480", "volume": "53" }, - "referencePositions": [ - "VARIANTS SPORADIC CANCERS" - ] + "referencePositions": ["VARIANTS SPORADIC CANCERS"] }, { "referenceNumber": 219, "citation": { "id": "8336944", "citationType": "journal article", - "authors": [ - "Hamelin R.", - "Jego N.", - "Laurent-Puig P.", - "Vidaud M.", - "Thomas G." - ], + "authors": ["Hamelin R.", "Jego N.", "Laurent-Puig P.", "Vidaud M.", "Thomas G."], "citationCrossReferences": [ { "database": "PubMed", @@ -49652,9 +45595,7 @@ "lastPage": "2220", "volume": "8" }, - "referencePositions": [ - "VARIANTS SPORADIC CANCERS" - ] + "referencePositions": ["VARIANTS SPORADIC CANCERS"] }, { "referenceNumber": 220, @@ -49694,24 +45635,14 @@ "lastPage": "1304", "volume": "54" }, - "referencePositions": [ - "VARIANTS", - "INVOLVEMENT IN LFL" - ] + "referencePositions": ["VARIANTS", "INVOLVEMENT IN LFL"] }, { "referenceNumber": 221, "citation": { "id": "8013454", "citationType": "journal article", - "authors": [ - "Zhang W.", - "Guo X.-Y.", - "Hu G.-Y.", - "Liu W.-B.", - "Shay J.W.", - "Deisseroth A.B." - ], + "authors": ["Zhang W.", "Guo X.-Y.", "Hu G.-Y.", "Liu W.-B.", "Shay J.W.", "Deisseroth A.B."], "citationCrossReferences": [ { "database": "PubMed", @@ -49729,9 +45660,7 @@ "lastPage": "2544", "volume": "13" }, - "referencePositions": [ - "CHARACTERIZATION OF VARIANT ALA-143" - ] + "referencePositions": ["CHARACTERIZATION OF VARIANT ALA-143"] }, { "referenceNumber": 222, @@ -49761,18 +45690,14 @@ "lastPage": "615", "volume": "56" }, - "referencePositions": [ - "VARIANTS LFS HIS-175; ARG-193; GLN-248; CYS-273 AND TYR-275" - ] + "referencePositions": ["VARIANTS LFS HIS-175; ARG-193; GLN-248; CYS-273 AND TYR-275"] }, { "referenceNumber": 223, "citation": { "id": "8718514", "citationType": "journal article", - "authors": [ - "Eeles R.A." - ], + "authors": ["Eeles R.A."], "citationCrossReferences": [ { "database": "PubMed", @@ -49786,10 +45711,7 @@ "lastPage": "124", "volume": "25" }, - "referencePositions": [ - "VARIANTS", - "INVOLVEMENT IN LFL" - ] + "referencePositions": ["VARIANTS", "INVOLVEMENT IN LFL"] }, { "referenceNumber": 224, @@ -49825,9 +45747,7 @@ "lastPage": "945", "volume": "32" }, - "referencePositions": [ - "VARIANT LFS HIS-175" - ] + "referencePositions": ["VARIANT LFS HIS-175"] }, { "referenceNumber": 225, @@ -49863,23 +45783,14 @@ "lastPage": "113", "volume": "7" }, - "referencePositions": [ - "VARIANTS SPORADIC CANCERS PHE-176; SER-245; TRP-248; TRP-282 AND GLN-286" - ] + "referencePositions": ["VARIANTS SPORADIC CANCERS PHE-176; SER-245; TRP-248; TRP-282 AND GLN-286"] }, { "referenceNumber": 226, "citation": { "id": "9101296", "citationType": "journal article", - "authors": [ - "Guldberg P.", - "Nedergaard T.", - "Nielsen H.J.", - "Olsen A.C.", - "Ahrenkiel V.", - "Zeuthen J." - ], + "authors": ["Guldberg P.", "Nedergaard T.", "Nielsen H.J.", "Olsen A.C.", "Ahrenkiel V.", "Zeuthen J."], "citationCrossReferences": [ { "database": "PubMed", @@ -49897,9 +45808,7 @@ "lastPage": "355", "volume": "9" }, - "referencePositions": [ - "VARIANTS SPORADIC CANCERS" - ] + "referencePositions": ["VARIANTS SPORADIC CANCERS"] }, { "referenceNumber": 227, @@ -49941,22 +45850,14 @@ "lastPage": "2881", "volume": "15" }, - "referencePositions": [ - "VARIANT SPORADIC CANCER ILE-157" - ] + "referencePositions": ["VARIANT SPORADIC CANCER ILE-157"] }, { "referenceNumber": 228, "citation": { "id": "9450901", "citationType": "journal article", - "authors": [ - "van Rensburg E.J.", - "Engelbrecht S.", - "van Heerden W.F.P.", - "Kotze M.J.", - "Raubenheimer E.J." - ], + "authors": ["van Rensburg E.J.", "Engelbrecht S.", "van Heerden W.F.P.", "Kotze M.J.", "Raubenheimer E.J."], "citationCrossReferences": [ { "database": "PubMed", @@ -49974,20 +45875,14 @@ "lastPage": "44", "volume": "11" }, - "referencePositions": [ - "VARIANTS SER-152; ILE-169; PHE-176; THR-195; CYS-220; ILE-230; CYS-273 AND SER-278" - ] + "referencePositions": ["VARIANTS SER-152; ILE-169; PHE-176; THR-195; CYS-220; ILE-230; CYS-273 AND SER-278"] }, { "referenceNumber": 229, "citation": { "id": "9452042", "citationType": "journal article", - "authors": [ - "Luca J.W.", - "Strong L.C.", - "Hansen M.F." - ], + "authors": ["Luca J.W.", "Strong L.C.", "Hansen M.F."], "citationCrossReferences": [ { "database": "PubMed", @@ -50005,20 +45900,14 @@ "lastPage": "S61", "volume": "1" }, - "referencePositions": [ - "VARIANT NON-CLASSICAL LFS CYS-337" - ] + "referencePositions": ["VARIANT NON-CLASSICAL LFS CYS-337"] }, { "referenceNumber": 230, "citation": { "id": "10484981", "citationType": "journal article", - "authors": [ - "Gueran S.", - "Tunca Y.", - "Imirzalioglu N." - ], + "authors": ["Gueran S.", "Tunca Y.", "Imirzalioglu N."], "citationCrossReferences": [ { "database": "PubMed", @@ -50036,19 +45925,14 @@ "lastPage": "151", "volume": "113" }, - "referencePositions": [ - "VARIANT LFS ILE-292" - ] + "referencePositions": ["VARIANT LFS ILE-292"] }, { "referenceNumber": 231, "citation": { "id": "10549356", "citationType": "journal article", - "authors": [ - "Hainaut P.", - "Hollstein M." - ], + "authors": ["Hainaut P.", "Hollstein M."], "citationCrossReferences": [ { "database": "PubMed", @@ -50066,9 +45950,7 @@ "lastPage": "137", "volume": "77" }, - "referencePositions": [ - "VARIANTS" - ] + "referencePositions": ["VARIANTS"] }, { "referenceNumber": 232, @@ -50107,9 +45989,7 @@ "lastPage": "9335", "volume": "98" }, - "referencePositions": [ - "VARIANT ADCC HIS-337" - ] + "referencePositions": ["VARIANT ADCC HIS-337"] }, { "referenceNumber": 233, @@ -50144,9 +46024,7 @@ "lastPage": "1596", "volume": "86" }, - "referencePositions": [ - "INVOLVEMENT IN CPP" - ] + "referencePositions": ["INVOLVEMENT IN CPP"] }, { "referenceNumber": 234, @@ -50276,9 +46154,7 @@ "lastPage": "629", "volume": "28" }, - "referencePositions": [ - "VARIANTS" - ] + "referencePositions": ["VARIANTS"] }, { "referenceNumber": 237, @@ -50357,9 +46233,7 @@ "lastPage": "e1002555", "volume": "14" }, - "referencePositions": [ - "VARIANTS VAL-138; HIS-175; ILE-237; TRP-248 AND PRO-273" - ] + "referencePositions": ["VARIANTS VAL-138; HIS-175; ILE-237; TRP-248 AND PRO-273"] } ], "uniProtKBCrossReferences": [ @@ -65891,4 +61765,4 @@ "uniParcId": "UPI000002ED67" } } -} \ No newline at end of file +} diff --git a/backend/cli/test/science/fixtures/fetch/wikipathways.json b/backend/cli/test/science/fixtures/fetch/wikipathways.json index ff539a1c..6787023a 100644 --- a/backend/cli/test/science/fixtures/fetch/wikipathways.json +++ b/backend/cli/test/science/fixtures/fetch/wikipathways.json @@ -12,4 +12,4 @@ "annotations": "signaling pathway, angiotensin signaling pathway, hypertension, ACE inhibitor drug pathway, mesangial cell", "citedIn": "" } -} \ No newline at end of file +} diff --git a/backend/cli/test/server/notebook.test.ts b/backend/cli/test/server/notebook.test.ts index 41f78b68..ae8eaff5 100644 --- a/backend/cli/test/server/notebook.test.ts +++ b/backend/cli/test/server/notebook.test.ts @@ -385,7 +385,7 @@ describe("/notebook routes", () => { requested: expect.any(Boolean), enforced: expect.any(Boolean), backend: expect.any(String), - network: expect.stringMatching(/^(allow|deny)$/), + network: expect.stringMatching(/^(allow|allowlist|deny)$/), platform: process.platform, }, }) @@ -572,7 +572,10 @@ describe("/notebook routes", () => { const waitForStarting = async (attempt = 0): Promise => { const result = (await (await status()).json()) as { state?: string } if (result.state === "starting") return - if (attempt >= 100) throw new Error("kernel did not start") + // 500 * 10ms = 5s: covers the loopback shim's own up-to-3s readiness + // wait under sandbox network "allowlist" (the default), plus normal + // interpreter startup, with margin. See sandbox.ts's shimScript. + if (attempt >= 500) throw new Error("kernel did not start") await Bun.sleep(10) return waitForStarting(attempt + 1) } @@ -971,7 +974,10 @@ describe("/notebook routes", () => { const waitForStarting = async (attempt = 0): Promise => { const result = (await (await status()).json()) as { state?: string } if (result.state === "starting") return - if (attempt >= 100) throw new Error("kernel did not start") + // 500 * 10ms = 5s: covers the loopback shim's own up-to-3s readiness + // wait under sandbox network "allowlist" (the default), plus normal + // interpreter startup, with margin. See sandbox.ts's shimScript. + if (attempt >= 500) throw new Error("kernel did not start") await Bun.sleep(10) return waitForStarting(attempt + 1) } @@ -1038,7 +1044,8 @@ describe("/notebook routes", () => { const waitForRunning = async (attempt = 0): Promise => { const response = (await (await status()).json()) as { state?: string } if (response.state === "running") return - if (attempt >= 100) throw new Error("kernel did not start running") + // See the "kernel did not start" budget above for why 500 * 10ms. + if (attempt >= 500) throw new Error("kernel did not start running") await Bun.sleep(10) return waitForRunning(attempt + 1) } @@ -1109,7 +1116,8 @@ describe("/notebook routes", () => { await app.request(`/kernels?sessionID=${encodeURIComponent(session.id)}`) ).json()) as typeof kernels if (current.kernels.find((value) => value.id === kernel.id)?.state === "running") return - if (attempt >= 100) throw new Error("kernel did not start running") + // See the "kernel did not start" budget above for why 500 * 10ms. + if (attempt >= 500) throw new Error("kernel did not start running") await Bun.sleep(10) return waitForRunning(attempt + 1) } diff --git a/backend/cli/test/shell/invocation.test.ts b/backend/cli/test/shell/invocation.test.ts new file mode 100644 index 00000000..ddbace2a --- /dev/null +++ b/backend/cli/test/shell/invocation.test.ts @@ -0,0 +1,84 @@ +import { expect, test } from "bun:test" +import { Shell } from "../../src/shell/shell" + +/** + * How a shell is handed one command. + * + * Measured on a real Windows 11 machine: `Sandbox.plan` composed + * `[shell, "-c", command]`, the shell resolved to `cmd.exe`, and cmd does not + * error on `-c` — it starts an INTERACTIVE shell. Every sandboxed command + * printed the cmd banner and a prompt, ran nothing, and exited 0. The sandbox + * self-test then read that banner as a process token and reported a containment + * failure that had not happened, which cost two debugging cycles. + * + * These run on Linux CI, which is the point: the machine that exposes this is + * not one the suite can run on, so `invocation` must not branch on + * `process.platform` to reach the Windows answer. + */ + +test.each([ + ["cmd.exe", ["/d", "/s", "/c", "echo hi"]], + ["C:\\Windows\\system32\\cmd.exe", ["/d", "/s", "/c", "echo hi"]], + // COMSPEC is what Shell.fallback() returns on Windows, and its casing varies. + ["C:\\WINDOWS\\SYSTEM32\\CMD.EXE", ["/d", "/s", "/c", "echo hi"]], + ["powershell.exe", ["-NoProfile", "-Command", "echo hi"]], + ["C:\\Program Files\\PowerShell\\7\\pwsh.exe", ["-NoProfile", "-Command", "echo hi"]], + // Git Bash is preferred over cmd by Shell.fallback(), and is POSIX. + ["C:\\Program Files\\Git\\bin\\bash.exe", ["-c", "echo hi"]], + ["/bin/sh", ["-c", "echo hi"]], + ["/bin/zsh", ["-c", "echo hi"]], + ["/usr/bin/fish", ["-c", "echo hi"]], +])("%s is invoked correctly", (shell, expected) => { + expect(Shell.invocation(shell, "echo hi")).toEqual(expected) +}) + +test("cmd never receives -c, whatever path it arrives by", () => { + // The specific regression. -c is not rejected by cmd; it is ignored, which is + // why this failed silently rather than loudly. + for (const shell of ["cmd", "cmd.exe", "C:\\Windows\\System32\\cmd.exe"]) + expect(Shell.invocation(shell, "whoami /groups")).not.toContain("-c") +}) + +test("the command is passed through untouched", () => { + // No quoting or escaping here: the argv is handed to spawn as separate + // arguments, so a shell-quoting pass would corrupt it. + const command = `printf 'a b' > "/tmp/x y" && echo "done"` + expect(Shell.invocation("/bin/sh", command)).toEqual(["-c", command]) + expect(Shell.invocation("cmd.exe", command)).toEqual(["/d", "/s", "/c", command]) +}) + +test.each([ + ["cmd.exe", "cmd"], + ["C:\\WINDOWS\\system32\\CMD.EXE", "cmd"], + ["powershell.exe", "powershell"], + ["C:\\Program Files\\PowerShell\\7\\pwsh.exe", "powershell"], + ["C:\\Program Files\\Git\\bin\\bash.exe", "posix"], + ["/bin/sh", "posix"], +] as Array<[string, ReturnType]>)("%s speaks %s", (shell, expected) => { + expect(Shell.family(shell)).toBe(expected) +}) + +test("family() exists because the flag alone is not enough", async () => { + // cmd.exe has no printf and no cat. The sandbox self-test probed with + // `printf hi > f && cat f`, which on Windows failed because neither command + // exists — and was reported as the sandbox being unable to write inside its + // own workspace, one layer after the /c fix had made commands run at all. + const source = await Bun.file(new URL("../../src/sandbox/sandbox.ts", import.meta.url).pathname).text() + const body = source.slice(source.indexOf("export async function selfTest")) + expect(body).toContain("Shell.family(shell)") + expect(body).toContain("type ") + // The POSIX spelling must still be there for Linux and macOS. + expect(body).toContain("printf ") +}) + +test("the self-test's own diagnostics never masquerade as the child's error", async () => { + // The launcher's debug dump shares stderr with the child, so firstLine() was + // returning the first line of the dump for every failure. + const source = await Bun.file(new URL("../../src/sandbox/sandbox.ts", import.meta.url).pathname).text() + const body = source.slice(source.indexOf("function firstLine"), source.indexOf("function runAsync")) + // Was a single prefix check. That proved insufficient: the structured logger + // writes INFO lines to the same stderr and one of them was reported as the + // reason a sandboxed curl failed. The rule is "anything we wrote", not "the + // one prefix noticed first". + expect(body).toContain("!ours(value)") +}) diff --git a/backend/cli/test/shell/shell.test.ts b/backend/cli/test/shell/shell.test.ts index 427ef8f8..ddf79de6 100644 --- a/backend/cli/test/shell/shell.test.ts +++ b/backend/cli/test/shell/shell.test.ts @@ -88,8 +88,17 @@ test("killTree SIGKILLs a detached group even after its leader exits", async () exited: () => true, }), ) - expect(groupKill).toHaveBeenNthCalledWith(1, -4321, "SIGTERM") - expect(groupKill).toHaveBeenNthCalledWith(2, -4321, "SIGKILL") + // Filter to this test's own pid rather than asserting on the mock's + // absolute call order: process.kill is one process-wide global, and under + // a full suite run a real sandboxed spawn elsewhere can land its own + // (real, unrelated) killTree cleanup on this same mock while it's active, + // interleaving with these two calls by index without changing what this + // test is actually verifying — that ITS SIGTERM precedes ITS SIGKILL. + const own = groupKill.mock.calls.filter(([pid]) => pid === -4321) + expect(own).toEqual([ + [-4321, "SIGTERM"], + [-4321, "SIGKILL"], + ]) expect(proc.kill).not.toHaveBeenCalled() } finally { groupKill.mockRestore() diff --git a/backend/cli/test/tool/bash-refusal.test.ts b/backend/cli/test/tool/bash-refusal.test.ts new file mode 100644 index 00000000..9e61ddc5 --- /dev/null +++ b/backend/cli/test/tool/bash-refusal.test.ts @@ -0,0 +1,180 @@ +import { expect, test } from "bun:test" +import path from "path" +import { Instance } from "../../src/project/instance" +import { Refuse } from "../../src/package/refuse" +import { BashTool } from "../../src/tool/bash" +import type { PermissionNext } from "../../src/permission/next" +import { executionSession, tmpdir } from "../fixture/fixture" + +async function context() { + const session = await executionSession() + return { + sessionID: session.id, + messageID: "", + callID: "", + agent: "research", + abort: AbortSignal.any([]), + messages: [], + metadata: () => {}, + ask: async () => {}, + } +} + +/** + * The tokenisation `bash.ts` performs, reproduced against the same parser it + * uses, so this breaks if that tokenisation changes shape. No mock: it is the + * real tree-sitter grammar, because the whole question this file answers is + * whether a real shell line reaches the refusal — a hand-built string array + * would assert nothing about that. + */ +async function commands(line: string) { + const { Parser, Language } = await import("web-tree-sitter") + const { default: treeWasm } = await import("web-tree-sitter/tree-sitter.wasm" as string, { + with: { type: "file" }, + }) + await Parser.init({ locateFile: () => treeWasm }) + const { default: bashWasm } = await import("tree-sitter-bash/tree-sitter-bash.wasm" as string, { + with: { type: "file" }, + }) + const parser = new Parser() + parser.setLanguage(await Language.load(bashWasm)) + const tree = parser.parse(line)! + const out: string[][] = [] + for (const node of tree.rootNode.descendantsOfType("command")) { + if (!node) continue + const command: string[] = [] + for (let i = 0; i < node.childCount; i++) { + const child = node.child(i) + if (!child) continue + if (!["command_name", "word", "string", "raw_string", "concatenation"].includes(child.type)) continue + command.push(child.text) + } + out.push(command) + } + return out +} + +test("a real parse of the venv bypass reaches the refusal", async () => { + // The exact line measured to succeed on feat/sandbox-network-policy, with no + // tool and no approval card. + const line = "python3 -m venv /w/venv && /w/venv/bin/pip install tqdm" + const parsed = await commands(line) + const refusals = parsed.map((c) => Refuse.installer(c)).filter(Boolean) + expect(refusals).toHaveLength(1) + expect(refusals[0]).toContain("package_install") +}) + +test("a compound command is refused on its installer clause, not its first clause", async () => { + const parsed = await commands("cd /w && pip install numpy") + expect(parsed.some((c) => Refuse.installer(c))).toBe(true) +}) + +test("ordinary shell work parses to no refusal", async () => { + const parsed = await commands("python analysis.py && pip list") + expect(parsed.every((c) => !Refuse.installer(c))).toBe(true) +}) + +test("an installer named only inside a quoted argument is not refused", async () => { + // `echo` is the command; the rest are its operands. A regex over the raw + // line would refuse this and be wrong. + const parsed = await commands(`echo "pip install numpy"`) + expect(parsed.every((c) => !Refuse.installer(c))).toBe(true) +}) + +// The tests above prove the matcher and the tokenisation. These prove the tool +// actually calls it — without them the refusal could be dead code and every +// other test in this file would still be green. + +test("the real bash tool refuses an install and never runs it", async () => { + await using tmp = await tmpdir({ git: true }) + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const bash = await BashTool.init() + const failure = await bash + .execute({ command: "pip install tqdm", description: "Install tqdm" }, await context()) + .then( + () => undefined, + (error: Error) => error, + ) + expect(failure?.message).toContain("package_install") + }, + }) +}) + +test("the refusal happens before the permission ask, not after", async () => { + // Otherwise the user is asked to approve a command that is then refused + // anyway — a prompt whose only possible outcome is an error. + await using tmp = await tmpdir({ git: true }) + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const bash = await BashTool.init() + const requests: Array> = [] + const ctx = { + ...(await context()), + ask: async (req: Omit) => { + requests.push(req) + }, + } + await bash.execute({ command: "pip install tqdm", description: "Install tqdm" }, ctx).catch(() => {}) + expect(requests).toHaveLength(0) + }, + }) +}) + +test("the real bash tool still runs an ordinary command", async () => { + // The refusal must not have become a blanket denial. + await using tmp = await tmpdir({ git: true }) + await Instance.provide({ + directory: tmp.path, + fn: async () => { + const bash = await BashTool.init() + const result = await bash.execute({ command: "echo ok", description: "Echo" }, await context()) + expect(result.metadata.output).toContain("ok") + }, + }) +}) + +test("sandbox status never claims confinement on a machine with no backend", async () => { + // Observed on Windows: "status enabled (agent shell commands are confined to + // the workspace)" printed on a machine where Sandbox.backend() is "none" and + // nothing confines anything. A false statement about a security property is + // the worst thing this command can print, so the sentence now keys off + // whether a backend EXISTS, not merely off the config being on. + const source = await Bun.file(new URL("../../src/cli/cmd/sandbox.ts", import.meta.url).pathname).text() + expect(source.includes("are NOT confined here: no backend on this platform")).toBe(true) + // Three states, not two — the ternary must consider availability. + expect(source.includes("d.available")).toBe(true) +}) + +test("nothing printed on a backend-less machine carries non-ASCII", async () => { + // A Windows console decodes our UTF-8 as its OEM code page. An em dash in the + // "unavailable" line arrived as mojibake in a real run. Everything reachable + // WITHOUT a backend — which is exactly the Windows path — stays ASCII. + const source = await Bun.file(new URL("../../src/cli/cmd/sandbox.ts", import.meta.url).pathname).text() + const printed = source + .split("\n") + .filter((l) => l.includes("UI.println") || l.includes("TEXT_WARNING") || l.includes("TEXT_DANGER")) + .filter((l) => !l.includes("c.skipped") && !l.trimStart().startsWith("//")) + .join("\n") + // eslint-disable-next-line no-control-regex + expect(printed).not.toMatch(/[^\x00-\x7F]/) +}) + +test("sandbox status does not claim containment it has not verified", async () => { + // It printed "are confined to the workspace" on a Windows run whose very next + // command, `sandbox test`, failed containment. A backend being AVAILABLE is not + // the same as it working, and status runs nothing that could tell the + // difference, so it must report only what it knows. + const source = await Bun.file(new URL("../../src/cli/cmd/sandbox.ts", import.meta.url).pathname).text() + // Comments are not code: this asserts the flag is not USED, and the comment + // explaining why it was removed must not trip it. + const code = source + .split("\n") + .filter((l) => !/^\s*(\/\/|\*|\/\*)/.test(l)) + .join("\n") + expect(code).not.toContain('"are confined to the workspace"') + expect(code).toContain("are launched through") + expect(code).toContain("sandbox test") +}) diff --git a/backend/cli/test/tool/command-runtime.test.ts b/backend/cli/test/tool/command-runtime.test.ts index eb8f5fa1..4b0895c3 100644 --- a/backend/cli/test/tool/command-runtime.test.ts +++ b/backend/cli/test/tool/command-runtime.test.ts @@ -35,7 +35,10 @@ test("bash registers only its live process in the project compute ledger", async const find = async (attempt = 0): Promise[number]> => { const command = CommandRuntime.list(Instance.project.id, session.id)[0] if (command) return command - if (attempt >= 100) throw new Error("Live command did not enter the compute ledger") + // 500 * 10ms = 5s: the real command doesn't start until the loopback + // shim signals ready (up to ~3s under sandbox network "allowlist", + // the default) or its wait caps out. See sandbox.ts's shimScript. + if (attempt >= 500) throw new Error("Live command did not enter the compute ledger") await Bun.sleep(10) return find(attempt + 1) } diff --git a/backend/cli/test/tool/find-python.test.ts b/backend/cli/test/tool/find-python.test.ts new file mode 100644 index 00000000..d9522f74 --- /dev/null +++ b/backend/cli/test/tool/find-python.test.ts @@ -0,0 +1,57 @@ +import { expect, test } from "bun:test" +import fs from "fs/promises" +import path from "path" +import os from "os" +import { findPython } from "../../src/tool/notebook" +import { Installer } from "../../src/package/installer" + +/** + * A managed environment's interpreter has to RUN, not merely exist. + * + * On Windows a venv's `Scripts\python.exe` is a redirector that resolves its + * base interpreter from `pyvenv.cfg` at startup. When that resolution fails the + * file is still there, so an existence check hands the kernel a binary that + * cannot start. Measured on a real machine as the redirector's own message — + * `No Python at '...'`, a string that appears nowhere in this repo — arriving + * inside a kernel-startup failure with nothing tying it to the environment that + * produced it. + */ + +async function scratch() { + const dir = await fs.mkdtemp(path.join(os.tmpdir(), "openscience-findpy-")) + await fs.mkdir(path.dirname(Installer.interpreter(dir)), { recursive: true }) + return dir +} + +test("an environment interpreter that cannot run is not returned", async () => { + const dir = await scratch() + try { + // Present, non-empty, and not executable — the shape of a broken redirector. + await fs.writeFile(Installer.interpreter(dir), "not an interpreter\n") + const found = (await findPython(undefined, dir)).binary + expect(found).not.toBe(Installer.interpreter(dir)) + // It falls through to a host interpreter rather than failing outright. + expect(["python3", "python"]).toContain(path.basename(found)) + } finally { + await fs.rm(dir, { recursive: true, force: true }) + } +}) + +test("an environment interpreter that runs is preferred over the host", async () => { + // The positive case matters just as much: the check must not have become a + // blanket rejection of managed environments, which would silently un-manage + // every install. + const dir = await scratch() + try { + const real = await Installer.select() + if (!real.binary) return + const target = Installer.interpreter(dir) + await fs.symlink(real.binary, target).catch(async () => { + await fs.copyFile(real.binary!, target) + await fs.chmod(target, 0o755) + }) + expect((await findPython(undefined, dir)).binary).toBe(target) + } finally { + await fs.rm(dir, { recursive: true, force: true }) + } +}, 60_000) diff --git a/docs/adr/0002-sandbox-network-policy.md b/docs/adr/0002-sandbox-network-policy.md new file mode 100644 index 00000000..0428c90f --- /dev/null +++ b/docs/adr/0002-sandbox-network-policy.md @@ -0,0 +1,73 @@ +# ADR 0002: Sandbox network policy becomes three-state, allowlist by default + +Status: accepted + +## Context + +`sandbox.network` is `"allow" | "deny"`. Deny locks kernels out of PyPI, NCBI, UniProt, PDB and +EBI, which is most of what a research tool is for. Allow is unrestricted egress. Neither state is +what the product needs: a kernel that cannot reach a package index or a sequence database cannot +do the work it exists to do, and a kernel with unrestricted egress can send agent-controlled data +anywhere on the internet. Because the installer needed the egress the kernel was denied, earlier +design work gave it a second, network-enabled sandbox purely so it could have that egress the +kernel could not. + +## Decision + +`sandbox.network` becomes three-state: `"deny" | "allowlist" | "allow"`, defaulting to +`"allowlist"`. + +Enforcement is `--unshare-net` plus a bind-mounted unix socket, established by two measurements +taken on the spike branch `proto/sandbox-allowlist-proxy` before the proxy itself was written: +inside `bwrap --unshare-net`, TCP to any host — including the host's own loopback — returns `000`, +and a unix socket bind-mounted into that same network namespace still crosses it. The socket is +therefore the only route out of the namespace. The proxy on the host end of that socket resolves +names itself, which is why the sandboxed process has no DNS of its own. + +One policy covers kernel and installer. There is no separate network-enabled install sandbox: +under `"allowlist"` the installer reaches the same allowlisted hosts through the same proxy the +kernel uses, so the asymmetry that motivated a second sandbox no longer exists. + +Proxy policy is not part of the `ExecutionAuthority.generation` hash. `generation` hashes trust, +filesystem grants, and sandbox policy, and changing it tears down and reboots every live kernel +bound to it. Editing the allowlist is not that kind of change: it takes effect on the next +connection through the running proxy, without tearing down live kernels. + +This ADR decides the default allowlist ships in code; per-project additions live in config. + +## Consequences + +This is a breaking change to a documented config key. Existing `"deny"` and `"allow"` values keep +working unchanged; only the default moves, from `"deny"` to `"allowlist"`. + +`HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` join `SAFE_ENV_PREFIXES` so they reach a kernel +process, and `Sandbox.wrapArgv` must compose a shim into the sandboxed argv, because pip, +requests and curl take an `http://host:port` proxy from those variables and none of them speak +unix sockets directly. Some process also has to start and stop the proxy across the CLI's own +lifecycle; this ADR does not fix that shape here, only that it is needed. + +The boundary is host-level, not content-level. The proxy pipes bytes after checking the +authority; it cannot see inside TLS, so an allowlisted host can still be sent anything a client +sends it. Allowlisting bounds where a kernel can talk, not what it says once it is talking. + +Unresolved: seatbelt has no namespace, so the _mechanism_ above — sever the network device, cross +back in only through a bind-mounted socket — does not transfer as written. That is not the same as +saying macOS cannot reach the same bounded-egress outcome: seatbelt can restrict +`network-outbound` to a specific local port via `(allow network-outbound (remote tcp +"localhost:PORT"))`, which is exactly the shape `anthropic-experimental/sandbox-runtime` ships (a +default network deny, then a selective allow for `network-bind`/`network-inbound`/ +`network-outbound` on the proxy's loopback port). Neither OS can filter by hostname at the +sandbox-profile level — that is what the proxy is for on Linux too — so this is achievable via a +different mechanism, not impossible. Task 7 built it: the host-side proxy listens on a loopback +TCP port directly on macOS (no bind-mounted socket, no bridge — seatbelt has no namespace to put +either behind), the profile permits `network-bind`/`network-inbound`/`network-outbound` on +exactly that port as described above, and — because a loopback port, unlike a unix socket, carries +no filesystem permissions of its own — every request to it must additionally carry a +`Proxy-Authorization` secret generated fresh per proxy start. This is unverified in the same sense +the rest of this ADR's Linux side was before it was measured: nobody on this project has run +`sandbox-exec`, so whether the profile text above is actually _accepted and enforced_ as written — +including whether `network-bind`/`network-inbound` are the right operations to permit at all, and +whether `(local ...)` is the right filter for them — is a real, open question, not merely +theoretical caution. See `.superpowers/sdd/2026-08-09-sandbox-network-policy/task-7-report.md` for +exactly what a Mac owner still needs to run to close it. Windows has no sandbox backend at all, so +the question does not apply there. diff --git a/docs/specs/kernel-execution-design.md b/docs/specs/kernel-execution-design.md new file mode 100644 index 00000000..45fbd417 --- /dev/null +++ b/docs/specs/kernel-execution-design.md @@ -0,0 +1,883 @@ +# Kernel execution and environments — design + +Status: draft, ready for review +Date: 2026-08-08 +Branch: `proto/kernel-package-install` + +## Problem + +A kernel cannot install a package. Three independent causes, each verified on an Arch box during +design, none of which the user can distinguish from the others: + +``` +$ which pip3 → not found +$ python3 -m pip --version → No module named pip +$ python3 -c "import site; ..." → /usr/lib/python3.14/site-packages writable=False +$ bwrap ... --unshare-net → sandbox denies the agent shell all network +``` + +The host interpreter may ship without pip (Arch does), the sandbox denies network, and +site-packages is read-only under `--ro-bind / /`. `findPython` (`tool/notebook.ts:201`) probes only +for a working `--version`, so it happily boots an interpreter that cannot install anything, and the +failure surfaces as an opaque error that reads like a broken machine. + +Two adjacent gaps found while investigating: + +- **GPU is unreachable.** `--dev /dev` mounts a fresh minimal devtmpfs, so no `/dev/nvidia*` reaches + the kernel. Verified: `nvidia-smi -L` inside the kernel's exact sandbox reports it cannot talk to + the driver. Meanwhile `KernelStatus.resources` declares `gpu_percent` and `vram_bytes` + (`science/kernel/registry.ts:99-100`) and `KernelCard.tsx:98-99` renders both — with **no sampler + anywhere**, so every card shows "Unavailable" permanently. +- **`package_install` already exists as a capability** in `project/trust.ts:25` and + `project/execution.ts:26`, with zero call sites. The slot was reserved and never filled. + +## Governing principle + +**Nothing is gated more strictly than arbitrary code execution unless it costs money.** + +`tool/notebook.ts:590` runs arbitrary agent-authored Python in a persistent kernel and asks for +`permission: "bash"` with `always: ["python*"]` — a standing grant covering all future execution. +`bash.ts` is the same shape. `tool/modal.ts` is stricter (exact-plan digest, `always: []`, plus a +`spendFilter` strip at `permission/next.ts:165-171`) for exactly one reason, stated in the comment +there: **paid actions**. + +An earlier draft of this design copied modal's contract. That was wrong — it would have made +installing a library stricter than running arbitrary code. Package installation costs nothing, so it +uses the ordinary contract. + +## Current architecture + +Five layers, unchanged by this spec except where noted. + +``` +NotebookTool / RKernelTool agent-facing tool, permission gate + ↓ +KernelRuntime (registry.ts) identity, persistence, provenance, authority + ↓ +KernelManager (per language) process pool, idle reaping + ↓ +Kernel (PythonKernel/RKernel) one child process, queue, wire protocol + ↓ +Sandbox.wrapArgv bwrap / seatbelt confinement +``` + +Facts this design relies on: + +- `KernelIdentity` is `{projectID, sessionID, name, language}`, hashed into a storage key at + `registry.ts:135`. Arbitrary names are already supported end to end — `POST /kernels` takes one, + provenance strips a `notebook:` prefix, the frontend strips it for display. +- `ExecutionAuthority.generation` (`project/execution.ts:95`) hashes trust, filesystem grants, and + sandbox policy. A change tears down and reboots live kernels at `registry.ts:337`. It takes + `{projectID?, sessionID, capability}` — **no kernel identity**, which constrains where env state + can live (see below). +- `KernelProcessIdentity` (`science/kernel/process.ts`) captures pid + a platform start token and + verifies both, guarding against pid reuse. + +## Decisions + +### Permission contract + +| | Decision | +| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Execution | Unchanged. `permission: "bash"`, `always: ["python*"]`. | +| Install | `permission: "package_install"` — the capability already declared in `trust.ts` and `execution.ts`. | +| Pattern | A canonical command string: `install numpy pandas → default [pypi.org/simple]`. This is both what the card shows and what the permission system matches. | +| `always` | `["install*"]` — a standing grant offered on the card from the start. Mirrors `notebook.ts:590`, which shows the specific `"python (notebook)"` and stores the broad `"python*"`. | +| Card fires | **On every install.** Allow-once approves that call and the next identical request prompts again; taking the broader grant runs subsequent installs without prompting. Friction is solved by the standing grant, not by exempting classes of install — no threshold to tune, and nothing enters an environment the user never saw once. | +| No digest | The command string does the job. Change the env, packages, or index and it is a different string, so the prompt reappears for free — and unlike a sha256 it is readable on the card. | +| No `spendFilter` entry | Installs are not a paid action. | + +### Environments + +| | Decision | +| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Shape | First-class, named, **language-scoped**. A python env and an r env are separate objects sharing one interface — mirrors `KernelManager`: one contract, per-language backends. | +| Conda | Off the table. Claude-science gets language-neutral envs free because conda unifies python and R in one directory; venv/uv unifies nothing, so neutrality would buy conda's abstraction without conda. | +| Manifest | The source of truth. `Global.Path.data/envs//.json`. | +| Directory | Derived, therefore cache. `Global.Path.cache/envs///`. | +| Kernel binding | A **property** of the registry entry, never part of `KernelIdentity` — adding it to the tuple rekeys every persisted record and orphans them. | +| Staleness | Compared at the **registry** level, not inside `ExecutionAuthority`, whose signature carries no kernel identity. | +| Default | A new kernel binds to `default` unless told otherwise. **The binding point is the tool, not the route** — `POST /kernels` was removed in #274/#275 and the agent now names kernels through the `kernel` parameter on `notebook`/`rkernel`, so `environment` belongs beside it, exactly as the reference carries `environment=` on every `python`/`bash`/`r` call. Reassignment is explicit, because it restarts the kernel. | +| Creation | No approval card. It writes a directory in our own cache and runs stdlib code. The install card notes the env will be created. A `uv venv --python X` that downloads an interpreter adds a line to that card. | + +Kernel reads are free — the cache directory is readable under `--ro-bind / /`. Only the installer +needs a writable bind. + +### Installer + +**Ladder**, probed in order: + +1. Existing env directory → use it +2. `Bun.which("uv")` → uv +3. `python3 -m venv` + ensurepip +4. Neither → fail with the exact remedy (`python3-venv` on Debian/Ubuntu, or install uv) + +Verified: on a host whose `python3` has no pip, `python3 -m venv` still bootstraps pip 26.1.2 from +`/usr/lib/python3.14/ensurepip/_bundled/pip-26.1.2-py3-none-any.whl`, and it works **offline inside +`--unshare-net`**. uv is a fast path, never a requirement. Only the Debian/Ubuntu ensurepip split is +a genuine dead end. + +Never auto-download uv. `compute/modal/volume.ts:112-116` is the house precedent: probe, use if +present, throw a remedy if not. + +**Containment.** _Superseded — see "Network policy" below._ Earlier drafts specified a **separate** +install sandbox, network-enabled, because the kernel's was network-denied and only the installer +needed egress. The allowlist proxy removes that asymmetry: under one policy the kernel can reach +PyPI too, so there is no second sandbox. The install runs in the **same** sandbox as the kernel, +differing only in what is writable: + +- egress via the allowlist proxy — identical to the kernel, not a relaxation +- writes confined to the env directory and a private `TMPDIR` +- a **writable package cache** bound in — without it pip disables its cache and every retry and + rebuild re-downloads +- `kernelSensitivePaths()` masked to `/dev/null` +- `--unshare-pid` + +Verified twice. A C-extension source build (`markupsafe --no-binary :all:`) completes inside this +sandbox, the compiler and headers arriving free on `--ro-bind / /`, with the credential mask holding +(`Permission denied` inside, real contents outside). And `pip install --only-binary :all: tqdm` +completes through the proxy with the same masks in place. An earlier claim that source builds would +break was reasoning, not evidence, and was wrong. + +**Wheels-only (`--only-binary :all:`) is the default**, escalating to source builds on request. This +is a speed and reliability default, _not_ a security boundary — if bwrap contains agent Python at +import time it contains `setup.py` at install time. + +### Install lifecycle + +| | Decision | +| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Resolution | Approve the **request**; resolve versions after. The card shows unversioned names. | +| Already satisfied | Skip outright — no card, no install, no restart. Nothing privileged happens, so nothing needs approving. | +| Dispatch | **`wait: true` by default** — the install runs inline and returns its result, so a two-second install costs one turn. `wait: false` returns an exec id to poll, mirroring `modal.ts` → `compute_job`. No notification channel. | +| Verification | After a successful install, import the installed names in the target env and report the versions. Catches an installer that exits 0 without producing a working module. | +| Busy kernel | **Queue** behind the running cell, display `queued behind `, offer a cancel. A cell that lazily imports a submodule mid-install can load a half-written file, so this is correctness, not scheduling. | +| Lock | **Per-env.** Other environments stay fully usable. | +| Restart | **Conditional on the change set** — see "Reopened by later evidence". Purely additive → no restart, namespace survives. Any removal, downgrade, or version change → restart every kernel bound to that env. Kernels on other envs are always untouched. Shared envs are enforced additive-only, so they never restart. | +| Reported failure | **Nothing landed.** Modern pip builds every wheel before running the install phase, so a build failure aborts before anything is committed — verified: a failing package's cleanly-resolving dependency was downloaded and still not installed. Report the log and stop; there is no subset to keep or retry. | +| Failure diagnosis | Surface the **cause**, not pip's summary line. `ERROR: Failed building wheel for X` names the package; the `fatal error:` line above it names a missing system header, which usually means the install is unachievable in a sandbox and a pure-Python alternative is the answer. | +| Wheels-only rejection | Translate it. `Could not find a version that satisfies the requirement X (from versions: none)` reads as "no such package" but means "no wheel for this policy". Say that, and offer the source-build escalation. | +| Interruption | Detach the installer (`detached: true`, no `--die-with-parent`) and **persist the lock with pid + start token**. On CLI start, reconcile: pid alive and token matches → still running; otherwise → unknown outcome. | +| Unknown outcome | New env → `rm -rf`. Existing env → mark dirty, rebuild from manifest. | +| No snapshots | pip has no transactions, and `cp --reflink=always` fails on ext4 here — a pre-install snapshot is a full multi-GB byte copy. Rollback is env-level only. | +| Constraints | Parse with a real PEP 508 parser. Splitting on `==` mishandles `numpy>=2.4`, extras, and markers. | +| Index credentials | Strip before matching, redact on the card. They are env config, not part of the approved action. | + +### Agent contract + +`PackagePrompt.system()` (written, `src/package/prompt.ts`) injected unconditionally into the system +array at `session/prompt.ts:863`, exactly as `SystemPrompt.compute()` is today. + +This is the mechanism that scales. 199 of 293 `SKILL.md` files mention `pip install`; 435 files do. +Editing them is neither necessary nor sufficient — the block pre-empts all of them, plus reference +files the skill tool never intercepts and third-party skills cloned from GitHub that this repo +cannot edit. + +**No skill-level override.** `ComputePrompt.skill()` is a whole-document replacement, appropriate +only when the whole document is wrong — true of the modal skills, whose subject _is_ the governed +mechanism. Every package-mentioning skill is about a domain (docking, geopandas, pydicom) with +install as scaffolding; replacing them would destroy correct content to fix a preamble. Add an +override only if a skill appears whose subject is environment setup. None exists today. + +**Refusal, not redirect — revised after the proxy landed.** This decision previously read "a shell +`pip install` must fail with a message naming `package_install`, not a DNS error", and rested on a +premise that is no longer true: that shell installs fail anyway, so the only job was replacing a +confusing error with a helpful one. + +Under `network: "allowlist"` they succeed. Measured on this branch, inside the agent's own sandbox, +with no tool and no approval card: + +``` +python3 -m venv /venv && /venv/bin/pip install tqdm → 4.70.0 +``` + +The workspace is writable and pypi is allowlisted, so system site-packages being read-only stops +nothing — the agent just builds its own venv beside the project. The proxy did not create the +intent to bypass; it removed the accident that used to prevent it. + +So the shell path has to be **refused**, not merely redirected, or the approval card is decorative: +an agent that never calls `package_install` never shows one. Requirements: + +- Refuse in the bash tool, before execution, matching the installer invocations named in + `PackagePrompt` — including `/bin/pip`, `python -m pip`, and `uv pip`, which a bare + `pip install` match misses. `tree-sitter-bash` is already a dependency and already used for + command parsing, so this is a parse, not a regex over the command line. +- Fail with the same message the contract uses, naming `package_install`. The redirect's original + value stands; it is now the message attached to a refusal rather than to a failure. +- Do **not** solve this by removing pypi from the allowlist. Notebook cells legitimately fetch from + allowlisted hosts, and an allowlist that differs per tool is a second policy to keep consistent. +- Refusal is a contract boundary, not a security boundary. An agent determined to bypass it can + vendor a wheel by hand over the same allowlisted egress. What refusal buys is that the _normal_ + path — the one every skill's `pip install` line leads to — arrives at the card. Treat it as + governance, and do not claim more for it. + +### GPU + +| | Decision | +| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | +| `sandbox.gpu` | `"none" \| "nvidia"`, default `"none"`. When set, `bubblewrapArgs` emits `--dev-bind-try` for the nvidia node set. | +| Not `allowWrite` | Verified: `--bind-try` mounts `nodev` and NVML reports Insufficient Permissions. Only `--dev-bind-try` works. No config-only workaround exists. | +| Authority | Reuses `kernel`. The policy already hashes into `generation`, so flipping it reboots live kernels under the new rules. | +| Metrics | `nvidia-smi --query-compute-apps=pid,used_gpu_memory` joins onto the process-group model in `metrics.ts`. Unavailable → omit the field, never 0. | +| Ordering | GPU access **before** metrics. With the flag off no kernel can be a CUDA app, so a sampler shipped first reads zero by construction. | + +Orthogonal to `--unshare-net`: GPU grants compute without touching the network boundary. + +## Reference implementation — Claude-science, as observed + +Recorded from screenshots and a transcript of a live session, not from documentation or source. It +is the closest shipping analogue to this design, and several decisions above are either copied from +it or deliberately diverge. Treated as evidence of what works in practice, not as a specification. + +### The sequence it runs + +The user typed a direct imperative — _"Can you pip install numpy"_ — and it **did not install**. + +1. **Probe first**, through `bash`. Two probe commands were captured verbatim; the scipy one is + known only by its reported result: + ``` + python -c "import numpy; print('numpy', numpy.__version__)" → numpy 2.4.6 + ‹scipy probe — command not captured› → scipy 1.17.1 + python -c "import torch; print('torch', torch.__version__, '| cuda available:', + torch.cuda.is_available())" 2>&1 || echo "NOT INSTALLED" + ``` +2. **Report and stop** when satisfied: _"numpy is already installed in the default python + environment — version 2.4.6 — so there's nothing to install."_ +3. **Volunteer the ambiguity unprompted** — a different project environment, or inside a Modal + container image, are named as separate targets with different answers. +4. **Reason about placement out loud** before acting, only for the absent package: _"It's a + heavyweight stack, and platform convention is to keep those out of the shared default env, so + I'll put it in a dedicated environment rather than the default one."_ +5. **Reason about hardware**: _"the local GPU isn't accessible from this sandbox, so a local install + would be CPU-only torch (useful for development/testing; actual GPU training goes through + Modal)."_ It then selects `pytorch-cpu` because of that, rather than installing the GPU build and + failing later. +6. **Only then** emit the typed call. + +### The call + +``` +mode create +name torch-cpu +python_version 3.13 +packages › 2 items [pytorch-cpu, torchvision-cpu] +channels › 1 item [pytorch] +background true +``` + +Typed parameters, not a shell string. The approval card renders them as a compact command line +rather than raw JSON: + +``` +create torch-cpu pytorch-cpu torchvision-cpu [channels: pytorch] [python=3.13] +``` + +Package names appear **unversioned**. No pins, no resolved set, no download size. + +### A second trace — installing into an existing environment + +Header: _"Ran 2 commands, set up an environment · 3 steps"_. + +**Step 1 — batched probe**, one `bash` call, `ENV python`: + +``` +python -c "import PIL, sys; print('pillow', PIL.__version__)" 2>&1 | tail -1; \ +python -c "import tqdm; print('tqdm', tqdm.__version__)" 2>&1 | tail -1; \ +pip index versions tdm 2>&1 | head -3 +``` + +``` +pillow 12.3.0 +ModuleNotFoundError: No module named 'tqdm' +tdm (0.1.0) +Available versions: 0.1.0 +``` + +Several probes chained with `;`, each normalised with `2>&1 | tail -1` so a missing module returns a +one-line result rather than a traceback. + +Note the third command: `tdm`, not `tqdm`. The typo resolved to a **real, unrelated package** and +returned its versions. Nothing was installed from it and the agent went on to install `tqdm` +correctly — but it is a live instance of the typosquat exposure that motivates sandboxing the +installer. + +**Step 2 — the install call:** + +``` +environment python +mode install +packages › 1 item [tqdm] +use_pip true +``` + +``` +Installed via pip in 'python': tqdm +``` + +**Step 3 — post-install verification**, again through `bash`: + +``` +python -c "import tqdm, PIL; print('tqdm', tqdm.__version__); print('pillow', PIL.__version__)" +→ tqdm 4.70.0 + pillow 12.3.0 +``` + +What this establishes that the first trace did not: + +- **One tool, several modes.** `mode: create` and `mode: install` are the same tool. The first trace + created a conda env with `channels` and `python_version`; this one installs with `use_pip: true` + and no channels. That corroborates the two backends named in the lock message — **path-venv** and + **conda-backed** — from a second, independent direction. +- **The installer backend is an explicit parameter**, not an internal choice. `use_pip: true` is on + the call. +- **Install is synchronous by default.** No `background: true`, no `exec_id`, no notification — just + a terse one-line result. Background is opt-in per call, so a two-second install stays inline and + only long ones go async. +- **The default environment is named `python`.** Not "default". Weak but real evidence toward + language-scoped naming. +- **Their agent shell has pip and network.** `pip index versions` reached the index from `bash`. + So probe-first is a **policy choice there, not something a sandbox forces** — unlike here, where + the shell has neither. +- **It verifies after installing** rather than trusting the installer's report. + +**Not established by this trace:** no approval cards appear in it. Either none were shown, or an +earlier "Allow for chat" on `bash` covered steps 1 and 3 and the install card was not captured. The +absence is not evidence that installs go unprompted. + +### Approval cards + +One card of each kind was captured. What is on them: + +| | Probe | Mutate | +| -------------- | ----------------------------------- | ------------------------------------------------------- | +| Title | "Run a shell command?" | "Create conda environment torch-cpu?" | +| Chips | `python` · `conda env` | none visible | +| Body | the code, under a `Code` disclosure | the rendered command line, under a `Details` disclosure | +| Primary button | **Allow for chat** | **Allow once** | +| Also | dropdown chevron, `Deny` | dropdown chevron, `Deny` | + +**Not established:** whether "read-only persists, mutation does not" is a systematic policy. That is +one sample of each, and both buttons carry a dropdown — so the label shown is the default offered, +not necessarily the only scope available. The pairing is suggestive and matches how +`permission/next.ts` already separates paid from ordinary actions, which is why this spec adopts the +shape; it is not evidence that they enforce it. + +### The dispatch response + +```json +{ "status": "running", "exec_id": "94d34ecc-f441-4dd8-ab8e-00702bbee577", "message": "…" } +``` + +The message is the interesting artefact. Decomposed: + +- **Async by default** for environment operations, returning immediately. +- **Permanent placeholder** — _"this placeholder is permanent"_. The tool result in the transcript + never updates; the outcome arrives later as a `notifications[]` entry of type `cell_result`, via + an explicit `wait_for_notification` or automatically at the start of a later turn. +- **No progress streaming** — _"Progress streaming (`exec_peek`) is not available for + package/environment operations."_ Stated rather than faked. +- **Honestly leaky interrupt** — `host.exec_interrupt(exec_id)` gives _"real termination for a + path-venv; for a conda-backed environment the wait is abandoned — lock released, subprocess + continues detached."_ It tells the agent that cancel does not always cancel. +- **Per-environment lock, stated to the agent** — \_"do NOT run python, r, or `manage\__` in that + environment until it finishes (its packages are being rewritten and **its kernel restarts on + completion**). A different environment or bash is fine."\* + +Two backends are named in that one sentence: **path-venv** and **conda-backed**. + +`python, r, manage_*` also appear together under one environment's lock. Two readings fit equally +well: the environment genuinely hosts both languages (conda can), or the message is a generic +template listing every execution tool regardless of what this environment contains. **The +screenshots do not distinguish them**, and an earlier draft of this spec asserted the first as fact. + +The language-scoped decision above does not rest on this either way — a venv cannot host R, which is +reason enough on its own. + +### A third trace — the network model + +Asked whether its sandbox has network, it reported: + +> _"Yes — the sandbox has network access, but it's filtered through an **allowlist proxy** rather +> than being open."_ + +- **Reachable (200):** `pypi.org`, `eutils.ncbi.nlm.nih.gov`, `rest.uniprot.org` +- **Blocked at the proxy:** `example.com`, `www.google.com` — connection fails outright +- **Mechanics:** all outbound goes through an HTTP/HTTPS proxy, `*_proxy` env vars set. Direct DNS + resolution returns nothing — name resolution happens _at the proxy_, so `getent hosts` is empty + even for domains that work. Connectivity must be tested over HTTP, never ping or DNS. +- **Allowlisted classes:** scientific APIs and package registries — NCBI, Ensembl, UniProt, PDB, + EBI, ChEMBL, arXiv, CRAN/Bioconductor, PyPI, conda, npm. Arbitrary browsing is not. +- **Adding a domain:** _"approval takes effect immediately without losing kernel state."_ + +This is categorically different from `--unshare-net`. Ours is binary — the kernel has all network or +none. Theirs is a **bounded** network: the kernel can reach the registries and data sources a +research tool actually needs, and cannot reach anywhere else. Because the policy lives in a proxy +rather than a namespace, changing it does not restart anything. + +### A fourth trace — the four install routes + +Asked how it installs packages, it enumerated: + +**1. `manage_packages` — the durable path.** Writes into the environment's real site-packages and +survives kernel restarts. conda by default, pip with `use_pip=true`. Accepts version pins, and _in +dedicated envs_ git URLs, wheel URLs, and extras. Also `mode="uninstall"` and `mode="list"`. + +> _"Installing does not restart the kernel — your variables and imports survive, and a new package +> is importable immediately. **Uninstalling does restart it.**"_ + +And the constraint that makes that safe: + +> _"the shared default `python` and `r` envs are **additive-only**. They accept bare names with at +> most an exact `==` pin; URLs, VCS refs, and version ranges are rejected, and uninstall is blocked. +> Conda there also runs `--freeze-installed`, so an install can never disturb what's already +> present."_ + +**2. `manage_environments` — a dedicated env**, for anything that may later need removing or +re-pinning, and for heavyweight stacks: + +``` +manage_environments(mode="create", name="cheminfo", packages=["rdkit", "scikit-learn"]) +``` + +Every subsequent `python`/`bash`/`r` call then carries `environment="cheminfo"`. Environments +present on that box: **`python`, `r`, `torch-cpu`, `compute-provider-modal`**. + +It can also **register an existing venv from a granted host path**, working against the user's own +repo interpreter with editable installs rather than a managed copy. + +**3. `pip install` inside a bash or python cell — ephemeral.** Gone when the kernel shuts down. And +worse in managed conda envs, where _"their site-packages are mounted read-only in the sandbox, so +`/bin/pip install` reports success and writes nothing."_ A silent no-op, not an error. + +**4. Remote and provider-side** — baked into the job's image on that side. + +Stated default: `manage_packages` into a purpose-built env, reserving the shared `python` env for +small additive things like `tqdm`. + +### What these two traces settle + +- **Environments are language-scoped.** `python` and `r` are _separate environments_ on the same + box. The earlier lock message naming `python, r, manage_*` together was a generic template listing + execution tools, exactly as the alternative reading suggested — not evidence of neutrality. The + hedge in this spec was correct and the question is now closed, in favour of the choice made here. +- **Adopting a user's existing venv is a real, shipped capability**, not a hypothetical. It was + listed as an open product question in this design; the reference answers it. + +### A fifth trace — two failure surfaces, neither partial + +**Blocked before any build.** `pyaudio` into the shared `python` env returned a `manage_packages` +error, not a compiler one: + +``` +ERROR: Could not find a version that satisfies the requirement pyaudio (from versions: none) +``` + +The shared env is **wheel-only as well as additive-only**, so an sdist-only package is filtered out +of the candidate list and never reaches a build step. Nothing downloaded, nothing changed. + +Note the message. `from versions: none` reads as _"this package does not exist"_ when it means +_"no wheel available for this policy"_. Under a wheels-only default that error will be common, and +raw is the wrong way to surface it. + +**A real build failure**, reproduced in a throwaway venv with a C extension against a nonexistent +header: + +``` +Building wheel for brokenpkg (pyproject.toml): finished with status 'error' + src/speed.c:2:10: fatal error: portaudio_that_does_not_exist.h: No such file or directory + error: Command '['gcc', ...]' returned non-zero exit status 1 + note: This error originates from a subprocess, and is likely not a problem with pip +ERROR: Failed building wheel for brokenpkg +``` + +Read bottom-up: pip's own `ERROR:` names only _which_ package failed; the cause is the `fatal error:` +line, and it is a missing **system** header, not a Python dependency. That signature means an +OS-level `-dev` package is required, which in a sandbox usually means the install is not achievable +and a pure-Python alternative is the answer. + +**The install was atomic.** `brokenpkg` was given a dependency on `six`, which installs cleanly as a +wheel. The log shows `Collecting six` — resolved and downloaded — yet: + +| | before | after | +| --------- | --------------------------------- | ----------- | +| installed | packaging, pip, setuptools, wheel | _identical_ | + +`six` was not left behind. Modern pip builds every wheel first and runs the install phase only after +all builds succeed, so a build failure aborts before anything is committed. The outcome is a clean +environment plus a log, never a half-installed one. + +**Where partial state does arise**, per the same trace: + +- a package that builds and installs fine but fails on **import** — wrong ABI, missing runtime `.so` +- a legacy `setup.py install` invoked directly +- an **interrupted** multi-package install, where earlier wheels already landed — and on conda, + `host.exec_interrupt` abandons the wait while the operation continues detached + +### A sixth trace — `mode="list"`, and the provider environment + +**`list` returns structured data, not text:** + +``` +{ environment_name, package_count, packages: [...], python_version, history: [...] } +``` + +`packages` is `name==version` for **everything the solver knows about**. The shared `python` env +reports 168 entries, most of them native libraries and fonts — `libgcc`, `harfbuzz`, `xorg-libx11`, +`qt6-main` — with importable Python packages a minority. + +_This is a bug in `src/package/prompt.ts` as first written_, which rendered the full package list +into the capability block on every request. Fixed: the block now shows only what was explicitly +requested, plus a `(+N deps)` count. A contract buried in font libraries teaches the agent nothing. + +**`history` is the field that matters** — an ordered record of how the env was built: + +``` +create numpy, pandas<3, scipy, matplotlib, seaborn, pillow, socksio, pysocks (py3.11) +install pypdfium2==5.9.0 (pip) +install nbformat (conda) +install tqdm (pip) +``` + +Seed spec, then every mutation, each tagged with its backend — _"the fastest way to answer 'where +did this package come from, and was it conda or pip?', which matters because mixing the two in one +env is where dependency resolution tends to break."_ + +**This sharpens the manifest decision.** This spec says the manifest is the truth and the directory +is derived. A _flat package list_ cannot be replayed faithfully — order matters and backend matters. +An ordered, backend-tagged history can. Rebuild-from-manifest becomes replay-the-history, and the +same record answers the provenance question that was listed here as out of scope. + +**`compute-provider-modal`** is infrastructure, not a workspace: 59 packages, history is one line +(`create python=3.11, pip`), payload is `modal==1.5.1` plus its gRPC and async plumbing. Nothing +scientific. It backs a `compute_provider` tool — _"the authenticated kernel where the Modal SDK is +pre-imported and wired to your token"_ — used for provisioning: building images, managing volumes, +inspecting workspace state. It **explicitly rejects `gpu=`**; job submission goes through a separate +path entirely. + +That is a different boundary from the one in ADR-0001, which holds that credentials are _"never +added to a generic agent, shell, kernel, or job environment"_ and routes everything through a +trusted JS adapter. Theirs puts the credential **inside one dedicated, capability-restricted +kernel** that can provision but cannot dispatch paid work. + +Worth noting because of a cost we already pay for our version: `compute/modal/volume.ts` exists — +a pinned Python bridge launched through `uv` — solely because the JS SDK cannot read Volumes and we +refused to have a credentialed Python environment. A capability-restricted provider env would make +that bridge unnecessary. Not a recommendation to change ADR-0001; a recorded alternative with a +known price on both sides. + +### Isolation + +The web UI is served on `:8000` behind a **single-use nonce** that expires in three minutes; sandbox +content is served separately on `:8001`. Remote access requires forwarding both. + +### Mapping to this design + +| Observed | This spec | Why | +| ----------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Probe before installing; report and stop when satisfied | **Adopted** — skip outright, no card, no restart | Confirms the principle independently | +| Unversioned names on the approval card | **Adopted** — approve the request, resolve after | Same trade | +| Read-only grant persists, mutation grant does not | **Adopted in shape**, on our own reasoning rather than theirs — the sample is one card each (see above) | Matches how `permission/next.ts` already separates paid from ordinary actions | +| A shell probe to answer "is numpy present?" | **Rejected** — the capability block carries the inventory, so nothing needs to run | Ours is a design choice about where inventory lives; why theirs probes is not observable | +| Typed parameters rendered as a command line on the card | **Adopted** — the canonical command string is both the display and the permission pattern | Readable, and it doubles as the match | +| Per-environment lock, named to the agent | **Adopted**, including surfacing it in the capability block | Directly copied | +| Kernel restarts on completion | **Adopted** | Confirms always-restart | +| Install is **synchronous by default**, `background: true` opt-in per call | **Adopted — revises this spec.** Default `wait: true`, returning the result inline; long installs pass `wait: false` and get an exec id to poll | An earlier draft made every install async, which is wrong for the `tqdm` case: two seconds of work behind a dispatch-and-poll round trip. `modal.ts` already has exactly this flag with exactly this default | +| Async dispatch with an exec id, when asked for | **Adopted** | — | +| Installer backend as an explicit call parameter (`use_pip: true`) | **Rejected** — the ladder picks it, and the choice is shown on the card | The agent has no basis for choosing; the host does. Exposing it invites the agent to pick badly and adds a field that changes nothing it can reason about | +| Verify by importing after installing | **Adopted** | Cheap, and it catches an installer that reports success without producing a working module | +| Batched probes in one shell call | **Not applicable** — the capability block carries the inventory, so there is nothing to probe | — | +| Notification delivery (`wait_for_notification`, `cell_result`) | **Rejected** — dispatch returns an id and is polled, mirroring `modal.ts` `wait: false` → `compute_job` | No notification channel exists here; polling already does | +| `python, r` under one environment lock — language-neutral, or a generic message template? | **Not adopted either way** — we are language-scoped | Decided on our own constraint: a venv cannot host R. Whether theirs is neutral is not established by the screenshots | +| Conda with channels | **Rejected** — venv/uv ladder | Heavy dependency and a second solver; the ladder is verified and needs neither | +| Channels on the approval card | **Adopted as `index`** | A channel is where the code comes from — the same reason index belongs in the pattern | +| Interrupt that abandons the wait and leaks a subprocess | **Rejected** — persist pid + start token and reconcile on startup | Copy the honesty, not the leak. `KernelProcessIdentity` already does exactly this for kernels | +| Combined create-and-install card | **Rejected** — no card for creating an empty env at all | Creating a directory in our own cache and running stdlib code is not privileged; a card that guards nothing devalues the ones that do | +| Separate origin for sandbox content, nonce on the app | **Deferred** — see Out of scope | Orthogonal; our `sandbox=""` iframe is stricter but blocks interactive output | +| No progress streaming for package operations | **Accepted as a constraint**, not a goal | If the closest analogue cannot do it, we should not promise it | + +## Reopened by later evidence + +Three decisions above predate the network and install-routes traces and are contradicted or +weakened by them. Flagged rather than silently rewritten, because two were explicit user calls. + +### 1. Restart policy — resolved: restart iff the change set is non-additive + +Initially read as a contradiction: the reference does not restart on install, only on uninstall. On +asking what happens when a **downgrade** lands in a dedicated env, where ranges are accepted and +`--freeze-installed` is not in force, the answer was that it still does not restart: + +> _"Install keeps the kernel alive, so you keep the stale module. Only `mode="uninstall"` restarts a +> kernel — and a downgrade goes through `mode="install"`, even though pip implements it internally +> as remove-then-reinstall. New files land in site-packages; your live interpreter never notices."_ + +So the reference has the hazard, and mitigates it with user discipline rather than mechanism. Its +own characterisation, from swapping a package's files under a running import: + +| | version | behaviour | +| --------------------------------------- | ------- | ------------ | +| before disk change | 2.0 | v2 | +| after disk change, no reload | 2.0 | v2 | +| submodule imported **after** the change | — | **v1** | +| after `importlib.reload` | 1.0 | v1 | +| name bound via `from … import compute` | — | **still v2** | + +**The hazard is mixed state, not staleness.** `sys.modules` caches only what was already imported. +Anything imported _later_ — a submodule, a lazy import inside a function, a dependency pulled in on +first use — reads the new files. The process ends up running 2.0's loaded modules beside 1.0's +freshly-loaded ones: a configuration neither version was tested in, failing in ways that do not +point back at the install. + +Two consequences worth carrying: + +- **`importlib.reload` is not a remedy.** It rebinds attributes on the module object, but names + bound directly (`from torch import foo`) still point at the old function, and live instances keep + their old classes. For compiled extensions it is worse — `torch._C` is a `.so` that CPython cannot + unload, so reload reuses the loaded one. _"A torch downgrade is not recoverable in-process."_ + Restart is the only recovery; never offer reload as an alternative. +- A bytecode-cache trap they hit: same-second mtime plus unchanged byte length let the `.pyc` be + considered valid, so the first reload returned the old version anyway. Real installs write fresh + sizes and mtimes, but hand-edited files will hit it. + +**This vindicates the always-restart call made here** — it eliminates by construction the hazard the +reference has to warn users about. But it overpays: a `tqdm` install discards a namespace for +nothing. + +The rule that dominates both: + +| Change set | Restart | +| --------------------------------------------------------------------- | --------------------------------------------------- | +| Purely additive — nothing existing replaced, removed, or re-versioned | **No.** No mix is possible, so nothing can go stale | +| Contains any removal, downgrade, or version change | **Yes**, unconditionally | +| Uninstall | **Yes** | + +Enforced additive-only on shared envs (their `--freeze-installed`) makes the common path _provably_ +additive, so it never restarts, by construction rather than by inspection. + +**Correction to an earlier draft of this table:** it said "restart only if a package being replaced +is already in `sys.modules`". That is wrong. A downgrade drags dependencies with it — if dependency +`B` was loaded as part of something else and the newly-installed `A` expects the older `B`, the mix +exists even though `A` itself was never imported. The trigger is the resolver's **whole change set** +containing anything non-additive, not an intersection with `sys.modules`. + +**Approval consequence:** because the restart is now conditional, the card must say so _before_ +approval — "this will replace numpy 2.3.4 with 2.2.0 and restart your kernel, discarding N +variables" — rather than the user discovering it afterwards. + +**And it must escape the standing grant.** A user who accepted `install*` to stop being asked about +`tqdm` has not consented to losing a namespace mid-session. A non-additive change is destructive, so +it prompts even when a standing allow is in force — the same carve-out `spendFilter` already +implements for a different reason at `permission/next.ts:167-171`, so the mechanism exists. +Recommended, not yet confirmed. + +### 2. Binary network policy — resolved, see "Network policy" + +Flagged here as a weakness, then prototyped and settled. The allowlist proxy is enforceable with no +root, it collapses the separate install sandbox, and it bounds exfiltration to a fixed host set +rather than the whole internet. Moved into its own section above with the measurements; this entry +remains only as the record of where the question came from. + +### 3. Keep-and-retry on partial install — premise withdrawn + +An explicit decision here was that a partial install keeps what landed and retries the failed subset +under the same approval, with a discard action for a twice-failed subset. The prototype carries an +`outstanding` field for it. + +**Build failures do not produce that state.** pip builds all wheels before installing any, so a +failed build commits nothing — demonstrated by a package whose clean wheel dependency was resolved +and downloaded and still absent afterwards. The premise was mine, not observed, and it was wrong. + +The three real sources of partial state are already covered by decisions taken for other reasons: + +| Source | Already handled by | +| ------------------------------------------------------- | ------------------------------------------------------------------ | +| Interrupted mid-install | Detach + reconcile; unknown outcome → rebuild from manifest | +| Installs, then fails on import (bad ABI, missing `.so`) | The post-install import verification adopted from the second trace | +| Legacy `setup.py install` | Wheels-only default | + +So `outstanding`, the retry-the-subset path, and the discard action have no remaining use case. +Recommend dropping all three rather than carrying machinery for a state pip does not produce. + +### 4. Silent-success on read-only site-packages + +_"`/bin/pip install` reports success and writes nothing."_ Our redirect-on-failure assumes a +shell install _fails_. Under a read-only mount it may exit 0 instead, which is worse than an error — +the agent believes it succeeded. The redirect must detect the no-op, not just the failure. + +## Re-analysed against `main` @ `74ee13cd` + +Three commits landed after this spec was drafted — #274 project-scoped inspector and kernel +lifecycle, #275 unified compute/results/artifact workflows, #276 minimised completed compute +results. `science/kernel/registry.ts` is **unchanged**, so the layer analysis above still holds. +Four things do move. + +### Named kernels became agent-driven + +`notebook` and `rkernel` now take `kernel` (a validated `[A-Za-z0-9][A-Za-z0-9._-]*` name, max 64) +and `action: "execute" | "stop"`. The tool description instructs the agent to issue several calls in +one response with distinct names for parallel analyses, to stop them when done, and — pointedly — +_"Never use shell subprocesses to imitate multiple kernels."_ A test asserts four named calls own +four live kernels concurrently. + +`POST /kernels` was **removed**. Kernels are created implicitly by naming one. + +Consequence for this spec: the environment binding must live on the **tool**, not the route. An +`environment` parameter beside `kernel`, which is exactly the shape the reference uses. Everything +else about binding — property not identity, registry-level generation, explicit reassignment — is +unaffected. + +### `CommandRuntime` is the tracking primitive we were about to build + +New at `science/command/registry.ts`, wired into `bash.ts`: every shell command registers with +`{id, projectID, sessionID, messageID, callID, description, command, process_id, started_at, +resources?}` and a `stop()` closure, and deregisters on exit. `list` / `owned(id, projectID, +sessionID)` / `stop` mirror `KernelRuntime` exactly. New routes `/commands` and +`/commands/:commandID/stop`. + +An install is a long-running command, so this is the live half of install-job tracking, already +built and already consistent with the rest of the codebase. Use it rather than inventing a parallel +registry. + +**But it is `new Map()` — in-memory only.** It does not survive a CLI restart, so the +detach-and-reconcile decision still needs its own persisted record with pid and start token. +`CommandRuntime` tracks what is running now; it cannot answer what was running before the crash. + +### Project trust flipped to trusted-by-default + +`ProjectTrust.status` inverted in #274: previously trusted only on an explicit persisted `trusted` +record, now trusted **unless** explicitly `revoked`. The tests were renamed to match — _"untrusted +project opens read-only…"_ became _"project code is enabled by default"_ — so this is deliberate, +not drift. + +Consequence: `canExecuteProjectCode` is true by default, so the `project_untrusted` branch of +`ExecutionAuthority.decide` is now rare. This spec should stop treating project trust as a +meaningful gate on installation. The real gates are the permission card, the sandbox, and +`sandbox_unavailable`. + +**One thing worth checking, not asserted.** The new condition is +`saved?.root !== canonical || saved.state !== "revoked"` → trusted. A record whose root no longer +matches evaluates the first clause true and yields _trusted_ even when its state is `revoked`. +Under the old code that case returned `revoked`. Whether a revoked project whose root moved should +re-trust silently is a question for whoever wrote #274; it may be intended, since a different root +is arguably a different project. Not tested here. + +## Network policy — supersedes the separate install sandbox + +`sandbox.network` is `"allow" | "deny"`. Deny (`--unshare-net`) is the default and locks kernels out +of PyPI, NCBI, UniProt, PDB and EBI — most of what a research tool is for. Allow is unrestricted +egress. Neither is what the product needs. + +A spike on `proto/sandbox-allowlist-proxy` established a third state, enforced rather than advisory: + +``` +--unshare-net → TCP to any host, incl. the host's own 127.0.0.1 000 blocked +bind-mounted unix socket, same namespace PONG crosses +``` + +The socket is therefore the only route out, and a proxy on the far end decides what is reachable, +resolving names itself. No root, no `pasta`, no `nftables`. + +``` +kernel ─TCP→ shim (127.0.0.1:3128, in-ns) ─unix socket→ proxy (host) ─→ allowlisted host only +``` + +Measured inside the sandbox: `pypi.org` 200, `eutils.ncbi.nlm.nih.gov` 200, +`rest.uniprot.org/uniprotkb/P00533.json` 200, `example.com` and `www.google.com` denied, direct +egress with the proxy unset 000, and `getent hosts pypi.org` empty — the reference implementation's +own signature, reproduced. Then `pip install --only-binary :all: tqdm` succeeded through it with +credentials masked. + +**Consequences for this spec:** + +- The **separate install sandbox is deleted**. One policy covers kernel and installer. +- `sandbox.network` becomes three-state: `"deny" | "allowlist" | "allow"`, defaulting to + `"allowlist"`. This is a **breaking change to a documented config key** and needs an ADR before + anything is built on it. +- Proxy policy must stay **out** of the `generation` hash, so adding a domain takes effect without + tearing down kernels — the property the reference advertises. +- `HTTP_PROXY` / `HTTPS_PROXY` / `NO_PROXY` join `SAFE_ENV_PREFIXES`; nothing reaches a kernel today. +- `Sandbox.wrapArgv` must compose the shim (`sh -c ' & exec '`) and bind the socket. + +Not yet done in the spike: any product wiring, a config surface, per-project domains, macOS seatbelt, +audit logging, port policy on CONNECT. And the proxy pipes bytes after checking the authority — it +cannot see inside TLS, so host-level allowlisting is the boundary, not content inspection. + +## Sequencing + +Rewritten. The previous order predated the proxy spike, the rebase onto `74ee13cd`, and the +withdrawal of keep-and-retry; it was stale in four places. + +**Phase 0 — independent, ships now.** No open questions, no dependencies on anything below. + +1. `sandbox.gpu` flag emitting `--dev-bind-try` +2. GPU metrics sampler — **depends on 1**, since with the flag off no kernel can be a CUDA app +3. Three-way install diagnostics in `findPython` +4. Wire `PackagePrompt.system()` into the system array — harmless before the tool exists, because + what it says about shell installs is already true + +**Phase 1 — network policy.** Determines the shape of everything after it, so it goes first. + +5. ADR: `sandbox.network` three-state, and the credential-boundary question raised by + `compute-provider-modal` against ADR-0001 +6. Proxy and shim into `src/sandbox/`, `Sandbox.wrapArgv` composition, `*_proxy` in the env + allowlist, config surface, policy kept out of `generation` + +**Phase 2 — environments.** + +7. Environment store: ordered backend-tagged history as the manifest, directory under + `Global.Path.cache`, per-env lock +8. `environment` parameter on `notebook`/`rkernel` beside `kernel`; `findPython` prefers the env; + registry-level generation comparison + +**Phase 3 — installation.** + +9. `package_install` tool: canonical command-string pattern, `always: ["install*"]`, non-additive + changes escaping the standing grant +10. Installer ladder, wheels-only default, message translation for both failure surfaces +11. `wait: true` default with `wait: false` dispatch; `CommandRuntime` for the live half plus a + persisted pid + start token for reconcile-on-restart +12. Post-install import verification +13. R parity — `R_LIBS_USER` is already in the kernel env allowlist and `install.packages` always + exists, so this is the simpler backend + +## Not verified + +Stated so nobody builds on them: + +- **CUDA compute under `--dev-bind`.** `nvidia-smi -L` works; no kernel launch was tested. +- **macOS.** The seatbelt profile allows `file-write*` on all of `/dev` (`sandbox.ts:267`), so Metal + may already work — untested. No install-sandbox profile has been written for seatbelt. +- **Whether the capability block beats an install-heavy skill.** The worst case is + `chemistry/molecular-docking/SKILL.md` with 9 `pip install` mentions. Empirical, answerable only + by running it. The sandbox and the redirect are the backstop. +- **The proxy at any scale.** _Partly resolved on `feat/sandbox-network-policy`._ Backpressure, + bounded buffers, dial timeouts and a per-client state machine were built and measured after this + was written; an 18 MB wheel now arrives byte-exact through a real sandbox in CI on both backends. + What remains unverified is the original sentence's tail: **concurrency is still uncapped** + (~64.7 KB per connection, no ceiling), there is no audit log, and behaviour under a slow or + hostile upstream is untested. `pip install torch` at gigabyte scale still has not been run. +- **The proxy on macOS.** _Resolved._ Seatbelt has no namespace, so the design changed rather than + transferred: the proxy listens on `127.0.0.1:` and the profile narrows `network-outbound` to + that one address, with a per-start secret in the proxy URL because every process on the machine + shares one loopback. Verified against a real `sandbox-exec` in CI — allowlisted host 200, denied + host refused, no direct route, no DNS, 18 MB byte-exact, and `pip install` through the + authenticated proxy. +- **`sandbox.gpu` alongside the proxy.** Both change `wrapArgv`; they have never been composed. + Still true — `sandbox.gpu` does not exist yet. +- **The release-mode shim.** In a compiled binary the binary is its own shim + (`Installation.isLocal()` false). Verified once by hand against a `bun run build --single` build — + a real `pip install` succeeded inside `--unshare-net` with the boundary intact — but every + automated test runs under `bun run`, which takes the dev-bundle branch instead. Nothing keeps it + working. + +## Out of scope + +- **Interactive kernel output on a second origin.** `NotebookView.tsx:764` renders kernel + `text/html` in `