Skip to content

feat(vultr): add Vultr provider with Compute.Instance and Database.Postgres#307

Open
zeyuri wants to merge 6 commits into
alchemy-run:mainfrom
zeyuri:feat/vultr-provider
Open

feat(vultr): add Vultr provider with Compute.Instance and Database.Postgres#307
zeyuri wants to merge 6 commits into
alchemy-run:mainfrom
zeyuri:feat/vultr-provider

Conversation

@zeyuri
Copy link
Copy Markdown
Contributor

@zeyuri zeyuri commented May 11, 2026

Adds an alchemy/Vultr provider with two resources, an example wiring them together end-to-end, and an extract of the Postgres helpers (applyMigrations, parsePostgresOrigin, listSqlFiles) out of Neon/ into a shared Postgres/ module so the new resource can reuse them without a cross-vendor import.

import * as Vultr from "alchemy/Vultr";

const db = yield* Vultr.Database.Postgres("AppDb", {
  region: "EWR",
  plan: "vultr-dbaas-hobbyist-cc-1-25-1",
  trustedIps: ["0.0.0.0/0"],
  migrationsDir: "./migrations",
});

const vm = yield* Vultr.Instance("HelloVm", {
  region: "ewr",
  plan: "vc2-1c-1gb",
  osId: 1743, // Ubuntu 22.04
  userData: buildUserData({ databaseUrl: Output.map(db.connectionUri, Redacted.value),}),
});

Resources

  • Vultr.Compute.Instance — shared-CPU VMs (vc2-*). Adoption fallback in read() searches by Alchemy branding tags so a partial create (state lost between createInstance and persist) doesn't spawn a duplicate.
  • Vultr.Database.Postgres — managed Postgres cluster. Exposes host, port, user, password (Redacted), databaseName, connectionUri (Redacted), and an origin: PostgresOrigin matching the shape Cloudflare.Hyperdrive accepts. migrationsDir + importFiles work the same way they do on Neon.Project. Adoption fallback uses the deterministic physical name from createPhysicalName({ id }) since Vultr's tag field is a single string (not an array).

Sub-resources for users / logical DBs / connection pools / read replicas are scoped out — endpoint coverage is in Database/api.ts but no resource wrappers yet. Not needed to use a cluster (admin user + default db are auto-created).

Verified live

A small example at examples/vultr-tunnel/ provisions both resources behind a Cloudflare.Tunnel + proxied CNAME:

GET https://vultr-demo.ktarz.com/
{
  "host": "vultr-demo.ktarz.com",
  "count": 3,
  "messages": [
    { "id": 1, "body": "hello from migrations" },
    { "id": 2, "body": "this row was seeded at deploy time" },
    { "id": 3, "body": "vultr managed postgres works" }
  ]
}

Path: browser → Cloudflare edge → tunnel → cloudflared on the Vultr VM → Bun HTTP server reading DATABASE_URL → Vultr Managed Postgres → rows seeded by ./migrations/0001_init.sql (applied by alchemy at deploy time).

Workaround included: local DnsRecord shim

examples/vultr-tunnel/src/DnsRecord.ts is a local workaround, not a piece of work to merge as-is. It exists only because alchemy/Cloudflare doesn't ship a DNS Record resource yet — and the moment you go off the all-Workers path (a Tunnel, an external origin, a static A record) you need one. The shim is CNAME-only, hand-rolled HTTP, custom CfApi Context.Service — the bare minimum to make the example deploy.

A proper Cloudflare.Dns.Record should:

  • Live at packages/alchemy/src/Cloudflare/Dns/Record.ts (next to Zone.ts)
  • Reuse Cloudflare.Credentials.fromAuthProvider() + the existing @distilled.cloud/cloudflare SDK's createRecord / getRecord / updateRecord / deleteRecord (all four verbs already there).
  • Support A / AAAA / CNAME / TXT / MX / SRV / CAA via a discriminated type field, not CNAME-only.
  • Accept either zoneId: string or zoneName: string and cache the resolved zone id in attrs.
  • Adopt by (zoneId, name, type) (uniquely identifies a record on CF) instead of always-creating on state loss.

Happy to spin a separate small PR for this if there's interest — keeping it out of this one so the Vultr conversation doesn't tangle with a Cloudflare-namespace decision. (Same TODO is on the source file as a TODO(upstream) block.)

Bugs caught while implementing — worth flagging

  • Vultr's managed-databases endpoint returns status capitalised ("Running") even though the rest of the JSON is lower-case. waitForRunning normalises before comparing — without that the poller loops forever and burns minutes / dollars before someone notices.
  • Bun's installer hard-requires $HOME. cloud-init doesn't set it, and combined with set -u the script dies silently at "HOME: unbound variable" with no obvious failure — every subsequent step is skipped. export HOME=/root before the installer fixes it (in examples/vultr-tunnel/src/userData.ts).

What I'd love a maintainer eye on

  1. Vultr.Database.Postgres adoption strategy. I'm relying on the deterministic label (${stack}-${id}-${stage}-<hash>) for read()'s recovery path because Vultr's tag field is a single string. This works for the realistic "create succeeded, state wasn't persisted" case (engine preserves instanceId), but won't work if state is entirely wiped (engine generates a new instanceId, label changes). Compute instance adoption uses multiple-tag matching since it supports tags: string[]. Open to suggestions if there's a cleaner pattern.

  2. The @/* alias in src. I wanted to use it to avoid ../../../Postgres/Migrations.ts-style imports but realised tsc -b doesn't rewrite paths in the emitted lib/, and external consumers using the "import" condition don't have the alias. I extracted to a shared Postgres/ module instead, which I think is the right structural fix — but the depth is still 3 levels. Curious if you've considered bundling the lib output via tsdown so aliases could be used internally.

  3. No tests yet. Pattern would mirror test/Cloudflare/Tunnel/Tunnel.test.ts — live API hitting a real VULTR_API_KEY. Happy to add as a follow-up, or in this PR if you'd prefer.

Commits

5acc588 docs(examples): flag the local DnsRecord shim as workaround-not-fix
3e6a29f feat(examples): vultr-tunnel — add Postgres-backed Bun app
e8a615b feat(vultr): add Vultr.Database.Postgres resource
8c66e66 refactor(postgres): extract shared Postgres helpers out of Neon
c5621d4 feat(examples): vultr-tunnel — Vultr VM behind a Cloudflare named tunnel
9aee9df feat(vultr): add provider with Compute.Instance resource

Reject as-is, scope down to just the compute side, or split into multiple PRs — happy to follow whichever direction makes review easiest.

@zeyuri zeyuri marked this pull request as ready for review May 11, 2026 16:48
zeyuri and others added 6 commits May 12, 2026 16:45
Vultr v2 provider following the Neon pattern: env-based credentials
(VULTR_API_KEY), typed HTTP wrapper with tagged errors, and one
resource — Vultr.Compute.Instance for shared-CPU VMs.

The reconciler does observe → ensure → sync with a tag-based adoption
fallback in read so partial creates (state lost between createInstance
and persist) don't duplicate instances on the next run. Plan/replace
classification covers region/osId/imageId/plan/hostname/sshKeys/ipv6
as immutables; label/tags/backups/ddos/firewall are mutable in place.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
End-to-end demo wiring the new Vultr.Compute.Instance to a
Cloudflare.Tunnel + proxied CNAME. cloud-init installs Caddy on :8080
and cloudflared bound to the tunnel token, so `https://<host>` flows
edge → tunnel → connector → caddy.

Includes a local Cloudflare.DnsRecord shim (~250 LOC, alchemy v2 has
no DnsRecord resource yet) so the stack is self-contained and the
shape is obvious if someone wants to land DnsRecord upstream.

Verified end-to-end against a real Vultr account + ktarz.com on
Cloudflare.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move applyMigrations / runSql / listSqlFiles / readSqlFile /
parsePostgresOrigin from packages/alchemy/src/Neon/ to a new shared
packages/alchemy/src/Postgres/ module so non-Neon Postgres-flavoured
providers (next commit: Vultr Managed Postgres) can reuse them without
a cross-vendor import.

The Neon module's own files become thin re-export shims so existing
imports (alchemy/Neon/Migrations, etc.) keep working — no public API
break.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A Managed Postgres cluster resource backed by Vultr's
`/v2/databases` surface. Verified end-to-end against a real Vultr
account by deploying a hobbyist-tier (\$15/mo) cluster, running SQL
migrations from a local `./migrations/` directory against the seeded
`defaultdb`, and consuming `db.connectionUri` from a Bun app running
on a sibling Vultr.Compute.Instance behind a Cloudflare Tunnel.

Resource type:

```ts
export const Postgres = Resource<Postgres>("Vultr.Database.Postgres");
```

Outputs `host`, `port`, `user`, `password` (Redacted), `databaseName`,
`connectionUri` (Redacted), and an `origin: PostgresOrigin` matching
the shape `Cloudflare.Hyperdrive` accepts — so the cluster slots into
Hyperdrive bindings without extra plumbing, mirroring the Neon
provider.

Migrations and seed files (`migrationsDir`, `migrationsTable`,
`importFiles`) use the shared `Postgres/` helpers landed in the
previous commit. Each migration runs in a transaction with bookkeeping
so partial application is impossible.

Notable bug-fixes-while-implementing surfaced in this commit:

1. Vultr returns the status field capitalised (`"Running"`) even
   though the rest of the JSON is lower-case. `waitForRunning`
   normalises before comparing so the poller actually terminates.
2. `read()` falls back to a label-based lookup (`?label=…`) when
   `output.databaseId` is missing — this is the recovery path Plan.ts
   triggers when state shows `status: "creating"` with no persisted
   attrs. Without it the engine would create a duplicate \$15/mo
   cluster on the next deploy. Vultr's `tag` field is a single string
   (not an array, unlike compute instances) so we rely on the
   deterministic physical name from `createPhysicalName({ id })` for
   identification, which works because the engine preserves
   `instanceId` across runs.

Sub-resources for users / logical DBs / connection pools / read
replicas have endpoint coverage in `Database/api.ts` but no resource
wrappers yet — that's the next PR. Not required to *use* a cluster
since the admin user + default db are created automatically.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends the example to provision a Vultr.Database.Postgres cluster
alongside the existing Vultr.Compute.Instance + Cloudflare.Tunnel.

The VM now runs a Bun HTTP server (instead of Caddy) that queries the
`messages` table over `import { sql } from "bun"`. cloud-init installs
Bun, writes the app, and registers it as a systemd unit with the
cluster's `connectionUri` (Redacted -> plain via Output.map) piped in
via `Environment=DATABASE_URL=…`.

End-to-end path verified live:

```
GET https://vultr-demo.ktarz.com/
{
  "host": "vultr-demo.ktarz.com",
  "count": 3,
  "messages": [
    { "id": 1, "body": "hello from migrations", … },
    { "id": 2, "body": "this row was seeded at deploy time", … },
    { "id": 3, "body": "vultr managed postgres works", … }
  ]
}
```

Two cloud-init gotchas surfaced and fixed:

- The Bun installer reads `$HOME`, which cloud-init does NOT set; combined
  with `set -u` the script silently dies at "HOME: unbound variable" and
  the rest of the steps (app + cloudflared) never run. Fixed by
  exporting `HOME=/root` before the installer.
- `trustedIps: ["0.0.0.0/0"]` is the simplest working default for
  cluster reachability from both this dev machine (for migrations) and
  the freshly-booted VM (whose mainIp isn't known until after the
  cluster is up). Documented as insecure-for-prod.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a TODO(upstream) header explaining that this file exists only
because `alchemy/Cloudflare` doesn't ship a DNS Record resource yet,
and sketches what the canonical `Cloudflare.Dns.Record` should look
like (record types beyond CNAME, reuse the existing `@distilled.cloud`
SDK verbs, adoption by `(zoneId, name, type)`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you plan on building an Effect-native Platform for this? We can lift the user data into something that is contributed to with bindings so we can plug in. E.g., I was planning on adding a Process or Daemon construct that would translate to a systemd service.

export default MyProcess extends Vultr.Daemon<MyProcess>()(
  "MyProcess",
  { main: import.meta.filename },
  Effect.gen(function*() {
    const db = yield* ...
    
   yield* serve(myServer)
 })
) {}

Comment on lines +31 to +32
const tunnelHostname = yield* Config.string("TUNNEL_HOSTNAME");
const zoneName = yield* Config.string("CLOUDFLARE_ZONE_NAME");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally these would be Cloudflare resources instead of config. Out of scope for this PR maybe, but integration should work via Resources and Outputs wherever possible

Comment on lines +39 to +44
const db = yield* Vultr.Database.Postgres("AppDb", {
region: "EWR",
plan: "vultr-dbaas-hobbyist-cc-1-25-1",
trustedIps: ["0.0.0.0/0"],
migrationsDir: "./migrations",
});
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might want to make Vultr.PostgresDatabase instead. Vultr.Database.Postgres won't tree shake as well.

region: "EWR",
plan: "vultr-dbaas-hobbyist-cc-1-25-1",
trustedIps: ["0.0.0.0/0"],
migrationsDir: "./migrations",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to see this wired up with Drizzle.Schema perhaps in a different example? Take a look at examples/cloudflare-neon-drizzle for inspiration

Comment on lines +69 to +79
const tunnelTokenPlain = Output.map(tunnel.token, Redacted.value);
const databaseUrlPlain = Output.map(db.connectionUri, Redacted.value);
const vm = yield* Vultr.Instance("HelloVm", {
region: "ewr",
plan: "vc2-1c-1gb",
osId: 1743, // Ubuntu 22.04 x64
tags: ["app:vultr-demo"],
userData: buildUserData({
hostname: tunnelHostname,
tunnelToken: tunnelTokenPlain,
databaseUrl: databaseUrlPlain,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

buildUserData should return Output<Redacted<string>> and accept Input<Redacted<string>> as input. We add extra envelope encryption for Redacted values when persisting state. And this helps protect users from mistakes.

@zeyuri
Copy link
Copy Markdown
Contributor Author

zeyuri commented May 14, 2026

@sam-goodwin one thing more important, I have opened this PR on the distilled repo, should I focus first on that work, then on this one? That PR supports a lot more services from Vultr, this ones only focused on the two I'm actually using

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants