feat(vultr): add Vultr provider with Compute.Instance and Database.Postgres#307
feat(vultr): add Vultr provider with Compute.Instance and Database.Postgres#307zeyuri wants to merge 6 commits into
Conversation
Vultr v2 provider following the Neon pattern: env-based credentials (VULTR_API_KEY), typed HTTP wrapper with tagged errors, and one resource — Vultr.Compute.Instance for shared-CPU VMs. The reconciler does observe → ensure → sync with a tag-based adoption fallback in read so partial creates (state lost between createInstance and persist) don't duplicate instances on the next run. Plan/replace classification covers region/osId/imageId/plan/hostname/sshKeys/ipv6 as immutables; label/tags/backups/ddos/firewall are mutable in place. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
End-to-end demo wiring the new Vultr.Compute.Instance to a Cloudflare.Tunnel + proxied CNAME. cloud-init installs Caddy on :8080 and cloudflared bound to the tunnel token, so `https://<host>` flows edge → tunnel → connector → caddy. Includes a local Cloudflare.DnsRecord shim (~250 LOC, alchemy v2 has no DnsRecord resource yet) so the stack is self-contained and the shape is obvious if someone wants to land DnsRecord upstream. Verified end-to-end against a real Vultr account + ktarz.com on Cloudflare. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move applyMigrations / runSql / listSqlFiles / readSqlFile / parsePostgresOrigin from packages/alchemy/src/Neon/ to a new shared packages/alchemy/src/Postgres/ module so non-Neon Postgres-flavoured providers (next commit: Vultr Managed Postgres) can reuse them without a cross-vendor import. The Neon module's own files become thin re-export shims so existing imports (alchemy/Neon/Migrations, etc.) keep working — no public API break. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A Managed Postgres cluster resource backed by Vultr's
`/v2/databases` surface. Verified end-to-end against a real Vultr
account by deploying a hobbyist-tier (\$15/mo) cluster, running SQL
migrations from a local `./migrations/` directory against the seeded
`defaultdb`, and consuming `db.connectionUri` from a Bun app running
on a sibling Vultr.Compute.Instance behind a Cloudflare Tunnel.
Resource type:
```ts
export const Postgres = Resource<Postgres>("Vultr.Database.Postgres");
```
Outputs `host`, `port`, `user`, `password` (Redacted), `databaseName`,
`connectionUri` (Redacted), and an `origin: PostgresOrigin` matching
the shape `Cloudflare.Hyperdrive` accepts — so the cluster slots into
Hyperdrive bindings without extra plumbing, mirroring the Neon
provider.
Migrations and seed files (`migrationsDir`, `migrationsTable`,
`importFiles`) use the shared `Postgres/` helpers landed in the
previous commit. Each migration runs in a transaction with bookkeeping
so partial application is impossible.
Notable bug-fixes-while-implementing surfaced in this commit:
1. Vultr returns the status field capitalised (`"Running"`) even
though the rest of the JSON is lower-case. `waitForRunning`
normalises before comparing so the poller actually terminates.
2. `read()` falls back to a label-based lookup (`?label=…`) when
`output.databaseId` is missing — this is the recovery path Plan.ts
triggers when state shows `status: "creating"` with no persisted
attrs. Without it the engine would create a duplicate \$15/mo
cluster on the next deploy. Vultr's `tag` field is a single string
(not an array, unlike compute instances) so we rely on the
deterministic physical name from `createPhysicalName({ id })` for
identification, which works because the engine preserves
`instanceId` across runs.
Sub-resources for users / logical DBs / connection pools / read
replicas have endpoint coverage in `Database/api.ts` but no resource
wrappers yet — that's the next PR. Not required to *use* a cluster
since the admin user + default db are created automatically.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends the example to provision a Vultr.Database.Postgres cluster
alongside the existing Vultr.Compute.Instance + Cloudflare.Tunnel.
The VM now runs a Bun HTTP server (instead of Caddy) that queries the
`messages` table over `import { sql } from "bun"`. cloud-init installs
Bun, writes the app, and registers it as a systemd unit with the
cluster's `connectionUri` (Redacted -> plain via Output.map) piped in
via `Environment=DATABASE_URL=…`.
End-to-end path verified live:
```
GET https://vultr-demo.ktarz.com/
{
"host": "vultr-demo.ktarz.com",
"count": 3,
"messages": [
{ "id": 1, "body": "hello from migrations", … },
{ "id": 2, "body": "this row was seeded at deploy time", … },
{ "id": 3, "body": "vultr managed postgres works", … }
]
}
```
Two cloud-init gotchas surfaced and fixed:
- The Bun installer reads `$HOME`, which cloud-init does NOT set; combined
with `set -u` the script silently dies at "HOME: unbound variable" and
the rest of the steps (app + cloudflared) never run. Fixed by
exporting `HOME=/root` before the installer.
- `trustedIps: ["0.0.0.0/0"]` is the simplest working default for
cluster reachability from both this dev machine (for migrations) and
the freshly-booted VM (whose mainIp isn't known until after the
cluster is up). Documented as insecure-for-prod.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a TODO(upstream) header explaining that this file exists only because `alchemy/Cloudflare` doesn't ship a DNS Record resource yet, and sketches what the canonical `Cloudflare.Dns.Record` should look like (record types beyond CNAME, reuse the existing `@distilled.cloud` SDK verbs, adoption by `(zoneId, name, type)`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5acc588 to
28e7d1f
Compare
There was a problem hiding this comment.
Do you plan on building an Effect-native Platform for this? We can lift the user data into something that is contributed to with bindings so we can plug in. E.g., I was planning on adding a Process or Daemon construct that would translate to a systemd service.
export default MyProcess extends Vultr.Daemon<MyProcess>()(
"MyProcess",
{ main: import.meta.filename },
Effect.gen(function*() {
const db = yield* ...
yield* serve(myServer)
})
) {}| const tunnelHostname = yield* Config.string("TUNNEL_HOSTNAME"); | ||
| const zoneName = yield* Config.string("CLOUDFLARE_ZONE_NAME"); |
There was a problem hiding this comment.
Ideally these would be Cloudflare resources instead of config. Out of scope for this PR maybe, but integration should work via Resources and Outputs wherever possible
| const db = yield* Vultr.Database.Postgres("AppDb", { | ||
| region: "EWR", | ||
| plan: "vultr-dbaas-hobbyist-cc-1-25-1", | ||
| trustedIps: ["0.0.0.0/0"], | ||
| migrationsDir: "./migrations", | ||
| }); |
There was a problem hiding this comment.
Might want to make Vultr.PostgresDatabase instead. Vultr.Database.Postgres won't tree shake as well.
| region: "EWR", | ||
| plan: "vultr-dbaas-hobbyist-cc-1-25-1", | ||
| trustedIps: ["0.0.0.0/0"], | ||
| migrationsDir: "./migrations", |
There was a problem hiding this comment.
I'd like to see this wired up with Drizzle.Schema perhaps in a different example? Take a look at examples/cloudflare-neon-drizzle for inspiration
| const tunnelTokenPlain = Output.map(tunnel.token, Redacted.value); | ||
| const databaseUrlPlain = Output.map(db.connectionUri, Redacted.value); | ||
| const vm = yield* Vultr.Instance("HelloVm", { | ||
| region: "ewr", | ||
| plan: "vc2-1c-1gb", | ||
| osId: 1743, // Ubuntu 22.04 x64 | ||
| tags: ["app:vultr-demo"], | ||
| userData: buildUserData({ | ||
| hostname: tunnelHostname, | ||
| tunnelToken: tunnelTokenPlain, | ||
| databaseUrl: databaseUrlPlain, |
There was a problem hiding this comment.
buildUserData should return Output<Redacted<string>> and accept Input<Redacted<string>> as input. We add extra envelope encryption for Redacted values when persisting state. And this helps protect users from mistakes.
|
@sam-goodwin one thing more important, I have opened this PR on the distilled repo, should I focus first on that work, then on this one? That PR supports a lot more services from Vultr, this ones only focused on the two I'm actually using |
Adds an
alchemy/Vultrprovider with two resources, an example wiring them together end-to-end, and an extract of the Postgres helpers (applyMigrations,parsePostgresOrigin,listSqlFiles) out ofNeon/into a sharedPostgres/module so the new resource can reuse them without a cross-vendor import.Resources
Vultr.Compute.Instance— shared-CPU VMs (vc2-*). Adoption fallback inread()searches by Alchemy branding tags so a partial create (state lost betweencreateInstanceand persist) doesn't spawn a duplicate.Vultr.Database.Postgres— managed Postgres cluster. Exposeshost,port,user,password(Redacted),databaseName,connectionUri(Redacted), and anorigin: PostgresOriginmatching the shapeCloudflare.Hyperdriveaccepts.migrationsDir+importFileswork the same way they do onNeon.Project. Adoption fallback uses the deterministic physical name fromcreatePhysicalName({ id })since Vultr'stagfield is a single string (not an array).Sub-resources for users / logical DBs / connection pools / read replicas are scoped out — endpoint coverage is in
Database/api.tsbut no resource wrappers yet. Not needed to use a cluster (admin user + default db are auto-created).Verified live
A small example at
examples/vultr-tunnel/provisions both resources behind aCloudflare.Tunnel+ proxied CNAME:Path: browser → Cloudflare edge → tunnel →
cloudflaredon the Vultr VM → Bun HTTP server readingDATABASE_URL→ Vultr Managed Postgres → rows seeded by./migrations/0001_init.sql(applied by alchemy at deploy time).Workaround included: local
DnsRecordshimexamples/vultr-tunnel/src/DnsRecord.tsis a local workaround, not a piece of work to merge as-is. It exists only becausealchemy/Cloudflaredoesn't ship a DNS Record resource yet — and the moment you go off the all-Workers path (a Tunnel, an external origin, a static A record) you need one. The shim is CNAME-only, hand-rolled HTTP, customCfApiContext.Service — the bare minimum to make the example deploy.A proper
Cloudflare.Dns.Recordshould:packages/alchemy/src/Cloudflare/Dns/Record.ts(next toZone.ts)Cloudflare.Credentials.fromAuthProvider()+ the existing@distilled.cloud/cloudflareSDK'screateRecord/getRecord/updateRecord/deleteRecord(all four verbs already there).typefield, not CNAME-only.zoneId: stringorzoneName: stringand cache the resolved zone id in attrs.(zoneId, name, type)(uniquely identifies a record on CF) instead of always-creating on state loss.Happy to spin a separate small PR for this if there's interest — keeping it out of this one so the Vultr conversation doesn't tangle with a Cloudflare-namespace decision. (Same TODO is on the source file as a
TODO(upstream)block.)Bugs caught while implementing — worth flagging
statuscapitalised ("Running") even though the rest of the JSON is lower-case.waitForRunningnormalises before comparing — without that the poller loops forever and burns minutes / dollars before someone notices.$HOME. cloud-init doesn't set it, and combined withset -uthe script dies silently at "HOME: unbound variable" with no obvious failure — every subsequent step is skipped.export HOME=/rootbefore the installer fixes it (inexamples/vultr-tunnel/src/userData.ts).What I'd love a maintainer eye on
Vultr.Database.Postgresadoption strategy. I'm relying on the deterministic label (${stack}-${id}-${stage}-<hash>) forread()'s recovery path because Vultr'stagfield is a single string. This works for the realistic "create succeeded, state wasn't persisted" case (engine preservesinstanceId), but won't work if state is entirely wiped (engine generates a newinstanceId, label changes). Compute instance adoption uses multiple-tag matching since it supportstags: string[]. Open to suggestions if there's a cleaner pattern.The
@/*alias in src. I wanted to use it to avoid../../../Postgres/Migrations.ts-style imports but realisedtsc -bdoesn't rewrite paths in the emittedlib/, and external consumers using the"import"condition don't have the alias. I extracted to a sharedPostgres/module instead, which I think is the right structural fix — but the depth is still 3 levels. Curious if you've considered bundling the lib output viatsdownso aliases could be used internally.No tests yet. Pattern would mirror
test/Cloudflare/Tunnel/Tunnel.test.ts— live API hitting a realVULTR_API_KEY. Happy to add as a follow-up, or in this PR if you'd prefer.Commits
Reject as-is, scope down to just the compute side, or split into multiple PRs — happy to follow whichever direction makes review easiest.