diff --git a/site/content/posts/ai-approved-the-resource-i-paid-the-bill.md b/site/content/posts/ai-approved-the-resource-i-paid-the-bill.md new file mode 100644 index 0000000..3de8e01 --- /dev/null +++ b/site/content/posts/ai-approved-the-resource-i-paid-the-bill.md @@ -0,0 +1,81 @@ +--- +title: "AI Approved the Resource. I Paid the Bill." +description: "How I burned through $1,000 of Azure credit on a project with zero customers — and why cost governance is the human-in-the-loop requirement I didn't see coming." +pubDate: 2026-06-09 +slug: "ai-approved-the-resource-i-paid-the-bill" +tags: + [ + "Azure", + "AI tooling", + "cost governance", + "lessons learned", + "building-in-public", + "AccessiShield", + ] +draft: true +featured: false +series: + id: "aiagentminder-codeveloping" + title: "Co-developing with AIAgentMinder" + order: 8 +--- + +Here's a number: $1,000. That's the Azure credit I started the AccessiShield project with, courtesy of a Microsoft promotion for solo developers. And here's another number: $300-plus per month. That's roughly what the SQL infrastructure I provisioned for a zero-customer MVP was costing to run. + +I'm going to tell you how those two numbers met, because the story is more instructive than embarrassing — though it is also a little embarrassing, and I think the honest version is the useful one. + +## The buffer that made the blind spot possible + +When you start a project with $1,000 of free credit, something shifts in how you think about costs. It's the developer equivalent of "it's on the company card." The money is real — it has real opportunity cost, it could have funded experimentation on other projects for months — but it doesn't *feel* real in the moment. You approved a sprint, the sprint ran, resources got created, and the bill went somewhere you weren't watching. + +I knew the credit was there. I knew the project was early-stage. I did not know, at any point during the build, what AccessiShield was costing per month to run. That sentence is the whole problem. + +## What the AI provisioned and why it wasn't wrong + +When I kicked off the AccessiShield build, the spec called for a proper production-grade SaaS: authentication, Stripe billing, customer dashboard, a scanning product underneath. The AI — working through AIAgentMinder's sprint workflow — proposed an architecture that fit those requirements. SQL on Azure, with a plan appropriate for a real application with real concurrent users and real availability requirements. + +Nothing about that decision was careless. Given the spec, it was correct. A proper SaaS needs a proper database, and the AI chose a tier that would handle the load the spec described. + +The problem is that the spec described a finished product. What I was actually building was an MVP with me as the only test user. The gap between those two things — in terms of actual database load — was enormous. A single developer poking at a dashboard doesn't need the same SQL tier as a product with paying customers. But the spec didn't say "optimize for zero users." So the AI didn't. + +That's not a failure of the model. It's a failure of the question I asked it. + +## The missing gate + +AccessiShield ran through the same sprint workflow I use for everything. Spec, plan, implement, PR, review, merge. The review step covered code quality, architecture, test coverage. It did not cover the line item on next month's Azure invoice. + +There was no moment in that workflow where anyone — me or the AI — asked: *what does this infrastructure cost per month at MVP scale?* It was simply not a question the process required. An infra PR landed, I read the code, it looked right, I approved it. "Approve" on a Bicep template and "I understand the monthly cost of what I just approved" turned out to be very different things. + +By the time I noticed the burn rate, a meaningful chunk of the credit was gone. The resources had been running for weeks, doing essentially nothing, at a rate that made no sense for the stage the project was at. I mothballed everything once the picture was clear, but the credit doesn't come back. + +## What the guardrail looks like now + +The fix isn't complicated. It just has to be explicit, because "I'll notice if something looks expensive" is not a process — it's optimism. + +Three things I've added: + +**Budget alerts before workloads.** Any new resource group that's going to run real infrastructure gets an Azure budget alert configured before the first resource is provisioned. Not after. The alert threshold is set conservatively — what I'd actually expect to spend at the current stage of the project, not what I might spend at maturity. If it fires, that's a conversation, not an emergency, and those are much better to have early. + +**A cost question at spec time.** My AIAgentMinder sprint spec template now includes a standing prompt: *"What is the estimated monthly cost of this infrastructure at the current usage level (users, requests, storage)?"* The AI fills this in as part of writing the spec. I read it before the sprint starts. This sounds obvious in retrospect. It wasn't in the workflow before. + +**SKU justification in infra PRs.** Any PR that creates or modifies compute or storage resources has to include the SKU, the estimated monthly cost at current scale, and a one-line justification for why that tier is appropriate now. This is a small addition to the PR template and it takes about two minutes to fill in. It would have caught the AccessiShield situation immediately — "SQL Business Critical, $310/month, justification: handles 500 concurrent users" is a sentence that stops a review dead when the project has zero users. + +## The principle behind the guardrail + +AI has no financial skin in the game. It will provision what is technically correct for the requirements you gave it, and it will do so confidently, because it's right. It doesn't know you have $1,000 of credit and an expectation that it'll last two years. It doesn't know you're the only user. It doesn't know that "production-grade" and "appropriate for this moment" are different problems. + +Cost-consciousness has to come from the human side of the loop. Not as an afterthought when the bill lands, but as a first-class question built into the workflow at spec time. The AI is optimizing for what you asked it to optimize for. If cost-appropriateness isn't in the spec, it's not in the output. + +I'd spent months thinking carefully about where humans need to stay in the loop: market judgment, audience sensing, the unglamorous account setup work. I had a tidy framework. Cost governance wasn't in it, because I'd never been burned by it before. Now it is. + +## What it cost and what it bought + +The credit is gone. That's the real loss — not the money itself, but the runway it represented for future experiments. Cloud-based projects I might have tried, infrastructure I might have spun up to play with, all now measured against actual spend instead of a comfortable buffer. + +What it bought was a lesson I won't need to learn twice, a template I can start the next product from at week eight, and a workflow with a guardrail in it that should have been there from the start. + +Some things you only learn by paying for them. This was one of them. + +--- + +*The AccessiShield SaaS starter template — auth, billing, email, CI/CD, everything except the product — is sitting on a shelf if I ever find the right thing to put on top of it.* diff --git a/site/content/posts/aiagentminder-v5-the-autonomy-retreat.md b/site/content/posts/aiagentminder-v5-the-autonomy-retreat.md new file mode 100644 index 0000000..9111b2c --- /dev/null +++ b/site/content/posts/aiagentminder-v5-the-autonomy-retreat.md @@ -0,0 +1,132 @@ +--- +title: "Mapping the Loop: Where AI Earns Its Keep and Where You Don't Get to Leave" +description: "Four months of building a local dev ecosystem where the repos talk to each other — an AI-run business experiment that mapped the leverage boundaries, a project-management layer that now describes itself to new projects, and a governance framework that got sharper by deciding what not to automate." +pubDate: 2026-05-28 +slug: "aiagentminder-v5-the-autonomy-retreat" +tags: + [ + "AIAgentMinder", + "Claude Code", + "AI tooling", + "building-in-public", + "AI governance", + "developer-productivity", + ] +draft: true +featured: false +series: + id: "aiagentminder-codeveloping" + title: "Co-developing with AIAgentMinder" + order: 7 +--- + +Here's the surface version of the last four months. I shipped a working SaaS — landing page, secure login, Stripe billing, transactional email, customer dashboard, the whole stack. I stood up a small fleet of monitoring bots that report to a Discord channel. I took [AIAgentMinder](https://github.com/lwalden/AIAgentMinder) from its first scrappy incarnation to version 5. I ran a local-AI audio proof of concept that actually worked. I started building a game. Across roughly two dozen repos, most of them small, the commit graphs look like someone who got a lot done in his off hours. + +I did. But the most interesting output wasn't any of those repos. It was a clearer map of where AI actually creates leverage — and where I genuinely can't leave the loop. + +That map matters because it changes how you build. Once you know which parts compress and which don't, you stop trying to automate the wrong things. The repos that came out of this stretch are more coherent as a system than anything I'd built before — and the clearest evidence of that is a small piece of infrastructure I added recently that made me realize the ecosystem had become self-describing. + +## Where AIAgentMinder was, and where it is + +AIAgentMinder didn't come from nowhere. I spent the first weeks of January working with vanilla Claude Code, and the friction of those early sessions — losing the thread between days, re-explaining the project every morning, watching it wander off the plan — is what told me what I actually wanted to build. By February there was a tool. + +When I [last wrote about the framework](/posts/aiagentminder-evolution-building-on-quicksand), it was at v1.2 and I was daydreaming about it becoming pure markdown — no runtime, no hooks, just rules. The thesis of that post was *design for your own obsolescence*: every feature I'd built that duplicated something Claude Code shipped natively eventually got deleted, and that was fine, because the ecosystem was maturing. + +That thesis held. It just got more interesting. + +What **stayed the same** is the spine: opinionated governance the platform won't ship for you. A planning interview that turns a rough idea into a brief. Decision logging so you stop re-debating settled architecture. A sprint state machine with approval gates. Mandatory tests before a PR. Scope enforcement that catches Claude cheerfully building things nobody asked for. That core has survived every version since v0.5, because it *extends* the model instead of patching around it. + +What **changed** is that I got sharper about the difference between automation I wanted and automation I could actually use — and the clearest lesson there came from the feature I was most excited about. + +## The feature I evolved instead of automating + +Somewhere around v4.4 I built context cycling. The motivation is real and well-documented: long context degrades model quality. As you pile tokens into the window, output gets worse — and not at the very end, but well before the limit. Chroma's testing of 18 frontier models found [every one of them degrades as input grows](https://www.trychroma.com/research/context-rot); a model with a 200K window can measurably slip at 50K. I'd *felt* this in long sessions — the slow drift where Claude starts losing the thread — before I had a name for it. + +So I built a system to get ahead of it. The idea was seductive: a session that notices it's getting heavy, wraps itself up, hands off its state, and starts fresh — automatically, so I could kick off long runs and check on them from my phone while I did something else. A `SessionEnd` hook built a continuation file, a `SessionStart` hook injected it into the next session, and a `PreToolUse` hook blocked tools once the context crossed a threshold to *force* the cycle. On paper, beautiful. Autonomous quality control. + +In practice it ran into the hard edges of how I actually work — and the biggest edge was that I like to *start a session on my PC and continue it on my phone*. Full autonomy fought that pattern at every turn. The cycle depended on slash commands I can't issue from the mobile app. When a session self-terminated to restart itself, it disconnected the very mobile view I was using to keep an eye on it. The whole protocol was dormant on Claude Code's web client. None of these were Claude Code doing anything wrong; they're just the shape of the platform, and the fully hands-off version I'd imagined didn't fit inside that shape. + +So I made a judgment call: what did I actually want *more* — autonomy I couldn't really use across my own devices, or a reliable nudge at the right moment plus a clean way to hand off? I picked the second, and I'm genuinely happy with where it landed. + +Here's what it became. A `Stop` hook watches the context budget and, when I cross the threshold, prints one advisory line: you're over, here are your options. Then *I* decide. Usually I run `/aiagentminder:handoff`, which writes a compact "Next Session" note into [Claude Code's native memory](https://code.claude.com/docs/en/memory), commit, and `/exit`. Then I open a fresh session — `claude --agent ` if I was mid-sprint, or just plain `claude` otherwise — and my first prompt is a single word: `resume`. Native memory carries the "Next Session" block forward into a clean, near-empty window, and the new session picks up exactly where the last one stopped. + +That's the pattern of v5 in a sentence: **I stopped chasing autonomy I couldn't actually use, and designed for the human who's actually there.** The warning fires at the right time, the handoff is one command, and the resume is one word. It's the part of the whole system I'm most quietly pleased with. + +## The two deletions the platform earned + +Context cycling evolved. Two other systems came out since v1.2, both for the same happy reason — the platform grew up and absorbed them: + +| What came out | Why | +| --- | --- | +| The npm installer and CLI | Claude Code shipped a real [plugin system](https://code.claude.com/docs/en/discover-plugins). AIAgentMinder is now a plugin you install and update with `/plugin`, not an npm package I have to maintain. | +| The correction-capture hook | Claude Code's native auto-memory now captures repeated mistakes and project habits across sessions on its own. My shell hook on every tool call was redundant weight. | + +This is the [obsolescence story](/posts/aiagentminder-v070-native-memory-migration) from last time, still playing out: every piece of AIAgentMinder that duplicates a native capability eventually goes, and that's the system working as intended. + +## The project-management workflow whose main output is deletion + +If you build across two dozen repos, your scarcest resource isn't compute or even time. It's attention. So the highest-leverage thing I built this stretch isn't a feature in any one app — it's a project-management layer that sits above all of them and, mostly, tells me what *not* to do. + +It runs on the same question I [used to cull my n8n hub](/posts/ai-learning-curve-n8n-automation-scaling-back): *if this project stopped tomorrow, would I actually notice, and would I care?* Applied across a portfolio, monthly, the answers are clarifying and a little brutal. In the last review it told me to archive a crypto project I had no domain urgency for, mothball the SaaS once it had no audience, pause a social-ops toolkit until a project needed it, merge a trading experiment into a consolidated codebase, and scale back the automation hub again. + +And then it did the one *additive* thing that mattered: it promoted a game project to my primary focus. That's the real function of good portfolio governance. It's not a dashboard that makes you feel busy. It's a mechanism for deciding, on purpose, where your limited attention goes. Most of its output is subtraction, and subtraction is the point. + +## What the ecosystem experiment actually built + +Here's the thing the commit count doesn't show. The repos have started to feed each other, and that compounding is the most valuable thing I've built. + +The clearest example is a small HTTP service that wraps my Claude *subscription* behind an API — [your subscription is an API you're probably not using](/posts/claude-subscription-as-api-cli-first-pattern). It started as a one-off so a single project could call Claude without paying per token. Now multiple projects call it. The monitoring bots use it for triage; a trading experiment uses it for analysis. One boring, single-purpose service, reused everywhere. + +The bots themselves prove that maturity buys speed. In a couple of evenings I stood up a handful of read-only monitors — they watch public sources, dedupe, and report to a Discord channel, and the daily digest asks Claude (through that gateway) to triage the day's signals by what actually matters. A couple of *evenings*. Two years ago that's a couple of weeks. The difference isn't that the AI got smarter; it's that I finally had governance, reusable infrastructure, and credentials plumbing that didn't make me start from zero. + +Then there's the ops toolkit: a set of agents for social publishing and content ops, exposed two ways — a CLI and an MCP server. *This very site* uses it to draft and post social updates; the note that announces this post on Bluesky goes out through that other project's tooling, wired into my workflow here. One repo's work becomes another repo's capability. + +The newest piece is the one I find most satisfying, because it closes the loop on the whole system. I now run a small local MCP server — a project-management registry — that knows about every service in the ecosystem: what each one does, what port it runs on, how to reach it, and what it prefers to consume. When I start working on any project, Claude can query that registry before recommending an integration and find out: *what's the preferred way to make an AI call around here?* The answer comes back: use claude-gateway. *What handles automation workflows?* n8n-automation-hub. *Social publishing?* The ops MCP server. + +That's not clever engineering. It's a boring preferences file with an MCP interface. But it means the ecosystem has become *self-describing*: a new project gets integration decisions that match the existing architecture, without me having to re-explain any of it. The governance I've been building at the sprint level, applied to the infrastructure level. I actually used it to fact-check this article — confirmed the services it mentions are still live and the descriptions are current. That's a first. + +The shape that keeps winning is the same one I landed on with n8n: small, boring, single-purpose, one clear consumer. Those are the pieces that compose. The clever, do-everything systems are the ones I keep paring back. + +## What the AI-run business experiment actually measured + +The SaaS deserves a clear-eyed look, because it wasn't just a product — it was a deliberate experiment in AI autonomy, and it returned useful data. + +The premise: could an AI *run a business*? I gave it a starting budget and a two-year horizon and told it to pick a product — I didn't even specify software — build it, and handle as much of the lifecycle as it could, only pulling me in when it genuinely had to. It chose a Shopify accessibility-compliance scanner, and it built the whole thing: marketing site, authentication, Stripe subscriptions, transactional email, a dashboard, and the scanning product underneath. It works. It's real software. + +The product found no market, and that's beside the point. What the experiment actually measured was *where in the business lifecycle AI creates leverage versus where it needs a human in the loop*. + +The leverage parts were real. AI compressed the build dramatically — far less effort than coding it solo from scratch. Architectural decisions, code generation, boilerplate infrastructure, even copy drafts — all faster than anything I could do unassisted. If you quantify "build effort," AI takes a large fraction of it. + +The essential-human parts were equally clear. Every account registration, every auth token, every OAuth app registration, every judgment call about who this was for and whether they'd pay — those required me. The thing it most needed a human for, audience-product fit, was the thing it got wrong. The model had no way to discover that the people it was targeting either didn't know they had the problem or already had a solution they weren't going to switch away from. That's not a research gap AI closes by reading more docs. It's a market-sensing judgment that comes from being a person in the world. + +There was a third category I didn't anticipate: **cost governance**. I had a significant Azure credit buffer when I started, which meant I wasn't watching the bill closely. The AI provisioned technically correct infrastructure — SQL resources on plans that were wildly over-provisioned for a zero-customer MVP. I approved the work without reviewing what I was approving in dollar terms. The credit burned through faster than I expected, and the resources were running for a while before I noticed. That's a failure mode I hadn't thought to put in the "human required" column, and I should have. The full story deserves its own post — but the short version is: "I have credit" is not a substitute for "I reviewed this bill," and approving an AI's infrastructure PR is not the same as understanding its monthly cost. + +So: AI owns the *build*. Human owns the *why*, the *who*, and the *bill*. + +The artifact that came out wasn't a customer. It was a template — all that plumbing, auth, billing, email, CI/CD, is now a reusable SaaS starter. The next product I want to build begins at week eight instead of week zero. And the cleaner map of where I belong in the loop is probably worth more than the template. + +## Where might the loop shrink next? + +The one gap that chafed most consistently — the seam between "AI can do this" and "human has to sit here and type" — was account setup. Generating and rotating auth tokens. Registering OAuth apps. Copy-pasting credentials out of sites with no API. Claude Code's [Chrome integration](https://code.claude.com/docs/en/chrome) is still young and constrained by design — it can't enter credentials for you, has no memory across sessions, and that's mostly the right call for security reasons. But browser automation is maturing fast. + +My read: credential bootstrapping is the next thing to compress. Not credential *storage* — that's been solved — but the ceremony of walking through a service's web UI to create the credential in the first place. As browser automation becomes safer and more reliable, that ceremony becomes scriptable. When it does, the last major gap between "kick it off and check in later" and "sit here while this happens" closes for a wide class of projects. + +What I don't expect to compress: taste, judgment, and market sensing. Those aren't harder versions of the same problem. They're different problems. + +## The friction nobody puts in the demo videos + +A quick honesty note, because the highlight reels lie by omission. The hard parts of AI-assisted solo dev aren't the coding. They're those seams between systems: setting up accounts, generating and rotating auth tokens, registering OAuth apps, copy-pasting data out of sites that actively block automation. I ended up building a whole Bitwarden-backed strategy just to get credentials to agents safely. The bottom line: I can't glance at my phone once a day and watch the empire run itself. The work is real work. It's *leveraged* work, sometimes dramatically — but it's hands-on. + +## Where the freed-up attention went + +I'll close with the project I'm least going to tell you about, because it's the one I care most about. + +All that subtraction — the archived repo, the mothballed SaaS, the paused toolkit, the culled automations — was, in the end, in service of clearing a runway. What it cleared it for is a game. I bought a domain. There's no store page, no website, no trailer, not a single screenshot to show you, and I'm not going to describe the design here. Talking up an unannounced game you haven't shown anyone is how you spend your enthusiasm before you've earned the right to. The successful local-AI audio POC I mentioned at the top feeds it. That's all I'll say. + +It's also the place where "AI does the work" is *least* true. A game needs taste and design and the kind of slow judgment that doesn't compress into a couple of evenings. Which, after four months, feels exactly right. The governance, the reusable infrastructure, the self-describing ecosystem — all of it exists to get the undifferentiated work out of my way so I can spend my actual attention on the thing that needs a human. + +That's the scorecard. Not "look how much the AI let me build." More like: I learned where the leverage lives, where the loop holds, and what that clarity is worth when you're building with limited time and unlimited ambition. + +--- + +*AIAgentMinder is open source under MIT. If you use Claude Code for projects that span more than a few sessions, it's at [github.com/lwalden/AIAgentMinder](https://github.com/lwalden/AIAgentMinder).*