From 36225cf0842d2d4c3a1ebb5f279f552adfe111d1 Mon Sep 17 00:00:00 2001 From: Ian Butterworth Date: Sun, 12 Apr 2026 20:51:40 -0400 Subject: [PATCH] Fix Lambda deployment: SSM permissions, cross-platform builds - template.yaml: Replace SSMParameterReadPolicy with explicit IAM statement to fix leading-slash ARN mismatch - Makefile: Cross-compile Linux x86_64 wheels via pip flags instead of requiring Docker; skip poetry export if requirements.txt exists - DEVGUIDE.md: Update deployment instructions, troubleshooting, log group names, and add token rotation docs - .samignore: Exclude .venv from SAM build context --- .samignore | 1 + DEVGUIDE.md | 48 +++++++++++++++++++++++++++++++++++++++++++----- Makefile | 6 ++++-- template.yaml | 6 ++++-- 4 files changed, 52 insertions(+), 9 deletions(-) create mode 100644 .samignore diff --git a/.samignore b/.samignore new file mode 100644 index 00000000..1d17dae1 --- /dev/null +++ b/.samignore @@ -0,0 +1 @@ +.venv diff --git a/DEVGUIDE.md b/DEVGUIDE.md index 61debd5a..cf88719b 100644 --- a/DEVGUIDE.md +++ b/DEVGUIDE.md @@ -95,7 +95,7 @@ The web service runs on AWS Lambda via AWS SAM. - AWS SAM CLI (`brew install aws-sam-cli` or `pip install aws-sam-cli`) - AWS credentials with deployment permissions -- Docker (for building Linux-compatible packages on macOS) +- Python 3.12+ venv (the Makefile uses `--platform manylinux2014_x86_64` pip flags to cross-install Linux wheels, so Docker is **not** required) ### Setup @@ -106,6 +106,12 @@ aws configure --profile julia_tagbot # region: us-east-1 ### Deployment ```bash +# Activate the venv (must be Python 3.12+ so pip markers match) +source .venv/bin/activate + +# Pre-generate requirements.txt (poetry is not available in Lambda build containers) +poetry export --extras web --without-hashes --output requirements.txt + # Production (with custom domain julia-tagbot.com) sam build && sam deploy --config-env prod \ --parameter-overrides "TagbotCommit=$(git rev-parse HEAD)" \ @@ -117,6 +123,8 @@ sam build && sam deploy \ --profile julia_tagbot ``` +> **Note**: Do NOT use `sam build --use-container`. SAM's CopySource follows symlinks in `.venv/` which don't exist inside the container, causing build failures. The Makefile's `--platform manylinux2014_x86_64 --only-binary=:all:` pip flags handle cross-compilation without Docker. + ### Configuration | File | Purpose | @@ -134,28 +142,58 @@ The GitHub token is stored in SSM Parameter Store as a SecureString at `/tagbot/ ### Troubleshooting -**Missing Python modules**: Check `poetry.lock`, try `rm -rf .aws-sam/` +**Missing Python modules on Lambda** (`No module named 'flask'`): Your local Python is likely < 3.12, so pip skips packages with `python_version >= "3.12"` markers. Use the `.venv` (Python 3.13) or ensure `--python-version 3.12` is passed to pip. + +**Invalid ELF header on Lambda** (`_rust.abi3.so: invalid ELF header`): Native extensions were built for macOS. The Makefile passes `--platform manylinux2014_x86_64 --only-binary=:all:` to pip, which installs Linux wheels. If this still happens, run `rm -rf .aws-sam/` and rebuild. -**Build issues**: `sam build --use-container` to build in a Docker container matching the Lambda runtime +**`sam build --use-container` fails with `.venv`**: SAM's CopySource tries to follow `.venv/bin/python3` symlinks inside the container where they don't exist. `.samignore` does not help. Use the default `sam build` (without `--use-container`) instead. + +**Stale cached state**: Try `rm -rf .aws-sam/` and rebuild. ### Checking Logs +Log group names include CloudFormation-generated suffixes. Discover them first: + +```bash +aws logs describe-log-groups --profile julia_tagbot --region us-east-1 \ + --log-group-name-prefix /aws/lambda/TagBotWeb-prod \ + --query 'logGroups[*].logGroupName' --output text +``` + +Then query with the actual names: + ```bash # Recent API function logs (last 5 min) aws logs filter-log-events --profile julia_tagbot --region us-east-1 \ - --log-group-name /aws/lambda/TagBotWeb-prod-api \ + --log-group-name /aws/lambda/TagBotWeb-prod-ApiFunction-XXXX \ --start-time $(($(date +%s) * 1000 - 300000)) \ --query 'events[*].message' --output text # Reports function logs aws logs filter-log-events --profile julia_tagbot --region us-east-1 \ - --log-group-name /aws/lambda/TagBotWeb-prod-reports \ + --log-group-name /aws/lambda/TagBotWeb-prod-ReportsFunction-XXXX \ --start-time $(($(date +%s) * 1000 - 300000)) \ --query 'events[*].message' --output text ``` Or view in [AWS Console](https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logsV2:log-groups). +### Updating the GitHub Token + +The reports Lambda reads the GitHub PAT from SSM at runtime and caches it globally. After updating the token, you must force a cold start: + +```bash +# Update the token +aws ssm put-parameter --profile julia_tagbot --region us-east-1 \ + --name /tagbot/github-token --type SecureString \ + --value "ghp_XXXXX" --overwrite + +# Force cold start (updates description to invalidate warm instances) +aws lambda update-function-configuration --profile julia_tagbot --region us-east-1 \ + --function-name TagBotWeb-prod-ReportsFunction-XXXX \ + --description "Force cold start $(date +%s)" +``` + --- ## Agent Guidelines diff --git a/Makefile b/Makefile index 051774b1..a9903ce8 100644 --- a/Makefile +++ b/Makefile @@ -1,8 +1,10 @@ .PHONY: test test-docker publish pytest black flake8 mypy build-ApiFunction build-ReportsFunction build-ApiFunction build-ReportsFunction: - poetry export --extras web --without-hashes --output requirements.txt - pip install -r requirements.txt -t $(ARTIFACTS_DIR)/ + test -f requirements.txt || poetry export --extras web --without-hashes --output requirements.txt + pip install -r requirements.txt -t $(ARTIFACTS_DIR)/ \ + --platform manylinux2014_x86_64 --only-binary=:all: \ + --python-version 3.12 --implementation cp cp -r tagbot $(ARTIFACTS_DIR)/ test: diff --git a/template.yaml b/template.yaml index 2eda99f9..adee388a 100644 --- a/template.yaml +++ b/template.yaml @@ -65,8 +65,10 @@ Resources: Variables: GITHUB_TOKEN_PARAM: !Ref GithubTokenParam Policies: - - SSMParameterReadPolicy: - ParameterName: !Ref GithubTokenParam + - Statement: + - Effect: Allow + Action: ssm:GetParameter + Resource: !Sub arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter${GithubTokenParam} Outputs: ApiUrl: