Pavel Tisner

AI_ASSIST.md

@@ -6,20 +6,45 @@
 
 ## The prompt I gave
 
-<!-- Paste the exact prompt you gave an LLM (ChatGPT, Claude, Copilot, etc.). -->
-
-TODO: paste your prompt here.
+My Python pipeline runs locally and inside Docker on my Apple Silicon Mac. I pushed the image to Azure Container Registry, but Azure Container Apps Job fails with: "no child with platform linux/amd64 in index". What does this mean and how should I fix it?
 
 ## The code or suggestion it returned
 
-<!-- Paste the suggestion verbatim — code, shell commands, or both. -->
+ChatGPT explained that the issue was not in my Python code. The problem was the Docker image platform. Because I built the image on an Apple Silicon Mac, the image was built for:
+
+linux/arm64
+
+But Azure Container Apps expected:
+
+linux/amd64
+
+The suggested fix was to rebuild and push the image with an explicit platform:
+
+docker buildx build \
+  --platform linux/amd64 \
+  -t hyfregistry.azurecr.io/week6-pavel-tisner-pipeline:latest \
+  --push .
 
-```text
-TODO: paste the AI output here.
-```
+It also suggested checking the job logs with the container name:
+
+az containerapp job logs show \
+  --name job-week6-pavel-tisner \
+  --resource-group rg-hyf-data \
+  --container job-week6-pavel-tisner
 
 ## What I changed after reviewing it
 
-<!-- Describe what you accepted, rejected, or modified, and why. -->
+I accepted the explanation because it matched the error message exactly. The pipeline and Dockerfile had already worked locally, so the most likely issue was the image architecture, not the application code.
+
+I rebuilt and pushed the image using docker buildx build --platform linux/amd64 --push. After that, the Azure Container Apps Job was created successfully.
+
+I verified the result by checking the job logs. The deployed container ran the pipeline successfully:
+
+starting pipeline
+uploaded blob raw/weather/2026-06-15.json
+wrote 3 rows to postgres
+pipeline complete (today=2026-06-15)
+
+I also checked Azure Blob Storage and PostgreSQL separately. The blob raw/weather/2026-06-15.json existed in the raw container, and the PostgreSQL table in my schema dev_pavel_tisner contained 3 rows. I reran the pipeline and confirmed that the row count stayed 3, so the ON CONFLICT DO UPDATE upsert worked correctly.
 
-TODO: describe your review here.
+I did not change the Python pipeline because of this AI suggestion. The fix was only in the Docker build and push step: I had to build the image for the platform Azure Container Apps expected.

Dockerfile

@@ -11,11 +11,10 @@ FROM python:3.11-slim
 
 WORKDIR /app
 
-# TODO Task 4: copy requirements.txt (must appear before any COPY src command)
+COPY requirements.txt .
 
-# TODO Task 4: install dependencies with pip
+RUN pip install --no-cache-dir -r requirements.txt
 
-# TODO Task 4: copy the src/ folder
+COPY src/ src/
 
-# TODO Task 4: set the CMD to run the pipeline (python -m src.pipeline)
-CMD ["python", "-c", "raise SystemExit('Dockerfile not finished: Task 4 still pending')"]
+CMD ["python", "-m", "src.pipeline"]

README.md

@@ -53,23 +53,18 @@ az login --use-device-code --tenant 07a14c4e-d88c-42f7-83b3-13af7e57ff3d
 ```bash
 python -m venv .venv && source .venv/bin/activate
 pip install -r requirements.txt
-cp .env.example .env  # fill in real connection strings, never commit them
+cp .env.example .env  # fill in real connection strings, never commit them; POSTGRES_URL must include sslmode=require
 set -a && source .env && set +a
 python -m src.pipeline
 ```
 
-## Verifying your deployment (Task 5)
+## Verification
 
-After deploying the Container App Job and triggering a run, capture proof:
+The Azure Container App Job `job-week6-pavel-tisner` was deployed and triggered successfully.
 
-1. Open the Azure portal, find your Container App Job, open the **Execution
-   history** blade.
-2. Screenshot the most recent successful run.
-3. Save the screenshot to `docs/`.
-4. Replace this whole section with one called `## Verification` and embed
-   your screenshot using a Markdown image link. The grader looks for the
-   `## Verification` heading and a `![alt](docs/your-file.png)` reference
-   pointing at the image you committed.
+The latest execution completed the pipeline and wrote raw JSON to Azure Blob Storage and 3 rows to Azure PostgreSQL.
+
+![Execution history](docs/execution_history.png)
 
 ## Check your score locally
 

docs/execution_history.txt

@@ -0,0 +1,68 @@
+Local pipeline execution
+
+Command:
+python -m src.pipeline
+
+Result:
+INFO starting pipeline
+INFO uploaded blob raw/weather/2026-06-15.json
+INFO wrote 3 rows to postgres
+INFO pipeline complete (today=2026-06-15)
+
+Blob verification:
+Command:
+az storage blob list --connection-string "$AZURE_STORAGE_CONNECTION_STRING" --container-name raw --prefix raw/weather/ -o table
+
+Verified blob:
+raw/weather/2026-06-15.json
+Content Type: application/json
+
+Postgres verification:
+Schema:
+dev_pavel_tisner
+
+Table:
+weather_readings
+
+Query result:
+count: 3
+('amsterdam', 2026-06-15 06:00:00+00:00, 18.5, 72)
+('utrecht', 2026-06-15 06:00:00+00:00, 17.8, 75)
+('rotterdam', 2026-06-15 06:00:00+00:00, 19.1, 70)
+
+Rerun behavior:
+The pipeline was run more than once on the same day. The row count stayed 3, confirming that ON CONFLICT DO UPDATE prevents duplicates.
+
+Docker local execution
+
+Build command:
+docker build -t week6-pavel-tisner-pipeline:local .
+
+Run command:
+docker run --rm \
+  -e POSTGRES_URL="$POSTGRES_URL" \
+  -e AZURE_STORAGE_CONNECTION_STRING="$AZURE_STORAGE_CONNECTION_STRING" \
+  week6-pavel-tisner-pipeline:local
+
+Result:
+INFO starting pipeline
+INFO uploaded blob raw/weather/2026-06-15.json
+INFO wrote 3 rows to postgres
+INFO pipeline complete (today=2026-06-15)
+
+Azure Container Apps Job execution
+
+Job name:
+job-week6-pavel-tisner
+
+Execution:
+job-week6-pavel-tisner-79lqj89
+
+Replica:
+job-week6-pavel-tisner-79lqj89-7zmr8
+
+Logs:
+starting pipeline
+uploaded blob raw/weather/2026-06-15.json
+wrote 3 rows to postgres
+pipeline complete (today=2026-06-15)

requirements.txt

@@ -6,8 +6,5 @@
 # The pipeline needs the Azure Blob SDK and a Postgres driver. Add them below
 # with explicit pins.
 
-# TODO: pin azure-storage-blob (uncomment and add a version)
-# azure-storage-blob==
-
-# TODO: pin psycopg2-binary (uncomment and add a version)
-# psycopg2-binary==
+azure-storage-blob==12.19.0
+psycopg2-binary==2.9.9

src/pipeline.py

@@ -13,12 +13,17 @@
 - Container Job:    Data Track/Week 6/week_6__5_container_apps_jobs.md
 """
 
+import json
 import logging
 import os
+from contextlib import closing
 from datetime import date
+import psycopg2
+from azure.storage.blob import BlobServiceClient
 
 logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
 logger = logging.getLogger(__name__)
+logging.getLogger("azure").setLevel(logging.WARNING)
 
 # TASK 3 hint: quiet the Azure SDK so its DEBUG output does not drown your own
 # pipeline logs. The right call lives in Chapter 5 (Viewing logs).
@@ -37,9 +42,22 @@ def get_config() -> dict:
 
     Raise RuntimeError with a clear message if a required variable is missing.
     """
-    raise NotImplementedError(
-        "Task 3: read POSTGRES_URL and AZURE_STORAGE_CONNECTION_STRING from os.environ"
-    )
+    postgres_url = os.environ.get("POSTGRES_URL")
+    azure_storage_connection_string = os.environ.get("AZURE_STORAGE_CONNECTION_STRING")
+
+    if not postgres_url:
+        raise RuntimeError("Missing required environment variable: POSTGRES_URL")
+
+    if not azure_storage_connection_string:
+        raise RuntimeError(
+            "Missing required environment variable: AZURE_STORAGE_CONNECTION_STRING"
+        )
+
+    return {
+        "postgres_url": postgres_url,
+        "azure_storage_connection_string": azure_storage_connection_string,
+        "source_name": os.environ.get("SOURCE_NAME", "weather"),
+    }
 
 
 def fetch_records() -> list[dict]:
@@ -49,7 +67,28 @@ def fetch_records() -> list[dict]:
     one dict with a stable key set (for example: station, timestamp,
     temperature_c, humidity_pct).
     """
-    raise NotImplementedError("Task 3: return a list of at least one record")
+    today = date.today().isoformat()
+
+    return [
+        {
+            "station": "amsterdam",
+            "timestamp": f"{today}T06:00:00Z",
+            "temperature_c": 18.5,
+            "humidity_pct": 72,
+        },
+        {
+            "station": "utrecht",
+            "timestamp": f"{today}T06:00:00Z",
+            "temperature_c": 17.8,
+            "humidity_pct": 75,
+        },
+        {
+            "station": "rotterdam",
+            "timestamp": f"{today}T06:00:00Z",
+            "temperature_c": 19.1,
+            "humidity_pct": 70,
+        },
+    ]
 
 
 def upload_raw_to_blob(records: list[dict], blob_conn_str: str, source: str) -> str:
@@ -62,7 +101,23 @@ def upload_raw_to_blob(records: list[dict], blob_conn_str: str, source: str) ->
     teacher has pre-created it). Overwrite if the blob already exists so
     same-day reruns succeed.
     """
-    raise NotImplementedError("Task 1 + Task 3: upload records to blob storage")
+    blob_name = f"raw/{source}/{date.today().isoformat()}.json"
+
+    blob_service_client = BlobServiceClient.from_connection_string(blob_conn_str)
+
+    container_client = blob_service_client.get_container_client("raw")
+
+    raw_json = json.dumps(records, ensure_ascii=False, indent=2)
+    raw_bytes = raw_json.encode("utf-8")
+
+    container_client.upload_blob(
+        name=blob_name,
+        data=raw_bytes,
+        overwrite=True,
+        content_type="application/json",
+    )
+
+    return blob_name
 
 
 def write_to_postgres(records: list[dict], postgres_url: str) -> int:
@@ -78,7 +133,55 @@ def write_to_postgres(records: list[dict], postgres_url: str) -> int:
 
     See Chapter 4 for the connection-and-cursor pattern this is based on.
     """
-    raise NotImplementedError("Task 2 + Task 3: insert rows into Azure Postgres")
+    with closing(psycopg2.connect(postgres_url)) as conn:
+        with conn:
+            with conn.cursor() as cur:
+                cur.execute("""
+                    CREATE SCHEMA IF NOT EXISTS dev_pavel_tisner
+                """)
+
+                cur.execute("""
+                    SET search_path TO dev_pavel_tisner
+                """)
+
+                cur.execute("""
+                    CREATE TABLE IF NOT EXISTS weather_readings (
+                        id SERIAL PRIMARY KEY,
+                        station TEXT NOT NULL,
+                        timestamp TIMESTAMPTZ NOT NULL,
+                        temperature_c DOUBLE PRECISION NOT NULL,
+                        humidity_pct INTEGER NOT NULL,
+                        ingested_at TIMESTAMPTZ DEFAULT NOW(),
+                        UNIQUE(station, timestamp)
+                    )
+                """)
+
+                for record in records:
+                    cur.execute(
+                        """
+                        INSERT INTO weather_readings (
+                            station,
+                            timestamp,
+                            temperature_c,
+                            humidity_pct
+                        )
+
+                        VALUES (%s, %s, %s, %s)
+                        ON CONFLICT (station, timestamp) DO UPDATE SET
+                            temperature_c = EXCLUDED.temperature_c,
+                            humidity_pct = EXCLUDED.humidity_pct,
+                            ingested_at = NOW()
+                        """,
+
+                        (
+                            record["station"],
+                            record["timestamp"],
+                            record["temperature_c"],
+                            record["humidity_pct"],
+                        ),
+                    )
+
+    return len(records)
 
 
 def run() -> None: