Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 73 additions & 17 deletions .claude/skills/run-e2e-pixel-sweep/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,25 @@
---
name: run-e2e-pixel-sweep
description: >-
Run the repeatable OpenLoop 3-emulator E2E sweep (Pixel 6, Pixel 8, Pixel 10 Pro Fold) using
the canonical imported test video: build once, then per device cold-boot → scripted
import → trim → speed → all four loop directions (real reverse previews) → B&W look → save →
share sheet → gallery playback, followed by a programmatic output-quality gate (per-half fps,
mirror SSIM, freeze/green/black scans) and a logcat scan. Use whenever the user says "pixel
sweep", "run the 3-device e2e", "/run-e2e-pixel-sweep", "regression-test the boomerang flow
on the emulators", or wants proof a media-pipeline change didn't break the import→save path
on Google devices. Scripts compute every gesture from uiautomator bounds, so they run
unmodified on any screen size.
Run the repeatable OpenLoop 4-emulator E2E sweep (Pixel 6, Pixel 8, Pixel 10 Pro Fold, Pixel 8
API 34) using the canonical imported test video: build once, then per device cold-boot →
scripted import → trim → speed → all four loop directions (real reverse previews) → B&W look →
save → share sheet → gallery playback, followed by a programmatic output-quality gate (per-half
fps, mirror SSIM, freeze/green/black scans) and a logcat scan. Use whenever the user says
"pixel sweep", "run the 4-device e2e", "/run-e2e-pixel-sweep", "api 34 fgs regression",
"samsung rtl sweep", "/samsung-rtl-sweep", "regression-test the boomerang flow on the emulators",
or wants proof a media-pipeline or foreground-service change didn't break the import→save path
on Google devices and Android 14. Scripts compute every gesture from uiautomator bounds, so they
run unmodified on any screen size.
---

# run-e2e-pixel-sweep — the repeatable 3-emulator OpenLoop sweep
# run-e2e-pixel-sweep — the repeatable 4-emulator OpenLoop sweep

Proven end-to-end on 2026-06-04 (fold-loop iterations 1–2 + scripted validation runs on
Pixel 6 and the Fold). One sweep ≈ 10 minutes/device, ~30 minutes total, fully scripted —
your job is to run the phases, read the PASS/FAIL lines, and apply judgment where the scripts
explicitly hand it back (borderline SSIM, any FAIL, churn growth).
Pixel 6 and the Fold) and 2026-06-22 (API-34 FGS fix verification on `Pixel_8_API34`).
One sweep ≈ 10 minutes/device, ~40 minutes total, fully scripted — your job is to run the
phases, read the PASS/FAIL lines, and apply judgment where the scripts explicitly hand it back
(borderline SSIM, any FAIL, churn growth).

`<skill>` = this directory. Sibling skill `run-e2e` provides `scripts/scan-logcat.ps1` and
`scripts/uiauto.ps1` (manual driving / debugging) — reuse them, don't duplicate.
Expand All @@ -34,8 +36,11 @@ tell the user; do not substitute another video.

## Preconditions (verify, never assume)

- AVDs `Pixel_6`, `Pixel_8`, `Pixel_10_Pro_Fold` exist
- AVDs `Pixel_6`, `Pixel_8`, `Pixel_10_Pro_Fold`, and **`Pixel_8_API34`** exist
(`& "$env:LOCALAPPDATA\Android\Sdk\emulator\emulator.exe" -list-avds`).
If `Pixel_8_API34` is missing, create it:
`pwsh <skill>\scripts\create-api34-avd.ps1` (requires the
`system-images;android-34;google_apis_playstore;x86_64` SDK image).
- ffmpeg/ffprobe on PATH (`winget install Gyan.FFmpeg` if not).
- `$env:JAVA_HOME = "C:\Program Files\Android\Android Studio\jbr"`.
- No emulator already running (`adb devices` empty) — sweep-prep refuses otherwise.
Expand All @@ -45,11 +50,19 @@ tell the user; do not substitute another video.

## The sweep (run for each AVD, sequentially, in this order)

Order: `Pixel_6` → `Pixel_8` → `Pixel_10_Pro_Fold`. One emulator at a time — parallel
emulators fight over host CPU/codecs and fake contention bugs.
Order: `Pixel_6` → `Pixel_8` → `Pixel_10_Pro_Fold` → **`Pixel_8_API34`**. One emulator at a
time — parallel emulators fight over host CPU/codecs and fake contention bugs.

**Why `Pixel_8_API34` last:** it runs **Android 14 (API 34)** — the OS level where
`FOREGROUND_SERVICE_TYPE_MEDIA_PROCESSING` (8192) is *not recognized* and v1.0.23 crashed every
Loopify save with `InvalidForegroundServiceTypeException: Starting FGS with type unknown`
(Crashlytics 9663c743, Galaxy A55). The fix gates to `dataSync` on API 29–34. After the three
API-35+ emulators pass, this device is the **mandatory FGS regression gate**: a green save +
`Worker result SUCCESS` for `BoomerangRenderWorker` and **zero** `InvalidForegroundServiceTypeException`
lines in logcat. See `docs/e2e/2026-06-22_082330-api34-fgs-fix.md` for the first verified run.

```powershell
$avd = "Pixel_6" # then Pixel_8, then Pixel_10_Pro_Fold
$avd = "Pixel_6" # then Pixel_8, then Pixel_10_Pro_Fold, then Pixel_8_API34
$dir = "$env:TEMP\openloop_sweep\$avd"

# 1. Cold-boot + install (fresh app state) + push video + start logcat capture + launch.
Expand All @@ -73,6 +86,48 @@ adb -s $serial emu kill
A device passes when: all drive-flow steps PASS + quality gate PASS + scan shows **0 in every
CRASH and TIMEOUT row**. Anything else is a finding — root-cause it (see "Reading failures").

## OEM regression lanes (API 34 / Samsung / LG)

Not all OEM bugs reproduce on stock emulators. Use the right lane:

| Lane | What it catches | How to run | Emulator? |
|------|-----------------|------------|-----------|
| **API 34 FGS** | `InvalidForegroundServiceTypeException` on Android 14 saves (9663c743) | 4th device in sweep: `Pixel_8_API34` | ✅ Stock AVD |
| **Samsung codec + preview cap** | Exynos/QC wedging, 480p preview cap, `c2.google` encoder order | **Samsung RTL sweep** (real Galaxy) | ❌ Use RTL |
| **Samsung app logic (identity)** | `isSamsungDevice()` branches, preview cap, encoder ordering policy | JVM: `DeviceMediaHintsOemRobolectricTest` + `SamsungReversePreviewRegressionTest` | ✅ Robolectric |
| **LG `start failed` recovery** | Hardware codec rejects `start()` → software fallback (47233ad7) | Instrumented: `VideoReverserTest.reverse_recoversFromCodecStartFailure_viaSoftwareFallback` | ✅ Any device/emulator |

**Do NOT** add fake `Pixel_8_Samsung` AVDs — Google Play images ignore `-prop ro.product.manufacturer`
and block `setprop`; the emulator still reports `Google` / goldfish codecs.

### Samsung RTL sweep (real Galaxy hardware)

Setup: [`docs/guides/samsung-rtl-steps.md`](../../docs/guides/samsung-rtl-steps.md) — reserve device,
start RDB, verify `adb devices` shows `localhost:<port>`.

```powershell
$env:JAVA_HOME = "C:\Program Files\Android\Android Studio\jbr"
& "<repo>\gradlew.bat" :app:assembleDebug
pwsh <skill>\scripts\samsung-rtl-sweep.ps1 -ArtifactDir $env:TEMP\openloop_rtl\s23
# Optional: -Serial localhost:52172 (port changes every RTL session)
```

Pass = drive-flow PASS + quality-gate PASS + scan **0 CRASH/TIMEOUT** + grep shows
`ensureReversed.ok` and `Worker result SUCCESS … BoomerangRenderWorker`.

### LG instrumented lane (headless on emulator)

LG has no public RTL. The recovery path is fault-injected on any attached device:

```powershell
& "<repo>\gradlew.bat" :app:assembleDebug :app:installDebug
adb shell am instrument -w -r `
-e class io.github.stozo04.openloop.media.VideoReverserTest#reverse_recoversFromCodecStartFailure_viaSoftwareFallback `
io.github.stozo04.openloop.test/androidx.test.runner.AndroidJUnitRunner
```

Pass = `tests=1 failures=0`. Literal LM-X540 hardware still needs a physical device if available.

## What the scripts verify (and what they hand back to you)

| Phase | Hard checks | Judgment handed to the agent |
Expand All @@ -95,6 +150,7 @@ device-specific Crashlytics issue is fixed from emulator evidence alone).
| first-half fps ≈ 15, second ≈ 30 | Pass-1 subsampling regression (BUG-1 family) — check `reverse pass1:` log line for `skipped=` ≠ 0 on this 30 fps source. |
| Moving-region macroblock smear in reversed half, static background clean | Compressed samples dropped before the decoder (broken P-frame references). Pull the cached reversed clip from `cache/scratch/reversed/` to isolate reverser vs Transformer. |
| `MPEG4Writer … encoded 0 frames` + tiny output | Zero-frame completion (S23 family, PR #62 validator territory) — different from a timeout. |
| `InvalidForegroundServiceTypeException` / `Starting FGS with type unknown` on API 34 | **FGS type regression** — the worker is passing `mediaProcessing` (8192) on Android 14. Check `BoomerangRenderNotifications.foregroundServiceTypeForSdk` gates on `VANILLA_ICE_CREAM` (35), not `UPSIDE_DOWN_CAKE` (34). Only fires on `Pixel_8_API34`; the three API-35+ emulators won't catch it. |
| `ChooserPreview: Could not read content://…fileprovider…` ×3 | Cosmetic — the OS share-sheet preview can't query FileProvider metadata. Ignore. |

## Field notes (each of these cost real time once — don't relearn them)
Expand Down
90 changes: 90 additions & 0 deletions .claude/skills/run-e2e-pixel-sweep/scripts/create-api34-avd.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
<#
.SYNOPSIS
Create (or verify) the Pixel_8_API34 AVD — Android 14 (API 34) regression target for
foreground-service-type bugs that only reproduce below API 35 (e.g. Crashlytics 9663c743).

.USAGE
pwsh create-api34-avd.ps1

.NOTES
Requires the system image already installed:
system-images;android-34;google_apis_playstore;x86_64
Install via Android Studio SDK Manager if missing. This script writes the AVD config files
directly (no avdmanager/cmdline-tools required).
#>
$ErrorActionPreference = 'Stop'
$avdId = 'Pixel_8_API34'
$avdHome = Join-Path $env:USERPROFILE '.android\avd'
$avdDir = Join-Path $avdHome "$avdId.avd"
$iniPath = Join-Path $avdHome "$avdId.ini"
$sdk = Join-Path $env:LOCALAPPDATA 'Android\Sdk'
$sysImg = Join-Path $sdk 'system-images\android-34\google_apis_playstore\x86_64'

if (-not (Test-Path $sysImg)) {
Write-Error "API 34 system image missing: $sysImg — install via Android Studio SDK Manager."
exit 1
}

New-Item -ItemType Directory -Force $avdDir | Out-Null

@(
"avd.ini.encoding=UTF-8",
"path=$avdDir",
"path.rel=avd\$avdId.avd",
'target=android-34'
) | Set-Content -Encoding UTF8 $iniPath

$config = @"
AvdId=$avdId
PlayStore.enabled=true
abi.type=x86_64
avd.ini.displayname=Pixel 8 (API 34)
avd.ini.encoding=UTF-8
disk.dataPartition.size=6G
fastboot.forceColdBoot=no
fastboot.forceFastBoot=yes
hw.accelerometer=yes
hw.arc=false
hw.audioInput=yes
hw.battery=yes
hw.camera.back=virtualscene
hw.camera.front=emulated
hw.cpu.arch=x86_64
hw.cpu.ncore=4
hw.dPad=no
hw.device.manufacturer=Google
hw.device.name=pixel_8
hw.gps=yes
hw.gpu.enabled=yes
hw.gpu.mode=auto
hw.gyroscope=yes
hw.initialOrientation=portrait
hw.keyboard=yes
hw.lcd.density=420
hw.lcd.height=2400
hw.lcd.width=1080
hw.mainKeys=no
hw.ramSize=2048
hw.sdCard=yes
hw.sensors.light=yes
hw.sensors.magnetic_field=yes
hw.sensors.orientation=yes
hw.sensors.pressure=yes
hw.sensors.proximity=yes
hw.trackBall=no
image.sysdir.1=system-images\android-34\google_apis_playstore\x86_64\
runtime.network.latency=none
runtime.network.speed=full
sdcard.size=512M
showDeviceFrame=yes
skin.dynamic=yes
skin.name=pixel_8
skin.path=$sdk\skins\pixel_8
tag.display=Google Play
tag.id=google_apis_playstore
target=android-34
vm.heapSize=228
"@
$config | Set-Content -Encoding UTF8 (Join-Path $avdDir 'config.ini')

Write-Host "AVD '$avdId' ready. Verify: emulator -list-avds"
9 changes: 7 additions & 2 deletions .claude/skills/run-e2e-pixel-sweep/scripts/drive-flow.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -122,8 +122,13 @@ Step "import" {
Start-Sleep -Seconds 4
if (-not (Tap "Video taken on")) { return $false } # the pushed clip in the system photo picker
Start-Sleep -Seconds 2
if (-not (Tap "Done")) { return $false }
Start-Sleep -Seconds 6
# Some OEM pickers (Samsung RTL) land on Trim immediately; others need "Done" first.
if (-not (Find-Node (Get-Nodes) "TRIM YOUR VIDEO")) {
if (-not (Tap "Done")) { return $false }
Start-Sleep -Seconds 6
} else {
Start-Sleep -Seconds 2
}
[bool](Find-Node (Get-Nodes) "TRIM YOUR VIDEO")
}

Expand Down
84 changes: 84 additions & 0 deletions .claude/skills/run-e2e-pixel-sweep/scripts/samsung-rtl-prep.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
<#
.SYNOPSIS
Prep a Samsung Remote Test Lab (RTL) device for the pixel-sweep flow — install debug APK,
fresh app state, push canonical test video, 3-button nav, logcat capture, launch app.

.USAGE
# After RDB connect (adb devices shows localhost:<port>):
pwsh samsung-rtl-prep.ps1 -Serial localhost:52172 -ArtifactDir $env:TEMP\openloop_rtl\s23

Auto-detects the first localhost:* serial when -Serial is omitted.

.NOTES
RTL devices are REAL Galaxy hardware — not emulators. See docs/guides/samsung-rtl-steps.md.
Does NOT start an emulator. Refuses non-Samsung manufacturers (LGE, Google, etc.).
#>
param(
[string]$Serial,
[Parameter(Mandatory = $true)][string]$ArtifactDir,
[string]$RepoRoot = "C:\Users\gates\Personal\OpenRang",
[string]$VideoName = "google-pro-fold-video.mp4"
)
$ErrorActionPreference = 'Stop'
$pkg = "io.github.stozo04.openloop"
$video = Join-Path $RepoRoot $VideoName
$apk = Join-Path $RepoRoot "app\build\outputs\apk\debug\app-debug.apk"

if (-not (Test-Path $video)) { Write-Error "Canonical test video missing: $video — STOP, do not substitute."; exit 1 }
if (-not (Test-Path $apk)) { Write-Error "APK not built: $apk — run :app:assembleDebug first."; exit 1 }
New-Item -ItemType Directory -Force $ArtifactDir | Out-Null

if (-not $Serial) {
$line = (adb devices) | Select-String 'localhost:\d+\s+device' | Select-Object -First 1
if (-not $line) { Write-Error "No RTL device (localhost:<port>) in adb devices. Connect RDB first — see docs/guides/samsung-rtl-steps.md"; exit 1 }
$Serial = ($line.ToString() -split '\s+')[0]
}

$state = (adb devices) | Select-String "$([regex]::Escape($Serial))\s+device"
if (-not $state) { Write-Error "Serial '$Serial' not in 'device' state. Run adb devices."; exit 1 }

$manufacturer = ((adb -s $Serial shell getprop ro.product.manufacturer) -join '').Trim()
$model = ((adb -s $Serial shell getprop ro.product.model) -join '').Trim()
$sdk = ((adb -s $Serial shell getprop ro.build.version.sdk) -join '').Trim()
if ($manufacturer -notmatch 'samsung') {
Write-Error "Refusing RTL prep: ro.product.manufacturer='$manufacturer' (expected samsung). Wrong device or not RTL Galaxy."
exit 1
}
Write-Host "RTL device: $manufacturer $model (API $sdk) serial=$Serial"

adb -s $Serial install -r -g $apk | Out-Null
adb -s $Serial shell pm clear $pkg | Out-Null
adb -s $Serial shell pm grant $pkg android.permission.CAMERA
adb -s $Serial push $video /sdcard/Download/ | Out-Null
adb -s $Serial shell am broadcast -a android.intent.action.MEDIA_SCANNER_SCAN_FILE -d "file:///sdcard/Download/$VideoName" | Out-Null
adb -s $Serial shell cmd overlay enable com.android.internal.systemui.navbar.threebutton 2>$null

adb -s $Serial logcat -c
Start-Process adb -ArgumentList "-s", $Serial, "logcat" `
-RedirectStandardOutput (Join-Path $ArtifactDir "logcat.txt") -WindowStyle Hidden
Start-Sleep -Seconds 1

# RTL devices often arrive locked / on the keyguard — drive-flow cannot see onboarding until unlocked.
adb -s $Serial shell input keyevent KEYCODE_WAKEUP | Out-Null
Start-Sleep -Seconds 1
adb -s $Serial shell wm dismiss-keyguard 2>$null | Out-Null
$sizeLine = (adb -s $Serial shell wm size) -join ''
if ($sizeLine -match '(\d+)x(\d+)') {
$w = [int]$Matches[1]; $h = [int]$Matches[2]
adb -s $Serial shell input swipe ([int]($w / 2)) ([int]($h * 0.78)) ([int]($w / 2)) ([int]($h * 0.34)) 300 | Out-Null
}
Start-Sleep -Seconds 2

Start-Sleep -Seconds 8
adb -s $Serial shell am start -n "$pkg/.MainActivity" | Out-Null
Start-Sleep -Seconds 4

@(
"serial=$Serial",
"manufacturer=$manufacturer",
"model=$model",
"sdk=$sdk"
) | Set-Content (Join-Path $ArtifactDir "device-info.txt")

Write-Host "SERIAL=$Serial"
exit 0
57 changes: 57 additions & 0 deletions .claude/skills/run-e2e-pixel-sweep/scripts/samsung-rtl-sweep.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
<#
.SYNOPSIS
Full Samsung Remote Test Lab (RTL) regression: prep → drive-flow → quality-gate → logcat scan.

.USAGE
# 1. Reserve a Galaxy on Samsung RTL, start RDB, connect in browser (see docs/guides/samsung-rtl-steps.md)
# 2. Build once:
# $env:JAVA_HOME = "C:\Program Files\Android\Android Studio\jbr"
# .\gradlew.bat :app:assembleDebug
# 3. Run:
pwsh samsung-rtl-sweep.ps1 -ArtifactDir $env:TEMP\openloop_rtl\s23
# Optional: -Serial localhost:52172 (auto-detects first localhost:* if omitted)

.NOTES
Reuses drive-flow.ps1, quality-gate.ps1, and scan-logcat.ps1 from this skill + run-e2e.
Keep the RTL browser tab awake — debugging drops when the remote screen sleeps.
#>
param(
[string]$Serial,
[Parameter(Mandatory = $true)][string]$ArtifactDir,
[string]$RepoRoot = "C:\Users\gates\Personal\OpenRang",
[switch]$SkipQualityGate
)
$ErrorActionPreference = 'Stop'
$skill = Split-Path -Parent $MyInvocation.MyCommand.Path
$repoSkill = Join-Path $RepoRoot ".claude\skills\run-e2e-pixel-sweep\scripts"
$e2eSkill = Join-Path $RepoRoot ".claude\skills\run-e2e\scripts"

foreach ($dir in @($skill, $repoSkill, $e2eSkill)) {
if (-not (Test-Path $dir)) { Write-Error "Skill scripts missing: $dir"; exit 1 }
}

Write-Host "=== Samsung RTL sweep ==="
& (Join-Path $skill "samsung-rtl-prep.ps1") -Serial $Serial -ArtifactDir $ArtifactDir -RepoRoot $RepoRoot
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }

$info = Get-Content (Join-Path $ArtifactDir "device-info.txt") -ErrorAction SilentlyContinue
$serialLine = $info | Where-Object { $_ -like 'serial=*' } | Select-Object -First 1
if ($serialLine) { $Serial = ($serialLine -split '=', 2)[1] }
if (-not $Serial) { Write-Error "Prep did not yield a serial."; exit 1 }

Write-Host "=== drive-flow on $Serial ==="
& (Join-Path $skill "drive-flow.ps1") -Serial $Serial -ArtifactDir $ArtifactDir
$flowExit = $LASTEXITCODE

if (-not $SkipQualityGate) {
Write-Host "=== quality-gate ==="
& (Join-Path $skill "quality-gate.ps1") -Serial $Serial -ArtifactDir $ArtifactDir
if ($LASTEXITCODE -ne 0) { $flowExit = [Math]::Max($flowExit, $LASTEXITCODE) }
}

Write-Host "=== scan-logcat ==="
& (Join-Path $e2eSkill "scan-logcat.ps1") -LogFile (Join-Path $ArtifactDir "logcat.txt")

Write-Host "=== RTL sweep complete (flow exit=$flowExit) ==="
Write-Host "Artifacts: $ArtifactDir"
exit $flowExit
Loading