Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,10 @@ checkpoints/
*.tflite
*.pb

# Exception: the models the mobile apps ship are part of the app, not local
# artifacts — without them the apps cannot be built or run.
!frontend/android/app/src/main/assets/second_look.tflite

# Notebook checkpoints
.ipynb_checkpoints/

Expand Down
28 changes: 28 additions & 0 deletions frontend/android/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# ---------------------------------------------------------------------------
# SecondLook — Android / Gradle .gitignore
# NOTE: the Gradle wrapper (gradlew, gradlew.bat, gradle/wrapper/*) IS
# committed, so a clone builds with the right Gradle version, and so is the
# bundled second_look.tflite model the app ships.
# ---------------------------------------------------------------------------

# Build output
build/
/captures/
.cxx/
.externalNativeBuild/

# Gradle / IDE local state
.gradle/
.kotlin/
.idea/
*.iml

# Machine-specific: SDK path, and any chaquopyBuildPython override
local.properties

# Python bytecode from running the vendored pipeline outside Android
__pycache__/
*.py[cod]

# macOS
.DS_Store
118 changes: 118 additions & 0 deletions frontend/android/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# SecondLook (Android)

An Android app that runs on-device triage of mammogram images. A user selects or
captures a mammogram, the image is preprocessed and passed through a bundled
TensorFlow Lite model, and the app returns a probability and a coarse tier
(Low / Moderate / Elevated).

This is the Android counterpart of the iOS app in `frontend/`, with the same
four-screen flow, the same palette (Apple's system colors), and the same model.

## ⚠️ Not a medical device

SecondLook is a research and educational prototype. It is **not** a diagnostic
tool, is **not** FDA/CE cleared, and must not be used to make clinical or
personal health decisions. The tier cut-points are provisional and uncalibrated.
The app shows a disclaimer gate on launch for this reason.

## How it works

1. **Capture / select** an image — `ui/SecondLookFlow.kt` (Compose): disclaimer
gate → capture → scanning → results.
2. **Preprocess** it by running the research pipeline's **actual Python code**
on-device through [Chaquopy](https://chaquo.com/chaquopy/): grayscale →
CLAHE → breast mask → pectoral-muscle removal → orientation normalization →
resize to 224×224 → float32 in `[0, 1]`.
3. **Classify** with `second_look.tflite` via `MammogramClassifier.kt`,
producing `P(worth second look)` and a tier.

Nothing is uploaded and nothing is written to shared storage: camera captures go
to the app's cache directory and the preprocessing scratch file is deleted after
each scan.

### Why Python instead of a port

The iOS app hand-ports the pipeline to Objective-C++/OpenCV, which means two
implementations that can silently drift apart — and a drift here feeds the model
inputs it was never trained on. Android can embed CPython, so this app runs the
training-time code verbatim:

```
app/src/main/python/
config/constants.py ┐
data_pipeline/_imaging_utils.py ├ copied verbatim from the research repo
data_pipeline/preprocessor.py │
data_pipeline/label_mapper.py ┘
data_pipeline/__init__.py trimmed: the repo's version imports the
whole training stack (pandas, GCS, TF)
second_look_bridge.py the only Android-specific Python
```

Refresh the copied files after changing them upstream:

```bash
./gradlew :app:syncPythonPipeline # defaults to the repo root (../..)
./gradlew :app:syncPythonPipeline -PsecondLookRepo=/path/to/secondlook
```

## Requirements

- Android Studio (AGP 9.2.x — pinned because Chaquopy 17 supports AGP ≤ 9.2)
- JDK 21 (Android Studio's bundled JBR is fine)
- **Python 3.10 on the build machine**, for Chaquopy's `buildPython`:

```bash
brew install python@3.10 # macOS
```

3.10 specifically: it is the only version with OpenCV wheels in Chaquopy's
Android package repository, and the pipeline is cv2-based. Chaquopy finds
`python3.10` on `PATH`; if Android Studio does not inherit your shell `PATH`,
set an explicit path instead:

```properties
# local.properties or gradle.properties
chaquopyBuildPython=/opt/homebrew/bin/python3.10
```

Minimum device: API 24, `arm64-v8a` or `x86_64` (the ABIs Chaquopy's NumPy and
OpenCV wheels are built for — see `abiFilters`).

## Build & run

```bash
./gradlew :app:installDebug # build + install on a running device
./gradlew :app:connectedAndroidTest # pipeline + inference tests, on-device
```

The first build downloads the CPython runtime plus NumPy and OpenCV wheels, and
the first launch unpacks them, so both are slower than usual.

Note: the Gradle configuration cache is disabled in `gradle.properties` —
Chaquopy shells out to `buildPython` during configuration, which the cache
forbids.

## The model

`app/src/main/assets/second_look.tflite` (~4.4 MB) is the trained classifier,
bundled into the APK and memory-mapped at runtime (hence `noCompress += "tflite"`).
It is trained separately in the Python pipeline. If you retrain, replace this
file and keep the 224×224×1 input contract — `MammogramClassifier` verifies the
preprocessed tensor against the model's declared input size and fails loudly if
they disagree.

## Project layout

```
app/src/main/java/com/hexogen/secondlook/
SecondLookApplication.kt Starts the CPython runtime once per process
MainActivity.kt Entry point
ui/SecondLookFlow.kt The four screens (disclaimer → capture → results)
ui/theme/ Colors, typography, tier palette
MammogramPreprocessor.kt Kotlin ↔ Python bridge
MammogramClassifier.kt TFLite interpreter wrapper + tiering
ImageLoading.kt EXIF orientation, preview decoding, capture URIs
app/src/main/python/ The preprocessing pipeline (see above)
app/src/main/assets/ second_look.tflite
app/src/androidTest/ On-device pipeline + inference tests
```
4 changes: 4 additions & 0 deletions frontend/android/app/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
/build
# Local bytecode from running the vendored pipeline outside Android
__pycache__/
*.py[cod]
120 changes: 120 additions & 0 deletions frontend/android/app/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
plugins {
alias(libs.plugins.android.application)
alias(libs.plugins.kotlin.compose)
alias(libs.plugins.chaquopy)
}

android {
namespace = "com.hexogen.secondlook"
compileSdk {
version = release(36) {
minorApiLevel = 1
}
}

defaultConfig {
applicationId = "com.hexogen.secondlook"
minSdk = 24
targetSdk = 36
versionCode = 1
versionName = "1.0"

testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"

// Chaquopy packs a CPython runtime plus native NumPy/OpenCV per ABI.
// Restrict to the two ABIs that matter (real devices + emulator) —
// each extra ABI adds tens of megabytes to the APK.
ndk {
abiFilters += listOf("arm64-v8a", "x86_64")
}
}

buildTypes {
release {
optimization {
enable = false
}
}
}
compileOptions {
sourceCompatibility = JavaVersion.VERSION_11
targetCompatibility = JavaVersion.VERSION_11
}
buildFeatures {
compose = true
}
androidResources {
// TFLite models are memory-mapped from the APK, so they must be stored
// uncompressed.
noCompress += "tflite"
}
}

chaquopy {
defaultConfig {
// 3.10 is the only Python version with OpenCV wheels in Chaquopy's
// package repository, and the preprocessing pipeline is cv2-based.
// buildPython must be a local Python 3.10 — see README.
version = "3.10"

// Chaquopy looks for `python3.10` on PATH. Android Studio launched from
// the Dock does not always inherit a shell PATH, so allow an override:
// ./gradlew -PchaquopyBuildPython=/opt/homebrew/bin/python3.10 ...
// or set chaquopyBuildPython in gradle.properties / local.properties.
providers.gradleProperty("chaquopyBuildPython").orNull?.let { buildPython(it) }

pip {
// Versions are the newest cp310 Android wheels Chaquopy publishes.
install("numpy==1.26.2")
install("opencv-python==4.5.1.48")
}
}
}

dependencies {
implementation(platform(libs.androidx.compose.bom))
implementation(libs.androidx.activity.compose)
implementation(libs.androidx.compose.material3)
implementation(libs.androidx.compose.ui)
implementation(libs.androidx.compose.ui.graphics)
implementation(libs.androidx.compose.ui.tooling.preview)
implementation(libs.androidx.core.ktx)
implementation(libs.androidx.exifinterface)
implementation(libs.androidx.lifecycle.runtime.ktx)
implementation(libs.tensorflow.lite)
testImplementation(libs.junit)
androidTestImplementation(platform(libs.androidx.compose.bom))
androidTestImplementation(libs.androidx.compose.ui.test.junit4)
androidTestImplementation(libs.androidx.espresso.core)
androidTestImplementation(libs.androidx.junit)
debugImplementation(libs.androidx.compose.ui.test.manifest)
debugImplementation(libs.androidx.compose.ui.tooling)
}

// Refresh the vendored copies of the Python pipeline from the Second Look
// research repo. The copies under src/main/python are the ones that ship; this
// task just keeps them from silently drifting. The default resolves to the repo
// root two levels up (frontend/android/ -> repo root); override with
// -PsecondLookRepo=... for a checkout somewhere else.
val secondLookRepo = providers.gradleProperty("secondLookRepo")
.orElse(rootProject.layout.projectDirectory.dir("../..").asFile.path)

tasks.register<Copy>("syncPythonPipeline") {
group = "second look"
description = "Copy preprocessing sources from the Second Look Python repo into src/main/python."

val repoDir = file(secondLookRepo.get())
onlyIf {
repoDir.isDirectory.also {
if (!it) logger.warn("Second Look repo not found at $repoDir — nothing to sync.")
}
}

from(repoDir) {
include("config/constants.py")
include("data_pipeline/_imaging_utils.py")
include("data_pipeline/preprocessor.py")
include("data_pipeline/label_mapper.py")
}
into(layout.projectDirectory.dir("src/main/python"))
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
package com.hexogen.secondlook

import android.content.Context
import android.graphics.Bitmap
import android.graphics.Canvas
import android.graphics.Color
import android.graphics.Paint
import android.graphics.RectF
import android.net.Uri
import androidx.test.ext.junit.runners.AndroidJUnit4
import androidx.test.platform.app.InstrumentationRegistry
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Test
import org.junit.runner.RunWith
import java.io.File

/**
* End-to-end checks for the on-device path: Python preprocessing through
* TensorFlow Lite inference. Instrumented rather than local because both the
* CPython runtime and the model live in the APK.
*/
@RunWith(AndroidJUnit4::class)
class MammogramPipelineTest {

private val context: Context = InstrumentationRegistry.getInstrumentation().targetContext

@Test
fun inputSizeComesFromTheSharedPythonConfig() {
assertEquals(224, MammogramPreprocessor.inputSize)
}

@Test
fun preprocessingProducesTheModelInputTensor() {
val size = MammogramPreprocessor.inputSize
val tensor = MammogramPreprocessor.preprocess(context, syntheticMammogram())

assertEquals("float32 [1, $size, $size, 1]", size * size * 4, tensor.capacity())

val floats = tensor.asFloatBuffer()
var sum = 0f
for (i in 0 until floats.limit()) {
val value = floats.get(i)
assertTrue("value $value at $i is outside [0, 1]", value in 0f..1f)
sum += value
}
// The pipeline masks background to zero but must not zero everything:
// an all-black tensor means masking ate the breast region.
assertTrue("preprocessed tensor is entirely zero", sum > 0f)
}

@Test
fun classificationReturnsAProbabilityAndMatchingTier() {
val result = MammogramClassifier(context).use { it.classify(syntheticMammogram()) }

assertTrue("probability ${result.probability} outside [0, 1]", result.probability in 0f..1f)
assertEquals(MammogramClassifier.tierFor(result.probability), result.tier)
}

@Test
fun tierCutPointsMatchTheSharedThresholds() {
// Mirrors data_pipeline.label_mapper.TIER_THRESHOLDS (provisional).
assertEquals("Low", MammogramClassifier.tierFor(0f))
assertEquals("Low", MammogramClassifier.tierFor(0.329f))
assertEquals("Moderate", MammogramClassifier.tierFor(0.33f))
assertEquals("Moderate", MammogramClassifier.tierFor(0.659f))
assertEquals("Elevated", MammogramClassifier.tierFor(0.66f))
assertEquals("Elevated", MammogramClassifier.tierFor(1f))
}

/**
* A stand-in mammogram: a bright half-ellipse of "tissue" against a dark
* background, plus a wedge in the top corner for the pectoral heuristic to
* find. Enough structure for every stage of the pipeline to do real work.
*/
private fun syntheticMammogram(): Uri {
val bitmap = Bitmap.createBitmap(400, 600, Bitmap.Config.ARGB_8888)
val canvas = Canvas(bitmap)
canvas.drawColor(Color.rgb(8, 8, 8))

val tissue = Paint().apply { color = Color.rgb(190, 190, 190); isAntiAlias = true }
canvas.drawOval(RectF(-160f, 60f, 280f, 540f), tissue)

val pectoral = Paint().apply { color = Color.rgb(245, 245, 245) }
canvas.drawRect(RectF(0f, 0f, 150f, 40f), pectoral)

val file = File.createTempFile("test_mammogram", ".png", context.cacheDir)
file.outputStream().use { bitmap.compress(Bitmap.CompressFormat.PNG, 100, it) }
return Uri.fromFile(file)
}
}
Loading