vertex_seniorpage - #1
Open
ecomperat wants to merge 2 commits into
Open
Conversation
Reviewer's GuideThis PR adds a TensorFlow dependency, introduces a new TFLite-based inference script for classifying senior page images, and includes related model assets and output scaffolding. Sequence diagram for senior page image classification pipelinesequenceDiagram
participant User
participant Script
participant Model
participant Output
User->>Script: Start run_tflite.py
Script->>Script: Locate model and label assets
Script->>Script: Load images from data/senior_page
loop For each image
Script->>Script: Preprocess image
Script->>Model: Run inference
Model-->>Script: Return raw scores
Script->>Output: Save raw scores and predictions
end
Script->>User: Print prediction summary
Class diagram for the new TFLite inference script (run_tflite.py)classDiagram
class Interpreter {
+allocate_tensors()
+set_tensor(index, value)
+invoke()
+get_tensor(index)
}
class run_tflite_py {
+main()
+preprocess(image_path, input_details)
+pick_artifact(provided, description, search_root, pattern)
+load_labels(label_path)
+sanitize_label(label)
+normalize_path(path)
+to_path(path)
+suppress_native_logs()
}
run_tflite_py --> Interpreter : uses
run_tflite_py --> Image_PIL : uses
run_tflite_py --> numpy_ndarray : uses
run_tflite_py --> Path_pathlib : uses
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey there - I've reviewed your changes - here's some feedback:
- Avoid hard-coding BASE_ROOT to a Windows path—make model/data/output roots configurable (e.g. via CLI args or environment variables) so the script is portable.
- Refactor the monolithic main into smaller reusable functions and use argparse to pass in model, label, data, and output directories instead of editing global constants.
- Replace bare print statements with a proper logging setup so you can control verbosity and direct output to files or stdout as needed.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Avoid hard-coding BASE_ROOT to a Windows path—make model/data/output roots configurable (e.g. via CLI args or environment variables) so the script is portable.
- Refactor the monolithic main into smaller reusable functions and use argparse to pass in model, label, data, and output directories instead of editing global constants.
- Replace bare print statements with a proper logging setup so you can control verbosity and direct output to files or stdout as needed.
## Individual Comments
### Comment 1
<location> `scripts/senior_page/run_tflite.py:212` </location>
<code_context>
+ output_details = interpreter.get_output_details()[0]
+
+ # Quantization parameters let us reverse the byte-level logits into floats.
+ output_scale, output_zero_point = output_details.get("quantization", (0.0, 0))
+
+ # Step 4: pre-create the reporting folders (summary, intermediary floats, raw bytes).
</code_context>
<issue_to_address>
**issue (bug_risk):** Default quantization parameters may not be valid for all models.
If quantization metadata is missing, defaulting to (0.0, 0) may cause all output values to be zero. Please validate quantization parameters or handle non-quantized outputs appropriately.
</issue_to_address>
### Comment 2
<location> `scripts/senior_page/run_tflite.py:261` </location>
<code_context>
+ # Step 5b: rank the calibrated confidences to pick the winning class.
+ best_idx = int(np.argmax(scores))
+ confidence = float(scores[best_idx])
+ label = labels[best_idx] if best_idx < len(labels) else f"class_{best_idx}"
+
+ line = f"{image_path.name}: {label} ({confidence:.3f})"
</code_context>
<issue_to_address>
**issue (bug_risk):** Fallback to 'class_{best_idx}' may indicate a label/model mismatch.
If best_idx is out of range for labels, log a warning or raise an error to detect model-label misalignment early.
</issue_to_address>
### Comment 3
<location> `scripts/senior_page/run_tflite.py:230-239` </location>
<code_context>
+ with output_path.open("w", encoding="utf-8") as handle:
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Writing predictions overwrites previous results without warning.
Please add an option to append, version, or warn before overwriting to help prevent accidental data loss.
Suggested implementation:
```python
# Option to control file writing mode: "write", "append", "warn", "version"
# You may want to expose this as a CLI argument or config variable.
output_mode = "write" # Options: "write", "append", "warn", "version"
# Determine file open mode and handle existence
file_exists = output_path.exists()
if output_mode == "append":
open_mode = "a"
elif output_mode == "warn":
if file_exists:
raise FileExistsError(f"Output file {output_path} already exists. Aborting to prevent overwrite.")
open_mode = "w"
elif output_mode == "version":
if file_exists:
# Find next available versioned filename
stem = output_path.stem
suffix = output_path.suffix
parent = output_path.parent
version = 1
while True:
versioned_path = parent / f"{stem}_v{version}{suffix}"
if not versioned_path.exists():
output_path = versioned_path
break
version += 1
open_mode = "w"
else: # "write"
open_mode = "w"
with output_path.open(open_mode, encoding="utf-8") as handle:
for image_path in images:
# Step 5a: feed input tensor and execute the forward pass.
input_data = preprocess(image_path, input_details=input_details)
interpreter.set_tensor(input_details["index"], input_data)
with suppress_native_logs():
interpreter.invoke()
# Raw model output is quantized uint8 logits centered at zero_point.
raw_scores = interpreter.get_tensor(output_details["index"])[0]
row = {"filename": image_path.name}
```
You may want to expose `output_mode` as a command-line argument or configuration option so users can select their preferred behavior. Also, ensure that any downstream code that reads the output file can handle versioned or appended files as needed.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| output_details = interpreter.get_output_details()[0] | ||
|
|
||
| # Quantization parameters let us reverse the byte-level logits into floats. | ||
| output_scale, output_zero_point = output_details.get("quantization", (0.0, 0)) |
There was a problem hiding this comment.
issue (bug_risk): Default quantization parameters may not be valid for all models.
If quantization metadata is missing, defaulting to (0.0, 0) may cause all output values to be zero. Please validate quantization parameters or handle non-quantized outputs appropriately.
| # Step 5b: rank the calibrated confidences to pick the winning class. | ||
| best_idx = int(np.argmax(scores)) | ||
| confidence = float(scores[best_idx]) | ||
| label = labels[best_idx] if best_idx < len(labels) else f"class_{best_idx}" |
There was a problem hiding this comment.
issue (bug_risk): Fallback to 'class_{best_idx}' may indicate a label/model mismatch.
If best_idx is out of range for labels, log a warning or raise an error to detect model-label misalignment early.
Comment on lines
+230
to
+239
| with output_path.open("w", encoding="utf-8") as handle: | ||
| for image_path in images: | ||
| # Step 5a: feed input tensor and execute the forward pass. | ||
| input_data = preprocess(image_path, input_details=input_details) | ||
| interpreter.set_tensor(input_details["index"], input_data) | ||
| with suppress_native_logs(): | ||
| interpreter.invoke() | ||
|
|
||
| # Raw model output is quantized uint8 logits centered at zero_point. | ||
| raw_scores = interpreter.get_tensor(output_details["index"])[0] |
There was a problem hiding this comment.
suggestion (bug_risk): Writing predictions overwrites previous results without warning.
Please add an option to append, version, or warn before overwriting to help prevent accidental data loss.
Suggested implementation:
# Option to control file writing mode: "write", "append", "warn", "version"
# You may want to expose this as a CLI argument or config variable.
output_mode = "write" # Options: "write", "append", "warn", "version"
# Determine file open mode and handle existence
file_exists = output_path.exists()
if output_mode == "append":
open_mode = "a"
elif output_mode == "warn":
if file_exists:
raise FileExistsError(f"Output file {output_path} already exists. Aborting to prevent overwrite.")
open_mode = "w"
elif output_mode == "version":
if file_exists:
# Find next available versioned filename
stem = output_path.stem
suffix = output_path.suffix
parent = output_path.parent
version = 1
while True:
versioned_path = parent / f"{stem}_v{version}{suffix}"
if not versioned_path.exists():
output_path = versioned_path
break
version += 1
open_mode = "w"
else: # "write"
open_mode = "w"
with output_path.open(open_mode, encoding="utf-8") as handle:
for image_path in images:
# Step 5a: feed input tensor and execute the forward pass.
input_data = preprocess(image_path, input_details=input_details)
interpreter.set_tensor(input_details["index"], input_data)
with suppress_native_logs():
interpreter.invoke()
# Raw model output is quantized uint8 logits centered at zero_point.
raw_scores = interpreter.get_tensor(output_details["index"])[0]
row = {"filename": image_path.name}You may want to expose output_mode as a command-line argument or configuration option so users can select their preferred behavior. Also, ensure that any downstream code that reads the output file can handle versioned or appended files as needed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by Sourcery
Add a standalone script to run the senior_page image classifier using TensorFlow Lite, encompassing model and label discovery, image preprocessing, inference execution, and result serialization
New Features:
Build: