feat: implement scoring module for segments#7
feat: implement scoring module for segments#7dhalmazna wants to merge 6 commits intofeat/model-predictorfrom
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request establishes the foundational scoring mechanisms for the explainer, enabling the evaluation of individual image segments and hyperpixels. By introducing a dedicated Highlights
Changelog
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new scoring module, which is a core component for evaluating image segment importance. The implementation is well-structured and includes efficient bitmask-based graph utilities. My review focuses on a couple of performance optimizations. Specifically, I've suggested removing some expensive logging operations that trigger redundant model forward passes and vectorizing a loop that applies masks to an image batch to better leverage PyTorch's capabilities. Overall, the changes are a solid addition to the project.
There was a problem hiding this comment.
Pull request overview
Introduces a new scoring/ module to quantify segment and hyperpixel importance for CIAO’s explainer workflow, plus bitmask-based graph traversal utilities to support efficient neighborhood masking.
Changes:
- Added surrogate dataset generation and per-segment score aggregation (
ciao/scoring/segments.py). - Added batched masking delta computation and top-k hyperpixel selection helpers (
ciao/scoring/hyperpixel.py). - Added bitmask graph primitives for fast neighborhood/frontier expansion (
ciao/algorithm/bitmask_graph.py) and exported new modules via package__init__.pyfiles; updated README tree.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| ciao/scoring/segments.py | Builds local neighborhood masking groups and produces a surrogate dataset + segment score aggregation. |
| ciao/scoring/hyperpixel.py | Computes masking deltas for candidate hyperpixels and provides a top-k selection helper. |
| ciao/scoring/init.py | Exposes scoring utilities as package-level exports. |
| ciao/algorithm/bitmask_graph.py | Adds iter_bits and get_frontier utilities for bitmask-based traversal. |
| ciao/algorithm/init.py | Exposes algorithm utilities as package-level exports. |
| README.md | Updates module layout documentation to reflect the new scoring/ package. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| def select_top_hyperpixels( | ||
| hyperpixels: list[dict[str, object]], max_hyperpixels: int = 10 | ||
| ) -> list[dict[str, object]]: | ||
| """Select top hyperpixels by their primary algorithm-specific score.""" | ||
| # Use hyperpixel_score | ||
| return sorted( | ||
| hyperpixels, | ||
| key=lambda hp: abs(hp.get("hyperpixel_score", 0)), # type: ignore[arg-type] | ||
| reverse=True, | ||
| )[:max_hyperpixels] |
There was a problem hiding this comment.
This is actually not a bad idea, but I feel like this would be a too complex solution for now. We can decide on it later after the reviewers will see how the hyperpixels dict looks like in different algorithms.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Context:
This PR introduces the
scoring/module, which provides the core mechanisms to evaluate the importance of individual image segments during the explainer's execution. It also establishes the foundation for selecting the initial starting points (seeds) for the main search algorithms (creation of the surrogate dataset).What was added:
scoring/segments.py: Contains the logic for scoring base segments (create_surrogate_dataset,calculate_segment_scores). It evaluates each individual segment by masking its local neighborhood and measuring the model's confidence drop. These importance scores are crucial for selecting the initial "seeds" from which the main algorithms (like MCTS or MCGS) will grow.scoring/hyperpixel.py: Handles the evaluation of discovered hyperpixels (calculating deltas and selecting the top-k most important regions).algorithm/bitmask_graph.py: Introduces highly efficient, low-level bitwise graph utilities (get_frontier,iter_bits). These primitives enable fast, memory-efficient BFS traversal during the surrogate dataset creation without relying on heavy external graph libraries. More bitwise utilities will be added in the next PR, these are just the ones necessary for thecreate_surrogate_datasetfunction to work.Related Task:
XAI-29