Skip to content

Deduplicate IOC identities and enforce uniqueness. Closes #1240#1344

Merged
regulartim merged 5 commits into
GreedyBear-Project:developfrom
cclts:deduplicate_ioc_identity_and_add_constraint
May 25, 2026
Merged

Deduplicate IOC identities and enforce uniqueness. Closes #1240#1344
regulartim merged 5 commits into
GreedyBear-Project:developfrom
cclts:deduplicate_ioc_identity_and_add_constraint

Conversation

@cclts
Copy link
Copy Markdown
Contributor

@cclts cclts commented May 20, 2026

Description

Deduplicate existing IOC rows by (name, type) before enforcing uniqueness at the database level.

This change updates the IOC model to declare a unique constraint on ("name", "type") and adds migration 0053 to safely prepare existing data first. The migration keeps the IOC with the highest attack_count for each duplicate group, breaks ties by keeping the row with the latest id, deletes the remaining duplicates, and then applies the unique constraint in a separate migration step.

Related issues

Closes #1240

Context:

  • Prevents duplicate IOC identities with the same (name, type) pair.
  • Ensures existing duplicate data does not cause the new database constraint to fail during migration.

Type of change

  • Bug fix (non-breaking change which fixes an issue).
  • New feature (non-breaking change which adds functionality).
  • Breaking change (fix or feature that would cause existing functionality to not work as expected).
  • Chore (refactoring, dependency updates, CI/CD changes, code cleanup, docs-only changes).

Checklist

Please complete this checklist carefully. It helps guide your contribution and lets maintainers verify that all requirements are met.

Formalities

  • I have read and understood the rules about how to Contribute to this project.
  • I chose an appropriate title for the pull request in the form: <feature name>. Closes #999
  • My branch is based on develop.
  • The pull request is for the branch develop.
  • I have reviewed and verified any LLM-generated code included in this PR.

Docs and tests

  • I documented my code changes with docstrings and/or comments.
  • I have checked if my changes affect user-facing behavior that is described in the docs. If so, I also included an update to the wiki in the description of this PR.
  • Linter (Ruff) gave 0 errors. If you have correctly installed pre-commit, it does these checks and adjustments on your behalf.
  • I have added tests for the feature/bug I solved.
  • All the tests gave 0 errors.

@cclts cclts marked this pull request as ready for review May 20, 2026 00:15
@cclts
Copy link
Copy Markdown
Contributor Author

cclts commented May 20, 2026

Would appreciate a review when you have a chance @regulartim

Copy link
Copy Markdown
Member

@regulartim regulartim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @cclts ! Yeah, I saw that the PR was ready but did not find time to review yesterday.

Comment thread greedybear/models.py Outdated
Comment thread greedybear/migrations/0053_deduplicate_ioc_identity_and_add_constraint.py Outdated
@cclts cclts requested a review from regulartim May 22, 2026 20:55
Copy link
Copy Markdown
Member

@regulartim regulartim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks! :) I'll make a small simplification to models.py before merging.

@regulartim regulartim merged commit c5cdec1 into GreedyBear-Project:develop May 25, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants