Skip to content

fix(structured): anonymize columns whose name is not an identifier#2142

Open
uwezkhan wants to merge 1 commit into
data-privacy-stack:mainfrom
uwezkhan:structured-nonidentifier-column
Open

fix(structured): anonymize columns whose name is not an identifier#2142
uwezkhan wants to merge 1 commit into
data-privacy-stack:mainfrom
uwezkhan:structured-nonidentifier-column

Conversation

@uwezkhan

@uwezkhan uwezkhan commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Change Description

PandasDataProcessor._process reads each cell with getattr(row, key) while iterating DataFrame.itertuples(). itertuples renames any column whose name is not a valid Python identifier (a space, a hyphen, a leading digit, a keyword) into a positional field like _1, so the lookup raises AttributeError for the very columns that usually hold PII, such as "Full Name" or "e-mail". The mapping comes straight from df.columns via the analysis builder, so StructuredEngine.anonymize hits this on ordinary input.

Before: only columns named as Python identifiers were anonymized; a "Full Name" column aborted the run, and because the frame is mutated column by column, any columns processed earlier were redacted while the failing PII column was left untouched. After: the cell is read by label with data.at[index, key], the same way the result is already written back, so read and write agree and the column name no longer matters. The tradeoff is a per-cell .at lookup in place of a namedtuple field access, negligible next to the operator call already run on every cell.

Issue reference

No linked issue.

Checklist

  • I have reviewed the contribution guidelines
  • I agree to follow this project's Code of Conduct
  • I confirm that I have the right to submit this contribution and that it does not knowingly contain proprietary or confidential code.
  • My code includes unit tests
  • All unit tests and lint checks pass locally
  • My PR contains documentation updates / additions if required

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant