fix(structured): anonymize columns whose name is not an identifier#2142
Open
uwezkhan wants to merge 1 commit into
Open
fix(structured): anonymize columns whose name is not an identifier#2142uwezkhan wants to merge 1 commit into
uwezkhan wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change Description
PandasDataProcessor._processreads each cell withgetattr(row, key)while iteratingDataFrame.itertuples(). itertuples renames any column whose name is not a valid Python identifier (a space, a hyphen, a leading digit, a keyword) into a positional field like_1, so the lookup raisesAttributeErrorfor the very columns that usually hold PII, such as "Full Name" or "e-mail". The mapping comes straight fromdf.columnsvia the analysis builder, soStructuredEngine.anonymizehits this on ordinary input.Before: only columns named as Python identifiers were anonymized; a "Full Name" column aborted the run, and because the frame is mutated column by column, any columns processed earlier were redacted while the failing PII column was left untouched. After: the cell is read by label with
data.at[index, key], the same way the result is already written back, so read and write agree and the column name no longer matters. The tradeoff is a per-cell.atlookup in place of a namedtuple field access, negligible next to the operator call already run on every cell.Issue reference
No linked issue.
Checklist