Feature/profanity redact - #9
Open
sdemanas wants to merge 6 commits into
Open
Conversation
Redacts profane words, replacing with *** Co-authored-by: Manas <194718544+sdemanas@users.noreply.github.com>
sanitizer.py imports better_profanity unconditionally, but it was never
added to pyproject.toml's dependencies and isn't installed anywhere -
pip install whatstext crashed on the very first import with
ModuleNotFoundError for every fresh install. Declared it properly so it's
pulled in automatically, removed the redundant manual "pip install
better_profanity" step, moved the feature from the roadmap's "Planned
next" to "Also added along the way" since it's actually implemented, and
noted the dependency under Dependencies.
Verified: pip install better_profanity + smoke test confirms censoring
works (sanitize_text("... damn ...") -> "... **** ..."), full test suite
passes.
Moves censoring out of sanitizer.py (which was censoring the entire raw file - usernames, timestamps context, everything - unconditionally before parsing) and into generator.py, applied only to each message's actual text at render time. Both the original and censored text are rendered for any message where they differ (skipped entirely for clean messages, to avoid bloating pages that don't need it); CSS shows/hides between them based on a data-profanity-filter attribute, exactly like the existing dark-mode mechanism, so switching is instant with no reload. Added a toggle button (🤬) in the topbar next to the theme toggle, defaulting to off (original text shown) and persisted via localStorage. This also fixes the original version's real behavioral bug: it censored the whole chat log text including <attached: ...> tags and other non-message content, since it ran before parsing separated them out. Added tests/test_generator.py covering both the dual-render helper and full page generation; added a sanitizer regression test locking in that sanitize_text no longer touches profanity. Full suite (17 tests) passes.
…ting Three mild swears inserted into testchat/chat.zip's _chat.txt at natural points in the conversation (shit/damn/hell - all in better_profanity's default word list), so dragging this fixture into the running app gives something real to toggle between original and censored text, not just the synthetic strings the automated tests use.
_render_text was skipping the .text-censored div entirely when a message had no profanity (an attempted size optimization). But the toggle CSS unconditionally hides .text-original and shows .text-censored when the filter is on - so any message without a .text-censored sibling rendered blank the moment the filter was switched on, even though it had nothing to redact. Found via manual testing: only messages that actually contained profanity survived toggling; everything else disappeared. Now always renders both divs, even when identical. Updated the two tests that had locked in the buggy single-div behavior for clean messages.
- Checklist feature
Owner
Author
|
@sandeshp Comments ? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reintroduces profanity redaction as a proper feature. The original version was merged directly to
mainwithout review and was actually broken - it crashed on import for every fresh install, sincebetter_profanitywas never declared as a dependency. It also censored unconditionally with no way to see original text.What changed
Dependency fix:
better_profanityis now declared inpyproject.toml. This alone fixes the crash currently live onmain.Opt-in, not forced: the original always censored every message. Now it's a 🤬 toggle button beside the existing 🌙 theme toggle — same instant, no-reload mechanic, persisted in
localStorage, defaulting to off so original text shows unless you opt in.Architecture correction: censoring moved from
sanitizer.pytogenerator.py. Previously it ran over the entire raw chat file before parsing - usernames,<attached: ...>tags, everything — rather than just message text. Now it's applied per-message, post-parsing.