perf(shift-type): stop and clean up last_sync_of_checkin versions - #396
Open
sagarvora wants to merge 3 commits into
Open
perf(shift-type): stop and clean up last_sync_of_checkin versions#396sagarvora wants to merge 3 commits into
last_sync_of_checkin versions#396sagarvora wants to merge 3 commits into
Conversation
The biometric checkin sync PUTs `last_sync_of_checkin` on every run, creating a `Version` record each time. Set `ignore_version` in `before_validate` when a resource PUT updates only that field.
Bail out before inspecting the request when `last_sync_of_checkin` hasn't actually changed.
sagarvora
marked this pull request as draft
August 13, 2026 13:28
Removes historical `tabVersion` rows that only record a `last_sync_of_checkin` bump on Shift Type — the versions the hook now prevents from being created. Not in `patches.txt`; run via `bench execute` (`estimate` first, it only reads). Keyset-paginated over the indexed `ref_doctype` slice and deleted in small autocommitted batches, so it is resumable and safe on a large table. Shape matching is done in Python since the `Version.data` key-set varies by the frappe version that wrote the row.
last_sync_of_checkin synclast_sync_of_checkin versions
sagarvora
marked this pull request as ready for review
August 13, 2026 16:28
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Biometric checkin sync PUTs
last_sync_of_checkinto/api/resource/Shift Type/<name>on every run — oneVersionrecord per sync, forever.Stop creating them
New
before_validatehook on Shift Type setsdoc.flags.ignore_versionwhen a resource PUT updates only that field.has_value_changed(last_sync_of_checkin), so the request is only inspected when the timestamp actually moved.frappe.api.v1.get_request_form_data— the sync sends a raw JSON body with noContent-Type, somake_form_dictleavesform_dictempty and onlyrequest.get_data()recovers it. Also covers form-encodeddata=<json>and query params.PUTon/api/resource/,/api/v1/resource/,/api/v2/document/. Desk saves, background jobs and/api/method/*are untouched.last_sync_of_checkinplus any other field versions normally.Clean up the ones already there
csf_tz/patches/delete_shift_type_checkin_sync_versions.py, deliberately not inpatches.txt— deleting history is a judgement call. Run viabench execute;estimatefirst, it only reads.nameover the indexedref_doctypeslice, deleted in small autocommitted batches. Resumable, and work is proportional to the Shift Type rows, not the table.Version.data's key-set depends on the frappe version that wrote the row (v11 emits four keys, v15 six, bare{"changed": …}also occurs), so it requireschangedto hold only our field and no other key to carry a value. An unrecognised key with content means the row is kept.reclaim_spacerebuilds the table afterwards to return space to the filesystem — optional, and wants a quiet window.Measured 7k–13k rows/s deleting locally, so under two hours for 35M rows — one bench, not a promise.
Verification
Hook: fires for the sync's exact shape (raw body, no content-type) on all three resource paths; stays off for unchanged values, extra fields, other-fields-only, empty/absent body, POST, GET,
/api/method/frappe.client.save, desk/background saves, and malformed payloads.Patch: deletes the v15/v11/
changed-only/null-previous shapes; keeps two-field diffs,row_changed/added/removed/data_import/updater_reference, unknown-key-with-content, insert markers, and malformed input. End-to-endVersioncounts and idempotent re-runs confirmed, plus batch boundaries across multi-batch walks.