The hash fast-path compared the JSON file hash to a stored value and exited
early when unchanged. But it only tracked the *file*, not the *table*: if rows
went missing from the DB (manual delete, restore from an older dump) while the
JSON stayed the same, the sync skipped entirely and never restored them — so
`bench migrate` silently failed to recreate missing master-data rows.
Always run the full diff now. It costs one SELECT plus an in-memory compare —
measured at ~0.1s for 13k rows — so there's no reason to skip it. The diff is
driven by DB state, making the sync self-healing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
create_item_groups() ran a 26-record exists-check loop on every migrate
(the "Группа X уже существует, пропускаем" spam in migrate logs). Replace
with the master_data hash-diff sync: fast-paths via stored file hash, applies
only the delta otherwise.
Item Group is a tree doctype shared with ERPNext/user groups, so the sync
gains two options: is_tree=True (insert via db_insert, rebuild lft/rgt once
at the end) and manage_deletes=False (never delete rows absent from the JSON,
since the JSON is only the AZ tax-specific subset).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous commit moved Main type of activity data to master_data/ but left
the original fixture file in place — Frappe's sync_fixtures reads every .json
in app/fixtures/ regardless of the hooks.py fixtures list, so the file was
still being imported on every migrate (3499 DELETE+INSERTs). Delete it.
Also adds a bulk-insert fast path (db_insert chunks of 500) to sync_master_data
for the large first-run case on fresh sites, so the new sync is faster than
the legacy fixture import even when populating an empty table.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fixture import does DELETE+INSERT for every row on every migrate (force=True
in frappe's import_doc), so 3499 rows of Main type of activity added minutes
to each migrate. Replace with a hash-diff sync that fast-paths via stored
file hash and otherwise applies only the delta (insert new, update changed,
smart-delete removed with disabled-fallback).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ali