The hash fast-path compared the JSON file hash to a stored value and exited
early when unchanged. But it only tracked the *file*, not the *table*: if rows
went missing from the DB while the JSON stayed the same, the sync skipped
entirely and never restored them.
Always run the full diff now. It costs one SELECT plus an in-memory compare —
~0.1s for 13k E-Taxes Item Group rows — so there's no reason to skip it. The
diff is driven by DB state, making the sync self-healing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These doctypes are now populated exclusively from master_data JSON via the
hash-diff sync, so block create/write/delete in the UI and API by dropping
those permissions. The sync itself writes at the DB level (db_insert /
set_value / delete_doc with ignore_permissions) so it is unaffected. Also
marks classification_name read_only and drops allow_import/allow_rename.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Frappe imports every .json under app/fixtures/ on every migrate via
DELETE+INSERT (force=True in import_doc), so 13,269 E-Taxes Item Group rows +
116 Classification code rows added ~35s to each migrate even though the
fixtures hook was empty. Replace with a hash-diff sync that fast-paths via a
stored file hash and otherwise applies only the delta (insert new with
db_insert() bulk path for first-run, update changed via set_value, smart-delete
removed with disabled fallback). Also removes the now-redundant
master_data_loader.py (install-only bulk insert that the sync supersedes) and
the original fixture files so they stop being imported on migrate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops the obl-pact-list integration entirely: the doctype directory,
loader/bulk-loader entries, REFERENCE_SECTIONS row, reference-list
allow-list entry, and the existence-check guard on Presented
Certificate's sazis_code. sazis_code is now a plain Data field —
no Link target to validate against.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The data loading dialog was opened via frappe.msgprint, which reuses the
shared frappe.msg_dialog. We then rewrote .modal-body via $.html(...) on
each progress tick, destroying the DOM that frappe.show_progress also
relies on. After hide(), Bootstrap left a modal-backdrop behind and the
whole page became unclickable until reload.
Switch to the dedicated show_loading_dialog_settings /
update_loading_message_settings / hide_loading_dialog_settings dialog
that already exists in this file, so msg_dialog is never touched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces "Azeri" / "Azerbaijani" / "Azerbaijan" with the native form
"Azərbaycan" in labels, descriptions, comments and docs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Recent ERPNext rejects assigning a group-type Customer Group to a Customer
("Cannot select a Group type Customer Group"); customer_group is also no
longer mandatory. The integration fell back to "All Customer Groups" (a
group node), so every auto-created party failed.
- add invoice_az/utils.py:resolve_customer_group() — returns the configured
group only if it exists and is not a group node, else None
- api.py / vat_api.py: skip customer_group when no valid non-group value;
create the Customer without a group instead of forcing "All Customer Groups"
- add link_filters (is_group=0) to all Customer/Supplier Group pickers in
E-Taxes Settings and the mapping child tables
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Convert add_menu_item to add_inner_button on E-Taxes Customer/Supplier/
Item/Unit lists, Purchase Order, Sales Order, and Journal Entry list
views so the load/import actions are visible directly instead of being
hidden under the "..." menu.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bulk import is now two-phase:
Phase A — parallel network. Up to BULK_IMPORT_PARALLELISM (=10)
ThreadPoolExecutor workers call get_employee_detail() concurrently.
Workers run in stateless mode via the new _post_amas helper and the
_state= parameter threaded through make_amas_request and the seven
detail helpers (get_edit_form_data, get_staff_data, get_person_data,
get_address_data, get_doc_main_data, get_contract_attachments,
download_contract_file). They share an in-memory snapshot of the
ƏMAS session/csrf/cookies and never touch the DB. If any worker is
rejected with HTTP 419 or a response-level CSRF code the whole
import aborts cleanly with a hint to lower BULK_IMPORT_PARALLELISM.
Phase B — sequential DB writes. create_single_employee_from_amas
takes a new prefetched_detail kwarg; the worker loop passes the
Phase-A payload through so the per-employee 7-call fetch is skipped.
Frappe ORM is not thread-safe, so writes stay sequential — that's
where the previous flow spent most of its time anyway.
Concurrency control:
- Asan Login picks up two new hidden Check fields,
amas_import_running and amas_import_cancel_requested.
- import_bulk_employees refuses to enqueue if the running flag is
already set, returning {success: False, already_running: True,
message}. The flag is set BEFORE frappe.enqueue (atomic guard
against fast double-clicks) and cleared in the worker's finally
block no matter how it exits.
- New whitelisted endpoints get_amas_import_status and
cancel_amas_import expose the flag to the UI. The worker checks
the cancel flag between Phase A futures and between Phase B
iterations.
Realtime events upgraded:
- amas_import_progress now carries phase: "fetch" | "save".
- amas_import_complete carries cancelled and aborted flags.
Frontend (employee.js):
- Bootstrap-4.6 stacked-modal fix re-applies modal-open class on
body when a nested modal closes (cancel-confirm over progress)
so the underlying backdrop isn't orphaned.
- reattach_amas_import_if_running re-binds the realtime listeners
on listview onload by polling get_connected_asan_logins +
get_amas_import_status, so refreshing /app/employee while an
import is in flight still shows the progress bar + Cancel button.
Plus the side fixes from the same session: connect_amas now opens
the org-picker dialog on success, error humanizer for common ƏMAS
codes, skip the dashboard prefetch when the cached CSRF is still
good. Schema changes need bench migrate.
- Always show "Select ƏMAS Organization" dialog after MyGovID re-auth in
the Employee → Load from AMAS flow. Previously auto-picked the first
cert, which on multi-cert users was the personal cert without rights
to the employee report — caused permission_error.
- humanize_amas_error() translates raw codes ('permission_error',
unknown) into user-facing messages. Permission failures return
permission_error: True so the frontend can offer a Reconnect button
that runs full re-auth + org selection.
- make_amas_request: drop the eager /core.dashboard fetch before every
call. Use the cached CSRF token + cookies (already updated from
Set-Cookie on each response). On real CSRF rejection (HTTP 419 or
response code CSRF/TOKENS_ARE_NOT_SAME/CSRF_TOKEN_MISMATCH), refresh
once and retry. Same simplification in get_amas_accounts.
- _process_bulk_employees_import: commit once after the loop instead of
per row, per CLAUDE.md guidance.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move E-Taxes Item Group (~13k rows) and Classification code (~116 rows)
out of the `fixtures` hook. Re-syncing these on every `bench migrate` was
the dominant cost of the fixture sync step.
Add invoice_az.master_data_loader.load_master_data() that bulk-inserts
from the existing JSON files; it runs once via after_install and is
idempotent (skips rows that already exist by primary key) so it can be
re-invoked from the bench console to refresh master data.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
load_company_bank_accounts now makes every non-closed e-taxes bank
record usable inside ERPNext, not just a read-only E-Taxes Bank Account
cache row:
- Bank (global, one per unique bankName) — idempotent insert.
- GL Account under parent "223 Bank hesablaşma hesabları", named
"<CUR> <IBAN>", account_type=Bank, account_currency from e-taxes.
Parent discovered by account_number=223 with a name-prefix fallback.
- Bank Account linking Bank + GL Account + Company, is_company_account=1,
iban + bank_account_no = the AZ IBAN.
Closed accounts (status=C) are left at cache-row only; no native
records are created for them.
After the loop, unused AZ CoA placeholder accounts (names like
"AZN AZXXXXXXXXXXXXXXXXXXXXXXXXXX" under the bank group) are deleted
so the chart isn't cluttered with unmatched templates. Per-record
failures go to the "E-Taxes Bank Native Materialize" error log and
don't abort the overall load.
Adds a "Data Loading" dialog on Company form that pulls 6 types of
taxpayer reference data from E-Taxes into dedicated doctypes, plus
in-form summary tables inside the Tax Policy tab.
New DocTypes (all with company link, Accounts User read-only,
System Manager full CRUD):
- E-Taxes Object (taxpayer registered locations)
- E-Taxes Cash Register (kassa aparatları with object FK)
- E-Taxes POS Terminal (with auto-derived object from registration number)
- E-Taxes Bank Account
- E-Taxes Obligation Pact (sazişlər / oil fields)
- E-Taxes Presented Certificate (with sazis_code FK to Obligation Pact)
Backend (invoice_az/company_api.py):
- 6 per-doctype loaders + shared _auth_headers / _etaxes_request helpers
implementing CLAUDE.md §10 401-retry-once pattern
- Cash Register / POS Terminal loaders walk the hasMore pagination
- load_company_data_bulk enqueues a background job that runs selected
loaders sequentially and streams progress over Socket.IO
(company_data_loading_progress / _complete events)
- get_company_reference_list exposes first N rows per doctype for the
Tax Policy tab tables
Frontend (invoice_az/client/company.js):
- Full ETaxes auth module inlined (CLAUDE.md §2) so ASAN Imza re-auth
works from Company form without relying on other forms' state
- "Data Loading" button opens a dialog of checkboxes (DATA_LOADERS
declarative array) — one bulk call, one floating show_progress bar
- Six reference tables rendered inside Tax Policy tab sections,
pattern copied from e-taxes_settings (card-section + Refresh +
"View All" only when total > rows)
CLAUDE.md:
- Rewrote §3 backend template with 401 retry + explicit commit
- Added §10–§16: 401 auto-refresh, two response shapes,
db.commit() standard, log title convention, frontend unauthorized
handling, field naming for reference vs fixture doctypes,
Socket.IO background job pattern
hooks.py:
- Register client/company.js under doctype_js["Company"]
.gitignore:
- Exclude *.har (may contain Bearer tokens / PII)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
vat_18_percent_with_amount stores the total INCLUDING VAT (e.g. 118),
but the e-taxes API vat18 field expects just the tax amount (e.g. 18).
Also calculates net cost/pricePerUnit for VAT-inclusive pricing (ƏDV daxil 18%).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Checkbox values (load_items, load_units, load_customers, load_suppliers)
were collected in the dialog but never passed through the loading pipeline,
causing all data types to always be created regardless of user selection.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove invoice_az.bundle.js (not needed, app uses doctype_js)
- Remove OPTIMIZATION_SUMMARY.md (outdated)
- Remove claude.md (replaced by CLAUDE.md)
The patches.txt file is sufficient to make the app recognizable by Frappe.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fixed two critical issues preventing app installation:
1. Removed non-existent e_taxes_parties_list.js from hooks.py
2. Added missing patches.txt file (required for Frappe app detection)
3. Added minimal bundle.js file to satisfy build system
Without patches.txt, the app was not recognized as a valid Frappe app,
causing esbuild to fail with "paths[0] must be of type string" error.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Ali