Track new import_bulk_employees response shape (parallel refactor)

invoice_az's amas_api was rewritten so import_bulk_employees:
  - takes a per-Asan-Login lock (Asan Login.amas_import_running)
    BEFORE enqueueing, returning {already_running: True} if the lock
    is held;
  - inside the worker, runs Phase A (parallel ThreadPool, 10 detail
    fetches concurrently) then Phase B (sequential DB writes with
    prefetched payloads), so realtime amas_import_progress now
    carries phase: "fetch" | "save".

Call signature is unchanged; only the return value differs and the
internal mechanism is faster. Wizard updates:

- _materialize_amas_employees now captures the response and dumps
  it into the "Jey Wizard trace: amas enqueued" breadcrumb (success,
  enqueued, total / already_running flag, full message). Spotting a
  stale lock at finalize-time on a fresh site is now possible from
  the Error Log alone.

- If the first call comes back with already_running: True (would be
  a stuck lock from a prior failed run — improbable on a freshly-
  dropped site but cheap to defend against), we force-clear the two
  Asan Login flags via direct db.set_value and retry once. The
  second outcome lands in the same trace.

- Comment on the function rewritten to describe the two-phase
  parallel/sequential flow so the next reader doesn't grep
  invoice_az to figure out where the work happens.

JS side: no changes — the wizard fires-and-forgets; in-app progress
UI lives in invoice_az/client/employee.js (already updated upstream).
This commit is contained in:
Ali 2026-05-01 13:56:14 +00:00
parent 38293a8de5
commit 10c07dfc0b
3 changed files with 53 additions and 7 deletions

View File

@ -1 +1 @@
__version__ = "0.1.18"
__version__ = "0.1.19"

View File

@ -963,9 +963,24 @@ def _extract_phone(profile, preferred=None):
def _materialize_amas_employees(company_name):
"""If the user loaded ƏMAS employees during the wizard, enqueue bulk import now
that Company exists. Skipped silently when the cache is empty (i.e. user skipped
the AMAS step)."""
"""If the user loaded ƏMAS employees during the wizard, enqueue bulk import
now that Company exists. Skipped silently when the cache is empty (i.e.
user skipped the AMAS step).
invoice_az's import_bulk_employees runs in two phases inside its background
job: a parallel ThreadPoolExecutor fetches per-employee detail (CSRF +
7 ƏMAS calls per row), then a sequential loop does the DB writes with the
prefetched payloads. For us, that's still fire-and-forget — the user lands
on /app and the Employee list shows progress via realtime
`amas_import_progress` / `amas_import_complete` events.
The bulk endpoint also takes a per-Asan-Login lock
(Asan Login.amas_import_running) before enqueueing. If that lock is set
(e.g. a stuck previous run), it returns {success: False, already_running:
True}. We capture and trace that response so we can spot it from the
Error Log alone at this point in the wizard a stale lock would be
surprising on a freshly-dropped site.
"""
cache = frappe.get_single("Jey Wizard Etaxes Cache")
raw = (cache.amas_selected_employees_json or "").strip()
if not raw:
@ -1005,14 +1020,45 @@ def _materialize_amas_employees(company_name):
try:
from invoice_az.amas_api import import_bulk_employees
import_bulk_employees(
result = import_bulk_employees(
asan_login_name=asan_login,
employees_data=json.dumps(employees, ensure_ascii=False),
company=company_name,
create_designation=create_designation,
)
# New (post-parallel-import refactor) response shape:
# {success: True, enqueued: True, total: N}
# {success: False, already_running: True, message: "..."}
# Old call signature is unchanged, only the return value matters.
# If the lock is stuck on a fresh site, force-clear and retry once.
if isinstance(result, dict) and result.get("already_running"):
try:
frappe.db.set_value(
"Asan Login", asan_login,
{"amas_import_running": 0, "amas_import_cancel_requested": 0},
update_modified=False,
)
frappe.db.commit()
result = import_bulk_employees(
asan_login_name=asan_login,
employees_data=json.dumps(employees, ensure_ascii=False),
company=company_name,
create_designation=create_designation,
)
except Exception as exc:
frappe.log_error(
f"amas import lock force-clear failed for {asan_login}: {exc}",
"Jey Wizard materialize",
)
frappe.log_error(
f"company={company_name} asan_login={asan_login} employees={len(employees)} create_designation={create_designation}",
json.dumps({
"company": company_name,
"asan_login": asan_login,
"employees": len(employees),
"create_designation": create_designation,
"result": result if isinstance(result, dict) else {"raw": str(result)},
}, ensure_ascii=False, indent=2),
"Jey Wizard trace: amas enqueued",
)
except Exception as exc:

View File

@ -10,7 +10,7 @@ frappe.provide("jey_wizard");
// Bump this string in every commit that changes wizard code. Displayed in the badge so
// we can tell at a glance which version is actually running on a given machine. Kept in
// sync with __version__ in jey_wizard/__init__.py.
const JEY_WIZARD_VERSION = "0.1.18";
const JEY_WIZARD_VERSION = "0.1.19";
// Wipe Frappe + ERPNext default slides so their `before_load`/`after_load` listeners
// don't try to mutate a wizard that isn't slide-based anymore.