26 KiB

Raw Blame History

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Overview

Invoice Az is a Frappe application that integrates with Azerbaijan's e-taxes.gov.az system for:

Downloading and importing purchase invoices from e-taxes
Downloading and importing sales invoices from e-taxes
Sending sales invoices to e-taxes with ASAN Imza signing
Importing VAT Account operations and creating Journal Entries
Managing authentication via ASAN Login with automatic token renewal

⚠️ CRITICAL RULES FOR E-TAXES INTEGRATION ⚠️

READ THIS BEFORE IMPLEMENTING ANY E-TAXES API CALLS!

1. Authentication Pattern (MANDATORY)

❌ WRONG - Direct API call:

function my_function(frm) {
    frappe.call({
        method: 'invoice_az.my_api.some_method',
        // ... this will fail with "Authentication expired"
    });
}

✅ CORRECT - Always wrap with authentication check:

function my_function(frm) {
    ETaxes.auth.checkAndProcess(function() {
        // Your code here - only runs after auth is verified
        frappe.call({
            method: 'invoice_az.my_api.some_method',
            // ... works correctly
        });
    });
}

Why? ETaxes.auth.checkAndProcess() does:

Checks if token is still valid
If expired → prompts user to login via ASAN Imza
Only after successful auth → executes your callback
Prevents "Authentication expired" errors

2. ETaxes Module Inclusion (MANDATORY)

⚠️ ETaxes is NOT a global module! It must be defined in EACH JavaScript file that uses it.

❌ WRONG - Assuming ETaxes exists:

// my_custom_form.js
frappe.ui.form.on('My DocType', {
    refresh: function(frm) {
        ETaxes.auth.checkAndProcess(function() {
            // ERROR: ETaxes is not defined
        });
    }
});

✅ CORRECT - Include ETaxes module at the top of your JS file:

Copy the entire ETaxes module from sales_invoice.js (lines 1-564) to the beginning of your JS file:

// ======= COPY THIS ENTIRE BLOCK =======
// From sales_invoice.js lines 1-564:
const ETaxes = {
    PROGRESS_UPDATE_DELAY: 50,
    AUTH_POLL_INTERVAL: 6000,
    AUTH_MAX_ATTEMPTS: 20,
    // ... (full module definition)
};
ETaxes.utils = { /* ... */ };
ETaxes.dialogs = { /* ... */ };
ETaxes.auth = { /* ... */ };
// ======= END COPY =======

// Now your code can use ETaxes:
frappe.ui.form.on('My DocType', {
    refresh: function(frm) {
        ETaxes.auth.checkAndProcess(function() {
            // ✅ Works correctly
        });
    }
});

Files that need ETaxes module:

✅ sales_invoice.js - has it (564 lines)
✅ purchase_order.js - has it
✅ supplier.js - has it (after fix)
❌ Any new JS file - MUST include it!

3. Backend API Pattern (MANDATORY)

All backend functions that call E-Taxes API MUST follow this pattern:

import frappe
import requests
from invoice_az.auth import record_etaxes_activity, get_default_asan_login

@frappe.whitelist()
def my_etaxes_function(doc_name):
    """My E-Taxes API function"""
    try:
        # Step 1: ALWAYS record activity (for token renewal)
        record_etaxes_activity()

        # Step 2: Get document
        doc = frappe.get_doc("My DocType", doc_name)

        # Step 3: Validate (business logic checks)
        validation = validate_my_document(doc)
        if not validation.get("valid"):
            return {
                "success": False,
                "message": validation.get("message")
            }

        # Step 4: Get authentication token
        asan_login = get_default_asan_login()
        if not asan_login.get("found") or not asan_login.get("main_token"):
            return {
                "success": False,
                "message": "E-Taxes authentication not found. Please login via ASAN."
            }

        token = asan_login.get("main_token")

        # Step 5: Build headers with token
        headers = DEFAULT_HEADERS.copy()
        headers["x-authorization"] = f"Bearer {token}"

        # Step 6: Make API request
        response = requests.post(
            "https://new.e-taxes.gov.az/api/...",
            data=json.dumps(payload),
            headers=headers,
            timeout=60
        )

        # Step 7: Handle common errors
        if response.status_code == 401:
            return {
                "success": False,
                "message": "Authentication expired. Please login again."
            }

        if response.status_code == 500:
            return {
                "success": False,
                "message": "E-Taxes service error. Please try again later."
            }

        if response.status_code == 404:
            return {
                "success": False,
                "message": "Resource not found."
            }

        response.raise_for_status()

        # Step 8: Process response
        result = response.json()

        # Step 9: Return success
        return {
            "success": True,
            "data": result
        }

    except requests.exceptions.RequestException as e:
        frappe.log_error(
            f"Network error: {str(e)}",
            "E-Taxes API Error"
        )
        return {
            "success": False,
            "message": f"Network error: {str(e)}"
        }
    except Exception as e:
        frappe.log_error(
            f"Unexpected error: {str(e)}\n{frappe.get_traceback()}",
            "E-Taxes API Error"
        )
        return {
            "success": False,
            "message": f"Error: {str(e)}"
        }

Key points:

@frappe.whitelist() - ALWAYS required for frontend calls
record_etaxes_activity() - ALWAYS first (enables token auto-renewal)
get_default_asan_login() - Get token
headers["x-authorization"] = f"Bearer {token}" - Required for auth
Handle 401, 500, 404 errors explicitly
ALWAYS return {"success": bool, "message": str} format
ALWAYS use try-except with proper error logging

4. Constants and URLs

Base URL:

BASE_URL = "https://new.e-taxes.gov.az"

Common endpoints:

Auth: /api/po/auth/public/v1/*
Invoices: /api/po/invoice/public/v2/invoice
Sign: /api/po/invoice/public/v1/invoice/sign/withAsanImza
Serial: /api/po/invoice/public/v1/generateSerialNumber/defaultInvoice
Agricultural Serial: /api/po/invoice/public/v1/act/generateSerialNumber/agriculturalProductsAct
Metal Scrap Serial: /api/po/invoice/public/v1/act/generateSerialNumber/metalProductsAct
Tire Disposal Serial: /api/po/invoice/public/v1/act/generateSerialNumber/tireProductsForDisposalAct
Plastic Disposal Serial: /api/po/invoice/public/v1/act/generateSerialNumber/plasticProductsForDisposalAct
Rawhide Supply Serial: /api/po/invoice/public/v1/act/generateSerialNumber/rawhideSupply
Other Product Receipt Serial: /api/po/invoice/public/v1/act/generateSerialNumber/otherProductReceiptAct
All Acts: /api/po/invoice/public/v1/act
Remove Drafts: /api/po/invoice/public/v1/common/removeDrafts
VAT Operations: /api/po/vatacc/public/v1/operation/find.outbox
Taxpayer by FIN: /api/po/profile/public/v1/taxpayer/findByFinAndPassport

Standard headers:

DEFAULT_HEADERS = {
    "Accept": "application/json, text/plain, */*",
    "Accept-Language": "en-US,en;q=0.9",
    "Content-Type": "application/json",
    "Cache-Control": "no-cache",
    "Pragma": "no-cache"
}

5. Error Handling

HTTP Status Codes:

200 - Success
401 - Token expired → Show "Please login again"
404 - Resource not found
500 - E-Taxes server error → Show "Try again later"

Always check in this order:

if response.status_code == 401:
    return {"success": False, "message": "Authentication expired. Please login again."}

if response.status_code == 500:
    return {"success": False, "message": "E-Taxes service error. Please try again later."}

if response.status_code == 404:
    return {"success": False, "message": "Not found"}

response.raise_for_status()  # For other errors

6. Frontend Button Pattern

Always show buttons conditionally:

frappe.ui.form.on('My DocType', {
    refresh: function(frm) {
        // Only for submitted documents
        if (frm.doc.docstatus === 1) {
            add_etaxes_buttons(frm);
        }
    }
});

function add_etaxes_buttons(frm) {
    // Check status and show appropriate buttons
    if (frm.doc.etaxes_status === 'Sent and Signed') {
        frm.add_custom_button(__('View Details'), function() {
            // Show details
        }, __('E-Taxes'));
    } else if (frm.doc.etaxes_status === 'Created, not signed') {
        frm.add_custom_button(__('Sign with ASAN Imza'), function() {
            // Sign document
        }, __('E-Taxes'));
    } else {
        frm.add_custom_button(__('Send to E-Taxes'), function() {
            // Send to E-Taxes
        }, __('E-Taxes'));
    }
}

7. Build Process

After modifying JavaScript files:

# Build assets
cd /home/frappe/frappe-bench
bench build --app invoice_az

# Clear cache
bench --site site1 clear-cache

After modifying Python files:

# Just clear cache (no build needed)
bench --site site1 clear-cache

8. Common Mistakes to Avoid

❌ Don't do this:

Call E-Taxes API without ETaxes.auth.checkAndProcess()
Forget to include ETaxes module in new JS files
Skip record_etaxes_activity() in backend functions
Forget to handle 401/500/404 errors
Use app_include_js in hooks.py (doesn't work for doctype-specific code)
Make API calls without Bearer token in headers

✅ Always do this:

Wrap ALL E-Taxes operations in ETaxes.auth.checkAndProcess()
Copy ETaxes module (564 lines) to every new JS file
Call record_etaxes_activity() first in every API function
Handle 401, 500, 404 explicitly
Return consistent {success, message} format
Add x-authorization: Bearer {token} header
Build assets after JS changes

9. Example Files to Reference

Backend (Python):

✅ send_sales_api.py - Perfect example of all patterns
✅ send_purchase_api.py - Agricultural act example
✅ supplier_api.py - Simple API call example
✅ auth.py - Authentication implementation

Frontend (JavaScript):

✅ sales_invoice.js - Complete ETaxes module (lines 1-564)
✅ purchase_invoice.js - Button patterns
✅ supplier.js - Minimal working example

When in doubt: Look at send_sales_api.py + sales_invoice.js - they contain all correct patterns!

Development Commands

Linting and Code Quality

# Run ruff for Python linting
ruff check invoice_az/

# Run ruff format
ruff format invoice_az/

# Run pre-commit hooks (includes ruff, eslint, prettier, pyupgrade)
pre-commit run --all-files

Testing

# Run unit tests for a specific doctype
bench run-tests --app invoice_az --doctype "E-Taxes Item"

# Run all tests for the app
bench run-tests --app invoice_az

Development

# Clear cache after making changes
bench clear-cache

# Restart workers after API changes
bench restart

# Watch logs
bench --site [site-name] console

Architecture

Core Modules

Authentication Module (invoice_az/auth.py)
- ASAN Login integration for Azerbaijan government authentication
- Token management with automatic renewal every 4 minutes via cron job
- Activity tracking to prevent unnecessary token renewals
- Retry logic with exponential backoff for failed requests
- Token validity checking and automatic refresh on 401 errors
- Full authentication flow: start auth → poll status → get certificates → select taxpayer
- Key functions: renew_token(), get_auth_token(), check_auth_status(), get_certificates(), choose_taxpayer(), check_token_validity(), handle_unauthorized_request(), handle_authentication(), poll_auth_status(), select_certificate(), select_taxpayer()
Purchase API (invoice_az/api.py)
- Download and import purchase invoices from e-taxes
- Party, item, and unit mapping system
- Separate Customer and Supplier mapping systems
- Create Purchase Orders and Purchase Invoices
- E-Taxes Purchase tracking records
- Reference data management (sync items, units, parties from e-taxes)
- Batch invoice processing for loading reference data
- Auto-matching for items, units, customers, suppliers by name similarity
Sales Import API (invoice_az/sales_api.py)
- Download and import sales invoices from e-taxes outbox
- Customer mapping system (separate from suppliers)
- Create Sales Orders and Sales Invoices
- E-Taxes Sales tracking records
- Azerbaijani character normalization for matching
Sales Send API (invoice_az/send_sales_api.py)
- Send Sales Invoices to e-taxes system
- Two-step workflow: create draft, then sign with ASAN Imza
- Generate serial numbers from e-taxes
- Build invoice payload with product codes and VAT fields
- E-Taxes Sales Outbox tracking
- Cancel/remove draft invoices functionality
- Retry signing for failed/pending invoices
- Get customer objects by TIN for delivery address selection
VAT Operations API (invoice_az/vat_api.py)
- Import VAT Account operations from e-taxes
- Import all operations (both income and expense types)
- Create Journal Entries with configurable account mappings
- VAT Account Mappings by operation type and expense/income type (Sub uçot hesabı → Sub uçot hesabı, Sub uçot hesabı → Büdcə, etc.)
- Classification Code support - allows specific account mappings per tax code with priority:
  - Priority 1: Mapping with matching operation_type + expense_income + classification_code
  - Priority 2: Default mapping with matching operation_type + expense_income (empty classification_code)
- Required mapping configuration - operation fails with error if mapping not found
- Smart party assignment: adds Customer/Supplier only for Receivable/Payable accounts
- Customer lookup by TIN
- E-Taxes VAT Operations tracking
- Key functions: get_accounts_for_operation_type(), find_account_by_number(), create_journal_entry_from_vat_operation(), map_operation_type()
VAT Operations Hooks (invoice_az/vat_operations.py)
- Delete related E-Taxes VAT Operations when Journal Entry is deleted/cancelled

Document Integration (hooks.py)

Extended Doctypes:

Purchase Order - import from e-taxes, delete hooks
Purchase Invoice - import from e-taxes
Sales Order - import from e-taxes, delete hooks
Sales Invoice - send to e-taxes, delete hooks
Journal Entry - import VAT operations, delete hooks
E-Taxes Settings - on_update hook to sync mapped statuses

Client-side Scripts (invoice_az/client/):

purchase_order.js - E-taxes import buttons and dialogs
purchase_invoice.js - E-taxes import functionality
sales_order.js - E-taxes import buttons and dialogs
sales_invoice.js - Send to e-taxes, sign document, retry signing, cancel buttons
journal_entry.js - VAT operations import functionality with operation type display
e_taxes_items_list.js - Sync items from e-taxes
e_taxes_suppliers_list.js - Sync suppliers from e-taxes
e_taxes_customers_list.js - Sync customers from e-taxes
e_taxes_unit_list.js - Sync units from e-taxes
etaxes_common.js - Shared utilities

Database Schema

Authentication:

Asan Login - Stores ASAN credentials, tokens, certificates, auth status

Settings:

E-Taxes Settings - Global settings, status mappings, item/party/customer/supplier/VAT account mappings

E-Taxes Entities (cached from e-taxes):

E-Taxes Item - Products/services from e-taxes with EQM codes
E-Taxes Parties - All parties (legacy, for purchases)
E-Taxes Suppliers - Supplier companies for purchases
E-Taxes Customers - Customer companies for sales
E-Taxes Unit - Units of measurement
E-Taxes Item Group - Product group codes (fixtures)
Classification code - Tax classification codes from e-taxes (code and description)

Mapping Tables:

E-Taxes Item Mapping - Links ERPNext Items to e-taxes items
E-Taxes Party Mapping - Links ERPNext parties to e-taxes parties (legacy)
E-Taxes Supplier Mappings - Links ERPNext Suppliers to e-taxes suppliers
E-Taxes Customer Mappings - Links ERPNext Customers to e-taxes customers
E-Taxes Unit Mapping - Links ERPNext UOM to e-taxes units
E-Taxes VAT Account Mapping - Maps VAT operation types to Chart of Accounts (debit/credit), with expense/income type and optional classification code filtering

Tracking Records:

E-Taxes Purchase - Tracks imported purchase documents
E-Taxes Sales - Tracks imported sales documents (from e-taxes)
E-Taxes Sales Outbox - Tracks sent sales invoices (to e-taxes)
E-Taxes VAT Operations - Tracks imported VAT operations

Key Features

Token Renewal: Automatic token renewal via scheduler (every 4 minutes) with activity check
Token Validity Check: Verify token validity before API calls, auto-refresh on 401 errors
Activity Tracking: Records user activity to optimize token renewals
Entity Mapping: Maps ERPNext items, parties, suppliers, customers, and units to e-taxes equivalents
Auto-Matching: Automatic matching of items, units, customers, suppliers by name similarity
Bulk Operations: Support for syncing multiple entities at once
Batch Processing: Load reference data from invoices in batches
Error Handling: Comprehensive error logging and user-friendly error messages
Azerbaijani Support: Character normalization for Ə, Ü, Ö, Ğ, İ, Ç, Ş characters
Two-step Signing: Create draft invoice, then sign with ASAN Imza
Retry Signing: Retry signing for failed/pending invoices
VAT Import: Import VAT account operations as Journal Entries with configurable account mappings
VAT Account Mappings: Configure debit/credit accounts by operation type and expense/income type (Sub uçot hesabı → Sub uçot hesabı, Sub uçot hesabı → Büdcə, etc.)
Smart Party Assignment: Automatically adds party (Customer/Supplier) only for Receivable/Payable accounts

API Endpoints (E-Taxes)

Base URL: https://new.e-taxes.gov.az

/api/po/auth/public/v1/* - Authentication endpoints
/api/po/invoice/public/v2/invoice/find.inbox - Purchase invoices
/api/po/invoice/public/v2/invoice/find.outbox - Sales invoices (sent)
/api/po/invoice/public/v2/invoice - Create invoice
/api/po/invoice/public/v1/invoice/sign/withAsanImza - Sign invoice
/api/po/invoice/public/v1/generateSerialNumber/defaultInvoice - Generate serial
/api/po/invoice/public/v1/common/removeDrafts - Remove draft invoices
/api/po/vatacc/public/v1/operation/find.outbox - VAT operations
/api/po/profile/public/v1/taxpayer/{tin}/object/find - Customer objects
/api/po/dictionary/public/v1/productGroups/find - Product groups dictionary

Fixtures

E-Taxes Item Group - Pre-populated product group codes

Install Hooks

after_install and after_migrate both call setup_token_renewal() to ensure scheduler is configured

Security Considerations

All API endpoints are whitelisted with @frappe.whitelist()
Token storage uses Frappe's password field type
Activity tracking prevents unnecessary API calls
Retry logic includes exponential backoff to prevent API flooding
Automatic token refresh on 401 errors

Important Files

invoice_az/hooks.py - App configuration, event hooks, scheduler setup, and fixtures (~310 lines)
invoice_az/auth.py - Authentication and token management (~970 lines)
invoice_az/api.py - Purchase invoice API and reference data management (~4460 lines)
invoice_az/sales_api.py - Sales invoice import API (~890 lines)
invoice_az/send_sales_api.py - Sales invoice send API (~1060 lines)
invoice_az/vat_api.py - VAT operations import API (~620 lines)
invoice_az/vat_operations.py - VAT operations hooks (~40 lines)
invoice_az/client/*.js - Client-side functionality for UI enhancements
invoice_az/invoice_az/doctype/*/ - Custom doctype definitions
invoice_az/fixtures/e_taxes_item_group.json - Item group fixtures

Workflow: Importing Purchases

User opens Purchase Order/Invoice list
Clicks "Import from E-Taxes" button
System fetches invoices from e-taxes inbox
User selects invoice to import
System validates mappings (items, suppliers, units)
Creates Purchase Order with items
Creates E-Taxes Purchase tracking record

Workflow: Importing Sales (from E-Taxes)

User opens Sales Order list
Clicks "Import from E-Taxes" button
System fetches invoices from e-taxes outbox
User selects invoice to import
System validates mappings (items, customers, units)
Creates Sales Order and Sales Invoice
Creates E-Taxes Sales tracking record

Workflow: Sending Sales Invoice to E-Taxes

User creates and submits Sales Invoice
Fills customer_object_name field (required for delivery address)
Clicks "Send to E-Taxes" button
System generates serial number
Creates draft invoice on e-taxes
User clicks "Sign Document" button
System signs with ASAN Imza
E-Taxes Sales Outbox record updated

Retry/Cancel:

If signing fails, user can click "Retry Signing" button
User can cancel draft invoice using "Cancel on E-Taxes" button

Workflow: Sending Purchase Acts to E-Taxes

Supported act types for Purchase Invoices with Individual suppliers:

Agricultural Products Act - For agricultural goods (kind: "agriculturalProductsAct")
- Tax types: "Tax Free" (0%) or "Taxable" (5%)
- Endpoint: /api/po/invoice/public/v1/act/generateSerialNumber/agriculturalProductsAct
Metal Scrap Reception Act - For ferrous/non-ferrous metal scrap (kind: "metalProductsAct")
- Tax types: Only "Taxable" (5%) - tax-free items not allowed
- Validation: All items must have tax_type = "Taxable"
- Endpoint: /api/po/invoice/public/v1/act/generateSerialNumber/metalProductsAct
Tire Products For Disposal Act - For tire disposal operations (kind: "tireProductsForDisposalAct")
- Tax types: Only "Taxable" (5%) - tax-free items not allowed
- Validation: All items must have tax_type = "Taxable"
- Endpoint: /api/po/invoice/public/v1/act/generateSerialNumber/tireProductsForDisposalAct
Plastic Products For Disposal Act - For plastic disposal operations (kind: "plasticProductsForDisposalAct")
- Tax types: Only "Taxable" (5%) - tax-free items not allowed
- Validation: All items must have tax_type = "Taxable"
- Endpoint: /api/po/invoice/public/v1/act/generateSerialNumber/plasticProductsForDisposalAct
Rawhide Supply - For rawhide supply operations (kind: "rawhideSupply")
- Tax types: Only "Taxable" (5%) - tax-free items not allowed
- Validation: All items must have tax_type = "Taxable"
- Endpoint: /api/po/invoice/public/v1/act/generateSerialNumber/rawhideSupply
Other Product Receipt Act - For other product receipts (kind: "otherProductReceiptAct")
- Tax types: "Tax Free" (0%) or "Taxable" (5%) - both allowed
- Validation: No tax type restrictions (same as Agricultural)
- Endpoint: /api/po/invoice/public/v1/act/generateSerialNumber/otherProductReceiptAct

Workflow:

User creates and submits Purchase Invoice
Selects Individual supplier (with FIN, passport, DOB)
Checks purchase_type checkbox to enable act fields
Selects act_kind (one of 6 available types)
Adds items with product_group_code and tax_type
Clicks "Send Act to E-Taxes" button (button text changes based on act_kind)
System generates serial number from appropriate endpoint
Creates draft act on e-taxes
User clicks "Sign Act with ASAN Imza" button
System signs with ASAN Imza
E-Taxes Purchase Outbox record updated