Deployment Change Log af4663d4c

This commit alters the cache retrieval logic in `get_cached_report_data` for temporary debugging.

The previous implementation iterated over a Redis hash to reconstruct the `final_stock_position` dictionary. This has been replaced with a single `pd_cache.get()` call to fetch the entire dictionary at once, simplifying the process for debugging purposes. The original hash-based retrieval code has been commented out but retained for reference.

Prefix log and exception messages with function names for better traceability.
Changed a log level from INFO to ERROR in `earliest_ship_date_for_sku_warehouse` for critical stock issues.

This commit introduces several enhancements to the promise date API endpoints to improve stability and debugging, particularly during cache rebuilds.

- **Handle CacheIsRefreshingException:** The API views (`pd1_ship_date_check_v1`, `pd1_ship_date_check_v2`, `get_promise_date_and_order_api_pd2`, `get_promise_date_and_order_customer_ref`) now catch the `CacheIsRefreshingException`. Instead of failing, they return a 503 Service Unavailable status with a `Retry-After` header, signaling to clients that the service is temporarily unavailable while data is being refreshed.

- **Enhance Logging:** Added more detailed exception logging and logging of the request body to `pd1_ship_date_check_v2` to facilitate easier debugging of client-side issues.

- **Refactor Stock Position Retrieval:** Temporarily adjusted the `when_can_it_ship_v2_promise_date_logic` function to fetch the entire `final_stock_position` object from the `database_cache`. This is a debugging measure to investigate potential issues with fetching partial data from the Redis hash.

- **Minor Fixes:**
    - Corrected the cache key in `pd1_ship_date_check_v2` for better specificity.
    - Replaced bare `except:` clauses with `except Exception:` for safer error handling.

This commit introduces a new Django management command `copy_pd_cache` within the `cached_dear` app.

WHAT CHANGED:
- A new management command `cached_dear/management/commands/copy_pd_cache.py` has been added. This command allows copying specific promise date related cache keys from the `database_cache` to the `default` cache (typically Redis/Valkey) for a given DEAR entity.
- Modifications were made to `cached_dear/promise_dates/dear_order_promise_date.py` to ensure that all promise date related data that is cached in the `default` cache is also mirrored in the `database_cache`. This includes `pd-soh_data`, `order-sources`, `pd-stock-data`, `pd-max-possible`, `pd-orderline_summary`, `pd-orderline_pick_summary`, `pd-unshippable-skus`, `pd-stock-data-available`, `pd-max-possible_allocated`, `promise_dates`, `allocation_promise_dates`, and `pd-final-stock-position-hash`.

WHY THE CHANGE IS NEEDED:
- The primary purpose of these changes is to facilitate the seeding of local development environments with promise date cache data. In a production environment, the `default` cache (Redis/Valkey) holds this critical data. However, in local development, it can be cumbersome to populate this cache. By mirroring the data in the `database_cache`, developers can easily copy this data to their local Redis/Valkey instance using the new management command, ensuring a consistent and functional development setup without needing to run full data synchronization processes.
- This addresses the need for a quick and reliable way to get promise date data into a local cache, which is essential for testing and development of features relying on this cached information.

TRADE-OFFS AND ALTERNATIVES:
- An alternative considered was to directly populate the Redis/Valkey cache from Dear API calls in development. This was rejected due to the time-consuming nature of API calls and the potential for rate limiting, making it inefficient for frequent local environment setups.
- Another alternative was to use fixtures or other database seeding mechanisms, but these would not directly populate the Redis/Valkey cache, requiring additional steps. The current approach directly addresses the cache seeding requirement.

RISK/IMPACT:
- The changes are low risk. The new management command is an opt-in utility for development environments.
- Mirroring cache data to the `database_cache` introduces a slight overhead during cache writes, but this is generally acceptable given the benefits for development and the relatively infrequent updates to some of these cache keys. The `database_cache` is not intended for high-performance reads in this context, but rather as a persistent store for seeding.
- No backward compatibility concerns are anticipated as existing cache read operations remain unchanged.

VALIDATION GUIDANCE:
- To validate, run the new management command in a development environment with an existing `database_cache` populated with promise date data.
  ```bash
  python manage.py copy_pd_cache <your_dear_entity_name>
  ```
- Verify that the `default` cache (e.g., by inspecting Redis/Valkey) now contains the copied keys and their corresponding data.
- Confirm that promise date calculations in the application function correctly after the cache copy.

Adds an info-level log message to the `earliest_ship_date_for_sku_warehouse` function.

This log is triggered when the function determines there is not enough stock for a given SKU/warehouse, which is an expected condition in greedy mode but an error otherwise. This provides better visibility for debugging order fulfillment scenarios.

Also includes minor import cleanup for the datetime module.

This commit refactors the intercompany journal processing by separating reconciliation and integrity checks from the main posting logic.

- Moves reconciliation functions (`reconcile_intercompany_invoices`, `reconcile_posted_journals_with_xero_report`, `find_and_void_invalid_xero_journals`) from `tilecloud_shopify_interentity.py` into a new, dedicated file `tilecloud_integrity_tools.py`.
- Extracts common helper functions (`_is_day_rate_limit`, `_format_xero_api_error`, `prepare_xero_journal_for_posting`) into a new `tilecloud_utils.py` file.
- Relocates the `PseudoManualJournal` helper to `core/xero_connector/xero_journal_helpers.py` as it is a generic Xero helper.

This improves separation of concerns and maintainability with no change in functionality.

- Adds a check to ensure that each line processed in the Xero report reconciliation starts with a valid date.
- Malformed lines that do not begin with a date are now skipped, and an informational message is logged.
- This prevents errors when processing journal entries with incorrect formatting.

Set `dry_run` fields in `FixIncorrectRefundJournalsForm`, `ReconcileJournalsForm`, and `XeroIntegrityRepairForm` to `initial=False`.
Enhance Xero report parsing in `tilecloud_shopify_interentity.py` to support tab-delimited data, skip "Total" lines, and adjust account code extraction.
Improve reconciliation logging and make amount comparisons slightly more tolerant using `abs_tol=0.01` in `math.isclose`.

- Fixes `F401` (unused import) by removing `collections.Counter`.
- Fixes `F821` (undefined name) by replacing `xero_transaction_ids` with `xero_journal_details.keys()` in the reconciliation logic.
- Fixes `F811` (redefinition of unused name) by removing the redundant import of `namedtuple`.
- Ensures the enhanced reconciliation logic in `reconcile_posted_journals_with_xero_report` adheres to project coding standards.

- Enhances the `reconcile_posted_journals_with_xero_report` function to perform a deep comparison of journal line data between the Xero CSV report and the local database.
- The function now parses the full financial details from the CSV, including account codes and amounts for each journal line.
- It compares this data against the `parent_journal_lines` or `child_journal_lines` in the `SaleIntercompanyJournal` records.
- If a content mismatch is found (in addition to journals that are missing entirely), the corresponding journal is reset to a `PENDING` state.
- This creates a self-healing mechanism that can now catch and trigger the re-processing of journals that failed to update correctly in Xero.

WHAT changed:
- Added `READY_TO_PROCESS` status to `three_pl.models.ProcessStatus`.
- Updated `three_pl.tables.ThreePL_order_fulfilmentsTable` to include a click-to-copy feature for process status details.
- Migrated `django_tables2` template from Bootstrap 4 to Bootstrap 5 in `three_pl_dermapen/templates/three_pl_dermapen/ThreePLFulfilmentsFilter.html`.
- Introduced `copyStatusToClipboard` JavaScript function for improved user experience.

WHY the change is needed:
- The new `READY_TO_PROCESS` status provides a clearer state for orders requiring manual intervention or being prepared for processing.
- The click-to-copy functionality for process status details (e.g., error messages) significantly improves usability, allowing users to easily extract and use these messages for debugging or communication.
- Upgrading to Bootstrap 5 for `django_tables2` ensures consistency with the project's overall Bootstrap 5 adoption and leverages newer styling and features.

Trade-offs and alternatives:
- No significant trade-offs. The change to Bootstrap 5 is a necessary upgrade. The copy-to-clipboard feature is a direct UX improvement.

Risk/impact:
- Low risk. The `READY_TO_PROCESS` status is additive. The JavaScript change is isolated to the `three_pl_dermapen` template and `three_pl` table rendering. The Bootstrap 5 template change should align with existing Bootstrap 5 styling.

Validation guidance:
- Verify that the "Process Status" column in the `ThreePLFulfilmentsFilter` table displays the new `READY_TO_PROCESS` status correctly.
- Confirm that hovering over status cells with details (e.g., `REJECTED`, `BAD_ADDRESS`, or any status with `latest_result`) shows the tooltip.
- Click on a status cell with a tooltip; verify that the tooltip content is copied to the clipboard and a temporary "Copied!" message appears.
- Ensure the table styling remains consistent and correct after the Bootstrap 5 template update.
- Check console for any JavaScript errors.

Safe rollback strategy:
- Revert the changes to the affected files.

- Introduce `READY_TO_PROCESS` status for 3PL orders.
- Improve error logging and status updates for failed 3PL order sends.
- Ensure `latest_result` is consistently used for tooltips in the UI.

This change enhances the visibility and handling of errors during the 3PL order
sending process. The new `READY_TO_PROCESS` status provides a clearer state
for orders awaiting processing. Error handling in `task_send_queued_orders`
now ensures that the `latest_result` field is populated with specific error
details, which are then displayed in the UI via the `render_process_status`
method. This provides better diagnostic information for rejected orders.

Key changes:
- When an address validation error occurs during fulfilment processing for Dermapen or Hexspoor, the status is now set to `BAD_ADDRESS` instead of the generic `REJECTED`.
- The UI table for fulfilments now displays a tooltip with error details for the `BAD_ADDRESS` status, mirroring the existing behavior for `REJECTED` items.
- The logic in `three_pl_logic_hexspoor` has been refactored to catch validation exceptions and update the fulfilment row's status directly.
- A minor label change from "Authpack" to "AuthPickPack" was made in the UI for clarity.

---
**Commit Message:**

feat(three_pl, three_pl_dermapen): Add BAD_ADDRESS status for fulfilment errors

Introduces a new `BAD_ADDRESS` process status to specifically handle address validation failures, separating them from generic rejections.

-   Updates business logic in `three_pl_dermapen` to catch address validation exceptions and set the fulfilment row status to `BAD_ADDRESS`, saving the error details.
-   Refactors `three_pl_logic_hexspoor` to catch Pydantic and custom validation errors, find the corresponding fulfilment row, and update its status to `BAD_ADDRESS`.
-   Modifies the `three_pl` table view to display a tooltip with the specific error message for rows with the `BAD_ADDRESS` status.
-   Updates button text in the status reset template from "Authpack" to "AuthPickPack" for improved clarity.

This commit introduces an optional end date filter to the "Prepare Refund Journal Corrections" functionality. Users can now specify both a start and end date to narrow down the journals targeted for correction.

- Adds an `end_date` field to `FixIncorrectRefundJournalsForm`.
- Updates `FixIncorrectRefundJournalsView` to process the new field and pass it to the background task.
- Modifies `task_prepare_refund_journal_corrections` to filter journals by `transaction_date__lte`.
- Adjusts the journal query to filter on `parent_correction_status` and `child_correction_status` separately for more accurate selection.
- Reorganizes the Intercompany menu in `index.html` for better clarity, grouping refund-related actions together.

Refactor the logic for updating manual journal lines to explicitly rename the 'amount' key to 'line_amount' before creating ManualJournalLine objects. This change improves clarity and ensures proper data mapping for the Xero API when correcting refund journals.

Refactor autofill logic to use DearCachedAPI for customer retrieval.
This change removes direct DearCache model access in autofill_logic.py
and leverages the get_customer_by_id method for better abstraction and maintainability.

Additionally, the 'Autopick B2C orders' periodic task in tasks.py has been
commented out and marked as redundant, as it will be replaced by a new
auto-fulfillment process (task_process_dermapen_autofill_queue).

# Conflicts:
#	deployment_support/includes/version.yml

- Fixes a `TypeError` in `task_fix_incorrect_refund_journals` that occurred when creating `ManualJournalLine` objects.
- The error was caused by a mismatch between the field name `amount` in the stored `SimpleJournalLine` data and the expected keyword argument `line_amount` in the `ManualJournalLine` constructor.
- The fix involves renaming the `amount` key to `line_amount` in the dictionary at the point of object creation, ensuring compatibility with the Xero SDK without breaking backward compatibility of stored data.
- The incorrect changes to `core/xero_connector/xero_journal_helpers.py` have been reverted.

- This commit includes the Django migration file for the changes made to the `SaleIntercompanyJournal` model.
- The migration adds the `parent_correction_status` and `child_correction_status` fields and marks the old `correction_status` field as deprecated.
- This database schema change is essential for the two-stage atomic correction process.

This commit refactors the refund correction process to be truly atomic and restartable by tracking the status of parent and child journal updates independently.

- **Model Changes:**
  - Replaced the single `correction_status` field on the `SaleIntercompanyJournal` model with two new fields: `parent_correction_status` and `child_correction_status`.
  - This allows for granular tracking of the correction state for each Xero instance.

- **Task Refactoring:**
  - The `task_prepare_refund_journal_corrections` task now sets both new status fields to `PENDING_CORRECTION`.
  - The `task_fix_incorrect_refund_journals` (the execution task) now queries for records where either status is pending.
  - The execution task processes parent and child updates independently, setting their respective statuses to `COMPLETE` or `ERROR`.
  - This ensures that if the task is interrupted, it can safely resume and only re-process the failed or pending part of the job.

This commit addresses the `InconsistentMigrationHistory` error encountered
on the `dear_analytics` database for the `three_pl` app.

The issue was resolved by:
- Iteratively adding missing migration records (0011 through 0015) for the
  `three_pl` app to the `dear_analytics` database using a new custom tool.
- Creating a new Django management command `fix_migration_history.py` in
  the `core` app. This tool allows for manual intervention in the
  `django_migrations` table to resolve similar inconsistencies in the future.
  It supports adding or removing migration records for a specified app and
  database.

Additionally, this commit includes:
- New migrations generated by `makemigrations` for `cached_dear` and `three_pl`
  reflecting recent model changes.
- Pre-existing modifications to `cached_dear/views_tilecloud.py` which were
  already staged.

- Removed the `!/tmp/` entry from `.gitignore` which was incorrectly including the `tmp/` directory.
- Added `tmp/` to `.gitignore` to ensure that all contents of the `tmp/` directory, including temporary commit message files, are properly ignored by Git.

This commit adds a critical safety check to the `task_prepare_refund_journal_corrections` task to prevent it from incorrectly processing journals that were already correct.

The initial identification logic was too broad and would have incorrectly matched correct standard payment journals as incorrect refund journals.

This has been fixed by adding a secondary validation step inside the task's loop. For each candidate journal, the task now inspects the original DEAR transaction data. It only proceeds with the correction if it can definitively confirm that the journal is for a refund by checking if the `Debit` account on the original transaction was the Accounts Receivable account.

This ensures that the corrective process is idempotent and will safely ignore any journals that were not affected by the original bug.

Adds a new helper function `stringify_carrier_service_dermapen` to extract carrier service information from Dear Sale or Fulfilment objects. This function is intended to be used for fulfilment rows, providing carrier details based on the available Dear data.

- Added a pytest run configuration for `test_make_journals` in `.run` directory.
- Updated the `order_number` in `test_make_journals` test to ensure accurate test scenarios.
- Enhanced `intercompany_payment_journal_lines` logic with detailed comments for better clarity and robustness.

This commit adds comprehensive documentation for the new two-stage corrective task pipeline.

- **Module Documentation:**
  - Added a detailed module-level docstring to `tasks_intercompany_corrections.py` explaining the purpose of the module and the two-stage (prepare/execute) architecture.
  - Enhanced the docstrings for `task_prepare_refund_journal_corrections` and `task_fix_incorrect_refund_journals` to clarify their specific roles, parameters, and state management.

- **Model Field Documentation:**
  - Updated the `help_text` for the new fields in the `SaleIntercompanyJournal` model (`parent_corrected_journal_lines`, `child_corrected_journal_lines`, `correction_status`) to clearly explain their function within the correction process.

This commit refactors the process for correcting incorrect refund journals into a more robust and restartable two-stage pipeline to ensure atomicity.

- **Model Changes:**
  - Added `parent_corrected_journal_lines`, `child_corrected_journal_lines`, and `correction_status` fields to the `SaleIntercompanyJournal` model to manage the state of the correction process.

- **Task Refactoring:**
  - Created a new `task_prepare_refund_journal_corrections` task to identify incorrect journals and prepare the corrected data without interacting with the Xero API.
  - Refactored `task_fix_incorrect_refund_journals` to be an "execution" task that only processes journals in the `PENDING_CORRECTION` state.

- **UI Updates:**
  - The "Fix Incorrect Refund Journals" view now triggers the preparation task.
  - A new "Execute Refund Journal Corrections" view and menu item have been added to trigger the execution task.
  - The forms and templates have been updated to reflect the new two-stage process.

This commit introduces a new UI to trigger the `task_fix_incorrect_refund_journals` Celery task, allowing users to correct historical refund journals.

- **Task Modification:** Updated `task_fix_incorrect_refund_journals` to accept an optional `order_number` for targeted testing.
- **Form Creation:** Added `FixIncorrectRefundJournalsForm` to `cached_dear/forms.py` for user input.
- **View Implementation:** Created `FixIncorrectRefundJournalsView` in `cached_dear/views_tilecloud.py` to handle form submission and task launching.
- **Template Creation:** Added `fix_incorrect_refund_journals.html` for the new view.
- **URL Configuration:** Added a new URL pattern to `cached_dear/url_includes/urls_tilecloud.py`.
- **Menu Integration:** Included a link to the new page in the "TileCloud" dropdown menu in `cached_dear/templates/cached_dear/index.html`.

- Adds a new Celery task `task_fix_incorrect_refund_journals` to identify and correct historical refund journals with incorrect accounting logic.
- The task is placed in a new, dedicated module `cached_dear/tasks_intercompany_corrections.py` to isolate it from routine tasks and follow project conventions for Celery discovery.
- The task allows for in-place updates of Xero manual journals, avoiding the need to void and repost.

This commit addresses several issues related to intercompany journal generation and code quality:

- **Fix(voiding): Make obsolete journal voiding comprehensive**
  - Modified `_void_obsolete_journal_entries` to remove date filtering, ensuring all obsolete journal entries for a sale are voided regardless of their transaction date.
  - Updated `process_dear_sale_transaction` to pass all transaction IDs from the DEAR sale payload to the voiding function.

- **Fix(forms): Correct `get_store_config_keys` import path**
  - Resolved `ImportError` in `cached_dear/forms.py` by changing the import source for `get_store_config_keys` from `tilecloud_shopify_interentity` to `tilecloud_shopify_configuration`.

- **Fix(forms): Resolve linting issues in `cached_dear/forms.py`**
  - Replaced `from typing import *` with specific imports (`List`, `Tuple`).
  - Removed unused local variables (`dear_instances`, `instance_set`).
  - Corrected misplaced and duplicate `settings` import.
  - Moved `UploadedDocument` import to the top of the file.
  - Explicitly imported `DearObjectType`.

- **Fix(interentity): Correct payment/refund journal signs in `_prepare_payment_lines`**
  - Implemented a targeted fix in `_prepare_payment_lines` to correctly handle parent journal entries for refunds.
  - Based on the 'reversal' paradigm, for refunds, the parent journal now correctly debits the payment account and credits the intercompany account.
  - Updated the docstring for `_prepare_payment_lines` to accurately reflect its purpose and the new accounting logic for payments and refunds.

Refined imports in `cached_dear/forms.py` for better clarity and consistency. Explicitly imported `List` and `Tuple` from `typing` instead of using a wildcard import. Introduced a logger for better debugging and traceability. Updated the import path for `get_store_config_keys` to reflect a module rename or move from `tilecloud_shopify_interentity` to `tilecloud_shopify_configuration`. Added imports for `DearObjectType` and `UploadedDocument` models. Removed redundant imports and commented-out code related to `SelectDearInstances` form initialization.

Remove date-based filtering from `_void_obsolete_journal_entries` to ensure all journal entries for a given sale are considered for voiding, regardless of transaction date.

This change makes the voiding process more robust by:
- Ensuring comprehensive voiding of obsolete journal entries.
- Aligning `current_transaction_ids` with the full Dear sale payload.
- Simplifying the function signature by removing redundant date parameters.

- Add `min_length=1` constraints to `postal_code`, `city`, `email`, and `phone` fields in the `Address` Pydantic model to prevent empty strings.
- Improve robustness of `unidecode_str` validator by ensuring operations are only applied to string values.
- Strip leading/trailing whitespace from `house_number_addition` and `house_number` fields.
- Minor stylistic adjustment for retrieving `first_name` and `last_name`.

# Conflicts:
#	deployment_support/includes/version.yml
#	three_pl_dermapen/business_logic/three_pl_logic_dermapen.py

The core changes are:
- **Function Renaming for Clarity:**
    - `compute_autofill_status_additional_attribute` is renamed to `compute_process_status_additional_attribute`.
    - `update_dear_order_status` is renamed to `update_dear_order_process_status`.
    These changes make the function names more descriptive of their purpose.

- **Improved Robustness:**
    - In `three_pl_dermapen`, the logic for creating CB XML orders now handles cases where a shipping carrier might be missing by accepting a default carrier.
    - Hardcoded status strings like `'QUEUED'` and `'REJECTED'` have been replaced with `ProcessStatus` enum members, improving type safety and maintainability.

- **Code Cleanup:**
    - Unused functions and redundant code have been removed from `three_pl_dermapen/business_logic/three_pl_logic_dermapen.py`.

- **New Developer Tool:**
    - A new script, `gemini_commit.sh`, has been added to automate the process of staging changes and generating commit messages.

**Commit Message:**

refactor(cached_dear, three_pl, three_pl_dermapen, three_pl_skyzer): Improve 3PL status function naming and carrier logic

This commit refactors key parts of the 3PL integration to enhance clarity and robustness.

- Renamed `compute_autofill_status_additional_attribute` to `compute_process_status_additional_attribute` and `update_dear_order_status` to `update_dear_order_process_status` across all relevant modules for better descriptiveness.
- Replaced hardcoded `process_status` string literals with `ProcessStatus` enum values to improve type safety and code maintainability.
- Enhanced the CB XML generation in `three_pl_dermapen` to accept a `default_carrier`, preventing errors when a carrier is not specified on the order.
- Removed unused functions and cleaned up carrier retrieval logic in `three_pl_logic_dermapen`.
- Added the `gemini_commit.sh` script to automate staging files and generating commit messages, streamlining the development workflow.

**WHAT changed:**
The changes are in `three_pl_dermapen/business_logic/cb_xml.py`.
- Modified how `ship_lines` are accessed from `dear_fulfilment` to prevent `KeyError` if the 'Ship' key is missing.
- Updated an error message related to missing ship lines to be more specific to the backend's requirements.
- Refined the logic for determining the source of the shipping address (`ship_to_source`) by checking for the presence of `Line1` in the shipping address from the fulfilment, falling back to the order header's shipping address if `Line1` is not present.

**WHY the change is needed:**
The previous code could raise a `KeyError` if `dear_fulfilment` did not contain a 'Ship' key, leading to unhandled exceptions. The updated logic safely accesses nested dictionary keys. The error message was clarified to provide better guidance for users. The shipping address source logic was improved to ensure a valid shipping address is used, prioritizing the fulfilment's address if complete, otherwise using the order header's address. This enhances robustness and error handling.

**Trade-offs and alternatives:**
No significant trade-offs were identified. The alternative would be to continue with the less robust key access, which could lead to runtime errors.

**Risk/impact:**
Low risk. These changes improve error handling and data access robustness without altering core business logic.

**Validation guidance:**
- Test scenarios where `dear_fulfilment` might not contain a 'Ship' key.
- Test scenarios where `dear_fulfilment['Ship']` might not contain 'Lines'.
- Test scenarios where `dear_fulfilment['Ship']['ShippingAddress']` might be present but lack a `Line1`.
- Verify that the correct shipping address is used and `ship_to_source` is set appropriately in all cases.

Refactors the Skyzer box packing and SSCC processing logic to run in a background Celery task. This prevents the main order update process from timing out while waiting for box packing information from the 3PL and document generation.

Key changes:
- The `post_pack_processing` method in `SkyzerManager` now launches the `task_process_skyzer_box_packing` task instead of performing the work synchronously.
- A `launch_task=False` parameter is added to `post_pack_processing` to allow the task itself to call the method without creating an infinite loop.
- The order status update process can now be filtered by a sales order number, allowing operators to re-check a specific order. This is implemented in the UI, view, task, and manager.
- When an order status is updated to shipped, the `shipped_date_3pl` is now recorded in the `ThreePL_order_fulfilments` model.
- The `convert_dear_pick_to_pick_lines_simplified` function was clarified to handle both Pick and Pack structures from Dear by renaming a parameter.

This commit adds the `tmp/` directory to the `.gitignore` file. This directory is used for temporary files that should not be committed to the repository.

This commit refactors the `DermapenAutoFillOrder` status management to improve clarity and robustness.

The `DermapenAutoFillOrderStatus` enum has been updated to include more granular statuses (`AUTOPICK_PROCESSED`, `QUEUED_3PL`), ensuring a clearer representation of the order's progress through the auto-fill pipeline.

The `task_process_dermapen_autofill_queue` task has been enhanced to handle orders in both `UNPROCESSED` and `AUTOPICK_PROCESSED` states. This allows the system to resume processing for orders that have been auto-picked but not yet queued for the 3PL.

The `ProcessAutofillQueueView` has been simplified to delegate location-based filtering to the task, making the view cleaner and the task more self-contained.

Additionally, this commit addresses several linting errors, including unused variables and imports, improving overall code quality.

Refactored `on_order_auth_equipmed` to improve readability and maintainability by extracting the main logic into a new function `_process_equipmed_order_logic`.

This new function is now used by both `on_order_auth_equipmed` and `reprocess_equipmed_order_task`.

The following fixes were also included:
- Corrected indentation errors causing syntax issues.
- Fixed a typo from `current_.dear_sale` to `current_dear_sale`.
- Removed an unused import of the `os` module.
- Updated the function call from `compute_autofill_status_additional_attribute` to the correct `compute_process_status_additional_attribute`.
- Simplified the `USE_V2_AUTOPROCESS` feature flag.

This commit refactors the `three_pl_dermapen` app to improve modularity and centralize 3PL dispatch logic.

Key changes:
-   **Centralized Dispatch Logic:** A new dispatch mapping in `definitions.py` centralizes the configuration for different 3PL backends (Hoxton, Hexspoor, Prism), making it easier to add new warehouses and manage existing ones.
-   **Prism XML Tasks:** Tasks related to Prism XML processing (`find_and_process_lrt_files_prism`, `stock_comparison_cache_prism`, etc.) have been moved from `tasks.py` to a new `tasks_prism_xml.py` file for better separation of concerns.
-   **Multi-Entity Fix:** Corrected a logic flaw in `task_process_dermapen_autofill_queue` where a location was incorrectly assumed to belong to a single Dear entity. The code now groups orders by a `(location, dear_entity)` tuple, ensuring correct entity context.
-   **Process Status Enum:** Replaced hardcoded status strings (e.g., QUEUED, PENDING_3PL) with the `ProcessStatus` enum for better consistency and maintainability.

Refactors the `bulk_update_additional_attribute` function to allow more targeted updates.

The function now uses the `write_blank_to_unseen_rows_if_existing_value_is_in` parameter to filter for orders that have specific, existing values in their additional attributes.

Previously, it would select all orders with any non-empty value. This change makes the process more precise, ensuring that only orders with designated values are considered for updates, and prevents unintentionally blanking out attributes when no specific values are targeted.

The _decompose_bom_for_one function was using the incorrect key 'ProductSKU' to access the component identifier from the BillOfMaterialsProducts list provided by the Dear API. The correct key is 'ProductCode'.

This change updates the logic to use the correct 'ProductCode' key, ensuring that component products are correctly looked up during BOM decomposition.

Moved the `default_promise_date_location_list` setting from `tasks_promise_dates.py` to a new, dedicated module at `cached_dear/promise_dates/promise_date_settings.py`.

This refactoring centralizes configuration, improves modularity, and avoids potential circular import issues between tasks, views, and promise date logic. All dependent files have been updated to import from the new settings module.

The location list was also expanded to include settings for `INTERIOR_ICONS` and `STATUS_ANXIETY`.

This commit introduces several improvements to the promise date calculation logic.

- **Refactors BOM Decomposition:** The `decompose_bom` function and its caching mechanism have been completely rewritten. The previous `LRUCache`-based implementation was replaced with a simpler in-memory dictionary cache. This cache is now explicitly cleared at the beginning of each promise date refresh to prevent stale data. The decomposition logic now caches the component list for a single unit of a parent product and then scales the quantities, which is more efficient and robust than the previous method.

- **Fixes Promise Date Updates:** The logic that updates promise dates on order headers now correctly excludes orders with a status of "VOIDED" or "COMPLETED", or a fulfillment status of "FULFILLED". This prevents unnecessary processing on orders that are already closed.

- **Performance Tweak:** Replaced `len(queryset)` with `queryset.count()` for more efficient database queries when checking the number of results.

- **Code Style:** Cleaned up import statements for better readability and consistency.

This commit enhances the promise date update process with improved logging, more robust queries, and better operational visibility.

- Adds detailed summary logging to `update_promise_date_order_attribute_and_send_webhook` and `bulk_update_additional_attribute`, reporting counts of updated, cleared, and newly set attributes.
- Introduces a safety check that logs a warning and a sample of affected orders if more than 1000 promise dates are about to be cleared.
- Refactors the query for finding existing promise dates to correctly handle both null and empty string values in the JSON field.
- Passes the `job` object into `get_cached_sale_row_by_order_nbr_or_id` to provide better logging context during order processing.
- Cleans up imports and removes a debug `print` statement.

- Move TileCloud/Shopify intercompany configuration from `tilecloud_shopify_interentity.py` to a new, dedicated module `tilecloud_shopify_configuration.py`. This refactoring separates configuration from business logic, improving modularity and maintainability.
- Add a new store configuration for "Dawn-Lighting-Pty".
- Update a log message in `tasks_interior_icons.py` for better clarity during order processing.

- Add pretty-printed JSON logging of the Hexspoor order data when processing a single order.
- Improve order number filtering to be case-insensitive and handle suffixes.

This change introduces a form to the `hexspoor_order_updates_and_processing` view, allowing a user to optionally specify a single Dear sale order number for processing. This enhances usability by enabling targeted updates for a single order, while retaining the ability to process all orders if no number is provided.

- Added `HexspoorOrderUpdatesForm` to `three_pl_dermapen/forms.py`.
- Modified `hexspoor_order_updates_and_processing` view in `three_pl_dermapen/views_dermapen.py` to use the new form.
- Created a new template for the form.
- Updated the `find_and_process_hexspoor_status_updates` task to accept and filter by `order_number`.
- Fixed various ruff linting issues in the modified files.

This commit refactors the DearStarshipitManager class to improve code quality, readability, and robustness without altering functionality.

Key changes include:
- Removed unused variables such as 'timestamp', 'ss_carrier', 'b2c', and assignments to 'auth_pack' and 'updated_packaging_order'.
- Renamed generic loop variables (e.g., 'l') to more descriptive names ('line', 'pick_line') in multiple methods to enhance clarity.
- Converted a lambda for fetching order details into a standard named function ('get_full_order_details') for better debugging and readability.
- Improved the robustness of location matching by making the comparison case-insensitive and stripping whitespace."

Adds entity, order number, and fulfilment number to the log message generated when a fulfilment's current DEAR state is not valid for the state being enforced by the 3PL manager.

This provides more detailed context, making it easier to identify the specific fulfilment that requires action, which is especially useful in multi-entity deployments.

The error logging for ThreePLAPIError in the Skyzer manager now includes the `path` and `message` from the API response, providing more context for debugging.

- Fix stale promise dates: Remove `exclude` filters for SHIPPED/COMPLETED orders in `update_promise_date_order_attribute_and_send_webhook` to ensure stale promise dates are correctly cleared.
- Add extensive logging: Add detailed logging to `promise_date_and_pick_updates_for_order_header` and `task_mark_header_ready_to_pick` to improve traceability of promise date and "Ready to Pick" status updates.
- Fix UI view: Resolve `NoReverseMatch` error in `view_revise_sale_header_attributes` by creating a job on GET requests.
- Improve UI text: Update job title in `view_revise_sale_header_attributes` for better user experience.

This commit significantly enhances the observability and robustness of the "Ready to Pick" status update process. It also refactors related code and fixes a UI bug.

- **Enhanced Logging:**
  - In `promise_date_order_header_status.py`, added extensive logging to trace the entire lifecycle of the status update, including cache refreshes, lock acquisition, promise date calculations, and detailed per-order SOH (Stock on Hand) evaluation.
  - In `tasks_interior_icons.py`, added verbose logging to the `task_mark_header_ready_to_pick` task to track progress, Shopify address checks, and 3PL performance data logging for each order.

- **Bug Fix:**
  - In `views_interior_icons.py`, fixed the `view_revise_sale_header_attributes` view to correctly render the `generic_websocket_job.html` template on GET requests. This ensures the job progress page loads and connects to the WebSocket for live updates as intended.

- **Refactoring & Cleanup:**
  - Removed unused imports (`ZoneInfo`), variables (`webhook_body`, `healthcheck_url`), and dead code blocks from `tasks_interior_icons.py`.
  - Deleted the legacy `ansible/includes/version.yml` file.

- **UI/Template Improvements:**
  - Renamed the "Refresh Sale Attributes" link to "Refresh Promise Date & Ready to Pick Status" for better clarity.
  - Made the `DEAR_INSTANCE` check in the template case-insensitive to prevent errors.

Correctly access the `date` attribute on Xero payment and credit note allocation objects.

The code was previously calling `.date.date()`, which is incorrect if the `.date` attribute is already a `date` object. This change removes the redundant `.date()` call to prevent errors and ensure proper date comparisons against period lock dates when voiding invoices.

This commit refactors the `threepl_common_logic.py` module to improve code quality, readability, and maintainability. No functional changes are introduced.

Key changes include:
- Renamed loop variables from `l` to `line` for better clarity in multiple functions.
- Removed several unused variables to reduce code clutter.
- Simplified a nested `try...except` block for more straightforward error handling.
- Updated a type check to use `isinstance()` for better adherence to Python best practices.

The journal reconciliation process was incorrectly matching reversed IC journals, leading to potential accounting discrepancies.

This change updates the regular expression used to identify journal headers, adding a negative lookahead to explicitly ignore journals prefixed with "Reversed: ". This ensures they are not processed during reconciliation.

- In the `three_pl` app, a log message has been added to record shipping line and address details. More importantly, the shipment creation API call is now wrapped in a `try...except` block to include the full payload in case of an error, which will aid in debugging.
- In the `starshipit` app, a minor comment has been added for clarity.

---
**Commit Message:**

**Subject:** `three_pl, starshipit: Enhance logging and error handling for shipment creation`

**Body:**

This commit improves the robustness and debuggability of the Dear shipment creation process within the `three_pl` application.

- In `create_and_post_authorised_ship_from_shiplines`, the Dear API call to create or update a shipment is now wrapped in a `try...except` block. If an `API_error` occurs, it is re-raised with the `ship_data` payload included, providing essential context for debugging failures.
- A new log message has been added in `create_dear_shipment_from_order_fulfilments_row` to capture the exact shipping lines and address being sent to Dear.
- A minor code comment was added to `starshipit_manager.py` for better readability.

Replaces the hardcoded 180-minute pickup delay in the
`get_pickup_and_closing_time` function with a named constant,
`MACHSHIP_DEFAULT_PICKUP_DELAY_MINUTES`, and increases the value
to 240 minutes.

The previous 3-hour buffer was found to be insufficient in some
scenarios, leading to API errors. Increasing the delay to 4 hours
provides a more reliable window for consignment booking. Using a
constant improves code readability and simplifies future adjustments.

This change re-enables the `make_company_and_contact_names` method within the `StatusAnxietyStarshipit` connector.

The method was previously commented out and is now restored to provide channel-specific logic for determining the recipient company and contact names on shipping labels.

Additionally, the type hint for the `dear_sale` parameter has been updated from `Dict` to the more specific `DearSale` TypedDict for improved type safety and clarity.

The task_run_intercompany_journal_sync_pipeline previously defaulted to a hardcoded Xero instance pair for "tilecloud".

This change updates the default behavior to be dynamic. If no instance pairs are specified, the task now generates pairs for all child Xero instances configured in the `child_xero_to_shopify` map. This allows new child entities to be included in the intercompany sync automatically without modifying the task itself.

Modified protect_ecr_images.py to report which Kubernetes namespaces are affected by a DEPLOYMENT-ECR MISMATCH.

The get_deployed_image_tags function now returns a dictionary mapping image tags to a set of namespaces, allowing sync_protection_tags to include the list of affected namespaces in the error log.

cached_dear:
The `recover_inflight_journals` function has been enhanced to correctly handle recovery for intercompany journals that were in an `IN_FLIGHT` state during a void operation.

Previously, the recovery logic only accounted for failed creations. If a crash occurred while voiding a journal, the row would remain in the `IN_FLIGHT` state without a clear resolution path. This change adds logic to detect if the failed operation was a void, ensures the corresponding journal is voided in Xero, and updates the local status to `VOIDED`.

dear_zoho_analytics:
Increased the `max_retries_per_chunk` from 5 to 8 in the Zoho backend `import_data` method. This makes the data upload process more tolerant of transient API errors or network issues, improving the overall reliability of data synchronization with Zoho Analytics.

- Enhances error logging in `refresh_sales_tables` to include the full traceback when processing a sale fails. This provides critical context for debugging.
- Refactors the sales table preparation logic by renaming loop variables for improved readability and removing several unused variables.
- Fixes a typo in an import path within the `zoho_backend` module.

Adds a try/except block around the `prepare_one_sale` call within the `refresh_sales_tables` task.

Previously, an exception raised while processing a single sale record would cause the entire background task to fail. This change makes the process more resilient by catching such exceptions, logging the error along with the specific sale order reference, and allowing the task to continue processing the remaining sales.

This commit introduces several improvements to the Zoho Analytics
integration, focusing on upload performance and user experience.

The core change is an enhancement to the `upload_data_to_zoho` method.
The data upload logic now dynamically increases the chunk size after a
series of successful uploads. This complements the existing back-off
strategy (which reduces chunk size on errors), allowing the system to
speed up uploads when the connection to Zoho is stable, improving
overall throughput.

User experience improvements include:
- The "Refresh Zoho Analytics" form now defaults to refreshing all sales
  tables, providing a more intuitive starting point for users.
- The warning on the manual integrity check page has been updated to
  clarify that this check is now integrated into the standard data
  refresh process.

Finally, this commit corrects a typo in an import path within the Zoho
backend and removes unused variable assignments.

Introduces a new UI to run an integrity check and repair process for Zoho Analytics. This allows users to select specific Cin7 Core instances and data topics to validate. The check is executed as a background task, with progress displayed on the standard job page.

Key changes:
- Added `ZohoIntegrityCheck` view and `ZohoIntegrityCheckForm`.
- Created a new URL and template for the integrity check page.
- The view launches the `tasks.zoho_analytics_integrity_check` background task with user-selected parameters.

Also includes significant code cleanup:
- Removed numerous unused variables and imports across the app.
- Replaced wildcard imports with specific ones for better clarity.
- Refined templates and exception handling.

Removes deprecated channel-specific logic from the StatusAnxietyStarshipit connector, relying on the parent class's implementation instead.

- The `make_company_and_contact_names` method, which contained logic based on sales channels (B2B, B2C), has been removed as it is no longer needed.
- Type hints have been improved, replacing generic `Dict` with the `DearSale` TypedDict for sale objects and clarifying `datetime.datetime` types.
- Django ORM queries have been reformatted for better readability.
- A duplicated `standard_pack_query_to_send` method was removed.

Refactors the `choose_starshipit_connector` method in the `StatusAnxietyStarshipit` class to simplify business logic and improve maintainability.

The key changes are:
- Deprecated channel-based routing (B2C, B2B, Wholesale) has been removed. The connector selection now defers to the parent `DearStarshipitManager`'s standard location-based logic.
- The customer exclusion check via `AdditionalAttribute7` is preserved as the primary business rule for Status Anxiety.
- The exception message for excluded customers is now more descriptive.

Minor improvements include:
- Renamed a loop variable from `f` to `fulfilment` in `DearStarshipitManager` for better readability.
- Added a docstring to `get_fulfilment_location`.
- Removed unused commented-out code and redundant line continuations.

Removes the hardcoded filter for `IntercompanyJournalTransactionType.INVOICE` from the `reconcile_posted_journals_with_xero_report` function.

This change allows the function to reconcile all types of intercompany journals (e.g., invoices, credit notes) against a Xero report, not just invoices. This makes the reconciliation process more flexible and comprehensive.

This commit introduces several improvements to the Status Anxiety business logic to enhance clarity, remove redundancy, and improve diagnostics.

- Refactors `classify_candidate_order` to simplify sales channel validation. The logic for checking "Shopify AUS" and "Status Anxiety Wholesale" representatives is now consolidated, reducing code duplication.
- Enhances the rejection message for invalid candidate orders to include specific details (SourceChannel, SalesRepresentative, Carrier), making it easier to diagnose why an order was not processed.
- Removes unused variables and ignored return values in `process_order_for_on_order_auth_status_anxiety`, `batch_process_orders_for_click_and_collect_location_change`, and `undo_fulfilment_and_change_location`.
- Replaces `type() == str` checks with the more idiomatic `isinstance(..., str)` for improved type checking.
- Cleans up unused imports and minor code formatting for better maintainability.

- Refactors `parse_dear_date` in `cached_dear` to be more resilient. It now attempts to parse multiple common datetime formats from the Dear API before falling back to the `dateparser` library for unknown formats. This improves robustness against API changes.
- Adds help text to the `start_date` field in `shopify.StartDateForm` to clarify that the filter applies to the `updated_at` field for Loop Returns, preventing user confusion.

- Added new run configurations for `starshipit.tests.starshipit_connector_tests_statusanxiety.test_get_order_live` and `three_pl_dermapen.tests.hexspoor_tests.test_hexspoor_address` to support pytest runs with environment-specific settings.
- Updated `.run/run_seedandsprout.run.xml` to include the `TESTING=1` environment variable and reorder existing environment variables for clarity.
- Updated credentials and test data in `starshipit_connector_tests_statusanxiety` for improved alignment with current configurations.

***

**Commit Message:**

**starshipit: Refactor shipping notes and improve API typing**

Refactor the Starshipit integration to improve type safety and align with API best practices.

- Change `StarshipitConnector.get_order` to return a `FullOrderModel` Pydantic model instead of a raw dictionary, ensuring type-safe access to order data.
- Modify `DearStarshipitManager` to send shipping notes in the `destination.delivery_instructions` field, removing them from the deprecated top-level `note` field in the API payload.
- Update Status Anxiety integration tests to use the `get_starshipit_manager_statusanxiety` factory, removing hardcoded credentials and improving test reliability.

Introduced a comprehensive set of `TypedDict` models for the Starshipit API in a new file, `starshipit/connectors/starshipit_models.py`. These models provide type-safe structures for API entities like orders, packages, and destinations, improving code clarity and reducing integration errors.

Additionally, a `make_shipping_notes` method was added to the `StatusAnxietyStarshipit` connector to concatenate the `Note` and `ShippingNotes` fields from a Dear sale. This ensures that both customer-facing and internal notes are passed to the carrier.

Introduces multi-entity capability to the 3PL fulfilment processing for Dermapen.

- Adds a `LOCATION_TO_ENTITY_MAPPING` in `three_pl_dermapen/business_logic/definitions.py` to define the relationship between warehouse locations and their managing Dear entities.

- Refactors the `refresh_three_pl_fulfilments_table_dermapen` task in `three_pl_dermapen/tasks.py` to use this new mapping.

The task now iterates over each unique Dear entity associated with the provided locations, allowing it to update fulfilment tables for multiple entities within a single run, instead of relying on a hardcoded entity.

Renames fields in the `PromiseDateReportData` dataclass to be more descriptive and consistent:
- `sku_to_order_lines_summary` is now `orderline_summary`
- `sku_to_pick_lines_summary` is now `orderline_pick_summary`

Improves code readability by renaming generic loop variables (`l`) to more descriptive names (`line`, `loc`) within list comprehensions and loops in `process_orders` and `summarise_next_available_data`.

Additionally, removes an unused `soh_date` variable assignment in `refresh_and_cache_promise_dates`. These are non-functional changes aimed at improving code maintainability.

    three_pl_dermapen: Refactor Hexspoor address parsing for NL/BE orders

    This commit significantly refactors the Hexspoor address creation logic to handle complex European address formats, particularly for the Netherlands (NL) and Belgium (BE), where the house number is often included in the primary address line.

    The previous implementation struggled with these formats, leading to validation errors and manual intervention. This change introduces a more robust parsing mechanism.

    Key changes:
    -   Introduced a new `AddressValidationError` for logical address validation failures, allowing for more specific error handling.
    -   Implemented a conservative regex-based splitter (`_conservative_address_split`) to parse a single address line into street, house number, and addition. It supports common patterns like "Street 11" and "11 Street".
    -   For NL/BE orders, failure to determine a house number now raises `AddressValidationError`, which is caught in the main fulfilment loop to correctly set the order status to `BAD_ADDRESS`.
    -   Added logic (`_split_and_overflow_field`) to handle overflowing long street names or address lines by moving the excess text into the second address line field, with validation to prevent data truncation.
    -   The main processing loop in `three_pl_logic_dermapen` was updated to catch and handle the new exception, providing clearer error logging and status updates.
    -   Performed minor code cleanup by removing several unused variables in `three_pl_logic_hexspoor`.

**Commit Message:**

three_pl_dermapen: Refactor country validation to use ValidationError

In `HoxtonManager`, the address validation logic for country lookups now raises `pydantic.ValidationError` instead of a generic `ValueError`.

This change allows the calling code to specifically catch validation errors related to the address structure, providing more robust and targeted error handling. The manually constructed `ValidationError` preserves field-specific error details for better diagnostics.

Type hints in `loop_business_logic.py` and `loop_connector.py` have been updated to use this new `LoopReturn` model, replacing generic dictionaries and enhancing type safety. Additionally, a new `LoopKeyValues` TypedDict was created to better define the output of the `set_key_values` function, and variable names in the `pass_two` function were improved for clarity.

***

**Commit Message:**

shopify: Refactor LoopReturn TypedDict into a dedicated models file

This commit moves the `LoopReturn` TypedDict and its associated data structures from `loop_connector.py` into a new `loop_models.py` file.

- **Improved Modularity:** Separates the data model from the API connection logic, making the `LoopReturn` model more reusable and the connector's responsibility clearer.
- **Enhanced Type Safety:** Replaces generic `dict` and `JsonDict` type hints with the specific `LoopReturn` model in `loop_business_logic.py` and `loop_connector.py`.
- **Improved Readability:** Introduces a new `LoopKeyValues` TypedDict for the return value of `set_key_values` and clarifies variable names within the `pass_two` function for better code comprehension.

Refactor the type hints in `loop_business_logic.py` for better clarity and type safety.

- Change the `loop_returns` parameter from `List[JsonDict]` to `List[LoopReturn]` in `process_returns`.
- Add an explicit `datetime.datetime` type hint to the `highest_change_dt` parameter in `pass_one`.
- Remove trailing whitespace in `loop_configurations.py`.

# Conflicts:
#	deployment_support/includes/version.yml

Feature/pricing staging

Previously, the verification for a stock transfer would stop at the first error. Now, it continues to check for all potential issues (like line count mismatches, SKU discrepancies, and quantity differences) for a given transfer, providing a complete report of all errors. A success message is now logged for each transfer that passes all checks.

The code is also updated to handle a change in the Dear Stock Transfer data structure, accessing order lines via `stock_transfer.get("Order", {}).get("Lines", [])` instead of directly from the root object. Finally, it adds a helpful log message when no duplicate purchase orders are found.

***

**Commit Message:**

**Subject:** `revel_pos: Improve robustness of restock PO verification`

**Body:**

Refactors the `verify_restock_pos` function to provide more thorough validation and clearer log output.

The key changes include:
- The verification for a single stock transfer no longer aborts on the first error. It now accumulates all discrepancies (e.g., line counts, missing SKUs, quantity mismatches) before moving to the next transfer, ensuring a comprehensive report.
- A success message is now explicitly logged for each transfer that passes all verification checks.
- An informational message is logged if no duplicate POs are found, improving user feedback.
- The data access pattern for Dear stock transfer lines is updated to `transfer["Order"]["Lines"]` to align with the correct `StockTransfer` TypedDict structure.
- The type hint is corrected from `StockTransferLine` to `StockTransferOrderLine`.

To support this change, the patch introduces a `GEMINI.md` file in the `revel_pos` app to document this key mapping. Additionally, it adds a progress log message when fetching POs and updates the final job status from `COMPLETED` to `OK` for consistency.

revel_pos: Fix product matching logic in PO verification

- Corrects the verification logic to use `vendor_item_id` instead of `barcode` to match Revel PO lines with Dear Stock Transfer SKUs.
- Adds a `GEMINI.md` file to document the mapping between Dear stock transfers and Revel purchase orders.
- Improves user feedback by adding a progress log message during the PO fetching loop.
- Standardizes background task completion status from `COMPLETED` to `OK` for consistency with core conventions.

- Fix VerifyRestockPOsView to render the websocket progress template, enabling users to see the job status after form submission.
- Create a specific VerifyRestockPOsForm to encapsulate form logic for the verification view.
- Add job_score progress reporting to the task_delete_revel_pos background task for improved user feedback during deletion.

Change the duplicate purchase order handling logic to perform a soft delete by setting an `is_deleted` flag rather than a hard delete. This approach is safer and preserves historical data in Revel.

- The `PurchaseOrderAPI.delete_purchase_order` method is modified to send a `PATCH` request that sets `is_deleted: True`.
- The `verify_restock_pos` function is updated to exclude soft-deleted POs from its initial query.
- When identifying duplicates, a survivor PO is now selected based on invoiced status, falling back to the oldest record.
- Logging is improved to clearly distinguish the survivor PO from the duplicates suggested for deletion.

When a timeout occurs while submitting a PO, the system now attempts to find the PO in Revel to confirm if it was created despite the error. If the PO exists, or if a "duplicate" error is received, the corresponding stock transfer is marked as processed to prevent redundant attempts.

Additionally, the verification process has been enhanced. If a stock transfer is found in Dear without a corresponding PO in Revel, it is now marked as processed in both the local cache and in Dear to prevent it from being picked up in future runs.

***

**Commit Message:**

**Subject:** `revel_pos: Improve robustness of restock PO creation`

**Body:**

Implement timeout and duplicate handling for Revel PO submission.

When a `ReadTimeout` occurs during PO creation, the system now queries Revel to verify if the PO was created successfully before raising an error. This prevents failed jobs for POs that were actually created.

If a PO is found after a timeout, or if a duplicate error is received, the stock transfer is marked as processed and the system continues.

Enhance the `verify_restock_pos` task to permanently mark transfers that are missing a corresponding PO in Revel. This is done by updating the Dear stock transfer `Reference` field and adding the transfer to a `KeyValueJson` processed list, preventing future re-attempts.

revel_pos:
- Adds a new verification tool to audit Revel Purchase Orders against Dear Stock Transfers. This tool identifies duplicate POs and discrepancies in line items.
- Adds a new deletion tool to remove duplicate or erroneous Purchase Orders from Revel by their IDs.
- Improves the robustness of the restock PO creation process by enhancing the initial query to exclude already processed transfers.
- Enhances type hinting in the verification logic to use specific data models for better code clarity and maintainability.
- Fixes several bugs related to incorrect API filter parameters, date formatting, and Python syntax errors.

.run:
- Corrects a typo in the filename for the seedandsprout run configuration.

cached_dear:
- The `task_void_xero_invoices` task now fetches Xero's period and end-of-year lock dates before processing.
- It skips deleting payments, refunds, and credit note allocations if their date falls within a locked period, logging a warning instead of failing. This prevents errors when attempting to modify transactions in a closed accounting period.

shopify:
- The Loop Returns configuration lookup is now case-insensitive against the Shopify store name to prevent mismatches.
- The `verify_loop_configuration` function is more robust, checking for the existence of the `loop_discount_service_product` before validating its attributes.
- Corrected a typo in the Loop configuration from "seed-spout-us" to "seed-sprout-us" and updated its account codes.

The `task_void_xero_invoices` task was failing when attempting to void Xero invoices that had credit notes allocated to them. Additionally, credit notes voided in Cin7 Core were not being voided in Xero.

This commit enhances the synchronization logic:

- Before voiding a Xero invoice, the task now iterates through any attached credit notes and removes their specific allocations to the invoice. This allows the subsequent payment deletion and invoice voiding to proceed without error.
- Adds a new section to the task that queries for recently voided credit notes in Cin7 Core. For each one found, it locates the corresponding credit note in Xero, removes any refunds (payments), and then updates its status to VOIDED.
- The initial sync from the Cin7 Core cache is updated to include credit notes to ensure the data is available.
- Adds a documentation link to the Xero connector module.

A new `customer_ref_regex` field has been added to the `IntercompanyConfig` data model to support this, and the filtering logic in `create_intercompany_journals` has been updated accordingly.

---

Here is a draft commit message:

**Subject:**
cached_dear: Use regex on CustomerReference to find intercompany sales

**Body:**
Refactors the intercompany journal creation process to improve the reliability of identifying sales.

Previously, sales were filtered by the `SalesRepresentative` field, which could be inconsistent or incorrectly assigned. This change switches the filtering logic to use a regular expression match on the `CustomerReference` field, which typically holds the Shopify order number (e.g., `#12345`, `#YB12345`).

- Adds a `customer_ref_regex` field to the `IntercompanyConfig` data model.
- Updates store mappings to define the specific customer reference regex for each store.
- Modifies `create_intercompany_journals` to query `DearCache` using the new regex filter instead of the sales rep name.

This approach provides a more robust and accurate method for identifying intercompany transactions, reducing reliance on mutable sales rep data.

The `MIXED_LOCATION` constant has been moved from `core.core_consts` to `cached_dear.dear_api_base.custom_logging`.

This change improves code organization by placing the constant in a more contextually relevant module, as it is primarily used within the logic for handling DEAR API data.

The `three_pl` app has been updated to reflect the new import path. No functional changes are introduced.

- Includes specific error reasons in log messages for `AccountingBadRequestException` when handling Xero payment deletions and invoice voiding.
- Improves visibility into API issues by appending `e.reason` to error logs.
- Removes an unused blank line in the payment processing loop for code cleanliness.

- Removes trailing whitespace across multiple modules.
- Standardizes formatting of function signatures and docstrings.
- Ensures consistent use of spacing around operators and parameters.
- Cleans up unused or extraneous blank lines.

These changes improve code readability and maintainability without altering functionality.

The Machship API appears to have changed its requirements for datetime formats, no longer accepting the 'Z' suffix for UTC timestamps.

This commit modifies the `datetime_in_machship_string` helper to make the inclusion of the trailing 'Z' optional, and defaults to omitting it. The `update_consignment` logic is updated to use this new default, ensuring despatch timestamps are sent in the correct format.

- Extends the default pickup time in `get_pickup_and_closing_time` from 20 minutes to 180 minutes. This provides a more realistic window for courier collection when a specific pickup time is not provided.
- Adds minor docstring clarifications to Machship-related functions in `machship` and `cached_dear` apps.
- Updates `aider_prompt.md` with more detailed guidelines for the AI assistant, focusing on system-wide context, risk analysis, and long-term maintainability.

Improves the reliability of the Cin7 stock cache update process and applies general code quality improvements to the zoho_crm app.

cin7_sync:
- Add a timeout and auto-renewal to the Redis lock in `task_update_stock_data_cache` to prevent tasks from getting stuck.
- Change the periodic stock cache rebuild to be incremental (`delete_all=False`) to reduce load.
- Adjust the cron schedule for the stock data API cache task from every 15 minutes to twice an hour.
- Fix cache hit logic in `api_stock_data_retrojan_v2` to correctly handle empty datasets.

zoho_crm:
- Remove unused variables (`full_sync`, `dear_cache`) from tasks.
- Replace wildcard `typing` imports with specific imports.
- Update boolean comparisons to use `is True`/`is False` for better style and correctness.

This commit introduces several improvements to the 3PL integration error handling and notification system.

- Sets `process_status` to `BAD_ADDRESS` on Pydantic validation errors for both Hexspoor and Hoxton integrations, providing more specific feedback for address-related issues.
- Implements location-based email routing for error notifications, allowing alerts to be sent to different recipients based on the order's location.
- Aligns string literal usage with project standards by replacing single quotes with double quotes in `three_pl/models.py`.

Refactor EDI-related logic and enhance Zoho Analytics integration. These changes improve system reliability and maintainability.

This commit removes a large number of unused imports, variables, and simplifies code by converting unnecessary f-strings to regular strings across the project. This improves code cleanliness and maintainability.

Affected apps: cached_dear, cin7_omni_analytics, cin7_sync, core, dear_capsule, dear_purchasing, dear_zoho_analytics, machship, microsoft_auth, payment_gateways, rest_api, revel_pos, salesforce, sample_skeleton, shopify, starshipit, three_pl, three_pl_amethyst, three_pl_aurora, three_pl_dermapen, three_pl_onemile, three_pl_skyzer, three_pl_torque, xysense, zoho_crm

The `api_stock_data_retrojan_v2` view is modified to ignore the `branchIDs` query parameter and always use the `DEFAULT_BRANCHES` constant.

This ensures a consistent cache key for the stock data, as allowing dynamic branch IDs from the request was incompatible with the caching strategy.

cin7_sync: Adds a comment to clarify that the branchIDs query parameter is ignored in the v2 stock data API. This is a temporary measure to ensure caching consistency, requiring the branch to be hardcoded.

The background task `task_update_stock_data_cache` has been simplified to no longer accept a `branch_ids` parameter, instead relying on the new `DEFAULT_BRANCHES` constant. This centralizes the branch logic and removes the now-unused branch `38` from the default API response.

***

**Commit Message:**

Refactor(cin7_sync): Use DEFAULT_BRANCHES constant for stock cache

Removes the hardcoded list of branch IDs in the `api_stock_data_retrojan_v2` view and the `task_update_stock_data_cache` task, replacing it with a new `DEFAULT_BRANCHES` constant.

- Introduces `DEFAULT_BRANCHES = [3,]` in `api_views.py` to define a single source of truth for the default branches.
- Removes branch `38` from the default stock data processing.
- Simplifies the signature of `task_update_stock_data_cache` by removing the `branch_ids` parameter, as it now consistently uses the default.

Removes the dynamic `locations_to_use` parameter from the promise date refresh task. Locations are now determined statically based on the entity, which simplifies the API and makes caching more reliable.

- Adds `stock_data_available` and `max_possible_stock_allocated` to the cached data and inspector view.
- Renames fields in `PromiseDateReportData` for clarity.
- Adds new PyCharm run configurations for different entities.
- Deletes an empty test file.

Corrects a logging message in the stock data API to accurately report the number of cached elements.

Refactors the `task_update_stock_data_cache` to accept a `refresh_cache` parameter instead of hardcoding the value to `False`. The API view now explicitly passes `refresh_cache=False` to the task, preserving the original performance optimization. This change improves the task's flexibility and makes the caching behavior more explicit.

fix different customer region

The stock data API endpoint now triggers a cache rebuild for all branches on a cache miss, ensuring the cache is comprehensive.

- Modified `api_stock_data_retrojan_v2` to call the update task with `branch_ids=None`.
- Updated `get_stock_data_cache_key` to handle an empty branch list by using a wildcard key.
- Added `JobMaster` logging to track cache hits and misses in the API view.

Key changes include:
- The process for deleting orphaned Zoho data now compares all historical sync dates from the local database against all dates in Zoho, ensuring a more thorough cleanup instead of only looking from a recent start date.
- A new scheduled task, `delete_old_stock_sync_entries`, has been added to automatically delete `StockSync` records older than two weeks.
- The stock comparison tasks for Prism and Hexspoor now default to using the current date if no specific date is provided, and this date is explicitly passed to the Zoho refresh function.

fix(three_pl_dermapen): Improve Zoho stock data cleanup and add maintenance task

Refactors the Zoho stock analysis to ensure a full comparison of sync dates for cleaning up orphaned data, rather than only checking from a specified start date.

- Modifies `refresh_zoho_threepl_stock_analysis` to fetch all sync dates from both the local DB and Zoho for a more robust orphan detection.
- Adds a new Celery task `delete_old_stock_sync_entries` to periodically purge records older than 14 days from the `StockSync` table.
- Updates `stock_comparison_cache_prism` and `stock_comparison_zoho_update_hexspoor` tasks to correctly handle date parameters, defaulting to the current date if none is specified.

Implements a data consistency mechanism for the Zoho stock analysis. The refresh logic now deletes orphaned rows from the Zoho table for dates that no longer exist in the local StockSync model.

A new daily task is added to prune StockSync records older than 14 days, ensuring the table does not grow indefinitely. A unit test for this new task is included.

Also, the Django Debug Toolbar is disabled in the project configuration.

Apps modified: three_pl_dermapen, project