mirror of
https://github.com/Stirling-Tools/Stirling-PDF.git
synced 2026-09-03 05:10:16 +03:00
Anthony Stirling
547dce4cf3
# Description of Changes - The multi-node compose stack + behave suite (11 features) - The nightly multinode-e2e job in build-enterprise.yml cuke features are cluster_health - both nodes boot healthy and join the Valkey backplane load_balancing - traffic spreads across nodes; no spurious 401 when bounced cross_node_auth - a token from one node validates on all nodes (shared DB keys) shared_state - teams/sources/org visible from every node policy_management - create/rename/delete a policy on any node, reflected everywhere source_management - source CRUD cross-node; referenced source can't be deleted anywhere connections - S3 connection resolves (secret masked) and deletes cluster-wide processor_ledger - files processed exactly once even when both nodes trigger together policy_run_coordination - a run on one node is visible from every node rate_limiting - rate-limit counters shared via Valkey, not per node failover - LB keeps serving when a node dies; recovered node accepts existing tokens can now start a full node system with export PREMIUM_KEY=<your licence key> ./start-multinode-test.sh starts a 40 person org DB install with multi node and database (--no-seed to have without DB on startup) 4 teams 1 s3 connection 1 policy --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details.
21 lines
1.1 KiB
Gherkin
21 lines
1.1 KiB
Gherkin
@multinode @processor
|
|
Feature: Processor pipeline and exactly-once ledger
|
|
|
|
A shared processed-file ledger with an atomic (identity_hash, policy_id) primary key ensures exactly-once processing when several nodes race the same scheduled policy; depends on the seeded "Compress incoming PDFs" policy.
|
|
|
|
@slow
|
|
Scenario: Files dropped in the source are processed across the cluster when both
|
|
nodes trigger the policy at the same instant
|
|
Given the multi-node stack is running
|
|
And I am logged in as admin
|
|
And the processor workspace is clean
|
|
When I drop 5 PDF file(s) into the S3 source under "incoming/"
|
|
And I trigger the policy "Compress incoming PDFs" on every node simultaneously
|
|
Then within 90s every dropped file should be processed across the cluster
|
|
|
|
Scenario: The ledger's atomic claim enforces exactly-once
|
|
Given the multi-node stack is running
|
|
Then a duplicate ledger claim for the same file and policy is rejected
|
|
|
|
# Note: output lands in the source's own bucket so it can be re-ingested; too flaky to assert under consume-mode pruning.
|