Needs the schema half: Stirling-Tools/Stirling-PDF-SaaS#327
## Current state
The PAYG free allowance is a one-time lifetime pool.
`pricing_policy.free_tier_units` is copied into
`payg_team_extensions.free_units_remaining` once, at team creation (V14
trigger, updated in V19), and the charge pipeline decrements it until it
reaches zero. Nothing ever puts it back.
## Problem
The product promises a monthly allowance the billing model does not
grant.
- The account-link connect dialog advertises "500 free per month". That
has **merged to main** (#7415), so the claim is live and unhonoured
until this lands.
- The wallet meter already read "Process 500 PDFs free, then $X/PDF",
which reads as an allowance-then-meter model.
- `SignupRequiredBootstrap`'s own doc comment described a "free
500-op/month allowance" while its copy said only "500 free operations".
Three separate comments asserted the opposite in code
(`billing/types.ts`, `WalletSnapshotResponse`, `TeamBillingContext`), so
the two halves of the repo disagreed about what a customer is owed.
## Solution
The grant now recurs each billing period, **for every team**. Paying
does not cost you the allowance: a subscribed team draws its grant first
each period and meters only the excess, which is what the meter's copy
always described. That also matches how the grant already worked at
charge time, where it reduced metered units regardless of subscription.
### The reset is lazy, with no scheduler
`payg_team_extensions` gains `free_units_period_start`: the period
`free_units_remaining` was last written for.
- A stamp older than the current period start, or absent as on every
existing row, means the reset is owed.
`TeamBillingService.remainingForPeriod` projects it to a full grant, so
the entitlement gate and the wallet both show it the instant the period
turns.
- `JobChargeService.consumeFreeGrant` persists it on the next charge,
under the pessimistic row lock that already makes the per-job free/paid
split exact.
One rule, both callers, so display and enforcement cannot drift onto
separate schedules. A team that runs nothing for a month has nothing to
write, and no job is needed to hand out the grant.
### One period definition
"Per period" is `TeamBillingContext.periodStart`: the Stripe
subscription's current period when subscribed, the calendar month
otherwise. It was already the only period notion in the system, so the
grant joined it rather than inventing its own:
- `InstanceEntitlement.periodCapUnits` is enforced over the same window.
- `localUsageService.currentPeriodUnsynced` already buckets a linked
instance's local usage by the `periodStart` it reads from the same
snapshot, and resets its counters on that boundary.
For an un-subscribed team, the only kind the grant gates, that window is
the calendar month, which is what the copy promises.
The period rule stays in Java by choice, not necessity: SQL could reach
the Stripe period through the sync engine, but restating the rule there
would give it a second home to drift from. Hence a nullable column and
no backfill in the migration — NULL already means "stale", so every
existing team reads as owed the current period's grant.
### Refunds
A refund landing after the period turned would have stacked last
period's units on top of the fresh grant.
`JobChargeService.restoreFreeGrant` now clamps the restore to one
period's grant, taking the same row lock, and the bulk-increment
`restoreFreeUnits` query is gone. Removing it also removed a `@Query`
string that no test would have parsed before application startup.
### Copy and comments
Every comment and user-facing string that asserted the lifetime model is
corrected. The strings that changed (code defaults and `en-US` TOML
updated together):
| Key | Now reads |
| --- | --- |
| `portal.billing.walletMeter.title` / `titleWithRate` | "500 free
credits every month, then $X per PDF" |
| `portal.billing.walletMeter.capSuffix` / `barAria` | "of 500 free
credits left this month" / "Free credits remaining" |
| `payg.free.hero.capSuffix` | "of 500 free PDFs left this month" |
| `plan.freeLimit.message` | "...this month. ... It resets next month,
or keep the momentum going now..." |
| `payg.signupRequired.body` | "500 free operations a month" |
Main rewrote these keys to "500 free credits to start" while this branch
was open. The merge keeps main's credits vocabulary and drops "to
start", which asserts the one-time grant this branch removes and which
main's own connect dialog already contradicts.
Also fixed in passing: `testing/compose/payg/saas-seed.sql` still
inserted `free_tier_units_per_cycle`, the pre-V19 column name, so that
INSERT had been failing since the rename.
## How to test
Backend:
```bash
STIRLING_FLAVOR=saas ./gradlew :saas:test spotlessCheck
```
Frontend:
```bash
task frontend:typecheck && task frontend:lint && task frontend:format:check
```
New coverage, 10 tests:
- `TeamBillingServiceMoreTest` — a past-period stamp reads as a fresh
grant, a current stamp reads the stored balance, an unstamped row reads
as a fresh grant, the grant follows the Stripe window rather than the
calendar month, plus the `remainingForPeriod` rule itself including a
future stamp and null/negative balances.
- `JobChargeServiceTest` — the first charge of a new period resets and
re-stamps, an unstamped row resets, a zero-grant policy still advances
the stamp, and a refund crossing a period boundary does not exceed the
grant.
Manually, against a team whose grant is spent: set
`free_units_period_start` back a month (or leave it NULL) and the
wallet, the sidebar meter and the entitlement gate should all show a
full grant before any job runs. The first billable job should then draw
from it and write the reset.
Three tests fail on a local Windows run and pass in CI, on files this
branch does not touch: `workbenchSession.test.ts`,
`notificationActions.test.tsx`, and `:proprietary`
`FolderIdentitiesTest.identityAgreesAcrossASymlinkedAliasOfTheDirectory`.
Nothing to do here — noted so a local run does not look like a
regression.
## Merge order
The migration is additive, and Hibernate `ddl-auto=update` will add the
column in a dev environment, so either order works locally. Beyond that
the schema goes first: Stirling-Tools/Stirling-PDF-SaaS#327 targets `v3`
(staging), so it needs to reach an environment before this lands there.
# Description of Changes
This PR modernizes the project's Python tooling across GitHub Actions by
migrating CI workflows from pip-based dependency management to `uv` and
aligning Python execution with the engine project's managed environment.
### What was changed
- Replaced `actions/setup-python` and ad-hoc `pip install` steps with
`astral-sh/setup-uv` across CI workflows.
- Configured shared `uv` dependency caching using
`engine/pyproject.toml` and `engine/uv.lock`.
- Updated Python script execution to use `uv run --project engine
--locked` for a consistent runtime environment.
- Replaced package installation steps with `uv sync` for the required
dependency groups (e.g. `tools` and `cucumber`).
- Added Docker image build validation for both production and
development AI engine images.
- Updated workflow cache configuration and Docker build context where
required.
- Removed obsolete Python requirements files that are no longer needed
after the migration.
- Applied minor Python code modernizations, including import cleanup,
modern built-in generic type annotations (`list[...]`, `tuple[...]`,
`float | None`), and small style improvements.
- Removed unnecessary Python formatter/linter extensions from the
development container configuration.
### Why the change was made
- Standardize Python dependency management across the repository.
- Reduce duplicated dependency installation logic in CI.
- Improve workflow performance through shared dependency caching.
- Ensure all Python utilities execute against the same locked dependency
set managed by the engine project.
- Simplify long-term maintenance by eliminating legacy requirements
files and pip-specific workflow steps.
---
## Checklist
### General
- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings
### Documentation
- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)
### Translations (if applicable)
- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)
### UI Changes (if applicable)
- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)
### Testing (if applicable)
- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
---------
Signed-off-by: Carsten Drewes <c.drewes@stud.uni-hannover.de>
Co-authored-by: albanobattistella <34811668+albanobattistella@users.noreply.github.com>
Co-authored-by: kastenherri <116314318+kastenherri@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: James Brunton <jbrunton96@gmail.com>
# Description of Changes
Currently when calculating the output file type for some tools, the
system will get it wrong because it doesn't know about what the default
parameters in tools are, so if it doesn't have a value for some key,
it'll just bail out and say "it might not be compatible". This PR adds
logic to `ToolIO` to read the default values set for the parameters if
the tool has `ToolIOCase`s and takes them into account when figuring out
the output type. I've built it with horrible Java reflection magic to
avoid having to specify the default for params twice, which will make it
impossible for the defaults to disagree with each other. This just runs
once at startup so there's negligible performance impact.
The change is easily tested with Change Parameters, which is just
`add-password` behind the scenes but with the password params omitted
(so Change Password is always PDF->PDF, never encrypted like Add
Password).
Also (somewhat hackily) fixes a bug I noticed where saving a Change
Permissions step then leaving and returning to the pipeline will cause
the step to be reloaded as Add Password. I've added a system to
disambiguate tools which share the same endpoint (which is only these
two currently).
## Currently
<img width="455" height="135" alt="image"
src="https://github.com/user-attachments/assets/c8867d6a-599b-4f21-a2db-4a1b6ac22d73"
/>
## Now
<img width="415" height="122" alt="image"
src="https://github.com/user-attachments/assets/bd8d1bab-f00a-421b-8c91-5af0d2ad5335"
/>
# Description of Changes
- The multi-node compose stack + behave suite (11 features)
- The nightly multinode-e2e job in build-enterprise.yml
cuke features are
cluster_health - both nodes boot healthy and join the Valkey backplane
load_balancing - traffic spreads across nodes; no spurious 401 when
bounced
cross_node_auth - a token from one node validates on all nodes (shared
DB keys)
shared_state - teams/sources/org visible from every node
policy_management - create/rename/delete a policy on any node, reflected
everywhere
source_management - source CRUD cross-node; referenced source can't be
deleted anywhere
connections - S3 connection resolves (secret masked) and deletes
cluster-wide
processor_ledger - files processed exactly once even when both nodes
trigger together
policy_run_coordination - a run on one node is visible from every node
rate_limiting - rate-limit counters shared via Valkey, not per node
failover - LB keeps serving when a node dies; recovered node accepts
existing tokens
can now start a full node system with
export PREMIUM_KEY=<your licence key> ./start-multinode-test.sh
starts a 40 person org DB install with multi node and database
(--no-seed to have without DB on startup)
4 teams
1 s3 connection
1 policy
---
## Checklist
### General
- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings
### Documentation
- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)
### Translations (if applicable)
- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)
### UI Changes (if applicable)
- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)
### Testing (if applicable)
- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
# Description of Changes
Multiple named **personal** API keys per user, replacing the single
opaque per-user key.
- Create (name + one-time secret), list, and revoke named keys from the
portal Infrastructure → API Keys tab. Works self-hosted and SaaS
(`X-API-KEY`).
- Per-key usage stats (today / trailing 30 days / lifetime);
API-processed documents are attributed to the specific key in the
processor's Documents feed.
- The legacy single per-user key keeps working and is lazily represented
as a named key. Rotating it revokes its migrated shadow row so the old
secret stops authenticating.
- Per-user (not per-key) rate limiting plus a per-user active-key cap,
so minting keys can't multiply the daily quota. Name-length cap;
race-safe migration and usage recording.
Keys are strictly personal: one owner, full access, no sharing.
Team-shared / scoped keys and per-key access levels were intentionally
left out of this PR to keep it small and easy to review; they can follow
as a separate, focused change.
> Note: the screenshots from the original revision showed an earlier
team-scoped design and need refreshing.
---
## Checklist
### General
- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings
### Documentation
- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)
### Translations (if applicable)
- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)
### UI Changes (if applicable)
- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)
### Testing (if applicable)
- [x] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
---------
Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
## What this PR does
Removes the **Flyway** migration system from the SaaS build:
- drops `flyway-core` + `flyway-database-postgresql` from
`app/saas/build.gradle`
- deletes all `Vxx__*.sql` files under
`app/saas/src/main/resources/db/migration/`
- removes the `spring.flyway.*` config from
`application-saas.properties`
- clears the now-inert `SPRING_FLYWAY_ENABLED` override and stale Flyway
comments in `testing/compose/docker-compose-saas.yml`
`ddl-auto` is **left on `update`** (unchanged) — that's a separate
decision (see "Not in this PR").
## Why — Flyway never actually ran anywhere
From a full schema-management review
(`notes/FLYWAY_MIGRATION_REVIEW.md`), verified against the live
databases:
- **Prod & dev (v3):** no `flyway_schema_history` table exists in any
schema → Flyway has never executed. Schema is authored by the Supabase
migrations in `Stirling-PDF-SaaS` and applied by that repo's **GitHub
integration** (merge to `main` → prod).
- **Tests:** the saas module has zero `@SpringBootTest`; the only
real-DB integration tests (in `proprietary`) use `ddl-auto=create-drop`
and don't have Flyway on the classpath.
- **The mock-DB harness** (`testing/compose/docker-compose-saas.yml`,
PAYG cucumber) *explicitly disabled* Flyway, because the `Vxx`
migrations can't run against a clean Postgres — they assume Supabase has
already provisioned `users`/`teams` (`V2` ALTERs `users`, `V5`
references `teams`).
So Flyway was dead weight, and its 4 duplicate versions
(`V25/V26/V30/V31`) were a latent trap: re-enabling it would crash boot
on the collision. Everything it contained (schema + seeds like the
default pricing policy) is already mirrored by the Supabase migrations,
and by `saas-seed.sql` for the cucumber stack.
## ⚠️ Required follow-up (item #1) — capture the Flyway-only tables into
Supabase migrations
**This is documentation of the next step, not done in this PR.**
Seven tables were defined in Flyway with **no matching Supabase
migration**. They exist in prod today only because `ddl-auto=update`
created them from their entities. Before `ddl-auto` is ever tightened to
`validate` (see #3 below), and so any fresh Supabase branch is complete,
they must be added as Supabase migrations in
`Stirling-PDF-SaaS/supabase/migrations/`.
**Capture (CREATE) — 6 live tables** (definitions are visible in the
deleted files in this PR's diff):
| Table | Source (deleted here) | Backing entity |
|---|---|---|
| `resource_grants` | `V25__resource_grants.sql` | `ResourceGrant` |
| `integration_configs` | `V26__integration_configs.sql` |
`IntegrationConfig` |
| `policy_sources` | `V22__policy_engine_tables.sql` | `SourceEntity` |
| `policy_source_doc_counts` | `V23__policy_source_doc_counts.sql` |
`SourceDocCountEntity` |
| `policy_source_doc_totals` | `V23__policy_source_doc_counts.sql` |
`SourceDocTotalEntity` |
| `saas_user_extensions` | `V9__saas_user_team_extensions.sql` |
`SaasUserExtensions` |
Write them as `CREATE TABLE IF NOT EXISTS stirling_pdf.<name> (...)`
(idempotent — no-op against the existing prod/v3 tables). Preserve
column types/defaults/constraints from the deleted `Vxx` files.
**Drop (do NOT recreate) — 1 orphaned table:**
- `classification_labels` — created by `V30` and dropped by `V39` within
Flyway; its `ClassificationLabel` is now a plain `record`, not a JPA
entity. It lingers in prod only because Flyway's `V39` drop never ran.
The follow-up should emit a `DROP TABLE IF EXISTS
stirling_pdf.classification_labels` (mirroring `V39`'s intent), after
confirming nothing reads it.
## Not in this PR (deliberately)
- **#3 — flip saas `ddl-auto` `update` → `validate`.** Held pending team
confirmation; it has boot-risk and should be gated by a CI
"validate-boot against a fresh Supabase branch" first. Self-hosted stays
on `update` regardless.
- **#4 — `billing_subscriptions` split-brain** (prod `public` has 23,164
rows, `stirling_pdf` has 0, Java reads the empty one). Tracked
separately.
## Verification
- `:saas:compileJava` succeeds with Flyway removed (no code imports
`org.flywaydb.*`).
- No test or ArchUnit rule references Flyway or the migration files.
- No runtime/data impact: Flyway never ran against any live database.
---------
Co-authored-by: James Brunton <jbrunton96@gmail.com>
# Description of Changes
* Remove complex port selection logic from `engine.yml`. It's
inconsistent with the frontend & backend task files, and caused issues
with Docker, which have been worked around but would be simpler to just
get rid of the problem altogether
* Fix Ruff formatting of Python script
* Remove payg tests which are failing and have drifted too far from the
implementation to save directly
Adds an optional MCP server (proprietary module) that exposes Stirling's
PDF operations and AI capabilities to MCP clients. Off by default, zero
footprint when disabled.
### What
- New `/mcp` endpoint: streamable-HTTP + JSON-RPC 2.0; 8 tools
(describe_operation, pages/convert/misc/security category tools, AI,
upload, download).
- Runs real operations over an internal loopback; results returned
inline as base64 (small) or by fileId (large).
### Auth (two modes)
- OAuth2 resource server: RFC 9728 protected-resource metadata, RFC 8707
audience binding, JWKS, `mcp.tools.read/write` scopes; binds each token
to a provisioned Stirling account.
- API-key mode: reuses Stirling per-user `X-API-KEY` (no IdP needed).
### Security
- Per-user file ownership in FileStorage: async/queued writes scoped to
the submitting user; legacy/owner-less files stay readable.
- Admin allow/block list controls which operations are exposed.
- Python engine gated behind a shared secret (`X-Engine-Auth`).
- MCP filter chain is isolated and cannot weaken the main app's
security.
- Hardened: no upstream error-body leakage, log injection sanitized,
fileId path/sidecar enumeration blocked.
### Config / footprint
- Disabled by default (`mcp.enabled=false`); all beans
`@ConditionalOnProperty`.
---
## Checklist
### General
- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings
### Documentation
- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)
### Translations (if applicable)
- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)
### UI Changes (if applicable)
- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)
### Testing (if applicable)
- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
## What this PR does
Bundles the **low-risk polish items** from the [multi-agent review of
#6519](https://github.com/Stirling-Tools/Stirling-PDF/pull/6519). Each
change is independent, mechanical, and ships with focused unit-test
coverage.
The medium-severity items (\`?async=true\` OUTPUT recording,
JSON-consumes endpoint coverage, SpringBootTest harness) are tracked
separately in [\`notes/PAYG_DESIGN.md\` §7.5
PR-S4](https://github.com/Stirling-Tools/Stirling-PDF/blob/payg-s4-hardening/notes/PAYG_DESIGN.md)
— they need design decisions + bigger infrastructure work, so this PR
sticks to the mechanical wins.
Stacked on #6519. When that merges to main, this rebases cleanly — no
code changes.
## Changes
| Area | What | Why |
|---|---|---|
| **\`tool_id\` becomes route pattern** |
\`PaygChargeInterceptor.resolveToolId()\` prefers
\`HandlerMapping.BEST_MATCHING_PATTERN_ATTRIBUTE\` over
\`request.getRequestURI()\`. Truncates to 128 + WARN log +
\`payg.filter.errors\` increment when truncation fires. | Audit rollups
aggregate by endpoint instead of by every individual request's
path-variable / matrix-param variant. Silent truncation now louder. |
| **Direct PDF magic-byte check** | \`PaygOutputExtractor.extract()\`
magic-checks the body even for direct \`application/pdf\` responses. |
Asymmetric with the ZIP-entry path which always magic-checks. A tool
that emits \`application/pdf\` for a JSON / HTML payload would otherwise
pollute \`job_artifact_hash\`. |
| **DESKTOP_APP detection** | \`X-Stirling-Client: desktop\` header →
\`JobSource.DESKTOP_APP\`. | The enum value was unreachable from
\`determineSource()\`; Tauri shell traffic was mis-classified as WEB. No
anti-spoof — V12 step limits are identical for WEB/DESKTOP_APP so the
worst-case abuse value is zero today. |
| **\`max-bytes\` sensible default** | 500 MiB instead of \`null\`
(unbounded). | Covers the largest realistic Stirling responses (full
split-to-ZIP on a 1000-page document) while preventing pathological
cases from tying up the interceptor for minutes. Set to \`null\` to
disable. |
| **\`BufferedOutputStream\` for spill** | Wraps the spill
\`OutputStream\` in 64 KiB \`BufferedOutputStream\`. | Previously every
Tomcat chunk (default 8 KiB) was a separate syscall. Big spilled
responses get a syscall-bound speedup. |
| **Duration timer per phase** | \`payg.filter.duration\` tagged
\`phase=preHandle\` vs \`phase=afterCompletion\`. | Two distinct latency
distributions were blended into one histogram; hard to alert on. |
## Tests
| Test | What it covers |
|---|---|
|
\`PaygOutputExtractorTest.pdfContentType_butBodyMissingPdfMagic_returnsEmpty\`
| New direct-PDF magic-byte gate. |
|
\`PaygChargeInterceptorTest.preHandle_desktopClientHeader_setsJobSourceDesktopApp\`
| New \`X-Stirling-Client: desktop\` → DESKTOP_APP path. |
|
\`PaygChargeInterceptorTest.preHandle_toolId_prefersBestMatchingPattern\`
| Route pattern wins over URI when both are set. |
|
\`PaygChargeInterceptorTest.preHandle_toolId_truncatesAndCountsWhenLongerThan128\`
| Oversized values truncate + increment errors counter. |
Full saas suite green (210 tests), coverage targets met.
## What's NOT in this PR (deliberately)
- **\`?async=true\` OUTPUT recording.** The JobExecutorService returns a
synchronous \`JobResponse{jobId}\` body before the async tool actually
runs; \`afterCompletion\` fires too early. Needs a design decision:
short-circuit PAYG when \`async=true\` OR hook into \`TaskManager\`
completion. Tracked in PR-S4 design doc.
- **JSON-consumes endpoint coverage.** The
\`MultipartHttpServletRequest\` cast skips endpoints with \`consumes =
APPLICATION_JSON_VALUE\` (e.g.
\`ConvertPdfJsonController.exportPartialPdf\`). Fix is either extract a
request-body hash for JSON or add a CI lint forbidding non-multipart
\`@AutoJobPostMapping\`. Design discussion needed.
- **SpringBootTest harness for filter + interceptor wiring.** Saas
module doesn't have one yet. Separate work — PR-S3 takes a different
approach (docker-compose + Behave); a SpringBootTest layer would be
additive in-process coverage.
These are tracked in \`notes/PAYG_DESIGN.md §7.5\` so they don't slip.
## Tracked in
\`notes/PAYG_DESIGN.md\` §7.5 PR-S4.
## What this PR is
End-to-end cucumber coverage for the PAYG shadow charging engine (the
filter + interceptor stack from #6519), wired into CI via a new
`docker-compose-tests-saas.yml` workflow that runs only on PAYG-touching
PRs.
Stacked on #6519.
## Automated scenarios (run by `docker-compose-tests-saas.yml`)
See
[`testing/cucumber/features/payg/shadow_charges.feature`](../tree/payg-s3-cucumber/testing/cucumber/features/payg/shadow_charges.feature):
| Scenario | Validates |
|---|---|
| First tool call writes a CHARGED row | Filter + interceptor fire
end-to-end |
| Lineage join — second call on output | `JobService.joinOrOpen`
matching; no new shadow row |
| 4xx leaves the row CHARGED | "Customer paid for the attempt" semantics
|
| ZIP-returning tool records per-PDF OUTPUT | `PaygOutputExtractor`
unpacks + records signatures |
| Multi-file input writes a single shadow row | Multi-input group sizing
|
| `X-Stirling-Automation` sets PIPELINE source | Header → `JobSource`
detection |
All 6 run locally via `./testing/test-payg.sh` and will run on CI for
any PR that touches `app/saas/**`, the PAYG cucumber features, the saas
compose stack, or the workflow itself.
## Manual-only scenarios — documented in design doc, not in this suite
Two parts of the shadow engine are deliberately not automated; the
engine paths are unit-tested in
`PaygChargeInterceptorTest.afterCompletion_5xx_opened_*`, and the manual
procedures (which require a temporary throw endpoint or a container
restart with a flag flipped) live in [`notes/PAYG_DESIGN.md` §7.5.2
"PAYG cucumber: manual-only
scenarios"](../tree/payg-s3-cucumber/notes/PAYG_DESIGN.md).
- **5xx first-step failure → REFUNDED + CLOSED.** No reliably-5xx-ing
endpoint exists; manual procedure adds a throw endpoint, runs, asserts,
removes.
- **Kill-switch (`PAYG_FILTER_ENABLED=false`).** Needs a container
restart mid-suite; manual procedure tears down, flips env, brings up,
asserts zero shadow rows.
If either gets a hot-reload path (test-only throw endpoint shipped
behind a profile gate, or admin endpoint for the kill switch), automate
it in a follow-up and drop the manual procedure.
## CI workflow
`.github/workflows/docker-compose-tests-saas.yml` (new) —
self-contained, not wired into `build.yml`'s `files-changed` matrix so
the saas-cucumber job fails and succeeds independently. Triggers only on
PAYG-relevant paths. No JaCoCo coverage in v1 (saas compose doesn't have
the coverage override; can add later).
## Test infrastructure (recap)
- **`testing/compose/docker-compose-saas.yml`** — Stirling-PDF backend
with `STIRLING_FLAVOR=saas` + Postgres holding the `stirling_pdf`
schema. Supabase JWT auto-config disabled; API-key auth via
`SECURITY_CUSTOMGLOBALAPIKEY` is the live path the cucumber tests
exercise.
- **`testing/compose/payg/saas-init.sql`** + **`saas-seed.sql`** —
schema bootstrap + idempotent seed (team / user / wallet_policy).
- **`testing/cucumber/features/payg/shadow_charges.feature`** — the 6
scenarios above.
- **`testing/cucumber/features/steps/payg_step_definitions.py`** — step
defs using `requests` (HTTP) + `psycopg` (direct DB inspection). Direct
DB reads are deliberate — we want to see the filter's side effects, not
relay them through another API layer.
- **`testing/test-payg.sh`** — companion runner to `testing/test.sh`.
Brings up the saas compose, waits for health, seeds, runs behave, tears
down.
- **`behave.ini`** excludes `features/payg` from the default behave run
(the saas-cucumber CI job invokes it explicitly).
## Why a separate harness from `testing/test.sh`
The existing `test.sh` covers the proprietary-flavour stack (no PAYG
tables, no saas profile). Coupling two CI matrices that fail and succeed
independently into one script is asking for trouble. Keep the
saas-cucumber job focused on its own concerns; once the harness is
mature, the wider team can decide whether to merge them.
## Tracked in
`notes/PAYG_DESIGN.md` §7.5 (PR-S3) + §7.5.2 (manual scenarios).
Bumps the pip group with 1 update in the /testing/cucumber directory:
[idna](https://github.com/kjd/idna).
Updates `idna` from 3.12 to 3.15
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/kjd/idna/blob/master/HISTORY.md">idna's
changelog</a>.</em></p>
<blockquote>
<h2>3.15 (2026-05-12)</h2>
<ul>
<li>Enforce DNS-length cap on individual labels early in
<code>check_label</code>,
short-circuiting contextual-rule processing for oversized input
while staying compatible with UTS 46 usage.</li>
<li>Tidy core helpers: hoist bidi category sets to module-level
frozensets (avoiding per-codepoint list construction), simplify
length checks, and reuse the shared <code>_unicode_dots_re</code> from
<code>idna.core</code> in the codec module.</li>
<li>Use <code>raise ... from err</code> for proper exception chaining
and
switch internal string formatting to f-strings.</li>
<li>Allow <code>flit_core</code> 4.x in the build backend.</li>
<li>Expand the ruff lint set (flake8-bugbear, flake8-simplify,
pyupgrade, perflint) and apply the surfaced fixes; pin lint CI
to Python 3.14.</li>
<li>Add Dependabot configuration for GitHub Actions.</li>
<li>Convert README and HISTORY from reStructuredText to Markdown.</li>
<li>Reference CVE-2026-45409 for the 3.14 advisory in place of the
initial GHSA identifier.</li>
</ul>
<p>Thanks to Felix Yan, Stan Ulbrych, and metsw24-max for
contributions to this release.</p>
<h2>3.14 (2026-05-10)</h2>
<ul>
<li>Removed opportunity to process long inputs into quadratic
time by rejecting oversize inputs up-front. Closes a bypass
of the CVE-2024-3651 mitigation. [CVE-2026-45409]</li>
</ul>
<p>Thanks to Stan Ulbrych for reporting the issue.</p>
<h2>3.13 (2026-04-22)</h2>
<ul>
<li>Correct classification error for codepoint U+A7F1</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/kjd/idna/commit/af30a092e158181d0b35ac66dfa813788126bdd8"><code>af30a09</code></a>
Release 3.15</li>
<li><a
href="https://github.com/kjd/idna/commit/30314d4628744ca14cf2b5820564e5127a9f86f2"><code>30314d4</code></a>
Pre-release 3.15rc0</li>
<li><a
href="https://github.com/kjd/idna/commit/05d4b219aa9eddc47371fcbd2000f0301016f3e9"><code>05d4b21</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/237">#237</a> from
kjd/convert-docs-to-markdown</li>
<li><a
href="https://github.com/kjd/idna/commit/2987fdba1962bbb2358399e0084ba062b98a0bee"><code>2987fdb</code></a>
Convert README and HISTORY from reStructuredText to Markdown</li>
<li><a
href="https://github.com/kjd/idna/commit/59fa8002d514bf4a5ce7b58f67b9ec587d53fa9c"><code>59fa800</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/236">#236</a> from
kjd/dependabot/github_actions/actions-f3e34333ea</li>
<li><a
href="https://github.com/kjd/idna/commit/def69834ced5d4b3c50439d8b99c4c856ec19ca2"><code>def6983</code></a>
Merge branch 'master' into
dependabot/github_actions/actions-f3e34333ea</li>
<li><a
href="https://github.com/kjd/idna/commit/bbd8004a797185d8c56bb555cd5c88fde05e0631"><code>bbd8004</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/234">#234</a> from
StanFromIreland/patch-1</li>
<li><a
href="https://github.com/kjd/idna/commit/edd07c05024344a6ccb517414ccb36683aee99fc"><code>edd07c0</code></a>
Bump github/codeql-action from 3.35.2 to 4.35.2 in the actions
group</li>
<li><a
href="https://github.com/kjd/idna/commit/5557db030c11bdec50d62aa5f631d705d33ba123"><code>5557db0</code></a>
Merge branch 'master' into patch-1</li>
<li><a
href="https://github.com/kjd/idna/commit/f11746cf4981d25123ef7830d3ee60f07de8ae3d"><code>f11746c</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/235">#235</a> from
StanFromIreland/patch-2</li>
<li>Additional commits viewable in <a
href="https://github.com/kjd/idna/compare/v3.12...v3.15">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/Stirling-Tools/Stirling-PDF/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
## Description
Consolidates Playwright running under cohesive Task namespaces, isolates
Playwright state from the developer's local working tree, and swaps CI's
frontend webserver from `vite` dev to `vite preview` against a pre-built
`dist/`.
### `e2e:*` namespace
Renames `.taskfiles/testing.yml` to `.taskfiles/e2e.yml` and
consolidates everything Playwright-related under one `e2e:` namespace:
- `e2e:stubbed` / `e2e:live` / `e2e:enterprise` / `e2e:cross-browser`:
project-specific runners
- `e2e:check` (no-Docker subset) and `e2e:check:all` (full)
- `e2e:oauth:up` / `:down`, `e2e:saml:up` / `:down`: symmetric lifecycle
for the keycloak compose stacks
- `e2e:install`: Playwright browser install
- `docker:test`: full Docker integration suite
The redundant `frontend:test:e2e:*` project shortcuts are removed. CI
workflows (`e2e-stubbed.yml`, `e2e-live.yml`, `build-enterprise.yml`,
`nightly.yml`) are updated to call the new task names.
### Isolated Playwright state
New `STIRLING_BASE_PATH` (and `-Dstirling.base-path=`) override in
`InstallationPathConfig` redirects the entire state tree (configs,
backups, customFiles, pipeline, logs) at startup. `task e2e:live` points
it at `.test-state/playwright/` (purged on every invocation) so the
suite never touches the developer's local DB, settings.yml or backups.
`task e2e:live` auto-spawns gradle, waits for `/api/v1/info/status` to
come up, runs Playwright, then tears down the whole backend process
tree.
### CI runs Playwright against `vite preview`
Builds the frontend up-front with `VITE_BUILD_FOR_PREVIEW=1` (forces
absolute base so deep SPA routes resolve `/assets/...`) and the
playwright `webServer` now uses `vite preview --port 5173 --strictPort`
in CI. Avoids the per-page on-demand transform cost that was blowing the
30s navigation timeout under `--workers=3` on
`all-tool-pages-load.spec.ts`. Local dev keeps `vite` dev for HMR.
### OAuth/SAML compose helpers
`start-oauth-test.sh` and `start-saml-test.sh` gain a `--license-key
<KEY>` (`-k`) flag so CI and scripted runs can skip the interactive
license prompt. `start-oauth-test.sh` also moves from `for arg in "$@"`
to a `while`-with-`shift` arg loop to support multi-arg flags
consistently with the SAML script.
### Backend gradlew unification
Drops the per-platform `cmd /c gradlew.bat` branches from `backend.yml`
and routes every gradle invocation through `bash gradlew`. Works
uniformly on Linux/macOS and Windows-with-Git-Bash.
### Compare.tsx flake fix (re-land of
[#6316](https://github.com/Stirling-Tools/Stirling-PDF/pull/6316))
Piggybacks Anthony's never-merged fix from #6316. Without it,
`e2e:stubbed` continues to flake under `--workers=3` on
`compare.spec.ts`'s second-upload case via a React "Maximum update depth
exceeded" infinite loop in the Compare auto-fill effect. CI traces from
recent failed runs match exactly; 10 local runs of `compare.spec.ts`
with `CI=1 --workers=3` pass cleanly with the fix applied.
---------
Co-authored-by: James Brunton <jbrunton96@gmail.com>
Bumps [reportlab](https://www.reportlab.com/) from 4.4.10 to 4.5.0.
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
# Description of Changes
Fixes share-link navigation for SSO users. Reported on v2.9.2 with
`SSOAutoLogin: true`: clicking a `/share/<token>` link in an email
redirected the user to the home page after SSO instead of the shared
file.
## Root cause
Three compounding issues had to be fixed together; the first was the
initial symptom but the other two only surfaced during live
verification.
1. **Spring Security blocked `/share/<token>` for unauthenticated
users.** The route wasn't in `RequestUriUtils.isPublicAuthEndpoint`, so
the server 302'd straight to `/login` before React could load
`ShareLinkPage`. The share URL was lost because `NullRequestCache` is
configured and never persisted the original destination.
2. **`httpErrorHandler` full-page-redirected to `/login?from=<path>` on
any unhandled 401** (fired by `LicenseContext`, `AppConfig`, etc. during
normal ShareLinkPage mount). That *did* preserve the return path — but
**Spring Security strips query strings from `/login`** (302 to bare
`/login`), so `?from=` never reached React. Confirmed via `curl -i
http://localhost:8080/login?from=xyz` → `Location: /login`.
3. **`AuthCallback.tsx` unconditionally `navigate("/")`** after the
SAML/OAuth round-trip, discarding any intended destination.
## Fix
**Backend** — make `/share/<token>` a public SPA bootstrap, data APIs
stay protected:
- `RequestUriUtils.isPublicAuthEndpoint` — permits `^/share/[^/]+/?$`
(tight regex, single token segment only; `/share/<token>/anything` stays
protected).
- `ReactRoutingController` — dedicated `@GetMapping("/share/{token}")`
mirroring `/auth/callback`.
- `/api/v1/storage/share-links/**` remains behind Spring Security with
its existing `canAccessShareLink` check.
**Frontend** — persist the return path across full-page redirects via
`sessionStorage` (same-origin, survives the SSO round-trip):
- `httpErrorHandler.ts` — stashes current pathname to
`stirling_post_login_path` before the 401 → `/login` redirect.
- `springAuthClient.ts` — new `isSafePostLoginRedirect` /
`setPostLoginRedirectPath` / `consumePostLoginRedirectPath` helpers
(rejects protocol-relative URLs and auth-plumbing paths to guard against
open-redirect abuse).
- `Login.tsx` — on explicit user sign-in, read path from
`location.state` or `?from=` query and stash it; don't clobber an
already-stashed value.
- `AuthCallback.tsx` — consume the stashed path (single-use) and
`navigate(target)` instead of always `/`.
---
## Checklist
### General
- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings
### Documentation
- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)
### Translations (if applicable)
- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)
### UI Changes (if applicable)
- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)
### Testing (if applicable)
- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
---------
Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
ConnorYoh