Stirling-Tools/Stirling-PDF
1
0
Fork 0
mirror of https://github.com/Stirling-Tools/Stirling-PDF.git synced 2026-09-03 05:10:16 +03:00
Code Issues Packages Projects Releases Wiki Activity
5,957 Commits 579 Branches 186 Tags
cbfc5cab481fd5ab4b587ce9d377a65bf4999f39
Commit Graph
21 Commits
Author SHA1 Message Date
ConnorYoh 3056e5ff44 Reset the PAYG free grant each billing period (#7709) 
Needs the schema half: Stirling-Tools/Stirling-PDF-SaaS#327

## Current state

The PAYG free allowance is a one-time lifetime pool.
`pricing_policy.free_tier_units` is copied into
`payg_team_extensions.free_units_remaining` once, at team creation (V14
trigger, updated in V19), and the charge pipeline decrements it until it
reaches zero. Nothing ever puts it back.

## Problem

The product promises a monthly allowance the billing model does not
grant.

- The account-link connect dialog advertises "500 free per month". That
has **merged to main** (#7415), so the claim is live and unhonoured
until this lands.
- The wallet meter already read "Process 500 PDFs free, then $X/PDF",
which reads as an allowance-then-meter model.
- `SignupRequiredBootstrap`'s own doc comment described a "free
500-op/month allowance" while its copy said only "500 free operations".

Three separate comments asserted the opposite in code
(`billing/types.ts`, `WalletSnapshotResponse`, `TeamBillingContext`), so
the two halves of the repo disagreed about what a customer is owed.

## Solution

The grant now recurs each billing period, **for every team**. Paying
does not cost you the allowance: a subscribed team draws its grant first
each period and meters only the excess, which is what the meter's copy
always described. That also matches how the grant already worked at
charge time, where it reduced metered units regardless of subscription.

### The reset is lazy, with no scheduler

`payg_team_extensions` gains `free_units_period_start`: the period
`free_units_remaining` was last written for.

- A stamp older than the current period start, or absent as on every
existing row, means the reset is owed.
`TeamBillingService.remainingForPeriod` projects it to a full grant, so
the entitlement gate and the wallet both show it the instant the period
turns.
- `JobChargeService.consumeFreeGrant` persists it on the next charge,
under the pessimistic row lock that already makes the per-job free/paid
split exact.

One rule, both callers, so display and enforcement cannot drift onto
separate schedules. A team that runs nothing for a month has nothing to
write, and no job is needed to hand out the grant.

### One period definition

"Per period" is `TeamBillingContext.periodStart`: the Stripe
subscription's current period when subscribed, the calendar month
otherwise. It was already the only period notion in the system, so the
grant joined it rather than inventing its own:

- `InstanceEntitlement.periodCapUnits` is enforced over the same window.
- `localUsageService.currentPeriodUnsynced` already buckets a linked
instance's local usage by the `periodStart` it reads from the same
snapshot, and resets its counters on that boundary.

For an un-subscribed team, the only kind the grant gates, that window is
the calendar month, which is what the copy promises.

The period rule stays in Java by choice, not necessity: SQL could reach
the Stripe period through the sync engine, but restating the rule there
would give it a second home to drift from. Hence a nullable column and
no backfill in the migration — NULL already means "stale", so every
existing team reads as owed the current period's grant.

### Refunds

A refund landing after the period turned would have stacked last
period's units on top of the fresh grant.
`JobChargeService.restoreFreeGrant` now clamps the restore to one
period's grant, taking the same row lock, and the bulk-increment
`restoreFreeUnits` query is gone. Removing it also removed a `@Query`
string that no test would have parsed before application startup.

### Copy and comments

Every comment and user-facing string that asserted the lifetime model is
corrected. The strings that changed (code defaults and `en-US` TOML
updated together):

| Key | Now reads |
| --- | --- |
| `portal.billing.walletMeter.title` / `titleWithRate` | "500 free
credits every month, then $X per PDF" |
| `portal.billing.walletMeter.capSuffix` / `barAria` | "of 500 free
credits left this month" / "Free credits remaining" |
| `payg.free.hero.capSuffix` | "of 500 free PDFs left this month" |
| `plan.freeLimit.message` | "...this month. ... It resets next month,
or keep the momentum going now..." |
| `payg.signupRequired.body` | "500 free operations a month" |

Main rewrote these keys to "500 free credits to start" while this branch
was open. The merge keeps main's credits vocabulary and drops "to
start", which asserts the one-time grant this branch removes and which
main's own connect dialog already contradicts.

Also fixed in passing: `testing/compose/payg/saas-seed.sql` still
inserted `free_tier_units_per_cycle`, the pre-V19 column name, so that
INSERT had been failing since the rename.

## How to test

Backend:

```bash
STIRLING_FLAVOR=saas ./gradlew :saas:test spotlessCheck
```

Frontend:

```bash
task frontend:typecheck && task frontend:lint && task frontend:format:check
```

New coverage, 10 tests:

- `TeamBillingServiceMoreTest` — a past-period stamp reads as a fresh
grant, a current stamp reads the stored balance, an unstamped row reads
as a fresh grant, the grant follows the Stripe window rather than the
calendar month, plus the `remainingForPeriod` rule itself including a
future stamp and null/negative balances.
- `JobChargeServiceTest` — the first charge of a new period resets and
re-stamps, an unstamped row resets, a zero-grant policy still advances
the stamp, and a refund crossing a period boundary does not exceed the
grant.

Manually, against a team whose grant is spent: set
`free_units_period_start` back a month (or leave it NULL) and the
wallet, the sidebar meter and the entitlement gate should all show a
full grant before any job runs. The first billable job should then draw
from it and write the reset.

Three tests fail on a local Windows run and pass in CI, on files this
branch does not touch: `workbenchSession.test.ts`,
`notificationActions.test.tsx`, and `:proprietary`
`FolderIdentitiesTest.identityAgreesAcrossASymlinkedAliasOfTheDirectory`.
Nothing to do here — noted so a local run does not look like a
regression.

## Merge order

The migration is additive, and Hibernate `ddl-auto=update` will add the
column in a dev environment, so either order works locally. Beyond that
the schema goes first: Stirling-Tools/Stirling-PDF-SaaS#327 targets `v3`
(staging), so it needs to reach an environment before this lands there.
 2026-09-02 13:57:56 +00:00
LudyalbanobattistellakastenherriAnthony StirlingCopilotJames Brunton
 		05eb74022a chore(ci): migrate Python tooling to uv and standardize workflow execution (#7386) 
# Description of Changes

This PR modernizes the project's Python tooling across GitHub Actions by
migrating CI workflows from pip-based dependency management to `uv` and
aligning Python execution with the engine project's managed environment.

### What was changed

- Replaced `actions/setup-python` and ad-hoc `pip install` steps with
`astral-sh/setup-uv` across CI workflows.
- Configured shared `uv` dependency caching using
`engine/pyproject.toml` and `engine/uv.lock`.
- Updated Python script execution to use `uv run --project engine
--locked` for a consistent runtime environment.
- Replaced package installation steps with `uv sync` for the required
dependency groups (e.g. `tools` and `cucumber`).
- Added Docker image build validation for both production and
development AI engine images.
- Updated workflow cache configuration and Docker build context where
required.
- Removed obsolete Python requirements files that are no longer needed
after the migration.
- Applied minor Python code modernizations, including import cleanup,
modern built-in generic type annotations (`list[...]`, `tuple[...]`,
`float | None`), and small style improvements.
- Removed unnecessary Python formatter/linter extensions from the
development container configuration.

### Why the change was made

- Standardize Python dependency management across the repository.
- Reduce duplicated dependency installation logic in CI.
- Improve workflow performance through shared dependency caching.
- Ensure all Python utilities execute against the same locked dependency
set managed by the engine project.
- Simplify long-term maintenance by eliminating legacy requirements
files and pip-specific workflow steps.


---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

---------

Signed-off-by: Carsten Drewes <c.drewes@stud.uni-hannover.de>
Co-authored-by: albanobattistella <34811668+albanobattistella@users.noreply.github.com>
Co-authored-by: kastenherri <116314318+kastenherri@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: James Brunton <jbrunton96@gmail.com>
 2026-08-11 08:09:21 +00:00
Anthony Stirling 547dce4cf3 Add multi-node cluster regression suite (compose stack + behave e2e) (#7026) 
# Description of Changes

- The multi-node compose stack + behave suite (11 features)
- The nightly multinode-e2e job in build-enterprise.yml


cuke features are

cluster_health - both nodes boot healthy and join the Valkey backplane
load_balancing - traffic spreads across nodes; no spurious 401 when
bounced
cross_node_auth - a token from one node validates on all nodes (shared
DB keys)
shared_state - teams/sources/org visible from every node
policy_management - create/rename/delete a policy on any node, reflected
everywhere
source_management - source CRUD cross-node; referenced source can't be
deleted anywhere
connections - S3 connection resolves (secret masked) and deletes
cluster-wide
processor_ledger - files processed exactly once even when both nodes
trigger together
policy_run_coordination - a run on one node is visible from every node
rate_limiting - rate-limit counters shared via Valkey, not per node
failover - LB keeps serving when a node dies; recovered node accepts
existing tokens


can now start a full node system with 
export PREMIUM_KEY=<your licence key> ./start-multinode-test.sh
starts a 40 person org DB install with multi node and database
(--no-seed to have without DB on startup)
4 teams
1 s3 connection
1 policy

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
 2026-08-01 17:09:11 +01:00
ConnorYohandJames Brunton f60a75c253 chore(saas): remove unused Flyway migration system (#7100) 
## What this PR does

Removes the **Flyway** migration system from the SaaS build:

- drops `flyway-core` + `flyway-database-postgresql` from
`app/saas/build.gradle`
- deletes all `Vxx__*.sql` files under
`app/saas/src/main/resources/db/migration/`
- removes the `spring.flyway.*` config from
`application-saas.properties`
- clears the now-inert `SPRING_FLYWAY_ENABLED` override and stale Flyway
comments in `testing/compose/docker-compose-saas.yml`

`ddl-auto` is **left on `update`** (unchanged) — that's a separate
decision (see "Not in this PR").

## Why — Flyway never actually ran anywhere

From a full schema-management review
(`notes/FLYWAY_MIGRATION_REVIEW.md`), verified against the live
databases:

- **Prod & dev (v3):** no `flyway_schema_history` table exists in any
schema → Flyway has never executed. Schema is authored by the Supabase
migrations in `Stirling-PDF-SaaS` and applied by that repo's **GitHub
integration** (merge to `main` → prod).
- **Tests:** the saas module has zero `@SpringBootTest`; the only
real-DB integration tests (in `proprietary`) use `ddl-auto=create-drop`
and don't have Flyway on the classpath.
- **The mock-DB harness** (`testing/compose/docker-compose-saas.yml`,
PAYG cucumber) *explicitly disabled* Flyway, because the `Vxx`
migrations can't run against a clean Postgres — they assume Supabase has
already provisioned `users`/`teams` (`V2` ALTERs `users`, `V5`
references `teams`).

So Flyway was dead weight, and its 4 duplicate versions
(`V25/V26/V30/V31`) were a latent trap: re-enabling it would crash boot
on the collision. Everything it contained (schema + seeds like the
default pricing policy) is already mirrored by the Supabase migrations,
and by `saas-seed.sql` for the cucumber stack.

## ⚠️ Required follow-up (item #1) — capture the Flyway-only tables into
Supabase migrations

**This is documentation of the next step, not done in this PR.**

Seven tables were defined in Flyway with **no matching Supabase
migration**. They exist in prod today only because `ddl-auto=update`
created them from their entities. Before `ddl-auto` is ever tightened to
`validate` (see #3 below), and so any fresh Supabase branch is complete,
they must be added as Supabase migrations in
`Stirling-PDF-SaaS/supabase/migrations/`.

**Capture (CREATE) — 6 live tables** (definitions are visible in the
deleted files in this PR's diff):

| Table | Source (deleted here) | Backing entity |
|---|---|---|
| `resource_grants` | `V25__resource_grants.sql` | `ResourceGrant` |
| `integration_configs` | `V26__integration_configs.sql` |
`IntegrationConfig` |
| `policy_sources` | `V22__policy_engine_tables.sql` | `SourceEntity` |
| `policy_source_doc_counts` | `V23__policy_source_doc_counts.sql` |
`SourceDocCountEntity` |
| `policy_source_doc_totals` | `V23__policy_source_doc_counts.sql` |
`SourceDocTotalEntity` |
| `saas_user_extensions` | `V9__saas_user_team_extensions.sql` |
`SaasUserExtensions` |

Write them as `CREATE TABLE IF NOT EXISTS stirling_pdf.<name> (...)`
(idempotent — no-op against the existing prod/v3 tables). Preserve
column types/defaults/constraints from the deleted `Vxx` files.

**Drop (do NOT recreate) — 1 orphaned table:**

- `classification_labels` — created by `V30` and dropped by `V39` within
Flyway; its `ClassificationLabel` is now a plain `record`, not a JPA
entity. It lingers in prod only because Flyway's `V39` drop never ran.
The follow-up should emit a `DROP TABLE IF EXISTS
stirling_pdf.classification_labels` (mirroring `V39`'s intent), after
confirming nothing reads it.

## Not in this PR (deliberately)

- **#3 — flip saas `ddl-auto` `update` → `validate`.** Held pending team
confirmation; it has boot-risk and should be gated by a CI
"validate-boot against a fresh Supabase branch" first. Self-hosted stays
on `update` regardless.
- **#4 — `billing_subscriptions` split-brain** (prod `public` has 23,164
rows, `stirling_pdf` has 0, Java reads the empty one). Tracked
separately.

## Verification

- `:saas:compileJava` succeeds with Flyway removed (no code imports
`org.flywaydb.*`).
- No test or ArchUnit rule references Flyway or the migration files.
- No runtime/data impact: Flyway never ran against any live database.

---------

Co-authored-by: James Brunton <jbrunton96@gmail.com>
 2026-07-21 09:32:56 +00:00
Anthony Stirling 1d598d5caa MCP OAuth discovery fix + Supabase consent page (#6608) 2026-06-11 18:30:49 +01:00
Anthony Stirling 3ecd95b779 Add MCP server with OAuth/API-key auth (#6570) 
Adds an optional MCP server (proprietary module) that exposes Stirling's
PDF operations and AI capabilities to MCP clients. Off by default, zero
footprint when disabled.

### What
- New `/mcp` endpoint: streamable-HTTP + JSON-RPC 2.0; 8 tools
(describe_operation, pages/convert/misc/security category tools, AI,
upload, download).
- Runs real operations over an internal loopback; results returned
inline as base64 (small) or by fileId (large).

### Auth (two modes)
- OAuth2 resource server: RFC 9728 protected-resource metadata, RFC 8707
audience binding, JWKS, `mcp.tools.read/write` scopes; binds each token
to a provisioned Stirling account.
- API-key mode: reuses Stirling per-user `X-API-KEY` (no IdP needed).

### Security
- Per-user file ownership in FileStorage: async/queued writes scoped to
the submitting user; legacy/owner-less files stay readable.
- Admin allow/block list controls which operations are exposed.
- Python engine gated behind a shared secret (`X-Engine-Auth`).
- MCP filter chain is isolated and cannot weaken the main app's
security.
- Hardened: no upstream error-body leakage, log injection sanitized,
fileId path/sidecar enumeration blocked.

### Config / footprint
- Disabled by default (`mcp.enabled=false`); all beans
`@ConditionalOnProperty`.
---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
 2026-06-10 09:46:25 +00:00
ConnorYoh 84aca12055 PR-S4: shadow-mode hardening (review follow-ups) (#6523) 
## What this PR does

Bundles the **low-risk polish items** from the [multi-agent review of
#6519](https://github.com/Stirling-Tools/Stirling-PDF/pull/6519). Each
change is independent, mechanical, and ships with focused unit-test
coverage.

The medium-severity items (\`?async=true\` OUTPUT recording,
JSON-consumes endpoint coverage, SpringBootTest harness) are tracked
separately in [\`notes/PAYG_DESIGN.md\` §7.5
PR-S4](https://github.com/Stirling-Tools/Stirling-PDF/blob/payg-s4-hardening/notes/PAYG_DESIGN.md)
— they need design decisions + bigger infrastructure work, so this PR
sticks to the mechanical wins.

Stacked on #6519. When that merges to main, this rebases cleanly — no
code changes.

## Changes

| Area | What | Why |
|---|---|---|
| **\`tool_id\` becomes route pattern** |
\`PaygChargeInterceptor.resolveToolId()\` prefers
\`HandlerMapping.BEST_MATCHING_PATTERN_ATTRIBUTE\` over
\`request.getRequestURI()\`. Truncates to 128 + WARN log +
\`payg.filter.errors\` increment when truncation fires. | Audit rollups
aggregate by endpoint instead of by every individual request's
path-variable / matrix-param variant. Silent truncation now louder. |
| **Direct PDF magic-byte check** | \`PaygOutputExtractor.extract()\`
magic-checks the body even for direct \`application/pdf\` responses. |
Asymmetric with the ZIP-entry path which always magic-checks. A tool
that emits \`application/pdf\` for a JSON / HTML payload would otherwise
pollute \`job_artifact_hash\`. |
| **DESKTOP_APP detection** | \`X-Stirling-Client: desktop\` header →
\`JobSource.DESKTOP_APP\`. | The enum value was unreachable from
\`determineSource()\`; Tauri shell traffic was mis-classified as WEB. No
anti-spoof — V12 step limits are identical for WEB/DESKTOP_APP so the
worst-case abuse value is zero today. |
| **\`max-bytes\` sensible default** | 500 MiB instead of \`null\`
(unbounded). | Covers the largest realistic Stirling responses (full
split-to-ZIP on a 1000-page document) while preventing pathological
cases from tying up the interceptor for minutes. Set to \`null\` to
disable. |
| **\`BufferedOutputStream\` for spill** | Wraps the spill
\`OutputStream\` in 64 KiB \`BufferedOutputStream\`. | Previously every
Tomcat chunk (default 8 KiB) was a separate syscall. Big spilled
responses get a syscall-bound speedup. |
| **Duration timer per phase** | \`payg.filter.duration\` tagged
\`phase=preHandle\` vs \`phase=afterCompletion\`. | Two distinct latency
distributions were blended into one histogram; hard to alert on. |

## Tests

| Test | What it covers |
|---|---|
|
\`PaygOutputExtractorTest.pdfContentType_butBodyMissingPdfMagic_returnsEmpty\`
| New direct-PDF magic-byte gate. |
|
\`PaygChargeInterceptorTest.preHandle_desktopClientHeader_setsJobSourceDesktopApp\`
| New \`X-Stirling-Client: desktop\` → DESKTOP_APP path. |
|
\`PaygChargeInterceptorTest.preHandle_toolId_prefersBestMatchingPattern\`
| Route pattern wins over URI when both are set. |
|
\`PaygChargeInterceptorTest.preHandle_toolId_truncatesAndCountsWhenLongerThan128\`
| Oversized values truncate + increment errors counter. |

Full saas suite green (210 tests), coverage targets met.

## What's NOT in this PR (deliberately)

- **\`?async=true\` OUTPUT recording.** The JobExecutorService returns a
synchronous \`JobResponse{jobId}\` body before the async tool actually
runs; \`afterCompletion\` fires too early. Needs a design decision:
short-circuit PAYG when \`async=true\` OR hook into \`TaskManager\`
completion. Tracked in PR-S4 design doc.
- **JSON-consumes endpoint coverage.** The
\`MultipartHttpServletRequest\` cast skips endpoints with \`consumes =
APPLICATION_JSON_VALUE\` (e.g.
\`ConvertPdfJsonController.exportPartialPdf\`). Fix is either extract a
request-body hash for JSON or add a CI lint forbidding non-multipart
\`@AutoJobPostMapping\`. Design discussion needed.
- **SpringBootTest harness for filter + interceptor wiring.** Saas
module doesn't have one yet. Separate work — PR-S3 takes a different
approach (docker-compose + Behave); a SpringBootTest layer would be
additive in-process coverage.

These are tracked in \`notes/PAYG_DESIGN.md §7.5\` so they don't slip.

## Tracked in

\`notes/PAYG_DESIGN.md\` §7.5 PR-S4.
 2026-06-10 09:08:43 +00:00
ConnorYoh ff96a80947 PAYG B-3 / S-3: cucumber suite for shadow-mode flows + CI workflow (#6522) 
## What this PR is

End-to-end cucumber coverage for the PAYG shadow charging engine (the
filter + interceptor stack from #6519), wired into CI via a new
`docker-compose-tests-saas.yml` workflow that runs only on PAYG-touching
PRs.

Stacked on #6519.

## Automated scenarios (run by `docker-compose-tests-saas.yml`)

See
[`testing/cucumber/features/payg/shadow_charges.feature`](../tree/payg-s3-cucumber/testing/cucumber/features/payg/shadow_charges.feature):

| Scenario | Validates |
|---|---|
| First tool call writes a CHARGED row | Filter + interceptor fire
end-to-end |
| Lineage join — second call on output | `JobService.joinOrOpen`
matching; no new shadow row |
| 4xx leaves the row CHARGED | "Customer paid for the attempt" semantics
|
| ZIP-returning tool records per-PDF OUTPUT | `PaygOutputExtractor`
unpacks + records signatures |
| Multi-file input writes a single shadow row | Multi-input group sizing
|
| `X-Stirling-Automation` sets PIPELINE source | Header → `JobSource`
detection |

All 6 run locally via `./testing/test-payg.sh` and will run on CI for
any PR that touches `app/saas/**`, the PAYG cucumber features, the saas
compose stack, or the workflow itself.

## Manual-only scenarios — documented in design doc, not in this suite

Two parts of the shadow engine are deliberately not automated; the
engine paths are unit-tested in
`PaygChargeInterceptorTest.afterCompletion_5xx_opened_*`, and the manual
procedures (which require a temporary throw endpoint or a container
restart with a flag flipped) live in [`notes/PAYG_DESIGN.md` §7.5.2
"PAYG cucumber: manual-only
scenarios"](../tree/payg-s3-cucumber/notes/PAYG_DESIGN.md).

- **5xx first-step failure → REFUNDED + CLOSED.** No reliably-5xx-ing
endpoint exists; manual procedure adds a throw endpoint, runs, asserts,
removes.
- **Kill-switch (`PAYG_FILTER_ENABLED=false`).** Needs a container
restart mid-suite; manual procedure tears down, flips env, brings up,
asserts zero shadow rows.

If either gets a hot-reload path (test-only throw endpoint shipped
behind a profile gate, or admin endpoint for the kill switch), automate
it in a follow-up and drop the manual procedure.

## CI workflow

`.github/workflows/docker-compose-tests-saas.yml` (new) —
self-contained, not wired into `build.yml`'s `files-changed` matrix so
the saas-cucumber job fails and succeeds independently. Triggers only on
PAYG-relevant paths. No JaCoCo coverage in v1 (saas compose doesn't have
the coverage override; can add later).

## Test infrastructure (recap)

- **`testing/compose/docker-compose-saas.yml`** — Stirling-PDF backend
with `STIRLING_FLAVOR=saas` + Postgres holding the `stirling_pdf`
schema. Supabase JWT auto-config disabled; API-key auth via
`SECURITY_CUSTOMGLOBALAPIKEY` is the live path the cucumber tests
exercise.
- **`testing/compose/payg/saas-init.sql`** + **`saas-seed.sql`** —
schema bootstrap + idempotent seed (team / user / wallet_policy).
- **`testing/cucumber/features/payg/shadow_charges.feature`** — the 6
scenarios above.
- **`testing/cucumber/features/steps/payg_step_definitions.py`** — step
defs using `requests` (HTTP) + `psycopg` (direct DB inspection). Direct
DB reads are deliberate — we want to see the filter's side effects, not
relay them through another API layer.
- **`testing/test-payg.sh`** — companion runner to `testing/test.sh`.
Brings up the saas compose, waits for health, seeds, runs behave, tears
down.
- **`behave.ini`** excludes `features/payg` from the default behave run
(the saas-cucumber CI job invokes it explicitly).

## Why a separate harness from `testing/test.sh`

The existing `test.sh` covers the proprietary-flavour stack (no PAYG
tables, no saas profile). Coupling two CI matrices that fail and succeed
independently into one script is asking for trouble. Keep the
saas-cucumber job focused on its own concerns; once the harness is
mature, the wider team can decide whether to merge them.

## Tracked in

`notes/PAYG_DESIGN.md` §7.5 (PR-S3) + §7.5.2 (manual scenarios).
 2026-06-09 14:47:40 +00:00
Anthony Stirling de9d6ad3f5 Add CI coverage summaries and aggregate JaCoCo report (#6451) 2026-06-02 14:59:10 +01:00
Anthony StirlingandJames Brunton f60a075443 Add Playwright/bootRun/test.sh tasks (#6244) 
## Description

Consolidates Playwright running under cohesive Task namespaces, isolates
Playwright state from the developer's local working tree, and swaps CI's
frontend webserver from `vite` dev to `vite preview` against a pre-built
`dist/`.

### `e2e:*` namespace

Renames `.taskfiles/testing.yml` to `.taskfiles/e2e.yml` and
consolidates everything Playwright-related under one `e2e:` namespace:

- `e2e:stubbed` / `e2e:live` / `e2e:enterprise` / `e2e:cross-browser`:
project-specific runners
- `e2e:check` (no-Docker subset) and `e2e:check:all` (full)
- `e2e:oauth:up` / `:down`, `e2e:saml:up` / `:down`: symmetric lifecycle
for the keycloak compose stacks
- `e2e:install`: Playwright browser install
- `docker:test`: full Docker integration suite

The redundant `frontend:test:e2e:*` project shortcuts are removed. CI
workflows (`e2e-stubbed.yml`, `e2e-live.yml`, `build-enterprise.yml`,
`nightly.yml`) are updated to call the new task names.

### Isolated Playwright state

New `STIRLING_BASE_PATH` (and `-Dstirling.base-path=`) override in
`InstallationPathConfig` redirects the entire state tree (configs,
backups, customFiles, pipeline, logs) at startup. `task e2e:live` points
it at `.test-state/playwright/` (purged on every invocation) so the
suite never touches the developer's local DB, settings.yml or backups.

`task e2e:live` auto-spawns gradle, waits for `/api/v1/info/status` to
come up, runs Playwright, then tears down the whole backend process
tree.

### CI runs Playwright against `vite preview`

Builds the frontend up-front with `VITE_BUILD_FOR_PREVIEW=1` (forces
absolute base so deep SPA routes resolve `/assets/...`) and the
playwright `webServer` now uses `vite preview --port 5173 --strictPort`
in CI. Avoids the per-page on-demand transform cost that was blowing the
30s navigation timeout under `--workers=3` on
`all-tool-pages-load.spec.ts`. Local dev keeps `vite` dev for HMR.

### OAuth/SAML compose helpers

`start-oauth-test.sh` and `start-saml-test.sh` gain a `--license-key
<KEY>` (`-k`) flag so CI and scripted runs can skip the interactive
license prompt. `start-oauth-test.sh` also moves from `for arg in "$@"`
to a `while`-with-`shift` arg loop to support multi-arg flags
consistently with the SAML script.

### Backend gradlew unification

Drops the per-platform `cmd /c gradlew.bat` branches from `backend.yml`
and routes every gradle invocation through `bash gradlew`. Works
uniformly on Linux/macOS and Windows-with-Git-Bash.

### Compare.tsx flake fix (re-land of
[#6316](https://github.com/Stirling-Tools/Stirling-PDF/pull/6316))

Piggybacks Anthony's never-merged fix from #6316. Without it,
`e2e:stubbed` continues to flake under `--workers=3` on
`compare.spec.ts`'s second-upload case via a React "Maximum update depth
exceeded" infinite loop in the Compare auto-fill effect. CI traces from
recent failed runs match exactly; 10 local runs of `compare.spec.ts`
with `CI=1 --workers=3` pass cleanly with the fix applied.

---------

Co-authored-by: James Brunton <jbrunton96@gmail.com>
 2026-05-11 14:50:07 +00:00
Anthony Stirling 8daee7303d playwright (#6025) 2026-04-27 11:35:50 +01:00
Anthony StirlingandEthanHealy01 c294e9b2cb fix file sharing bug (#6161) 
# Description of Changes

Fixes share-link navigation for SSO users. Reported on v2.9.2 with
`SSOAutoLogin: true`: clicking a `/share/<token>` link in an email
redirected the user to the home page after SSO instead of the shared
file.

## Root cause

Three compounding issues had to be fixed together; the first was the
initial symptom but the other two only surfaced during live
verification.

1. **Spring Security blocked `/share/<token>` for unauthenticated
users.** The route wasn't in `RequestUriUtils.isPublicAuthEndpoint`, so
the server 302'd straight to `/login` before React could load
`ShareLinkPage`. The share URL was lost because `NullRequestCache` is
configured and never persisted the original destination.

2. **`httpErrorHandler` full-page-redirected to `/login?from=<path>` on
any unhandled 401** (fired by `LicenseContext`, `AppConfig`, etc. during
normal ShareLinkPage mount). That *did* preserve the return path — but
**Spring Security strips query strings from `/login`** (302 to bare
`/login`), so `?from=` never reached React. Confirmed via `curl -i
http://localhost:8080/login?from=xyz` → `Location: /login`.

3. **`AuthCallback.tsx` unconditionally `navigate("/")`** after the
SAML/OAuth round-trip, discarding any intended destination.

## Fix

**Backend** — make `/share/<token>` a public SPA bootstrap, data APIs
stay protected:
- `RequestUriUtils.isPublicAuthEndpoint` — permits `^/share/[^/]+/?$`
(tight regex, single token segment only; `/share/<token>/anything` stays
protected).
- `ReactRoutingController` — dedicated `@GetMapping("/share/{token}")`
mirroring `/auth/callback`.
- `/api/v1/storage/share-links/**` remains behind Spring Security with
its existing `canAccessShareLink` check.

**Frontend** — persist the return path across full-page redirects via
`sessionStorage` (same-origin, survives the SSO round-trip):
- `httpErrorHandler.ts` — stashes current pathname to
`stirling_post_login_path` before the 401 → `/login` redirect.
- `springAuthClient.ts` — new `isSafePostLoginRedirect` /
`setPostLoginRedirectPath` / `consumePostLoginRedirectPath` helpers
(rejects protocol-relative URLs and auth-plumbing paths to guard against
open-redirect abuse).
- `Login.tsx` — on explicit user sign-in, read path from
`location.state` or `?from=` query and stash it; don't clobber an
already-stashed value.
- `AuthCallback.tsx` — consume the stashed path (single-use) and
`navigate(target)` instead of always `/`.


---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

---------

Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
 2026-04-23 14:52:25 +01:00
unlair 3d17f0409f Fix healthcheck in Docker files when SYSTEM_ROOTURIPATH is specified (#5954) 2026-04-12 22:44:04 +01:00
Balázs SzücsandAnthony Stirling 1f9b90ad57 feat(docker): update base images to Java 25, Spring 4, Jackson 3, Gradle 9 and optimize JVM options (Project Lilliput) (#5725) 
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
 2026-02-24 20:06:32 +00:00
Anthony Stirling 8d5b3eb36b Fix SAML login "something went wrong" when language list = 1 (#5750) 
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
 2026-02-18 08:49:08 +00:00
James Brunton 5df466266a Enhance SSO SAML in desktop app (#5705) 
# Description of Changes
Change the SAML support for SSO to understand when a request is coming
from the desktop app, and use the alternate auth flow that the desktop
app requires.
 2026-02-11 16:07:06 +00:00
Anthony Stirling 00136f9e20 Saml fix (#5651) 
# Description of Changes
When password login is disabled UI changes to have central style SSO
button

<img width="2057" height="1369" alt="image"
src="https://github.com/user-attachments/assets/8f65f778-0809-4c54-a9c4-acf3a67cfa63"
/>

Auto SSO login functionality

Massively increases auth debugging visibility: verbose console logging
in ErrorBoundary, AuthProvider, Landing, AuthCallback.

Improves OAuth/SAML testability: adds Keycloak docker-compose setups +
realm JSON exports + start/validate scripts for OAuth and SAML
environments.

Hardens license upload path handling: better logs + safer directory
traversal protection by normalizing absolute paths before startsWith
check.

UI polish for SSO-only login: new “single provider” centered layout +
updated button styles (pill buttons, variants, icon wrapper, arrow).


<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
 2026-02-05 12:26:41 +00:00
Anthony Stirling 772ed6f52b OOM logs (#5405) 
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
 2026-01-07 17:35:45 +00:00
stirlingbot[bot] c990ab3216 🤖 format everything with pre-commit by stirlingbot (#5144) 
Auto-generated by [create-pull-request][1] with **stirlingbot**

[1]: https://github.com/peter-evans/create-pull-request

Signed-off-by: stirlingbot[bot] <stirlingbot[bot]@users.noreply.github.com>
Co-authored-by: stirlingbot[bot] <195170888+stirlingbot[bot]@users.noreply.github.com>
 2025-12-22 15:44:38 +00:00
Anthony StirlingandCopilot 768ece6921 Remove backend UI (#4023) 
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-08-11 12:27:18 +01:00
Anthony Stirling 3fd52ed564 testing and docker replacements 2025-07-16 23:09:26 +01:00