Compare commits

...
Author SHA1 Message Date
Anthony Stirling e6d476297d Clean up update dialog UI and fix desktop external links (#6727) 2026-06-18 22:20:02 +01:00
stirlingbot[bot] 3456316569 Update Backend 3rd Party Licenses (#6719)
Auto-generated by stirlingbot[bot]

This PR updates the backend license report based on dependency changes.

Signed-off-by: stirlingbot[bot] <stirlingbot[bot]@users.noreply.github.com>
Co-authored-by: stirlingbot[bot] <195170888+stirlingbot[bot]@users.noreply.github.com>
2026-06-18 19:54:02 +00:00
Anthony StirlingandEthanHealy01 215bba39bc Give SaaS users their own team and harden the user list endpoint (#6717)
# Description of Changes

Previously, new SaaS users were placed on a shared Default team and then
migrated to their own. A race (or a failed migration, or an
anonymous→registered upgrade) could leave them stuck on that shared
team, where unrelated users could see each other
Instead they now get their own personal team during creation so
unrelated users no longer collide on one team. SaaS-only
(@Profile("saas")); self-host's Default behaviour is untouched.
Also happens during call to avoid uncaught users

Scope GET /api/v1/user/users. Anonymous callers get 403; a caller on a
system team (Default/Internal) gets only themselves, not the team's
members.

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

---------

Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
2026-06-18 16:27:22 +00:00
ConnorYohandEthanHealy01 900b66b030 chore(saas): remove dead ErrorTrackingService island + credits path exclusion (#6718)
## What

Residual dead-code cleanup following the credits engine teardown
(#6687).

- **Delete the `ErrorTrackingService` dead island** (6 files): the
service, `UserErrorTrackerRepository`, `UserErrorTracker`,
`ProcessingErrorType`, `CreditsProperties`, and
`ErrorTrackingServiceTest`. These formed a self-referential cluster with
**zero external callers** once the credit machinery was removed.
- **Remove `/api/v1/credits/**`** from both `excludePathPatterns` blocks
in `PaygWebMvcConfig` — the credits controller no longer exists, so the
exclusion is defunct. (spotless collapsed the lists to one line.)

## Verification

- `./gradlew :saas:compileJava :saas:compileTestJava` → **BUILD
SUCCESSFUL**
- grep confirms zero dangling references to the deleted types

## Not in scope (deliberately deferred)

Destructive DB drops
(`user_credits`/`team_credits`/`user_subscription_plans` tables, dead
`payg_shadow_charge` columns, `user_error_tracker` table) are gated
behind the post-release soak (`live_ratio==1.0 ≥7d`) and tracked in a
separate bundle.

Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
2026-06-18 14:32:54 +00:00
c3795c1a3c fix(viewer): wire Ctrl+A to select all text in the PDF (#6517)
# Description of Changes

Allow Ctrl A support in viewer and fix select text to copy issues via a
hovering copy button

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

---------

Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
Co-authored-by: James Brunton <jbrunton96@gmail.com>
2026-06-18 13:50:08 +00:00
Anthony StirlingandEthanHealy01 c8925acee7 add prerendered Open Graph previews and OG card generator (#6661)
# Description of Changes

add prerendered Open Graph previews and OG card generator
so that /compresss etc shows a pre generated static html file (Since
google etc doenst render javascript)


---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

---------

Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
2026-06-18 13:49:16 +00:00
James Brunton eb08e60d67 Redesign pre-commit commands to run through Task (#6670)
# Description of Changes
The `pre-commit` commands in this repo are inconsistent with the rest of
the dev workflow, as they are impossible to run through Task and they
can cause CI to fail with no way for a developer to run the `pre-commit`
scripts after they've failed. This PR adds `task pre-commit` (and `task
pre-commit:fix`) and then hooks up the existing `pre-commit` hooks and
CI to call the Task rule, so if developers are using pre-commit hooks
then they should still work, but they're also runnable without using
pre-commit at all.

I think it'd be worth reviewing what we're actually running at
pre-commit in the future because I'm not entirely convinced by all of
the scripts that we are running, but this should at least make what we
have properly enforced and usable by all devs.
2026-06-18 12:55:53 +00:00
EthanHealy01 18da914bf9 fix theme issues, remove dead rainbow mode code, standardize theme us… (#6668)
Fix issues with the theme of the app that caused some things to persist
in light mode/dark mode whilst the rest of the app was the opposite
theme.

Removed dead rainbow mode code.

Added system theme option to settings.
2026-06-18 12:42:19 +00:00
Reece Browne 8f46ca0d92 feat(policies): lock policies to the SaaS build + profile (#6702)
## What & why

Policies (automation-backed enforcement) execute and bill through the
cloud backend, so the feature should only be available in the hosted
**SaaS** product — not in self-hosted proprietary or core builds. Today
it's enabled in the proprietary build (and the API is exposed in any
proprietary backend), so this locks it to SaaS on both layers.

## Frontend (build-flavor gate)

`POLICIES_ENABLED` is the single gate `usePoliciesEnabled` uses (rail +
auto-run controller).

- `proprietary` flag → **`false`** (self-hosted web no longer shows
policies)
- new `src/saas/constants/featureFlags.ts` → re-exports proprietary
flags, overrides `POLICIES_ENABLED = true`
- new `src/desktop/constants/featureFlags.ts` → same `true` override —
**required**: desktop's `@app` alias has no saas layer, and desktop
already gates policies on `POLICIES_ENABLED && useConfirmedSaaSMode()`,
so without a `true` here that runtime gate could never be satisfied.
Behaviour unchanged: desktop shows policies only when connected to SaaS.
- `PoliciesSidebar.test` mocks the flag on (it tests the component, not
the build gate — same pattern the existing `usePolicyAutoRun.retry.test`
uses).

## Backend (`@Profile("saas")` gate)

The saas backend runs under the `saas` Spring profile (as
`EntitlementGuard`, the AI controllers, etc. already do). The policy
beans are now `@Profile("saas")`, so `/api/v1/policies/*` and the
auto-run triggers exist **only** in the saas backend:

`PolicyController`, `PolicyEngine`, `PolicyRunner`, `PolicyRunRegistry`,
`PolicyValidator`, `JpaPolicyStore`, `FolderInputSource`,
`FolderOutputSink`, `InlineOutputSink`, `PolicyAccessGuard`,
`FolderAccessGuard`, `FolderWatchTrigger`, `ScheduleTrigger`,
`PolicyTriggerManager`.

**Deliberately *not* gated:** `PolicyExecutor` — `AiWorkflowService`
(always-on) injects it to run ad-hoc pipelines, so it stays
profile-free. It only depends on shared infra (`InternalApiClient`,
`ToolMetadataService`, `TempFileManager`, `ObjectMapper`), so leaving it
on is safe. Gating the engine/store/triggers as a set keeps wiring
consistent (nothing un-gated depends on a gated bean).

The saas `PolicyManagementAuthority` impl
(`TeamLeaderPolicyManagementAuthority`, `@Profile("saas")`) satisfies
`PolicyAccessGuard` in the saas context.
`AdminPolicyManagementAuthority` (`@Profile("!saas")`) becomes an unused
orphan in non-saas builds — harmless; left as-is rather than expanding
this PR's scope.

## Testing
- Frontend: full suite **869 pass**; typecheck clean on
proprietary/saas/core.
- Backend: `:proprietary` compiles, spotless clean, policy tests pass,
and the proprietary (non-saas) Spring context still boots with the
policy beans gated out (verified via the MCP `@SpringBootTest`
integration tests — no missing-bean failures).

Net: SaaS web build + desktop-in-SaaS-mode get policies (UI + API);
self-hosted proprietary and core get neither the UI nor the
`/api/v1/policies` endpoints.
2026-06-18 11:05:55 +00:00
Anthony Stirling 9a3bc6b47f Add cloud-aware delete and version history to My Files (#6704)
## /files
- Cloud-aware delete: choose device / cloud / both
- `/files` left rail always collapsed
- Details panel: pinned buttons, collapsible info, smaller preview
- Removed redundant Quick view
- New i18n keys added to en-US/en-GB

## Sidebar 
- Sidebar kebab: upload to server, delete, version history (+ cloud
badge)
- Version history modal everywhere files appear
2026-06-18 10:33:18 +00:00
Reece Browne 2b05865a84 Portal: unified-design surfaces (Policies, Users, Components, Agent Builder, Editor deploy) + Settings rebuild (#6696)
Builds the remaining developer-portal surfaces from the unified design
and rebuilds Settings, on top of the portal scaffold merged in #6686.
All tier-aware, mock-driven (MSW), componentised with Storybook
coverage. Touches only `frontend/portal` + `frontend/shared` — the
editor is untouched.

## New surfaces
- **Policies** — org-wide governance across the five categories
(Ingestion / Security / Compliance / Routing / Retention) with a
designer + per-doc-type overrides
- **Users** — members, roles, invite, tier-scaled SSO/SCIM access
- **Components** — embeddable `@stirling/*` SDK catalogue with
per-action pricing
- **Getting Started** — three-step funnel (use case → analyse a document
→ API key + snippets)
- **Agent Builder** — agent lifecycle (scenarios, tool modes,
evals/golden-sets, versions), reached from Sources
- **Editor deployment** — deploy/pair/operate the editor (targets,
pairing, health, credential rotation, air-gapped bundle), reached from
Infrastructure

## Reworks
- **Documents** → review/approval queue (confidence, extractions, audit
drawer, zero-standing-access elevation); the doc-type catalogue is
retained as a second tab
- **Pipelines** → golden-set pass column + "Promoted from the Editor"
section
- **Infrastructure** → new **Models** tab; deeper **Security** (managed
/ BYOK / HYOK + SOC 2 / ISO 27001 / HIPAA / GDPR / PCI attestations)
- **Home** → "What runs on your PDFs" policy summary + tier-aware
processing-status strip + pipeline-fork wizard

## Settings & shared
- New shared **`SettingsShell`** (grouped left-nav + content pane),
modelled on the editor's account-settings modal so both apps can
converge on one layout
- Portal **Settings** rebuilt on it as scoped sections — Account /
Workspace / Admin (Authentication, Active sessions, Early access)

## Brand
- Adopt the editor's brand mark + favicon; sidebar reads **Stirling
Processor**; app-switcher labels the active app "Processor"

## Mock contract
- Every surface follows the 3-layer pattern (typed `api/*` → MSW handler
→ fixtures); new endpoints documented in `MOCKS.md`. The read contract
is backend-ready; writes are marked `// TODO(backend): <METHOD> <path>`.

## Verification
- tsc (portal + shared) ✓ · eslint ✓ · dpdm (no circular) ✓ ·
`build:portal` ✓ · `storybook:build` ✓ · Prettier ✓

## Deferred (noted, not in scope)
- Unified shell / auth / role→surface routing / Workspace=Plan
(architectural epic)
- Tier rename (Editor / Processor / Bespoke) and the Usage flat-pricing
+ PAYG quick-amounts + Bespoke modal
- Editor adopting the shared `SettingsShell`; converting marked
write-stubs into live `api/` seams
2026-06-18 10:28:03 +00:00
James Brunton 0c503cc41d Fix all top-level dev tasks treating engine as enabled (#6705)
# Description of Changes
Currently, `task dev` explicitly calls the backend with
`AIENGINE_ENABLED=true` even though it isn't being spawned, so you just
get a dead FAB in the UI. This PR fixes it so that the engine will only
be enabled for tasks that will actually spawn the engine.

It also fixes a bug with the chat which makes it unusable locally. The
API path was not going through `apiClient` so for local dev you end up
with `//api/v1/...` which is not a valid path, so you get CORS errors
when trying to connect to the AI engine.
2026-06-18 10:08:50 +00:00
albanobattistella b1fef4c647 Update Italian translations (#6713) 2026-06-18 08:35:31 +00:00
EthanHealy01 06254853af allow drag and drop onto left files section and make top bar slightly smaller (#6711)
<img width="1261" height="984" alt="Screenshot 2026-06-17 at 5 59 30 PM"
src="https://github.com/user-attachments/assets/849dee17-1927-4336-81fc-dff7e91e55e7"
/>
2026-06-18 08:28:11 +00:00
Anthony Stirling d9e6041a75 set z-index on config dropdowns so they render above the modal (#6674) 2026-06-18 09:08:50 +01:00
Anthony Stirling 8f81fdc762 Use glibc base for ultra-lite and bundle per-arch JPDFium natives (#6706) 2026-06-18 08:32:55 +01:00
James Brunton 13af10a6d1 Redesign policy running (#6609)
# Description of Changes
Redesign policy running so the server is in charge of policy IDs and
running, to make it impossible to have the frontend miss the results.
This solves a minor bug that we currently have in policies, where if you
load a file and then refresh while the policy is running, you'll never
receive the outputted file.
2026-06-17 16:18:50 +00:00
EthanHealy01 3750111ffc fix agent overlay chat position when workbench size changes (#6682)
<img width="2056" height="1047" alt="Screenshot 2026-06-16 at 12 42
05 AM"
src="https://github.com/user-attachments/assets/74a38b93-f31f-4263-bb62-24c2334a22e8"
/>
<img width="1443" height="1051" alt="Screenshot 2026-06-16 at 12 42
33 AM"
src="https://github.com/user-attachments/assets/adb3ba47-f3e6-44a7-bbc3-2097e15843b6"
/>
2026-06-17 15:54:09 +00:00
ConnorYoh 20c88feabb refactor(saas): remove the legacy credits engine (FE + Java) (#6687)
Complete legacy-credits teardown ("Group 3"). The per-user/per-team
credit model is fully superseded by PAYG (`wallet_ledger`) — confirmed
no PAYG code references it. Authorized to also remove the `TeamCredit`
pool + its monthly reset.

## Frontend (saas)
- Deleted `saas/hooks/useCredits.ts`, `apiKeys/hooks/useCredits.ts`,
`types/credits.ts`, `apiKeys/UsageSection.tsx`.
- `UseSession.tsx`: removed credit members (`creditBalance`,
`creditSummary`, `hasSufficientCredits`, `updateCredits`,
`refreshCredits`, `fetchCredits`) + the credit types + global
credit-update callback. **Kept** `isPro`/`refreshProStatus` and the
Supabase auth subscription listener.
- `services/apiClient.ts`: removed the dead `x-credits-remaining`
handler + low-credit plumbing (token-refresh / PAYG / 401 logic
untouched).
- Credit refs removed from `ApiKeys.tsx`, `AppConfigModal.tsx`,
`auth/teamSession.ts`.

## Java (:saas)
**Deleted (15):** `UserCredit`(+repo),
`TeamCredit`(+repo)+`TeamCreditService`, `CreditService`,
`CreditHeaderUtils`, `CreditResetScheduler`, `CreditController`,
`CreditInterceptorConfig`, `UnifiedCreditInterceptor`,
`CreditSuccessAdvice`, `CreditErrorAdvice`, `CreditConsumptionResult` (+
the CreditController test).

**Edited — stripped legacy credit side-effects, preserved
auth/role/AI/PAYG logic:**
- `AiCreate`/`AiProxyController`: dropped the
`X-Credits-Remaining`/`X-Credit-Source` response header (its only
consumer, the desktop credit system, was already removed).
- `SaasTeamService`: dropped UserCredit/TeamCredit init on team-create +
seat-update.
- `SupabaseAuthenticationFilter` / `SupabaseSecurityConfig`: dropped
`getOrCreateUserCredits` on signup + the credit field/CORS header.
- `UserRoleService`: dropped `resetCycleAllocationForRoleChange`;
`ROLE_PRO_USER` grant/revoke preserved.
- proprietary `UserRepository`: dropped
`findUsersWithApiKeyButNoCredits()`.
- Tests updated to drop credit mocks/refs.

## Kept / scope
- `isPro` / `is_pro` RPC / `ROLE_PRO_USER` (that's the separate Group-4
/ EE effort) and **all PAYG** are untouched.
- **No DB tables dropped.** `user_credits`/`team_credits` stay until a
later **gated** migration — which this PR unblocks (the JPA entities
that pinned them are gone).

## Verify
`:saas:compileJava` + `:saas:compileTestJava` pass; FE `tsc --noEmit`
(saas) + eslint clean; 0 stray artifacts; no residual source refs to the
deleted classes.

## Follow-up (not in this PR)
`ErrorTrackingService` (+
`UserErrorTracker`/`ProcessingErrorType`/`CreditsProperties`) is now a
dead island — its only callers were the deleted interceptors. Safe to
delete, but it cascades beyond the credit scope, so it's a separate
tidy-up.

Targets `feat/desktop-cloud-saas-reuse`.
2026-06-17 14:11:06 +00:00
ConnorYoh 4f26fdeb5c feat(desktop): show the AI assistant in SaaS mode via the cloud kill switch (#6666)
## What & why

Chained on top of #6649 (the `cloud/` refactor). The AI assistant was
effectively dead on desktop:

1. **Hidden** — `ChatFAB` gates on `aiEngineEnabled`, which desktop
reads from the **local** bundled backend's `/api/v1/config/app-config`.
The local backend has no AI engine, so the flag is always `false` and
the FAB never renders.
2. **Mis-routed** — even if shown, AI calls used `getApiBaseUrl()`,
which is empty/local on desktop, so the orchestrate stream and AI
result-file download missed the engine (which only runs in the cloud).

This PR wires AI properly **without hardcoding it on**, so the cloud
keeps the kill switch: flip `aiEngineEnabled` server-side and the
desktop FAB disappears on the next load — no desktop release required.
(Deliberately *not* assume-on, so a future "turn AI off" doesn't strand
shipped versions.)

## Changes

**General SaaS app-config service** (reusable for any cloud flag, not
just AI):
- `desktop/services/saasAppConfigService.ts` — SaaS-mode-only fetch +
5-min cache of the **public** `/api/v1/config/app-config` from the
**SaaS** backend over native HTTP (`@tauri-apps/plugin-http`, no CORS).
Returns `null` outside SaaS mode.
- `desktop/hooks/useSaasAppConfig.ts` — hook over it; reloads on
connection-mode change.

**AI gating + routing seams:**
- `useAiEngineEnabled()` — core reads `useAppConfig()` (web), desktop
reads `useSaasAppConfig()`. `ChatFAB` consumes it.
- `getAiBaseUrl()` — core uses the normal API base (web), desktop points
AI calls at the SaaS backend. `ChatContext` uses it for the orchestrate
stream + result-file download.
- `operationRouter` — route `/api/v1/ai/*` to the SaaS backend
(cloud-only prefix).

**Docs:** AGENTS.md gains a short "cloud feature flags on desktop" note
so the pattern is maintained.

## Verification
- `tsc --noEmit` green for saas / desktop / cloud flavors
- `eslint --max-warnings=0` clean (cloud-layer guardrail respected — the
platform-coupled bits live in `desktop/`)
- New `saasAppConfigService.test.ts` (3 tests) + existing
`operationRouter` / `tauriHttpClient` / `httpErrorHandler` suites green
- 0 stray compiled artifacts

## Not headlessly verifiable — needs a live Tauri smoke
The orchestrate **SSE stream** uses the webview's global `fetch` (native
HTTP can't stream the body the same way), so it's subject to browser
CORS to the SaaS backend. The `SupabaseSecurityConfig` tauri-origin
allowance (from #6649) covers it, but please confirm on a real build:
open the FAB in SaaS mode, run an agent task, watch the stream + a
result-file download succeed.
2026-06-17 14:10:35 +00:00
Anthony Stirling df9dbc5179 MCP token rejection reason and stop logging the raw tokens (#6700)
- Surface the real reason an MCP token is rejected: the 401's
WWW-Authenticate header now includes error_description
(audience/issuer/expiry), and a present-but-rejected token logs the
concrete OAuth2 reason. Tokenless 401s (the normal discovery handshake)
stay at debug.
- Add McpConfigValidator that sanity-checks MCP config at startup and
logs actionable warnings (missing issuer-uri/resource-id, unrecognized
auth mode, sub + require-existing-account, open access, scopes,
allow/block overlap) so misconfig shows up in the logs before a client
ever connects.
- Align the audience-rejection message to mention both resource-id and
accepted-audiences.
- Harden audit writes: hash JWT-shaped or over-long principals
(token:<sha256-prefix>) so the insert fits the column and never stores a
raw bearer token, and stop logging the raw principal on persist failure.
---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-17 11:06:05 +00:00
Anthony Stirling de9242c4f7 Add JUnit tests for saas module coverage (#6699)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-17 11:04:53 +00:00
Anthony Stirling 460c037bbb Prefer JBoss mirror over shibboleth repo for opensaml (#6701)
# Description of Changes

Jboss not shibboleth first

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-17 10:55:59 +00:00
ConnorYohandJames Brunton cd7264a76a refactor(fe): share the SaaS PAYG experience with desktop via a cloud/ layer (#6649)
Co-authored-by: James Brunton <jbrunton96@gmail.com>
2026-06-17 11:12:05 +01:00
James Brunton ef0deef4f2 Skip flaky Playwright test (#6698)
# Description of Changes
One of the Playwright tests is flaky, despite several attempts to fix it
before it made it into main. This disables the test for now so a
followup PR can try to fix it again.
2026-06-17 09:12:09 +00:00
65fcc036fe Fix inverted link toolbar in rotated PDFs (#6518) (#6684)
Closes #6518 

# Cause of the bug 
This is a fix to the #6518 issue. The bug happened because the link
toolbar was rendered inside the PDF page layer. That layer can be
affected by the viewer/page rotation transform, so the toolbar was laid
out using local page coordinates and then visually transformed together
with the page.

As a result, the placement logic could calculate a position that was
correct in the page’s local coordinate space, such as above or below the
link, but the parent transform would rotate or shift that result after
layout. On rotated pages, this could make the toolbar appear on the
wrong side, inverted, or misaligned relative to the link.

More specifically, in the PDF that exposed the bug, the page content
appears to have been authored upside down and then corrected with a
180-degree page/viewer rotation so it looks normal to the user.

Because the toolbar was rendered inside the same transformed page layer,
it inherited that 180-degree rotation as well. The PDF content looked
upright because the rotation was part of how the page was displayed, but
the toolbar is viewer UI and should not be rotated with the page. As a
result, the tooltip appeared upside down even though the PDF itself
looked correct.


# Description of Changes

Fixes the inverted link tooltip/toolbar positioning in rotated PDF
viewer pages.

The link toolbar is now rendered through a body portal and positioned
from the link element’s real viewport bounds, so page rotation
transforms no longer flip or misalign it.

The update also keeps the toolbar within the viewport during scroll,
resize, zoom, and rotation changes, preserves the hover delay between
the link and toolbar, centralizes the z-index in a shared constant, and
improves label sizing to avoid clipped text.

Note: The link hover styling was also changed from an underline to a
subtle rectangular highlight based on the PDF link annotation bounds.

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [X] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [X] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [X] I have performed a self-review of my own code
- [X] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [X] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

UI behaviour before the changes :

<img width="1256" height="868" alt="Captura de tela de 2026-06-16
00-12-10"
src="https://github.com/user-attachments/assets/321edbb3-42a2-4bc3-96ad-3ccc70a355b8"
/>

<img width="762" height="496" alt="Captura de tela de 2026-06-16
00-11-46"
src="https://github.com/user-attachments/assets/be4c1af4-5488-4a54-9b6f-675e3bea73b8"
/>

<img width="1256" height="868" alt="Captura de tela de 2026-06-16
00-12-57"
src="https://github.com/user-attachments/assets/60f44cd5-c772-44a8-97c8-bde135764e53"
/>


UI behaviour after the changes :
 
<img width="1256" height="868" alt="Captura de tela de 2026-06-16
00-22-02"
src="https://github.com/user-attachments/assets/dda77bda-0780-4807-a70d-3bbc60683e5a"
/>

<img width="1256" height="868" alt="Captura de tela de 2026-06-16
00-23-07"
src="https://github.com/user-attachments/assets/5745c37e-438a-4bbe-ba1e-c6f2098421de"
/>

<img width="1256" height="868" alt="Captura de tela de 2026-06-16
00-23-24"
src="https://github.com/user-attachments/assets/85932541-4a6f-48e4-879f-41f34a6d79e6"
/>


### Testing (if applicable)

- [X] I have run `task check` to verify linters, typechecks, and tests
pass
- [X] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

---------

Co-authored-by: James Brunton <jbrunton96@gmail.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-06-17 08:15:31 +00:00
Reece Browne f127d4f575 fix(policies): poll runs to completion with progress, soft-retry when queue is full (#6690)
## What & why

Production reports of policy enforcement "hanging" traced to
large/many-page documents: the watermark step's flatten-to-image
(`convertPDFToImage`) on a 500+ page PDF takes minutes, exceeding both
the client poll cap and the backend per-step timeout. This makes the
slow case graceful instead of looking broken, and makes load-shedding
non-fatal.

### Poll runs to completion (no false "hang")
The client poll loop used a flat ~150s cap that was **shorter than the
backend's 300s per-step timeout**, so it abandoned long-but-healthy runs
mid-flight. The budget is now sized to the backend's real worst case —
`stepCount × per-step timeout + grace`, learned from the first status
report — so the client always polls long enough to surface the run's
**actual** terminal state (success or the backend's real error) rather
than a misleading client-side timeout.

### Per-step progress
The activity feed now shows `Enforcing… · step n/m` (from
`currentStep`/`stepCount`), so a slow step shows movement instead of a
dead spinner.

### Soft-retry on queue rejection
Under load the shared `JobQueue` rejects runs ("queue full"), which
previously surfaced as a hard failure needing a manual Retry. The
backend now tags that rejection with a stable `POLICY_QUEUE_FULL`
errorCode; the client treats it as transient backpressure and
**auto-retries the file in place** with exponential backoff (≈4s→64s, ~2
min), shown as a soft "Busy — retrying…" row, falling back to the manual
Retry only once the retry budget is spent.

## Testing
- **Frontend unit tests** (30 pass across the policies suite), including
a new `usePolicyAutoRun.retry.test.tsx` that drives the real controller
orchestration (poll → `POLICY_QUEUE_FULL` → relabel → backoff → in-place
re-dispatch), plus poll-budget, step-progress, and activity-feed relabel
cases.
- **Backend** `PolicyEngineTest` case asserting a queue-rejected run
carries the `POLICY_QUEUE_FULL` code.
- Typecheck clean on all three flavors (proprietary/saas/core); prettier
+ spotless clean.
- Poll-budget + progress + real-error surfacing were also verified live
end-to-end against a 599-page run (survived past the old cap, showed
step progress, reported the backend's real 300s-timeout failure,
recovered after a simulated network drop).

## Not included (follow-ups)
- The underlying flatten-to-image cost itself (bounded-memory/streaming
flatten, revisiting `convertPDFToImage` default and the 300s timeout) —
the real perf fix, deliberately out of scope here.
2026-06-16 17:32:12 +00:00
Anthony Stirling f33f4f8f75 Add JUnit tests for common and core module coverage (#6675)
# Description of Changes

JUNITS!
They JUnits were 100% AI generated however no code was touched

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-16 16:23:34 +00:00
7e67bfc459 Fix SaaS issues (#6694)
# Description of Changes

Fixes several SaaS issues,  was integration branch for saas release

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: James Brunton <jbrunton96@gmail.com>
Co-authored-by: Reece Browne <74901996+reecebrowne@users.noreply.github.com>
Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
Co-authored-by: Reece <reece@stirlingpdf.com>
Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
Co-authored-by: Ludy <Ludy87@users.noreply.github.com>
2026-06-16 17:16:07 +01:00
Anthony Stirling 686fb1fb50 Revert "SaaS fixes" (#6693)
Reverts Stirling-Tools/Stirling-PDF#6578 due to mistaken squash merge
not normal merge
2026-06-16 17:13:10 +01:00
Anthony Stirling 5389e39cfc Revert "SaaS fixes (#6578)"
This reverts commit ddf78d11ae.
2026-06-16 16:48:30 +01:00
ddf78d11ae SaaS fixes (#6578)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: James Brunton <jbrunton96@gmail.com>
Co-authored-by: Reece Browne <74901996+reecebrowne@users.noreply.github.com>
Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
Co-authored-by: Reece <reece@stirlingpdf.com>
Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
Co-authored-by: Ludy <Ludy87@users.noreply.github.com>
2026-06-16 16:41:25 +01:00
James Brunton 10b4551449 Fix failing Playwright test in SaaS (#6688)
# Description of Changes
Fix Playwright failing test in SaaS (I think this is my third attempt
now so who knows if this will actually fix it for real this time, but
hopefully it does)
2026-06-16 15:56:30 +01:00
Reece Browne 96accea984 Portal: full mock-driven surfaces, demonolithed components, backend-ready mocks (#6686) 2026-06-16 12:20:35 +01:00
James Brunton 9a883be697 Cleanup of SaaS code (#6669)
# Description of Changes
De-AI comments and fix ridiculously indented code
2026-06-16 11:49:13 +01:00
dependabot[bot]andAnthony Stirling 6716398ccb build(deps): bump go-task/setup-task from 2.0.0 to 2.1.0 (#6429)
Bumps [go-task/setup-task](https://github.com/go-task/setup-task) from
2.0.0 to 2.1.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/go-task/setup-task/releases">go-task/setup-task's
releases</a>.</em></p>
<blockquote>
<h2>v2.1.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Replaced <code>typed-rest-client</code> with
<code>@actions/http-client</code> for GitHub API calls
to eliminate the Node 24 <code>DEP0169</code> deprecation warning about
<code>url.parse()</code> (<a
href="https://redirect.github.com/go-task/setup-task/issues/5">#5</a> by
<a href="https://github.com/vmaerten"><code>@​vmaerten</code></a>).</li>
<li>Modernized the TypeScript tooling stack (vitest, oxlint,
<code>@actions/core@2</code>,
<code>@actions/io@2</code>, updated <code>@types/node</code>,
<code>@vercel/ncc</code>, <code>prettier</code>, etc.) (<a
href="https://redirect.github.com/go-task/setup-task/issues/5">#5</a> by
<a href="https://github.com/vmaerten"><code>@​vmaerten</code></a>).</li>
<li>Migrated the project to ESM (sources + bundle). Aligns with the new
<code>@actions/*</code> ESM-only majors and produces a ~47% smaller
<code>dist/index.js</code> (<a
href="https://redirect.github.com/go-task/setup-task/issues/5">#5</a> by
<a href="https://github.com/vmaerten"><code>@​vmaerten</code></a>).</li>
<li>Upgraded <code>@actions/core</code> 2 → 3,
<code>@actions/http-client</code> 2 → 4,
<code>@actions/io</code> 2 → 3, <code>@actions/tool-cache</code> 2 → 4,
<code>typescript</code> 5 → 6, and
<code>markdownlint-cli</code> 0.47 → 0.48 (<a
href="https://redirect.github.com/go-task/setup-task/issues/5">#5</a> by
<a href="https://github.com/vmaerten"><code>@​vmaerten</code></a>).</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/go-task/setup-task/blob/main/CHANGELOG.md">go-task/setup-task's
changelog</a>.</em></p>
<blockquote>
<h1>Changelog</h1>
<h2>Unreleased</h2>
<h2>v2.1.0 - 2026-05-17</h2>
<ul>
<li>Replaced <code>typed-rest-client</code> with
<code>@actions/http-client</code> for GitHub API calls
to eliminate the Node 24 <code>DEP0169</code> deprecation warning about
<code>url.parse()</code>.</li>
<li>Modernized the TypeScript tooling stack (vitest, oxlint,
<code>@actions/core@2</code>,
<code>@actions/io@2</code>, updated <code>@types/node</code>,
<code>@vercel/ncc</code>, <code>prettier</code>, etc.).</li>
<li>Migrated the project to ESM (sources + bundle). Aligns with the new
<code>@actions/*</code> ESM-only majors and produces a ~47% smaller
<code>dist/index.js</code>.</li>
<li>Upgraded <code>@actions/core</code> 2 → 3,
<code>@actions/http-client</code> 2 → 4,
<code>@actions/io</code> 2 → 3, <code>@actions/tool-cache</code> 2 → 4,
<code>typescript</code> 5 → 6, and
<code>markdownlint-cli</code> 0.47 → 0.48.</li>
</ul>
<h2>v2.0.0 - 2026-03-18</h2>
<ul>
<li><strong>BREAKING</strong>: Upgraded to Node 24. Requires a GitHub
Actions runner with
Node.js 24 support
(<a
href="https://redirect.github.com/go-task/setup-task/pull/10">#10</a> by
<a href="https://github.com/vmaerten"><code>@​vmaerten</code></a>).</li>
</ul>
<h2>v1.1.0 - 2026-03-17</h2>
<ul>
<li>Added configurable HTTP retry for API requests
(<a href="https://redirect.github.com/go-task/setup-task/pull/7">#7</a>
by <a
href="https://github.com/vmaerten"><code>@​vmaerten</code></a>).</li>
</ul>
<h2>v1.0.0 - 2025-09-12</h2>
<ul>
<li>Forked <a
href="https://github.com/arduino/setup-task">arduino/setup-task</a> (by
<a href="https://github.com/pd93"><code>@​pd93</code></a>).</li>
<li>Default <code>repo-token</code> to <code>{{github.token}}</code>
(<a
href="https://redirect.github.com/arduino/setup-task/pull/642">arduino/setup-task#642</a>
by
<a href="https://github.com/shrink"><code>@​shrink</code></a>).</li>
<li>Fixed a bug where the action would fail is Task pushed a tag without
a release
(<a
href="https://redirect.github.com/arduino/setup-task/pull/490">arduino/setup-task#490</a>,
<a
href="https://redirect.github.com/arduino/setup-task/pull/1193">arduino/setup-task#1193</a>
by
<a href="https://github.com/trim21"><code>@​trim21</code></a>).</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/go-task/setup-task/commit/01a4adf9db2d14c1de7a560f09170b6e0df736aa"><code>01a4adf</code></a>
chore: release v2.1.0</li>
<li><a
href="https://github.com/go-task/setup-task/commit/56fc0886350e15a75ed1e6f4b7a83d3b831b3054"><code>56fc088</code></a>
fix(taskfile): make mktemp utilities portable across BSD and GNU</li>
<li><a
href="https://github.com/go-task/setup-task/commit/4de50203767624993e228e2135c1d13cc156b095"><code>4de5020</code></a>
chore(release): add release task automation (<a
href="https://redirect.github.com/go-task/setup-task/issues/14">#14</a>)</li>
<li><a
href="https://github.com/go-task/setup-task/commit/f95f6c5aebc71143d70361ab79d87c447fd4c48f"><code>f95f6c5</code></a>
chore(deps): update all dependencies (<a
href="https://redirect.github.com/go-task/setup-task/issues/12">#12</a>)</li>
<li><a
href="https://github.com/go-task/setup-task/commit/dc4f00abd355059e622d428a1a905dfcd1169477"><code>dc4f00a</code></a>
chore(deps): update all dependencies (<a
href="https://redirect.github.com/go-task/setup-task/issues/2">#2</a>)</li>
<li><a
href="https://github.com/go-task/setup-task/commit/035a5f11fa6bbb298cc436d4173402c6ebfbc411"><code>035a5f1</code></a>
chore: modernize stack (<a
href="https://redirect.github.com/go-task/setup-task/issues/5">#5</a>)</li>
<li><a
href="https://github.com/go-task/setup-task/commit/099972a06751959896ae32ae844ed17001f59da5"><code>099972a</code></a>
docs: mark v2.0.0 release in changelog</li>
<li>See full diff in <a
href="https://github.com/go-task/setup-task/compare/3be4020d41929789a01026e0e427a4321ce0ad44...01a4adf9db2d14c1de7a560f09170b6e0df736aa">compare
view</a></li>
</ul>
</details>
<br />

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-06-15 22:04:16 +00:00
dependabot[bot]andAnthony Stirling 5b20257dea build(deps): bump actions/stale from 10.2.0 to 10.3.0 (#6487)
Bumps [actions/stale](https://github.com/actions/stale) from 10.2.0 to
10.3.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/stale/releases">actions/stale's
releases</a>.</em></p>
<blockquote>
<h2>v10.3.0</h2>
<h2>What's Changed</h2>
<h3>Bug Fix</h3>
<ul>
<li>Enhancement: ignore stale labeling events by <a
href="https://github.com/shamoon"><code>@​shamoon</code></a> in <a
href="https://redirect.github.com/actions/stale/pull/1311">actions/stale#1311</a></li>
</ul>
<h3>Dependency Updates</h3>
<ul>
<li>Upgrade dependencies (<code>@​actions/core</code>,
<code>@​octokit/plugin-retry</code>, <a
href="https://github.com/typescript-eslint"><code>@​typescript-eslint</code></a>)
by <a href="https://github.com/Copilot"><code>@​Copilot</code></a> in <a
href="https://redirect.github.com/actions/stale/pull/1335">actions/stale#1335</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/shamoon"><code>@​shamoon</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/stale/pull/1311">actions/stale#1311</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/stale/compare/v10...v10.3.0">https://github.com/actions/stale/compare/v10...v10.3.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/actions/stale/commit/eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899"><code>eb5cf3a</code></a>
chore: upgrade dependencies and bump version to 10.3.0 (<a
href="https://redirect.github.com/actions/stale/issues/1335">#1335</a>)</li>
<li><a
href="https://github.com/actions/stale/commit/db5d06a4c82d5e94513c09c406638111df61f63e"><code>db5d06a</code></a>
Enhancement: ignore stale labeling events (<a
href="https://redirect.github.com/actions/stale/issues/1311">#1311</a>)</li>
<li>See full diff in <a
href="https://github.com/actions/stale/compare/b5d41d4e1d5dceea10e7104786b73624c18a190f...eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/stale&package-manager=github_actions&previous-version=10.2.0&new-version=10.3.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-06-15 22:04:09 +00:00
dependabot[bot]andAnthony Stirling 2f5fc7be4e build(deps): bump depot/build-push-action from 1.17.0 to 1.18.0 (#6488)
Bumps
[depot/build-push-action](https://github.com/depot/build-push-action)
from 1.17.0 to 1.18.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/depot/build-push-action/releases">depot/build-push-action's
releases</a>.</em></p>
<blockquote>
<h2>v1.18.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Upgrade action runtime to Node 24 (<a
href="https://redirect.github.com/depot/build-push-action/issues/48">#48</a>)
<a href="https://github.com/Akatama"><code>@​Akatama</code></a></li>
<li>Add Depot Registry save example (<a
href="https://redirect.github.com/depot/build-push-action/issues/47">#47</a>)
<a href="https://github.com/maschwenk"><code>@​maschwenk</code></a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/depot/build-push-action/commit/98e78adca7817480b8185f474a400b451d74e287"><code>98e78ad</code></a>
Merge pull request <a
href="https://redirect.github.com/depot/build-push-action/issues/48">#48</a>
from depot/upgrade-node-24-runtime</li>
<li><a
href="https://github.com/depot/build-push-action/commit/e97ebff18729ac91be067461138674a006ab9bff"><code>e97ebff</code></a>
Remove Node 24 compatibility docs</li>
<li><a
href="https://github.com/depot/build-push-action/commit/2db929fa768ebb3ad332ae8fc530412bf0964782"><code>2db929f</code></a>
Upgrade action runtime to Node 24</li>
<li><a
href="https://github.com/depot/build-push-action/commit/f78af826a1c272c4b60c485e934974b515094928"><code>f78af82</code></a>
Merge pull request <a
href="https://redirect.github.com/depot/build-push-action/issues/47">#47</a>
from maschwenk/maschwenk/add-depot-registry-example</li>
<li><a
href="https://github.com/depot/build-push-action/commit/6855818d5954fa4361879bb0b0e5c32856fc6703"><code>6855818</code></a>
Update action.yml</li>
<li><a
href="https://github.com/depot/build-push-action/commit/b984f6a1944d5420eefb2b012d6eb856249bd225"><code>b984f6a</code></a>
Clarify save/save-tag/save-tags input descriptions</li>
<li><a
href="https://github.com/depot/build-push-action/commit/1a34abd3707433f4f7b6d594e49e56b4b9f4d6d0"><code>1a34abd</code></a>
Add Depot Registry save example</li>
<li>See full diff in <a
href="https://github.com/depot/build-push-action/compare/5f3b3c2e5a00f0093de47f657aeaefcedff27d18...98e78adca7817480b8185f474a400b451d74e287">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=depot/build-push-action&package-manager=github_actions&previous-version=1.17.0&new-version=1.18.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-06-15 22:04:03 +00:00
Anthony Stirling d18caf6116 Fix PDF text selection locked out on touch devices (#6656) 2026-06-15 23:04:43 +01:00
dependabot[bot]andAnthony Stirling 3bafbb1919 build(deps): bump docker/metadata-action from 6.0.0 to 6.1.0 (#6490)
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
2026-06-15 23:03:55 +01:00
James Brunton 42c1cce56d Fix Java formatting 2026-06-15 15:05:26 +01:00
fb6a118be9 Update SaaS to latest main (#6667)
# Description of Changes
> [!warning]
> **Do not** squash this on merge. It should be merged via a merge
commit

Fixes conflicts in `pgvector_store.py`. 

Also since codespell is failing, add comments to ignore the errors in
`sync_en_us_spelling.py`

---------

Co-authored-by: Ludy <Ludy87@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-06-15 14:53:22 +01:00
James Brunton c55beacead Shut codespell up 2026-06-15 14:20:25 +01:00
James Brunton 04d68c650a Merge remote-tracking branch 'origin/main' into saas-update
# Conflicts:
#	engine/src/stirling/documents/pgvector_store.py
2026-06-15 14:10:21 +01:00
Anthony Stirling 9d7467cf90 Scope signing user picker to team for multi-tenant SaaS (#6583) 2026-06-15 13:44:34 +01:00
James Brunton 2a905c01c3 SaaS tidying (#6665)
# Description of Changes
* Remove complex port selection logic from `engine.yml`. It's
inconsistent with the frontend & backend task files, and caused issues
with Docker, which have been worked around but would be simpler to just
get rid of the problem altogether
* Fix Ruff formatting of Python script
* Remove payg tests which are failing and have drifted too far from the
implementation to save directly
2026-06-15 13:21:33 +01:00
Anthony Stirling d6a5777c69 Fix pgvector (#6591) 2026-06-15 13:09:40 +01:00
James Brunton c1a637d764 Fix CI errors in SaaS (#6662)
# Description of Changes
Fix CI errors in #6578 to make SaaS branch ready for merge into main
2026-06-15 11:26:29 +01:00
LudyandJames Brunton 085ad6c784 build(taskfile): use platform-specific Gradle wrapper command (#6355)
Co-authored-by: James Brunton <jbrunton96@gmail.com>
2026-06-14 15:50:31 +01:00
Ludy cb13102117 chore(frontend): make Vite allowed hosts configurable (#6354) 2026-06-14 15:49:45 +01:00
Ludy 1fa1293b39 chore(build): upgrade Gradle wrapper and Docker build images to 9.5.1 (#6501) 2026-06-14 15:48:44 +01:00
Ludy 1ce765ab1e refactor: replace legacy Paths usage with Path.of (#6441) 2026-06-14 15:48:28 +01:00
Ludy dad2425c27 docs: Document pluralization suffix usage for translations (#6650) 2026-06-14 15:47:57 +01:00
Anthony Stirling eefa8eff61 Route mobile scanner API and vendor loads through the app base path (#6648) 2026-06-12 16:00:11 +01:00
Reece Browne 63ecbe3b6d Policies: centre the collapsed-rail policy button between its dividers (#6646) 2026-06-12 13:39:03 +01:00
Anthony Stirling f1ed850a73 Fix SaaS mobile scanner being auth-gated under /app base path (#6642) 2026-06-12 13:13:49 +01:00
Anthony Stirling b11c272e87 Feature/v2/guest action gating (#6643) 2026-06-12 13:13:38 +01:00
Reece Browne f5e697347b Policies: drop the Pro-license gate from the policy API (#6645)
`PolicyController` was annotated `@PremiumEndpoint` (requires a
Pro-or-higher server license). Policies don't need a server-license
gate:

- On SaaS the server runs in Pro mode, so the check is always satisfied
anyway — it gates nothing in practice.
- Access is already governed by team scoping (#6632) plus the per-user /
guest gates.

So the annotation is dead weight and misleading. This removes it (and
its import) — a 2-line change.

## Verification
- `:proprietary:compileJava` succeeds; spotless clean. No other premium
gate on policy classes.
2026-06-12 13:13:01 +01:00
Reece Browne 4e880c7510 Policies: summon the guest sign-up banner when a guest clicks a policy (#6644)
Guests (anonymous users on a login-enabled deployment) could open a
policy's setup/detail. Policies are an account feature, so a guest
clicking a policy should be nudged to sign up rather than opening it.

## Behaviour
A guest clicking a policy row — or a collapsed-rail icon — now
**re-summons the existing guest sign-up banner** ("You're using Stirling
PDF as a guest!…") and does **not** open the policy.

- `GuestUserBanner` listens for a `stirling:show-guest-banner` window
event and re-shows (even if previously dismissed; the render guard still
hides it for non-anonymous users).
- The policy sidebar dispatches that event on a guest click (cross-layer
via `CustomEvent`, same pattern as `payg:signupRequired`; a no-op on
builds without the banner).
- `usePolicyGuestBlocked()` gates it: `config.enableLogin === true &&
user.is_anonymous === true`.
- **Login-disabled single-user** deployments have an anonymous local
operator with full access → not gated.

## Verification
- Typecheck clean (proprietary + saas); eslint clean; sidebar tests
pass.

## Note
No dedicated guest unit test — the suite mocks `useAppConfig` at module
scope, and making it per-test controllable needs `vi.hoisted` plumbing
that risked the existing tests. Easy follow-up.
2026-06-12 13:10:37 +01:00
James Brunton 511b92b321 De-AI the onboarding prose (#6641)
# Description of Changes
De-AI the onboarding prose.
2026-06-12 11:39:19 +01:00
ConnorYoh 87723d3ce2 fix(payg): fire the usage-limit modal when an AI agent run hits the limit (#6638)
## Problem

We're getting 402s when an AI **agent** (chat) run hits the free
allowance / spending cap, but the frontend handles them poorly and never
pops the usage-limit modal.

The agent runs its tool calls **server-side** (loopback HTTP via
`PolicyExecutor`), so the 402 never reaches the `apiClient` interceptor
that pops the modal for direct calls. It was caught by the generic
tool-failure handler and flattened into a `CANNOT_CONTINUE` reason
string (`"The /api/v1/… tool failed: 402…"`), streamed as a `result`
event, and rendered as a scary chat bubble. This is the same gap the
policy auto-run path bridges (#6626) — one layer up.

## Fix

**Backend** (`proprietary`)
- `AiWorkflowResponse` gains `errorCode` + `errorSubscribed`.
- `AiWorkflowService` detects a downstream 401/402 entitlement sentinel
in its three tool-exec catch sites (`onToolCall`, `runPlan`,
`onConvertMarkdown`) and surfaces the structured code (+ `subscribed`)
on the terminal response instead of the raw failure text.
- Factored the 401/402 body extraction `PolicyEngine` already had into a
shared `DownstreamEntitlementError` util so the two server-side paths
can't drift.

**Frontend**
- New `usageLimitBridge` (`PAYG_LIMIT_REACHED_EVENT` +
`dispatchPaygLimitReached`) generalises the previously policy-only
bridge. Proprietary can't import the saas modal API (layering), so
server-side limit hits broadcast a window event the saas
`UsageLimitModalHost` opens the modal from. Migrated the policy path
onto it.
- `ChatContext` fires the matching modal (free → subscribe, subscribed →
raise cap) on the limit result **and** on a direct 402, replacing the
raw reason with a brief friendly line
(`chat.responses.usage_limit_reached`).

No Python engine changes — the charge/402 happens on the Java tool
endpoint that Java itself calls.

## Test plan

- [x] `:proprietary:compileJava` + `spotlessCheck` clean
- [x] `AiWorkflowServiceTest` + `PolicyEngineTest` green
- [x] eslint, proprietary + saas typechecks clean
- [ ] Manual: drive an agent run over the limit → brief line in chat +
the right modal (free vs cap)

> Note: proprietary test compilation is currently blocked on the
pre-existing `InitialSecuritySetupTest` 6-arg ctor break (unrelated,
tracked separately); verified locally by temporarily patching it.
2026-06-12 11:38:07 +01:00
EthanHealy01 eb2527fc7f Properly sync US and GB translation files (#6635)
add en-US changes to SaaS, previously merged into main. So this is
effectively a main -> SaaS PR also. It seems to be all additive.

Also take the 230 ish missing translations from en-GB over to en-US
using a script, and also make and english spellings American when adding
them to the en-US file, and fix any existing American spellings in the
en-GB file.
2026-06-12 11:18:25 +01:00
James Brunton d363a1e957 Improve search logic (#6637)
# Description of Changes
Search has got significantly worse since #6581, where I added all the
missing tags for tools that should have been there for months. Turns out
that the fuzzy matching search logic has always been way too permissive
to match words with Levenshtein distances way too far away from the
target word, so long searches include way too much stuff. The new tags
just exposed that underlying logic issue. This PR makes the Levenshtein
logic much stricter, so it is still tolerant to minor typos in tool
names, but doesn't match completely inappropriate strings.
2026-06-12 10:34:11 +01:00
James Brunton ea102cdb93 Merge pull request #6636 from Stirling-Tools/SaaS-update
Update SaaS to latest main
2026-06-12 10:19:35 +01:00
James Brunton d995471a55 Merge remote-tracking branch 'origin/main' into SaaS-update
# Conflicts:
#	frontend/editor/src/proprietary/components/chat/ChatContext.tsx
#	frontend/editor/src/saas/components/shared/TrialStatusBanner.tsx
2026-06-12 09:58:40 +01:00
Reece Browne e3e49c07ae Policies: let team leaders configure policies in the UI (#6634)
Frontend follow-up to #6632 (team-scoped policies, editing gated to team
leaders on the backend). Brings the UI's edit gate in line.

## Problem
The policy config UI gated editing to `config.isAdmin`. On SaaS, org
users are never the single global admin, so **no one could open the
policy editor** — the same lockout #6632 fixed on the backend.

## Fix
`usePolicies` now allows a **team leader** to configure, falling back to
a global admin self-hosted:

```ts
canConfigure =
  config != null && (!config.enableLogin || isTeamLeader || config.isAdmin === true);
```

- SaaS → `isTeamLeader` (from `useSaaSTeam()`) — team leaders can
configure; members get the read-only surface.
- Self-hosted → `config.isAdmin` (the core `useSaaSTeam` stub returns
`false`, so admins aren't locked out).
- Login disabled (single-user) → always allowed.
- The `config != null` guard keeps the gate closed until app-config
resolves, so edit controls never flash for users who can't use them.

The two locked-policy banners now read "Contact a team leader to change
this policy" (updated in the `t()` defaults and the `en-GB`
translations).

## Verification
- Typecheck clean (proprietary + saas); eslint clean.
- Tests pass: `usePolicies`, `PoliciesSidebar`.
2026-06-11 23:51:26 +01:00
EthanHealy01 962119e14f UI ux/add ai warning and change style (#6633)
<img width="394" height="426" alt="Screenshot 2026-06-11 at 11 39 50 PM"
src="https://github.com/user-attachments/assets/15805931-73fd-416b-841b-99a556468433"
/>
bottom text input is sticky even when shrunk down
2026-06-11 23:49:52 +01:00
Reece BrowneandClaude Opus 4.8 cc1235bbf2 i18n(policies): route policy UI strings through i18n (English only) (#6628)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 23:37:43 +01:00
Reece Browne e88d22d2fc Policies: scope to the owning team; editing restricted to team leaders (#6632) 2026-06-11 23:21:48 +01:00
Anthony Stirling ddf10f0aaf Stop advertising mcp.tools scopes in OAuth metadata when scope enforcement is disabled 2026-06-11 23:03:43 +01:00
EthanHealy01 b756b5befb add agent warning and update style (#6629) 2026-06-11 21:55:51 +01:00
ConnorYoh eddc54c6c0 fix(payg): land usage-limit modal CTAs on the Plan section (#6630) 2026-06-11 21:42:59 +01:00
ConnorYoh 22379fd5ab fe(payg): show the usage-limit modal when the limit is hit (direct + policy) (#6626) 2026-06-11 21:28:44 +01:00
Reece Browne 6f1c19c179 Policies: enforce input on uploads only; badge follows edited files (#6627) 2026-06-11 21:27:44 +01:00
Reece BrowneandClaude Opus 4.8 ef65e6b015 feat(policies): org-wide policies with admin-only editing (#6625)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 21:23:06 +01:00
Anthony Stirling 47e5977a31 Drop startup credit-reset catch-up (bulk per-user loop; lazy reset covers it) 2026-06-11 21:21:28 +01:00
Anthony Stirling 3a4b340313 Remove legacy CreditBackfillRunner (PAYG replaces it; rows created lazily) 2026-06-11 21:03:09 +01:00
EthanHealy01 41d2aa8174 UI ux/move footer links to settings (#6606)
<img width="2056" height="1044" alt="Screenshot 2026-06-11 at 2 15
34 PM"
src="https://github.com/user-attachments/assets/e58a9f8f-7172-4f30-ab28-0760b66249c9"
/>
<img width="2056" height="1045" alt="Screenshot 2026-06-11 at 2 15
43 PM"
src="https://github.com/user-attachments/assets/890b7a0b-740f-4c7f-9a48-c9a2c28e8ded"
/>
2026-06-11 20:43:33 +01:00
ConnorYoh ee9fdeed6b fix(payg): run the entitlement guard before the charge interceptor (#6622)
## Problem

When the `EntitlementGuard` refuses a request with **402** (team is over
its free allowance / spending cap, or has no subscription to bill), the
handler never runs — so it must not charge. But it did: the guard (order
**1100**) ran *after* the charge interceptor (**1000**), so
`openProcess` had already written the charge before the 402, and
`afterCompletion` then billed it as "customer paid for the attempt" — a
ledger debit, and a **Stripe meter for a subscribed-over-cap team**.

## Fix

**Run the guard first** (order **900**, before the charge interceptor at
1000). Spring runs interceptors in ascending order on the way in and
**skips a later interceptor's `preHandle` (and `afterCompletion`)
entirely once an earlier one returns `false`** — so a refused request
short-circuits with its 402 *before* the charge interceptor runs at all.
A blocked request never opens a process, materialises inputs, or writes
a charge.

This replaces the earlier attribute-flag + `afterCompletion`-refund
approach with a simpler reorder (per review): the no-charge-on-block
guarantee is now **structural**, and it also avoids the wasted
open-then-refund churn (no temp-file write, no debit/refund pair) for
refused requests.

### Why the reorder is safe
- `EntitlementGuard` reads no `PaygChargeInterceptor` state and has **no
`afterCompletion`** (only `preHandle`), so reverse-order teardown is a
non-issue.
- The legacy `UnifiedCreditInterceptor` (default order **0**, and only
registered under the `legacy-credits` profile) still runs first, so any
legacy rejection wins.
- For *admitted* requests both interceptors still run (guard then
charge) — behaviour is unchanged; only refused requests now
short-circuit before the charge.

## Tests

`PaygWebMvcConfigTest` locks the `ENTITLEMENT_GUARD_ORDER <
INTERCEPTOR_ORDER` invariant (if it's ever reversed, refused requests
would bill again — this fails first). Existing `EntitlementGuardTest`
already proves the guard returns 402 on a degraded/billable request.
`:saas:test` + spotless green.

## Related (separate, in progress)

The **fail-cleanly + fire-the-modal** half (suppress the error toast,
trigger the subscribe/raise-cap modal via the existing
`subscribed`/`category` signal — catching the 402 centrally so direct
API usage is handled, and propagating the entitlement reason through the
async policy-run status) lands **with Ethan's modal** so we don't remove
the toast before there's a popup to replace it.
2026-06-11 20:42:40 +01:00
Anthony Stirling b1a960a240 Skip self-host user-table bootstrap (team backfill, grandfathering) in SaaS 2026-06-11 20:39:48 +01:00
Anthony Stirling 946c032fb5 Change default language to en-US and add US language (#6621) 2026-06-11 20:36:23 +01:00
EthanHealy01 7e493226c4 add popups for free limit hit and spend cap hit (#6623) 2026-06-11 20:34:59 +01:00
Anthony Stirling d48017a5b5 Skip engine free-port probe in containers (fixed port) 2026-06-11 20:18:54 +01:00
ConnorYoh 37b4d24a95 fix(payg): gate + charge AI document tools and AI Create sessions (#6617)
## Problem

Two AI surfaces slipped through PAYG unbilled:

1. **AI document tools** — `/api/v1/ai/tools/**`
(`PdfCommentAgentController`, `MathAuditorAgentController`) live in the
**proprietary** module, which can't depend on `saas` and so can't carry
the saas-only `@RequiresFeature`. They also lacked
`@AutoJobPostMapping`, so the charge interceptor's scope gate
short-circuited them **before** category resolution: **not charged, and
not even entitlement-gated** — whether called directly or dispatched by
the orchestrator.
2. **AI Create** — `/api/v1/ai/create` is JSON/session-based with no
file input, so the multipart charge path never fired. The old
per-generation charge ran through the now-dead legacy credit system, so
it currently charges nothing.

## Fix

- **`AiToolRoutes`** (new, saas) — single source of truth for the
`/api/v1/ai/tools/**` prefix. The proprietary controllers stay
untouched; the saas hot-path recognises them by path:
- **`PaygChargeInterceptor`**: brings these routes into scope and bills
them **AI** on a direct call. An orchestrator-dispatched call still
resolves to **AUTOMATION** first (the `X-Stirling-Automation` header is
checked before the path rule), so AI-tool-inside-a-workflow keeps
billing as automation.
  - **`EntitlementGuard`**: gates them on **`AI_SUPPORT`**.
- This keeps the `proprietary → saas` layering intact (no backwards
dependency).
- **`JobChargeService.chargeStandalone(ctx, units)`** — charges a fixed
unit count for a non-file billable action, reusing the existing
free-grant split + shadow row + ledger debit + `close()`→meter path on a
standalone bookkeeping job (no lineage inputs, so nothing lineage-joins
it). **`JobService.open(ctx, docUnits)`** opens that bare job.
- **`AiCreateController.createSession`** — charges **one document per
session** at creation (best-effort; entitlement is already enforced
upstream by the class-level `@RequiresFeature(AI_SUPPORT)`). Follow-up
edits (`outline` / `reprompt` / `draft` / `template` / `stream`) carry
**no** charge — they have no charge hook, so "charge on create,
follow-ups free" falls out naturally.

Per the agreed scope: charge AI Create on create now; we can optimise
follow-up handling later. **AI workflow categorisation (AUTOMATION vs
AI) intentionally left as-is** (the orchestrator's automation header
dominates by design).

## Tests

- `PaygChargeInterceptorTest`: AI-tool route (no annotations) is in
scope + **AI** category; same route with the automation header →
**AUTOMATION**; a plain non-AI route still short-circuits.
- `EntitlementGuardTest`: AI-tool route is in scope + gated on
**AI_SUPPORT** (degraded team → 402; anonymous → 401 with `category:
AI`).
- `JobChargeServiceTest`: `chargeStandalone` charges + meters the paid
portion for a subscribed team, draws the free grant (no meter) for an
unsubscribed team, and rejects `BYPASSED`.

`:saas:test` + `:saas:spotlessCheck` green; coverage gates met.

## Follow-ups (not in this PR)

- Make AI Create follow-ups explicitly cheaper / chained if we want
(currently free by absence of a hook).
- Decide whether AI-tool-inside-a-workflow should bill as AI rather than
AUTOMATION.
2026-06-11 20:10:44 +01:00
ConnorYoh aaa2599e23 fix(saas): block accepting an invite when it would orphan a paid team (#6616)
## Problem

A **free team can invite a paid team's leader** to join. The leader
accepts, and:
- `acceptInvitation` moves them onto the inviting team and removes their
membership from the old team, but only deletes the old team if it's
**personal**.
- Their paid (non-personal) team is left **memberless but still
subscribed** — an orphaned Stripe subscription billing for a team nobody
is in.

## Root cause

Two gaps in `SaasTeamService.acceptInvitation`:

1. The pre-accept guard checks `hasPaidSubscription(acceptingUser)` →
`existsActivePaidSubscriptionForUser(supabaseId)`, keyed on
**`user_id`**. A team's plan is keyed on **`team_id`**
(`existsActiveSubscriptionForTeam`), so a paid team's *leader* isn't
caught and accepts freely.
2. The 'leave existing teams' loop deletes the membership directly and
only marks **personal** teams for deletion — it bypasses the last-leader
protection `leaveTeam` already enforces (`"Cannot leave as the last team
leader. Transfer leadership first."`), and never cleans up / cancels the
non-personal team.

There is no in-app subscription-cancel path (cancellation is
Stripe-portal/webhook driven), so nothing reconciles the orphan after
the fact — it has to be prevented.

## Fix

Add `assertCanLeaveCurrentTeamsToJoinAnother(user)`, called in
`acceptInvitation` before any membership changes. For each non-personal
team where the user is the **last leader**, the accept is rejected:
- team has an active subscription → *"Cancel the plan or transfer
leadership before joining another team."*
- otherwise → *"Transfer leadership before joining another team."*

This mirrors the protection `leaveTeam` already has and is
team/leadership-aware, closing the `user_id`-vs-`team_id` gap. Regular
members and teams with another leader are unaffected.

## Verification

- `ENABLE_SAAS=true ./gradlew :saas:compileJava` — passes.
- Manual: with a paid team leader, accepting an invite to another team
should now be rejected with the message above; verify a non-leader
member can still accept.

## Note

This prevents *new* orphans. Any teams already orphaned by this bug
(memberless, still subscribed) would need a one-off reconciliation —
happy to follow up with a query/cleanup if useful.
2026-06-11 20:10:21 +01:00
EthanHealy01 33026e1a82 update saas onboarding (#6619)
<img width="1002" height="487" alt="Screenshot 2026-06-11 at 6 20 10 PM"
src="https://github.com/user-attachments/assets/5ee3cfc2-6c4f-4b35-9586-ef45fa216c6a"
/>
2026-06-11 19:39:02 +01:00
Anthony Stirling 1d598d5caa MCP OAuth discovery fix + Supabase consent page (#6608) 2026-06-11 18:30:49 +01:00
ConnorYoh 9e5fe2f4ca fix(payg): attribute policy runs to the owner so usage is charged (#6620)
## Problem

A policy ran over a file but the owner's **free usage was never
consumed**.

A policy run executes on a **background virtual thread**
(`PolicyEngine.submit` → `asyncExecutor`), and Spring's
`SecurityContextHolder` is thread-local — so the worker thread has no
identity. When `PolicyExecutor` → `InternalApiClient.post` resolves the
tool-call API key via `UserService.getCurrentUsername()`, it finds
nothing and falls back to the **`INTERNAL_API_USER`** key. The loopback
tool calls then authenticate as that system account, so
`PaygChargeInterceptor` attributes the charge to *its* team (or none) —
the real owner's free grant is untouched. Folder-watch / scheduled
triggers are even further removed (fired from a background watch loop
with no request context at all).

The charging *mechanism* was fine (AUTOMATION, multipart,
`openProcess`); only the **attribution** was wrong.

## Fix

Propagate the acting identity onto the worker thread using the
**audit-principal MDC key** that `UserService.getCurrentUsername()`
already reads as its documented async fallback (the same mechanism used
for other async jobs). No new plumbing through the executor.

- **`runPolicy`** (stored policies — covers triggers *and* manual
`runWith`) → bill the **policy owner**. `Policy.owner` is the username
stamped at creation, so `getApiKeyForUser(owner)` resolves it.
- **`submit`** (ad-hoc Automate/AI one-offs) → bill the **submitting
user**, captured on the request thread (it doesn't survive the hop to
the worker otherwise).

With the principal set, `InternalApiClient` dispatches each tool call as
that user → the interceptor resolves the right team → free grant draws /
Stripe meters correctly.

## Tests

`PolicyEngineTest`:
- `runPolicyDispatchesToolCallsAsTheOwner` — asserts MDC
`auditPrincipal` == the policy owner at the moment
`InternalApiClient.post` is invoked.
- `adHocRunDispatchesToolCallsAsTheSubmittingUser` — asserts it's the
submitting user for an ad-hoc run.

`:proprietary:test` + `:saas:test` + spotless green; coverage gates met.

## Heads-up (not in this PR)

Once attributed, **automatic folder-watch / scheduled runs consume free
grant (or bill) per file** — set up once, runs forever. That's
automation-is-billable working as intended, but a set-and-forget policy
can drain an allowance fast, so it may warrant a per-policy cap or a
heads-up in the UI. Flagging for a product decision.
2026-06-11 18:28:58 +01:00
Reece Browne 9ee0bc4b32 Policies: enforce on upload or export (#6614)
Follow-up to #6604 (merged). Builds the Security policy out so it
actually enforces, driven from the editor.

## What it does
- **Run on upload or export** — a single choice in the wizard: enforce
when a file is uploaded, or just before it's exported.
- **Output** — enforced result is a **new version** of the file
(default) or a **new file**, with optional filename
prefix/suffix/auto-number ("Output filename" subsection; auto-number
only for new files).
- **Export enforcement** — exporting an export-mode file runs the policy
first and downloads the enforced result; never hard-blocks (on failure
the original downloads). For new-version policies the in-editor file is
versioned too. Covers every export path incl. multi-file ZIP. A toast
(glowing in the policy's accent while it runs) reports progress and
fades after ~10s.
- **Affordances** — a freshly enforced file briefly glows its policy
accent and carries a shield badge.
- **Config tidy-up** — removed the unwired Security setting fields + the
wizard's review step; "Upgrade to enterprise" on locked categories;
category accent in the detail/wizard headers.

## Notes
- Builds on the manual-only (client-driven) policy model from #6587
(`trigger: null`, metadata in `output.options`), adding the `runOn`
field + export-time enforcement.
- The page-editor merge-export (no single source file) enforces +
downloads but doesn't version in place.

## Verification
typecheck (core + proprietary), eslint, prettier; proprietary suite
(105) green; flows checked in-app.
2026-06-11 18:12:01 +01:00
ConnorYoh 5fa5e12c64 fix(saas): show team invitation banner in SaaS web build (#6612)
## Problem

When a user is invited to a team, the SaaS web app shows **no invitation
banner** — even though the pending invite is returned by
`/api/v1/team/invitations/pending` on refresh.

## Root causes

1. **Never rendered in SaaS.** `TeamInvitationBanner` only existed in
`desktop/`, wired solely into `DesktopBannerInitializer`. The SaaS
banner stack rendered only `<UpgradeBanner />`.
2. **Single banner slot.** `BannerContext` holds one node; `setBanner`
replaces it. `TrialStatusBanner` called `setBanner(null)` when there was
no active trial (and re-fired once `trialStatus` resolved async), wiping
any other banner.
3. **Shadowing was too fragile.** A first attempt shadowed the
proprietary `UpgradeBannerInitializer` from the saas layer, but
`vite-tsconfig-paths` resolves the `@app` specifier once at dev-server
start — a newly-added shadow of an already-resolved module isn't picked
up on a browser refresh, only a full restart. So the proprietary
initializer kept running and no invite banner appeared (while the SaaS
team context still fetched + populated the invite, which is why the
pending call was visible).

## Fix

- Add `saas/components/shared/TeamInvitationBanner.tsx` — ported from
desktop, minus the desktop `connectionMode` gate and explicit billing
refresh (SaaS `acceptInvitation` already refreshes credits + session).
- Render it **inline in `saas/routes/Landing.tsx`** next to
`GuestUserBanner` — a new import specifier in an existing file
(HMR-friendly), unambiguously inside `SaaSTeamProvider`, mirroring the
proven `GuestUserBanner` pattern. No dependency on the single banner
slot.
- **Remove `TrialStatusBanner`** (trials are being retired) so it can't
clobber banners. Also drops the stale mention from the stripe-lazy-load
test comment.

## Verification

- `tsc --noEmit -p tsconfig.saas.vite.json`: clean in touched files;
total unchanged from baseline (37 pre-existing, unrelated).
- Manual: pull + verify the Accept/Decline banner appears for an account
with a pending invite.
2026-06-11 18:01:58 +01:00
James Brunton 34ead60194 Kill off agents pane now that we have FAB (#6613)
# Description of Changes
Kill off agents pane now that we have the FAB. Also fixes a bug with the
FAB where it would sometimes fail to render the chat, and fixes a
duplicated entry in the Vite config which was throwing a warning
2026-06-11 17:28:05 +01:00
ConnorYoh 5bc7ae626d fix(payg): cancelled subscription left team gated as subscribed (#6611)
## Problem

A team that **cancelled** its PAYG subscription kept full subscribed
access:

- **UI didn't reflect cancellation** — the Plan tab still rendered the
subscribed view, never the free/upgrade view.
- **Automation wasn't stopped** — automation / AI / API kept running
without ever falling back to the free-grant gate.

## Root cause

`TeamBillingService.compute` decided `subscribed` as:

```java
boolean subscribed =
    subscriptionId != null
        || extOpt.map(PaygTeamExtensions::getStripeCustomerId).filter(s -> !s.isBlank()).isPresent();
```

On cancellation, the `customer.subscription.deleted` webhook calls
`payg_unlink_subscription`, which nulls `payg_subscription_id` but
**deliberately keeps `stripe_customer_id`** (so a future re-subscribe
can reuse the Stripe customer).

`payg_link_subscription` is the **only** writer of
`payg_team_extensions.stripe_customer_id`, and it writes it in the
*same* `UPDATE` as `payg_subscription_id` (on
`customer.subscription.created`). So the customer id is never set before
the subscription id — the "pre-webhook stand-in" the old comment claimed
**cannot happen**. The fallback only ever pinned a team that *ever*
subscribed to `subscribed` forever, because the Stripe customer outlives
the subscription.

Both symptoms are this one flag:
- `PaygWalletController` status → `SUBSCRIBED` vs `FREE`
- `EntitlementService` gate branch → monthly-cap vs free-grant

## Fix

Gate `subscribed` purely on `payg_subscription_id != null`. A cancelled
team now correctly drops to free (UI shows free; billable ops gate on
the one-time grant). This aligns the wallet/entitlement read with the
**meter path** (`JobChargeService.close`), which already gated on
`payg_subscription_id`.

Handles both Stripe cancel modes: "cancel at period end" keeps the sub
`active` (id stays set) until `.deleted` fires at period end → access
through the paid period; immediate cancel fires `.deleted` now → flips
to free now.

**No data migration / backfill** — already-cancelled teams have
`payg_subscription_id = NULL`, so they flip to free as soon as this
ships (within the 30s billing-cache TTL).

## Tests

Adds `TeamBillingServiceTest` — the `subscribed` computation previously
had **no** unit coverage (which is how this shipped). Covers: subscribed
iff subscription id present; **cancelled team (customer id remains,
subscription id null) ≠ subscribed** + free grant survives;
no-subscription/no-customer; no extension row.

`:saas:test` + `:saas:spotlessCheck` green; coverage gates met.

## Not included (optional hardening, can fast-follow)

- Cross-check the synced `stripe.subscriptions.status` to guard a
*missed* `.deleted` webhook leaving `payg_subscription_id` stale.
- Push cache-invalidation from the webhook (currently ≤30s TTL
staleness).
2026-06-11 17:26:58 +01:00
ConnorYoh f16ca4795c fe(payg): remove em dashes from Plan page copy (#6610)
## What

Removes all em dash (`—`) characters from the **user-facing text** on
the Plan page (PAYG section), replacing them with colons, commas, or
restructured punctuation so the copy reads naturally.

## Changes

- `frontend/editor/public/locales/en-GB/translation.toml` — all `payg.*`
strings (this is what actually renders on the page)
- `PaygFree.tsx` — `t()` default fallbacks + the `{" — "}` JSX
benefit-list separators (now `{": "}`)
- `Payg.tsx` — `t()` default fallback for the editor-plan body

## Notes

- The en-dash range separator (`{{start}} – {{end}}`) in the
billing-period string is intentionally **kept** — only em dashes were
targeted.
- JSDoc / code comments containing em dashes were **left unchanged**,
since they aren't rendered text on the page.
<img width="990" height="502" alt="image"
src="https://github.com/user-attachments/assets/13d89b0f-007c-4b4c-b72d-1d912f968bc7"
/>
2026-06-11 16:50:27 +01:00
James Brunton d52c7ced7c Improvements to Stirling Engine to prepare for SaaS release (#6603)
# Description of Changes
- Use pool for postgres connections
- Add ability to require user ID to be set on API calls to the engine
- Add process-wide concurrency cap on AI access (in addition to existing
user caps)
- Allow number of workers (threads) to be specified for stirling engine
- Update env var names to reflect that the DB is not just for RAG
2026-06-11 16:31:35 +01:00
James Brunton 606964ee52 Fix Teams and MCP settings pages (#6605)
# Description of Changes
Remove the Pro guards from the Team settings page and also fix the
styling of the MCP settings screen (the code sections were black text on
black background in light mode)
2026-06-11 16:26:02 +01:00
ConnorYohandReece cf513c255b PAYG: pay-as-you-go billing — metered automation/AI/API + one-time free grant (#6589)
## Summary

Pay-as-you-go (PAYG) billing for Stirling-PDF SaaS. Manual PDF editing
stays free forever; only **automation, AI, and API** usage is metered.
Every team gets a **one-time lifetime free grant** (default 500 PDFs)
before any billing; past that, a team adds a card and pays per metered
document, with a self-set monthly spending cap.

This branch combines and supersedes the in-flight BE (#6574) and FE
(#6579) work plus the SaaS edge functions (Stirling-PDF-SaaS PR, now on
`v3`), hardened into a single reviewable feature after a pre-merge
dead-code/security review.

## Billing model

- **Always free:** manual / JWT web-tool usage is `BYPASSED` — never
metered, no matter where it's triggered.
- **Billable categories:** `AUTOMATION`, `AI`, `API`.
- **One-time lifetime free grant** (`pricing_policy.free_tier_units`,
default 500): never resets, survives subscribing. It gates unsubscribed
teams (billable API calls hard-stop with a 402 once exhausted) and
decides the free-vs-paid split of every job.
- **Subscribed:** paid documents (beyond the grant) are metered to a
Stripe Billing Meter; an optional monthly spending cap degrades billable
categories when reached.
- **Dedup:** the same file pushed through several steps within a
workflow window counts **once** (lineage join), so API/AI chaining on
one file isn't double-charged.

## What's included

**Database** — Flyway migrations `V11`→`V21` with matching Supabase
twins: pricing policy + per-team sidecar (`payg_team_extensions`:
subscription id, Stripe customer, free-grant counter), append-only
`wallet_ledger`, shadow charges, subscription-state RPCs (`V14`), audit
logs (`V15`), billing category (`V16`), one-time lifetime free grant
(`V19`), launch-grant seed (`V20`), drop of the unused
`wallet_category_summary` view (`V21`).

**Charge pipeline** — `PaygChargeInterceptor` (open/join a process,
split the free grant, write the ledger DEBIT), `JobChargeService`
(consume the grant under a row lock, restore it on a first-step refund,
meter only the paid portion on completion), `StaleJobCloser` fallback
(idempotent close → meter).

**Entitlement** — `EntitlementService` (per-team cached snapshot:
grant-gated for free teams, monthly-cap-gated for subscribed) +
`EntitlementGuard` (401 `SIGNUP_REQUIRED` / 402 `FEATURE_DEGRADED` /
`PAYG_LIMIT_REACHED`).

**Metering** — `PaygMeterReportingService` writes a durable
`payg_meter_event_log` row around every POST to the `meter-payg-units`
edge fn (pending → posted/failed); `PaygMeterReconcileScheduler` retries
unposted events under the same idempotency key inside Stripe's 24h dedup
window.

**Billing facts** — `TeamBillingService` reads the synced `stripe.*`
mirror (subscription window, per-document rate; the unsubscribed-team
estimate resolves the rate by Price `lookup_key = plan:processor`).

**Wallet API** — `PaygWalletController`: `GET /api/v1/payg/wallet`,
`PATCH /api/v1/payg/cap`.

**Frontend** — PAYG Plan page (two-card free layout + subscribed views),
`useWallet`, upgrade modal with lazy-loaded Stripe Embedded Checkout and
a shared `SpendCapControl`, customer-portal link, 402/401 interceptor
toast, en-GB i18n. (Per-member usage shows each teammate's spend; the
activity feed is behind a flag until polished.)

**SaaS edge functions** (`Stirling-PDF-SaaS` `v3`) —
`create-checkout-session`, `create-payg-team-subscription`,
`create-customer-portal-session`, `meter-payg-units`,
`payg-subscription-webhook`, `stripe-sync`, plus the stripe-sync
`migrate` + scoped-`backfill` scripts. All price lookup is DB-driven (no
`STRIPE_PAYG_PRICE_ID_*` env vars).

## Release prerequisites (prod)

1. Apply Flyway migrations (`V11`→`V21`) and the Supabase migration
twins.
2. Stripe Sync Engine: run `stripe-sync:migrate`, then a **scoped**
backfill — `product`, `price`, `customer`, `subscription` only (not
`all`, which rate-limits).
3. Register 2 PAYG webhook endpoints (each its own signing secret):
`stripe-sync` (product/price/customer/subscription `.*`) and
`payg-subscription-webhook` (`customer.subscription.created`/`.deleted`
drive state; `.updated` + `invoice.*` observed). Keep the legacy
`stripe-webhook` only if credits/self-hosted flows still run.
4. Stripe Billing Meter: `event_name = payg_doc_units`, value key
`processed_documents`.
5. Env: `PAYG_METER_ENDPOINT` + `SUPABASE_EDGE_FUNCTION_SECRET`
(backend); the webhook signing secrets (edge fns). The default pricing
policy must point at the PAYG Stripe Price(s); `V20` seeds
`free_tier_units = 500`.

## Testing

- `:saas:test` green, `:saas:spotlessCheck` clean, edge-fn Deno tests
green, FE saas typecheck clean (the remaining errors are pre-existing
`proprietary/*` + `prototypes/*`, untouched here). Cucumber shadow-mode
suite + CI workflow included.

## Pre-merge review

An independent dead-code/security pass came back **clean on security**
(team-derived authz / no IDOR, leader-only cap mutation, no
billing-category downgrade, dev/mock hooks gated to
`import.meta.env.DEV` + `/dev/`, no secrets/injection, fail-open
metering by design). The dead/unwired code it flagged has been removed
in this branch (unenforced sub-cap control, an unused JDBC DAO + its
view, dead methods).

## Follow-ups (tracked, not blocking)

- **Enforce per-member sub-caps** — the control was removed because it
read for display but never gated a request; the per-member usage display
and `cap_units` column are retained for when enforcement is wired.
- **API/AI chaining billing model + `ProcessType` enum** — confirm
same-file dedup covers API chaining; define per-tool AI charging; decide
whether the unused enum values stay.
- **Activity feed** — hidden behind a flag until the meter-event surface
is polished.

---------

Co-authored-by: Reece <reece@stirlingpdf.com>
2026-06-11 15:56:01 +01:00
EthanHealy01andAnthony Stirling 88adb7adad create agent (#6520)
Added the create agent. Use [these
prompts](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/blob/main/docgen/backend/default_templates/sample_prompts.md)
to test or try your own :)

Here’s the one I use

```
Hey, I need to generate an employee expense report for reimbursement.
Company: Summit Consulting Partners Company address: 88 Riverside Plaza, Suite 1400, New York, NY 10069 Accounting department email: expenses@example.com
Employee details:
* Employee Name: Michael Tran
* Employee ID: EMP-1047
* Department: Client Services
* Report Date: January 20th, 2026
* Reporting Period: January 5th, 2026 – January 16th, 2026
* Manager Approver: Laura Simmons
Trip purpose: Client onsite meetings with Atlantic Energy Solutions in Boston, MA.
Expense items:
* Flight (NYC to Boston roundtrip) — $325.40 — January 5th, 2026 — Airline ticket
* Hotel (3 nights at Harborview Hotel) — $822.75 — January 5th-8th, 2026
* Taxi from airport to hotel — $48.00 — January 5th, 2026
* Client dinner (3 attendees) — $186.20 — January 6th, 2026
* Parking at JFK Airport — $72.00 — January 5th-8th, 2026
* Breakfast (per diem not used) — $18.50 — January 7th, 2026
* Uber to client office — $22.10 — January 7th, 2026
* Printing + presentation materials — $46.90 — January 8th, 2026
* Lunch with client — $39.75 — January 8th, 2026
* Office supplies (notebooks, pens) — $27.60 — January 10th, 2026
* Mileage reimbursement (client visit in NJ, 42 miles @ $0.67/mile) — $28.14 — January 14th, 2026
* Team lunch meeting (internal) — $64.30 — January 15th, 2026
Reimbursement method should be direct deposit.
Add a notes section stating: "All receipts attached. Expenses are business-related and comply with company travel policy."
```

---------

Co-authored-by: Anthony Stirling <77850077+frooodle@users.noreply.github.com>
2026-06-11 14:18:13 +00:00
Reece Browne 11ab762f57 feat(policies): config refinements + new-version output (post-#6598) (#6604)
Follow-up to #6598 (squash-merged into `SaaS`). These are the policy
refinements made after that merge, against the current `SaaS` tip.

## Changes
- **Simplify Security config + plain-language info buttons** — Redact
config reduced to the PII field; Sanitise has no config
(JavaScript-removal only) with a non-technical info button; per-tool
info buttons reworded to match the tool-steps style.
- **Hide 'Flatten PDF pages to images' from the watermark policy
config** — new `PolicyWatermarkConfig` wrapping the watermark settings
with the flatten checkbox gated off.
- **Flatten-to-image on by default for redact + watermark** — both
normalise `convertPDFToImage: true` on mount.
- **Self-heal a stale backing folder** — `ensurePolicyFolder` recreates
a backing folder whose `folderId` no longer resolves (preferring the
backend's stored automation), instead of hanging Edit Settings on a
permanent "Loading…".
- **Version the input file on 'new version' output mode** — completed
runs whose policy output mode is `new_version` replace the input file
with a versioned child (origin tool `automate`) rather than adding a
separate file; falls back to a new file if the input is gone.
`outputMode` is plumbed through `PolicyState`, the local-cache default,
and backend reconciliation.

## Verification
- `typecheck:proprietary` + `typecheck:core` clean
- policy + hooks vitest: 17 passing
- eslint + prettier clean on all changed files
2026-06-11 14:45:22 +01:00
James Brunton 68e031ac55 Policies tidying (#6587)
# Description of Changes
* Improve typing of API (breaking change but unreleased, frontend also
updated in this PR)
* Add ownership concept to policies
* De-AI the comments
* Update the `task dev:saas` rule to spawn the engine as well
2026-06-11 13:20:01 +01:00
Anthony Stirling c722b9f6ad fix: MCP copy buttons read as proper buttons in dark mode
Subtle/gray compact buttons rendered as low-contrast floating text;
use the default variant (adaptive surface+border) idle, light teal when
copied.
2026-06-10 17:26:43 +01:00
Anthony Stirling 36c68fb69e fix: doubled base path in mobile-scanner QR URL
A configured frontendUrl/server_url already includes the subpath (e.g.
/bpp), but the code also applied withBasePath, producing /bpp/bpp/...
Append the route directly to a configured URL; reserve withBasePath for
the bare-origin fallback. Matches the ShareFileModal convention.
2026-06-10 17:21:42 +01:00
Anthony Stirling d3c359f923 reword MCP usage tip to reference the API and Automation 2026-06-10 16:38:16 +01:00
Anthony Stirling 4947ab12fd remove the 'What your assistant can do' tool-category badges from MCP section 2026-06-10 16:33:48 +01:00
Reece Browne 8dde4262ec feat(policies): backend-driven policy enforcement (frontend) (#6598)
## Summary
Adds the **Policies** feature (proprietary, behind the
`POLICIES_ENABLED` flag): backend-driven enforcement that runs a fixed
tool pipeline on documents, docked in the right tool sidebar alongside
Tools.

## Highlights
- **Policy catalog** — 5 categories; **Security** is wired (redact PII +
sanitize), the others are marked "Coming soon".
- **Backend as source of truth** — policies persist via the Policies
engine (`/api/v1/policies`), one policy per category, with a local cache
+ offline fallback.
- **Auto-run** — enabled policies run on every uploaded file: dispatch →
poll → import outputs into the workspace.
- **Security redact config** — PII preset dropdown + custom word/regex
entry + advanced options; tool params map to the backend endpoint
fields.
- **Activity feed** with retry on failures; **file badges** showing
which policies ran on a file (sidebar + files page), tinted to the
policy colour.
- Reuses the **Watched Folders** engine for each policy's backing
folder; policy-owned folders are filtered out of the Watched Folders UI.

## Notes
- Gated by `POLICIES_ENABLED` (true in proprietary, false in core) —
unreachable in the open-source build.
- Frontend-only diff; depends on the backend Policies engine and the
merged Watched Folders feature.
2026-06-10 15:57:08 +01:00
James Brunton ebc28b0a14 Add team settings to SaaS (#6601)
# Description of Changes
Add team settings UI to SaaS, which is currently only available in
desktop. It'd be nice to refactor this so they're more shared, but
they're slightly different so needs to be done with some care. Leaving
for followup work.
2026-06-10 15:54:18 +01:00
Anthony Stirling 56862cc1d3 Merge branch 'main' into SaaS 2026-06-10 15:51:43 +01:00
EthanHealy01 9b877d4f8d Move agent section to fab (#6597) 2026-06-10 15:47:47 +01:00
Reece Browne da4b84962c drop type-aware ESLint to stop the lint OOM (#6602) 2026-06-10 15:44:52 +01:00
Anthony Stirling d6306f51e1 fix: no blue disc behind the sidebar profile picture
Keep the colored background only for the initials fallback; a real
photo fills the circle with a transparent backing.
2026-06-10 15:05:29 +01:00
Anthony Stirling 9a1804ce04 Merge branch 'main' into SaaS 2026-06-10 14:58:44 +01:00
Anthony Stirling 611468b972 Add SaaS MCP usage tab (#6590) 2026-06-10 14:58:33 +01:00
EthanHealy01 5fca2f199a Feature/pdf ingestion jpdfium (#6525) 2026-06-10 14:51:41 +01:00
Anthony Stirling be0db3fd8a fix: show profile picture in the FileSidebar bottom bar
The home page's bottom-left settings button is FileSidebar's bottom
bar, which hardcoded an initials circle - the avatar work in
useConfigButtonIcon only affects the QuickAccessBar rail, which the
home page doesn't render. Add a layered useProfilePictureUrl hook
(core stub returns null; saas returns the auth context URL) and render
the picture inside the existing avatar circle, falling back to the
initial when absent or on image load failure.
2026-06-10 14:49:16 +01:00
EthanHealy01 2aa6768921 show chat progress and other UX improvements (#6576) 2026-06-10 14:47:57 +01:00
Anthony Stirling 90bda6b4b4 fix: User principal was discarded by the resource server re-authentication
The saas chain authenticates bearer requests twice:
SupabaseAuthenticationFilter builds an EnhancedJwtAuthenticationToken
with the resolved User principal, but BearerTokenAuthenticationFilter
(oauth2ResourceServer) then re-authenticates the same token through the
static toAuthentication converter and overwrites the SecurityContext
with a token whose principal is the raw Jwt - so storage endpoints kept
returning 401 "Unsupported user principal" despite the principal fix.

Carry the User across in the converter: when the context already holds
an EnhancedJwtAuthenticationToken for the same subject with a User
principal, attach that User to the converter-built token. No extra DB
lookups; anonymous sessions and API-key auth unchanged. Covered by new
unit tests (carry, no-context, subject mismatch).
2026-06-10 14:37:12 +01:00
Anthony Stirling 5b412c0fed cleanup: trim oversized comments across recent SaaS fixes
Reduce multi-paragraph comment blocks to short two-line notes and drop
history-style references; no behaviour changes.
2026-06-10 14:30:29 +01:00
Anthony Stirling bf18af4708 fix: show user avatar on the home page settings button
The bottom-left settings button and the settings page both read
profilePictureUrl, but only the settings page had a fallback (initials
avatar) - the button silently fell back to a gear. The URL itself was
usually null because fetchProfilePicture raced the background OAuth
avatar sync with a fixed 500ms delay and never retried, and a missing
bucket object simply resolved to null.

- useConfigButtonIcon: fall back to the same initials avatar as the
  settings page instead of the gear when no picture URL is available.
- UseSession: fetch the profile picture when syncOAuthAvatar settles
  (init and SIGNED_IN) instead of after an arbitrary 500ms.
- fetchProfilePicture: when the bucket copy is missing, fall back to
  the OAuth provider's own photo URL so the picture shows immediately
  on first login - unless the user explicitly uploaded/removed a
  picture (metadata source 'upload'), preserving the remove flow.
2026-06-10 14:25:19 +01:00
EthanHealy01andAnthony Stirling f15e405759 changes to the login and signup, similar to in the saas repo (#6577)
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-06-10 13:49:27 +01:00
Anthony StirlingandClaude Opus 4.8 d29059e6fb fix: storage APIs 401'd valid Supabase sessions (principal type mismatch)
FileStorageService.requireAuthenticatedUser and
FolderService.requireAuthenticatedUser authorize via
'principal instanceof User', but EnhancedJwtAuthenticationToken extends
JwtAuthenticationToken whose principal is the decoded Jwt - so every
/api/v1/storage/* request 401'd for JWT users AFTER Spring Security had
already authenticated them. This persistent 401-with-valid-session was
the trigger feeding the frontend login loop.

Attach the filter-resolved local User as the token principal for full
accounts (User implements UserDetails, matching the form-login
convention every shared instanceof check expects). Anonymous sessions
keep the raw Jwt principal, preserving their existing exclusions. All
other principal consumers verified safe: AuthenticationUtils checks
instanceof User first, extractSupabaseId/CreditController/Team
SecurityExpressions switch on the authentication type, not the
principal.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 13:44:22 +01:00
Anthony StirlingandClaude Opus 4.8 e7bbbb4702 fix: make the 401 login redirect loop structurally impossible
Audit of every code path that can produce the login->/->login cycle
found the observed loop was one instance of a repeatable class: any
automatic API call that persistently 401s while the Supabase session is
valid triggers httpErrorHandler's hard redirect to /login, which sees
the valid session and bounces back. Close the class, not just the
instance:

- saas apiClient: a 401 that survives a refresh-and-retry means the
  backend rejected a valid token (authz bug / wrong origin), not an
  expired session - never redirect to /login for it. Also fix the stale
  publicEndpoints entry ('endpoints-enabled' matched nothing; the real
  routes are endpoints-availability and endpoint-enabled).
- httpErrorHandler: sessionStorage loop breaker - if a 401 redirect
  already fired within 10s, suppress the repeat instead of cycling.
- Guard the remaining unflagged automatic callers: /api/v1/credits
  (fires on session init and TOKEN_REFRESHED), endpoints-availability
  (fires on app load), and ui-data/login (auto-called when a stale
  stirling_jwt is present).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 13:44:21 +01:00
Anthony StirlingandClaude Opus 4.8 247ef6313c fix: stop login loop caused by unguarded folder-sync 401
The deployed app looped /login -> / -> /login forever: Login sees a
valid Supabase session and navigates to /, the global FolderProvider
pulls GET /api/v1/storage/folders, the backend rejects it with 401, and
the global error handler hard-redirects back to /login?from=/bpp.

fileSyncService's /api/v1/storage/files pull already opts out via
suppressErrorToast + skipAuthRedirect, so its 401 fails silently;
folderSyncService.list() passed neither flag, so its 401 fell through to
the redirect. Add the same flags - FolderContext.pullFromServer already
handles 4xx locally (flips serverReachable, suppresses the banner).

Note: the underlying 401 on /api/v1/storage/* with a valid session is a
backend/deployment issue (storage endpoints rejecting the Supabase
token); this change makes the frontend resilient so it degrades to
"folder sync unavailable" instead of an auth loop.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 12:13:20 +01:00
Anthony StirlingandClaude Opus 4.8 7f7c865888 fix: stop unauthenticated storage calls on /login + fix subpath manifest 404
Two issues seen on the hosted /bpp login screen:

1. GET /api/v1/storage/folders fired (and 401'd) on the login page. The
   global FolderProvider pulls from the server whenever
   appConfig.storageEnabled is true, with no auth gate, so it hits the
   authenticated storage API before the user has signed in. Skip the pull
   on auth routes (/login, /signup, /auth/*, /invite, /reset-password),
   mirroring the existing LicenseContext / AppConfigContext guards. Tests
   wrap FolderProvider in MemoryRouter (now uses useLocation).

2. manifest.json and modern-logo/favicon.ico 404'd from the domain root
   instead of /bpp/. vite base for RUN_SUBPATH deploys was "/bpp" with no
   trailing slash, so <base href="/bpp"> made the browser resolve relative
   links against the parent (root). Use "/bpp/"; getBasePath() strips the
   trailing slash, so BASE_PATH, routing and asset URLs are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 11:56:48 +01:00
James Brunton 3675db5907 Add new triggers, sources and sinks to policies (#6543)
# Description of Changes
Add new triggers:
- Schedule (fires every X amount of time)
- Folder watch (fires whenever the OS tells us a folder has a new file
in it; on Mac this is technically a 2s schedule but that's just how Java
implements it)

Add new sources:
- Folder (reads from this directory)

Add new sinks:
- Inline (stores in FileStorage)
- Folder (stores in specified directory)

Still want to do S3 buckets and web hooks and stuff, but they can come
in a future PR. I'm hoping this should make it sufficient to be able to
integrate with processing folders frontend etc. I've also changed it so
that policies can have multiple sources and triggers at once, which
seems like it might be useful.
2026-06-10 10:55:29 +00:00
Anthony Stirling 2101b4028c Merge branch 'main' into SaaS 2026-06-10 11:42:25 +01:00
Anthony StirlingandClaude Opus 4.8 06476ea69e fix(saas): collapse duplicate Supabase client to one GoTrueClient
The console warned "Multiple GoTrueClient instances detected in the same
browser context" and storage endpoints (/api/v1/storage/folders,
/files) kept 401ing even after a successful token refresh.

Cause: the SaaS bundle instantiated TWO Supabase clients on the same
sb-<ref>-auth-token storage key. :saas/auth/supabase.ts creates the
primary client (used by UseSession + apiClient), while billing /
licensing / user-management code imports @app/services/supabaseClient,
which fell through to :proprietary/services/supabaseClient.ts and called
createClient() again. Each client runs its own autoRefreshToken timer,
so they rotate the refresh token out from under each other → "Already
Used" refresh failures and spurious 401s, plus a residual /login flash.

Add a :saas override of @app/services/supabaseClient that re-exports the
single instance from @app/auth/supabase. The path mapping
(@app/* → src/saas/* → src/proprietary/* → src/core/*) now resolves
every consumer to the same client, so the :proprietary createClient() is
never bundled in the SaaS build.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 11:26:27 +01:00
Reece Browneandaikido-pr-checks[bot] e2536daeb8 Feature/v2/smartfolders rebuild (#6480)
Co-authored-by: aikido-pr-checks[bot] <169896070+aikido-pr-checks[bot]@users.noreply.github.com>
2026-06-10 11:18:14 +01:00
Anthony StirlingandClaude Opus 4.8 1135bd9b63 fix(saas): stop login→logout→login bounce on cold load
On returning to the app with an expired Supabase access token, bootstrap
requests fired with the stale token and 401'd before Supabase finished
refreshing. The global 401 handler then hard-redirected to
/login?from=… (a full window.location navigation), and once the refresh
landed the app sent the user straight back in — the login/logout/login
flicker.

Two holes in the SaaS apiClient response interceptor caused it:

1. "public" endpoints (e.g. /api/v1/config/app-config) skipped the
   refresh-and-retry path. The backend 401s any expired Bearer token
   regardless of route, so those bootstrap calls 401'd and fell through
   to handleHttpError, which redirected to /login. Now public endpoints
   also refresh-and-retry, and a 401 on a public endpoint sets
   skipAuthRedirect so it can never trigger the global login redirect.

2. Concurrent 401s each called supabase.auth.refreshSession()
   independently. Supabase rotates the refresh token on first use, so
   the racing refreshes failed with "Invalid Refresh Token: Already
   Used" and bounced the app even though the session was recoverable.
   Refreshes are now de-duplicated through a single in-flight promise.

Existing apiClient unit tests (refresh-and-retry on protected 401, bare
/login redirect on genuine refresh failure) are preserved.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 11:13:28 +01:00
Anthony Stirling bbfe29c2ef Merge remote-tracking branch 'origin/main' into SaaS
# Conflicts:
#	frontend/editor/src/core/components/shared/AppConfigModal.tsx
2026-06-10 10:51:06 +01:00
James Brunton e0fc5061de Delete dead translations (#6581)
# Description of Changes
Adds all the missing translations that I could find (they're all dynamic
ones that the existing test can't detect are required) and adds a new
test to find as many unused translations as possible. The test has an
ignore list for translations that are used, but dynamically so the test
can't find them (most of the settings UI translations are built up
dynamically like that).

This PR is scoped to just include en-GB translation changes, since
that's the main supported language. We'll need to do a translation PR to
trim all the dead keys from the other languages, and add the missing
ones.
2026-06-10 09:49:00 +00:00
Anthony Stirling 3ecd95b779 Add MCP server with OAuth/API-key auth (#6570)
Adds an optional MCP server (proprietary module) that exposes Stirling's
PDF operations and AI capabilities to MCP clients. Off by default, zero
footprint when disabled.

### What
- New `/mcp` endpoint: streamable-HTTP + JSON-RPC 2.0; 8 tools
(describe_operation, pages/convert/misc/security category tools, AI,
upload, download).
- Runs real operations over an internal loopback; results returned
inline as base64 (small) or by fileId (large).

### Auth (two modes)
- OAuth2 resource server: RFC 9728 protected-resource metadata, RFC 8707
audience binding, JWKS, `mcp.tools.read/write` scopes; binds each token
to a provisioned Stirling account.
- API-key mode: reuses Stirling per-user `X-API-KEY` (no IdP needed).

### Security
- Per-user file ownership in FileStorage: async/queued writes scoped to
the submitting user; legacy/owner-less files stay readable.
- Admin allow/block list controls which operations are exposed.
- Python engine gated behind a shared secret (`X-Engine-Auth`).
- MCP filter chain is isolated and cannot weaken the main app's
security.
- Hardened: no upstream error-body leakage, log injection sanitized,
fileId path/sidecar enumeration blocked.

### Config / footprint
- Disabled by default (`mcp.enabled=false`); all beans
`@ConditionalOnProperty`.
---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-10 09:46:25 +00:00
ConnorYoh 84aca12055 PR-S4: shadow-mode hardening (review follow-ups) (#6523)
## What this PR does

Bundles the **low-risk polish items** from the [multi-agent review of
#6519](https://github.com/Stirling-Tools/Stirling-PDF/pull/6519). Each
change is independent, mechanical, and ships with focused unit-test
coverage.

The medium-severity items (\`?async=true\` OUTPUT recording,
JSON-consumes endpoint coverage, SpringBootTest harness) are tracked
separately in [\`notes/PAYG_DESIGN.md\` §7.5
PR-S4](https://github.com/Stirling-Tools/Stirling-PDF/blob/payg-s4-hardening/notes/PAYG_DESIGN.md)
— they need design decisions + bigger infrastructure work, so this PR
sticks to the mechanical wins.

Stacked on #6519. When that merges to main, this rebases cleanly — no
code changes.

## Changes

| Area | What | Why |
|---|---|---|
| **\`tool_id\` becomes route pattern** |
\`PaygChargeInterceptor.resolveToolId()\` prefers
\`HandlerMapping.BEST_MATCHING_PATTERN_ATTRIBUTE\` over
\`request.getRequestURI()\`. Truncates to 128 + WARN log +
\`payg.filter.errors\` increment when truncation fires. | Audit rollups
aggregate by endpoint instead of by every individual request's
path-variable / matrix-param variant. Silent truncation now louder. |
| **Direct PDF magic-byte check** | \`PaygOutputExtractor.extract()\`
magic-checks the body even for direct \`application/pdf\` responses. |
Asymmetric with the ZIP-entry path which always magic-checks. A tool
that emits \`application/pdf\` for a JSON / HTML payload would otherwise
pollute \`job_artifact_hash\`. |
| **DESKTOP_APP detection** | \`X-Stirling-Client: desktop\` header →
\`JobSource.DESKTOP_APP\`. | The enum value was unreachable from
\`determineSource()\`; Tauri shell traffic was mis-classified as WEB. No
anti-spoof — V12 step limits are identical for WEB/DESKTOP_APP so the
worst-case abuse value is zero today. |
| **\`max-bytes\` sensible default** | 500 MiB instead of \`null\`
(unbounded). | Covers the largest realistic Stirling responses (full
split-to-ZIP on a 1000-page document) while preventing pathological
cases from tying up the interceptor for minutes. Set to \`null\` to
disable. |
| **\`BufferedOutputStream\` for spill** | Wraps the spill
\`OutputStream\` in 64 KiB \`BufferedOutputStream\`. | Previously every
Tomcat chunk (default 8 KiB) was a separate syscall. Big spilled
responses get a syscall-bound speedup. |
| **Duration timer per phase** | \`payg.filter.duration\` tagged
\`phase=preHandle\` vs \`phase=afterCompletion\`. | Two distinct latency
distributions were blended into one histogram; hard to alert on. |

## Tests

| Test | What it covers |
|---|---|
|
\`PaygOutputExtractorTest.pdfContentType_butBodyMissingPdfMagic_returnsEmpty\`
| New direct-PDF magic-byte gate. |
|
\`PaygChargeInterceptorTest.preHandle_desktopClientHeader_setsJobSourceDesktopApp\`
| New \`X-Stirling-Client: desktop\` → DESKTOP_APP path. |
|
\`PaygChargeInterceptorTest.preHandle_toolId_prefersBestMatchingPattern\`
| Route pattern wins over URI when both are set. |
|
\`PaygChargeInterceptorTest.preHandle_toolId_truncatesAndCountsWhenLongerThan128\`
| Oversized values truncate + increment errors counter. |

Full saas suite green (210 tests), coverage targets met.

## What's NOT in this PR (deliberately)

- **\`?async=true\` OUTPUT recording.** The JobExecutorService returns a
synchronous \`JobResponse{jobId}\` body before the async tool actually
runs; \`afterCompletion\` fires too early. Needs a design decision:
short-circuit PAYG when \`async=true\` OR hook into \`TaskManager\`
completion. Tracked in PR-S4 design doc.
- **JSON-consumes endpoint coverage.** The
\`MultipartHttpServletRequest\` cast skips endpoints with \`consumes =
APPLICATION_JSON_VALUE\` (e.g.
\`ConvertPdfJsonController.exportPartialPdf\`). Fix is either extract a
request-body hash for JSON or add a CI lint forbidding non-multipart
\`@AutoJobPostMapping\`. Design discussion needed.
- **SpringBootTest harness for filter + interceptor wiring.** Saas
module doesn't have one yet. Separate work — PR-S3 takes a different
approach (docker-compose + Behave); a SpringBootTest layer would be
additive in-process coverage.

These are tracked in \`notes/PAYG_DESIGN.md §7.5\` so they don't slip.

## Tracked in

\`notes/PAYG_DESIGN.md\` §7.5 PR-S4.
2026-06-10 09:08:43 +00:00
stirlingbot[bot]andAnthony Stirling be914c7135 Sync Translations + tauri fix for get info (#6484)
### Description of Changes

This Pull Request was automatically generated to synchronize updates to
translation files and documentation. Below are the details of the
changes made:

#### **1. Synchronization of Translation Files**
- Updated translation files
(`frontend/editor/public/locales/*/translation.toml`) to reflect changes
in the reference file `en-GB/translation.toml`.
- Ensured consistency and synchronization across all supported language
files.
- Highlighted any missing or incomplete translations.
- **Format**: TOML

#### **2. Update README.md**
- Generated the translation progress table in `README.md` using
`counter_translation_v3.py`.
- Added a summary of the current translation status for all supported
languages.
- Included up-to-date statistics on translation coverage.

#### **Why these changes are necessary**
- Keeps translation files aligned with the latest reference updates.
- Ensures the documentation reflects the current translation progress.

---

Auto-generated by [create-pull-request][1].

[1]: https://github.com/peter-evans/create-pull-request

---------

Co-authored-by: stirlingbot[bot] <195170888+stirlingbot[bot]@users.noreply.github.com>
Co-authored-by: Anthony Stirling <anthony@stirlingpdf.com>
2026-06-09 21:31:33 +01:00
Anthony Stirling 71361f0d33 Minor: Office doc changes (#6571) 2026-06-09 18:00:34 +01:00
Anthony Stirling 6478c400db Fix font loss in rearrange/overlay/autosplit/OCR from PDFBox (#6545) 2026-06-09 18:00:20 +01:00
Anthony Stirling 502f6c1e4d fix folder causing 500 toast when deleted on another machine (#6551) 2026-06-09 18:00:02 +01:00
Anthony Stirling 1a0beaffc2 stop background flash on tab switches, unblock Audit/Usage demos (#6562) 2026-06-09 17:59:53 +01:00
Anthony Stirling 1e739b6f6f SaaS-aware API landing page (#6585)
# Description of Changes

OLD  (and still current in selfhosted)
<img width="610" height="869" alt="image"
src="https://github.com/user-attachments/assets/f8019298-b4ee-4a68-b928-a9746b64ac1c"
/>


New (in SaaS mode)

<img width="635" height="876" alt="image"
src="https://github.com/user-attachments/assets/6ee4946f-1d7b-42ec-a6f7-75e85739e348"
/>


---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-09 14:49:20 +00:00
ConnorYoh 98967bfa86 PAYG: V14 + V15 — subscription_id, free-tier, RPCs, audit logs (#6532)
## Summary

Two Flyway migrations + matching JPA entity updates. **Part 1 of 2** in
the Stripe/Supabase wire-up (PR-SB-1 in
`payg-stripe-supabase-plan.html`); the companion SaaS PR carries the
twin Supabase migrations + new edge functions.

### V14 — payg_subscription_state.sql

- `payg_team_extensions.payg_subscription_id` — the single switch that
decides whether a team is billed. NULL = free-tier or block; NOT NULL =
post Stripe meter events.
- `pricing_policy.free_tier_units_per_cycle` — per-policy free allowance
before a card is required. Default 0.
- `payg_link_subscription(team_id, customer_id, sub_id)` RPC —
idempotent.
- `payg_unlink_subscription(team_id, reason)` RPC — called on
`subscription.deleted`.
- AFTER-INSERT trigger on `teams` so every new signup gets a
`payg_team_extensions` sidecar row from creation.
- Backfill for existing teams without a sidecar row.
- RLS: SELECT permissive (any team member), UPDATE restricted to LEADER.
Service-role bypasses (backend reads + day-1 migration writes).

### V15 — payg_audit_logs.sql

- `payg_meter_event_log` — backend audit of every Stripe meter event
POST attempt (idempotency-key UNIQUE; index on unposted rows for nightly
reconcile).
- `payg_subscription_change_log` — written by V14 RPCs on every
link/unlink.

### Entity updates

- `PaygTeamExtensions.paygSubscriptionId` — read-only field; RPC
functions are the only writers.
- `PricingPolicy.freeTierUnitsPerCycle` — read by upcoming
`PaygTeamUsageService` (PR-SB-4).

### Behaviour change

**None yet.** The columns + functions sit unused until PR-SB-4 wires
`PaygMeterReportingService` and the free-tier gate into
`JobChargeService`. This PR is pure schema + JPA wiring.

## Test plan

- [x] `./gradlew :saas:test` — BUILD SUCCESSFUL
- [x] Manual schema review: column types, FK directions, RLS scope
- [ ] Apply against v3-Supabase via `supabase db push` (after companion
SaaS PR merges)
- [ ] Smoke-test the trigger: `INSERT INTO teams(...)` → assert
`payg_team_extensions` row appears
- [ ] Smoke-test RPCs: SQL-only test of `payg_link_subscription` +
`payg_unlink_subscription` produces expected row + audit entries

## References

- `notes/PAYG_DESIGN.md` (revision note 2026-06-03)
- `payg-stripe-supabase-plan.html` §3.1 (RPC functions), §3.5 (RLS),
§3.10 (audit-log tables)
2026-06-09 14:48:05 +00:00
ConnorYoh ff96a80947 PAYG B-3 / S-3: cucumber suite for shadow-mode flows + CI workflow (#6522)
## What this PR is

End-to-end cucumber coverage for the PAYG shadow charging engine (the
filter + interceptor stack from #6519), wired into CI via a new
`docker-compose-tests-saas.yml` workflow that runs only on PAYG-touching
PRs.

Stacked on #6519.

## Automated scenarios (run by `docker-compose-tests-saas.yml`)

See
[`testing/cucumber/features/payg/shadow_charges.feature`](../tree/payg-s3-cucumber/testing/cucumber/features/payg/shadow_charges.feature):

| Scenario | Validates |
|---|---|
| First tool call writes a CHARGED row | Filter + interceptor fire
end-to-end |
| Lineage join — second call on output | `JobService.joinOrOpen`
matching; no new shadow row |
| 4xx leaves the row CHARGED | "Customer paid for the attempt" semantics
|
| ZIP-returning tool records per-PDF OUTPUT | `PaygOutputExtractor`
unpacks + records signatures |
| Multi-file input writes a single shadow row | Multi-input group sizing
|
| `X-Stirling-Automation` sets PIPELINE source | Header → `JobSource`
detection |

All 6 run locally via `./testing/test-payg.sh` and will run on CI for
any PR that touches `app/saas/**`, the PAYG cucumber features, the saas
compose stack, or the workflow itself.

## Manual-only scenarios — documented in design doc, not in this suite

Two parts of the shadow engine are deliberately not automated; the
engine paths are unit-tested in
`PaygChargeInterceptorTest.afterCompletion_5xx_opened_*`, and the manual
procedures (which require a temporary throw endpoint or a container
restart with a flag flipped) live in [`notes/PAYG_DESIGN.md` §7.5.2
"PAYG cucumber: manual-only
scenarios"](../tree/payg-s3-cucumber/notes/PAYG_DESIGN.md).

- **5xx first-step failure → REFUNDED + CLOSED.** No reliably-5xx-ing
endpoint exists; manual procedure adds a throw endpoint, runs, asserts,
removes.
- **Kill-switch (`PAYG_FILTER_ENABLED=false`).** Needs a container
restart mid-suite; manual procedure tears down, flips env, brings up,
asserts zero shadow rows.

If either gets a hot-reload path (test-only throw endpoint shipped
behind a profile gate, or admin endpoint for the kill switch), automate
it in a follow-up and drop the manual procedure.

## CI workflow

`.github/workflows/docker-compose-tests-saas.yml` (new) —
self-contained, not wired into `build.yml`'s `files-changed` matrix so
the saas-cucumber job fails and succeeds independently. Triggers only on
PAYG-relevant paths. No JaCoCo coverage in v1 (saas compose doesn't have
the coverage override; can add later).

## Test infrastructure (recap)

- **`testing/compose/docker-compose-saas.yml`** — Stirling-PDF backend
with `STIRLING_FLAVOR=saas` + Postgres holding the `stirling_pdf`
schema. Supabase JWT auto-config disabled; API-key auth via
`SECURITY_CUSTOMGLOBALAPIKEY` is the live path the cucumber tests
exercise.
- **`testing/compose/payg/saas-init.sql`** + **`saas-seed.sql`** —
schema bootstrap + idempotent seed (team / user / wallet_policy).
- **`testing/cucumber/features/payg/shadow_charges.feature`** — the 6
scenarios above.
- **`testing/cucumber/features/steps/payg_step_definitions.py`** — step
defs using `requests` (HTTP) + `psycopg` (direct DB inspection). Direct
DB reads are deliberate — we want to see the filter's side effects, not
relay them through another API layer.
- **`testing/test-payg.sh`** — companion runner to `testing/test.sh`.
Brings up the saas compose, waits for health, seeds, runs behave, tears
down.
- **`behave.ini`** excludes `features/payg` from the default behave run
(the saas-cucumber CI job invokes it explicitly).

## Why a separate harness from `testing/test.sh`

The existing `test.sh` covers the proprietary-flavour stack (no PAYG
tables, no saas profile). Coupling two CI matrices that fail and succeed
independently into one script is asking for trouble. Keep the
saas-cucumber job focused on its own concerns; once the harness is
mature, the wider team can decide whether to merge them.

## Tracked in

`notes/PAYG_DESIGN.md` §7.5 (PR-S3) + §7.5.2 (manual scenarios).
2026-06-09 14:47:40 +00:00
Anthony Stirling 347ae9ebbf fix many UI issues (#6569)
# Description of Changes

- Tool action button truncation - fixed by allowing Mantine <Button>
label to wrap (whiteSpace: normal, height: auto) instead of clipping
- Role badge truncation on People page - fixed by dropping the column's
fixed w={100} and letting the badge size to its content
- Settings nav item wraps to 3 lines - fixed by hiding the inline ALPHA
badge by default and revealing it on :hover/:focus-within/.active
- Zoom slider cramped on narrow desktop - fixed by removing the
toolbar's hardcoded minWidth: 30rem and giving the slider flexShrink: 0
+ minWidth: 6rem
- "Swipe left or right" hint on desktop - fixed by adding a useIsTouch()
hook (pointer: coarse) and gating the hint on isMobile && isTouch
- Logout doesn't redirect - fixed by replacing navigate('/login') with
window.location.assign('/login') in a finally block so auth context
fully re-bootstraps
- Viewer top toolbar clips icons on mobile - fixed by switching the
wrapped state to justify-content: flex-start + overflow-x: auto so the
icon strip is momentum-scrollable
- Mobile bottom toolbar overflows - fixed by gating layout on
useIsPhone() and reducing the inline bar to prev / page / next / ⋮ only
- Lost controls when shrinking mobile toolbar - fixed by adding a
Mantine <Menu> behind ⋮ that groups First/Last page, Zoom in/out (with
%), Dual-page, Dark/Sepia filter under Page navigation / Zoom / View
labels
- "Upload from computer" label clipped on hover - fixed by unmounting
the Add Files button entirely while Upload is hovered, so Upload claims
width: 100%
- Settings rows clip controls off-screen - fixed by adding flex: 1,
minWidth: 0 to the inner text-block <div> on 44 rows across 10 -files,
so labels shrink and wrap while controls stay anchored to the right

---
Screenshots 

[report-before-after.html](https://github.com/user-attachments/files/28687621/report-before-after.html)

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-09 13:18:57 +00:00
Anthony Stirling 800a411167 Hide endpoints (#6586)
# Description of Changes

Hides the /api/v1/credits endpoints from the generated OpenAPI/Swagger
docs. The root GET /api/v1/credits and GET /api/v1/credits/usage now
carry @Hidden (the 8 admin credit endpoints were already hidden), so the
whole Credit Management controller is gone from the docs.

Adds a single global AI tag to the OpenAPI definition.


Why
We don't want the credits or AI endpoints surfaced in the public API
docs yet as they are not ready for public use, but we do want the AI
endpoints pre-grouped under one AI tag so they land cleanly when we
later un-hide them but dont clutter PDF APIs.

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-09 13:18:36 +00:00
Anthony Stirling 66f431a2b7 Lazy-load Stripe SDK so it only loads on checkout (#6546)
# Description of Changes

- Stripe SDK (`@stripe/*` + `js.stripe.com/v3`) was loading on every
page; now it only loads when an upgrade/checkout modal actually opens.
- Converted every import site to `React.lazy()` + `<Suspense>`, gated by
the existing `opened` state.
- Adds a Playwright spec that asserts no Stripe requests on landing or
settings.



---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-09 12:36:48 +00:00
James Brunton 0e3cbb3cf2 Explicitly test for console warnings & errors (#6502)
# Description of Changes
Disallow warnings and errors from being thrown in the browser console
during tests unless explicitly expected in the test. Also adds a
Playwright test to prod around some main UI areas and checks that no
warnings/errors have been thrown.
2026-06-09 08:34:02 +00:00
Anthony Stirling 92376b7382 fix: prettier format on AppConfigModal 2026-06-08 21:28:13 +01:00
Anthony Stirling 1d5ce8a1d2 chore: shorten verbose block comments across SaaS branch 2026-06-08 18:38:00 +01:00
Anthony Stirling 8b2baaf0a0 Merge remote-tracking branch 'origin/saas-docker-split' into SaaS 2026-06-08 18:10:02 +01:00
Anthony Stirling d9651f7065 fix(engine): match Dockerfile layout to root Taskfile dir: engine 2026-06-08 18:02:02 +01:00
James Brunton 002de06411 Fix desktop app not being able to load pdfium (#6575)
# Description of Changes
The changes in
[#6279](https://github.com/Stirling-Tools/Stirling-PDF/pull/6279) broke
the desktop app because the wasm URL handling didn't deal with
`tauri://` paths. Also I noticed that `task desktop:build:dev:mac`
failed locally because it was attempting to sign the app with
credentials that developers won't have (and shouldn't need), so I fixed
that too.
2026-06-08 16:21:56 +00:00
Anthony Stirling 4cd03be87a fix: send Supabase token on raw fetch in SaaS chat 2026-06-08 16:41:09 +01:00
Anthony Stirling 02d923f378 chore: remove env var debug from vite.config 2026-06-08 16:29:47 +01:00
Anthony Stirling e7d3430134 merge: pull latest main into SaaS 2026-06-08 16:28:14 +01:00
Anthony Stirling 4b2be58fab debug: fuzzy-match env var names that look like VITE_API_BASE_URL 2026-06-08 16:18:19 +01:00
Anthony Stirling 290c8c2c8b debug: enumerate VITE_/RUN_ env var names in build log 2026-06-08 16:06:56 +01:00
Anthony Stirling 90d6ecd7e1 debug: log env vars at build, write build-info.txt with masked markers 2026-06-08 15:55:40 +01:00
Anthony Stirling a0b7daca52 trigger: rebuild after VITE_API_BASE_URL scope fix 2026-06-08 13:37:30 +01:00
James Brunton 51478e5051 Policies backend (#6527)
# Description of Changes
Add a backend for running any multi-step PDF operations. This is
designed to be used for the upcoming Policies feature, along with
anything else that will require automated running of PDF operations,
like the Automate tool or Processing Folders.

The implementation is not complete. I've tried to get all the
infrastructure in there so that we can add in whichever triggers we need
in the future (like cron triggers or watching folders on disk) but
currently it just supports manual triggering of the policy.

The basis of this work was the operation running from the Stirling
Engine, which this PR removes in favour of this new system. The only
currently accessible frontend way to test this work is to ask the AI
chat to execute multiple operations on a PDF, but I've also extensively
tested with direct API calls to make sure that the policies work and
persist properly.
2026-06-08 10:50:55 +00:00
Anthony Stirling 69e62d8949 exclude unused Redis auto-config (#6547)
fix(health): exclude unused Redis auto-config so default
/actuator/health stays UP

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-08 10:41:38 +00:00
Anthony Stirling 2f6b113a13 feat(settings): link ENTERPRISE badges to plan page (#6560)
## Summary
Make the remaining static ENTERPRISE badges in the admin settings
clickable so they navigate the user to `/settings/adminPlan`, matching
the pattern already used by the PRO badges in Connections / Features /
General sections.

### Before
Two ENTERPRISE badges were inert text chips with no affordance:
- `AdminSecuritySection.tsx` - Audit Logging
- `AdminDatabaseSection.tsx` - Database section header

### After
Both now use the same pattern as the existing clickable PRO badges:
- `cursor: pointer`
- `onClick={() => navigate("/settings/adminPlan")}`
- `title` tooltip with the existing
`admin.settings.badge.clickToUpgrade` i18n key ("Click to view plan
details")

No new strings, no new components - just wiring up existing behavior to
the two badges that were missing it.

### Existing already-clickable badges (kept identical for reference)
- `AdminConnectionsSection.tsx:585-596` - SSO Auto Login PRO
- `AdminFeaturesSection.tsx:175-186` - Server Certificate PRO
- `AdminGeneralSection.tsx:920-931` - Custom Metadata PRO
2026-06-08 10:40:35 +00:00
Anthony Stirling af52134811 fix(automate): flip AutomationEntry tooltip to position=left (#6550)
# Description of Changes

automate description tooltip was facing wrong way

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-08 10:38:00 +00:00
Anthony Stirling 8a2474ff60 fix(desktop): enable in-page drag-drop in Tauri build (#6548)
## Summary

- Set `dragDropEnabled: false` on the Tauri window so HTML5 drag events
reach the WebView. Previously the default `true` made Tauri intercept
all drag-drop at the OS level, silently breaking in-page drag-to-reorder
(Pragmatic Drag and Drop in `FileEditorThumbnail` /
`useFileItemDragDrop`) in the desktop build. The Active Files tab
reorder, which feeds Merge ordering, was the user-visible symptom.
- Browser builds are unaffected (tauri.conf.json is desktop-only).
- The OS file-drop pipeline now flows through the existing Mantine
`Dropzone` in `FileEditor.tsx` via HTML5 events instead of the Rust
`WindowEvent::DragDrop` handler in `lib.rs:215`. Verified working.

## Test plan

- [x] Desktop: drag a thumbnail in Active Files past another - row goes
semi-transparent, order updates on drop.
- [x] Desktop: drag a PDF from File Explorer onto the window - file is
added.
- [x] Web build: drag-to-reorder still works (unchanged code path; flag
is desktop-only).
- [x] Merge tool: order set by drag in Active Files is the order used by
the merge output.

## Follow-up (not in this PR)

- `WindowEvent::DragDrop` arm in
`frontend/editor/src-tauri/src/lib.rs:215-229` is now unreachable for
window drops. The `forward_files_to_window` helper still serves the
macOS Finder "Open With" path (`RunEvent::Opened` at lib.rs:230), so
only the DragDrop arm can be deleted. Worth a small cleanup pass later.
2026-06-08 10:36:58 +00:00
Anthony Stirling d202c9c32f Minor: sanitize SVG (#6572)
# Description of Changes

Sanitize SVG

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-08 10:31:41 +00:00
Anthony Stirling 1ef03c43b4 fix(i18n): wrap hard-coded English strings in t() across UI (#6566)
## Summary
Audit + bulk fix of hard-coded English UI strings - `aria-label`,
`title`, `placeholder`, `label`, and raw JSX literals that bypassed i18n
entirely. Each literal now goes through `t("key", "English Default")`
from `react-i18next`, and every new key has a corresponding entry in
`en-GB/translation.toml` so translators can pick it up.

## What this fixes
Strings were rendered untranslated in every non-EN locale because they
never went through `t()` at all (not just "value not translated yet").
Affects screen-reader labels, tooltips, form placeholders, empty/loading
states, plan card content, and the entire workflow ParticipantView.

## Coverage (~143 keys / 50 files)
- **Viewer chrome** - search bar (close, clear, prev/next, "of N"
results), link/signature/redaction actions, viewer error state, zoom
labels
- **Page editor** - undo/redo/rotate/delete toolbar tooltips, empty
state, bulk selection operator chip tooltips
- **Shared primitives** - Tooltip close, InfoBanner dismiss, TextInput
clear, Toast dismiss/toggle, UpdateModal close, EditableSecretField
edit, DropdownListWithFooter search, FileCard/FileDropdownMenu actions,
EmptyFilesState + AddFileCard upload
- **Tools** - Image upload + hint, ColorControl eyedropper, sign Use
Signature, CompressSettings, OCR loading, PageLayout
margin/border/row/col placeholders, FormFill switch + save + re-scan
- **Proprietary admin** - OverviewHeader signed-in line + logout,
AdminPremiumSection moved-features list (via `<Trans>`),
AdminPlanSection no-data alert, AdminAdvancedSection temp-dir
placeholders, AdminEndpointsSection multiselect placeholders,
AdminMailSection + AdminDatabaseSection password placeholders
- **Onboarding** - MFASetupSlide QR loading + auth code label,
SecurityCheckSlide role select + options
- **ParticipantView** - entire sign-document UI (~30 strings: loading,
error, badges, headings, cert-type Select, all input labels and
placeholders, action buttons, completion + expired alerts) - file
previously imported `useTranslation` but only used `t()` for cert
validation
- **planConstants.ts refactor** - replaced `PLAN_FEATURES` /
`PLAN_HIGHLIGHTS` const exports with `usePlanFeatures()` /
`usePlanHighlights()` hooks. Service layer (`licenseService.getPlans`)
updated to accept feature/highlight maps so it stays hook-free. Callers
(`usePlans`, `CheckoutContext`) resolve the hooks at the React boundary
- **Previously catalogued offenders** - `FileSidebarFileItem`
open/close-viewer aria-labels, `quickAccessBar/ActiveToolButton` "Back
to all tools" tooltip + aria, `AppConfigModal` close button

## Notes
- One small refactor in `usePageSelectionTips.ts` was needed to resolve
a TOML key-shape conflict: the existing scalar keys
`bulkSelection.operators.{and,not,comma}` needed to become tables to
hold the new `.title` subkeys for OperatorsSection's chip tooltips. The
existing descriptions moved to `[bulkSelection.operators.descriptions]`
and the three i18n key paths in usePageSelectionTips were updated to
match.
- Viewer sidebar close buttons
(Bookmark/Layer/Thumbnail/Attachment/Comments) were on the audit list
but are NOT on main - they're added by the unmerged PR #6552
(feat/viewer-sidebar-ux). Those particular strings will need wrapping
when that PR lands.
- TOML hook (`toml-sort-fix`) ran and re-sorted the translation file.

## Test plan
- [ ] `task frontend:typecheck` passes (core + proprietary + desktop
variants)
- [ ] `task frontend:lint` passes
- [ ] Switching language to Deutsch / Русский: previously-English
`aria-label`s + tooltips + placeholders + plan card bullets now render
translated (when the locale has values) or fall back to the English
default (when it doesn't)
- [ ] Plan page bullet points in EN render unchanged
- [ ] Sign-document flow (ParticipantView) renders unchanged in EN
2026-06-08 10:11:43 +00:00
Anthony Stirling 0b575ed841 fix: respect BASE_PATH in AI chat fetch and pdfjs worker assets 2026-06-06 21:21:24 +01:00
Anthony Stirling 940cb2fc44 chore: trigger Cloudflare deploy 2026-06-06 19:26:08 +01:00
Anthony Stirling 9da0a0d020 fix: respect BASE_PATH in redirects, comparisons, and cookie consent paths 2026-06-06 19:20:07 +01:00
brios 0dff192281 perf(compression): add vite-plugin-compression for gzip and Brotli support (#6279)
Signed-off-by: Balázs Szücs <bszucs1209@gmail.com>
2026-06-05 18:49:09 +01:00
Anthony Stirling 9866d6e12d Fix desktop updater latest.json generation for releases (#6540) 2026-06-05 15:41:41 +01:00
James Brunton e79f4a044f Make any typing linting opt-out instead of opt-in (#6542)
# Description of Changes
Reconfigure linting of `any` type to an opt-out instead of an opt-in
strategy now that we're close enough to everything supporting it. Also
slightly expands the scope of things included in the linting.
2026-06-05 14:11:26 +00:00
James Brunton 9ab404b2e6 Fix intermittently failing Playwright tests in main (#6541)
# Description of Changes
[#6474](https://github.com/Stirling-Tools/Stirling-PDF/pull/6474)
updated the IndexedDB schema number to v9, but a couple of Playwright
tests were explicitly creating a DB in v4 schema, which then caused
inconsistently failing tests because the DB upgrade process is
asynchronous and sometimes was too slow to upgrade, causing the test to
get into an invalid state.

Also fixes the screenshots directory exclusion since the frontend folder
was restructured.
2026-06-05 11:34:32 +00:00
EthanHealy01 a61fe012d7 chore: i18n time utils and use TFunction type (#6507) (#6539)
## Summary

Addresses two review comments from #6507:

- **`timeUtils.ts`** — route relative time strings (`just now`, `Xm
ago`, `Xh ago`, `Xd ago`) through i18n by accepting a `TFunction`
parameter and using new `time.relative.*` keys in `en-GB`
- **`ChatPanel.tsx`** — replace `ReturnType<typeof useTranslation>["t"]`
with `TFunction` from `i18next`

## Test plan
- [x] `task frontend:check` passes (695 tests)
2026-06-05 09:45:27 +00:00
Anthony Stirling c93776e297 Fix z-index conflicts: Google Drive picker, automate dropdowns, tooltips (#6513)
## Summary

## What changed

### 1. Google Drive Picker now renders above the FileManager modal
`frontend/editor/src/core/services/googleDrivePickerService.ts`

The picker is opened from inside the FileManager modal
(`Z_INDEX_FILE_MANAGER_MODAL = 1200`), but Google's Picker defaults to
z-index ~1001 - so it landed *behind* the modal that invoked it. Added
`setZIndex(Z_INDEX_OVER_FILE_MANAGER_MODAL)` to the builder.

### 2. `Z_INDEX_AUTOMATE_DROPDOWN` no longer collides with
`Z_INDEX_FILE_MANAGER_MODAL`
`frontend/editor/src/core/styles/zIndex.ts`

Both constants were `1200`. The automate dropdown only needs to sit
above the automate modal (1100), so dropped it to `1150`. This keeps
automate dropdowns above their parent modal but reliably below the file
manager scrim when the two overlap.

Confirmed callers - all are dropdowns inside automate-modal tool
settings:
- `DropdownListWithFooter.tsx`
- `AddPageNumbersAppearanceSettings.tsx`
- `AddPasswordSettings.tsx`
- `StampPositionFormattingSettings.tsx`
- and several other `*Settings.tsx` files

All keep working as intended (1150 > 1100).

### 3. Tooltip z-index honours the documented hierarchy again
`frontend/editor/src/core/components/shared/tooltip/Tooltip.module.css`

`Tooltip.tsx` sets `zIndex: Z_INDEX_OVER_FULLSCREEN_SURFACE` (1300)
inline, but the CSS module had a hardcoded `z-index: 9999` that overrode
it. Removed the stale CSS rule so tooltips render at the intended 1300
level rather than floating above almost everything.
2026-06-05 08:16:21 +00:00
EthanHealy01 1698769928 Improvements to agent chat markdown rendering. (#6507)
### To test

- Ask the agent to “list all the things you can do and put them in a
markdown table”. I know we’re explicitly asking it for markdown, but I
don’t want to update the system prompt to ask it to make tables when
necessary because it’ll probably turn everything into a table, not sure
though, we can test in future.
     -  Notice how the loading is different
- Notice how the user chat is in a bubble but the agent chat is flat
(super standard design practice in AI tools, and looks much better when
the agent outputs mardown, expecially tables and needs room to do so)
- Ask it to do something different, then close the chat, and see that
the agent is marked as running and has a green outline and a green dot.
     - Play around with resizing the chat to make it bigger/smaller    
    
Open to any and all criticisms on any of the design choices, and of
course the usual, code etc.


Resizing
<img width="1572" height="812" alt="Screenshot 2026-06-01 at 2 47 53 PM"
src="https://github.com/user-attachments/assets/ec0ac1d0-01da-4025-bf7e-eea4eb544181"
/>

Loading (cool animation not visible through screenshot obviously)
<img width="559" height="141" alt="Screenshot 2026-06-01 at 2 53 41 PM"
src="https://github.com/user-attachments/assets/99f0b1f5-1719-4d78-8947-21b142293052"
/>

Removed bubbles for agent chat (maybe controversial, let me know) and
markdown now renders properly again
<img width="654" height="1060" alt="Screenshot 2026-06-01 at 2 55 01 PM"
src="https://github.com/user-attachments/assets/445f0889-a632-4751-9a16-f80ae388c632"
/>
2026-06-04 18:26:19 +00:00
Anthony StirlingandReece Browne bd9ef0586b fix: harden multi-file response detection so merge can't fail silently (#6516)
Co-authored-by: Reece Browne <74901996+reecebrowne@users.noreply.github.com>
2026-06-04 18:17:36 +01:00
Anthony Stirling cb687fbf99 fix(e2e): stop files-page tests racing the skeleton-grid render (#6533) 2026-06-04 18:00:30 +01:00
Anthony Stirling 69ee39fa6e Fix settings: dark borders, update dropdown z-index, dead accessibility link (#6528) 2026-06-04 17:59:22 +01:00
Anthony Stirling 353b5c807c sort comments sidebar in visual reading order (#6439) (#6514) 2026-06-04 17:58:55 +01:00
ConnorYoh 22dacbed01 PAYG B-2: shadow-mode filter + interceptor (engine activation) (#6519)
## What this PR does

Wires the **B-1 shadow charging engine** into real HTTP request flow.
After this lands, flipping an internal team to ``PAYG_SHADOW`` via SQL
begins populating ``payg_shadow_charge`` automatically — with **zero
impact** on the legacy credit deduction path.

**This is the load-bearing PR for shadow mode.** Without it, B-1's
engine sits idle — nothing in the codebase calls
``JobChargeService.openProcess()`` from a real HTTP request.

Stacks on top of #6477 (PR B-1).

## Components

| Class | Role |
|---|---|
| ``PaygResponseBodyWrapperFilter`` | Servlet filter, installs tee'ing
response wrapper. Defers wrapper close to ``AsyncListener`` for
``DeferredResult`` / ``CompletableFuture`` controllers so the lifetime
spans the async window. |
| ``PaygResponseBodyWrapper`` | ``HttpServletResponseWrapper`` —
in-memory ``ByteArrayOutputStream`` up to 10 MiB; spills to ``TempFile``
above. ``materialisedPath()`` always returns a uniform ``Path``
interface. |
| ``PaygChargeInterceptor`` | ``AsyncHandlerInterceptor`` mirroring
``UnifiedCreditInterceptor`` shape. ``preHandle`` gates on
``@AutoJobPostMapping``, materialises multipart inputs, calls
``JobChargeService.openProcess``. ``afterCompletion`` branches on HTTP
status. |
| ``PaygOutputExtractor`` | Pulls PDFs out of the response body. Direct
``application/pdf`` returns body verbatim; ``application/zip`` iterates
entries and keeps each ``.pdf`` entry whose first bytes match the
``%PDF-`` magic. |
| ``PaygWebMvcConfig`` | Registers filter at end of Spring filter chain
(after security); interceptor after ``UnifiedCreditInterceptor``. |
| ``PaygFilterProperties`` | ``payg.filter.enabled`` master switch +
in-memory threshold + optional max-bytes ceiling. |

## Status branching in afterCompletion

| HTTP status | Action |
|---|---|
| **2xx** | Append OK step; extract PDFs from response;
``JobService.recordOutput`` per PDF |
| **4xx** | Append FAILED step with ``errorCode``. No refund — customer
paid for the attempt. No OUTPUT recording. |
| **5xx + OPENED** (first-step) |
``JobChargeService.markFirstStepFailed`` → shadow row flipped to
``REFUNDED``, process CLOSED. Refund counter incremented. |
| **5xx + JOINED** (mid-chain) | ``JobChargeService.decrementStepCount``
— step slot returned without resetting ``lastStepAt`` (workflow window
stays active for retry). |

## New ``JobChargeService`` methods

- **``markFirstStepFailed(jobId, reason)``** — flips shadow row to
``REFUNDED`` with ``refundedAt`` + ``refundReason``, closes the process.
Idempotent. Mimics the eventual Stripe
``meter_event_adjustment(cancel)`` flow that real-mode will invoke at
the same callsite. **Refund implies close** so a same-input retry can't
lineage-join into a refunded chain for free work.
- **``decrementStepCount(jobId)``** — defensive floor at 1; never drives
count negative.

## Schema

- Backend: ``V13__payg_shadow_charge_status.sql`` adds ``status``
(``CHARGED`` | ``REFUNDED``) + ``refunded_at`` + ``refund_reason``.
``DEFAULT 'CHARGED'`` so existing B-1 rows stay correct without
backfill.
- Supabase: matching migration in
[Stirling-PDF-SaaS#payg-shadow-charge-status](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/tree/payg-shadow-charge-status)

## Fail-open semantics in shadow

Any ``RuntimeException`` in ``preHandle`` / ``afterCompletion`` is
logged at WARN, increments ``payg.filter.errors``, and lets the
customer's tool call proceed unbilled. This **reverses to fail-closed**
when ``wallet_policy.engine = PAYG`` (real charging) — that reversal
lives inside ``JobChargeService`` and ships with the cap evaluator PR
(PR-C1 in PAYG_DESIGN.md).

## Observability

Micrometer metrics:
- ``payg.filter.errors`` Counter — internal failures (preHandle +
afterCompletion). Alert source.
- ``payg.filter.calls`` Counter, tagged ``disposition`` (``OPENED`` |
``JOINED`` | ``SHORT_CIRCUIT``)
- ``payg.filter.refunds`` Counter — first-step 5xx refunds
- ``payg.filter.duration`` Timer — preHandle + afterCompletion
wall-clock per request

## Test coverage (38 tests across 4 classes)

- **PaygResponseBodyWrapperTest** (12 tests) — in-memory, spill,
threshold crossing mid-chunk, writer vs outputStream exclusivity,
``resetBuffer`` with and without spill, close idempotency, single-byte
writes across threshold.
- **PaygOutputExtractorTest** (7 tests) — direct PDF, parametrised
content type, ZIP with mixed entries + magic-byte gate, corrupt ZIP
fail-open, empty ZIP.
- **PaygChargeInterceptorTest** (13 tests) — all preHandle
short-circuits, OPENED disposition stash, fail-open on chargeService
exception, 2xx recordOutputs path, 5xx OPENED → markFirstStepFailed, 5xx
JOINED → decrementStepCount, 4xx FAILED step append, max-bytes ceiling
skip, PIPELINE header detection.
- **JobChargeServiceTest extended** (+6 tests) — markFirstStepFailed
happy path, idempotency, missing-shadow-row case, long-reason trim;
decrementStepCount happy path, floor-at-1 defence, missing-job no-op.

## What's NOT in this PR (deliberate)

- **No SpringBootTest layer.** The saas module doesn't have bootstrap
test infrastructure (Supabase JWT config + H2 schema harness).
Integration confidence comes from the manual staging deploy + SQL-flip
of an internal team. Bootstrap-test infra is a focused follow-up if
needed.
- **No saas-mode Behave / docker-compose.** Per design §17 — deferred.
Existing ``testing/cucumber/`` infrastructure doesn't yet have a
saas-profile compose target; that's its own PR when warranted.
- **No CreditService wire-in** (per design §13 decision). Per-row
comparison data moves to the reconciliation report PR (PR-S2).
``legacy_credits_charged`` + ``diff_pct`` columns stay at 0 in shadow
rows.
- **No reconciliation report endpoint.** Direct SQL queries against
``payg_shadow_charge`` cover the data-access need until patterns emerge.

## Rollback levers

| Symptom | Lever |
|---|---|
| Some / all tool calls breaking due to filter |
``payg.filter.enabled=false`` + restart (~20s) |
| Shadow rows look wrong for a specific team | ``UPDATE wallet_policy
SET engine = 'LEGACY' WHERE team_id = ?`` |
| Mass shadow weirdness | ``UPDATE wallet_policy SET engine = 'LEGACY'``
|
| Memory exhaustion from response tee | Lower
``payg.filter.response.in-memory-threshold-bytes`` |

## Test plan

- [ ] CI green (build + tests)
- [ ] Aikido / Snyk / SonarCloud clean
- [ ] Manual: deploy to staging
- [ ] Manual: flip one internal team via ``UPDATE wallet_policy SET
engine = 'PAYG_SHADOW' WHERE team_id = ?``
- [ ] Manual: hit ``/api/v1/security/add-password`` with that team's
JWT; verify a ``payg_shadow_charge`` row appears with
``status='CHARGED'``
- [ ] Manual: trigger a 503 (e.g. via temporary backend kill
mid-request); verify the resulting row is ``status='REFUNDED'`` + the
process is ``CLOSED``
- [ ] Manual: hit ``/api/v1/general/split`` with a multi-page PDF;
verify one OUTPUT signature per inner PDF appears in
``job_artifact_hash``
- [ ] Manual: chain ``add-password`` → ``compress`` on the output;
verify the second call JOINS the first process (no new shadow row) and
the inner output OUTPUT signature is what drove the lineage join

## Stacks on / references

- Stacks on: #6477 (B-1 — shadow charging engine)
- Schema mirror: Stirling-PDF-SaaS#payg-shadow-charge-status branch
- Design doc: ``notes/PAYG_FILTER_DESIGN.md`` (all 19 decisions DECIDED)
2026-06-04 15:07:58 +00:00
ConnorYoh 3807cdfbc6 PAYG: process tracking + shadow charging engine (PR B-1) (#6477)
> 📌 **Stacked on
[#6464](https://github.com/Stirling-Tools/Stirling-PDF/pull/6464)**
(lineage primitives, still in review). #6469 has merged so its commits
are no longer in this PR's diff. Once #6464 merges, a final rebase
collapses the lineage-primitives commits out of this diff too — leaving
only the B-1 work.

## What this is

Process tracking + shadow charging engine. Bundles PR-I7 service half
with the non-filter piece of PR-I7a so the pieces ship together — none
of them is useful in isolation.

**Review focus:** the new files in:
- \`app/saas/src/main/java/stirling/software/saas/payg/job/\`
(\`JobService\`, \`JobContext\`, \`JoinOrOpenResult\`,
\`StaleJobCloser\`)
- \`app/saas/src/main/java/stirling/software/saas/payg/charge/\`
(\`JobChargeService\`, \`ChargeContext\`, \`ChargeOutcome\`,
\`JobInput\`)
-
\`app/saas/src/main/java/stirling/software/saas/payg/lineage/LineagePruneScheduler.java\`
- their tests

The 8 files inherited from #6464 (lineage primitives) are unchanged from
there — they ride along in this diff until #6464 lands.

The remaining work for shadow-in-staging is the ingress/egress filter
that wires controllers into this engine — that's PR B-2.

## Scope

### \`JobService\` — persistence + lineage policy

- **\`joinOrOpen\`** — the multi-input "any-match-joins, newest wins"
rule. Hash every input via the lineage detector; if any matches an open
process in the workflow window, attach to the one with the freshest
\`lastStepAt\`. Step-limit overflow on the matched job spawns a fresh
process; the new job's input signatures are still recorded so
\`mostRecentMatchWins\` routes future calls forward.
- **\`recordOutput\`** — post-tool-success path. Records OUTPUT
signatures so the next call that takes this file as input
lineage-matches into the same process.
- **\`appendStep\`** — audit-trail step row written after a tool
completes.
- **\`close\`** — idempotent; safe to call from multiple paths
(explicit, FE on-unload, scheduler). Returns the same row on re-close,
no state mutation.
- **\`findStale\` / \`closeStale\`** — workflow-window-based stale
closure used by the scheduler.

### \`JobChargeService\` — the orchestrator (shadow variant)

\`openProcess\` resolves the effective policy via
\`PricingPolicyService\` (now in main via #6469), derives the step-limit
for the current \`JobSource\` (with a defensive fallback if the policy
is missing an entry), delegates to \`JobService.joinOrOpen\`, and on
OPENED runs the \`DocumentClassifier\` + writes a \`payg_shadow_charge\`
row. Applies the policy-level \`minChargeUnits\` floor per design § 3.4.

Shadow variant only — never debits the ledger, never posts a Stripe
meter event. The real-charging follow-up reuses the same orchestration
and swaps the side-effect. \`legacyCreditsCharged\` on the shadow row
stays \`0\` until the legacy \`CreditService\` is wired in (PR B-2),
where the comparison becomes meaningful.

### Schedulers (both plain \`@Scheduled\`)

- **\`StaleJobCloser\`** — fixed-rate 60 s. Closes \`OPEN\` jobs idle
past the workflow window. API users never have to call close explicitly
— this is the safety net.
- **\`LineagePruneScheduler\`** — hourly cron, retention 1 h. Deletes
\`job_artifact_hash\` rows older than the retention window.
- **No \`@SchedulerLock\` / no \`shedlock\` table** — consistent with
the 5 existing unguarded \`@Scheduled\` tasks in \`:saas\`
(\`CreditResetScheduler\` and friends, none of which are guarded today).
Cluster-correctness across all 7 saas schedulers is tracked in design §
9 as a separate focused cleanup. Underlying operations are idempotent —
duplicate firings on multi-pod would be wasted DB load, not data
corruption.

### Records (call-shape glue for PR B-2's filter)

- \`JobContext\` / \`JoinOrOpenResult\` — input/output for
\`JobService\`.
- \`ChargeContext\` / \`ChargeOutcome\` — input/output for
\`JobChargeService\`.
- \`JobInput\` — paired \`(MultipartFile, materialised Path)\` so the
upcoming ingress filter can pass both views without re-materialising.

## Tests

**26 new, all green.**

- 14 × \`JobServiceTest\` — no-match → opens new, single-match → joins
existing, multi-input any-match-joins, multi-match newest-wins (older
job never even looked up), step-limit hit spawns fresh job (and original
\`stepCount\` is NOT mutated), empty inputs reject, stale-signature
handling, recordOutput delegation, close idempotency, closeStale,
appendStep persistence.
- 7 × \`JobChargeServiceTest\` — JOINED skips classifier + shadow write
entirely, OPENED writes shadow row + classifies (single + multi file
paths), \`minChargeUnits\` floor applied, step-limit resolved
per-\`JobSource\` from policy, missing source entry falls back to
conservative default of 10.
- 2 × \`StaleJobCloserTest\`, 3 × \`LineagePruneSchedulerTest\` —
scheduler-wiring smoke + constructor-validation tests.

\`ENABLE_SAAS=true ./gradlew :saas:test\` — BUILD SUCCESSFUL.

## What's not in this PR (lands in PR B-2)

- **Tool ingress/egress servlet filter.** The highest-risk piece —
materialises the request body into a \`JobInput\`, calls
\`JobChargeService.openProcess\` from every \`@AutoJobPostMapping\`,
records OUTPUT after success. Edge cases to validate: multipart parts,
async controllers, streaming responses, errored 5xx paths, very large
files. Design decisions for the filter are being worked through in
\`notes/PAYG_FILTER_DESIGN.md\` before any code is written.
- **Wire shadow path into legacy \`CreditService\`.** Every legacy debit
writes a comparison row carrying both the PAYG would-be units and the
legacy actual credits, populating \`diffPct\`.

## Design doc

\`notes/PAYG_DESIGN.md\` — PR-I7 + PR-I7a status updated to reflect this
bundle. § 9 carries the cluster-correctness deferral note alongside the
existing LISTEN/NOTIFY trade-off note. § 7.5.1 readiness summary shows
the path now needs just **1 more PR** (the filter half + CreditService
wire-in, both bundled into PR B-2).
2026-06-04 11:09:25 +00:00
EthanHealy01andJames Brunton 35a712a278 smart redaction (#6195)
Co-authored-by: James Brunton <jbrunton96@gmail.com>
2026-06-03 16:16:33 +00:00
James Brunton 7f3ca7ea70 Fix mockServiceWorker.js reformatting (#6526)
# Description of Changes
`mockServiceWorker.js` is a third-party managed file, which is included
in our `.prettierignore` file, and is rewritten to be in the module's
standard format whenever `msw` runs. At some point, it was reformatted
in our style, but shouldn't have been. This puts it back to `msw`'s
style, which should make it stop appearing in diffs.
2026-06-03 15:47:03 +00:00
stirlingbot[bot]andAnthony Stirling 895dcbbafd Update Backend 3rd Party Licenses (#6407)
Co-authored-by: Anthony Stirling <anthony@stirlingpdf.com>
Signed-off-by: stirlingbot[bot] <stirlingbot[bot]@users.noreply.github.com>
2026-06-03 15:52:28 +01:00
James Brunton b705c5b84c Switch to use JPDFium v1.0.2, which signs the Mac binaries (#6521)
# Description of Changes
Currently, it's not possible to develop the backend on Mac without
manually signing the JPDFium binaries yourself since macOS will reject
running the unsigned binaries. [We've now updated JPDFium to sign the
Mac binaries in
v1.0.2](https://github.com/Stirling-Tools/JPDFium/releases/tag/v1.0.2),
so update to use that version.
2026-06-03 11:55:38 +00:00
James Brunton 1264f4cfed Set up document management for Stirling Engine (#6476)
# Description of Changes
Change Stirling Engine to support deleting documents automatically. This
happens both on user logout and after an amount of time specified by the
Java when ingesting a document (allowing for personal documents to have
short lifetimes but org documents to be left in the db with no expiry
date). Also sets up an [ACL
policy](https://en.wikipedia.org/wiki/Access-control_list) for the
documents so the database knows which users have access to which
documents. This is not fully implemented in the Java, so currently all
docs are treated as having a single owner, the uploader, but
theoretically when we need to support org storage, we shouldn't need to
change the db schema.
2026-06-03 11:52:11 +00:00
James Brunton 71633861d0 Make zoom key command behave the same regardless of mouse position (#6508)
# Description of Changes
Make zoom key command behave the same regardless of mouse position.
Previously only zoomed the editor if the mouse was over the editor.
2026-06-03 11:09:55 +00:00
ConnorYoh e6974d52f7 PAYG: hash-lineage detection primitives (modular extractor / store / detector) (#6464)
## What this is

Three orthogonal interfaces — each with one production impl — for
detecting whether an incoming tool call should join an existing process
via content-hash lineage. Groundwork for PR-I7a: nothing in this PR
calls the detector yet; the ingress/egress filter that wires it into
every controller lands separately.

Built to be modular along three axes. Swapping any of them should not
require changes elsewhere:

| Axis | Interface | V1 impl | Plausible future impl |
|---|---|---|---|
| Hash algorithm | `LineageSignatureExtractor` |
`ByteHashSignatureExtractor` (SHA-256) | `PdfMetadataSignatureExtractor`
(PDF `/ID`, content-stream hash) |
| Storage backend | `JobLineageStore` | `JpaJobLineageStore` |
`RedisJobLineageStore` |
| Matching policy | `HashLineageDetector` | `DefaultHashLineageDetector`
| strategy-driven variant (any-match-joins for multi-input — lives in
JobService) |

## Interfaces

### `LineageSignatureExtractor` — what counts as a fingerprint

```java
public interface LineageSignatureExtractor {
    Set<LineageSignature> extract(Path file) throws IOException;
    String name();
}
```

File-based (not stream-based) so a future PDF-aware extractor can open
the same file via jpdfium / PDFBox and pull `/ID[0]` or a content-stream
hash. Multiple extractors compose at the detector layer — Spring
auto-wires all `LineageSignatureExtractor` beans, the detector unions
their results.

Production impl: **`ByteHashSignatureExtractor`** — SHA-256 over the
file via 64 KiB-buffered `DigestInputStream`. Hardware-accelerated by
the JVM (Intel SHA-NI, ARM SHA).

### `JobLineageStore` — where signatures live

```java
public interface JobLineageStore {
    void record(UUID jobId, Set<LineageSignature> signatures, ArtifactKind kind);
    Optional<LineageMatch> findOpenJobForSignatures(Long userId, Set<LineageSignature> candidates, Duration window);
    int pruneOlderThan(Instant cutoff);
}
```

Knows nothing about storage technology. Production impl
**`JpaJobLineageStore`** runs a single joined query against
`job_artifact_hash` ⋈ `processing_job` — status + window filtering
happen at the database. The query is bounded by `Limit.of(1)` on the hot
path so a job set sharing a popular signature doesn't materialise
unwanted rows. A future `RedisJobLineageStore` (or write-through hybrid)
is a drop-in.

### `HashLineageDetector` — the high-level API

```java
public interface HashLineageDetector {
    Optional<LineageMatch> detect(Long userId, Path inputFile) throws IOException;
    void record(UUID jobId, Path file, ArtifactKind kind) throws IOException;
}
```

**`DefaultHashLineageDetector`** delegates extraction to every
registered `LineageSignatureExtractor`, storage to the configured
`JobLineageStore`, and reads `payg.lineage.workflow-window` (default
`PT5M`) from config. When a single extractor throws (e.g. a future
PDF-aware extractor against a malformed PDF), the other extractors still
contribute — failures don't block the byte-hash from landing.

## Profile gating

All three `@Component` beans (`JpaJobLineageStore`,
`ByteHashSignatureExtractor`, `DefaultHashLineageDetector`) are
`@Profile("saas")` — consistent with every other `:saas` bean. Without
this guard the JPA store would fail to wire against its profile-gated
repository in non-saas profiles that pull `:saas` onto the classpath.

## Tests

Run entirely in-memory; no database required.

- **`LineageSignatureTest`** — storage-key encoding round-trips, rejects
malformed `"type:value"` keys.
- **`ByteHashSignatureExtractorTest`** — identical bytes → identical
sigs; empty file hashes to the well-known SHA-256-of-empty constant; 10
MiB file streams without OOM.
- **`DefaultHashLineageDetectorTest`** — same-user / within-window /
status=OPEN filtering, multi-signature matching (one extractor sees
`pdf-id` and matches even when bytes differ), most-recent-job-wins,
record+detect round-trip, extractor-throwing-doesn't-break-others.

**`InMemoryJobLineageStore`** (in test sources) implements the same
`JobLineageStore` interface as the JPA impl, plus a `registerJob` hook
for tests to model job state. Same contract — proves the abstraction is
portable. When the Redis impl lands it gets the same contract tests.

## What's not in this PR (deliberate)

- The tool ingress/egress filter that wires the detector into every
controller — separate, focused review.
- `JobChargeService.openProcess()` — uses the detector, part of the
charging machinery, separate PR.
- Prune scheduler that calls `pruneOlderThan` — small follow-up
alongside the `shedlock` foundational table.
- PDF-aware extractor (`PdfMetadataSignatureExtractor`) — to be added
when we measure how often byte-hash-only misses real workflows.
- Multi-input "any-match-joins" lineage policy — that's a `JobService`
decision (PR-I7), not a primitive.

## Self-review pass applied

An independent code-review on this PR caught:

- **HIGH:** Missing `@Profile("saas")` on the three `@Component` beans →
fixed.
- **MEDIUM:** `pruneOlderThan` missing `@Transactional` (its
`@Modifying` query would have thrown
`InvalidDataAccessApiUsageException`) → fixed.
- **MEDIUM:** `findOpenJobsForSignatures` fetching the whole match set
just to `get(0)` → now takes `Limit`, JPA store passes `Limit.of(1)` on
the hot path.
- **LOW:** `InMemoryJobLineageStore` used both `synchronized` methods
and `ConcurrentHashMap` → dropped the redundant `ConcurrentHashMap`.

Deferred: project-wide UTC unification (`LocalDateTime.now()`
system-zone is the established convention; flipping one file mid-stack
caused a real test failure — proper fix needs its own audit).

## Rollback

Straight `git revert`. No callers yet; deleting these classes wouldn't
break anything.

---

## Checklist

- [x] Tests pass: `ENABLE_SAAS=true ./gradlew :saas:test`
- [x] No new warnings
- [x] Self-review performed (HIGH + MEDIUM findings addressed)
2026-06-03 11:00:08 +00:00
Anthony Stirling 0b944a29a7 Prefer Maven Central over jboss/shibboleth mirrors for resilience 2026-06-03 09:10:08 +01:00
Anthony Stirling 58aeba2bf7 Add backend-only and SaaS-aware frontend Dockerfiles 2026-06-03 09:03:58 +01:00
Anthony StirlingandClaude Opus 4.6 256d1a86d2 UI changes to update and support auto updating (#6075)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-06-02 23:11:37 +01:00
Reece BrowneandClaude Opus 4.8 919f0ade99 Portal (#6391)
# Description of Changes

## What & why

This PR introduces the **Stirling developer portal** — a new
control-plane frontend that sits alongside the existing PDF editor —
plus the shared design system and workspace structure needed to host
both apps in one frontend.

The portal is the parent product surface: where users connect sources,
compose pipelines, wire agents, and manage usage / billing /
infrastructure, with the PDF editor as one capability inside it. This PR
lays the **foundation** — workspace reshape, design system, app shell,
navigation, and a mock-driven home — rather than wiring real backends
(those surfaces are placeholders for follow-up phases).

## What's in this PR

**1. Frontend repo reshape (`frontend/src/` → `frontend/editor/`)**
The existing editor app moved under `frontend/editor/`, so `editor`,
`portal`, and `shared` are siblings in one workspace. All references
were updated accordingly: `LICENSE`, `.dockerignore`, `.gitignore`,
build/sign shell scripts, the GH language-check script, the Taskfile,
and Docker config. **No editor source logic changed — path references
only.**

**2. New shared design system (`frontend/shared/`)**
- **Design tokens** in `tokens.css` as the single runtime source of
truth (light/dark, category accents, gradients). `tokens.ts` now holds
only the `Tier` type — the old JS palette mirror was removed (nothing
consumed it and it had drifted).
- ~30 framework-light **components** (Card, Button, Input, Select, Tabs,
Modal, Drawer, Toast, MetricCard, StatusBadge, Skeleton, EmptyState, …)
with Storybook stories.
- **Typed data catalogues**: `endpoints.ts` (10 verticals / 64
endpoints) and `ops.ts`.

**3. New developer portal app (`frontend/portal/`)**
- App shell: `Header`, `Sidebar`, `AssistantPanel`, search modal,
notifications, tier switcher, theme toggle, MSW toggle.
- **Tier-aware** home (free / pay-as-you-go / enterprise): KPI strip,
30-day usage chart, onboarding checklist, quick actions, recent
activity, region health, product grid, and a curated **"Popular use
cases"** teaser.
- **Documents** view hosting the full, tab-filterable endpoint
catalogue.
- Placeholder views for Sources / Pipelines / Agents / Editor /
Infrastructure / Usage & Billing / Developer Docs / Settings (follow-up
phases).
- **MSW-mocked** API layer: `api/*` issues real `fetch`, intercepted by
mocks in dev/Storybook; pointing at a real backend is just a matter of
not registering MSW. `react-router` URLs; Tier / View / UI contexts.

**4. Tooling & guardrails**
- ESLint extended to `portal` + `shared`, with **layering-boundary
rules**: `shared/` may depend only on third-party packages and itself
(no `@app` / `@portal` / `@core` / `@proprietary` / Tauri), so it stays
cleanly extractable into a standalone package later.
- `dpdm` circular-dependency check now walks editor + portal + shared
(the old glob matched only 2 files).
- New **devDependencies only** — Storybook (+ a11y/docs/themes addons),
MSW. No runtime dependencies added.
- New tasks: `frontend:dev:portal`, `frontend:build:portal`.

## Testing done locally

- `tsc` for both `portal` and `shared` projects — clean
- `eslint --max-warnings=0` across the whole frontend — clean
- `dpdm` circular-dependency check — no cycles
- Editor builds clean: `vite build editor --mode core` (✓ built, only
the pre-existing >500 kB chunk-size advisory)
- Editor runs in dev (core mode) with **zero console errors**; portal
runs in dev across all three tiers

## Notes for reviewers

- The change is overwhelmingly **additive**: `shared/` and `portal/` are
brand-new; the existing editor is path-reference changes only.
- The portal is intentionally **mock-driven** at this stage — real
backends and the remaining views land in follow-up phases.

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 16:08:24 +00:00
Anthony Stirling b355ccec9e Add Valkey cluster backplane and sticky-410 ownership (clusters) (#6472) 2026-06-02 14:59:20 +01:00
Anthony Stirling de9d6ad3f5 Add CI coverage summaries and aggregate JaCoCo report (#6451) 2026-06-02 14:59:10 +01:00
brios 2c0ebc28a7 perf(api): optimize static asset caching, enable ETag support, and expand response compression mime types. (#6273) 2026-06-01 16:37:56 +01:00
Ludy d1486c7762 ci(github-actions): replace deprecated app-id input with client-id (#6485)
# Description of Changes

This change updates the GitHub App token generation step in the custom
`setup-bot` GitHub Action to use the new `client-id` input instead of
the deprecated `app-id` input when invoking
`actions/create-github-app-token`.

### What was changed

- Replaced the deprecated `app-id` parameter with `client-id` in
`.github/actions/setup-bot/action.yml`.
- Continued sourcing the value from the existing `inputs.app-id` action
input to avoid broader interface changes.

### Why the change was made

- `actions/create-github-app-token` has deprecated the `app-id` input
and now expects `client-id`.
- Updating the workflow removes the deprecation warning and ensures
compatibility with current and future versions of the action.


---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-05-31 08:07:30 +01:00
Ludy 2ccff6f73f fix(update-service): correct GitHub branch reference for version retrieval (#6333) 2026-05-30 22:44:42 +01:00
Ludy 78da227eba fix: Use frontend/editor for locales paths (#6483) 2026-05-30 22:00:17 +01:00
Anthony Stirling 30e782e29c Disable Save-to-server when storage off, fix QR port 0 (#6473) 2026-05-29 19:37:42 +01:00
Anthony Stirling 2b0905887b Add desktop multi-window support (#6463) 2026-05-29 19:35:54 +01:00
ConnorYoh 28b81828b5 PAYG: PricingPolicyService + admin REST + 30s read cache (#6469)
## What this is

PR-I1 service half from `notes/PAYG_DESIGN.md`. Built on top of the data
model from #6460 — answers "what pricing policy applies to this team
right now?" with a fast cache and an admin write surface.

## Scope

| Piece | Where |
|---|---|
| `PricingPolicyService` — `getEffectivePolicy(teamId)` with 30s
Caffeine cache + admin write paths |
`app/saas/.../payg/policy/PricingPolicyService.java` |
| `PolicyChangedEvent` — published after admin writes for in-process
cache invalidation | `app/saas/.../payg/policy/PolicyChangedEvent.java`
|
| Admin REST — list / get / create / set-default / set team override /
get effective |
`app/saas/.../payg/policy/admin/PricingPolicyAdminController.java` +
DTOs |
| `PricingPolicyRepository.clearDefaultFlag()` — atomic clear for
set-default | repository update |
| `SaasJpaConfigScanTest` — drift guard against the JPA scan paths going
stale (carried over from the #6460 review concern) | new test |
| V12 default-policy seed (`v1-initial`, 25 pages/unit, 5 MiB/unit,
per-`JobSource` step limits) | `V12__seed_default_payg_policy.sql` |

## Lookup precedence

1. `PaygTeamExtensions.pricingPolicyId` set → return that policy
2. Else return the `pricing_policy` row with `is_default = TRUE`
3. Override row points at a deleted policy → log warn, fall back to
default (safety net for racing deletes)
4. No default → `IllegalStateException` (V12 seed guarantees one exists)

## Cache behaviour

- 30s `expireAfterWrite` Caffeine, max 10k entries, keyed by `teamId`.
- **Single correctness model: the TTL.** Cross-instance propagation is
at-most-30-seconds. The writer instance sees its own change immediately
via the `PolicyChangedEvent` after-commit publish. Other instances pick
it up on the next TTL expiry.
- Admin reads use `getEffectivePolicyUncached` so admins always see
their own write straight back.

**Why no LISTEN/NOTIFY runner.** An earlier cut of this PR included a
Postgres `LISTEN policy_changed` runner so cross-instance propagation
was instant. Dropped — admin policy changes are events-per-week and the
30s TTL is already the correctness floor; the listener was ~250 lines of
nontrivial code (raw JDBC outside HikariCP, daemon thread, reconnect
loop, lock-protected connection lifecycle) for a use case that isn't on
the hot path. Trade-off is documented in `notes/PAYG_DESIGN.md` §9 with
three concrete triggers that would justify reintroducing it (aggressive
cap enforcement, Redis landing for other reasons, real-time admin UI).

## Writes — transactional, fire `PolicyChangedEvent` after commit

- `create(draft)` — rejects pre-set `policy_id` or `is_default=true`
(promotion must go through `setDefault` so the partial unique idx is
freed first).
- `setDefault(id)` — atomically clears the existing default via
`clearDefaultFlag()` then flips the new row. Idempotent: silent no-op if
the row is already default.
- `setTeamOverride(teamId, policyId | null)` — validates the policy
exists before save; `null` clears the override.

`publishOnCommit` uses `TransactionSynchronizationManager.afterCommit`
so listeners never see pre-commit state. Outside a transaction (test
paths) falls through to immediate publish.

## Admin REST surface — `/api/v1/admin/payg/...`

All endpoints `@PreAuthorize("hasRole('ADMIN')")`:

- `GET  /policies` — list all
- `GET  /policies/{id}` — read one
- `POST /policies` — create new (non-default)
- `POST /policies/{id}/set-default` — atomic promote
- `PUT /teams/{teamId}/policy-override` — set or clear per-team override
- `GET  /teams/{teamId}/effective-policy` — cache-bypassing live read

Validation errors → 400, unknown rows → 404.

## Counterpart Supabase PR


[`Stirling-PDF-SaaS#298`](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/298)
— seeds the same V1 default policy on the Supabase side via
`20260528000002_payg_seed_default_policy.sql`.

## Tests

- 17 × `PricingPolicyServiceTest` — lookup precedence, cache hit/miss,
invalidation on event, mutation paths publishing event, error cases.
- 14 × `PricingPolicyAdminControllerTest` — every endpoint's happy path
+ error mapping, DTO defensive-copy invariant.
- 2 × `SaasJpaConfigScanTest` — reflection-based guard that
`payg.repository` is in `@EnableJpaRepositories` and `payg` is in
`@EntityScan`. Without this, new sub-packages can silently fail to wire
at runtime — same class of bug that the #6460 review caught.

Full `:saas:test` BUILD SUCCESSFUL.

## Design doc

`notes/PAYG_DESIGN.md` §7.4 PR-I1 — completes the service half (the
schema half landed in #6460). §9 carries the 30s-TTL trade-off note.
2026-05-29 16:27:01 +00:00
James Brunton 2c01f41142 Update indexeddb to v9 to unify SaaS and OSS users (#6474)
# Description of Changes
The production SaaS is currently on v8 of IndexedDB due to various
schema changes for Smart Folders, which haven't made their way into OSS.
OSS is currently on v4 of IndexedDB, so if we release an OSS build to
the SaaS deployment, existing users will not be able to use it because
the DB version is 'too old'.

This PR updates the IDB version number to v9 so both OSS and SaaS users
will be able to upgrade to it. Theoretically both types of user should
be able to keep their IDB files without issue. SaaS previously actively
wiped the user's files in an old version (v6/v7) and users who haven't
used it since then will have their DBs wiped, but that'd happen anyway
if they use current SaaS so I don't think that matters.
2026-05-29 15:23:40 +00:00
James Brunton 4d5eeb103f Fix username display issues (#6471)
# Description of Changes
Main fixes:
- Fix the display of the username in the bottom left
- Now displays as "User" when not logged in on self-hosted (desktop) and
"Guest" on SaaS when logged in anonymously
- Now updates properly when the user logs in/out in SaaS, desktop and
self-hosted
- Fix incremental build issues in the desktop app that have been here
since the start (I hope at least - I think the issue is that the JLink
is built read-only and then on subsequent builds you get OS errors when
trying to override the JLink with the new version. There's no real need
for it to be read-only that I know of, so we might as well just make it
R/W and ship like that)
2026-05-29 14:35:47 +00:00
ConnorYoh 83ea07ed6a saas: DocumentClassifier + PAYG data model (#6460)
# Description of Changes

Two layers — the `DocumentClassifier` utility plus the full data model
for the new billing engine. Nothing wires the entities into application
behaviour yet; services and controllers land in follow-up PRs.

**Companion PR:**
[Stirling-PDF-SaaS#296](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/296)
— Supabase migration for the v3 dev branch, schema-equivalent to the
Flyway migration in this PR.

## 1. DocumentClassifier (under `payg.docs`)

`DocumentClassifier` computes the doc-unit cost of an uploaded file (or
multi-file input) under a `PricingPolicy`. PDFs read page count via
`stirling.software.jpdfium.PdfDocument`; non-PDFs are bytes-only.
Formula: `max(ceil(pages / docPagesPerUnit), ceil(bytes /
docBytesPerUnit))` clamped to `[1, fileUnitCap]`. Multi-file is the sum
of raw per-file units capped at `fileUnitCap × file_count`.

Two floors, by design: the classifier returns `docUnits` with an
absolute `1` floor for non-empty input; the policy-level
`minChargeUnits` is intentionally applied later, at process-open time in
`JobChargeService`, per design § 3.4 (`unitsForProcess =
max(policy.min_charge_units, docUnits)`). Documented in the interface +
impl javadoc.

Upload bytes are materialised through
`TempFileManager.createManagedTempFile` so jpdfium gets a `Path`; the
temp file auto-deletes on close.

Twelve tests, all in-memory fixtures generated with PDFBox at test time
— no committed binary blobs.

## 2. PAYG data model (under `payg.*`)

JPA entities, repositories, and a Flyway migration covering the full
schema in §6 of the design.

**Enums** (`payg.model`):

`JobSource`, `ProcessType`, `JobStatus`, `JobStepStatus`,
`ArtifactKind`, `LedgerEntryType`, `LedgerBucket`, `ReferenceType`,
`EntitlementState`, `FeatureSet`, `FeatureGate`, `WalletEngine`,
`CapPeriod`, `AutoGroupStrategy`.

**Entities + repositories:**

| Entity | Table | Notes |
|---|---|---|
| `PricingPolicy` | `pricing_policy` | Promoted from a record.
`stepLimits` is `Map<JobSource, Integer>` persisted via normalised child
table `pricing_policy_step_limit`. `stripePriceIds` is `Set<String>`
persisted via `pricing_policy_stripe_price` — currency comes from
`stripe.prices` via Sync Engine, not stored locally. |
| `ProcessingJob` | `processing_job` | UUID PK. Tracks lineage window
via `step_count` and `last_step_at`. |
| `ProcessingJobStep` | `processing_job_step` | Per-tool-call audit. |
| `JobArtifactHash` | `job_artifact_hash` | Composite key `(job_id,
content_hash, kind)`. `content_hash VARCHAR(128)` so multiple signature
schemes coexist as `"type:value"` storage keys. Lineage detector queries
this. |
| `WalletLedgerEntry` | `wallet_ledger` | Append-only, signed
`amount_units`. Two unique indexes kill double-posting. |
| `WalletPolicy` | `wallet_policy` | Per-team engine + cap + degradation
rules + lineage strategy. No `@Version` — admin-only writes (documented
in javadoc). |
| `WalletEntitlementSnapshot` | `wallet_entitlement_snapshot` |
Composite key `(team_id, user_id)`; `user_id = 0` is the team-wide
sentinel. No `@Version` — full-row recompute via
`EntitlementService.recompute` (documented in javadoc). |
| `PaygShadowCharge` | `payg_shadow_charge` | Per-job diff while in
`PAYG_SHADOW` engine mode. |
| `PaygTeamExtensions` | `payg_team_extensions` | Sidecar 1:1 with
`teams` carrying `pricing_policy_id` (per-team override) +
`stripe_customer_id`. Sidecar pattern (mirrors `saas_team_extensions`)
so OSS Hibernate ddl-auto never sees PAYG columns on `teams`. |

**Column adds:**

- `team_memberships.cap_units` (optional per-member sub-cap)

**Width split (intentional, documented in V11):** per-row deltas
(`wallet_ledger.amount_units`, `processing_job.charged_units`) are
`INTEGER` because no single charge realistically approaches 2B units.
Cap and period-rollup columns (`team_memberships.cap_units`,
`wallet_policy.cap_units`,
`wallet_entitlement_snapshot.period_spend_units / period_cap_units`) are
`BIGINT` because they accumulate across a billing period and admins may
legitimately set headroom-cap values into the millions.

**JPA wiring:** `SaasJpaConfig` was updated to include
`stirling.software.saas.payg.repository` in
`@EnableJpaRepositories.basePackages` and `stirling.software.saas.payg`
in `@EntityScan` (covers `payg.policy` / `payg.job` / `payg.wallet` /
`payg.entitlement` / `payg.shadow` recursively). New
`SaasJpaConfigScanTest` reads the annotations reflectively and asserts
every expected package is wired — catches the next time someone adds a
new sub-package without updating the scan paths.

**Migration:** `V11__saas_payg_model.sql` (purely additive).
Schema-equivalent to the Supabase migration in the companion PR —
including the `VARCHAR(128) content_hash` width that's needed for the
multi-signature-scheme storage encoding the lineage layer uses.

## 3. Smoke tests

`PaygEntitiesSmokeTest` exercises each entity via the no-arg ctor JPA
requires, plus getter/setter round-trips and composite-key equality —
catches Lombok/annotation regressions without needing a database.
Real-DB integration coverage lands alongside the services that consume
each entity.

## Why this is safe to land now

- All schema changes are additive — no existing rows modified, no
columns dropped.
- The entities are not yet referenced from any production code path;
they exist for the next PRs to build on.
- The v3 Supabase dev branch picks up the schema via the companion PR;
the main repo's Flyway migration applies the same shape when an instance
boots against a freshly-migrated v3 database.

## Open decisions made

- **Step-limits keyed by `JobSource`** rather than by `ProcessType`.
Captures the "self-hosted gets a different knob" framing in earlier
feedback. Trivially overridable per pricing policy version.
- **Step limits + Stripe price IDs normalised into child tables** rather
than JSONB on `pricing_policy` (per Connor's review on #296). Typed
columns, queryable directly, no JSON parsing.
- **Currency dropped from `pricing_policy_stripe_price`** — it lives on
`stripe.prices.currency` and is resolved via Sync Engine. App is
currency-blind.

## Rollback

Straight `git revert` on this PR. The Supabase migration in #296 is
additive and can be left in place safely — the running app ignores
tables it doesn't reference.

---

## Checklist

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings
- [x] I have run `task check` (via `./gradlew :saas:test` with
`ENABLE_SAAS=true`) — passes
2026-05-29 12:03:01 +00:00
James Brunton 61ebe977d3 Auto-delete CI linting comments on success (#6465)
# Description of Changes
Set CI backend & engine comments to auto-delete once the CI has passed. 

Also redesign the engine CI to call `task engine:check` like it should
have been, and make it post a comment when the tool models need to be
updated.

Also makes the comment wording more consistent between the three
languages.
2026-05-29 10:13:12 +00:00
EthanHealy01 763595a5a3 feat: add Agents UI to proprietary right sidebar (#6454)
Update UI to include agents

Run `task dev:all` to test
2026-05-28 17:26:23 +00:00
Anthony Stirling 398617391b Fix SSO auto-login and custom metadata settings not persisting on restart (#6468) 2026-05-28 17:39:05 +01:00
ConnorYoh a0e0e88f07 saas: harden CreditService Stripe ordering + lint @AutoJobPostMapping weights (#6458)
# Description of Changes

Two narrowly-scoped hardening changes to the credits engine.

## 1. CreditService — move Stripe meter call to `afterCommit`

The Stripe metered-usage call sits inside the surrounding
`@Transactional`, holding the `user_credits` row lock for the duration
of an HTTP round-trip to Supabase. Under load this starves concurrent
debits; a transient Stripe blip rolls back a (correct) free-credit
consumption and forces the caller to retry.

The Stripe call now runs in a `TransactionSynchronization.afterCommit`
hook — DB commits first, Stripe fires immediately after. If Stripe fails
after commit, we log + increment a new `credits.stripe_report.failures`
counter; the idempotency key is stable, so a manual replay recovers
without double-charging.

Applied to both `consumeCreditBySupabaseId` and
`consumeCreditWithWaterfall`.

**Dead-code removed:**
- Unreachable UUID fallback for MDC `requestId` — `CorrelationIdFilter`
already guarantees the key on every request.
- The `"Unable to report usage to Stripe"` `RuntimeException` and its
catch block — the afterCommit refactor eliminates the throw path.
- `StripeRollbackOnFailureTest` — pinned the rollback-on-Stripe-fail
behaviour this refactor replaces.

## 2. `@AutoJobPostMapping` — build-time lint for `resourceWeight`

`UnifiedCreditInterceptor` multiplies `resourceWeight` into the per-call
charge. An endpoint that falls through to the annotation default
produces a charge derived from a value nobody chose.

- Annotation default flipped from `1` to `Integer.MIN_VALUE` (sentinel).
Both runtime readers (`UnifiedCreditInterceptor`, `AutoJobAspect`)
already clamp into `[1, 100]` so behaviour is unchanged.
- New `AutoJobPostMappingWeightTest` scans the classpath and fails the
build if any method leaves the sentinel.
- Initial run caught 11 endpoints relying on the default. Explicit
weights now declared, chosen by comparing to peer endpoints:
  - `EditTextController` — LARGE
  - `EmailController#sendEmailWithAttachment` — SMALL
  - `ConvertPDFToMarkdown` — MEDIUM
  - `AttachmentController` (extract/list/rename/delete) — SMALL × 4
  - `ConvertImgPDFController` (cbr/cbz ↔ pdf) — MEDIUM × 2, LARGE × 2

## Tests

- `StripeUsageIdempotencyKeyTest` — pins the `(supabaseId, overage,
requestId)` idempotency key shape so Stripe always dedupes a retry.
- `StripeAfterCommitOrderingTest` — pins that `afterCommit` fires after
commit and NOT on rollback.
- `AutoJobPostMappingWeightTest` — the lint itself, plus a self-check
that the classpath scan finds at least 10 `@AutoJobPostMapping` methods
(guards against the lint passing vacuously).

Build verified: `ENABLE_SAAS=true ./gradlew :stirling-pdf:test
:saas:test`.

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable) — no translation changes
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
— internal-billing change, no public docs impact
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
— N/A

### Translations (if applicable)

- [ ] Not applicable

### UI Changes (if applicable)

- [ ] Not applicable

### Testing (if applicable)

- [x] I have run `task check` (via `./gradlew :stirling-pdf:test
:saas:test` with `ENABLE_SAAS=true`) — passes
- [x] I have tested my changes locally
2026-05-28 14:57:59 +00:00
Anthony Stirling c80a5db5f5 folder and file fixes (#6461) 2026-05-28 15:57:35 +01:00
Anthony Stirling 4fa67afc3d Fix Tauri artifact copy path so installers upload (smoke + release) (#6466)
## Summary
Regression from #6404 (Restructure/frontend editor). Two CI workflows
copy the built installers to the wrong directory, so installer artifacts
(MSI / DMG / DEB / RPM / AppImage) silently vanish:

- **`tauri-build.yml`** (PR/desktop smoke builds) - uploads zero
installer artifacts.
- **`multiOSReleases.yml`** (production releases) - the empty artifacts
are downloaded by `create-release` and fed to `action-gh-release`, so a
release would publish **only the JARs, no desktop installers**.

## Root cause
#6404 moved the Tauri project from `frontend/` to `frontend/editor/` and
updated every **absolute** path (`projectPath`, `cd`, `Get-ChildItem`)
to add the `editor/` segment - but left the **relative** copy targets
`../../../dist`. Those resolve against the (now one level deeper)
working dir after `cd ./frontend/editor/src-tauri/target`:

| | resolves to |
|---|---|
| before #6404 (`frontend/src-tauri/target`) | repo-root `dist/`  |
| after #6404 (`frontend/editor/src-tauri/target`) | `frontend/dist/` 
(missing) |

The `cp` fails, repo-root `dist/` (from `mkdir -p ./dist`) stays empty,
and the upload finds nothing. `find -exec cp` failing is non-fatal, so
jobs still report success - that's why it went unnoticed. No release has
shipped broken yet: the last release (v2.11.0, 2026-05-19) predates
#6404 (2026-05-22).

## Fix
Copy to an absolute `$GITHUB_WORKSPACE/dist` in both workflows so the
`cd` can't drift the destination again. This matches where the upload /
signature-verify steps already read from.

## Evidence (run 26574078559, all 3 OS legs)
```
cp: cannot create regular file '../../../dist/Stirling-PDF-windows-x86_64.msi': No such file or directory
##[warning]No files were found with the provided path: ./dist/*. No artifacts will be uploaded.
```
The Tauri builds themselves succeeded - only the copy/upload was broken.

## Test plan
- [ ] `tauri-build` on this PR uploads non-empty `Stirling-PDF-<name>`
artifacts on Windows/macOS/Linux.
- [ ] Next release (or a `workflow_dispatch` of multiOSReleases)
attaches MSI/DMG/DEB/RPM/AppImage to the release.
2026-05-28 15:57:01 +01:00
Anthony StirlingandConnorYoh 8bd78d2624 Add landscape page size options (#6248)
# Description of Changes

Adds orientation (portrait/landscape) to the Adjust Page Scale tool.

- Orientation as a separate parameter (per review), sent through to the
backend
- ScalePagesController simplified; PDFWithPageSize gains the orientation
field
- Regenerated tool_models.py; frontend + backend tests added

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

---------

Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
2026-05-28 14:16:15 +00:00
Anthony Stirling b3c4b8b463 Add S3 storage and cluster artifact backend (#6457) 2026-05-28 13:06:27 +01:00
James Brunton 57af5b9dc2 Fix Tauri testing (#6462)
# Description of Changes
#6402 introduced a Rust test `refresh_token_fallback.rs`, but it wasn't
moved properly after the restructure of the `frontend/` folder in #6404.
This PR moves the file to the right place, and also hooks up Task and CI
rules for `cargo test` since nothing was actually running the test in
the first place.
2026-05-28 11:05:56 +00:00
James Brunton 44fbf8c587 Various bug fixes found while testing SaaS build (#6459)
# Description of Changes
Various fixes and improvements I made while testing the SaaS code:
- Changes the new `.env.saas` file to live in `app/` and match the
semantics of the other `.env` files
- Adds top-level `task dev:saas` command to spawn SaaS frontend &
backend
- Deletes dead SaaS code and improves some overriding logic
- Fixes refreshing issue when coming back to the tab
- Fix the Compare tool's selection logic
- Make Compare handle error cases properly
- Fixes the location of the "Dismiss All Errors" button (was rendering
on top of the top-bar with a transparent background previously so it
looked rubbish)
- Fixes file selection in PDF Editor
2026-05-28 11:05:30 +00:00
Anthony Stirling 76840d8a57 Add CI DB migration smoke test against v2.0/v2.5/v2.10 updates (#6453) 2026-05-28 11:36:07 +01:00
James Brunton d459ded168 Add cancel button to kill long-running AI tasks (#6351)
# Description of Changes
Adds a cancel button to the AI chat to allow the user to abort
long-running AI tasks. Just disconnects the SSE stream (all the backend
code already interrupts when it notices the stream is dead).
2026-05-28 09:25:23 +00:00
ConnorYoh 43b67d213d feat(oauth2): opt-in claim-dump diagnostics for OIDC login failures (#6456)
# Description of Changes

## What & why

Customers using ADFS (or any generic OIDC provider that doesn't emit
`email`) hit `Attribute value for 'email' cannot be null` during OAuth2
login with no visibility into what claims the provider actually sent.
The only available remedy was guessing at
`security.oauth2.useAsUsername` until something worked.

This PR adds a new opt-in `security.oauth2.debugLogging` flag (default
`false`). When enabled, `CustomOAuth2UserService` logs:

- All ID token claims (sorted, with values)
- All UserInfo endpoint claims (if any)
- The merged attribute key set Spring exposes to `getAttribute()`
- The value the configured `useAsUsername` actually resolved to
- A **`Hint:`** line listing the claim keys present in the token that
map to a valid `UsernameAttribute` enum value — i.e. exactly what the
operator could put in `useAsUsername` to make login work

Logged at `INFO` on the success path and `ERROR` on failure (inside the
existing `catch (IllegalArgumentException)` block that throws
`OAuth2AuthenticationException`). The block is wrapped with a `[OAUTH2
DEBUG] ... [/OAUTH2 DEBUG]` banner and ends with a PII warning so
operators don't leave it on in production.

Default off → zero observable change for anyone not actively
troubleshooting.

## Files changed

| File | Why |
|---|---|
| `app/common/.../ApplicationProperties.java` | New `debugLogging` field
on the `OAUTH2` config class with javadoc warning about PII |
| `app/core/src/main/resources/settings.yml.template` | Documents
`oauth2.debugLogging` so it appears on next startup |
| `app/proprietary/.../security/service/CustomOAuth2UserService.java` |
Emits the claim dump + suggestion hint when the flag is on |
|
`app/proprietary/.../security/service/CustomOAuth2UserServiceDebugLoggingTest.java`
(new) | Unit test: mocks the OIDC delegate, asserts off-path is silent
and on-path emits the dump with the right Hint contents |

## End-to-end verification

Ran the bundled `testing/compose/docker-compose-keycloak-oauth.yml`
Keycloak realm, configured `security.oauth2.useAsUsername: mail`
(Keycloak emits `email`, not `mail`) and `provider: demarest` (matches
the original customer bug report). Triggered the OAuth flow at
`http://localhost:8080/oauth2/authorization/demarest` and confirmed:

- The ERROR-level dump fires with the full 19-claim ID token decoded
- `-- Value at 'mail' : <NULL — this is why login fails>` correctly
identifies the missing claim
- `-- Hint:` correctly suggests `[email, family_name, given_name,
preferred_username]` (the four keys present that map to valid
`UsernameAttribute` values)
- Auth still fails with the original `OAuth2AuthenticationException` —
no change to control flow, just added diagnostic logging

Unit test (`CustomOAuth2UserServiceDebugLoggingTest`) covers both
branches.

## Reviewer notes

- **No new public APIs.** The flag is config-only; no servlet endpoints
exposed.
- **PII is logged when the flag is on.** This is the whole point —
operators need to see the claims to fix their config — but it's gated,
defaults off, and the dump self-documents with a `WARNING: ... Set
security.oauth2.debugLogging=false once troubleshooting is complete.`
footer.
- **Why log everything, not just sub/email?** Because the operator
doesn't know in advance which claim they actually want. ADFS uses `upn`
in some configs and `preferred_username` in others; Azure AD uses `oid`;
the customer here had neither. Dumping the full set is the only way to
make the diagnostic self-service.
- **Out of scope for this PR (follow-ups):**
- The `UsernameAttribute` enum doesn't include `upn` / `unique_name`
(common ADFS claims). If the customer's token only has `upn`, the Hint
will be empty even though the operator can see `upn` in the dump. Worth
a separate PR to extend the enum.
- The known-provider validator in `Provider.java` (rejects e.g.
`useAsUsername: mail` for `provider: keycloak` at startup) bypasses our
diagnostic for those provider names. ADFS customers using `provider:
<name>` fall into the `default` branch so are not affected — but it's a
sharp edge worth documenting.

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable) — N/A, backend-only change
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] Doc-repo update (if functionality has heavily changed) —
diagnostic flag is self-documenting via the `settings.yml.template`
comment and the in-log warning; happy to add a doc-repo entry if
reviewers want one
- [ ] Translation tags — N/A

### UI Changes (if applicable)

- [ ] N/A — backend-only

### Testing (if applicable)

- [x] Unit test added (`CustomOAuth2UserServiceDebugLoggingTest`)
covering on/off paths and Hint correctness
- [x] End-to-end verified locally against bundled Keycloak compose with
intentionally misconfigured `useAsUsername`
- [x] Full `:proprietary:test` suite passes
2026-05-27 13:01:51 +00:00
Anthony Stirling d42b779644 Add server-side folders and files page UI (#6383) 2026-05-27 12:52:46 +01:00
1930 changed files with 223010 additions and 25047 deletions
+4 -4
View File
@@ -20,8 +20,8 @@ set -e
# - To build the project, use:
# ./gradlew build
#
# - For running pre-commit hooks (if configured), use:
# pre-commit run --all-files
# - To run the lint/format/secret checks, use:
# task pre-commit
#
# Make sure you are in the project root directory after this script executes.
# =============================================================================
@@ -70,6 +70,6 @@ echo ""
echo " To build the project: "
echo -e "\e[34m gradle build\e[0m"
echo ""
echo " To run pre-commit hooks (if configured):"
echo -e "\e[34m pre-commit run --all-files -c .pre-commit-config.yaml\e[0m"
echo " To run the lint/format/secret checks:"
echo -e "\e[34m task pre-commit\e[0m"
echo "=================================================================="
+3 -2
View File
@@ -26,8 +26,9 @@ version_builds/
node_modules/
**/node_modules/
frontend/node_modules/
frontend/dist/
frontend/playwright-report/
frontend/editor/dist/
frontend/dist-portal/
frontend/editor/playwright-report/
.npm/
.yarn/
+1 -1
View File
@@ -24,7 +24,7 @@ runs:
id: generate-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ inputs.app-id }}
client-id: ${{ inputs.app-id }}
private-key: ${{ inputs.private-key }}
- name: Configure Git
run: |
+1 -1
View File
@@ -1,6 +1,6 @@
# Maintainer: Stirling PDF Inc <contact@stirlingpdf.com>
pkgname=stirling-pdf-desktop
pkgver=2.11.0
pkgver=2.13.0
pkgrel=1
pkgdesc="Locally hosted, web-based PDF manipulation tool (Tauri desktop app, official Stirling PDF Inc build)"
arch=('x86_64')
+1 -1
View File
@@ -1,6 +1,6 @@
# Maintainer: Stirling PDF Inc <contact@stirlingpdf.com>
pkgname=stirling-pdf-server-bin
pkgver=2.11.0
pkgver=2.13.0
pkgrel=1
pkgdesc="Locally hosted, web-based PDF manipulation tool (server JAR, prebuilt)"
arch=('any')
+15 -1
View File
@@ -1,6 +1,8 @@
build: &build
- build.gradle
- app/(common|core|proprietary)/build.gradle
- Taskfile.yml
- .taskfiles/backend.yml
openapi: &openapi
- *build
@@ -38,6 +40,11 @@ project: &project
- frontend/**
- docker/**
- scripts/RestartHelper.java
- Taskfile.yml
- .taskfiles/backend.yml
- .taskfiles/docker.yml
- scripts/db-migration/**
- .github/workflows/db-migration-test.yml
frontend: &frontend
- frontend/**
@@ -53,6 +60,9 @@ frontend: &frontend
- scripts/summarize_type3_signatures.py
- scripts/type3_to_cff.py
- scripts/update_type3_library.py
- Taskfile.yml
- .taskfiles/frontend.yml
- .taskfiles/e2e.yml
# Files that affect the Tauri desktop bundle. Gate the multi-OS Tauri build
# job on changes to any of these.
@@ -64,6 +74,8 @@ tauri: &tauri
- frontend/package-lock.json
- frontend/editor/vite.config.ts
- .github/workflows/tauri-build.yml
- Taskfile.yml
- .taskfiles/desktop.yml
# Files that affect the AI engine (Python tool models, fixers, tests). Gate
# the engine validation job on changes to engine sources or to the Java
@@ -72,6 +84,8 @@ engine: &engine
- engine/**
- app/(common|core|proprietary)/src/main/java/**
- .github/workflows/ai-engine.yml
- Taskfile.yml
- .taskfiles/engine.yml
licenses-frontend: &licenses-frontend
- ".github/workflows/frontend-backend-licenses-update.yml"
@@ -100,4 +114,4 @@ proprietary: &proprietary
- configs/settings.yml.template
- build.gradle
- app/proprietary/build.gradle
- .github/workflows/build-enterprise.yml
- .github/workflows/build-enterprise.yml
+5 -4
View File
@@ -13,7 +13,7 @@ Usage:
"""
# Sample for Windows:
# python .github/scripts/check_language_toml.py --reference-file frontend/public/locales/en-GB/translation.toml --branch "" --files frontend/public/locales/de-DE/translation.toml frontend/public/locales/fr-FR/translation.toml
# python .github/scripts/check_language_toml.py --reference-file frontend/editor/public/locales/en-US/translation.toml --branch "" --files frontend/editor/public/locales/de-DE/translation.toml frontend/editor/public/locales/fr-FR/translation.toml
import argparse
import glob
@@ -184,7 +184,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
if len(file_list) == 1:
file_arr = file_list[0].split()
base_dir = Path.cwd() / "frontend" / "public" / "locales"
base_dir = Path.cwd() / "frontend" / "editor" / "public" / "locales"
for file_path in file_arr:
file_path = Path(file_path)
@@ -211,7 +211,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
)
continue
if basename_current_file == basename_reference_file and locale_dir == "en-GB":
if basename_current_file == basename_reference_file and locale_dir == "en-US":
continue
if (
@@ -308,7 +308,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
report.append("## ❌ Overall Check Status: **_Failed_**")
report.append("")
report.append(
f"@{actor} please check your translation if it conforms to the standard. Follow the format of [en-GB/translation.toml](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/frontend/public/locales/en-GB/translation.toml)"
f"@{actor} please check your translation if it conforms to the standard. Follow the format of [en-US/translation.toml](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/frontend/editor/public/locales/en-US/translation.toml)"
)
else:
report.append("## ✅ Overall Check Status: **_Success_**")
@@ -372,6 +372,7 @@ if __name__ == "__main__":
os.path.join(
os.getcwd(),
"frontend",
"editor",
"public",
"locales",
"*",
@@ -1 +0,0 @@
pre-commit
-121
View File
@@ -1,121 +0,0 @@
#
# This file is autogenerated by pip-compile with Python 3.12
# by the following command:
#
# pip-compile --generate-hashes --output-file='.github\scripts\requirements_pre_commit.txt' --strip-extras '.github\scripts\requirements_pre_commit.in'
#
cfgv==3.5.0 \
--hash=sha256:a8dc6b26ad22ff227d2634a65cb388215ce6cc96bbcc5cfde7641ae87e8dacc0 \
--hash=sha256:d5b1034354820651caa73ede66a6294d6e95c1b00acc5e9b098e917404669132
# via pre-commit
distlib==0.4.0 \
--hash=sha256:9659f7d87e46584a30b5780e43ac7a2143098441670ff0a49d5f9034c54a6c16 \
--hash=sha256:feec40075be03a04501a973d81f633735b4b69f98b05450592310c0f401a4e0d
# via virtualenv
filelock==3.29.0 \
--hash=sha256:69974355e960702e789734cb4871f884ea6fe50bd8404051a3530bc07809cf90 \
--hash=sha256:96f5f6344709aa1572bbf631c640e4ebeeb519e08da902c39a001882f30ac258
# via
# python-discovery
# virtualenv
identify==2.6.19 \
--hash=sha256:20e6a87f786f768c092a721ad107fc9df0eb89347be9396cadf3f4abbd1fb78a \
--hash=sha256:6be5020c38fcb07da56c53733538a3081ea5aa70d36a156f83044bfbf9173842
# via pre-commit
nodeenv==1.10.0 \
--hash=sha256:5bb13e3eed2923615535339b3c620e76779af4cb4c6a90deccc9e36b274d3827 \
--hash=sha256:996c191ad80897d076bdfba80a41994c2b47c68e224c542b48feba42ba00f8bb
# via pre-commit
platformdirs==4.9.6 \
--hash=sha256:3bfa75b0ad0db84096ae777218481852c0ebc6c727b3168c1b9e0118e458cf0a \
--hash=sha256:e61adb1d5e5cb3441b4b7710bea7e4c12250ca49439228cc1021c00dcfac0917
# via
# python-discovery
# virtualenv
pre-commit==4.6.0 \
--hash=sha256:718d2208cef53fdc38206e40524a6d4d9576d103eb16f0fec11c875e7716e9d9 \
--hash=sha256:e2cf246f7299edcabcf15f9b0571fdce06058527f0a06535068a86d38089f29b
# via -r .github/scripts/requirements_pre_commit.in
python-discovery==1.2.2 \
--hash=sha256:876e9c57139eb757cb5878cbdd9ae5379e5d96266c99ef731119e04fffe533bb \
--hash=sha256:e1ae95d9af875e78f15e19aed0c6137ab1bb49c200f21f5061786490c9585c7a
# via virtualenv
pyyaml==6.0.3 \
--hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \
--hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \
--hash=sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3 \
--hash=sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956 \
--hash=sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6 \
--hash=sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c \
--hash=sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65 \
--hash=sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a \
--hash=sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0 \
--hash=sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b \
--hash=sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1 \
--hash=sha256:22ba7cfcad58ef3ecddc7ed1db3409af68d023b7f940da23c6c2a1890976eda6 \
--hash=sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7 \
--hash=sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e \
--hash=sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007 \
--hash=sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310 \
--hash=sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4 \
--hash=sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9 \
--hash=sha256:3ff07ec89bae51176c0549bc4c63aa6202991da2d9a6129d7aef7f1407d3f295 \
--hash=sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea \
--hash=sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0 \
--hash=sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e \
--hash=sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac \
--hash=sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9 \
--hash=sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7 \
--hash=sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35 \
--hash=sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb \
--hash=sha256:5cf4e27da7e3fbed4d6c3d8e797387aaad68102272f8f9752883bc32d61cb87b \
--hash=sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69 \
--hash=sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5 \
--hash=sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b \
--hash=sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c \
--hash=sha256:6344df0d5755a2c9a276d4473ae6b90647e216ab4757f8426893b5dd2ac3f369 \
--hash=sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd \
--hash=sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824 \
--hash=sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198 \
--hash=sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065 \
--hash=sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c \
--hash=sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c \
--hash=sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764 \
--hash=sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196 \
--hash=sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b \
--hash=sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00 \
--hash=sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac \
--hash=sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8 \
--hash=sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e \
--hash=sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28 \
--hash=sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3 \
--hash=sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5 \
--hash=sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4 \
--hash=sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b \
--hash=sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf \
--hash=sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5 \
--hash=sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702 \
--hash=sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8 \
--hash=sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788 \
--hash=sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da \
--hash=sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d \
--hash=sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc \
--hash=sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c \
--hash=sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba \
--hash=sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f \
--hash=sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917 \
--hash=sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5 \
--hash=sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26 \
--hash=sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f \
--hash=sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b \
--hash=sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be \
--hash=sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c \
--hash=sha256:efd7b85f94a6f21e4932043973a7ba2613b059c4a000551892ac9f1d11f5baf3 \
--hash=sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6 \
--hash=sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926 \
--hash=sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0
# via pre-commit
virtualenv==21.2.4 \
--hash=sha256:29d21e941795206138d0f22f4e45ff7050e5da6c6472299fb7103318763861ac \
--hash=sha256:b294ef68192638004d72524ce7ef303e9d0cf5a44c95ce2e54a7500a6381cada
# via pre-commit
@@ -0,0 +1,83 @@
#!/usr/bin/env python3
"""Verify Tauri updater .sig files against plugins.updater.pubkey in tauri.conf.json.
Usage: verify-updater-signatures.py <dir-to-scan> [tauri.conf.json]
"""
import binascii
import sys
import json
import base64
import hashlib
from pathlib import Path
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
from cryptography.exceptions import InvalidSignature
ART_ROOT = Path(sys.argv[1])
CONF = Path(
sys.argv[2] if len(sys.argv) > 2 else "frontend/editor/src-tauri/tauri.conf.json"
)
def load_pubkey():
# tauri pubkey = base64 of a minisign .pub file; last line is base64 of
# [2 algo][8 key-id][32 ed25519 public key].
raw = json.loads(CONF.read_text())["plugins"]["updater"]["pubkey"]
blob = base64.b64decode(base64.b64decode(raw).decode().splitlines()[-1])
return blob[2:10], Ed25519PublicKey.from_public_bytes(blob[10:])
def hash_file(path: Path) -> bytes:
h = hashlib.blake2b(digest_size=64)
with path.open("rb") as f:
for chunk in iter(lambda: f.read(1 << 16), b""):
h.update(chunk)
return h.digest()
def verify(artifact: Path, sig_file: Path, keyid_pub, pub) -> str:
# tauri .sig = base64 of a minisign signature file (4 lines).
try:
lines = base64.b64decode(sig_file.read_text()).decode().splitlines()
sig_blob = base64.b64decode(lines[1])
except (binascii.Error, IndexError, UnicodeDecodeError) as e:
return f"FAIL malformed sig ({type(e).__name__})"
algo, keyid, sig = sig_blob[:2], sig_blob[2:10], sig_blob[10:74]
if keyid != keyid_pub:
return f"FAIL key-id mismatch (sig {keyid.hex()} vs pub {keyid_pub.hex()})"
# 'ED' = prehashed (BLAKE2b-512), 'Ed' = legacy (raw message).
msg = hash_file(artifact) if algo == b"ED" else artifact.read_bytes()
try:
pub.verify(sig, msg)
except InvalidSignature:
return f"FAIL signature invalid (algo={algo.decode()})"
# Global signature covers sig + trusted_comment.
gc = "global-sig FAIL"
try:
tc = lines[2].split("trusted comment: ", 1)[1]
pub.verify(base64.b64decode(lines[3]), sig + tc.encode())
gc = "global-sig OK"
except (InvalidSignature, IndexError, binascii.Error):
pass
return f"VALID (algo={algo.decode()}, keyid={keyid.hex()}, {gc})"
keyid_pub, pub = load_pubkey()
print(f"updater pubkey keyid={keyid_pub.hex()}\n")
sigs = sorted(ART_ROOT.rglob("*.sig"))
if not sigs:
print(f"WARN: no .sig files under {ART_ROOT} - nothing to verify")
sys.exit(0)
bad = 0
for sig_file in sigs:
artifact = sig_file.with_suffix("")
if not artifact.exists():
print(f" ? {sig_file.name}: artifact missing")
bad += 1
continue
res = verify(artifact, sig_file, keyid_pub, pub)
print(f" {artifact.name}: {res}")
if not res.startswith("VALID") or "global-sig FAIL" in res:
bad += 1
print(f"\n{'ALL SIGNATURES VALID' if bad == 0 else f'{bad} SIGNATURE(S) FAILED'}")
sys.exit(1 if bad else 0)
+4 -3
View File
@@ -239,7 +239,7 @@ jobs:
- name: Build and push V2 image (Depot)
if: env.USE_DEPOT == 'true' && steps.check-image.outputs.exists == 'false'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
@@ -287,12 +287,13 @@ jobs:
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/data:/usr/share/tessdata:rw
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/config:/configs:rw
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/logs:/logs:rw
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/storage:/storage:rw
environment:
DISABLE_ADDITIONAL_FEATURES: "false"
SECURITY_ENABLELOGIN: "true"
SECURITY_INITIALLOGIN_USERNAME: "${{ secrets.TEST_LOGIN_USERNAME }}"
SECURITY_INITIALLOGIN_PASSWORD: "${{ secrets.TEST_LOGIN_PASSWORD }}"
SYSTEM_DEFAULTLOCALE: en-GB
SYSTEM_DEFAULTLOCALE: en-US
UI_APPNAME: "Stirling-PDF V2 PR#${{ needs.check-pr.outputs.pr_number }}"
UI_HOMEDESCRIPTION: "V2 PR#${{ needs.check-pr.outputs.pr_number }} - Embedded Architecture"
UI_APPNAMENAVBAR: "V2 PR#${{ needs.check-pr.outputs.pr_number }}"
@@ -309,7 +310,7 @@ jobs:
ssh -i ../private.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -T ${{ secrets.NEW_VPS_USERNAME }}@${{ secrets.NEW_VPS_HOST }} << ENDSSH
# Create V2 PR-specific directories
mkdir -p /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/{data,config,logs}
mkdir -p /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/{data,config,logs,storage}
# Move docker-compose file to correct location
mv /tmp/docker-compose-v2.yml /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/docker-compose.yml
@@ -222,10 +222,10 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Run Gradle Command
run: |
if [ "${{ needs.check-comment.outputs.disable_security }}" == "true" ]; then
@@ -256,7 +256,7 @@ jobs:
- name: Build and push PR-specific image (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
@@ -285,7 +285,7 @@ jobs:
- name: Build and push engine image (Depot)
if: env.USE_DEPOT == 'true' && needs.check-comment.outputs.enable_prototypes == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: ./engine
@@ -388,7 +388,7 @@ jobs:
environment:
DISABLE_ADDITIONAL_FEATURES: "${DISABLE_ADDITIONAL_FEATURES}"
SECURITY_ENABLELOGIN: "${LOGIN_SECURITY}"
SYSTEM_DEFAULTLOCALE: en-GB
SYSTEM_DEFAULTLOCALE: en-US
UI_APPNAME: "Stirling-PDF PR#${PR_NUMBER}"
UI_HOMEDESCRIPTION: "PR#${PR_NUMBER} for Stirling-PDF Latest"
UI_APPNAMENAVBAR: "PR#${PR_NUMBER}"
+126 -29
View File
@@ -1,8 +1,9 @@
name: AI Engine CI
# Validates the Python AI engine: regenerates tool models, runs fixers,
# lint, type-check, and tests. Called from build.yml on PRs and merge_group;
# also runs directly on push to main as a post-merge safety net.
# Validates the Python AI engine: regenerates tool models and runs the
# engine quality gate (lint, type-check, format-check, tests). Called from
# build.yml on PRs and merge_group; also runs directly on push to main as
# a post-merge safety net.
on:
workflow_call:
push:
@@ -42,36 +43,104 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Regenerate tool models
run: task engine:tool-models
- name: Verify tool models are up to date
id: tool-models-check
continue-on-error: true
run: git diff --exit-code engine/src/stirling/models/tool_models.py
- name: Comment on tool models check failure
# Only post a comment on PRs. github-script's PR helpers need an
# issue/PR number, which doesn't exist on merge_group runs.
if: steps.tool-models-check.outcome == 'failure' && github.event_name == 'pull_request'
continue-on-error: true
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const marker = '<!-- tool-models-check -->';
const body = [
marker,
'### Tool Models Check Failed',
'',
'The generated `engine/src/stirling/models/tool_models.py` is out of date with the Java OpenAPI spec and will need to be regenerated before it can be merged in.',
'',
'Run `task engine:tool-models` to regenerate, then commit the updated file.',
].join('\n');
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body.includes(marker));
if (existing) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
body,
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body,
});
}
- name: Fail if tool models check failed
if: steps.tool-models-check.outcome == 'failure'
run: |
if ! git diff --exit-code engine/src/stirling/models/tool_models.py; then
echo "tool_models.py is out of date."
echo "Run 'task engine:tool-models' locally and commit the updated file."
exit 1
fi
echo "============================================"
echo " Tool Models Check Failed"
echo "============================================"
echo ""
echo "The generated engine/src/stirling/models/tool_models.py"
echo "is out of date with the Java OpenAPI spec and will"
echo "need to be regenerated before it can be merged in."
echo ""
echo "Run 'task engine:tool-models' to regenerate, then"
echo "commit the updated file."
echo "============================================"
exit 1
- name: Run fixers
run: task engine:fix
- name: Remove tool models check comment on success
if: steps.tool-models-check.outcome == 'success' && github.event_name == 'pull_request'
continue-on-error: true
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const marker = '<!-- tool-models-check -->';
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body.includes(marker));
if (existing) {
await github.rest.issues.deleteComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
});
}
- name: Verify fixes are committed
id: fixer_changes
run: |
if ! git diff --quiet; then
git --no-pager diff --stat
echo "::error::There are issues with your Python code that will need to be fixed before they can be merged in. Run 'task engine:fix' to auto-fix what can be fixed automatically, then run 'task engine:check' to see what still needs fixing manually."
exit 1
fi
- name: Quality-check engine
id: engine-check
run: task engine:check
continue-on-error: true
- name: Comment on fixer failures
if: steps.fixer_changes.outcome == 'failure' && github.event_name == 'pull_request'
- name: Comment on engine check failure
# Only post a comment on PRs. github-script's PR helpers need an
# issue/PR number, which doesn't exist on merge_group runs.
if: steps.engine-check.outcome == 'failure' && github.event_name == 'pull_request'
continue-on-error: true
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
@@ -107,11 +176,39 @@ jobs:
});
}
- name: Run linting
run: task engine:lint
- name: Fail if engine check failed
if: steps.engine-check.outcome == 'failure'
run: |
echo "============================================"
echo " Engine Check Failed"
echo "============================================"
echo ""
echo "There are issues with your Python code that"
echo "will need to be fixed before they can be merged in."
echo ""
echo "Run 'task engine:fix' to auto-fix what can be"
echo "fixed automatically, then run 'task engine:check'"
echo "to see what still needs fixing manually."
echo "============================================"
exit 1
- name: Run type checking
run: task engine:typecheck
- name: Run tests
run: task engine:test
- name: Remove engine check comment on success
if: steps.engine-check.outcome == 'success' && github.event_name == 'pull_request'
continue-on-error: true
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const marker = '<!-- engine-check -->';
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body.includes(marker));
if (existing) {
await github.rest.issues.deleteComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
});
}
+100 -24
View File
@@ -1,8 +1,14 @@
name: Backend build, format check, and coverage
# Reusable workflow called from build.yml. Runs the backend build matrix
# (JDK 25 × spring-security on/off), Spotless formatting check, JUnit, and
# (JDK 25 × every flavor), Spotless formatting check, JUnit, and
# posts Jacoco coverage to PRs.
#
# Flavor axis (maps to STIRLING_FLAVOR in settings.gradle):
# core - DISABLE_ADDITIONAL_FEATURES=true, no proprietary, no saas
# proprietary - default build, no saas
# saas - proprietary + the saas subproject (build + JUnit only,
# never any runtime/integration testing)
on:
workflow_call:
@@ -25,7 +31,7 @@ jobs:
fail-fast: false
matrix:
jdk-version: [25]
spring-security: [true, false]
flavor: [core, proprietary, saas]
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -52,13 +58,16 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
cache-disabled: true
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Check Java formatting (Spotless)
if: matrix.jdk-version == 25 && matrix.spring-security == false
# Runs once per matrix combination - pick the cheapest leg
# (core - no proprietary, no saas) so we don't wait for the
# heavier flavors just to fail formatting.
if: matrix.jdk-version == 25 && matrix.flavor == 'core'
id: spotless-check
run: task backend:format:check
continue-on-error: true
@@ -67,7 +76,7 @@ jobs:
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
- name: Comment on Java formatting failure
- name: Comment on backend format check failure
# Only post a comment on PRs. github-script's PR helpers need an
# issue/PR number, which doesn't exist on merge_group runs.
if: steps.spotless-check.outcome == 'failure' && github.event_name == 'pull_request'
@@ -78,15 +87,11 @@ jobs:
const marker = '<!-- java-formatting-check -->';
const body = [
marker,
'### Java Formatting Check Failed',
'### Backend Format Check Failed',
'',
'Your code has formatting issues. Run the following command to fix them:',
'There are formatting issues in your Java code that will need to be fixed before they can be merged in.',
'',
'```bash',
'task backend:format',
'```',
'',
'Then commit and push the changes.',
'Run `task backend:format` to auto-fix, then commit and push the changes.',
].join('\n');
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
@@ -110,33 +115,61 @@ jobs:
});
}
- name: Fail if Java formatting issues found
- name: Fail if backend format check failed
if: steps.spotless-check.outcome == 'failure'
run: |
echo "============================================"
echo " Java Formatting Check Failed"
echo " Backend Format Check Failed"
echo "============================================"
echo ""
echo "Your code has formatting issues."
echo "Run the following command to fix them:"
echo "There are formatting issues in your Java code"
echo "that will need to be fixed before they can be"
echo "merged in."
echo ""
echo " task backend:format"
echo ""
echo "Then commit and push the changes."
echo "Run 'task backend:format' to auto-fix, then"
echo "commit and push the changes."
echo "============================================"
exit 1
- name: Build with Gradle and spring security ${{ matrix.spring-security }}
- name: Remove backend format check comment on success
if: steps.spotless-check.outcome == 'success' && github.event_name == 'pull_request'
continue-on-error: true
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const marker = '<!-- java-formatting-check -->';
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body.includes(marker));
if (existing) {
await github.rest.issues.deleteComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
});
}
- name: Build with Gradle (flavor=${{ matrix.flavor }})
# STIRLING_FLAVOR is read by settings.gradle and expands into the
# right combination of DISABLE_ADDITIONAL_FEATURES + ENABLE_SAAS
# so we don't have to set them by hand. The saas flavor pulls in
# the app/saas subproject (unit tests only - no runtime tests).
run: task backend:build:ci
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
DISABLE_ADDITIONAL_FEATURES: ${{ matrix.spring-security }}
STIRLING_FLAVOR: ${{ matrix.flavor }}
- name: Check Test Reports Exist
if: always()
run: |
# Common + core + proprietary always build (proprietary is
# excluded only at runtime, not from the gradle subproject
# graph). Saas builds add a fourth report dir.
declare -a dirs=(
"app/core/build/reports/tests/"
"app/core/build/test-results/"
@@ -145,6 +178,9 @@ jobs:
"app/proprietary/build/reports/tests/"
"app/proprietary/build/test-results/"
)
if [ "${{ matrix.flavor }}" = "saas" ]; then
dirs+=("app/saas/build/reports/tests/" "app/saas/build/test-results/")
fi
for dir in "${dirs[@]}"; do
if [ ! -d "$dir" ]; then
echo "Missing $dir"
@@ -156,7 +192,7 @@ jobs:
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: test-reports-jdk-${{ matrix.jdk-version }}-spring-security-${{ matrix.spring-security }}
name: test-reports-jdk-${{ matrix.jdk-version }}-flavor-${{ matrix.flavor }}
path: |
app/**/build/reports/jacoco/test
app/**/build/reports/tests/
@@ -166,7 +202,47 @@ jobs:
retention-days: 3
if-no-files-found: warn
- name: Add coverage to PR with spring security ${{ matrix.spring-security }} and JDK ${{ matrix.jdk-version }}
- name: Install defusedxml for coverage summary
# coverage-summary.py parses JaCoCo XML through defusedxml to
# silence security scanners that pattern-match on the stdlib
# xml.etree.ElementTree.parse call.
if: always() && matrix.flavor == 'saas'
run: python -m pip install --quiet defusedxml
- name: JaCoCo coverage step summary
# Only the saas leg posts the JUnit summary - it's a strict
# superset of the core + proprietary legs (same .exec files plus
# the saas subproject). Posting from all three would mean three
# near-identical tables crowding out the aggregate report.
if: always() && matrix.flavor == 'saas'
run: |
python scripts/coverage-summary.py \
--title "Backend JUnit coverage (JDK ${{ matrix.jdk-version }})" \
--jacoco "common=app/common/build/reports/jacoco/test/jacocoTestReport.xml" \
--jacoco "core=app/core/build/reports/jacoco/test/jacocoTestReport.xml" \
--jacoco "proprietary=app/proprietary/build/reports/jacoco/test/jacocoTestReport.xml" \
--jacoco "saas=app/saas/build/reports/jacoco/test/jacocoTestReport.xml" \
--github-step-summary
- name: Upload raw JUnit .exec for aggregate merge
# Same dedup rationale as the summary step: upload from the saas
# leg only (the most complete set, includes app/saas/.../test.exec)
# so the aggregate workflow merges the union rather than three
# overlapping subsets.
#
# Separate artifact from the HTML reports so the aggregate
# workflow can grab just the .exec files with a name pattern
# (`jacoco-exec-*`) instead of unpacking the whole test-reports
# tarball.
if: always() && matrix.flavor == 'saas'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: jacoco-exec-junit-jdk-${{ matrix.jdk-version }}
path: app/*/build/jacoco/*.exec
retention-days: 7
if-no-files-found: warn
- name: Add coverage to PR (flavor=${{ matrix.flavor }}, JDK=${{ matrix.jdk-version }})
# The action only supports the pull_request event (it posts a PR comment),
# so skip it for merge_group runs and workflow_dispatch.
if: github.event_name == 'pull_request'
+1 -1
View File
@@ -78,7 +78,7 @@ jobs:
cache: "npm"
cache-dependency-path: frontend/package-lock.json
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Install Playwright (chromium only)
run: task e2e:install -- chromium
- name: Build frontend (needed for playwright's vite preview webServer)
+32
View File
@@ -68,6 +68,18 @@ jobs:
uses: ./.github/workflows/backend-build.yml
secrets: inherit
db-migration-test:
# Boots the current bootJar against H2 fixtures captured from past
# releases (v2.0.0 / v2.5.0 / v2.10.0) and verifies admin login still
# works after Hibernate's ddl-auto=update migrates the schema. Gated on
# the `project` filter so doc-only PRs skip this ~5-minute job.
if: needs.files-changed.outputs.project == 'true'
needs: [files-changed]
permissions:
contents: read
uses: ./.github/workflows/db-migration-test.yml
secrets: inherit
check-generateOpenApiDocs:
if: needs.files-changed.outputs.openapi == 'true'
needs: [files-changed]
@@ -173,6 +185,24 @@ jobs:
uses: ./.github/workflows/dependency-review.yml
secrets: inherit
# Coverage aggregate: merges the JUnit + e2e:live + cucumber .exec
# artifacts produced by the jobs above into one report, plus pulls
# in vitest + Playwright frontend coverage for the per-area matrix.
# `if: always()` so a producer failing partway still gets credit
# for whatever did record. Advisory only - intentionally NOT in
# all-checks-passed, so a flaky aggregate run never blocks merging.
coverage-aggregate:
if: always()
needs:
- build
- playwright-e2e-live
- docker-compose-tests
- frontend-validation
permissions:
contents: read
uses: ./.github/workflows/coverage-aggregate.yml
secrets: inherit
# Single status check that branch protection should mark as required.
# Succeeds when every upstream job is either `success` or `skipped` (path-
# gated jobs that didn't apply this run). Any `failure` or `cancelled`
@@ -184,6 +214,7 @@ jobs:
needs:
- files-changed
- build
- db-migration-test
- check-generateOpenApiDocs
- frontend-validation
- playwright-e2e
@@ -208,6 +239,7 @@ jobs:
RESULTS: |
files-changed=${{ needs.files-changed.result }}
build=${{ needs.build.result }}
db-migration-test=${{ needs.db-migration-test.result }}
check-generateOpenApiDocs=${{ needs.check-generateOpenApiDocs.result }}
frontend-validation=${{ needs.frontend-validation.result }}
playwright-e2e=${{ needs.playwright-e2e.result }}
+2 -2
View File
@@ -40,11 +40,11 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
cache-disabled: true
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Check licenses for compatibility
run: task backend:licenses:check
env:
+2 -2
View File
@@ -45,11 +45,11 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
cache-disabled: true
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Generate OpenAPI documentation
run: task backend:swagger
env:
+6 -6
View File
@@ -166,16 +166,16 @@ jobs:
// Determine reference file
let referenceFilePath;
if (changedFiles.includes("frontend/editor/public/locales/en-GB/translation.toml")) {
if (changedFiles.includes("frontend/editor/public/locales/en-US/translation.toml")) {
console.log("Using PR branch reference file.");
const { data: fileContent } = await github.rest.repos.getContent({
owner: prRepoOwner,
repo: prRepoName,
path: "frontend/editor/public/locales/en-GB/translation.toml",
path: "frontend/editor/public/locales/en-US/translation.toml",
ref: branch,
});
referenceFilePath = "pr-branch-translation-en-GB.toml";
referenceFilePath = "pr-branch-translation-en-US.toml";
const content = Buffer.from(fileContent.content, "base64").toString("utf-8");
fs.writeFileSync(referenceFilePath, content);
} else {
@@ -183,11 +183,11 @@ jobs:
const { data: fileContent } = await github.rest.repos.getContent({
owner: repoOwner,
repo: repoName,
path: "frontend/editor/public/locales/en-GB/translation.toml",
path: "frontend/editor/public/locales/en-US/translation.toml",
ref: "main",
});
referenceFilePath = "main-branch-translation-en-GB.toml";
referenceFilePath = "main-branch-translation-en-US.toml";
const content = Buffer.from(fileContent.content, "base64").toString("utf-8");
fs.writeFileSync(referenceFilePath, content);
}
@@ -293,6 +293,6 @@ jobs:
run: |
echo "Cleaning up temporary files..."
rm -rf pr-branch
rm -f pr-branch-translation-en-GB.toml main-branch-translation-en-GB.toml changed_files.txt result.txt
rm -f pr-branch-translation-en-US.toml main-branch-translation-en-US.toml changed_files.txt result.txt
echo "Cleanup complete."
continue-on-error: true # Ensure cleanup runs even if previous steps fail
+230
View File
@@ -0,0 +1,230 @@
name: Aggregate backend coverage
# Reusable workflow called from build.yml after every backend coverage
# producer (JUnit, e2e:live, cucumber) has run. Downloads each job's raw
# .exec, merges them into one JaCoCo report, and posts a combined step
# summary alongside the per-source ones.
#
# Kept separate from the per-source jobs so:
# - the per-source jobs stay fast and independent (no cross-job waits)
# - this job can `if: always()` and still produce something useful when
# one of the producers fails partway through
# - frontend producers can be added later without touching the
# producers themselves
on:
workflow_call:
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
aggregate:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
timeout-minutes: 15
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up JDK 25
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
with:
java-version: "25"
distribution: "temurin"
- name: Cache Gradle dependency artifacts
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: |
~/.gradle/wrapper
~/.gradle/caches/modules-2/files-2.1
~/.gradle/caches/modules-2/metadata-2.*
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
cache-disabled: true
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.12"
- name: Install defusedxml for coverage scripts
# Both coverage-summary.py and coverage-matrix.py parse JaCoCo
# XML through defusedxml - see the script headers for context.
run: python -m pip install --quiet defusedxml
# Pattern matches every artifact this PR's producers might upload:
# jacoco-exec-junit-jdk-25 (uploaded only by the saas
# leg of backend-build, which
# is a strict superset of the
# core + proprietary legs)
# jacoco-exec-e2e-live
# jacoco-exec-cucumber
# Each lands as a sibling dir under coverage-execs/, with the .exec
# files preserving their original relative paths.
- name: Download all .exec artifacts
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v6.0.0
with:
pattern: jacoco-exec-*
path: coverage-execs/
merge-multiple: false
continue-on-error: true
- name: Inventory .exec files
id: inventory
# Splits the downloaded artifacts into two buckets:
# * e2e-only = cucumber + Playwright live (user-flow coverage)
# * all = the above plus JUnit (everything we test)
#
# Bucketing is by artifact-name prefix: download-artifact preserves
# the artifact name as the top-level dir, so JUnit's `.exec`s live
# under coverage-execs/jacoco-exec-junit-*/... while the others
# are under coverage-execs/jacoco-exec-{e2e-live,cucumber}/...
#
# If nothing was uploaded (e.g. all producers crashed before
# writing) we exit gracefully so this advisory job never fails CI.
run: |
mapfile -t all_execs < <(find coverage-execs -name '*.exec' -type f | sort)
mapfile -t e2e_execs < <(find coverage-execs -name '*.exec' -type f -not -path '*/jacoco-exec-junit-*' | sort)
if [ "${#all_execs[@]}" -eq 0 ]; then
echo "::warning::No .exec artifacts found - skipping aggregate report"
echo "found_all=false" >> "$GITHUB_OUTPUT"
echo "found_e2e=false" >> "$GITHUB_OUTPUT"
exit 0
fi
printf 'All %d .exec files:\n' "${#all_execs[@]}"
printf ' %s\n' "${all_execs[@]}"
IFS=','; all_joined="${all_execs[*]}"
echo "files_all=$all_joined" >> "$GITHUB_OUTPUT"
echo "found_all=true" >> "$GITHUB_OUTPUT"
if [ "${#e2e_execs[@]}" -eq 0 ]; then
echo "::notice::No e2e/cucumber .exec files - e2e-only report will be skipped"
echo "found_e2e=false" >> "$GITHUB_OUTPUT"
else
printf 'E2E-only %d .exec files:\n' "${#e2e_execs[@]}"
printf ' %s\n' "${e2e_execs[@]}"
unset IFS
IFS=','; e2e_joined="${e2e_execs[*]}"
echo "files_e2e=$e2e_joined" >> "$GITHUB_OUTPUT"
echo "found_e2e=true" >> "$GITHUB_OUTPUT"
fi
- name: Compile classes for JaCoCo class lookup
# jacocoReportFromExec only needs the compiled .class files
# under each subproject's build/classes/java/main/. `classes`
# (compileJava + processResources) is enough; we skipped the
# heavier `assemble` to avoid building bootJar / fat jars that
# add 60+ seconds per run for no gain to the report.
if: steps.inventory.outputs.found_all == 'true'
run: ./gradlew classes -PnoSpotless
- name: Generate e2e-only JaCoCo report
# "Real user-flow" coverage: only counts code reached by an actual
# HTTP request from cucumber or live Playwright. Useful for
# questions like "how much of our backend does a user actually
# hit?". Skipped when neither producer uploaded a .exec.
if: steps.inventory.outputs.found_e2e == 'true'
run: |
./gradlew jacocoReportFromExec \
-PexecFile="${{ steps.inventory.outputs.files_e2e }}" \
-PreportDir=build/reports/jacoco/aggregate-e2e \
-PnoSpotless
- name: Generate combined JaCoCo report (everything)
if: steps.inventory.outputs.found_all == 'true'
run: |
./gradlew jacocoReportFromExec \
-PexecFile="${{ steps.inventory.outputs.files_all }}" \
-PreportDir=build/reports/jacoco/aggregate-all \
-PnoSpotless
- name: E2E-only step summary
# Rendered first so it gets prime real estate in the Summary
# tab - this is the number most readers actually want
# ("how much of the backend do real user flows cover?").
if: steps.inventory.outputs.found_e2e == 'true'
run: |
python scripts/coverage-summary.py \
--title "Real user-flow backend coverage (e2e:live + cucumber)" \
--jacoco "merged=build/reports/jacoco/aggregate-e2e/jacocoTestReport.xml" \
--github-step-summary
- name: ALL-sources step summary
# Separate call (not a multi-input one) because the helper's
# rightmost "Aggregate" column would sum the two reports - which
# is meaningless when one is a strict superset of the other.
if: steps.inventory.outputs.found_all == 'true'
run: |
python scripts/coverage-summary.py \
--title "Combined backend coverage (JUnit + e2e:live + cucumber)" \
--jacoco "merged=build/reports/jacoco/aggregate-all/jacocoTestReport.xml" \
--github-step-summary
- name: Upload combined aggregate report
if: steps.inventory.outputs.found_all == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: jacoco-aggregate-all-${{ github.run_id }}
path: build/reports/jacoco/aggregate-all/
retention-days: 14
- name: Upload e2e-only aggregate report
if: steps.inventory.outputs.found_e2e == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: jacoco-aggregate-e2e-${{ github.run_id }}
path: build/reports/jacoco/aggregate-e2e/
retention-days: 14
# --------------------------------------------------------------
# Per-area matrix: rolls backend + frontend coverage into one
# table indexed by core/proprietary/saas/desktop. Pulls the
# frontend artifacts now (after the JaCoCo step has done its
# work) so the per-source backend summaries still render first
# even if the matrix step fails.
# --------------------------------------------------------------
- name: Download vitest coverage artifact
# frontend-validation uploads as `frontend-coverage`. Tolerate
# absence so a backend-only PR still produces the matrix with
# just backend rows populated.
if: always()
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v6.0.0
with:
name: frontend-coverage
path: matrix-inputs/vitest/
continue-on-error: true
- name: Download Playwright frontend coverage artifact
# e2e-live uploads as `playwright-frontend-coverage-<run_id>`.
# Same tolerance as vitest - matrix script handles missing inputs.
if: always()
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v6.0.0
with:
name: playwright-frontend-coverage-${{ github.run_id }}
path: matrix-inputs/playwright/
continue-on-error: true
- name: Coverage matrix step summary
if: always()
# Matrix references the two aggregate JaCoCo XMLs (already
# generated above) plus whichever frontend artifacts landed.
# Every input is optional; missing ones render as "-".
run: |
python scripts/coverage-matrix.py \
${{ steps.inventory.outputs.found_all == 'true' && '--jacoco-all build/reports/jacoco/aggregate-all/jacocoTestReport.xml' || '' }} \
${{ steps.inventory.outputs.found_e2e == 'true' && '--jacoco-e2e build/reports/jacoco/aggregate-e2e/jacocoTestReport.xml' || '' }} \
--vitest matrix-inputs/vitest/coverage-summary.json \
--playwright-frontend matrix-inputs/playwright/coverage-pw-summary/coverage-summary.json \
--title "Coverage matrix (per-area, e2e vs all)" \
--github-step-summary
+93
View File
@@ -0,0 +1,93 @@
name: DB migration smoke test
# Boots the current Stirling-PDF JAR against H2 fixtures captured from past
# releases (v2.0.0 / v2.5.0 / v2.10.0) and verifies admin login still works.
# Catches schema changes that would break existing user databases under
# Hibernate's `ddl-auto=update` upgrade path.
on:
workflow_call:
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
migration-test:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-8' }}
timeout-minutes: 30
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up JDK 25
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
with:
java-version: 25
distribution: temurin
- name: Cache Gradle dependency artifacts
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: |
~/.gradle/wrapper
~/.gradle/caches/modules-2/files-2.1
~/.gradle/caches/modules-2/metadata-2.*
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.5.1
cache-disabled: true
# No `-PnoSpotless` here yet because the upstream cache layer matches the
# backend build's; reuse keeps cold-cache cost identical.
- name: Build Stirling-PDF JAR
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
run: ./gradlew :stirling-pdf:bootJar -PnoSpotless --no-daemon
- name: Locate built JAR
id: jar
run: |
jar=$(find app/core/build/libs -maxdepth 1 -name 'Stirling-PDF*.jar' -o -name 'stirling-pdf*.jar' 2>/dev/null \
| grep -vE '(-plain|-sources)\.jar$' | head -n 1)
if [[ -z "$jar" ]]; then
echo "::error::No JAR under app/core/build/libs"
ls -lah app/core/build/libs || true
exit 1
fi
# Absolute path - the migration script pushd's into a temp workdir
# before invoking java, which would dangle a relative path.
jar=$(realpath "$jar")
echo "path=$jar" >> "$GITHUB_OUTPUT"
echo "Built JAR: $jar"
- name: Run migration smoke test
env:
STIRLING_JAR: ${{ steps.jar.outputs.path }}
run: bash scripts/db-migration/run-migration-test.sh
- name: Upload app logs on failure
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: db-migration-app-logs
# Path matches the preserved workdir in run-migration-test.sh -
# only failing fixtures leave a directory behind.
path: /tmp/stirling-migration-failed-*/app.log
retention-days: 7
if-no-files-found: warn
+3 -3
View File
@@ -107,7 +107,7 @@ jobs:
- name: Build and push frontend image (Depot)
if: env.USE_DEPOT == 'true' && steps.check-frontend.outputs.exists == 'false'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
@@ -136,7 +136,7 @@ jobs:
- name: Build and push backend image (Depot)
if: env.USE_DEPOT == 'true' && steps.check-backend.outputs.exists == 'false'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
@@ -188,7 +188,7 @@ jobs:
environment:
DISABLE_ADDITIONAL_FEATURES: "true"
SECURITY_ENABLELOGIN: "false"
SYSTEM_DEFAULTLOCALE: en-GB
SYSTEM_DEFAULTLOCALE: en-US
UI_APPNAME: "Stirling-PDF V2"
UI_HOMEDESCRIPTION: "V2 Frontend/Backend Split"
UI_APPNAMENAVBAR: "V2 Deployment"
+63 -1
View File
@@ -61,7 +61,7 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
cache-disabled: true
- name: Set up Docker Buildx
@@ -87,6 +87,12 @@ jobs:
run: |
pip install --require-hashes --only-binary=:all: -r ./testing/cucumber/requirements.txt
- name: Extract JaCoCo agent for cucumber coverage
# Stages build/jacoco/jacocoagent.jar where the coverage override
# file bind-mounts it into the cucumber container. The agent jar
# never goes into the published image - this is host-only.
run: ./gradlew copyJacocoAgent -PnoSpotless
- name: Run Docker Compose Tests
run: |
chmod +x ./testing/test_webpages.sh
@@ -98,6 +104,62 @@ jobs:
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
DOCKER_BASE_CHANGED: ${{ inputs.docker-base-changed }}
# Tells test.sh to layer testing/compose/docker-compose-coverage.override.yml
# over the cucumber compose so the container starts with the
# JaCoCo agent attached via JAVA_CUSTOM_OPTS.
STIRLING_PDF_TEST_COVERAGE: "1"
- name: Generate cucumber JaCoCo report
# `if: always()` so a behave failure still produces partial
# coverage from whatever endpoints did run. The exec file only
# exists when the container shut down cleanly - guard so the step
# is silent on the (rare) crash path.
if: always()
id: cucumber-coverage
run: |
if [ -s testing/cucumber-coverage/cucumber.exec ]; then
./gradlew jacocoReportFromExec \
-PexecFile=testing/cucumber-coverage/cucumber.exec \
-PreportDir=build/reports/jacoco/cucumber \
-PnoSpotless
echo "report=true" >> "$GITHUB_OUTPUT"
else
echo "::warning::No cucumber .exec at testing/cucumber-coverage/cucumber.exec (container may have crashed before flushing)"
echo "report=false" >> "$GITHUB_OUTPUT"
fi
- name: Install defusedxml for coverage summary
# coverage-summary.py parses JaCoCo XML through defusedxml -
# see the script header for context.
if: always() && steps.cucumber-coverage.outputs.report == 'true'
run: python -m pip install --quiet defusedxml
- name: Cucumber coverage step summary
if: always() && steps.cucumber-coverage.outputs.report == 'true'
run: |
python scripts/coverage-summary.py \
--title "Cucumber (docker) JaCoCo coverage" \
--jacoco "cucumber=build/reports/jacoco/cucumber/jacocoTestReport.xml" \
--github-step-summary
- name: Upload cucumber JaCoCo report
if: always() && steps.cucumber-coverage.outputs.report == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: jacoco-cucumber-${{ github.run_id }}
path: build/reports/jacoco/cucumber/
retention-days: 7
- name: Upload raw cucumber .exec for aggregate merge
# Picked up by the coverage-aggregate workflow via the
# `jacoco-exec-*` artifact name pattern.
if: always() && steps.cucumber-coverage.outputs.report == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: jacoco-exec-cucumber
path: testing/cucumber-coverage/cucumber.exec
retention-days: 7
if-no-files-found: warn
- name: Upload Cucumber Report
if: always()
+147 -4
View File
@@ -42,16 +42,139 @@ jobs:
cache: "npm"
cache-dependency-path: frontend/package-lock.json
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Install Playwright (chromium only)
run: task e2e:install -- chromium
- name: Build frontend (production bundle for vite preview)
env:
VITE_BUILD_FOR_PREVIEW: "1"
run: task frontend:build
- name: Run live E2E tests (chromium)
- name: Run live E2E tests (chromium) with coverage
id: live-tests
env:
# Attaches the JaCoCo agent to the bootRun JVM (see
# .taskfiles/e2e.yml live:backend). The .exec gets flushed on
# graceful shutdown when the runner traps EXIT/INT/TERM, so the
# report step below sees a populated file.
COVERAGE: "1"
# Tells the Playwright fixture (test-base.ts) to capture per-test
# V8 JS coverage. Raw dumps land under
# .test-state/playwright/coverage-pw/ for the post-process step
# to aggregate. Chromium-only - other engines silently skip.
PW_COVERAGE: "1"
run: task e2e:live
- name: Generate JaCoCo report from e2e:live .exec
if: always()
id: live-coverage
# `if: always()` so even a failed test run still produces a
# report from whatever flows did exercise the backend before
# the failure. The task itself tolerates a missing .exec
# (jacoco emits an empty report rather than crashing) but we
# guard with `test -s` to keep the job log clean.
run: |
if [ -s .test-state/playwright/jacoco.exec ]; then
./gradlew jacocoReportFromExec \
-PexecFile=.test-state/playwright/jacoco.exec \
-PreportDir=build/reports/jacoco/e2e-live \
-PnoSpotless
echo "report=true" >> "$GITHUB_OUTPUT"
else
echo "::warning::No e2e:live .exec found at .test-state/playwright/jacoco.exec; skipping report"
echo "report=false" >> "$GITHUB_OUTPUT"
fi
- name: Set up Python for coverage summary
if: always() && steps.live-coverage.outputs.report == 'true'
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.12"
- name: Install defusedxml for coverage summary
# coverage-summary.py uses defusedxml instead of stdlib xml.etree
# to dodge XXE / billion-laughs scanner findings.
if: always() && steps.live-coverage.outputs.report == 'true'
run: python -m pip install --quiet defusedxml
- name: e2e:live coverage step summary
if: always() && steps.live-coverage.outputs.report == 'true'
run: |
python scripts/coverage-summary.py \
--title "Playwright (live backend) JaCoCo coverage" \
--jacoco "e2e-live=build/reports/jacoco/e2e-live/jacocoTestReport.xml" \
--github-step-summary
- name: Upload e2e:live JaCoCo report
if: always() && steps.live-coverage.outputs.report == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: jacoco-e2e-live-${{ github.run_id }}
path: build/reports/jacoco/e2e-live/
retention-days: 7
- name: Upload raw e2e:live .exec for aggregate merge
# Picked up by the coverage-aggregate workflow via the
# `jacoco-exec-*` artifact name pattern.
if: always() && steps.live-coverage.outputs.report == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: jacoco-exec-e2e-live
path: .test-state/playwright/jacoco.exec
retention-days: 7
if-no-files-found: warn
- name: Set up Python for frontend coverage summary
# Separate from the backend-coverage python step because the
# frontend path doesn't depend on a JaCoCo report - it produces
# a summary even on backend failure, as long as some Playwright
# tests ran far enough to dump V8 coverage.
if: always()
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.12"
- name: Install defusedxml for frontend coverage summary
# Idempotent re-install: the backend-coverage step may have
# installed it already, but this leg can run on its own when the
# backend report step skips (e.g. .exec missing).
if: always()
run: python -m pip install --quiet defusedxml
- name: Aggregate Playwright frontend (V8) coverage
# Rolls per-test V8 dumps from the test-base fixture into one
# vitest-shaped coverage-summary.json. Tolerates a missing dump
# dir (firefox/webkit runs, or a failure before any test got
# far enough to dump).
if: always()
id: pw-frontend-coverage
run: |
if [ -d .test-state/playwright/coverage-pw ] && \
find .test-state/playwright/coverage-pw -name '*.json' -type f | grep -q .; then
python scripts/playwright-coverage-summary.py \
.test-state/playwright/coverage-pw \
--out .test-state/playwright/coverage-pw-summary/coverage-summary.json
echo "summary=true" >> "$GITHUB_OUTPUT"
else
echo "::notice::No Playwright frontend coverage dumps found (chromium-only feature)"
echo "summary=false" >> "$GITHUB_OUTPUT"
fi
- name: Playwright frontend coverage step summary
if: always() && steps.pw-frontend-coverage.outputs.summary == 'true'
run: |
python scripts/coverage-summary.py \
--title "Playwright (live) frontend coverage" \
--vitest .test-state/playwright/coverage-pw-summary/coverage-summary.json \
--github-step-summary
- name: Upload Playwright frontend coverage
# Bundle both the aggregated summary and the raw V8 dumps so
# someone debugging "why is this function showing as covered"
# can trace it back to the source dump.
if: always() && steps.pw-frontend-coverage.outputs.summary == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: playwright-frontend-coverage-${{ github.run_id }}
path: |
.test-state/playwright/coverage-pw-summary/
.test-state/playwright/coverage-pw/
retention-days: 7
- name: Print backend log on failure
if: failure() && steps.live-tests.conclusion == 'failure'
run: |
@@ -65,10 +188,30 @@ jobs:
name: backend-log-live-${{ github.run_id }}
path: .test-state/playwright/backend.log
retention-days: 7
- name: Upload Playwright report
- name: List Playwright output locations (debug)
if: always()
run: |
echo "::group::Playwright output dirs"
# Playwright anchors its default outputDir + HTML report to the
# nearest package.json, which is frontend/ (frontend/editor has
# none), so artifacts land under frontend/, not frontend/editor/.
ls -la frontend/playwright-report 2>/dev/null \
|| echo "no playwright-report at frontend/"
ls -la frontend/test-results 2>/dev/null \
|| echo "no test-results at frontend/"
find . -name node_modules -prune -o -name 'trace.zip' -print 2>/dev/null || true
echo "::endgroup::"
- name: Upload Playwright report + traces
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: playwright-report-live-${{ github.run_id }}
path: frontend/editor/playwright-report/
# test-results/ holds the per-test trace.zip (with browser console
# logs) + screenshots/video; playwright-report/ is the HTML report.
# Both live under frontend/ (Playwright anchors them to the nearest
# package.json, which is frontend/; frontend/editor has none).
path: |
frontend/playwright-report/
frontend/test-results/
retention-days: 7
if-no-files-found: warn
+1 -1
View File
@@ -36,7 +36,7 @@ jobs:
cache: "npm"
cache-dependency-path: frontend/package-lock.json
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Install Playwright (chromium only)
run: task e2e:install -- chromium
- name: Build frontend (production bundle for vite preview)
@@ -97,7 +97,7 @@ jobs:
run: npm ci --ignore-scripts --audit=false --fund=false
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Generate frontend license report (internal PR)
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false
env:
@@ -110,8 +110,8 @@ jobs:
NPM_CONFIG_IGNORE_SCRIPTS: "true"
working-directory: frontend
run: |
mkdir -p src/assets
npx --yes license-report --only=prod --output=json > src/assets/3rdPartyLicenses.json
mkdir -p editor/src/assets
npx --yes license-report --only=prod --output=json > editor/src/assets/3rdPartyLicenses.json
- name: Postprocess with project script (BASE version)
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == true
@@ -349,10 +349,10 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Check licenses and generate report
id: license-check
run: task backend:licenses:generate || echo "LICENSE_CHECK_FAILED=true" >> $GITHUB_ENV
+36 -1
View File
@@ -31,7 +31,7 @@ jobs:
cache: "npm"
cache-dependency-path: frontend/package-lock.json
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Quality-check frontend
id: frontend-check
run: task frontend:check:all
@@ -109,6 +109,41 @@ jobs:
comment_id: existing.id,
});
}
- name: Vitest coverage
# Separate from `frontend:check:all` so the quality-gate run stays
# uninstrumented (faster signal) and coverage stays an informational
# follow-up. Continue-on-error keeps the workflow green even when
# a handful of test files refuse to import (e.g. missing icon
# specifiers) - the summary still gets posted with whatever
# vitest managed to instrument.
id: frontend-coverage
continue-on-error: true
run: task frontend:test:coverage
- name: Set up Python for coverage summary
if: always()
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.12"
- name: Install defusedxml for coverage summary
# See coverage-summary.py header - it parses XML through defusedxml
# to dodge the stdlib parser's exposure to XXE / billion-laughs.
if: always()
run: python -m pip install --quiet defusedxml
- name: Vitest coverage step summary
if: always()
run: |
python scripts/coverage-summary.py \
--title "Frontend Vitest coverage" \
--vitest frontend/editor/coverage/coverage-summary.json \
--github-step-summary
- name: Upload vitest coverage report
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: frontend-coverage
path: frontend/editor/coverage/
retention-days: 7
if-no-files-found: warn
- name: Upload frontend build artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
+173 -22
View File
@@ -73,10 +73,10 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Get version number
id: versionNumber
run: |
@@ -148,7 +148,7 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Setup Node.js
if: matrix.variant.build_frontend == true
@@ -159,7 +159,7 @@ jobs:
cache-dependency-path: frontend/package-lock.json
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Build JAR
run: ./gradlew build ${{ matrix.variant.build_frontend && '-PbuildWithFrontend=true' || '' }} -x spotlessApply -x spotlessCheck -x test -x sonarqube
@@ -252,10 +252,10 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
# Build the universal JRE before desktop:prepare so the jlink:runtime
# task short-circuits on its `test -d runtime/jre` status check.
@@ -442,10 +442,6 @@ jobs:
echo "Generated tauri.windows.conf.json (alias masked):"
sed "s/${KEYPAIR_ALIAS}/***/g" ./frontend/editor/src-tauri/tauri.windows.conf.json
- name: Sign JPDFium dylibs inside bootJar (macOS only)
if: matrix.platform == 'macos-15' && env.APPLE_CERTIFICATE != ''
run: bash frontend/scripts/sign-jpdfium-dylibs-in-bootjar.sh
- name: Import release GPG signing key (Linux)
if: matrix.platform == 'ubuntu-22.04' && env.RELEASE_GPG_PRIVATE_KEY != '' && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master')
run: |
@@ -495,6 +491,7 @@ jobs:
projectPath: ./frontend/editor
tauriScript: npx tauri
args: ${{ matrix.args }}
updaterJsonKeepUniversal: true
- name: Clear release GPG key from runner keyring (Linux)
if: always() && matrix.platform == 'ubuntu-22.04' && env.RELEASE_GPG_PRIVATE_KEY != '' && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master')
@@ -586,21 +583,37 @@ jobs:
if: always() && steps.digicert-setup.conclusion != 'failure'
shell: bash
run: |
mkdir -p ./dist
# Absolute dist path so the cd below can't break the copy targets.
DIST="$GITHUB_WORKSPACE/dist"
mkdir -p "$DIST"
cd ./frontend/editor/src-tauri/target
# Find and rename artifacts based on platform
echo "=== tauri bundle artifacts ==="
find . -path "*/bundle/*" \( -name "*.msi" -o -name "*.deb" \
-o -name "*.rpm" -o -name "*.AppImage" -o -name "*.dmg" \
-o -name "*.app.tar.gz" -o -name "*.sig" \) 2>/dev/null | sort || true
echo "=============================="
# createUpdaterArtifacts:true signs the native installers in place;
# each <bundle> ships with a sibling <bundle>.sig consumed by latest.json.
if [ "${{ matrix.platform }}" = "windows-latest" ]; then
# Only ship the MSI installer on Windows. The loose exe and WiX toolset exes
# are not the user-facing installer - the MSI contains the signed inner exe.
find . -name "*.msi" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.msi" \;
find . -name "*.msi" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.msi" \;
find . -name "*.msi.sig" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.msi.sig" \;
elif [ "${{ matrix.platform }}" = "macos-15" ]; then
find . -name "*.dmg" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.dmg" \;
find . -name "*.app" -exec cp -r {} "../../../dist/Stirling-PDF-${{ matrix.name }}.app" \;
# DMG = manual install; .app.tar.gz (+ .sig) = updater payload.
# Raw .app is intentionally not shipped (hundreds of MB of uncompressed input).
find . -name "*.dmg" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.dmg" \;
find . -name "*.app.tar.gz" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.app.tar.gz" \;
find . -name "*.app.tar.gz.sig" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.app.tar.gz.sig" \;
else
find . -name "*.deb" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.rpm" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.AppImage" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.AppImage" \;
# The raw .AppImage IS its updater payload (signed -> .AppImage.sig),
# not a .tar.gz wrapper - that's only produced under v1Compatible.
find . -name "*.deb" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.deb.sig" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.deb.sig" \;
find . -name "*.rpm" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.rpm.sig" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.rpm.sig" \;
find . -name "*.AppImage" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.AppImage" \;
find . -name "*.AppImage.sig" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.AppImage.sig" \;
fi
- name: Upload build artifacts
@@ -611,8 +624,7 @@ jobs:
path: ./dist/*
retention-days: 1
create-release:
if: (github.event_name == 'workflow_dispatch' && github.event.inputs.test_mode != 'true') || github.event_name == 'release' || github.ref == 'refs/heads/V2-master'
collect-and-release:
needs: [pick, determine-matrix, build, build-jars]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
@@ -623,6 +635,16 @@ jobs:
with:
egress-policy: audit
# Sparse-check out the verifier + pubkey before the artifact downloads
# so the checkout cannot clobber ./artifacts.
- name: Checkout updater verifier
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
sparse-checkout: |
.github/scripts/verify-updater-signatures.py
frontend/editor/src-tauri/tauri.conf.json
sparse-checkout-cone-mode: false
- name: Download all Tauri artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
@@ -650,17 +672,146 @@ jobs:
- name: Display structure of downloaded files
run: ls -R ./artifacts
# tauri-action only emits latest.json when it also publishes the release
# (tagName/releaseId set). We publish separately via action-gh-release,
# so build latest.json here from the per-platform .sig files.
- name: Generate updater latest.json
env:
VERSION: ${{ needs.determine-matrix.outputs.version }}
TAG: v${{ needs.determine-matrix.outputs.version }}
REPO: ${{ github.repository }}
run: |
python3 - << 'PYEOF'
import json, os, sys
from pathlib import Path
from datetime import datetime, timezone
VERSION = os.environ['VERSION']
TAG = os.environ['TAG']
REPO = os.environ['REPO']
ART = Path('./artifacts/tauri')
# Tauri updater looks up {os}-{arch}-{installer} (e.g. linux-x86_64-deb)
# before bare {os}-{arch}, so per-format Linux keys let deb/rpm/appimage
# each self-update from their matching file. macOS universal serves both
# arches from the one .app.tar.gz.
PLATFORM_MAP = [
{
'bundles': ['Stirling-PDF-linux-x86_64.deb'],
'targets': ['linux-x86_64-deb'],
},
{
'bundles': ['Stirling-PDF-linux-x86_64.rpm'],
'targets': ['linux-x86_64-rpm'],
},
{
'bundles': ['Stirling-PDF-linux-x86_64.AppImage'],
'targets': ['linux-x86_64-appimage'],
},
{
'bundles': ['Stirling-PDF-windows-x86_64.msi'],
'targets': ['windows-x86_64-msi', 'windows-x86_64'],
},
{
'bundles': ['Stirling-PDF-macos-universal.app.tar.gz'],
'targets': ['darwin-x86_64', 'darwin-aarch64'],
},
]
# rglob() because download-artifact varies layout: one artifact -> flat,
# many -> nested under <artifact-name>/.
def find_signed(name):
for bundle_path in sorted(ART.rglob(name)):
sig_path = bundle_path.with_name(bundle_path.name + '.sig')
if sig_path.exists():
return bundle_path, sig_path
return None
platforms = {}
skipped = []
for entry in PLATFORM_MAP:
picked = None
for name in entry['bundles']:
picked = find_signed(name)
if picked:
break
if not picked:
skipped.append(
f"{entry['targets']} (no signed bundle among "
f"{entry['bundles']} - TAURI_SIGNING_PRIVATE_KEY unset "
f"or createUpdaterArtifacts disabled?)"
)
continue
bundle_path, sig_path = picked
signature = sig_path.read_text(encoding='utf-8').strip()
url = f"https://github.com/{REPO}/releases/download/{TAG}/{bundle_path.name}"
for target in entry['targets']:
platforms[target] = {'signature': signature, 'url': url}
print(f"Added {entry['targets']} from {bundle_path.name}")
if skipped:
print("Skipped platforms:")
for s in skipped:
print(f" - {s}")
if not platforms:
print(
"WARN: no signed updater bundles found - "
"skipping latest.json generation"
)
sys.exit(0)
manifest = {
'version': VERSION,
'notes': f"See https://github.com/{REPO}/releases/tag/{TAG}",
'pub_date': datetime.now(timezone.utc).strftime('%Y-%m-%dT%H:%M:%SZ'),
'platforms': platforms,
}
out = Path('./artifacts/latest.json')
out.write_text(json.dumps(manifest, indent=2) + '\n', encoding='utf-8')
print(f"Generated {out} with platforms: {sorted(platforms.keys())}")
PYEOF
- name: Upload merged artifacts for review
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: release-artifacts
path: ./artifacts/
retention-days: 7
# Gate publish on valid updater sigs. Runs after the review upload (so
# artifacts survive for debugging) and before action-gh-release.
- name: Verify updater signatures
run: |
python3 -m pip install --quiet 'cryptography==44.0.0'
python3 .github/scripts/verify-updater-signatures.py \
./artifacts/tauri frontend/editor/src-tauri/tauri.conf.json
# workflow_dispatch path requires platform=='all' so a single-platform
# dispatch can't overwrite an existing release's full latest.json with a
# partial one (action-gh-release defaults overwrite_files:true).
# release / V2-master always build the full matrix so no extra guard needed.
# fail_on_unmatched_files makes a missing latest.json or installer fail loudly
# instead of silently shipping a broken auto-update.
- name: Upload binaries to Release
if: (github.event_name == 'workflow_dispatch' && github.event.inputs.test_mode != 'true' && github.event.inputs.platform == 'all') || github.event_name == 'release' || github.ref == 'refs/heads/V2-master'
uses: softprops/action-gh-release@b4309332981a82ec1c5618f44dd2e27cc8bfbfda # v3.0.0
with:
tag_name: v${{ needs.determine-matrix.outputs.version }}
generate_release_notes: true
fail_on_unmatched_files: true
# Installers + updater payloads + manifest. .sig contents are embedded
# in latest.json so the .sig files themselves are not uploaded.
files: |
./artifacts/**/*.jar
./artifacts/**/*.msi
./artifacts/**/*.dmg
./artifacts/**/*.app.tar.gz
./artifacts/**/*.deb
./artifacts/**/*.rpm
./artifacts/**/*.AppImage
./artifacts/latest.json
draft: false
prerelease: false
+1 -1
View File
@@ -37,7 +37,7 @@ jobs:
cache-dependency-path: frontend/package-lock.json
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Install all Playwright browsers
run: task e2e:install
+9 -24
View File
@@ -1,8 +1,7 @@
name: Pre-commit
# Runs `pre-commit run` for ruff / codespell / gitleaks / EOF / trailing-ws.
# Called from build.yml on PRs and merge_group; also runnable on demand via
# workflow_dispatch for manual local-equivalent linting.
# Runs the repo-wide lint/format/secret checks via `task pre-commit`.
# Called from build.yml on PRs and merge_group; also runnable on demand via workflow_dispatch.
on:
workflow_call:
workflow_dispatch:
@@ -13,10 +12,6 @@ permissions:
jobs:
pre-commit:
runs-on: ubuntu-latest
env:
# Prevents sdist builds → no tar extraction
PIP_ONLY_BINARY: ":all:"
PIP_DISABLE_PIP_VERSION_CHECK: "1"
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -29,23 +24,13 @@ jobs:
fetch-depth: 0
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
- name: Install uv
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
with:
python-version: 3.12
cache: "pip" # caching pip dependencies
cache-dependency-path: ./.github/scripts/requirements_pre_commit.txt
enable-cache: true
- name: Run Pre-Commit Hooks
run: |
pip install --require-hashes --only-binary=:all: -r ./.github/scripts/requirements_pre_commit.txt
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Run Pre-Commit
run: |
pre-commit run ruff --all-files -c .pre-commit-config.yaml
pre-commit run ruff-format --all-files -c .pre-commit-config.yaml
pre-commit run codespell --all-files -c .pre-commit-config.yaml
pre-commit run gitleaks --all-files -c .pre-commit-config.yaml
pre-commit run end-of-file-fixer --all-files -c .pre-commit-config.yaml
pre-commit run trailing-whitespace --all-files -c .pre-commit-config.yaml
git diff --exit-code
- name: Run pre-commit checks
run: task pre-commit
+1 -1
View File
@@ -75,7 +75,7 @@ jobs:
- name: Generate tags for base image
id: meta
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6.0.0
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
with:
images: |
${{ secrets.DOCKER_HUB_ORG_USERNAME }}/stirling-pdf-base
+5 -5
View File
@@ -78,14 +78,14 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Get version number
id: versionNumber
run: echo "versionNumber=$(./gradlew printVersion --quiet | tail -1)" >> $GITHUB_OUTPUT
@@ -129,7 +129,7 @@ jobs:
- name: Generate tags for latest
id: meta
if: env.RUN_MAIN_APP == 'true'
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6.0.0
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
with:
images: |
${{ secrets.DOCKER_HUB_USERNAME }}/s-pdf
@@ -178,7 +178,7 @@ jobs:
- name: Generate tags for latest-fat
id: meta-fat
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6.0.0
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
if: env.RUN_MAIN_APP == 'true' && github.ref != 'refs/heads/main' && github.ref != 'refs/heads/testMain'
with:
images: |
@@ -222,7 +222,7 @@ jobs:
- name: Generate tags for ultra-lite
id: meta-lite
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6.0.0
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
if: env.RUN_MAIN_APP == 'true' && github.ref != 'refs/heads/main' && github.ref != 'refs/heads/testMain'
with:
images: |
+1 -1
View File
@@ -22,7 +22,7 @@ jobs:
egress-policy: audit
- name: 30 days stale issues
uses: actions/stale@b5d41d4e1d5dceea10e7104786b73624c18a190f # v10.2.0
uses: actions/stale@eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899 # v10.3.0
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
days-before-stale: 30
+2 -2
View File
@@ -48,7 +48,7 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Generate Swagger documentation
run: ./gradlew :stirling-pdf:generateOpenApiDocs
@@ -63,7 +63,7 @@ jobs:
SWAGGERHUB_USER: "Frooodle"
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Get version number
id: versionNumber
run: echo "versionNumber=$(./gradlew printVersion --quiet | tail -1)" >> $GITHUB_OUTPUT
+13 -5
View File
@@ -58,15 +58,23 @@ jobs:
- name: Install Python dependencies
run: |
pip install --require-hashes --only-binary=:all: -r ./.github/scripts/requirements_sync_readme.txt -r ./.github/scripts/requirements_pre_commit.txt
pip install --require-hashes --only-binary=:all: -r ./.github/scripts/requirements_sync_readme.txt
- name: Install uv
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
with:
enable-cache: true
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Sync translation TOML files
run: |
python .github/scripts/check_language_toml.py --reference-file "frontend/editor/public/locales/en-GB/translation.toml" --branch main
python .github/scripts/check_language_toml.py --reference-file "frontend/editor/public/locales/en-US/translation.toml" --branch main
- name: pre-commit run
- name: Sort translation TOML files
run: |
pre-commit run toml-sort-fix --all-files
task pre-commit:toml-sort FIX=1
- name: Commit translation files
run: |
@@ -100,7 +108,7 @@ jobs:
This Pull Request was automatically generated to synchronize updates to translation files and documentation. Below are the details of the changes made:
#### **1. Synchronization of Translation Files**
- Updated translation files (`frontend/editor/public/locales/*/translation.toml`) to reflect changes in the reference file `en-GB/translation.toml`.
- Updated translation files (`frontend/editor/public/locales/*/translation.toml`) to reflect changes in the reference file `en-US/translation.toml`.
- Ensured consistency and synchronization across all supported language files.
- Highlighted any missing or incomplete translations.
- **Format**: TOML
+13 -12
View File
@@ -136,10 +136,10 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Setup Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Build universal macOS JRE
if: matrix.platform == 'macos-15'
@@ -157,6 +157,9 @@ jobs:
JPDFIUM_PLATFORMS: ${{ matrix.jpdfium_platforms }}
run: task desktop:prepare
- name: Run Tauri/Cargo tests
run: task desktop:test
# DigiCert KeyLocker Setup (Cloud HSM)
- name: Setup DigiCert KeyLocker
id: digicert-setup
@@ -268,10 +271,6 @@ jobs:
echo "APPLE_SIGNING_IDENTITY=$CERT_ID" >> $GITHUB_ENV
echo "Certificate imported successfully."
- name: Sign JPDFium dylibs inside bootJar (macOS only)
if: matrix.platform == 'macos-15' && env.APPLE_CERTIFICATE != ''
run: bash frontend/scripts/sign-jpdfium-dylibs-in-bootjar.sh
- name: Check DMG creation dependencies (macOS only)
if: matrix.platform == 'macos-15'
run: |
@@ -417,20 +416,22 @@ jobs:
- name: Rename artifacts
shell: bash
run: |
mkdir -p ./dist
# Absolute dist path so the cd below can't break the copy targets.
DIST="$GITHUB_WORKSPACE/dist"
mkdir -p "$DIST"
cd ./frontend/editor/src-tauri/target
# Find and rename artifacts based on platform
if [ "${{ matrix.platform }}" = "windows-latest" ]; then
# Only ship the MSI installer. The loose exe and WiX toolset exes
# are not the user-facing installer - the MSI contains the signed inner exe.
find . -name "*.msi" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.msi" \;
find . -name "*.msi" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.msi" \;
elif [ "${{ matrix.platform }}" = "macos-15" ]; then
find . -name "*.dmg" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.dmg" \;
find . -name "*.dmg" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.dmg" \;
else
find . -name "*.deb" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.rpm" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.AppImage" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.AppImage" \;
find . -name "*.deb" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.rpm" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.AppImage" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.AppImage" \;
fi
# Verify the MSI AND the inner exe extracted from it are signed.
+4 -4
View File
@@ -106,11 +106,11 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
cache-disabled: true
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Build application
run: task backend:build
env:
@@ -157,7 +157,7 @@ jobs:
- name: Build ${{ matrix.docker-rev }} (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
@@ -230,7 +230,7 @@ jobs:
- name: Build docker/unoserver/Dockerfile (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
+3 -3
View File
@@ -51,7 +51,7 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Build with Gradle
run: ./gradlew build
@@ -83,7 +83,7 @@ jobs:
- name: Build and push test image (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
@@ -129,7 +129,7 @@ jobs:
environment:
DISABLE_ADDITIONAL_FEATURES: "true"
SECURITY_ENABLELOGIN: "false"
SYSTEM_DEFAULTLOCALE: en-GB
SYSTEM_DEFAULTLOCALE: en-US
UI_APPNAME: "Stirling-PDF Test"
UI_HOMEDESCRIPTION: "Test Deployment"
UI_APPNAMENAVBAR: "Test"
+18 -3
View File
@@ -23,6 +23,10 @@ customFiles/
configs/
watchedFolders/
clientWebUI/
# Scratch dir used by local fixture-regeneration runs (see
# app/proprietary/src/test/resources/db-migration-fixtures/README.md).
# Holds downloaded JARs and disposable workdirs. Never committed.
.alpha-local/
!cucumber/
!cucumber/exampleFiles/
!cucumber/exampleFiles/example_html.zip
@@ -42,6 +46,11 @@ app/core/storage/
# These are generated by npm build and should not be committed
app/core/src/main/resources/static/assets/
app/core/src/main/resources/static/index.html
# Prerendered per-route SPA pages (OG/social-preview), e.g. compress.html. api-landing.html is source.
app/core/src/main/resources/static/*.html
!app/core/src/main/resources/static/api-landing.html
# Prerendered nested-route pages (e.g. settings/people.html)
app/core/src/main/resources/static/settings/
app/core/src/main/resources/static/locales/
app/core/src/main/resources/static/Login/
app/core/src/main/resources/static/classic-logo/
@@ -49,10 +58,14 @@ app/core/src/main/resources/static/modern-logo/
app/core/src/main/resources/static/og_images/
app/core/src/main/resources/static/samples/
app/core/src/main/resources/static/manifest-classic.json
app/core/src/main/resources/static/og-metadata.json
app/core/src/main/resources/static/sw-folder-retry.js
app/core/src/main/resources/static/robots.txt
app/core/src/main/resources/static/pdfium/
app/core/src/main/resources/static/pdfjs/
app/core/src/main/resources/static/vendor/
app/core/src/main/resources/static/**/*.gz
app/core/src/main/resources/static/**/*.br
# Note: Keep backend-managed files like fonts/, css/, js/, pdfjs/, etc.
# Gradle
@@ -174,7 +187,6 @@ venv.bak/
# Env files (secrets / local overrides). Subproject .gitignore files whitelist any committed defaults.
.env*
!.env.saas.example
# VS Code
/.vscode/**/*
@@ -209,7 +221,7 @@ out/
*.asc
# Allow test fixture certificates (synthetic, no real credentials)
!frontend/src/core/tests/test-fixtures/certs/**
!frontend/editor/src/core/tests/test-fixtures/certs/**
# SSH Keys
*.pub
@@ -251,7 +263,7 @@ node_modules/
*compact*.json
test_batch.json
*.backup.*.json
frontend/public/locales/*/translation.backup*.json
frontend/editor/public/locales/*/translation.backup*.json
# Development/build artifacts
.gradle-cache/
@@ -274,3 +286,6 @@ docs/type3/signatures/
# Playwright MCP screenshots / traces
.playwright-mcp/
*.playwright-mcp.png
# Local screenshot artifacts from *-screenshots.spec.ts
frontend/editor/screenshots/
+21
View File
@@ -0,0 +1,21 @@
# PostHog project-level key - phc_ prefix keys are public/client-side by design
# (PostHog client-side tracking embeds them in the browser bundle). Committed
# intentionally in #6150 so engine/.env has a working default, with real
# credentials overridden via engine/.env.local.
engine/.env:generic-api-key:41
# MCP test fixtures / harness - no real secrets:
# - test-only API key constant in an integration test
# - JDBC URL + throwaway Keycloak creds in the local test compose
# - placeholder / shell-variable Bearer headers in curl-based validation scripts
app/proprietary/src/test/java/stirling/software/proprietary/mcp/security/McpApiKeyIntegrationTest.java:generic-api-key:40
testing/compose/docker-compose-keycloak-mcp.yml:generic-api-key:25
testing/compose/validate-mcp-apikey.sh:curl-auth-header:73
testing/compose/validate-mcp-test.sh:curl-auth-header:92
testing/compose/validate-mcp-test.sh:curl-auth-header:116
# Storybook example showing curl with a fake Bearer token placeholder (sk_live_a3f8...).
frontend/shared/components/CodeBlock.stories.tsx:curl-auth-header:4
# Truncated placeholder API key in portal docs example (sk_live_8f2c...e10) - not a real secret.
frontend/portal/src/components/docs/GettingStartedSection.tsx:generic-api-key:31
+12 -50
View File
@@ -1,52 +1,14 @@
# The actual checks live in .taskfiles/pre-commit.yml (with helper scripts under
# scripts/pre-commit/) and are driven by Task. This hook just delegates to `task
# pre-commit` so the git pre-commit hook, CI and a manual `task pre-commit` all
# run the exact same thing. Requires `task` and `uv` on PATH. To auto-fix instead
# of only checking, run `task pre-commit:fix`.
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.14
- repo: local
hooks:
- id: ruff
args:
- --fix
- --line-length=127
files: ^((\.github/scripts|scripts|app/core/src/main/resources/static/python)/.+)?[^/]+\.py$
exclude: (split_photos.py)
- id: ruff-format
files: ^((\.github/scripts|scripts|app/core/src/main/resources/static/python)/.+)?[^/]+\.py$
exclude: (split_photos.py)
- repo: https://github.com/codespell-project/codespell
rev: v2.4.2
hooks:
- id: codespell
args:
- --ignore-words-list=thirdParty,tabEl,tabEls,Sie,ist,fulfilment
- --skip="./.*,*.csv,*.json,*.ambr"
- --quiet-level=2
files: \.(html|css|js|py|md)$
exclude: (.vscode|.devcontainer|app/core/src/main/resources|app/proprietary/src/main/resources|frontend/editor/public/vendor|Dockerfile|.*/pdfjs.*|.*/thirdParty.*|bootstrap.*|.*\.min\..*|.*diff\.js)
- repo: https://github.com/gitleaks/gitleaks
rev: v8.30.0
hooks:
- id: gitleaks
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
hooks:
- id: end-of-file-fixer
files: ^.*(\.js|\.java|\.py|\.yml)$
exclude: ^(.*/pdfjs.*|.*/thirdParty.*|bootstrap.*|.*\.min\..*|.*diff\.js|\.github/workflows/.*$)
- id: trailing-whitespace
files: ^.*(\.js|\.java|\.py|\.yml)$
exclude: ^(.*/pdfjs.*|.*/thirdParty.*|bootstrap.*|.*\.min\..*|.*diff\.js|\.github/workflows/.*$)
- repo: https://github.com/pappasam/toml-sort
rev: v0.24.4
hooks:
- id: toml-sort-fix
files: frontend/editor/public/locales/.*\.toml$
args: ['--in-place', '--all', '--ignore-case']
# - repo: https://github.com/thibaudcolas/pre-commit-stylelint
# rev: v16.21.1
# hooks:
# - id: stylelint
# additional_dependencies:
# - stylelint@16.21.1
# - stylelint-config-standard@38.0.0
# - "@stylistic/stylelint-plugin@3.1.3"
# files: \.(css)$
# args: [--fix]
- id: task-pre-commit
name: task pre-commit
entry: task pre-commit
language: system
pass_filenames: false
always_run: true
+23 -5
View File
@@ -18,16 +18,28 @@ version: '3'
tasks:
dev:
desc: "Start backend dev server"
cmds:
- task: dev:proprietary
vars:
PORT: '{{.PORT}}'
AIENGINE_URL: '{{.AIENGINE_URL}}'
AIENGINE_ENABLED: '{{.AIENGINE_ENABLED}}'
AIENGINE_TIMEOUTSECONDS: '{{.AIENGINE_TIMEOUTSECONDS}}'
dev:proprietary:
desc: "Start backend dev server in proprietary mode"
ignore_error: true
vars:
PORT: '{{.PORT | default "8080"}}'
AIENGINE_URL: '{{.AIENGINE_URL | default ""}}'
AIENGINE_ENABLED: '{{.AIENGINE_ENABLED | default "false"}}'
AIENGINE_TIMEOUTSECONDS: '{{.AIENGINE_TIMEOUTSECONDS | default "120"}}'
env:
SERVER_PORT: '{{.PORT}}'
cmds:
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED=true {{end}}cmd /c ".\gradlew.bat :stirling-pdf:bootRun"'
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED={{.AIENGINE_ENABLED}} AIENGINE_TIMEOUTSECONDS={{.AIENGINE_TIMEOUTSECONDS}} {{end}}cmd /c ".\gradlew.bat :stirling-pdf:bootRun"'
platforms: [windows]
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED=true {{end}}./gradlew :stirling-pdf:bootRun'
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED={{.AIENGINE_ENABLED}} AIENGINE_TIMEOUTSECONDS={{.AIENGINE_TIMEOUTSECONDS}} {{end}}./gradlew :stirling-pdf:bootRun'
platforms: [linux, darwin]
dev:bundled:
@@ -40,18 +52,24 @@ tasks:
platforms: [linux, darwin]
dev:saas:
desc: "Start backend in SaaS flavor against Supabase (loads .env.saas.local)"
desc: "Start backend in SaaS flavor against Supabase"
# `dotenv:` reads from the root Taskfile's directory (".") because this
# subtaskfile is included with `dir: .`. Drop the file at the repo root.
dotenv: ['.env.saas.local']
# subtaskfile is included with `dir: .`.
dotenv: ['app/.env.saas.local', 'app/.env.saas']
ignore_error: true
vars:
PORT: '{{.PORT | default "8080"}}'
# Override to "" to run the pure `saas` profile against your own SAAS_DB_*.
PROFILES: '{{.PROFILES | default "dev"}}'
AIENGINE_URL: '{{.AIENGINE_URL | default ""}}'
AIENGINE_ENABLED: '{{.AIENGINE_ENABLED | default "false"}}'
AIENGINE_TIMEOUTSECONDS: '{{.AIENGINE_TIMEOUTSECONDS | default "120"}}'
env:
SERVER_PORT: '{{.PORT}}'
STIRLING_FLAVOR: saas
AIENGINE_URL: '{{.AIENGINE_URL}}'
AIENGINE_ENABLED: '{{.AIENGINE_ENABLED}}'
AIENGINE_TIMEOUTSECONDS: '{{.AIENGINE_TIMEOUTSECONDS}}'
cmds:
- cmd: cmd /c ".\gradlew.bat :stirling-pdf:bootRun {{if .PROFILES}}--args=\"--spring.profiles.include={{.PROFILES}}\"{{end}}"
platforms: [windows]
+44 -13
View File
@@ -1,7 +1,9 @@
version: '3'
vars:
JLINK_MODULES: "java.base,java.compiler,java.desktop,java.instrument,java.logging,java.management,java.naming,java.net.http,java.prefs,java.rmi,java.scripting,java.security.jgss,java.security.sasl,java.sql,java.transaction.xa,java.xml,java.xml.crypto,jdk.crypto.ec,jdk.crypto.cryptoki,jdk.unsupported"
# jdk.dynalink is required by VeraPDF (PDF/A validation); without it the bundled JRE throws
# NoClassDefFoundError: jdk/dynalink/Namespace at runtime in get-info-on-pdf and verify-pdf
JLINK_MODULES: "java.base,java.compiler,java.desktop,java.instrument,java.logging,java.management,java.naming,java.net.http,java.prefs,java.rmi,java.scripting,java.security.jgss,java.security.sasl,java.sql,java.transaction.xa,java.xml,java.xml.crypto,jdk.crypto.ec,jdk.crypto.cryptoki,jdk.unsupported,jdk.dynalink"
# Override via JPDFIUM_PLATFORMS env (csv of platform keys, or 'all').
JPDFIUM_PLATFORMS:
@@ -62,21 +64,28 @@ tasks:
deps: [prepare]
dir: editor
cmds:
- npx tauri build --bundles app
- npx tauri build --bundles app --config '{"bundle":{"createUpdaterArtifacts":false}}'
build:dev:windows:
desc: "Build Tauri desktop NSIS installer (Windows)"
deps: [prepare]
dir: editor
cmds:
- npx tauri build --bundles nsis
- npx tauri build --bundles nsis --config '{"bundle":{"createUpdaterArtifacts":false}}'
build:dev:linux:
desc: "Build Tauri desktop AppImage (Linux)"
deps: [prepare]
dir: editor
cmds:
- npx tauri build --bundles appimage
- npx tauri build --bundles appimage --config '{"bundle":{"createUpdaterArtifacts":false}}'
test:
desc: "Run Tauri/Cargo tests"
deps: [prepare]
dir: editor/src-tauri
cmds:
- cargo test
clean:
desc: "Clean Tauri/Cargo build artifacts"
@@ -118,16 +127,38 @@ tasks:
cmds:
- rm -rf runtime/jre
- mkdir -p runtime
- >-
jlink
--add-modules {{.JLINK_MODULES}}
--strip-debug
--compress=zip-6
--no-header-files
--no-man-pages
--output runtime/jre
- |
JLINK_COMPRESS="$(jlink --help 2>&1 | grep -q 'zip-\[0-9\]' && echo zip-6 || echo 2)"
jlink \
--add-modules {{.JLINK_MODULES}} \
--strip-debug \
--compress="$JLINK_COMPRESS" \
--no-header-files \
--no-man-pages \
--output runtime/jre
# jlink emits its files mode 444 (read-only). Tauri's build-script
# resource copier preserves source permissions when staging
# `runtime/jre/**/*` into `target/<profile>/runtime/jre/...`, so the
# staged copies are read-only too. On any subsequent incremental
# build the copier tries to overwrite them and fails with a bare
# `Permission denied (os error 13)` (Rust's io::Error Display drops
# the path, so the failure is opaque). Make the source writable here
# so the staged destinations are writable and can be overwritten.
#
# Trade-off: this task runs for both `task desktop:dev` and
# `task desktop:build`, so production bundles also ship mode-644
# JRE files instead of 444. Functionally harmless on POSIX (the
# `other` bit is `r--` either way, and on macOS code signing is the
# real integrity check) and on Windows the DOS read-only attribute
# isn't load-bearing for the bundled JDK. If we ever need strict
# 444 in production, split the chmod into a dev-only step and have
# `desktop:build` run `jlink:clean` first to force a fresh build.
- cmd: chmod -R u+w runtime/jre
platforms: [linux, darwin]
- cmd: powershell -NoProfile -Command "Get-ChildItem -Recurse runtime/jre | ForEach-Object { $_.IsReadOnly = $false }"
platforms: [windows]
status:
- test -d editor/src-tauri/runtime/jre
- test -f runtime/jre/release
jlink:clean:
desc: "Remove JLink runtime and bundled JARs"
+5
View File
@@ -20,6 +20,11 @@ tasks:
cmds:
- docker build -t stirling-pdf-ultra-lite -f {{.EMBEDDED_DIR}}/Dockerfile.ultra-lite .
build:backend:
desc: "Build backend-only Docker image (no embedded frontend)"
cmds:
- docker build -t stirling-pdf-backend -f docker/backend/Dockerfile .
build:frontend:
desc: "Build frontend-only Docker image"
cmds:
+63 -1
View File
@@ -34,6 +34,9 @@ tasks:
ignore_error: true
vars:
BASE_DIR: '{{.ROOT_DIR}}/.test-state/playwright'
# COVERAGE=1 in the calling environment attaches the JaCoCo agent to
# the bootRun JVM and writes to BASE_DIR/jacoco.exec on shutdown.
# Off by default to keep local dev runs uninstrumented; CI flips it.
env:
STIRLING_BASE_PATH: '{{.BASE_DIR}}'
# Suppress the analytics opt-in modal that fires on first admin login.
@@ -58,12 +61,19 @@ tasks:
set -e
rm -rf "{{.BASE_DIR}}"
mkdir -p "{{.BASE_DIR}}"
GRADLE_ARGS=":stirling-pdf:bootRun"
if [ -n "${COVERAGE:-}" ]; then
# copyJacocoAgent is wired as a dependency of bootRun when
# -PjacocoAgent=true, so we do not need to invoke it separately.
GRADLE_ARGS="$GRADLE_ARGS -PjacocoAgent=true -PjacocoExec={{.BASE_DIR}}/jacoco.exec"
echo "JaCoCo coverage enabled, writing to {{.BASE_DIR}}/jacoco.exec"
fi
# Background gradle and record its PID so the runner can clean up
# the exact process tree (wrapper + forked Spring Boot JVM) without
# resorting to fuzzy `pkill -f` patterns. `wait` keeps this script
# alive for the lifetime of gradle so Task'"'"'s parallel deps stay
# synchronised.
bash gradlew :stirling-pdf:bootRun > "{{.BASE_DIR}}/backend.log" 2>&1 &
bash gradlew $GRADLE_ARGS > "{{.BASE_DIR}}/backend.log" 2>&1 &
GRADLE_PID=$!
echo $GRADLE_PID > "{{.BASE_DIR}}/backend.pid"
wait $GRADLE_PID
@@ -202,3 +212,55 @@ tasks:
desc: "Stop the SAML keycloak test environment"
cmds:
- docker compose -f testing/compose/docker-compose-keycloak-saml.yml down -v
mcp:up:
desc: "Start the MCP keycloak test environment (Stirling as OAuth resource server)"
summary: |
Brings up Keycloak (OAuth authorization server) + Stirling configured as an
MCP resource server, then you can exercise /mcp with real Keycloak tokens.
Set LICENSE_KEY=<KEY> to skip the interactive license prompt:
task e2e:mcp:up LICENSE_KEY=abc123
Pass extra flags via -- :
task e2e:mcp:up -- --validate --nobuild
ignore_error: true
cmds:
- bash testing/compose/start-mcp-test.sh {{if .LICENSE_KEY}}--license-key "{{.LICENSE_KEY}}"{{end}} {{.CLI_ARGS}}
mcp:manual:
desc: "Start the MCP keycloak test env in manual mode (prints URLs + a live token for your client)"
summary: |
Brings the stack up and prints copy-paste URLs/commands plus a freshly minted
access token so you can drive your own MCP client (Inspector, curl, ...).
task e2e:mcp:manual LICENSE_KEY=<your-license-key>
Add --nobuild if the images are already built:
task e2e:mcp:manual LICENSE_KEY=<your-license-key> -- --nobuild
ignore_error: true
cmds:
- bash testing/compose/start-mcp-test.sh --manual {{if .LICENSE_KEY}}--license-key "{{.LICENSE_KEY}}"{{end}} {{.CLI_ARGS}}
mcp:apikey:
desc: "Start the MCP test env in API-KEY manual mode (no OAuth/IdP): mints a key + prints client settings"
summary: |
Brings Stirling up in apikey auth mode and prints copy-paste client settings with a freshly
minted X-API-KEY - ideal for clients whose OAuth layer can't reach localhost.
task e2e:mcp:apikey LICENSE_KEY=<your-license-key>
Add --nobuild if images are already built:
task e2e:mcp:apikey LICENSE_KEY=<your-license-key> -- --nobuild
ignore_error: true
cmds:
- bash testing/compose/start-mcp-test.sh --apikey {{if .LICENSE_KEY}}--license-key "{{.LICENSE_KEY}}"{{end}} {{.CLI_ARGS}}
mcp:validate:
desc: "Validate the running MCP keycloak test environment end-to-end (oauth mode + real MCP SDK client)"
cmds:
- bash testing/compose/validate-mcp-test.sh
mcp:validate-apikey:
desc: "Validate the MCP server in API-KEY auth mode (mints a key + real MCP SDK client), then restore oauth"
cmds:
- bash testing/compose/validate-mcp-apikey.sh
mcp:down:
desc: "Stop the MCP keycloak test environment"
cmds:
- docker compose -f testing/compose/docker-compose-keycloak-mcp.yml down -v
+1 -1
View File
@@ -33,7 +33,7 @@ tasks:
env:
PYTHONUNBUFFERED: "1"
cmds:
- uv run uvicorn stirling.api.app:app --host 0.0.0.0 --port {{.PORT}}
- uv run uvicorn stirling.api.app:app --host 0.0.0.0 --port {{.PORT}} --workers "${STIRLING_ENGINE_WORKERS:-4}"
dev:
desc: "Start engine dev server with hot reload"
+118 -6
View File
@@ -40,6 +40,21 @@ tasks:
cmds:
- node editor/scripts/generate-icons.js
prepare:og:
internal: true
run: when_changed
desc: "Regenerate OG/social-preview metadata from the tool registry"
cmds:
- node editor/scripts/generate-og-metadata.mjs
sources:
- editor/src/core/types/toolId.ts
- editor/src/core/utils/urlMapping.ts
- editor/src/core/data/useTranslatedToolRegistry.tsx
- editor/public/og_images/*.png
generates:
- editor/src/core/data/ogImageMap.json
- editor/public/og-metadata.json
prepare:
desc: "Set up dev environment"
run: when_changed
@@ -49,6 +64,7 @@ tasks:
- task: prepare:env
vars: { MODE: '{{.MODE}}' }
- prepare:icons
- prepare:og
# ============================================================
# Development
@@ -112,6 +128,12 @@ tasks:
- task: dev:_run
vars: { MODE: prototypes, PORT: '{{.PORT}}', BACKEND_URL: '{{.BACKEND_URL}}', OPEN: '{{.OPEN}}' }
dev:portal:
desc: "Start developer portal dev server"
deps: [install]
cmds:
- npx vite portal --port {{.PORT | default "5173"}}{{if .OPEN}} --open{{end}}
# ============================================================
# Build
# ============================================================
@@ -156,6 +178,24 @@ tasks:
cmds:
- npx vite build editor --mode prototypes
build:portal:
desc: "Build developer portal"
deps: [install]
cmds:
- npx vite build portal
storybook:
desc: "Start Storybook dev server"
deps: [install]
cmds:
- npx storybook dev -p 6006 {{.CLI_ARGS}}
storybook:build:
desc: "Build static Storybook"
deps: [install]
cmds:
- npx storybook build {{.CLI_ARGS}}
# ============================================================
# Code quality
# ============================================================
@@ -163,9 +203,23 @@ tasks:
lint:
desc: "Run linting"
deps: [install]
cmds:
- task: lint:eslint
- task: lint:dpdm
lint:eslint:
desc: "Run ESLint linting"
deps: [install]
cmds:
- npx eslint --max-warnings=0
- npx dpdm editor/src --circular --no-warning --no-tree --exit-code circular:1
lint:dpdm:
desc: "Run circular import linting"
deps: [install]
cmds:
# Globs so dpdm walks the whole tree. dpdm expands the braces itself, so this is
# shell-agnostic. Covers editor, portal, and the shared design system.
- npx dpdm "editor/src/**/*.{ts,tsx}" "portal/src/**/*.{ts,tsx}" "shared/**/*.{ts,tsx}" --circular --no-warning --no-tree --exit-code circular:1
lint:fix:
desc: "Auto-fix lint issues"
@@ -224,6 +278,12 @@ tasks:
cmds:
- npx tsc --noEmit --project editor/src/desktop/tsconfig.json
typecheck:cloud:
desc: "Typecheck cloud shared layer (standalone)"
deps: [prepare]
cmds:
- npx tsc --noEmit --project editor/src/cloud/tsconfig.json
typecheck:scripts:
desc: "Typecheck scripts"
deps: [prepare]
@@ -236,6 +296,18 @@ tasks:
cmds:
- npx tsc --noEmit --project editor/src/prototypes/tsconfig.json
typecheck:portal:
desc: "Typecheck developer portal build variant"
deps: [install]
cmds:
- npx tsc --noEmit --project portal/tsconfig.json
typecheck:shared:
desc: "Typecheck the shared design system"
deps: [install]
cmds:
- npx tsc --noEmit --project shared/tsconfig.json
typecheck:all:
desc: "Typecheck all build variants"
cmds:
@@ -243,8 +315,11 @@ tasks:
- task: typecheck:proprietary
- task: typecheck:saas
- task: typecheck:desktop
- task: typecheck:cloud
- task: typecheck:scripts
- task: typecheck:prototypes
- task: typecheck:portal
- task: typecheck:shared
# ============================================================
# Quality Gate
@@ -258,14 +333,24 @@ tasks:
- task: format:check
- task: test
og:check:
desc: "Fail if committed OG/social-preview metadata is out of date"
cmds:
- node editor/scripts/generate-og-metadata.mjs --check
check:all:
desc: "Full CI quality gate"
cmds:
# Runs first, before prepare regenerates: guards the committed og-metadata.json /
# ogImageMap.json that the Cloudflare Pages (plain `vite build`) deploy relies on.
- task: og:check
- task: typecheck:all
- task: lint
- task: format:check
- task: build
- task: build:portal
- task: test
- task: storybook:build
# ============================================================
# Test
@@ -273,21 +358,36 @@ tasks:
test:
desc: "Run tests"
deps: [install]
deps: [prepare]
cmds:
- npx vitest run --root editor
test:watch:
desc: "Run tests in watch mode"
deps: [install]
deps: [prepare]
cmds:
- npx vitest --watch --root editor
test:coverage:
desc: "Run tests with coverage"
deps: [install]
desc: "Run tests with coverage (one-shot; CI-friendly)."
deps: [prepare]
cmds:
- npx vitest --coverage --root editor
# `vitest run` makes this CI-safe (the bare `vitest` form enters watch
# mode). Explicit reporter list because v8 + json-summary is what the
# coverage-summary.py helper consumes; html/text are kept for humans.
#
# reportsDirectory is pinned to ./coverage relative to vitest's root
# (--root editor), so output lands at frontend/editor/coverage/. The
# CI upload step reads from that path. An earlier attempt with
# `./editor/coverage` double-nested into frontend/editor/editor/coverage;
# pinning future-proofs against vitest changing the default.
- >
npx vitest run --root editor --coverage
--coverage.provider=v8
--coverage.reporter=text-summary
--coverage.reporter=json-summary
--coverage.reporter=html
--coverage.reportsDirectory=./coverage
# ============================================================
# Code Generation
@@ -298,3 +398,15 @@ tasks:
deps: [install]
cmds:
- node editor/scripts/generate-licenses.js
# ============================================================
# Clean
# ============================================================
clean:
desc: "Clean build artifacts and caches"
cmds:
- cmd: powershell rm -Recurse -Force -ErrorAction SilentlyContinue node_modules/.vite, editor/dist, dist, dist-portal
platforms: [windows]
- cmd: rm -rf node_modules/.vite editor/dist dist dist-portal
platforms: [linux, darwin]
+159
View File
@@ -0,0 +1,159 @@
version: '3'
# Repo-wide lint/format/secret checks - the single source of truth that the git
# pre-commit hook (.pre-commit-config.yaml) and CI (pre_commit.yml) both call.
vars:
GITLEAKS: '8.30.0'
# File selections as git pathspecs: git does the include/exclude matching, so
# there is no grep/xargs and it behaves identically on every platform.
PY_FILES: >-
'scripts/*.py'
'.github/scripts/*.py'
'app/core/src/main/resources/static/python/*.py'
':(exclude)*split_photos.py'
SPELL_FILES: >-
'*.html'
'*.css'
'*.js'
'*.py'
'*.md'
':(exclude).vscode/*'
':(exclude).devcontainer/*'
':(exclude)app/core/src/main/resources/*'
':(exclude)app/proprietary/src/main/resources/*'
':(exclude)frontend/editor/public/vendor/*'
':(exclude)*Dockerfile*'
':(exclude)*pdfjs*'
':(exclude)*thirdParty*'
':(exclude)*bootstrap*'
':(exclude)*.min.*'
':(exclude)*diff.js'
WS_FILES: >-
'*.js'
'*.java'
'*.py'
'*.yml'
':(exclude)*pdfjs*'
':(exclude)*thirdParty*'
':(exclude)*bootstrap*'
':(exclude)*.min.*'
':(exclude)*diff.js'
':(exclude).github/workflows/*'
LOCALE_TOML: 'frontend/editor/public/locales/*/translation.toml'
GITLEAKS_BIN: '.task/bin/gitleaks-{{.GITLEAKS}}{{if eq OS "windows"}}.exe{{end}}'
tasks:
default:
desc: "Check formatting, spelling, and secrets across the repo"
cmds:
- task: ruff
- task: ruff-format
- task: codespell
- task: gitleaks
- task: whitespace
- task: toml-sort
fix:
desc: "Auto-fix formatting, spelling, and secrets issues across the repo"
cmds:
# Auto-fixers first, then the report-only tools (codespell, gitleaks) so a
# finding there does not stop the fixers from running.
- task: ruff
vars: { FIX: '1' }
- task: ruff-format
vars: { FIX: '1' }
- task: whitespace
vars: { FIX: '1' }
- task: toml-sort
vars: { FIX: '1' }
- task: codespell
- task: gitleaks
install:
desc: "Install the pinned pre-commit Python tools (ruff, codespell, toml-sort)"
run: once
cmds:
- uv sync --project scripts/pre-commit --locked
sources:
- scripts/pre-commit/uv.lock
- scripts/pre-commit/pyproject.toml
status:
- test -d scripts/pre-commit/.venv
clean:
desc: "Remove the cache/build artifacts"
cmds:
- task: '{{if eq OS "windows"}}clean-windows{{else}}clean-unix{{end}}'
clean-unix:
internal: true
cmds:
- rm -rf scripts/pre-commit/.venv .task/bin/gitleaks-*
# On Windows, use PowerShell so it matches the same paths and tolerates absent
# files without erroring.
clean-windows:
internal: true
ignore_error: true
cmds:
- powershell -NoProfile -Command "Remove-Item -Recurse -Force -ErrorAction SilentlyContinue scripts/pre-commit/.venv, .task/bin/gitleaks-*"
# Individual checks (hidden from `task --list`, but callable, e.g.
# `task pre-commit:toml-sort FIX=1`). Pass FIX=1 to auto-fix where supported.
ruff:
deps: [install]
cmds:
- uv run --project scripts/pre-commit --no-sync ruff check --line-length=127 {{if .FIX}}--fix {{end}}$(git ls-files {{.PY_FILES}})
ruff-format:
deps: [install]
cmds:
- uv run --project scripts/pre-commit --no-sync ruff format {{if .FIX}}{{else}}--check {{end}}$(git ls-files {{.PY_FILES}})
codespell:
deps: [install]
cmds:
- uv run --project scripts/pre-commit --no-sync codespell --ignore-words-list=thirdParty,tabEl,tabEls,Sie,ist,fulfilment --quiet-level=2 $(git ls-files {{.SPELL_FILES}})
toml-sort:
deps: [install]
cmds:
- uv run --project scripts/pre-commit --no-sync toml-sort --all --ignore-case {{if .FIX}}--in-place{{else}}--check{{end}} {{.LOCALE_TOML}}
whitespace:
cmds:
- uv run --no-project python scripts/pre-commit/whitespace.py {{if .FIX}}--fix {{end}}$(git ls-files {{.WS_FILES}})
gitleaks:
deps: [gitleaks-bin]
# Scan staged changes only, matching the old hook: the git-mode fingerprints
# in .gitleaksignore (file:rule:line) still apply, and with nothing staged
# this is a no-op. Secrets are never auto-fixed, so FIX has no effect.
cmds:
- "{{.GITLEAKS_BIN}} git --pre-commit --redact --staged --verbose"
gitleaks-bin:
internal: true
desc: "Ensure the pinned gitleaks binary is cached in .task/bin"
status:
- test -f {{.GITLEAKS_BIN}}
vars:
GL_ARCH: '{{if eq ARCH "amd64"}}x64{{else if eq ARCH "arm64"}}arm64{{else if eq ARCH "386"}}x32{{else}}{{ARCH}}{{end}}'
GL_PLATFORM: '{{OS}}_{{.GL_ARCH}}'
GL_URL: 'https://github.com/gitleaks/gitleaks/releases/download/v{{.GITLEAKS}}/gitleaks_{{.GITLEAKS}}_{{.GL_PLATFORM}}'
# SHA-256 of each release asset, from gitleaks_{{.GITLEAKS}}_checksums.txt.
GL_SHA: >-
{{if eq .GL_PLATFORM "linux_x64"}}79a3ab579b53f71efd634f3aaf7e04a0fa0cf206b7ed434638d1547a2470a66e
{{- else if eq .GL_PLATFORM "linux_arm64"}}b4cbbb6ddf7d1b2a603088cd03a4e3f7ce48ee7fd449b51f7de6ee2906f5fa2f
{{- else if eq .GL_PLATFORM "darwin_x64"}}ca221d012d247080c2f6f61f4b7a83bffa2453806b0c195c795bbe9a8c775ed5
{{- else if eq .GL_PLATFORM "darwin_arm64"}}b251ab2bcd4cd8ba9e56ff37698c033ebf38582b477d21ebd86586d927cf87e7
{{- else if eq .GL_PLATFORM "windows_x64"}}54fe94f644b832dd08e8c3a5915efb3bfa862386d59fb27ca0792cb687a83573
{{- end}}
cmds:
- cmd: bash scripts/pre-commit/install-gitleaks.sh "{{.GL_URL}}.tar.gz" "{{.GL_SHA}}" "{{.GITLEAKS_BIN}}"
platforms: [linux, darwin]
- cmd: powershell -NoProfile -File scripts/pre-commit/install-gitleaks.ps1 -Url "{{.GL_URL}}.zip" -Sha "{{.GL_SHA}}" -Dest "{{.GITLEAKS_BIN}}"
platforms: [windows]
+3 -3
View File
@@ -200,9 +200,9 @@ const [ToolName] = (props: BaseToolProps) => {
```
## 5. Add Translations
Update translation files. **Important: Only update `en-GB` files** - other languages are handled separately.
Update translation files. **Important: Only update `en-US` files** - other languages are handled separately.
**File to update:** `frontend/editor/public/locales/en-GB/translation.toml`
**File to update:** `frontend/editor/public/locales/en-US/translation.toml`
**Required Translation Keys**:
```toml
@@ -251,7 +251,7 @@ Update translation files. **Important: Only update `en-GB` files** - other langu
```
**Translation Notes:**
- **Only update `en-GB/translation.toml`** - other locale files are managed separately
- **Only update `en-US/translation.toml`** - other locale files are managed separately
- Use descriptive keys that match your component's `t()` calls
- Include tooltip translations if you created tooltip hooks
- Add `options.*` keys if your tool has settings with descriptions
+26 -2
View File
@@ -169,7 +169,31 @@ import { useFileContext } from "@proprietary/contexts/FileContext";
- Building layer-specific override that wraps a lower layer's component
- Example: `import { AppProviders as CoreAppProviders } from "@core/components/AppProviders"` when creating proprietary/AppProviders.tsx that extends the core version
The `@app/*` alias automatically resolves to the correct layer based on build target (core/proprietary/desktop) and handles the fallback cascade.
The `@app/*` alias automatically resolves to the correct layer based on build target (core/proprietary/saas/desktop/cloud) and handles the fallback cascade — see "Frontend `cloud/` Layer" below for the full per-flavor order.
#### Frontend `cloud/` Layer
`@app/*` resolves through a per-flavor cascade — first existing file wins (shadow/override):
- **core** → core
- **proprietary** → proprietary → core
- **saas** → saas → cloud → proprietary → core
- **desktop** → desktop → cloud → proprietary → core
- **cloud** → cloud → proprietary → core
What goes where:
- **core** — OSS base.
- **proprietary** — licensed / offline features.
- **cloud** — the SHARED hosted/SaaS experience used by BOTH saas + desktop: PAYG, wallet, plan, billing, usage meters, cloud config/team/onboarding.
- **saas** — web-only: Supabase web auth, AuthCallback, avatar canvas, `window.location`.
- **desktop** — Tauri-only: keyring authService, tauriHttpClient, native files/windows, backend routing.
`cloud/` MUST NOT import `@supabase/*`, `@tauri-apps/*`, raw `fetch`, `window.location`, `localStorage`, `sessionStorage`, or `import.meta.env.VITE_*` (enforced by ESLint). It reaches platform-specific things only via `@app/*` seams: `services/apiClient`, `auth/session.getAccessToken`, `auth/supabase`, `platform/openExternal`, `services/billing`, `hooks/useSaaSMode` — each provided per-platform in `saas/` and `desktop/`.
Rule of thumb — **move, don't copy**: share via `cloud/`, override by shadowing the same `@app/*` path in a leaf (`saas/` or `desktop/`).
**Cloud feature flags on desktop.** The local `AppConfigContext` reads `/api/v1/config/app-config` from the LOCAL bundled backend, so cloud-only flags (`aiEngineEnabled`, `premiumEnabled`, …) are never seen on desktop. To read the cloud's view, use `useSaasAppConfig()` (`desktop/hooks/useSaasAppConfig.ts`, backed by the general `saasAppConfigService` — SaaS-mode-only, public endpoint, native HTTP, 5-min cache). It returns `null` outside SaaS mode, so cloud features stay off in local/self-hosted and the server keeps the on/off switch (no desktop release needed to flip a flag). Gate a feature behind a per-platform seam — e.g. `useAiEngineEnabled()` (core reads `useAppConfig()`, desktop reads `useSaasAppConfig()`) — rather than hardcoding the flag on.
#### Component Override Pattern (Stub/Shadow)
Use this pattern for desktop-specific or proprietary-specific features WITHOUT runtime checks or conditionals.
@@ -426,7 +450,7 @@ The frontend is organized with a clear separation of concerns:
## Translation Rules
- **CRITICAL**: Always update translations in `en-GB` only, never `en-US`
- **CRITICAL**: Always update translations in `en-US` only - all other languages (including `en-GB`) are handled separately
- Translation files are located in `frontend/editor/public/locales/`
## Important Notes
+12 -1
View File
@@ -52,6 +52,17 @@ This guide focuses on developing for Stirling 2.0, including both the React fron
- Rust and Cargo (required for Tauri desktop app development)
- Tauri CLI (install with `cargo install tauri-cli`)
### Optional System Dependencies
These are not required to run the app but enable specific features. The app detects them at startup and disables the relevant features if they are missing.
| Dependency | Feature | Install |
|---|---|---|
| LibreOffice | File-to-PDF conversions | `brew install libreoffice` / `apt install libreoffice` |
| Tesseract | OCR | `brew install tesseract` / `apt install tesseract-ocr` |
| WeasyPrint | AI document creation | `brew install weasyprint` / `apt install weasyprint` |
| qpdf | PDF optimisation | `brew install qpdf` / `apt install qpdf` |
### Setup Steps
1. Clone the repository:
@@ -576,7 +587,7 @@ When adding a new feature or modifying existing ones in Stirling-PDF, you'll nee
Find the existing `messages.properties` files in the `stirling-pdf/src/main/resources` directory. You'll see files like:
- `messages.properties` (default, usually English)
- `messages_en_GB.properties`
- `messages_en_US.properties`
- `messages_fr_FR.properties`
- `messages_de_DE.properties`
- etc.
+12 -8
View File
@@ -10,14 +10,18 @@ if that directory exists, is licensed under the license defined in "app/propriet
if that directory exists, is licensed under the license defined in "app/saas/LICENSE".
* All content that resides under the "engine/" directory of this repository,
if that directory exists, is licensed under the license defined in "engine/LICENSE".
* All content that resides under the "frontend/src/proprietary/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/src/proprietary/LICENSE".
* All content that resides under the "frontend/src/desktop/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/src/desktop/LICENSE".
* All content that resides under the "frontend/src/saas/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/src/saas/LICENSE".
* All content that resides under the "frontend/src/prototypes/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/src/prototypes/LICENSE".
* All content that resides under the "frontend/editor/src/proprietary/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/editor/src/proprietary/LICENSE".
* All content that resides under the "frontend/editor/src/desktop/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/editor/src/desktop/LICENSE".
* All content that resides under the "frontend/editor/src/saas/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/editor/src/saas/LICENSE".
* All content that resides under the "frontend/editor/src/cloud/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/editor/src/cloud/LICENSE".
* All content that resides under the "frontend/editor/src/prototypes/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/editor/src/prototypes/LICENSE".
* All content that resides under the "frontend/portal/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/portal/LICENSE".
* Content outside of the above mentioned directories or restrictions above is
available under the MIT License as defined below.
+21 -2
View File
@@ -25,6 +25,9 @@ includes:
e2e:
taskfile: .taskfiles/e2e.yml
dir: .
pre-commit:
taskfile: .taskfiles/pre-commit.yml
dir: .
tasks:
# ============================================================
@@ -58,9 +61,22 @@ tasks:
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
OPEN: "true"
dev:saas:
desc: "Start SaaS backend + frontend concurrently on free ports"
cmds:
- task: dev:_all
vars: { FRONTEND: saas, BACKEND: saas }
dev:all:
desc: "Start backend + frontend + engine concurrently on free ports"
cmds:
- task: dev:_all
dev:_all:
internal: true
vars:
FRONTEND: '{{.FRONTEND | default "proprietary"}}'
BACKEND: '{{.BACKEND | default "proprietary"}}'
PORTS:
sh: '{{if eq OS "windows"}}{{.FIND_FREE_PORT_PS}} 8080 5173 5001{{else}}{{.FIND_FREE_PORT_SH}} 8080 5173 5001{{end}}'
BACKEND_PORT: '{{index (splitList "\n" .PORTS) 0}}'
@@ -70,11 +86,12 @@ tasks:
- task: engine:dev
vars:
PORT: '{{.ENGINE_PORT}}'
- task: backend:dev
- task: 'backend:dev:{{.BACKEND}}'
vars:
PORT: '{{.BACKEND_PORT}}'
AIENGINE_URL: 'http://localhost:{{.ENGINE_PORT}}'
- task: frontend:dev:prototypes
AIENGINE_ENABLED: "true"
- task: 'frontend:dev:{{.FRONTEND}}'
vars:
PORT: '{{.FRONTEND_PORT}}'
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
@@ -158,4 +175,6 @@ tasks:
desc: "Clean all build artifacts"
cmds:
- task: backend:clean
- task: frontend:clean
- task: engine:clean
- task: pre-commit:clean
+9 -9
View File
@@ -1,20 +1,20 @@
###############################################################################
# Stirling-PDF SaaS local environment template.
# Stirling-PDF SaaS environment defaults.
#
# Copy this file to `.env.saas.local` (gitignored) and fill in real values.
# Loaded by `task backend:dev:saas` via Taskfile's `dotenv:` directive, then
# read by Spring Boot's `${...}` placeholders in application-saas.properties
# and application-dev.properties.
# This file is committed and provides non-secret defaults loaded by
# `task backend:dev:saas`. Put real values for secrets (passwords, project
# refs, edge function secrets) in `.env.saas.local` - any variable set there
# takes precedence over what's defined here.
#
# DO NOT commit `.env.saas.local`. Only `.env.saas.example` is checked in.
# DO NOT commit `.env.saas.local`. Only `.env.saas` is checked in.
###############################################################################
# ---------- Supabase project ----------
# Project reference (the subdomain part of <ref>.supabase.co). Required.
# Example dev project:
# Set in .env.saas.local.
SAAS_DB_PROJECT_REF=
# Edge function secret used by billing/license rollup calls.
# Edge function secret used by billing/license rollup calls. Set in .env.saas.local.
SUPABASE_EDGE_FUNCTION_SECRET=
# ---------- Database (saas profile) ----------
@@ -28,7 +28,7 @@ SAAS_DB_PASSWORD=
# ---------- Database (dev profile overrides) ----------
# Used when `--spring.profiles.include=dev` is active. The dev profile
# defaults the URL/username to the shared dev Supabase project, but the
# password must still be provided here.
# password must still be provided in .env.saas.local.
SAAS_DEV_DB_URL=
SAAS_DEV_DB_USERNAME=postgres
SAAS_DEV_DB_PASSWORD=
+3
View File
@@ -0,0 +1,3 @@
# Whitelist committed env defaults. `.env.saas.local` (and any other .env*)
# stays ignored via the root .gitignore.
!.env.saas
+4
View File
@@ -44,6 +44,10 @@
"moduleName": ".*",
"moduleLicense": "The MIT License"
},
{
"moduleName": ".*",
"moduleLicense": "MIT-0"
},
{
"moduleName": "com.github.jai-imageio:jai-imageio-core",
"moduleLicense": "LICENSE.txt"
+2 -2
View File
@@ -60,7 +60,7 @@ dependencies {
exclude group: 'com.google.code.gson', module: 'gson'
}
api 'com.stirling:jpdfium:1.0.1'
api 'com.stirling:jpdfium:1.0.2'
// -PjpdfiumPlatforms=all|<csv of linux-x64,linux-arm64,darwin-x64,darwin-arm64,windows-x64>
def jpdfiumPlatformsProp = (project.findProperty('jpdfiumPlatforms') ?: 'all').toString().trim()
@@ -75,7 +75,7 @@ dependencies {
}
logger.lifecycle("JPDFium native platforms: ${jpdfiumPlatforms.join(', ')}")
jpdfiumPlatforms.each { platform ->
runtimeOnly "com.stirling:jpdfium-natives-${platform}:1.0.1"
runtimeOnly "com.stirling:jpdfium-natives-${platform}:1.0.2"
}
// Bucket4j (local in-process token bucket for RateLimitStore default impl)
@@ -1,73 +0,0 @@
package stirling.software.SPDF.pdf.parser;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.io.IOException;
import java.util.List;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.springframework.context.annotation.Primary;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
/**
* Chains table parsers in priority order: Tabula lattice → Tabula stream → {@link
* LineAlignmentTableParser}. The first parser returning a result above {@link
* #TABULA_CONFIDENCE_THRESHOLD} wins; results from different parsers are never mixed on one page.
*/
@Service
@Primary
@RequiredArgsConstructor
@Slf4j
public class CompositeTableParser implements TableParser {
/** Min Tabula confidence to accept results; below this LineAlignment is tried instead. */
static final float TABULA_CONFIDENCE_THRESHOLD = 0.5f;
private final TabulaTableParser tabulaParser;
private final LineAlignmentTableParser lineAlignmentParser;
@Override
public List<TableFragment> parse(PDDocument document, RawPage rawPage) throws IOException {
// Step 1: Tabula lattice mode (ruled/bordered tables).
List<TableFragment> latticeResults = filterConfident(tabulaParser.parse(document, rawPage));
if (!latticeResults.isEmpty()) {
log.debug(
"Page {}: using Tabula lattice ({} table(s))",
rawPage.pageNumber(),
latticeResults.size());
return latticeResults;
}
// Step 2: Tabula stream mode (borderless/whitespace-delimited tables).
// parseStream is not on the TableParser interface — this intentionally couples to the
// concrete TabulaTableParser since stream mode is a Tabula-specific concept.
List<TableFragment> streamResults =
filterConfident(tabulaParser.parseStream(document, rawPage));
if (!streamResults.isEmpty()) {
log.debug(
"Page {}: using Tabula stream ({} table(s))",
rawPage.pageNumber(),
streamResults.size());
return streamResults;
}
// Step 3: Geometry-based line-alignment fallback.
List<TableFragment> lineResults = lineAlignmentParser.parse(document, rawPage);
if (!lineResults.isEmpty()) {
log.debug(
"Page {}: using LineAlignment ({} table(s))",
rawPage.pageNumber(),
lineResults.size());
return lineResults;
}
return List.of();
}
private List<TableFragment> filterConfident(List<TableFragment> tables) {
return tables.stream().filter(t -> t.confidence() >= TABULA_CONFIDENCE_THRESHOLD).toList();
}
}
@@ -1,528 +0,0 @@
package stirling.software.SPDF.pdf.parser;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.TreeMap;
import java.util.regex.Pattern;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.springframework.stereotype.Service;
import lombok.extern.slf4j.Slf4j;
/**
* Fallback {@link TableParser} for borderless financial tables using text geometry.
*
* <p>Identifies "anchor lines" (≥2 numeric tokens), builds a column grid from their right-edge
* positions, groups vertically proximate anchor lines into table candidates, then scores each group
* on column consistency and anchor density (confidence ceiling 0.85).
*/
@Service
@Slf4j
public class LineAlignmentTableParser implements TableParser {
/** Width in points of each column position bucket. */
static final float COLUMN_BUCKET_PT = 5f;
/** Tolerance in buckets when matching a token's right-edge to a confirmed column position. */
private static final int COLUMN_MATCH_BUCKETS = 2;
/** Maximum gap (as a multiple of modal line spacing) before splitting a group. */
private static final float MAX_GAP_FACTOR = 2.5f;
/** Minimum anchor rows (numeric-heavy) to form a valid table. */
static final int MIN_TABLE_ROWS = 3;
/** Minimum confirmed column positions to form a valid table. */
static final int MIN_COLUMNS = 2;
/**
* Min fraction of anchor lines a column must appear on to be confirmed (permissive for N/A
* rows).
*/
private static final double COLUMN_MIN_FREQUENCY = 0.40;
/**
* Matches financial numeric tokens: integers, decimals, parenthetical negatives, currency,
* percent, nil dashes.
*/
private static final Pattern NUMERIC =
Pattern.compile("^[\\(\\-\\$£€¥]?\\d[\\d,\\.]*[\\)%]?$|^[-–—]$");
/**
* Lines within this y-distance are merged into one row (restores rows split by LineBuilder's
* column-gap logic).
*/
static final float ROW_MERGE_TOLERANCE_PT = 2f;
// ── public API ───────────────────────────────────────────────────────────────────────────────
@Override
public List<TableFragment> parse(PDDocument document, RawPage rawPage) throws IOException {
List<RawLine> lines = rawPage.lines();
if (lines.size() < MIN_TABLE_ROWS) return List.of();
float modalSpacing = computeModalSpacing(lines);
List<TokenizedLine> tokenized =
mergeCoincidentLines(lines.stream().map(this::tokenize).toList());
List<TokenizedLine> anchors = tokenized.stream().filter(TokenizedLine::isAnchor).toList();
if (anchors.size() < MIN_TABLE_ROWS) return List.of();
List<Float> columnGrid = buildColumnGrid(anchors);
if (columnGrid.size() < MIN_COLUMNS) {
log.debug(
"Page {}: LineAlignment — fewer than {} confirmed columns, skipping",
rawPage.pageNumber(),
MIN_COLUMNS);
return List.of();
}
List<List<TokenizedLine>> groups = groupRows(tokenized, columnGrid, modalSpacing);
List<TableFragment> results = new ArrayList<>();
for (int i = 0; i < groups.size(); i++) {
buildFragment(groups.get(i), columnGrid, rawPage.pageNumber(), i)
.ifPresent(results::add);
}
log.debug(
"Page {}: LineAlignment detected {} table(s) ({} anchor lines, {} columns)",
rawPage.pageNumber(),
results.size(),
anchors.size(),
columnGrid.size());
return results;
}
// ── coincident-line merging ──────────────────────────────────────────────────────────────────
/**
* Merges tokenised lines sharing the same y-position into one row, rejoining label/value halves
* split by LineBuilder.
*/
List<TokenizedLine> mergeCoincidentLines(List<TokenizedLine> tokenized) {
if (tokenized.size() < 2) return tokenized;
List<TokenizedLine> result = new ArrayList<>();
int i = 0;
while (i < tokenized.size()) {
float baseY = tokenized.get(i).line().bounds().y();
int j = i + 1;
while (j < tokenized.size()
&& Math.abs(tokenized.get(j).line().bounds().y() - baseY)
<= ROW_MERGE_TOLERANCE_PT) {
j++;
}
if (j == i + 1) {
result.add(tokenized.get(i));
} else {
result.add(mergeGroup(tokenized.subList(i, j)));
}
i = j;
}
return result;
}
private TokenizedLine mergeGroup(List<TokenizedLine> group) {
List<TextFragment> mergedFragments =
group.stream()
.flatMap(tl -> tl.line().fragments().stream())
.sorted(Comparator.comparingDouble(f -> f.bounds().x()))
.toList();
Bounds mergedBounds =
group.stream()
.map(tl -> tl.line().bounds())
.reduce(Bounds::merge)
.orElse(group.get(0).line().bounds());
RawLine mergedLine =
new RawLine(
group.get(0).line().lineId(),
mergedFragments,
mergedBounds,
group.get(0).line().pageNumber());
return tokenize(mergedLine);
}
// ── tokenisation ─────────────────────────────────────────────────────────────────────────────
/**
* Splits fragments into word-level tokens; x-positions are estimated linearly within each
* fragment.
*/
TokenizedLine tokenize(RawLine line) {
List<LineToken> tokens = new ArrayList<>();
for (TextFragment frag : line.fragments()) {
tokens.addAll(tokensFromFragment(frag));
}
List<LineToken> numeric = tokens.stream().filter(LineToken::numeric).toList();
return new TokenizedLine(line, tokens, numeric);
}
private List<LineToken> tokensFromFragment(TextFragment frag) {
String raw = frag.text();
if (raw == null || raw.isBlank()) return List.of();
float fragX = frag.bounds().x();
float fragWidth = frag.bounds().width();
int rawLen = raw.length();
List<LineToken> result = new ArrayList<>();
int offset = 0;
for (String part : raw.split("\\s+")) {
if (part.isEmpty()) {
offset++;
continue;
}
int idx = raw.indexOf(part, offset);
if (idx < 0) idx = offset;
float tokenX = rawLen > 0 ? fragX + ((float) idx / rawLen) * fragWidth : fragX;
float tokenRight =
rawLen > 0
? fragX + ((float) (idx + part.length()) / rawLen) * fragWidth
: fragX + fragWidth;
result.add(new LineToken(part, tokenX, tokenRight, NUMERIC.matcher(part).matches()));
offset = idx + part.length();
}
return result;
}
// ── column grid ──────────────────────────────────────────────────────────────────────────────
/**
* Returns confirmed column right-edge positions — those appearing on ≥ {@value
* #COLUMN_MIN_FREQUENCY} × N anchor lines.
*/
private List<Float> buildColumnGrid(List<TokenizedLine> anchors) {
// bucket → set of line indices that contributed a numeric token to that bucket
Map<Integer, List<Integer>> bucketLines = new HashMap<>();
for (int i = 0; i < anchors.size(); i++) {
for (LineToken t : anchors.get(i).numeric()) {
int bucket = bucket(t.right());
bucketLines.computeIfAbsent(bucket, k -> new ArrayList<>()).add(i);
}
}
int minHits =
Math.max(MIN_TABLE_ROWS, (int) Math.ceil(anchors.size() * COLUMN_MIN_FREQUENCY));
// Confirmed buckets → average right-edge for that bucket
TreeMap<Integer, Float> confirmed = new TreeMap<>();
for (Map.Entry<Integer, List<Integer>> entry : bucketLines.entrySet()) {
// Count distinct lines
long distinctLines = entry.getValue().stream().distinct().count();
if (distinctLines >= minHits) {
double avg =
entry.getValue().stream()
.distinct() // weight each line equally regardless of token count
.mapToDouble(
lineIdx ->
avgRightEdgeForBucket(
anchors, lineIdx, entry.getKey()))
.average()
.orElse(entry.getKey() * (double) COLUMN_BUCKET_PT);
confirmed.put(entry.getKey(), (float) avg);
}
}
return new ArrayList<>(confirmed.values()); // already sorted by bucket (left to right)
}
/**
* Returns the average right-edge position of tokens in {@code line} whose bucket matches {@code
* targetBucket}, falling back to the bucket's nominal centre when no tokens match.
*/
private double avgRightEdgeForBucket(
List<TokenizedLine> anchors, int lineIdx, int targetBucket) {
return anchors.get(lineIdx).numeric().stream()
.filter(t -> bucket(t.right()) == targetBucket)
.mapToDouble(LineToken::right)
.average()
.orElse(targetBucket * (double) COLUMN_BUCKET_PT);
}
// ── grouping ─────────────────────────────────────────────────────────────────────────────────
/**
* Groups anchor lines into table candidates, including adjacent label rows; a gap &gt;
* MAX_GAP_FACTOR × modal spacing splits groups.
*/
private List<List<TokenizedLine>> groupRows(
List<TokenizedLine> all, List<Float> columnGrid, float modalSpacing) {
float maxGap = modalSpacing > 0 ? modalSpacing * MAX_GAP_FACTOR : 30f;
List<List<TokenizedLine>> groups = new ArrayList<>();
List<TokenizedLine> current = new ArrayList<>();
for (int i = 0; i < all.size(); i++) {
TokenizedLine tl = all.get(i);
boolean fits = tl.isAnchor() && matchesGrid(tl, columnGrid);
if (current.isEmpty()) {
if (fits) current.add(tl);
continue;
}
float gap =
tl.line().bounds().y()
- current.get(current.size() - 1).line().bounds().bottom();
if (gap > maxGap) {
groups.add(current);
current = new ArrayList<>();
if (fits) current.add(tl);
continue;
}
if (fits) {
current.add(tl);
} else if (!tl.line().text().isBlank()) {
// Include non-anchor lines (labels) only if they have text and are within
// proximity.
current.add(tl);
}
}
if (!current.isEmpty()) groups.add(current);
return groups.stream().filter(g -> hasEnoughAnchorRows(g, columnGrid)).toList();
}
private boolean hasEnoughAnchorRows(List<TokenizedLine> group, List<Float> columnGrid) {
return group.stream().filter(r -> r.isAnchor() && matchesGrid(r, columnGrid)).count()
>= MIN_TABLE_ROWS;
}
/** A line "matches" the grid when ≥ 60 % of its numeric tokens land in confirmed columns. */
private boolean matchesGrid(TokenizedLine tl, List<Float> columnGrid) {
if (tl.numeric().isEmpty()) return false;
long matches =
tl.numeric().stream()
.filter(t -> nearestColumnIndex(t.right(), columnGrid) >= 0)
.count();
return (double) matches / tl.numeric().size() >= 0.60;
}
private boolean hasInconsistentColumnMatch(TokenizedLine tl, List<Float> columnGrid) {
if (tl.numeric().isEmpty()) return false;
long hits =
tl.numeric().stream()
.filter(t -> nearestColumnIndex(t.right(), columnGrid) >= 0)
.count();
return (double) hits / tl.numeric().size() < 0.60;
}
// ── fragment assembly ────────────────────────────────────────────────────────────────────────
private Optional<TableFragment> buildFragment(
List<TokenizedLine> group, List<Float> columnGrid, int pageNumber, int tableIndex) {
long anchorCount =
group.stream().filter(r -> r.isAnchor() && matchesGrid(r, columnGrid)).count();
if (anchorCount < MIN_TABLE_ROWS) return Optional.empty();
List<String> warnings = new ArrayList<>();
List<List<String>> rawRows = new ArrayList<>();
List<TableRow> rows = new ArrayList<>();
for (int rowIdx = 0; rowIdx < group.size(); rowIdx++) {
TokenizedLine tl = group.get(rowIdx);
List<String> rawRow = buildRawRow(tl, columnGrid);
rawRows.add(Collections.unmodifiableList(rawRow));
rows.add(buildTableRow(rowIdx, tl, rawRow, columnGrid));
}
// Column count = 1 label column + confirmed numeric columns
int colCount = columnGrid.size() + 1;
Bounds bounds = computeGroupBounds(group);
float confidence = computeConfidence(group, columnGrid, warnings);
return Optional.of(
new TableFragment(
"tbl-la-p" + pageNumber + "-" + tableIndex,
pageNumber,
bounds,
List.of(),
Collections.unmodifiableList(rows),
Collections.unmodifiableList(rawRows),
colCount,
confidence,
Collections.unmodifiableList(warnings),
null));
}
/**
* Builds a raw row as a list of strings: index 0 = label text, indices 1..N = column values.
*/
private List<String> buildRawRow(TokenizedLine tl, List<Float> columnGrid) {
String[] cells = new String[columnGrid.size() + 1];
Arrays.fill(cells, "");
// Separate label tokens (those not landing in any confirmed column) from column tokens.
List<String> labelParts = new ArrayList<>();
for (LineToken token : tl.all()) {
int col = nearestColumnIndex(token.right(), columnGrid);
if (col >= 0 && token.numeric()) {
int cellIdx = col + 1;
cells[cellIdx] =
cells[cellIdx].isEmpty()
? token.text()
: cells[cellIdx] + " " + token.text();
} else {
labelParts.add(token.text());
}
}
cells[0] = String.join(" ", labelParts).trim();
return Arrays.asList(cells);
}
private TableRow buildTableRow(
int rowIdx, TokenizedLine tl, List<String> rawRow, List<Float> columnGrid) {
List<TableCell> cells = new ArrayList<>(rawRow.size());
// Label cell: use the line's full bounds as an approximation.
cells.add(TableCell.of(0, rawRow.get(0), tl.line().bounds()));
for (int col = 0; col < columnGrid.size(); col++) {
String text = col + 1 < rawRow.size() ? rawRow.get(col + 1) : "";
float right = columnGrid.get(col);
float left = col > 0 ? columnGrid.get(col - 1) : right - 50f;
Bounds cellBounds =
new Bounds(
left,
tl.line().bounds().y(),
right - left,
tl.line().bounds().height());
cells.add(TableCell.of(col + 1, text, cellBounds));
}
return new TableRow(rowIdx, Collections.unmodifiableList(cells));
}
// ── confidence scoring ───────────────────────────────────────────────────────────────────────
/**
* Heuristic score in [0.0, 0.85] (ceiling keeps results below Tabula lattice which starts at
* 1.0). Base 0.70; +0.05/col beyond 2 (max +0.10); +0.05 at ≥5 anchors, +0.05 at ≥8; 0.15 if
* &gt;30 % of anchors have inconsistent columns; 0.10 if non-anchors outnumber anchors.
*/
private float computeConfidence(
List<TokenizedLine> group, List<Float> columnGrid, List<String> warnings) {
float score = 0.70f;
long anchorCount =
group.stream().filter(r -> r.isAnchor() && matchesGrid(r, columnGrid)).count();
long totalRows = group.size();
// More columns
int extraCols = Math.min(columnGrid.size() - MIN_COLUMNS, 2);
score += extraCols * 0.05f;
// More anchor rows
if (anchorCount >= 5) score += 0.05f;
if (anchorCount >= 8) score += 0.05f;
// Inconsistent column matching
long inconsistent =
group.stream()
.filter(TokenizedLine::isAnchor)
.filter(tl -> hasInconsistentColumnMatch(tl, columnGrid))
.count();
if (inconsistent > anchorCount * 0.30) {
score -= 0.15f;
warnings.add(
"Column match inconsistent on "
+ inconsistent
+ "/"
+ anchorCount
+ " anchor rows");
}
// Label-heavy
long nonAnchor = totalRows - anchorCount;
if (nonAnchor > anchorCount) {
score -= 0.10f;
warnings.add(
"Non-anchor rows ("
+ nonAnchor
+ ") outnumber anchor rows ("
+ anchorCount
+ ")");
}
return Math.max(0f, Math.min(0.85f, score));
}
// ── utility ──────────────────────────────────────────────────────────────────────────────────
/**
* Returns the grid index nearest to {@code rightEdge}, or -1 if none is within {@value
* #COLUMN_MATCH_BUCKETS} buckets.
*/
private int nearestColumnIndex(float rightEdge, List<Float> grid) {
int nearest = -1;
float minDist = COLUMN_MATCH_BUCKETS * COLUMN_BUCKET_PT + 1f;
for (int i = 0; i < grid.size(); i++) {
float dist = Math.abs(rightEdge - grid.get(i));
if (dist < minDist) {
minDist = dist;
nearest = i;
}
}
return nearest;
}
private Bounds computeGroupBounds(List<TokenizedLine> group) {
return group.stream()
.map(tl -> tl.line().bounds())
.reduce(Bounds::merge)
.orElse(new Bounds(0, 0, 0, 0));
}
/** Modal gap between consecutive line edges, used to calibrate the group-split threshold. */
private float computeModalSpacing(List<RawLine> lines) {
if (lines.size() < 2) return 0f;
Map<Float, Long> freq = new HashMap<>();
for (int i = 1; i < lines.size(); i++) {
float gap = lines.get(i).bounds().y() - lines.get(i - 1).bounds().bottom();
if (gap > 0) freq.merge(Math.round(gap / 2f) * 2f, 1L, Long::sum);
}
return freq.entrySet().stream()
.max(Map.Entry.comparingByValue())
.map(Map.Entry::getKey)
.orElse(0f);
}
private static int bucket(float x) {
return Math.round(x / COLUMN_BUCKET_PT);
}
// ── private data types ───────────────────────────────────────────────────────────────────────
/** A word-level token with an approximate right-edge x-position. */
record LineToken(String text, float x, float right, boolean numeric) {}
/** A {@link RawLine} with tokens pre-computed; an "anchor" has ≥ 2 numeric tokens. */
record TokenizedLine(RawLine line, List<LineToken> all, List<LineToken> numeric) {
boolean isAnchor() {
return numeric.size() >= 2;
}
}
}
@@ -1,139 +0,0 @@
package stirling.software.SPDF.pdf.parser;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import org.springframework.stereotype.Service;
import lombok.extern.slf4j.Slf4j;
/**
* Groups {@link TextFragment} objects into visual {@link RawLine}s using baseline proximity.
*
* <p>Fragments are on the same line when their baselines are within a font-size-derived tolerance.
* A new line starts whenever the horizontal gap exceeds an adaptive column-gap threshold ({@code
* max(effectiveWidth * COLUMN_GAP_RATIO, COLUMN_GAP_MIN_PT)}), splitting two-column text.
*/
@Service
@Slf4j
public class LineBuilder {
/** Baseline tolerance as a fraction of font size; 0.5 keeps mixed-size text on one line. */
private static final float BASELINE_TOLERANCE_FACTOR = 0.5f;
/** Absolute minimum tolerance so tiny font sizes don't collapse multi-line content. */
private static final float MIN_BASELINE_TOLERANCE = 2f;
/**
* Column-gap threshold as a fraction of page width; 0.10 clears tab stops but stays below
* two-column gutters.
*/
static final float COLUMN_GAP_RATIO = 0.10f;
/** Floor for the column-gap threshold so narrow pages don't over-split lines. */
static final float COLUMN_GAP_MIN_PT = 40f;
public List<RawLine> build(List<TextFragment> fragments, int pageNumber) {
if (fragments.isEmpty()) return List.of();
float effectiveWidth = inferEffectiveWidth(fragments);
float columnGapThreshold = Math.max(effectiveWidth * COLUMN_GAP_RATIO, COLUMN_GAP_MIN_PT);
log.debug(
"LineBuilder page {}: effectiveWidth={:.1f}pt, columnGapThreshold={:.1f}pt",
pageNumber,
effectiveWidth,
columnGapThreshold);
// Sort top-to-bottom first, then left-to-right within the same baseline band.
List<TextFragment> sorted =
fragments.stream()
.sorted(
Comparator.comparingDouble(TextFragment::baseline)
.thenComparingDouble(f -> f.bounds().x()))
.toList();
List<List<TextFragment>> groups = groupByBaseline(sorted, columnGapThreshold);
List<RawLine> lines = new ArrayList<>(groups.size());
for (int i = 0; i < groups.size(); i++) {
List<TextFragment> group =
groups.get(i).stream()
.sorted(Comparator.comparingDouble(f -> f.bounds().x()))
.toList();
Bounds lineBounds =
group.stream()
.map(TextFragment::bounds)
.reduce(Bounds::merge)
.orElse(new Bounds(0, 0, 0, 0));
lines.add(new RawLine("ln-p" + pageNumber + "-" + i, group, lineBounds, pageNumber));
}
return lines;
}
private List<List<TextFragment>> groupByBaseline(
List<TextFragment> sorted, float columnGapThreshold) {
List<List<TextFragment>> groups = new ArrayList<>();
List<TextFragment> current = new ArrayList<>();
float currentBaseline = Float.NaN;
for (TextFragment fragment : sorted) {
if (current.isEmpty()) {
current.add(fragment);
currentBaseline = fragment.baseline();
continue;
}
float maxFontSize =
Math.max(
fragment.fontSize(),
(float)
current.stream()
.mapToDouble(TextFragment::fontSize)
.max()
.orElse(0));
float tolerance =
Math.max(maxFontSize * BASELINE_TOLERANCE_FACTOR, MIN_BASELINE_TOLERANCE);
boolean sameBaseline = Math.abs(fragment.baseline() - currentBaseline) <= tolerance;
boolean columnGap = sameBaseline && hasColumnGap(fragment, current, columnGapThreshold);
if (sameBaseline && !columnGap) {
current.add(fragment);
// Anchor to the weighted mean baseline so long lines stay stable.
currentBaseline =
(currentBaseline * (current.size() - 1) + fragment.baseline())
/ current.size();
} else {
groups.add(current);
current = new ArrayList<>();
current.add(fragment);
currentBaseline = fragment.baseline();
}
}
if (!current.isEmpty()) groups.add(current);
return groups;
}
/**
* True when the gap from the rightmost fragment in {@code group} to {@code next} exceeds {@code
* threshold}.
*/
private static boolean hasColumnGap(
TextFragment next, List<TextFragment> group, float threshold) {
float lastRight = group.get(group.size() - 1).bounds().right();
return next.bounds().x() - lastRight > threshold;
}
/** Infers effective page width from the rightmost fragment right-edge plus a 10 % margin. */
private static float inferEffectiveWidth(List<TextFragment> fragments) {
double maxRight =
fragments.stream().mapToDouble(f -> f.bounds().right()).max().orElse(500.0);
return (float) maxRight * 1.10f;
}
}
@@ -0,0 +1,142 @@
package stirling.software.SPDF.pdf.parser;
import java.util.ArrayList;
import java.util.List;
/**
* Detects whether a page is one- or two-column from per-line bounding boxes, and classifies an
* X-span into the column it belongs to. Detection is a midpoint vote at {@code pageWidth / 2}.
*
* <p>Capped at two columns by design — sufficient for the redaction target set (single-column
* documents and IEEE-style two-column papers). 3+ column layouts (newspapers, magazines) and
* off-centre gutters (asymmetric two-column) would need a histogram or clustering approach to
* detect the actual gutter X. (future work)
*
* <p>Coordinates are PDFTextStripper screen space (top-left origin, Y increases downward).
*/
public final class PageColumnLayout {
/**
* Slack when checking "crosses a gutter" so single-pixel overshoots don't mark a line as
* spanning.
*/
public static final float SPAN_SLACK_PT = 2f;
/**
* Slack on each side of the page midpoint inside which a line is considered "spanning"
* (covering both columns) rather than belonging to one side.
*/
private static final float MIDPOINT_SLACK_PT = 30f;
/**
* Minimum line width (points) for a line to count toward the two-column tally. Avoids false
* positives where right-aligned dates, page numbers, or short "Link" fragments next to a
* heading look like a second column when they're really just inline metadata.
*/
private static final float MIN_COLUMN_LINE_WIDTH_PT = 100f;
/**
* Minimum number of clearly leftish AND clearly rightish lines (each of width &ge; {@link
* #MIN_COLUMN_LINE_WIDTH_PT}) required to call the page two-column. Anything below this falls
* back to single-column.
*/
private static final int MIN_SIDE_LINES = 3;
private final List<float[]> columns;
private final List<float[]> gutters;
private PageColumnLayout(List<float[]> columns, List<float[]> gutters) {
this.columns = columns;
this.gutters = gutters;
}
/**
* Determines column layout from per-line bounding boxes ({@code [x1, _, x2, _]}). Counts lines
* whose X-midpoint sits clearly left of, or clearly right of, the page midpoint (with {@link
* #MIDPOINT_SLACK_PT} slack each side). If both sides have at least {@link #MIN_SIDE_LINES}
* lines, the page is treated as two-column with the gutter at the page midpoint. Otherwise it's
* single-column.
*
* <p>Cross-column lines must already be split: callers should feed boxes from a line extractor
* that splits same-Y glyphs at large X gaps (see {@code AllTextLineExtractor}). Without that
* split, IEEE-style aligned-baseline 2-column PDFs produce one wide merged box per row and the
* side tallies all end up classified as "spanning", falling to single-column.
*/
public static PageColumnLayout fromLineBoxes(List<float[]> lineBoxes, float pageWidth) {
if (lineBoxes == null || lineBoxes.isEmpty()) {
return new PageColumnLayout(List.of(new float[] {0f, pageWidth}), List.of());
}
float pageMid = pageWidth / 2f;
int left = 0, right = 0;
for (float[] lb : lineBoxes) {
if (lb == null || lb.length < 3) continue;
float width = lb[2] - lb[0];
// Skip narrow lines — dates, page numbers, "Link" labels next to a heading should
// not, on their own, make a single-column doc look two-column.
if (width < MIN_COLUMN_LINE_WIDTH_PT) continue;
float mid = (lb[0] + lb[2]) * 0.5f;
if (mid < pageMid - MIDPOINT_SLACK_PT) left++;
else if (mid > pageMid + MIDPOINT_SLACK_PT) right++;
}
if (left < MIN_SIDE_LINES || right < MIN_SIDE_LINES) {
return new PageColumnLayout(List.of(new float[] {0f, pageWidth}), List.of());
}
float gutterL = pageMid - MIDPOINT_SLACK_PT;
float gutterR = pageMid + MIDPOINT_SLACK_PT;
return new PageColumnLayout(
List.of(new float[] {0f, gutterL}, new float[] {gutterR, pageWidth}),
List.of(new float[] {gutterL, gutterR}));
}
/** All columns, left-to-right, as {@code [leftX, rightX]} pairs. Never empty. */
public List<float[]> columns() {
return columns;
}
/** Gutters between columns, left-to-right, as {@code [leftX, rightX]} pairs. */
public List<float[]> gutters() {
return gutters;
}
public int columnCount() {
return columns.size();
}
/**
* Returns the column index containing the X-midpoint of {@code [x1, x2]}, falling back to the
* closest column if the midpoint sits inside a gutter.
*/
public int columnOf(float x1, float x2) {
float mid = (x1 + x2) * 0.5f;
int best = 0;
float bestDist = Float.MAX_VALUE;
for (int i = 0; i < columns.size(); i++) {
float[] c = columns.get(i);
if (mid >= c[0] && mid <= c[1]) return i;
float dist = mid < c[0] ? c[0] - mid : mid - c[1];
if (dist < bestDist) {
bestDist = dist;
best = i;
}
}
return best;
}
/**
* Returns every column index whose X-range overlaps {@code [x1, x2]} with at least {@link
* #SPAN_SLACK_PT} of intrusion. A normal in-column line returns one index; a line crossing a
* gutter returns two or more.
*/
public int[] columnsCrossing(float x1, float x2) {
List<Integer> hits = new ArrayList<>();
for (int i = 0; i < columns.size(); i++) {
float[] c = columns.get(i);
float overlap = Math.min(x2, c[1]) - Math.max(x1, c[0]);
if (overlap > SPAN_SLACK_PT) hits.add(i);
}
if (hits.isEmpty()) hits.add(columnOf(x1, x2));
int[] out = new int[hits.size()];
for (int i = 0; i < hits.size(); i++) out[i] = hits.get(i);
return out;
}
}
@@ -0,0 +1,133 @@
package stirling.software.SPDF.pdf.parser;
import java.awt.geom.Point2D;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.pdfbox.contentstream.PDFGraphicsStreamEngine;
import org.apache.pdfbox.cos.COSName;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.graphics.image.PDImage;
import org.apache.pdfbox.util.Matrix;
/**
* PDFGraphicsStreamEngine that intercepts {@code drawImage} calls and records each image's bounding
* box in PDF user-space (origin bottom-left, Y up) by transforming the unit square through the
* current transformation matrix (CTM).
*
* <p>Usage:
*
* <pre>{@code
* PageImageLocator locator = new PageImageLocator(page, pageIndex);
* locator.processPage(page);
* List<ImageBox> boxes = locator.getImageBoxes();
* }</pre>
*
* <p>Each {@link ImageBox} carries the 0-based page index and the axis-aligned bounding box {@code
* (x1, y1, x2, y2)} in PDF user-space coordinates.
*/
public final class PageImageLocator extends PDFGraphicsStreamEngine {
/**
* Bounding box of a raster or vector image found on a PDF page.
*
* @param pageIndex 0-based page index
* @param x1 left edge in PDF user-space (origin bottom-left)
* @param y1 bottom edge in PDF user-space
* @param x2 right edge
* @param y2 top edge
*/
public record ImageBox(int pageIndex, float x1, float y1, float x2, float y2) {}
private final int pageIndex;
private final List<ImageBox> imageBoxes = new ArrayList<>();
private final Point2D.Float currentPoint = new Point2D.Float();
/**
* @param page the PDPage to process
* @param pageIndex 0-based index of this page in the document (stored on each returned {@link
* ImageBox})
*/
public PageImageLocator(PDPage page, int pageIndex) {
super(page);
this.pageIndex = pageIndex;
}
/** Returns all image bounding boxes collected during {@link #processPage}. */
public List<ImageBox> getImageBoxes() {
return imageBoxes;
}
@Override
public void drawImage(PDImage pdImage) throws IOException {
Matrix ctm = getGraphicsState().getCurrentTransformationMatrix();
// An image occupies the unit square (0,0)→(1,1) in image space.
// Transform all four corners through the CTM to get the page-space bounding box.
float a = ctm.getScaleX();
float b = ctm.getShearY();
float c = ctm.getShearX();
float d = ctm.getScaleY();
float e = ctm.getTranslateX();
float f = ctm.getTranslateY();
float[] xs = {e, a + e, c + e, a + c + e};
float[] ys = {f, b + f, d + f, b + d + f};
float x1 = Float.MAX_VALUE, y1 = Float.MAX_VALUE;
float x2 = -Float.MAX_VALUE, y2 = -Float.MAX_VALUE;
for (float x : xs) {
x1 = Math.min(x1, x);
x2 = Math.max(x2, x);
}
for (float y : ys) {
y1 = Math.min(y1, y);
y2 = Math.max(y2, y);
}
imageBoxes.add(new ImageBox(pageIndex, x1, y1, x2, y2));
}
// ---------- required abstract methods (no-op for path operations) ----------
@Override
public void appendRectangle(Point2D p0, Point2D p1, Point2D p2, Point2D p3) {}
@Override
public void clip(int windingRule) {}
@Override
public void moveTo(float x, float y) {
currentPoint.setLocation(x, y);
}
@Override
public void lineTo(float x, float y) {
currentPoint.setLocation(x, y);
}
@Override
public void curveTo(float x1, float y1, float x2, float y2, float x3, float y3) {
currentPoint.setLocation(x3, y3);
}
@Override
public Point2D getCurrentPoint() {
return currentPoint;
}
@Override
public void closePath() {}
@Override
public void endPath() {}
@Override
public void strokePath() {}
@Override
public void fillPath(int windingRule) {}
@Override
public void fillAndStrokePath(int windingRule) {}
@Override
public void shadingFill(COSName shadingName) {}
}
@@ -1,79 +0,0 @@
package stirling.software.SPDF.pdf.parser;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
/**
* Runs the per-page ingestion pipeline: {@link WordExtractingStripper} → {@link LineBuilder} →
* {@link TableParser}, producing a {@link PdfModels.ParsedPage} per page. The caller owns the
* {@link PDDocument} lifecycle.
*/
@Service
@RequiredArgsConstructor
@Slf4j
public class PdfIngester {
private final LineBuilder lineBuilder;
private final TableParser tableParser;
public List<ParsedPage> parse(PDDocument document) throws IOException {
return parse(document, document.getNumberOfPages());
}
public List<ParsedPage> parse(PDDocument document, int maxPages) throws IOException {
int pageCount = Math.min(document.getNumberOfPages(), maxPages);
List<ParsedPage> pages = new ArrayList<>(pageCount);
long fragmentsMs = 0;
long tablesMs = 0;
long t0 = System.currentTimeMillis();
for (int p = 1; p <= pageCount; p++) {
long ft = System.currentTimeMillis();
List<TextFragment> fragments = extractFragments(document, p);
fragmentsMs += System.currentTimeMillis() - ft;
PDPage page = document.getPage(p - 1);
PDRectangle mediaBox = page.getMediaBox();
List<RawLine> lines = lineBuilder.build(fragments, p);
RawPage rawPage = new RawPage(p, mediaBox.getWidth(), mediaBox.getHeight(), lines);
long tt = System.currentTimeMillis();
List<TableFragment> tables = tableParser.parse(document, rawPage);
tablesMs += System.currentTimeMillis() - tt;
log.debug(
"Page {}: {} fragments → {} lines, {} table(s)",
p,
fragments.size(),
lines.size(),
tables.size());
pages.add(new ParsedPage(p, mediaBox.getWidth(), mediaBox.getHeight(), tables, lines));
}
log.info(
"[timing] parse pages={} total={}ms fragments={}ms tables={}ms",
pageCount,
System.currentTimeMillis() - t0,
fragmentsMs,
tablesMs);
return pages;
}
private List<TextFragment> extractFragments(PDDocument document, int pageNumber)
throws IOException {
WordExtractingStripper stripper = new WordExtractingStripper(pageNumber);
stripper.getText(document);
return stripper.getFragments();
}
}
@@ -1,113 +0,0 @@
package stirling.software.SPDF.pdf.parser;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.font.PDFont;
import org.apache.pdfbox.text.PDFTextStripper;
import org.apache.pdfbox.text.TextPosition;
/**
* Extends {@link PDFTextStripper} to capture per-fragment geometry and font metadata.
*
* <p>Overrides {@link #writeString} to split each content-stream string into word-level {@link
* TextFragment}s with bounding boxes, baseline, font name, and bold flag. Coordinates are in
* PDFTextStripper space: (0,0) top-left, Y increases downward, {@code getY()} is the baseline.
*/
class WordExtractingStripper extends PDFTextStripper {
private final int targetPage;
private final List<TextFragment> fragments = new ArrayList<>();
private int fragmentIndex = 0;
WordExtractingStripper(int pageNumber) throws IOException {
this.targetPage = pageNumber;
setStartPage(pageNumber);
setEndPage(pageNumber);
setSortByPosition(true);
}
@Override
protected void startPage(PDPage page) throws IOException {
super.startPage(page);
fragments.clear();
fragmentIndex = 0;
}
@Override
protected void writeString(String text, List<TextPosition> textPositions) throws IOException {
if (text == null || text.isBlank()) return;
// Fast path: no whitespace → emit one fragment (most financial PDFs have each
// number as its own string operation, so this is the common case).
if (text.indexOf(' ') < 0) {
emitFragment(text, textPositions);
return;
}
// Per-word splitting requires 1:1 text-char to TextPosition correspondence.
// Fall back to one fragment when sizes differ (ligatures, encoding edge cases).
if (textPositions.size() != text.length()) {
emitFragment(text, textPositions);
return;
}
// Emit one TextFragment per whitespace-delimited word with accurate per-word bounds.
int start = 0;
for (int i = 0; i <= text.length(); i++) {
if (i == text.length() || text.charAt(i) == ' ') {
if (start < i) {
emitFragment(text.substring(start, i), textPositions.subList(start, i));
}
start = i + 1;
}
}
}
private void emitFragment(String text, List<TextPosition> positions) {
if (positions.isEmpty()) return;
float minX = Float.MAX_VALUE;
float minY = Float.MAX_VALUE;
float maxRight = -Float.MAX_VALUE;
float maxBaseline = -Float.MAX_VALUE;
TextPosition first = null;
for (TextPosition tp : positions) {
if (tp == null) continue;
if (first == null) first = tp;
float x = tp.getX();
// getY() is the baseline; top of character = getY() - getHeight().
float top = tp.getY() - tp.getHeight();
float right = x + tp.getWidth();
float baseline = tp.getY();
minX = Math.min(minX, x);
minY = Math.min(minY, top);
maxRight = Math.max(maxRight, right);
maxBaseline = Math.max(maxBaseline, baseline);
}
if (first == null) return;
PDFont font = first.getFont();
String fontName = font != null ? font.getName() : "";
boolean bold = fontName != null && fontName.toLowerCase().contains("bold");
// getHeight() gives the rendered glyph height, which is the most reliable visual size.
float fontSize = first.getHeight();
Bounds bounds = new Bounds(minX, minY, maxRight - minX, maxBaseline - minY);
String id = "tf-p" + targetPage + "-" + fragmentIndex++;
fragments.add(new TextFragment(id, text, bounds, maxBaseline, fontSize, fontName, bold));
}
List<TextFragment> getFragments() {
return Collections.unmodifiableList(fragments);
}
}
@@ -77,6 +77,10 @@ public @interface AutoJobPostMapping {
/**
* Relative resource weight (1-100). See {@link
* stirling.software.common.enumeration.ResourceWeight} for the standard tiers.
*
* <p>The default is a sentinel ({@link Integer#MIN_VALUE}); {@code
* AutoJobPostMappingWeightTest} fails the build if any endpoint leaves it unset. Runtime
* readers clamp the value into {@code [1, 100]}.
*/
int resourceWeight() default 1;
int resourceWeight() default Integer.MIN_VALUE;
}
@@ -11,23 +11,39 @@ public interface FileStore {
/** Stored file record. */
record Stored(String fileId, long size) {}
/** Store the given stream and return a generated file id and total bytes written. */
Stored store(InputStream in, String originalName) throws IOException;
/**
* Store the given stream and return a generated file id and total bytes written. {@code owner}
* may be null to indicate the file has no associated user (anonymous / desktop / async job with
* no propagated security context); a non-null value is persisted alongside the data so {@link
* #getOwner(String)} can return it later for authorization checks.
*/
Stored store(InputStream in, String originalName, String owner) throws IOException;
/** Store with no owner. Equivalent to {@link #store(InputStream, String, String)} with null. */
default Stored store(InputStream in, String originalName) throws IOException {
return store(in, originalName, null);
}
/**
* Store the file at {@code source} and return a generated file id and total bytes written.
*
* <p>Default implementation opens {@code source} as a stream and delegates to {@link
* #store(InputStream, String)}. Local-disk implementations should override to use a direct
* file-to-file copy ({@code Files.copy(source, dest)} can use {@code sendfile(2)} on Linux),
* which avoids the two-memory-copy hit of streaming a disk-backed upload through the JVM heap.
* #store(InputStream, String, String)}. Local-disk implementations should override to use a
* direct file-to-file copy ({@code Files.copy(source, dest)} can use {@code sendfile(2)} on
* Linux), which avoids the two-memory-copy hit of streaming a disk-backed upload through the
* JVM heap.
*/
default Stored store(Path source, String originalName) throws IOException {
default Stored store(Path source, String originalName, String owner) throws IOException {
try (InputStream in = Files.newInputStream(source)) {
return store(in, originalName);
return store(in, originalName, owner);
}
}
/** Store with no owner. Equivalent to {@link #store(Path, String, String)} with null. */
default Stored store(Path source, String originalName) throws IOException {
return store(source, originalName, null);
}
/** Open the stored file for streaming reads. Caller closes. */
InputStream retrieve(String fileId) throws IOException;
@@ -42,4 +58,12 @@ public interface FileStore {
/** Whether the file id exists in the store. */
boolean exists(String fileId);
/**
* Returns the owner identifier recorded at store time, or {@code null} if the file does not
* exist or was stored without an owner. Implementations must not throw when the file is missing
* or when the owner record is absent; they should return null so callers can treat "no owner"
* as a non-authoritative case.
*/
String getOwner(String fileId) throws IOException;
}
@@ -2,7 +2,13 @@ package stirling.software.common.cluster;
import java.time.Duration;
/** Token-bucket rate limiting backed by the cluster backplane. */
/**
* Token-bucket rate limiting backed by the cluster backplane.
*
* <p>In-process implementations enforce a per-JVM limit; distributed implementations enforce a
* single global limit across every node. Both use a Bucket4j greedy-refill token bucket so the
* semantics match across single-node and cluster deployments.
*/
public interface RateLimitStore {
/**
@@ -3,9 +3,12 @@ package stirling.software.common.cluster.inprocess;
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.UUID;
import java.util.concurrent.locks.ReentrantLock;
import java.util.regex.Pattern;
import lombok.extern.slf4j.Slf4j;
@@ -15,33 +18,47 @@ import stirling.software.common.cluster.FileStore;
@Slf4j
public class LocalDiskFileStore implements FileStore {
private static final String OWNER_SUFFIX = ".owner";
// File ids are generated as random UUIDs; reject anything else so a tainted id can never reach
// Files.* APIs (defence in depth on top of the resolve() prefix check, and silences CodeQL's
// path-injection finding on the resolveOwner sidecar lookup).
private static final Pattern UUID_PATTERN =
Pattern.compile(
"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$");
private final String baseDirPath;
// Fixed-size lock stripes so concurrent store/delete on the same (or colliding) fileId
// serialise the data-file + owner-sidecar pair as one critical section. Striped (not
// per-id) so the map never has to be cleaned up; collisions across unrelated ids are
// harmless contention.
private static final int LOCK_STRIPES = 64;
private final ReentrantLock[] stripes = new ReentrantLock[LOCK_STRIPES];
public LocalDiskFileStore(String baseDirPath) {
this.baseDirPath = baseDirPath;
for (int i = 0; i < LOCK_STRIPES; i++) {
stripes[i] = new ReentrantLock();
}
}
@Override
public Stored store(InputStream in, String originalName) throws IOException {
public Stored store(InputStream in, String originalName, String owner) throws IOException {
String fileId = UUID.randomUUID().toString();
Path filePath = resolve(fileId);
Files.createDirectories(filePath.getParent());
ReentrantLock lock = acquire(fileId);
boolean success = false;
try {
long size = Files.copy(in, filePath);
writeOwner(fileId, owner);
success = true;
return new Stored(fileId, size);
} finally {
if (!success) {
try {
Files.deleteIfExists(filePath);
} catch (IOException cleanupEx) {
log.warn(
"Failed to clean up partial file {} after store failure",
filePath,
cleanupEx);
}
cleanupAfterFailedStore(fileId, filePath);
}
release(fileId, lock);
}
}
@@ -52,27 +69,44 @@ public class LocalDiskFileStore implements FileStore {
* the source size before copying so the post-copy stat is unnecessary.
*/
@Override
public Stored store(Path source, String originalName) throws IOException {
public Stored store(Path source, String originalName, String owner) throws IOException {
String fileId = UUID.randomUUID().toString();
Path filePath = resolve(fileId);
Files.createDirectories(filePath.getParent());
long size = Files.size(source);
ReentrantLock lock = acquire(fileId);
boolean success = false;
try {
Files.copy(source, filePath);
writeOwner(fileId, owner);
success = true;
return new Stored(fileId, size);
} finally {
if (!success) {
try {
Files.deleteIfExists(filePath);
} catch (IOException cleanupEx) {
log.warn(
"Failed to clean up partial file {} after store failure",
filePath,
cleanupEx);
}
cleanupAfterFailedStore(fileId, filePath);
}
release(fileId, lock);
}
}
private void writeOwner(String fileId, String owner) throws IOException {
if (owner == null || owner.isBlank()) {
return;
}
Path ownerPath = resolveOwner(fileId);
Files.write(ownerPath, owner.getBytes(StandardCharsets.UTF_8));
}
private void cleanupAfterFailedStore(String fileId, Path filePath) {
try {
Files.deleteIfExists(filePath);
} catch (IOException cleanupEx) {
log.warn("Failed to clean up partial file {} after store failure", filePath, cleanupEx);
}
try {
Files.deleteIfExists(resolveOwner(fileId));
} catch (IOException cleanupEx) {
log.warn("Failed to clean up owner sidecar for {} after store failure", fileId);
}
}
@@ -101,11 +135,26 @@ public class LocalDiskFileStore implements FileStore {
@Override
public boolean delete(String fileId) {
ReentrantLock lock = acquire(fileId);
try {
return Files.deleteIfExists(resolve(fileId));
} catch (IOException e) {
log.error("Error deleting file with ID: {}", fileId, e);
return false;
// Data first, owner second: a concurrent retrieve that observes the transient
// (data-gone, owner-still-present) window simply fails with IOException; the inverse
// order would briefly look like an unowned file and could grant cross-user access.
boolean removed;
try {
removed = Files.deleteIfExists(resolve(fileId));
} catch (IOException e) {
log.error("Error deleting file with ID: {}", fileId, e);
return false;
}
try {
Files.deleteIfExists(resolveOwner(fileId));
} catch (IOException e) {
log.warn("Error deleting owner sidecar for file ID: {}", fileId, e);
}
return removed;
} finally {
release(fileId, lock);
}
}
@@ -114,8 +163,21 @@ public class LocalDiskFileStore implements FileStore {
return Files.exists(resolve(fileId));
}
@Override
public String getOwner(String fileId) throws IOException {
Path ownerPath = resolveOwner(fileId);
if (!Files.exists(ownerPath)) {
return null;
}
byte[] bytes = Files.readAllBytes(ownerPath);
if (bytes.length == 0) {
return null;
}
return new String(bytes, StandardCharsets.UTF_8);
}
public Path resolve(String fileId) {
if (fileId.contains("..") || fileId.contains("/") || fileId.contains("\\")) {
if (fileId == null || !UUID_PATTERN.matcher(fileId).matches()) {
throw new IllegalArgumentException("Invalid file ID");
}
Path basePath = Path.of(baseDirPath).normalize().toAbsolutePath();
@@ -125,4 +187,19 @@ public class LocalDiskFileStore implements FileStore {
}
return resolvedPath;
}
private Path resolveOwner(String fileId) {
Path data = resolve(fileId);
return data.resolveSibling(data.getFileName().toString() + OWNER_SUFFIX);
}
private ReentrantLock acquire(String fileId) {
ReentrantLock lock = stripes[(fileId.hashCode() & Integer.MAX_VALUE) % LOCK_STRIPES];
lock.lock();
return lock;
}
private void release(String fileId, ReentrantLock lock) {
lock.unlock();
}
}
@@ -3,7 +3,6 @@ package stirling.software.common.configuration;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.List;
import java.util.Locale;
import java.util.Properties;
@@ -122,17 +121,17 @@ public class AppConfig {
@Bean(name = "RunningInDocker")
public boolean runningInDocker() {
return Files.exists(Paths.get("/.dockerenv"));
return Files.exists(Path.of("/.dockerenv"));
}
@Bean(name = "configDirMounted")
public boolean isRunningInDockerWithConfig() {
Path dockerEnv = Paths.get("/.dockerenv");
Path dockerEnv = Path.of("/.dockerenv");
// default to true if not docker
if (!Files.exists(dockerEnv)) {
return true;
}
Path mountInfo = Paths.get("/proc/1/mountinfo");
Path mountInfo = Path.of("/proc/1/mountinfo");
// this should always exist, if not some unknown usecase
if (!Files.exists(mountInfo)) {
return true;
@@ -7,7 +7,6 @@ import java.net.URISyntaxException;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.util.List;
@@ -27,7 +26,7 @@ public class ConfigInitializer {
public void ensureConfigExists() throws IOException, URISyntaxException {
// 1) If settings file doesn't exist, create from template
Path destPath = Paths.get(InstallationPathConfig.getSettingsPath());
Path destPath = Path.of(InstallationPathConfig.getSettingsPath());
boolean settingsFileExists = Files.exists(destPath);
@@ -39,7 +38,7 @@ public class ConfigInitializer {
if (settingsFileExists) {
// move settings.yml to settings.yml.{timestamp}.bak
Path backupPath =
Paths.get(
Path.of(
InstallationPathConfig.getSettingsPath()
+ "."
+ System.currentTimeMillis()
@@ -80,6 +79,7 @@ public class ConfigInitializer {
YamlHelper settingsFile = new YamlHelper(settingTempPath);
migrateEnterpriseEditionToPremium(settingsFile, settingsTemplateFile);
migrateProFeaturesKeyCasing(settingsFile, settingsTemplateFile);
boolean changesMade =
settingsTemplateFile.updateValuesFromYaml(settingsFile, settingsTemplateFile);
@@ -95,7 +95,7 @@ public class ConfigInitializer {
}
// 3) Ensure custom settings file exists
Path customSettingsPath = Paths.get(InstallationPathConfig.getCustomSettingsPath());
Path customSettingsPath = Path.of(InstallationPathConfig.getCustomSettingsPath());
if (Files.notExists(customSettingsPath)) {
Files.createFile(customSettingsPath);
log.info("Created custom_settings file: {}", customSettingsPath);
@@ -116,31 +116,52 @@ public class ConfigInitializer {
}
if (yaml.getValueByExactKeyPath("enterpriseEdition", "SSOAutoLogin") != null) {
template.updateValue(
List.of("premium", "proFeatures", "SSOAutoLogin"),
List.of("premium", "proFeatures", "ssoAutoLogin"),
yaml.getValueByExactKeyPath("enterpriseEdition", "SSOAutoLogin"));
}
if (yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "autoUpdateMetadata")
!= null) {
template.updateValue(
List.of("premium", "proFeatures", "CustomMetadata", "autoUpdateMetadata"),
List.of("premium", "proFeatures", "customMetadata", "autoUpdateMetadata"),
yaml.getValueByExactKeyPath(
"enterpriseEdition", "CustomMetadata", "autoUpdateMetadata"));
}
if (yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "author") != null) {
template.updateValue(
List.of("premium", "proFeatures", "CustomMetadata", "author"),
List.of("premium", "proFeatures", "customMetadata", "author"),
yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "author"));
}
if (yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "creator") != null) {
template.updateValue(
List.of("premium", "proFeatures", "CustomMetadata", "creator"),
List.of("premium", "proFeatures", "customMetadata", "creator"),
yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "creator"));
}
if (yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "producer")
!= null) {
template.updateValue(
List.of("premium", "proFeatures", "CustomMetadata", "producer"),
List.of("premium", "proFeatures", "customMetadata", "producer"),
yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "producer"));
}
}
// TODO: Remove post migration
// settings.yml.template renamed the two non-camelCase proFeatures keys
// ("SSOAutoLogin" -> "ssoAutoLogin", "CustomMetadata" -> "customMetadata") so the whole
// settings pipeline is consistent camelCase. The save path (YamlHelper.updateValue) matches
// keys case-sensitively, so without this carry-forward an existing install's values written
// under the old PascalCase keys would be dropped on upgrade and reset to template defaults.
void migrateProFeaturesKeyCasing(YamlHelper yaml, YamlHelper template) {
Object ssoAutoLogin = yaml.getValueByExactKeyPath("premium", "proFeatures", "SSOAutoLogin");
if (ssoAutoLogin != null) {
template.updateValue(List.of("premium", "proFeatures", "ssoAutoLogin"), ssoAutoLogin);
}
for (String field : List.of("autoUpdateMetadata", "author", "creator", "producer")) {
Object value =
yaml.getValueByExactKeyPath("premium", "proFeatures", "CustomMetadata", field);
if (value != null) {
template.updateValue(
List.of("premium", "proFeatures", "customMetadata", field), value);
}
}
}
}
@@ -3,7 +3,6 @@ package stirling.software.common.configuration;
import java.nio.file.Files;
import java.nio.file.InvalidPathException;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Collections;
import java.util.LinkedHashSet;
@@ -201,7 +200,7 @@ public class RuntimePathConfig {
try {
// Normalize to absolute path
Path path = Paths.get(pathStr.trim()).toAbsolutePath().normalize();
Path path = Path.of(pathStr.trim()).toAbsolutePath().normalize();
String normalizedPath = path.toString();
// Check for duplicates
@@ -224,9 +223,9 @@ public class RuntimePathConfig {
private void detectOverlappingPaths(List<String> paths) {
for (int i = 0; i < paths.size(); i++) {
Path path1 = Paths.get(paths.get(i));
Path path1 = Path.of(paths.get(i));
for (int j = i + 1; j < paths.size(); j++) {
Path path2 = Paths.get(paths.get(j));
Path path2 = Path.of(paths.get(j));
// Check if one path is a parent of the other
if (path1.startsWith(path2)) {
@@ -246,10 +245,10 @@ public class RuntimePathConfig {
private void validatePipelinePaths() {
try {
Path finishedPath = Paths.get(pipelineFinishedFoldersPath).toAbsolutePath().normalize();
Path finishedPath = Path.of(pipelineFinishedFoldersPath).toAbsolutePath().normalize();
for (String watchedPathStr : pipelineWatchedFoldersPaths) {
Path watchedPath = Paths.get(watchedPathStr).toAbsolutePath().normalize();
Path watchedPath = Path.of(watchedPathStr).toAbsolutePath().normalize();
// Check if watched folder is same as finished folder
if (watchedPath.equals(finishedPath)) {
@@ -77,8 +77,10 @@ public class ApplicationProperties {
private ProcessExecutor processExecutor = new ProcessExecutor();
private PdfEditor pdfEditor = new PdfEditor();
private AiEngine aiEngine = new AiEngine();
private Mcp mcp = new Mcp();
private InternalApi internalApi = new InternalApi();
private Cluster cluster = new Cluster();
private Policies policies = new Policies();
@Bean
public PropertySource<?> dynamicYamlPropertySource(ConfigurableEnvironment environment)
@@ -202,6 +204,45 @@ public class ApplicationProperties {
}
}
@Data
public static class Policies {
/**
* Absolute directories that policy folder input sources and output sinks may read from or
* write to. Empty (the default) disables folder access entirely, so a policy can never be
* pointed at an arbitrary server path. Stirling's own config directory is always
* off-limits, and folder access is always disabled in SaaS mode regardless of this list.
*/
private List<String> allowedFolderRoots = new java.util.ArrayList<>();
/** How often (seconds) the schedule trigger checks for policies whose schedule is due. */
private long scheduleSweepSeconds = 60;
/**
* How often (seconds) the folder-watch trigger reconciles its watch registrations and
* re-runs every folder-watch policy as a safety net for filesystem events that were missed
* (NFS, bind mounts, inotify-queue overflow).
*/
private long watchReconcileSeconds = 300;
/**
* How long (milliseconds) the folder-watch trigger keeps draining filesystem events after
* the first, so a burst from a single file copy coalesces into one run instead of many.
*/
private long watchQuietPeriodMs = 500;
/**
* SSE emitter timeout (milliseconds) for streamed runs; generous for long multi-step runs.
*/
private long streamTimeoutMs = 1800000;
/**
* How long (minutes) a finished run's in-memory state is retained before eviction,
* mirroring the job-result expiry so rich run state does not outlive the process. Active
* and paused runs are kept regardless of age.
*/
private int runExpiryMinutes = 30;
}
@Data
public static class PdfEditor {
private Cache cache = new Cache();
@@ -256,6 +297,103 @@ public class ApplicationProperties {
private int longRunningTimeoutSeconds = 600;
}
/**
* Model Context Protocol (MCP) server configuration. All keys live under the top-level {@code
* mcp.*} prefix. {@link #enabled} defaults to {@code false}: when off, no MCP beans are wired,
* no /mcp endpoint exists, and no protected-resource metadata is published.
*/
@Data
public static class Mcp {
/** Master switch. When {@code false} (default), no MCP beans are wired. */
private boolean enabled = false;
/**
* When {@code true} (default), invocations require an OAuth scope: {@code mcp.tools.read}
* for read-style operations and {@code mcp.tools.write} for write/destructive ones. When
* {@code false}, scope checks are skipped (use only if your IdP issues a single coarse
* scope).
*/
private boolean scopesEnabled = true;
/** How often to refresh the AI capabilities manifest from the engine. */
private int engineCapabilityRefreshMinutes = 5;
/**
* Tool allow-list (operation ids, e.g. {@code compress-pdf}). When non-empty, ONLY these
* operations are exposed over MCP; everything else is hidden, undescribable, and
* uninvocable - on top of the global endpoint enable/disable config. Empty = allow all.
*/
private List<String> allowedOperations = new ArrayList<>();
/**
* Tool deny-list (operation ids). Any operation listed here is removed from MCP even if it
* would otherwise be allowed. Applied after {@link #allowedOperations}.
*/
private List<String> blockedOperations = new ArrayList<>();
/** Max MCP request body size in bytes; inline file uploads ride in the JSON-RPC body. */
private long maxRequestBytes = 10L * 1024 * 1024;
/** Results up to this size return inline as base64; larger ones return a fileId only. */
private long maxInlineResponseBytes = 10L * 1024 * 1024;
private Auth auth = new Auth();
@Data
public static class Auth {
/**
* Authentication mode for the MCP endpoint. {@code oauth} (default) runs a full OAuth2
* resource server (JWT, RFC 8707 audience, RFC 9728 metadata). {@code apikey} accepts a
* Stirling per-user API key via the {@code X-API-KEY} header (or {@code Authorization:
* Bearer <key>}) and binds the request to that user - the low-friction self-host path,
* no external IdP required.
*/
private String mode = "oauth";
/** OAuth2 issuer URI, e.g. {@code http://localhost:9000}. Required when MCP is on. */
private String issuerUri = "";
/**
* JWKS URI. When blank, derived from the issuer's {@code
* /.well-known/openid-configuration} document.
*/
private String jwksUri = "";
/**
* RFC 8707 resource identifier of THIS MCP server, e.g. {@code
* http://localhost:8080/mcp}. Tokens that do not list this id in their {@code aud}
* claim are rejected with HTTP 401.
*/
private String resourceId = "";
/**
* Additional JWT audiences accepted at the MCP endpoint, on top of {@link #resourceId}.
* Empty (default) keeps strict RFC 8707 binding. Some IdPs cannot mint
* resource-specific audiences - e.g. Supabase's OAuth server always issues {@code
* aud=authenticated} - so operators list the audience their IdP actually emits here
* (env: {@code MCP_AUTH_ACCEPTEDAUDIENCES}, comma-separated).
*/
private List<String> acceptedAudiences = new ArrayList<>();
/**
* JWT claim whose value is matched against a provisioned Stirling username. Defaults to
* {@code sub}; set to {@code email} or {@code preferred_username} to match how your IdP
* maps users to Stirling accounts.
*/
private String usernameClaim = "sub";
/**
* When {@code true} (default), a validated token is accepted only if its {@link
* #usernameClaim} value resolves to an existing, enabled Stirling user account. Tokens
* whose subject has no Stirling account (or a disabled one) are rejected with HTTP 403.
* Set to {@code false} only if you intentionally want any IdP-valid token to use MCP
* without a local account.
*/
private boolean requireExistingAccount = true;
}
}
/**
* Cluster backplane configuration. All keys live under the top-level {@code cluster.*} prefix
* (e.g. env var {@code CLUSTER_ENABLED}). The master switch is {@link #enabled} and defaults to
@@ -528,6 +666,16 @@ public class ApplicationProperties {
private String provider;
private Client client = new Client();
/**
* When true, the OAuth2/OIDC login flow logs the full set of ID token and UserInfo
* claims at INFO level (and again at ERROR level if the username attribute cannot be
* resolved). Used to diagnose provider misconfiguration (for example ADFS not returning
* an {@code email} claim). WARNING: writes PII (sub, email, name) to application logs.
* Leave disabled in production; enable only while actively troubleshooting and disable
* again afterwards.
*/
private Boolean debugLogging = false;
public void setScopes(String scopes) {
List<String> scopesList =
Arrays.stream(scopes.split(",")).map(String::trim).toList();
@@ -778,6 +926,7 @@ public class ApplicationProperties {
private boolean enabled = false;
private String provider = "local";
private Local local = new Local();
private S3 s3 = new S3();
private Quotas quotas = new Quotas();
private Sharing sharing = new Sharing();
private Signing signing = new Signing();
@@ -787,6 +936,57 @@ public class ApplicationProperties {
private String basePath = InstallationPathConfig.getPath() + "storage";
}
@Data
public static class S3 {
/**
* Optional custom endpoint (e.g. {@code https://<account>.r2.cloudflarestorage.com},
* {@code https://<project>.supabase.co/storage/v1/s3}, or {@code http://localhost:9000}
* for MinIO). Blank = use AWS regional default.
*/
private String endpoint = "";
private String bucket = "";
private String region = "us-east-1";
private String accessKey = "";
private String secretKey = "";
/**
* When {@code true} use path-style URLs ({@code <endpoint>/<bucket>/<key>}) instead of
* virtual-hosted ({@code <bucket>.<endpoint>/<key>}). MinIO and most S3-compatible
* gateways require path-style; AWS S3 prefers virtual-hosted.
*/
private boolean pathStyleAccess = false;
/**
* When {@code false} (default), {@code endpoint} hostnames that resolve to private,
* loopback, or link-local addresses are rejected at startup to block SSRF attacks via
* the cloud metadata service (e.g. {@code http://169.254.169.254/}). Set to {@code
* true} to opt in for MinIO / in-cluster S3 endpoints on private networks.
*/
private boolean allowPrivateEndpoints = false;
/**
* Controls when the SDK adds an {@code x-amz-checksum-*} header on PUT/UploadPart.
* Default {@code WHEN_SUPPORTED} (the SDK default since 2.30) makes the SDK send a
* CRC32 checksum on every upload - this works on AWS S3, MinIO, current Supabase,
* Backblaze B2 (post-July-2025), and modern R2. Set to {@code WHEN_REQUIRED} to
* suppress the auto-checksum on vendors that reject unknown {@code x-amz-checksum-*}
* headers (older Backblaze B2, some R2 corner cases, GCS S3 endpoint). Invalid values
* fall back to {@code WHEN_SUPPORTED}.
*/
private String requestChecksumCalculation = "WHEN_SUPPORTED";
/**
* Controls when the SDK validates returned {@code x-amz-checksum-*} headers on GET
* responses. Default {@code WHEN_SUPPORTED}. Set to {@code WHEN_REQUIRED} if your
* vendor never returns these headers and you see false-positive checksum-mismatch
* errors. Invalid values fall back to {@code WHEN_SUPPORTED}.
*/
private String responseChecksumValidation = "WHEN_SUPPORTED";
}
@Data
public static class Sharing {
private boolean enabled = false;
@@ -805,6 +1005,10 @@ public class ApplicationProperties {
@Data
public static class Signing {
private boolean enabled = false;
// Signing user-picker scope: 'org' (default) = whole instance, anything else =
// caller's team only (fail-closed). The saas profile pins 'team'.
private String userListScope = "org";
}
}
@@ -1,7 +1,6 @@
package stirling.software.common.model;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.Locale;
@@ -24,7 +23,7 @@ public class FileInfo {
// Converts the file path string to a Path object.
public Path getFilePathAsPath() {
return Paths.get(filePath);
return Path.of(filePath);
}
// Formats the file size into a human-readable string.
@@ -0,0 +1,191 @@
package stirling.software.common.pdf;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import stirling.software.jpdfium.text.PageText;
import stirling.software.jpdfium.text.TextChar;
import stirling.software.jpdfium.text.TextLine;
import stirling.software.jpdfium.text.TextWord;
final class HeadingDetector {
private HeadingDetector() {}
/** A heading is at most this many words; longer lines are treated as body text. */
private static final int MAX_HEADING_WORDS = 12;
/**
* Returns the Markdown heading prefix for a line. The decision combines several signals, never
* text matching, so a plain line that merely shares text with a heading is never promoted:
*
* <ul>
* <li><b>Size</b> — dominant glyph font size vs. the document body median (primary signal).
* Some PDFs encode visual size in the text matrix, so every glyph reports ~1.0; for those
* the line height is used as the proxy instead.
* <li><b>Brevity</b> — headings are short labels; a line over {@value #MAX_HEADING_WORDS}
* words is body text regardless of size.
* <li><b>Not a sentence</b> — a line ending in {@code . ! ?} reads as prose, not a heading.
* </ul>
*
* <p>Boldness is deliberately <em>not</em> a heading signal — a bold-but-not-larger line is
* emphasis, not a heading (see {@link #isBoldLabel}); promoting it to {@code #}/{@code ##} is
* the main source of false-positive headings.
*
* <ul>
* <li>size &gt; baseline * 1.4 → {@code "# "}
* <li>size &gt; baseline * 1.2 → {@code "## "}
* <li>otherwise → {@code ""}
* </ul>
*/
static String headingPrefix(TextLine line, float medianBodySize, float medianBodyHeight) {
String text = line.text().strip();
if (text.isEmpty() || wordCount(text) > MAX_HEADING_WORDS || endsLikeSentence(text)) {
return "";
}
float dominant = dominantFontSize(line);
float value;
float baseline;
if (dominant > 2f && medianBodySize > 2f) {
value = dominant;
baseline = medianBodySize;
} else {
value = line.height();
baseline = medianBodyHeight;
}
if (baseline <= 0f) {
return "";
}
float ratio = value / baseline;
if (ratio > 1.4f) {
return "# ";
}
if (ratio > 1.2f) {
return "## ";
}
return "";
}
/**
* True when a line should be emphasised as bold (rendered {@code **like this**}) rather than
* promoted to a heading: it is bold, short, and not a full sentence. Used for bold labels that
* are not large enough to be headings.
*/
static boolean isBoldLabel(TextLine line) {
String text = line.text().strip();
if (text.isEmpty() || wordCount(text) > MAX_HEADING_WORDS || endsLikeSentence(text)) {
return false;
}
return isBold(line);
}
private static int wordCount(String text) {
return text.split("\\s+").length;
}
private static boolean endsLikeSentence(String text) {
char last = text.charAt(text.length() - 1);
return last == '.' || last == '!' || last == '?';
}
/** True when the line's dominant font is bold, inferred from PostScript font names. */
private static boolean isBold(TextLine line) {
Map<String, Integer> counts = new HashMap<>();
for (TextWord word : line.words()) {
for (TextChar ch : word.chars()) {
if (ch.isWhitespace() || ch.isNewline()) {
continue;
}
String name = ch.fontName();
if (name != null && !name.isBlank()) {
counts.merge(name, 1, Integer::sum);
}
}
}
String dominantFont = "";
int max = -1;
for (Map.Entry<String, Integer> e : counts.entrySet()) {
if (e.getValue() > max) {
max = e.getValue();
dominantFont = e.getKey();
}
}
String lower = dominantFont.toLowerCase(java.util.Locale.ROOT);
return lower.contains("bold")
|| lower.contains("black")
|| lower.contains("heavy")
|| lower.contains("semibold");
}
/** Computes the median glyph font size across all pages. */
static float medianFontSize(List<PageText> allPages) {
List<Float> sizes = new ArrayList<>();
for (PageText page : allPages) {
for (TextChar ch : page.chars()) {
if (!ch.isWhitespace() && !ch.isNewline() && ch.fontSize() > 0f) {
sizes.add(ch.fontSize());
}
}
}
return median(sizes, 12f);
}
/** Computes the median TextLine height across all pages. Used when font size is degenerate. */
static float medianLineHeight(List<PageText> allPages) {
List<Float> heights = new ArrayList<>();
for (PageText page : allPages) {
for (TextLine line : page.lines()) {
if (line.height() > 0f && !line.text().isBlank()) {
heights.add(line.height());
}
}
}
return median(heights, 12f);
}
private static float median(List<Float> values, float fallback) {
if (values.isEmpty()) {
return fallback;
}
Collections.sort(values);
int mid = values.size() / 2;
if (values.size() % 2 == 0) {
return (values.get(mid - 1) + values.get(mid)) / 2f;
}
return values.get(mid);
}
/**
* Returns the font size that appears most often (by character count) in the given line. Ties
* are broken in favour of the larger size.
*/
private static float dominantFontSize(TextLine line) {
Map<Float, Integer> counts = new HashMap<>();
for (TextWord word : line.words()) {
for (TextChar ch : word.chars()) {
if (!ch.isWhitespace() && !ch.isNewline() && ch.fontSize() > 0f) {
counts.merge(ch.fontSize(), 1, Integer::sum);
}
}
}
if (counts.isEmpty()) {
return 0f;
}
float dominant = 0f;
int maxCount = -1;
for (Map.Entry<Float, Integer> entry : counts.entrySet()) {
int count = entry.getValue();
float size = entry.getKey();
if (count > maxCount || (count == maxCount && size > dominant)) {
maxCount = count;
dominant = size;
}
}
return dominant;
}
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,82 @@
package stirling.software.common.pdf;
import stirling.software.jpdfium.text.Table;
final class TableRenderer {
private TableRenderer() {}
/** Renders a Table as a GitHub-Flavoured Markdown table string. */
static String render(Table table) {
if (table.rowCount() == 0) {
return "";
}
String[][] grid = table.asGrid();
if (table.rowCount() < 2) {
// No separator row possible — return plain lines
StringBuilder sb = new StringBuilder();
for (int c = 0; c < grid[0].length; c++) {
if (c > 0) sb.append('\n');
sb.append(escape(grid[0][c].trim()));
}
return sb.toString();
}
int cols = grid[0].length;
// Compute column widths: max(3, max content length across all rows)
int[] widths = new int[cols];
for (int c = 0; c < cols; c++) {
widths[c] = 3;
}
for (String[] row : grid) {
for (int c = 0; c < cols; c++) {
String cell = c < row.length ? row[c].trim() : "";
widths[c] = Math.max(widths[c], escape(cell).length());
}
}
StringBuilder sb = new StringBuilder();
// Header row
sb.append(buildRow(grid[0], widths, cols));
sb.append('\n');
// Separator row
sb.append('|');
for (int c = 0; c < cols; c++) {
sb.append('-').append("-".repeat(widths[c])).append('-').append('|');
}
sb.append('\n');
// Data rows
for (int r = 1; r < grid.length; r++) {
sb.append(buildRow(grid[r], widths, cols));
if (r < grid.length - 1) {
sb.append('\n');
}
}
return sb.toString();
}
private static String buildRow(String[] row, int[] widths, int cols) {
StringBuilder sb = new StringBuilder();
sb.append('|');
for (int c = 0; c < cols; c++) {
String cell = c < row.length ? escape(row[c].trim()) : "";
sb.append(' ').append(padRight(cell, widths[c])).append(' ').append('|');
}
return sb.toString();
}
private static String escape(String cell) {
return cell.replace("|", "\\|");
}
private static String padRight(String s, int width) {
if (s.length() >= width) return s;
return s + " ".repeat(width - s.length());
}
}
@@ -5,6 +5,7 @@ import java.io.IOException;
import java.io.InputStream;
import java.io.PipedInputStream;
import java.io.PipedOutputStream;
import java.util.Optional;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicReference;
@@ -17,6 +18,7 @@ import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.cluster.FileStore;
import stirling.software.common.util.JobContext;
/**
* Service for storing and retrieving files with unique file IDs. Used by the AutoJobPostMapping
@@ -32,8 +34,10 @@ public class FileStorage {
private final FileOrUploadService fileOrUploadService;
private final FileStore fileStore;
private final Optional<JobOwnershipService> jobOwnershipService;
public String storeFile(MultipartFile file) throws IOException {
String owner = resolveOwner();
// Fast path: when Spring buffered the multipart to disk (typical for large uploads), the
// backing Resource exposes a real File. Hand the Path to the FileStore so it can do a
// file-to-file copy (Linux sendfile, no copy through Java heap) rather than streaming
@@ -48,7 +52,7 @@ public class FileStorage {
if (res != null && res.isFile()) {
try {
FileStore.Stored stored =
fileStore.store(res.getFile().toPath(), file.getOriginalFilename());
fileStore.store(res.getFile().toPath(), file.getOriginalFilename(), owner);
log.debug("Stored file with ID: {} (fast path)", stored.fileId());
return stored.fileId();
} catch (IOException ex) {
@@ -57,40 +61,45 @@ public class FileStorage {
}
}
try (InputStream in = file.getInputStream()) {
FileStore.Stored stored = fileStore.store(in, file.getOriginalFilename());
FileStore.Stored stored = fileStore.store(in, file.getOriginalFilename(), owner);
log.debug("Stored file with ID: {}", stored.fileId());
return stored.fileId();
}
}
public String storeBytes(byte[] bytes, String originalName) throws IOException {
FileStore.Stored stored = fileStore.store(new ByteArrayInputStream(bytes), originalName);
FileStore.Stored stored =
fileStore.store(new ByteArrayInputStream(bytes), originalName, resolveOwner());
log.debug("Stored byte array with ID: {}", stored.fileId());
return stored.fileId();
}
public MultipartFile retrieveFile(String fileId) throws IOException {
enforceOwnership(fileId);
byte[] fileData = fileStore.retrieveBytes(fileId);
return fileOrUploadService.toMockMultipartFile(fileId, fileData);
}
public byte[] retrieveBytes(String fileId) throws IOException {
enforceOwnership(fileId);
return fileStore.retrieveBytes(fileId);
}
public InputStream retrieveInputStream(String fileId) throws IOException {
enforceOwnership(fileId);
return fileStore.retrieve(fileId);
}
public StoredFile storeInputStream(InputStream inputStream, String originalName)
throws IOException {
FileStore.Stored stored = fileStore.store(inputStream, originalName);
FileStore.Stored stored = fileStore.store(inputStream, originalName, resolveOwner());
log.debug("Stored input stream with ID: {}", stored.fileId());
return new StoredFile(stored.fileId(), stored.size());
}
public String storeFromStreamingBody(StreamingResponseBody body, String originalName)
throws IOException {
String owner = resolveOwner();
// Hold Throwable not IOException: an unchecked failure (NPE, IllegalState, OOM, etc.)
// from the body writer would otherwise close the pipe with EOF and the consumer would
// return a truncated file with no error surfaced to the caller.
@@ -115,7 +124,7 @@ public class FileStorage {
}
}
});
FileStore.Stored stored = fileStore.store(in, originalName);
FileStore.Stored stored = fileStore.store(in, originalName, owner);
Throwable writerErr = bodyError.get();
if (writerErr != null) {
// Body failed mid-write: the FileStore persisted a truncated entry.
@@ -159,21 +168,62 @@ public class FileStorage {
public String storeFromResource(Resource resource, String originalName) throws IOException {
try (InputStream in = resource.getInputStream()) {
FileStore.Stored stored = fileStore.store(in, originalName);
FileStore.Stored stored = fileStore.store(in, originalName, resolveOwner());
log.debug("Stored Resource with ID: {}", stored.fileId());
return stored.fileId();
}
}
public boolean deleteFile(String fileId) {
enforceOwnership(fileId);
return fileStore.delete(fileId);
}
public boolean fileExists(String fileId) {
enforceOwnership(fileId);
return fileStore.exists(fileId);
}
public long getFileSize(String fileId) throws IOException {
enforceOwnership(fileId);
return fileStore.size(fileId);
}
private String resolveOwner() {
String propagated = JobContext.getOwner();
if (propagated != null) {
return propagated;
}
return jobOwnershipService.flatMap(JobOwnershipService::getCurrentUserId).orElse(null);
}
private void enforceOwnership(String fileId) {
if (jobOwnershipService.isEmpty()) {
return;
}
Optional<String> currentUser = jobOwnershipService.get().getCurrentUserId();
if (currentUser.isEmpty()) {
return;
}
String owner;
try {
owner = fileStore.getOwner(fileId);
} catch (IOException e) {
log.warn("Failed to read owner for file {}: {}", fileId, e.getMessage());
throw new SecurityException(
"Access denied: could not verify ownership of the requested file");
}
if (owner == null) {
return;
}
if (!owner.equals(currentUser.get())) {
log.warn(
"Access denied: user {} attempted to access file {} owned by {}",
currentUser.get(),
fileId,
owner);
throw new SecurityException(
"Access denied: you do not have permission to access this file");
}
}
}
@@ -50,6 +50,16 @@ public class InternalApiClient {
"^/api/v1/(general|misc|security|convert|filter)(/[A-Za-z0-9_-]+)+$"
+ "|^/api/v1/ai/tools(/[A-Za-z0-9_-]+)+$");
/**
* Marker propagated on every internal sub-step dispatch so the saas PAYG interceptor classifies
* the call as {@code BillingCategory.AUTOMATION}. By construction every {@link
* InternalApiClient#post} caller is an automation surface (pipeline executor, AI workflow,
* policy runner) running a child tool inside a parent automation flow — see the saas {@code
* PaygChargeInterceptor.determineCategory} precedence chain, where this header dominates any
* per-tool {@code @RequiresFeature} annotation.
*/
public static final String AUTOMATION_HEADER = "X-Stirling-Automation";
private final ServletContext servletContext;
private final UserServiceInterface userService;
private final TempFileManager tempFileManager;
@@ -96,7 +106,23 @@ public class InternalApiClient {
if (apiKey != null && !apiKey.isEmpty()) {
headers.add("X-API-KEY", apiKey);
}
// Tag the sub-step as automation so PAYG bills it under AUTOMATION regardless of which
// tool-level @RequiresFeature annotation the dispatched controller carries (e.g. an AI-OCR
// step inside a policy run must bill as AUTOMATION, not AI). Set unconditionally because
// every caller of this dispatcher is an automation surface by design.
headers.add(AUTOMATION_HEADER, "true");
// A no-file ai/tools call (e.g. create-pdf-from-html-agent) sends only string params, so
// without this RestTemplate would use urlencoded instead of the multipart the controller
// expects. File-bearing calls get the right multipart content-type from RestTemplate.
boolean isAiTool = endpointPath.startsWith("/api/v1/ai/tools/");
boolean hasFilePart =
body.values().stream()
.flatMap(java.util.List::stream)
.anyMatch(v -> v instanceof Resource);
if (isAiTool && !hasFilePart) {
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
}
HttpEntity<MultiValueMap<String, Object>> entity = new HttpEntity<>(body, headers);
RequestCallback requestCallback = restTemplate.httpEntityCallback(entity, Resource.class);
@@ -57,85 +57,43 @@ public class JobExecutorService {
this.resourceMonitor = resourceMonitor;
this.jobQueue = jobQueue;
// Parse session timeout and calculate effective timeout once during initialization
long sessionTimeoutMs = parseSessionTimeout(sessionTimeout);
this.effectiveTimeoutMs = Math.min(asyncRequestTimeoutMs, sessionTimeoutMs);
log.debug(
"Job executor configured with effective timeout of {} ms", this.effectiveTimeoutMs);
}
/**
* Run a job either asynchronously or synchronously
*
* @param async Whether to run the job asynchronously
* @param work The work to be done
* @return The response
*/
public ResponseEntity<?> runJobGeneric(boolean async, Supplier<Object> work) {
return runJobGeneric(async, work, -1);
}
/**
* Run a job either asynchronously or synchronously with a custom timeout
*
* @param async Whether to run the job asynchronously
* @param work The work to be done
* @param customTimeoutMs Custom timeout in milliseconds, or -1 to use the default
* @return The response
*/
public ResponseEntity<?> runJobGeneric(
boolean async, Supplier<Object> work, long customTimeoutMs) {
return runJobGeneric(async, work, customTimeoutMs, false, 50);
}
/**
* Run a job either asynchronously or synchronously with custom parameters
*
* @param async Whether to run the job asynchronously
* @param work The work to be done
* @param customTimeoutMs Custom timeout in milliseconds, or -1 to use the default
* @param queueable Whether this job can be queued when system resources are limited
* @param resourceWeight The resource weight of this job (1-100)
* @return The response
*/
public ResponseEntity<?> runJobGeneric(
boolean async,
Supplier<Object> work,
long customTimeoutMs,
boolean queueable,
int resourceWeight) {
// Generate base UUID
String baseJobId = UUID.randomUUID().toString();
// Scope job to authenticated user if security is enabled
String scopedJobKey = getScopedJobKey(baseJobId);
log.debug("Generated jobId: {} (base: {})", scopedJobKey, baseJobId);
// Store the scoped job ID in the request for potential use by other components
if (request != null) {
request.setAttribute("jobId", scopedJobKey);
// Also track this job ID in the user's session for authorization purposes
// This ensures users can only cancel their own jobs
if (request.getSession() != null) {
@SuppressWarnings("unchecked")
java.util.Set<String> userJobIds =
(java.util.Set<String>) request.getSession().getAttribute("userJobIds");
if (userJobIds == null) {
userJobIds = new java.util.concurrent.ConcurrentSkipListSet<>();
request.getSession().setAttribute("userJobIds", userJobIds);
}
userJobIds.add(scopedJobKey);
log.debug("Added scoped job ID {} to user session", scopedJobKey);
}
}
String jobId = scopedJobKey;
// Determine which timeout to use
final String jobOwner =
jobOwnershipService != null
? jobOwnershipService.getCurrentUserId().orElse(null)
: null;
long timeoutToUse = customTimeoutMs > 0 ? customTimeoutMs : effectiveTimeoutMs;
log.debug(
@@ -146,7 +104,6 @@ public class JobExecutorService {
queueable,
resourceWeight);
// Check if we need to queue this job based on resource availability
boolean shouldQueue =
queueable
&& async
@@ -154,7 +111,6 @@ public class JobExecutorService {
resourceMonitor.shouldQueueJob(resourceWeight);
if (shouldQueue) {
// Queue the job instead of executing immediately
log.debug(
"Queueing job {} due to resource constraints (weight: {})",
jobId,
@@ -162,18 +118,13 @@ public class JobExecutorService {
taskManager.createTask(jobId);
// Create a specialized wrapper that updates the TaskManager
final String capturedJobIdForQueue = jobId;
Supplier<Object> wrappedWork =
() -> {
try {
// Set jobId in ThreadLocal context for the queued job
stirling.software.common.util.JobContext.setJobId(
capturedJobIdForQueue);
log.debug(
"Set jobId {} in JobContext for queued job execution",
capturedJobIdForQueue);
stirling.software.common.util.JobContext.setOwner(jobOwner);
Object result = work.get();
processJobResult(capturedJobIdForQueue, result);
return result;
@@ -186,21 +137,17 @@ public class JobExecutorService {
taskManager.setError(capturedJobIdForQueue, e.getMessage());
throw e;
} finally {
// Clean up ThreadLocal to avoid memory leaks
stirling.software.common.util.JobContext.clear();
}
};
// Queue the job and get the future
CompletableFuture<ResponseEntity<?>> future =
jobQueue.queueJob(jobId, resourceWeight, wrappedWork, timeoutToUse);
// Return immediately with job ID
return ResponseEntity.ok().body(new JobResponse<>(true, jobId, null));
} else if (async) {
taskManager.createTask(jobId);
// Capture the jobId for the async thread
final String capturedJobId = jobId;
executor.execute(
@@ -211,13 +158,8 @@ public class JobExecutorService {
capturedJobId,
timeoutToUse);
// Set jobId in ThreadLocal context for the async thread
stirling.software.common.util.JobContext.setJobId(capturedJobId);
log.debug(
"Set jobId {} in JobContext for async execution",
capturedJobId);
// Execute with timeout
stirling.software.common.util.JobContext.setOwner(jobOwner);
Object result = executeWithTimeout(() -> work.get(), timeoutToUse);
processJobResult(capturedJobId, result);
} catch (TimeoutException te) {
@@ -227,7 +169,6 @@ public class JobExecutorService {
log.error("Error executing job {}: {}", jobId, e.getMessage(), e);
taskManager.setError(jobId, e.getMessage());
} finally {
// Clean up ThreadLocal to avoid memory leaks
stirling.software.common.util.JobContext.clear();
}
});
@@ -237,27 +178,19 @@ public class JobExecutorService {
try {
log.debug("Running sync job with timeout {} ms", timeoutToUse);
// Make jobId available to downstream components on the worker thread
stirling.software.common.util.JobContext.setJobId(jobId);
log.debug("Set jobId {} in JobContext for sync execution", jobId);
// Execute with timeout
Object result = executeWithTimeout(() -> work.get(), timeoutToUse);
// If the result is already a ResponseEntity, return it directly
if (result instanceof ResponseEntity) {
return (ResponseEntity<?>) result;
}
// Process different result types
return handleResultForSyncJob(result);
} catch (TimeoutException te) {
log.error("Synchronous job timed out after {} ms", timeoutToUse);
return ResponseEntity.internalServerError()
.body(Map.of("error", "Job timed out after " + timeoutToUse + " ms"));
} catch (RuntimeException e) {
// Check if this is a typed exception that should be handled by
// GlobalExceptionHandler (either directly or wrapped)
Throwable cause = e.getCause();
if (e instanceof IllegalArgumentException
|| cause
@@ -267,16 +200,13 @@ public class JobExecutorService {
instanceof
stirling.software.common.util.ExceptionUtils
.BaseValidationException) {
// Rethrow so GlobalExceptionHandler can handle with proper HTTP status codes
throw e;
}
// Handle other RuntimeExceptions as generic errors
log.error("Error executing synchronous job: {}", e.getMessage(), e);
return ResponseEntity.internalServerError()
.body(Map.of("error", "Job failed: " + e.getMessage()));
} catch (Exception e) {
log.error("Error executing synchronous job: {}", e.getMessage(), e);
// Construct a JSON error response
return ResponseEntity.internalServerError()
.body(Map.of("error", "Job failed: " + e.getMessage()));
} finally {
@@ -285,23 +215,13 @@ public class JobExecutorService {
}
}
/**
* Process the result of an asynchronous job
*
* @param jobId The job ID
* @param result The result
*/
private void processJobResult(String jobId, Object result) {
try {
if (result instanceof byte[]) {
// Store byte array directly to disk to avoid double memory consumption
String fileId = fileStorage.storeBytes((byte[]) result, "result.pdf");
taskManager.setFileResult(
jobId, fileId, "result.pdf", MediaType.APPLICATION_PDF_VALUE);
log.debug("Stored byte[] result with fileId: {}", fileId);
// Let the byte array get collected naturally in the next GC cycle
// We don't need to force System.gc() which can be harmful
} else if (result instanceof ResponseEntity) {
ResponseEntity<?> response = (ResponseEntity<?>) result;
Object body = response.getBody();
@@ -330,16 +250,13 @@ public class JobExecutorService {
taskManager.setFileResult(jobId, fileId, filename, contentType);
log.debug("Stored ResponseEntity<Resource> result with fileId: {}", fileId);
} else {
// Check if the response body contains a fileId
if (body != null && body.toString().contains("fileId")) {
try {
// Try to extract fileId using reflection
java.lang.reflect.Method getFileId =
body.getClass().getMethod("getFileId");
String fileId = (String) getFileId.invoke(body);
if (fileId != null && !fileId.isEmpty()) {
// Try to get filename and content type
String filename = "result.pdf";
String contentType = MediaType.APPLICATION_PDF_VALUE;
@@ -379,7 +296,6 @@ public class JobExecutorService {
}
}
// Store generic result
taskManager.setResult(jobId, body);
}
} else if (result instanceof MultipartFile file) {
@@ -388,16 +304,13 @@ public class JobExecutorService {
jobId, fileId, file.getOriginalFilename(), file.getContentType());
log.debug("Stored MultipartFile result with fileId: {}", fileId);
} else {
// Check if result has a fileId field
if (result != null) {
try {
// Try to extract fileId using reflection
java.lang.reflect.Method getFileId =
result.getClass().getMethod("getFileId");
String fileId = (String) getFileId.invoke(result);
if (fileId != null && !fileId.isEmpty()) {
// Try to get filename and content type
String filename = "result.pdf";
String contentType = MediaType.APPLICATION_PDF_VALUE;
@@ -435,7 +348,6 @@ public class JobExecutorService {
}
}
// Default case: store the result as is
taskManager.setResult(jobId, result);
}
@@ -446,16 +358,8 @@ public class JobExecutorService {
}
}
/**
* Handle different result types for synchronous jobs
*
* @param result The result object
* @return The appropriate ResponseEntity
* @throws IOException If there is an error processing the result
*/
private ResponseEntity<?> handleResultForSyncJob(Object result) throws IOException {
if (result instanceof byte[]) {
// Return byte array as PDF
return ResponseEntity.ok()
.contentType(MediaType.APPLICATION_PDF)
.header(
@@ -463,7 +367,6 @@ public class JobExecutorService {
"form-data; name=\"attachment\"; filename=\"result.pdf\"")
.body(result);
} else if (result instanceof MultipartFile file) {
// Return MultipartFile content
return ResponseEntity.ok()
.contentType(MediaType.parseMediaType(file.getContentType()))
.header(
@@ -473,7 +376,6 @@ public class JobExecutorService {
+ "\"")
.body(file.getBytes());
} else {
// Default case: return as JSON
return ResponseEntity.ok(result);
}
}
@@ -493,15 +395,9 @@ public class JobExecutorService {
return mediaType != null ? mediaType.toString() : MediaType.APPLICATION_PDF_VALUE;
}
/**
* Parse session timeout string (e.g., "30m", "1h") to milliseconds
*
* @param timeout The timeout string
* @return The timeout in milliseconds
*/
private long parseSessionTimeout(String timeout) {
if (timeout == null || timeout.isEmpty()) {
return 30 * 60 * 1000; // Default: 30 minutes
return 30 * 60 * 1000;
}
try {
@@ -523,27 +419,16 @@ public class JobExecutorService {
case "m" -> (long) (numericValue * 60 * 1000);
case "h" -> (long) (numericValue * 60 * 60 * 1000);
case "d" -> (long) (numericValue * 24 * 60 * 60 * 1000);
default -> (long) (numericValue * 60 * 1000); // Default to minutes
default -> (long) (numericValue * 60 * 1000);
};
} catch (Exception e) {
log.warn("Could not parse session timeout '{}', using default", timeout);
return 30 * 60 * 1000; // Default: 30 minutes
return 30 * 60 * 1000;
}
}
/**
* Execute a supplier with a timeout
*
* @param supplier The supplier to execute
* @param timeoutMs The timeout in milliseconds
* @return The result from the supplier
* @throws TimeoutException If the execution times out
* @throws Exception If the supplier throws an exception
*/
private <T> T executeWithTimeout(Supplier<T> supplier, long timeoutMs)
throws TimeoutException, Exception {
// Use the same executor as other async jobs for consistency
// This ensures all operations run on the same thread pool
String currentJobId = stirling.software.common.util.JobContext.getJobId();
java.util.concurrent.CompletableFuture<T> future =
@@ -577,17 +462,10 @@ public class JobExecutorService {
}
}
/**
* Get a scoped job key that includes user ownership when security is enabled.
*
* @param baseJobId the base job identifier
* @return scoped job key, or just baseJobId if no ownership service available
*/
private String getScopedJobKey(String baseJobId) {
if (jobOwnershipService != null) {
return jobOwnershipService.createScopedJobKey(baseJobId);
}
// Security disabled, return unsecured job key
return baseJobId;
}
}
@@ -3,7 +3,6 @@ package stirling.software.common.service;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
@@ -35,7 +34,7 @@ public class MobileScannerService {
public MobileScannerService() throws IOException {
// Create temp directory for mobile scanner uploads
this.tempDirectory =
Paths.get(System.getProperty("java.io.tmpdir"), "stirling-mobile-scanner");
Path.of(System.getProperty("java.io.tmpdir"), "stirling-mobile-scanner");
Files.createDirectories(tempDirectory);
log.info("Mobile scanner temp directory: {}", tempDirectory);
}
@@ -8,7 +8,7 @@ import java.lang.management.OperatingSystemMXBean;
import java.lang.management.RuntimeMXBean;
import java.lang.management.ThreadMXBean;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.Path;
import java.util.HashMap;
import java.util.Locale;
import java.util.Map;
@@ -160,7 +160,7 @@ public class PostHogService {
}
private boolean isRunningInDocker() {
return Files.exists(Paths.get("/.dockerenv"));
return Files.exists(Path.of("/.dockerenv"));
}
private Map<String, Object> getDockerMetrics() {
@@ -1,11 +1,21 @@
package stirling.software.common.service;
import java.util.List;
/** Provides metadata about tool endpoints for internal dispatch. */
public interface ToolMetadataService {
/** Returns true if the given operation path accepts multiple input files. */
boolean isMultiInput(String operationPath);
/**
* Returns the file extensions (lowercase, no leading dot, e.g. {@code "pdf"}) that the
* operation accepts as input ({@code output=false}) or produces as output ({@code
* output=true}), derived from the endpoint's declared type. Returns {@code null} when the
* endpoint declares no specific type, which callers should treat as "any type accepted".
*/
List<String> getExtensionTypes(boolean output, String operationPath);
/**
* Returns true when the endpoint's ZIP response is a transport for multiple typed results and
* should be unpacked: multi-output endpoints (Type:SIMO / Type:MIMO) and wrapper declarations
@@ -255,7 +255,7 @@ public class GeneralUtils {
String pattern = locationPattern;
if (pattern.startsWith("file:")) {
String rawPath = pattern.substring(5).replace("\\*", "").replace("/*", "");
Path normalizePath = Paths.get(rawPath).normalize();
Path normalizePath = Path.of(rawPath).normalize();
pattern = "file:" + normalizePath.toString().replace("\\", "/") + "/*";
}
return ResourcePatternUtils.getResourcePatternResolver(resourceLoader)
@@ -837,7 +837,7 @@ public class GeneralUtils {
}
public boolean createDir(String path) {
Path folder = Paths.get(path);
Path folder = Path.of(path);
if (!Files.exists(folder)) {
try {
Files.createDirectories(folder);
@@ -867,7 +867,7 @@ public class GeneralUtils {
public void saveKeyToSettings(String key, Object newValue) throws IOException {
String[] keyArray = key.split("\\.");
Path settingsPath = Paths.get(InstallationPathConfig.getSettingsPath());
Path settingsPath = Path.of(InstallationPathConfig.getSettingsPath());
YamlHelper settingsYaml = new YamlHelper(settingsPath);
settingsYaml.updateValue(Arrays.asList(keyArray), newValue);
settingsYaml.saveOverride(settingsPath);
@@ -888,7 +888,7 @@ public class GeneralUtils {
return;
}
Path settingsPath = Paths.get(InstallationPathConfig.getSettingsPath());
Path settingsPath = Path.of(InstallationPathConfig.getSettingsPath());
YamlHelper settingsYaml = new YamlHelper(settingsPath);
// Apply all updates to the same YamlHelper instance
@@ -974,11 +974,11 @@ public class GeneralUtils {
*/
public void extractPipeline() throws IOException {
Path pipelineDir =
Paths.get(InstallationPathConfig.getPipelinePath(), DEFAULT_WEBUI_CONFIGS_DIR);
Path.of(InstallationPathConfig.getPipelinePath(), DEFAULT_WEBUI_CONFIGS_DIR);
Files.createDirectories(pipelineDir);
for (String name : DEFAULT_VALID_PIPELINE) {
if (!Paths.get(name).getFileName().toString().equals(name)) {
if (!Path.of(name).getFileName().toString().equals(name)) {
log.error("Invalid pipeline file name: {}", name);
throw new IllegalArgumentException("Invalid pipeline file name: " + name);
}
@@ -1014,7 +1014,7 @@ public class GeneralUtils {
throw new IllegalArgumentException(
"scriptName must not contain path traversal characters");
}
if (!Paths.get(scriptName).getFileName().toString().equals(scriptName)) {
if (!Path.of(scriptName).getFileName().toString().equals(scriptName)) {
throw new IllegalArgumentException(
"scriptName must not contain path traversal characters");
}
@@ -1024,7 +1024,7 @@ public class GeneralUtils {
"scriptName must be either 'png_to_webp.py' or 'split_photos.py'");
}
Path scriptsDir = Paths.get(InstallationPathConfig.getScriptsPath(), PYTHON_SCRIPTS_DIR);
Path scriptsDir = Path.of(InstallationPathConfig.getScriptsPath(), PYTHON_SCRIPTS_DIR);
Files.createDirectories(scriptsDir);
Path target = scriptsDir.resolve(scriptName);
@@ -4,7 +4,6 @@ import java.io.File;
import java.net.URISyntaxException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import lombok.extern.slf4j.Slf4j;
@@ -20,7 +19,7 @@ public class JarPathUtil {
public static Path currentJar() {
try {
Path jar =
Paths.get(
Path.of(
JarPathUtil.class
.getProtectionDomain()
.getCodeSource()
@@ -61,14 +60,14 @@ public class JarPathUtil {
}
// Location 2: ./build/libs/ (development build)
possibleLocations[1] = Paths.get("build", "libs", "restart-helper.jar").toAbsolutePath();
possibleLocations[1] = Path.of("build", "libs", "restart-helper.jar").toAbsolutePath();
// Location 3: app/common/build/libs/ (multi-module build)
possibleLocations[2] =
Paths.get("app", "common", "build", "libs", "restart-helper.jar").toAbsolutePath();
Path.of("app", "common", "build", "libs", "restart-helper.jar").toAbsolutePath();
// Location 4: Current working directory
possibleLocations[3] = Paths.get("restart-helper.jar").toAbsolutePath();
possibleLocations[3] = Path.of("restart-helper.jar").toAbsolutePath();
// Check each location
for (Path location : possibleLocations) {
@@ -1,8 +1,9 @@
package stirling.software.common.util;
/** Thread-local context for passing job ID across async boundaries */
/** Thread-local context for passing job ID and owner across async boundaries */
public class JobContext {
private static final ThreadLocal<String> CURRENT_JOB_ID = new ThreadLocal<>();
private static final ThreadLocal<String> CURRENT_OWNER = new ThreadLocal<>();
public static void setJobId(String jobId) {
CURRENT_JOB_ID.set(jobId);
@@ -12,7 +13,16 @@ public class JobContext {
return CURRENT_JOB_ID.get();
}
public static void setOwner(String owner) {
CURRENT_OWNER.set(owner);
}
public static String getOwner() {
return CURRENT_OWNER.get();
}
public static void clear() {
CURRENT_JOB_ID.remove();
CURRENT_OWNER.remove();
}
}
@@ -0,0 +1,310 @@
package stirling.software.common.util;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.Set;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import java.util.zip.ZipOutputStream;
import javax.xml.XMLConstants;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import org.springframework.stereotype.Component;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.NamedNodeMap;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.SAXException;
import io.github.pixee.security.ZipSecurity;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.SsrfProtectionService;
// Strips external refs from OOXML/ODF uploads so LibreOffice can't be made to fetch them.
@Component
@Slf4j
public class OfficeDocumentSanitizer {
private static final Set<String> OOXML_EXTENSIONS =
Set.of(
"docx", "docm", "dotx", "dotm", "xlsx", "xlsm", "xltx", "xltm", "pptx", "pptm",
"potx", "potm", "ppsx", "ppsm");
private static final Set<String> ODF_EXTENSIONS =
Set.of(
"odt", "ott", "ods", "ots", "odp", "otp", "odg", "otg", "odf", "odc", "odi",
"odm");
private static final Set<String> ODF_XML_PARTS =
Set.of("content.xml", "styles.xml", "meta.xml", "settings.xml");
private final SsrfProtectionService ssrfProtectionService;
private final ApplicationProperties applicationProperties;
public OfficeDocumentSanitizer(
SsrfProtectionService ssrfProtectionService,
ApplicationProperties applicationProperties) {
this.ssrfProtectionService = ssrfProtectionService;
this.applicationProperties = applicationProperties;
}
public boolean isSanitizableExtension(String extension) {
if (extension == null) {
return false;
}
String lower = extension.toLowerCase(Locale.ROOT);
return OOXML_EXTENSIONS.contains(lower) || ODF_EXTENSIONS.contains(lower);
}
public byte[] sanitize(byte[] documentBytes, String extension) throws IOException {
if (documentBytes == null || documentBytes.length == 0) {
throw new IOException("Office document input is empty or null");
}
if (applicationProperties.getSystem().isDisableSanitize()) {
log.debug("Office document sanitization disabled by configuration");
return documentBytes;
}
if (!isSanitizableExtension(extension)) {
return documentBytes;
}
ByteArrayOutputStream out = new ByteArrayOutputStream(documentBytes.length);
try (ZipInputStream zipIn =
ZipSecurity.createHardenedInputStream(
new ByteArrayInputStream(documentBytes));
ZipOutputStream zipOut = new ZipOutputStream(out)) {
ZipEntry entry;
while ((entry = zipIn.getNextEntry()) != null) {
String name = entry.getName();
byte[] bytes = entry.isDirectory() ? new byte[0] : zipIn.readAllBytes();
if (!entry.isDirectory()) {
bytes = sanitizeEntry(name, bytes);
}
ZipEntry outEntry = new ZipEntry(name);
if (entry.getComment() != null) {
outEntry.setComment(entry.getComment());
}
if (entry.getExtra() != null) {
outEntry.setExtra(entry.getExtra());
}
zipOut.putNextEntry(outEntry);
if (!entry.isDirectory()) {
zipOut.write(bytes);
}
zipOut.closeEntry();
}
}
return out.toByteArray();
}
private byte[] sanitizeEntry(String entryName, byte[] entryBytes) {
String lower = entryName.toLowerCase(Locale.ROOT);
try {
if (lower.endsWith(".rels")) {
return sanitizeOoxmlRels(entryBytes);
}
if (isOdfXmlPart(lower)) {
return sanitizeOdfXml(entryBytes);
}
} catch (ParserConfigurationException
| SAXException
| IOException
| TransformerException e) {
log.warn(
"Failed to parse XML part '{}' for sanitization, leaving as-is: {}",
entryName,
e.getMessage());
}
return entryBytes;
}
private boolean isOdfXmlPart(String lowerName) {
int slash = lowerName.lastIndexOf('/');
String base = slash >= 0 ? lowerName.substring(slash + 1) : lowerName;
return ODF_XML_PARTS.contains(base);
}
private byte[] sanitizeOoxmlRels(byte[] xmlBytes)
throws IOException, ParserConfigurationException, SAXException, TransformerException {
Document doc = parseSecurely(xmlBytes);
Element root = doc.getDocumentElement();
if (root == null) {
return xmlBytes;
}
NodeList relationships = root.getElementsByTagNameNS("*", "Relationship");
List<Node> toRemove = new ArrayList<>();
for (int i = 0; i < relationships.getLength(); i++) {
Node node = relationships.item(i);
NamedNodeMap attrs = node.getAttributes();
if (attrs == null) {
continue;
}
Node targetMode = attrs.getNamedItem("TargetMode");
if (targetMode == null || !"external".equalsIgnoreCase(targetMode.getNodeValue())) {
continue;
}
Node target = attrs.getNamedItem("Target");
String targetValue = target == null ? "" : target.getNodeValue();
if (isAdminAllowed(targetValue)) {
continue;
}
log.warn(
"Stripping OOXML external relationship target: {}",
truncateForLog(targetValue));
toRemove.add(node);
}
if (toRemove.isEmpty()) {
return xmlBytes;
}
for (Node n : toRemove) {
n.getParentNode().removeChild(n);
}
return serializeDocument(doc);
}
private byte[] sanitizeOdfXml(byte[] xmlBytes)
throws IOException, ParserConfigurationException, SAXException, TransformerException {
Document doc = parseSecurely(xmlBytes);
Element root = doc.getDocumentElement();
if (root == null) {
return xmlBytes;
}
boolean modified = stripExternalHrefs(root);
if (!modified) {
return xmlBytes;
}
return serializeDocument(doc);
}
private boolean stripExternalHrefs(Node node) {
boolean modified = false;
if (node.getNodeType() == Node.ELEMENT_NODE) {
NamedNodeMap attrs = node.getAttributes();
List<String> hrefAttrsToRemove = new ArrayList<>();
for (int i = 0; i < attrs.getLength(); i++) {
Node attr = attrs.item(i);
String name = attr.getNodeName();
if (name == null) {
continue;
}
String lower = name.toLowerCase(Locale.ROOT);
if (!(lower.equals("xlink:href")
|| lower.endsWith(":href")
|| lower.equals("href"))) {
continue;
}
String value = attr.getNodeValue();
if (!isExternalUrl(value)) {
continue;
}
if (isAdminAllowed(value)) {
continue;
}
log.warn(
"Stripping ODF external href attribute ({}): {}",
name,
truncateForLog(value));
hrefAttrsToRemove.add(name);
}
Element element = (Element) node;
for (String attrName : hrefAttrsToRemove) {
element.removeAttribute(attrName);
modified = true;
}
}
NodeList children = node.getChildNodes();
for (int i = 0; i < children.getLength(); i++) {
if (stripExternalHrefs(children.item(i))) {
modified = true;
}
}
return modified;
}
private boolean isExternalUrl(String url) {
if (url == null) {
return false;
}
String trimmed = url.trim().toLowerCase(Locale.ROOT);
if (trimmed.isEmpty() || trimmed.startsWith("#") || trimmed.startsWith("../")) {
return false;
}
return trimmed.startsWith("http://")
|| trimmed.startsWith("https://")
|| trimmed.startsWith("ftp://")
|| trimmed.startsWith("ftps://")
|| trimmed.startsWith("file:")
|| trimmed.startsWith("smb:")
|| trimmed.startsWith("\\\\")
|| trimmed.startsWith("//");
}
// Preserved only with an explicit allowedDomains entry; MEDIUM default would admit public URLs.
private boolean isAdminAllowed(String url) {
if (ssrfProtectionService == null || url == null || url.isBlank()) {
return false;
}
ApplicationProperties.Html.UrlSecurity config =
applicationProperties.getSystem().getHtml().getUrlSecurity();
if (config == null
|| config.getAllowedDomains() == null
|| config.getAllowedDomains().isEmpty()) {
return false;
}
return ssrfProtectionService.isUrlAllowed(url);
}
private Document parseSecurely(byte[] xmlBytes)
throws ParserConfigurationException, SAXException, IOException {
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
factory.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);
factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
factory.setFeature("http://xml.org/sax/features/external-general-entities", false);
factory.setFeature("http://xml.org/sax/features/external-parameter-entities", false);
factory.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);
factory.setXIncludeAware(false);
factory.setExpandEntityReferences(false);
factory.setNamespaceAware(true);
DocumentBuilder builder = factory.newDocumentBuilder();
return builder.parse(new ByteArrayInputStream(xmlBytes));
}
private byte[] serializeDocument(Document doc) throws TransformerException {
TransformerFactory tf = TransformerFactory.newInstance();
tf.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);
Transformer transformer = tf.newTransformer();
transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8");
transformer.setOutputProperty(OutputKeys.INDENT, "no");
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "no");
ByteArrayOutputStream baos = new ByteArrayOutputStream();
transformer.transform(new DOMSource(doc), new StreamResult(baos));
return baos.toByteArray();
}
private String truncateForLog(String value) {
if (value == null) {
return "null";
}
return value.length() > 80 ? value.substring(0, 80) + "..." : value;
}
}
@@ -83,7 +83,16 @@ public class RequestUriUtils {
return false;
}
// Blocklist of backend/non-frontend paths that should still go through filters
// Blocklist of backend/non-frontend paths that should still go through filters.
//
// `/files` was historically a backend route; it is now a frontend route
// owned by HomePage / FileManagerView. Direct-nav or refresh on /files
// (or /files/<folder-uuid>) was returning the Spring auth filter's 401
// JSON instead of serving index.html, so the SPA never got a chance to
// mount and the user saw a raw error response. There are no `/files`
// backend mappings at the servlet root - the real storage endpoints
// live under `/api/v1/storage/files`, which is filtered out a few lines
// up by the `startsWith("/api/")` guard.
String[] backendOnlyPrefixes = {
"/register",
"/pipeline",
@@ -91,7 +100,6 @@ public class RequestUriUtils {
"/pdfjs-legacy",
"/fonts",
"/images",
"/files",
"/css",
"/js",
"/swagger",
@@ -181,7 +189,7 @@ public class RequestUriUtils {
|| trimmedUri.startsWith(
"/api/v1/mobile-scanner/") // Mobile scanner endpoints (no auth)
|| trimmedUri.startsWith("/v1/api-docs")
// Workflow participant endpoints access controlled by share tokens, not login
// Workflow participant endpoints - access controlled by share tokens, not login
|| trimmedUri.startsWith("/api/v1/workflow/participant/")
// Share-link SPA bootstrap; data APIs remain protected
|| trimmedUri.matches("^/share/[^/]+/?$");
@@ -0,0 +1,48 @@
package stirling.software.common.util.propertyeditor;
import java.beans.PropertyEditorSupport;
import java.util.ArrayList;
import java.util.List;
import lombok.extern.slf4j.Slf4j;
import tools.jackson.core.type.TypeReference;
import tools.jackson.databind.DeserializationFeature;
import tools.jackson.databind.ObjectMapper;
import tools.jackson.databind.json.JsonMapper;
/**
* Binds a multipart form value containing a JSON array into a typed {@code List<T>}. Used for
* endpoints that accept structured list parameters via {@code @ModelAttribute} — the form field
* carries the full JSON array as its value and the editor parses it once.
*/
@Slf4j
public class JsonListPropertyEditor<T> extends PropertyEditorSupport {
private static final ObjectMapper OBJECT_MAPPER =
JsonMapper.builder()
.enable(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY)
.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
.build();
private final TypeReference<? extends List<T>> typeRef;
public JsonListPropertyEditor(TypeReference<? extends List<T>> typeRef) {
this.typeRef = typeRef;
}
@Override
public void setAsText(String text) throws IllegalArgumentException {
if (text == null || text.trim().isEmpty()) {
setValue(new ArrayList<T>());
return;
}
try {
setValue(OBJECT_MAPPER.readValue(text, typeRef));
} catch (Exception e) {
log.error("Failed to parse JSON list value", e);
throw new IllegalArgumentException(
"Expected a JSON array but could not parse: " + e.getMessage());
}
}
}
@@ -0,0 +1,41 @@
package stirling.software.common.util.propertyeditor;
import java.beans.PropertyEditorSupport;
import lombok.extern.slf4j.Slf4j;
import tools.jackson.databind.DeserializationFeature;
import tools.jackson.databind.ObjectMapper;
import tools.jackson.databind.json.JsonMapper;
/**
* Binds a multipart form value containing a JSON object into a typed {@code T}. Companion to {@link
* JsonListPropertyEditor} for single-object nested fields on {@code @ModelAttribute} endpoints.
*/
@Slf4j
public class JsonObjectPropertyEditor<T> extends PropertyEditorSupport {
private static final ObjectMapper OBJECT_MAPPER =
JsonMapper.builder().disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES).build();
private final Class<T> type;
public JsonObjectPropertyEditor(Class<T> type) {
this.type = type;
}
@Override
public void setAsText(String text) throws IllegalArgumentException {
if (text == null || text.trim().isEmpty()) {
setValue(null);
return;
}
try {
setValue(OBJECT_MAPPER.readValue(text, type));
} catch (Exception e) {
log.error("Failed to parse JSON object value", e);
throw new IllegalArgumentException(
"Expected a JSON object but could not parse: " + e.getMessage());
}
}
}
@@ -1,53 +0,0 @@
package stirling.software.common.util.propertyeditor;
import java.beans.PropertyEditorSupport;
import java.util.ArrayList;
import java.util.List;
import lombok.extern.slf4j.Slf4j;
import tools.jackson.databind.DeserializationFeature;
import tools.jackson.databind.JavaType;
import tools.jackson.databind.ObjectMapper;
import tools.jackson.databind.json.JsonMapper;
/**
* Spring property editor that decodes a JSON string into a typed {@link ArrayList}. Used to bind
* complex list parameters (e.g. {@code List<RedactionArea>}, {@code List<EditTextOperation>}) from
* multipart form fields, where Spring's default binding cannot deserialize a JSON array.
*/
@Slf4j
public class StringToArrayListPropertyEditor<T> extends PropertyEditorSupport {
private final ObjectMapper objectMapper =
JsonMapper.builder()
.enable(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY)
.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
.build();
private final Class<T> elementType;
public StringToArrayListPropertyEditor(Class<T> elementType) {
this.elementType = elementType;
}
@Override
public void setAsText(String text) throws IllegalArgumentException {
if (text == null || text.trim().isEmpty()) {
setValue(new ArrayList<>());
return;
}
try {
JavaType listType =
objectMapper
.getTypeFactory()
.constructCollectionType(ArrayList.class, elementType);
List<T> list = objectMapper.readValue(text, listType);
setValue(list);
} catch (Exception e) {
log.error("Exception while converting {}", e);
throw new IllegalArgumentException(
"Failed to convert java.lang.String to java.util.List");
}
}
}
@@ -0,0 +1,481 @@
package stirling.software.SPDF.config;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.util.List;
import java.util.Set;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import stirling.software.SPDF.config.EndpointConfiguration.DisableReason;
import stirling.software.SPDF.config.EndpointConfiguration.EndpointAvailability;
import stirling.software.common.model.ApplicationProperties;
/**
* Unit tests for {@link EndpointConfiguration}. The class wires up its endpoint/group registry in
* {@code init()} during construction and then applies environment overrides. We build it with a
* real {@link ApplicationProperties} (whose System/Endpoints sub-objects are non-null by default)
* so the constructor runs cleanly without any mocking.
*/
class EndpointConfigurationGapTest {
private ApplicationProperties applicationProperties;
/**
* Construct an EndpointConfiguration with the given pro flag and current applicationProperties.
*/
private EndpointConfiguration build(boolean runningProOrHigher) {
return new EndpointConfiguration(applicationProperties, runningProOrHigher);
}
/** Default config: not pro, no removals, url-to-pdf disabled (default System flag is false). */
private EndpointConfiguration buildDefault() {
return build(false);
}
@BeforeEach
void setUp() {
applicationProperties = new ApplicationProperties();
}
@Nested
@DisplayName("endpointKeyForUri (static)")
class EndpointKeyForUriTests {
@Test
@DisplayName("returns null for null uri")
void nullUri() {
assertNull(EndpointConfiguration.endpointKeyForUri(null));
}
@Test
@DisplayName("returns null when uri does not contain /api/v1")
void notApiPath() {
assertNull(EndpointConfiguration.endpointKeyForUri("/foo/bar/baz"));
assertNull(EndpointConfiguration.endpointKeyForUri("https://example.com/home"));
}
@Test
@DisplayName("returns null when uri has too few path segments")
void tooFewSegments() {
// "/api/v1/general" splits to ["", "api", "v1", "general"] -> length 4, not > 4
assertNull(EndpointConfiguration.endpointKeyForUri("/api/v1/general"));
}
@Test
@DisplayName("extracts plain endpoint key from a standard /api/v1/<group>/<endpoint> uri")
void plainEndpoint() {
assertEquals(
"remove-pages",
EndpointConfiguration.endpointKeyForUri("/api/v1/general/remove-pages"));
}
@Test
@DisplayName("builds a <from>-to-<to> key for convert endpoints")
void convertEndpoint() {
assertEquals(
"pdf-to-img",
EndpointConfiguration.endpointKeyForUri("/api/v1/convert/pdf/img"));
}
@Test
@DisplayName("convert path without a target segment falls back to the segment after group")
void convertWithoutTarget() {
// "/api/v1/convert/pdf" -> length 5, the convert branch needs length > 5
assertEquals("pdf", EndpointConfiguration.endpointKeyForUri("/api/v1/convert/pdf"));
}
}
@Nested
@DisplayName("enable / disable endpoint")
class EnableDisableEndpointTests {
@Test
@DisplayName("a freshly registered endpoint is enabled by default")
void enabledByDefault() {
EndpointConfiguration config = buildDefault();
assertTrue(config.isEndpointEnabled("merge-pdfs"));
}
@Test
@DisplayName("disableEndpoint marks the endpoint disabled")
void disableEndpoint() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("merge-pdfs");
assertFalse(config.isEndpointEnabled("merge-pdfs"));
}
@Test
@DisplayName("enableEndpoint re-enables a previously disabled endpoint")
void reEnableEndpoint() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("merge-pdfs");
assertFalse(config.isEndpointEnabled("merge-pdfs"));
config.enableEndpoint("merge-pdfs");
assertTrue(config.isEndpointEnabled("merge-pdfs"));
}
@Test
@DisplayName("leading slash is normalized away on disable")
void leadingSlashNormalizedOnDisable() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("/merge-pdfs");
// both forms resolve to the same key
assertFalse(config.isEndpointEnabled("merge-pdfs"));
assertFalse(config.isEndpointEnabled("/merge-pdfs"));
}
@Test
@DisplayName("isEndpointEnabled tolerates a leading slash on the query")
void leadingSlashOnQuery() {
EndpointConfiguration config = buildDefault();
assertTrue(config.isEndpointEnabled("/merge-pdfs"));
}
@Test
@DisplayName("disabling clears with enable, removing the disable reason")
void enableClearsReason() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("split-pages", DisableReason.DEPENDENCY);
assertEquals(
DisableReason.DEPENDENCY,
config.getEndpointAvailability("split-pages").getReason());
config.enableEndpoint("split-pages");
EndpointAvailability availability = config.getEndpointAvailability("split-pages");
assertTrue(availability.isEnabled());
assertNull(availability.getReason());
}
}
@Nested
@DisplayName("isEndpointEnabledForUri")
class IsEndpointEnabledForUriTests {
@Test
@DisplayName("translates a /api/v1 uri to a key and reports its status")
void translatesUri() {
EndpointConfiguration config = buildDefault();
assertTrue(config.isEndpointEnabledForUri("/api/v1/general/merge-pdfs"));
config.disableEndpoint("merge-pdfs");
assertFalse(config.isEndpointEnabledForUri("/api/v1/general/merge-pdfs"));
}
@Test
@DisplayName("falls back to treating a non-api uri as a raw key")
void fallsBackToRawKey() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("merge-pdfs");
// non-api path: key resolution returns null, so the uri itself is used as the key
assertFalse(config.isEndpointEnabledForUri("merge-pdfs"));
}
}
@Nested
@DisplayName("group enable / disable")
class GroupTests {
@Test
@DisplayName("a functional group with all endpoints enabled reports enabled")
void functionalGroupEnabled() {
EndpointConfiguration config = buildDefault();
assertTrue(config.isGroupEnabled("PageOps"));
}
@Test
@DisplayName("disabling a functional group cascades to all its endpoints")
void disableFunctionalGroupCascades() {
EndpointConfiguration config = buildDefault();
config.disableGroup("PageOps");
assertFalse(config.isGroupEnabled("PageOps"));
assertFalse(config.isEndpointEnabled("remove-pages"));
assertFalse(config.isEndpointEnabled("split-pages"));
}
@Test
@DisplayName("re-enabling a functional group re-enables its endpoints")
void enableFunctionalGroupRestores() {
EndpointConfiguration config = buildDefault();
config.disableGroup("PageOps");
assertFalse(config.isEndpointEnabled("remove-pages"));
config.enableGroup("PageOps");
assertTrue(config.isEndpointEnabled("remove-pages"));
assertTrue(config.isGroupEnabled("PageOps"));
}
@Test
@DisplayName("a functional group with one disabled endpoint is not enabled")
void functionalGroupWithDisabledEndpoint() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("remove-pages");
assertFalse(config.isGroupEnabled("PageOps"));
}
@Test
@DisplayName("disabledGroups reflects disabled groups and getDisabledGroups returns a copy")
void getDisabledGroupsReturnsCopy() {
EndpointConfiguration config = buildDefault();
config.disableGroup("PageOps");
Set<String> disabled = config.getDisabledGroups();
assertTrue(disabled.contains("PageOps"));
// mutating the returned set must not affect internal state
disabled.clear();
assertTrue(config.getDisabledGroups().contains("PageOps"));
}
@Test
@DisplayName("an unknown group with no endpoints is not enabled")
void unknownGroupNotEnabled() {
EndpointConfiguration config = buildDefault();
assertFalse(config.isGroupEnabled("NoSuchGroupXyz"));
}
}
@Nested
@DisplayName("tool group semantics")
class ToolGroupTests {
@Test
@DisplayName("a tool group is enabled until explicitly disabled")
void toolGroupEnabledUntilDisabled() {
EndpointConfiguration config = buildDefault();
assertTrue(config.isGroupEnabled("qpdf"));
config.disableGroup("qpdf");
assertFalse(config.isGroupEnabled("qpdf"));
}
@Test
@DisplayName("disabling a tool group does NOT cascade to its endpoints directly")
void toolGroupNoCascade() {
EndpointConfiguration config = buildDefault();
// repair has alternatives (qpdf, Ghostscript); disabling only qpdf keeps it enabled
config.disableGroup("qpdf");
assertTrue(config.isEndpointEnabled("repair"));
}
@Test
@DisplayName("endpoint with alternatives is disabled only when all tool groups are gone")
void allAlternativesDisabled() {
EndpointConfiguration config = buildDefault();
config.disableGroup("qpdf");
config.disableGroup("Ghostscript");
// repair's only alternatives are qpdf and Ghostscript
assertFalse(config.isEndpointEnabled("repair"));
}
@Test
@DisplayName("endpoint with a still-enabled alternative stays enabled")
void oneAlternativeRemains() {
EndpointConfiguration config = buildDefault();
// compress-pdf alternatives: qpdf, Ghostscript, Java
config.disableGroup("qpdf");
config.disableGroup("Ghostscript");
assertTrue(config.isEndpointEnabled("compress-pdf"));
config.disableGroup("Java");
assertFalse(config.isEndpointEnabled("compress-pdf"));
}
@Test
@DisplayName("single-dependency endpoint (no alternatives) disabled when its tool group is")
void singleDependencyDisabled() {
EndpointConfiguration config = buildDefault();
// pdf-to-epub depends on Calibre, no alternatives registered
assertTrue(config.isEndpointEnabled("pdf-to-epub"));
config.disableGroup("Calibre");
assertFalse(config.isEndpointEnabled("pdf-to-epub"));
}
}
@Nested
@DisplayName("addEndpointToGroup / addEndpointAlternative")
class RegistrationTests {
@Test
@DisplayName("addEndpointToGroup makes the endpoint part of the group")
void addEndpointToGroup() {
EndpointConfiguration config = buildDefault();
config.addEndpointToGroup("CustomGroup", "custom-endpoint");
Set<String> endpoints = config.getEndpointsForGroup("CustomGroup");
assertTrue(endpoints.contains("custom-endpoint"));
}
@Test
@DisplayName("disabling a custom functional group disables its added endpoint")
void customFunctionalGroupCascades() {
EndpointConfiguration config = buildDefault();
config.addEndpointToGroup("CustomGroup", "custom-endpoint");
assertTrue(config.isEndpointEnabled("custom-endpoint"));
config.disableGroup("CustomGroup");
assertFalse(config.isEndpointEnabled("custom-endpoint"));
}
@Test
@DisplayName("getEndpointsForGroup returns an empty set for unknown groups")
void unknownGroupEmptySet() {
EndpointConfiguration config = buildDefault();
Set<String> endpoints = config.getEndpointsForGroup("NoSuchGroupXyz");
assertNotNull(endpoints);
assertTrue(endpoints.isEmpty());
}
}
@Nested
@DisplayName("getEndpointAvailability / determineDisableReason")
class AvailabilityTests {
@Test
@DisplayName("an enabled endpoint has a null disable reason")
void enabledHasNullReason() {
EndpointConfiguration config = buildDefault();
EndpointAvailability availability = config.getEndpointAvailability("merge-pdfs");
assertTrue(availability.isEnabled());
assertNull(availability.getReason());
}
@Test
@DisplayName("explicit disable preserves the supplied reason")
void explicitDisableReason() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("merge-pdfs", DisableReason.DEPENDENCY);
EndpointAvailability availability = config.getEndpointAvailability("merge-pdfs");
assertFalse(availability.isEnabled());
assertEquals(DisableReason.DEPENDENCY, availability.getReason());
}
@Test
@DisplayName("default disableEndpoint reason is CONFIG")
void defaultDisableReasonIsConfig() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("merge-pdfs");
assertEquals(
DisableReason.CONFIG, config.getEndpointAvailability("merge-pdfs").getReason());
}
@Test
@DisplayName("endpoint disabled via functional group reports the group's reason")
void functionalGroupReason() {
EndpointConfiguration config = buildDefault();
config.disableGroup("PageOps", DisableReason.DEPENDENCY);
EndpointAvailability availability = config.getEndpointAvailability("crop");
assertFalse(availability.isEnabled());
// crop is disabled both via group cascade and group membership; reason is DEPENDENCY
assertEquals(DisableReason.DEPENDENCY, availability.getReason());
}
}
@Nested
@DisplayName("getAllEndpoints")
class GetAllEndpointsTests {
@Test
@DisplayName("aggregates endpoints across all groups")
void aggregatesAcrossGroups() {
EndpointConfiguration config = buildDefault();
Set<String> all = config.getAllEndpoints();
assertTrue(all.contains("merge-pdfs"));
assertTrue(all.contains("compress-pdf"));
assertTrue(all.contains("ocr-pdf"));
assertFalse(all.isEmpty());
}
@Test
@DisplayName("custom endpoints registered after init appear in getAllEndpoints")
void includesCustomEndpoints() {
EndpointConfiguration config = buildDefault();
config.addEndpointToGroup("CustomGroup", "brand-new-endpoint");
assertTrue(config.getAllEndpoints().contains("brand-new-endpoint"));
}
}
@Nested
@DisplayName("environment / constructor driven configuration")
class EnvironmentConfigTests {
@Test
@DisplayName("url-to-pdf is disabled when enableUrlToPDF is false (default)")
void urlToPdfDisabledByDefault() {
EndpointConfiguration config = buildDefault();
assertFalse(config.isEndpointEnabled("url-to-pdf"));
}
@Test
@DisplayName("url-to-pdf stays enabled when enableUrlToPDF is true")
void urlToPdfEnabledWhenFlagSet() {
applicationProperties.getSystem().setEnableUrlToPDF(true);
EndpointConfiguration config = build(false);
assertTrue(config.isEndpointEnabled("url-to-pdf"));
}
@Test
@DisplayName("endpoints.toRemove disables the listed endpoints at construction")
void endpointsToRemove() {
applicationProperties
.getEndpoints()
.setToRemove(List.of(" merge-pdfs ", "split-pages"));
EndpointConfiguration config = build(false);
// values are trimmed before disabling
assertFalse(config.isEndpointEnabled("merge-pdfs"));
assertFalse(config.isEndpointEnabled("split-pages"));
}
@Test
@DisplayName("endpoints.groupsToRemove disables the listed groups at construction")
void groupsToRemove() {
applicationProperties.getEndpoints().setGroupsToRemove(List.of(" PageOps "));
EndpointConfiguration config = build(false);
assertTrue(config.getDisabledGroups().contains("PageOps"));
assertFalse(config.isEndpointEnabled("remove-pages"));
}
@Test
@DisplayName("non-pro build disables the enterprise group")
void nonProDisablesEnterprise() {
EndpointConfiguration config = build(false);
assertTrue(config.getDisabledGroups().contains("enterprise"));
}
@Test
@DisplayName("pro build does not disable the enterprise group")
void proDoesNotDisableEnterprise() {
EndpointConfiguration config = build(true);
assertFalse(config.getDisabledGroups().contains("enterprise"));
}
}
@Nested
@DisplayName("getEndpointStatuses (Lombok getter) and logging summary")
class MiscTests {
@Test
@DisplayName("getEndpointStatuses reflects explicit disable state")
void endpointStatusesReflectDisable() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("merge-pdfs");
assertEquals(Boolean.FALSE, config.getEndpointStatuses().get("merge-pdfs"));
}
@Test
@DisplayName("logDisabledEndpointsSummary runs without throwing")
void logSummaryDoesNotThrow() {
EndpointConfiguration config = buildDefault();
config.disableGroup("PageOps");
config.disableGroup("qpdf");
// purely a smoke test of the logging branch coverage
config.logDisabledEndpointsSummary();
}
@Test
@DisplayName("logDisabledEndpointsSummary runs when nothing is disabled")
void logSummaryNothingDisabled() {
applicationProperties.getSystem().setEnableUrlToPDF(true);
EndpointConfiguration config = build(true);
config.logDisabledEndpointsSummary();
}
}
}
@@ -1,153 +0,0 @@
package stirling.software.SPDF.pdf.parser;
import static org.assertj.core.api.Assertions.assertThat;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.util.List;
import org.junit.jupiter.api.Test;
/**
* Unit tests for {@link LineAlignmentTableParser}, focused on the coincident-line merge logic and
* column-grid construction.
*/
class LineAlignmentTableParserTest {
private final LineAlignmentTableParser parser = new LineAlignmentTableParser();
// ── mergeCoincidentLines ─────────────────────────────────────────────────────────────────────
@Test
void mergeCoincidentLines_singleLine_unchanged() {
var lines = List.of(tokenized(rawLine(10f, 100f, "Revenue")));
assertThat(parser.mergeCoincidentLines(lines)).hasSize(1);
}
@Test
void mergeCoincidentLines_distinctYLines_unchanged() {
// Two lines at different y positions — must NOT be merged.
var lines =
List.of(
tokenized(rawLine(10f, 100f, "Revenue")),
tokenized(rawLine(10f, 115f, "Cost")));
assertThat(parser.mergeCoincidentLines(lines)).hasSize(2);
}
@Test
void mergeCoincidentLines_sameY_merged() {
// Simulates a financial-table row split by LineBuilder at the column gap:
// label fragment at x=72 → "Revenue"
// value fragment at x=350 → "1,234"
// Both have y=100. After merge they should form one TokenizedLine.
var label = rawLine(72f, 100f, "Revenue");
var value = rawLine(350f, 100f, "1,234");
var merged = parser.mergeCoincidentLines(List.of(tokenized(label), tokenized(value)));
assertThat(merged).hasSize(1);
// The merged line should contain tokens from both halves.
var tokens = merged.get(0).all();
assertThat(tokens.stream().map(t -> t.text()).toList())
.containsExactlyInAnyOrder("Revenue", "1,234");
}
@Test
void mergeCoincidentLines_sameY_mergedLineHasCorrectBounds() {
var label = rawLine(72f, 100f, "Revenue"); // 7 chars × 6pt = 42pt wide → right = 114
var value = rawLine(350f, 100f, "1,234"); // 5 chars × 6pt = 30pt wide → right = 380
var merged = parser.mergeCoincidentLines(List.of(tokenized(label), tokenized(value)));
var bounds = merged.get(0).line().bounds();
assertThat(bounds.x()).isEqualTo(72f);
assertThat(bounds.right()).isEqualTo(380f);
}
@Test
void mergeCoincidentLines_withinTolerance_merged() {
// Lines 1.5pt apart (within ROW_MERGE_TOLERANCE_PT = 2pt) should merge.
var a = rawLine(10f, 100.0f, "Alpha");
var b = rawLine(200f, 101.5f, "99");
var merged = parser.mergeCoincidentLines(List.of(tokenized(a), tokenized(b)));
assertThat(merged).hasSize(1);
}
@Test
void mergeCoincidentLines_beyondTolerance_notMerged() {
// Lines 3pt apart (beyond ROW_MERGE_TOLERANCE_PT = 2pt) should NOT merge.
var a = rawLine(10f, 100.0f, "Alpha");
var b = rawLine(200f, 103.0f, "99");
var merged = parser.mergeCoincidentLines(List.of(tokenized(a), tokenized(b)));
assertThat(merged).hasSize(2);
}
@Test
void mergeCoincidentLines_threeCoincident_allMerged() {
// Three fragments at the same y (e.g. wide financial table with two value columns).
var a = rawLine(72f, 100f, "Revenue");
var b = rawLine(300f, 100f, "1,234");
var c = rawLine(400f, 100f, "5,678");
var merged = parser.mergeCoincidentLines(List.of(tokenized(a), tokenized(b), tokenized(c)));
assertThat(merged).hasSize(1);
assertThat(merged.get(0).all()).hasSize(3);
}
@Test
void mergeCoincidentLines_coincidentPairFollowedByDistinctLine_twoGroups() {
var a = rawLine(72f, 100f, "Revenue");
var b = rawLine(350f, 100f, "1,234"); // same y as a → merges with a
var c = rawLine(10f, 115f, "Expenses"); // different y → stays separate
var merged = parser.mergeCoincidentLines(List.of(tokenized(a), tokenized(b), tokenized(c)));
assertThat(merged).hasSize(2);
}
@Test
void mergeCoincidentLines_numericAnchorStatus_correctAfterMerge() {
// After merging, the combined line should be an anchor (≥2 numeric tokens).
// "Revenue" alone → not an anchor. "1,234 567" alone → anchor.
// Merged → anchor with at least 2 numerics.
var label = rawLine(72f, 100f, "Revenue");
var values = rawLineMultiWord(350f, 100f, "1,234", 30f, "567", 30f);
var merged = parser.mergeCoincidentLines(List.of(tokenized(label), tokenized(values)));
assertThat(merged).hasSize(1);
assertThat(merged.get(0).isAnchor()).isTrue();
}
// ── helpers ──────────────────────────────────────────────────────────────────────────────────
/** Creates a RawLine with a single TextFragment of the given text at the given position. */
private static RawLine rawLine(float x, float y, String text) {
float width = text.length() * 6f; // ~6pt per char — rough but consistent
float height = 12f;
Bounds bounds = new Bounds(x, y, width, height);
TextFragment fragment =
new TextFragment("tf-test", text, bounds, y + height, 11f, "Helvetica", false);
return new RawLine("ln-test", List.of(fragment), bounds, 1);
}
/**
* Creates a RawLine with two TextFragments representing two words separated by a small gap.
* Used to simulate a values-only line with multiple numeric tokens.
*/
private static RawLine rawLineMultiWord(
float x, float y, String word1, float w1, String word2, float w2) {
float height = 12f;
Bounds b1 = new Bounds(x, y, w1, height);
Bounds b2 = new Bounds(x + w1 + 5f, y, w2, height);
TextFragment f1 = new TextFragment("tf-1", word1, b1, y + height, 11f, "Helvetica", false);
TextFragment f2 = new TextFragment("tf-2", word2, b2, y + height, 11f, "Helvetica", false);
Bounds lineBounds = new Bounds(x, y, x + w1 + 5f + w2 - x, height);
return new RawLine("ln-test", List.of(f1, f2), lineBounds, 1);
}
/** Tokenises a RawLine via the parser's own tokenise logic (package-private access). */
private LineAlignmentTableParser.TokenizedLine tokenized(RawLine line) {
return parser.tokenize(line);
}
}
@@ -0,0 +1,149 @@
package stirling.software.SPDF.pdf.parser;
import static org.assertj.core.api.Assertions.assertThat;
import java.util.ArrayList;
import java.util.List;
import org.junit.jupiter.api.Test;
/** Unit tests for {@link PageColumnLayout} gutter detection and column classification. */
class PageColumnLayoutTest {
private static final float PAGE_WIDTH = 612f; // Letter portrait
// ── single-column ────────────────────────────────────────────────────────────────────────────
@Test
void singleColumn_oneColumnNoGutters() {
List<float[]> lines = List.of(lineBox(72f, 396f));
PageColumnLayout layout = PageColumnLayout.fromLineBoxes(lines, PAGE_WIDTH);
assertThat(layout.columnCount()).isEqualTo(1);
assertThat(layout.gutters()).isEmpty();
}
@Test
void singleColumn_classifyAnchor_returnsZero() {
PageColumnLayout layout =
PageColumnLayout.fromLineBoxes(List.of(lineBox(72f, 396f)), PAGE_WIDTH);
assertThat(layout.columnOf(100f, 200f)).isEqualTo(0);
}
// ── two-column ───────────────────────────────────────────────────────────────────────────────
@Test
void twoColumn_detectsGutter() {
PageColumnLayout layout =
PageColumnLayout.fromLineBoxes(buildTwoColumnLines(3), PAGE_WIDTH);
assertThat(layout.columnCount()).isEqualTo(2);
assertThat(layout.gutters()).hasSize(1);
float[] gutter = layout.gutters().get(0);
// Gutter is centered on pageWidth/2 with PageColumnLayout.MIDPOINT_SLACK_PT slack each
// side.
float pageMid = PAGE_WIDTH / 2f;
assertThat(gutter[0]).isBetween(pageMid - 40f, pageMid - 20f);
assertThat(gutter[1]).isBetween(pageMid + 20f, pageMid + 40f);
}
@Test
void twoColumn_classifyLeftAndRightAnchors() {
PageColumnLayout layout =
PageColumnLayout.fromLineBoxes(buildTwoColumnLines(3), PAGE_WIDTH);
assertThat(layout.columnOf(100f, 200f)).isEqualTo(0);
assertThat(layout.columnOf(380f, 460f)).isEqualTo(1);
}
@Test
void twoColumn_columnsCrossing_leftLineOnlyHitsLeft() {
PageColumnLayout layout =
PageColumnLayout.fromLineBoxes(buildTwoColumnLines(3), PAGE_WIDTH);
assertThat(layout.columnsCrossing(72f, 280f)).containsExactly(0);
assertThat(layout.columnsCrossing(320f, 540f)).containsExactly(1);
}
@Test
void twoColumn_spanningLine_returnsBothColumns() {
List<float[]> lines = new ArrayList<>(buildTwoColumnLines(3));
// Full-width header that crosses pageWidth/2.
lines.add(lineBox(72f, 396f));
PageColumnLayout layout = PageColumnLayout.fromLineBoxes(lines, PAGE_WIDTH);
assertThat(layout.columnsCrossing(72f, 540f)).containsExactly(0, 1);
assertThat(layout.columnsCrossing(72f, 280f)).containsExactly(0);
}
private static List<float[]> buildTwoColumnLines(int rowsPerColumn) {
List<float[]> lines = new ArrayList<>();
for (int i = 0; i < rowsPerColumn; i++) {
lines.add(lineBox(72f, 136f)); // left column body (72..208)
lines.add(lineBox(320f, 220f)); // right column body (320..540)
}
return lines;
}
// ── three-column ─────────────────────────────────────────────────────────────────────────────
@Test
void threeColumn_collapsesToLeftRightSplit() {
// The midpoint-based detector splits the page at pageWidth/2 and treats anything else as
// single-column or spanning. A genuine 3-column layout collapses to 2 columns; the middle
// column's content ends up classified by midpoint as left or right of pageMid.
List<float[]> lines = new ArrayList<>();
for (int i = 0; i < 6; i++) {
lines.add(lineBox(72f, 150f)); // 72..222
lines.add(lineBox(252f, 150f)); // 252..402
lines.add(lineBox(432f, 150f)); // 432..582
}
PageColumnLayout layout = PageColumnLayout.fromLineBoxes(lines, PAGE_WIDTH);
assertThat(layout.columnCount()).isEqualTo(2);
assertThat(layout.gutters()).hasSize(1);
}
// ── empty page ───────────────────────────────────────────────────────────────────────────────
@Test
void emptyPage_singleColumnFallback() {
PageColumnLayout layout = PageColumnLayout.fromLineBoxes(List.of(), PAGE_WIDTH);
assertThat(layout.columnCount()).isEqualTo(1);
assertThat(layout.gutters()).isEmpty();
}
@Test
void onlyShortFragments_singleColumnFallback() {
// Page numbers / decorations — too narrow to vote either side.
List<float[]> lines = List.of(lineBox(300f, 6f));
PageColumnLayout layout = PageColumnLayout.fromLineBoxes(lines, PAGE_WIDTH);
assertThat(layout.columnCount()).isEqualTo(1);
}
// ── narrow gap should not be confused for a gutter ───────────────────────────────────────────
@Test
void narrowInternalGap_doesNotProduceGutter() {
// Both halves sit left of the page midpoint, so no line votes for a right column and
// detection falls back to single-column.
List<float[]> lines = new ArrayList<>();
lines.add(lineBox(72f, 100f));
lines.add(lineBox(180f, 100f));
for (int i = 0; i < 5; i++) {
lines.add(lineBox(72f, 208f));
}
PageColumnLayout layout = PageColumnLayout.fromLineBoxes(lines, PAGE_WIDTH);
assertThat(layout.columnCount()).isEqualTo(1);
}
// ── helpers ──────────────────────────────────────────────────────────────────────────────────
/** Builds a line bounding box {@code [x1, 0, x1+width, 0]}; Y is unused by detection. */
private static float[] lineBox(float x1, float width) {
return new float[] {x1, 0f, x1 + width, 0f};
}
}
@@ -0,0 +1,345 @@
package stirling.software.SPDF.pdf.parser;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertSame;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static stirling.software.SPDF.pdf.parser.PdfModels.RawPage;
import static stirling.software.SPDF.pdf.parser.PdfModels.TableCell;
import static stirling.software.SPDF.pdf.parser.PdfModels.TableFragment;
import static stirling.software.SPDF.pdf.parser.PdfModels.TableRow;
import java.awt.Color;
import java.io.ByteArrayOutputStream;
import java.util.List;
import org.apache.pdfbox.Loader;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDPageContentStream;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
import org.apache.pdfbox.pdmodel.font.PDType1Font;
import org.apache.pdfbox.pdmodel.font.Standard14Fonts;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
/**
* Unit tests for {@link TabulaTableParser}. Tables are built in-memory with PDFBox so the tests are
* deterministic and need no fixtures, network, or external processes.
*/
class TabulaTableParserGapTest {
private final TabulaTableParser parser = new TabulaTableParser();
// ── error / empty branches ───────────────────────────────────────────────
@Nested
@DisplayName("Empty and error branches")
class EmptyAndErrorBranches {
@Test
@DisplayName("page number 0 is out of Tabula's 1-based range -> empty list, no throw")
void pageNumberZeroReturnsEmpty() throws Exception {
byte[] pdf = pdfWithText(new String[] {"hello"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> result = parser.parse(doc, 0);
assertNotNull(result);
assertTrue(result.isEmpty());
}
}
@Test
@DisplayName("page number beyond the document -> empty list, exception swallowed")
void pageNumberOutOfRangeReturnsEmpty() throws Exception {
byte[] pdf = pdfWithText(new String[] {"hello"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> result = parser.parse(doc, 99);
assertNotNull(result);
assertTrue(result.isEmpty());
}
}
@Test
@DisplayName("negative page number -> empty list")
void negativePageNumberReturnsEmpty() throws Exception {
byte[] pdf = pdfWithText(new String[] {"hello"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
assertTrue(parser.parse(doc, -5).isEmpty());
}
}
@Test
@DisplayName("lattice mode on a page with no ruled lines -> no tables")
void latticeWithNoRulingsReturnsEmpty() throws Exception {
byte[] pdf = pdfWithText(new String[] {"just some prose", "no table here"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> result = parser.parse(doc, new RawPage(1, 0f, 0f, List.of()));
assertNotNull(result);
assertTrue(
result.isEmpty(), "borderless text must not be detected in lattice mode");
}
}
@Test
@DisplayName("blank page in lattice mode -> empty list")
void blankPageLatticeReturnsEmpty() throws Exception {
byte[] pdf = blankPdf();
try (PDDocument doc = Loader.loadPDF(pdf)) {
assertTrue(parser.parse(doc, new RawPage(1, 0f, 0f, List.of())).isEmpty());
}
}
}
// ── stream mode (BasicExtractionAlgorithm) ───────────────────────────────
@Nested
@DisplayName("Stream mode")
class StreamMode {
@Test
@DisplayName("page with text yields at least one well-formed fragment")
void streamOnTextProducesFragment() throws Exception {
byte[] pdf =
pdfWithText(new String[] {"Name Age City", "Alice 30 Paris", "Bob 25 Rome"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> fragments =
parser.parseStream(doc, new RawPage(1, 0f, 0f, List.of()));
assertNotNull(fragments);
assertFalse(fragments.isEmpty(), "stream mode always builds a table from text");
assertFragmentWellFormed(fragments.get(0), 1, 0);
}
}
@Test
@DisplayName("fragment ids encode page and index")
void streamFragmentIdFormat() throws Exception {
byte[] pdf = pdfWithText(new String[] {"col1 col2", "a b"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> fragments =
parser.parseStream(doc, new RawPage(1, 0f, 0f, List.of()));
assertFalse(fragments.isEmpty());
assertEquals("tbl-p1-0", fragments.get(0).tableId());
assertEquals(1, fragments.get(0).pageNumber());
}
}
@Test
@DisplayName("rawRows and the parsed rows stay in lockstep")
void streamRowsMatchRawRows() throws Exception {
byte[] pdf = pdfWithText(new String[] {"x y", "1 2", "3 4"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> fragments =
parser.parseStream(doc, new RawPage(1, 0f, 0f, List.of()));
assertFalse(fragments.isEmpty());
TableFragment f = fragments.get(0);
assertEquals(f.rawRows().size(), f.rows().size());
}
}
}
// ── lattice mode with a real bordered grid ───────────────────────────────
@Nested
@DisplayName("Lattice mode")
class LatticeMode {
@Test
@DisplayName("bordered grid is detected and produces well-formed fragments")
void latticeDetectsBorderedTable() throws Exception {
byte[] pdf = pdfWithGrid();
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> fragments =
parser.parse(doc, new RawPage(1, 0f, 0f, List.of()));
assertNotNull(fragments);
assertFalse(
fragments.isEmpty(), "a clean ruled grid must be detected in lattice mode");
TableFragment f = fragments.get(0);
assertFragmentWellFormed(f, 1, 0);
assertTrue(f.columnCount() >= 1, "a detected grid must have at least one column");
assertFalse(f.rawRows().isEmpty(), "a detected grid must have rows");
}
}
@Test
@DisplayName("convenience overload with page number routes to lattice mode")
void parseByPageNumberDetectsGrid() throws Exception {
byte[] pdf = pdfWithGrid();
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> fragments = parser.parse(doc, 1);
assertNotNull(fragments);
assertFalse(fragments.isEmpty());
assertEquals(1, fragments.get(0).pageNumber());
}
}
@Test
@DisplayName("cell text is normalised (trimmed, newlines collapsed)")
void latticeCellTextIsNormalised() throws Exception {
byte[] pdf = pdfWithGrid();
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> fragments =
parser.parse(doc, new RawPage(1, 0f, 0f, List.of()));
assertFalse(fragments.isEmpty());
for (List<String> row : fragments.get(0).rawRows()) {
for (String cell : row) {
assertNotNull(cell);
assertFalse(cell.contains("\n"), "newlines must be collapsed");
assertFalse(cell.contains("\r"), "carriage returns must be collapsed");
assertEquals(cell.trim(), cell, "cell text must be trimmed");
}
}
}
}
}
// ── contract invariants ──────────────────────────────────────────────────
@Nested
@DisplayName("Contract invariants")
class ContractInvariants {
@Test
@DisplayName("parse never returns null")
void parseNeverReturnsNull() throws Exception {
byte[] pdf = pdfWithText(new String[] {"abc"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
assertNotNull(parser.parse(doc, new RawPage(1, 0f, 0f, List.of())));
assertNotNull(parser.parse(doc, 1));
assertNotNull(parser.parseStream(doc, new RawPage(1, 0f, 0f, List.of())));
}
}
@Test
@DisplayName("the document is not closed by the parser")
void documentRemainsOpenAfterParse() throws Exception {
byte[] pdf = pdfWithText(new String[] {"keep me open"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
parser.parse(doc, new RawPage(1, 0f, 0f, List.of()));
parser.parseStream(doc, new RawPage(1, 0f, 0f, List.of()));
// ObjectExtractor.close() would close the underlying COSDocument; the parser must
// not.
assertFalse(
doc.getDocument().isClosed(),
"parser must not close the caller's document");
assertEquals(1, doc.getNumberOfPages());
}
}
}
// ── helpers ──────────────────────────────────────────────────────────────
/** Asserts every field of a fragment satisfies the documented contract. */
private static void assertFragmentWellFormed(
TableFragment f, int expectedPage, int expectedIndex) {
assertNotNull(f);
assertEquals(expectedPage, f.pageNumber());
assertEquals("tbl-p" + expectedPage + "-" + expectedIndex, f.tableId());
assertNotNull(f.bounds());
assertNotNull(f.headers());
assertTrue(f.headers().isEmpty(), "headers are deferred to v2 and must be empty");
assertNotNull(f.rows());
assertNotNull(f.rawRows());
assertNotNull(f.warnings());
assertSame(null, f.continuedFromPage(), "continuedFromPage is deferred to v2");
assertTrue(f.columnCount() >= 0);
assertTrue(f.confidence() >= 0f && f.confidence() <= 1f, "confidence must be within [0,1]");
assertEquals(f.rawRows().size(), f.rows().size());
for (TableRow row : f.rows()) {
assertNotNull(row.cells());
for (TableCell cell : row.cells()) {
assertNotNull(cell.text());
assertNotNull(cell.bounds());
assertEquals(1, cell.colSpan(), "colSpan is always 1 in v1");
assertEquals(1, cell.rowSpan(), "rowSpan is always 1 in v1");
}
}
}
private static byte[] pdfWithText(String[] lines) throws Exception {
try (PDDocument doc = new PDDocument()) {
PDPage page = new PDPage(PDRectangle.A4);
doc.addPage(page);
try (PDPageContentStream cs = new PDPageContentStream(doc, page)) {
cs.setFont(new PDType1Font(Standard14Fonts.FontName.HELVETICA), 12);
cs.setNonStrokingColor(Color.BLACK);
float y = 720f;
for (String line : lines) {
cs.beginText();
cs.newLineAtOffset(72f, y);
cs.showText(line);
cs.endText();
y -= 20f;
}
}
return save(doc);
}
}
private static byte[] blankPdf() throws Exception {
try (PDDocument doc = new PDDocument()) {
doc.addPage(new PDPage(PDRectangle.A4));
return save(doc);
}
}
/**
* Builds a small 3-row x 3-column ruled grid with text in each cell. The ruled lines make the
* table detectable by lattice mode.
*/
private static byte[] pdfWithGrid() throws Exception {
try (PDDocument doc = new PDDocument()) {
PDPage page = new PDPage(PDRectangle.A4);
doc.addPage(page);
float left = 100f;
float right = 400f;
float top = 700f;
float bottom = 550f;
int cols = 3;
int rows = 3;
float colStep = (right - left) / cols;
float rowStep = (top - bottom) / rows;
try (PDPageContentStream cs = new PDPageContentStream(doc, page)) {
cs.setStrokingColor(Color.BLACK);
cs.setLineWidth(1f);
// vertical lines
for (int c = 0; c <= cols; c++) {
float x = left + c * colStep;
cs.moveTo(x, bottom);
cs.lineTo(x, top);
}
// horizontal lines
for (int r = 0; r <= rows; r++) {
float yLine = bottom + r * rowStep;
cs.moveTo(left, yLine);
cs.lineTo(right, yLine);
}
cs.stroke();
// cell text
cs.setFont(new PDType1Font(Standard14Fonts.FontName.HELVETICA), 10);
cs.setNonStrokingColor(Color.BLACK);
for (int r = 0; r < rows; r++) {
for (int c = 0; c < cols; c++) {
cs.beginText();
cs.newLineAtOffset(left + c * colStep + 5f, top - (r + 1) * rowStep + 6f);
cs.showText("R" + r + "C" + c);
cs.endText();
}
}
}
return save(doc);
}
}
private static byte[] save(PDDocument doc) throws Exception {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
doc.save(baos);
return baos.toByteArray();
}
}
@@ -24,7 +24,9 @@ class InProcessDistributedLockTest {
}
@Test
void reentryFromSameThreadFails() {
void reentryFromSameThreadFails_parityWithValkey() {
// The Valkey impl refuses reentry (SET NX semantics); the in-process impl must match,
// otherwise code working in single-instance silently breaks in cluster mode.
DistributedLock lock = new InProcessDistributedLock();
DistributedLock.LockHandle h1 = lock.tryAcquire("k", Duration.ofSeconds(30)).orElseThrow();
Optional<DistributedLock.LockHandle> reentry = lock.tryAcquire("k", Duration.ofSeconds(30));
@@ -3,11 +3,13 @@ package stirling.software.common.cluster.inprocess;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import org.junit.jupiter.api.Test;
@@ -39,4 +41,46 @@ class LocalDiskFileStoreTest {
assertThrows(IllegalArgumentException.class, () -> store.resolve("a/b"));
assertThrows(IllegalArgumentException.class, () -> store.resolve("a\\b"));
}
@Test
void ownerSidecarCannotBeReadAsFileId(@TempDir Path dir) throws IOException {
LocalDiskFileStore store = new LocalDiskFileStore(dir.toString());
FileStore.Stored stored =
store.store(new ByteArrayInputStream("hi".getBytes()), "f.bin", "alice");
String sidecarId = stored.fileId() + ".owner";
assertThrows(IllegalArgumentException.class, () -> store.resolve(sidecarId));
assertThrows(IllegalArgumentException.class, () -> store.retrieveBytes(sidecarId));
}
@Test
void ownerIsPersistedAndReturnedByGetOwner(@TempDir Path dir) throws IOException {
LocalDiskFileStore store = new LocalDiskFileStore(dir.toString());
FileStore.Stored stored =
store.store(new ByteArrayInputStream("hi".getBytes()), "f.bin", "alice");
assertEquals("alice", store.getOwner(stored.fileId()));
}
@Test
void getOwnerReturnsNullWhenNoOwnerWasRecorded(@TempDir Path dir) throws IOException {
LocalDiskFileStore store = new LocalDiskFileStore(dir.toString());
FileStore.Stored stored =
store.store(new ByteArrayInputStream("hi".getBytes()), "f.bin", null);
assertNull(store.getOwner(stored.fileId()));
}
@Test
void getOwnerReturnsNullForUnknownFileId(@TempDir Path dir) throws IOException {
LocalDiskFileStore store = new LocalDiskFileStore(dir.toString());
assertNull(store.getOwner("00000000-0000-0000-0000-000000000000"));
}
@Test
void deleteRemovesOwnerSidecar(@TempDir Path dir) throws IOException {
LocalDiskFileStore store = new LocalDiskFileStore(dir.toString());
FileStore.Stored stored =
store.store(new ByteArrayInputStream("hi".getBytes()), "f.bin", "alice");
assertTrue(store.delete(stored.fileId()));
assertFalse(Files.exists(dir.resolve(stored.fileId() + ".owner")));
assertNull(store.getOwner(stored.fileId()));
}
}
@@ -0,0 +1,95 @@
package stirling.software.common.configuration;
import static org.junit.jupiter.api.Assertions.*;
import org.junit.jupiter.api.Test;
import org.snakeyaml.engine.v2.api.LoadSettings;
import stirling.software.common.util.YamlHelper;
class ConfigInitializerTest {
private static final LoadSettings LOAD_SETTINGS =
LoadSettings.builder()
.setUseMarks(true)
.setMaxAliasesForCollections(Integer.MAX_VALUE)
.setAllowRecursiveKeys(true)
.setParseComments(true)
.build();
// Mirrors the proFeatures block of settings.yml.template after the camelCase rename.
private static final String CAMEL_CASE_TEMPLATE =
"""
premium:
proFeatures:
ssoAutoLogin: false
customMetadata:
autoUpdateMetadata: false
author: username
creator: Stirling-PDF
producer: Stirling-PDF
""";
@Test
void migrateProFeaturesKeyCasing_carriesForwardLegacyPascalCaseValues() {
// An existing install whose settings.yml still uses the old PascalCase keys.
String legacy =
"""
premium:
proFeatures:
SSOAutoLogin: true
CustomMetadata:
autoUpdateMetadata: true
author: alice
creator: bob
producer: carol
""";
YamlHelper template = new YamlHelper(LOAD_SETTINGS, CAMEL_CASE_TEMPLATE);
YamlHelper existing = new YamlHelper(LOAD_SETTINGS, legacy);
new ConfigInitializer().migrateProFeaturesKeyCasing(existing, template);
assertEquals(
"true", template.getValueByExactKeyPath("premium", "proFeatures", "ssoAutoLogin"));
assertEquals(
"true",
template.getValueByExactKeyPath(
"premium", "proFeatures", "customMetadata", "autoUpdateMetadata"));
assertEquals(
"alice",
template.getValueByExactKeyPath(
"premium", "proFeatures", "customMetadata", "author"));
assertEquals(
"bob",
template.getValueByExactKeyPath(
"premium", "proFeatures", "customMetadata", "creator"));
assertEquals(
"carol",
template.getValueByExactKeyPath(
"premium", "proFeatures", "customMetadata", "producer"));
}
@Test
void migrateProFeaturesKeyCasing_withoutLegacyKeys_keepsTemplateDefaults() {
// No PascalCase keys present -> this migration step must be a no-op.
String alreadyCamel =
"""
premium:
proFeatures:
ssoAutoLogin: true
customMetadata:
author: dave
""";
YamlHelper template = new YamlHelper(LOAD_SETTINGS, CAMEL_CASE_TEMPLATE);
YamlHelper existing = new YamlHelper(LOAD_SETTINGS, alreadyCamel);
new ConfigInitializer().migrateProFeaturesKeyCasing(existing, template);
assertEquals(
"false", template.getValueByExactKeyPath("premium", "proFeatures", "ssoAutoLogin"));
assertEquals(
"username",
template.getValueByExactKeyPath(
"premium", "proFeatures", "customMetadata", "author"));
}
}
@@ -0,0 +1,520 @@
package stirling.software.common.configuration;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertSame;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.ApplicationProperties.CustomPaths.Operations;
import stirling.software.common.model.ApplicationProperties.CustomPaths.Pipeline;
import stirling.software.common.model.ApplicationProperties.ProcessExecutor.UnoServerEndpoint;
/**
* Unit tests for {@link RuntimePathConfig}. All of the resolution logic lives in the constructor,
* so each test builds a real {@link ApplicationProperties} (a plain @Data POJO with sensible
* defaults), constructs the config, and asserts on the exposed getters.
*/
class RuntimePathConfigTest {
/** The base path the production code derives from {@link InstallationPathConfig#getPath()}. */
private static final String BASE_PATH = InstallationPathConfig.getPath();
private static ApplicationProperties newProperties() {
return new ApplicationProperties();
}
private static RuntimePathConfig build(ApplicationProperties properties) {
return new RuntimePathConfig(properties);
}
@Nested
@DisplayName("Pipeline directory resolution")
class PipelinePaths {
@Test
@DisplayName("Defaults to <basePath>/pipeline and derived sub-folders")
void defaultPipelinePaths() {
RuntimePathConfig config = build(newProperties());
String expectedPipeline = Path.of(BASE_PATH, "pipeline").toString();
assertEquals(expectedPipeline, config.getPipelinePath());
// Watched folders are resolved to an absolute, normalized path by the production code.
assertEquals(
Path.of(expectedPipeline, "watchedFolders")
.toAbsolutePath()
.normalize()
.toString(),
config.getPipelineWatchedFoldersPath());
assertEquals(
Path.of(expectedPipeline, "finishedFolders").toString(),
config.getPipelineFinishedFoldersPath());
assertEquals(
Path.of(expectedPipeline, "defaultWebUIConfigs").toString(),
config.getPipelineDefaultWebUiConfigs());
}
@Test
@DisplayName("Custom pipelineDir overrides the default pipeline path")
void customPipelineDir() {
ApplicationProperties properties = newProperties();
Pipeline pipeline = properties.getSystem().getCustomPaths().getPipeline();
pipeline.setPipelineDir("/custom/pipeline");
RuntimePathConfig config = build(properties);
assertEquals("/custom/pipeline", config.getPipelinePath());
// Sub-folders are derived from the (already-resolved) custom pipeline path.
assertEquals(
Path.of("/custom/pipeline", "finishedFolders").toString(),
config.getPipelineFinishedFoldersPath());
assertEquals(
Path.of("/custom/pipeline", "defaultWebUIConfigs").toString(),
config.getPipelineDefaultWebUiConfigs());
}
@Test
@DisplayName("Blank pipelineDir falls back to the default")
void blankPipelineDirFallsBackToDefault() {
ApplicationProperties properties = newProperties();
properties.getSystem().getCustomPaths().getPipeline().setPipelineDir(" ");
RuntimePathConfig config = build(properties);
assertEquals(Path.of(BASE_PATH, "pipeline").toString(), config.getPipelinePath());
}
@Test
@DisplayName("Custom finished and webUI configs dirs override defaults")
void customFinishedAndWebUiDirs() {
ApplicationProperties properties = newProperties();
Pipeline pipeline = properties.getSystem().getCustomPaths().getPipeline();
pipeline.setFinishedFoldersDir("/custom/finished");
pipeline.setWebUIConfigsDir("/custom/webui");
RuntimePathConfig config = build(properties);
assertEquals("/custom/finished", config.getPipelineFinishedFoldersPath());
assertEquals("/custom/webui", config.getPipelineDefaultWebUiConfigs());
}
}
@Nested
@DisplayName("Watched folder resolution")
class WatchedFolders {
@Test
@DisplayName("Default watched folder is <pipeline>/watchedFolders and list has one entry")
void defaultWatchedFolder() {
RuntimePathConfig config = build(newProperties());
// Watched folders are resolved to an absolute, normalized path by the production code.
String expected =
Path.of(Path.of(BASE_PATH, "pipeline").toString(), "watchedFolders")
.toAbsolutePath()
.normalize()
.toString();
assertEquals(expected, config.getPipelineWatchedFoldersPath());
assertEquals(1, config.getPipelineWatchedFoldersPaths().size());
assertEquals(expected, config.getPipelineWatchedFoldersPaths().get(0));
}
@Test
@DisplayName("Legacy single watchedFoldersDir is used when no list is provided")
void legacyWatchedFolder() {
ApplicationProperties properties = newProperties();
properties
.getSystem()
.getCustomPaths()
.getPipeline()
.setWatchedFoldersDir("relativeWatched");
RuntimePathConfig config = build(properties);
// Legacy paths are normalized to absolute.
String expected = Path.of("relativeWatched").toAbsolutePath().normalize().toString();
assertEquals(1, config.getPipelineWatchedFoldersPaths().size());
assertEquals(expected, config.getPipelineWatchedFoldersPath());
}
@Test
@DisplayName("New list config takes precedence over the legacy single dir")
void listTakesPrecedenceOverLegacy() {
ApplicationProperties properties = newProperties();
Pipeline pipeline = properties.getSystem().getCustomPaths().getPipeline();
pipeline.setWatchedFoldersDir("legacyDir");
pipeline.setWatchedFoldersDirs(new ArrayList<>(Arrays.asList("listDirA", "listDirB")));
RuntimePathConfig config = build(properties);
List<String> paths = config.getPipelineWatchedFoldersPaths();
assertEquals(2, paths.size());
assertEquals(Path.of("listDirA").toAbsolutePath().normalize().toString(), paths.get(0));
assertEquals(Path.of("listDirB").toAbsolutePath().normalize().toString(), paths.get(1));
// The legacy value must NOT appear when the list is present.
assertFalse(
paths.contains(Path.of("legacyDir").toAbsolutePath().normalize().toString()));
}
@Test
@DisplayName("Duplicate paths in the list are de-duplicated after normalization")
void duplicatePathsAreDeduplicated() {
ApplicationProperties properties = newProperties();
properties
.getSystem()
.getCustomPaths()
.getPipeline()
.setWatchedFoldersDirs(
new ArrayList<>(Arrays.asList("dupDir", "dupDir", "otherDir")));
RuntimePathConfig config = build(properties);
List<String> paths = config.getPipelineWatchedFoldersPaths();
assertEquals(2, paths.size());
assertEquals(Path.of("dupDir").toAbsolutePath().normalize().toString(), paths.get(0));
assertEquals(Path.of("otherDir").toAbsolutePath().normalize().toString(), paths.get(1));
}
@Test
@DisplayName("Blank and whitespace-only list entries are sanitized out")
void blankListEntriesAreFiltered() {
ApplicationProperties properties = newProperties();
properties
.getSystem()
.getCustomPaths()
.getPipeline()
.setWatchedFoldersDirs(
new ArrayList<>(Arrays.asList(" ", "", "validDir", " ")));
RuntimePathConfig config = build(properties);
List<String> paths = config.getPipelineWatchedFoldersPaths();
assertEquals(1, paths.size());
assertEquals(Path.of("validDir").toAbsolutePath().normalize().toString(), paths.get(0));
}
@Test
@DisplayName("List entries are trimmed before resolution")
void listEntriesAreTrimmed() {
ApplicationProperties properties = newProperties();
properties
.getSystem()
.getCustomPaths()
.getPipeline()
.setWatchedFoldersDirs(new ArrayList<>(Arrays.asList(" spacedDir ")));
RuntimePathConfig config = build(properties);
assertEquals(
Path.of("spacedDir").toAbsolutePath().normalize().toString(),
config.getPipelineWatchedFoldersPath());
}
@Test
@DisplayName("An all-blank list falls back to the legacy dir, then default")
void allBlankListFallsBackToDefault() {
ApplicationProperties properties = newProperties();
properties
.getSystem()
.getCustomPaths()
.getPipeline()
.setWatchedFoldersDirs(new ArrayList<>(Arrays.asList("", " ")));
RuntimePathConfig config = build(properties);
// sanitizePathList strips everything -> empty -> falls through to default watched
// folder.
// The default is also resolved to an absolute, normalized path by the production code.
String expectedDefault =
Path.of(Path.of(BASE_PATH, "pipeline").toString(), "watchedFolders")
.toAbsolutePath()
.normalize()
.toString();
assertEquals(1, config.getPipelineWatchedFoldersPaths().size());
assertEquals(expectedDefault, config.getPipelineWatchedFoldersPath());
}
@Test
@DisplayName("First watched folder path is always exposed via the singular getter")
void singularGetterReturnsFirstEntry() {
ApplicationProperties properties = newProperties();
properties
.getSystem()
.getCustomPaths()
.getPipeline()
.setWatchedFoldersDirs(new ArrayList<>(Arrays.asList("firstDir", "secondDir")));
RuntimePathConfig config = build(properties);
assertEquals(
config.getPipelineWatchedFoldersPaths().get(0),
config.getPipelineWatchedFoldersPath());
assertEquals(
Path.of("firstDir").toAbsolutePath().normalize().toString(),
config.getPipelineWatchedFoldersPath());
}
}
@Nested
@DisplayName("Operation tool path resolution")
class OperationPaths {
@Test
@DisplayName("Defaults to bare command names when not running in Docker")
void defaultOperationPaths() {
// The test host has no /.dockerenv, so the non-docker defaults apply.
RuntimePathConfig config = build(newProperties());
assertEquals("weasyprint", config.getWeasyPrintPath());
assertEquals("unoconvert", config.getUnoConvertPath());
assertEquals("ebook-convert", config.getCalibrePath());
assertEquals("ocrmypdf", config.getOcrMyPdfPath());
assertEquals("soffice", config.getSOfficePath());
}
@Test
@DisplayName("Custom operation paths override the defaults")
void customOperationPaths() {
ApplicationProperties properties = newProperties();
Operations operations = properties.getSystem().getCustomPaths().getOperations();
operations.setWeasyprint("/opt/custom/weasyprint");
operations.setUnoconvert("/opt/custom/unoconvert");
operations.setCalibre("/opt/custom/ebook-convert");
operations.setOcrmypdf("/opt/custom/ocrmypdf");
operations.setSoffice("/opt/custom/soffice");
RuntimePathConfig config = build(properties);
assertEquals("/opt/custom/weasyprint", config.getWeasyPrintPath());
assertEquals("/opt/custom/unoconvert", config.getUnoConvertPath());
assertEquals("/opt/custom/ebook-convert", config.getCalibrePath());
assertEquals("/opt/custom/ocrmypdf", config.getOcrMyPdfPath());
assertEquals("/opt/custom/soffice", config.getSOfficePath());
}
@Test
@DisplayName("Blank custom operation path falls back to the default")
void blankOperationPathFallsBack() {
ApplicationProperties properties = newProperties();
properties.getSystem().getCustomPaths().getOperations().setWeasyprint(" ");
RuntimePathConfig config = build(properties);
assertEquals("weasyprint", config.getWeasyPrintPath());
}
@Test
@DisplayName("A single custom path leaves the other operation paths at defaults")
void partialOperationOverride() {
ApplicationProperties properties = newProperties();
properties
.getSystem()
.getCustomPaths()
.getOperations()
.setSoffice("/usr/local/soffice");
RuntimePathConfig config = build(properties);
assertEquals("/usr/local/soffice", config.getSOfficePath());
assertEquals("weasyprint", config.getWeasyPrintPath());
assertEquals("unoconvert", config.getUnoConvertPath());
}
}
@Nested
@DisplayName("Tesseract data path resolution")
class TessdataPath {
@Test
@DisplayName("Explicit tessdataDir config wins over env var and default")
void configuredTessdataDirWins() {
ApplicationProperties properties = newProperties();
properties.getSystem().setTessdataDir("/my/tessdata");
RuntimePathConfig config = build(properties);
// Config setting has the highest priority regardless of TESSDATA_PREFIX env state.
assertEquals("/my/tessdata", config.getTessDataPath());
}
@Test
@DisplayName("tessDataPath is never null even with no config")
void tessDataPathNeverNull() {
RuntimePathConfig config = build(newProperties());
// With no config setting, the value comes from TESSDATA_PREFIX or the hard default,
// either of which is non-null.
assertNotNull(config.getTessDataPath());
assertFalse(config.getTessDataPath().isEmpty());
}
}
@Nested
@DisplayName("UNO server endpoint resolution")
class UnoServerEndpoints {
@Test
@DisplayName("Auto mode builds one endpoint when session limit is unset (defaults to 1)")
void autoSingleEndpointByDefault() {
// Default ApplicationProperties: autoUnoServer = true, libreOfficeSessionLimit = 0 ->
// 1.
RuntimePathConfig config = build(newProperties());
List<UnoServerEndpoint> endpoints = config.getUnoServerEndpoints();
assertEquals(1, endpoints.size());
assertEquals("127.0.0.1", endpoints.get(0).getHost());
assertEquals(2003, endpoints.get(0).getPort());
}
@Test
@DisplayName("Auto mode builds N endpoints on consecutive even ports")
void autoMultipleEndpoints() {
ApplicationProperties properties = newProperties();
properties.getProcessExecutor().getSessionLimit().setLibreOfficeSessionLimit(3);
RuntimePathConfig config = build(properties);
List<UnoServerEndpoint> endpoints = config.getUnoServerEndpoints();
assertEquals(3, endpoints.size());
assertEquals(2003, endpoints.get(0).getPort());
assertEquals(2005, endpoints.get(1).getPort());
assertEquals(2007, endpoints.get(2).getPort());
for (UnoServerEndpoint endpoint : endpoints) {
assertEquals("127.0.0.1", endpoint.getHost());
}
}
@Test
@DisplayName("Manual mode returns the configured (valid) endpoints")
void manualEndpointsAreUsed() {
ApplicationProperties properties = newProperties();
ApplicationProperties.ProcessExecutor processExecutor = properties.getProcessExecutor();
processExecutor.setAutoUnoServer(false);
UnoServerEndpoint endpoint = new UnoServerEndpoint();
endpoint.setHost("10.0.0.5");
endpoint.setPort(4000);
processExecutor.setUnoServerEndpoints(new ArrayList<>(Arrays.asList(endpoint)));
RuntimePathConfig config = build(properties);
List<UnoServerEndpoint> endpoints = config.getUnoServerEndpoints();
assertEquals(1, endpoints.size());
assertEquals("10.0.0.5", endpoints.get(0).getHost());
assertEquals(4000, endpoints.get(0).getPort());
}
@Test
@DisplayName("Manual mode filters out endpoints with blank host or non-positive port")
void manualEndpointsAreSanitized() {
ApplicationProperties properties = newProperties();
ApplicationProperties.ProcessExecutor processExecutor = properties.getProcessExecutor();
processExecutor.setAutoUnoServer(false);
UnoServerEndpoint valid = new UnoServerEndpoint();
valid.setHost("192.168.1.10");
valid.setPort(5000);
UnoServerEndpoint blankHost = new UnoServerEndpoint();
blankHost.setHost(" ");
blankHost.setPort(5001);
UnoServerEndpoint badPort = new UnoServerEndpoint();
badPort.setHost("192.168.1.11");
badPort.setPort(0);
processExecutor.setUnoServerEndpoints(
new ArrayList<>(Arrays.asList(valid, blankHost, badPort)));
RuntimePathConfig config = build(properties);
List<UnoServerEndpoint> endpoints = config.getUnoServerEndpoints();
assertEquals(1, endpoints.size());
assertEquals("192.168.1.10", endpoints.get(0).getHost());
assertEquals(5000, endpoints.get(0).getPort());
}
@Test
@DisplayName("Manual mode with no usable endpoints falls back to a single default endpoint")
void manualModeNoEndpointsFallsBackToDefault() {
ApplicationProperties properties = newProperties();
ApplicationProperties.ProcessExecutor processExecutor = properties.getProcessExecutor();
processExecutor.setAutoUnoServer(false);
processExecutor.setUnoServerEndpoints(new ArrayList<>());
RuntimePathConfig config = build(properties);
List<UnoServerEndpoint> endpoints = config.getUnoServerEndpoints();
assertEquals(1, endpoints.size());
assertEquals("127.0.0.1", endpoints.get(0).getHost());
assertEquals(2003, endpoints.get(0).getPort());
}
@Test
@DisplayName("Null processExecutor defaults to a single UNO endpoint")
void nullProcessExecutorDefaultsToSingleEndpoint() {
ApplicationProperties properties = newProperties();
properties.setProcessExecutor(null);
RuntimePathConfig config = build(properties);
List<UnoServerEndpoint> endpoints = config.getUnoServerEndpoints();
assertEquals(1, endpoints.size());
assertEquals("127.0.0.1", endpoints.get(0).getHost());
assertEquals(2003, endpoints.get(0).getPort());
}
}
@Nested
@DisplayName("General contract")
class GeneralContract {
@Test
@DisplayName("getProperties returns the same instance passed to the constructor")
void propertiesAccessorReturnsSameInstance() {
ApplicationProperties properties = newProperties();
RuntimePathConfig config = build(properties);
assertSame(properties, config.getProperties());
}
@Test
@DisplayName("basePath matches InstallationPathConfig.getPath()")
void basePathMatchesInstallationPath() {
RuntimePathConfig config = build(newProperties());
assertEquals(BASE_PATH, config.getBasePath());
}
@Test
@DisplayName("All resolved path getters are non-null")
void allPathsNonNull() {
RuntimePathConfig config = build(newProperties());
assertNotNull(config.getPipelinePath());
assertNotNull(config.getPipelineWatchedFoldersPath());
assertNotNull(config.getPipelineWatchedFoldersPaths());
assertNotNull(config.getPipelineFinishedFoldersPath());
assertNotNull(config.getPipelineDefaultWebUiConfigs());
assertNotNull(config.getWeasyPrintPath());
assertNotNull(config.getUnoConvertPath());
assertNotNull(config.getCalibrePath());
assertNotNull(config.getOcrMyPdfPath());
assertNotNull(config.getSOfficePath());
assertNotNull(config.getTessDataPath());
assertNotNull(config.getUnoServerEndpoints());
assertTrue(config.getUnoServerEndpoints().size() >= 1);
}
}
}
@@ -2,7 +2,7 @@ package stirling.software.common.model;
import static org.junit.jupiter.api.Assertions.*;
import java.nio.file.Paths;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
@@ -31,18 +31,33 @@ class ApplicationPropertiesLogicTest {
assertTrue(sys.isAnalyticsEnabled());
}
@Test
void storageSigning_userListScope_defaultsToOrg_andIsSettable() {
// Self-host backward-compat: scope must default to "org" (saas profile pins "team").
ApplicationProperties.Storage.Signing signing = new ApplicationProperties.Storage.Signing();
assertFalse(signing.isEnabled());
assertEquals("org", signing.getUserListScope());
signing.setUserListScope("team");
assertEquals("team", signing.getUserListScope());
// Reachable from the full tree as storage.signing.userListScope.
assertEquals(
"org", new ApplicationProperties().getStorage().getSigning().getUserListScope());
}
@Test
void tempFileManagement_defaults_and_overrides() {
Function<String, String> normalize = s -> Paths.get(s).normalize().toString();
Function<String, String> normalize = s -> Path.of(s).normalize().toString();
ApplicationProperties.TempFileManagement tfm =
new ApplicationProperties.TempFileManagement();
String expectedBase =
Paths.get(java.lang.System.getProperty("java.io.tmpdir"), "stirling-pdf")
.toString();
Path.of(java.lang.System.getProperty("java.io.tmpdir"), "stirling-pdf").toString();
assertEquals(expectedBase, tfm.getBaseTmpDir());
String expectedLibre = Paths.get(expectedBase, "libreoffice").toString();
String expectedLibre = Path.of(expectedBase, "libreoffice").toString();
assertEquals(expectedLibre, tfm.getLibreofficeDir());
tfm.setBaseTmpDir("/custom/base");
@@ -0,0 +1,269 @@
package stirling.software.common.pdf;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.fail;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Stream;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;
import stirling.software.jpdfium.PdfDocument;
import stirling.software.jpdfium.text.TextLine;
import stirling.software.jpdfium.text.TextWord;
/**
* Accuracy and robustness tests for {@link PdfMarkdownConverter}, comparing conversion output
* against hand-authored golden Markdown for a set of owned/synthetic fixtures.
*
* <p>The {@link #gatedFixtures()} set is enforced in CI: those fixtures currently convert within
* the accuracy threshold and guard against regressions. Fixtures still being iterated on live in
* {@link #wipFixtures()} under a {@link Disabled} test so the goldens stay in the tree without
* breaking the build. Enable the WIP test locally to see per-fixture scores while working on the
* converter.
*/
class PdfMarkdownConverterTest {
/** Accuracy threshold: output must share at least this fraction of content with the golden. */
private static final double THRESHOLD = 0.95;
@TempDir Path tmp;
/** Fixtures that meet the accuracy threshold today and therefore gate CI. */
static Stream<Arguments> gatedFixtures() {
return Stream.of(
Arguments.of("multi-column-test_lorem.pdf", "multi-column-test_lorem.md"),
Arguments.of("bordered-table-test_widget.pdf", "bordered-table-test_widget.md"),
Arguments.of("many-tables-test_stress.pdf", "many-tables-test_stress.md"));
}
/** Fixtures still below the threshold; tracked here, enable locally to iterate. */
static Stream<Arguments> wipFixtures() {
return Stream.of(
Arguments.of(
"wrapped-cell-test_expense-report.pdf",
"wrapped-cell-test_expense-report.md"));
}
@ParameterizedTest(name = "{0}")
@MethodSource("gatedFixtures")
void convertMatchesGoldenMarkdown(String pdfName, String mdName) throws IOException {
assertConversionMatchesGolden(pdfName, mdName);
}
@Disabled("WIP fixtures below the accuracy threshold; enable locally to iterate")
@ParameterizedTest(name = "{0}")
@MethodSource("wipFixtures")
void convertMatchesGoldenMarkdownWip(String pdfName, String mdName) throws IOException {
assertConversionMatchesGolden(pdfName, mdName);
}
/**
* Degenerate/extreme geometry must not crash the converter. A crafted or malformed PDF can
* position text anywhere via a text matrix, so a row's words can span from near the origin to a
* coordinate beyond {@link Integer#MAX_VALUE}. The old column-detection code sized an {@code
* int[]} straight from {@code (int) Math.ceil(maxX) - lo}, which either allocated a multi-GB
* array (OutOfMemoryError) or overflowed to a negative length (NegativeArraySizeException) —
* taking down the request thread. Detection must instead bail out and return no columns.
*/
@Test
void columnDetectionSurvivesDegenerateGeometry() {
// x ≈ 2.5e9 is past Integer.MAX_VALUE; combined with a near-origin word it yields an
// implausible span that the pre-fix code turned into a fatal array allocation.
List<TextLine> rows = new ArrayList<>();
for (int r = 0; r < 4; r++) {
float y = 400f - r * 12f;
TextWord near = new TextWord(List.of(), 50f, y, 30f, 10f);
TextWord far = new TextWord(List.of(), 2_500_000_000f, y, 30f, 10f);
rows.add(new TextLine(List.of(near, far), 50f, y, 2_499_999_980f, 10f));
}
List<float[]> columns =
assertDoesNotThrow(() -> PdfMarkdownConverter.findColumnRangesFromLines(rows));
assertTrue(
columns.isEmpty(),
"implausible page span should disable column detection, not allocate from it");
}
private void assertConversionMatchesGolden(String pdfName, String mdName) throws IOException {
Path pdfPath = tmp.resolve(pdfName);
try (InputStream in =
getClass().getResourceAsStream("/pdf-ingestion-fixtures/" + pdfName)) {
if (in == null) {
fail("Fixture not found on classpath: /pdf-ingestion-fixtures/" + pdfName);
}
Files.copy(in, pdfPath);
}
String actual;
try (PdfDocument doc = PdfDocument.open(pdfPath)) {
actual = new PdfMarkdownConverter().convert(doc);
}
String expected;
try (InputStream in = getClass().getResourceAsStream("/pdf-ingestion-fixtures/" + mdName)) {
if (in == null) {
fail("Golden file not found on classpath: /pdf-ingestion-fixtures/" + mdName);
}
expected = new String(in.readAllBytes(), StandardCharsets.UTF_8);
}
// Image placeholders are not scored: their body text is a TODO ("ideally, add the info
// available about the image...") rather than real content, so comparing it would penalise
// output for matching a placeholder we intend to replace. Drop those lines from both sides.
expected = stripImagePlaceholders(expected);
actual = stripImagePlaceholders(actual);
double similarity = similarity(expected, actual);
if (similarity < THRESHOLD) {
fail(
String.format(
"Markdown output differs from golden file '%s' by %.1f%% (threshold %.0f%%):%n%s",
mdName,
(1.0 - similarity) * 100,
(1.0 - THRESHOLD) * 100,
unifiedDiff(expected, actual)));
}
}
/** Substring identifying an image-placeholder line, which is excluded from scoring. */
private static final String IMAGE_PLACEHOLDER_MARKER = "Image intentionally redacted";
/**
* Removes non-content lines from the comparison: image placeholders (TODO text we intend to
* replace) and GFM table separator rows (the {@code |---|---|} divider, whose exact dash count
* is cosmetic — any run of three or more dashes is valid Markdown).
*/
private static String stripImagePlaceholders(String md) {
StringBuilder sb = new StringBuilder();
for (String line : md.split("\n", -1)) {
if (line.contains(IMAGE_PLACEHOLDER_MARKER)
|| line.strip().startsWith("<image redacted")
|| isTableSeparatorRow(line)) {
continue;
}
if (sb.length() > 0) {
sb.append('\n');
}
sb.append(line);
}
return sb.toString();
}
/** True for a GFM table separator row, e.g. {@code |---|:--:|---|} (only |, -, :, space). */
private static boolean isTableSeparatorRow(String line) {
String t = line.strip();
if (!t.contains("-")) {
return false;
}
return t.chars().allMatch(c -> c == '|' || c == '-' || c == ':' || c == ' ');
}
/**
* Character-level similarity: proportion of expected characters that appear in the LCS. O(n*m)
* but golden files are small enough that this is fine.
*/
private static double similarity(String expected, String actual) {
if (expected.isEmpty() && actual.isEmpty()) return 1.0;
if (expected.isEmpty() || actual.isEmpty()) return 0.0;
// Strip all whitespace for a content-focused comparison
String e = expected.replaceAll("\\s+", " ").strip();
String a = actual.replaceAll("\\s+", " ").strip();
int lcs = lcsLength(e, a);
return (double) lcs / Math.max(e.length(), a.length());
}
private static int lcsLength(String a, String b) {
// Use two-row DP to keep memory reasonable
int m = a.length(), n = b.length();
int[] prev = new int[n + 1];
int[] curr = new int[n + 1];
for (int i = 1; i <= m; i++) {
for (int j = 1; j <= n; j++) {
if (a.charAt(i - 1) == b.charAt(j - 1)) {
curr[j] = prev[j - 1] + 1;
} else {
curr[j] = Math.max(curr[j - 1], prev[j]);
}
}
int[] tmp = prev;
prev = curr;
curr = tmp;
java.util.Arrays.fill(curr, 0);
}
return prev[n];
}
private static String unifiedDiff(String expected, String actual) {
String[] expectedLines = expected.split("\n", -1);
String[] actualLines = actual.split("\n", -1);
List<String> diff = new ArrayList<>();
diff.add("--- expected");
diff.add("+++ actual");
int maxLines = Math.max(expectedLines.length, actualLines.length);
int context = 3;
boolean inHunk = false;
int hunkStart = -1;
List<String> hunkLines = new ArrayList<>();
for (int i = 0; i < maxLines; i++) {
String exp = i < expectedLines.length ? expectedLines[i] : null;
String act = i < actualLines.length ? actualLines[i] : null;
boolean changed = exp == null || act == null || !exp.equals(act);
if (changed) {
if (!inHunk) {
inHunk = true;
hunkStart = Math.max(0, i - context);
// add context lines before change
for (int c = hunkStart; c < i; c++) {
hunkLines.add(" " + (c < expectedLines.length ? expectedLines[c] : ""));
}
}
if (exp != null) hunkLines.add("-" + exp);
if (act != null) hunkLines.add("+" + act);
} else {
if (inHunk) {
hunkLines.add(" " + exp);
// check if we're far enough past the last change to close the hunk
boolean moreChanges = false;
for (int j = i + 1; j < Math.min(i + context, maxLines); j++) {
String e2 = j < expectedLines.length ? expectedLines[j] : null;
String a2 = j < actualLines.length ? actualLines[j] : null;
if (e2 == null || a2 == null || !e2.equals(a2)) {
moreChanges = true;
break;
}
}
if (!moreChanges && (i - hunkStart) >= context) {
diff.add("@@ -" + (hunkStart + 1) + " @@");
diff.addAll(hunkLines);
hunkLines.clear();
inHunk = false;
}
}
}
}
if (inHunk && !hunkLines.isEmpty()) {
diff.add("@@ -" + (hunkStart + 1) + " @@");
diff.addAll(hunkLines);
}
return String.join("\n", diff);
}
}

Some files were not shown because too many files have changed in this diff Show More