Compare commits

...
Author SHA1 Message Date
stirlingbot[bot]andAnthony Stirling be914c7135 Sync Translations + tauri fix for get info (#6484)
### Description of Changes

This Pull Request was automatically generated to synchronize updates to
translation files and documentation. Below are the details of the
changes made:

#### **1. Synchronization of Translation Files**
- Updated translation files
(`frontend/editor/public/locales/*/translation.toml`) to reflect changes
in the reference file `en-GB/translation.toml`.
- Ensured consistency and synchronization across all supported language
files.
- Highlighted any missing or incomplete translations.
- **Format**: TOML

#### **2. Update README.md**
- Generated the translation progress table in `README.md` using
`counter_translation_v3.py`.
- Added a summary of the current translation status for all supported
languages.
- Included up-to-date statistics on translation coverage.

#### **Why these changes are necessary**
- Keeps translation files aligned with the latest reference updates.
- Ensures the documentation reflects the current translation progress.

---

Auto-generated by [create-pull-request][1].

[1]: https://github.com/peter-evans/create-pull-request

---------

Co-authored-by: stirlingbot[bot] <195170888+stirlingbot[bot]@users.noreply.github.com>
Co-authored-by: Anthony Stirling <anthony@stirlingpdf.com>
2026-06-09 21:31:33 +01:00
Anthony Stirling 71361f0d33 Minor: Office doc changes (#6571) 2026-06-09 18:00:34 +01:00
Anthony Stirling 6478c400db Fix font loss in rearrange/overlay/autosplit/OCR from PDFBox (#6545) 2026-06-09 18:00:20 +01:00
Anthony Stirling 502f6c1e4d fix folder causing 500 toast when deleted on another machine (#6551) 2026-06-09 18:00:02 +01:00
Anthony Stirling 1a0beaffc2 stop background flash on tab switches, unblock Audit/Usage demos (#6562) 2026-06-09 17:59:53 +01:00
Anthony Stirling 1e739b6f6f SaaS-aware API landing page (#6585)
# Description of Changes

OLD  (and still current in selfhosted)
<img width="610" height="869" alt="image"
src="https://github.com/user-attachments/assets/f8019298-b4ee-4a68-b928-a9746b64ac1c"
/>


New (in SaaS mode)

<img width="635" height="876" alt="image"
src="https://github.com/user-attachments/assets/6ee4946f-1d7b-42ec-a6f7-75e85739e348"
/>


---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-09 14:49:20 +00:00
ConnorYoh 98967bfa86 PAYG: V14 + V15 — subscription_id, free-tier, RPCs, audit logs (#6532)
## Summary

Two Flyway migrations + matching JPA entity updates. **Part 1 of 2** in
the Stripe/Supabase wire-up (PR-SB-1 in
`payg-stripe-supabase-plan.html`); the companion SaaS PR carries the
twin Supabase migrations + new edge functions.

### V14 — payg_subscription_state.sql

- `payg_team_extensions.payg_subscription_id` — the single switch that
decides whether a team is billed. NULL = free-tier or block; NOT NULL =
post Stripe meter events.
- `pricing_policy.free_tier_units_per_cycle` — per-policy free allowance
before a card is required. Default 0.
- `payg_link_subscription(team_id, customer_id, sub_id)` RPC —
idempotent.
- `payg_unlink_subscription(team_id, reason)` RPC — called on
`subscription.deleted`.
- AFTER-INSERT trigger on `teams` so every new signup gets a
`payg_team_extensions` sidecar row from creation.
- Backfill for existing teams without a sidecar row.
- RLS: SELECT permissive (any team member), UPDATE restricted to LEADER.
Service-role bypasses (backend reads + day-1 migration writes).

### V15 — payg_audit_logs.sql

- `payg_meter_event_log` — backend audit of every Stripe meter event
POST attempt (idempotency-key UNIQUE; index on unposted rows for nightly
reconcile).
- `payg_subscription_change_log` — written by V14 RPCs on every
link/unlink.

### Entity updates

- `PaygTeamExtensions.paygSubscriptionId` — read-only field; RPC
functions are the only writers.
- `PricingPolicy.freeTierUnitsPerCycle` — read by upcoming
`PaygTeamUsageService` (PR-SB-4).

### Behaviour change

**None yet.** The columns + functions sit unused until PR-SB-4 wires
`PaygMeterReportingService` and the free-tier gate into
`JobChargeService`. This PR is pure schema + JPA wiring.

## Test plan

- [x] `./gradlew :saas:test` — BUILD SUCCESSFUL
- [x] Manual schema review: column types, FK directions, RLS scope
- [ ] Apply against v3-Supabase via `supabase db push` (after companion
SaaS PR merges)
- [ ] Smoke-test the trigger: `INSERT INTO teams(...)` → assert
`payg_team_extensions` row appears
- [ ] Smoke-test RPCs: SQL-only test of `payg_link_subscription` +
`payg_unlink_subscription` produces expected row + audit entries

## References

- `notes/PAYG_DESIGN.md` (revision note 2026-06-03)
- `payg-stripe-supabase-plan.html` §3.1 (RPC functions), §3.5 (RLS),
§3.10 (audit-log tables)
2026-06-09 14:48:05 +00:00
ConnorYoh ff96a80947 PAYG B-3 / S-3: cucumber suite for shadow-mode flows + CI workflow (#6522)
## What this PR is

End-to-end cucumber coverage for the PAYG shadow charging engine (the
filter + interceptor stack from #6519), wired into CI via a new
`docker-compose-tests-saas.yml` workflow that runs only on PAYG-touching
PRs.

Stacked on #6519.

## Automated scenarios (run by `docker-compose-tests-saas.yml`)

See
[`testing/cucumber/features/payg/shadow_charges.feature`](../tree/payg-s3-cucumber/testing/cucumber/features/payg/shadow_charges.feature):

| Scenario | Validates |
|---|---|
| First tool call writes a CHARGED row | Filter + interceptor fire
end-to-end |
| Lineage join — second call on output | `JobService.joinOrOpen`
matching; no new shadow row |
| 4xx leaves the row CHARGED | "Customer paid for the attempt" semantics
|
| ZIP-returning tool records per-PDF OUTPUT | `PaygOutputExtractor`
unpacks + records signatures |
| Multi-file input writes a single shadow row | Multi-input group sizing
|
| `X-Stirling-Automation` sets PIPELINE source | Header → `JobSource`
detection |

All 6 run locally via `./testing/test-payg.sh` and will run on CI for
any PR that touches `app/saas/**`, the PAYG cucumber features, the saas
compose stack, or the workflow itself.

## Manual-only scenarios — documented in design doc, not in this suite

Two parts of the shadow engine are deliberately not automated; the
engine paths are unit-tested in
`PaygChargeInterceptorTest.afterCompletion_5xx_opened_*`, and the manual
procedures (which require a temporary throw endpoint or a container
restart with a flag flipped) live in [`notes/PAYG_DESIGN.md` §7.5.2
"PAYG cucumber: manual-only
scenarios"](../tree/payg-s3-cucumber/notes/PAYG_DESIGN.md).

- **5xx first-step failure → REFUNDED + CLOSED.** No reliably-5xx-ing
endpoint exists; manual procedure adds a throw endpoint, runs, asserts,
removes.
- **Kill-switch (`PAYG_FILTER_ENABLED=false`).** Needs a container
restart mid-suite; manual procedure tears down, flips env, brings up,
asserts zero shadow rows.

If either gets a hot-reload path (test-only throw endpoint shipped
behind a profile gate, or admin endpoint for the kill switch), automate
it in a follow-up and drop the manual procedure.

## CI workflow

`.github/workflows/docker-compose-tests-saas.yml` (new) —
self-contained, not wired into `build.yml`'s `files-changed` matrix so
the saas-cucumber job fails and succeeds independently. Triggers only on
PAYG-relevant paths. No JaCoCo coverage in v1 (saas compose doesn't have
the coverage override; can add later).

## Test infrastructure (recap)

- **`testing/compose/docker-compose-saas.yml`** — Stirling-PDF backend
with `STIRLING_FLAVOR=saas` + Postgres holding the `stirling_pdf`
schema. Supabase JWT auto-config disabled; API-key auth via
`SECURITY_CUSTOMGLOBALAPIKEY` is the live path the cucumber tests
exercise.
- **`testing/compose/payg/saas-init.sql`** + **`saas-seed.sql`** —
schema bootstrap + idempotent seed (team / user / wallet_policy).
- **`testing/cucumber/features/payg/shadow_charges.feature`** — the 6
scenarios above.
- **`testing/cucumber/features/steps/payg_step_definitions.py`** — step
defs using `requests` (HTTP) + `psycopg` (direct DB inspection). Direct
DB reads are deliberate — we want to see the filter's side effects, not
relay them through another API layer.
- **`testing/test-payg.sh`** — companion runner to `testing/test.sh`.
Brings up the saas compose, waits for health, seeds, runs behave, tears
down.
- **`behave.ini`** excludes `features/payg` from the default behave run
(the saas-cucumber CI job invokes it explicitly).

## Why a separate harness from `testing/test.sh`

The existing `test.sh` covers the proprietary-flavour stack (no PAYG
tables, no saas profile). Coupling two CI matrices that fail and succeed
independently into one script is asking for trouble. Keep the
saas-cucumber job focused on its own concerns; once the harness is
mature, the wider team can decide whether to merge them.

## Tracked in

`notes/PAYG_DESIGN.md` §7.5 (PR-S3) + §7.5.2 (manual scenarios).
2026-06-09 14:47:40 +00:00
Anthony Stirling 347ae9ebbf fix many UI issues (#6569)
# Description of Changes

- Tool action button truncation - fixed by allowing Mantine <Button>
label to wrap (whiteSpace: normal, height: auto) instead of clipping
- Role badge truncation on People page - fixed by dropping the column's
fixed w={100} and letting the badge size to its content
- Settings nav item wraps to 3 lines - fixed by hiding the inline ALPHA
badge by default and revealing it on :hover/:focus-within/.active
- Zoom slider cramped on narrow desktop - fixed by removing the
toolbar's hardcoded minWidth: 30rem and giving the slider flexShrink: 0
+ minWidth: 6rem
- "Swipe left or right" hint on desktop - fixed by adding a useIsTouch()
hook (pointer: coarse) and gating the hint on isMobile && isTouch
- Logout doesn't redirect - fixed by replacing navigate('/login') with
window.location.assign('/login') in a finally block so auth context
fully re-bootstraps
- Viewer top toolbar clips icons on mobile - fixed by switching the
wrapped state to justify-content: flex-start + overflow-x: auto so the
icon strip is momentum-scrollable
- Mobile bottom toolbar overflows - fixed by gating layout on
useIsPhone() and reducing the inline bar to prev / page / next / ⋮ only
- Lost controls when shrinking mobile toolbar - fixed by adding a
Mantine <Menu> behind ⋮ that groups First/Last page, Zoom in/out (with
%), Dual-page, Dark/Sepia filter under Page navigation / Zoom / View
labels
- "Upload from computer" label clipped on hover - fixed by unmounting
the Add Files button entirely while Upload is hovered, so Upload claims
width: 100%
- Settings rows clip controls off-screen - fixed by adding flex: 1,
minWidth: 0 to the inner text-block <div> on 44 rows across 10 -files,
so labels shrink and wrap while controls stay anchored to the right

---
Screenshots 

[report-before-after.html](https://github.com/user-attachments/files/28687621/report-before-after.html)

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-09 13:18:57 +00:00
Anthony Stirling 800a411167 Hide endpoints (#6586)
# Description of Changes

Hides the /api/v1/credits endpoints from the generated OpenAPI/Swagger
docs. The root GET /api/v1/credits and GET /api/v1/credits/usage now
carry @Hidden (the 8 admin credit endpoints were already hidden), so the
whole Credit Management controller is gone from the docs.

Adds a single global AI tag to the OpenAPI definition.


Why
We don't want the credits or AI endpoints surfaced in the public API
docs yet as they are not ready for public use, but we do want the AI
endpoints pre-grouped under one AI tag so they land cleanly when we
later un-hide them but dont clutter PDF APIs.

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-09 13:18:36 +00:00
Anthony Stirling 66f431a2b7 Lazy-load Stripe SDK so it only loads on checkout (#6546)
# Description of Changes

- Stripe SDK (`@stripe/*` + `js.stripe.com/v3`) was loading on every
page; now it only loads when an upgrade/checkout modal actually opens.
- Converted every import site to `React.lazy()` + `<Suspense>`, gated by
the existing `opened` state.
- Adds a Playwright spec that asserts no Stripe requests on landing or
settings.



---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-09 12:36:48 +00:00
James Brunton 0e3cbb3cf2 Explicitly test for console warnings & errors (#6502)
# Description of Changes
Disallow warnings and errors from being thrown in the browser console
during tests unless explicitly expected in the test. Also adds a
Playwright test to prod around some main UI areas and checks that no
warnings/errors have been thrown.
2026-06-09 08:34:02 +00:00
James Brunton 002de06411 Fix desktop app not being able to load pdfium (#6575)
# Description of Changes
The changes in
[#6279](https://github.com/Stirling-Tools/Stirling-PDF/pull/6279) broke
the desktop app because the wasm URL handling didn't deal with
`tauri://` paths. Also I noticed that `task desktop:build:dev:mac`
failed locally because it was attempting to sign the app with
credentials that developers won't have (and shouldn't need), so I fixed
that too.
2026-06-08 16:21:56 +00:00
James Brunton 51478e5051 Policies backend (#6527)
# Description of Changes
Add a backend for running any multi-step PDF operations. This is
designed to be used for the upcoming Policies feature, along with
anything else that will require automated running of PDF operations,
like the Automate tool or Processing Folders.

The implementation is not complete. I've tried to get all the
infrastructure in there so that we can add in whichever triggers we need
in the future (like cron triggers or watching folders on disk) but
currently it just supports manual triggering of the policy.

The basis of this work was the operation running from the Stirling
Engine, which this PR removes in favour of this new system. The only
currently accessible frontend way to test this work is to ask the AI
chat to execute multiple operations on a PDF, but I've also extensively
tested with direct API calls to make sure that the policies work and
persist properly.
2026-06-08 10:50:55 +00:00
Anthony Stirling 69e62d8949 exclude unused Redis auto-config (#6547)
fix(health): exclude unused Redis auto-config so default
/actuator/health stays UP

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-08 10:41:38 +00:00
Anthony Stirling 2f6b113a13 feat(settings): link ENTERPRISE badges to plan page (#6560)
## Summary
Make the remaining static ENTERPRISE badges in the admin settings
clickable so they navigate the user to `/settings/adminPlan`, matching
the pattern already used by the PRO badges in Connections / Features /
General sections.

### Before
Two ENTERPRISE badges were inert text chips with no affordance:
- `AdminSecuritySection.tsx` - Audit Logging
- `AdminDatabaseSection.tsx` - Database section header

### After
Both now use the same pattern as the existing clickable PRO badges:
- `cursor: pointer`
- `onClick={() => navigate("/settings/adminPlan")}`
- `title` tooltip with the existing
`admin.settings.badge.clickToUpgrade` i18n key ("Click to view plan
details")

No new strings, no new components - just wiring up existing behavior to
the two badges that were missing it.

### Existing already-clickable badges (kept identical for reference)
- `AdminConnectionsSection.tsx:585-596` - SSO Auto Login PRO
- `AdminFeaturesSection.tsx:175-186` - Server Certificate PRO
- `AdminGeneralSection.tsx:920-931` - Custom Metadata PRO
2026-06-08 10:40:35 +00:00
Anthony Stirling af52134811 fix(automate): flip AutomationEntry tooltip to position=left (#6550)
# Description of Changes

automate description tooltip was facing wrong way

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-08 10:38:00 +00:00
Anthony Stirling 8a2474ff60 fix(desktop): enable in-page drag-drop in Tauri build (#6548)
## Summary

- Set `dragDropEnabled: false` on the Tauri window so HTML5 drag events
reach the WebView. Previously the default `true` made Tauri intercept
all drag-drop at the OS level, silently breaking in-page drag-to-reorder
(Pragmatic Drag and Drop in `FileEditorThumbnail` /
`useFileItemDragDrop`) in the desktop build. The Active Files tab
reorder, which feeds Merge ordering, was the user-visible symptom.
- Browser builds are unaffected (tauri.conf.json is desktop-only).
- The OS file-drop pipeline now flows through the existing Mantine
`Dropzone` in `FileEditor.tsx` via HTML5 events instead of the Rust
`WindowEvent::DragDrop` handler in `lib.rs:215`. Verified working.

## Test plan

- [x] Desktop: drag a thumbnail in Active Files past another - row goes
semi-transparent, order updates on drop.
- [x] Desktop: drag a PDF from File Explorer onto the window - file is
added.
- [x] Web build: drag-to-reorder still works (unchanged code path; flag
is desktop-only).
- [x] Merge tool: order set by drag in Active Files is the order used by
the merge output.

## Follow-up (not in this PR)

- `WindowEvent::DragDrop` arm in
`frontend/editor/src-tauri/src/lib.rs:215-229` is now unreachable for
window drops. The `forward_files_to_window` helper still serves the
macOS Finder "Open With" path (`RunEvent::Opened` at lib.rs:230), so
only the DragDrop arm can be deleted. Worth a small cleanup pass later.
2026-06-08 10:36:58 +00:00
Anthony Stirling d202c9c32f Minor: sanitize SVG (#6572)
# Description of Changes

Sanitize SVG

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-08 10:31:41 +00:00
Anthony Stirling 1ef03c43b4 fix(i18n): wrap hard-coded English strings in t() across UI (#6566)
## Summary
Audit + bulk fix of hard-coded English UI strings - `aria-label`,
`title`, `placeholder`, `label`, and raw JSX literals that bypassed i18n
entirely. Each literal now goes through `t("key", "English Default")`
from `react-i18next`, and every new key has a corresponding entry in
`en-GB/translation.toml` so translators can pick it up.

## What this fixes
Strings were rendered untranslated in every non-EN locale because they
never went through `t()` at all (not just "value not translated yet").
Affects screen-reader labels, tooltips, form placeholders, empty/loading
states, plan card content, and the entire workflow ParticipantView.

## Coverage (~143 keys / 50 files)
- **Viewer chrome** - search bar (close, clear, prev/next, "of N"
results), link/signature/redaction actions, viewer error state, zoom
labels
- **Page editor** - undo/redo/rotate/delete toolbar tooltips, empty
state, bulk selection operator chip tooltips
- **Shared primitives** - Tooltip close, InfoBanner dismiss, TextInput
clear, Toast dismiss/toggle, UpdateModal close, EditableSecretField
edit, DropdownListWithFooter search, FileCard/FileDropdownMenu actions,
EmptyFilesState + AddFileCard upload
- **Tools** - Image upload + hint, ColorControl eyedropper, sign Use
Signature, CompressSettings, OCR loading, PageLayout
margin/border/row/col placeholders, FormFill switch + save + re-scan
- **Proprietary admin** - OverviewHeader signed-in line + logout,
AdminPremiumSection moved-features list (via `<Trans>`),
AdminPlanSection no-data alert, AdminAdvancedSection temp-dir
placeholders, AdminEndpointsSection multiselect placeholders,
AdminMailSection + AdminDatabaseSection password placeholders
- **Onboarding** - MFASetupSlide QR loading + auth code label,
SecurityCheckSlide role select + options
- **ParticipantView** - entire sign-document UI (~30 strings: loading,
error, badges, headings, cert-type Select, all input labels and
placeholders, action buttons, completion + expired alerts) - file
previously imported `useTranslation` but only used `t()` for cert
validation
- **planConstants.ts refactor** - replaced `PLAN_FEATURES` /
`PLAN_HIGHLIGHTS` const exports with `usePlanFeatures()` /
`usePlanHighlights()` hooks. Service layer (`licenseService.getPlans`)
updated to accept feature/highlight maps so it stays hook-free. Callers
(`usePlans`, `CheckoutContext`) resolve the hooks at the React boundary
- **Previously catalogued offenders** - `FileSidebarFileItem`
open/close-viewer aria-labels, `quickAccessBar/ActiveToolButton` "Back
to all tools" tooltip + aria, `AppConfigModal` close button

## Notes
- One small refactor in `usePageSelectionTips.ts` was needed to resolve
a TOML key-shape conflict: the existing scalar keys
`bulkSelection.operators.{and,not,comma}` needed to become tables to
hold the new `.title` subkeys for OperatorsSection's chip tooltips. The
existing descriptions moved to `[bulkSelection.operators.descriptions]`
and the three i18n key paths in usePageSelectionTips were updated to
match.
- Viewer sidebar close buttons
(Bookmark/Layer/Thumbnail/Attachment/Comments) were on the audit list
but are NOT on main - they're added by the unmerged PR #6552
(feat/viewer-sidebar-ux). Those particular strings will need wrapping
when that PR lands.
- TOML hook (`toml-sort-fix`) ran and re-sorted the translation file.

## Test plan
- [ ] `task frontend:typecheck` passes (core + proprietary + desktop
variants)
- [ ] `task frontend:lint` passes
- [ ] Switching language to Deutsch / Русский: previously-English
`aria-label`s + tooltips + placeholders + plan card bullets now render
translated (when the locale has values) or fall back to the English
default (when it doesn't)
- [ ] Plan page bullet points in EN render unchanged
- [ ] Sign-document flow (ParticipantView) renders unchanged in EN
2026-06-08 10:11:43 +00:00
brios 0dff192281 perf(compression): add vite-plugin-compression for gzip and Brotli support (#6279)
Signed-off-by: Balázs Szücs <bszucs1209@gmail.com>
2026-06-05 18:49:09 +01:00
Anthony Stirling 9866d6e12d Fix desktop updater latest.json generation for releases (#6540) 2026-06-05 15:41:41 +01:00
James Brunton e79f4a044f Make any typing linting opt-out instead of opt-in (#6542)
# Description of Changes
Reconfigure linting of `any` type to an opt-out instead of an opt-in
strategy now that we're close enough to everything supporting it. Also
slightly expands the scope of things included in the linting.
2026-06-05 14:11:26 +00:00
James Brunton 9ab404b2e6 Fix intermittently failing Playwright tests in main (#6541)
# Description of Changes
[#6474](https://github.com/Stirling-Tools/Stirling-PDF/pull/6474)
updated the IndexedDB schema number to v9, but a couple of Playwright
tests were explicitly creating a DB in v4 schema, which then caused
inconsistently failing tests because the DB upgrade process is
asynchronous and sometimes was too slow to upgrade, causing the test to
get into an invalid state.

Also fixes the screenshots directory exclusion since the frontend folder
was restructured.
2026-06-05 11:34:32 +00:00
EthanHealy01 a61fe012d7 chore: i18n time utils and use TFunction type (#6507) (#6539)
## Summary

Addresses two review comments from #6507:

- **`timeUtils.ts`** — route relative time strings (`just now`, `Xm
ago`, `Xh ago`, `Xd ago`) through i18n by accepting a `TFunction`
parameter and using new `time.relative.*` keys in `en-GB`
- **`ChatPanel.tsx`** — replace `ReturnType<typeof useTranslation>["t"]`
with `TFunction` from `i18next`

## Test plan
- [x] `task frontend:check` passes (695 tests)
2026-06-05 09:45:27 +00:00
Anthony Stirling c93776e297 Fix z-index conflicts: Google Drive picker, automate dropdowns, tooltips (#6513)
## Summary

## What changed

### 1. Google Drive Picker now renders above the FileManager modal
`frontend/editor/src/core/services/googleDrivePickerService.ts`

The picker is opened from inside the FileManager modal
(`Z_INDEX_FILE_MANAGER_MODAL = 1200`), but Google's Picker defaults to
z-index ~1001 - so it landed *behind* the modal that invoked it. Added
`setZIndex(Z_INDEX_OVER_FILE_MANAGER_MODAL)` to the builder.

### 2. `Z_INDEX_AUTOMATE_DROPDOWN` no longer collides with
`Z_INDEX_FILE_MANAGER_MODAL`
`frontend/editor/src/core/styles/zIndex.ts`

Both constants were `1200`. The automate dropdown only needs to sit
above the automate modal (1100), so dropped it to `1150`. This keeps
automate dropdowns above their parent modal but reliably below the file
manager scrim when the two overlap.

Confirmed callers - all are dropdowns inside automate-modal tool
settings:
- `DropdownListWithFooter.tsx`
- `AddPageNumbersAppearanceSettings.tsx`
- `AddPasswordSettings.tsx`
- `StampPositionFormattingSettings.tsx`
- and several other `*Settings.tsx` files

All keep working as intended (1150 > 1100).

### 3. Tooltip z-index honours the documented hierarchy again
`frontend/editor/src/core/components/shared/tooltip/Tooltip.module.css`

`Tooltip.tsx` sets `zIndex: Z_INDEX_OVER_FULLSCREEN_SURFACE` (1300)
inline, but the CSS module had a hardcoded `z-index: 9999` that overrode
it. Removed the stale CSS rule so tooltips render at the intended 1300
level rather than floating above almost everything.
2026-06-05 08:16:21 +00:00
EthanHealy01 1698769928 Improvements to agent chat markdown rendering. (#6507)
### To test

- Ask the agent to “list all the things you can do and put them in a
markdown table”. I know we’re explicitly asking it for markdown, but I
don’t want to update the system prompt to ask it to make tables when
necessary because it’ll probably turn everything into a table, not sure
though, we can test in future.
     -  Notice how the loading is different
- Notice how the user chat is in a bubble but the agent chat is flat
(super standard design practice in AI tools, and looks much better when
the agent outputs mardown, expecially tables and needs room to do so)
- Ask it to do something different, then close the chat, and see that
the agent is marked as running and has a green outline and a green dot.
     - Play around with resizing the chat to make it bigger/smaller    
    
Open to any and all criticisms on any of the design choices, and of
course the usual, code etc.


Resizing
<img width="1572" height="812" alt="Screenshot 2026-06-01 at 2 47 53 PM"
src="https://github.com/user-attachments/assets/ec0ac1d0-01da-4025-bf7e-eea4eb544181"
/>

Loading (cool animation not visible through screenshot obviously)
<img width="559" height="141" alt="Screenshot 2026-06-01 at 2 53 41 PM"
src="https://github.com/user-attachments/assets/99f0b1f5-1719-4d78-8947-21b142293052"
/>

Removed bubbles for agent chat (maybe controversial, let me know) and
markdown now renders properly again
<img width="654" height="1060" alt="Screenshot 2026-06-01 at 2 55 01 PM"
src="https://github.com/user-attachments/assets/445f0889-a632-4751-9a16-f80ae388c632"
/>
2026-06-04 18:26:19 +00:00
Anthony StirlingandReece Browne bd9ef0586b fix: harden multi-file response detection so merge can't fail silently (#6516)
Co-authored-by: Reece Browne <74901996+reecebrowne@users.noreply.github.com>
2026-06-04 18:17:36 +01:00
Anthony Stirling cb687fbf99 fix(e2e): stop files-page tests racing the skeleton-grid render (#6533) 2026-06-04 18:00:30 +01:00
Anthony Stirling 69ee39fa6e Fix settings: dark borders, update dropdown z-index, dead accessibility link (#6528) 2026-06-04 17:59:22 +01:00
Anthony Stirling 353b5c807c sort comments sidebar in visual reading order (#6439) (#6514) 2026-06-04 17:58:55 +01:00
ConnorYoh 22dacbed01 PAYG B-2: shadow-mode filter + interceptor (engine activation) (#6519)
## What this PR does

Wires the **B-1 shadow charging engine** into real HTTP request flow.
After this lands, flipping an internal team to ``PAYG_SHADOW`` via SQL
begins populating ``payg_shadow_charge`` automatically — with **zero
impact** on the legacy credit deduction path.

**This is the load-bearing PR for shadow mode.** Without it, B-1's
engine sits idle — nothing in the codebase calls
``JobChargeService.openProcess()`` from a real HTTP request.

Stacks on top of #6477 (PR B-1).

## Components

| Class | Role |
|---|---|
| ``PaygResponseBodyWrapperFilter`` | Servlet filter, installs tee'ing
response wrapper. Defers wrapper close to ``AsyncListener`` for
``DeferredResult`` / ``CompletableFuture`` controllers so the lifetime
spans the async window. |
| ``PaygResponseBodyWrapper`` | ``HttpServletResponseWrapper`` —
in-memory ``ByteArrayOutputStream`` up to 10 MiB; spills to ``TempFile``
above. ``materialisedPath()`` always returns a uniform ``Path``
interface. |
| ``PaygChargeInterceptor`` | ``AsyncHandlerInterceptor`` mirroring
``UnifiedCreditInterceptor`` shape. ``preHandle`` gates on
``@AutoJobPostMapping``, materialises multipart inputs, calls
``JobChargeService.openProcess``. ``afterCompletion`` branches on HTTP
status. |
| ``PaygOutputExtractor`` | Pulls PDFs out of the response body. Direct
``application/pdf`` returns body verbatim; ``application/zip`` iterates
entries and keeps each ``.pdf`` entry whose first bytes match the
``%PDF-`` magic. |
| ``PaygWebMvcConfig`` | Registers filter at end of Spring filter chain
(after security); interceptor after ``UnifiedCreditInterceptor``. |
| ``PaygFilterProperties`` | ``payg.filter.enabled`` master switch +
in-memory threshold + optional max-bytes ceiling. |

## Status branching in afterCompletion

| HTTP status | Action |
|---|---|
| **2xx** | Append OK step; extract PDFs from response;
``JobService.recordOutput`` per PDF |
| **4xx** | Append FAILED step with ``errorCode``. No refund — customer
paid for the attempt. No OUTPUT recording. |
| **5xx + OPENED** (first-step) |
``JobChargeService.markFirstStepFailed`` → shadow row flipped to
``REFUNDED``, process CLOSED. Refund counter incremented. |
| **5xx + JOINED** (mid-chain) | ``JobChargeService.decrementStepCount``
— step slot returned without resetting ``lastStepAt`` (workflow window
stays active for retry). |

## New ``JobChargeService`` methods

- **``markFirstStepFailed(jobId, reason)``** — flips shadow row to
``REFUNDED`` with ``refundedAt`` + ``refundReason``, closes the process.
Idempotent. Mimics the eventual Stripe
``meter_event_adjustment(cancel)`` flow that real-mode will invoke at
the same callsite. **Refund implies close** so a same-input retry can't
lineage-join into a refunded chain for free work.
- **``decrementStepCount(jobId)``** — defensive floor at 1; never drives
count negative.

## Schema

- Backend: ``V13__payg_shadow_charge_status.sql`` adds ``status``
(``CHARGED`` | ``REFUNDED``) + ``refunded_at`` + ``refund_reason``.
``DEFAULT 'CHARGED'`` so existing B-1 rows stay correct without
backfill.
- Supabase: matching migration in
[Stirling-PDF-SaaS#payg-shadow-charge-status](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/tree/payg-shadow-charge-status)

## Fail-open semantics in shadow

Any ``RuntimeException`` in ``preHandle`` / ``afterCompletion`` is
logged at WARN, increments ``payg.filter.errors``, and lets the
customer's tool call proceed unbilled. This **reverses to fail-closed**
when ``wallet_policy.engine = PAYG`` (real charging) — that reversal
lives inside ``JobChargeService`` and ships with the cap evaluator PR
(PR-C1 in PAYG_DESIGN.md).

## Observability

Micrometer metrics:
- ``payg.filter.errors`` Counter — internal failures (preHandle +
afterCompletion). Alert source.
- ``payg.filter.calls`` Counter, tagged ``disposition`` (``OPENED`` |
``JOINED`` | ``SHORT_CIRCUIT``)
- ``payg.filter.refunds`` Counter — first-step 5xx refunds
- ``payg.filter.duration`` Timer — preHandle + afterCompletion
wall-clock per request

## Test coverage (38 tests across 4 classes)

- **PaygResponseBodyWrapperTest** (12 tests) — in-memory, spill,
threshold crossing mid-chunk, writer vs outputStream exclusivity,
``resetBuffer`` with and without spill, close idempotency, single-byte
writes across threshold.
- **PaygOutputExtractorTest** (7 tests) — direct PDF, parametrised
content type, ZIP with mixed entries + magic-byte gate, corrupt ZIP
fail-open, empty ZIP.
- **PaygChargeInterceptorTest** (13 tests) — all preHandle
short-circuits, OPENED disposition stash, fail-open on chargeService
exception, 2xx recordOutputs path, 5xx OPENED → markFirstStepFailed, 5xx
JOINED → decrementStepCount, 4xx FAILED step append, max-bytes ceiling
skip, PIPELINE header detection.
- **JobChargeServiceTest extended** (+6 tests) — markFirstStepFailed
happy path, idempotency, missing-shadow-row case, long-reason trim;
decrementStepCount happy path, floor-at-1 defence, missing-job no-op.

## What's NOT in this PR (deliberate)

- **No SpringBootTest layer.** The saas module doesn't have bootstrap
test infrastructure (Supabase JWT config + H2 schema harness).
Integration confidence comes from the manual staging deploy + SQL-flip
of an internal team. Bootstrap-test infra is a focused follow-up if
needed.
- **No saas-mode Behave / docker-compose.** Per design §17 — deferred.
Existing ``testing/cucumber/`` infrastructure doesn't yet have a
saas-profile compose target; that's its own PR when warranted.
- **No CreditService wire-in** (per design §13 decision). Per-row
comparison data moves to the reconciliation report PR (PR-S2).
``legacy_credits_charged`` + ``diff_pct`` columns stay at 0 in shadow
rows.
- **No reconciliation report endpoint.** Direct SQL queries against
``payg_shadow_charge`` cover the data-access need until patterns emerge.

## Rollback levers

| Symptom | Lever |
|---|---|
| Some / all tool calls breaking due to filter |
``payg.filter.enabled=false`` + restart (~20s) |
| Shadow rows look wrong for a specific team | ``UPDATE wallet_policy
SET engine = 'LEGACY' WHERE team_id = ?`` |
| Mass shadow weirdness | ``UPDATE wallet_policy SET engine = 'LEGACY'``
|
| Memory exhaustion from response tee | Lower
``payg.filter.response.in-memory-threshold-bytes`` |

## Test plan

- [ ] CI green (build + tests)
- [ ] Aikido / Snyk / SonarCloud clean
- [ ] Manual: deploy to staging
- [ ] Manual: flip one internal team via ``UPDATE wallet_policy SET
engine = 'PAYG_SHADOW' WHERE team_id = ?``
- [ ] Manual: hit ``/api/v1/security/add-password`` with that team's
JWT; verify a ``payg_shadow_charge`` row appears with
``status='CHARGED'``
- [ ] Manual: trigger a 503 (e.g. via temporary backend kill
mid-request); verify the resulting row is ``status='REFUNDED'`` + the
process is ``CLOSED``
- [ ] Manual: hit ``/api/v1/general/split`` with a multi-page PDF;
verify one OUTPUT signature per inner PDF appears in
``job_artifact_hash``
- [ ] Manual: chain ``add-password`` → ``compress`` on the output;
verify the second call JOINS the first process (no new shadow row) and
the inner output OUTPUT signature is what drove the lineage join

## Stacks on / references

- Stacks on: #6477 (B-1 — shadow charging engine)
- Schema mirror: Stirling-PDF-SaaS#payg-shadow-charge-status branch
- Design doc: ``notes/PAYG_FILTER_DESIGN.md`` (all 19 decisions DECIDED)
2026-06-04 15:07:58 +00:00
ConnorYoh 3807cdfbc6 PAYG: process tracking + shadow charging engine (PR B-1) (#6477)
> 📌 **Stacked on
[#6464](https://github.com/Stirling-Tools/Stirling-PDF/pull/6464)**
(lineage primitives, still in review). #6469 has merged so its commits
are no longer in this PR's diff. Once #6464 merges, a final rebase
collapses the lineage-primitives commits out of this diff too — leaving
only the B-1 work.

## What this is

Process tracking + shadow charging engine. Bundles PR-I7 service half
with the non-filter piece of PR-I7a so the pieces ship together — none
of them is useful in isolation.

**Review focus:** the new files in:
- \`app/saas/src/main/java/stirling/software/saas/payg/job/\`
(\`JobService\`, \`JobContext\`, \`JoinOrOpenResult\`,
\`StaleJobCloser\`)
- \`app/saas/src/main/java/stirling/software/saas/payg/charge/\`
(\`JobChargeService\`, \`ChargeContext\`, \`ChargeOutcome\`,
\`JobInput\`)
-
\`app/saas/src/main/java/stirling/software/saas/payg/lineage/LineagePruneScheduler.java\`
- their tests

The 8 files inherited from #6464 (lineage primitives) are unchanged from
there — they ride along in this diff until #6464 lands.

The remaining work for shadow-in-staging is the ingress/egress filter
that wires controllers into this engine — that's PR B-2.

## Scope

### \`JobService\` — persistence + lineage policy

- **\`joinOrOpen\`** — the multi-input "any-match-joins, newest wins"
rule. Hash every input via the lineage detector; if any matches an open
process in the workflow window, attach to the one with the freshest
\`lastStepAt\`. Step-limit overflow on the matched job spawns a fresh
process; the new job's input signatures are still recorded so
\`mostRecentMatchWins\` routes future calls forward.
- **\`recordOutput\`** — post-tool-success path. Records OUTPUT
signatures so the next call that takes this file as input
lineage-matches into the same process.
- **\`appendStep\`** — audit-trail step row written after a tool
completes.
- **\`close\`** — idempotent; safe to call from multiple paths
(explicit, FE on-unload, scheduler). Returns the same row on re-close,
no state mutation.
- **\`findStale\` / \`closeStale\`** — workflow-window-based stale
closure used by the scheduler.

### \`JobChargeService\` — the orchestrator (shadow variant)

\`openProcess\` resolves the effective policy via
\`PricingPolicyService\` (now in main via #6469), derives the step-limit
for the current \`JobSource\` (with a defensive fallback if the policy
is missing an entry), delegates to \`JobService.joinOrOpen\`, and on
OPENED runs the \`DocumentClassifier\` + writes a \`payg_shadow_charge\`
row. Applies the policy-level \`minChargeUnits\` floor per design § 3.4.

Shadow variant only — never debits the ledger, never posts a Stripe
meter event. The real-charging follow-up reuses the same orchestration
and swaps the side-effect. \`legacyCreditsCharged\` on the shadow row
stays \`0\` until the legacy \`CreditService\` is wired in (PR B-2),
where the comparison becomes meaningful.

### Schedulers (both plain \`@Scheduled\`)

- **\`StaleJobCloser\`** — fixed-rate 60 s. Closes \`OPEN\` jobs idle
past the workflow window. API users never have to call close explicitly
— this is the safety net.
- **\`LineagePruneScheduler\`** — hourly cron, retention 1 h. Deletes
\`job_artifact_hash\` rows older than the retention window.
- **No \`@SchedulerLock\` / no \`shedlock\` table** — consistent with
the 5 existing unguarded \`@Scheduled\` tasks in \`:saas\`
(\`CreditResetScheduler\` and friends, none of which are guarded today).
Cluster-correctness across all 7 saas schedulers is tracked in design §
9 as a separate focused cleanup. Underlying operations are idempotent —
duplicate firings on multi-pod would be wasted DB load, not data
corruption.

### Records (call-shape glue for PR B-2's filter)

- \`JobContext\` / \`JoinOrOpenResult\` — input/output for
\`JobService\`.
- \`ChargeContext\` / \`ChargeOutcome\` — input/output for
\`JobChargeService\`.
- \`JobInput\` — paired \`(MultipartFile, materialised Path)\` so the
upcoming ingress filter can pass both views without re-materialising.

## Tests

**26 new, all green.**

- 14 × \`JobServiceTest\` — no-match → opens new, single-match → joins
existing, multi-input any-match-joins, multi-match newest-wins (older
job never even looked up), step-limit hit spawns fresh job (and original
\`stepCount\` is NOT mutated), empty inputs reject, stale-signature
handling, recordOutput delegation, close idempotency, closeStale,
appendStep persistence.
- 7 × \`JobChargeServiceTest\` — JOINED skips classifier + shadow write
entirely, OPENED writes shadow row + classifies (single + multi file
paths), \`minChargeUnits\` floor applied, step-limit resolved
per-\`JobSource\` from policy, missing source entry falls back to
conservative default of 10.
- 2 × \`StaleJobCloserTest\`, 3 × \`LineagePruneSchedulerTest\` —
scheduler-wiring smoke + constructor-validation tests.

\`ENABLE_SAAS=true ./gradlew :saas:test\` — BUILD SUCCESSFUL.

## What's not in this PR (lands in PR B-2)

- **Tool ingress/egress servlet filter.** The highest-risk piece —
materialises the request body into a \`JobInput\`, calls
\`JobChargeService.openProcess\` from every \`@AutoJobPostMapping\`,
records OUTPUT after success. Edge cases to validate: multipart parts,
async controllers, streaming responses, errored 5xx paths, very large
files. Design decisions for the filter are being worked through in
\`notes/PAYG_FILTER_DESIGN.md\` before any code is written.
- **Wire shadow path into legacy \`CreditService\`.** Every legacy debit
writes a comparison row carrying both the PAYG would-be units and the
legacy actual credits, populating \`diffPct\`.

## Design doc

\`notes/PAYG_DESIGN.md\` — PR-I7 + PR-I7a status updated to reflect this
bundle. § 9 carries the cluster-correctness deferral note alongside the
existing LISTEN/NOTIFY trade-off note. § 7.5.1 readiness summary shows
the path now needs just **1 more PR** (the filter half + CreditService
wire-in, both bundled into PR B-2).
2026-06-04 11:09:25 +00:00
EthanHealy01andJames Brunton 35a712a278 smart redaction (#6195)
Co-authored-by: James Brunton <jbrunton96@gmail.com>
2026-06-03 16:16:33 +00:00
James Brunton 7f3ca7ea70 Fix mockServiceWorker.js reformatting (#6526)
# Description of Changes
`mockServiceWorker.js` is a third-party managed file, which is included
in our `.prettierignore` file, and is rewritten to be in the module's
standard format whenever `msw` runs. At some point, it was reformatted
in our style, but shouldn't have been. This puts it back to `msw`'s
style, which should make it stop appearing in diffs.
2026-06-03 15:47:03 +00:00
stirlingbot[bot]andAnthony Stirling 895dcbbafd Update Backend 3rd Party Licenses (#6407)
Co-authored-by: Anthony Stirling <anthony@stirlingpdf.com>
Signed-off-by: stirlingbot[bot] <stirlingbot[bot]@users.noreply.github.com>
2026-06-03 15:52:28 +01:00
James Brunton b705c5b84c Switch to use JPDFium v1.0.2, which signs the Mac binaries (#6521)
# Description of Changes
Currently, it's not possible to develop the backend on Mac without
manually signing the JPDFium binaries yourself since macOS will reject
running the unsigned binaries. [We've now updated JPDFium to sign the
Mac binaries in
v1.0.2](https://github.com/Stirling-Tools/JPDFium/releases/tag/v1.0.2),
so update to use that version.
2026-06-03 11:55:38 +00:00
James Brunton 1264f4cfed Set up document management for Stirling Engine (#6476)
# Description of Changes
Change Stirling Engine to support deleting documents automatically. This
happens both on user logout and after an amount of time specified by the
Java when ingesting a document (allowing for personal documents to have
short lifetimes but org documents to be left in the db with no expiry
date). Also sets up an [ACL
policy](https://en.wikipedia.org/wiki/Access-control_list) for the
documents so the database knows which users have access to which
documents. This is not fully implemented in the Java, so currently all
docs are treated as having a single owner, the uploader, but
theoretically when we need to support org storage, we shouldn't need to
change the db schema.
2026-06-03 11:52:11 +00:00
James Brunton 71633861d0 Make zoom key command behave the same regardless of mouse position (#6508)
# Description of Changes
Make zoom key command behave the same regardless of mouse position.
Previously only zoomed the editor if the mouse was over the editor.
2026-06-03 11:09:55 +00:00
ConnorYoh e6974d52f7 PAYG: hash-lineage detection primitives (modular extractor / store / detector) (#6464)
## What this is

Three orthogonal interfaces — each with one production impl — for
detecting whether an incoming tool call should join an existing process
via content-hash lineage. Groundwork for PR-I7a: nothing in this PR
calls the detector yet; the ingress/egress filter that wires it into
every controller lands separately.

Built to be modular along three axes. Swapping any of them should not
require changes elsewhere:

| Axis | Interface | V1 impl | Plausible future impl |
|---|---|---|---|
| Hash algorithm | `LineageSignatureExtractor` |
`ByteHashSignatureExtractor` (SHA-256) | `PdfMetadataSignatureExtractor`
(PDF `/ID`, content-stream hash) |
| Storage backend | `JobLineageStore` | `JpaJobLineageStore` |
`RedisJobLineageStore` |
| Matching policy | `HashLineageDetector` | `DefaultHashLineageDetector`
| strategy-driven variant (any-match-joins for multi-input — lives in
JobService) |

## Interfaces

### `LineageSignatureExtractor` — what counts as a fingerprint

```java
public interface LineageSignatureExtractor {
    Set<LineageSignature> extract(Path file) throws IOException;
    String name();
}
```

File-based (not stream-based) so a future PDF-aware extractor can open
the same file via jpdfium / PDFBox and pull `/ID[0]` or a content-stream
hash. Multiple extractors compose at the detector layer — Spring
auto-wires all `LineageSignatureExtractor` beans, the detector unions
their results.

Production impl: **`ByteHashSignatureExtractor`** — SHA-256 over the
file via 64 KiB-buffered `DigestInputStream`. Hardware-accelerated by
the JVM (Intel SHA-NI, ARM SHA).

### `JobLineageStore` — where signatures live

```java
public interface JobLineageStore {
    void record(UUID jobId, Set<LineageSignature> signatures, ArtifactKind kind);
    Optional<LineageMatch> findOpenJobForSignatures(Long userId, Set<LineageSignature> candidates, Duration window);
    int pruneOlderThan(Instant cutoff);
}
```

Knows nothing about storage technology. Production impl
**`JpaJobLineageStore`** runs a single joined query against
`job_artifact_hash` ⋈ `processing_job` — status + window filtering
happen at the database. The query is bounded by `Limit.of(1)` on the hot
path so a job set sharing a popular signature doesn't materialise
unwanted rows. A future `RedisJobLineageStore` (or write-through hybrid)
is a drop-in.

### `HashLineageDetector` — the high-level API

```java
public interface HashLineageDetector {
    Optional<LineageMatch> detect(Long userId, Path inputFile) throws IOException;
    void record(UUID jobId, Path file, ArtifactKind kind) throws IOException;
}
```

**`DefaultHashLineageDetector`** delegates extraction to every
registered `LineageSignatureExtractor`, storage to the configured
`JobLineageStore`, and reads `payg.lineage.workflow-window` (default
`PT5M`) from config. When a single extractor throws (e.g. a future
PDF-aware extractor against a malformed PDF), the other extractors still
contribute — failures don't block the byte-hash from landing.

## Profile gating

All three `@Component` beans (`JpaJobLineageStore`,
`ByteHashSignatureExtractor`, `DefaultHashLineageDetector`) are
`@Profile("saas")` — consistent with every other `:saas` bean. Without
this guard the JPA store would fail to wire against its profile-gated
repository in non-saas profiles that pull `:saas` onto the classpath.

## Tests

Run entirely in-memory; no database required.

- **`LineageSignatureTest`** — storage-key encoding round-trips, rejects
malformed `"type:value"` keys.
- **`ByteHashSignatureExtractorTest`** — identical bytes → identical
sigs; empty file hashes to the well-known SHA-256-of-empty constant; 10
MiB file streams without OOM.
- **`DefaultHashLineageDetectorTest`** — same-user / within-window /
status=OPEN filtering, multi-signature matching (one extractor sees
`pdf-id` and matches even when bytes differ), most-recent-job-wins,
record+detect round-trip, extractor-throwing-doesn't-break-others.

**`InMemoryJobLineageStore`** (in test sources) implements the same
`JobLineageStore` interface as the JPA impl, plus a `registerJob` hook
for tests to model job state. Same contract — proves the abstraction is
portable. When the Redis impl lands it gets the same contract tests.

## What's not in this PR (deliberate)

- The tool ingress/egress filter that wires the detector into every
controller — separate, focused review.
- `JobChargeService.openProcess()` — uses the detector, part of the
charging machinery, separate PR.
- Prune scheduler that calls `pruneOlderThan` — small follow-up
alongside the `shedlock` foundational table.
- PDF-aware extractor (`PdfMetadataSignatureExtractor`) — to be added
when we measure how often byte-hash-only misses real workflows.
- Multi-input "any-match-joins" lineage policy — that's a `JobService`
decision (PR-I7), not a primitive.

## Self-review pass applied

An independent code-review on this PR caught:

- **HIGH:** Missing `@Profile("saas")` on the three `@Component` beans →
fixed.
- **MEDIUM:** `pruneOlderThan` missing `@Transactional` (its
`@Modifying` query would have thrown
`InvalidDataAccessApiUsageException`) → fixed.
- **MEDIUM:** `findOpenJobsForSignatures` fetching the whole match set
just to `get(0)` → now takes `Limit`, JPA store passes `Limit.of(1)` on
the hot path.
- **LOW:** `InMemoryJobLineageStore` used both `synchronized` methods
and `ConcurrentHashMap` → dropped the redundant `ConcurrentHashMap`.

Deferred: project-wide UTC unification (`LocalDateTime.now()`
system-zone is the established convention; flipping one file mid-stack
caused a real test failure — proper fix needs its own audit).

## Rollback

Straight `git revert`. No callers yet; deleting these classes wouldn't
break anything.

---

## Checklist

- [x] Tests pass: `ENABLE_SAAS=true ./gradlew :saas:test`
- [x] No new warnings
- [x] Self-review performed (HIGH + MEDIUM findings addressed)
2026-06-03 11:00:08 +00:00
Anthony StirlingandClaude Opus 4.6 256d1a86d2 UI changes to update and support auto updating (#6075)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-06-02 23:11:37 +01:00
Reece BrowneandClaude Opus 4.8 919f0ade99 Portal (#6391)
# Description of Changes

## What & why

This PR introduces the **Stirling developer portal** — a new
control-plane frontend that sits alongside the existing PDF editor —
plus the shared design system and workspace structure needed to host
both apps in one frontend.

The portal is the parent product surface: where users connect sources,
compose pipelines, wire agents, and manage usage / billing /
infrastructure, with the PDF editor as one capability inside it. This PR
lays the **foundation** — workspace reshape, design system, app shell,
navigation, and a mock-driven home — rather than wiring real backends
(those surfaces are placeholders for follow-up phases).

## What's in this PR

**1. Frontend repo reshape (`frontend/src/` → `frontend/editor/`)**
The existing editor app moved under `frontend/editor/`, so `editor`,
`portal`, and `shared` are siblings in one workspace. All references
were updated accordingly: `LICENSE`, `.dockerignore`, `.gitignore`,
build/sign shell scripts, the GH language-check script, the Taskfile,
and Docker config. **No editor source logic changed — path references
only.**

**2. New shared design system (`frontend/shared/`)**
- **Design tokens** in `tokens.css` as the single runtime source of
truth (light/dark, category accents, gradients). `tokens.ts` now holds
only the `Tier` type — the old JS palette mirror was removed (nothing
consumed it and it had drifted).
- ~30 framework-light **components** (Card, Button, Input, Select, Tabs,
Modal, Drawer, Toast, MetricCard, StatusBadge, Skeleton, EmptyState, …)
with Storybook stories.
- **Typed data catalogues**: `endpoints.ts` (10 verticals / 64
endpoints) and `ops.ts`.

**3. New developer portal app (`frontend/portal/`)**
- App shell: `Header`, `Sidebar`, `AssistantPanel`, search modal,
notifications, tier switcher, theme toggle, MSW toggle.
- **Tier-aware** home (free / pay-as-you-go / enterprise): KPI strip,
30-day usage chart, onboarding checklist, quick actions, recent
activity, region health, product grid, and a curated **"Popular use
cases"** teaser.
- **Documents** view hosting the full, tab-filterable endpoint
catalogue.
- Placeholder views for Sources / Pipelines / Agents / Editor /
Infrastructure / Usage & Billing / Developer Docs / Settings (follow-up
phases).
- **MSW-mocked** API layer: `api/*` issues real `fetch`, intercepted by
mocks in dev/Storybook; pointing at a real backend is just a matter of
not registering MSW. `react-router` URLs; Tier / View / UI contexts.

**4. Tooling & guardrails**
- ESLint extended to `portal` + `shared`, with **layering-boundary
rules**: `shared/` may depend only on third-party packages and itself
(no `@app` / `@portal` / `@core` / `@proprietary` / Tauri), so it stays
cleanly extractable into a standalone package later.
- `dpdm` circular-dependency check now walks editor + portal + shared
(the old glob matched only 2 files).
- New **devDependencies only** — Storybook (+ a11y/docs/themes addons),
MSW. No runtime dependencies added.
- New tasks: `frontend:dev:portal`, `frontend:build:portal`.

## Testing done locally

- `tsc` for both `portal` and `shared` projects — clean
- `eslint --max-warnings=0` across the whole frontend — clean
- `dpdm` circular-dependency check — no cycles
- Editor builds clean: `vite build editor --mode core` (✓ built, only
the pre-existing >500 kB chunk-size advisory)
- Editor runs in dev (core mode) with **zero console errors**; portal
runs in dev across all three tiers

## Notes for reviewers

- The change is overwhelmingly **additive**: `shared/` and `portal/` are
brand-new; the existing editor is path-reference changes only.
- The portal is intentionally **mock-driven** at this stage — real
backends and the remaining views land in follow-up phases.

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 16:08:24 +00:00
Anthony Stirling b355ccec9e Add Valkey cluster backplane and sticky-410 ownership (clusters) (#6472) 2026-06-02 14:59:20 +01:00
Anthony Stirling de9d6ad3f5 Add CI coverage summaries and aggregate JaCoCo report (#6451) 2026-06-02 14:59:10 +01:00
brios 2c0ebc28a7 perf(api): optimize static asset caching, enable ETag support, and expand response compression mime types. (#6273) 2026-06-01 16:37:56 +01:00
Ludy d1486c7762 ci(github-actions): replace deprecated app-id input with client-id (#6485)
# Description of Changes

This change updates the GitHub App token generation step in the custom
`setup-bot` GitHub Action to use the new `client-id` input instead of
the deprecated `app-id` input when invoking
`actions/create-github-app-token`.

### What was changed

- Replaced the deprecated `app-id` parameter with `client-id` in
`.github/actions/setup-bot/action.yml`.
- Continued sourcing the value from the existing `inputs.app-id` action
input to avoid broader interface changes.

### Why the change was made

- `actions/create-github-app-token` has deprecated the `app-id` input
and now expects `client-id`.
- Updating the workflow removes the deprecation warning and ensures
compatibility with current and future versions of the action.


---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-05-31 08:07:30 +01:00
Ludy 2ccff6f73f fix(update-service): correct GitHub branch reference for version retrieval (#6333) 2026-05-30 22:44:42 +01:00
Ludy 78da227eba fix: Use frontend/editor for locales paths (#6483) 2026-05-30 22:00:17 +01:00
Anthony Stirling 30e782e29c Disable Save-to-server when storage off, fix QR port 0 (#6473) 2026-05-29 19:37:42 +01:00
Anthony Stirling 2b0905887b Add desktop multi-window support (#6463) 2026-05-29 19:35:54 +01:00
ConnorYoh 28b81828b5 PAYG: PricingPolicyService + admin REST + 30s read cache (#6469)
## What this is

PR-I1 service half from `notes/PAYG_DESIGN.md`. Built on top of the data
model from #6460 — answers "what pricing policy applies to this team
right now?" with a fast cache and an admin write surface.

## Scope

| Piece | Where |
|---|---|
| `PricingPolicyService` — `getEffectivePolicy(teamId)` with 30s
Caffeine cache + admin write paths |
`app/saas/.../payg/policy/PricingPolicyService.java` |
| `PolicyChangedEvent` — published after admin writes for in-process
cache invalidation | `app/saas/.../payg/policy/PolicyChangedEvent.java`
|
| Admin REST — list / get / create / set-default / set team override /
get effective |
`app/saas/.../payg/policy/admin/PricingPolicyAdminController.java` +
DTOs |
| `PricingPolicyRepository.clearDefaultFlag()` — atomic clear for
set-default | repository update |
| `SaasJpaConfigScanTest` — drift guard against the JPA scan paths going
stale (carried over from the #6460 review concern) | new test |
| V12 default-policy seed (`v1-initial`, 25 pages/unit, 5 MiB/unit,
per-`JobSource` step limits) | `V12__seed_default_payg_policy.sql` |

## Lookup precedence

1. `PaygTeamExtensions.pricingPolicyId` set → return that policy
2. Else return the `pricing_policy` row with `is_default = TRUE`
3. Override row points at a deleted policy → log warn, fall back to
default (safety net for racing deletes)
4. No default → `IllegalStateException` (V12 seed guarantees one exists)

## Cache behaviour

- 30s `expireAfterWrite` Caffeine, max 10k entries, keyed by `teamId`.
- **Single correctness model: the TTL.** Cross-instance propagation is
at-most-30-seconds. The writer instance sees its own change immediately
via the `PolicyChangedEvent` after-commit publish. Other instances pick
it up on the next TTL expiry.
- Admin reads use `getEffectivePolicyUncached` so admins always see
their own write straight back.

**Why no LISTEN/NOTIFY runner.** An earlier cut of this PR included a
Postgres `LISTEN policy_changed` runner so cross-instance propagation
was instant. Dropped — admin policy changes are events-per-week and the
30s TTL is already the correctness floor; the listener was ~250 lines of
nontrivial code (raw JDBC outside HikariCP, daemon thread, reconnect
loop, lock-protected connection lifecycle) for a use case that isn't on
the hot path. Trade-off is documented in `notes/PAYG_DESIGN.md` §9 with
three concrete triggers that would justify reintroducing it (aggressive
cap enforcement, Redis landing for other reasons, real-time admin UI).

## Writes — transactional, fire `PolicyChangedEvent` after commit

- `create(draft)` — rejects pre-set `policy_id` or `is_default=true`
(promotion must go through `setDefault` so the partial unique idx is
freed first).
- `setDefault(id)` — atomically clears the existing default via
`clearDefaultFlag()` then flips the new row. Idempotent: silent no-op if
the row is already default.
- `setTeamOverride(teamId, policyId | null)` — validates the policy
exists before save; `null` clears the override.

`publishOnCommit` uses `TransactionSynchronizationManager.afterCommit`
so listeners never see pre-commit state. Outside a transaction (test
paths) falls through to immediate publish.

## Admin REST surface — `/api/v1/admin/payg/...`

All endpoints `@PreAuthorize("hasRole('ADMIN')")`:

- `GET  /policies` — list all
- `GET  /policies/{id}` — read one
- `POST /policies` — create new (non-default)
- `POST /policies/{id}/set-default` — atomic promote
- `PUT /teams/{teamId}/policy-override` — set or clear per-team override
- `GET  /teams/{teamId}/effective-policy` — cache-bypassing live read

Validation errors → 400, unknown rows → 404.

## Counterpart Supabase PR


[`Stirling-PDF-SaaS#298`](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/298)
— seeds the same V1 default policy on the Supabase side via
`20260528000002_payg_seed_default_policy.sql`.

## Tests

- 17 × `PricingPolicyServiceTest` — lookup precedence, cache hit/miss,
invalidation on event, mutation paths publishing event, error cases.
- 14 × `PricingPolicyAdminControllerTest` — every endpoint's happy path
+ error mapping, DTO defensive-copy invariant.
- 2 × `SaasJpaConfigScanTest` — reflection-based guard that
`payg.repository` is in `@EnableJpaRepositories` and `payg` is in
`@EntityScan`. Without this, new sub-packages can silently fail to wire
at runtime — same class of bug that the #6460 review caught.

Full `:saas:test` BUILD SUCCESSFUL.

## Design doc

`notes/PAYG_DESIGN.md` §7.4 PR-I1 — completes the service half (the
schema half landed in #6460). §9 carries the 30s-TTL trade-off note.
2026-05-29 16:27:01 +00:00
James Brunton 2c01f41142 Update indexeddb to v9 to unify SaaS and OSS users (#6474)
# Description of Changes
The production SaaS is currently on v8 of IndexedDB due to various
schema changes for Smart Folders, which haven't made their way into OSS.
OSS is currently on v4 of IndexedDB, so if we release an OSS build to
the SaaS deployment, existing users will not be able to use it because
the DB version is 'too old'.

This PR updates the IDB version number to v9 so both OSS and SaaS users
will be able to upgrade to it. Theoretically both types of user should
be able to keep their IDB files without issue. SaaS previously actively
wiped the user's files in an old version (v6/v7) and users who haven't
used it since then will have their DBs wiped, but that'd happen anyway
if they use current SaaS so I don't think that matters.
2026-05-29 15:23:40 +00:00
James Brunton 4d5eeb103f Fix username display issues (#6471)
# Description of Changes
Main fixes:
- Fix the display of the username in the bottom left
- Now displays as "User" when not logged in on self-hosted (desktop) and
"Guest" on SaaS when logged in anonymously
- Now updates properly when the user logs in/out in SaaS, desktop and
self-hosted
- Fix incremental build issues in the desktop app that have been here
since the start (I hope at least - I think the issue is that the JLink
is built read-only and then on subsequent builds you get OS errors when
trying to override the JLink with the new version. There's no real need
for it to be read-only that I know of, so we might as well just make it
R/W and ship like that)
2026-05-29 14:35:47 +00:00
ConnorYoh 83ea07ed6a saas: DocumentClassifier + PAYG data model (#6460)
# Description of Changes

Two layers — the `DocumentClassifier` utility plus the full data model
for the new billing engine. Nothing wires the entities into application
behaviour yet; services and controllers land in follow-up PRs.

**Companion PR:**
[Stirling-PDF-SaaS#296](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/296)
— Supabase migration for the v3 dev branch, schema-equivalent to the
Flyway migration in this PR.

## 1. DocumentClassifier (under `payg.docs`)

`DocumentClassifier` computes the doc-unit cost of an uploaded file (or
multi-file input) under a `PricingPolicy`. PDFs read page count via
`stirling.software.jpdfium.PdfDocument`; non-PDFs are bytes-only.
Formula: `max(ceil(pages / docPagesPerUnit), ceil(bytes /
docBytesPerUnit))` clamped to `[1, fileUnitCap]`. Multi-file is the sum
of raw per-file units capped at `fileUnitCap × file_count`.

Two floors, by design: the classifier returns `docUnits` with an
absolute `1` floor for non-empty input; the policy-level
`minChargeUnits` is intentionally applied later, at process-open time in
`JobChargeService`, per design § 3.4 (`unitsForProcess =
max(policy.min_charge_units, docUnits)`). Documented in the interface +
impl javadoc.

Upload bytes are materialised through
`TempFileManager.createManagedTempFile` so jpdfium gets a `Path`; the
temp file auto-deletes on close.

Twelve tests, all in-memory fixtures generated with PDFBox at test time
— no committed binary blobs.

## 2. PAYG data model (under `payg.*`)

JPA entities, repositories, and a Flyway migration covering the full
schema in §6 of the design.

**Enums** (`payg.model`):

`JobSource`, `ProcessType`, `JobStatus`, `JobStepStatus`,
`ArtifactKind`, `LedgerEntryType`, `LedgerBucket`, `ReferenceType`,
`EntitlementState`, `FeatureSet`, `FeatureGate`, `WalletEngine`,
`CapPeriod`, `AutoGroupStrategy`.

**Entities + repositories:**

| Entity | Table | Notes |
|---|---|---|
| `PricingPolicy` | `pricing_policy` | Promoted from a record.
`stepLimits` is `Map<JobSource, Integer>` persisted via normalised child
table `pricing_policy_step_limit`. `stripePriceIds` is `Set<String>`
persisted via `pricing_policy_stripe_price` — currency comes from
`stripe.prices` via Sync Engine, not stored locally. |
| `ProcessingJob` | `processing_job` | UUID PK. Tracks lineage window
via `step_count` and `last_step_at`. |
| `ProcessingJobStep` | `processing_job_step` | Per-tool-call audit. |
| `JobArtifactHash` | `job_artifact_hash` | Composite key `(job_id,
content_hash, kind)`. `content_hash VARCHAR(128)` so multiple signature
schemes coexist as `"type:value"` storage keys. Lineage detector queries
this. |
| `WalletLedgerEntry` | `wallet_ledger` | Append-only, signed
`amount_units`. Two unique indexes kill double-posting. |
| `WalletPolicy` | `wallet_policy` | Per-team engine + cap + degradation
rules + lineage strategy. No `@Version` — admin-only writes (documented
in javadoc). |
| `WalletEntitlementSnapshot` | `wallet_entitlement_snapshot` |
Composite key `(team_id, user_id)`; `user_id = 0` is the team-wide
sentinel. No `@Version` — full-row recompute via
`EntitlementService.recompute` (documented in javadoc). |
| `PaygShadowCharge` | `payg_shadow_charge` | Per-job diff while in
`PAYG_SHADOW` engine mode. |
| `PaygTeamExtensions` | `payg_team_extensions` | Sidecar 1:1 with
`teams` carrying `pricing_policy_id` (per-team override) +
`stripe_customer_id`. Sidecar pattern (mirrors `saas_team_extensions`)
so OSS Hibernate ddl-auto never sees PAYG columns on `teams`. |

**Column adds:**

- `team_memberships.cap_units` (optional per-member sub-cap)

**Width split (intentional, documented in V11):** per-row deltas
(`wallet_ledger.amount_units`, `processing_job.charged_units`) are
`INTEGER` because no single charge realistically approaches 2B units.
Cap and period-rollup columns (`team_memberships.cap_units`,
`wallet_policy.cap_units`,
`wallet_entitlement_snapshot.period_spend_units / period_cap_units`) are
`BIGINT` because they accumulate across a billing period and admins may
legitimately set headroom-cap values into the millions.

**JPA wiring:** `SaasJpaConfig` was updated to include
`stirling.software.saas.payg.repository` in
`@EnableJpaRepositories.basePackages` and `stirling.software.saas.payg`
in `@EntityScan` (covers `payg.policy` / `payg.job` / `payg.wallet` /
`payg.entitlement` / `payg.shadow` recursively). New
`SaasJpaConfigScanTest` reads the annotations reflectively and asserts
every expected package is wired — catches the next time someone adds a
new sub-package without updating the scan paths.

**Migration:** `V11__saas_payg_model.sql` (purely additive).
Schema-equivalent to the Supabase migration in the companion PR —
including the `VARCHAR(128) content_hash` width that's needed for the
multi-signature-scheme storage encoding the lineage layer uses.

## 3. Smoke tests

`PaygEntitiesSmokeTest` exercises each entity via the no-arg ctor JPA
requires, plus getter/setter round-trips and composite-key equality —
catches Lombok/annotation regressions without needing a database.
Real-DB integration coverage lands alongside the services that consume
each entity.

## Why this is safe to land now

- All schema changes are additive — no existing rows modified, no
columns dropped.
- The entities are not yet referenced from any production code path;
they exist for the next PRs to build on.
- The v3 Supabase dev branch picks up the schema via the companion PR;
the main repo's Flyway migration applies the same shape when an instance
boots against a freshly-migrated v3 database.

## Open decisions made

- **Step-limits keyed by `JobSource`** rather than by `ProcessType`.
Captures the "self-hosted gets a different knob" framing in earlier
feedback. Trivially overridable per pricing policy version.
- **Step limits + Stripe price IDs normalised into child tables** rather
than JSONB on `pricing_policy` (per Connor's review on #296). Typed
columns, queryable directly, no JSON parsing.
- **Currency dropped from `pricing_policy_stripe_price`** — it lives on
`stripe.prices.currency` and is resolved via Sync Engine. App is
currency-blind.

## Rollback

Straight `git revert` on this PR. The Supabase migration in #296 is
additive and can be left in place safely — the running app ignores
tables it doesn't reference.

---

## Checklist

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings
- [x] I have run `task check` (via `./gradlew :saas:test` with
`ENABLE_SAAS=true`) — passes
2026-05-29 12:03:01 +00:00
James Brunton 61ebe977d3 Auto-delete CI linting comments on success (#6465)
# Description of Changes
Set CI backend & engine comments to auto-delete once the CI has passed. 

Also redesign the engine CI to call `task engine:check` like it should
have been, and make it post a comment when the tool models need to be
updated.

Also makes the comment wording more consistent between the three
languages.
2026-05-29 10:13:12 +00:00
EthanHealy01 763595a5a3 feat: add Agents UI to proprietary right sidebar (#6454)
Update UI to include agents

Run `task dev:all` to test
2026-05-28 17:26:23 +00:00
Anthony Stirling 398617391b Fix SSO auto-login and custom metadata settings not persisting on restart (#6468) 2026-05-28 17:39:05 +01:00
ConnorYoh a0e0e88f07 saas: harden CreditService Stripe ordering + lint @AutoJobPostMapping weights (#6458)
# Description of Changes

Two narrowly-scoped hardening changes to the credits engine.

## 1. CreditService — move Stripe meter call to `afterCommit`

The Stripe metered-usage call sits inside the surrounding
`@Transactional`, holding the `user_credits` row lock for the duration
of an HTTP round-trip to Supabase. Under load this starves concurrent
debits; a transient Stripe blip rolls back a (correct) free-credit
consumption and forces the caller to retry.

The Stripe call now runs in a `TransactionSynchronization.afterCommit`
hook — DB commits first, Stripe fires immediately after. If Stripe fails
after commit, we log + increment a new `credits.stripe_report.failures`
counter; the idempotency key is stable, so a manual replay recovers
without double-charging.

Applied to both `consumeCreditBySupabaseId` and
`consumeCreditWithWaterfall`.

**Dead-code removed:**
- Unreachable UUID fallback for MDC `requestId` — `CorrelationIdFilter`
already guarantees the key on every request.
- The `"Unable to report usage to Stripe"` `RuntimeException` and its
catch block — the afterCommit refactor eliminates the throw path.
- `StripeRollbackOnFailureTest` — pinned the rollback-on-Stripe-fail
behaviour this refactor replaces.

## 2. `@AutoJobPostMapping` — build-time lint for `resourceWeight`

`UnifiedCreditInterceptor` multiplies `resourceWeight` into the per-call
charge. An endpoint that falls through to the annotation default
produces a charge derived from a value nobody chose.

- Annotation default flipped from `1` to `Integer.MIN_VALUE` (sentinel).
Both runtime readers (`UnifiedCreditInterceptor`, `AutoJobAspect`)
already clamp into `[1, 100]` so behaviour is unchanged.
- New `AutoJobPostMappingWeightTest` scans the classpath and fails the
build if any method leaves the sentinel.
- Initial run caught 11 endpoints relying on the default. Explicit
weights now declared, chosen by comparing to peer endpoints:
  - `EditTextController` — LARGE
  - `EmailController#sendEmailWithAttachment` — SMALL
  - `ConvertPDFToMarkdown` — MEDIUM
  - `AttachmentController` (extract/list/rename/delete) — SMALL × 4
  - `ConvertImgPDFController` (cbr/cbz ↔ pdf) — MEDIUM × 2, LARGE × 2

## Tests

- `StripeUsageIdempotencyKeyTest` — pins the `(supabaseId, overage,
requestId)` idempotency key shape so Stripe always dedupes a retry.
- `StripeAfterCommitOrderingTest` — pins that `afterCommit` fires after
commit and NOT on rollback.
- `AutoJobPostMappingWeightTest` — the lint itself, plus a self-check
that the classpath scan finds at least 10 `@AutoJobPostMapping` methods
(guards against the lint passing vacuously).

Build verified: `ENABLE_SAAS=true ./gradlew :stirling-pdf:test
:saas:test`.

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable) — no translation changes
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
— internal-billing change, no public docs impact
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
— N/A

### Translations (if applicable)

- [ ] Not applicable

### UI Changes (if applicable)

- [ ] Not applicable

### Testing (if applicable)

- [x] I have run `task check` (via `./gradlew :stirling-pdf:test
:saas:test` with `ENABLE_SAAS=true`) — passes
- [x] I have tested my changes locally
2026-05-28 14:57:59 +00:00
Anthony Stirling c80a5db5f5 folder and file fixes (#6461) 2026-05-28 15:57:35 +01:00
Anthony Stirling 4fa67afc3d Fix Tauri artifact copy path so installers upload (smoke + release) (#6466)
## Summary
Regression from #6404 (Restructure/frontend editor). Two CI workflows
copy the built installers to the wrong directory, so installer artifacts
(MSI / DMG / DEB / RPM / AppImage) silently vanish:

- **`tauri-build.yml`** (PR/desktop smoke builds) - uploads zero
installer artifacts.
- **`multiOSReleases.yml`** (production releases) - the empty artifacts
are downloaded by `create-release` and fed to `action-gh-release`, so a
release would publish **only the JARs, no desktop installers**.

## Root cause
#6404 moved the Tauri project from `frontend/` to `frontend/editor/` and
updated every **absolute** path (`projectPath`, `cd`, `Get-ChildItem`)
to add the `editor/` segment - but left the **relative** copy targets
`../../../dist`. Those resolve against the (now one level deeper)
working dir after `cd ./frontend/editor/src-tauri/target`:

| | resolves to |
|---|---|
| before #6404 (`frontend/src-tauri/target`) | repo-root `dist/`  |
| after #6404 (`frontend/editor/src-tauri/target`) | `frontend/dist/` 
(missing) |

The `cp` fails, repo-root `dist/` (from `mkdir -p ./dist`) stays empty,
and the upload finds nothing. `find -exec cp` failing is non-fatal, so
jobs still report success - that's why it went unnoticed. No release has
shipped broken yet: the last release (v2.11.0, 2026-05-19) predates
#6404 (2026-05-22).

## Fix
Copy to an absolute `$GITHUB_WORKSPACE/dist` in both workflows so the
`cd` can't drift the destination again. This matches where the upload /
signature-verify steps already read from.

## Evidence (run 26574078559, all 3 OS legs)
```
cp: cannot create regular file '../../../dist/Stirling-PDF-windows-x86_64.msi': No such file or directory
##[warning]No files were found with the provided path: ./dist/*. No artifacts will be uploaded.
```
The Tauri builds themselves succeeded - only the copy/upload was broken.

## Test plan
- [ ] `tauri-build` on this PR uploads non-empty `Stirling-PDF-<name>`
artifacts on Windows/macOS/Linux.
- [ ] Next release (or a `workflow_dispatch` of multiOSReleases)
attaches MSI/DMG/DEB/RPM/AppImage to the release.
2026-05-28 15:57:01 +01:00
Anthony StirlingandConnorYoh 8bd78d2624 Add landscape page size options (#6248)
# Description of Changes

Adds orientation (portrait/landscape) to the Adjust Page Scale tool.

- Orientation as a separate parameter (per review), sent through to the
backend
- ScalePagesController simplified; PDFWithPageSize gains the orientation
field
- Regenerated tool_models.py; frontend + backend tests added

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

---------

Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
2026-05-28 14:16:15 +00:00
Anthony Stirling b3c4b8b463 Add S3 storage and cluster artifact backend (#6457) 2026-05-28 13:06:27 +01:00
James Brunton 57af5b9dc2 Fix Tauri testing (#6462)
# Description of Changes
#6402 introduced a Rust test `refresh_token_fallback.rs`, but it wasn't
moved properly after the restructure of the `frontend/` folder in #6404.
This PR moves the file to the right place, and also hooks up Task and CI
rules for `cargo test` since nothing was actually running the test in
the first place.
2026-05-28 11:05:56 +00:00
James Brunton 44fbf8c587 Various bug fixes found while testing SaaS build (#6459)
# Description of Changes
Various fixes and improvements I made while testing the SaaS code:
- Changes the new `.env.saas` file to live in `app/` and match the
semantics of the other `.env` files
- Adds top-level `task dev:saas` command to spawn SaaS frontend &
backend
- Deletes dead SaaS code and improves some overriding logic
- Fixes refreshing issue when coming back to the tab
- Fix the Compare tool's selection logic
- Make Compare handle error cases properly
- Fixes the location of the "Dismiss All Errors" button (was rendering
on top of the top-bar with a transparent background previously so it
looked rubbish)
- Fixes file selection in PDF Editor
2026-05-28 11:05:30 +00:00
Anthony Stirling 76840d8a57 Add CI DB migration smoke test against v2.0/v2.5/v2.10 updates (#6453) 2026-05-28 11:36:07 +01:00
James Brunton d459ded168 Add cancel button to kill long-running AI tasks (#6351)
# Description of Changes
Adds a cancel button to the AI chat to allow the user to abort
long-running AI tasks. Just disconnects the SSE stream (all the backend
code already interrupts when it notices the stream is dead).
2026-05-28 09:25:23 +00:00
ConnorYoh 43b67d213d feat(oauth2): opt-in claim-dump diagnostics for OIDC login failures (#6456)
# Description of Changes

## What & why

Customers using ADFS (or any generic OIDC provider that doesn't emit
`email`) hit `Attribute value for 'email' cannot be null` during OAuth2
login with no visibility into what claims the provider actually sent.
The only available remedy was guessing at
`security.oauth2.useAsUsername` until something worked.

This PR adds a new opt-in `security.oauth2.debugLogging` flag (default
`false`). When enabled, `CustomOAuth2UserService` logs:

- All ID token claims (sorted, with values)
- All UserInfo endpoint claims (if any)
- The merged attribute key set Spring exposes to `getAttribute()`
- The value the configured `useAsUsername` actually resolved to
- A **`Hint:`** line listing the claim keys present in the token that
map to a valid `UsernameAttribute` enum value — i.e. exactly what the
operator could put in `useAsUsername` to make login work

Logged at `INFO` on the success path and `ERROR` on failure (inside the
existing `catch (IllegalArgumentException)` block that throws
`OAuth2AuthenticationException`). The block is wrapped with a `[OAUTH2
DEBUG] ... [/OAUTH2 DEBUG]` banner and ends with a PII warning so
operators don't leave it on in production.

Default off → zero observable change for anyone not actively
troubleshooting.

## Files changed

| File | Why |
|---|---|
| `app/common/.../ApplicationProperties.java` | New `debugLogging` field
on the `OAUTH2` config class with javadoc warning about PII |
| `app/core/src/main/resources/settings.yml.template` | Documents
`oauth2.debugLogging` so it appears on next startup |
| `app/proprietary/.../security/service/CustomOAuth2UserService.java` |
Emits the claim dump + suggestion hint when the flag is on |
|
`app/proprietary/.../security/service/CustomOAuth2UserServiceDebugLoggingTest.java`
(new) | Unit test: mocks the OIDC delegate, asserts off-path is silent
and on-path emits the dump with the right Hint contents |

## End-to-end verification

Ran the bundled `testing/compose/docker-compose-keycloak-oauth.yml`
Keycloak realm, configured `security.oauth2.useAsUsername: mail`
(Keycloak emits `email`, not `mail`) and `provider: demarest` (matches
the original customer bug report). Triggered the OAuth flow at
`http://localhost:8080/oauth2/authorization/demarest` and confirmed:

- The ERROR-level dump fires with the full 19-claim ID token decoded
- `-- Value at 'mail' : <NULL — this is why login fails>` correctly
identifies the missing claim
- `-- Hint:` correctly suggests `[email, family_name, given_name,
preferred_username]` (the four keys present that map to valid
`UsernameAttribute` values)
- Auth still fails with the original `OAuth2AuthenticationException` —
no change to control flow, just added diagnostic logging

Unit test (`CustomOAuth2UserServiceDebugLoggingTest`) covers both
branches.

## Reviewer notes

- **No new public APIs.** The flag is config-only; no servlet endpoints
exposed.
- **PII is logged when the flag is on.** This is the whole point —
operators need to see the claims to fix their config — but it's gated,
defaults off, and the dump self-documents with a `WARNING: ... Set
security.oauth2.debugLogging=false once troubleshooting is complete.`
footer.
- **Why log everything, not just sub/email?** Because the operator
doesn't know in advance which claim they actually want. ADFS uses `upn`
in some configs and `preferred_username` in others; Azure AD uses `oid`;
the customer here had neither. Dumping the full set is the only way to
make the diagnostic self-service.
- **Out of scope for this PR (follow-ups):**
- The `UsernameAttribute` enum doesn't include `upn` / `unique_name`
(common ADFS claims). If the customer's token only has `upn`, the Hint
will be empty even though the operator can see `upn` in the dump. Worth
a separate PR to extend the enum.
- The known-provider validator in `Provider.java` (rejects e.g.
`useAsUsername: mail` for `provider: keycloak` at startup) bypasses our
diagnostic for those provider names. ADFS customers using `provider:
<name>` fall into the `default` branch so are not affected — but it's a
sharp edge worth documenting.

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable) — N/A, backend-only change
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] Doc-repo update (if functionality has heavily changed) —
diagnostic flag is self-documenting via the `settings.yml.template`
comment and the in-log warning; happy to add a doc-repo entry if
reviewers want one
- [ ] Translation tags — N/A

### UI Changes (if applicable)

- [ ] N/A — backend-only

### Testing (if applicable)

- [x] Unit test added (`CustomOAuth2UserServiceDebugLoggingTest`)
covering on/off paths and Hint correctness
- [x] End-to-end verified locally against bundled Keycloak compose with
intentionally misconfigured `useAsUsername`
- [x] Full `:proprietary:test` suite passes
2026-05-27 13:01:51 +00:00
Anthony Stirling d42b779644 Add server-side folders and files page UI (#6383) 2026-05-27 12:52:46 +01:00
906 changed files with 103248 additions and 7383 deletions
+3 -2
View File
@@ -26,8 +26,9 @@ version_builds/
node_modules/
**/node_modules/
frontend/node_modules/
frontend/dist/
frontend/playwright-report/
frontend/editor/dist/
frontend/dist-portal/
frontend/editor/playwright-report/
.npm/
.yarn/
+1 -1
View File
@@ -24,7 +24,7 @@ runs:
id: generate-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ inputs.app-id }}
client-id: ${{ inputs.app-id }}
private-key: ${{ inputs.private-key }}
- name: Configure Git
run: |
+1 -1
View File
@@ -1,6 +1,6 @@
# Maintainer: Stirling PDF Inc <contact@stirlingpdf.com>
pkgname=stirling-pdf-desktop
pkgver=2.11.0
pkgver=2.12.0
pkgrel=1
pkgdesc="Locally hosted, web-based PDF manipulation tool (Tauri desktop app, official Stirling PDF Inc build)"
arch=('x86_64')
+1 -1
View File
@@ -1,6 +1,6 @@
# Maintainer: Stirling PDF Inc <contact@stirlingpdf.com>
pkgname=stirling-pdf-server-bin
pkgver=2.11.0
pkgver=2.12.0
pkgrel=1
pkgdesc="Locally hosted, web-based PDF manipulation tool (server JAR, prebuilt)"
arch=('any')
+2
View File
@@ -38,6 +38,8 @@ project: &project
- frontend/**
- docker/**
- scripts/RestartHelper.java
- scripts/db-migration/**
- .github/workflows/db-migration-test.yml
frontend: &frontend
- frontend/**
+4 -3
View File
@@ -13,7 +13,7 @@ Usage:
"""
# Sample for Windows:
# python .github/scripts/check_language_toml.py --reference-file frontend/public/locales/en-GB/translation.toml --branch "" --files frontend/public/locales/de-DE/translation.toml frontend/public/locales/fr-FR/translation.toml
# python .github/scripts/check_language_toml.py --reference-file frontend/editor/public/locales/en-GB/translation.toml --branch "" --files frontend/editor/public/locales/de-DE/translation.toml frontend/editor/public/locales/fr-FR/translation.toml
import argparse
import glob
@@ -184,7 +184,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
if len(file_list) == 1:
file_arr = file_list[0].split()
base_dir = Path.cwd() / "frontend" / "public" / "locales"
base_dir = Path.cwd() / "frontend" / "editor" / "public" / "locales"
for file_path in file_arr:
file_path = Path(file_path)
@@ -308,7 +308,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
report.append("## ❌ Overall Check Status: **_Failed_**")
report.append("")
report.append(
f"@{actor} please check your translation if it conforms to the standard. Follow the format of [en-GB/translation.toml](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/frontend/public/locales/en-GB/translation.toml)"
f"@{actor} please check your translation if it conforms to the standard. Follow the format of [en-GB/translation.toml](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/frontend/editor/public/locales/en-GB/translation.toml)"
)
else:
report.append("## ✅ Overall Check Status: **_Success_**")
@@ -372,6 +372,7 @@ if __name__ == "__main__":
os.path.join(
os.getcwd(),
"frontend",
"editor",
"public",
"locales",
"*",
@@ -0,0 +1,83 @@
#!/usr/bin/env python3
"""Verify Tauri updater .sig files against plugins.updater.pubkey in tauri.conf.json.
Usage: verify-updater-signatures.py <dir-to-scan> [tauri.conf.json]
"""
import binascii
import sys
import json
import base64
import hashlib
from pathlib import Path
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
from cryptography.exceptions import InvalidSignature
ART_ROOT = Path(sys.argv[1])
CONF = Path(
sys.argv[2] if len(sys.argv) > 2 else "frontend/editor/src-tauri/tauri.conf.json"
)
def load_pubkey():
# tauri pubkey = base64 of a minisign .pub file; last line is base64 of
# [2 algo][8 key-id][32 ed25519 public key].
raw = json.loads(CONF.read_text())["plugins"]["updater"]["pubkey"]
blob = base64.b64decode(base64.b64decode(raw).decode().splitlines()[-1])
return blob[2:10], Ed25519PublicKey.from_public_bytes(blob[10:])
def hash_file(path: Path) -> bytes:
h = hashlib.blake2b(digest_size=64)
with path.open("rb") as f:
for chunk in iter(lambda: f.read(1 << 16), b""):
h.update(chunk)
return h.digest()
def verify(artifact: Path, sig_file: Path, keyid_pub, pub) -> str:
# tauri .sig = base64 of a minisign signature file (4 lines).
try:
lines = base64.b64decode(sig_file.read_text()).decode().splitlines()
sig_blob = base64.b64decode(lines[1])
except (binascii.Error, IndexError, UnicodeDecodeError) as e:
return f"FAIL malformed sig ({type(e).__name__})"
algo, keyid, sig = sig_blob[:2], sig_blob[2:10], sig_blob[10:74]
if keyid != keyid_pub:
return f"FAIL key-id mismatch (sig {keyid.hex()} vs pub {keyid_pub.hex()})"
# 'ED' = prehashed (BLAKE2b-512), 'Ed' = legacy (raw message).
msg = hash_file(artifact) if algo == b"ED" else artifact.read_bytes()
try:
pub.verify(sig, msg)
except InvalidSignature:
return f"FAIL signature invalid (algo={algo.decode()})"
# Global signature covers sig + trusted_comment.
gc = "global-sig FAIL"
try:
tc = lines[2].split("trusted comment: ", 1)[1]
pub.verify(base64.b64decode(lines[3]), sig + tc.encode())
gc = "global-sig OK"
except (InvalidSignature, IndexError, binascii.Error):
pass
return f"VALID (algo={algo.decode()}, keyid={keyid.hex()}, {gc})"
keyid_pub, pub = load_pubkey()
print(f"updater pubkey keyid={keyid_pub.hex()}\n")
sigs = sorted(ART_ROOT.rglob("*.sig"))
if not sigs:
print(f"WARN: no .sig files under {ART_ROOT} - nothing to verify")
sys.exit(0)
bad = 0
for sig_file in sigs:
artifact = sig_file.with_suffix("")
if not artifact.exists():
print(f" ? {sig_file.name}: artifact missing")
bad += 1
continue
res = verify(artifact, sig_file, keyid_pub, pub)
print(f" {artifact.name}: {res}")
if not res.startswith("VALID") or "global-sig FAIL" in res:
bad += 1
print(f"\n{'ALL SIGNATURES VALID' if bad == 0 else f'{bad} SIGNATURE(S) FAILED'}")
sys.exit(1 if bad else 0)
+2 -1
View File
@@ -287,6 +287,7 @@ jobs:
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/data:/usr/share/tessdata:rw
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/config:/configs:rw
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/logs:/logs:rw
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/storage:/storage:rw
environment:
DISABLE_ADDITIONAL_FEATURES: "false"
SECURITY_ENABLELOGIN: "true"
@@ -309,7 +310,7 @@ jobs:
ssh -i ../private.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -T ${{ secrets.NEW_VPS_USERNAME }}@${{ secrets.NEW_VPS_HOST }} << ENDSSH
# Create V2 PR-specific directories
mkdir -p /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/{data,config,logs}
mkdir -p /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/{data,config,logs,storage}
# Move docker-compose file to correct location
mv /tmp/docker-compose-v2.yml /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/docker-compose.yml
+124 -27
View File
@@ -1,8 +1,9 @@
name: AI Engine CI
# Validates the Python AI engine: regenerates tool models, runs fixers,
# lint, type-check, and tests. Called from build.yml on PRs and merge_group;
# also runs directly on push to main as a post-merge safety net.
# Validates the Python AI engine: regenerates tool models and runs the
# engine quality gate (lint, type-check, format-check, tests). Called from
# build.yml on PRs and merge_group; also runs directly on push to main as
# a post-merge safety net.
on:
workflow_call:
push:
@@ -51,27 +52,95 @@ jobs:
run: task engine:tool-models
- name: Verify tool models are up to date
id: tool-models-check
continue-on-error: true
run: git diff --exit-code engine/src/stirling/models/tool_models.py
- name: Comment on tool models check failure
# Only post a comment on PRs. github-script's PR helpers need an
# issue/PR number, which doesn't exist on merge_group runs.
if: steps.tool-models-check.outcome == 'failure' && github.event_name == 'pull_request'
continue-on-error: true
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const marker = '<!-- tool-models-check -->';
const body = [
marker,
'### Tool Models Check Failed',
'',
'The generated `engine/src/stirling/models/tool_models.py` is out of date with the Java OpenAPI spec and will need to be regenerated before it can be merged in.',
'',
'Run `task engine:tool-models` to regenerate, then commit the updated file.',
].join('\n');
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body.includes(marker));
if (existing) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
body,
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body,
});
}
- name: Fail if tool models check failed
if: steps.tool-models-check.outcome == 'failure'
run: |
if ! git diff --exit-code engine/src/stirling/models/tool_models.py; then
echo "tool_models.py is out of date."
echo "Run 'task engine:tool-models' locally and commit the updated file."
exit 1
fi
echo "============================================"
echo " Tool Models Check Failed"
echo "============================================"
echo ""
echo "The generated engine/src/stirling/models/tool_models.py"
echo "is out of date with the Java OpenAPI spec and will"
echo "need to be regenerated before it can be merged in."
echo ""
echo "Run 'task engine:tool-models' to regenerate, then"
echo "commit the updated file."
echo "============================================"
exit 1
- name: Run fixers
run: task engine:fix
- name: Remove tool models check comment on success
if: steps.tool-models-check.outcome == 'success' && github.event_name == 'pull_request'
continue-on-error: true
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const marker = '<!-- tool-models-check -->';
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body.includes(marker));
if (existing) {
await github.rest.issues.deleteComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
});
}
- name: Verify fixes are committed
id: fixer_changes
run: |
if ! git diff --quiet; then
git --no-pager diff --stat
echo "::error::There are issues with your Python code that will need to be fixed before they can be merged in. Run 'task engine:fix' to auto-fix what can be fixed automatically, then run 'task engine:check' to see what still needs fixing manually."
exit 1
fi
- name: Quality-check engine
id: engine-check
run: task engine:check
continue-on-error: true
- name: Comment on fixer failures
if: steps.fixer_changes.outcome == 'failure' && github.event_name == 'pull_request'
- name: Comment on engine check failure
# Only post a comment on PRs. github-script's PR helpers need an
# issue/PR number, which doesn't exist on merge_group runs.
if: steps.engine-check.outcome == 'failure' && github.event_name == 'pull_request'
continue-on-error: true
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
@@ -107,11 +176,39 @@ jobs:
});
}
- name: Run linting
run: task engine:lint
- name: Fail if engine check failed
if: steps.engine-check.outcome == 'failure'
run: |
echo "============================================"
echo " Engine Check Failed"
echo "============================================"
echo ""
echo "There are issues with your Python code that"
echo "will need to be fixed before they can be merged in."
echo ""
echo "Run 'task engine:fix' to auto-fix what can be"
echo "fixed automatically, then run 'task engine:check'"
echo "to see what still needs fixing manually."
echo "============================================"
exit 1
- name: Run type checking
run: task engine:typecheck
- name: Run tests
run: task engine:test
- name: Remove engine check comment on success
if: steps.engine-check.outcome == 'success' && github.event_name == 'pull_request'
continue-on-error: true
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const marker = '<!-- engine-check -->';
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body.includes(marker));
if (existing) {
await github.rest.issues.deleteComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
});
}
+98 -22
View File
@@ -1,8 +1,14 @@
name: Backend build, format check, and coverage
# Reusable workflow called from build.yml. Runs the backend build matrix
# (JDK 25 × spring-security on/off), Spotless formatting check, JUnit, and
# (JDK 25 × every flavor), Spotless formatting check, JUnit, and
# posts Jacoco coverage to PRs.
#
# Flavor axis (maps to STIRLING_FLAVOR in settings.gradle):
# core - DISABLE_ADDITIONAL_FEATURES=true, no proprietary, no saas
# proprietary - default build, no saas
# saas - proprietary + the saas subproject (build + JUnit only,
# never any runtime/integration testing)
on:
workflow_call:
@@ -25,7 +31,7 @@ jobs:
fail-fast: false
matrix:
jdk-version: [25]
spring-security: [true, false]
flavor: [core, proprietary, saas]
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -58,7 +64,10 @@ jobs:
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Check Java formatting (Spotless)
if: matrix.jdk-version == 25 && matrix.spring-security == false
# Runs once per matrix combination - pick the cheapest leg
# (core - no proprietary, no saas) so we don't wait for the
# heavier flavors just to fail formatting.
if: matrix.jdk-version == 25 && matrix.flavor == 'core'
id: spotless-check
run: task backend:format:check
continue-on-error: true
@@ -67,7 +76,7 @@ jobs:
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
- name: Comment on Java formatting failure
- name: Comment on backend format check failure
# Only post a comment on PRs. github-script's PR helpers need an
# issue/PR number, which doesn't exist on merge_group runs.
if: steps.spotless-check.outcome == 'failure' && github.event_name == 'pull_request'
@@ -78,15 +87,11 @@ jobs:
const marker = '<!-- java-formatting-check -->';
const body = [
marker,
'### Java Formatting Check Failed',
'### Backend Format Check Failed',
'',
'Your code has formatting issues. Run the following command to fix them:',
'There are formatting issues in your Java code that will need to be fixed before they can be merged in.',
'',
'```bash',
'task backend:format',
'```',
'',
'Then commit and push the changes.',
'Run `task backend:format` to auto-fix, then commit and push the changes.',
].join('\n');
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
@@ -110,33 +115,61 @@ jobs:
});
}
- name: Fail if Java formatting issues found
- name: Fail if backend format check failed
if: steps.spotless-check.outcome == 'failure'
run: |
echo "============================================"
echo " Java Formatting Check Failed"
echo " Backend Format Check Failed"
echo "============================================"
echo ""
echo "Your code has formatting issues."
echo "Run the following command to fix them:"
echo "There are formatting issues in your Java code"
echo "that will need to be fixed before they can be"
echo "merged in."
echo ""
echo " task backend:format"
echo ""
echo "Then commit and push the changes."
echo "Run 'task backend:format' to auto-fix, then"
echo "commit and push the changes."
echo "============================================"
exit 1
- name: Build with Gradle and spring security ${{ matrix.spring-security }}
- name: Remove backend format check comment on success
if: steps.spotless-check.outcome == 'success' && github.event_name == 'pull_request'
continue-on-error: true
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const marker = '<!-- java-formatting-check -->';
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body.includes(marker));
if (existing) {
await github.rest.issues.deleteComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
});
}
- name: Build with Gradle (flavor=${{ matrix.flavor }})
# STIRLING_FLAVOR is read by settings.gradle and expands into the
# right combination of DISABLE_ADDITIONAL_FEATURES + ENABLE_SAAS
# so we don't have to set them by hand. The saas flavor pulls in
# the app/saas subproject (unit tests only - no runtime tests).
run: task backend:build:ci
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
DISABLE_ADDITIONAL_FEATURES: ${{ matrix.spring-security }}
STIRLING_FLAVOR: ${{ matrix.flavor }}
- name: Check Test Reports Exist
if: always()
run: |
# Common + core + proprietary always build (proprietary is
# excluded only at runtime, not from the gradle subproject
# graph). Saas builds add a fourth report dir.
declare -a dirs=(
"app/core/build/reports/tests/"
"app/core/build/test-results/"
@@ -145,6 +178,9 @@ jobs:
"app/proprietary/build/reports/tests/"
"app/proprietary/build/test-results/"
)
if [ "${{ matrix.flavor }}" = "saas" ]; then
dirs+=("app/saas/build/reports/tests/" "app/saas/build/test-results/")
fi
for dir in "${dirs[@]}"; do
if [ ! -d "$dir" ]; then
echo "Missing $dir"
@@ -156,7 +192,7 @@ jobs:
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: test-reports-jdk-${{ matrix.jdk-version }}-spring-security-${{ matrix.spring-security }}
name: test-reports-jdk-${{ matrix.jdk-version }}-flavor-${{ matrix.flavor }}
path: |
app/**/build/reports/jacoco/test
app/**/build/reports/tests/
@@ -166,7 +202,47 @@ jobs:
retention-days: 3
if-no-files-found: warn
- name: Add coverage to PR with spring security ${{ matrix.spring-security }} and JDK ${{ matrix.jdk-version }}
- name: Install defusedxml for coverage summary
# coverage-summary.py parses JaCoCo XML through defusedxml to
# silence security scanners that pattern-match on the stdlib
# xml.etree.ElementTree.parse call.
if: always() && matrix.flavor == 'saas'
run: python -m pip install --quiet defusedxml
- name: JaCoCo coverage step summary
# Only the saas leg posts the JUnit summary - it's a strict
# superset of the core + proprietary legs (same .exec files plus
# the saas subproject). Posting from all three would mean three
# near-identical tables crowding out the aggregate report.
if: always() && matrix.flavor == 'saas'
run: |
python scripts/coverage-summary.py \
--title "Backend JUnit coverage (JDK ${{ matrix.jdk-version }})" \
--jacoco "common=app/common/build/reports/jacoco/test/jacocoTestReport.xml" \
--jacoco "core=app/core/build/reports/jacoco/test/jacocoTestReport.xml" \
--jacoco "proprietary=app/proprietary/build/reports/jacoco/test/jacocoTestReport.xml" \
--jacoco "saas=app/saas/build/reports/jacoco/test/jacocoTestReport.xml" \
--github-step-summary
- name: Upload raw JUnit .exec for aggregate merge
# Same dedup rationale as the summary step: upload from the saas
# leg only (the most complete set, includes app/saas/.../test.exec)
# so the aggregate workflow merges the union rather than three
# overlapping subsets.
#
# Separate artifact from the HTML reports so the aggregate
# workflow can grab just the .exec files with a name pattern
# (`jacoco-exec-*`) instead of unpacking the whole test-reports
# tarball.
if: always() && matrix.flavor == 'saas'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: jacoco-exec-junit-jdk-${{ matrix.jdk-version }}
path: app/*/build/jacoco/*.exec
retention-days: 7
if-no-files-found: warn
- name: Add coverage to PR (flavor=${{ matrix.flavor }}, JDK=${{ matrix.jdk-version }})
# The action only supports the pull_request event (it posts a PR comment),
# so skip it for merge_group runs and workflow_dispatch.
if: github.event_name == 'pull_request'
+32
View File
@@ -68,6 +68,18 @@ jobs:
uses: ./.github/workflows/backend-build.yml
secrets: inherit
db-migration-test:
# Boots the current bootJar against H2 fixtures captured from past
# releases (v2.0.0 / v2.5.0 / v2.10.0) and verifies admin login still
# works after Hibernate's ddl-auto=update migrates the schema. Gated on
# the `project` filter so doc-only PRs skip this ~5-minute job.
if: needs.files-changed.outputs.project == 'true'
needs: [files-changed]
permissions:
contents: read
uses: ./.github/workflows/db-migration-test.yml
secrets: inherit
check-generateOpenApiDocs:
if: needs.files-changed.outputs.openapi == 'true'
needs: [files-changed]
@@ -173,6 +185,24 @@ jobs:
uses: ./.github/workflows/dependency-review.yml
secrets: inherit
# Coverage aggregate: merges the JUnit + e2e:live + cucumber .exec
# artifacts produced by the jobs above into one report, plus pulls
# in vitest + Playwright frontend coverage for the per-area matrix.
# `if: always()` so a producer failing partway still gets credit
# for whatever did record. Advisory only - intentionally NOT in
# all-checks-passed, so a flaky aggregate run never blocks merging.
coverage-aggregate:
if: always()
needs:
- build
- playwright-e2e-live
- docker-compose-tests
- frontend-validation
permissions:
contents: read
uses: ./.github/workflows/coverage-aggregate.yml
secrets: inherit
# Single status check that branch protection should mark as required.
# Succeeds when every upstream job is either `success` or `skipped` (path-
# gated jobs that didn't apply this run). Any `failure` or `cancelled`
@@ -184,6 +214,7 @@ jobs:
needs:
- files-changed
- build
- db-migration-test
- check-generateOpenApiDocs
- frontend-validation
- playwright-e2e
@@ -208,6 +239,7 @@ jobs:
RESULTS: |
files-changed=${{ needs.files-changed.result }}
build=${{ needs.build.result }}
db-migration-test=${{ needs.db-migration-test.result }}
check-generateOpenApiDocs=${{ needs.check-generateOpenApiDocs.result }}
frontend-validation=${{ needs.frontend-validation.result }}
playwright-e2e=${{ needs.playwright-e2e.result }}
+230
View File
@@ -0,0 +1,230 @@
name: Aggregate backend coverage
# Reusable workflow called from build.yml after every backend coverage
# producer (JUnit, e2e:live, cucumber) has run. Downloads each job's raw
# .exec, merges them into one JaCoCo report, and posts a combined step
# summary alongside the per-source ones.
#
# Kept separate from the per-source jobs so:
# - the per-source jobs stay fast and independent (no cross-job waits)
# - this job can `if: always()` and still produce something useful when
# one of the producers fails partway through
# - frontend producers can be added later without touching the
# producers themselves
on:
workflow_call:
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
aggregate:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
timeout-minutes: 15
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up JDK 25
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
with:
java-version: "25"
distribution: "temurin"
- name: Cache Gradle dependency artifacts
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: |
~/.gradle/wrapper
~/.gradle/caches/modules-2/files-2.1
~/.gradle/caches/modules-2/metadata-2.*
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
cache-disabled: true
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.12"
- name: Install defusedxml for coverage scripts
# Both coverage-summary.py and coverage-matrix.py parse JaCoCo
# XML through defusedxml - see the script headers for context.
run: python -m pip install --quiet defusedxml
# Pattern matches every artifact this PR's producers might upload:
# jacoco-exec-junit-jdk-25 (uploaded only by the saas
# leg of backend-build, which
# is a strict superset of the
# core + proprietary legs)
# jacoco-exec-e2e-live
# jacoco-exec-cucumber
# Each lands as a sibling dir under coverage-execs/, with the .exec
# files preserving their original relative paths.
- name: Download all .exec artifacts
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v6.0.0
with:
pattern: jacoco-exec-*
path: coverage-execs/
merge-multiple: false
continue-on-error: true
- name: Inventory .exec files
id: inventory
# Splits the downloaded artifacts into two buckets:
# * e2e-only = cucumber + Playwright live (user-flow coverage)
# * all = the above plus JUnit (everything we test)
#
# Bucketing is by artifact-name prefix: download-artifact preserves
# the artifact name as the top-level dir, so JUnit's `.exec`s live
# under coverage-execs/jacoco-exec-junit-*/... while the others
# are under coverage-execs/jacoco-exec-{e2e-live,cucumber}/...
#
# If nothing was uploaded (e.g. all producers crashed before
# writing) we exit gracefully so this advisory job never fails CI.
run: |
mapfile -t all_execs < <(find coverage-execs -name '*.exec' -type f | sort)
mapfile -t e2e_execs < <(find coverage-execs -name '*.exec' -type f -not -path '*/jacoco-exec-junit-*' | sort)
if [ "${#all_execs[@]}" -eq 0 ]; then
echo "::warning::No .exec artifacts found - skipping aggregate report"
echo "found_all=false" >> "$GITHUB_OUTPUT"
echo "found_e2e=false" >> "$GITHUB_OUTPUT"
exit 0
fi
printf 'All %d .exec files:\n' "${#all_execs[@]}"
printf ' %s\n' "${all_execs[@]}"
IFS=','; all_joined="${all_execs[*]}"
echo "files_all=$all_joined" >> "$GITHUB_OUTPUT"
echo "found_all=true" >> "$GITHUB_OUTPUT"
if [ "${#e2e_execs[@]}" -eq 0 ]; then
echo "::notice::No e2e/cucumber .exec files - e2e-only report will be skipped"
echo "found_e2e=false" >> "$GITHUB_OUTPUT"
else
printf 'E2E-only %d .exec files:\n' "${#e2e_execs[@]}"
printf ' %s\n' "${e2e_execs[@]}"
unset IFS
IFS=','; e2e_joined="${e2e_execs[*]}"
echo "files_e2e=$e2e_joined" >> "$GITHUB_OUTPUT"
echo "found_e2e=true" >> "$GITHUB_OUTPUT"
fi
- name: Compile classes for JaCoCo class lookup
# jacocoReportFromExec only needs the compiled .class files
# under each subproject's build/classes/java/main/. `classes`
# (compileJava + processResources) is enough; we skipped the
# heavier `assemble` to avoid building bootJar / fat jars that
# add 60+ seconds per run for no gain to the report.
if: steps.inventory.outputs.found_all == 'true'
run: ./gradlew classes -PnoSpotless
- name: Generate e2e-only JaCoCo report
# "Real user-flow" coverage: only counts code reached by an actual
# HTTP request from cucumber or live Playwright. Useful for
# questions like "how much of our backend does a user actually
# hit?". Skipped when neither producer uploaded a .exec.
if: steps.inventory.outputs.found_e2e == 'true'
run: |
./gradlew jacocoReportFromExec \
-PexecFile="${{ steps.inventory.outputs.files_e2e }}" \
-PreportDir=build/reports/jacoco/aggregate-e2e \
-PnoSpotless
- name: Generate combined JaCoCo report (everything)
if: steps.inventory.outputs.found_all == 'true'
run: |
./gradlew jacocoReportFromExec \
-PexecFile="${{ steps.inventory.outputs.files_all }}" \
-PreportDir=build/reports/jacoco/aggregate-all \
-PnoSpotless
- name: E2E-only step summary
# Rendered first so it gets prime real estate in the Summary
# tab - this is the number most readers actually want
# ("how much of the backend do real user flows cover?").
if: steps.inventory.outputs.found_e2e == 'true'
run: |
python scripts/coverage-summary.py \
--title "Real user-flow backend coverage (e2e:live + cucumber)" \
--jacoco "merged=build/reports/jacoco/aggregate-e2e/jacocoTestReport.xml" \
--github-step-summary
- name: ALL-sources step summary
# Separate call (not a multi-input one) because the helper's
# rightmost "Aggregate" column would sum the two reports - which
# is meaningless when one is a strict superset of the other.
if: steps.inventory.outputs.found_all == 'true'
run: |
python scripts/coverage-summary.py \
--title "Combined backend coverage (JUnit + e2e:live + cucumber)" \
--jacoco "merged=build/reports/jacoco/aggregate-all/jacocoTestReport.xml" \
--github-step-summary
- name: Upload combined aggregate report
if: steps.inventory.outputs.found_all == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: jacoco-aggregate-all-${{ github.run_id }}
path: build/reports/jacoco/aggregate-all/
retention-days: 14
- name: Upload e2e-only aggregate report
if: steps.inventory.outputs.found_e2e == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: jacoco-aggregate-e2e-${{ github.run_id }}
path: build/reports/jacoco/aggregate-e2e/
retention-days: 14
# --------------------------------------------------------------
# Per-area matrix: rolls backend + frontend coverage into one
# table indexed by core/proprietary/saas/desktop. Pulls the
# frontend artifacts now (after the JaCoCo step has done its
# work) so the per-source backend summaries still render first
# even if the matrix step fails.
# --------------------------------------------------------------
- name: Download vitest coverage artifact
# frontend-validation uploads as `frontend-coverage`. Tolerate
# absence so a backend-only PR still produces the matrix with
# just backend rows populated.
if: always()
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v6.0.0
with:
name: frontend-coverage
path: matrix-inputs/vitest/
continue-on-error: true
- name: Download Playwright frontend coverage artifact
# e2e-live uploads as `playwright-frontend-coverage-<run_id>`.
# Same tolerance as vitest - matrix script handles missing inputs.
if: always()
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v6.0.0
with:
name: playwright-frontend-coverage-${{ github.run_id }}
path: matrix-inputs/playwright/
continue-on-error: true
- name: Coverage matrix step summary
if: always()
# Matrix references the two aggregate JaCoCo XMLs (already
# generated above) plus whichever frontend artifacts landed.
# Every input is optional; missing ones render as "-".
run: |
python scripts/coverage-matrix.py \
${{ steps.inventory.outputs.found_all == 'true' && '--jacoco-all build/reports/jacoco/aggregate-all/jacocoTestReport.xml' || '' }} \
${{ steps.inventory.outputs.found_e2e == 'true' && '--jacoco-e2e build/reports/jacoco/aggregate-e2e/jacocoTestReport.xml' || '' }} \
--vitest matrix-inputs/vitest/coverage-summary.json \
--playwright-frontend matrix-inputs/playwright/coverage-pw-summary/coverage-summary.json \
--title "Coverage matrix (per-area, e2e vs all)" \
--github-step-summary
+93
View File
@@ -0,0 +1,93 @@
name: DB migration smoke test
# Boots the current Stirling-PDF JAR against H2 fixtures captured from past
# releases (v2.0.0 / v2.5.0 / v2.10.0) and verifies admin login still works.
# Catches schema changes that would break existing user databases under
# Hibernate's `ddl-auto=update` upgrade path.
on:
workflow_call:
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
migration-test:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-8' }}
timeout-minutes: 30
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up JDK 25
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
with:
java-version: 25
distribution: temurin
- name: Cache Gradle dependency artifacts
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: |
~/.gradle/wrapper
~/.gradle/caches/modules-2/files-2.1
~/.gradle/caches/modules-2/metadata-2.*
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
cache-disabled: true
# No `-PnoSpotless` here yet because the upstream cache layer matches the
# backend build's; reuse keeps cold-cache cost identical.
- name: Build Stirling-PDF JAR
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
run: ./gradlew :stirling-pdf:bootJar -PnoSpotless --no-daemon
- name: Locate built JAR
id: jar
run: |
jar=$(find app/core/build/libs -maxdepth 1 -name 'Stirling-PDF*.jar' -o -name 'stirling-pdf*.jar' 2>/dev/null \
| grep -vE '(-plain|-sources)\.jar$' | head -n 1)
if [[ -z "$jar" ]]; then
echo "::error::No JAR under app/core/build/libs"
ls -lah app/core/build/libs || true
exit 1
fi
# Absolute path - the migration script pushd's into a temp workdir
# before invoking java, which would dangle a relative path.
jar=$(realpath "$jar")
echo "path=$jar" >> "$GITHUB_OUTPUT"
echo "Built JAR: $jar"
- name: Run migration smoke test
env:
STIRLING_JAR: ${{ steps.jar.outputs.path }}
run: bash scripts/db-migration/run-migration-test.sh
- name: Upload app logs on failure
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: db-migration-app-logs
# Path matches the preserved workdir in run-migration-test.sh -
# only failing fixtures leave a directory behind.
path: /tmp/stirling-migration-failed-*/app.log
retention-days: 7
if-no-files-found: warn
@@ -0,0 +1,136 @@
name: Docker Compose Cucumber tests (saas / PAYG)
# Self-contained CI job for the PAYG shadow-mode cucumber scenarios.
# Triggers only on PAYG-relevant paths so we don't add CI minutes to every PR
# that doesn't touch the saas flavour.
#
# Companion to `docker-compose-tests.yml` (which runs against the
# proprietary-flavour stack and skips features/payg via behave.ini's
# exclude_re). Kept as a separate workflow so the saas matrix can fail and
# succeed independently without touching the main cucumber harness.
on:
pull_request:
paths:
- "app/saas/**"
- "testing/cucumber/features/payg/**"
- "testing/cucumber/features/steps/payg_step_definitions.py"
- "testing/cucumber/requirements.txt"
- "testing/compose/docker-compose-saas.yml"
- "testing/compose/payg/**"
- "testing/test-payg.sh"
- ".github/workflows/docker-compose-tests-saas.yml"
push:
branches: [main]
paths:
- "app/saas/**"
- "testing/cucumber/features/payg/**"
- "testing/cucumber/features/steps/payg_step_definitions.py"
- "testing/cucumber/requirements.txt"
- "testing/compose/docker-compose-saas.yml"
- "testing/compose/payg/**"
- "testing/test-payg.sh"
- ".github/workflows/docker-compose-tests-saas.yml"
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
docker-compose-tests-saas:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
actions: write
contents: read
checks: write
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout Repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up JDK 25
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
with:
java-version: "25"
distribution: "temurin"
- name: Cache Gradle dependency artifacts
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: |
~/.gradle/wrapper
~/.gradle/caches/modules-2/files-2.1
~/.gradle/caches/modules-2/metadata-2.*
key: gradle-deps-saas-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
cache-disabled: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Expose GitHub runtime for Buildx cache
uses: crazy-max/ghaction-github-runtime@04d248b84655b509d8c44dc1d6f990c879747487 # v4.0.0
# No "Install Docker Compose" step: Ubuntu runners ship with `docker compose`
# v2 (built into the Docker CLI). test-payg.sh uses the v2 form throughout
# (`docker compose …`, no hyphen), so the legacy v1 `docker-compose` binary
# isn't needed. Avoids a `curl | sudo install` without checksum verification
# (Aikido flagged this when copy-pasted from docker-compose-tests.yml).
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.12"
cache: "pip"
cache-dependency-path: ./testing/cucumber/requirements.txt
- name: Pip requirements
run: |
pip install --require-hashes --only-binary=:all: -r ./testing/cucumber/requirements.txt
- name: Run PAYG Cucumber Tests
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
run: |
chmod +x ./testing/test-payg.sh
./testing/test-payg.sh
- name: Dump saas container logs on failure
if: failure()
run: |
docker compose -f testing/compose/docker-compose-saas.yml logs --tail 500 stirling-pdf-saas || true
docker compose -f testing/compose/docker-compose-saas.yml logs --tail 200 postgres-saas || true
- name: Upload PAYG Cucumber Report
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: payg-cucumber-report
path: testing/cucumber/report-payg.html
retention-days: 7
if-no-files-found: warn
- name: PAYG Cucumber Test Report
if: always()
uses: dorny/test-reporter@a43b3a5f7366b97d083190328d2c652e1a8b6aa2 # v3.0.0
with:
name: PAYG Cucumber Tests
path: testing/cucumber/junit-payg/*.xml
reporter: java-junit
fail-on-error: false
@@ -87,6 +87,12 @@ jobs:
run: |
pip install --require-hashes --only-binary=:all: -r ./testing/cucumber/requirements.txt
- name: Extract JaCoCo agent for cucumber coverage
# Stages build/jacoco/jacocoagent.jar where the coverage override
# file bind-mounts it into the cucumber container. The agent jar
# never goes into the published image - this is host-only.
run: ./gradlew copyJacocoAgent -PnoSpotless
- name: Run Docker Compose Tests
run: |
chmod +x ./testing/test_webpages.sh
@@ -98,6 +104,62 @@ jobs:
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
DOCKER_BASE_CHANGED: ${{ inputs.docker-base-changed }}
# Tells test.sh to layer testing/compose/docker-compose-coverage.override.yml
# over the cucumber compose so the container starts with the
# JaCoCo agent attached via JAVA_CUSTOM_OPTS.
STIRLING_PDF_TEST_COVERAGE: "1"
- name: Generate cucumber JaCoCo report
# `if: always()` so a behave failure still produces partial
# coverage from whatever endpoints did run. The exec file only
# exists when the container shut down cleanly - guard so the step
# is silent on the (rare) crash path.
if: always()
id: cucumber-coverage
run: |
if [ -s testing/cucumber-coverage/cucumber.exec ]; then
./gradlew jacocoReportFromExec \
-PexecFile=testing/cucumber-coverage/cucumber.exec \
-PreportDir=build/reports/jacoco/cucumber \
-PnoSpotless
echo "report=true" >> "$GITHUB_OUTPUT"
else
echo "::warning::No cucumber .exec at testing/cucumber-coverage/cucumber.exec (container may have crashed before flushing)"
echo "report=false" >> "$GITHUB_OUTPUT"
fi
- name: Install defusedxml for coverage summary
# coverage-summary.py parses JaCoCo XML through defusedxml -
# see the script header for context.
if: always() && steps.cucumber-coverage.outputs.report == 'true'
run: python -m pip install --quiet defusedxml
- name: Cucumber coverage step summary
if: always() && steps.cucumber-coverage.outputs.report == 'true'
run: |
python scripts/coverage-summary.py \
--title "Cucumber (docker) JaCoCo coverage" \
--jacoco "cucumber=build/reports/jacoco/cucumber/jacocoTestReport.xml" \
--github-step-summary
- name: Upload cucumber JaCoCo report
if: always() && steps.cucumber-coverage.outputs.report == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: jacoco-cucumber-${{ github.run_id }}
path: build/reports/jacoco/cucumber/
retention-days: 7
- name: Upload raw cucumber .exec for aggregate merge
# Picked up by the coverage-aggregate workflow via the
# `jacoco-exec-*` artifact name pattern.
if: always() && steps.cucumber-coverage.outputs.report == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: jacoco-exec-cucumber
path: testing/cucumber-coverage/cucumber.exec
retention-days: 7
if-no-files-found: warn
- name: Upload Cucumber Report
if: always()
+124 -1
View File
@@ -49,9 +49,132 @@ jobs:
env:
VITE_BUILD_FOR_PREVIEW: "1"
run: task frontend:build
- name: Run live E2E tests (chromium)
- name: Run live E2E tests (chromium) with coverage
id: live-tests
env:
# Attaches the JaCoCo agent to the bootRun JVM (see
# .taskfiles/e2e.yml live:backend). The .exec gets flushed on
# graceful shutdown when the runner traps EXIT/INT/TERM, so the
# report step below sees a populated file.
COVERAGE: "1"
# Tells the Playwright fixture (test-base.ts) to capture per-test
# V8 JS coverage. Raw dumps land under
# .test-state/playwright/coverage-pw/ for the post-process step
# to aggregate. Chromium-only - other engines silently skip.
PW_COVERAGE: "1"
run: task e2e:live
- name: Generate JaCoCo report from e2e:live .exec
if: always()
id: live-coverage
# `if: always()` so even a failed test run still produces a
# report from whatever flows did exercise the backend before
# the failure. The task itself tolerates a missing .exec
# (jacoco emits an empty report rather than crashing) but we
# guard with `test -s` to keep the job log clean.
run: |
if [ -s .test-state/playwright/jacoco.exec ]; then
./gradlew jacocoReportFromExec \
-PexecFile=.test-state/playwright/jacoco.exec \
-PreportDir=build/reports/jacoco/e2e-live \
-PnoSpotless
echo "report=true" >> "$GITHUB_OUTPUT"
else
echo "::warning::No e2e:live .exec found at .test-state/playwright/jacoco.exec; skipping report"
echo "report=false" >> "$GITHUB_OUTPUT"
fi
- name: Set up Python for coverage summary
if: always() && steps.live-coverage.outputs.report == 'true'
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.12"
- name: Install defusedxml for coverage summary
# coverage-summary.py uses defusedxml instead of stdlib xml.etree
# to dodge XXE / billion-laughs scanner findings.
if: always() && steps.live-coverage.outputs.report == 'true'
run: python -m pip install --quiet defusedxml
- name: e2e:live coverage step summary
if: always() && steps.live-coverage.outputs.report == 'true'
run: |
python scripts/coverage-summary.py \
--title "Playwright (live backend) JaCoCo coverage" \
--jacoco "e2e-live=build/reports/jacoco/e2e-live/jacocoTestReport.xml" \
--github-step-summary
- name: Upload e2e:live JaCoCo report
if: always() && steps.live-coverage.outputs.report == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: jacoco-e2e-live-${{ github.run_id }}
path: build/reports/jacoco/e2e-live/
retention-days: 7
- name: Upload raw e2e:live .exec for aggregate merge
# Picked up by the coverage-aggregate workflow via the
# `jacoco-exec-*` artifact name pattern.
if: always() && steps.live-coverage.outputs.report == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: jacoco-exec-e2e-live
path: .test-state/playwright/jacoco.exec
retention-days: 7
if-no-files-found: warn
- name: Set up Python for frontend coverage summary
# Separate from the backend-coverage python step because the
# frontend path doesn't depend on a JaCoCo report - it produces
# a summary even on backend failure, as long as some Playwright
# tests ran far enough to dump V8 coverage.
if: always()
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.12"
- name: Install defusedxml for frontend coverage summary
# Idempotent re-install: the backend-coverage step may have
# installed it already, but this leg can run on its own when the
# backend report step skips (e.g. .exec missing).
if: always()
run: python -m pip install --quiet defusedxml
- name: Aggregate Playwright frontend (V8) coverage
# Rolls per-test V8 dumps from the test-base fixture into one
# vitest-shaped coverage-summary.json. Tolerates a missing dump
# dir (firefox/webkit runs, or a failure before any test got
# far enough to dump).
if: always()
id: pw-frontend-coverage
run: |
if [ -d .test-state/playwright/coverage-pw ] && \
find .test-state/playwright/coverage-pw -name '*.json' -type f | grep -q .; then
python scripts/playwright-coverage-summary.py \
.test-state/playwright/coverage-pw \
--out .test-state/playwright/coverage-pw-summary/coverage-summary.json
echo "summary=true" >> "$GITHUB_OUTPUT"
else
echo "::notice::No Playwright frontend coverage dumps found (chromium-only feature)"
echo "summary=false" >> "$GITHUB_OUTPUT"
fi
- name: Playwright frontend coverage step summary
if: always() && steps.pw-frontend-coverage.outputs.summary == 'true'
run: |
python scripts/coverage-summary.py \
--title "Playwright (live) frontend coverage" \
--vitest .test-state/playwright/coverage-pw-summary/coverage-summary.json \
--github-step-summary
- name: Upload Playwright frontend coverage
# Bundle both the aggregated summary and the raw V8 dumps so
# someone debugging "why is this function showing as covered"
# can trace it back to the source dump.
if: always() && steps.pw-frontend-coverage.outputs.summary == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: playwright-frontend-coverage-${{ github.run_id }}
path: |
.test-state/playwright/coverage-pw-summary/
.test-state/playwright/coverage-pw/
retention-days: 7
- name: Print backend log on failure
if: failure() && steps.live-tests.conclusion == 'failure'
run: |
@@ -110,8 +110,8 @@ jobs:
NPM_CONFIG_IGNORE_SCRIPTS: "true"
working-directory: frontend
run: |
mkdir -p src/assets
npx --yes license-report --only=prod --output=json > src/assets/3rdPartyLicenses.json
mkdir -p editor/src/assets
npx --yes license-report --only=prod --output=json > editor/src/assets/3rdPartyLicenses.json
- name: Postprocess with project script (BASE version)
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == true
+35
View File
@@ -109,6 +109,41 @@ jobs:
comment_id: existing.id,
});
}
- name: Vitest coverage
# Separate from `frontend:check:all` so the quality-gate run stays
# uninstrumented (faster signal) and coverage stays an informational
# follow-up. Continue-on-error keeps the workflow green even when
# a handful of test files refuse to import (e.g. missing icon
# specifiers) - the summary still gets posted with whatever
# vitest managed to instrument.
id: frontend-coverage
continue-on-error: true
run: task frontend:test:coverage
- name: Set up Python for coverage summary
if: always()
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.12"
- name: Install defusedxml for coverage summary
# See coverage-summary.py header - it parses XML through defusedxml
# to dodge the stdlib parser's exposure to XXE / billion-laughs.
if: always()
run: python -m pip install --quiet defusedxml
- name: Vitest coverage step summary
if: always()
run: |
python scripts/coverage-summary.py \
--title "Frontend Vitest coverage" \
--vitest frontend/editor/coverage/coverage-summary.json \
--github-step-summary
- name: Upload vitest coverage report
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: frontend-coverage
path: frontend/editor/coverage/
retention-days: 7
if-no-files-found: warn
- name: Upload frontend build artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
+167 -16
View File
@@ -442,10 +442,6 @@ jobs:
echo "Generated tauri.windows.conf.json (alias masked):"
sed "s/${KEYPAIR_ALIAS}/***/g" ./frontend/editor/src-tauri/tauri.windows.conf.json
- name: Sign JPDFium dylibs inside bootJar (macOS only)
if: matrix.platform == 'macos-15' && env.APPLE_CERTIFICATE != ''
run: bash frontend/scripts/sign-jpdfium-dylibs-in-bootjar.sh
- name: Import release GPG signing key (Linux)
if: matrix.platform == 'ubuntu-22.04' && env.RELEASE_GPG_PRIVATE_KEY != '' && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master')
run: |
@@ -495,6 +491,7 @@ jobs:
projectPath: ./frontend/editor
tauriScript: npx tauri
args: ${{ matrix.args }}
updaterJsonKeepUniversal: true
- name: Clear release GPG key from runner keyring (Linux)
if: always() && matrix.platform == 'ubuntu-22.04' && env.RELEASE_GPG_PRIVATE_KEY != '' && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master')
@@ -586,21 +583,37 @@ jobs:
if: always() && steps.digicert-setup.conclusion != 'failure'
shell: bash
run: |
mkdir -p ./dist
# Absolute dist path so the cd below can't break the copy targets.
DIST="$GITHUB_WORKSPACE/dist"
mkdir -p "$DIST"
cd ./frontend/editor/src-tauri/target
# Find and rename artifacts based on platform
echo "=== tauri bundle artifacts ==="
find . -path "*/bundle/*" \( -name "*.msi" -o -name "*.deb" \
-o -name "*.rpm" -o -name "*.AppImage" -o -name "*.dmg" \
-o -name "*.app.tar.gz" -o -name "*.sig" \) 2>/dev/null | sort || true
echo "=============================="
# createUpdaterArtifacts:true signs the native installers in place;
# each <bundle> ships with a sibling <bundle>.sig consumed by latest.json.
if [ "${{ matrix.platform }}" = "windows-latest" ]; then
# Only ship the MSI installer on Windows. The loose exe and WiX toolset exes
# are not the user-facing installer - the MSI contains the signed inner exe.
find . -name "*.msi" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.msi" \;
find . -name "*.msi" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.msi" \;
find . -name "*.msi.sig" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.msi.sig" \;
elif [ "${{ matrix.platform }}" = "macos-15" ]; then
find . -name "*.dmg" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.dmg" \;
find . -name "*.app" -exec cp -r {} "../../../dist/Stirling-PDF-${{ matrix.name }}.app" \;
# DMG = manual install; .app.tar.gz (+ .sig) = updater payload.
# Raw .app is intentionally not shipped (hundreds of MB of uncompressed input).
find . -name "*.dmg" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.dmg" \;
find . -name "*.app.tar.gz" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.app.tar.gz" \;
find . -name "*.app.tar.gz.sig" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.app.tar.gz.sig" \;
else
find . -name "*.deb" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.rpm" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.AppImage" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.AppImage" \;
# The raw .AppImage IS its updater payload (signed -> .AppImage.sig),
# not a .tar.gz wrapper - that's only produced under v1Compatible.
find . -name "*.deb" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.deb.sig" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.deb.sig" \;
find . -name "*.rpm" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.rpm.sig" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.rpm.sig" \;
find . -name "*.AppImage" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.AppImage" \;
find . -name "*.AppImage.sig" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.AppImage.sig" \;
fi
- name: Upload build artifacts
@@ -611,8 +624,7 @@ jobs:
path: ./dist/*
retention-days: 1
create-release:
if: (github.event_name == 'workflow_dispatch' && github.event.inputs.test_mode != 'true') || github.event_name == 'release' || github.ref == 'refs/heads/V2-master'
collect-and-release:
needs: [pick, determine-matrix, build, build-jars]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
@@ -623,6 +635,16 @@ jobs:
with:
egress-policy: audit
# Sparse-check out the verifier + pubkey before the artifact downloads
# so the checkout cannot clobber ./artifacts.
- name: Checkout updater verifier
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
sparse-checkout: |
.github/scripts/verify-updater-signatures.py
frontend/editor/src-tauri/tauri.conf.json
sparse-checkout-cone-mode: false
- name: Download all Tauri artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
@@ -650,17 +672,146 @@ jobs:
- name: Display structure of downloaded files
run: ls -R ./artifacts
# tauri-action only emits latest.json when it also publishes the release
# (tagName/releaseId set). We publish separately via action-gh-release,
# so build latest.json here from the per-platform .sig files.
- name: Generate updater latest.json
env:
VERSION: ${{ needs.determine-matrix.outputs.version }}
TAG: v${{ needs.determine-matrix.outputs.version }}
REPO: ${{ github.repository }}
run: |
python3 - << 'PYEOF'
import json, os, sys
from pathlib import Path
from datetime import datetime, timezone
VERSION = os.environ['VERSION']
TAG = os.environ['TAG']
REPO = os.environ['REPO']
ART = Path('./artifacts/tauri')
# Tauri updater looks up {os}-{arch}-{installer} (e.g. linux-x86_64-deb)
# before bare {os}-{arch}, so per-format Linux keys let deb/rpm/appimage
# each self-update from their matching file. macOS universal serves both
# arches from the one .app.tar.gz.
PLATFORM_MAP = [
{
'bundles': ['Stirling-PDF-linux-x86_64.deb'],
'targets': ['linux-x86_64-deb'],
},
{
'bundles': ['Stirling-PDF-linux-x86_64.rpm'],
'targets': ['linux-x86_64-rpm'],
},
{
'bundles': ['Stirling-PDF-linux-x86_64.AppImage'],
'targets': ['linux-x86_64-appimage'],
},
{
'bundles': ['Stirling-PDF-windows-x86_64.msi'],
'targets': ['windows-x86_64-msi', 'windows-x86_64'],
},
{
'bundles': ['Stirling-PDF-macos-universal.app.tar.gz'],
'targets': ['darwin-x86_64', 'darwin-aarch64'],
},
]
# rglob() because download-artifact varies layout: one artifact -> flat,
# many -> nested under <artifact-name>/.
def find_signed(name):
for bundle_path in sorted(ART.rglob(name)):
sig_path = bundle_path.with_name(bundle_path.name + '.sig')
if sig_path.exists():
return bundle_path, sig_path
return None
platforms = {}
skipped = []
for entry in PLATFORM_MAP:
picked = None
for name in entry['bundles']:
picked = find_signed(name)
if picked:
break
if not picked:
skipped.append(
f"{entry['targets']} (no signed bundle among "
f"{entry['bundles']} - TAURI_SIGNING_PRIVATE_KEY unset "
f"or createUpdaterArtifacts disabled?)"
)
continue
bundle_path, sig_path = picked
signature = sig_path.read_text(encoding='utf-8').strip()
url = f"https://github.com/{REPO}/releases/download/{TAG}/{bundle_path.name}"
for target in entry['targets']:
platforms[target] = {'signature': signature, 'url': url}
print(f"Added {entry['targets']} from {bundle_path.name}")
if skipped:
print("Skipped platforms:")
for s in skipped:
print(f" - {s}")
if not platforms:
print(
"WARN: no signed updater bundles found - "
"skipping latest.json generation"
)
sys.exit(0)
manifest = {
'version': VERSION,
'notes': f"See https://github.com/{REPO}/releases/tag/{TAG}",
'pub_date': datetime.now(timezone.utc).strftime('%Y-%m-%dT%H:%M:%SZ'),
'platforms': platforms,
}
out = Path('./artifacts/latest.json')
out.write_text(json.dumps(manifest, indent=2) + '\n', encoding='utf-8')
print(f"Generated {out} with platforms: {sorted(platforms.keys())}")
PYEOF
- name: Upload merged artifacts for review
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: release-artifacts
path: ./artifacts/
retention-days: 7
# Gate publish on valid updater sigs. Runs after the review upload (so
# artifacts survive for debugging) and before action-gh-release.
- name: Verify updater signatures
run: |
python3 -m pip install --quiet 'cryptography==44.0.0'
python3 .github/scripts/verify-updater-signatures.py \
./artifacts/tauri frontend/editor/src-tauri/tauri.conf.json
# workflow_dispatch path requires platform=='all' so a single-platform
# dispatch can't overwrite an existing release's full latest.json with a
# partial one (action-gh-release defaults overwrite_files:true).
# release / V2-master always build the full matrix so no extra guard needed.
# fail_on_unmatched_files makes a missing latest.json or installer fail loudly
# instead of silently shipping a broken auto-update.
- name: Upload binaries to Release
if: (github.event_name == 'workflow_dispatch' && github.event.inputs.test_mode != 'true' && github.event.inputs.platform == 'all') || github.event_name == 'release' || github.ref == 'refs/heads/V2-master'
uses: softprops/action-gh-release@b4309332981a82ec1c5618f44dd2e27cc8bfbfda # v3.0.0
with:
tag_name: v${{ needs.determine-matrix.outputs.version }}
generate_release_notes: true
fail_on_unmatched_files: true
# Installers + updater payloads + manifest. .sig contents are embedded
# in latest.json so the .sig files themselves are not uploaded.
files: |
./artifacts/**/*.jar
./artifacts/**/*.msi
./artifacts/**/*.dmg
./artifacts/**/*.app.tar.gz
./artifacts/**/*.deb
./artifacts/**/*.rpm
./artifacts/**/*.AppImage
./artifacts/latest.json
draft: false
prerelease: false
+11 -10
View File
@@ -157,6 +157,9 @@ jobs:
JPDFIUM_PLATFORMS: ${{ matrix.jpdfium_platforms }}
run: task desktop:prepare
- name: Run Tauri/Cargo tests
run: task desktop:test
# DigiCert KeyLocker Setup (Cloud HSM)
- name: Setup DigiCert KeyLocker
id: digicert-setup
@@ -268,10 +271,6 @@ jobs:
echo "APPLE_SIGNING_IDENTITY=$CERT_ID" >> $GITHUB_ENV
echo "Certificate imported successfully."
- name: Sign JPDFium dylibs inside bootJar (macOS only)
if: matrix.platform == 'macos-15' && env.APPLE_CERTIFICATE != ''
run: bash frontend/scripts/sign-jpdfium-dylibs-in-bootjar.sh
- name: Check DMG creation dependencies (macOS only)
if: matrix.platform == 'macos-15'
run: |
@@ -417,20 +416,22 @@ jobs:
- name: Rename artifacts
shell: bash
run: |
mkdir -p ./dist
# Absolute dist path so the cd below can't break the copy targets.
DIST="$GITHUB_WORKSPACE/dist"
mkdir -p "$DIST"
cd ./frontend/editor/src-tauri/target
# Find and rename artifacts based on platform
if [ "${{ matrix.platform }}" = "windows-latest" ]; then
# Only ship the MSI installer. The loose exe and WiX toolset exes
# are not the user-facing installer - the MSI contains the signed inner exe.
find . -name "*.msi" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.msi" \;
find . -name "*.msi" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.msi" \;
elif [ "${{ matrix.platform }}" = "macos-15" ]; then
find . -name "*.dmg" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.dmg" \;
find . -name "*.dmg" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.dmg" \;
else
find . -name "*.deb" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.rpm" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.AppImage" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.AppImage" \;
find . -name "*.deb" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.rpm" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.AppImage" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.AppImage" \;
fi
# Verify the MSI AND the inner exe extracted from it are signed.
+11 -3
View File
@@ -23,6 +23,10 @@ customFiles/
configs/
watchedFolders/
clientWebUI/
# Scratch dir used by local fixture-regeneration runs (see
# app/proprietary/src/test/resources/db-migration-fixtures/README.md).
# Holds downloaded JARs and disposable workdirs. Never committed.
.alpha-local/
!cucumber/
!cucumber/exampleFiles/
!cucumber/exampleFiles/example_html.zip
@@ -53,6 +57,8 @@ app/core/src/main/resources/static/robots.txt
app/core/src/main/resources/static/pdfium/
app/core/src/main/resources/static/pdfjs/
app/core/src/main/resources/static/vendor/
app/core/src/main/resources/static/**/*.gz
app/core/src/main/resources/static/**/*.br
# Note: Keep backend-managed files like fonts/, css/, js/, pdfjs/, etc.
# Gradle
@@ -174,7 +180,6 @@ venv.bak/
# Env files (secrets / local overrides). Subproject .gitignore files whitelist any committed defaults.
.env*
!.env.saas.example
# VS Code
/.vscode/**/*
@@ -209,7 +214,7 @@ out/
*.asc
# Allow test fixture certificates (synthetic, no real credentials)
!frontend/src/core/tests/test-fixtures/certs/**
!frontend/editor/src/core/tests/test-fixtures/certs/**
# SSH Keys
*.pub
@@ -251,7 +256,7 @@ node_modules/
*compact*.json
test_batch.json
*.backup.*.json
frontend/public/locales/*/translation.backup*.json
frontend/editor/public/locales/*/translation.backup*.json
# Development/build artifacts
.gradle-cache/
@@ -274,3 +279,6 @@ docs/type3/signatures/
# Playwright MCP screenshots / traces
.playwright-mcp/
*.playwright-mcp.png
# Local screenshot artifacts from *-screenshots.spec.ts
frontend/editor/screenshots/
+5
View File
@@ -0,0 +1,5 @@
# PostHog project-level key — phc_ prefix keys are public/client-side by design
# (PostHog client-side tracking embeds them in the browser bundle). Committed
# intentionally in #6150 so engine/.env has a working default, with real
# credentials overridden via engine/.env.local.
engine/.env:generic-api-key:41
+6 -5
View File
@@ -22,12 +22,13 @@ tasks:
vars:
PORT: '{{.PORT | default "8080"}}'
AIENGINE_URL: '{{.AIENGINE_URL | default ""}}'
AIENGINE_TIMEOUTSECONDS: '{{.AIENGINE_TIMEOUTSECONDS | default "120"}}'
env:
SERVER_PORT: '{{.PORT}}'
cmds:
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED=true {{end}}cmd /c ".\gradlew.bat :stirling-pdf:bootRun"'
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED=true AIENGINE_TIMEOUTSECONDS={{.AIENGINE_TIMEOUTSECONDS}} {{end}}cmd /c ".\gradlew.bat :stirling-pdf:bootRun"'
platforms: [windows]
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED=true {{end}}./gradlew :stirling-pdf:bootRun'
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED=true AIENGINE_TIMEOUTSECONDS={{.AIENGINE_TIMEOUTSECONDS}} {{end}}./gradlew :stirling-pdf:bootRun'
platforms: [linux, darwin]
dev:bundled:
@@ -40,10 +41,10 @@ tasks:
platforms: [linux, darwin]
dev:saas:
desc: "Start backend in SaaS flavor against Supabase (loads .env.saas.local)"
desc: "Start backend in SaaS flavor against Supabase"
# `dotenv:` reads from the root Taskfile's directory (".") because this
# subtaskfile is included with `dir: .`. Drop the file at the repo root.
dotenv: ['.env.saas.local']
# subtaskfile is included with `dir: .`.
dotenv: ['app/.env.saas.local', 'app/.env.saas']
ignore_error: true
vars:
PORT: '{{.PORT | default "8080"}}'
+35 -5
View File
@@ -1,7 +1,9 @@
version: '3'
vars:
JLINK_MODULES: "java.base,java.compiler,java.desktop,java.instrument,java.logging,java.management,java.naming,java.net.http,java.prefs,java.rmi,java.scripting,java.security.jgss,java.security.sasl,java.sql,java.transaction.xa,java.xml,java.xml.crypto,jdk.crypto.ec,jdk.crypto.cryptoki,jdk.unsupported"
# jdk.dynalink is required by VeraPDF (PDF/A validation); without it the bundled JRE throws
# NoClassDefFoundError: jdk/dynalink/Namespace at runtime in get-info-on-pdf and verify-pdf
JLINK_MODULES: "java.base,java.compiler,java.desktop,java.instrument,java.logging,java.management,java.naming,java.net.http,java.prefs,java.rmi,java.scripting,java.security.jgss,java.security.sasl,java.sql,java.transaction.xa,java.xml,java.xml.crypto,jdk.crypto.ec,jdk.crypto.cryptoki,jdk.unsupported,jdk.dynalink"
# Override via JPDFIUM_PLATFORMS env (csv of platform keys, or 'all').
JPDFIUM_PLATFORMS:
@@ -62,21 +64,28 @@ tasks:
deps: [prepare]
dir: editor
cmds:
- npx tauri build --bundles app
- npx tauri build --bundles app --config '{"bundle":{"createUpdaterArtifacts":false}}'
build:dev:windows:
desc: "Build Tauri desktop NSIS installer (Windows)"
deps: [prepare]
dir: editor
cmds:
- npx tauri build --bundles nsis
- npx tauri build --bundles nsis --config '{"bundle":{"createUpdaterArtifacts":false}}'
build:dev:linux:
desc: "Build Tauri desktop AppImage (Linux)"
deps: [prepare]
dir: editor
cmds:
- npx tauri build --bundles appimage
- npx tauri build --bundles appimage --config '{"bundle":{"createUpdaterArtifacts":false}}'
test:
desc: "Run Tauri/Cargo tests"
deps: [prepare]
dir: editor/src-tauri
cmds:
- cargo test
clean:
desc: "Clean Tauri/Cargo build artifacts"
@@ -126,8 +135,29 @@ tasks:
--no-header-files
--no-man-pages
--output runtime/jre
# jlink emits its files mode 444 (read-only). Tauri's build-script
# resource copier preserves source permissions when staging
# `runtime/jre/**/*` into `target/<profile>/runtime/jre/...`, so the
# staged copies are read-only too. On any subsequent incremental
# build the copier tries to overwrite them and fails with a bare
# `Permission denied (os error 13)` (Rust's io::Error Display drops
# the path, so the failure is opaque). Make the source writable here
# so the staged destinations are writable and can be overwritten.
#
# Trade-off: this task runs for both `task desktop:dev` and
# `task desktop:build`, so production bundles also ship mode-644
# JRE files instead of 444. Functionally harmless on POSIX (the
# `other` bit is `r--` either way, and on macOS code signing is the
# real integrity check) and on Windows the DOS read-only attribute
# isn't load-bearing for the bundled JDK. If we ever need strict
# 444 in production, split the chmod into a dev-only step and have
# `desktop:build` run `jlink:clean` first to force a fresh build.
- cmd: chmod -R u+w runtime/jre
platforms: [linux, darwin]
- cmd: powershell -NoProfile -Command "Get-ChildItem -Recurse runtime/jre | ForEach-Object { $_.IsReadOnly = $false }"
platforms: [windows]
status:
- test -d editor/src-tauri/runtime/jre
- test -f runtime/jre/release
jlink:clean:
desc: "Remove JLink runtime and bundled JARs"
+11 -1
View File
@@ -34,6 +34,9 @@ tasks:
ignore_error: true
vars:
BASE_DIR: '{{.ROOT_DIR}}/.test-state/playwright'
# COVERAGE=1 in the calling environment attaches the JaCoCo agent to
# the bootRun JVM and writes to BASE_DIR/jacoco.exec on shutdown.
# Off by default to keep local dev runs uninstrumented; CI flips it.
env:
STIRLING_BASE_PATH: '{{.BASE_DIR}}'
# Suppress the analytics opt-in modal that fires on first admin login.
@@ -58,12 +61,19 @@ tasks:
set -e
rm -rf "{{.BASE_DIR}}"
mkdir -p "{{.BASE_DIR}}"
GRADLE_ARGS=":stirling-pdf:bootRun"
if [ -n "${COVERAGE:-}" ]; then
# copyJacocoAgent is wired as a dependency of bootRun when
# -PjacocoAgent=true, so we do not need to invoke it separately.
GRADLE_ARGS="$GRADLE_ARGS -PjacocoAgent=true -PjacocoExec={{.BASE_DIR}}/jacoco.exec"
echo "JaCoCo coverage enabled, writing to {{.BASE_DIR}}/jacoco.exec"
fi
# Background gradle and record its PID so the runner can clean up
# the exact process tree (wrapper + forked Spring Boot JVM) without
# resorting to fuzzy `pkill -f` patterns. `wait` keeps this script
# alive for the lifetime of gradle so Task'"'"'s parallel deps stay
# synchronised.
bash gradlew :stirling-pdf:bootRun > "{{.BASE_DIR}}/backend.log" 2>&1 &
bash gradlew $GRADLE_ARGS > "{{.BASE_DIR}}/backend.log" 2>&1 &
GRADLE_PID=$!
echo $GRADLE_PID > "{{.BASE_DIR}}/backend.pid"
wait $GRADLE_PID
+72 -3
View File
@@ -112,6 +112,12 @@ tasks:
- task: dev:_run
vars: { MODE: prototypes, PORT: '{{.PORT}}', BACKEND_URL: '{{.BACKEND_URL}}', OPEN: '{{.OPEN}}' }
dev:portal:
desc: "Start developer portal dev server"
deps: [install]
cmds:
- npx vite portal --port {{.PORT | default "5173"}}{{if .OPEN}} --open{{end}}
# ============================================================
# Build
# ============================================================
@@ -156,6 +162,24 @@ tasks:
cmds:
- npx vite build editor --mode prototypes
build:portal:
desc: "Build developer portal"
deps: [install]
cmds:
- npx vite build portal
storybook:
desc: "Start Storybook dev server"
deps: [install]
cmds:
- npx storybook dev -p 6006 {{.CLI_ARGS}}
storybook:build:
desc: "Build static Storybook"
deps: [install]
cmds:
- npx storybook build {{.CLI_ARGS}}
# ============================================================
# Code quality
# ============================================================
@@ -163,9 +187,23 @@ tasks:
lint:
desc: "Run linting"
deps: [install]
cmds:
- task: lint:eslint
- task: lint:dpdm
lint:eslint:
desc: "Run ESLint linting"
deps: [install]
cmds:
- npx eslint --max-warnings=0
- npx dpdm editor/src --circular --no-warning --no-tree --exit-code circular:1
lint:dpdm:
desc: "Run circular import linting"
deps: [install]
cmds:
# Globs so dpdm walks the whole tree. dpdm expands the braces itself, so this is
# shell-agnostic. Covers editor, portal, and the shared design system.
- npx dpdm "editor/src/**/*.{ts,tsx}" "portal/src/**/*.{ts,tsx}" "shared/**/*.{ts,tsx}" --circular --no-warning --no-tree --exit-code circular:1
lint:fix:
desc: "Auto-fix lint issues"
@@ -236,6 +274,18 @@ tasks:
cmds:
- npx tsc --noEmit --project editor/src/prototypes/tsconfig.json
typecheck:portal:
desc: "Typecheck developer portal build variant"
deps: [install]
cmds:
- npx tsc --noEmit --project portal/tsconfig.json
typecheck:shared:
desc: "Typecheck the shared design system"
deps: [install]
cmds:
- npx tsc --noEmit --project shared/tsconfig.json
typecheck:all:
desc: "Typecheck all build variants"
cmds:
@@ -245,6 +295,8 @@ tasks:
- task: typecheck:desktop
- task: typecheck:scripts
- task: typecheck:prototypes
- task: typecheck:portal
- task: typecheck:shared
# ============================================================
# Quality Gate
@@ -265,7 +317,9 @@ tasks:
- task: lint
- task: format:check
- task: build
- task: build:portal
- task: test
- task: storybook:build
# ============================================================
# Test
@@ -284,10 +338,25 @@ tasks:
- npx vitest --watch --root editor
test:coverage:
desc: "Run tests with coverage"
desc: "Run tests with coverage (one-shot; CI-friendly)."
deps: [install]
cmds:
- npx vitest --coverage --root editor
# `vitest run` makes this CI-safe (the bare `vitest` form enters watch
# mode). Explicit reporter list because v8 + json-summary is what the
# coverage-summary.py helper consumes; html/text are kept for humans.
#
# reportsDirectory is pinned to ./coverage relative to vitest's root
# (--root editor), so output lands at frontend/editor/coverage/. The
# CI upload step reads from that path. An earlier attempt with
# `./editor/coverage` double-nested into frontend/editor/editor/coverage;
# pinning future-proofs against vitest changing the default.
- >
npx vitest run --root editor --coverage
--coverage.provider=v8
--coverage.reporter=text-summary
--coverage.reporter=json-summary
--coverage.reporter=html
--coverage.reportsDirectory=./coverage
# ============================================================
# Code Generation
+10 -8
View File
@@ -10,14 +10,16 @@ if that directory exists, is licensed under the license defined in "app/propriet
if that directory exists, is licensed under the license defined in "app/saas/LICENSE".
* All content that resides under the "engine/" directory of this repository,
if that directory exists, is licensed under the license defined in "engine/LICENSE".
* All content that resides under the "frontend/src/proprietary/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/src/proprietary/LICENSE".
* All content that resides under the "frontend/src/desktop/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/src/desktop/LICENSE".
* All content that resides under the "frontend/src/saas/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/src/saas/LICENSE".
* All content that resides under the "frontend/src/prototypes/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/src/prototypes/LICENSE".
* All content that resides under the "frontend/editor/src/proprietary/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/editor/src/proprietary/LICENSE".
* All content that resides under the "frontend/editor/src/desktop/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/editor/src/desktop/LICENSE".
* All content that resides under the "frontend/editor/src/saas/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/editor/src/saas/LICENSE".
* All content that resides under the "frontend/editor/src/prototypes/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/editor/src/prototypes/LICENSE".
* All content that resides under the "frontend/portal/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/portal/LICENSE".
* Content outside of the above mentioned directories or restrictions above is
available under the MIT License as defined below.
+18 -1
View File
@@ -58,6 +58,23 @@ tasks:
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
OPEN: "true"
dev:saas:
desc: "Start SaaS backend + frontend concurrently on free ports"
vars:
PORTS:
sh: '{{if eq OS "windows"}}{{.FIND_FREE_PORT_PS}} 8080 5173{{else}}{{.FIND_FREE_PORT_SH}} 8080 5173{{end}}'
BACKEND_PORT: '{{index (splitList "\n" .PORTS) 0}}'
FRONTEND_PORT: '{{index (splitList "\n" .PORTS) 1}}'
deps:
- task: backend:dev:saas
vars:
PORT: '{{.BACKEND_PORT}}'
- task: frontend:dev:saas
vars:
PORT: '{{.FRONTEND_PORT}}'
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
OPEN: "true"
dev:all:
desc: "Start backend + frontend + engine concurrently on free ports"
vars:
@@ -74,7 +91,7 @@ tasks:
vars:
PORT: '{{.BACKEND_PORT}}'
AIENGINE_URL: 'http://localhost:{{.ENGINE_PORT}}'
- task: frontend:dev:prototypes
- task: frontend:dev
vars:
PORT: '{{.FRONTEND_PORT}}'
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
+9 -9
View File
@@ -1,20 +1,20 @@
###############################################################################
# Stirling-PDF SaaS local environment template.
# Stirling-PDF SaaS environment defaults.
#
# Copy this file to `.env.saas.local` (gitignored) and fill in real values.
# Loaded by `task backend:dev:saas` via Taskfile's `dotenv:` directive, then
# read by Spring Boot's `${...}` placeholders in application-saas.properties
# and application-dev.properties.
# This file is committed and provides non-secret defaults loaded by
# `task backend:dev:saas`. Put real values for secrets (passwords, project
# refs, edge function secrets) in `.env.saas.local` - any variable set there
# takes precedence over what's defined here.
#
# DO NOT commit `.env.saas.local`. Only `.env.saas.example` is checked in.
# DO NOT commit `.env.saas.local`. Only `.env.saas` is checked in.
###############################################################################
# ---------- Supabase project ----------
# Project reference (the subdomain part of <ref>.supabase.co). Required.
# Example dev project:
# Set in .env.saas.local.
SAAS_DB_PROJECT_REF=
# Edge function secret used by billing/license rollup calls.
# Edge function secret used by billing/license rollup calls. Set in .env.saas.local.
SUPABASE_EDGE_FUNCTION_SECRET=
# ---------- Database (saas profile) ----------
@@ -28,7 +28,7 @@ SAAS_DB_PASSWORD=
# ---------- Database (dev profile overrides) ----------
# Used when `--spring.profiles.include=dev` is active. The dev profile
# defaults the URL/username to the shared dev Supabase project, but the
# password must still be provided here.
# password must still be provided in .env.saas.local.
SAAS_DEV_DB_URL=
SAAS_DEV_DB_USERNAME=postgres
SAAS_DEV_DB_PASSWORD=
+3
View File
@@ -0,0 +1,3 @@
# Whitelist committed env defaults. `.env.saas.local` (and any other .env*)
# stays ignored via the root .gitignore.
!.env.saas
+4
View File
@@ -44,6 +44,10 @@
"moduleName": ".*",
"moduleLicense": "The MIT License"
},
{
"moduleName": ".*",
"moduleLicense": "MIT-0"
},
{
"moduleName": "com.github.jai-imageio:jai-imageio-core",
"moduleLicense": "LICENSE.txt"
+2 -2
View File
@@ -60,7 +60,7 @@ dependencies {
exclude group: 'com.google.code.gson', module: 'gson'
}
api 'com.stirling:jpdfium:1.0.1'
api 'com.stirling:jpdfium:1.0.2'
// -PjpdfiumPlatforms=all|<csv of linux-x64,linux-arm64,darwin-x64,darwin-arm64,windows-x64>
def jpdfiumPlatformsProp = (project.findProperty('jpdfiumPlatforms') ?: 'all').toString().trim()
@@ -75,7 +75,7 @@ dependencies {
}
logger.lifecycle("JPDFium native platforms: ${jpdfiumPlatforms.join(', ')}")
jpdfiumPlatforms.each { platform ->
runtimeOnly "com.stirling:jpdfium-natives-${platform}:1.0.1"
runtimeOnly "com.stirling:jpdfium-natives-${platform}:1.0.2"
}
// Bucket4j (local in-process token bucket for RateLimitStore default impl)
@@ -0,0 +1,142 @@
package stirling.software.SPDF.pdf.parser;
import java.util.ArrayList;
import java.util.List;
/**
* Detects whether a page is one- or two-column from per-line bounding boxes, and classifies an
* X-span into the column it belongs to. Detection is a midpoint vote at {@code pageWidth / 2}.
*
* <p>Capped at two columns by design — sufficient for the redaction target set (single-column
* documents and IEEE-style two-column papers). 3+ column layouts (newspapers, magazines) and
* off-centre gutters (asymmetric two-column) would need a histogram or clustering approach to
* detect the actual gutter X. (future work)
*
* <p>Coordinates are PDFTextStripper screen space (top-left origin, Y increases downward).
*/
public final class PageColumnLayout {
/**
* Slack when checking "crosses a gutter" so single-pixel overshoots don't mark a line as
* spanning.
*/
public static final float SPAN_SLACK_PT = 2f;
/**
* Slack on each side of the page midpoint inside which a line is considered "spanning"
* (covering both columns) rather than belonging to one side.
*/
private static final float MIDPOINT_SLACK_PT = 30f;
/**
* Minimum line width (points) for a line to count toward the two-column tally. Avoids false
* positives where right-aligned dates, page numbers, or short "Link" fragments next to a
* heading look like a second column when they're really just inline metadata.
*/
private static final float MIN_COLUMN_LINE_WIDTH_PT = 100f;
/**
* Minimum number of clearly leftish AND clearly rightish lines (each of width &ge; {@link
* #MIN_COLUMN_LINE_WIDTH_PT}) required to call the page two-column. Anything below this falls
* back to single-column.
*/
private static final int MIN_SIDE_LINES = 3;
private final List<float[]> columns;
private final List<float[]> gutters;
private PageColumnLayout(List<float[]> columns, List<float[]> gutters) {
this.columns = columns;
this.gutters = gutters;
}
/**
* Determines column layout from per-line bounding boxes ({@code [x1, _, x2, _]}). Counts lines
* whose X-midpoint sits clearly left of, or clearly right of, the page midpoint (with {@link
* #MIDPOINT_SLACK_PT} slack each side). If both sides have at least {@link #MIN_SIDE_LINES}
* lines, the page is treated as two-column with the gutter at the page midpoint. Otherwise it's
* single-column.
*
* <p>Cross-column lines must already be split: callers should feed boxes from a line extractor
* that splits same-Y glyphs at large X gaps (see {@code AllTextLineExtractor}). Without that
* split, IEEE-style aligned-baseline 2-column PDFs produce one wide merged box per row and the
* side tallies all end up classified as "spanning", falling to single-column.
*/
public static PageColumnLayout fromLineBoxes(List<float[]> lineBoxes, float pageWidth) {
if (lineBoxes == null || lineBoxes.isEmpty()) {
return new PageColumnLayout(List.of(new float[] {0f, pageWidth}), List.of());
}
float pageMid = pageWidth / 2f;
int left = 0, right = 0;
for (float[] lb : lineBoxes) {
if (lb == null || lb.length < 3) continue;
float width = lb[2] - lb[0];
// Skip narrow lines — dates, page numbers, "Link" labels next to a heading should
// not, on their own, make a single-column doc look two-column.
if (width < MIN_COLUMN_LINE_WIDTH_PT) continue;
float mid = (lb[0] + lb[2]) * 0.5f;
if (mid < pageMid - MIDPOINT_SLACK_PT) left++;
else if (mid > pageMid + MIDPOINT_SLACK_PT) right++;
}
if (left < MIN_SIDE_LINES || right < MIN_SIDE_LINES) {
return new PageColumnLayout(List.of(new float[] {0f, pageWidth}), List.of());
}
float gutterL = pageMid - MIDPOINT_SLACK_PT;
float gutterR = pageMid + MIDPOINT_SLACK_PT;
return new PageColumnLayout(
List.of(new float[] {0f, gutterL}, new float[] {gutterR, pageWidth}),
List.of(new float[] {gutterL, gutterR}));
}
/** All columns, left-to-right, as {@code [leftX, rightX]} pairs. Never empty. */
public List<float[]> columns() {
return columns;
}
/** Gutters between columns, left-to-right, as {@code [leftX, rightX]} pairs. */
public List<float[]> gutters() {
return gutters;
}
public int columnCount() {
return columns.size();
}
/**
* Returns the column index containing the X-midpoint of {@code [x1, x2]}, falling back to the
* closest column if the midpoint sits inside a gutter.
*/
public int columnOf(float x1, float x2) {
float mid = (x1 + x2) * 0.5f;
int best = 0;
float bestDist = Float.MAX_VALUE;
for (int i = 0; i < columns.size(); i++) {
float[] c = columns.get(i);
if (mid >= c[0] && mid <= c[1]) return i;
float dist = mid < c[0] ? c[0] - mid : mid - c[1];
if (dist < bestDist) {
bestDist = dist;
best = i;
}
}
return best;
}
/**
* Returns every column index whose X-range overlaps {@code [x1, x2]} with at least {@link
* #SPAN_SLACK_PT} of intrusion. A normal in-column line returns one index; a line crossing a
* gutter returns two or more.
*/
public int[] columnsCrossing(float x1, float x2) {
List<Integer> hits = new ArrayList<>();
for (int i = 0; i < columns.size(); i++) {
float[] c = columns.get(i);
float overlap = Math.min(x2, c[1]) - Math.max(x1, c[0]);
if (overlap > SPAN_SLACK_PT) hits.add(i);
}
if (hits.isEmpty()) hits.add(columnOf(x1, x2));
int[] out = new int[hits.size()];
for (int i = 0; i < hits.size(); i++) out[i] = hits.get(i);
return out;
}
}
@@ -0,0 +1,133 @@
package stirling.software.SPDF.pdf.parser;
import java.awt.geom.Point2D;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.pdfbox.contentstream.PDFGraphicsStreamEngine;
import org.apache.pdfbox.cos.COSName;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.graphics.image.PDImage;
import org.apache.pdfbox.util.Matrix;
/**
* PDFGraphicsStreamEngine that intercepts {@code drawImage} calls and records each image's bounding
* box in PDF user-space (origin bottom-left, Y up) by transforming the unit square through the
* current transformation matrix (CTM).
*
* <p>Usage:
*
* <pre>{@code
* PageImageLocator locator = new PageImageLocator(page, pageIndex);
* locator.processPage(page);
* List<ImageBox> boxes = locator.getImageBoxes();
* }</pre>
*
* <p>Each {@link ImageBox} carries the 0-based page index and the axis-aligned bounding box {@code
* (x1, y1, x2, y2)} in PDF user-space coordinates.
*/
public final class PageImageLocator extends PDFGraphicsStreamEngine {
/**
* Bounding box of a raster or vector image found on a PDF page.
*
* @param pageIndex 0-based page index
* @param x1 left edge in PDF user-space (origin bottom-left)
* @param y1 bottom edge in PDF user-space
* @param x2 right edge
* @param y2 top edge
*/
public record ImageBox(int pageIndex, float x1, float y1, float x2, float y2) {}
private final int pageIndex;
private final List<ImageBox> imageBoxes = new ArrayList<>();
private final Point2D.Float currentPoint = new Point2D.Float();
/**
* @param page the PDPage to process
* @param pageIndex 0-based index of this page in the document (stored on each returned {@link
* ImageBox})
*/
public PageImageLocator(PDPage page, int pageIndex) {
super(page);
this.pageIndex = pageIndex;
}
/** Returns all image bounding boxes collected during {@link #processPage}. */
public List<ImageBox> getImageBoxes() {
return imageBoxes;
}
@Override
public void drawImage(PDImage pdImage) throws IOException {
Matrix ctm = getGraphicsState().getCurrentTransformationMatrix();
// An image occupies the unit square (0,0)→(1,1) in image space.
// Transform all four corners through the CTM to get the page-space bounding box.
float a = ctm.getScaleX();
float b = ctm.getShearY();
float c = ctm.getShearX();
float d = ctm.getScaleY();
float e = ctm.getTranslateX();
float f = ctm.getTranslateY();
float[] xs = {e, a + e, c + e, a + c + e};
float[] ys = {f, b + f, d + f, b + d + f};
float x1 = Float.MAX_VALUE, y1 = Float.MAX_VALUE;
float x2 = -Float.MAX_VALUE, y2 = -Float.MAX_VALUE;
for (float x : xs) {
x1 = Math.min(x1, x);
x2 = Math.max(x2, x);
}
for (float y : ys) {
y1 = Math.min(y1, y);
y2 = Math.max(y2, y);
}
imageBoxes.add(new ImageBox(pageIndex, x1, y1, x2, y2));
}
// ---------- required abstract methods (no-op for path operations) ----------
@Override
public void appendRectangle(Point2D p0, Point2D p1, Point2D p2, Point2D p3) {}
@Override
public void clip(int windingRule) {}
@Override
public void moveTo(float x, float y) {
currentPoint.setLocation(x, y);
}
@Override
public void lineTo(float x, float y) {
currentPoint.setLocation(x, y);
}
@Override
public void curveTo(float x1, float y1, float x2, float y2, float x3, float y3) {
currentPoint.setLocation(x3, y3);
}
@Override
public Point2D getCurrentPoint() {
return currentPoint;
}
@Override
public void closePath() {}
@Override
public void endPath() {}
@Override
public void strokePath() {}
@Override
public void fillPath(int windingRule) {}
@Override
public void fillAndStrokePath(int windingRule) {}
@Override
public void shadingFill(COSName shadingName) {}
}
@@ -77,6 +77,10 @@ public @interface AutoJobPostMapping {
/**
* Relative resource weight (1-100). See {@link
* stirling.software.common.enumeration.ResourceWeight} for the standard tiers.
*
* <p>The default is a sentinel ({@link Integer#MIN_VALUE}); {@code
* AutoJobPostMappingWeightTest} fails the build if any endpoint leaves it unset. Runtime
* readers clamp the value into {@code [1, 100]}.
*/
int resourceWeight() default 1;
int resourceWeight() default Integer.MIN_VALUE;
}
@@ -2,7 +2,13 @@ package stirling.software.common.cluster;
import java.time.Duration;
/** Token-bucket rate limiting backed by the cluster backplane. */
/**
* Token-bucket rate limiting backed by the cluster backplane.
*
* <p>In-process implementations enforce a per-JVM limit; distributed implementations enforce a
* single global limit across every node. Both use a Bucket4j greedy-refill token bucket so the
* semantics match across single-node and cluster deployments.
*/
public interface RateLimitStore {
/**
@@ -80,6 +80,7 @@ public class ConfigInitializer {
YamlHelper settingsFile = new YamlHelper(settingTempPath);
migrateEnterpriseEditionToPremium(settingsFile, settingsTemplateFile);
migrateProFeaturesKeyCasing(settingsFile, settingsTemplateFile);
boolean changesMade =
settingsTemplateFile.updateValuesFromYaml(settingsFile, settingsTemplateFile);
@@ -116,31 +117,52 @@ public class ConfigInitializer {
}
if (yaml.getValueByExactKeyPath("enterpriseEdition", "SSOAutoLogin") != null) {
template.updateValue(
List.of("premium", "proFeatures", "SSOAutoLogin"),
List.of("premium", "proFeatures", "ssoAutoLogin"),
yaml.getValueByExactKeyPath("enterpriseEdition", "SSOAutoLogin"));
}
if (yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "autoUpdateMetadata")
!= null) {
template.updateValue(
List.of("premium", "proFeatures", "CustomMetadata", "autoUpdateMetadata"),
List.of("premium", "proFeatures", "customMetadata", "autoUpdateMetadata"),
yaml.getValueByExactKeyPath(
"enterpriseEdition", "CustomMetadata", "autoUpdateMetadata"));
}
if (yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "author") != null) {
template.updateValue(
List.of("premium", "proFeatures", "CustomMetadata", "author"),
List.of("premium", "proFeatures", "customMetadata", "author"),
yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "author"));
}
if (yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "creator") != null) {
template.updateValue(
List.of("premium", "proFeatures", "CustomMetadata", "creator"),
List.of("premium", "proFeatures", "customMetadata", "creator"),
yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "creator"));
}
if (yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "producer")
!= null) {
template.updateValue(
List.of("premium", "proFeatures", "CustomMetadata", "producer"),
List.of("premium", "proFeatures", "customMetadata", "producer"),
yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "producer"));
}
}
// TODO: Remove post migration
// settings.yml.template renamed the two non-camelCase proFeatures keys
// ("SSOAutoLogin" -> "ssoAutoLogin", "CustomMetadata" -> "customMetadata") so the whole
// settings pipeline is consistent camelCase. The save path (YamlHelper.updateValue) matches
// keys case-sensitively, so without this carry-forward an existing install's values written
// under the old PascalCase keys would be dropped on upgrade and reset to template defaults.
void migrateProFeaturesKeyCasing(YamlHelper yaml, YamlHelper template) {
Object ssoAutoLogin = yaml.getValueByExactKeyPath("premium", "proFeatures", "SSOAutoLogin");
if (ssoAutoLogin != null) {
template.updateValue(List.of("premium", "proFeatures", "ssoAutoLogin"), ssoAutoLogin);
}
for (String field : List.of("autoUpdateMetadata", "author", "creator", "producer")) {
Object value =
yaml.getValueByExactKeyPath("premium", "proFeatures", "CustomMetadata", field);
if (value != null) {
template.updateValue(
List.of("premium", "proFeatures", "customMetadata", field), value);
}
}
}
}
@@ -528,6 +528,16 @@ public class ApplicationProperties {
private String provider;
private Client client = new Client();
/**
* When true, the OAuth2/OIDC login flow logs the full set of ID token and UserInfo
* claims at INFO level (and again at ERROR level if the username attribute cannot be
* resolved). Used to diagnose provider misconfiguration (for example ADFS not returning
* an {@code email} claim). WARNING: writes PII (sub, email, name) to application logs.
* Leave disabled in production; enable only while actively troubleshooting and disable
* again afterwards.
*/
private Boolean debugLogging = false;
public void setScopes(String scopes) {
List<String> scopesList =
Arrays.stream(scopes.split(",")).map(String::trim).toList();
@@ -778,6 +788,7 @@ public class ApplicationProperties {
private boolean enabled = false;
private String provider = "local";
private Local local = new Local();
private S3 s3 = new S3();
private Quotas quotas = new Quotas();
private Sharing sharing = new Sharing();
private Signing signing = new Signing();
@@ -787,6 +798,57 @@ public class ApplicationProperties {
private String basePath = InstallationPathConfig.getPath() + "storage";
}
@Data
public static class S3 {
/**
* Optional custom endpoint (e.g. {@code https://<account>.r2.cloudflarestorage.com},
* {@code https://<project>.supabase.co/storage/v1/s3}, or {@code http://localhost:9000}
* for MinIO). Blank = use AWS regional default.
*/
private String endpoint = "";
private String bucket = "";
private String region = "us-east-1";
private String accessKey = "";
private String secretKey = "";
/**
* When {@code true} use path-style URLs ({@code <endpoint>/<bucket>/<key>}) instead of
* virtual-hosted ({@code <bucket>.<endpoint>/<key>}). MinIO and most S3-compatible
* gateways require path-style; AWS S3 prefers virtual-hosted.
*/
private boolean pathStyleAccess = false;
/**
* When {@code false} (default), {@code endpoint} hostnames that resolve to private,
* loopback, or link-local addresses are rejected at startup to block SSRF attacks via
* the cloud metadata service (e.g. {@code http://169.254.169.254/}). Set to {@code
* true} to opt in for MinIO / in-cluster S3 endpoints on private networks.
*/
private boolean allowPrivateEndpoints = false;
/**
* Controls when the SDK adds an {@code x-amz-checksum-*} header on PUT/UploadPart.
* Default {@code WHEN_SUPPORTED} (the SDK default since 2.30) makes the SDK send a
* CRC32 checksum on every upload - this works on AWS S3, MinIO, current Supabase,
* Backblaze B2 (post-July-2025), and modern R2. Set to {@code WHEN_REQUIRED} to
* suppress the auto-checksum on vendors that reject unknown {@code x-amz-checksum-*}
* headers (older Backblaze B2, some R2 corner cases, GCS S3 endpoint). Invalid values
* fall back to {@code WHEN_SUPPORTED}.
*/
private String requestChecksumCalculation = "WHEN_SUPPORTED";
/**
* Controls when the SDK validates returned {@code x-amz-checksum-*} headers on GET
* responses. Default {@code WHEN_SUPPORTED}. Set to {@code WHEN_REQUIRED} if your
* vendor never returns these headers and you see false-positive checksum-mismatch
* errors. Invalid values fall back to {@code WHEN_SUPPORTED}.
*/
private String responseChecksumValidation = "WHEN_SUPPORTED";
}
@Data
public static class Sharing {
private boolean enabled = false;
@@ -57,85 +57,38 @@ public class JobExecutorService {
this.resourceMonitor = resourceMonitor;
this.jobQueue = jobQueue;
// Parse session timeout and calculate effective timeout once during initialization
long sessionTimeoutMs = parseSessionTimeout(sessionTimeout);
this.effectiveTimeoutMs = Math.min(asyncRequestTimeoutMs, sessionTimeoutMs);
log.debug(
"Job executor configured with effective timeout of {} ms", this.effectiveTimeoutMs);
}
/**
* Run a job either asynchronously or synchronously
*
* @param async Whether to run the job asynchronously
* @param work The work to be done
* @return The response
*/
public ResponseEntity<?> runJobGeneric(boolean async, Supplier<Object> work) {
return runJobGeneric(async, work, -1);
}
/**
* Run a job either asynchronously or synchronously with a custom timeout
*
* @param async Whether to run the job asynchronously
* @param work The work to be done
* @param customTimeoutMs Custom timeout in milliseconds, or -1 to use the default
* @return The response
*/
public ResponseEntity<?> runJobGeneric(
boolean async, Supplier<Object> work, long customTimeoutMs) {
return runJobGeneric(async, work, customTimeoutMs, false, 50);
}
/**
* Run a job either asynchronously or synchronously with custom parameters
*
* @param async Whether to run the job asynchronously
* @param work The work to be done
* @param customTimeoutMs Custom timeout in milliseconds, or -1 to use the default
* @param queueable Whether this job can be queued when system resources are limited
* @param resourceWeight The resource weight of this job (1-100)
* @return The response
*/
public ResponseEntity<?> runJobGeneric(
boolean async,
Supplier<Object> work,
long customTimeoutMs,
boolean queueable,
int resourceWeight) {
// Generate base UUID
String baseJobId = UUID.randomUUID().toString();
// Scope job to authenticated user if security is enabled
String scopedJobKey = getScopedJobKey(baseJobId);
log.debug("Generated jobId: {} (base: {})", scopedJobKey, baseJobId);
// Store the scoped job ID in the request for potential use by other components
if (request != null) {
request.setAttribute("jobId", scopedJobKey);
// Also track this job ID in the user's session for authorization purposes
// This ensures users can only cancel their own jobs
if (request.getSession() != null) {
@SuppressWarnings("unchecked")
java.util.Set<String> userJobIds =
(java.util.Set<String>) request.getSession().getAttribute("userJobIds");
if (userJobIds == null) {
userJobIds = new java.util.concurrent.ConcurrentSkipListSet<>();
request.getSession().setAttribute("userJobIds", userJobIds);
}
userJobIds.add(scopedJobKey);
log.debug("Added scoped job ID {} to user session", scopedJobKey);
}
}
String jobId = scopedJobKey;
// Determine which timeout to use
long timeoutToUse = customTimeoutMs > 0 ? customTimeoutMs : effectiveTimeoutMs;
log.debug(
@@ -146,7 +99,6 @@ public class JobExecutorService {
queueable,
resourceWeight);
// Check if we need to queue this job based on resource availability
boolean shouldQueue =
queueable
&& async
@@ -154,7 +106,6 @@ public class JobExecutorService {
resourceMonitor.shouldQueueJob(resourceWeight);
if (shouldQueue) {
// Queue the job instead of executing immediately
log.debug(
"Queueing job {} due to resource constraints (weight: {})",
jobId,
@@ -162,18 +113,12 @@ public class JobExecutorService {
taskManager.createTask(jobId);
// Create a specialized wrapper that updates the TaskManager
final String capturedJobIdForQueue = jobId;
Supplier<Object> wrappedWork =
() -> {
try {
// Set jobId in ThreadLocal context for the queued job
stirling.software.common.util.JobContext.setJobId(
capturedJobIdForQueue);
log.debug(
"Set jobId {} in JobContext for queued job execution",
capturedJobIdForQueue);
Object result = work.get();
processJobResult(capturedJobIdForQueue, result);
return result;
@@ -186,21 +131,17 @@ public class JobExecutorService {
taskManager.setError(capturedJobIdForQueue, e.getMessage());
throw e;
} finally {
// Clean up ThreadLocal to avoid memory leaks
stirling.software.common.util.JobContext.clear();
}
};
// Queue the job and get the future
CompletableFuture<ResponseEntity<?>> future =
jobQueue.queueJob(jobId, resourceWeight, wrappedWork, timeoutToUse);
// Return immediately with job ID
return ResponseEntity.ok().body(new JobResponse<>(true, jobId, null));
} else if (async) {
taskManager.createTask(jobId);
// Capture the jobId for the async thread
final String capturedJobId = jobId;
executor.execute(
@@ -211,13 +152,7 @@ public class JobExecutorService {
capturedJobId,
timeoutToUse);
// Set jobId in ThreadLocal context for the async thread
stirling.software.common.util.JobContext.setJobId(capturedJobId);
log.debug(
"Set jobId {} in JobContext for async execution",
capturedJobId);
// Execute with timeout
Object result = executeWithTimeout(() -> work.get(), timeoutToUse);
processJobResult(capturedJobId, result);
} catch (TimeoutException te) {
@@ -227,7 +162,6 @@ public class JobExecutorService {
log.error("Error executing job {}: {}", jobId, e.getMessage(), e);
taskManager.setError(jobId, e.getMessage());
} finally {
// Clean up ThreadLocal to avoid memory leaks
stirling.software.common.util.JobContext.clear();
}
});
@@ -237,27 +171,19 @@ public class JobExecutorService {
try {
log.debug("Running sync job with timeout {} ms", timeoutToUse);
// Make jobId available to downstream components on the worker thread
stirling.software.common.util.JobContext.setJobId(jobId);
log.debug("Set jobId {} in JobContext for sync execution", jobId);
// Execute with timeout
Object result = executeWithTimeout(() -> work.get(), timeoutToUse);
// If the result is already a ResponseEntity, return it directly
if (result instanceof ResponseEntity) {
return (ResponseEntity<?>) result;
}
// Process different result types
return handleResultForSyncJob(result);
} catch (TimeoutException te) {
log.error("Synchronous job timed out after {} ms", timeoutToUse);
return ResponseEntity.internalServerError()
.body(Map.of("error", "Job timed out after " + timeoutToUse + " ms"));
} catch (RuntimeException e) {
// Check if this is a typed exception that should be handled by
// GlobalExceptionHandler (either directly or wrapped)
Throwable cause = e.getCause();
if (e instanceof IllegalArgumentException
|| cause
@@ -267,16 +193,13 @@ public class JobExecutorService {
instanceof
stirling.software.common.util.ExceptionUtils
.BaseValidationException) {
// Rethrow so GlobalExceptionHandler can handle with proper HTTP status codes
throw e;
}
// Handle other RuntimeExceptions as generic errors
log.error("Error executing synchronous job: {}", e.getMessage(), e);
return ResponseEntity.internalServerError()
.body(Map.of("error", "Job failed: " + e.getMessage()));
} catch (Exception e) {
log.error("Error executing synchronous job: {}", e.getMessage(), e);
// Construct a JSON error response
return ResponseEntity.internalServerError()
.body(Map.of("error", "Job failed: " + e.getMessage()));
} finally {
@@ -285,23 +208,13 @@ public class JobExecutorService {
}
}
/**
* Process the result of an asynchronous job
*
* @param jobId The job ID
* @param result The result
*/
private void processJobResult(String jobId, Object result) {
try {
if (result instanceof byte[]) {
// Store byte array directly to disk to avoid double memory consumption
String fileId = fileStorage.storeBytes((byte[]) result, "result.pdf");
taskManager.setFileResult(
jobId, fileId, "result.pdf", MediaType.APPLICATION_PDF_VALUE);
log.debug("Stored byte[] result with fileId: {}", fileId);
// Let the byte array get collected naturally in the next GC cycle
// We don't need to force System.gc() which can be harmful
} else if (result instanceof ResponseEntity) {
ResponseEntity<?> response = (ResponseEntity<?>) result;
Object body = response.getBody();
@@ -330,16 +243,13 @@ public class JobExecutorService {
taskManager.setFileResult(jobId, fileId, filename, contentType);
log.debug("Stored ResponseEntity<Resource> result with fileId: {}", fileId);
} else {
// Check if the response body contains a fileId
if (body != null && body.toString().contains("fileId")) {
try {
// Try to extract fileId using reflection
java.lang.reflect.Method getFileId =
body.getClass().getMethod("getFileId");
String fileId = (String) getFileId.invoke(body);
if (fileId != null && !fileId.isEmpty()) {
// Try to get filename and content type
String filename = "result.pdf";
String contentType = MediaType.APPLICATION_PDF_VALUE;
@@ -379,7 +289,6 @@ public class JobExecutorService {
}
}
// Store generic result
taskManager.setResult(jobId, body);
}
} else if (result instanceof MultipartFile file) {
@@ -388,16 +297,13 @@ public class JobExecutorService {
jobId, fileId, file.getOriginalFilename(), file.getContentType());
log.debug("Stored MultipartFile result with fileId: {}", fileId);
} else {
// Check if result has a fileId field
if (result != null) {
try {
// Try to extract fileId using reflection
java.lang.reflect.Method getFileId =
result.getClass().getMethod("getFileId");
String fileId = (String) getFileId.invoke(result);
if (fileId != null && !fileId.isEmpty()) {
// Try to get filename and content type
String filename = "result.pdf";
String contentType = MediaType.APPLICATION_PDF_VALUE;
@@ -435,7 +341,6 @@ public class JobExecutorService {
}
}
// Default case: store the result as is
taskManager.setResult(jobId, result);
}
@@ -446,16 +351,8 @@ public class JobExecutorService {
}
}
/**
* Handle different result types for synchronous jobs
*
* @param result The result object
* @return The appropriate ResponseEntity
* @throws IOException If there is an error processing the result
*/
private ResponseEntity<?> handleResultForSyncJob(Object result) throws IOException {
if (result instanceof byte[]) {
// Return byte array as PDF
return ResponseEntity.ok()
.contentType(MediaType.APPLICATION_PDF)
.header(
@@ -463,7 +360,6 @@ public class JobExecutorService {
"form-data; name=\"attachment\"; filename=\"result.pdf\"")
.body(result);
} else if (result instanceof MultipartFile file) {
// Return MultipartFile content
return ResponseEntity.ok()
.contentType(MediaType.parseMediaType(file.getContentType()))
.header(
@@ -473,7 +369,6 @@ public class JobExecutorService {
+ "\"")
.body(file.getBytes());
} else {
// Default case: return as JSON
return ResponseEntity.ok(result);
}
}
@@ -493,15 +388,9 @@ public class JobExecutorService {
return mediaType != null ? mediaType.toString() : MediaType.APPLICATION_PDF_VALUE;
}
/**
* Parse session timeout string (e.g., "30m", "1h") to milliseconds
*
* @param timeout The timeout string
* @return The timeout in milliseconds
*/
private long parseSessionTimeout(String timeout) {
if (timeout == null || timeout.isEmpty()) {
return 30 * 60 * 1000; // Default: 30 minutes
return 30 * 60 * 1000;
}
try {
@@ -523,27 +412,16 @@ public class JobExecutorService {
case "m" -> (long) (numericValue * 60 * 1000);
case "h" -> (long) (numericValue * 60 * 60 * 1000);
case "d" -> (long) (numericValue * 24 * 60 * 60 * 1000);
default -> (long) (numericValue * 60 * 1000); // Default to minutes
default -> (long) (numericValue * 60 * 1000);
};
} catch (Exception e) {
log.warn("Could not parse session timeout '{}', using default", timeout);
return 30 * 60 * 1000; // Default: 30 minutes
return 30 * 60 * 1000;
}
}
/**
* Execute a supplier with a timeout
*
* @param supplier The supplier to execute
* @param timeoutMs The timeout in milliseconds
* @return The result from the supplier
* @throws TimeoutException If the execution times out
* @throws Exception If the supplier throws an exception
*/
private <T> T executeWithTimeout(Supplier<T> supplier, long timeoutMs)
throws TimeoutException, Exception {
// Use the same executor as other async jobs for consistency
// This ensures all operations run on the same thread pool
String currentJobId = stirling.software.common.util.JobContext.getJobId();
java.util.concurrent.CompletableFuture<T> future =
@@ -577,17 +455,10 @@ public class JobExecutorService {
}
}
/**
* Get a scoped job key that includes user ownership when security is enabled.
*
* @param baseJobId the base job identifier
* @return scoped job key, or just baseJobId if no ownership service available
*/
private String getScopedJobKey(String baseJobId) {
if (jobOwnershipService != null) {
return jobOwnershipService.createScopedJobKey(baseJobId);
}
// Security disabled, return unsecured job key
return baseJobId;
}
}
@@ -1,11 +1,21 @@
package stirling.software.common.service;
import java.util.List;
/** Provides metadata about tool endpoints for internal dispatch. */
public interface ToolMetadataService {
/** Returns true if the given operation path accepts multiple input files. */
boolean isMultiInput(String operationPath);
/**
* Returns the file extensions (lowercase, no leading dot, e.g. {@code "pdf"}) that the
* operation accepts as input ({@code output=false}) or produces as output ({@code
* output=true}), derived from the endpoint's declared type. Returns {@code null} when the
* endpoint declares no specific type, which callers should treat as "any type accepted".
*/
List<String> getExtensionTypes(boolean output, String operationPath);
/**
* Returns true when the endpoint's ZIP response is a transport for multiple typed results and
* should be unpacked: multi-output endpoints (Type:SIMO / Type:MIMO) and wrapper declarations
@@ -0,0 +1,310 @@
package stirling.software.common.util;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.Set;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import java.util.zip.ZipOutputStream;
import javax.xml.XMLConstants;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import org.springframework.stereotype.Component;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.NamedNodeMap;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.SAXException;
import io.github.pixee.security.ZipSecurity;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.SsrfProtectionService;
// Strips external refs from OOXML/ODF uploads so LibreOffice can't be made to fetch them.
@Component
@Slf4j
public class OfficeDocumentSanitizer {
private static final Set<String> OOXML_EXTENSIONS =
Set.of(
"docx", "docm", "dotx", "dotm", "xlsx", "xlsm", "xltx", "xltm", "pptx", "pptm",
"potx", "potm", "ppsx", "ppsm");
private static final Set<String> ODF_EXTENSIONS =
Set.of(
"odt", "ott", "ods", "ots", "odp", "otp", "odg", "otg", "odf", "odc", "odi",
"odm");
private static final Set<String> ODF_XML_PARTS =
Set.of("content.xml", "styles.xml", "meta.xml", "settings.xml");
private final SsrfProtectionService ssrfProtectionService;
private final ApplicationProperties applicationProperties;
public OfficeDocumentSanitizer(
SsrfProtectionService ssrfProtectionService,
ApplicationProperties applicationProperties) {
this.ssrfProtectionService = ssrfProtectionService;
this.applicationProperties = applicationProperties;
}
public boolean isSanitizableExtension(String extension) {
if (extension == null) {
return false;
}
String lower = extension.toLowerCase(Locale.ROOT);
return OOXML_EXTENSIONS.contains(lower) || ODF_EXTENSIONS.contains(lower);
}
public byte[] sanitize(byte[] documentBytes, String extension) throws IOException {
if (documentBytes == null || documentBytes.length == 0) {
throw new IOException("Office document input is empty or null");
}
if (applicationProperties.getSystem().isDisableSanitize()) {
log.debug("Office document sanitization disabled by configuration");
return documentBytes;
}
if (!isSanitizableExtension(extension)) {
return documentBytes;
}
ByteArrayOutputStream out = new ByteArrayOutputStream(documentBytes.length);
try (ZipInputStream zipIn =
ZipSecurity.createHardenedInputStream(
new ByteArrayInputStream(documentBytes));
ZipOutputStream zipOut = new ZipOutputStream(out)) {
ZipEntry entry;
while ((entry = zipIn.getNextEntry()) != null) {
String name = entry.getName();
byte[] bytes = entry.isDirectory() ? new byte[0] : zipIn.readAllBytes();
if (!entry.isDirectory()) {
bytes = sanitizeEntry(name, bytes);
}
ZipEntry outEntry = new ZipEntry(name);
if (entry.getComment() != null) {
outEntry.setComment(entry.getComment());
}
if (entry.getExtra() != null) {
outEntry.setExtra(entry.getExtra());
}
zipOut.putNextEntry(outEntry);
if (!entry.isDirectory()) {
zipOut.write(bytes);
}
zipOut.closeEntry();
}
}
return out.toByteArray();
}
private byte[] sanitizeEntry(String entryName, byte[] entryBytes) {
String lower = entryName.toLowerCase(Locale.ROOT);
try {
if (lower.endsWith(".rels")) {
return sanitizeOoxmlRels(entryBytes);
}
if (isOdfXmlPart(lower)) {
return sanitizeOdfXml(entryBytes);
}
} catch (ParserConfigurationException
| SAXException
| IOException
| TransformerException e) {
log.warn(
"Failed to parse XML part '{}' for sanitization, leaving as-is: {}",
entryName,
e.getMessage());
}
return entryBytes;
}
private boolean isOdfXmlPart(String lowerName) {
int slash = lowerName.lastIndexOf('/');
String base = slash >= 0 ? lowerName.substring(slash + 1) : lowerName;
return ODF_XML_PARTS.contains(base);
}
private byte[] sanitizeOoxmlRels(byte[] xmlBytes)
throws IOException, ParserConfigurationException, SAXException, TransformerException {
Document doc = parseSecurely(xmlBytes);
Element root = doc.getDocumentElement();
if (root == null) {
return xmlBytes;
}
NodeList relationships = root.getElementsByTagNameNS("*", "Relationship");
List<Node> toRemove = new ArrayList<>();
for (int i = 0; i < relationships.getLength(); i++) {
Node node = relationships.item(i);
NamedNodeMap attrs = node.getAttributes();
if (attrs == null) {
continue;
}
Node targetMode = attrs.getNamedItem("TargetMode");
if (targetMode == null || !"external".equalsIgnoreCase(targetMode.getNodeValue())) {
continue;
}
Node target = attrs.getNamedItem("Target");
String targetValue = target == null ? "" : target.getNodeValue();
if (isAdminAllowed(targetValue)) {
continue;
}
log.warn(
"Stripping OOXML external relationship target: {}",
truncateForLog(targetValue));
toRemove.add(node);
}
if (toRemove.isEmpty()) {
return xmlBytes;
}
for (Node n : toRemove) {
n.getParentNode().removeChild(n);
}
return serializeDocument(doc);
}
private byte[] sanitizeOdfXml(byte[] xmlBytes)
throws IOException, ParserConfigurationException, SAXException, TransformerException {
Document doc = parseSecurely(xmlBytes);
Element root = doc.getDocumentElement();
if (root == null) {
return xmlBytes;
}
boolean modified = stripExternalHrefs(root);
if (!modified) {
return xmlBytes;
}
return serializeDocument(doc);
}
private boolean stripExternalHrefs(Node node) {
boolean modified = false;
if (node.getNodeType() == Node.ELEMENT_NODE) {
NamedNodeMap attrs = node.getAttributes();
List<String> hrefAttrsToRemove = new ArrayList<>();
for (int i = 0; i < attrs.getLength(); i++) {
Node attr = attrs.item(i);
String name = attr.getNodeName();
if (name == null) {
continue;
}
String lower = name.toLowerCase(Locale.ROOT);
if (!(lower.equals("xlink:href")
|| lower.endsWith(":href")
|| lower.equals("href"))) {
continue;
}
String value = attr.getNodeValue();
if (!isExternalUrl(value)) {
continue;
}
if (isAdminAllowed(value)) {
continue;
}
log.warn(
"Stripping ODF external href attribute ({}): {}",
name,
truncateForLog(value));
hrefAttrsToRemove.add(name);
}
Element element = (Element) node;
for (String attrName : hrefAttrsToRemove) {
element.removeAttribute(attrName);
modified = true;
}
}
NodeList children = node.getChildNodes();
for (int i = 0; i < children.getLength(); i++) {
if (stripExternalHrefs(children.item(i))) {
modified = true;
}
}
return modified;
}
private boolean isExternalUrl(String url) {
if (url == null) {
return false;
}
String trimmed = url.trim().toLowerCase(Locale.ROOT);
if (trimmed.isEmpty() || trimmed.startsWith("#") || trimmed.startsWith("../")) {
return false;
}
return trimmed.startsWith("http://")
|| trimmed.startsWith("https://")
|| trimmed.startsWith("ftp://")
|| trimmed.startsWith("ftps://")
|| trimmed.startsWith("file:")
|| trimmed.startsWith("smb:")
|| trimmed.startsWith("\\\\")
|| trimmed.startsWith("//");
}
// Preserved only with an explicit allowedDomains entry; MEDIUM default would admit public URLs.
private boolean isAdminAllowed(String url) {
if (ssrfProtectionService == null || url == null || url.isBlank()) {
return false;
}
ApplicationProperties.Html.UrlSecurity config =
applicationProperties.getSystem().getHtml().getUrlSecurity();
if (config == null
|| config.getAllowedDomains() == null
|| config.getAllowedDomains().isEmpty()) {
return false;
}
return ssrfProtectionService.isUrlAllowed(url);
}
private Document parseSecurely(byte[] xmlBytes)
throws ParserConfigurationException, SAXException, IOException {
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
factory.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);
factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
factory.setFeature("http://xml.org/sax/features/external-general-entities", false);
factory.setFeature("http://xml.org/sax/features/external-parameter-entities", false);
factory.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);
factory.setXIncludeAware(false);
factory.setExpandEntityReferences(false);
factory.setNamespaceAware(true);
DocumentBuilder builder = factory.newDocumentBuilder();
return builder.parse(new ByteArrayInputStream(xmlBytes));
}
private byte[] serializeDocument(Document doc) throws TransformerException {
TransformerFactory tf = TransformerFactory.newInstance();
tf.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);
Transformer transformer = tf.newTransformer();
transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8");
transformer.setOutputProperty(OutputKeys.INDENT, "no");
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "no");
ByteArrayOutputStream baos = new ByteArrayOutputStream();
transformer.transform(new DOMSource(doc), new StreamResult(baos));
return baos.toByteArray();
}
private String truncateForLog(String value) {
if (value == null) {
return "null";
}
return value.length() > 80 ? value.substring(0, 80) + "..." : value;
}
}
@@ -83,7 +83,16 @@ public class RequestUriUtils {
return false;
}
// Blocklist of backend/non-frontend paths that should still go through filters
// Blocklist of backend/non-frontend paths that should still go through filters.
//
// `/files` was historically a backend route; it is now a frontend route
// owned by HomePage / FileManagerView. Direct-nav or refresh on /files
// (or /files/<folder-uuid>) was returning the Spring auth filter's 401
// JSON instead of serving index.html, so the SPA never got a chance to
// mount and the user saw a raw error response. There are no `/files`
// backend mappings at the servlet root - the real storage endpoints
// live under `/api/v1/storage/files`, which is filtered out a few lines
// up by the `startsWith("/api/")` guard.
String[] backendOnlyPrefixes = {
"/register",
"/pipeline",
@@ -91,7 +100,6 @@ public class RequestUriUtils {
"/pdfjs-legacy",
"/fonts",
"/images",
"/files",
"/css",
"/js",
"/swagger",
@@ -181,7 +189,7 @@ public class RequestUriUtils {
|| trimmedUri.startsWith(
"/api/v1/mobile-scanner/") // Mobile scanner endpoints (no auth)
|| trimmedUri.startsWith("/v1/api-docs")
// Workflow participant endpoints access controlled by share tokens, not login
// Workflow participant endpoints - access controlled by share tokens, not login
|| trimmedUri.startsWith("/api/v1/workflow/participant/")
// Share-link SPA bootstrap; data APIs remain protected
|| trimmedUri.matches("^/share/[^/]+/?$");
@@ -0,0 +1,48 @@
package stirling.software.common.util.propertyeditor;
import java.beans.PropertyEditorSupport;
import java.util.ArrayList;
import java.util.List;
import lombok.extern.slf4j.Slf4j;
import tools.jackson.core.type.TypeReference;
import tools.jackson.databind.DeserializationFeature;
import tools.jackson.databind.ObjectMapper;
import tools.jackson.databind.json.JsonMapper;
/**
* Binds a multipart form value containing a JSON array into a typed {@code List<T>}. Used for
* endpoints that accept structured list parameters via {@code @ModelAttribute} — the form field
* carries the full JSON array as its value and the editor parses it once.
*/
@Slf4j
public class JsonListPropertyEditor<T> extends PropertyEditorSupport {
private static final ObjectMapper OBJECT_MAPPER =
JsonMapper.builder()
.enable(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY)
.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
.build();
private final TypeReference<? extends List<T>> typeRef;
public JsonListPropertyEditor(TypeReference<? extends List<T>> typeRef) {
this.typeRef = typeRef;
}
@Override
public void setAsText(String text) throws IllegalArgumentException {
if (text == null || text.trim().isEmpty()) {
setValue(new ArrayList<T>());
return;
}
try {
setValue(OBJECT_MAPPER.readValue(text, typeRef));
} catch (Exception e) {
log.error("Failed to parse JSON list value", e);
throw new IllegalArgumentException(
"Expected a JSON array but could not parse: " + e.getMessage());
}
}
}
@@ -0,0 +1,41 @@
package stirling.software.common.util.propertyeditor;
import java.beans.PropertyEditorSupport;
import lombok.extern.slf4j.Slf4j;
import tools.jackson.databind.DeserializationFeature;
import tools.jackson.databind.ObjectMapper;
import tools.jackson.databind.json.JsonMapper;
/**
* Binds a multipart form value containing a JSON object into a typed {@code T}. Companion to {@link
* JsonListPropertyEditor} for single-object nested fields on {@code @ModelAttribute} endpoints.
*/
@Slf4j
public class JsonObjectPropertyEditor<T> extends PropertyEditorSupport {
private static final ObjectMapper OBJECT_MAPPER =
JsonMapper.builder().disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES).build();
private final Class<T> type;
public JsonObjectPropertyEditor(Class<T> type) {
this.type = type;
}
@Override
public void setAsText(String text) throws IllegalArgumentException {
if (text == null || text.trim().isEmpty()) {
setValue(null);
return;
}
try {
setValue(OBJECT_MAPPER.readValue(text, type));
} catch (Exception e) {
log.error("Failed to parse JSON object value", e);
throw new IllegalArgumentException(
"Expected a JSON object but could not parse: " + e.getMessage());
}
}
}
@@ -1,53 +0,0 @@
package stirling.software.common.util.propertyeditor;
import java.beans.PropertyEditorSupport;
import java.util.ArrayList;
import java.util.List;
import lombok.extern.slf4j.Slf4j;
import tools.jackson.databind.DeserializationFeature;
import tools.jackson.databind.JavaType;
import tools.jackson.databind.ObjectMapper;
import tools.jackson.databind.json.JsonMapper;
/**
* Spring property editor that decodes a JSON string into a typed {@link ArrayList}. Used to bind
* complex list parameters (e.g. {@code List<RedactionArea>}, {@code List<EditTextOperation>}) from
* multipart form fields, where Spring's default binding cannot deserialize a JSON array.
*/
@Slf4j
public class StringToArrayListPropertyEditor<T> extends PropertyEditorSupport {
private final ObjectMapper objectMapper =
JsonMapper.builder()
.enable(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY)
.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
.build();
private final Class<T> elementType;
public StringToArrayListPropertyEditor(Class<T> elementType) {
this.elementType = elementType;
}
@Override
public void setAsText(String text) throws IllegalArgumentException {
if (text == null || text.trim().isEmpty()) {
setValue(new ArrayList<>());
return;
}
try {
JavaType listType =
objectMapper
.getTypeFactory()
.constructCollectionType(ArrayList.class, elementType);
List<T> list = objectMapper.readValue(text, listType);
setValue(list);
} catch (Exception e) {
log.error("Exception while converting {}", e);
throw new IllegalArgumentException(
"Failed to convert java.lang.String to java.util.List");
}
}
}
@@ -0,0 +1,149 @@
package stirling.software.SPDF.pdf.parser;
import static org.assertj.core.api.Assertions.assertThat;
import java.util.ArrayList;
import java.util.List;
import org.junit.jupiter.api.Test;
/** Unit tests for {@link PageColumnLayout} gutter detection and column classification. */
class PageColumnLayoutTest {
private static final float PAGE_WIDTH = 612f; // Letter portrait
// ── single-column ────────────────────────────────────────────────────────────────────────────
@Test
void singleColumn_oneColumnNoGutters() {
List<float[]> lines = List.of(lineBox(72f, 396f));
PageColumnLayout layout = PageColumnLayout.fromLineBoxes(lines, PAGE_WIDTH);
assertThat(layout.columnCount()).isEqualTo(1);
assertThat(layout.gutters()).isEmpty();
}
@Test
void singleColumn_classifyAnchor_returnsZero() {
PageColumnLayout layout =
PageColumnLayout.fromLineBoxes(List.of(lineBox(72f, 396f)), PAGE_WIDTH);
assertThat(layout.columnOf(100f, 200f)).isEqualTo(0);
}
// ── two-column ───────────────────────────────────────────────────────────────────────────────
@Test
void twoColumn_detectsGutter() {
PageColumnLayout layout =
PageColumnLayout.fromLineBoxes(buildTwoColumnLines(3), PAGE_WIDTH);
assertThat(layout.columnCount()).isEqualTo(2);
assertThat(layout.gutters()).hasSize(1);
float[] gutter = layout.gutters().get(0);
// Gutter is centered on pageWidth/2 with PageColumnLayout.MIDPOINT_SLACK_PT slack each
// side.
float pageMid = PAGE_WIDTH / 2f;
assertThat(gutter[0]).isBetween(pageMid - 40f, pageMid - 20f);
assertThat(gutter[1]).isBetween(pageMid + 20f, pageMid + 40f);
}
@Test
void twoColumn_classifyLeftAndRightAnchors() {
PageColumnLayout layout =
PageColumnLayout.fromLineBoxes(buildTwoColumnLines(3), PAGE_WIDTH);
assertThat(layout.columnOf(100f, 200f)).isEqualTo(0);
assertThat(layout.columnOf(380f, 460f)).isEqualTo(1);
}
@Test
void twoColumn_columnsCrossing_leftLineOnlyHitsLeft() {
PageColumnLayout layout =
PageColumnLayout.fromLineBoxes(buildTwoColumnLines(3), PAGE_WIDTH);
assertThat(layout.columnsCrossing(72f, 280f)).containsExactly(0);
assertThat(layout.columnsCrossing(320f, 540f)).containsExactly(1);
}
@Test
void twoColumn_spanningLine_returnsBothColumns() {
List<float[]> lines = new ArrayList<>(buildTwoColumnLines(3));
// Full-width header that crosses pageWidth/2.
lines.add(lineBox(72f, 396f));
PageColumnLayout layout = PageColumnLayout.fromLineBoxes(lines, PAGE_WIDTH);
assertThat(layout.columnsCrossing(72f, 540f)).containsExactly(0, 1);
assertThat(layout.columnsCrossing(72f, 280f)).containsExactly(0);
}
private static List<float[]> buildTwoColumnLines(int rowsPerColumn) {
List<float[]> lines = new ArrayList<>();
for (int i = 0; i < rowsPerColumn; i++) {
lines.add(lineBox(72f, 136f)); // left column body (72..208)
lines.add(lineBox(320f, 220f)); // right column body (320..540)
}
return lines;
}
// ── three-column ─────────────────────────────────────────────────────────────────────────────
@Test
void threeColumn_collapsesToLeftRightSplit() {
// The midpoint-based detector splits the page at pageWidth/2 and treats anything else as
// single-column or spanning. A genuine 3-column layout collapses to 2 columns; the middle
// column's content ends up classified by midpoint as left or right of pageMid.
List<float[]> lines = new ArrayList<>();
for (int i = 0; i < 6; i++) {
lines.add(lineBox(72f, 150f)); // 72..222
lines.add(lineBox(252f, 150f)); // 252..402
lines.add(lineBox(432f, 150f)); // 432..582
}
PageColumnLayout layout = PageColumnLayout.fromLineBoxes(lines, PAGE_WIDTH);
assertThat(layout.columnCount()).isEqualTo(2);
assertThat(layout.gutters()).hasSize(1);
}
// ── empty page ───────────────────────────────────────────────────────────────────────────────
@Test
void emptyPage_singleColumnFallback() {
PageColumnLayout layout = PageColumnLayout.fromLineBoxes(List.of(), PAGE_WIDTH);
assertThat(layout.columnCount()).isEqualTo(1);
assertThat(layout.gutters()).isEmpty();
}
@Test
void onlyShortFragments_singleColumnFallback() {
// Page numbers / decorations — too narrow to vote either side.
List<float[]> lines = List.of(lineBox(300f, 6f));
PageColumnLayout layout = PageColumnLayout.fromLineBoxes(lines, PAGE_WIDTH);
assertThat(layout.columnCount()).isEqualTo(1);
}
// ── narrow gap should not be confused for a gutter ───────────────────────────────────────────
@Test
void narrowInternalGap_doesNotProduceGutter() {
// Both halves sit left of the page midpoint, so no line votes for a right column and
// detection falls back to single-column.
List<float[]> lines = new ArrayList<>();
lines.add(lineBox(72f, 100f));
lines.add(lineBox(180f, 100f));
for (int i = 0; i < 5; i++) {
lines.add(lineBox(72f, 208f));
}
PageColumnLayout layout = PageColumnLayout.fromLineBoxes(lines, PAGE_WIDTH);
assertThat(layout.columnCount()).isEqualTo(1);
}
// ── helpers ──────────────────────────────────────────────────────────────────────────────────
/** Builds a line bounding box {@code [x1, 0, x1+width, 0]}; Y is unused by detection. */
private static float[] lineBox(float x1, float width) {
return new float[] {x1, 0f, x1 + width, 0f};
}
}
@@ -24,7 +24,9 @@ class InProcessDistributedLockTest {
}
@Test
void reentryFromSameThreadFails() {
void reentryFromSameThreadFails_parityWithValkey() {
// The Valkey impl refuses reentry (SET NX semantics); the in-process impl must match,
// otherwise code working in single-instance silently breaks in cluster mode.
DistributedLock lock = new InProcessDistributedLock();
DistributedLock.LockHandle h1 = lock.tryAcquire("k", Duration.ofSeconds(30)).orElseThrow();
Optional<DistributedLock.LockHandle> reentry = lock.tryAcquire("k", Duration.ofSeconds(30));
@@ -0,0 +1,95 @@
package stirling.software.common.configuration;
import static org.junit.jupiter.api.Assertions.*;
import org.junit.jupiter.api.Test;
import org.snakeyaml.engine.v2.api.LoadSettings;
import stirling.software.common.util.YamlHelper;
class ConfigInitializerTest {
private static final LoadSettings LOAD_SETTINGS =
LoadSettings.builder()
.setUseMarks(true)
.setMaxAliasesForCollections(Integer.MAX_VALUE)
.setAllowRecursiveKeys(true)
.setParseComments(true)
.build();
// Mirrors the proFeatures block of settings.yml.template after the camelCase rename.
private static final String CAMEL_CASE_TEMPLATE =
"""
premium:
proFeatures:
ssoAutoLogin: false
customMetadata:
autoUpdateMetadata: false
author: username
creator: Stirling-PDF
producer: Stirling-PDF
""";
@Test
void migrateProFeaturesKeyCasing_carriesForwardLegacyPascalCaseValues() {
// An existing install whose settings.yml still uses the old PascalCase keys.
String legacy =
"""
premium:
proFeatures:
SSOAutoLogin: true
CustomMetadata:
autoUpdateMetadata: true
author: alice
creator: bob
producer: carol
""";
YamlHelper template = new YamlHelper(LOAD_SETTINGS, CAMEL_CASE_TEMPLATE);
YamlHelper existing = new YamlHelper(LOAD_SETTINGS, legacy);
new ConfigInitializer().migrateProFeaturesKeyCasing(existing, template);
assertEquals(
"true", template.getValueByExactKeyPath("premium", "proFeatures", "ssoAutoLogin"));
assertEquals(
"true",
template.getValueByExactKeyPath(
"premium", "proFeatures", "customMetadata", "autoUpdateMetadata"));
assertEquals(
"alice",
template.getValueByExactKeyPath(
"premium", "proFeatures", "customMetadata", "author"));
assertEquals(
"bob",
template.getValueByExactKeyPath(
"premium", "proFeatures", "customMetadata", "creator"));
assertEquals(
"carol",
template.getValueByExactKeyPath(
"premium", "proFeatures", "customMetadata", "producer"));
}
@Test
void migrateProFeaturesKeyCasing_withoutLegacyKeys_keepsTemplateDefaults() {
// No PascalCase keys present -> this migration step must be a no-op.
String alreadyCamel =
"""
premium:
proFeatures:
ssoAutoLogin: true
customMetadata:
author: dave
""";
YamlHelper template = new YamlHelper(LOAD_SETTINGS, CAMEL_CASE_TEMPLATE);
YamlHelper existing = new YamlHelper(LOAD_SETTINGS, alreadyCamel);
new ConfigInitializer().migrateProFeaturesKeyCasing(existing, template);
assertEquals(
"false", template.getValueByExactKeyPath("premium", "proFeatures", "ssoAutoLogin"));
assertEquals(
"username",
template.getValueByExactKeyPath(
"premium", "proFeatures", "customMetadata", "author"));
}
}
@@ -86,6 +86,8 @@ class TaskManagerJobStoreDelegationTest {
@Override
public boolean shouldRunLocalCleanup() {
// Distributed backplanes own job TTL eviction themselves; this mock
// mirrors the real ValkeyClusterBackplane override of the default true.
return false;
}
};
@@ -2,6 +2,8 @@ package stirling.software.common.util;
import static org.junit.jupiter.api.Assertions.*;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
@@ -12,9 +14,47 @@ import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.mockito.MockedStatic;
import org.mockito.Mockito;
import stirling.software.common.configuration.InstallationPathConfig;
public class GeneralUtilsTest {
// Regression guard for the SSO auto-login persistence bug: the admin UI writes camelCase
// proFeatures keys, so saveKeyToSettings must match (and persist) them against the camelCase
// settings.yml.template. A case mismatch makes YamlHelper.updateValue silently no-op.
@Test
void saveKeyToSettings_persistsCamelCaseProFeatureKeys(@TempDir Path tempDir) throws Exception {
Path settings = tempDir.resolve("settings.yml");
Files.writeString(
settings,
"""
premium:
proFeatures:
ssoAutoLogin: false
customMetadata:
author: username
""");
try (MockedStatic<InstallationPathConfig> mocked =
Mockito.mockStatic(InstallationPathConfig.class)) {
mocked.when(InstallationPathConfig::getSettingsPath).thenReturn(settings.toString());
GeneralUtils.saveKeyToSettings("premium.proFeatures.ssoAutoLogin", true);
GeneralUtils.saveKeyToSettings("premium.proFeatures.customMetadata.author", "alice");
}
YamlHelper reloaded = new YamlHelper(settings);
assertEquals(
"true", reloaded.getValueByExactKeyPath("premium", "proFeatures", "ssoAutoLogin"));
assertEquals(
"alice",
reloaded.getValueByExactKeyPath(
"premium", "proFeatures", "customMetadata", "author"));
}
@Test
void testParsePageListWithAll() {
List<Integer> result = GeneralUtils.parsePageList(new String[] {"all"}, 5, false);
@@ -0,0 +1,370 @@
package stirling.software.common.util;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.lenient;
import static org.mockito.Mockito.mock;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import java.util.zip.ZipOutputStream;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.SsrfProtectionService;
class OfficeDocumentSanitizerTest {
private static final String EXTERNAL_URL = "https://webhook.site/ssrf-callback";
private static final String INTERNAL_TARGET = "media/image1.png";
private static final String DOCX_RELS =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<Relationships xmlns=\"http://schemas.openxmlformats.org/package/2006/relationships\">"
+ "<Relationship Id=\"rId1\" Type=\"http://schemas.openxmlformats.org/officeDocument/2006/relationships/image\""
+ " Target=\""
+ EXTERNAL_URL
+ "\" TargetMode=\"External\"/>"
+ "<Relationship Id=\"rId2\" Type=\"http://schemas.openxmlformats.org/officeDocument/2006/relationships/image\""
+ " Target=\""
+ INTERNAL_TARGET
+ "\"/>"
+ "</Relationships>";
private static final String DOCX_DOCUMENT =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<w:document xmlns:w=\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\">"
+ "<w:body><w:p/></w:body></w:document>";
private static final String ODF_CONTENT_EXTERNAL =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<office:document-content"
+ " xmlns:office=\"urn:oasis:names:tc:opendocument:xmlns:office:1.0\""
+ " xmlns:draw=\"urn:oasis:names:tc:opendocument:xmlns:drawing:1.0\""
+ " xmlns:xlink=\"http://www.w3.org/1999/xlink\">"
+ "<office:body><office:text>"
+ "<draw:frame><draw:image xlink:href=\""
+ EXTERNAL_URL
+ "\" xlink:type=\"simple\"/></draw:frame>"
+ "<draw:frame><draw:image xlink:href=\"Pictures/image1.png\" xlink:type=\"simple\"/></draw:frame>"
+ "</office:text></office:body></office:document-content>";
private SsrfProtectionService ssrfProtectionService;
private ApplicationProperties applicationProperties;
private OfficeDocumentSanitizer sanitizer;
@BeforeEach
void setUp() {
applicationProperties = new ApplicationProperties();
ssrfProtectionService = mock(SsrfProtectionService.class);
sanitizer = new OfficeDocumentSanitizer(ssrfProtectionService, applicationProperties);
}
@Test
void isSanitizableExtension_recognizesOoxmlAndOdf() {
assertTrue(sanitizer.isSanitizableExtension("docx"));
assertTrue(sanitizer.isSanitizableExtension("DOCX"));
assertTrue(sanitizer.isSanitizableExtension("xlsx"));
assertTrue(sanitizer.isSanitizableExtension("pptx"));
assertTrue(sanitizer.isSanitizableExtension("odt"));
assertTrue(sanitizer.isSanitizableExtension("ods"));
assertTrue(sanitizer.isSanitizableExtension("odp"));
assertFalse(sanitizer.isSanitizableExtension("pdf"));
assertFalse(sanitizer.isSanitizableExtension("html"));
assertFalse(sanitizer.isSanitizableExtension(""));
assertFalse(sanitizer.isSanitizableExtension(null));
}
@Test
void sanitize_stripsOoxmlExternalRelationship() throws IOException {
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("word/_rels/document.xml.rels", DOCX_RELS.getBytes(StandardCharsets.UTF_8));
entries.put("word/document.xml", DOCX_DOCUMENT.getBytes(StandardCharsets.UTF_8));
byte[] docx = zip(entries);
byte[] cleaned = sanitizer.sanitize(docx, "docx");
Map<String, byte[]> result = unzip(cleaned);
String rels =
new String(result.get("word/_rels/document.xml.rels"), StandardCharsets.UTF_8);
assertFalse(rels.contains(EXTERNAL_URL), "External URL should be stripped from .rels");
assertFalse(
rels.toLowerCase().contains("targetmode=\"external\""),
"TargetMode=External relationship should be removed");
assertTrue(rels.contains(INTERNAL_TARGET), "Internal image target should be preserved");
assertArrayEquals(
DOCX_DOCUMENT.getBytes(StandardCharsets.UTF_8),
result.get("word/document.xml"),
"Non-rels entries must be untouched");
}
@Test
void sanitize_pptxExternalImageRelStripped() throws IOException {
String pptxRels =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<Relationships xmlns=\"http://schemas.openxmlformats.org/package/2006/relationships\">"
+ "<Relationship Id=\"rId1\" Type=\"http://schemas.openxmlformats.org/officeDocument/2006/relationships/image\""
+ " Target=\""
+ EXTERNAL_URL
+ "\" TargetMode=\"External\"/>"
+ "</Relationships>";
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("ppt/slides/_rels/slide1.xml.rels", pptxRels.getBytes(StandardCharsets.UTF_8));
byte[] pptx = zip(entries);
byte[] cleaned = sanitizer.sanitize(pptx, "pptx");
Map<String, byte[]> result = unzip(cleaned);
String rels =
new String(result.get("ppt/slides/_rels/slide1.xml.rels"), StandardCharsets.UTF_8);
assertFalse(rels.contains(EXTERNAL_URL));
}
@Test
void sanitize_xlsxExternalImageRelStripped() throws IOException {
String xlsxRels =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<Relationships xmlns=\"http://schemas.openxmlformats.org/package/2006/relationships\">"
+ "<Relationship Id=\"rId1\" Type=\"http://schemas.openxmlformats.org/officeDocument/2006/relationships/image\""
+ " Target=\""
+ EXTERNAL_URL
+ "\" TargetMode=\"External\"/>"
+ "</Relationships>";
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put(
"xl/drawings/_rels/drawing1.xml.rels", xlsxRels.getBytes(StandardCharsets.UTF_8));
byte[] xlsx = zip(entries);
byte[] cleaned = sanitizer.sanitize(xlsx, "xlsx");
Map<String, byte[]> result = unzip(cleaned);
String rels =
new String(
result.get("xl/drawings/_rels/drawing1.xml.rels"), StandardCharsets.UTF_8);
assertFalse(rels.contains(EXTERNAL_URL));
}
@Test
void sanitize_odtStripsExternalXlinkHrefButKeepsInternal() throws IOException {
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("content.xml", ODF_CONTENT_EXTERNAL.getBytes(StandardCharsets.UTF_8));
String manifestXml =
"<?xml version=\"1.0\"?><manifest:manifest"
+ " xmlns:manifest=\"urn:oasis:names:tc:opendocument:xmlns:manifest:1.0\"/>";
entries.put("META-INF/manifest.xml", manifestXml.getBytes(StandardCharsets.UTF_8));
byte[] odt = zip(entries);
byte[] cleaned = sanitizer.sanitize(odt, "odt");
Map<String, byte[]> result = unzip(cleaned);
String content = new String(result.get("content.xml"), StandardCharsets.UTF_8);
assertFalse(content.contains(EXTERNAL_URL), "External xlink:href should be stripped");
assertTrue(content.contains("Pictures/image1.png"), "Internal href should be preserved");
}
@Test
void sanitize_odsStripsExternalXlinkHref() throws IOException {
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("content.xml", ODF_CONTENT_EXTERNAL.getBytes(StandardCharsets.UTF_8));
byte[] ods = zip(entries);
byte[] cleaned = sanitizer.sanitize(ods, "ods");
Map<String, byte[]> result = unzip(cleaned);
String content = new String(result.get("content.xml"), StandardCharsets.UTF_8);
assertFalse(content.contains(EXTERNAL_URL));
}
@Test
void sanitize_odpStripsExternalXlinkHrefInStylesXml() throws IOException {
String stylesXml =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<office:document-styles"
+ " xmlns:office=\"urn:oasis:names:tc:opendocument:xmlns:office:1.0\""
+ " xmlns:draw=\"urn:oasis:names:tc:opendocument:xmlns:drawing:1.0\""
+ " xmlns:xlink=\"http://www.w3.org/1999/xlink\">"
+ "<draw:image xlink:href=\""
+ EXTERNAL_URL
+ "\"/></office:document-styles>";
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("styles.xml", stylesXml.getBytes(StandardCharsets.UTF_8));
byte[] odp = zip(entries);
byte[] cleaned = sanitizer.sanitize(odp, "odp");
Map<String, byte[]> result = unzip(cleaned);
String content = new String(result.get("styles.xml"), StandardCharsets.UTF_8);
assertFalse(content.contains(EXTERNAL_URL));
}
@Test
void sanitize_disabledByConfigReturnsOriginal() throws IOException {
applicationProperties.getSystem().setDisableSanitize(true);
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("word/_rels/document.xml.rels", DOCX_RELS.getBytes(StandardCharsets.UTF_8));
byte[] docx = zip(entries);
byte[] result = sanitizer.sanitize(docx, "docx");
assertArrayEquals(docx, result);
}
@Test
void sanitize_unrecognizedExtensionReturnsOriginal() throws IOException {
byte[] original = "irrelevant".getBytes(StandardCharsets.UTF_8);
byte[] result = sanitizer.sanitize(original, "pdf");
assertArrayEquals(original, result);
}
@Test
void sanitize_emptyInputThrows() {
assertThrows(IOException.class, () -> sanitizer.sanitize(new byte[0], "docx"));
}
@Test
void sanitize_nullInputThrows() {
assertThrows(IOException.class, () -> sanitizer.sanitize(null, "docx"));
}
@Test
void sanitize_preservesEntryWithExternalRefWhenAdminAllowsDomain() throws IOException {
applicationProperties
.getSystem()
.getHtml()
.getUrlSecurity()
.getAllowedDomains()
.add("webhook.site");
lenient().when(ssrfProtectionService.isUrlAllowed(eq(EXTERNAL_URL))).thenReturn(true);
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("word/_rels/document.xml.rels", DOCX_RELS.getBytes(StandardCharsets.UTF_8));
byte[] docx = zip(entries);
byte[] cleaned = sanitizer.sanitize(docx, "docx");
Map<String, byte[]> result = unzip(cleaned);
String rels =
new String(result.get("word/_rels/document.xml.rels"), StandardCharsets.UTF_8);
assertTrue(rels.contains(EXTERNAL_URL), "Allow-listed external URL should be preserved");
}
@Test
void sanitize_doesNotConsultSsrfServiceWhenAllowedDomainsEmpty() throws IOException {
// Even if mock would say allowed, we should not invoke it when there is no allow-list,
// because MEDIUM default would let public URLs through and re-introduce the vulnerability.
lenient().when(ssrfProtectionService.isUrlAllowed(eq(EXTERNAL_URL))).thenReturn(true);
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("word/_rels/document.xml.rels", DOCX_RELS.getBytes(StandardCharsets.UTF_8));
byte[] docx = zip(entries);
byte[] cleaned = sanitizer.sanitize(docx, "docx");
Map<String, byte[]> result = unzip(cleaned);
String rels =
new String(result.get("word/_rels/document.xml.rels"), StandardCharsets.UTF_8);
assertFalse(rels.contains(EXTERNAL_URL));
}
@Test
void sanitize_handlesNonXmlEntriesSafely() throws IOException {
Map<String, byte[]> entries = new LinkedHashMap<>();
byte[] imageBytes = new byte[] {(byte) 0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a};
entries.put("word/media/image1.png", imageBytes);
entries.put("word/_rels/document.xml.rels", DOCX_RELS.getBytes(StandardCharsets.UTF_8));
byte[] docx = zip(entries);
byte[] cleaned = sanitizer.sanitize(docx, "docx");
Map<String, byte[]> result = unzip(cleaned);
assertArrayEquals(imageBytes, result.get("word/media/image1.png"));
}
@Test
void sanitize_internalLinksKeptWhenNoExternalPresent() throws IOException {
String internalOnlyRels =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<Relationships xmlns=\"http://schemas.openxmlformats.org/package/2006/relationships\">"
+ "<Relationship Id=\"rId1\" Type=\"http://schemas.openxmlformats.org/officeDocument/2006/relationships/image\""
+ " Target=\"media/image1.png\"/>"
+ "</Relationships>";
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put(
"word/_rels/document.xml.rels", internalOnlyRels.getBytes(StandardCharsets.UTF_8));
byte[] docx = zip(entries);
byte[] cleaned = sanitizer.sanitize(docx, "docx");
Map<String, byte[]> result = unzip(cleaned);
String rels =
new String(result.get("word/_rels/document.xml.rels"), StandardCharsets.UTF_8);
assertTrue(rels.contains("media/image1.png"));
}
@Test
void sanitize_corruptZipProducesSafeOutput() throws IOException {
byte[] garbage = "this is not a zip file".getBytes(StandardCharsets.UTF_8);
byte[] result = sanitizer.sanitize(garbage, "docx");
Map<String, byte[]> entries = unzip(result);
assertTrue(entries.isEmpty(), "Garbage input must not yield exploitable entries");
}
@Test
void sanitize_relativeOdfPathsArePreserved() throws IOException {
String content =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<office:document-content"
+ " xmlns:office=\"urn:oasis:names:tc:opendocument:xmlns:office:1.0\""
+ " xmlns:draw=\"urn:oasis:names:tc:opendocument:xmlns:drawing:1.0\""
+ " xmlns:xlink=\"http://www.w3.org/1999/xlink\">"
+ "<draw:image xlink:href=\"../Pictures/image1.png\"/>"
+ "<draw:image xlink:href=\"#anchor\"/>"
+ "</office:document-content>";
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("content.xml", content.getBytes(StandardCharsets.UTF_8));
byte[] odt = zip(entries);
byte[] cleaned = sanitizer.sanitize(odt, "odt");
Map<String, byte[]> result = unzip(cleaned);
String out = new String(result.get("content.xml"), StandardCharsets.UTF_8);
assertTrue(out.contains("../Pictures/image1.png"));
assertTrue(out.contains("#anchor"));
}
private static byte[] zip(Map<String, byte[]> entries) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (ZipOutputStream zos = new ZipOutputStream(baos)) {
for (Map.Entry<String, byte[]> e : entries.entrySet()) {
ZipEntry entry = new ZipEntry(e.getKey());
zos.putNextEntry(entry);
zos.write(e.getValue());
zos.closeEntry();
}
}
return baos.toByteArray();
}
private static Map<String, byte[]> unzip(byte[] data) throws IOException {
Map<String, byte[]> entries = new HashMap<>();
try (ZipInputStream zis = new ZipInputStream(new ByteArrayInputStream(data))) {
ZipEntry e;
while ((e = zis.getNextEntry()) != null) {
entries.put(e.getName(), zis.readAllBytes());
zis.closeEntry();
}
}
return entries;
}
}
@@ -98,6 +98,17 @@ class RequestUriUtilsTest {
assertTrue(RequestUriUtils.isFrontendRoute("", "/split-pdf"));
}
@Test
void testIsFrontendRoute_filesRouteOwnedByFrontend() {
// /files and /files/<folder-uuid> are FileManagerView routes - they
// must fall through to the SPA index.html, not get blocked by the
// backend auth filter. Regression test for direct-nav/refresh on
// the file manager returning a 401 JSON.
assertTrue(RequestUriUtils.isFrontendRoute("", "/files"));
assertTrue(
RequestUriUtils.isFrontendRoute("", "/files/3331910a-4155-4f71-8111-e38c896bc458"));
}
@Test
void testIsFrontendRoute_pathWithExtension() {
assertFalse(RequestUriUtils.isFrontendRoute("", "/some/file.pdf"));
@@ -183,7 +194,7 @@ class RequestUriUtilsTest {
@Test
void testIsPublicAuthEndpoint_shareRootNotPublic() {
// Avoid matching bare "/share" or "/share/" must have a token segment
// Avoid matching bare "/share" or "/share/" - must have a token segment
assertFalse(RequestUriUtils.isPublicAuthEndpoint("/share", ""));
assertFalse(RequestUriUtils.isPublicAuthEndpoint("/share/", ""));
}
@@ -197,7 +208,7 @@ class RequestUriUtilsTest {
@Test
void testIsPublicAuthEndpoint_shareApiStillProtected() {
// Share-link data APIs must NOT be public they enforce auth + access checks
// Share-link data APIs must NOT be public - they enforce auth + access checks
assertFalse(RequestUriUtils.isPublicAuthEndpoint("/api/v1/storage/share-links/abc123", ""));
assertFalse(
RequestUriUtils.isPublicAuthEndpoint(
@@ -1,163 +0,0 @@
package stirling.software.common.util.propertyeditor;
import static org.junit.jupiter.api.Assertions.*;
import java.util.List;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import stirling.software.common.model.api.security.RedactionArea;
class StringToArrayListPropertyEditorTest {
private StringToArrayListPropertyEditor<RedactionArea> editor;
@BeforeEach
void setUp() {
editor = new StringToArrayListPropertyEditor<>(RedactionArea.class);
}
@Test
void testSetAsText_ValidJson() {
// Arrange
String json =
"[{\"x\":10.5,\"y\":20.5,\"width\":100.0,\"height\":50.0,\"page\":1,\"color\":\"#FF0000\"}]";
// Act
editor.setAsText(json);
Object value = editor.getValue();
// Assert
assertNotNull(value, "Value should not be null");
assertInstanceOf(List.class, value, "Value should be a List");
@SuppressWarnings("unchecked")
List<RedactionArea> list = (List<RedactionArea>) value;
assertEquals(1, list.size(), "List should have 1 entry");
RedactionArea area = list.get(0);
assertEquals(10.5, area.getX(), "X should be 10.5");
assertEquals(20.5, area.getY(), "Y should be 20.5");
assertEquals(100.0, area.getWidth(), "Width should be 100.0");
assertEquals(50.0, area.getHeight(), "Height should be 50.0");
assertEquals(1, area.getPage(), "Page should be 1");
assertEquals("#FF0000", area.getColor(), "Color should be #FF0000");
}
@Test
void testSetAsText_MultipleItems() {
// Arrange
String json =
"["
+ "{\"x\":10.0,\"y\":20.0,\"width\":100.0,\"height\":50.0,\"page\":1,\"color\":\"#FF0000\"},"
+ "{\"x\":30.0,\"y\":40.0,\"width\":200.0,\"height\":150.0,\"page\":2,\"color\":\"#00FF00\"}"
+ "]";
// Act
editor.setAsText(json);
Object value = editor.getValue();
// Assert
assertNotNull(value, "Value should not be null");
assertInstanceOf(List.class, value, "Value should be a List");
@SuppressWarnings("unchecked")
List<RedactionArea> list = (List<RedactionArea>) value;
assertEquals(2, list.size(), "List should have 2 entries");
RedactionArea area1 = list.get(0);
assertEquals(10.0, area1.getX(), "X should be 10.0");
assertEquals(20.0, area1.getY(), "Y should be 20.0");
assertEquals(1, area1.getPage(), "Page should be 1");
RedactionArea area2 = list.get(1);
assertEquals(30.0, area2.getX(), "X should be 30.0");
assertEquals(40.0, area2.getY(), "Y should be 40.0");
assertEquals(2, area2.getPage(), "Page should be 2");
}
@Test
void testSetAsText_EmptyString() {
// Arrange
String json = "";
// Act
editor.setAsText(json);
Object value = editor.getValue();
// Assert
assertNotNull(value, "Value should not be null");
assertInstanceOf(List.class, value, "Value should be a List");
@SuppressWarnings("unchecked")
List<RedactionArea> list = (List<RedactionArea>) value;
assertTrue(list.isEmpty(), "List should be empty");
}
@Test
void testSetAsText_NullString() {
// Act
editor.setAsText(null);
Object value = editor.getValue();
// Assert
assertNotNull(value, "Value should not be null");
assertInstanceOf(List.class, value, "Value should be a List");
@SuppressWarnings("unchecked")
List<RedactionArea> list = (List<RedactionArea>) value;
assertTrue(list.isEmpty(), "List should be empty");
}
@Test
void testSetAsText_SingleItemAsArray() {
// Arrange - note this is a single object, not an array
String json =
"{\"x\":10.0,\"y\":20.0,\"width\":100.0,\"height\":50.0,\"page\":1,\"color\":\"#FF0000\"}";
// Act
editor.setAsText(json);
Object value = editor.getValue();
// Assert
assertNotNull(value, "Value should not be null");
assertInstanceOf(List.class, value, "Value should be a List");
@SuppressWarnings("unchecked")
List<RedactionArea> list = (List<RedactionArea>) value;
assertEquals(1, list.size(), "List should have 1 entry");
RedactionArea area = list.get(0);
assertEquals(10.0, area.getX(), "X should be 10.0");
assertEquals(20.0, area.getY(), "Y should be 20.0");
}
@Test
void testSetAsText_InvalidJson() {
// Arrange
String json = "invalid json";
// Act & Assert
assertThrows(IllegalArgumentException.class, () -> editor.setAsText(json));
}
@Test
void testSetAsText_UnknownProperties() {
// Arrange - this JSON contains properties not in RedactionArea
// With FAIL_ON_UNKNOWN_PROPERTIES disabled, this should ignore the unknown properties
String json = "[{\"invalid\":\"structure\"}]";
// Act
editor.setAsText(json);
Object value = editor.getValue();
// Assert
assertNotNull(value, "Value should not be null");
assertInstanceOf(List.class, value, "Value should be a List");
@SuppressWarnings("unchecked")
List<RedactionArea> list = (List<RedactionArea>) value;
assertEquals(1, list.size(), "List should have 1 entry (empty object)");
}
}
+17 -1
View File
@@ -207,13 +207,29 @@ def resourcesStaticDir = file('src/main/resources/static')
def generatedFrontendPaths = [
'assets',
'index.html',
'index.html.gz',
'index.html.br',
'sw.js',
'sw.js.gz',
'sw.js.br',
'manifest.json.gz',
'manifest.json.br',
'site.webmanifest.gz',
'site.webmanifest.br',
'browserconfig.xml.gz',
'browserconfig.xml.br',
'manifest-classic.json',
'manifest-classic.json.gz',
'manifest-classic.json.br',
'locales',
'Login',
'classic-logo',
'modern-logo',
'og_images',
'samples',
'manifest-classic.json'
'pdfium',
'vendor',
'pdfjs'
]
tasks.register('npmInstall', Exec) {
@@ -21,6 +21,13 @@ public class EndpointInterceptor implements HandlerInterceptor {
HttpServletRequest request, HttpServletResponse response, Object handler)
throws Exception {
String requestURI = request.getRequestURI();
// Prevent API responses from being stored by browsers or intermediary caches by default
String servletPath = request.getServletPath();
if (servletPath != null && servletPath.startsWith("/api/")) {
response.setHeader("Cache-Control", "private, no-store");
}
boolean isEnabled = endpointConfiguration.isEndpointEnabledForUri(requestURI);
if (!isEnabled) {
response.sendError(HttpServletResponse.SC_FORBIDDEN, "This endpoint is disabled");
@@ -18,6 +18,7 @@ import io.swagger.v3.oas.models.media.StringSchema;
import io.swagger.v3.oas.models.security.SecurityRequirement;
import io.swagger.v3.oas.models.security.SecurityScheme;
import io.swagger.v3.oas.models.servers.Server;
import io.swagger.v3.oas.models.tags.Tag;
import lombok.RequiredArgsConstructor;
@@ -60,6 +61,15 @@ public class OpenApiConfig {
OpenAPI openAPI = new OpenAPI().info(info).openapi("3.0.3");
// Register a single global "AI" tag so every AI endpoint groups under it in the docs.
// The AI controllers are currently @Hidden, so they don't emit this tag themselves yet;
// defining it here keeps the grouping ready for when those endpoints are unhidden.
openAPI.addTagsItem(
new Tag()
.name("AI")
.description(
"AI-powered document creation, editing, and assistant endpoints."));
// Add server configuration from environment variable
String swaggerServerUrl = System.getenv("SWAGGER_SERVER_URL");
Server server;
@@ -1,5 +1,8 @@
package stirling.software.SPDF.config;
import java.time.Duration;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeUnit;
import org.slf4j.Logger;
@@ -10,6 +13,7 @@ import org.springframework.web.servlet.config.annotation.CorsRegistry;
import org.springframework.web.servlet.config.annotation.InterceptorRegistry;
import org.springframework.web.servlet.config.annotation.ResourceHandlerRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
import org.springframework.web.servlet.resource.EncodedResourceResolver;
import lombok.RequiredArgsConstructor;
@@ -24,6 +28,10 @@ public class WebMvcConfig implements WebMvcConfigurer {
private static final Logger logger = LoggerFactory.getLogger(WebMvcConfig.class);
private static final CacheControl NO_CACHE = CacheControl.noCache();
private static final CacheControl IMMUTABLE_ONE_YEAR =
CacheControl.maxAge(365, TimeUnit.DAYS).cachePublic().immutable();
@Override
public void addInterceptors(InterceptorRegistry registry) {
registry.addInterceptor(endpointInterceptor);
@@ -31,37 +39,106 @@ public class WebMvcConfig implements WebMvcConfigurer {
@Override
public void addResourceHandlers(ResourceHandlerRegistry registry) {
// Cache hashed assets (JS/CSS with content hashes) for 1 year
// These files have names like index-ChAS4tCC.js that change when content changes
// Check customFiles/static first, then fall back to classpath
String staticPath =
"file:"
+ stirling.software.common.configuration.InstallationPathConfig
.getStaticPath();
// 1. Service worker and PWA metadata (never store)
// Browsers revalidate SW bytes anyway; no-store is the safest for atomic updates.
registry.addResourceHandler(
"/sw.js", "/manifest.json", "/site.webmanifest", "/browserconfig.xml")
.addResourceLocations(staticPath, "classpath:/static/")
.setCacheControl(CacheControl.noStore())
.resourceChain(true)
.addResolver(new EncodedResourceResolver());
// 2. Vite fingerprinted assets (immutable)
// These already have content hashes in filenames (e.g. index-ChAS4tCC.js)
registry.addResourceHandler("/assets/**")
.addResourceLocations(
"file:"
+ stirling.software.common.configuration.InstallationPathConfig
.getStaticPath()
+ "assets/",
"classpath:/static/assets/")
.setCacheControl(CacheControl.maxAge(365, TimeUnit.DAYS).cachePublic());
.addResourceLocations(staticPath + "assets/", "classpath:/static/assets/")
.setCacheControl(IMMUTABLE_ONE_YEAR)
.resourceChain(true)
.addResolver(new EncodedResourceResolver());
// Don't cache index.html - it needs to be fresh to reference latest hashed assets
// Note: index.html is handled by ReactRoutingController for dynamic processing
registry.addResourceHandler("/index.html")
// 3. Media and fonts (immutable)
registry.addResourceHandler("/images/**", "/fonts/**")
.addResourceLocations(
"file:"
+ stirling.software.common.configuration.InstallationPathConfig
.getStaticPath(),
"classpath:/static/")
.setCacheControl(CacheControl.noCache().mustRevalidate());
staticPath + "images/",
"classpath:/static/images/",
staticPath + "fonts/",
"classpath:/static/fonts/")
.setCacheControl(IMMUTABLE_ONE_YEAR)
.resourceChain(true)
.addResolver(new EncodedResourceResolver());
// Handle all other static resources (js, css, images, fonts, etc.)
// Check customFiles/static first for user overrides
// 4. Branding and stable non-fingerprinted assets (1 day + SWR)
// Use stale-while-revalidate to improve perceived performance.
registry.addResourceHandler(
"/favicon.*",
"/apple-touch-icon.png",
"/android-chrome-*.png",
"/mstile-*.png",
"/safari-pinned-tab.svg",
"/icons/**",
"/modern-logo/**",
"/classic-logo/**",
"/robots.txt",
"/3rdPartyLicenses.json",
"/pdfjs/**",
"/pdfjs-legacy/**",
"/pdfium/**",
"/locales/**",
"/css/**",
"/js/**",
"/vendor/**",
"/samples/**",
"/og_images/**",
"/Login/**",
"/manifest-classic.json")
.addResourceLocations(
staticPath,
"classpath:/static/",
staticPath + "pdfjs/",
"classpath:/static/pdfjs/",
staticPath + "pdfjs-legacy/",
"classpath:/static/pdfjs-legacy/",
staticPath + "pdfium/",
"classpath:/static/pdfium/",
staticPath + "locales/",
"classpath:/static/locales/",
staticPath + "css/",
"classpath:/static/css/",
staticPath + "js/",
"classpath:/static/js/",
staticPath + "vendor/",
"classpath:/static/vendor/",
staticPath + "samples/",
"classpath:/static/samples/",
staticPath + "og_images/",
"classpath:/static/og_images/",
staticPath + "Login/",
"classpath:/static/Login/",
staticPath + "icons/",
"classpath:/static/icons/",
staticPath + "modern-logo/",
"classpath:/static/modern-logo/",
staticPath + "classic-logo/",
"classpath:/static/classic-logo/")
.setCacheControl(
CacheControl.maxAge(Duration.ofDays(1))
.cachePublic()
.staleWhileRevalidate(Duration.ofDays(7)))
.resourceChain(true)
.addResolver(new EncodedResourceResolver());
// 5. Catch-all (SPA fallback)
// Must check with server to ensure index.html is always fresh.
registry.addResourceHandler("/**")
.addResourceLocations(
"file:"
+ stirling.software.common.configuration.InstallationPathConfig
.getStaticPath(),
"classpath:/static/")
.setCacheControl(CacheControl.maxAge(1, TimeUnit.HOURS));
.addResourceLocations(staticPath, "classpath:/static/")
.setCacheControl(NO_CACHE)
.resourceChain(true)
.addResolver(new EncodedResourceResolver());
}
@Override
@@ -115,9 +192,8 @@ public class WebMvcConfig implements WebMvcConfigurer {
applicationProperties.getSystem().getCorsAllowedOrigins());
// Combine user-configured origins with Tauri origins
java.util.List<String> allOrigins =
new java.util.ArrayList<>(
applicationProperties.getSystem().getCorsAllowedOrigins());
List<String> allOrigins =
new ArrayList<>(applicationProperties.getSystem().getCorsAllowedOrigins());
// Always include Tauri origins for desktop app compatibility
// Tauri v1 uses tauri://localhost, v2 uses http(s)://tauri.localhost
@@ -158,7 +234,8 @@ public class WebMvcConfig implements WebMvcConfigurer {
} else {
// Default to allowing all origins when nothing is configured
logger.debug(
"No CORS allowed origins configured in settings.yml (system.corsAllowedOrigins); WebMvcConfig allowing all origins.");
"No CORS allowed origins configured in settings.yml"
+ " (system.corsAllowedOrigins); WebMvcConfig allowing all origins.");
registry.addMapping("/**")
.allowedOriginPatterns("*")
.allowedMethods("GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS")
@@ -32,13 +32,16 @@ import stirling.software.SPDF.model.json.PdfJsonTextElement;
import stirling.software.SPDF.service.PdfJsonConversionService;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.model.api.general.EditTextOperation;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import stirling.software.common.util.propertyeditor.StringToArrayListPropertyEditor;
import stirling.software.common.util.propertyeditor.JsonListPropertyEditor;
import tools.jackson.core.type.TypeReference;
/**
* Find/replace text editing for PDFs. Round-trips through {@link PdfJsonConversionService}: the
@@ -72,10 +75,13 @@ public class EditTextController {
binder.registerCustomEditor(
List.class,
"edits",
new StringToArrayListPropertyEditor<>(EditTextOperation.class));
new JsonListPropertyEditor<>(new TypeReference<List<EditTextOperation>>() {}));
}
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/edit-text")
@AutoJobPostMapping(
consumes = "multipart/form-data",
value = "/edit-text",
resourceWeight = ResourceWeight.LARGE_WEIGHT)
@StandardPdfResponse
@Operation(
summary = "Edit text in a PDF via find and replace",
@@ -10,6 +10,7 @@ import java.util.Map;
import org.apache.pdfbox.Loader;
import org.apache.pdfbox.multipdf.Overlay;
import org.apache.pdfbox.pdfwriter.compress.CompressParameters;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.springframework.core.io.Resource;
import org.springframework.http.MediaType;
@@ -157,7 +158,10 @@ public class PdfOverlayController {
PDDocument singlePageDocument = new PDDocument()) {
singlePageDocument.addPage(overlayPdf.getPage(pageCountInCurrentOverlay));
File tempFile = Files.createTempFile("overlay-page-", ".pdf").toFile();
singlePageDocument.save(tempFile);
// NO_COMPRESSION: this single-page doc holds a page copied from overlayPdf.
// PDFBox 3.0.7's compressed writer (PDFBOX-6203) drops shared resources imported
// across documents, corrupting overlay fonts. Revert once on 3.0.8.
singlePageDocument.save(tempFile, CompressParameters.NO_COMPRESSION);
overlayGuide.put(basePageIndex, tempFile.getAbsolutePath());
tempFiles.add(tempFile); // Keep track of the temporary file for cleanup
@@ -6,11 +6,9 @@ import java.util.Collections;
import java.util.List;
import java.util.Locale;
import org.apache.pdfbox.cos.COSName;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDDocumentCatalog;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.interactive.form.PDAcroForm;
import org.apache.pdfbox.pdmodel.PDPageTree;
import org.springframework.core.io.Resource;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
@@ -262,38 +260,31 @@ public class RearrangePagesPDFController {
}
log.info("newPageOrder = {}", newPageOrder);
log.info("totalPages = {}", totalPages);
// Create a new list to hold the pages in the new order
List<PDPage> newPages = new ArrayList<>();
for (int i = 0; i < newPageOrder.size(); i++) {
newPages.add(document.getPage(newPageOrder.get(i)));
// Snapshot the desired pages before mutating the source document's page tree.
List<PDPage> newPages = new ArrayList<>(newPageOrder.size());
for (Integer idx : newPageOrder) {
newPages.add(document.getPage(idx));
}
// Create a new document based on the original one
try (PDDocument rearrangedDocument =
pdfDocumentFactory.createNewDocumentBasedOnOldDocument(document)) {
// Add the pages in the new order
for (PDPage page : newPages) {
rearrangedDocument.addPage(page);
}
PDDocumentCatalog sourceCatalog = document.getDocumentCatalog();
if (sourceCatalog != null) {
PDAcroForm sourceForm = sourceCatalog.getAcroForm(null);
if (sourceForm != null) {
rearrangedDocument
.getDocumentCatalog()
.getCOSObject()
.setItem(COSName.ACRO_FORM, sourceForm.getCOSObject());
}
}
return WebResponseUtils.pdfDocToWebResponse(
rearrangedDocument,
GeneralUtils.generateFilename(
pdfFile.getOriginalFilename(), "_rearranged.pdf"),
tempFileManager);
// Rearrange in-place on the source document rather than copying pages into a
// freshly-created PDDocument. Copying pages across documents triggers a PDFBox
// 3.0.7 compressed-save regression (PDFBOX-6203, fixed for 3.0.8) where shared
// resource objects (fonts, etc.) imported from the source can be silently
// dropped from the output, producing pages with "font not found" errors.
PDPageTree pages = document.getPages();
for (int i = totalPages - 1; i >= 0; i--) {
pages.remove(i);
}
for (PDPage page : newPages) {
pages.add(page);
}
return WebResponseUtils.pdfDocToWebResponse(
document,
GeneralUtils.generateFilename(
pdfFile.getOriginalFilename(), "_rearranged.pdf"),
tempFileManager);
}
} catch (IOException e) {
ExceptionUtils.logException("document rearrangement", e);
@@ -40,7 +40,8 @@ public class ScalePagesController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
private static PDRectangle getTargetSize(String targetPDRectangle, PDDocument sourceDocument) {
private static PDRectangle getTargetSize(
String targetPDRectangle, String orientation, PDDocument sourceDocument) {
if ("KEEP".equals(targetPDRectangle)) {
if (sourceDocument.getNumberOfPages() == 0) {
throw ExceptionUtils.createInvalidPageSizeException("KEEP");
@@ -57,18 +58,19 @@ public class ScalePagesController {
}
Map<String, PDRectangle> sizeMap = getSizeMap();
if (sizeMap.containsKey(targetPDRectangle)) {
return sizeMap.get(targetPDRectangle);
PDRectangle base = sizeMap.get(targetPDRectangle);
if (base == null) {
throw ExceptionUtils.createInvalidPageSizeException(targetPDRectangle);
}
throw ExceptionUtils.createInvalidPageSizeException(targetPDRectangle);
if ("LANDSCAPE".equalsIgnoreCase(orientation)) {
return new PDRectangle(base.getHeight(), base.getWidth());
}
return base;
}
private static Map<String, PDRectangle> getSizeMap() {
Map<String, PDRectangle> sizeMap = new HashMap<>();
// Portrait sizes (A0-A6)
sizeMap.put("A0", PDRectangle.A0);
sizeMap.put("A1", PDRectangle.A1);
sizeMap.put("A2", PDRectangle.A2);
@@ -76,42 +78,8 @@ public class ScalePagesController {
sizeMap.put("A4", PDRectangle.A4);
sizeMap.put("A5", PDRectangle.A5);
sizeMap.put("A6", PDRectangle.A6);
// Landscape sizes (A0-A6)
sizeMap.put(
"A0_LANDSCAPE",
new PDRectangle(PDRectangle.A0.getHeight(), PDRectangle.A0.getWidth()));
sizeMap.put(
"A1_LANDSCAPE",
new PDRectangle(PDRectangle.A1.getHeight(), PDRectangle.A1.getWidth()));
sizeMap.put(
"A2_LANDSCAPE",
new PDRectangle(PDRectangle.A2.getHeight(), PDRectangle.A2.getWidth()));
sizeMap.put(
"A3_LANDSCAPE",
new PDRectangle(PDRectangle.A3.getHeight(), PDRectangle.A3.getWidth()));
sizeMap.put(
"A4_LANDSCAPE",
new PDRectangle(PDRectangle.A4.getHeight(), PDRectangle.A4.getWidth()));
sizeMap.put(
"A5_LANDSCAPE",
new PDRectangle(PDRectangle.A5.getHeight(), PDRectangle.A5.getWidth()));
sizeMap.put(
"A6_LANDSCAPE",
new PDRectangle(PDRectangle.A6.getHeight(), PDRectangle.A6.getWidth()));
// Portrait US sizes
sizeMap.put("LETTER", PDRectangle.LETTER);
sizeMap.put("LEGAL", PDRectangle.LEGAL);
// Landscape US sizes
sizeMap.put(
"LETTER_LANDSCAPE",
new PDRectangle(PDRectangle.LETTER.getHeight(), PDRectangle.LETTER.getWidth()));
sizeMap.put(
"LEGAL_LANDSCAPE",
new PDRectangle(PDRectangle.LEGAL.getHeight(), PDRectangle.LEGAL.getWidth()));
return sizeMap;
}
@@ -128,13 +96,14 @@ public class ScalePagesController {
throws IOException {
MultipartFile file = request.getFileInput();
String targetPDRectangle = request.getPageSize();
String orientation = request.getOrientation();
float scaleFactor = request.getScaleFactor();
try (PDDocument sourceDocument = pdfDocumentFactory.load(file);
PDDocument outputDocument =
pdfDocumentFactory.createNewDocumentBasedOnOldDocument(sourceDocument)) {
PDRectangle targetSize = getTargetSize(targetPDRectangle, sourceDocument);
PDRectangle targetSize = getTargetSize(targetPDRectangle, orientation, sourceDocument);
// Create LayerUtility once outside the loop for better performance
LayerUtility layerUtility = new LayerUtility(outputDocument);
@@ -275,7 +275,10 @@ public class ConvertImgPDFController {
GeneralUtils.generateFilename(file[0].getOriginalFilename(), "_converted.pdf"));
}
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/cbz/pdf")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/cbz/pdf",
resourceWeight = ResourceWeight.MEDIUM_WEIGHT)
@Operation(
summary = "Convert CBZ comic book archive to PDF",
description =
@@ -301,7 +304,10 @@ public class ConvertImgPDFController {
return WebResponseUtils.pdfFileToWebResponse(pdfFile, filename);
}
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/pdf/cbz")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/pdf/cbz",
resourceWeight = ResourceWeight.LARGE_WEIGHT)
@Operation(
summary = "Convert PDF to CBZ comic book archive",
description =
@@ -324,7 +330,10 @@ public class ConvertImgPDFController {
return WebResponseUtils.zipFileToWebResponse(cbzFile, filename);
}
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/cbr/pdf")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/cbr/pdf",
resourceWeight = ResourceWeight.MEDIUM_WEIGHT)
@Operation(
summary = "Convert CBR comic book archive to PDF",
description =
@@ -350,7 +359,10 @@ public class ConvertImgPDFController {
return WebResponseUtils.bytesToWebResponse(pdfBytes, filename);
}
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/pdf/cbr")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/pdf/cbr",
resourceWeight = ResourceWeight.LARGE_WEIGHT)
@Operation(
summary = "Convert PDF to CBR comic book archive",
description =
@@ -35,6 +35,7 @@ import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.CustomHtmlSanitizer;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.OfficeDocumentSanitizer;
import stirling.software.common.util.ProcessExecutor;
import stirling.software.common.util.ProcessExecutor.ProcessExecutorResult;
import stirling.software.common.util.RegexPatternUtils;
@@ -50,6 +51,7 @@ public class ConvertOfficeController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final RuntimePathConfig runtimePathConfig;
private final CustomHtmlSanitizer customHtmlSanitizer;
private final OfficeDocumentSanitizer officeDocumentSanitizer;
private final EndpointConfiguration endpointConfiguration;
private final TempFileManager tempFileManager;
@@ -83,14 +85,16 @@ public class ConvertOfficeController {
Path inputPath = workDir.resolve(baseName + "." + extensionLower);
Path outputPath = workDir.resolve(baseName + ".pdf");
// Check if the file is HTML and apply sanitization if needed
// Sanitize input before LibreOffice sees it so embedded URLs can't trigger SSRF.
if ("html".equals(extensionLower) || "htm".equals(extensionLower)) {
// Read and sanitize HTML content
String htmlContent = new String(inputFile.getBytes(), StandardCharsets.UTF_8);
String sanitizedHtml = customHtmlSanitizer.sanitize(htmlContent);
Files.writeString(inputPath, sanitizedHtml, StandardCharsets.UTF_8);
} else if (officeDocumentSanitizer.isSanitizableExtension(extensionLower)) {
byte[] sanitized =
officeDocumentSanitizer.sanitize(inputFile.getBytes(), extensionLower);
Files.write(inputPath, sanitized);
} else {
// copy file content
Files.copy(inputFile.getInputStream(), inputPath, StandardCopyOption.REPLACE_EXISTING);
}
@@ -141,7 +141,8 @@ public class AttachmentController {
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/extract-attachments")
value = "/extract-attachments",
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@Operation(
summary = "Extract attachments from PDF",
description =
@@ -176,7 +177,10 @@ public class AttachmentController {
}
}
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/list-attachments")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/list-attachments",
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@Operation(
summary = "List attachments in PDF",
description =
@@ -193,7 +197,8 @@ public class AttachmentController {
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/rename-attachment")
value = "/rename-attachment",
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@StandardPdfResponse
@Operation(
summary = "Rename attachment in PDF",
@@ -228,7 +233,8 @@ public class AttachmentController {
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/delete-attachment")
value = "/delete-attachment",
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@StandardPdfResponse
@Operation(
summary = "Delete attachment from PDF",
@@ -14,6 +14,7 @@ import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
import org.apache.pdfbox.cos.COSName;
import org.apache.pdfbox.pdfwriter.compress.CompressParameters;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.graphics.image.PDImageXObject;
@@ -357,7 +358,10 @@ public class AutoSplitPdfController {
for (int i = 0; i < splitDocuments.size(); i++) {
String fileName = filename + "_" + (i + 1) + ".pdf";
zipOut.putNextEntry(new ZipEntry(fileName));
splitDocuments.get(i).save(zipOut);
// NO_COMPRESSION: split docs are built by addPage()-ing pages copied from the
// source document. PDFBox 3.0.7's compressed writer (PDFBOX-6203) drops shared
// resources imported across documents, corrupting fonts. Revert once on 3.0.8.
splitDocuments.get(i).save(zipOut, CompressParameters.NO_COMPRESSION);
zipOut.closeEntry();
}
}
@@ -12,6 +12,8 @@ import org.springframework.web.bind.annotation.RequestParam;
import io.swagger.v3.oas.annotations.Hidden;
import jakarta.servlet.http.HttpServletRequest;
import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.config.EndpointConfiguration;
@@ -91,6 +93,63 @@ public class ConfigController {
return null;
}
/**
* Resolve the frontend URL the client should advertise to phones / share-link recipients.
* Priority: explicit system.frontendUrl, then the Host the user is already using to reach this
* server (works for Docker, reverse proxies, and bare-metal LANs), then a detected site-local
* IPv4, then empty.
*/
// visible for testing
String resolveFrontendUrl(HttpServletRequest request, AppConfig appConfig) {
String configured = applicationProperties.getSystem().getFrontendUrl();
if (configured != null && !configured.isBlank()) {
return configured;
}
if (request != null) {
String host = request.getServerName();
if (host != null && !host.isBlank() && !isLoopbackHost(host)) {
String scheme = request.getScheme();
int port = request.getServerPort();
boolean defaultPort =
("http".equals(scheme) && port == 80)
|| ("https".equals(scheme) && port == 443);
return defaultPort ? scheme + "://" + host : scheme + "://" + host + ":" + port;
}
}
String localIp = GeneralUtils.getLocalNetworkIp();
if (localIp != null) {
String scheme = appConfig.getBackendUrl().startsWith("https") ? "https" : "http";
return scheme + "://" + localIp + ":" + resolveEffectiveServerPort(appConfig);
}
return "";
}
/**
* The port the embedded server is actually listening on. With {@code server.port=0} (an
* ephemeral port, which the desktop bundle uses to dodge port clashes) the configured value
* stays {@code "0"} while Spring publishes the real bound port as {@code local.server.port}
* once the server is up. Advertised URLs (the mobile-scanner QR, share links) must carry the
* real port - a literal {@code :0} is unreachable and browsers reject it as ERR_UNSAFE_PORT.
*/
// visible for testing
String resolveEffectiveServerPort(AppConfig appConfig) {
String configured = appConfig.getServerPort();
if (configured == null || "0".equals(configured.trim())) {
String actual = applicationContext.getEnvironment().getProperty("local.server.port");
if (actual != null && !actual.isBlank()) {
return actual;
}
}
return configured;
}
private static boolean isLoopbackHost(String host) {
return "localhost".equalsIgnoreCase(host)
|| "127.0.0.1".equals(host)
|| "::1".equals(host)
|| "0:0:0:0:0:0:0:1".equals(host);
}
/** Check if running Enterprise edition dynamically. */
private Boolean isRunningEE() {
// Use LicenseService for fresh license status if available
@@ -107,7 +166,7 @@ public class ConfigController {
}
@GetMapping("/app-config")
public ResponseEntity<Map<String, Object>> getAppConfig() {
public ResponseEntity<Map<String, Object>> getAppConfig(HttpServletRequest request) {
Map<String, Object> configData = new HashMap<>();
try {
@@ -121,20 +180,10 @@ public class ConfigController {
// Note: Frontend expects "baseUrl" field name for compatibility
configData.put("baseUrl", appConfig.getBackendUrl());
configData.put("contextPath", appConfig.getContextPath());
configData.put("serverPort", appConfig.getServerPort());
configData.put("serverPort", resolveEffectiveServerPort(appConfig));
String frontendUrl = applicationProperties.getSystem().getFrontendUrl();
if ((frontendUrl == null || frontendUrl.isBlank())
&& Boolean.parseBoolean(
System.getProperty("STIRLING_PDF_TAURI_MODE", "false"))) {
String localIp = GeneralUtils.getLocalNetworkIp();
if (localIp != null) {
String scheme =
appConfig.getBackendUrl().startsWith("https") ? "https" : "http";
frontendUrl = scheme + "://" + localIp + ":" + appConfig.getServerPort();
}
}
configData.put("frontendUrl", frontendUrl != null ? frontendUrl : "");
configData.put("frontendUrl", resolveFrontendUrl(request, appConfig));
// Add mobile scanner settings
configData.put(
@@ -277,6 +326,9 @@ public class ConfigController {
// Premium/Enterprise settings
configData.put("premiumEnabled", applicationProperties.getPremium().isEnabled());
// AI Engine settings
configData.put("aiEngineEnabled", applicationProperties.getAiEngine().isEnabled());
// Timestamp TSA settings — single source of truth for presets + admin URLs
ApplicationProperties.Security.Timestamp tsConfig =
applicationProperties.getSecurity().getTimestamp();
@@ -17,6 +17,7 @@ import javax.imageio.ImageIO;
import org.apache.pdfbox.io.IOUtils;
import org.apache.pdfbox.multipdf.PDFMergerUtility;
import org.apache.pdfbox.pdfwriter.compress.CompressParameters;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.rendering.PDFRenderer;
@@ -427,7 +428,10 @@ public class OCRController {
// Save original page without OCR as fallback
try (PDDocument pageDoc = new PDDocument()) {
pageDoc.addPage(page);
pageDoc.save(pageOutputPath);
// NO_COMPRESSION: page is copied from another document;
// PDFBox 3.0.7 compressed writer (PDFBOX-6203) drops shared
// resources, corrupting fonts. Revert once on 3.0.8.
pageDoc.save(pageOutputPath, CompressParameters.NO_COMPRESSION);
}
}
@@ -437,7 +441,10 @@ public class OCRController {
// Save original page without OCR
try (PDDocument pageDoc = new PDDocument()) {
pageDoc.addPage(page);
pageDoc.save(pageOutputPath);
// NO_COMPRESSION: page is copied from another document; PDFBox 3.0.7
// compressed writer (PDFBOX-6203) drops shared resources, corrupting
// fonts on retained text pages. Revert once on 3.0.8.
pageDoc.save(pageOutputPath, CompressParameters.NO_COMPRESSION);
merger.addSource(pageOutputPath);
}
}
@@ -25,6 +25,7 @@ import stirling.software.common.annotations.api.MiscApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.SvgSanitizer;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@@ -36,6 +37,7 @@ public class OverlayImageController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
private final SvgSanitizer svgSanitizer;
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
@@ -61,6 +63,9 @@ public class OverlayImageController {
byte[] imageBytes = imageFile.getBytes();
boolean isSvg = SvgOverlayUtil.isSvgImage(imageBytes);
if (isSvg) {
imageBytes = svgSanitizer.sanitize(imageBytes);
}
try (PDDocument document = pdfDocumentFactory.load(pdfBytes)) {
int pages = document.getNumberOfPages();
@@ -0,0 +1,127 @@
package stirling.software.SPDF.controller.api.security;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.text.PDFTextStripper;
import org.apache.pdfbox.text.TextPosition;
/**
* PDFTextStripper subclass that collects all text positions and groups them into line-level
* bounding boxes.
*
* <p>Two outputs are maintained in parallel:
*
* <ul>
* <li>{@link #getLineBoxes()} returns {@code [x1, pdfYbottom, x2, pdfYtop]} in PDF user-space
* (origin bottom-left, Y up). This is what existing callers expect.
* <li>{@link #getScreenLineBoxes()} returns {@code [x1, screenYtop, x2, screenYbottom]} computed
* directly from glyph positions without a PDF↔screen round-trip — used by column-aware
* redaction where ulp-level drift in the round-trip caused false rejects against anchors.
* </ul>
*
* <p>Lines are flushed not only on Y jumps but also on large X gaps within the same Y row. That way
* left-column glyphs and right-column glyphs that happen to share a baseline (common in IEEE
* conference templates) get emitted as two distinct line boxes instead of one wide merged box.
*/
final class AllTextLineExtractor extends PDFTextStripper {
/** Min vertical jump (screen Y) before the next glyph is treated as a new line. */
private static final float LINE_Y_TOLERANCE = 3.0f;
/**
* Min horizontal gap (screen X) between consecutive glyphs on the same Y that indicates a
* column boundary. Chosen large enough to not split normal inter-word spacing (~610pt for 11pt
* text) but small enough to catch standard column gutters (typically ≥15pt).
*/
private static final float COLUMN_GAP_X = 14f;
private final float pageHeight;
private final List<float[]> lineBoxes = new ArrayList<>();
private final List<float[]> screenLineBoxes = new ArrayList<>();
private final List<TextPosition> currentLine = new ArrayList<>();
private float lastScreenY = Float.NaN;
private float lastGlyphRight = Float.NaN;
AllTextLineExtractor(int pageNumber, float pageHeight) throws IOException {
this.pageHeight = pageHeight;
setStartPage(pageNumber);
setEndPage(pageNumber);
setSortByPosition(true);
}
List<float[]> getLineBoxes() {
return lineBoxes;
}
/**
* Returns line boxes as {@code [x1, screenYtop, x2, screenYbottom]}. {@code screenYtop} is the
* minimum {@code TextPosition.getY() - getHeight()} on the line and {@code screenYbottom} is
* the maximum {@code getY()} (the line's baseline). These values come straight from PDFBox
* without going through {@code pageHeight - …}, so they're stable for ulp-sensitive comparisons
* against anchor screen Ys.
*/
List<float[]> getScreenLineBoxes() {
return screenLineBoxes;
}
@Override
protected void writeString(String text, List<TextPosition> positions) {
for (TextPosition tp : positions) {
// Skip whitespace-only positions (spaces, newline markers, indent characters).
// These have a TextPosition but no visible glyph; including them causes
// space-only "lines" to produce degenerate segments that appear as thin
// black bars after redaction.
String unicode = tp.getUnicode();
if (unicode == null || unicode.isBlank()) {
continue;
}
float screenY = tp.getY();
float screenX = tp.getX();
boolean yJump =
!Float.isNaN(lastScreenY) && Math.abs(screenY - lastScreenY) > LINE_Y_TOLERANCE;
boolean xJump =
!Float.isNaN(lastGlyphRight) && (screenX - lastGlyphRight) > COLUMN_GAP_X;
if (yJump || xJump) {
flushLine();
}
lastScreenY = screenY;
lastGlyphRight = screenX + tp.getWidth();
currentLine.add(tp);
}
}
@Override
protected void endPage(PDPage page) throws IOException {
flushLine();
super.endPage(page);
}
private void flushLine() {
if (currentLine.isEmpty()) {
return;
}
float minX = Float.MAX_VALUE, maxX = -Float.MAX_VALUE;
float minScreenY = Float.MAX_VALUE, maxScreenY = -Float.MAX_VALUE;
for (TextPosition tp : currentLine) {
minX = Math.min(minX, tp.getX());
maxX = Math.max(maxX, tp.getX() + tp.getWidth());
minScreenY = Math.min(minScreenY, tp.getY() - tp.getHeight());
maxScreenY = Math.max(maxScreenY, tp.getY());
}
emitSegment(minX, maxX, minScreenY, maxScreenY);
currentLine.clear();
lastScreenY = Float.NaN;
lastGlyphRight = Float.NaN;
}
private void emitSegment(float minX, float maxX, float minScreenY, float maxScreenY) {
float pdfY1 = pageHeight - maxScreenY; // bottom in PDF coords
float pdfY2 = pageHeight - minScreenY; // top in PDF coords
lineBoxes.add(new float[] {minX, pdfY1, maxX, pdfY2});
screenLineBoxes.add(new float[] {minX, minScreenY, maxX, maxScreenY});
}
}
@@ -0,0 +1,403 @@
package stirling.software.SPDF.controller.api.security;
import java.awt.Color;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDPageContentStream;
import org.apache.pdfbox.pdmodel.PDPageTree;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
import org.apache.pdfbox.pdmodel.interactive.annotation.PDAnnotation;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.model.PDFText;
import stirling.software.SPDF.model.api.security.ManualRedactPdfRequest;
import stirling.software.SPDF.pdf.parser.PageImageLocator;
import stirling.software.common.model.api.security.RedactionArea;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.PdfUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
@Service
@Slf4j
@RequiredArgsConstructor
class ManualRedactionService {
private static final float DEFAULT_TEXT_PADDING_MULTIPLIER = 0.6f;
private static final float REDACTION_WIDTH_REDUCTION_FACTOR = 0.9f;
private final TempFileManager tempFileManager;
// -----------------------------------------------------------------------
// Area and page redaction
// -----------------------------------------------------------------------
void redactAreas(List<RedactionArea> redactionAreas, PDDocument document, PDPageTree allPages)
throws IOException {
if (redactionAreas == null || redactionAreas.isEmpty()) {
return;
}
Map<Integer, List<RedactionArea>> redactionsByPage = new HashMap<>();
for (RedactionArea redactionArea : redactionAreas) {
if (redactionArea.getPage() == null
|| redactionArea.getPage() <= 0
|| redactionArea.getHeight() == null
|| redactionArea.getHeight() <= 0.0D
|| redactionArea.getWidth() == null
|| redactionArea.getWidth() <= 0.0D) {
continue;
}
redactionsByPage
.computeIfAbsent(redactionArea.getPage(), k -> new ArrayList<>())
.add(redactionArea);
}
for (Map.Entry<Integer, List<RedactionArea>> entry : redactionsByPage.entrySet()) {
Integer pageNumber = entry.getKey();
List<RedactionArea> areasForPage = entry.getValue();
if (pageNumber > allPages.getCount()) {
continue;
}
PDPage page = allPages.get(pageNumber - 1);
try (PDPageContentStream contentStream =
new PDPageContentStream(
document, page, PDPageContentStream.AppendMode.APPEND, true, true)) {
contentStream.saveGraphicsState();
for (RedactionArea redactionArea : areasForPage) {
Color redactColor = decodeOrDefault(redactionArea.getColor());
contentStream.setNonStrokingColor(redactColor);
float x = redactionArea.getX().floatValue();
float y = redactionArea.getY().floatValue();
float width = redactionArea.getWidth().floatValue();
float height = redactionArea.getHeight().floatValue();
float pdfY = page.getBBox().getHeight() - y - height;
contentStream.addRect(x, pdfY, width, height);
contentStream.fill();
}
contentStream.restoreGraphicsState();
}
}
}
void redactPages(ManualRedactPdfRequest request, PDDocument document, PDPageTree allPages)
throws IOException {
Color redactColor = decodeOrDefault(request.getPageRedactionColor());
List<Integer> pageNumbers = getPageNumbers(request, allPages.getCount());
for (Integer pageNumber : pageNumbers) {
PDPage page = allPages.get(pageNumber);
try (PDPageContentStream contentStream =
new PDPageContentStream(
document, page, PDPageContentStream.AppendMode.APPEND, true, true)) {
contentStream.setNonStrokingColor(redactColor);
PDRectangle box = page.getBBox();
contentStream.addRect(0, 0, box.getWidth(), box.getHeight());
contentStream.fill();
}
}
}
// -----------------------------------------------------------------------
// Overlay drawing
// -----------------------------------------------------------------------
void redactFoundText(
PDDocument document,
List<PDFText> blocks,
float customPadding,
Color redactColor,
boolean isTextRemovalMode)
throws IOException {
var allPages = document.getDocumentCatalog().getPages();
Map<Integer, List<PDFText>> blocksByPage = new HashMap<>();
for (PDFText block : blocks) {
blocksByPage.computeIfAbsent(block.getPageIndex(), k -> new ArrayList<>()).add(block);
}
for (Map.Entry<Integer, List<PDFText>> entry : blocksByPage.entrySet()) {
Integer pageIndex = entry.getKey();
List<PDFText> pageBlocks = entry.getValue();
if (pageIndex >= allPages.getCount()) {
continue;
}
var page = allPages.get(pageIndex);
try (PDPageContentStream contentStream =
new PDPageContentStream(
document, page, PDPageContentStream.AppendMode.APPEND, true, true)) {
contentStream.saveGraphicsState();
try {
contentStream.setNonStrokingColor(redactColor);
PDRectangle pageBox = page.getBBox();
for (PDFText block : pageBlocks) {
float padding =
(block.getY2() - block.getY1()) * DEFAULT_TEXT_PADDING_MULTIPLIER
+ customPadding;
float originalWidth = block.getX2() - block.getX1();
float boxWidth;
float boxX;
if (isTextRemovalMode) {
boxWidth = originalWidth * REDACTION_WIDTH_REDUCTION_FACTOR;
float widthReduction = originalWidth - boxWidth;
boxX = block.getX1() + (widthReduction / 2);
} else {
boxWidth = originalWidth;
boxX = block.getX1();
}
contentStream.addRect(
boxX,
pageBox.getHeight() - block.getY2() - padding,
boxWidth,
block.getY2() - block.getY1() + 2 * padding);
}
contentStream.fill();
} finally {
contentStream.restoreGraphicsState();
}
}
// Remove annotations whose bounding rect overlaps a redacted block, to prevent
// users from hovering over redacted URLs and seeing the underlying destination.
try {
float pageH = page.getBBox().getHeight();
List<PDAnnotation> kept = new ArrayList<>();
for (PDAnnotation ann : page.getAnnotations()) {
PDRectangle ar = ann.getRectangle();
boolean overlaps = false;
if (ar != null) {
for (PDFText block : pageBlocks) {
float padding =
(block.getY2() - block.getY1())
* DEFAULT_TEXT_PADDING_MULTIPLIER
+ customPadding;
float bx1 = block.getX1();
float bx2 = block.getX2();
float by1 = pageH - block.getY2() - padding;
float by2 = pageH - block.getY1() + padding;
if (ar.getLowerLeftX() < bx2
&& ar.getUpperRightX() > bx1
&& ar.getLowerLeftY() < by2
&& ar.getUpperRightY() > by1) {
overlaps = true;
break;
}
}
}
if (!overlaps) {
kept.add(ann);
}
}
page.setAnnotations(kept);
} catch (Exception e) {
log.debug(
"[redact] could not remove annotations on page {}: {}",
pageIndex,
e.getMessage());
}
}
}
void redactImageBoxes(PDDocument document, List<float[]> imageBoxes, Color color)
throws IOException {
Map<Integer, List<float[]>> byPage = new HashMap<>();
for (float[] box : imageBoxes) {
byPage.computeIfAbsent((int) box[0], k -> new ArrayList<>()).add(box);
}
PDPageTree pages = document.getDocumentCatalog().getPages();
for (Map.Entry<Integer, List<float[]>> entry : byPage.entrySet()) {
int pageIdx = entry.getKey();
if (pageIdx < 0 || pageIdx >= pages.getCount()) {
log.warn("[redact/execute] image box references out-of-range page {}", pageIdx);
continue;
}
PDPage page = pages.get(pageIdx);
try (PDPageContentStream cs =
new PDPageContentStream(
document, page, PDPageContentStream.AppendMode.APPEND, true, true)) {
cs.saveGraphicsState();
cs.setNonStrokingColor(color);
for (float[] box : entry.getValue()) {
float x1 = box[1], y1 = box[2], x2 = box[3], y2 = box[4];
cs.addRect(x1, y1, x2 - x1, y2 - y1);
}
cs.fill();
cs.restoreGraphicsState();
}
}
}
// -----------------------------------------------------------------------
// Page element extraction
// -----------------------------------------------------------------------
/**
* Returns bounding boxes for every text line and image on {@code page} in PDF user-space
* coordinates: {@code [x1, y1, x2, y2]} (origin bottom-left, Y increases upward).
*/
List<float[]> extractPageElementBoxes(PDDocument document, PDPage page, int pageIndex)
throws IOException {
List<float[]> boxes = new ArrayList<>();
AllTextLineExtractor textExtractor =
new AllTextLineExtractor(pageIndex + 1, page.getBBox().getHeight());
textExtractor.getText(document);
boxes.addAll(textExtractor.getLineBoxes());
PageImageLocator imgLocator = new PageImageLocator(page, pageIndex);
imgLocator.processPage(page);
for (PageImageLocator.ImageBox imgBox : imgLocator.getImageBoxes()) {
boxes.add(new float[] {imgBox.x1(), imgBox.y1(), imgBox.x2(), imgBox.y2()});
}
return boxes;
}
// -----------------------------------------------------------------------
// Finalization
// -----------------------------------------------------------------------
TempFile finalizeRedaction(
PDDocument document,
Map<Integer, List<PDFText>> allFoundTextsByPage,
String colorString,
float customPadding,
Boolean convertToImage,
boolean isTextRemovalMode)
throws IOException {
List<PDFText> allFoundTexts = new ArrayList<>();
for (List<PDFText> pageTexts : allFoundTextsByPage.values()) {
allFoundTexts.addAll(pageTexts);
}
if (!allFoundTexts.isEmpty()) {
Color redactColor = decodeOrDefault(colorString);
redactFoundText(document, allFoundTexts, customPadding, redactColor, isTextRemovalMode);
cleanDocumentMetadata(document);
}
if (Boolean.TRUE.equals(convertToImage)) {
try (PDDocument convertedPdf = PdfUtils.convertPdfToPdfImage(document)) {
cleanDocumentMetadata(convertedPdf);
TempFile tempOut = tempFileManager.createManagedTempFile(".pdf");
try {
convertedPdf.save(tempOut.getFile());
} catch (IOException e) {
tempOut.close();
throw e;
}
log.info(
"Redaction finalized (image mode): {} pages ➜ {} KB",
convertedPdf.getNumberOfPages(),
tempOut.getFile().length() / 1024);
return tempOut;
}
}
TempFile tempOut = tempFileManager.createManagedTempFile(".pdf");
try {
document.save(tempOut.getFile());
} catch (IOException e) {
tempOut.close();
throw e;
}
log.info(
"Redaction finalized: {} pages ➜ {} KB",
document.getNumberOfPages(),
tempOut.getFile().length() / 1024);
return tempOut;
}
private void cleanDocumentMetadata(PDDocument document) {
try {
var documentInfo = document.getDocumentInformation();
if (documentInfo != null) {
documentInfo.setAuthor(null);
documentInfo.setSubject(null);
documentInfo.setKeywords(null);
documentInfo.setModificationDate(java.util.Calendar.getInstance());
log.debug("Cleaned document metadata for security");
}
if (document.getDocumentCatalog() != null) {
try {
document.getDocumentCatalog().setMetadata(null);
} catch (Exception e) {
log.debug("Could not clear XMP metadata: {}", e.getMessage());
}
}
} catch (Exception e) {
log.warn("Failed to clean document metadata: {}", e.getMessage());
}
}
// -----------------------------------------------------------------------
// Utilities
// -----------------------------------------------------------------------
static Color decodeOrDefault(String hex) {
if (hex == null) {
return Color.BLACK;
}
String colorString = hex.startsWith("#") ? hex : "#" + hex;
try {
return Color.decode(colorString);
} catch (NumberFormatException e) {
return Color.BLACK;
}
}
private List<Integer> getPageNumbers(ManualRedactPdfRequest request, int pagesCount) {
String pageNumbersInput = request.getPageNumbers();
String[] parsedPageNumbers =
pageNumbersInput != null ? pageNumbersInput.split(",") : new String[0];
List<Integer> pageNumbers =
GeneralUtils.parsePageList(parsedPageNumbers, pagesCount, false);
Collections.sort(pageNumbers);
return pageNumbers;
}
}
@@ -0,0 +1,174 @@
package stirling.software.SPDF.controller.api.security;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.text.PDFTextStripper;
import org.apache.pdfbox.text.TextPosition;
import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.model.PDFText;
/**
* Scans a PDF document once and matches all provided patterns in a single pass, collecting
* bounding-box positions for every match. Use in place of creating one {@code TextFinder} per
* search term to avoid O(n) full-document scans.
*/
@Slf4j
final class MultiPatternTextFinder extends PDFTextStripper {
private static final long REGEX_MATCH_TIMEOUT_SECONDS = 30;
private static final ExecutorService REGEX_EXECUTOR =
Executors.newVirtualThreadPerTaskExecutor();
private final List<Pattern> patterns;
private final Map<Integer, List<PDFText>> foundTextsByPage = new HashMap<>();
private final List<TextPosition> pageTextPositions = new ArrayList<>();
private final StringBuilder pageTextBuilder = new StringBuilder();
MultiPatternTextFinder(List<Pattern> patterns) throws IOException {
this.patterns = patterns;
this.setWordSeparator(" ");
this.setLineSeparator("\n");
}
Map<Integer, List<PDFText>> getFoundTextsByPage() {
return foundTextsByPage;
}
@Override
protected void startPage(PDPage page) throws IOException {
super.startPage(page);
pageTextPositions.clear();
pageTextBuilder.setLength(0);
}
@Override
protected void writeString(String text, List<TextPosition> textPositions) {
pageTextBuilder.append(text);
pageTextPositions.addAll(textPositions);
}
@Override
protected void writeWordSeparator() {
pageTextBuilder.append(getWordSeparator());
pageTextPositions.add(null);
}
@Override
protected void writeLineSeparator() {
pageTextBuilder.append(getLineSeparator());
pageTextPositions.add(null);
}
@Override
protected void endPage(PDPage page) throws IOException {
String text = pageTextBuilder.toString();
if (!text.isEmpty()) {
int pageIndex = getCurrentPageNo() - 1;
for (Pattern pattern : patterns) {
Matcher matcher = pattern.matcher(text);
while (safeFind(matcher)) {
PDFText pdfText = resolveMatchPosition(matcher, pageIndex);
if (pdfText != null) {
foundTextsByPage
.computeIfAbsent(pageIndex, k -> new ArrayList<>())
.add(pdfText);
}
}
}
}
super.endPage(page);
}
/**
* Wraps a single {@code matcher.find()} call with a {@value #REGEX_MATCH_TIMEOUT_SECONDS}
* second timeout. Prevents pathological regex backtracking from blocking the request
* indefinitely; per-match timeout so fast legitimate scans are unaffected.
*/
private static boolean safeFind(Matcher matcher) throws IOException {
Future<Boolean> future =
REGEX_EXECUTOR.submit((java.util.concurrent.Callable<Boolean>) matcher::find);
try {
return future.get(REGEX_MATCH_TIMEOUT_SECONDS, TimeUnit.SECONDS);
} catch (TimeoutException e) {
future.cancel(true);
throw new IOException(
"Regex match timed out after "
+ REGEX_MATCH_TIMEOUT_SECONDS
+ "s — pattern may cause catastrophic backtracking");
} catch (InterruptedException e) {
future.cancel(true);
Thread.currentThread().interrupt();
throw new IOException("Regex match interrupted", e);
} catch (ExecutionException e) {
Throwable cause = e.getCause();
if (cause instanceof IOException ioEx) throw ioEx;
throw new IOException("Regex match failed: " + cause.getMessage(), cause);
}
}
private PDFText resolveMatchPosition(Matcher matcher, int pageIndex) {
int matchStart = matcher.start();
int matchEnd = matcher.end();
float minX = Float.MAX_VALUE;
float minY = Float.MAX_VALUE;
float maxX = Float.MIN_VALUE;
float maxY = Float.MIN_VALUE;
boolean foundPosition = false;
for (int i = matchStart; i < matchEnd; i++) {
if (i >= pageTextPositions.size()) break;
TextPosition pos = pageTextPositions.get(i);
if (pos != null) {
foundPosition = true;
minX = Math.min(minX, pos.getX());
maxX = Math.max(maxX, pos.getX() + pos.getWidth());
minY = Math.min(minY, pos.getY() - pos.getHeight());
maxY = Math.max(maxY, pos.getY());
}
}
if (!foundPosition && matchStart < pageTextPositions.size()) {
for (int i = Math.max(0, matchStart - 5);
i < Math.min(pageTextPositions.size(), matchEnd + 5);
i++) {
TextPosition pos = pageTextPositions.get(i);
if (pos != null) {
foundPosition = true;
minX = Math.min(minX, pos.getX());
maxX = Math.max(maxX, pos.getX() + pos.getWidth());
minY = Math.min(minY, pos.getY() - pos.getHeight());
maxY = Math.max(maxY, pos.getY());
break;
}
}
}
if (!foundPosition) {
log.warn(
"Found text match '{}' but no valid position data at {}-{}",
matcher.group(),
matchStart,
matchEnd);
return null;
}
return new PDFText(pageIndex, minX, minY, maxX, maxY, matcher.group());
}
}
@@ -0,0 +1,850 @@
package stirling.software.SPDF.controller.api.security;
import java.awt.Color;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.regex.Pattern;
import org.apache.pdfbox.cos.COSName;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDPageContentStream;
import org.apache.pdfbox.pdmodel.PDPageTree;
import org.apache.pdfbox.pdmodel.PDResources;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.model.PDFText;
import stirling.software.SPDF.model.api.security.RedactExecuteRequest;
import stirling.software.SPDF.model.api.security.RedactExecuteRequest.ImageBox;
import stirling.software.SPDF.model.api.security.RedactExecuteRequest.RedactStyle;
import stirling.software.SPDF.model.api.security.RedactExecuteRequest.TextRange;
import stirling.software.SPDF.pdf.parser.PageColumnLayout;
import stirling.software.SPDF.pdf.parser.PageImageLocator;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.TempFile;
@Service
@Slf4j
@RequiredArgsConstructor
class RedactExecuteService {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final ManualRedactionService manualRedactionService;
private final TextRedactionService textRedactionService;
TempFile execute(RedactExecuteRequest request) throws IOException {
RedactStyle style = request.getStyle() != null ? request.getStyle() : new RedactStyle();
List<String> textValues = orEmpty(request.getTextValues());
List<String> regexPatterns = orEmpty(request.getRegexPatterns());
List<Integer> wipePages = orEmpty(request.getWipePages());
List<TextRange> ranges = orEmpty(request.getRanges());
List<ImageBox> imageBoxes = orEmpty(request.getImageBoxes());
boolean hasTargets =
!textValues.isEmpty()
|| !regexPatterns.isEmpty()
|| !wipePages.isEmpty()
|| !ranges.isEmpty()
|| !imageBoxes.isEmpty()
|| request.getRedactImagePages() != null;
if (!hasTargets) {
throw ExceptionUtils.createIllegalArgumentException(
"error.redaction.no.targets", "No redaction targets provided");
}
boolean overlayOnly =
RedactExecuteRequest.RedactionStrategy.OVERLAY_ONLY.equals(style.getStrategy());
boolean imageFinalize =
RedactExecuteRequest.RedactionStrategy.IMAGE_FINALIZE.equals(style.getStrategy());
boolean convertToImage = imageFinalize || style.isConvertToImage();
boolean hasTextOps = !textValues.isEmpty() || !regexPatterns.isEmpty();
log.info(
"[redact/execute] strategy={} textValues={} regexPatterns={} wipePages={} ranges={} imageBoxes={} imagePages={}",
style.getStrategy(),
textValues.size(),
regexPatterns.size(),
wipePages.size(),
ranges.size(),
imageBoxes.size(),
request.getRedactImagePages());
if (request.getFileInput() == null) {
throw ExceptionUtils.createFileNullOrEmptyException();
}
PDDocument document = null;
try {
document = pdfDocumentFactory.load(request.getFileInput());
// Single-pass text scan: collect all text-based targets so we run the PDF
// stripper only once across the entire execute() call rather than once per target.
Map<Integer, List<PDFText>> foundTexts =
hasTextOps ? collectTextMatches(document, request) : new HashMap<>();
int totalMatches = foundTexts.values().stream().mapToInt(List::size).sum();
log.info(
"[redact/execute] scan complete: {} text matches across {} pages",
totalMatches,
foundTexts.size());
// Text removal (content-stream rewriting) — skipped in overlay-only mode.
boolean needsOverlayOnly = overlayOnly;
if (hasTextOps && !foundTexts.isEmpty() && !overlayOnly) {
needsOverlayOnly = applyTextRemoval(document, request);
} else if (overlayOnly) {
log.info(
"[redact/execute] overlay-only mode requested — skipping content-stream rewriting");
}
// Reload fresh document on fallback so we overlay onto clean content.
if (needsOverlayOnly && !foundTexts.isEmpty()) {
log.info("[redact/execute] reloading document for clean overlay pass");
document.close();
document = pdfDocumentFactory.load(request.getFileInput());
foundTexts.clear();
if (hasTextOps) {
foundTexts.putAll(collectTextMatches(document, request));
}
}
// Non-text operations.
Map<Integer, PageColumnLayout> layoutCache = new HashMap<>();
if (!wipePages.isEmpty()) {
applyPageWipe(document, wipePages, style);
}
for (TextRange range : ranges) {
applyRangeRedaction(document, range, style, layoutCache);
}
for (ImageBox box : imageBoxes) {
applyImageBoxRedaction(document, box, style);
}
if (request.getRedactImagePages() != null) {
applyAllImagesRedaction(document, request.getRedactImagePages(), style);
}
return manualRedactionService.finalizeRedaction(
document,
foundTexts,
style.getColor(),
style.getPadding(),
convertToImage,
!needsOverlayOnly);
} catch (Exception e) {
log.error("Execute redaction failed: {}", e.getMessage(), e);
throw new RuntimeException("Failed to perform PDF redaction: " + e.getMessage(), e);
} finally {
if (document != null) {
try {
document.close();
} catch (IOException e) {
log.warn("Failed to close document: {}", e.getMessage());
}
}
}
}
// -----------------------------------------------------------------------
// Single-pass text scan (one stripper pass per execute() call)
// -----------------------------------------------------------------------
/**
* Runs a single PDF text-stripper pass over all text-based targets and returns the merged hit
* map.
*/
private Map<Integer, List<PDFText>> collectTextMatches(
PDDocument document, RedactExecuteRequest request) {
Map<Integer, List<PDFText>> found = new HashMap<>();
String[] terms = cleanStrings(request.getTextValues());
if (terms.length > 0) {
textRedactionService
.findTextToRedact(document, terms, false, false)
.forEach(
(page, hits) ->
found.computeIfAbsent(page, k -> new ArrayList<>())
.addAll(hits));
}
String[] patterns = cleanStrings(request.getRegexPatterns());
if (patterns.length > 0) {
textRedactionService
.findTextToRedact(document, patterns, true, false)
.forEach(
(page, hits) ->
found.computeIfAbsent(page, k -> new ArrayList<>())
.addAll(hits));
}
return found;
}
// -----------------------------------------------------------------------
// Text removal (content-stream rewriting)
// -----------------------------------------------------------------------
/**
* Attempts content-stream text removal for all text/regex targets. Returns {@code true} if the
* document fell back to overlay-only mode.
*/
private boolean applyTextRemoval(PDDocument document, RedactExecuteRequest request) {
try {
boolean fallback = false;
String[] terms = cleanStrings(request.getTextValues());
if (terms.length > 0) {
Map<Integer, List<PDFText>> exactFound =
textRedactionService.findTextToRedact(document, terms, false, false);
if (!exactFound.isEmpty()) {
fallback |=
textRedactionService.performTextReplacement(
document, exactFound, terms, false, false);
}
}
String[] patterns = cleanStrings(request.getRegexPatterns());
if (patterns.length > 0) {
Map<Integer, List<PDFText>> regexFound =
textRedactionService.findTextToRedact(document, patterns, true, false);
if (!regexFound.isEmpty()) {
fallback |=
textRedactionService.performTextReplacement(
document, regexFound, patterns, true, false);
}
}
if (fallback) {
log.warn(
"[redact/execute] font compatibility issue — falling back to overlay-only");
} else {
log.info("[redact/execute] content-stream text removal applied successfully");
}
return fallback;
} catch (Exception e) {
log.warn(
"[redact/execute] text removal failed, falling back to overlay: {}",
e.getMessage());
return true;
}
}
// -----------------------------------------------------------------------
// Per-operation dispatch methods
// -----------------------------------------------------------------------
private void applyPageWipe(PDDocument document, List<Integer> pageNumbers, RedactStyle style)
throws IOException {
List<Integer> pageIndices = toZeroBasedIndices(pageNumbers);
if (pageIndices.isEmpty()) return;
PDPageTree allPages = document.getDocumentCatalog().getPages();
Color pageColor = ManualRedactionService.decodeOrDefault(style.getColor());
Collections.sort(pageIndices);
log.info("[redact/execute] full-page wipe: {} pages ({})", pageIndices.size(), pageIndices);
Map<Integer, List<float[]>> pageElementBoxes = new HashMap<>();
for (Integer idx : pageIndices) {
if (idx >= 0 && idx < allPages.getCount()) {
try {
pageElementBoxes.put(
idx,
manualRedactionService.extractPageElementBoxes(
document, allPages.get(idx), idx));
} catch (Exception e) {
log.warn(
"[redact/execute] element extraction failed for page {}: {}",
idx,
e.getMessage());
}
}
}
for (Integer idx : pageIndices) {
if (idx >= 0 && idx < allPages.getCount()) {
PDPage page = allPages.get(idx);
List<float[]> elementBoxes =
pageElementBoxes.getOrDefault(idx, Collections.emptyList());
page.getCOSObject().removeItem(COSName.CONTENTS);
page.setResources(new PDResources());
try (PDPageContentStream cs = new PDPageContentStream(document, page)) {
cs.setNonStrokingColor(pageColor);
if (elementBoxes.isEmpty()) {
PDRectangle box = page.getBBox();
cs.addRect(0, 0, box.getWidth(), box.getHeight());
} else {
log.info(
"[redact/execute] page {}: drawing {} element boxes",
idx + 1,
elementBoxes.size());
for (float[] r : elementBoxes) {
cs.addRect(r[0], r[1], r[2] - r[0], r[3] - r[1]);
}
}
cs.fill();
}
}
}
}
private void applyRangeRedaction(
PDDocument document,
TextRange range,
RedactStyle style,
Map<Integer, PageColumnLayout> layoutCache)
throws IOException {
String rangeStart = trimOrEmpty(range.startString());
String rangeEnd = trimOrEmpty(range.endString());
log.info("[redact/execute] range redaction: start='{}' end='{}'", rangeStart, rangeEnd);
try {
List<PDFText> blocks = collectRangeBlocks(document, rangeStart, rangeEnd, layoutCache);
if (!blocks.isEmpty()) {
manualRedactionService.redactFoundText(
document,
blocks,
style.getPadding(),
ManualRedactionService.decodeOrDefault(style.getColor()),
false);
} else {
log.warn(
"[redact/execute] range not found: start='{}' end='{}'",
rangeStart,
rangeEnd);
}
} catch (Exception e) {
log.warn("[redact/execute] range redaction failed: {}", e.getMessage());
}
}
private void applyImageBoxRedaction(PDDocument document, ImageBox box, RedactStyle style)
throws IOException {
List<float[]> boxes =
List.of(
new float[] {
(float) box.pageIndex(), box.x1(), box.y1(), box.x2(), box.y2()
});
log.info("[redact/execute] image box overlay on page {}", box.pageIndex());
Color boxColor = ManualRedactionService.decodeOrDefault(style.getColor());
manualRedactionService.redactImageBoxes(document, boxes, boxColor);
}
private void applyAllImagesRedaction(
PDDocument document, List<Integer> pageNumbers, RedactStyle style) throws IOException {
PDPageTree allPages = document.getDocumentCatalog().getPages();
Color imgColor = ManualRedactionService.decodeOrDefault(style.getColor());
List<Integer> imagePageIndices = toZeroBasedIndices(pageNumbers);
if (imagePageIndices.isEmpty()) {
imagePageIndices = new ArrayList<>();
for (int i = 0; i < allPages.getCount(); i++) {
imagePageIndices.add(i);
}
}
List<float[]> detectedBoxes = new ArrayList<>();
for (int pageIdx : imagePageIndices) {
if (pageIdx < 0 || pageIdx >= allPages.getCount()) continue;
try {
PDPage page = allPages.get(pageIdx);
PageImageLocator locator = new PageImageLocator(page, pageIdx);
locator.processPage(page);
for (PageImageLocator.ImageBox ib : locator.getImageBoxes()) {
detectedBoxes.add(new float[] {pageIdx, ib.x1(), ib.y1(), ib.x2(), ib.y2()});
}
} catch (Exception e) {
log.warn(
"[redact/execute] image detection failed for page {}: {}",
pageIdx + 1,
e.getMessage());
}
}
log.info(
"[redact/execute] auto image detection: {} images across {} pages",
detectedBoxes.size(),
imagePageIndices.size());
if (!detectedBoxes.isEmpty()) {
manualRedactionService.redactImageBoxes(document, detectedBoxes, imgColor);
}
}
// -----------------------------------------------------------------------
// Range collection helpers
// -----------------------------------------------------------------------
/**
* Locates {@code startStr} in the document and returns {@link PDFText} blocks for every text
* line and image from that point up to (but NOT including) the line where {@code endStr}
* begins. If {@code endStr} is blank, redacts from {@code startStr} to the end of the document.
*
* <p>Multi-column pages follow reading order: down the start column, jump to the top of the
* next column, continue to the end anchor. Single-column pages reduce to a plain Y-band check.
*/
List<PDFText> collectRangeBlocks(
PDDocument document,
String startStr,
String endStr,
Map<Integer, PageColumnLayout> layoutCache)
throws IOException {
PDPageTree allPages = document.getDocumentCatalog().getPages();
int totalPages = allPages.getCount();
Map<Integer, List<PDFText>> startMatchesByPage = findWithFallbacks(document, startStr);
if (startMatchesByPage.isEmpty()) {
log.warn("[redact/execute] range start not found: '{}'", startStr);
return Collections.emptyList();
}
List<Anchor> starts = toAnchors(document, startMatchesByPage, layoutCache);
starts.sort(READING_ORDER);
log.info(
"[redact/execute] start='{}' matched {} anchor(s): {}",
startStr,
starts.size(),
anchorSummary(starts));
boolean openEnded = (endStr == null || endStr.isBlank());
List<Anchor> ends = new ArrayList<>();
if (!openEnded) {
Map<Integer, List<PDFText>> endMatchesByPage = findWithFallbacks(document, endStr);
if (endMatchesByPage.isEmpty()) {
log.warn(
"[redact/execute] range end '{}' not found in document - skipping range"
+ " (start='{}')",
endStr,
startStr);
return Collections.emptyList();
}
ends = toAnchors(document, endMatchesByPage, layoutCache);
ends.sort(READING_ORDER);
log.info(
"[redact/execute] end='{}' matched {} anchor(s): {}",
endStr,
ends.size(),
anchorSummary(ends));
}
List<PDFText> blocks = new ArrayList<>();
for (Anchor start : starts) {
Anchor end = null;
int endPage;
if (openEnded) {
endPage = totalPages - 1;
} else {
for (Anchor candidate : ends) {
if (READING_ORDER.compare(candidate, start) > 0) {
end = candidate;
break;
}
}
if (end == null) {
log.warn(
"[redact/execute] no end anchor after start at (page={}, col={}, y={}) — skipping",
start.page + 1,
start.col,
start.y);
continue;
}
endPage = end.page;
}
log.info(
"[redact/execute] range pages {}-{}: start='{}' (col {}) end='{}'",
start.page + 1,
endPage + 1,
startStr,
start.col,
openEnded ? "<end of document>" : endStr);
collectBlocksForRange(document, allPages, start, end, openEnded, blocks, layoutCache);
}
log.info(
"[redact/execute] range '{}'→'{}': {} total blocks",
startStr,
openEnded ? "<end of document>" : endStr,
blocks.size());
return blocks;
}
/**
* Collects all redactable content (text line segments and images) between two anchor positions.
*
* <p>Line boxes are cached per page number in {@code lineBoxCache} and reused across range
* iterations within one execute() call, avoiding redundant {@link AllTextLineExtractor} passes.
*/
private void collectBlocksForRange(
PDDocument document,
PDPageTree allPages,
Anchor start,
Anchor end,
boolean openEnded,
List<PDFText> blocks,
Map<Integer, PageColumnLayout> layoutCache)
throws IOException {
int startPage = start.page;
int endPage = openEnded ? allPages.getCount() - 1 : end.page;
int endCol =
openEnded ? layoutFor(document, endPage, layoutCache).columnCount() - 1 : end.col;
float startY = start.y;
// Use bottom of end anchor so the end anchor line itself is included (inclusive range).
float endY = openEnded ? Float.POSITIVE_INFINITY : end.text.getY2();
// Line-box cache: populated lazily per page, reused across range iterations.
// Cannot use computeIfAbsent because AllTextLineExtractor's constructor throws IOException.
Map<Integer, List<float[]>> lineBoxCache = new HashMap<>();
for (int pageIdx = startPage; pageIdx <= endPage; pageIdx++) {
PDPage page = allPages.get(pageIdx);
float pageHeight = page.getBBox().getHeight();
PageColumnLayout layout = layoutFor(document, pageIdx, layoutCache);
List<float[]> screenLineBoxes = lineBoxCache.get(pageIdx);
if (screenLineBoxes == null) {
AllTextLineExtractor textExtractor =
new AllTextLineExtractor(pageIdx + 1, pageHeight);
textExtractor.getText(document);
screenLineBoxes = textExtractor.getScreenLineBoxes();
lineBoxCache.put(pageIdx, screenLineBoxes);
}
for (float[] sb : screenLineBoxes) {
emitColumnSlices(
pageIdx, layout, sb[0], sb[2], sb[1], sb[3], start.col, startPage, startY,
endCol, endPage, endY, blocks);
}
PageImageLocator imgLocator = new PageImageLocator(page, pageIdx);
imgLocator.processPage(page);
for (PageImageLocator.ImageBox ib : imgLocator.getImageBoxes()) {
// ImageBox coordinates are in PDF user-space (Y up); convert to screen-Y (Y down).
float screenY1 = pageHeight - ib.y2();
float screenY2 = pageHeight - ib.y1();
emitColumnSlices(
pageIdx, layout, ib.x1(), ib.x2(), screenY1, screenY2, start.col, startPage,
startY, endCol, endPage, endY, blocks);
}
}
}
/** Emits each per-column sub-box accepted by the reading-order predicate. */
private static void emitColumnSlices(
int pageIdx,
PageColumnLayout layout,
float x1,
float x2,
float yTop,
float yBottom,
int startCol,
int startPage,
float startY,
int endCol,
int endPage,
float endY,
List<PDFText> blocks) {
int[] cols = layout.columnsCrossing(x1, x2);
if (cols.length == 1) {
if (inColumnZone(
pageIdx, cols[0], yTop, yBottom, startPage, startCol, startY, endPage, endCol,
endY)) {
blocks.add(new PDFText(pageIdx, x1, yTop, x2, yBottom, ""));
}
return;
}
for (int col : cols) {
if (!inColumnZone(
pageIdx, col, yTop, yBottom, startPage, startCol, startY, endPage, endCol,
endY)) {
return;
}
}
blocks.add(new PDFText(pageIdx, x1, yTop, x2, yBottom, ""));
}
/**
* Reading-order predicate: true when (col, yBottom) on page {@code pageIdx} sits between the
* start anchor (inclusive) and end anchor (inclusive).
*/
static boolean inColumnZone(
int pageIdx,
int col,
float yTop,
float yBottom,
int startPage,
int startCol,
float startY,
int endPage,
int endCol,
float endY) {
if (pageIdx > startPage && pageIdx < endPage) return true;
if (pageIdx == startPage && pageIdx == endPage) {
if (startCol == endCol) {
return col == startCol && yBottom >= startY && yBottom <= endY;
}
if (startCol < endCol) {
if (col < startCol || col > endCol) return false;
if (col == startCol) return yBottom >= startY;
if (col == endCol) return yBottom <= endY;
return true;
}
return col == startCol && yBottom >= startY;
}
if (pageIdx == startPage) {
if (col == startCol) return yBottom >= startY;
return col > startCol;
}
if (pageIdx == endPage) {
if (col == endCol) return yBottom <= endY;
return col < endCol;
}
return false;
}
/** Lazily builds and caches the column layout for a single page. */
private PageColumnLayout layoutFor(
PDDocument document, int pageIdx, Map<Integer, PageColumnLayout> cache)
throws IOException {
PageColumnLayout cached = cache.get(pageIdx);
if (cached != null) return cached;
PDPage page = document.getDocumentCatalog().getPages().get(pageIdx);
float pageWidth = page.getBBox().getWidth();
float pageHeight = page.getBBox().getHeight();
AllTextLineExtractor extractor = new AllTextLineExtractor(pageIdx + 1, pageHeight);
extractor.getText(document);
PageColumnLayout layout =
PageColumnLayout.fromLineBoxes(extractor.getLineBoxes(), pageWidth);
if (layout.columnCount() > 1) {
float[] g = layout.gutters().get(0);
log.info(
"[redact/execute] page {} layout: 2 cols, gutter x=[{}, {}]",
pageIdx + 1,
g[0],
g[1]);
} else {
log.info("[redact/execute] page {} layout: 1 col (single-column mode)", pageIdx + 1);
}
cache.put(pageIdx, layout);
return layout;
}
private List<Anchor> toAnchors(
PDDocument document,
Map<Integer, List<PDFText>> matchesByPage,
Map<Integer, PageColumnLayout> layoutCache)
throws IOException {
List<Anchor> out = new ArrayList<>();
for (int page : matchesByPage.keySet().stream().sorted().toList()) {
PageColumnLayout layout = layoutFor(document, page, layoutCache);
for (PDFText hit : matchesByPage.get(page)) {
int col = layout.columnOf(hit.getX1(), hit.getX2());
out.add(new Anchor(page, col, hit.getY1(), hit));
}
}
return out;
}
/** Lexicographic ordering by (page, column, screenY). */
private static final Comparator<Anchor> READING_ORDER =
Comparator.comparingInt((Anchor a) -> a.page)
.thenComparingInt(a -> a.col)
.thenComparingDouble(a -> a.y);
private static String anchorSummary(List<Anchor> anchors) {
StringBuilder sb = new StringBuilder();
int max = Math.min(anchors.size(), 5);
for (int i = 0; i < max; i++) {
Anchor a = anchors.get(i);
if (i > 0) sb.append(", ");
sb.append(String.format("(p=%d,c=%d,y=%.1f)", a.page + 1, a.col, a.y));
}
if (anchors.size() > max) sb.append(", …");
return sb.toString();
}
private record Anchor(int page, int col, float y, PDFText text) {}
/**
* Tries progressively more permissive variants: raw (regex then literal), letter-spacing
* collapsed, then a punctuation-tolerant regex over alphanumeric runs.
*/
private Map<Integer, List<PDFText>> findWithFallbacks(PDDocument document, String raw) {
String trimmed = raw.trim();
String collapsed = collapseLetterSpacing(trimmed);
String tolerant = punctuationTolerantRegex(trimmed);
List<Candidate> candidates = new ArrayList<>();
candidates.add(new Candidate(trimmed, true));
candidates.add(new Candidate(trimmed, false));
if (!collapsed.equals(trimmed)) {
candidates.add(new Candidate(collapsed, true));
candidates.add(new Candidate(collapsed, false));
}
if (tolerant != null && !tolerant.equals(trimmed)) {
candidates.add(new Candidate(tolerant, true));
}
// If the anchor spans multiple lines (model provided entire paragraph instead of a short
// phrase), try just the first non-empty line — it's usually sufficient to locate the
// position and avoids mismatches from mid-paragraph text extraction artifacts.
if (trimmed.contains("\n")) {
String firstLine =
Arrays.stream(trimmed.split("\n"))
.map(String::trim)
.filter(s -> !s.isEmpty())
.findFirst()
.orElse(null);
if (firstLine != null && firstLine.length() >= 4) {
String firstLineCollapsed = collapseLetterSpacing(firstLine);
String firstLineTolerant = punctuationTolerantRegex(firstLine);
candidates.add(new Candidate(firstLine, false));
if (!firstLineCollapsed.equals(firstLine)) {
candidates.add(new Candidate(firstLineCollapsed, false));
}
if (firstLineTolerant != null && !firstLineTolerant.equals(firstLine)) {
candidates.add(new Candidate(firstLineTolerant, true));
}
}
}
for (Candidate c : candidates) {
Map<Integer, List<PDFText>> m =
textRedactionService.findTextToRedact(
document, new String[] {c.pattern}, c.useRegex, false);
if (!m.isEmpty()) {
if (!c.pattern.equals(trimmed)) {
log.info(
"[redact/execute] range boundary matched via fallback: '{}' → '{}'",
trimmed,
c.pattern);
}
return m;
}
}
return Collections.emptyMap();
}
private record Candidate(String pattern, boolean useRegex) {}
// -----------------------------------------------------------------------
// Static helpers
// -----------------------------------------------------------------------
/**
* Joins {@code raw}'s alphanumeric runs with {@code \W*} so anchors match across punctuation
* drift. Returns {@code null} when fewer than two tokens exist.
*/
private static String punctuationTolerantRegex(String raw) {
List<String> tokens = new ArrayList<>();
StringBuilder current = new StringBuilder();
for (int i = 0; i < raw.length(); i++) {
char ch = raw.charAt(i);
if (Character.isLetterOrDigit(ch)) {
current.append(ch);
} else if (current.length() > 0) {
tokens.add(current.toString());
current.setLength(0);
}
}
if (current.length() > 0) tokens.add(current.toString());
if (tokens.size() < 2) return null;
StringBuilder out = new StringBuilder();
for (int i = 0; i < tokens.size(); i++) {
if (i > 0) out.append("\\W*");
out.append(Pattern.quote(tokens.get(i)));
}
return out.toString();
}
/**
* Collapses letter-spaced text produced by position-sorted text extraction.
*
* <p>When a PDF text stripper runs with {@code setSortByPosition(true)}, letter-spaced headings
* come out as {@code "T a b l e o f c o n t e n t s"}. This method converts the spaced form
* back to words.
*/
private static String collapseLetterSpacing(String text) {
String[] tokens = text.split(" ", -1);
StringBuilder result = new StringBuilder();
StringBuilder current = new StringBuilder();
for (String token : tokens) {
if (token.isEmpty()) {
if (current.length() > 0) {
if (result.length() > 0) result.append(' ');
result.append(current);
current.setLength(0);
}
} else if (token.length() == 1) {
current.append(token);
} else {
if (current.length() > 0) {
if (result.length() > 0) result.append(' ');
result.append(current);
current.setLength(0);
}
if (result.length() > 0) result.append(' ');
result.append(token);
}
}
if (current.length() > 0) {
if (result.length() > 0) result.append(' ');
result.append(current);
}
return result.toString().trim();
}
private static <T> List<T> orEmpty(List<T> list) {
return list != null ? list : List.of();
}
private static String[] cleanStrings(List<String> input) {
if (input == null || input.isEmpty()) {
return new String[0];
}
return input.stream()
.filter(s -> s != null)
.map(String::trim)
.filter(s -> !s.isEmpty())
.toArray(String[]::new);
}
/**
* Converts 1-based page numbers from the request to the 0-based indices used internally.
* Out-of-range and non-positive values are silently dropped.
*/
private static List<Integer> toZeroBasedIndices(List<Integer> oneBasedPageNumbers) {
if (oneBasedPageNumbers == null || oneBasedPageNumbers.isEmpty()) {
return new ArrayList<>();
}
List<Integer> result = new ArrayList<>();
for (Integer page : oneBasedPageNumbers) {
if (page != null && page > 0) {
result.add(page - 1);
}
}
return result;
}
private static String trimOrEmpty(String s) {
return s == null ? "" : s.trim();
}
}
@@ -12,6 +12,7 @@ import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.FileSystemResource;
import org.springframework.core.io.Resource;
import org.springframework.http.CacheControl;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Controller;
@@ -40,6 +41,8 @@ public class ReactRoutingController {
private boolean indexHtmlExists = false;
private boolean useExternalIndexHtml = false;
private boolean loggedMissingIndex = false;
private String cachedSaasLandingHtml;
private boolean saasLandingExists = false;
@PostConstruct
public void init() {
@@ -48,6 +51,20 @@ public class ReactRoutingController {
// Always initialize callback HTML (used for OAuth desktop flow)
this.cachedCallbackHtml = buildCallbackHtml();
// SaaS landing page: only present on the classpath when the :saas module is bundled
// (app/saas/src/main/resources/static/saas-landing.html). When present it replaces the
// root page so the SaaS API host shows its own landing instead of the OSS API-only page.
ClassPathResource saasLanding = new ClassPathResource("static/saas-landing.html");
if (saasLanding.exists()) {
try (InputStream in = saasLanding.getInputStream()) {
this.cachedSaasLandingHtml = new String(in.readAllBytes(), StandardCharsets.UTF_8);
this.saasLandingExists = true;
log.info("SaaS landing page detected; serving it at '/' and '/index.html'");
} catch (Exception ex) {
log.warn("Failed to read saas-landing.html; falling back to index.html", ex);
}
}
// Check for external index.html first (customFiles/static/)
Path externalIndexPath = Paths.get(InstallationPathConfig.getStaticPath(), "index.html");
log.debug("Checking for custom index.html at: {}", externalIndexPath);
@@ -131,16 +148,37 @@ public class ReactRoutingController {
@GetMapping(
value = {"/", "/index.html"},
produces = MediaType.TEXT_HTML_VALUE)
public ResponseEntity<String> serveRootPage(HttpServletRequest request) {
// Swap ONLY the root page for SaaS. SPA entry points that delegate to serveIndexHtml
// (/auth/callback, /share/{token}, forwarded routes) keep serving the normal shell.
if (saasLandingExists && cachedSaasLandingHtml != null) {
return ResponseEntity.ok()
.cacheControl(CacheControl.noCache().mustRevalidate())
.contentType(MediaType.TEXT_HTML)
.body(cachedSaasLandingHtml);
}
return serveIndexHtml(request);
}
public ResponseEntity<String> serveIndexHtml(HttpServletRequest request) {
try {
if (indexHtmlExists && cachedIndexHtml != null) {
return ResponseEntity.ok().contentType(MediaType.TEXT_HTML).body(cachedIndexHtml);
return ResponseEntity.ok()
.cacheControl(CacheControl.noCache().mustRevalidate())
.contentType(MediaType.TEXT_HTML)
.body(cachedIndexHtml);
}
// Fallback: process on each request (dev mode or cache failed)
return ResponseEntity.ok().contentType(MediaType.TEXT_HTML).body(processIndexHtml());
return ResponseEntity.ok()
.cacheControl(CacheControl.noCache().mustRevalidate())
.contentType(MediaType.TEXT_HTML)
.body(processIndexHtml());
} catch (Exception ex) {
log.error("Failed to serve index.html, returning fallback", ex);
return ResponseEntity.ok().contentType(MediaType.TEXT_HTML).body(buildFallbackHtml());
return ResponseEntity.ok()
.cacheControl(CacheControl.noCache().mustRevalidate())
.contentType(MediaType.TEXT_HTML)
.body(buildFallbackHtml());
}
}
@@ -160,14 +198,19 @@ public class ReactRoutingController {
return ResponseEntity.ok().contentType(MediaType.TEXT_HTML).body(cachedCallbackHtml);
}
// `files` was historically a backend static-asset directory and was therefore
// in the exclusion list - removing it lets /files and /files/<folder-uuid>
// forward to the SPA index.html, which is what FileManagerView expects.
// (Real storage endpoints live under /api/v1/storage/files, already
// excluded by the leading `api` token in the same regex.)
@GetMapping(
"/{path:^(?!api|static|robots\\.txt|favicon\\.ico|manifest.*\\.json|pipeline|pdfjs|pdfjs-legacy|pdfium|vendor|fonts|images|files|css|js|assets|locales|modern-logo|classic-logo|Login|og_images|samples)[^\\.]*$}")
"/{path:^(?!api|static|robots\\.txt|favicon\\.ico|manifest.*\\.json|pipeline|pdfjs|pdfjs-legacy|pdfium|vendor|fonts|images|css|js|assets|locales|modern-logo|classic-logo|Login|og_images|samples)[^\\.]*$}")
public ResponseEntity<String> forwardRootPaths(HttpServletRequest request) throws IOException {
return serveIndexHtml(request);
}
@GetMapping(
"/{path:^(?!api|static|pipeline|pdfjs|pdfjs-legacy|pdfium|vendor|fonts|images|files|css|js|assets|locales|modern-logo|classic-logo|Login|og_images|samples)[^\\.]*}/{subpath:^(?!.*\\.).*$}")
"/{path:^(?!api|static|pipeline|pdfjs|pdfjs-legacy|pdfium|vendor|fonts|images|css|js|assets|locales|modern-logo|classic-logo|Login|og_images|samples)[^\\.]*}/{subpath:^(?!.*\\.).*$}")
public ResponseEntity<String> forwardNestedPaths(HttpServletRequest request)
throws IOException {
return serveIndexHtml(request);
@@ -22,7 +22,9 @@ import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.RestControllerAdvice;
import org.springframework.web.multipart.MaxUploadSizeExceededException;
import org.springframework.web.multipart.support.MissingServletRequestPartException;
import org.springframework.web.server.ResponseStatusException;
import org.springframework.web.servlet.NoHandlerFoundException;
import org.springframework.web.servlet.resource.NoResourceFoundException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
@@ -196,12 +198,12 @@ public class GlobalExceptionHandler {
/**
* Checks whether the given IOException indicates that the client disconnected before the
* response could be written (broken pipe, connection reset, etc.). When this happens there is
* no point in serialising a {@link ProblemDetail} body because the socket is already closed
* no point in serialising a {@link ProblemDetail} body because the socket is already closed -
* and attempting to do so may trigger a secondary {@code HttpMessageNotWritableException} if
* the response Content-Type was already committed as a non-JSON type (e.g. image/png).
*/
private static boolean isClientDisconnectException(IOException ex) {
// Walk the causal chain Jetty/Tomcat may wrap the low-level SocketException
// Walk the causal chain - Jetty/Tomcat may wrap the low-level SocketException
Throwable current = ex;
while (current != null) {
String msg = current.getMessage();
@@ -992,6 +994,49 @@ public class GlobalExceptionHandler {
.body(problemDetail);
}
/** Unmapped path → clean 404 instead of falling through to the generic 500 catch-all. */
@ExceptionHandler(NoResourceFoundException.class)
public ResponseEntity<ProblemDetail> handleNoResourceFound(
NoResourceFoundException ex, HttpServletRequest request) {
// /api/* miss = likely missing controller (operator-relevant); other paths = favicons,
// robots.txt, scanner noise. Demote the latter so prod logs aren't flooded.
String uri = request.getRequestURI();
if (uri != null && uri.startsWith("/api/")) {
log.warn("No resource at {}: {}", uri, ex.getMessage());
} else {
log.debug("No resource at {}: {}", uri, ex.getMessage());
}
String title = getLocalizedMessage("error.notFound.title", ErrorTitles.NOT_FOUND_DEFAULT);
String detail =
getLocalizedMessage(
"error.notFound.detail",
String.format(
"No endpoint found for %s %s",
request.getMethod(), request.getRequestURI()),
request.getMethod(),
request.getRequestURI());
ProblemDetail problemDetail =
createBaseProblemDetail(HttpStatus.NOT_FOUND, detail, request);
problemDetail.setType(URI.create(ErrorTypes.NOT_FOUND));
problemDetail.setTitle(title);
problemDetail.setProperty("title", title);
problemDetail.setProperty("method", request.getMethod());
addStandardHints(
problemDetail,
"error.notFound.hints",
List.of(
"Verify the URL path and HTTP method are correct.",
"Check the API base path and version if applicable.",
"Ensure there are no typos in the endpoint path."));
problemDetail.setProperty("actionRequired", "Use a valid endpoint URL and method.");
return ResponseEntity.status(HttpStatus.NOT_FOUND)
.contentType(PROBLEM_JSON)
.body(problemDetail);
}
/**
* Handle IllegalArgumentException.
*
@@ -1040,6 +1085,43 @@ public class GlobalExceptionHandler {
* @param request the HTTP servlet request
* @return ProblemDetail with appropriate HTTP status
*/
/**
* Handle ResponseStatusException explicitly so its embedded HTTP status reaches the client
* instead of being swallowed by the {@code RuntimeException} catch-all (which would downgrade
* every controller-thrown 400/404/409 to a generic 500). Folder/file storage controllers and
* any other code that throws {@code ResponseStatusException} relies on this handler taking
* precedence.
*/
@ExceptionHandler(ResponseStatusException.class)
public ResponseEntity<ProblemDetail> handleResponseStatusException(
ResponseStatusException ex, HttpServletRequest request) {
HttpStatus status =
HttpStatus.resolve(ex.getStatusCode().value()) != null
? HttpStatus.valueOf(ex.getStatusCode().value())
: HttpStatus.INTERNAL_SERVER_ERROR;
String reason = ex.getReason() != null ? ex.getReason() : status.getReasonPhrase();
ProblemDetail problemDetail = createBaseProblemDetail(status, reason, request);
problemDetail.setType(URI.create("/errors/" + status.value()));
problemDetail.setTitle(status.getReasonPhrase());
problemDetail.setProperty("title", status.getReasonPhrase());
// 5xx is operator-relevant; 4xx is a normal client-rejection - log at the right level.
if (status.is5xxServerError()) {
log.error(
"ResponseStatusException {} at {}: {}",
status.value(),
request.getRequestURI(),
reason,
ex);
} else {
log.debug(
"ResponseStatusException {} at {}: {}",
status.value(),
request.getRequestURI(),
reason);
}
return ResponseEntity.status(status).contentType(PROBLEM_JSON).body(problemDetail);
}
@ExceptionHandler(RuntimeException.class)
public ResponseEntity<ProblemDetail> handleRuntimeException(
RuntimeException ex, HttpServletRequest request) {
@@ -18,4 +18,11 @@ public class PDFWithPageSize extends PDFFile {
requiredMode = Schema.RequiredMode.REQUIRED,
allowableValues = {"A0", "A1", "A2", "A3", "A4", "A5", "A6", "LETTER", "LEGAL", "KEEP"})
private String pageSize;
@Schema(
description =
"Orientation to apply to the target page size. Ignored when pageSize is KEEP.",
defaultValue = "PORTRAIT",
allowableValues = {"PORTRAIT", "LANDSCAPE"})
private String orientation = "PORTRAIT";
}
@@ -13,6 +13,7 @@ import lombok.RequiredArgsConstructor;
import stirling.software.SPDF.config.swagger.MarkdownConversionResponse;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.ConvertApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.model.api.PDFFile;
import stirling.software.common.util.PDFToFile;
import stirling.software.common.util.TempFileManager;
@@ -23,7 +24,10 @@ public class ConvertPDFToMarkdown {
private final TempFileManager tempFileManager;
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/pdf/markdown")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/pdf/markdown",
resourceWeight = ResourceWeight.MEDIUM_WEIGHT)
@MarkdownConversionResponse
@Operation(
summary = "Convert PDF to Markdown",
@@ -0,0 +1,132 @@
package stirling.software.SPDF.model.api.security;
import java.util.ArrayList;
import java.util.List;
import io.swagger.v3.oas.annotations.media.Schema;
import lombok.Data;
import lombok.EqualsAndHashCode;
import stirling.software.common.model.api.PDFFile;
@Data
@EqualsAndHashCode(callSuper = true)
public class RedactExecuteRequest extends PDFFile {
@Schema(
description =
"Exact strings to find and black out. One entry per phrase to redact."
+ " Best for known names, identifiers, and specific text found in the document.")
private List<String> textValues = new ArrayList<>();
@Schema(
description =
"Regex patterns to match and redact. Each match anywhere in the document is blacked out."
+ " Uses Java/PCRE regex syntax. Well-suited for strings that follow known patterns, like"
+ " phone numbers, email addresses, national ID numbers, or"
+ " dates (which can appear with different separators, optional country codes,"
+ " etc.). For fixed known strings such as names, use textValues instead.")
private List<String> regexPatterns = new ArrayList<>();
@Schema(
description =
"1-indexed page numbers to wipe entirely (all content removed from those pages).")
private List<Integer> wipePages = new ArrayList<>();
@Schema(
description =
"Text ranges to redact by specifying a start and end anchor phrase. All"
+ " content between the two phrases (inclusive) is redacted. Anchors"
+ " work best when short and unique. They must appear"
+ " verbatim in the document.")
private List<TextRange> ranges = new ArrayList<>();
@Schema(
description =
"Rectangular areas to black out, each defined by a page number and bounding box coordinates.")
private List<ImageBox> imageBoxes = new ArrayList<>();
@Schema(
description =
"1-indexed page numbers to redact all detected images from. Pass an empty list to redact images from every page. Omit or pass null to skip image redaction entirely.")
private List<Integer> redactImagePages;
@Schema(description = "Redaction style options")
private RedactStyle style = new RedactStyle();
public record TextRange(
@Schema(
description =
"A short, distinctive phrase (515 words) that marks where"
+ " redaction begins (inclusive). Must appear verbatim in"
+ " the document — e.g. a section heading or a unique"
+ " sentence fragment.",
requiredMode = Schema.RequiredMode.REQUIRED,
minLength = 1)
String startString,
@Schema(
description =
"A short, distinctive phrase (515 words) that marks where"
+ " redaction ends (inclusive). Must appear verbatim in the"
+ " document. Shorter phrases match more reliably.",
requiredMode = Schema.RequiredMode.REQUIRED,
minLength = 1)
String endString) {
public TextRange {
if (endString == null) endString = "";
}
}
public record ImageBox(
@Schema(
description = "0-indexed page number (first page = 0).",
requiredMode = Schema.RequiredMode.REQUIRED)
int pageIndex,
@Schema(
description =
"Left x coordinate of the redaction rectangle in PDF user-space points.",
requiredMode = Schema.RequiredMode.REQUIRED)
float x1,
@Schema(
description =
"Top y coordinate of the redaction rectangle in PDF user-space points.",
requiredMode = Schema.RequiredMode.REQUIRED)
float y1,
@Schema(
description =
"Right x coordinate of the redaction rectangle in PDF user-space points.",
requiredMode = Schema.RequiredMode.REQUIRED)
float x2,
@Schema(
description =
"Bottom y coordinate of the redaction rectangle in PDF user-space points.",
requiredMode = Schema.RequiredMode.REQUIRED)
float y2) {}
public enum RedactionStrategy {
AUTO,
OVERLAY_ONLY,
IMAGE_FINALIZE
}
@Data
public static class RedactStyle {
@Schema(description = "Hex redaction box color", defaultValue = "#000000")
private String color = "#000000";
@Schema(
description = "Extra padding around each box in points",
type = "number",
defaultValue = "0")
private float padding = 0f;
@Schema(description = "Rasterize output to prevent text extraction", defaultValue = "false")
private boolean convertToImage = false;
@Schema(
description = "Execution strategy hint for the redaction pipeline",
defaultValue = "AUTO")
private RedactionStrategy strategy = RedactionStrategy.AUTO;
}
}
@@ -63,6 +63,7 @@ public class ApiDocService implements stirling.software.common.service.ToolMetad
return "http://localhost:" + port + contextPath + "/v1/api-docs";
}
@Override
public List<String> getExtensionTypes(boolean output, String operationName) {
if (outputToFileTypes.isEmpty()) {
outputToFileTypes.put("PDF", List.of("pdf"));
@@ -10,6 +10,7 @@ import org.apache.batik.bridge.GVTBuilder;
import org.apache.batik.bridge.UserAgent;
import org.apache.batik.bridge.UserAgentAdapter;
import org.apache.batik.gvt.GraphicsNode;
import org.apache.batik.util.ParsedURL;
import org.apache.batik.util.XMLResourceDescriptor;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
@@ -39,7 +40,16 @@ public class SvgOverlayUtil {
svgDoc = factory.createSVGDocument("file:///overlay.svg", inputStream);
}
UserAgent userAgent = new UserAgentAdapter();
UserAgent userAgent =
new UserAgentAdapter() {
@Override
public void checkLoadExternalResource(
ParsedURL resourceURL, ParsedURL docURL) {
throw new SecurityException(
"External resource loading is disabled for SVG overlays: "
+ resourceURL);
}
};
DocumentLoader loader = new DocumentLoader(userAgent);
BridgeContext ctx = new BridgeContext(userAgent, loader);
ctx.setDynamicState(BridgeContext.DYNAMIC);
@@ -4,6 +4,7 @@ import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.MediaType;
@@ -22,6 +23,10 @@ import jakarta.servlet.http.HttpServletRequest;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.cluster.ClusterBackplane;
import stirling.software.common.cluster.JobStore;
import stirling.software.common.cluster.JobStoreEntry;
import stirling.software.common.cluster.StickyMissRecorder;
import stirling.software.common.model.job.JobResult;
import stirling.software.common.model.job.ResultFile;
import stirling.software.common.service.FileStorage;
@@ -30,7 +35,6 @@ import stirling.software.common.service.JobQueue;
import stirling.software.common.service.TaskManager;
import stirling.software.common.util.RegexPatternUtils;
/** REST controller for job-related endpoints */
@RestController
@RequiredArgsConstructor
@Slf4j
@@ -42,20 +46,29 @@ public class JobController {
private final FileStorage fileStorage;
private final JobQueue jobQueue;
private final HttpServletRequest request;
private final ClusterBackplane clusterBackplane;
private final JobStore jobStore;
// Short-TTL local cache fronting JobStore.get() on the sticky-410 path to avoid a Valkey
// HGETALL round-trip on every download retry for the same job.
private final JobOwnershipCache ownershipCache = new JobOwnershipCache();
@Autowired(required = false)
private JobOwnershipService jobOwnershipService;
/**
* Get the status of a job
*
* @param jobId The job ID
* @return The job result
*/
@Autowired(required = false)
private StickyMissRecorder stickyMissRecorder;
@GetMapping("/job/{jobId}")
@Operation(summary = "Get job status")
public ResponseEntity<?> getJobStatus(@PathVariable("jobId") String jobId) {
// Validate job ownership
// Sticky-410 must run before user-auth: a 403 here would leak job existence and defeat
// LB re-routing. The owner node is where the real auth check should happen.
Optional<ResponseEntity<?>> peerOwned = guardNonOwner(jobId);
if (peerOwned.isPresent()) {
return peerOwned.get();
}
if (!validateJobAccess(jobId)) {
log.warn("Unauthorized attempt to access job status: {}", jobId);
return ResponseEntity.status(403)
@@ -67,7 +80,6 @@ public class JobController {
return ResponseEntity.notFound().build();
}
// Check if the job is in the queue and add queue information
if (!result.isComplete() && jobQueue.isJobQueued(jobId)) {
int position = jobQueue.getJobPosition(jobId);
Map<String, Object> resultWithQueueInfo =
@@ -82,16 +94,14 @@ public class JobController {
return ResponseEntity.ok(result);
}
/**
* Get the result of a job
*
* @param jobId The job ID
* @return The job result
*/
@GetMapping("/job/{jobId}/result")
@Operation(summary = "Get job result")
public ResponseEntity<?> getJobResult(@PathVariable("jobId") String jobId) {
// Validate job ownership
Optional<ResponseEntity<?>> peerOwned = guardNonOwner(jobId);
if (peerOwned.isPresent()) {
return peerOwned.get();
}
if (!validateJobAccess(jobId)) {
log.warn("Unauthorized attempt to access job result: {}", jobId);
return ResponseEntity.status(403)
@@ -111,7 +121,6 @@ public class JobController {
return ResponseEntity.badRequest().body("Job failed: " + result.getError());
}
// Handle multiple files - return metadata for client to download individually
if (result.hasMultipleFiles()) {
return ResponseEntity.ok()
.contentType(MediaType.APPLICATION_JSON)
@@ -125,11 +134,11 @@ public class JobController {
result.getAllResultFiles()));
}
// Handle single file (download directly)
if (result.hasFiles() && !result.hasMultipleFiles()) {
try {
List<ResultFile> files = result.getAllResultFiles();
ResultFile singleFile = files.get(0);
byte[] fileContent = fileStorage.retrieveBytes(singleFile.getFileId());
return ResponseEntity.ok()
.header("Content-Type", singleFile.getContentType())
@@ -147,30 +156,22 @@ public class JobController {
return ResponseEntity.ok(result.getResult());
}
// Admin-only endpoints have been moved to AdminJobController in the proprietary package
/**
* Cancel a job by its ID
*
* <p>This method should only allow cancellation of jobs that were created by the current user.
* The jobId should be part of the user's session or otherwise linked to their identity.
*
* @param jobId The job ID
* @return Response indicating whether the job was cancelled
*/
@DeleteMapping("/job/{jobId}")
@Operation(summary = "Cancel a job")
public ResponseEntity<?> cancelJob(@PathVariable("jobId") String jobId) {
log.debug("Request to cancel job: {}", jobId);
// Validate job ownership
Optional<ResponseEntity<?>> peerOwned = guardNonOwner(jobId);
if (peerOwned.isPresent()) {
return peerOwned.get();
}
if (!validateJobAccess(jobId)) {
log.warn("Unauthorized attempt to cancel job: {}", jobId);
return ResponseEntity.status(403)
.body(Map.of("message", "You are not authorized to cancel this job"));
}
// First check if the job is in the queue
boolean cancelled = false;
int queuePosition = -1;
@@ -180,11 +181,9 @@ public class JobController {
log.info("Cancelled queued job: {} (was at position {})", jobId, queuePosition);
}
// If not in queue or couldn't cancel, try to cancel in TaskManager
if (!cancelled) {
JobResult result = taskManager.getJobResult(jobId);
if (result != null && !result.isComplete()) {
// Mark as error with cancellation message
taskManager.setError(jobId, "Job was cancelled by user");
cancelled = true;
log.info("Marked job as cancelled in TaskManager: {}", jobId);
@@ -201,7 +200,6 @@ public class JobController {
"queuePosition",
queuePosition >= 0 ? queuePosition : "n/a"));
} else {
// Job not found or already complete
JobResult result = taskManager.getJobResult(jobId);
if (result == null) {
return ResponseEntity.notFound().build();
@@ -215,16 +213,14 @@ public class JobController {
}
}
/**
* Get the list of files for a job
*
* @param jobId The job ID
* @return List of files for the job
*/
@GetMapping("/job/{jobId}/result/files")
@Operation(summary = "Get job result files")
public ResponseEntity<?> getJobFiles(@PathVariable("jobId") String jobId) {
// Validate job ownership
Optional<ResponseEntity<?>> peerOwned = guardNonOwner(jobId);
if (peerOwned.isPresent()) {
return peerOwned.get();
}
if (!validateJobAccess(jobId)) {
log.warn("Unauthorized attempt to access job files: {}", jobId);
return ResponseEntity.status(403)
@@ -252,28 +248,31 @@ public class JobController {
"files", files));
}
/**
* Get metadata for an individual file by its file ID
*
* @param fileId The file ID
* @return The file metadata
*/
@GetMapping("/files/{fileId}/metadata")
@Operation(summary = "Get file metadata")
public ResponseEntity<?> getFileMetadata(@PathVariable("fileId") String fileId) {
try {
String jobKey = taskManager.findJobKeyByFileId(fileId);
String jobKey;
try {
jobKey = taskManager.findJobKeyByFileId(fileId);
} catch (RuntimeException backplaneEx) {
return backplaneUnavailable(fileId, backplaneEx);
}
if (jobKey == null) {
return ResponseEntity.notFound().build();
}
Optional<ResponseEntity<?>> notOwner = guardNonOwner(jobKey);
if (notOwner.isPresent()) {
return notOwner.get();
}
if (!validateJobAccess(jobKey)) {
log.warn("Unauthorized attempt to access file metadata: {}", fileId);
return ResponseEntity.status(403)
.body(Map.of("message", "You are not authorized to access this file"));
}
// Find the file metadata from any job that contains this file
ResultFile resultFile = taskManager.findResultFileByFileId(fileId);
if (resultFile != null) {
@@ -281,12 +280,10 @@ public class JobController {
}
if (!isSecurityEnabled()) {
// Backwards compatibility when ownership service is unavailable
if (!fileStorage.fileExists(fileId)) {
return ResponseEntity.notFound().build();
}
// File exists but no metadata found, get basic info efficiently
long fileSize = fileStorage.getFileSize(fileId);
return ResponseEntity.ok(
Map.of(
@@ -308,32 +305,31 @@ public class JobController {
}
}
/**
* Download an individual file by its file ID
*
* @param fileId The file ID
* @return The file content
*/
@GetMapping("/files/{fileId}")
@Operation(summary = "Download a file")
public ResponseEntity<?> downloadFile(@PathVariable("fileId") String fileId) {
try {
String jobKey = taskManager.findJobKeyByFileId(fileId);
String jobKey;
try {
jobKey = taskManager.findJobKeyByFileId(fileId);
} catch (RuntimeException backplaneEx) {
return backplaneUnavailable(fileId, backplaneEx);
}
if (jobKey == null) {
return ResponseEntity.notFound().build();
}
Optional<ResponseEntity<?>> notOwner = guardNonOwner(jobKey);
if (notOwner.isPresent()) {
return notOwner.get();
}
if (!validateJobAccess(jobKey)) {
log.warn("Unauthorized attempt to download file: {}", fileId);
return ResponseEntity.status(403)
.body(Map.of("message", "You are not authorized to access this file"));
}
// Retrieve file content
byte[] fileContent = fileStorage.retrieveBytes(fileId);
// Find the file metadata from any job that contains this file
// This is for getting the original filename and content type
ResultFile resultFile = taskManager.findResultFileByFileId(fileId);
String fileName = resultFile != null ? resultFile.getFileName() : "download";
@@ -342,6 +338,8 @@ public class JobController {
? resultFile.getContentType()
: MediaType.APPLICATION_OCTET_STREAM_VALUE;
byte[] fileContent = fileStorage.retrieveBytes(fileId);
return ResponseEntity.ok()
.header("Content-Type", contentType)
.header("Content-Disposition", createContentDispositionHeader(fileName))
@@ -357,11 +355,88 @@ public class JobController {
}
/**
* Create Content-Disposition header with UTF-8 filename support
*
* @param fileName The filename to encode
* @return Content-Disposition header value
* Returns 410 Gone when the job is owned by a peer node, empty otherwise. Uses a short-TTL
* local cache to avoid repeated Valkey lookups on the hot download path. When the backplane is
* unreachable, a locally-held job is still served and anything else gets a retryable 503.
*/
private Optional<ResponseEntity<?>> guardNonOwner(String jobId) {
if (clusterBackplane == null || jobStore == null) {
return Optional.empty();
}
Optional<JobStoreEntry> entry;
Optional<Optional<JobStoreEntry>> cached = ownershipCache.get(jobId);
if (cached.isPresent()) {
entry = cached.get();
} else {
try {
entry = jobStore.get(jobId);
} catch (RuntimeException ex) {
// Backplane unreachable: if we hold the job locally serve it, otherwise return a
// retryable 503 (same contract as the file endpoints) instead of a misleading 404.
if (taskManager.getJobResult(jobId) == null) {
return Optional.of(backplaneUnavailable(jobId, ex));
}
log.warn(
"JobStore lookup failed for jobId={}; serving locally-held job: {}",
jobId,
ex.getMessage());
return Optional.empty();
}
ownershipCache.put(jobId, entry);
}
if (entry.isEmpty()) {
return Optional.empty();
}
String owner = entry.get().owningNodeId();
if (owner == null || owner.isBlank()) {
return Optional.empty();
}
String localId = clusterBackplane.localNodeId();
if (owner.equals(localId)) {
return Optional.empty();
}
log.info(
"Sticky-session miss for jobId={} (owner={}, local={}); returning 410 so client"
+ " retries via LB affinity",
jobId,
owner,
localId);
if (stickyMissRecorder != null) {
stickyMissRecorder.recordStickyMiss();
}
return Optional.of(
ResponseEntity.status(410)
.header("Retry-After", "0")
.body(
Map.of(
"message",
"Result lives on another node. Retry to be routed there"
+ " by the load balancer's sticky-session"
+ " affinity, or re-run the job.",
"ownedBy",
owner,
"currentNode",
localId == null ? "" : localId)));
}
/**
* When the backplane is unreachable we cannot resolve ownership or existence, and serving
* without that check would be unsafe - so return a retryable 503 (consistent with the
* sticky-410 retry model) rather than a misleading 404 or a generic 500.
*/
private ResponseEntity<?> backplaneUnavailable(String id, RuntimeException ex) {
log.warn(
"Backplane lookup failed for {}; returning 503 (retryable): {}",
id,
ex.getMessage());
return ResponseEntity.status(503)
.header("Retry-After", "1")
.body(
Map.of(
"message",
"Cluster backplane temporarily unavailable; retry shortly."));
}
private String createContentDispositionHeader(String fileName) {
try {
String encodedFileName =
@@ -371,19 +446,11 @@ public class JobController {
.replaceAll("%20"); // URLEncoder uses + for spaces, but we want %20
return "attachment; filename=\"" + fileName + "\"; filename*=UTF-8''" + encodedFileName;
} catch (Exception e) {
// Fallback to basic filename if encoding fails
return "attachment; filename=\"" + fileName + "\"";
}
}
/**
* Validate that the current user has access to the given job.
*
* @param jobId the job identifier to validate
* @return true if user has access, false otherwise
*/
private boolean validateJobAccess(String jobId) {
// If JobOwnershipService is available (security enabled), use it
if (jobOwnershipService != null) {
try {
return jobOwnershipService.validateJobAccess(jobId);
@@ -393,8 +460,6 @@ public class JobController {
}
}
// Security disabled - allow all access (backwards compatibility)
// When security is not enabled, any user can access any job by jobId
return true;
}
}
@@ -0,0 +1,47 @@
package stirling.software.common.controller;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import stirling.software.common.cluster.JobStoreEntry;
/**
* Process-local TTL cache for {@link JobStoreEntry} lookups to suppress redundant Valkey HGETALL
* round-trips on the hot result-download path (sticky-410 ownership check).
*
* <p>5 second TTL is short enough that a job's lifecycle transitions (RUNNING -> COMPLETE -> TTL
* expiry) propagate to all nodes within the LB's sticky-session window, and short enough that a
* mistakenly-cached "not found" recovers quickly when an entry actually shows up. Cap the map at
* 2048 entries to bound memory; eviction is best-effort (clear-and-restart) since the cache is
* advisory.
*/
final class JobOwnershipCache {
private static final long TTL_NANOS = 5L * 1_000_000_000L; // 5 s
private static final int MAX_ENTRIES = 2048;
private final ConcurrentMap<String, Entry> entries = new ConcurrentHashMap<>();
Optional<Optional<JobStoreEntry>> get(String jobId) {
Entry e = entries.get(jobId);
if (e == null) {
return Optional.empty();
}
if (System.nanoTime() - e.storedAtNanos > TTL_NANOS) {
entries.remove(jobId, e);
return Optional.empty();
}
return Optional.of(e.value);
}
void put(String jobId, Optional<JobStoreEntry> value) {
if (entries.size() >= MAX_ENTRIES) {
// Best-effort eviction; under burst the cache simply rebuilds.
entries.clear();
}
entries.put(jobId, new Entry(value, System.nanoTime()));
}
private record Entry(Optional<JobStoreEntry> value, long storedAtNanos) {}
}
@@ -33,7 +33,7 @@ spring.security.filter.dispatcher-types=REQUEST,ERROR
# Response compression
server.compression.enabled=true
server.compression.min-response-size=1024
server.compression.mime-types=application/json,application/xml,text/html,text/plain,text/css,application/javascript
server.compression.mime-types=application/json,application/xml,text/html,text/plain,text/css,application/javascript,image/svg+xml,application/x-font-ttf,font/opentype,application/vnd.ms-fontobject,font/woff,font/woff2,application/font-woff,application/font-woff2,application/wasm
spring.web.error.path=/error
spring.web.error.whitelabel.enabled=false
@@ -93,6 +93,11 @@ posthog.host=https://eu.i.posthog.com
spring.main.allow-bean-definition-overriding=true
# spring-data-redis is on the classpath only for the optional Valkey backplane (which wires its own
# factory); exclude Spring Boot's stock Redis auto-config so a default install doesn't create a dead
# localhost:6379 factory that flips /actuator/health to DOWN.
spring.autoconfigure.exclude=org.springframework.boot.data.redis.autoconfigure.DataRedisAutoConfiguration,org.springframework.boot.data.redis.autoconfigure.DataRedisReactiveAutoConfiguration
# Set up a consistent temporary directory location
java.io.tmpdir=${stirling.tempfiles.directory:${java.io.tmpdir}/stirling-pdf}
@@ -20,6 +20,7 @@ security:
password: "" # initial password for the first login
oauth2:
enabled: false # set to 'true' to enable login (Note: enableLogin must also be 'true' for this to work)
debugLogging: false # set to 'true' to log full ID token and UserInfo claims during OAuth2/OIDC login. Use this to diagnose claim issues (e.g. "Attribute value for 'email' cannot be null" with ADFS). WARNING: writes PII (sub, email, name) to logs; disable after troubleshooting.
client:
keycloak:
issuer: "" # URL of the Keycloak realm's OpenID Connect Discovery endpoint
@@ -93,8 +94,8 @@ premium:
key: 00000000-0000-0000-0000-000000000000
enabled: false # Enable license key checks for pro/enterprise features
proFeatures:
SSOAutoLogin: false
CustomMetadata:
ssoAutoLogin: false
customMetadata:
autoUpdateMetadata: false
author: username
creator: Stirling-PDF
@@ -245,6 +246,39 @@ storage:
provider: local # storage provider: 'local' for filesystem storage, 'database' for DB-backed storage
local:
basePath: './storage' # base directory for stored files
# ====================================================================================
# S3-COMPATIBLE OBJECT STORAGE - PRO / ENTERPRISE LICENSE REQUIRED
# storage.provider=s3, storage.provider=database, and cluster.artifactStore=s3 all
# require a valid Pro or Enterprise license.
# ====================================================================================
# Used when provider=s3 (persistent user uploads) and/or cluster.artifactStore=s3
# (transient cluster artifacts). The two consumers share this block.
# Vendor cheat sheet (set the highlighted flags to taste):
# AWS S3 -> endpoint='' region='<your-region>' pathStyleAccess=false
# Cloudflare R2 -> endpoint='https://<acct>.r2.cloudflarestorage.com' region='auto'
# pathStyleAccess=false; if uploads fail with 'unsupported header
# x-amz-checksum-*' set requestChecksumCalculation=WHEN_REQUIRED
# Supabase Storage -> endpoint='https://<project>.supabase.co/storage/v1/s3'
# region='<project-region>' pathStyleAccess=true
# (filenames with non-ASCII display fine - the storage key is opaque)
# MinIO (in-cluster) -> endpoint='http://minio:9000' region='us-east-1'
# pathStyleAccess=true allowPrivateEndpoints=true
# Backblaze B2 -> endpoint='https://s3.<region>.backblazeb2.com'
# If on a B2 deployment older than July-2025 and uploads return
# 'Unsupported header x-amz-checksum-crc32', set
# requestChecksumCalculation=WHEN_REQUIRED
# DigitalOcean Spaces -> endpoint='https://<region>.digitaloceanspaces.com'
# Note: 5GB per-object cap (regardless of multipart)
s3:
endpoint: "" # blank = use AWS regional default; otherwise full URL incl. https://
bucket: "" # required when provider=s3 or cluster.artifactStore=s3
region: us-east-1
accessKey: "" # blank = fall back to AWS DefaultCredentialsProvider (env / profile / IMDS)
secretKey: ""
pathStyleAccess: false # true for MinIO and Supabase; false for AWS/R2/most CDNs
allowPrivateEndpoints: false # true required when endpoint resolves to a private/loopback IP (e.g. in-cluster MinIO). SSRF guard - leave false for any internet-facing vendor.
requestChecksumCalculation: WHEN_SUPPORTED # WHEN_SUPPORTED|WHEN_REQUIRED|DISABLED. Set WHEN_REQUIRED if your vendor rejects auto-added x-amz-checksum-* headers (older Backblaze B2, some R2 corner cases).
responseChecksumValidation: WHEN_SUPPORTED # WHEN_SUPPORTED|WHEN_REQUIRED|DISABLED. Set WHEN_REQUIRED if you see false-positive checksum-mismatch errors on GET from a vendor that never returns checksum headers.
quotas:
maxStorageMbPerUser: -1 # Max storage per user in MB; -1 disables per-user cap
maxStorageMbTotal: -1 # Max storage across all users in MB; -1 disables total cap
@@ -335,6 +369,8 @@ cluster:
enabled: false # Master switch. 'false' (default) wires the in-process backplane and skips all cluster checks. Single-instance installs do not need to change anything here.
backplane: inprocess # Backplane implementation: 'inprocess' (single JVM only) or 'valkey' (multi-node via Valkey/Redis)
artifactStore: local # Transient cluster job-artifact backend: 'local' (per-node disk; single-node only) or 's3' (shared object store; required for multi-node). Distinct from 'storage.provider' which controls persistent user uploads - when both are 's3' they share the storage.s3.* credentials block. Multi-node deployments MUST set this to 's3'.
s3:
keyPrefix: transient/ # Bucket key prefix used by the cluster artifact store when artifactStore=s3. Trailing slash recommended. Lets a single bucket host both persistent uploads (storage.s3.*) and transient job artifacts under separate prefixes.
valkey:
url: "" # Valkey/Redis URL, e.g. 'redis://valkey:6379' or 'rediss://...' for TLS. Required when enabled=true and backplane=valkey.
tls:
@@ -24,7 +24,21 @@
{
"moduleName": "com.bucket4j:bucket4j_jdk17-core",
"moduleUrl": "http://github.com/bucket4j/bucket4j/bucket4j_jdk17-core",
"moduleVersion": "8.18.0",
"moduleVersion": "8.19.0",
"moduleLicense": "The Apache Software License, Version 2.0",
"moduleLicenseUrl": "http://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "com.bucket4j:bucket4j_jdk17-lettuce",
"moduleUrl": "http://github.com/bucket4j/bucket4j/bucket4j_jdk17-redis/bucket4j_jdk17-lettuce",
"moduleVersion": "8.19.0",
"moduleLicense": "The Apache Software License, Version 2.0",
"moduleLicenseUrl": "http://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "com.bucket4j:bucket4j_jdk17-redis-common",
"moduleUrl": "http://github.com/bucket4j/bucket4j/bucket4j_jdk17-redis/bucket4j_jdk17-redis-common",
"moduleVersion": "8.19.0",
"moduleLicense": "The Apache Software License, Version 2.0",
"moduleLicenseUrl": "http://www.apache.org/licenses/LICENSE-2.0"
},
@@ -297,6 +311,48 @@
"moduleLicense": "MIT License",
"moduleLicenseUrl": "https://opensource.org/licenses/mit-license.php"
},
{
"moduleName": "com.stirling:jpdfium",
"moduleUrl": "https://github.com/Stirling-Tools/JPDFium",
"moduleVersion": "1.0.2",
"moduleLicense": "MIT License",
"moduleLicenseUrl": "https://opensource.org/licenses/MIT"
},
{
"moduleName": "com.stirling:jpdfium-natives-darwin-arm64",
"moduleUrl": "https://github.com/Stirling-Tools/JPDFium",
"moduleVersion": "1.0.2",
"moduleLicense": "MIT License",
"moduleLicenseUrl": "https://opensource.org/licenses/MIT"
},
{
"moduleName": "com.stirling:jpdfium-natives-darwin-x64",
"moduleUrl": "https://github.com/Stirling-Tools/JPDFium",
"moduleVersion": "1.0.2",
"moduleLicense": "MIT License",
"moduleLicenseUrl": "https://opensource.org/licenses/MIT"
},
{
"moduleName": "com.stirling:jpdfium-natives-linux-arm64",
"moduleUrl": "https://github.com/Stirling-Tools/JPDFium",
"moduleVersion": "1.0.2",
"moduleLicense": "MIT License",
"moduleLicenseUrl": "https://opensource.org/licenses/MIT"
},
{
"moduleName": "com.stirling:jpdfium-natives-linux-x64",
"moduleUrl": "https://github.com/Stirling-Tools/JPDFium",
"moduleVersion": "1.0.2",
"moduleLicense": "MIT License",
"moduleLicenseUrl": "https://opensource.org/licenses/MIT"
},
{
"moduleName": "com.stirling:jpdfium-natives-windows-x64",
"moduleUrl": "https://github.com/Stirling-Tools/JPDFium",
"moduleVersion": "1.0.2",
"moduleLicense": "MIT License",
"moduleLicenseUrl": "https://opensource.org/licenses/MIT"
},
{
"moduleName": "com.sun.activation:jakarta.activation",
"moduleUrl": "https://www.eclipse.org",
@@ -640,6 +696,13 @@
"moduleLicense": "Apache-2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.lettuce:lettuce-core",
"moduleUrl": "https://github.com/redis/lettuce",
"moduleVersion": "6.8.2.RELEASE",
"moduleLicense": "MIT",
"moduleLicenseUrl": "https://github.com/redis/lettuce/blob/main/LICENSE"
},
{
"moduleName": "io.micrometer:micrometer-commons",
"moduleUrl": "https://github.com/micrometer-metrics/micrometer",
@@ -675,6 +738,125 @@
"moduleLicense": "The Apache Software License, Version 2.0",
"moduleLicenseUrl": "http://www.apache.org/licenses/LICENSE-2.0.txt"
},
{
"moduleName": "io.netty:netty-buffer",
"moduleUrl": "https://netty.io/",
"moduleVersion": "4.2.12.Final",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.netty:netty-codec",
"moduleUrl": "https://netty.io/",
"moduleVersion": "4.2.12.Final",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.netty:netty-codec-base",
"moduleUrl": "https://netty.io/",
"moduleVersion": "4.2.12.Final",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.netty:netty-codec-compression",
"moduleUrl": "https://netty.io/",
"moduleVersion": "4.2.12.Final",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.netty:netty-codec-dns",
"moduleUrl": "https://netty.io/",
"moduleVersion": "4.2.12.Final",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.netty:netty-codec-http",
"moduleUrl": "https://netty.io/",
"moduleVersion": "4.2.12.Final",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.netty:netty-codec-http2",
"moduleUrl": "https://netty.io/",
"moduleVersion": "4.2.12.Final",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.netty:netty-codec-marshalling",
"moduleUrl": "https://netty.io/",
"moduleVersion": "4.2.12.Final",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.netty:netty-codec-protobuf",
"moduleUrl": "https://netty.io/",
"moduleVersion": "4.2.12.Final",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.netty:netty-common",
"moduleUrl": "https://netty.io/",
"moduleVersion": "4.2.12.Final",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.netty:netty-handler",
"moduleUrl": "https://netty.io/",
"moduleVersion": "4.2.12.Final",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.netty:netty-resolver",
"moduleUrl": "https://netty.io/",
"moduleVersion": "4.2.12.Final",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.netty:netty-resolver-dns",
"moduleUrl": "https://netty.io/",
"moduleVersion": "4.2.12.Final",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.netty:netty-transport",
"moduleUrl": "https://netty.io/",
"moduleVersion": "4.2.12.Final",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.netty:netty-transport-classes-epoll",
"moduleUrl": "https://netty.io/",
"moduleVersion": "4.2.12.Final",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.netty:netty-transport-native-unix-common",
"moduleUrl": "https://netty.io/",
"moduleVersion": "4.2.12.Final",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "io.projectreactor:reactor-core",
"moduleUrl": "https://github.com/reactor/reactor-core",
"moduleVersion": "3.8.5",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0.txt"
},
{
"moduleName": "io.prometheus:prometheus-metrics-config",
"moduleVersion": "1.4.3",
@@ -1017,6 +1199,13 @@
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0.txt"
},
{
"moduleName": "org.apache.httpcomponents:httpclient",
"moduleUrl": "http://hc.apache.org/httpcomponents-client",
"moduleVersion": "4.5.13",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "http://www.apache.org/licenses/LICENSE-2.0.txt"
},
{
"moduleName": "org.apache.httpcomponents:httpclient",
"moduleUrl": "http://hc.apache.org/httpcomponents-client-ga",
@@ -1281,21 +1470,21 @@
{
"moduleName": "org.bouncycastle:bcpkix-jdk18on",
"moduleUrl": "https://www.bouncycastle.org/download/bouncy-castle-java/",
"moduleVersion": "1.83",
"moduleVersion": "1.84",
"moduleLicense": "Bouncy Castle Licence",
"moduleLicenseUrl": "https://www.bouncycastle.org/licence.html"
},
{
"moduleName": "org.bouncycastle:bcprov-jdk18on",
"moduleUrl": "https://www.bouncycastle.org/download/bouncy-castle-java/",
"moduleVersion": "1.83",
"moduleVersion": "1.84",
"moduleLicense": "Bouncy Castle Licence",
"moduleLicenseUrl": "https://www.bouncycastle.org/licence.html"
},
{
"moduleName": "org.bouncycastle:bcutil-jdk18on",
"moduleUrl": "https://www.bouncycastle.org/download/bouncy-castle-java/",
"moduleVersion": "1.83",
"moduleVersion": "1.84",
"moduleLicense": "Bouncy Castle Licence",
"moduleLicenseUrl": "https://www.bouncycastle.org/licence.html"
},
@@ -1787,6 +1976,20 @@
"moduleLicense": "BSD-2-Clause",
"moduleLicenseUrl": "https://jdbc.postgresql.org/about/license.html"
},
{
"moduleName": "org.postgresql:postgresql",
"moduleUrl": "https://jdbc.postgresql.org/",
"moduleVersion": "42.7.11",
"moduleLicense": "BSD-2-Clause",
"moduleLicenseUrl": "https://jdbc.postgresql.org/about/license.html"
},
{
"moduleName": "org.reactivestreams:reactive-streams",
"moduleUrl": "http://www.reactive-streams.org/",
"moduleVersion": "1.0.4",
"moduleLicense": "MIT-0",
"moduleLicenseUrl": "https://spdx.org/licenses/MIT-0.html"
},
{
"moduleName": "org.simplejavamail:core-module",
"moduleVersion": "8.12.6",
@@ -1900,6 +2103,13 @@
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "org.springframework.boot:spring-boot-data-redis",
"moduleUrl": "https://spring.io/projects/spring-boot",
"moduleVersion": "4.0.6",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "org.springframework.boot:spring-boot-devtools",
"moduleUrl": "https://spring.io/projects/spring-boot",
@@ -1977,6 +2187,13 @@
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "org.springframework.boot:spring-boot-netty",
"moduleUrl": "https://spring.io/projects/spring-boot",
"moduleVersion": "4.0.6",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "org.springframework.boot:spring-boot-persistence",
"moduleUrl": "https://spring.io/projects/spring-boot",
@@ -2047,6 +2264,13 @@
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "org.springframework.boot:spring-boot-starter-data-redis",
"moduleUrl": "https://spring.io/projects/spring-boot",
"moduleVersion": "4.0.6",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "org.springframework.boot:spring-boot-starter-jackson",
"moduleUrl": "https://spring.io/projects/spring-boot",
@@ -2159,6 +2383,19 @@
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "org.springframework.data:spring-data-keyvalue",
"moduleVersion": "4.0.5",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "org.springframework.data:spring-data-redis",
"moduleUrl": "https://spring.io/projects/spring-data-redis",
"moduleVersion": "4.0.5",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "org.springframework.security:spring-security-config",
"moduleUrl": "https://spring.io/projects/spring-security",
@@ -2285,6 +2522,13 @@
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "org.springframework:spring-oxm",
"moduleUrl": "https://github.com/spring-projects/spring-framework",
"moduleVersion": "7.0.7",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
},
{
"moduleName": "org.springframework:spring-tx",
"moduleUrl": "https://github.com/spring-projects/spring-framework",
@@ -2384,6 +2628,226 @@
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "http://www.apache.org/licenses/LICENSE-2.0.txt"
},
{
"moduleName": "redis.clients.authentication:redis-authx-core",
"moduleUrl": "https://github.com/redis/redis-authx-core",
"moduleVersion": "0.1.1-beta2",
"moduleLicense": "MIT",
"moduleLicenseUrl": "https://github.com/redis/redis-authx-core/blob/master/LICENSE"
},
{
"moduleName": "software.amazon.awssdk:annotations",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:apache-client",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:arns",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:auth",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:aws-core",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:aws-query-protocol",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:aws-xml-protocol",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:checksums",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:checksums-spi",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:crt-core",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:endpoints-spi",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:http-auth",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:http-auth-aws",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:http-auth-aws-eventstream",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:http-auth-spi",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:http-client-spi",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:identity-spi",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:json-utils",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:metrics-spi",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:netty-nio-client",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:profiles",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:protocol-core",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:regions",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:retries",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:retries-spi",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:s3",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:sdk-core",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:third-party-jackson-core",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:url-connection-client",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:utils",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.awssdk:utils-lite",
"moduleUrl": "https://aws.amazon.com/sdkforjava",
"moduleVersion": "2.44.12",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "software.amazon.eventstream:eventstream",
"moduleUrl": "https://github.com/awslabs/aws-eventstream-java",
"moduleVersion": "1.0.1",
"moduleLicense": "Apache License, Version 2.0",
"moduleLicenseUrl": "https://aws.amazon.com/apache2.0"
},
{
"moduleName": "technology.tabula:tabula",
"moduleUrl": "http://github.com/tabulapdf/tabula-java",
@@ -0,0 +1,132 @@
package stirling.software.SPDF.config;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.io.IOException;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;
import org.junit.jupiter.api.Test;
import org.springframework.core.io.Resource;
import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
import org.springframework.core.io.support.ResourcePatternResolver;
import org.springframework.core.type.classreading.CachingMetadataReaderFactory;
import org.springframework.core.type.classreading.MetadataReader;
import org.springframework.core.type.classreading.MetadataReaderFactory;
import org.springframework.core.type.filter.TypeFilter;
import stirling.software.common.annotations.AutoJobPostMapping;
/**
* Build-time guardrail: every {@link AutoJobPostMapping} method must declare an explicit {@code
* resourceWeight}.
*
* <p>The credits interceptor multiplies {@code resourceWeight} into the per-call charge. An
* endpoint that falls through to the annotation default produces a charge derived from a value
* nobody chose — silently under- or over-billing depending on the endpoint's true cost. Forcing
* each method to pick a value from {@link stirling.software.common.enumeration.ResourceWeight}
* keeps the choice deliberate.
*
* <p>The annotation's default is {@link Integer#MIN_VALUE} (a sentinel). Runtime readers clamp the
* value into {@code [1, 100]}, so a missed declaration can't crash production — this test is the
* contract, the clamp is the safety net.
*
* <p>Lives in {@code :stirling-pdf} (core) because that's the module whose compile classpath
* transitively sees every other module's controllers ({@code :common}, {@code :proprietary}, and
* {@code :saas} when enabled).
*/
class AutoJobPostMappingWeightTest {
private static final String SCAN_BASE_PACKAGE = "stirling.software";
@Test
void everyAutoJobPostMappingDeclaresExplicitResourceWeight() throws Exception {
List<String> offenders = findOffendingMethods();
assertTrue(
offenders.isEmpty(),
() ->
"The following @AutoJobPostMapping methods do not declare an explicit"
+ " resourceWeight. Pick a value from"
+ " stirling.software.common.enumeration.ResourceWeight (SMALL,"
+ " MEDIUM, LARGE, XLARGE) and add it to the annotation:\n - "
+ String.join("\n - ", offenders));
}
private List<String> findOffendingMethods() throws IOException, ClassNotFoundException {
List<String> offenders = new ArrayList<>();
for (Class<?> candidate : scanForCandidateClasses()) {
for (Method method : candidate.getDeclaredMethods()) {
AutoJobPostMapping annotation = method.getAnnotation(AutoJobPostMapping.class);
if (annotation == null) {
continue;
}
if (annotation.resourceWeight() == Integer.MIN_VALUE) {
offenders.add(candidate.getName() + "#" + method.getName());
}
}
}
return offenders;
}
/**
* Returns every class under {@link #SCAN_BASE_PACKAGE} that has an @AutoJobPostMapping method.
*/
private List<Class<?>> scanForCandidateClasses() throws IOException, ClassNotFoundException {
ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
MetadataReaderFactory metadataReaderFactory = new CachingMetadataReaderFactory(resolver);
String pattern = "classpath*:" + SCAN_BASE_PACKAGE.replace('.', '/') + "/**/*.class";
Resource[] resources = resolver.getResources(pattern);
// Pre-filter by reading annotation metadata from the class file so we don't have to load
// every class on the test classpath just to find the few that are annotated.
TypeFilter mentionsAutoJobPostMapping =
(reader, factory) ->
reader.getAnnotationMetadata()
.getAnnotatedMethods(AutoJobPostMapping.class.getName())
.size()
> 0;
List<Class<?>> matches = new ArrayList<>();
for (Resource resource : resources) {
if (!resource.isReadable()) {
continue;
}
MetadataReader reader = metadataReaderFactory.getMetadataReader(resource);
if (!mentionsAutoJobPostMapping.match(reader, metadataReaderFactory)) {
continue;
}
matches.add(Class.forName(reader.getClassMetadata().getClassName()));
}
return matches;
}
/**
* Sanity check that the classpath scan returns non-empty; otherwise the main test passes
* vacuously.
*/
@Test
void scannerFindsAtLeastOneAutoJobPostMapping() throws Exception {
long count =
scanForCandidateClasses().stream()
.flatMap(c -> java.util.Arrays.stream(c.getDeclaredMethods()))
.filter(m -> m.isAnnotationPresent(AutoJobPostMapping.class))
.count();
assertTrue(
count > 10,
() ->
"Expected the classpath scan to find many @AutoJobPostMapping methods but"
+ " found only "
+ count
+ ". Scanner regression?");
}
@SuppressWarnings("unused")
private static String describeCandidates(List<Class<?>> candidates) {
return candidates.stream().map(Class::getName).collect(Collectors.joining(", "));
}
}
@@ -4,10 +4,14 @@ import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.*;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.List;
import org.apache.pdfbox.Loader;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.junit.jupiter.api.BeforeEach;
@@ -56,6 +60,38 @@ class RearrangePagesPDFControllerTest {
"fileInput", "test.pdf", MediaType.APPLICATION_PDF_VALUE, new byte[] {1, 2, 3});
}
/** Build a real, in-memory PDDocument with the requested number of blank pages. */
private PDDocument buildRealPdf(int pageCount) throws IOException {
PDDocument doc = new PDDocument();
for (int i = 0; i < pageCount; i++) {
doc.addPage(new PDPage());
}
return doc;
}
/**
* Returns the underlying {@link org.apache.pdfbox.cos.COSDictionary} for each page in document
* order. PDPageTree returns a fresh PDPage wrapper per get(), so comparing wrappers with
* assertSame is unreliable - the COSDictionary identity is the stable handle.
*/
private List<Object> snapshotCosPages(PDDocument doc) {
List<Object> snapshot = new ArrayList<>();
for (PDPage p : doc.getPages()) {
snapshot.add(p.getCOSObject());
}
return snapshot;
}
private List<Object> reloadAndSnapshot(ResponseEntity<Resource> response) throws IOException {
try (var in = response.getBody().getInputStream();
var baos = new ByteArrayOutputStream()) {
in.transferTo(baos);
try (PDDocument out = Loader.loadPDF(baos.toByteArray())) {
return snapshotCosPages(out);
}
}
}
@Test
void testDeletePages_Success() throws IOException {
MockMultipartFile file = createMockPdf();
@@ -83,27 +119,23 @@ class RearrangePagesPDFControllerTest {
request.setPageNumbers("");
request.setCustomMode("REVERSE_ORDER");
PDDocument mockDoc = mock(PDDocument.class);
PDDocument mockNewDoc = mock(PDDocument.class);
PDPage page0 = mock(PDPage.class);
PDPage page1 = mock(PDPage.class);
PDPage page2 = mock(PDPage.class);
try (PDDocument realDoc = buildRealPdf(3)) {
List<Object> originals = snapshotCosPages(realDoc);
when(pdfDocumentFactory.load(file)).thenReturn(realDoc);
when(pdfDocumentFactory.load(file)).thenReturn(mockDoc);
when(mockDoc.getNumberOfPages()).thenReturn(3);
when(mockDoc.getPage(0)).thenReturn(page0);
when(mockDoc.getPage(1)).thenReturn(page1);
when(mockDoc.getPage(2)).thenReturn(page2);
when(pdfDocumentFactory.createNewDocumentBasedOnOldDocument(mockDoc))
.thenReturn(mockNewDoc);
ResponseEntity<Resource> response = controller.rearrangePages(request);
ResponseEntity<Resource> response = controller.rearrangePages(request);
assertNotNull(response);
assertEquals(200, response.getStatusCode().value());
verify(mockNewDoc).addPage(page2);
verify(mockNewDoc).addPage(page1);
verify(mockNewDoc).addPage(page0);
assertNotNull(response);
assertEquals(200, response.getStatusCode().value());
List<Object> finalOrder = reloadAndSnapshot(response);
assertEquals(3, finalOrder.size());
// We can no longer compare references after a save/reload, so compare via
// the in-memory snapshot taken *after* the controller mutated the source.
List<Object> mutatedSource = snapshotCosPages(realDoc);
assertSame(originals.get(2), mutatedSource.get(0));
assertSame(originals.get(1), mutatedSource.get(1));
assertSame(originals.get(0), mutatedSource.get(2));
}
}
@Test
@@ -114,25 +146,18 @@ class RearrangePagesPDFControllerTest {
request.setPageNumbers("");
request.setCustomMode("REMOVE_FIRST");
PDDocument mockDoc = mock(PDDocument.class);
PDDocument mockNewDoc = mock(PDDocument.class);
PDPage page0 = mock(PDPage.class);
PDPage page1 = mock(PDPage.class);
PDPage page2 = mock(PDPage.class);
try (PDDocument realDoc = buildRealPdf(3)) {
List<Object> originals = snapshotCosPages(realDoc);
when(pdfDocumentFactory.load(file)).thenReturn(realDoc);
when(pdfDocumentFactory.load(file)).thenReturn(mockDoc);
when(mockDoc.getNumberOfPages()).thenReturn(3);
when(mockDoc.getPage(1)).thenReturn(page1);
when(mockDoc.getPage(2)).thenReturn(page2);
when(pdfDocumentFactory.createNewDocumentBasedOnOldDocument(mockDoc))
.thenReturn(mockNewDoc);
ResponseEntity<Resource> response = controller.rearrangePages(request);
ResponseEntity<Resource> response = controller.rearrangePages(request);
assertNotNull(response);
verify(mockNewDoc).addPage(page1);
verify(mockNewDoc).addPage(page2);
verify(mockNewDoc, never()).addPage(page0);
assertNotNull(response);
List<Object> mutated = snapshotCosPages(realDoc);
assertEquals(2, mutated.size());
assertSame(originals.get(1), mutated.get(0));
assertSame(originals.get(2), mutated.get(1));
}
}
@Test
@@ -143,23 +168,18 @@ class RearrangePagesPDFControllerTest {
request.setPageNumbers("");
request.setCustomMode("REMOVE_LAST");
PDDocument mockDoc = mock(PDDocument.class);
PDDocument mockNewDoc = mock(PDDocument.class);
PDPage page0 = mock(PDPage.class);
PDPage page1 = mock(PDPage.class);
try (PDDocument realDoc = buildRealPdf(3)) {
List<Object> originals = snapshotCosPages(realDoc);
when(pdfDocumentFactory.load(file)).thenReturn(realDoc);
when(pdfDocumentFactory.load(file)).thenReturn(mockDoc);
when(mockDoc.getNumberOfPages()).thenReturn(3);
when(mockDoc.getPage(0)).thenReturn(page0);
when(mockDoc.getPage(1)).thenReturn(page1);
when(pdfDocumentFactory.createNewDocumentBasedOnOldDocument(mockDoc))
.thenReturn(mockNewDoc);
ResponseEntity<Resource> response = controller.rearrangePages(request);
ResponseEntity<Resource> response = controller.rearrangePages(request);
assertNotNull(response);
verify(mockNewDoc).addPage(page0);
verify(mockNewDoc).addPage(page1);
assertNotNull(response);
List<Object> mutated = snapshotCosPages(realDoc);
assertEquals(2, mutated.size());
assertSame(originals.get(0), mutated.get(0));
assertSame(originals.get(1), mutated.get(1));
}
}
@Test
@@ -170,21 +190,19 @@ class RearrangePagesPDFControllerTest {
request.setPageNumbers("");
request.setCustomMode("REMOVE_FIRST_AND_LAST");
PDDocument mockDoc = mock(PDDocument.class);
PDDocument mockNewDoc = mock(PDDocument.class);
PDPage page1 = mock(PDPage.class);
try (PDDocument realDoc = buildRealPdf(4)) {
List<Object> originals = snapshotCosPages(realDoc);
when(pdfDocumentFactory.load(file)).thenReturn(realDoc);
when(pdfDocumentFactory.load(file)).thenReturn(mockDoc);
when(mockDoc.getNumberOfPages()).thenReturn(4);
when(mockDoc.getPage(1)).thenReturn(page1);
when(mockDoc.getPage(2)).thenReturn(page1);
when(pdfDocumentFactory.createNewDocumentBasedOnOldDocument(mockDoc))
.thenReturn(mockNewDoc);
ResponseEntity<Resource> response = controller.rearrangePages(request);
ResponseEntity<Resource> response = controller.rearrangePages(request);
assertNotNull(response);
assertEquals(200, response.getStatusCode().value());
assertNotNull(response);
assertEquals(200, response.getStatusCode().value());
List<Object> mutated = snapshotCosPages(realDoc);
assertEquals(2, mutated.size());
assertSame(originals.get(1), mutated.get(0));
assertSame(originals.get(2), mutated.get(1));
}
}
@Test
@@ -195,23 +213,15 @@ class RearrangePagesPDFControllerTest {
request.setPageNumbers("");
request.setCustomMode("DUPLEX_SORT");
PDDocument mockDoc = mock(PDDocument.class);
PDDocument mockNewDoc = mock(PDDocument.class);
PDPage page0 = mock(PDPage.class);
PDPage page1 = mock(PDPage.class);
PDPage page2 = mock(PDPage.class);
PDPage page3 = mock(PDPage.class);
try (PDDocument realDoc = buildRealPdf(4)) {
when(pdfDocumentFactory.load(file)).thenReturn(realDoc);
when(pdfDocumentFactory.load(file)).thenReturn(mockDoc);
when(mockDoc.getNumberOfPages()).thenReturn(4);
when(mockDoc.getPage(anyInt())).thenReturn(page0);
when(pdfDocumentFactory.createNewDocumentBasedOnOldDocument(mockDoc))
.thenReturn(mockNewDoc);
ResponseEntity<Resource> response = controller.rearrangePages(request);
ResponseEntity<Resource> response = controller.rearrangePages(request);
assertNotNull(response);
assertEquals(200, response.getStatusCode().value());
assertNotNull(response);
assertEquals(200, response.getStatusCode().value());
assertEquals(4, realDoc.getNumberOfPages());
}
}
@Test
@@ -222,20 +232,15 @@ class RearrangePagesPDFControllerTest {
request.setPageNumbers("");
request.setCustomMode("BOOKLET_SORT");
PDDocument mockDoc = mock(PDDocument.class);
PDDocument mockNewDoc = mock(PDDocument.class);
PDPage page = mock(PDPage.class);
try (PDDocument realDoc = buildRealPdf(4)) {
when(pdfDocumentFactory.load(file)).thenReturn(realDoc);
when(pdfDocumentFactory.load(file)).thenReturn(mockDoc);
when(mockDoc.getNumberOfPages()).thenReturn(4);
when(mockDoc.getPage(anyInt())).thenReturn(page);
when(pdfDocumentFactory.createNewDocumentBasedOnOldDocument(mockDoc))
.thenReturn(mockNewDoc);
ResponseEntity<Resource> response = controller.rearrangePages(request);
ResponseEntity<Resource> response = controller.rearrangePages(request);
assertNotNull(response);
assertEquals(200, response.getStatusCode().value());
assertNotNull(response);
assertEquals(200, response.getStatusCode().value());
assertEquals(4, realDoc.getNumberOfPages());
}
}
@Test
@@ -246,20 +251,15 @@ class RearrangePagesPDFControllerTest {
request.setPageNumbers("");
request.setCustomMode("ODD_EVEN_SPLIT");
PDDocument mockDoc = mock(PDDocument.class);
PDDocument mockNewDoc = mock(PDDocument.class);
PDPage page = mock(PDPage.class);
try (PDDocument realDoc = buildRealPdf(4)) {
when(pdfDocumentFactory.load(file)).thenReturn(realDoc);
when(pdfDocumentFactory.load(file)).thenReturn(mockDoc);
when(mockDoc.getNumberOfPages()).thenReturn(4);
when(mockDoc.getPage(anyInt())).thenReturn(page);
when(pdfDocumentFactory.createNewDocumentBasedOnOldDocument(mockDoc))
.thenReturn(mockNewDoc);
ResponseEntity<Resource> response = controller.rearrangePages(request);
ResponseEntity<Resource> response = controller.rearrangePages(request);
assertNotNull(response);
assertEquals(200, response.getStatusCode().value());
assertNotNull(response);
assertEquals(200, response.getStatusCode().value());
assertEquals(4, realDoc.getNumberOfPages());
}
}
@Test
@@ -270,24 +270,20 @@ class RearrangePagesPDFControllerTest {
request.setPageNumbers("3,1,2");
request.setCustomMode("custom");
PDDocument mockDoc = mock(PDDocument.class);
PDDocument mockNewDoc = mock(PDDocument.class);
PDPage page0 = mock(PDPage.class);
PDPage page1 = mock(PDPage.class);
PDPage page2 = mock(PDPage.class);
try (PDDocument realDoc = buildRealPdf(3)) {
List<Object> originals = snapshotCosPages(realDoc);
when(pdfDocumentFactory.load(file)).thenReturn(realDoc);
when(pdfDocumentFactory.load(file)).thenReturn(mockDoc);
when(mockDoc.getNumberOfPages()).thenReturn(3);
when(mockDoc.getPage(0)).thenReturn(page0);
when(mockDoc.getPage(1)).thenReturn(page1);
when(mockDoc.getPage(2)).thenReturn(page2);
when(pdfDocumentFactory.createNewDocumentBasedOnOldDocument(mockDoc))
.thenReturn(mockNewDoc);
ResponseEntity<Resource> response = controller.rearrangePages(request);
ResponseEntity<Resource> response = controller.rearrangePages(request);
assertNotNull(response);
assertEquals(200, response.getStatusCode().value());
assertNotNull(response);
assertEquals(200, response.getStatusCode().value());
List<Object> mutated = snapshotCosPages(realDoc);
assertEquals(3, mutated.size());
assertSame(originals.get(2), mutated.get(0));
assertSame(originals.get(0), mutated.get(1));
assertSame(originals.get(1), mutated.get(2));
}
}
@Test
@@ -298,21 +294,15 @@ class RearrangePagesPDFControllerTest {
request.setPageNumbers("3");
request.setCustomMode("DUPLICATE");
PDDocument mockDoc = mock(PDDocument.class);
PDDocument mockNewDoc = mock(PDDocument.class);
PDPage page = mock(PDPage.class);
try (PDDocument realDoc = buildRealPdf(2)) {
when(pdfDocumentFactory.load(file)).thenReturn(realDoc);
when(pdfDocumentFactory.load(file)).thenReturn(mockDoc);
when(mockDoc.getNumberOfPages()).thenReturn(2);
when(mockDoc.getPage(anyInt())).thenReturn(page);
when(pdfDocumentFactory.createNewDocumentBasedOnOldDocument(mockDoc))
.thenReturn(mockNewDoc);
ResponseEntity<Resource> response = controller.rearrangePages(request);
ResponseEntity<Resource> response = controller.rearrangePages(request);
assertNotNull(response);
// 2 pages * 3 duplicates = 6 addPage calls
verify(mockNewDoc, times(6)).addPage(page);
assertNotNull(response);
// 2 pages * 3 duplicates = 6 final pages
assertEquals(6, realDoc.getNumberOfPages());
}
}
@Test
@@ -323,19 +313,14 @@ class RearrangePagesPDFControllerTest {
request.setPageNumbers("");
request.setCustomMode("SIDE_STITCH_BOOKLET_SORT");
PDDocument mockDoc = mock(PDDocument.class);
PDDocument mockNewDoc = mock(PDDocument.class);
PDPage page = mock(PDPage.class);
try (PDDocument realDoc = buildRealPdf(4)) {
when(pdfDocumentFactory.load(file)).thenReturn(realDoc);
when(pdfDocumentFactory.load(file)).thenReturn(mockDoc);
when(mockDoc.getNumberOfPages()).thenReturn(4);
when(mockDoc.getPage(anyInt())).thenReturn(page);
when(pdfDocumentFactory.createNewDocumentBasedOnOldDocument(mockDoc))
.thenReturn(mockNewDoc);
ResponseEntity<Resource> response = controller.rearrangePages(request);
ResponseEntity<Resource> response = controller.rearrangePages(request);
assertNotNull(response);
assertEquals(200, response.getStatusCode().value());
assertNotNull(response);
assertEquals(200, response.getStatusCode().value());
assertEquals(4, realDoc.getNumberOfPages());
}
}
}
@@ -237,7 +237,8 @@ class ScalePagesControllerTest {
ScalePagesRequest request = new ScalePagesRequest();
request.setFileInput(file);
request.setPageSize("A4_LANDSCAPE");
request.setPageSize("A4");
request.setOrientation("LANDSCAPE");
request.setScaleFactor(1.0f);
setupFactory();
@@ -15,10 +15,14 @@ import org.springframework.context.ApplicationContext;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import jakarta.servlet.http.HttpServletRequest;
import stirling.software.SPDF.config.EndpointConfiguration;
import stirling.software.SPDF.config.EndpointConfiguration.DisableReason;
import stirling.software.SPDF.config.EndpointConfiguration.EndpointAvailability;
import stirling.software.common.configuration.AppConfig;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.ApplicationProperties.System;
import stirling.software.common.service.LicenseServiceInterface;
import stirling.software.common.service.ServerCertificateServiceInterface;
import stirling.software.common.service.UserServiceInterface;
@@ -173,4 +177,119 @@ class ConfigControllerTest {
assertEquals(HttpStatus.OK, response.getStatusCode());
verify(endpointConfiguration).getAllEndpoints();
}
@Test
void resolveFrontendUrl_prefersExplicitConfiguredValue() {
System sys = mock(System.class);
when(applicationProperties.getSystem()).thenReturn(sys);
when(sys.getFrontendUrl()).thenReturn("https://pdf.example.com");
// Request would say something else, but configured wins.
HttpServletRequest req = mock(HttpServletRequest.class);
AppConfig appConfig = mock(AppConfig.class);
assertEquals(
"https://pdf.example.com", configController.resolveFrontendUrl(req, appConfig));
}
@Test
void resolveFrontendUrl_usesRequestHostWhenNotConfigured() {
System sys = mock(System.class);
when(applicationProperties.getSystem()).thenReturn(sys);
when(sys.getFrontendUrl()).thenReturn(null);
HttpServletRequest req = mock(HttpServletRequest.class);
when(req.getServerName()).thenReturn("192.168.1.100");
when(req.getScheme()).thenReturn("http");
when(req.getServerPort()).thenReturn(8080);
assertEquals(
"http://192.168.1.100:8080",
configController.resolveFrontendUrl(req, mock(AppConfig.class)));
}
@Test
void resolveFrontendUrl_elidesDefaultHttpsPort() {
System sys = mock(System.class);
when(applicationProperties.getSystem()).thenReturn(sys);
when(sys.getFrontendUrl()).thenReturn("");
HttpServletRequest req = mock(HttpServletRequest.class);
when(req.getServerName()).thenReturn("pdf.example.com");
when(req.getScheme()).thenReturn("https");
when(req.getServerPort()).thenReturn(443);
assertEquals(
"https://pdf.example.com",
configController.resolveFrontendUrl(req, mock(AppConfig.class)));
}
@Test
void resolveFrontendUrl_fallsThroughOnLoopbackHost() {
System sys = mock(System.class);
when(applicationProperties.getSystem()).thenReturn(sys);
when(sys.getFrontendUrl()).thenReturn(null);
HttpServletRequest req = mock(HttpServletRequest.class);
when(req.getServerName()).thenReturn("localhost");
AppConfig appConfig = mock(AppConfig.class);
when(appConfig.getBackendUrl()).thenReturn("http://localhost:8080");
when(appConfig.getServerPort()).thenReturn("8080");
// Detected IP (if any) wins over loopback request host. We can't assert the
// exact value (depends on the host running the test) but we can assert it
// never returns "localhost".
String result = configController.resolveFrontendUrl(req, appConfig);
assertNotNull(result);
assertFalse(result.contains("localhost"));
}
@Test
void resolveFrontendUrl_usesActualPortWhenServerPortIsEphemeral() {
System sys = mock(System.class);
when(applicationProperties.getSystem()).thenReturn(sys);
when(sys.getFrontendUrl()).thenReturn(null);
// Loopback host forces the detected-LAN-IP branch, which is where an
// ephemeral server.port=0 would otherwise leak through as ":0".
HttpServletRequest req = mock(HttpServletRequest.class);
when(req.getServerName()).thenReturn("localhost");
AppConfig appConfig = mock(AppConfig.class);
when(appConfig.getBackendUrl()).thenReturn("http://localhost");
when(appConfig.getServerPort()).thenReturn("0");
org.springframework.core.env.Environment environment =
mock(org.springframework.core.env.Environment.class);
when(applicationContext.getEnvironment()).thenReturn(environment);
when(environment.getProperty("local.server.port")).thenReturn("54321");
String result = configController.resolveFrontendUrl(req, appConfig);
assertNotNull(result);
assertTrue(result.endsWith(":54321"));
assertFalse(result.contains(":0"));
}
@Test
void resolveEffectiveServerPort_prefersActualBoundPortWhenConfiguredZero() {
AppConfig appConfig = mock(AppConfig.class);
when(appConfig.getServerPort()).thenReturn("0");
org.springframework.core.env.Environment environment =
mock(org.springframework.core.env.Environment.class);
when(applicationContext.getEnvironment()).thenReturn(environment);
when(environment.getProperty("local.server.port")).thenReturn("54321");
assertEquals("54321", configController.resolveEffectiveServerPort(appConfig));
}
@Test
void resolveEffectiveServerPort_keepsConfiguredNonZeroPort() {
AppConfig appConfig = mock(AppConfig.class);
when(appConfig.getServerPort()).thenReturn("8080");
// Non-zero configured port is authoritative; the runtime env is never consulted.
assertEquals("8080", configController.resolveEffectiveServerPort(appConfig));
}
}
@@ -32,6 +32,7 @@ import org.springframework.mock.web.MockMultipartFile;
import stirling.software.SPDF.model.api.misc.OverlayImageRequest;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.SvgSanitizer;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@@ -52,6 +53,7 @@ class OverlayImageControllerTest {
@Mock private CustomPDFDocumentFactory pdfDocumentFactory;
@Mock private TempFileManager tempFileManager;
@Mock private SvgSanitizer svgSanitizer;
@InjectMocks private OverlayImageController controller;
@@ -205,6 +207,52 @@ class OverlayImageControllerTest {
mockDoc.close();
}
@Test
void overlayImage_svgInput_sanitizedBeforeOverlay() throws Exception {
byte[] maliciousSvg =
("<svg xmlns=\"http://www.w3.org/2000/svg\""
+ " xmlns:xlink=\"http://www.w3.org/1999/xlink\""
+ " width=\"10\" height=\"10\">"
+ "<image x=\"0\" y=\"0\" width=\"10\" height=\"10\""
+ " xlink:href=\"file:///etc/passwd\"/>"
+ "</svg>")
.getBytes();
byte[] sanitized =
("<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"10\" height=\"10\">"
+ "<image x=\"0\" y=\"0\" width=\"10\" height=\"10\"/>"
+ "</svg>")
.getBytes();
when(svgSanitizer.sanitize(maliciousSvg)).thenReturn(sanitized);
MockMultipartFile svgFile =
new MockMultipartFile("imageFile", "overlay.svg", "image/svg+xml", maliciousSvg);
OverlayImageRequest request = new OverlayImageRequest();
request.setFileInput(pdfFile);
request.setImageFile(svgFile);
request.setX(0);
request.setY(0);
request.setEveryPage(false);
PDDocument mockDoc = new PDDocument();
mockDoc.addPage(new PDPage(PDRectangle.A4));
when(pdfDocumentFactory.load(any(byte[].class))).thenReturn(mockDoc);
try (MockedStatic<WebResponseUtils> mockedWebResponse =
mockStatic(WebResponseUtils.class)) {
mockedWebResponse
.when(
() ->
WebResponseUtils.pdfFileToWebResponse(
any(TempFile.class), anyString()))
.thenReturn(streamingOk("result".getBytes()));
controller.overlayImage(request);
}
mockDoc.close();
verify(svgSanitizer).sanitize(maliciousSvg);
}
@Test
void overlayImage_withCoordinates_usesXY() throws Exception {
OverlayImageRequest request = new OverlayImageRequest();
@@ -37,7 +37,6 @@ import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.mockito.junit.jupiter.MockitoSettings;
@@ -77,8 +76,11 @@ class RedactControllerTest {
@Mock private CustomPDFDocumentFactory pdfDocumentFactory;
@Mock private TempFileManager tempFileManager;
@Mock private RedactExecuteService redactExecuteService;
@InjectMocks private RedactController redactController;
private TextRedactionService textRedactionService;
private ManualRedactionService manualRedactionService;
private RedactController redactController;
private MockMultipartFile mockPdfFile;
private PDDocument mockDocument;
@@ -201,7 +203,17 @@ class RedactControllerTest {
.save(any(File.class));
doNothing().when(mockDocument).close();
// Initialize a real document for unit tests
// Build real service instances so tests exercise actual logic
textRedactionService = new TextRedactionService();
manualRedactionService = new ManualRedactionService(tempFileManager);
redactController =
new RedactController(
pdfDocumentFactory,
tempFileManager,
manualRedactionService,
textRedactionService,
redactExecuteService);
setupRealDocument();
}
@@ -819,9 +831,9 @@ class RedactControllerTest {
contentStream.newLineAtOffset(50, 750);
contentStream.showText("This is ");
contentStream.newLineAtOffset(-10, 0); // Simulate positioning
contentStream.newLineAtOffset(-10, 0);
contentStream.showText("secret");
contentStream.newLineAtOffset(10, 0); // Reset positioning
contentStream.newLineAtOffset(10, 0);
contentStream.showText(" information");
contentStream.endText();
}
@@ -1005,7 +1017,7 @@ class RedactControllerTest {
contentStream.showText("Original content");
contentStream.endText();
}
return redactController.createTokensWithoutTargetText(
return textRedactionService.createTokensWithoutTargetText(
realDocument, pageForTokenExtraction, Collections.emptySet(), false, false);
}
@@ -1016,28 +1028,28 @@ class RedactControllerTest {
@Test
@DisplayName("Should decode valid hex color with hash")
void decodeValidHexColorWithHash() {
Color result = redactController.decodeOrDefault("#FF0000");
Color result = ManualRedactionService.decodeOrDefault("#FF0000");
assertEquals(Color.RED, result);
}
@Test
@DisplayName("Should decode valid hex color without hash")
void decodeValidHexColorWithoutHash() {
Color result = redactController.decodeOrDefault("FF0000");
Color result = ManualRedactionService.decodeOrDefault("FF0000");
assertEquals(Color.RED, result);
}
@Test
@DisplayName("Should default to black for null color")
void defaultToBlackForNullColor() {
Color result = redactController.decodeOrDefault(null);
Color result = ManualRedactionService.decodeOrDefault(null);
assertEquals(Color.BLACK, result);
}
@Test
@DisplayName("Should default to black for invalid color")
void defaultToBlackForInvalidColor() {
Color result = redactController.decodeOrDefault("invalid-color");
Color result = ManualRedactionService.decodeOrDefault("invalid-color");
assertEquals(Color.BLACK, result);
}
@@ -1049,7 +1061,7 @@ class RedactControllerTest {
})
@DisplayName("Should handle various valid color formats")
void handleVariousValidColorFormats(String colorInput) {
Color result = redactController.decodeOrDefault(colorInput);
Color result = ManualRedactionService.decodeOrDefault(colorInput);
assertNotNull(result);
assertTrue(
result.getRed() >= 0 && result.getRed() <= 255,
@@ -1065,8 +1077,8 @@ class RedactControllerTest {
@Test
@DisplayName("Should handle short hex codes appropriately")
void handleShortHexCodes() {
Color result1 = redactController.decodeOrDefault("123");
Color result2 = redactController.decodeOrDefault("#12");
Color result1 = ManualRedactionService.decodeOrDefault("123");
Color result2 = ManualRedactionService.decodeOrDefault("#12");
assertNotNull(result1);
assertNotNull(result2);
@@ -1094,7 +1106,7 @@ class RedactControllerTest {
Set<String> targetWords = Set.of("confidential");
List<Object> tokens =
redactController.createTokensWithoutTargetText(
textRedactionService.createTokensWithoutTargetText(
realDocument, realPage, targetWords, false, false);
assertNotNull(tokens);
@@ -1115,7 +1127,7 @@ class RedactControllerTest {
Set<String> targetWords = Set.of("secret");
List<Object> tokens =
redactController.createTokensWithoutTargetText(
textRedactionService.createTokensWithoutTargetText(
realDocument, realPage, targetWords, false, false);
assertNotNull(tokens);
@@ -1148,7 +1160,7 @@ class RedactControllerTest {
List<Object> originalTokens = getOriginalTokens();
List<Object> filteredTokens =
redactController.createTokensWithoutTargetText(
textRedactionService.createTokensWithoutTargetText(
realDocument, realPage, targetWords, false, false);
long originalNonTextCount =
@@ -1156,7 +1168,7 @@ class RedactControllerTest {
.filter(
token ->
token instanceof Operator op
&& !redactController.isTextShowingOperator(
&& !textRedactionService.isTextShowingOperator(
op.getName()))
.count();
@@ -1165,7 +1177,7 @@ class RedactControllerTest {
.filter(
token ->
token instanceof Operator op
&& !redactController.isTextShowingOperator(
&& !textRedactionService.isTextShowingOperator(
op.getName()))
.count();
@@ -1184,7 +1196,7 @@ class RedactControllerTest {
Set<String> targetWords = Set.of("\\d{3}-\\d{2}-\\d{4}"); // SSN pattern
List<Object> tokens =
redactController.createTokensWithoutTargetText(
textRedactionService.createTokensWithoutTargetText(
realDocument, realPage, targetWords, true, false);
String reconstructedText = extractTextFromTokens(tokens);
@@ -1200,7 +1212,7 @@ class RedactControllerTest {
Set<String> targetWords = Set.of("test");
List<Object> tokens =
redactController.createTokensWithoutTargetText(
textRedactionService.createTokensWithoutTargetText(
realDocument, realPage, targetWords, false, true);
String reconstructedText = extractTextFromTokens(tokens);
@@ -1217,7 +1229,7 @@ class RedactControllerTest {
Set<String> targetWords = Set.of("sensitive");
List<Object> tokens =
redactController.createTokensWithoutTargetText(
textRedactionService.createTokensWithoutTargetText(
realDocument, realPage, targetWords, false, false);
String reconstructedText = extractTextFromTokens(tokens);
@@ -1231,7 +1243,7 @@ class RedactControllerTest {
void shouldWriteTokensToNewContentStream() throws Exception {
List<Object> tokens = createSampleTokenList();
redactController.writeFilteredContentStream(realDocument, realPage, tokens);
textRedactionService.writeFilteredContentStream(realDocument, realPage, tokens);
assertNotNull(realPage.getContents(), "Page should have content stream");
@@ -1249,7 +1261,7 @@ class RedactControllerTest {
assertDoesNotThrow(
() ->
redactController.writeFilteredContentStream(
textRedactionService.writeFilteredContentStream(
realDocument, realPage, emptyTokens));
assertNotNull(realPage.getContents(), "Page should still have content stream");
@@ -1262,7 +1274,7 @@ class RedactControllerTest {
String originalContent = extractTextFromModifiedPage(realPage);
List<Object> newTokens = createSampleTokenList();
redactController.writeFilteredContentStream(realDocument, realPage, newTokens);
textRedactionService.writeFilteredContentStream(realDocument, realPage, newTokens);
String newContent = extractTextFromModifiedPage(realPage);
assertNotEquals(originalContent, newContent, "Content stream should be replaced");
@@ -1273,7 +1285,7 @@ class RedactControllerTest {
void shouldCreateWidthMatchingPlaceholder() {
String originalText = "confidential";
String placeholder =
redactController.createPlaceholderWithFont(
textRedactionService.createPlaceholderWithFont(
originalText, new PDType1Font(Standard14Fonts.FontName.HELVETICA));
assertEquals(
@@ -1287,7 +1299,7 @@ class RedactControllerTest {
void shouldHandleSpecialCharactersInPlaceholder() {
String originalText = "café naïve";
String placeholder =
redactController.createPlaceholderWithFont(
textRedactionService.createPlaceholderWithFont(
originalText, new PDType1Font(Standard14Fonts.FontName.HELVETICA));
assertEquals(originalText.length(), placeholder.length());
@@ -1303,10 +1315,10 @@ class RedactControllerTest {
Set<String> targetWords = Set.of("secret");
List<Object> filteredTokens =
redactController.createTokensWithoutTargetText(
textRedactionService.createTokensWithoutTargetText(
realDocument, realPage, targetWords, false, false);
redactController.writeFilteredContentStream(realDocument, realPage, filteredTokens);
textRedactionService.writeFilteredContentStream(realDocument, realPage, filteredTokens);
assertNotNull(realPage.getContents());
String finalText = extractTextFromModifiedPage(realPage);
@@ -1322,7 +1334,7 @@ class RedactControllerTest {
Set<String> targetWords = Set.of("confidential");
List<Object> filteredTokens =
redactController.createTokensWithoutTargetText(
textRedactionService.createTokensWithoutTargetText(
realDocument, realPage, targetWords, false, false);
long filteredPositioning =
@@ -1377,7 +1389,7 @@ class RedactControllerTest {
Set<String> targetWords = Set.of("confidential");
List<Object> tokens =
redactController.createTokensWithoutTargetText(
textRedactionService.createTokensWithoutTargetText(
realDocument, realPage, targetWords, false, false);
assertNotNull(tokens);
@@ -1404,14 +1416,12 @@ class RedactControllerTest {
@Test
@DisplayName("Should handle documents with multiple text blocks")
void shouldHandleDocumentsWithMultipleTextBlocks() throws Exception {
// Create a document with multiple text blocks
realPage = new PDPage(PDRectangle.A4);
while (realDocument.getNumberOfPages() > 0) {
realDocument.removePage(0);
}
realDocument.addPage(realPage);
// Create resources
PDResources resources = new PDResources();
resources.put(
COSName.getPDFName("F1"), new PDType1Font(Standard14Fonts.FontName.HELVETICA));
@@ -0,0 +1,430 @@
package stirling.software.SPDF.controller.api.security;
import static org.assertj.core.api.Assertions.assertThat;
import java.io.IOException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDPageContentStream;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
import org.apache.pdfbox.pdmodel.font.PDType1Font;
import org.apache.pdfbox.pdmodel.font.Standard14Fonts;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import stirling.software.SPDF.model.PDFText;
import stirling.software.SPDF.pdf.parser.PageColumnLayout;
/**
* Integration tests for {@link RedactExecuteService#collectRangeBlocks(PDDocument, String, String,
* Map)}. Each test builds a synthetic PDF (single-column or two-column) with text-positioning that
* matches what a real document would produce, then asserts that the redaction range produces blocks
* confined to the expected X/Y region.
*/
class RedactExecuteServiceTest {
private static final float PAGE_WIDTH = PDRectangle.LETTER.getWidth(); // 612
private static final float PAGE_HEIGHT = PDRectangle.LETTER.getHeight(); // 792
private static final float LEFT_X = 72f;
private static final float RIGHT_X = 330f;
private static final float COL_WIDTH = 220f;
private static final float LINE_HEIGHT = 14f;
private static final float TOP_Y = PAGE_HEIGHT - 80f;
private static final float FONT_SIZE = 11f;
private final RedactExecuteService service =
new RedactExecuteService(null, null, new TextRedactionService());
@Nested
@DisplayName("Single-column documents")
class SingleColumn {
@Test
void redactBetweenMarkers_inclusive() throws IOException {
try (PDDocument doc = buildSingleColumnDoc()) {
Map<Integer, PageColumnLayout> cache = new HashMap<>();
List<PDFText> blocks =
service.collectRangeBlocks(doc, "START-HERE", "STOP-HERE", cache);
assertThat(blocks)
.as("blocks should be produced for single-column range")
.isNotEmpty();
// Blocks are in screen coords (top-left, Y down). START-HERE is drawn at the top
// of the page; STOP-HERE four lines below. Screen Y grows downward, so the
// anchors' screen-Y tops sit roughly around screenTop(0) and screenTop(4).
// The end anchor is inclusive, so blocks may extend to the bottom of line 4.
float screenTopOfStart = screenTopOfLine(0);
float screenBottomOfEnd = screenTopOfLine(4) + LINE_HEIGHT;
for (PDFText block : blocks) {
assertThat(block.getY1())
.as("block top must be at or below the start anchor's top")
.isGreaterThanOrEqualTo(screenTopOfStart - 1f);
assertThat(block.getY2())
.as(
"block bottom must not extend past the end anchor's bottom (end is inclusive)")
.isLessThanOrEqualTo(screenBottomOfEnd + 1f);
assertThat(block.getX2())
.as("block should not extend into a hypothetical right column")
.isLessThan(PAGE_WIDTH / 2f + 50f);
}
}
}
@Test
void missingStartString_noBlocks() throws IOException {
try (PDDocument doc = buildSingleColumnDoc()) {
Map<Integer, PageColumnLayout> cache = new HashMap<>();
List<PDFText> blocks =
service.collectRangeBlocks(doc, "MISSING-START", "STOP-HERE", cache);
assertThat(blocks).isEmpty();
}
}
@Test
void cvStyleHeadingPlusRightAlignedDate_stillTreatedAsSingleColumn() throws IOException {
// CV-style page: single-column body, but each section heading shares its row with a
// right-aligned date. The X-gap splitter emits the heading and the date as separate
// line boxes; this must NOT trip 2-column detection (the date is too narrow to be a
// real column), otherwise the cross-page redaction predicate over-includes wrong
// regions.
try (PDDocument doc = buildCvStyleDoc()) {
Map<Integer, PageColumnLayout> cache = new HashMap<>();
List<PDFText> blocks =
service.collectRangeBlocks(doc, "SECTION-A", "SECTION-C", cache);
assertThat(blocks)
.as("CV-style redaction between section headings must produce blocks")
.isNotEmpty();
PageColumnLayout layout = cache.get(0);
assertThat(layout.columnCount())
.as("CV-style page with heading+date rows must remain single-column")
.isEqualTo(1);
}
}
@Test
void punctuationDriftInAnchors_stillMatchesViaTolerantFallback() throws IOException {
// Simulates the LLM paraphrasing the heading by inserting a colon that isn't in the
// source ("#3 Character substitution" → "#3: Character substitution"). The
// punctuation-tolerant regex fallback should still find the line.
try (PDDocument doc = buildHeadingPdf()) {
Map<Integer, PageColumnLayout> cache = new HashMap<>();
List<PDFText> blocks =
service.collectRangeBlocks(
doc, "#3: Character substitution", "#6: Image resolution", cache);
assertThat(blocks)
.as("anchor with extra punctuation should still resolve via fallback")
.isNotEmpty();
}
}
}
@Nested
@DisplayName("Two-column documents")
class TwoColumn {
@Test
void rangeInLeftColumn_redactsOnlyLeftColumn() throws IOException {
try (PDDocument doc = buildTwoColumnDoc()) {
Map<Integer, PageColumnLayout> cache = new HashMap<>();
List<PDFText> blocks = service.collectRangeBlocks(doc, "L-START", "L-END", cache);
assertThat(blocks).as("left-only range must produce blocks").isNotEmpty();
float gutterMid = (LEFT_X + COL_WIDTH + RIGHT_X) / 2f;
for (PDFText block : blocks) {
float midX = (block.getX1() + block.getX2()) / 2f;
assertThat(midX)
.as("every block must sit in the left column, never the right")
.isLessThan(gutterMid);
}
}
}
@Test
void rangeInRightColumn_redactsOnlyRightColumn() throws IOException {
try (PDDocument doc = buildTwoColumnDoc()) {
Map<Integer, PageColumnLayout> cache = new HashMap<>();
List<PDFText> blocks = service.collectRangeBlocks(doc, "R-START", "R-END", cache);
assertThat(blocks).as("right-only range must produce blocks").isNotEmpty();
float gutterMid = (LEFT_X + COL_WIDTH + RIGHT_X) / 2f;
for (PDFText block : blocks) {
float midX = (block.getX1() + block.getX2()) / 2f;
assertThat(midX)
.as("every block must sit in the right column, never the left")
.isGreaterThan(gutterMid);
}
}
}
@Test
void twoColumnWithTocAbove_pairsAcrossColumns() throws IOException {
// Reproduces magic.pdf-style stacked layout: a multi-line TOC near the top, then a
// 2-column body where the start anchor is in left col (lower screen Y) and the end
// anchor is in right col (higher screen Y). Original pairing failed here because
// end.y < start.y in screen coords.
try (PDDocument doc = buildTwoColumnWithTocDoc()) {
Map<Integer, PageColumnLayout> cache = new HashMap<>();
List<PDFText> blocks =
service.collectRangeBlocks(doc, "BODY-L-3", "BODY-R-1", cache);
assertThat(blocks)
.as("cross-column body redaction must produce blocks despite stacked TOC")
.isNotEmpty();
}
}
@Test
void crossColumnReadingOrder_leftBottomToRightTop_producesBothSides() throws IOException {
// This is the case the original code couldn't handle at all: end Y < start Y.
try (PDDocument doc = buildTwoColumnDoc()) {
Map<Integer, PageColumnLayout> cache = new HashMap<>();
List<PDFText> blocks =
service.collectRangeBlocks(doc, "L-MIDDLE", "R-MIDDLE", cache);
assertThat(blocks)
.as("cross-column range must produce blocks, not be silently dropped")
.isNotEmpty();
float gutterMid = (LEFT_X + COL_WIDTH + RIGHT_X) / 2f;
boolean sawLeft = false;
boolean sawRight = false;
for (PDFText block : blocks) {
float midX = (block.getX1() + block.getX2()) / 2f;
if (midX < gutterMid) sawLeft = true;
else sawRight = true;
}
assertThat(sawLeft).as("left column should contain at least one block").isTrue();
assertThat(sawRight).as("right column should contain at least one block").isTrue();
}
}
}
// ── document fixtures ────────────────────────────────────────────────────────────────────────
/**
* Single-column page laid out as one column starting at LEFT_X. Lines: 0: START-HERE (start
* anchor) 1: line one 2: line two 3: line three 4: STOP-HERE (end anchor) 5: line five (must
* NOT be redacted)
*/
private PDDocument buildSingleColumnDoc() throws IOException {
PDDocument doc = new PDDocument();
PDPage page = new PDPage(PDRectangle.LETTER);
doc.addPage(page);
try (PDPageContentStream cs = new PDPageContentStream(doc, page)) {
cs.setFont(new PDType1Font(Standard14Fonts.FontName.HELVETICA), FONT_SIZE);
String[] lines = {
"START-HERE", "line one", "line two", "line three", "STOP-HERE", "line five"
};
for (int i = 0; i < lines.length; i++) {
cs.beginText();
cs.newLineAtOffset(LEFT_X, yForLine(i));
cs.showText(lines[i]);
cs.endText();
}
}
return doc;
}
/**
* Two-column page. Lines per column, top to bottom: Left: L-TOP, L-START, L-MIDDLE, L-END,
* L-BOTTOM Right: R-TOP, R-MIDDLE, R-START, R-END, R-BOTTOM
*/
private PDDocument buildTwoColumnDoc() throws IOException {
PDDocument doc = new PDDocument();
PDPage page = new PDPage(PDRectangle.LETTER);
doc.addPage(page);
try (PDPageContentStream cs = new PDPageContentStream(doc, page)) {
cs.setFont(new PDType1Font(Standard14Fonts.FontName.HELVETICA), FONT_SIZE);
// Body lines are padded to make each column genuinely wide enough that column
// detection (which ignores narrow lines) treats both sides as real columns.
String fill = " " + "x".repeat(26);
String[] left = {
"L-TOP" + fill,
"L-START" + fill,
"L-MIDDLE" + fill,
"L-END" + fill,
"L-BOTTOM" + fill
};
String[] right = {
"R-TOP" + fill,
"R-MIDDLE" + fill,
"R-START" + fill,
"R-END" + fill,
"R-BOTTOM" + fill
};
for (int i = 0; i < left.length; i++) {
cs.beginText();
cs.newLineAtOffset(LEFT_X, yForLine(i));
cs.showText(left[i]);
cs.endText();
}
// Aligned baselines per row (IEEE template style) — AllTextLineExtractor must split
// these at the column gap rather than merge same-row left+right glyphs into a wide
// box.
for (int i = 0; i < right.length; i++) {
cs.beginText();
cs.newLineAtOffset(RIGHT_X, yForLine(i));
cs.showText(right[i]);
cs.endText();
}
}
return doc;
}
/**
* Single-column page with feature headings: #1..#7 each followed by body text. The PDF text is
* exactly "#3 Character substitution" (no colon) — the test then queries with a colon to
* exercise the punctuation-tolerant fallback.
*/
private PDDocument buildHeadingPdf() throws IOException {
PDDocument doc = new PDDocument();
PDPage page = new PDPage(PDRectangle.LETTER);
doc.addPage(page);
try (PDPageContentStream cs = new PDPageContentStream(doc, page)) {
cs.setFont(new PDType1Font(Standard14Fonts.FontName.HELVETICA), FONT_SIZE);
String[] lines = {
"#1 Auto layout",
"Body about auto layout.",
"#2 Smart selection",
"Body about smart selection.",
"#3 Character substitution",
"Body about character substitution.",
"#4 Rounded borders",
"Body about rounded borders.",
"#5 Auto contrast",
"Body about auto contrast.",
"#6 Image resolution",
"Body about image resolution.",
"#7 Columns",
"Body about columns."
};
for (int i = 0; i < lines.length; i++) {
cs.beginText();
cs.newLineAtOffset(LEFT_X, yForLine(i));
cs.showText(lines[i]);
cs.endText();
}
}
return doc;
}
/**
* Two-column page like {@code magic.pdf}: a few full-width header lines, a 2-column TOC stacked
* on top of the 2-column body, where TOC's right half lives inside what would otherwise be the
* body's gutter. Body left column has BODY-L-1..3, right column has BODY-R-1..3.
*/
private PDDocument buildTwoColumnWithTocDoc() throws IOException {
PDDocument doc = new PDDocument();
PDPage page = new PDPage(PDRectangle.LETTER);
doc.addPage(page);
try (PDPageContentStream cs = new PDPageContentStream(doc, page)) {
cs.setFont(new PDType1Font(Standard14Fonts.FontName.HELVETICA), FONT_SIZE);
// Header — full width, lines 0..1.
for (int i = 0; i < 2; i++) {
cs.beginText();
cs.newLineAtOffset(LEFT_X, yForLine(i));
cs.showText("FULL WIDTH HEADER LINE " + i + " ACROSS BOTH COLUMNS OF THE PAGE");
cs.endText();
}
// TOC, 2 columns of entries. TOC right half sits where the body gutter would be —
// exactly the layout that broke the histogram-based detector on magic.pdf.
float tocLeftX = 101f;
float tocRightX = 230f;
for (int i = 0; i < 5; i++) {
float y = yForLine(3 + i);
cs.beginText();
cs.newLineAtOffset(tocLeftX, y);
cs.showText("TOC entry left " + i);
cs.endText();
cs.beginText();
cs.newLineAtOffset(tocRightX, y);
cs.showText("TOC entry right " + i);
cs.endText();
}
// Body — 2-column with aligned baselines per row (IEEE-style).
String fill = " " + "x".repeat(26);
String[] bodyLeft = {"BODY-L-1" + fill, "BODY-L-2" + fill, "BODY-L-3" + fill};
String[] bodyRight = {"BODY-R-1" + fill, "BODY-R-2" + fill, "BODY-R-3" + fill};
for (int i = 0; i < bodyLeft.length; i++) {
cs.beginText();
cs.newLineAtOffset(LEFT_X, yForLine(10 + i));
cs.showText(bodyLeft[i]);
cs.endText();
cs.beginText();
cs.newLineAtOffset(RIGHT_X, yForLine(10 + i));
cs.showText(bodyRight[i]);
cs.endText();
}
}
return doc;
}
/**
* CV-style page: single-column body with a few section headings, each followed on the same
* baseline by a right-aligned date string. {@link AllTextLineExtractor} will split each
* heading+date row into two line boxes; column detection must reject this as a fake two-column
* layout because the dates are too narrow to be a real column body.
*/
private PDDocument buildCvStyleDoc() throws IOException {
PDDocument doc = new PDDocument();
PDPage page = new PDPage(PDRectangle.LETTER);
doc.addPage(page);
try (PDPageContentStream cs = new PDPageContentStream(doc, page)) {
cs.setFont(new PDType1Font(Standard14Fonts.FontName.HELVETICA), FONT_SIZE);
float dateX = PAGE_WIDTH - 144f; // right-aligned dates near the right margin
// Section A: heading + date, then 3 body lines.
writeAt(cs, LEFT_X, yForLine(0), "SECTION-A");
writeAt(cs, dateX, yForLine(0), "Jan 2020");
writeAt(cs, LEFT_X, yForLine(1), "Body line A1 with enough width to look like body");
writeAt(cs, LEFT_X, yForLine(2), "Body line A2 with enough width to look like body");
writeAt(cs, LEFT_X, yForLine(3), "Body line A3 with enough width to look like body");
// Section B (in the redact range): heading + date + 3 body lines.
writeAt(cs, LEFT_X, yForLine(5), "SECTION-B");
writeAt(cs, dateX, yForLine(5), "Feb 2021");
writeAt(cs, LEFT_X, yForLine(6), "Body line B1 with enough width to look like body");
writeAt(cs, LEFT_X, yForLine(7), "Body line B2 with enough width to look like body");
writeAt(cs, LEFT_X, yForLine(8), "Body line B3 with enough width to look like body");
// Section C (end anchor): heading + date.
writeAt(cs, LEFT_X, yForLine(10), "SECTION-C");
writeAt(cs, dateX, yForLine(10), "Mar 2022");
}
return doc;
}
private static void writeAt(PDPageContentStream cs, float x, float y, String text)
throws IOException {
cs.beginText();
cs.newLineAtOffset(x, y);
cs.showText(text);
cs.endText();
}
/** PDF user-space Y baseline for line index {@code i} (0-based, top to bottom). */
private static float yForLine(int lineIndex) {
return TOP_Y - lineIndex * LINE_HEIGHT;
}
/** Approximate screen-Y of the top of line {@code i} (top-left origin). */
private static float screenTopOfLine(int lineIndex) {
// baseline_pdf → baseline_screen flips against page height; glyph top ≈ baseline - font
// size.
return PAGE_HEIGHT - yForLine(lineIndex) - FONT_SIZE;
}
}
@@ -18,6 +18,7 @@ import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.context.MessageSource;
import org.springframework.core.env.Environment;
import org.springframework.http.HttpMethod;
import org.springframework.http.HttpStatus;
import org.springframework.http.ProblemDetail;
import org.springframework.http.ResponseEntity;
@@ -27,6 +28,7 @@ import org.springframework.web.bind.MissingServletRequestParameterException;
import org.springframework.web.multipart.MaxUploadSizeExceededException;
import org.springframework.web.multipart.support.MissingServletRequestPartException;
import org.springframework.web.servlet.NoHandlerFoundException;
import org.springframework.web.servlet.resource.NoResourceFoundException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
@@ -208,6 +210,19 @@ class GlobalExceptionHandlerTest {
assertEquals(HttpStatus.NOT_FOUND, resp.getStatusCode());
}
// ---- NoResourceFoundException ----
// Regression guard: was falling through to the 500 catch-all.
@Test
void handleNoResourceFound_returns_404_not_500() {
when(request.getMethod()).thenReturn("GET");
NoResourceFoundException ex =
new NoResourceFoundException(HttpMethod.GET, "/api/v1/storage/folders", "");
ResponseEntity<ProblemDetail> resp = handler.handleNoResourceFound(ex, request);
assertEquals(HttpStatus.NOT_FOUND, resp.getStatusCode());
assertEquals("GET", resp.getBody().getProperties().get("method"));
}
// ---- IllegalArgumentException ----
@Test
@@ -0,0 +1,93 @@
package stirling.software.common.configuration;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.Mockito.mockStatic;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.mockito.MockedStatic;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.YamlHelper;
/**
* End-to-end check of the container-restart path. {@link ConfigInitializer#ensureConfigExists()} is
* what runs on every startup, merging the on-disk settings.yml with the bundled
* settings.yml.template. These tests exercise it against the real template on the classpath to
* prove admin-saved proFeatures values survive a restart - the bug behind "the SSO auto-login
* button resets every time the container resets".
*/
class ConfigInitializerRestartTest {
private static String read(Path settings, String... keyPath) throws IOException {
return String.valueOf(new YamlHelper(settings).getValueByExactKeyPath(keyPath));
}
@Test
void ssoAutoLoginAndCustomMetadata_persistAcrossRestart(@TempDir Path tmp) throws Exception {
Path settings = tmp.resolve("settings.yml");
Path custom = tmp.resolve("custom_settings.yml");
try (MockedStatic<InstallationPathConfig> paths =
mockStatic(InstallationPathConfig.class)) {
paths.when(InstallationPathConfig::getSettingsPath).thenReturn(settings.toString());
paths.when(InstallationPathConfig::getCustomSettingsPath).thenReturn(custom.toString());
ConfigInitializer init = new ConfigInitializer();
// First boot: settings.yml created from the bundled template (camelCase, default off).
init.ensureConfigExists();
assertEquals("false", read(settings, "premium", "proFeatures", "ssoAutoLogin"));
// Admin enables SSO auto-login and edits custom metadata via the exact save path the
// admin settings controller uses.
GeneralUtils.saveKeyToSettings("premium.proFeatures.ssoAutoLogin", true);
GeneralUtils.saveKeyToSettings("premium.proFeatures.customMetadata.author", "acme");
// Container restart: ensureConfigExists merges the saved file with the template again.
init.ensureConfigExists();
assertEquals("true", read(settings, "premium", "proFeatures", "ssoAutoLogin"));
assertEquals(
"acme", read(settings, "premium", "proFeatures", "customMetadata", "author"));
}
}
@Test
void legacyPascalCaseConfig_isMigratedAndPreservedOnRestart(@TempDir Path tmp)
throws Exception {
Path settings = tmp.resolve("settings.yml");
Path custom = tmp.resolve("custom_settings.yml");
try (MockedStatic<InstallationPathConfig> paths =
mockStatic(InstallationPathConfig.class)) {
paths.when(InstallationPathConfig::getSettingsPath).thenReturn(settings.toString());
paths.when(InstallationPathConfig::getCustomSettingsPath).thenReturn(custom.toString());
ConfigInitializer init = new ConfigInitializer();
// Seed a full settings.yml as an OLD install would have written it: PascalCase keys
// with
// SSO auto-login enabled.
init.ensureConfigExists();
String legacy =
Files.readString(settings)
.replace("ssoAutoLogin: false", "SSOAutoLogin: true")
.replace("customMetadata:", "CustomMetadata:");
Files.writeString(settings, legacy);
// Upgrade restart.
init.ensureConfigExists();
// Value carried forward onto the new camelCase key; the legacy PascalCase key is gone.
assertEquals("true", read(settings, "premium", "proFeatures", "ssoAutoLogin"));
assertNull(
new YamlHelper(settings)
.getValueByExactKeyPath("premium", "proFeatures", "SSOAutoLogin"));
}
}
}
@@ -0,0 +1,479 @@
package stirling.software.common.controller;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertInstanceOf;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.Mockito.lenient;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import java.time.Instant;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.stream.Stream;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.test.util.ReflectionTestUtils;
import jakarta.servlet.http.HttpServletRequest;
import stirling.software.common.cluster.ClusterBackplane;
import stirling.software.common.cluster.JobStore;
import stirling.software.common.cluster.JobStoreEntry;
import stirling.software.common.cluster.StickyMissRecorder;
import stirling.software.common.model.job.JobResult;
import stirling.software.common.service.FileStorage;
import stirling.software.common.service.JobOwnershipService;
import stirling.software.common.service.JobQueue;
import stirling.software.common.service.TaskManager;
/**
* Sticky-410 ownership contract for {@link JobController}: peer-owned jobs return 410 Gone with
* ownedBy/currentNode fields; locally-owned and no-entry cases return 200. FileStorage is never
* touched on the 410 path. Manual mock construction so tests can vary backplane/jobstore combos.
*/
class JobControllerOwnershipTest {
private TaskManager taskManager;
private FileStorage fileStorage;
private JobQueue jobQueue;
private HttpServletRequest request;
private JobOwnershipService jobOwnershipService;
private ClusterBackplane clusterBackplane;
private JobStore jobStore;
private StickyMissRecorder stickyMissRecorder;
private static final String JOB_ID = "job-42";
private static final String FILE_ID = "file-abc";
private static final String LOCAL_NODE = "node-self";
private static final String PEER_NODE = "node-peer";
@BeforeEach
void setUp() {
taskManager = mock(TaskManager.class);
fileStorage = mock(FileStorage.class);
jobQueue = mock(JobQueue.class);
request = mock(HttpServletRequest.class);
jobOwnershipService = mock(JobOwnershipService.class);
clusterBackplane = mock(ClusterBackplane.class);
jobStore = mock(JobStore.class);
stickyMissRecorder = mock(StickyMissRecorder.class);
}
private JobController makeController(ClusterBackplane backplane, JobStore store) {
JobController c =
new JobController(taskManager, fileStorage, jobQueue, request, backplane, store);
ReflectionTestUtils.setField(c, "stickyMissRecorder", stickyMissRecorder);
return c;
}
private JobController makeController() {
return makeController(clusterBackplane, jobStore);
}
private JobStoreEntry entryOwnedBy(String ownerNodeId) {
return new JobStoreEntry(
JOB_ID,
JobStoreEntry.JobState.COMPLETE,
ownerNodeId,
Instant.now(),
Instant.now(),
null,
List.of(FILE_ID),
Map.of());
}
private JobResult completedJobWithFile() {
JobResult result = new JobResult();
result.setJobId(JOB_ID);
// completeWithSingleFile populates the resultFiles list, sets complete=true,
// and sets completedAt - all required for the getJobResult single-file branch.
result.completeWithSingleFile(FILE_ID, "out.pdf", "application/pdf", 7L);
return result;
}
@Test
@DisplayName(
"downloadFile peer-owned → full sticky-410 contract"
+ " (status + Retry-After + payload + metric + storage untouched)")
void downloadFile_peerOwned_fullStickyContract() throws Exception {
when(clusterBackplane.localNodeId()).thenReturn(LOCAL_NODE);
when(taskManager.findJobKeyByFileId(FILE_ID)).thenReturn(JOB_ID);
when(jobStore.get(JOB_ID)).thenReturn(Optional.of(entryOwnedBy(PEER_NODE)));
ResponseEntity<?> response = makeController().downloadFile(FILE_ID);
assertEquals(HttpStatus.GONE, response.getStatusCode());
assertEquals("0", response.getHeaders().getFirst("Retry-After"));
assertInstanceOf(Map.class, response.getBody());
Map<?, ?> body = (Map<?, ?>) response.getBody();
assertEquals(3, body.size(), "exactly: message, ownedBy, currentNode");
assertEquals(PEER_NODE, body.get("ownedBy"));
assertEquals(LOCAL_NODE, body.get("currentNode"));
assertNotNull(body.get("message"));
assertTrue(((String) body.get("message")).toLowerCase().contains("retry"));
assertNull(body.get("internalSecret"));
assertNull(body.get("filePath"));
verify(stickyMissRecorder).recordStickyMiss();
verify(fileStorage, never()).retrieveBytes(FILE_ID);
}
private static Stream<Arguments> downloadHappyPathScenarios() {
return Stream.of(
Arguments.of("locallyOwned", LOCAL_NODE, true),
Arguments.of("noJobStoreEntry", null, false),
Arguments.of("blankOwningNodeId", "", true));
}
@ParameterizedTest(name = "downloadFile {0} -> 200, no sticky-miss")
@MethodSource("downloadHappyPathScenarios")
void downloadFile_happyPath_returnsOkAndNoMetric(
String scenario, String ownerNodeId, boolean entryPresent) throws Exception {
when(clusterBackplane.localNodeId()).thenReturn(LOCAL_NODE);
when(taskManager.findJobKeyByFileId(FILE_ID)).thenReturn(JOB_ID);
when(jobStore.get(JOB_ID))
.thenReturn(
entryPresent ? Optional.of(entryOwnedBy(ownerNodeId)) : Optional.empty());
when(fileStorage.retrieveBytes(FILE_ID)).thenReturn("payload".getBytes());
ResponseEntity<?> response = makeController().downloadFile(FILE_ID);
assertEquals(HttpStatus.OK, response.getStatusCode(), scenario);
verify(fileStorage).retrieveBytes(FILE_ID);
verify(stickyMissRecorder, never()).recordStickyMiss();
}
@Test
@DisplayName("getJobResult: locally-owned single-file result → reads from FileStorage, 200 OK")
void getJobResult_singleFile_locallyOwned_readsFromStorage() throws Exception {
when(clusterBackplane.localNodeId()).thenReturn(LOCAL_NODE);
when(taskManager.getJobResult(JOB_ID)).thenReturn(completedJobWithFile());
when(jobStore.get(JOB_ID)).thenReturn(Optional.of(entryOwnedBy(LOCAL_NODE)));
when(fileStorage.retrieveBytes(FILE_ID)).thenReturn("payload".getBytes());
ResponseEntity<?> response = makeController().getJobResult(JOB_ID);
assertEquals(HttpStatus.OK, response.getStatusCode());
}
private enum Endpoint {
DOWNLOAD_FILE,
GET_JOB_RESULT,
GET_JOB_STATUS,
GET_JOB_FILES,
GET_FILE_METADATA,
CANCEL_JOB
}
private static Stream<Arguments> peerOwned410Scenarios() {
return Stream.of(
Arguments.of(Endpoint.DOWNLOAD_FILE),
Arguments.of(Endpoint.GET_JOB_RESULT),
Arguments.of(Endpoint.GET_JOB_STATUS),
Arguments.of(Endpoint.GET_JOB_FILES),
Arguments.of(Endpoint.GET_FILE_METADATA),
Arguments.of(Endpoint.CANCEL_JOB));
}
@ParameterizedTest(name = "{0} peer-owned -> 410, ownedBy=peer, metric++")
@MethodSource("peerOwned410Scenarios")
void endpoint_peerOwned_returns410(Endpoint endpoint) throws Exception {
when(clusterBackplane.localNodeId()).thenReturn(LOCAL_NODE);
when(jobStore.get(JOB_ID)).thenReturn(Optional.of(entryOwnedBy(PEER_NODE)));
switch (endpoint) {
case DOWNLOAD_FILE, GET_FILE_METADATA ->
when(taskManager.findJobKeyByFileId(FILE_ID)).thenReturn(JOB_ID);
case GET_JOB_RESULT ->
when(taskManager.getJobResult(JOB_ID)).thenReturn(completedJobWithFile());
case GET_JOB_STATUS, GET_JOB_FILES ->
when(taskManager.getJobResult(JOB_ID)).thenReturn(null);
case CANCEL_JOB -> {
when(jobQueue.isJobQueued(JOB_ID)).thenReturn(false);
when(taskManager.getJobResult(JOB_ID)).thenReturn(null);
}
}
ResponseEntity<?> response =
switch (endpoint) {
case DOWNLOAD_FILE -> makeController().downloadFile(FILE_ID);
case GET_JOB_RESULT -> makeController().getJobResult(JOB_ID);
case GET_JOB_STATUS -> makeController().getJobStatus(JOB_ID);
case GET_JOB_FILES -> makeController().getJobFiles(JOB_ID);
case GET_FILE_METADATA -> makeController().getFileMetadata(FILE_ID);
case CANCEL_JOB -> makeController().cancelJob(JOB_ID);
};
assertEquals(HttpStatus.GONE, response.getStatusCode());
Map<?, ?> body = (Map<?, ?>) response.getBody();
assertEquals(PEER_NODE, body.get("ownedBy"));
assertEquals(LOCAL_NODE, body.get("currentNode"));
verify(stickyMissRecorder).recordStickyMiss();
verify(fileStorage, never()).retrieveBytes(FILE_ID);
if (endpoint == Endpoint.CANCEL_JOB) {
verify(taskManager, never()).setError(JOB_ID, "Job was cancelled by user");
}
}
private static Stream<Arguments> unknownJob404Scenarios() {
return Stream.of(Arguments.of(Endpoint.GET_JOB_STATUS), Arguments.of(Endpoint.CANCEL_JOB));
}
@ParameterizedTest(name = "{0} unknown jobId -> 404 (not 410), no metric")
@MethodSource("unknownJob404Scenarios")
void endpoint_unknownJob_returns404(Endpoint endpoint) {
when(taskManager.getJobResult(JOB_ID)).thenReturn(null);
when(jobStore.get(JOB_ID)).thenReturn(Optional.empty());
if (endpoint == Endpoint.CANCEL_JOB) {
when(jobQueue.isJobQueued(JOB_ID)).thenReturn(false);
}
ResponseEntity<?> response =
switch (endpoint) {
case GET_JOB_STATUS -> makeController().getJobStatus(JOB_ID);
case CANCEL_JOB -> makeController().cancelJob(JOB_ID);
default -> throw new IllegalArgumentException(endpoint.name());
};
assertEquals(HttpStatus.NOT_FOUND, response.getStatusCode());
verify(stickyMissRecorder, never()).recordStickyMiss();
}
@Test
@DisplayName("Single-instance install (no ClusterBackplane bean): no 410, no NPE")
void singleInstance_noClusterBackplane_noGoneResponse() throws Exception {
when(taskManager.findJobKeyByFileId(FILE_ID)).thenReturn(JOB_ID);
when(fileStorage.retrieveBytes(FILE_ID)).thenReturn("payload".getBytes());
ResponseEntity<?> response = makeController(null, jobStore).downloadFile(FILE_ID);
assertEquals(HttpStatus.OK, response.getStatusCode());
verify(fileStorage).retrieveBytes(FILE_ID);
}
@Test
@DisplayName("Single-instance install (no JobStore bean): no 410, no NPE")
void singleInstance_noJobStore_noGoneResponse() throws Exception {
when(taskManager.findJobKeyByFileId(FILE_ID)).thenReturn(JOB_ID);
when(fileStorage.retrieveBytes(FILE_ID)).thenReturn("payload".getBytes());
ResponseEntity<?> response = makeController(clusterBackplane, null).downloadFile(FILE_ID);
assertEquals(HttpStatus.OK, response.getStatusCode());
verify(fileStorage).retrieveBytes(FILE_ID);
}
@Test
@DisplayName("Single-instance (no StickyMissRecorder bean) → no NPE, still 200 OK")
void noStickyMissRecorder_works() throws Exception {
when(clusterBackplane.localNodeId()).thenReturn(LOCAL_NODE);
when(taskManager.findJobKeyByFileId(FILE_ID)).thenReturn(JOB_ID);
when(jobStore.get(JOB_ID)).thenReturn(Optional.of(entryOwnedBy(LOCAL_NODE)));
when(fileStorage.retrieveBytes(FILE_ID)).thenReturn("payload".getBytes());
JobController c = makeController();
ReflectionTestUtils.setField(c, "stickyMissRecorder", null);
ResponseEntity<?> response = c.downloadFile(FILE_ID);
assertEquals(HttpStatus.OK, response.getStatusCode());
}
@Test
@DisplayName(
"cluster-mode but localNodeId is null → no NPE; 410 because owner is set and"
+ " differs from blank")
void clusterBackplanePresent_butLocalNodeIdNull_falsBackGracefully() throws Exception {
when(clusterBackplane.localNodeId()).thenReturn(null);
when(taskManager.findJobKeyByFileId(FILE_ID)).thenReturn(JOB_ID);
when(jobStore.get(JOB_ID)).thenReturn(Optional.of(entryOwnedBy(PEER_NODE)));
// We still 410: owner is "node-peer", local is null → they don't match. Rather than
// silently 200-from-wrong-disk (which would serve garbage), we surface the mismatch.
ResponseEntity<?> response = makeController().downloadFile(FILE_ID);
assertEquals(HttpStatus.GONE, response.getStatusCode());
Map<?, ?> body = (Map<?, ?>) response.getBody();
assertEquals("", body.get("currentNode"), "blank when localNodeId is null");
assertEquals(PEER_NODE, body.get("ownedBy"));
}
@Test
@DisplayName("Owner returns 410 even when JobOwnershipService allows access (orthogonal)")
void ownershipService_passes_butStickyStillReturns410() throws Exception {
when(clusterBackplane.localNodeId()).thenReturn(LOCAL_NODE);
when(taskManager.findJobKeyByFileId(FILE_ID)).thenReturn(JOB_ID);
when(jobStore.get(JOB_ID)).thenReturn(Optional.of(entryOwnedBy(PEER_NODE)));
lenient().when(jobOwnershipService.validateJobAccess(JOB_ID)).thenReturn(true);
JobController c = makeController();
ReflectionTestUtils.setField(c, "jobOwnershipService", jobOwnershipService);
ResponseEntity<?> response = c.downloadFile(FILE_ID);
assertEquals(HttpStatus.GONE, response.getStatusCode());
}
@Test
@DisplayName(
"downloadFile: peer-owned + ownership-denied → 410 (NOT 403) so we don't leak"
+ " file existence")
void downloadFile_peerOwned_ownershipDenied_returns410NotForbidden() throws Exception {
when(clusterBackplane.localNodeId()).thenReturn(LOCAL_NODE);
when(taskManager.findJobKeyByFileId(FILE_ID)).thenReturn(JOB_ID);
when(jobStore.get(JOB_ID)).thenReturn(Optional.of(entryOwnedBy(PEER_NODE)));
lenient().when(jobOwnershipService.validateJobAccess(JOB_ID)).thenReturn(false);
JobController c = makeController();
ReflectionTestUtils.setField(c, "jobOwnershipService", jobOwnershipService);
ResponseEntity<?> response = c.downloadFile(FILE_ID);
assertEquals(HttpStatus.GONE, response.getStatusCode());
Map<?, ?> body = (Map<?, ?>) response.getBody();
assertEquals(PEER_NODE, body.get("ownedBy"));
verify(fileStorage, never()).retrieveBytes(FILE_ID);
}
@Test
@DisplayName(
"getJobStatus: peer-owned + ownership-denied → 410 (NOT 403) so we don't leak"
+ " job existence")
void getJobStatus_peerOwned_ownershipDenied_returns410NotForbidden() {
when(clusterBackplane.localNodeId()).thenReturn(LOCAL_NODE);
when(jobStore.get(JOB_ID)).thenReturn(Optional.of(entryOwnedBy(PEER_NODE)));
lenient().when(jobOwnershipService.validateJobAccess(JOB_ID)).thenReturn(false);
JobController c = makeController();
ReflectionTestUtils.setField(c, "jobOwnershipService", jobOwnershipService);
ResponseEntity<?> response = c.getJobStatus(JOB_ID);
assertEquals(HttpStatus.GONE, response.getStatusCode());
Map<?, ?> body = (Map<?, ?>) response.getBody();
assertEquals(PEER_NODE, body.get("ownedBy"));
}
@Test
@DisplayName(
"cancelJob: peer-owned + ownership-denied → 410 (NOT 403) so we don't leak job"
+ " existence")
void cancelJob_peerOwned_ownershipDenied_returns410NotForbidden() {
when(clusterBackplane.localNodeId()).thenReturn(LOCAL_NODE);
when(jobQueue.isJobQueued(JOB_ID)).thenReturn(false);
when(jobStore.get(JOB_ID)).thenReturn(Optional.of(entryOwnedBy(PEER_NODE)));
lenient().when(jobOwnershipService.validateJobAccess(JOB_ID)).thenReturn(false);
JobController c = makeController();
ReflectionTestUtils.setField(c, "jobOwnershipService", jobOwnershipService);
ResponseEntity<?> response = c.cancelJob(JOB_ID);
assertEquals(HttpStatus.GONE, response.getStatusCode());
Map<?, ?> body = (Map<?, ?>) response.getBody();
assertEquals(PEER_NODE, body.get("ownedBy"));
verify(taskManager, never()).setError(JOB_ID, "Job was cancelled by user");
}
@Test
@DisplayName(
"guardNonOwner caches JobStore.get within TTL window: second call same jobId hits"
+ " cache, not Valkey")
void guardNonOwner_cachesJobStoreLookupWithinTtl() throws Exception {
when(clusterBackplane.localNodeId()).thenReturn(LOCAL_NODE);
when(taskManager.findJobKeyByFileId(FILE_ID)).thenReturn(JOB_ID);
when(jobStore.get(JOB_ID)).thenReturn(Optional.of(entryOwnedBy(LOCAL_NODE)));
when(fileStorage.retrieveBytes(FILE_ID)).thenReturn("payload".getBytes());
JobController c = makeController();
c.downloadFile(FILE_ID);
c.downloadFile(FILE_ID);
c.downloadFile(FILE_ID);
verify(jobStore, times(1)).get(JOB_ID);
}
@Test
@DisplayName(
"guardNonOwner: JobStore.get throws (Valkey timeout) → falls through to local-disk"
+ " path, no 500 leaks to caller")
void guardNonOwner_jobStoreException_fallsThroughToLocalPath() throws Exception {
when(taskManager.findJobKeyByFileId(FILE_ID)).thenReturn(JOB_ID);
when(jobStore.get(JOB_ID)).thenThrow(new RuntimeException("Valkey command timeout"));
when(taskManager.getJobResult(JOB_ID)).thenReturn(completedJobWithFile());
when(fileStorage.retrieveBytes(FILE_ID)).thenReturn("payload".getBytes());
ResponseEntity<?> response = makeController().downloadFile(FILE_ID);
assertEquals(HttpStatus.OK, response.getStatusCode());
verify(fileStorage).retrieveBytes(FILE_ID);
verify(stickyMissRecorder, never()).recordStickyMiss();
}
@Test
@DisplayName("backplane down + job NOT held locally → 503 retryable (not a misleading 404)")
void jobEndpoint_backplaneDown_notLocal_returns503() {
when(jobStore.get(JOB_ID)).thenThrow(new RuntimeException("Valkey command timeout"));
when(taskManager.getJobResult(JOB_ID)).thenReturn(null);
ResponseEntity<?> response = makeController().getJobStatus(JOB_ID);
assertEquals(HttpStatus.SERVICE_UNAVAILABLE, response.getStatusCode());
assertEquals("1", response.getHeaders().getFirst("Retry-After"));
verify(stickyMissRecorder, never()).recordStickyMiss();
}
@Test
@DisplayName("backplane down but job held locally → owner still serves (not 503)")
void jobEndpoint_backplaneDown_local_servesLocally() {
when(jobStore.get(JOB_ID)).thenThrow(new RuntimeException("Valkey command timeout"));
when(taskManager.getJobResult(JOB_ID)).thenReturn(completedJobWithFile());
ResponseEntity<?> response = makeController().getJobStatus(JOB_ID);
assertEquals(HttpStatus.OK, response.getStatusCode());
}
@Test
@DisplayName(
"downloadFile: findJobKeyByFileId throws (backplane down) → 503 + Retry-After,"
+ " not 404/500, storage untouched")
void downloadFile_findJobKeyThrows_returns503Retryable() throws Exception {
when(taskManager.findJobKeyByFileId(FILE_ID))
.thenThrow(new RuntimeException("Valkey command timeout"));
ResponseEntity<?> response = makeController().downloadFile(FILE_ID);
assertEquals(HttpStatus.SERVICE_UNAVAILABLE, response.getStatusCode());
assertEquals("1", response.getHeaders().getFirst("Retry-After"));
Map<?, ?> body = (Map<?, ?>) response.getBody();
assertTrue(((String) body.get("message")).toLowerCase().contains("unavailable"));
verify(fileStorage, never()).retrieveBytes(FILE_ID);
}
@Test
@DisplayName(
"getFileMetadata: findJobKeyByFileId throws (backplane down) → 503 + Retry-After,"
+ " not 404/500")
void getFileMetadata_findJobKeyThrows_returns503Retryable() throws Exception {
when(taskManager.findJobKeyByFileId(FILE_ID))
.thenThrow(new RuntimeException("Valkey command timeout"));
ResponseEntity<?> response = makeController().getFileMetadata(FILE_ID);
assertEquals(HttpStatus.SERVICE_UNAVAILABLE, response.getStatusCode());
assertEquals("1", response.getHeaders().getFirst("Retry-After"));
Map<?, ?> body = (Map<?, ?>) response.getBody();
assertTrue(((String) body.get("message")).toLowerCase().contains("unavailable"));
verify(fileStorage, never()).retrieveBytes(FILE_ID);
}
}
@@ -19,6 +19,8 @@ import org.springframework.test.util.ReflectionTestUtils;
import jakarta.servlet.http.HttpServletRequest;
import stirling.software.common.cluster.ClusterBackplane;
import stirling.software.common.cluster.JobStore;
import stirling.software.common.model.job.JobResult;
import stirling.software.common.service.FileStorage;
import stirling.software.common.service.JobOwnershipService;
@@ -37,6 +39,10 @@ class JobControllerTest {
@Mock private JobOwnershipService jobOwnershipService;
@Mock private ClusterBackplane clusterBackplane;
@Mock private JobStore jobStore;
private MockHttpSession session;
@InjectMocks private JobController controller;
+2
View File
@@ -123,6 +123,8 @@ SwaggerDoc.json
*.tar.gz
*.rar
*.db
# Whitelist the H2 fixtures that feed the version-migration CI smoke test.
!src/test/resources/db-migration-fixtures/*.mv.db
/build
/app/proprietary/build/
+18 -1
View File
@@ -5,6 +5,8 @@ repositories {
ext {
jwtVersion = '0.13.0'
awsSdkVersion = '2.44.12'
testcontainersMinioVersion = '1.21.4'
}
bootRun {
@@ -53,8 +55,13 @@ dependencies {
api 'org.springframework.boot:spring-boot-starter-mail'
api 'org.springframework.boot:spring-boot-starter-cache'
api 'com.github.ben-manes.caffeine:caffeine'
implementation 'org.springframework.boot:spring-boot-starter-data-redis'
api 'io.swagger.core.v3:swagger-core-jakarta:2.2.46'
implementation 'com.bucket4j:bucket4j_jdk17-core:8.18.0'
implementation 'com.bucket4j:bucket4j_jdk17-core:8.19.0'
// Lettuce-backed Bucket4j ProxyManager used by ValkeyRateLimitStore for cluster-wide
// token-bucket rate limiting (parity with in-process Bucket4j semantics; no fixed-window
// boundary doubling).
implementation 'com.bucket4j:bucket4j_jdk17-lettuce:8.19.0'
// https://mvnrepository.com/artifact/com.bucket4j/bucket4j_jdk17
implementation "org.bouncycastle:bcprov-jdk18on:$bouncycastleVersion"
@@ -71,6 +78,16 @@ dependencies {
implementation('com.coveo:saml-client:5.0.0') {
exclude group: 'org.opensaml', module: 'opensaml-core'
}
implementation "software.amazon.awssdk:s3:$awsSdkVersion"
implementation "software.amazon.awssdk:url-connection-client:$awsSdkVersion"
// Testcontainers: real MinIO/LocalStack (S3) and Valkey for integration tests in CI without
// manually-started instances. Tests skip cleanly when Docker is unavailable.
testImplementation "org.testcontainers:testcontainers:$testcontainersMinioVersion"
testImplementation "org.testcontainers:minio:$testcontainersMinioVersion"
testImplementation "org.testcontainers:localstack:$testcontainersMinioVersion"
testImplementation "org.testcontainers:junit-jupiter:$testcontainersMinioVersion"
}
tasks.register('prepareKotlinBuildScriptModel') {}
@@ -0,0 +1,40 @@
package stirling.software.proprietary.cluster;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Configuration;
import jakarta.annotation.PostConstruct;
import lombok.extern.slf4j.Slf4j;
/**
* Runtime license gate for cluster mode. Cluster mode requires a SERVER or ENTERPRISE license; the
* SaaS flavor bypasses (no {@code runningProOrHigher} bean is published). The Valkey connection
* config {@code @DependsOn} this bean, so it runs before any Valkey bean is constructed.
*/
@Configuration
@ConditionalOnProperty(name = "cluster.enabled", havingValue = "true")
@Slf4j
public class ClusterLicenseGate {
@Autowired(required = false)
@Qualifier("runningProOrHigher")
private Boolean runningProOrHigher;
@PostConstruct
void verifyLicense() {
if (runningProOrHigher == null) {
return; // saas flavor - licensed via Stripe elsewhere
}
if (!runningProOrHigher) {
throw new IllegalStateException(
"Cluster mode (cluster.enabled=true) requires a SERVER or"
+ " ENTERPRISE license. Configure stirling.premium.key with a valid"
+ " license key (contact sales@stirlingpdf.com to obtain one), or set"
+ " cluster.enabled=false.");
}
log.info("Cluster license gate: SERVER/ENTERPRISE license verified, cluster mode allowed.");
}
}

Some files were not shown because too many files have changed in this diff Show More