Compare commits

...
Author SHA1 Message Date
EthanHealy01 fc9960b6e6 fast path added 2026-06-02 01:20:35 +01:00
EthanHealy01 f8ce30ff37 Add AGPL-3.0 license text and fix formatting
- Add licenses/AGPL-3.0.txt (full license text for scripts/pymupdf_convert.py)
- Update root LICENSE to reference it
- Fix Spotless formatting in PyMuPdfConverter and ExternalAppDepConfig
2026-06-02 00:58:39 +01:00
EthanHealy01 f0a7f9af78 Collapse pymupdf-worker package into a single script
The pymupdf-worker/ directory existed only to register a pip entry point
for pymupdf-convert. Replace it with scripts/pymupdf_convert.py — a
standalone script installed directly onto PATH in the Dockerfile.

- Add scripts/pymupdf_convert.py (AGPL-3.0, same logic, shebang added)
- Update docker/base/Dockerfile to copy + install the script directly
  instead of pip-installing the package; remove now-redundant symlink
- Delete pymupdf-worker/ and its taskfile
- Update root LICENSE to reference the script file instead of directory
2026-06-02 00:48:55 +01:00
EthanHealy01 6309ca7234 Clean up pymupdf-convert integration per review feedback
- Remove AGPL/license comments from main codebase (Dockerfile, Java
  classes, javadocs) — legal isolation is structural, comments were noise
- PyMuPdfConverter: check CLI availability once at @PostConstruct instead
  of caching with a TTL; simpler and consistent with how LibreOffice/
  Tesseract availability is handled
2026-06-02 00:44:41 +01:00
EthanHealy01 f3f17d6381 fix: non-root user in Dockerfile, update stale HTTP references in worker 2026-06-01 19:45:35 +01:00
EthanHealy01 4487f23ff7 feat(pdf-to-markdown): add pymupdf-convert CLI for faster PDF→Markdown conversion
Introduces a new  directory containing a small standalone
CLI tool () that converts PDFs to Markdown using PyMuPDF
and pymupdf4llm. When installed, it replaces the slower pdftohtml-based
converter with a significantly faster path.
The tool is invoked as an OS subprocess, following the same pattern already
used for LibreOffice and Tesseract. A new  service handles
discovery (PATH probe, cached 30 s), temp file I/O, and process execution via
the existing  infrastructure. Auto-detected at startup via
 — no configuration flag required. Falls back
transparently to the bundled pdftohtml converter if not found.
 is baked into the Docker base image via the existing
Python venv build stage and symlinked onto PATH, so all Docker users get
the faster path automatically with no manual steps.
AGPL notice:  is licensed under AGPL-3.0-or-later because
it depends on PyMuPDF (AGPL-3.0 / Artifex commercial). It is a SEPARATE
program from the rest of Stirling PDF and shares no source code with the
MIT-licensed core or the proprietary engine. The subprocess boundary
ensures the AGPL copyleft attaches to this tool alone and does not extend
to Stirling PDF. The LICENSE file and pymupdf-worker/LICENSE document this
separation explicitly. Do not replace the subprocess invocation with an
in-process import of PyMuPDF.
2026-06-01 19:42:49 +01:00
brios 2c0ebc28a7 perf(api): optimize static asset caching, enable ETag support, and expand response compression mime types. (#6273) 2026-06-01 16:37:56 +01:00
Ludy d1486c7762 ci(github-actions): replace deprecated app-id input with client-id (#6485)
# Description of Changes

This change updates the GitHub App token generation step in the custom
`setup-bot` GitHub Action to use the new `client-id` input instead of
the deprecated `app-id` input when invoking
`actions/create-github-app-token`.

### What was changed

- Replaced the deprecated `app-id` parameter with `client-id` in
`.github/actions/setup-bot/action.yml`.
- Continued sourcing the value from the existing `inputs.app-id` action
input to avoid broader interface changes.

### Why the change was made

- `actions/create-github-app-token` has deprecated the `app-id` input
and now expects `client-id`.
- Updating the workflow removes the deprecation warning and ensures
compatibility with current and future versions of the action.


---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-05-31 08:07:30 +01:00
Ludy 2ccff6f73f fix(update-service): correct GitHub branch reference for version retrieval (#6333) 2026-05-30 22:44:42 +01:00
Ludy 78da227eba fix: Use frontend/editor for locales paths (#6483) 2026-05-30 22:00:17 +01:00
Anthony Stirling 30e782e29c Disable Save-to-server when storage off, fix QR port 0 (#6473) 2026-05-29 19:37:42 +01:00
Anthony Stirling 2b0905887b Add desktop multi-window support (#6463) 2026-05-29 19:35:54 +01:00
ConnorYoh 28b81828b5 PAYG: PricingPolicyService + admin REST + 30s read cache (#6469)
## What this is

PR-I1 service half from `notes/PAYG_DESIGN.md`. Built on top of the data
model from #6460 — answers "what pricing policy applies to this team
right now?" with a fast cache and an admin write surface.

## Scope

| Piece | Where |
|---|---|
| `PricingPolicyService` — `getEffectivePolicy(teamId)` with 30s
Caffeine cache + admin write paths |
`app/saas/.../payg/policy/PricingPolicyService.java` |
| `PolicyChangedEvent` — published after admin writes for in-process
cache invalidation | `app/saas/.../payg/policy/PolicyChangedEvent.java`
|
| Admin REST — list / get / create / set-default / set team override /
get effective |
`app/saas/.../payg/policy/admin/PricingPolicyAdminController.java` +
DTOs |
| `PricingPolicyRepository.clearDefaultFlag()` — atomic clear for
set-default | repository update |
| `SaasJpaConfigScanTest` — drift guard against the JPA scan paths going
stale (carried over from the #6460 review concern) | new test |
| V12 default-policy seed (`v1-initial`, 25 pages/unit, 5 MiB/unit,
per-`JobSource` step limits) | `V12__seed_default_payg_policy.sql` |

## Lookup precedence

1. `PaygTeamExtensions.pricingPolicyId` set → return that policy
2. Else return the `pricing_policy` row with `is_default = TRUE`
3. Override row points at a deleted policy → log warn, fall back to
default (safety net for racing deletes)
4. No default → `IllegalStateException` (V12 seed guarantees one exists)

## Cache behaviour

- 30s `expireAfterWrite` Caffeine, max 10k entries, keyed by `teamId`.
- **Single correctness model: the TTL.** Cross-instance propagation is
at-most-30-seconds. The writer instance sees its own change immediately
via the `PolicyChangedEvent` after-commit publish. Other instances pick
it up on the next TTL expiry.
- Admin reads use `getEffectivePolicyUncached` so admins always see
their own write straight back.

**Why no LISTEN/NOTIFY runner.** An earlier cut of this PR included a
Postgres `LISTEN policy_changed` runner so cross-instance propagation
was instant. Dropped — admin policy changes are events-per-week and the
30s TTL is already the correctness floor; the listener was ~250 lines of
nontrivial code (raw JDBC outside HikariCP, daemon thread, reconnect
loop, lock-protected connection lifecycle) for a use case that isn't on
the hot path. Trade-off is documented in `notes/PAYG_DESIGN.md` §9 with
three concrete triggers that would justify reintroducing it (aggressive
cap enforcement, Redis landing for other reasons, real-time admin UI).

## Writes — transactional, fire `PolicyChangedEvent` after commit

- `create(draft)` — rejects pre-set `policy_id` or `is_default=true`
(promotion must go through `setDefault` so the partial unique idx is
freed first).
- `setDefault(id)` — atomically clears the existing default via
`clearDefaultFlag()` then flips the new row. Idempotent: silent no-op if
the row is already default.
- `setTeamOverride(teamId, policyId | null)` — validates the policy
exists before save; `null` clears the override.

`publishOnCommit` uses `TransactionSynchronizationManager.afterCommit`
so listeners never see pre-commit state. Outside a transaction (test
paths) falls through to immediate publish.

## Admin REST surface — `/api/v1/admin/payg/...`

All endpoints `@PreAuthorize("hasRole('ADMIN')")`:

- `GET  /policies` — list all
- `GET  /policies/{id}` — read one
- `POST /policies` — create new (non-default)
- `POST /policies/{id}/set-default` — atomic promote
- `PUT /teams/{teamId}/policy-override` — set or clear per-team override
- `GET  /teams/{teamId}/effective-policy` — cache-bypassing live read

Validation errors → 400, unknown rows → 404.

## Counterpart Supabase PR


[`Stirling-PDF-SaaS#298`](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/298)
— seeds the same V1 default policy on the Supabase side via
`20260528000002_payg_seed_default_policy.sql`.

## Tests

- 17 × `PricingPolicyServiceTest` — lookup precedence, cache hit/miss,
invalidation on event, mutation paths publishing event, error cases.
- 14 × `PricingPolicyAdminControllerTest` — every endpoint's happy path
+ error mapping, DTO defensive-copy invariant.
- 2 × `SaasJpaConfigScanTest` — reflection-based guard that
`payg.repository` is in `@EnableJpaRepositories` and `payg` is in
`@EntityScan`. Without this, new sub-packages can silently fail to wire
at runtime — same class of bug that the #6460 review caught.

Full `:saas:test` BUILD SUCCESSFUL.

## Design doc

`notes/PAYG_DESIGN.md` §7.4 PR-I1 — completes the service half (the
schema half landed in #6460). §9 carries the 30s-TTL trade-off note.
2026-05-29 16:27:01 +00:00
James Brunton 2c01f41142 Update indexeddb to v9 to unify SaaS and OSS users (#6474)
# Description of Changes
The production SaaS is currently on v8 of IndexedDB due to various
schema changes for Smart Folders, which haven't made their way into OSS.
OSS is currently on v4 of IndexedDB, so if we release an OSS build to
the SaaS deployment, existing users will not be able to use it because
the DB version is 'too old'.

This PR updates the IDB version number to v9 so both OSS and SaaS users
will be able to upgrade to it. Theoretically both types of user should
be able to keep their IDB files without issue. SaaS previously actively
wiped the user's files in an old version (v6/v7) and users who haven't
used it since then will have their DBs wiped, but that'd happen anyway
if they use current SaaS so I don't think that matters.
2026-05-29 15:23:40 +00:00
James Brunton 4d5eeb103f Fix username display issues (#6471)
# Description of Changes
Main fixes:
- Fix the display of the username in the bottom left
- Now displays as "User" when not logged in on self-hosted (desktop) and
"Guest" on SaaS when logged in anonymously
- Now updates properly when the user logs in/out in SaaS, desktop and
self-hosted
- Fix incremental build issues in the desktop app that have been here
since the start (I hope at least - I think the issue is that the JLink
is built read-only and then on subsequent builds you get OS errors when
trying to override the JLink with the new version. There's no real need
for it to be read-only that I know of, so we might as well just make it
R/W and ship like that)
2026-05-29 14:35:47 +00:00
ConnorYoh 83ea07ed6a saas: DocumentClassifier + PAYG data model (#6460)
# Description of Changes

Two layers — the `DocumentClassifier` utility plus the full data model
for the new billing engine. Nothing wires the entities into application
behaviour yet; services and controllers land in follow-up PRs.

**Companion PR:**
[Stirling-PDF-SaaS#296](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/296)
— Supabase migration for the v3 dev branch, schema-equivalent to the
Flyway migration in this PR.

## 1. DocumentClassifier (under `payg.docs`)

`DocumentClassifier` computes the doc-unit cost of an uploaded file (or
multi-file input) under a `PricingPolicy`. PDFs read page count via
`stirling.software.jpdfium.PdfDocument`; non-PDFs are bytes-only.
Formula: `max(ceil(pages / docPagesPerUnit), ceil(bytes /
docBytesPerUnit))` clamped to `[1, fileUnitCap]`. Multi-file is the sum
of raw per-file units capped at `fileUnitCap × file_count`.

Two floors, by design: the classifier returns `docUnits` with an
absolute `1` floor for non-empty input; the policy-level
`minChargeUnits` is intentionally applied later, at process-open time in
`JobChargeService`, per design § 3.4 (`unitsForProcess =
max(policy.min_charge_units, docUnits)`). Documented in the interface +
impl javadoc.

Upload bytes are materialised through
`TempFileManager.createManagedTempFile` so jpdfium gets a `Path`; the
temp file auto-deletes on close.

Twelve tests, all in-memory fixtures generated with PDFBox at test time
— no committed binary blobs.

## 2. PAYG data model (under `payg.*`)

JPA entities, repositories, and a Flyway migration covering the full
schema in §6 of the design.

**Enums** (`payg.model`):

`JobSource`, `ProcessType`, `JobStatus`, `JobStepStatus`,
`ArtifactKind`, `LedgerEntryType`, `LedgerBucket`, `ReferenceType`,
`EntitlementState`, `FeatureSet`, `FeatureGate`, `WalletEngine`,
`CapPeriod`, `AutoGroupStrategy`.

**Entities + repositories:**

| Entity | Table | Notes |
|---|---|---|
| `PricingPolicy` | `pricing_policy` | Promoted from a record.
`stepLimits` is `Map<JobSource, Integer>` persisted via normalised child
table `pricing_policy_step_limit`. `stripePriceIds` is `Set<String>`
persisted via `pricing_policy_stripe_price` — currency comes from
`stripe.prices` via Sync Engine, not stored locally. |
| `ProcessingJob` | `processing_job` | UUID PK. Tracks lineage window
via `step_count` and `last_step_at`. |
| `ProcessingJobStep` | `processing_job_step` | Per-tool-call audit. |
| `JobArtifactHash` | `job_artifact_hash` | Composite key `(job_id,
content_hash, kind)`. `content_hash VARCHAR(128)` so multiple signature
schemes coexist as `"type:value"` storage keys. Lineage detector queries
this. |
| `WalletLedgerEntry` | `wallet_ledger` | Append-only, signed
`amount_units`. Two unique indexes kill double-posting. |
| `WalletPolicy` | `wallet_policy` | Per-team engine + cap + degradation
rules + lineage strategy. No `@Version` — admin-only writes (documented
in javadoc). |
| `WalletEntitlementSnapshot` | `wallet_entitlement_snapshot` |
Composite key `(team_id, user_id)`; `user_id = 0` is the team-wide
sentinel. No `@Version` — full-row recompute via
`EntitlementService.recompute` (documented in javadoc). |
| `PaygShadowCharge` | `payg_shadow_charge` | Per-job diff while in
`PAYG_SHADOW` engine mode. |
| `PaygTeamExtensions` | `payg_team_extensions` | Sidecar 1:1 with
`teams` carrying `pricing_policy_id` (per-team override) +
`stripe_customer_id`. Sidecar pattern (mirrors `saas_team_extensions`)
so OSS Hibernate ddl-auto never sees PAYG columns on `teams`. |

**Column adds:**

- `team_memberships.cap_units` (optional per-member sub-cap)

**Width split (intentional, documented in V11):** per-row deltas
(`wallet_ledger.amount_units`, `processing_job.charged_units`) are
`INTEGER` because no single charge realistically approaches 2B units.
Cap and period-rollup columns (`team_memberships.cap_units`,
`wallet_policy.cap_units`,
`wallet_entitlement_snapshot.period_spend_units / period_cap_units`) are
`BIGINT` because they accumulate across a billing period and admins may
legitimately set headroom-cap values into the millions.

**JPA wiring:** `SaasJpaConfig` was updated to include
`stirling.software.saas.payg.repository` in
`@EnableJpaRepositories.basePackages` and `stirling.software.saas.payg`
in `@EntityScan` (covers `payg.policy` / `payg.job` / `payg.wallet` /
`payg.entitlement` / `payg.shadow` recursively). New
`SaasJpaConfigScanTest` reads the annotations reflectively and asserts
every expected package is wired — catches the next time someone adds a
new sub-package without updating the scan paths.

**Migration:** `V11__saas_payg_model.sql` (purely additive).
Schema-equivalent to the Supabase migration in the companion PR —
including the `VARCHAR(128) content_hash` width that's needed for the
multi-signature-scheme storage encoding the lineage layer uses.

## 3. Smoke tests

`PaygEntitiesSmokeTest` exercises each entity via the no-arg ctor JPA
requires, plus getter/setter round-trips and composite-key equality —
catches Lombok/annotation regressions without needing a database.
Real-DB integration coverage lands alongside the services that consume
each entity.

## Why this is safe to land now

- All schema changes are additive — no existing rows modified, no
columns dropped.
- The entities are not yet referenced from any production code path;
they exist for the next PRs to build on.
- The v3 Supabase dev branch picks up the schema via the companion PR;
the main repo's Flyway migration applies the same shape when an instance
boots against a freshly-migrated v3 database.

## Open decisions made

- **Step-limits keyed by `JobSource`** rather than by `ProcessType`.
Captures the "self-hosted gets a different knob" framing in earlier
feedback. Trivially overridable per pricing policy version.
- **Step limits + Stripe price IDs normalised into child tables** rather
than JSONB on `pricing_policy` (per Connor's review on #296). Typed
columns, queryable directly, no JSON parsing.
- **Currency dropped from `pricing_policy_stripe_price`** — it lives on
`stripe.prices.currency` and is resolved via Sync Engine. App is
currency-blind.

## Rollback

Straight `git revert` on this PR. The Supabase migration in #296 is
additive and can be left in place safely — the running app ignores
tables it doesn't reference.

---

## Checklist

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings
- [x] I have run `task check` (via `./gradlew :saas:test` with
`ENABLE_SAAS=true`) — passes
2026-05-29 12:03:01 +00:00
James Brunton 61ebe977d3 Auto-delete CI linting comments on success (#6465)
# Description of Changes
Set CI backend & engine comments to auto-delete once the CI has passed. 

Also redesign the engine CI to call `task engine:check` like it should
have been, and make it post a comment when the tool models need to be
updated.

Also makes the comment wording more consistent between the three
languages.
2026-05-29 10:13:12 +00:00
EthanHealy01 763595a5a3 feat: add Agents UI to proprietary right sidebar (#6454)
Update UI to include agents

Run `task dev:all` to test
2026-05-28 17:26:23 +00:00
Anthony Stirling 398617391b Fix SSO auto-login and custom metadata settings not persisting on restart (#6468) 2026-05-28 17:39:05 +01:00
ConnorYoh a0e0e88f07 saas: harden CreditService Stripe ordering + lint @AutoJobPostMapping weights (#6458)
# Description of Changes

Two narrowly-scoped hardening changes to the credits engine.

## 1. CreditService — move Stripe meter call to `afterCommit`

The Stripe metered-usage call sits inside the surrounding
`@Transactional`, holding the `user_credits` row lock for the duration
of an HTTP round-trip to Supabase. Under load this starves concurrent
debits; a transient Stripe blip rolls back a (correct) free-credit
consumption and forces the caller to retry.

The Stripe call now runs in a `TransactionSynchronization.afterCommit`
hook — DB commits first, Stripe fires immediately after. If Stripe fails
after commit, we log + increment a new `credits.stripe_report.failures`
counter; the idempotency key is stable, so a manual replay recovers
without double-charging.

Applied to both `consumeCreditBySupabaseId` and
`consumeCreditWithWaterfall`.

**Dead-code removed:**
- Unreachable UUID fallback for MDC `requestId` — `CorrelationIdFilter`
already guarantees the key on every request.
- The `"Unable to report usage to Stripe"` `RuntimeException` and its
catch block — the afterCommit refactor eliminates the throw path.
- `StripeRollbackOnFailureTest` — pinned the rollback-on-Stripe-fail
behaviour this refactor replaces.

## 2. `@AutoJobPostMapping` — build-time lint for `resourceWeight`

`UnifiedCreditInterceptor` multiplies `resourceWeight` into the per-call
charge. An endpoint that falls through to the annotation default
produces a charge derived from a value nobody chose.

- Annotation default flipped from `1` to `Integer.MIN_VALUE` (sentinel).
Both runtime readers (`UnifiedCreditInterceptor`, `AutoJobAspect`)
already clamp into `[1, 100]` so behaviour is unchanged.
- New `AutoJobPostMappingWeightTest` scans the classpath and fails the
build if any method leaves the sentinel.
- Initial run caught 11 endpoints relying on the default. Explicit
weights now declared, chosen by comparing to peer endpoints:
  - `EditTextController` — LARGE
  - `EmailController#sendEmailWithAttachment` — SMALL
  - `ConvertPDFToMarkdown` — MEDIUM
  - `AttachmentController` (extract/list/rename/delete) — SMALL × 4
  - `ConvertImgPDFController` (cbr/cbz ↔ pdf) — MEDIUM × 2, LARGE × 2

## Tests

- `StripeUsageIdempotencyKeyTest` — pins the `(supabaseId, overage,
requestId)` idempotency key shape so Stripe always dedupes a retry.
- `StripeAfterCommitOrderingTest` — pins that `afterCommit` fires after
commit and NOT on rollback.
- `AutoJobPostMappingWeightTest` — the lint itself, plus a self-check
that the classpath scan finds at least 10 `@AutoJobPostMapping` methods
(guards against the lint passing vacuously).

Build verified: `ENABLE_SAAS=true ./gradlew :stirling-pdf:test
:saas:test`.

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable) — no translation changes
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
— internal-billing change, no public docs impact
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
— N/A

### Translations (if applicable)

- [ ] Not applicable

### UI Changes (if applicable)

- [ ] Not applicable

### Testing (if applicable)

- [x] I have run `task check` (via `./gradlew :stirling-pdf:test
:saas:test` with `ENABLE_SAAS=true`) — passes
- [x] I have tested my changes locally
2026-05-28 14:57:59 +00:00
Anthony Stirling c80a5db5f5 folder and file fixes (#6461) 2026-05-28 15:57:35 +01:00
Anthony Stirling 4fa67afc3d Fix Tauri artifact copy path so installers upload (smoke + release) (#6466)
## Summary
Regression from #6404 (Restructure/frontend editor). Two CI workflows
copy the built installers to the wrong directory, so installer artifacts
(MSI / DMG / DEB / RPM / AppImage) silently vanish:

- **`tauri-build.yml`** (PR/desktop smoke builds) - uploads zero
installer artifacts.
- **`multiOSReleases.yml`** (production releases) - the empty artifacts
are downloaded by `create-release` and fed to `action-gh-release`, so a
release would publish **only the JARs, no desktop installers**.

## Root cause
#6404 moved the Tauri project from `frontend/` to `frontend/editor/` and
updated every **absolute** path (`projectPath`, `cd`, `Get-ChildItem`)
to add the `editor/` segment - but left the **relative** copy targets
`../../../dist`. Those resolve against the (now one level deeper)
working dir after `cd ./frontend/editor/src-tauri/target`:

| | resolves to |
|---|---|
| before #6404 (`frontend/src-tauri/target`) | repo-root `dist/`  |
| after #6404 (`frontend/editor/src-tauri/target`) | `frontend/dist/` 
(missing) |

The `cp` fails, repo-root `dist/` (from `mkdir -p ./dist`) stays empty,
and the upload finds nothing. `find -exec cp` failing is non-fatal, so
jobs still report success - that's why it went unnoticed. No release has
shipped broken yet: the last release (v2.11.0, 2026-05-19) predates
#6404 (2026-05-22).

## Fix
Copy to an absolute `$GITHUB_WORKSPACE/dist` in both workflows so the
`cd` can't drift the destination again. This matches where the upload /
signature-verify steps already read from.

## Evidence (run 26574078559, all 3 OS legs)
```
cp: cannot create regular file '../../../dist/Stirling-PDF-windows-x86_64.msi': No such file or directory
##[warning]No files were found with the provided path: ./dist/*. No artifacts will be uploaded.
```
The Tauri builds themselves succeeded - only the copy/upload was broken.

## Test plan
- [ ] `tauri-build` on this PR uploads non-empty `Stirling-PDF-<name>`
artifacts on Windows/macOS/Linux.
- [ ] Next release (or a `workflow_dispatch` of multiOSReleases)
attaches MSI/DMG/DEB/RPM/AppImage to the release.
2026-05-28 15:57:01 +01:00
Anthony StirlingandConnorYoh 8bd78d2624 Add landscape page size options (#6248)
# Description of Changes

Adds orientation (portrait/landscape) to the Adjust Page Scale tool.

- Orientation as a separate parameter (per review), sent through to the
backend
- ScalePagesController simplified; PDFWithPageSize gains the orientation
field
- Regenerated tool_models.py; frontend + backend tests added

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

---------

Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
2026-05-28 14:16:15 +00:00
Anthony Stirling b3c4b8b463 Add S3 storage and cluster artifact backend (#6457) 2026-05-28 13:06:27 +01:00
James Brunton 57af5b9dc2 Fix Tauri testing (#6462)
# Description of Changes
#6402 introduced a Rust test `refresh_token_fallback.rs`, but it wasn't
moved properly after the restructure of the `frontend/` folder in #6404.
This PR moves the file to the right place, and also hooks up Task and CI
rules for `cargo test` since nothing was actually running the test in
the first place.
2026-05-28 11:05:56 +00:00
James Brunton 44fbf8c587 Various bug fixes found while testing SaaS build (#6459)
# Description of Changes
Various fixes and improvements I made while testing the SaaS code:
- Changes the new `.env.saas` file to live in `app/` and match the
semantics of the other `.env` files
- Adds top-level `task dev:saas` command to spawn SaaS frontend &
backend
- Deletes dead SaaS code and improves some overriding logic
- Fixes refreshing issue when coming back to the tab
- Fix the Compare tool's selection logic
- Make Compare handle error cases properly
- Fixes the location of the "Dismiss All Errors" button (was rendering
on top of the top-bar with a transparent background previously so it
looked rubbish)
- Fixes file selection in PDF Editor
2026-05-28 11:05:30 +00:00
Anthony Stirling 76840d8a57 Add CI DB migration smoke test against v2.0/v2.5/v2.10 updates (#6453) 2026-05-28 11:36:07 +01:00
James Brunton d459ded168 Add cancel button to kill long-running AI tasks (#6351)
# Description of Changes
Adds a cancel button to the AI chat to allow the user to abort
long-running AI tasks. Just disconnects the SSE stream (all the backend
code already interrupts when it notices the stream is dead).
2026-05-28 09:25:23 +00:00
ConnorYoh 43b67d213d feat(oauth2): opt-in claim-dump diagnostics for OIDC login failures (#6456)
# Description of Changes

## What & why

Customers using ADFS (or any generic OIDC provider that doesn't emit
`email`) hit `Attribute value for 'email' cannot be null` during OAuth2
login with no visibility into what claims the provider actually sent.
The only available remedy was guessing at
`security.oauth2.useAsUsername` until something worked.

This PR adds a new opt-in `security.oauth2.debugLogging` flag (default
`false`). When enabled, `CustomOAuth2UserService` logs:

- All ID token claims (sorted, with values)
- All UserInfo endpoint claims (if any)
- The merged attribute key set Spring exposes to `getAttribute()`
- The value the configured `useAsUsername` actually resolved to
- A **`Hint:`** line listing the claim keys present in the token that
map to a valid `UsernameAttribute` enum value — i.e. exactly what the
operator could put in `useAsUsername` to make login work

Logged at `INFO` on the success path and `ERROR` on failure (inside the
existing `catch (IllegalArgumentException)` block that throws
`OAuth2AuthenticationException`). The block is wrapped with a `[OAUTH2
DEBUG] ... [/OAUTH2 DEBUG]` banner and ends with a PII warning so
operators don't leave it on in production.

Default off → zero observable change for anyone not actively
troubleshooting.

## Files changed

| File | Why |
|---|---|
| `app/common/.../ApplicationProperties.java` | New `debugLogging` field
on the `OAUTH2` config class with javadoc warning about PII |
| `app/core/src/main/resources/settings.yml.template` | Documents
`oauth2.debugLogging` so it appears on next startup |
| `app/proprietary/.../security/service/CustomOAuth2UserService.java` |
Emits the claim dump + suggestion hint when the flag is on |
|
`app/proprietary/.../security/service/CustomOAuth2UserServiceDebugLoggingTest.java`
(new) | Unit test: mocks the OIDC delegate, asserts off-path is silent
and on-path emits the dump with the right Hint contents |

## End-to-end verification

Ran the bundled `testing/compose/docker-compose-keycloak-oauth.yml`
Keycloak realm, configured `security.oauth2.useAsUsername: mail`
(Keycloak emits `email`, not `mail`) and `provider: demarest` (matches
the original customer bug report). Triggered the OAuth flow at
`http://localhost:8080/oauth2/authorization/demarest` and confirmed:

- The ERROR-level dump fires with the full 19-claim ID token decoded
- `-- Value at 'mail' : <NULL — this is why login fails>` correctly
identifies the missing claim
- `-- Hint:` correctly suggests `[email, family_name, given_name,
preferred_username]` (the four keys present that map to valid
`UsernameAttribute` values)
- Auth still fails with the original `OAuth2AuthenticationException` —
no change to control flow, just added diagnostic logging

Unit test (`CustomOAuth2UserServiceDebugLoggingTest`) covers both
branches.

## Reviewer notes

- **No new public APIs.** The flag is config-only; no servlet endpoints
exposed.
- **PII is logged when the flag is on.** This is the whole point —
operators need to see the claims to fix their config — but it's gated,
defaults off, and the dump self-documents with a `WARNING: ... Set
security.oauth2.debugLogging=false once troubleshooting is complete.`
footer.
- **Why log everything, not just sub/email?** Because the operator
doesn't know in advance which claim they actually want. ADFS uses `upn`
in some configs and `preferred_username` in others; Azure AD uses `oid`;
the customer here had neither. Dumping the full set is the only way to
make the diagnostic self-service.
- **Out of scope for this PR (follow-ups):**
- The `UsernameAttribute` enum doesn't include `upn` / `unique_name`
(common ADFS claims). If the customer's token only has `upn`, the Hint
will be empty even though the operator can see `upn` in the dump. Worth
a separate PR to extend the enum.
- The known-provider validator in `Provider.java` (rejects e.g.
`useAsUsername: mail` for `provider: keycloak` at startup) bypasses our
diagnostic for those provider names. ADFS customers using `provider:
<name>` fall into the `default` branch so are not affected — but it's a
sharp edge worth documenting.

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable) — N/A, backend-only change
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] Doc-repo update (if functionality has heavily changed) —
diagnostic flag is self-documenting via the `settings.yml.template`
comment and the in-log warning; happy to add a doc-repo entry if
reviewers want one
- [ ] Translation tags — N/A

### UI Changes (if applicable)

- [ ] N/A — backend-only

### Testing (if applicable)

- [x] Unit test added (`CustomOAuth2UserServiceDebugLoggingTest`)
covering on/off paths and Hint correctness
- [x] End-to-end verified locally against bundled Keycloak compose with
intentionally misconfigured `useAsUsername`
- [x] Full `:proprietary:test` suite passes
2026-05-27 13:01:51 +00:00
Anthony Stirling d42b779644 Add server-side folders and files page UI (#6383) 2026-05-27 12:52:46 +01:00
Anthony Stirling 4564ed5bec Add cluster backplane abstraction and interfaces (#6449) 2026-05-27 11:54:59 +01:00
Matt Van HornandMatt Van Horn 7743720f0a chore: fix "Endoints" typo in health route comment (#6446)
`app/common/src/main/java/stirling/software/common/util/RequestUriUtils.java:175`
section comment said "Health Endoints" → "Health Endpoints".
Comment-only.

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
2026-05-26 23:30:57 +01:00
Saul Ifshin 620f246ca4 test(frontend): add URL-encoded filename parsing case (#6436)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [x] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [x] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [x] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [x] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [x] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [x] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-05-26 23:29:32 +01:00
Saul Ifshin dd24440b73 test(frontend): cover MIME precedence in non-PDF type detection (#6438)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [x] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [x] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [x] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [x] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [x] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [x] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-05-26 23:28:17 +01:00
Ludy c0374266e7 ci: harden workflow runners and update checkout action pins (#6445)
# Description of Changes

Add runner hardening and housekeeping across workflows.

- .github/dependabot.yml: add /docker/unoserver and /engine to
Dependabot update paths.
- .github/workflows/_runner-pick.yml and .github/workflows/build.yml:
add step-security/harden-runner (egress-policy: audit) to audit outbound
calls from runners.
- .github/workflows/ai-engine.yml, .github/workflows/aur-publish.yml,
.github/workflows/package-managers.yml: update actions/checkout usage to
the newer v6.0.2 reference.
- .github/workflows/package-managers.yml: minor YAML formatting tidy
(release types array).

These changes improve CI security by auditing egress, update the
checkout action to a newer release, and expand Dependabot coverage.

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-05-26 23:26:20 +01:00
Anthony Stirling 730ed43af9 Fix login loop (#6402) 2026-05-26 20:05:25 +01:00
Anthony Stirling 56ff1e5092 impl migration to pdfium for split (#6410)
## Summary

Migrates `SplitPDFController`, `SplitPdfByChaptersController`,
`SplitPdfBySizeController` from PDFBox to JPDFium.
`SplitPdfBySectionsController` and `AutoSplitPdfController` are
intentionally left on PDFBox (require JPDFium 1.0.2 features that don't
exist yet).

## Benchmark (audited on `audit/jpdfium-split`, file
`app/core/src/test/java/stirling/software/SPDF/bench/SplitBenchmark.java`)

| Workload | PDFBox heap | JPDFium heap | PDFBox wall | JPDFium wall |
|---|---|---|---|---|
| 100 pp, chunk 10 | +21-26 MB | **+0.02 MB** | 80-106 ms | **25 ms** |
| 300 pp, chunk 10 | +59 MB | **+1.0 MB** | 232 ms | **76 ms** |

**98-99.9% heap reduction. 3-4.2x faster wall.**

## Hybrid

- AcroForm-bearing splits keep PDFBox
`FormUtils.pruneOrphanedFormFields` post-pass (FPDF_ImportPagesByIndex
drops AcroForm dict). Sub-bench shows +1.0 MB / +27 ms - tightly
bounded.
- Metadata extraction stays on PDFBox.
- `SplitPdfBySectionsController` - JPDFium `PdfPageSplitter` only does
2-up halving, not arbitrary MxN.
- `AutoSplitPdfController` - needs PDFRenderer + zxing for QR markers.

## Test plan

- [ ] 30/30 unit tests pass with PDFBox `Loader.loadPDF` as the oracle
(assert page counts + document totals)
- [ ] Existing cucumber feature `split.feature` continues to pass
- [ ] AcroForm-bearing PDF round-trips without orphaned widgets (covered
by existing FormUtils tests)
2026-05-26 17:50:13 +01:00
ConnorYoh 05b80fbe4f Working local Saas (#6450)
Env file for setting backend saas and taskfile for running it
2026-05-26 14:40:44 +00:00
Reece Browne 4047d02086 chore: address restructure PR review feedback (#6423) 2026-05-26 14:12:05 +01:00
Ludy 5f78083470 Fix unresolved Material Symbols icon names in frontend (#6443) 2026-05-25 17:02:59 +01:00
dependabot[bot]andAnthony Stirling fc048872d7 build(deps): bump idna from 3.12 to 3.15 in /testing/cucumber in the pip group across 1 directory (#6403)
Bumps the pip group with 1 update in the /testing/cucumber directory:
[idna](https://github.com/kjd/idna).

Updates `idna` from 3.12 to 3.15
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/kjd/idna/blob/master/HISTORY.md">idna's
changelog</a>.</em></p>
<blockquote>
<h2>3.15 (2026-05-12)</h2>
<ul>
<li>Enforce DNS-length cap on individual labels early in
<code>check_label</code>,
short-circuiting contextual-rule processing for oversized input
while staying compatible with UTS 46 usage.</li>
<li>Tidy core helpers: hoist bidi category sets to module-level
frozensets (avoiding per-codepoint list construction), simplify
length checks, and reuse the shared <code>_unicode_dots_re</code> from
<code>idna.core</code> in the codec module.</li>
<li>Use <code>raise ... from err</code> for proper exception chaining
and
switch internal string formatting to f-strings.</li>
<li>Allow <code>flit_core</code> 4.x in the build backend.</li>
<li>Expand the ruff lint set (flake8-bugbear, flake8-simplify,
pyupgrade, perflint) and apply the surfaced fixes; pin lint CI
to Python 3.14.</li>
<li>Add Dependabot configuration for GitHub Actions.</li>
<li>Convert README and HISTORY from reStructuredText to Markdown.</li>
<li>Reference CVE-2026-45409 for the 3.14 advisory in place of the
initial GHSA identifier.</li>
</ul>
<p>Thanks to Felix Yan, Stan Ulbrych, and metsw24-max for
contributions to this release.</p>
<h2>3.14 (2026-05-10)</h2>
<ul>
<li>Removed opportunity to process long inputs into quadratic
time by rejecting oversize inputs up-front. Closes a bypass
of the CVE-2024-3651 mitigation. [CVE-2026-45409]</li>
</ul>
<p>Thanks to Stan Ulbrych for reporting the issue.</p>
<h2>3.13 (2026-04-22)</h2>
<ul>
<li>Correct classification error for codepoint U+A7F1</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/kjd/idna/commit/af30a092e158181d0b35ac66dfa813788126bdd8"><code>af30a09</code></a>
Release 3.15</li>
<li><a
href="https://github.com/kjd/idna/commit/30314d4628744ca14cf2b5820564e5127a9f86f2"><code>30314d4</code></a>
Pre-release 3.15rc0</li>
<li><a
href="https://github.com/kjd/idna/commit/05d4b219aa9eddc47371fcbd2000f0301016f3e9"><code>05d4b21</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/237">#237</a> from
kjd/convert-docs-to-markdown</li>
<li><a
href="https://github.com/kjd/idna/commit/2987fdba1962bbb2358399e0084ba062b98a0bee"><code>2987fdb</code></a>
Convert README and HISTORY from reStructuredText to Markdown</li>
<li><a
href="https://github.com/kjd/idna/commit/59fa8002d514bf4a5ce7b58f67b9ec587d53fa9c"><code>59fa800</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/236">#236</a> from
kjd/dependabot/github_actions/actions-f3e34333ea</li>
<li><a
href="https://github.com/kjd/idna/commit/def69834ced5d4b3c50439d8b99c4c856ec19ca2"><code>def6983</code></a>
Merge branch 'master' into
dependabot/github_actions/actions-f3e34333ea</li>
<li><a
href="https://github.com/kjd/idna/commit/bbd8004a797185d8c56bb555cd5c88fde05e0631"><code>bbd8004</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/234">#234</a> from
StanFromIreland/patch-1</li>
<li><a
href="https://github.com/kjd/idna/commit/edd07c05024344a6ccb517414ccb36683aee99fc"><code>edd07c0</code></a>
Bump github/codeql-action from 3.35.2 to 4.35.2 in the actions
group</li>
<li><a
href="https://github.com/kjd/idna/commit/5557db030c11bdec50d62aa5f631d705d33ba123"><code>5557db0</code></a>
Merge branch 'master' into patch-1</li>
<li><a
href="https://github.com/kjd/idna/commit/f11746cf4981d25123ef7830d3ee60f07de8ae3d"><code>f11746c</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/235">#235</a> from
StanFromIreland/patch-2</li>
<li>Additional commits viewable in <a
href="https://github.com/kjd/idna/compare/v3.12...v3.15">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=idna&package-manager=pip&previous-version=3.12&new-version=3.15)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/Stirling-Tools/Stirling-PDF/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-05-24 09:51:19 +00:00
dependabot[bot] 6ee533c427 build(deps): bump ws from 8.20.0 to 8.20.1 in /frontend (#6400)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:47:49 +01:00
dependabot[bot] a13683e84a build(deps): bump protobufjs from 7.5.6 to 7.6.0 in /frontend (#6401)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:44:02 +01:00
dependabot[bot]andAnthony Stirling 914b474d84 build(deps-dev): bump ip-address from 10.1.0 to 10.2.0 in /frontend (#6319)
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:43:11 +01:00
dependabot[bot] 51f69a718f build(deps): bump svelte from 5.55.0 to 5.55.9 in /frontend (#6409)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:42:51 +01:00
dependabot[bot] 7f597051e4 build(deps): bump reportlab from 4.5.0 to 4.5.1 in /testing/cucumber (#6414)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:42:26 +01:00
dependabot[bot] e113df6252 build(deps): bump requests from 2.33.1 to 2.34.2 in /testing/cucumber (#6416)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:42:11 +01:00
dependabot[bot] 06b4a4184b build(deps): bump com.diffplug.spotless from 8.4.0 to 8.5.0 (#6417)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:41:52 +01:00
dependabot[bot] 19982d7207 build(deps): bump actions/ai-inference from 2.0.7 to 2.1.0 (#6415)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:30:44 +01:00
dependabot[bot] cdfefb02bc build(deps): bump github/codeql-action from 4.35.4 to 4.35.5 (#6421)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:30:28 +01:00
dependabot[bot] 5ba6a65c1a build(deps): bump step-security/harden-runner from 2.19.1 to 2.19.3 (#6422)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:30:02 +01:00
ConnorYoh 017c8d59fa feat(ai): add Contradiction Agent on a new ChunkedMapper primitive (#6369)
## Summary

Adds a new AI specialist that finds **textual contradictions** across
one or more PDFs — conflicting claims, recommendations, points of view,
contested facts — built entirely in Python on top of the new
`DocumentService` + `ChunkedReasoner` stack from #6314. Replaces the
closed #6304, which was started before #6314 landed and therefore
over-engineered (Java orchestrator, two-round handshake, resume
artifact, discriminated-union lift).

Two commits:
1. **`refactor(engine): extract ChunkedMapper[T] from ChunkedReasoner`**
— pure refactor, public API of ChunkedReasoner unchanged. New
`ChunkedMapper[T: BaseModel]` is a generic parallel-chunk primitive
(slicing, semaphore, time-bounded extraction, cancellation drain,
progress events) that's now a peer to ChunkedReasoner rather than locked
inside it. The compression loop stays on ChunkedReasoner where it
belongs.
2. **`feat(ai): add Contradiction Agent on ChunkedMapper`** — the agent
itself, plus integrations into `PdfReviewAgent` and `PdfQuestionAgent`.

## Architecture

- **Python-only.** No Java code. No `AgentToolId.CONTRADICTION_AGENT`.
No dedicated HTTP endpoint. No resume artifact, no discriminated-union
lift in `contracts/common.py`. Detector runs inside the Python engine
and the Python engine alone.
- **Review path** (`PdfReviewAgent`): a new
`ContradictionIntentClassifier` fires on contradiction-flavoured
prompts; agent runs detection synchronously and emits a single
`EditPlanResponse(steps=[ADD_COMMENTS])`. Single-turn flow — no resume.
- **Question path** (`PdfQuestionAgent`): a new
`ContradictionCapability` joins `RagCapability` and
`WholeDocReaderCapability` in the smart-model toolset, exposing
`find_contradictions(query)`. The smart model picks it from the toolset
alongside `search_knowledge` and `read_full_document`.

## Inside `ContradictionDetector.detect()`

1. `DocumentService.read_pages(file_id)` → ordered `list[Page]`.
2. `ChunkedMapper[_ExtractedClaims].map_pages(...)` — char-budgeted
multi-page slicing; each slice runs the claim-extractor LLM in parallel
under a semaphore.
3. Page-traceability: the extractor returns `_ExtractedClaim.page`
(which `[Page N]` marker the claim came from). The wrapper validates
`page ∈ chunk.pages`; if not, mechanical fallback searches the chunk's
page text for the verbatim quote and reassigns. If still no match, drop
the claim.
4. `Claim.anchor_quality: Literal[\"verbatim\", \"paraphrased\"]` is set
by a substring check against the declared page's text. Verbatim quotes
feed `anchor_text` for snap-to-quote add-comments placement; paraphrased
ones fall back to margin geometry.
5. Subject canonicalisation: ONE fast-model LLM call collapses synonyms
across the document. Fails open to lexical bucketing.
6. Pre-filters: drop identical-quote pairs; drop same-page same-polarity
paraphrases.
7. Per-bucket pair detection in parallel (separate semaphore, cap 5).
Buckets > 12 claims chunk into windows of 12 with overlap 2; pairs
deduped across overlapping windows by frozen `(i, j)` index pair.
8. Summary fast-model call with fallback string on error.

## Prompt-injection hardening

Every prompt that interpolates user-supplied or PDF-extracted text wraps
content in `<user_message>` / `<verdict>` / `<content>` tags with an
explicit SECURITY preamble instructing the model to treat tagged content
as data only.

## Limitations

- **Combined math + contradiction intent**: when both intent classifiers
fire on the same prompt, contradiction takes precedence and the math
intent is silently dropped. Documented in the Review module docstring
and pinned by
`test_review_integration.py::test_contradiction_precedence_over_math`.
- **Cross-window contradiction reach**: within a subject bucket, pairs
more than ~10 claim indices apart in the same chunked window may be
missed by the overlap-2 strategy. Documented in `test_detector.py`.
Acceptable for v1.

## Settings (engine/src/stirling/config/settings.py)

```python
contradiction_detect_concurrency = 5     # per-bucket detector semaphore
contradiction_bucket_chunk_size = 12     # max claims per detector call
contradiction_bucket_chunk_overlap = 2   # overlap for >threshold buckets
```

`chars_per_slice` and extraction concurrency are reused from the
existing `chunked_reasoner_*` settings.

## Test plan

- [x] `uv run pytest tests/ -v` — **245/245 pass** (210 pre-existing +
35 new)
- [x] `uv run ruff check src/ tests/` — clean
- [x] `uv run pyright src/stirling/agents/contradiction/
src/stirling/contracts/contradiction.py` — 0 errors
- [x] `./gradlew :proprietary:test` — green; no Java was touched, but
verified untouched
- [x] Page-traceability tests cover: valid page kept, hallucinated page
dropped, mechanical-reassign on misattribution, anchor-quality verbatim
vs paraphrased
- [x] Review integration: ADD_COMMENTS plan with two paired CommentSpecs
per contradiction; NeedIngestResponse precheck; precedence vs math
intent pinned
- [x] Question integration: all three capabilities wired into
smart-model toolset; `find_contradictions` returns formatted report text
- [x] ChunkedMapper standalone: slicing, multi-chunk ordering, worker
failures, timeouts, cancellation drain, semaphore saturation
- [x] ChunkedReasoner regression: all pre-existing tests pass unchanged
after the internal split

## Relationship to closed #6304

#6304 was closed in favour of this PR. The closed PR predated #6314 and
modelled the agent as a Java-orchestrated two-round examine/deliberate
flow with its own HTTP endpoint and a discriminated-union resume
artifact. With #6314 making full ordered page text available to the
engine via `DocumentService.read_pages`, none of that is needed. Net
effect: drop ~600 lines of Java, drop the two-round handshake, drop the
`ToolReportArtifact` lift, while ending up with a more scalable agent
(chunk-based instead of page-based extraction; tested to
ChunkedReasoner-equivalent scale).
2026-05-22 13:23:52 +00:00
Reece BrowneandClaude Opus 4.7 0a50e765b7 Restructure/frontend editor (#6404)
## Move editor under `frontend/editor/`

Pure restructure: `frontend/` becomes the workspace, `frontend/editor/`
holds
  the PDF editor. 1775 file renames + 40 wiring edits. No logic changes.

  ### Why

`frontend/` is currently the editor — its `src/`, `public/`,
`src-tauri/`,
  config files all sit at the root. Promoting `frontend/` to a
workspace and putting the editor in a sibling folder leaves room for
future
apps to drop in alongside it, sharing one `package.json` /
`node_modules` /
  lint config / Storybook.

  ### What moves

  frontend/
  ├── editor/                ← NEW: everything editor-specific
  │   ├── src/               ← was frontend/src/
  │   ├── public/            ← was frontend/public/
  │   ├── src-tauri/         ← was frontend/src-tauri/
│ ├── index.html, vite.config.ts, vitest.config.ts, playwright.config.ts
  │   ├── tsconfig*.json, tailwind.config.js, postcss.config.js
  │   ├── scripts/
  │   ├── .env, .env.desktop, .env.saas
  │   └── DeveloperGuide.md
├── package.json, package-lock.json, node_modules/ ← workspace install
  ├── eslint.config.mjs, .prettierrc, .prettierignore ← shared tooling
  ├── .gitignore
  └── README.md

  ### Wiring edits (40 files)

  - `.taskfiles/frontend.yml`, `desktop.yml`, `e2e.yml`
  - `build.gradle`, `app/core/build.gradle`
- `eslint.config.mjs`, `frontend/package.json`, `.gitignore`,
`.prettierignore`
  - `docker/frontend/Dockerfile`
  - 8 `.github/workflows/*.yml`, plus `.github/dependabot.yml`,
    `.github/config/.files.yaml`, `.github/labeler-config-srvaroa.yml`
  - `scripts/translations/**`
- Docs: `AGENTS.md`, `CLAUDE.md`, `ADDING_TOOLS.md`,
`DeveloperGuide.md`,
`WINDOWS_SIGNING.md`, `devGuide/HowToAddNewLanguage.md`,
`frontend/README.md`,
    `frontend/editor/DeveloperGuide.md`

Plus 3 renamed + edited: `editor/vite.config.ts` (env path +
node_modules
  walk-up), `editor/scripts/setup-env.mts` (renamed from `.ts` for
`import.meta.url`), `editor/scripts/build-provisioner.mjs` (resolve
src-tauri
  relative to script).

  ### Verification

  | Check | Result |
  |---|---|
  | `task frontend:typecheck:all` (6 variants) | exit 0 |
  | `task frontend:lint` (eslint + dpdm) | exit 0 |
  | `task frontend:format:check` | exit 0 |
  | `task frontend:test` | 657 tests pass, 50 files |
| `task frontend:build:{core,proprietary,saas,desktop,prototypes}` | all
green |
| `task desktop:build` | full Tauri pipeline →
`Stirling-PDF_2.11.0_x64_en-US.msi` |
  | `playwright test --list --project=stubbed` | 172 tests discovered |

`task desktop:build` exercises the heaviest path — Rust + WiX + MSI
bundle
against the moved `editor/src-tauri/`. If anything in the restructure
was
  wrong it wouldn't have built.

  ### Test plan

  - [ ] `frontend-validation.yml` green
  - [ ] `e2e-stubbed.yml` green
  - [ ] `tauri-build.yml` green on at least one platform
  - [ ] `check_toml.yml` runs on a translation-touching PR

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 13:40:34 +01:00
Anthony Stirling 48027ee9d6 JDK25 - Integrate Stirling-Tools JPDFium for PDF operations (#6387) 2026-05-22 13:19:46 +01:00
Anthony StirlingandConnorYoh 22d56bb1fb test Depot runners for team commits (#6394)
Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
2026-05-21 16:54:20 +01:00
Anthony StirlingandConnorYoh 9d081d1792 SaaS Consolidation (#6384)
Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
2026-05-21 16:05:35 +01:00
089de247b4 Updated Spanish translation (#6309)
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Ludy <Ludy87@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-05-21 13:16:07 +01:00
dependabot[bot] 0c3a512b21 build(deps): bump softprops/action-gh-release from 2.6.1 to 3.0.0 (#6346)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:34:16 +01:00
dependabot[bot] 707cfa3140 build(deps): bump devalue from 5.6.4 to 5.8.1 in /frontend (#6367)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:18:43 +01:00
dependabot[bot] ea91a35518 build(deps): bump org.postgresql:postgresql from 42.7.10 to 42.7.11 (#6375)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:17:36 +01:00
dependabot[bot] 5a5aada2d8 build(deps): bump pypdf from 6.10.2 to 6.11.0 in /testing/cucumber (#6376)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:17:18 +01:00
dependabot[bot] a4ef70eaf8 build(deps): bump sigstore/cosign-installer from 4.1.1 to 4.1.2 (#6377)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:17:04 +01:00
dependabot[bot] d76d25654a build(deps): bump actions/dependency-review-action from 4.9.0 to 5.0.0 (#6378)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:16:48 +01:00
dependabot[bot] 31b00b7d06 build(deps): bump github/codeql-action from 4.35.3 to 4.35.4 (#6379)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:16:42 +01:00
dependabot[bot] f00c12ad8a build(deps): bump step-security/harden-runner from 2.19.0 to 2.19.1 (#6380)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:16:38 +01:00
dependabot[bot] 1f5c046362 build(deps): bump tokio from 1.49.0 to 1.50.0 in /frontend/src-tauri in the tokio group across 1 directory (#6296)
Bumps the tokio group with 1 update in the /frontend/src-tauri
directory: [tokio](https://github.com/tokio-rs/tokio).

Updates `tokio` from 1.49.0 to 1.50.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/tokio-rs/tokio/releases">tokio's
releases</a>.</em></p>
<blockquote>
<h2>Tokio v1.50.0</h2>
<h1>1.50.0 (Mar 3rd, 2026)</h1>
<h3>Added</h3>
<ul>
<li>net: add <code>TcpStream::set_zero_linger</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7837">#7837</a>)</li>
<li>rt: add <code>is_rt_shutdown_err</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7771">#7771</a>)</li>
</ul>
<h3>Changed</h3>
<ul>
<li>io: add optimizer hint that <code>memchr</code> returns in-bounds
pointer (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7792">#7792</a>)</li>
<li>io: implement vectored writes for <code>write_buf</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7871">#7871</a>)</li>
<li>runtime: panic when <code>event_interval</code> is set to 0 (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7838">#7838</a>)</li>
<li>runtime: shorten default thread name to fit in Linux limit (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7880">#7880</a>)</li>
<li>signal: remember the result of <code>SetConsoleCtrlHandler</code>
(<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7833">#7833</a>)</li>
<li>signal: specialize windows <code>Registry</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7885">#7885</a>)</li>
</ul>
<h3>Fixed</h3>
<ul>
<li>io: always cleanup <code>AsyncFd</code> registration list on
deregister (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7773">#7773</a>)</li>
<li>macros: remove (most) local <code>use</code> declarations in
<code>tokio::select!</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7929">#7929</a>)</li>
<li>net: fix <code>GET_BUF_SIZE</code> constant for <code>target_os =
&quot;android&quot;</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7889">#7889</a>)</li>
<li>runtime: avoid redundant unpark in current_thread scheduler (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7834">#7834</a>)</li>
<li>runtime: don't park in <code>current_thread</code> if
<code>before_park</code> defers waker (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7835">#7835</a>)</li>
<li>io: fix write readiness on ESP32 on short writes (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7872">#7872</a>)</li>
<li>runtime: wake deferred tasks before entering
<code>block_in_place</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7879">#7879</a>)</li>
<li>sync: drop rx waker when oneshot receiver is dropped (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7886">#7886</a>)</li>
<li>runtime: fix double increment of <code>num_idle_threads</code> on
shutdown (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7910">#7910</a>,
<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7918">#7918</a>,
<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7922">#7922</a>)</li>
</ul>
<h3>Unstable</h3>
<ul>
<li>fs: check for io-uring opcode support (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7815">#7815</a>)</li>
<li>runtime: avoid lock acquisition after uring init (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7850">#7850</a>)</li>
</ul>
<h3>Documented</h3>
<ul>
<li>docs: update outdated unstable features section (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7839">#7839</a>)</li>
<li>io: clarify the behavior of <code>AsyncWriteExt::shutdown()</code>
(<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7908">#7908</a>)</li>
<li>io: explain how to flush stdout/stderr (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7904">#7904</a>)</li>
<li>io: fix incorrect and confusing <code>AsyncWrite</code>
documentation (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7875">#7875</a>)</li>
<li>rt: clarify the documentation of <code>Runtime::spawn</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7803">#7803</a>)</li>
<li>rt: fix missing quotation in docs (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7925">#7925</a>)</li>
<li>runtime: correct the default thread name in docs (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7896">#7896</a>)</li>
<li>runtime: fix <code>event_interval</code> doc (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7932">#7932</a>)</li>
<li>sync: clarify RwLock fairness documentation (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7919">#7919</a>)</li>
<li>sync: clarify that <code>recv</code> returns <code>None</code> once
closed and no more messages (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7920">#7920</a>)</li>
<li>task: clarify when to use <code>spawn_blocking</code> vs dedicated
threads (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7923">#7923</a>)</li>
<li>task: doc that task drops before <code>JoinHandle</code> completion
(<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7825">#7825</a>)</li>
<li>signal: guarantee that listeners never return <code>None</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7869">#7869</a>)</li>
<li>task: fix task module feature flags in docs (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7891">#7891</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/tokio-rs/tokio/commit/0273e45ead199dac7725faee1e3dc35a9c8753ab"><code>0273e45</code></a>
chore: prepare Tokio v1.50.0 (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7934">#7934</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/e3ee4e58dc9bb7accf26dfd51b0a2146922b5269"><code>e3ee4e5</code></a>
chore: prepare tokio-macros v2.6.1 (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7943">#7943</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/8c980ea75a0f8dd2799403777db700c2e8f4cda4"><code>8c980ea</code></a>
io: add <code>write_all_vectored</code> to <code>tokio-util</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7768">#7768</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/e35fd6d6b7d9a8ba37ee621835ef91372c2565cb"><code>e35fd6d</code></a>
ci: fix patch during clippy step (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7935">#7935</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/03fe44c10302fdb55c29dbe5b08d4f8769c80272"><code>03fe44c</code></a>
runtime: fix <code>event_interval</code> doc (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7932">#7932</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/d18e5dfbb0cdc28725bebb28cde80a6c11ee32bc"><code>d18e5df</code></a>
io: fix race in <code>Mock::poll_write</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7882">#7882</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/f21f2693f02aec9a876ac2bd21566c85e15b682e"><code>f21f269</code></a>
runtime: fix race condition during the blocking pool shutdown (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7922">#7922</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/d81e8f0acbdd7d866bce4f733b3545fd834c7840"><code>d81e8f0</code></a>
macros: remove (most) local <code>use</code> declarations in
<code>tokio::select!</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7929">#7929</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/25e7f2641ef2555d688c267059431a2802805f1d"><code>25e7f26</code></a>
rt: fix missing quotation in docs (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7925">#7925</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/e1a91ef114a301b542d810abab9956f2868861b9"><code>e1a91ef</code></a>
util: fix typo in docs (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7926">#7926</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/tokio-rs/tokio/compare/tokio-1.49.0...tokio-1.50.0">compare
view</a></li>
</ul>
</details>
<br />

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-20 14:16:18 +01:00
dependabot[bot] 1787443102 build(deps): bump fast-uri from 3.0.6 to 3.1.2 in /devTools (#6332)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:15:08 +01:00
dependabot[bot] 8fb281525b build(deps): bump quinn-proto from 0.11.13 to 0.11.14 in /frontend/src-tauri (#6322)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:13:31 +01:00
Ludy 4c3be4992f deps: align Bouncy Castle modules to shared version property (#6362) 2026-05-20 14:11:43 +01:00
dependabot[bot] 2790467241 build(deps): bump urllib3 from 2.6.3 to 2.7.0 in /testing/cucumber in the pip group across 1 directory (#6352)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:11:11 +01:00
dependabot[bot] dd770dfa76 build(deps): bump rustls-webpki from 0.103.9 to 0.103.13 in /frontend/src-tauri (#6323)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:10:56 +01:00
ConnorYoh b146d9994d fix(task): make task dev / task dev:all work on Windows (#6392)
## Summary

`#6145` (port picker) and `#6244` (gradle unification) combined to break
`task dev` / `task dev:all` on Windows. Two independent regressions,
both addressed here.

### 1. `find-free-port.ps1` panic (`#6145`)

```
$ task --dry dev
The argument 'scriptsfind-free-port.ps1' to the -File parameter does not exist.
panic: ended up with a non-nil exitStatus.err but a zero exitStatus.code
```

- **Backslash stripped.** `Taskfile.yml` had
`scripts\find-free-port.ps1`. go-task pipes the `sh:` block through
mvdan/sh, which treats `\` as a POSIX escape and silently drops it,
leaving `scriptsfind-free-port.ps1`. PowerShell can't find the file,
exits non-zero, mvdan/sh panics on the inconsistent exit status.
Switched to a forward slash.
- **`-Preferred 8080,5173` is brittle.** Relies on PowerShell parsing
the comma-list into `[int[]]`. Dropped the named flag; switched the
script's `param` to `[Parameter(ValueFromRemainingArguments =
$true)][int[]]$Preferred`; pass each port as its own positional token
(matching how the bash variant is called).

### 2. `bash gradlew` can't find Java on Windows (`#6244`)

```
[backend:dev] ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
```

`#6244` unified all gradle invocations under `bash gradlew` on the
assumption that Git-Bash inherits Windows-side env. It doesn't (and
neither does WSL bash, which can also shadow `bash` on a developer's
PATH). The standard Adoptium/Temurin installer sets `JAVA_HOME` in the
Windows env only, so `bash gradlew` fails before Spring Boot even loads.

Restored the per-platform branch for every backend task: `cmd /c
".\gradlew.bat ..."` on Windows, `./gradlew ...` on Linux/macOS. Two
Windows-specific gotchas to be aware of for future edits:

- mvdan/sh strips `\` outside of double quotes, so `cmd /c
.\gradlew.bat` ends up as `.gradlew.bat`. The entire payload must be
wrapped in double quotes (`cmd /c "..."`).
- Modern Windows excludes cwd from cmd.exe's search path, so the leading
`.\` is required — bare `cmd /c gradlew.bat` errors with "is not
recognized" even from the repo root.

## Test plan

- [x] `task --dry dev` / `task --dry dev:all` on Windows resolve ports
cleanly and dispatch the inner tasks
- [x] `powershell -NoProfile -File scripts/find-free-port.ps1 8080 5173
5001` prints three ports
- [x] `task backend:dev PORT=8081` on Windows starts Spring Boot in ~9s;
`/api/v1/info/status` returns `{"status":"UP"}`
- [x] `task dev:all` on Windows brings up all three services healthy:
backend `/api/v1/info/status` 200, engine `/health` 200, frontend `/`
200
- [ ] Linux / macOS path unaffected (only the Windows branches changed
in behaviour)
2026-05-20 12:57:41 +00:00
Anthony StirlingandConnorYoh 2399da6893 migrate exportUpdatedPages from bytes to stream (#6201)
Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
2026-05-19 23:03:44 +01:00
EthanHealy01 2c15044dfa Snag/disable logo drag (#6393) 2026-05-19 18:13:58 +00:00
ConnorYoh 31f6ea4b25 perf(frontend): stabilize hot-path context subscriptions to fix excessive rerenders (#6373)
## Summary

The frontend was rerendering excessively across many interactions —
typing, clicking tools, opening modals, toggling the sidebar — because
**multiple compounding ref-instability cascades defeated `memo()` checks
in hot paths**. This PR fixes the cascades structurally.

Seven focused commits, low-to-high blast radius:

1. `perf(contexts): memoize BannerContext provider value`
2. `perf(contexts): memoize CommentAuthor and ActiveDocument provider
values`
3. `perf(contexts): memoize AppConfigContext provider value`
4. `perf(useToolManagement): stop spreading tool entries to keep refs
stable` — **root-cause fix**
5. `refactor(ToolPicker): hoist module-scope styles and helpers`
6. `feat(ToolWorkflowContext): add ref-stable Actions and Data subset
contexts` — additive
7. `perf(tools): migrate hot consumers to slim contexts and wrap in
memo()`

(Plus `style: apply prettier formatting` for CI.)

## What was wrong

Whenever something high-up in the tree caused a render, a chain of
unstable references propagated downward and forced every `ToolButton` to
re-execute its full body (hooks, derived computations, hook
subscriptions to other contexts). The chain:

- **4 unstable Context providers** (`Banner`, `CommentAuthor`,
`ActiveDocument`, `AppConfig`) were passing fresh `value={{ … }}`
objects on every render. Every consumer rerendered on every ancestor
render.
- **`useToolManagement.toolRegistry`** spread `{...baseTool, name,
description}` — a no-op spread that manufactured a new tool object
identity on every memo recompute.
- **The big `ToolWorkflowContext`** (25+ fields including
`state.searchQuery`) rebuilt its entire value on every
keystroke/click/toggle, forcing every `useToolWorkflow()` consumer (~36
files) to rerender.
- **`useToolNavigation`** transitively subscribed every `ToolButton` to
the full workflow context.
- **`ToolButton` & `ToolPicker`** weren't `memo()`-wrapped, so nothing
checked.
- **`ToolPanel`** passed inline `onSelect={(id) =>
handleToolSelect(...)}` — fresh ref every render, defeats child
memoization.
- **`ToolPicker`** allocated inline styles / `[]` / `toTitleCase` inside
the function body — churned `useToolSections`'s internal memo.

## Interaction matrix — what improves

The PR fixes the underlying ref-stability problem; the same fix benefits
*every* interaction that previously triggered the cascade:

| Interaction | Before | After |
|---|---|---|
| **Typing in tool search** | All visible buttons rerender per keystroke
| Only buttons whose matched-text changes rerender |
| **Clicking a tool** | All 36 `useToolWorkflow()` consumers rerender |
Only previously-selected and newly-selected buttons rerender (via
`isSelected` prop) |
| **Toggling sidebar / panel mode / reader mode** | Every tool button
rerenders | Tool components stay still (slim context doesn't see UI
state) |
| **Switching workbench / navigation** | `handleToolSelect` identity
changes → cascades through `onSelect` props | Ref-stabilized in Actions
context. Identity stable. Children's memo bails |
| **Modal/dialog open/close** | AppConfig churns → every `useAppConfig`
consumer rerenders (ToolButton reads `premiumEnabled`) | AppConfig
memoized; consumers rerender only when config changes |
| **Banner show/hide** | BannerProvider value churns → every consumer
rerenders on any ancestor render | Memoized; AppLayout rerenders only
when banner content changes |
| **Any state update high in the tree** | Compounding cascade defeats
memo everywhere | Stable subscriptions; memo bails out |

## Evidence

Per-keystroke prop instability on `ToolButton` (cleanest measurable
signal, captured via custom memo comparators logging which prop refs
differ):

| | `tool` ref diffs | `onSelect` ref diffs | `matchedSynonym` value
diffs | Total |
|---|---|---|---|---|
| Before | 18 | 18 | 6 | **42** |
| After  | 0  | 0  | 6 | **6 (all legitimate)** |

→ **86% reduction** in spurious per-keystroke prop instability. The 6
remaining matched-synonym diffs are correct (different substring
highlighted per keystroke).

Context value rebuild counts during a keystroke (verified with
instrumented `useMemo` factories): `useToolWorkflowData=0`,
`useToolWorkflowActions=0`, `AppConfigContext=0`.

The same stabilization applies to click/toggle/modal interactions — they
were all driven by the same cascading invalidations.

## Honest caveat on render-count metrics

`React.Profiler` counts and function-body execution counts in **dev
mode** came back identical before vs after (StrictMode + concurrent
rendering + Mantine internal commits dominate the numbers). The PR's
value is measured against the **prop-stability signal** above, not
Profiler counts. Production builds — where StrictMode doesn't
double-render and Mantine internals aren't constantly committing — will
show memo bail out properly.

## Risk × benefit

| # | Commit | Risk | Benefit |
|---|--------|------|---------|
| 1 | BannerContext memo |  Trivial | 🟦 Small |
| 2 | CommentAuthor + ActiveDocument memo |  Trivial | 🟦 Small |
| 3 | AppConfig memo |  Trivial | 🟦 Moderate (wide consumer base) |
| 4 | useToolManagement spread removal |  Trivial | 🟥 **High (root
cause)** |
| 5 | ToolPicker hoist |  Trivial | 🟦 Small |
| 6 | ToolWorkflowContext split | 🟧 Low-Med | 🟥 **High (foundation)** |
| 7 | Hot consumer migration + memo | 🟧 Low-Med | 🟥 **High
(actualization)** |

Commit 6 introduces an invariant: ref-stabilized callbacks in the
Actions context must only be invoked from event handlers (post-commit),
never during render. All current call sites comply.

## Test plan

- [x] `npx playwright test --project=stubbed` — 145 / 6 skipped / 0
failed before and after.
- [x] Targeted regression: `main-dashboard`, `tool-search`, `navigation`
— 11/11 passing.
- [x] CI passing on commits (one infrastructure flake on
`docker-compose-tests` — "No space left on device" — unrelated;
rerunning).
- [ ] Manual sanity check in a dev build after merge.

## What this enables

The same Actions + Data subset-context pattern can be applied to
`FileContext`, `NavigationContext`, and other big contexts. The
foundation is in place.
2026-05-19 15:56:50 +00:00
Anthony Stirling f9fbc37800 add translations (#6390) 2026-05-19 11:25:13 +01:00
2325 changed files with 60135 additions and 6563 deletions
+2 -2
View File
@@ -34,8 +34,8 @@ frontend/playwright-report/
# Tauri/desktop builds
src-tauri/target/
src-tauri/dist/
frontend/src-tauri/target/
frontend/src-tauri/dist/
frontend/editor/src-tauri/target/
frontend/editor/src-tauri/dist/
# IDE and editor
.idea/
+1 -1
View File
@@ -24,7 +24,7 @@ runs:
id: generate-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ inputs.app-id }}
client-id: ${{ inputs.app-id }}
private-key: ${{ inputs.private-key }}
- name: Configure Git
run: |
+9 -8
View File
@@ -8,7 +8,6 @@ openapi: &openapi
docker-base: &docker-base
- docker/base/Dockerfile
- ".github/workflows/push-docker-base.yml"
docker: &docker
- docker/embedded/Dockerfile
@@ -39,6 +38,8 @@ project: &project
- frontend/**
- docker/**
- scripts/RestartHelper.java
- scripts/db-migration/**
- .github/workflows/db-migration-test.yml
frontend: &frontend
- frontend/**
@@ -58,12 +59,12 @@ frontend: &frontend
# Files that affect the Tauri desktop bundle. Gate the multi-OS Tauri build
# job on changes to any of these.
tauri: &tauri
- frontend/src-tauri/**
- frontend/src/desktop/**
- frontend/tsconfig.desktop.json
- frontend/editor/src-tauri/**
- frontend/editor/src/desktop/**
- frontend/editor/tsconfig.desktop.vite.json
- frontend/package.json
- frontend/package-lock.json
- frontend/vite.config.ts
- frontend/editor/vite.config.ts
- .github/workflows/tauri-build.yml
# Files that affect the AI engine (Python tool models, fixers, tests). Gate
@@ -78,7 +79,7 @@ licenses-frontend: &licenses-frontend
- ".github/workflows/frontend-backend-licenses-update.yml"
- "frontend/package.json"
- "frontend/package-lock.json"
- "frontend/scripts/generate-licenses.js"
- "frontend/editor/scripts/generate-licenses.js"
licenses-backend: &licenses-backend
- ".github/workflows/frontend-backend-licenses-update.yml"
@@ -88,8 +89,8 @@ licenses-backend: &licenses-backend
# Playwright job on changes to any of these on PRs.
proprietary: &proprietary
- app/proprietary/**
- frontend/src/proprietary/**
- frontend/src/core/tests/enterprise/**
- frontend/editor/src/proprietary/**
- frontend/editor/src/core/tests/enterprise/**
- testing/compose/docker-compose-keycloak-oauth.yml
- testing/compose/docker-compose-keycloak-saml.yml
- testing/compose/keycloak-realm-oauth.json
+5 -3
View File
@@ -25,6 +25,8 @@ updates:
- "/docker/frontend"
- "/docker/base"
- "/docker/engine"
- "/docker/unoserver"
- "/engine"
schedule:
interval: "weekly"
cooldown:
@@ -126,9 +128,9 @@ updates:
- package-ecosystem: cargo
directories:
- /frontend/src-tauri
- /frontend/src-tauri/thumbnail-handler
- /frontend/src-tauri/provisioner
- /frontend/editor/src-tauri
- /frontend/editor/src-tauri/thumbnail-handler
- /frontend/editor/src-tauri/provisioner
schedule:
interval: "weekly"
cooldown:
+3 -3
View File
@@ -51,7 +51,7 @@ labels:
- label: 'Translation'
files:
- 'frontend/public/locales/[a-zA-Z]{2}-[a-zA-Z\-]{2,7}/translation.toml'
- 'frontend/editor/public/locales/[a-zA-Z]{2}-[a-zA-Z\-]{2,7}/translation.toml'
- 'scripts/ignore_translation.toml'
- 'scripts/remove_translation_keys.sh'
- 'scripts/replace_translation_line.sh'
@@ -69,8 +69,8 @@ labels:
- label: 'Tauri'
files:
- 'frontend/src-tauri/**'
- 'frontend/src-tauri/.*'
- 'frontend/editor/src-tauri/**'
- 'frontend/editor/src-tauri/.*'
- label: 'engine'
files:
+4 -3
View File
@@ -13,7 +13,7 @@ Usage:
"""
# Sample for Windows:
# python .github/scripts/check_language_toml.py --reference-file frontend/public/locales/en-GB/translation.toml --branch "" --files frontend/public/locales/de-DE/translation.toml frontend/public/locales/fr-FR/translation.toml
# python .github/scripts/check_language_toml.py --reference-file frontend/editor/public/locales/en-GB/translation.toml --branch "" --files frontend/editor/public/locales/de-DE/translation.toml frontend/editor/public/locales/fr-FR/translation.toml
import argparse
import glob
@@ -184,7 +184,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
if len(file_list) == 1:
file_arr = file_list[0].split()
base_dir = Path.cwd() / "frontend" / "public" / "locales"
base_dir = Path.cwd() / "frontend" / "editor" / "public" / "locales"
for file_path in file_arr:
file_path = Path(file_path)
@@ -308,7 +308,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
report.append("## ❌ Overall Check Status: **_Failed_**")
report.append("")
report.append(
f"@{actor} please check your translation if it conforms to the standard. Follow the format of [en-GB/translation.toml](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/frontend/public/locales/en-GB/translation.toml)"
f"@{actor} please check your translation if it conforms to the standard. Follow the format of [en-GB/translation.toml](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/frontend/editor/public/locales/en-GB/translation.toml)"
)
else:
report.append("## ✅ Overall Check Status: **_Success_**")
@@ -372,6 +372,7 @@ if __name__ == "__main__":
os.path.join(
os.getcwd(),
"frontend",
"editor",
"public",
"locales",
"*",
+37 -10
View File
@@ -23,9 +23,13 @@ permissions:
pull-requests: write
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
check-pr:
if: (github.event_name == 'pull_request' && github.event.action != 'closed') || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
outputs:
should_deploy: ${{ steps.decide.outputs.should_deploy }}
is_fork: ${{ steps.resolve.outputs.is_fork }}
@@ -35,7 +39,7 @@ jobs:
pr_ref: ${{ steps.resolve.outputs.ref }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -97,8 +101,8 @@ jobs:
echo "allow_fork=${allow_fork:-false}" >> $GITHUB_OUTPUT
deploy-v2-pr:
needs: check-pr
runs-on: ubuntu-latest
needs: [pick, check-pr]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
if: needs.check-pr.outputs.should_deploy == 'true' && (needs.check-pr.outputs.is_fork == 'false' || needs.check-pr.outputs.allow_fork == 'true')
# Concurrency control - only one deployment per PR at a time
concurrency:
@@ -108,10 +112,14 @@ jobs:
contents: read
issues: write
pull-requests: write
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -179,7 +187,12 @@ jobs:
token: ${{ secrets.GITHUB_TOKEN }}
fetch-depth: 0 # Fetch full history for commit hash detection
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Get version number
@@ -224,8 +237,20 @@ jobs:
echo "Image needs to be built"
fi
- name: Build and push V2 image
if: steps.check-image.outputs.exists == 'false'
- name: Build and push V2 image (Depot)
if: env.USE_DEPOT == 'true' && steps.check-image.outputs.exists == 'false'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/embedded/Dockerfile
push: true
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-${{ steps.commit-hash.outputs.app_short }}
build-args: VERSION_TAG=v2-alpha
platforms: linux/amd64
- name: Build and push V2 image (Docker fork fallback)
if: env.USE_DEPOT != 'true' && steps.check-image.outputs.exists == 'false'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: .
@@ -262,6 +287,7 @@ jobs:
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/data:/usr/share/tessdata:rw
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/config:/configs:rw
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/logs:/logs:rw
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/storage:/storage:rw
environment:
DISABLE_ADDITIONAL_FEATURES: "false"
SECURITY_ENABLELOGIN: "true"
@@ -284,7 +310,7 @@ jobs:
ssh -i ../private.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -T ${{ secrets.NEW_VPS_USERNAME }}@${{ secrets.NEW_VPS_HOST }} << ENDSSH
# Create V2 PR-specific directories
mkdir -p /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/{data,config,logs}
mkdir -p /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/{data,config,logs,storage}
# Move docker-compose file to correct location
mv /tmp/docker-compose-v2.yml /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/docker-compose.yml
@@ -351,7 +377,8 @@ jobs:
cleanup-v2-deployment:
if: github.event.action == 'closed'
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
contents: read
issues: write
@@ -359,7 +386,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -34,8 +34,12 @@ permissions:
pull-requests: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
check-comment:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
issues: write
if: |
@@ -70,7 +74,7 @@ jobs:
enable_prototypes: ${{ steps.check-prototypes-flag.outputs.enable_prototypes }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -175,15 +179,19 @@ jobs:
}
deploy-pr:
needs: check-comment
runs-on: ubuntu-latest
needs: [pick, check-comment]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
issues: write
pull-requests: write
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -232,7 +240,12 @@ jobs:
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
STIRLING_PDF_DESKTOP_UI: false
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Login to Docker Hub
@@ -241,7 +254,22 @@ jobs:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_API }}
- name: Build and push PR-specific image
- name: Build and push PR-specific image (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/embedded/Dockerfile
push: true
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:pr-${{ needs.check-comment.outputs.pr_number }}
build-args: |
VERSION_TAG=alpha
PROTOTYPES_BUILD=${{ needs.check-comment.outputs.enable_prototypes }}
platforms: linux/amd64
- name: Build and push PR-specific image (Docker fork fallback)
if: env.USE_DEPOT != 'true'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: .
@@ -255,8 +283,19 @@ jobs:
PROTOTYPES_BUILD=${{ needs.check-comment.outputs.enable_prototypes }}
platforms: linux/amd64
- name: Build and push engine image
if: needs.check-comment.outputs.enable_prototypes == 'true'
- name: Build and push engine image (Depot)
if: env.USE_DEPOT == 'true' && needs.check-comment.outputs.enable_prototypes == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: ./engine
file: ./engine/Dockerfile
push: true
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:engine-pr-${{ needs.check-comment.outputs.pr_number }}
platforms: linux/amd64
- name: Build and push engine image (Docker fork fallback)
if: env.USE_DEPOT != 'true' && needs.check-comment.outputs.enable_prototypes == 'true'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: ./engine
@@ -471,10 +510,11 @@ jobs:
handle-label-commands:
if: ${{ github.event.issue.pull_request != null }}
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+70
View File
@@ -0,0 +1,70 @@
name: _runner-pick
# Tiny reusable workflow that classifies the trigger as either a "fork PR
# from an untrusted contributor" or a "trusted commit" so downstream jobs
# can pick a runner class without each one duplicating the 200-char gate
# expression in their own `runs-on:`.
#
# Caller pattern:
#
# jobs:
# pick:
# uses: ./.github/workflows/_runner-pick.yml
#
# real-work:
# needs: pick
# runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-8' }}
# steps: [...]
#
# Output:
# is_fork: "true" when the trigger is a pull_request from a fork or an
# untrusted author_association, "false" otherwise.
on:
workflow_call:
outputs:
is_fork:
description: '"true" if the trigger is an untrusted fork PR.'
value: ${{ jobs.pick.outputs.is_fork }}
permissions:
contents: read
jobs:
pick:
runs-on: ubuntu-latest
timeout-minutes: 1
outputs:
is_fork: ${{ steps.decide.outputs.is_fork }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Classify the trigger
id: decide
env:
PR_NUMBER: ${{ github.event.pull_request.number }}
HEAD_REPO_FORK: ${{ github.event.pull_request.head.repo.fork }}
AUTHOR_ASSOC: ${{ github.event.pull_request.author_association }}
run: |
set -eu
if [ -z "${PR_NUMBER:-}" ]; then
# Not a pull_request event at all (push, schedule, workflow_dispatch,
# workflow_call from a non-PR trigger) -> trusted by default.
echo "is_fork=false" >> "$GITHUB_OUTPUT"
exit 0
fi
if [ "${HEAD_REPO_FORK}" = "true" ]; then
echo "is_fork=true" >> "$GITHUB_OUTPUT"
exit 0
fi
case "${AUTHOR_ASSOC}" in
OWNER|MEMBER|COLLABORATOR)
echo "is_fork=false" >> "$GITHUB_OUTPUT"
;;
*)
echo "is_fork=true" >> "$GITHUB_OUTPUT"
;;
esac
+128 -29
View File
@@ -1,8 +1,9 @@
name: AI Engine CI
# Validates the Python AI engine: regenerates tool models, runs fixers,
# lint, type-check, and tests. Called from build.yml on PRs and merge_group;
# also runs directly on push to main as a post-merge safety net.
# Validates the Python AI engine: regenerates tool models and runs the
# engine quality gate (lint, type-check, format-check, tests). Called from
# build.yml on PRs and merge_group; also runs directly on push to main as
# a post-merge safety net.
on:
workflow_call:
push:
@@ -17,14 +18,16 @@ jobs:
permissions:
contents: read
pull-requests: write
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install uv
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
@@ -49,27 +52,95 @@ jobs:
run: task engine:tool-models
- name: Verify tool models are up to date
id: tool-models-check
continue-on-error: true
run: git diff --exit-code engine/src/stirling/models/tool_models.py
- name: Comment on tool models check failure
# Only post a comment on PRs. github-script's PR helpers need an
# issue/PR number, which doesn't exist on merge_group runs.
if: steps.tool-models-check.outcome == 'failure' && github.event_name == 'pull_request'
continue-on-error: true
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const marker = '<!-- tool-models-check -->';
const body = [
marker,
'### Tool Models Check Failed',
'',
'The generated `engine/src/stirling/models/tool_models.py` is out of date with the Java OpenAPI spec and will need to be regenerated before it can be merged in.',
'',
'Run `task engine:tool-models` to regenerate, then commit the updated file.',
].join('\n');
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body.includes(marker));
if (existing) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
body,
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body,
});
}
- name: Fail if tool models check failed
if: steps.tool-models-check.outcome == 'failure'
run: |
if ! git diff --exit-code engine/src/stirling/models/tool_models.py; then
echo "tool_models.py is out of date."
echo "Run 'task engine:tool-models' locally and commit the updated file."
exit 1
fi
echo "============================================"
echo " Tool Models Check Failed"
echo "============================================"
echo ""
echo "The generated engine/src/stirling/models/tool_models.py"
echo "is out of date with the Java OpenAPI spec and will"
echo "need to be regenerated before it can be merged in."
echo ""
echo "Run 'task engine:tool-models' to regenerate, then"
echo "commit the updated file."
echo "============================================"
exit 1
- name: Run fixers
run: task engine:fix
- name: Remove tool models check comment on success
if: steps.tool-models-check.outcome == 'success' && github.event_name == 'pull_request'
continue-on-error: true
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const marker = '<!-- tool-models-check -->';
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body.includes(marker));
if (existing) {
await github.rest.issues.deleteComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
});
}
- name: Verify fixes are committed
id: fixer_changes
run: |
if ! git diff --quiet; then
git --no-pager diff --stat
echo "::error::There are issues with your Python code that will need to be fixed before they can be merged in. Run 'task engine:fix' to auto-fix what can be fixed automatically, then run 'task engine:check' to see what still needs fixing manually."
exit 1
fi
- name: Quality-check engine
id: engine-check
run: task engine:check
continue-on-error: true
- name: Comment on fixer failures
if: steps.fixer_changes.outcome == 'failure' && github.event_name == 'pull_request'
- name: Comment on engine check failure
# Only post a comment on PRs. github-script's PR helpers need an
# issue/PR number, which doesn't exist on merge_group runs.
if: steps.engine-check.outcome == 'failure' && github.event_name == 'pull_request'
continue-on-error: true
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
@@ -105,11 +176,39 @@ jobs:
});
}
- name: Run linting
run: task engine:lint
- name: Fail if engine check failed
if: steps.engine-check.outcome == 'failure'
run: |
echo "============================================"
echo " Engine Check Failed"
echo "============================================"
echo ""
echo "There are issues with your Python code that"
echo "will need to be fixed before they can be merged in."
echo ""
echo "Run 'task engine:fix' to auto-fix what can be"
echo "fixed automatically, then run 'task engine:check'"
echo "to see what still needs fixing manually."
echo "============================================"
exit 1
- name: Run type checking
run: task engine:typecheck
- name: Run tests
run: task engine:test
- name: Remove engine check comment on success
if: steps.engine-check.outcome == 'success' && github.event_name == 'pull_request'
continue-on-error: true
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const marker = '<!-- engine-check -->';
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body.includes(marker));
if (existing) {
await github.rest.issues.deleteComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
});
}
+2 -2
View File
@@ -19,7 +19,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -87,7 +87,7 @@ jobs:
- name: AI PR Title Analysis
if: steps.actor.outputs.is_repo_dev == 'true'
id: ai-title-analysis
uses: actions/ai-inference@e09e65981758de8b2fdab13c2bfb7c7d5493b0b6 # v2.0.7
uses: actions/ai-inference@17ff458cb182449bbb2e43701fcd98f6af8f6570 # v2.1.0
with:
model: openai/gpt-4o
system-prompt-file: ".github/config/system-prompt.txt"
+4 -4
View File
@@ -26,7 +26,7 @@ jobs:
jar_sha256: ${{ steps.hashes.outputs.jar_sha256 }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -70,12 +70,12 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository (for PKGBUILD templates)
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Update stirling-pdf-desktop PKGBUILD
env:
@@ -106,7 +106,7 @@ jobs:
- name: Publish stirling-pdf-desktop to AUR
if: ${{ github.event_name == 'release' || inputs.dry_run == false }}
uses: KSXGitHub/github-actions-deploy-aur@v4.1.3
uses: KSXGitHub/github-actions-deploy-aur@da03e160361ce01bf087e790b6ffd196d7dccff7 # v4.1.3
with:
pkgname: stirling-pdf-desktop
pkgbuild: .github/aur/stirling-pdf-desktop/PKGBUILD
+1 -1
View File
@@ -16,7 +16,7 @@ jobs:
pull-requests: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+43 -20
View File
@@ -1,7 +1,7 @@
name: Backend build, format check, and coverage
# Reusable workflow called from build.yml. Runs the full backend build matrix
# (JDK 21/25 × spring-security on/off), Spotless formatting check, JUnit, and
# Reusable workflow called from build.yml. Runs the backend build matrix
# (JDK 25 × spring-security on/off), Spotless formatting check, JUnit, and
# posts Jacoco coverage to PRs.
on:
workflow_call:
@@ -13,16 +13,22 @@ permissions:
pull-requests: write
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
build:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-8' }}
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
strategy:
fail-fast: false
matrix:
jdk-version: [21, 25]
jdk-version: [25]
spring-security: [true, false]
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
@@ -61,7 +67,7 @@ jobs:
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
- name: Comment on Java formatting failure
- name: Comment on backend format check failure
# Only post a comment on PRs. github-script's PR helpers need an
# issue/PR number, which doesn't exist on merge_group runs.
if: steps.spotless-check.outcome == 'failure' && github.event_name == 'pull_request'
@@ -72,15 +78,11 @@ jobs:
const marker = '<!-- java-formatting-check -->';
const body = [
marker,
'### Java Formatting Check Failed',
'### Backend Format Check Failed',
'',
'Your code has formatting issues. Run the following command to fix them:',
'There are formatting issues in your Java code that will need to be fixed before they can be merged in.',
'',
'```bash',
'task backend:format',
'```',
'',
'Then commit and push the changes.',
'Run `task backend:format` to auto-fix, then commit and push the changes.',
].join('\n');
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
@@ -104,22 +106,43 @@ jobs:
});
}
- name: Fail if Java formatting issues found
- name: Fail if backend format check failed
if: steps.spotless-check.outcome == 'failure'
run: |
echo "============================================"
echo " Java Formatting Check Failed"
echo " Backend Format Check Failed"
echo "============================================"
echo ""
echo "Your code has formatting issues."
echo "Run the following command to fix them:"
echo "There are formatting issues in your Java code"
echo "that will need to be fixed before they can be"
echo "merged in."
echo ""
echo " task backend:format"
echo ""
echo "Then commit and push the changes."
echo "Run 'task backend:format' to auto-fix, then"
echo "commit and push the changes."
echo "============================================"
exit 1
- name: Remove backend format check comment on success
if: steps.spotless-check.outcome == 'success' && github.event_name == 'pull_request'
continue-on-error: true
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const marker = '<!-- java-formatting-check -->';
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body.includes(marker));
if (existing) {
await github.rest.issues.deleteComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
});
}
- name: Build with Gradle and spring security ${{ matrix.spring-security }}
run: task backend:build:ci
env:
+20 -3
View File
@@ -17,11 +17,23 @@ name: Enterprise E2E (Playwright)
on:
workflow_call:
inputs:
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
required: false
type: string
default: "8"
push:
branches: ["main"]
schedule:
- cron: "0 4 * * *"
workflow_dispatch:
inputs:
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
required: false
type: string
default: "8"
# No `concurrency:` block here on purpose. When this workflow is called via
# workflow_call from build.yml, ${{ github.workflow }}/event_name/pr_number
@@ -35,16 +47,21 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
playwright-e2e-enterprise:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
timeout-minutes: 45
env:
PREMIUM_KEY: ${{ secrets.PREMIUM_KEY_ENTERPRISE }}
PREMIUM_ENABLED: "true"
SYSTEM_ENABLEANALYTICS: "false"
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
@@ -268,5 +285,5 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: playwright-report-enterprise-${{ github.run_id }}
path: frontend/playwright-report/
path: frontend/editor/playwright-report/
retention-days: 7
+21 -1
View File
@@ -46,7 +46,7 @@ jobs:
proprietary: ${{ steps.changes.outputs.proprietary }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
@@ -68,6 +68,18 @@ jobs:
uses: ./.github/workflows/backend-build.yml
secrets: inherit
db-migration-test:
# Boots the current bootJar against H2 fixtures captured from past
# releases (v2.0.0 / v2.5.0 / v2.10.0) and verifies admin login still
# works after Hibernate's ddl-auto=update migrates the schema. Gated on
# the `project` filter so doc-only PRs skip this ~5-minute job.
if: needs.files-changed.outputs.project == 'true'
needs: [files-changed]
permissions:
contents: read
uses: ./.github/workflows/db-migration-test.yml
secrets: inherit
check-generateOpenApiDocs:
if: needs.files-changed.outputs.openapi == 'true'
needs: [files-changed]
@@ -135,6 +147,7 @@ jobs:
permissions:
contents: read
packages: read
id-token: write
uses: ./.github/workflows/test-build-docker.yml
secrets: inherit
with:
@@ -183,6 +196,7 @@ jobs:
needs:
- files-changed
- build
- db-migration-test
- check-generateOpenApiDocs
- frontend-validation
- playwright-e2e
@@ -197,11 +211,17 @@ jobs:
- dependency-review
runs-on: ubuntu-latest
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Verify every required job passed (or was legitimately skipped)
env:
RESULTS: |
files-changed=${{ needs.files-changed.result }}
build=${{ needs.build.result }}
db-migration-test=${{ needs.db-migration-test.result }}
check-generateOpenApiDocs=${{ needs.check-generateOpenApiDocs.result }}
frontend-validation=${{ needs.frontend-validation.result }}
playwright-e2e=${{ needs.playwright-e2e.result }}
+3 -1
View File
@@ -11,9 +11,11 @@ permissions:
jobs:
check-licence:
runs-on: ubuntu-latest
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+8 -2
View File
@@ -10,11 +10,17 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
check-generate-openapi-docs:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+6 -6
View File
@@ -6,7 +6,7 @@ on:
pull_request_target:
types: [opened, synchronize, reopened]
paths:
- "frontend/public/locales/*/translation.toml"
- "frontend/editor/public/locales/*/translation.toml"
- ".github/scripts/check_language_toml.py"
- ".github/workflows/check_toml.yml"
@@ -27,7 +27,7 @@ jobs:
pull-requests: write # Allow writing to pull requests
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -76,7 +76,7 @@ jobs:
exit 1
fi
# Get changed files and filter for TOML translation files
gh pr view ${{ steps.get-pr-data.outputs.pr_number }} --json files -q ".files[].path" | grep -E '^frontend/public/locales/[a-zA-Z-]+/translation\.toml$' > changed_files.txt || echo "No matching TOML files found in PR"
gh pr view ${{ steps.get-pr-data.outputs.pr_number }} --json files -q ".files[].path" | grep -E '^frontend/editor/public/locales/[a-zA-Z-]+/translation\.toml$' > changed_files.txt || echo "No matching TOML files found in PR"
# Check if any files were found
if [ ! -s changed_files.txt ]; then
echo "No TOML translation files changed in this PR"
@@ -166,12 +166,12 @@ jobs:
// Determine reference file
let referenceFilePath;
if (changedFiles.includes("frontend/public/locales/en-GB/translation.toml")) {
if (changedFiles.includes("frontend/editor/public/locales/en-GB/translation.toml")) {
console.log("Using PR branch reference file.");
const { data: fileContent } = await github.rest.repos.getContent({
owner: prRepoOwner,
repo: prRepoName,
path: "frontend/public/locales/en-GB/translation.toml",
path: "frontend/editor/public/locales/en-GB/translation.toml",
ref: branch,
});
@@ -183,7 +183,7 @@ jobs:
const { data: fileContent } = await github.rest.repos.getContent({
owner: repoOwner,
repo: repoName,
path: "frontend/public/locales/en-GB/translation.toml",
path: "frontend/editor/public/locales/en-GB/translation.toml",
ref: "main",
});
+93
View File
@@ -0,0 +1,93 @@
name: DB migration smoke test
# Boots the current Stirling-PDF JAR against H2 fixtures captured from past
# releases (v2.0.0 / v2.5.0 / v2.10.0) and verifies admin login still works.
# Catches schema changes that would break existing user databases under
# Hibernate's `ddl-auto=update` upgrade path.
on:
workflow_call:
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
migration-test:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-8' }}
timeout-minutes: 30
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up JDK 25
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
with:
java-version: 25
distribution: temurin
- name: Cache Gradle dependency artifacts
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: |
~/.gradle/wrapper
~/.gradle/caches/modules-2/files-2.1
~/.gradle/caches/modules-2/metadata-2.*
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
cache-disabled: true
# No `-PnoSpotless` here yet because the upstream cache layer matches the
# backend build's; reuse keeps cold-cache cost identical.
- name: Build Stirling-PDF JAR
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
run: ./gradlew :stirling-pdf:bootJar -PnoSpotless --no-daemon
- name: Locate built JAR
id: jar
run: |
jar=$(find app/core/build/libs -maxdepth 1 -name 'Stirling-PDF*.jar' -o -name 'stirling-pdf*.jar' 2>/dev/null \
| grep -vE '(-plain|-sources)\.jar$' | head -n 1)
if [[ -z "$jar" ]]; then
echo "::error::No JAR under app/core/build/libs"
ls -lah app/core/build/libs || true
exit 1
fi
# Absolute path - the migration script pushd's into a temp workdir
# before invoking java, which would dangle a relative path.
jar=$(realpath "$jar")
echo "path=$jar" >> "$GITHUB_OUTPUT"
echo "Built JAR: $jar"
- name: Run migration smoke test
env:
STIRLING_JAR: ${{ steps.jar.outputs.path }}
run: bash scripts/db-migration/run-migration-test.sh
- name: Upload app logs on failure
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: db-migration-app-logs
# Path matches the preserved workdir in run-migration-test.sh -
# only failing fixtures leave a directory behind.
path: /tmp/stirling-migration-failed-*/app.log
retention-days: 7
if-no-files-found: warn
+2 -2
View File
@@ -14,13 +14,13 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: "Checkout Repository"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: "Dependency Review"
uses: actions/dependency-review-action@2031cfc080254a8a887f58cffee85186f0e49e48 # v4.9.0
uses: actions/dependency-review-action@a1d282b36b6f3519aa1f3fc636f609c47dddb294 # v5.0.0
with:
config-file: "./.github/config/dependency-review-config.yml"
+49 -6
View File
@@ -10,22 +10,37 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
deploy-v2-on-push:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
concurrency:
group: deploy-v2-push-V2
cancel-in-progress: true
permissions:
contents: read
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Get commit hashes for frontend and backend
@@ -90,8 +105,22 @@ jobs:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_API }}
- name: Build and push frontend image
if: steps.check-frontend.outputs.exists == 'false'
- name: Build and push frontend image (Depot)
if: env.USE_DEPOT == 'true' && steps.check-frontend.outputs.exists == 'false'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/frontend/Dockerfile
push: true
tags: |
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-frontend-${{ steps.commit-hashes.outputs.frontend_short }}
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-frontend-latest
build-args: VERSION_TAG=v2-alpha
platforms: linux/amd64
- name: Build and push frontend image (Docker fork fallback)
if: env.USE_DEPOT != 'true' && steps.check-frontend.outputs.exists == 'false'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: .
@@ -105,8 +134,22 @@ jobs:
build-args: VERSION_TAG=v2-alpha
platforms: linux/amd64
- name: Build and push backend image
if: steps.check-backend.outputs.exists == 'false'
- name: Build and push backend image (Depot)
if: env.USE_DEPOT == 'true' && steps.check-backend.outputs.exists == 'false'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/backend/Dockerfile
push: true
tags: |
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-backend-${{ steps.commit-hashes.outputs.backend_short }}
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-backend-latest
build-args: VERSION_TAG=v2-alpha
platforms: linux/amd64
- name: Build and push backend image (Docker fork fallback)
if: env.USE_DEPOT != 'true' && steps.check-backend.outputs.exists == 'false'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: .
+13 -2
View File
@@ -11,21 +11,32 @@ on:
required: false
type: string
default: "false"
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking. Tuned to 4 because bench showed 16 was within noise of 4."
required: false
type: string
default: "4"
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
docker-compose-tests:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '4') }}
permissions:
actions: write
contents: read
checks: write
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+13 -3
View File
@@ -5,17 +5,27 @@ name: Playwright E2E (live backend)
# server.
on:
workflow_call:
inputs:
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
required: false
type: string
default: "8"
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
playwright-e2e-live:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
timeout-minutes: 30
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
@@ -60,5 +70,5 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: playwright-report-live-${{ github.run_id }}
path: frontend/playwright-report/
path: frontend/editor/playwright-report/
retention-days: 7
+13 -3
View File
@@ -5,16 +5,26 @@ name: Playwright E2E (stubbed)
# mocks API responses in the browser.
on:
workflow_call:
inputs:
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking. Tuned to 8 to match the other playwright workflows; bench showed flat scaling above 8."
required: false
type: string
default: "8"
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
playwright-e2e:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
@@ -40,5 +50,5 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: playwright-report-stubbed-${{ github.run_id }}
path: frontend/playwright-report/
path: frontend/editor/playwright-report/
retention-days: 7
@@ -19,16 +19,20 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
files-changed:
name: detect what files changed
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
timeout-minutes: 3
outputs:
licenses-frontend: ${{ steps.changes.outputs.licenses-frontend }}
licenses-backend: ${{ steps.changes.outputs.licenses-backend }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -44,15 +48,15 @@ jobs:
generate-frontend-license-report:
if: needs.files-changed.outputs.licenses-frontend == 'true'
name: Generate Frontend License Report
needs: files-changed
runs-on: ubuntu-latest
needs: [pick, files-changed]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
contents: write
pull-requests: write
repository-projects: write # Required for enabling automerge
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -114,23 +118,23 @@ jobs:
env:
PR_IS_FORK: "true"
run: |
node base/frontend/scripts/generate-licenses.js \
--input frontend/src/assets/3rdPartyLicenses.json
node base/frontend/editor/scripts/generate-licenses.js \
--input frontend/editor/src/assets/3rdPartyLicenses.json
- name: Copy postprocessed artifacts back (fork PRs)
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == true
run: |
mkdir -p frontend/src/assets
if [ -f "base/frontend/src/assets/3rdPartyLicenses.json" ]; then
cp base/frontend/src/assets/3rdPartyLicenses.json frontend/src/assets/3rdPartyLicenses.json
mkdir -p frontend/editor/src/assets
if [ -f "base/frontend/editor/src/assets/3rdPartyLicenses.json" ]; then
cp base/frontend/editor/src/assets/3rdPartyLicenses.json frontend/editor/src/assets/3rdPartyLicenses.json
fi
if [ -f "base/frontend/src/assets/license-warnings.json" ]; then
cp base/frontend/src/assets/license-warnings.json frontend/src/assets/license-warnings.json
if [ -f "base/frontend/editor/src/assets/license-warnings.json" ]; then
cp base/frontend/editor/src/assets/license-warnings.json frontend/editor/src/assets/license-warnings.json
fi
- name: Check for license warnings
run: |
if [ -f "frontend/src/assets/license-warnings.json" ]; then
if [ -f "frontend/editor/src/assets/license-warnings.json" ]; then
echo "LICENSE_WARNINGS_EXIST=true" >> $GITHUB_ENV
else
echo "LICENSE_WARNINGS_EXIST=false" >> $GITHUB_ENV
@@ -178,10 +182,10 @@ jobs:
echo ""
if [ "${LICENSE_WARNINGS_EXIST}" = "true" ]; then
echo "❌ **Failed** incompatible or unknown licenses found."
if [ -f "frontend/src/assets/license-warnings.json" ]; then
if [ -f "frontend/editor/src/assets/license-warnings.json" ]; then
echo ""
echo "### Warnings"
jq -r '.warnings[] | "- \(.message)"' frontend/src/assets/license-warnings.json || true
jq -r '.warnings[] | "- \(.message)"' frontend/editor/src/assets/license-warnings.json || true
fi
else
echo "✅ **Passed** no license warnings detected."
@@ -207,7 +211,7 @@ jobs:
const fs = require('fs');
let warningDetails = '';
try {
const warnings = JSON.parse(fs.readFileSync('frontend/src/assets/license-warnings.json', 'utf8'));
const warnings = JSON.parse(fs.readFileSync('frontend/editor/src/assets/license-warnings.json', 'utf8'));
warningDetails = warnings.warnings.map(w => `- ${w.message}`).join('\n');
} catch (e) {
warningDetails = 'Unable to read warning details';
@@ -247,7 +251,7 @@ jobs:
- name: Commit changes (Push only)
if: github.event_name == 'push'
run: |
git add frontend/src/assets/3rdPartyLicenses.json
git add frontend/editor/src/assets/3rdPartyLicenses.json
# Note: Do NOT commit license-warnings.json - it's only for PR review
git diff --staged --quiet || echo "CHANGES_DETECTED=true" >> $GITHUB_ENV
@@ -265,7 +269,7 @@ jobs:
The following licenses may require review for corporate compatibility:
$(cat frontend/src/assets/license-warnings.json | jq -r '.warnings[].message')
$(cat frontend/editor/src/assets/license-warnings.json | jq -r '.warnings[].message')
Please review these licenses to ensure they are acceptable for your use case."
fi
@@ -307,16 +311,18 @@ jobs:
generate-backend-license-report:
if: needs.files-changed.outputs.licenses-backend == 'true'
needs: files-changed
needs: [pick, files-changed]
name: Generate Backend License Report
runs-on: ubuntu-latest
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
contents: write
pull-requests: write
repository-projects: write # Required for enabling automerge
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+7 -3
View File
@@ -11,11 +11,15 @@ permissions:
pull-requests: write
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
frontend-validation:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
@@ -109,5 +113,5 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: frontend-build
path: frontend/dist/
path: frontend/editor/dist/
retention-days: 3
+1 -1
View File
@@ -15,7 +15,7 @@ jobs:
issues: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+43 -28
View File
@@ -36,15 +36,19 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
determine-matrix:
if: ${{ vars.CI_PROFILE != 'lite' }}
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
version: ${{ steps.versionNumber.outputs.versionNumber }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -90,26 +94,28 @@ jobs:
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
case "${{ github.event.inputs.platform }}" in
"windows")
echo 'matrix={"include":[{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64"}]}' >> $GITHUB_OUTPUT
echo 'matrix={"include":[{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64","jpdfium_platforms":"windows-x64"}]}' >> $GITHUB_OUTPUT
;;
"macos")
echo 'matrix={"include":[{"platform":"macos-15","args":"--target universal-apple-darwin","name":"macos-universal"}]}' >> $GITHUB_OUTPUT
echo 'matrix={"include":[{"platform":"macos-15","args":"--target universal-apple-darwin","name":"macos-universal","jpdfium_platforms":"darwin-arm64,darwin-x64"}]}' >> $GITHUB_OUTPUT
;;
"linux")
echo 'matrix={"include":[{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64"}]}' >> $GITHUB_OUTPUT
echo 'matrix={"include":[{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64","jpdfium_platforms":"linux-x64"}]}' >> $GITHUB_OUTPUT
;;
*)
echo 'matrix={"include":[{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64"},{"platform":"macos-15","args":"--target universal-apple-darwin","name":"macos-universal"},{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64"}]}' >> $GITHUB_OUTPUT
echo 'matrix={"include":[{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64","jpdfium_platforms":"windows-x64"},{"platform":"macos-15","args":"--target universal-apple-darwin","name":"macos-universal","jpdfium_platforms":"darwin-arm64,darwin-x64"},{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64","jpdfium_platforms":"linux-x64"}]}' >> $GITHUB_OUTPUT
;;
esac
else
# For push/release events, build all platforms
echo 'matrix={"include":[{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64"},{"platform":"macos-15","args":"--target universal-apple-darwin","name":"macos-universal"},{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64"}]}' >> $GITHUB_OUTPUT
echo 'matrix={"include":[{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64","jpdfium_platforms":"windows-x64"},{"platform":"macos-15","args":"--target universal-apple-darwin","name":"macos-universal","jpdfium_platforms":"darwin-arm64,darwin-x64"},{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64","jpdfium_platforms":"linux-x64"}]}' >> $GITHUB_OUTPUT
fi
build-jars:
needs: determine-matrix
runs-on: ubuntu-latest
needs: [pick, determine-matrix]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
strategy:
matrix:
variant:
@@ -127,7 +133,7 @@ jobs:
file_suffix: "-server"
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -189,9 +195,10 @@ jobs:
SM_API_KEY: ${{ secrets.SM_API_KEY }}
WINDOWS_CERTIFICATE: ${{ secrets.WINDOWS_CERTIFICATE }}
RELEASE_GPG_PRIVATE_KEY: ${{ secrets.RELEASE_GPG_PRIVATE_KEY }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
allowed-endpoints: >
@@ -256,15 +263,17 @@ jobs:
if: matrix.platform == 'macos-15'
env:
AARCH64_JAVA_HOME: ${{ env.JAVA_HOME }}
JPDFIUM_PLATFORMS: ${{ matrix.jpdfium_platforms }}
run: task desktop:jlink:universal-mac
- name: Prepare desktop build
run: task desktop:prepare
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
DISABLE_ADDITIONAL_FEATURES: true
JPDFIUM_PLATFORMS: ${{ matrix.jpdfium_platforms }}
run: task desktop:prepare
# DigiCert KeyLocker Setup (Cloud HSM)
- name: Setup DigiCert KeyLocker
@@ -418,7 +427,7 @@ jobs:
env:
KEYPAIR_ALIAS: ${{ secrets.SM_KEYPAIR_ALIAS }}
run: |
cat > ./frontend/src-tauri/tauri.windows.conf.json <<EOF
cat > ./frontend/editor/src-tauri/tauri.windows.conf.json <<EOF
{
"bundle": {
"windows": {
@@ -431,7 +440,11 @@ jobs:
}
EOF
echo "Generated tauri.windows.conf.json (alias masked):"
sed "s/${KEYPAIR_ALIAS}/***/g" ./frontend/src-tauri/tauri.windows.conf.json
sed "s/${KEYPAIR_ALIAS}/***/g" ./frontend/editor/src-tauri/tauri.windows.conf.json
- name: Sign JPDFium dylibs inside bootJar (macOS only)
if: matrix.platform == 'macos-15' && env.APPLE_CERTIFICATE != ''
run: bash frontend/scripts/sign-jpdfium-dylibs-in-bootjar.sh
- name: Import release GPG signing key (Linux)
if: matrix.platform == 'ubuntu-22.04' && env.RELEASE_GPG_PRIVATE_KEY != '' && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master')
@@ -479,7 +492,7 @@ jobs:
SM_CODE_SIGNING_CERT_SHA1_HASH: ${{ secrets.SM_CODE_SIGNING_CERT_SHA1_HASH }}
CI: true
with:
projectPath: ./frontend
projectPath: ./frontend/editor
tauriScript: npx tauri
args: ${{ matrix.args }}
@@ -505,7 +518,7 @@ jobs:
$allSigned = $true
# Check MSI installer (outer wrapper - what users download)
$msiFiles = Get-ChildItem -Path "./frontend/src-tauri/target" -Filter "*.msi" -Recurse -File
$msiFiles = Get-ChildItem -Path "./frontend/editor/src-tauri/target" -Filter "*.msi" -Recurse -File
if ($msiFiles.Count -eq 0) {
Write-Host "[ERROR] No MSI found under target/"
exit 1
@@ -573,21 +586,23 @@ jobs:
if: always() && steps.digicert-setup.conclusion != 'failure'
shell: bash
run: |
mkdir -p ./dist
cd ./frontend/src-tauri/target
# Absolute dist path so the cd below can't break the copy targets.
DIST="$GITHUB_WORKSPACE/dist"
mkdir -p "$DIST"
cd ./frontend/editor/src-tauri/target
# Find and rename artifacts based on platform
if [ "${{ matrix.platform }}" = "windows-latest" ]; then
# Only ship the MSI installer on Windows. The loose exe and WiX toolset exes
# are not the user-facing installer - the MSI contains the signed inner exe.
find . -name "*.msi" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.msi" \;
find . -name "*.msi" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.msi" \;
elif [ "${{ matrix.platform }}" = "macos-15" ]; then
find . -name "*.dmg" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.dmg" \;
find . -name "*.app" -exec cp -r {} "../../../dist/Stirling-PDF-${{ matrix.name }}.app" \;
find . -name "*.dmg" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.dmg" \;
find . -name "*.app" -exec cp -r {} "$DIST/Stirling-PDF-${{ matrix.name }}.app" \;
else
find . -name "*.deb" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.rpm" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.AppImage" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.AppImage" \;
find . -name "*.deb" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.rpm" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.AppImage" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.AppImage" \;
fi
- name: Upload build artifacts
@@ -600,13 +615,13 @@ jobs:
create-release:
if: (github.event_name == 'workflow_dispatch' && github.event.inputs.test_mode != 'true') || github.event_name == 'release' || github.ref == 'refs/heads/V2-master'
needs: [determine-matrix, build, build-jars]
runs-on: ubuntu-latest
needs: [pick, determine-matrix, build, build-jars]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
contents: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -638,7 +653,7 @@ jobs:
run: ls -R ./artifacts
- name: Upload binaries to Release
uses: softprops/action-gh-release@153bb8e04406b158c6c84fc1615b65b24149a1fe # v2.6.1
uses: softprops/action-gh-release@b4309332981a82ec1c5618f44dd2e27cc8bfbfda # v3.0.0
with:
tag_name: v${{ needs.determine-matrix.outputs.version }}
generate_release_notes: true
+7 -3
View File
@@ -13,12 +13,16 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
playwright-all-browsers:
name: Playwright (chromium + firefox + webkit)
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -45,5 +49,5 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: playwright-nightly-${{ github.run_id }}
path: frontend/playwright-report/
path: frontend/editor/playwright-report/
retention-days: 14
+4 -4
View File
@@ -2,7 +2,7 @@ name: Update Package Manager Manifests
on:
release:
types: [ released ]
types: [released]
workflow_dispatch:
inputs:
version:
@@ -28,7 +28,7 @@ jobs:
jar_sha256: ${{ steps.hashes.outputs.jar_sha256 }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -79,12 +79,12 @@ jobs:
contents: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout homebrew-stirling-pdf tap (also hosts Scoop bucket)
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: Stirling-Tools/homebrew-stirling-pdf
token: ${{ secrets.HOMEBREW_TAP_TOKEN }}
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
PIP_DISABLE_PIP_VERSION_CHECK: "1"
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+2 -2
View File
@@ -43,7 +43,7 @@ jobs:
echo "version=${VERSION}" >> $GITHUB_OUTPUT
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -100,7 +100,7 @@ jobs:
sbom: true
- name: Install cosign
uses: sigstore/cosign-installer@cad07c2e89fa2edd6e2d7bab4c1aa38e53f76003 # v4.1.1
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
with:
cosign-release: "v2.4.1"
+3 -3
View File
@@ -53,7 +53,7 @@ jobs:
RUN_UNOSERVER: ${{ github.event_name != 'workflow_dispatch' || inputs.build_unoserver }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -96,13 +96,13 @@ jobs:
- name: Install cosign
if: github.ref == 'refs/heads/master' || github.ref == 'refs/heads/V2-master'
uses: sigstore/cosign-installer@cad07c2e89fa2edd6e2d7bab4c1aa38e53f76003 # v4.1.1
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
with:
cosign-release: "v2.4.1"
- name: Install cosign
if: github.ref == 'refs/heads/master' || github.ref == 'refs/heads/V2-master'
uses: sigstore/cosign-installer@cad07c2e89fa2edd6e2d7bab4c1aa38e53f76003 # v4.1.1
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
with:
cosign-release: "v2.4.1"
+1 -1
View File
@@ -18,7 +18,7 @@ jobs:
packages: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+2 -2
View File
@@ -35,7 +35,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -75,6 +75,6 @@ jobs:
# Upload the results to GitHub's code scanning dashboard.
- name: "Upload to code-scanning"
uses: github/codeql-action/upload-sarif@e46ed2cbd01164d986452f91f178727624ae40d7 # v3.29.5
uses: github/codeql-action/upload-sarif@9e0d7b8d25671d64c341c19c0152d693099fb5ba # v3.29.5
with:
sarif_file: results.sarif
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
pull-requests: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+8 -2
View File
@@ -22,12 +22,18 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
push:
if: ${{ vars.CI_PROFILE != 'lite' }}
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+6 -6
View File
@@ -11,7 +11,7 @@ on:
- "app/core/build.gradle"
- "app/proprietary/build.gradle"
- "README.md"
- "frontend/public/locales/*/translation.toml"
- "frontend/editor/public/locales/*/translation.toml"
- "app/core/src/main/resources/static/3rdPartyLicenses.json"
- "scripts/ignore_translation.toml"
@@ -35,7 +35,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -62,7 +62,7 @@ jobs:
- name: Sync translation TOML files
run: |
python .github/scripts/check_language_toml.py --reference-file "frontend/public/locales/en-GB/translation.toml" --branch main
python .github/scripts/check_language_toml.py --reference-file "frontend/editor/public/locales/en-GB/translation.toml" --branch main
- name: pre-commit run
run: |
@@ -70,7 +70,7 @@ jobs:
- name: Commit translation files
run: |
git add frontend/public/locales/*/translation.toml
git add frontend/editor/public/locales/*/translation.toml
git diff --staged --quiet || git commit -m ":memo: Sync translation files (TOML)" || echo "No changes detected"
- name: Sync README.md
@@ -100,7 +100,7 @@ jobs:
This Pull Request was automatically generated to synchronize updates to translation files and documentation. Below are the details of the changes made:
#### **1. Synchronization of Translation Files**
- Updated translation files (`frontend/public/locales/*/translation.toml`) to reflect changes in the reference file `en-GB/translation.toml`.
- Updated translation files (`frontend/editor/public/locales/*/translation.toml`) to reflect changes in the reference file `en-GB/translation.toml`.
- Ensured consistency and synchronization across all supported language files.
- Highlighted any missing or incomplete translations.
- **Format**: TOML
@@ -125,5 +125,5 @@ jobs:
sign-commits: true
add-paths: |
README.md
frontend/public/locales/*/translation.toml
frontend/editor/public/locales/*/translation.toml
scripts/ignore_translation.toml
+37 -29
View File
@@ -4,6 +4,10 @@ name: Build Tauri Applications
# Linux). Called from build.yml on PRs that touch desktop sources (gated
# via the `tauri` filter in .github/config/.files.yaml). Also runnable
# on demand via workflow_dispatch with a per-platform selector.
#
# Note: editing this file is itself enough to make the `tauri` path filter
# match, which is how non-desktop PRs (e.g. backend-only fixes) opt into a
# desktop smoke build.
on:
workflow_call:
inputs:
@@ -37,7 +41,7 @@ jobs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -47,12 +51,10 @@ jobs:
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
PLATFORM: ${{ inputs.platform }}
run: |
WINDOWS='{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64"}'
MACOS='{"platform":"macos-15","args":"--target universal-apple-darwin","name":"macos-universal"}'
LINUX='{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64"}'
WINDOWS='{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64","jpdfium_platforms":"windows-x64"}'
MACOS='{"platform":"macos-15","args":"--target universal-apple-darwin","name":"macos-universal","jpdfium_platforms":"darwin-arm64,darwin-x64"}'
LINUX='{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64","jpdfium_platforms":"linux-x64"}'
# Resolve requested platform — populated by either workflow_dispatch
# or workflow_call inputs; both paths default to "all".
case "$PLATFORM" in
windows) ENTRIES=("$WINDOWS") ;;
macos) ENTRIES=("$MACOS") ;;
@@ -84,9 +86,10 @@ jobs:
WINDOWS_CERTIFICATE: ${{ secrets.WINDOWS_CERTIFICATE }}
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
RELEASE_GPG_PRIVATE_KEY: ${{ secrets.RELEASE_GPG_PRIVATE_KEY }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -112,10 +115,6 @@ jobs:
toolchain: stable
targets: ${{ matrix.platform == 'macos-15' && 'aarch64-apple-darwin,x86_64-apple-darwin' || '' }}
# x86_64 JDK is set up first so the aarch64 step below can leave its
# JAVA_HOME as the active one. The macOS universal JRE build needs
# jmods from both arches; the x64 path is captured into the env
# before the second setup-java overwrites JAVA_HOME.
- name: Set up x86_64 JDK 25 (macOS universal JRE)
if: matrix.platform == 'macos-15'
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
@@ -142,21 +141,24 @@ jobs:
- name: Setup Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
# Build the universal JRE before desktop:prepare so the jlink:runtime
# task short-circuits on its `test -d runtime/jre` status check.
- name: Build universal macOS JRE
if: matrix.platform == 'macos-15'
env:
AARCH64_JAVA_HOME: ${{ env.JAVA_HOME }}
JPDFIUM_PLATFORMS: ${{ matrix.jpdfium_platforms }}
run: task desktop:jlink:universal-mac
- name: Prepare desktop build
run: task desktop:prepare
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
DISABLE_ADDITIONAL_FEATURES: true
JPDFIUM_PLATFORMS: ${{ matrix.jpdfium_platforms }}
run: task desktop:prepare
- name: Run Tauri/Cargo tests
run: task desktop:test
# DigiCert KeyLocker Setup (Cloud HSM)
- name: Setup DigiCert KeyLocker
@@ -269,6 +271,10 @@ jobs:
echo "APPLE_SIGNING_IDENTITY=$CERT_ID" >> $GITHUB_ENV
echo "Certificate imported successfully."
- name: Sign JPDFium dylibs inside bootJar (macOS only)
if: matrix.platform == 'macos-15' && env.APPLE_CERTIFICATE != ''
run: bash frontend/scripts/sign-jpdfium-dylibs-in-bootjar.sh
- name: Check DMG creation dependencies (macOS only)
if: matrix.platform == 'macos-15'
run: |
@@ -299,7 +305,7 @@ jobs:
env:
KEYPAIR_ALIAS: ${{ secrets.SM_KEYPAIR_ALIAS }}
run: |
cat > ./frontend/src-tauri/tauri.windows.conf.json <<EOF
cat > ./frontend/editor/src-tauri/tauri.windows.conf.json <<EOF
{
"bundle": {
"windows": {
@@ -357,7 +363,7 @@ jobs:
SM_CODE_SIGNING_CERT_SHA1_HASH: ${{ secrets.SM_CODE_SIGNING_CERT_SHA1_HASH }}
CI: true
with:
projectPath: ./frontend
projectPath: ./frontend/editor
tauriScript: npx tauri
# Linux: build deb+rpm only here. AppImage runs in its own
# continue-on-error step below so its persistent linuxdeploy
@@ -382,7 +388,7 @@ jobs:
VITE_SAAS_BACKEND_API_URL: ${{ secrets.VITE_SAAS_BACKEND_API_URL || 'https://api.stirlingpdf.com' }}
CI: true
with:
projectPath: ./frontend
projectPath: ./frontend/editor
tauriScript: npx tauri
args: --bundles appimage
@@ -400,7 +406,7 @@ jobs:
if: matrix.platform == 'macos-15'
run: |
echo "🔍 Verifying notarization status..."
cd ./frontend/src-tauri/target
cd ./frontend/editor/src-tauri/target
DMG_FILE=$(find . -name "*.dmg" | head -1)
if [ -n "$DMG_FILE" ]; then
echo "Found DMG: $DMG_FILE"
@@ -414,20 +420,22 @@ jobs:
- name: Rename artifacts
shell: bash
run: |
mkdir -p ./dist
cd ./frontend/src-tauri/target
# Absolute dist path so the cd below can't break the copy targets.
DIST="$GITHUB_WORKSPACE/dist"
mkdir -p "$DIST"
cd ./frontend/editor/src-tauri/target
# Find and rename artifacts based on platform
if [ "${{ matrix.platform }}" = "windows-latest" ]; then
# Only ship the MSI installer. The loose exe and WiX toolset exes
# are not the user-facing installer - the MSI contains the signed inner exe.
find . -name "*.msi" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.msi" \;
find . -name "*.msi" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.msi" \;
elif [ "${{ matrix.platform }}" = "macos-15" ]; then
find . -name "*.dmg" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.dmg" \;
find . -name "*.dmg" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.dmg" \;
else
find . -name "*.deb" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.rpm" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.AppImage" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.AppImage" \;
find . -name "*.deb" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.rpm" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.AppImage" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.AppImage" \;
fi
# Verify the MSI AND the inner exe extracted from it are signed.
@@ -505,7 +513,7 @@ jobs:
- name: Verify build artifacts
shell: bash
run: |
cd ./frontend/src-tauri/target
cd ./frontend/editor/src-tauri/target
# Check for expected artifacts based on platform
if [ "${{ matrix.platform }}" = "windows-latest" ]; then
@@ -536,7 +544,7 @@ jobs:
- name: Test artifact sizes
shell: bash
run: |
cd ./frontend/src-tauri/target
cd ./frontend/editor/src-tauri/target
echo "Artifact sizes for ${{ matrix.name }}:"
find . -name "*.exe" -o -name "*.dmg" -o -name "*.deb" -o -name "*.rpm" -o -name "*.AppImage" -o -name "*.msi" | while read file; do
if [ -f "$file" ]; then
@@ -557,7 +565,7 @@ jobs:
pull-requests: write
steps:
- name: Harden the runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -648,7 +656,7 @@ jobs:
if: always()
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+70 -6
View File
@@ -12,11 +12,19 @@ on:
required: false
type: string
default: "false"
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
required: false
type: string
default: "8"
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
# TODO: extract a pre-matrix `prepare` job that runs once and produces
# shared artifacts for the three matrix entries below to consume:
# 1. `task backend:build` — currently runs 3× in parallel with
@@ -32,7 +40,14 @@ jobs:
# spring-security=true matrix entry if `task backend:build` and
# `task backend:build:ci` produce equivalent JARs (verify before wiring).
test-build-docker-images:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
permissions:
contents: read
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' && inputs.docker-base-changed != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
strategy:
fail-fast: false
matrix:
@@ -48,7 +63,7 @@ jobs:
cache-scope: stirling-pdf-fat
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -105,10 +120,16 @@ jobs:
DISABLE_ADDITIONAL_FEATURES: true
STIRLING_PDF_DESKTOP_UI: false
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up QEMU
if: env.USE_DEPOT != 'true'
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
id: buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
@@ -134,7 +155,22 @@ jobs:
echo "platforms=linux/amd64,linux/arm64/v8" >> "$GITHUB_OUTPUT"
fi
- name: Build ${{ matrix.docker-rev }}
- name: Build ${{ matrix.docker-rev }} (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./${{ matrix.docker-rev }}
push: false
platforms: ${{ steps.build-params.outputs.platforms }}
build-args: |
BASE_IMAGE=${{ steps.build-params.outputs.base_image }}
provenance: true
sbom: true
- name: Build ${{ matrix.docker-rev }} (Docker fork fallback)
if: env.USE_DEPOT != 'true'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
builder: ${{ steps.buildx.outputs.name }}
@@ -162,24 +198,52 @@ jobs:
if-no-files-found: warn
test-build-unoserver-image:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
permissions:
contents: read
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' && inputs.docker-base-changed != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout Repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up QEMU
if: env.USE_DEPOT != 'true'
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
id: buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Build docker/unoserver/Dockerfile
- name: Build docker/unoserver/Dockerfile (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/unoserver/Dockerfile
push: false
load: true
platforms: linux/amd64
tags: stirling-unoserver:pr-test
provenance: false
sbom: false
- name: Build docker/unoserver/Dockerfile (Docker fork fallback)
if: env.USE_DEPOT != 'true'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
builder: ${{ steps.buildx.outputs.name }}
+40 -12
View File
@@ -20,12 +20,22 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
deploy:
if: ${{ vars.CI_PROFILE != 'lite' }}
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
contents: read
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -51,7 +61,12 @@ jobs:
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
DISABLE_ADDITIONAL_FEATURES: true
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Get version number
@@ -66,7 +81,20 @@ jobs:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_API }}
- name: Build and push test image
- name: Build and push test image (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/embedded/Dockerfile
push: true
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:test-${{ github.sha }}
build-args: VERSION_TAG=${{ steps.versionNumber.outputs.versionNumber }}
platforms: linux/amd64
- name: Build and push test image (Docker fork fallback)
if: env.USE_DEPOT != 'true'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: .
@@ -125,13 +153,14 @@ jobs:
files-changed:
if: always()
name: detect what files changed
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
timeout-minutes: 3
outputs:
frontend: ${{ steps.changes.outputs.frontend }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -145,12 +174,11 @@ jobs:
test:
if: needs.files-changed.outputs.frontend == 'true'
needs: [deploy, files-changed]
runs-on: ubuntu-latest
needs: [pick, deploy, files-changed]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -180,13 +208,13 @@ jobs:
FORCE_COLOR: "3"
cleanup:
needs: [deploy, test]
runs-on: ubuntu-latest
needs: [pick, deploy, test]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
if: always()
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+10 -1
View File
@@ -23,6 +23,10 @@ customFiles/
configs/
watchedFolders/
clientWebUI/
# Scratch dir used by local fixture-regeneration runs (see
# app/proprietary/src/test/resources/db-migration-fixtures/README.md).
# Holds downloaded JARs and disposable workdirs. Never committed.
.alpha-local/
!cucumber/
!cucumber/exampleFiles/
!cucumber/exampleFiles/example_html.zip
@@ -157,7 +161,7 @@ app/proprietary/build
common/build
proprietary/build
stirling-pdf/build
frontend/src-tauri/provisioner/target
frontend/editor/src-tauri/provisioner/target
# Byte-compiled / optimized / DLL files
__pycache__/
@@ -265,9 +269,14 @@ docs/type3/signatures/
# Type3 sample PDFs (development only)
**/type3/samples/
**/application-dev-local.properties
# Claude
.claude/
# Playwright MCP screenshots / traces
.playwright-mcp/
*.playwright-mcp.png
# Local screenshot artifacts from *-screenshots.spec.ts
frontend/screenshots/
+3 -3
View File
@@ -1,6 +1,6 @@
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.11
rev: v0.15.14
hooks:
- id: ruff
args:
@@ -20,7 +20,7 @@ repos:
- --skip="./.*,*.csv,*.json,*.ambr"
- --quiet-level=2
files: \.(html|css|js|py|md)$
exclude: (.vscode|.devcontainer|app/core/src/main/resources|app/proprietary/src/main/resources|frontend/public/vendor|Dockerfile|.*/pdfjs.*|.*/thirdParty.*|bootstrap.*|.*\.min\..*|.*diff\.js)
exclude: (.vscode|.devcontainer|app/core/src/main/resources|app/proprietary/src/main/resources|frontend/editor/public/vendor|Dockerfile|.*/pdfjs.*|.*/thirdParty.*|bootstrap.*|.*\.min\..*|.*diff\.js)
- repo: https://github.com/gitleaks/gitleaks
rev: v8.30.0
hooks:
@@ -38,7 +38,7 @@ repos:
rev: v0.24.4
hooks:
- id: toml-sort-fix
files: frontend/public/locales/.*\.toml$
files: frontend/editor/public/locales/.*\.toml$
args: ['--in-place', '--all', '--ignore-case']
# - repo: https://github.com/thibaudcolas/pre-commit-stylelint
# rev: v16.21.1
+86 -15
View File
@@ -1,7 +1,19 @@
version: '3'
# All gradle invocations go through `bash gradlew` (the sh wrapper) so they work
# uniformly on Linux/macOS (native bash) and Windows-with-Git-Bash.
# Gradle invocation strategy:
# - Linux/macOS: `./gradlew` runs the POSIX shell wrapper natively.
# - Windows: `cmd /c ".\gradlew.bat ..."` invokes the .bat wrapper through
# cmd.exe so it inherits the user's Windows-side `JAVA_HOME`
# and PATH. Routing through `bash gradlew` on Windows ends up
# under WSL or Git-Bash, neither of which inherits
# Adoptium/Temurin's default Windows-only Java env - gradlew
# then errors with "JAVA_HOME is not set and no 'java' command
# could be found".
#
# The entire `.\gradlew.bat ...` payload is double-quoted so
# the leading `.\` survives mvdan/sh's POSIX backslash
# stripping; cmd.exe also requires `.\` (not bare `gradlew.bat`)
# because modern Windows excludes cwd from cmd's search path.
tasks:
dev:
@@ -10,46 +22,90 @@ tasks:
vars:
PORT: '{{.PORT | default "8080"}}'
AIENGINE_URL: '{{.AIENGINE_URL | default ""}}'
AIENGINE_TIMEOUTSECONDS: '{{.AIENGINE_TIMEOUTSECONDS | default "120"}}'
env:
SERVER_PORT: '{{.PORT}}'
cmds:
- '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED=true {{end}}bash gradlew :stirling-pdf:bootRun'
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED=true AIENGINE_TIMEOUTSECONDS={{.AIENGINE_TIMEOUTSECONDS}} {{end}}cmd /c ".\gradlew.bat :stirling-pdf:bootRun"'
platforms: [windows]
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED=true AIENGINE_TIMEOUTSECONDS={{.AIENGINE_TIMEOUTSECONDS}} {{end}}./gradlew :stirling-pdf:bootRun'
platforms: [linux, darwin]
dev:bundled:
desc: "Clean + bootRun with frontend bundled into the backend (single :8080 server)"
ignore_error: true
cmds:
- bash gradlew clean bootRun -PbuildWithFrontend=true
- cmd: cmd /c ".\gradlew.bat clean bootRun -PbuildWithFrontend=true"
platforms: [windows]
- cmd: ./gradlew clean bootRun -PbuildWithFrontend=true
platforms: [linux, darwin]
dev:saas:
desc: "Start backend in SaaS flavor against Supabase"
# `dotenv:` reads from the root Taskfile's directory (".") because this
# subtaskfile is included with `dir: .`.
dotenv: ['app/.env.saas.local', 'app/.env.saas']
ignore_error: true
vars:
PORT: '{{.PORT | default "8080"}}'
# Override to "" to run the pure `saas` profile against your own SAAS_DB_*.
PROFILES: '{{.PROFILES | default "dev"}}'
env:
SERVER_PORT: '{{.PORT}}'
STIRLING_FLAVOR: saas
cmds:
- cmd: cmd /c ".\gradlew.bat :stirling-pdf:bootRun {{if .PROFILES}}--args=\"--spring.profiles.include={{.PROFILES}}\"{{end}}"
platforms: [windows]
- cmd: ./gradlew :stirling-pdf:bootRun {{if .PROFILES}}--args='--spring.profiles.include={{.PROFILES}}'{{end}}
platforms: [linux, darwin]
build:
desc: "Full backend build"
cmds:
- bash gradlew clean build
- cmd: cmd /c ".\gradlew.bat clean build"
platforms: [windows]
- cmd: ./gradlew clean build
platforms: [linux, darwin]
build:fast:
desc: "Build without tests"
cmds:
- bash gradlew clean build -x test
- cmd: cmd /c ".\gradlew.bat clean build -x test"
platforms: [windows]
- cmd: ./gradlew clean build -x test
platforms: [linux, darwin]
build:ci:
desc: "Build for CI (formatting checked separately)"
cmds:
- bash gradlew build -PnoSpotless
- cmd: cmd /c ".\gradlew.bat build -PnoSpotless"
platforms: [windows]
- cmd: ./gradlew build -PnoSpotless
platforms: [linux, darwin]
test:
desc: "Run backend tests"
cmds:
- bash gradlew test
- cmd: cmd /c ".\gradlew.bat test"
platforms: [windows]
- cmd: ./gradlew test
platforms: [linux, darwin]
format:
desc: "Auto-fix code formatting"
cmds:
- bash gradlew spotlessApply
- cmd: cmd /c ".\gradlew.bat spotlessApply"
platforms: [windows]
- cmd: ./gradlew spotlessApply
platforms: [linux, darwin]
format:check:
desc: "Check code formatting"
cmds:
- bash gradlew spotlessCheck
- cmd: cmd /c ".\gradlew.bat spotlessCheck"
platforms: [windows]
- cmd: ./gradlew spotlessCheck
platforms: [linux, darwin]
fix:
desc: "Auto-fix backend"
@@ -59,7 +115,10 @@ tasks:
swagger:
desc: "Generate OpenAPI docs"
cmds:
- bash gradlew :stirling-pdf:copySwaggerDoc
- cmd: cmd /c ".\gradlew.bat :stirling-pdf:copySwaggerDoc"
platforms: [windows]
- cmd: ./gradlew :stirling-pdf:copySwaggerDoc
platforms: [linux, darwin]
sources:
- app/core/src/main/java/**/*.java
- app/proprietary/src/main/java/**/*.java
@@ -77,19 +136,31 @@ tasks:
desc: "Print project version"
silent: true
cmds:
- bash gradlew printVersion --quiet | tail -1
- cmd: cmd /c ".\gradlew.bat printVersion --quiet" | tail -1
platforms: [windows]
- cmd: ./gradlew printVersion --quiet | tail -1
platforms: [linux, darwin]
licenses:check:
desc: "Check dependency licenses"
cmds:
- bash gradlew checkLicense --no-parallel
- cmd: cmd /c ".\gradlew.bat checkLicense --no-parallel"
platforms: [windows]
- cmd: ./gradlew checkLicense --no-parallel
platforms: [linux, darwin]
licenses:generate:
desc: "Check and generate dependency license report"
cmds:
- bash gradlew checkLicense generateLicenseReport --no-parallel
- cmd: cmd /c ".\gradlew.bat checkLicense generateLicenseReport --no-parallel"
platforms: [windows]
- cmd: ./gradlew checkLicense generateLicenseReport --no-parallel
platforms: [linux, darwin]
clean:
desc: "Clean build artifacts"
cmds:
- bash gradlew clean
- cmd: cmd /c ".\gradlew.bat clean"
platforms: [windows]
- cmd: ./gradlew clean
platforms: [linux, darwin]
+67 -11
View File
@@ -3,6 +3,22 @@ version: '3'
vars:
JLINK_MODULES: "java.base,java.compiler,java.desktop,java.instrument,java.logging,java.management,java.naming,java.net.http,java.prefs,java.rmi,java.scripting,java.security.jgss,java.security.sasl,java.sql,java.transaction.xa,java.xml,java.xml.crypto,jdk.crypto.ec,jdk.crypto.cryptoki,jdk.unsupported"
# Override via JPDFIUM_PLATFORMS env (csv of platform keys, or 'all').
JPDFIUM_PLATFORMS:
sh: |
if [ -n "${JPDFIUM_PLATFORMS:-}" ]; then
echo "$JPDFIUM_PLATFORMS"
else
case "{{OS}}-{{ARCH}}" in
darwin-arm64) echo "darwin-arm64";;
darwin-amd64) echo "darwin-x64";;
linux-amd64) echo "linux-x64";;
linux-arm64) echo "linux-arm64";;
windows-amd64) echo "windows-x64";;
*) echo "all";;
esac
fi
tasks:
prepare:
desc: "Prepare desktop build dependencies"
@@ -15,6 +31,7 @@ tasks:
provisioner:
desc: "Build installer provisioner"
platforms: [windows]
dir: editor
cmds:
- node scripts/build-provisioner.mjs
@@ -22,41 +39,55 @@ tasks:
desc: "Start Tauri desktop dev mode"
deps: [prepare]
ignore_error: true
dir: editor
cmds:
- npx tauri dev --no-watch
build:
desc: "Build Tauri desktop app (production)"
deps: [prepare]
dir: editor
cmds:
- npx tauri build
build:dev:
desc: "Build Tauri desktop app (dev, no bundling)"
deps: [prepare]
dir: editor
cmds:
- npx tauri build --no-bundle
build:dev:mac:
desc: "Build Tauri desktop .app bundle (macOS)"
deps: [prepare]
dir: editor
cmds:
- npx tauri build --bundles app
build:dev:windows:
desc: "Build Tauri desktop NSIS installer (Windows)"
deps: [prepare]
dir: editor
cmds:
- npx tauri build --bundles nsis
build:dev:linux:
desc: "Build Tauri desktop AppImage (Linux)"
deps: [prepare]
dir: editor
cmds:
- npx tauri build --bundles appimage
test:
desc: "Run Tauri/Cargo tests"
deps: [prepare]
dir: editor/src-tauri
cmds:
- cargo test
clean:
desc: "Clean Tauri/Cargo build artifacts"
dir: editor
cmds:
- task: jlink:clean
- cd src-tauri && cargo clean
@@ -71,27 +102,29 @@ tasks:
deps: [jlink:jar, jlink:runtime]
jlink:jar:
desc: "Build backend JAR for Tauri bundling"
desc: "Build backend JAR for Tauri bundling (host-OS natives only by default)"
run: once
dir: ..
env:
DISABLE_ADDITIONAL_FEATURES: "true"
cmds:
- cmd: cmd /c gradlew.bat bootJar --no-daemon
- echo "Building bootJar with JPDFium natives for {{.JPDFIUM_PLATFORMS}}"
- cmd: cmd /c gradlew.bat bootJar --no-daemon -PjpdfiumPlatforms={{.JPDFIUM_PLATFORMS}}
platforms: [windows]
- cmd: ./gradlew bootJar --no-daemon
- cmd: ./gradlew bootJar --no-daemon -PjpdfiumPlatforms={{.JPDFIUM_PLATFORMS}}
platforms: [linux, darwin]
- mkdir -p frontend/src-tauri/libs
- cp app/core/build/libs/stirling-pdf-*.jar frontend/src-tauri/libs/
- mkdir -p frontend/editor/src-tauri/libs
- cp app/core/build/libs/stirling-pdf-*.jar frontend/editor/src-tauri/libs/
status:
- test -f frontend/src-tauri/libs/stirling-pdf-*.jar
- test -f frontend/editor/src-tauri/libs/stirling-pdf-*.jar
jlink:runtime:
desc: "Create custom JRE with jlink"
deps: [jlink:jar]
dir: editor/src-tauri
cmds:
- rm -rf src-tauri/runtime/jre
- mkdir -p src-tauri/runtime
- rm -rf runtime/jre
- mkdir -p runtime
- >-
jlink
--add-modules {{.JLINK_MODULES}}
@@ -99,14 +132,36 @@ tasks:
--compress=zip-6
--no-header-files
--no-man-pages
--output src-tauri/runtime/jre
--output runtime/jre
# jlink emits its files mode 444 (read-only). Tauri's build-script
# resource copier preserves source permissions when staging
# `runtime/jre/**/*` into `target/<profile>/runtime/jre/...`, so the
# staged copies are read-only too. On any subsequent incremental
# build the copier tries to overwrite them and fails with a bare
# `Permission denied (os error 13)` (Rust's io::Error Display drops
# the path, so the failure is opaque). Make the source writable here
# so the staged destinations are writable and can be overwritten.
#
# Trade-off: this task runs for both `task desktop:dev` and
# `task desktop:build`, so production bundles also ship mode-644
# JRE files instead of 444. Functionally harmless on POSIX (the
# `other` bit is `r--` either way, and on macOS code signing is the
# real integrity check) and on Windows the DOS read-only attribute
# isn't load-bearing for the bundled JDK. If we ever need strict
# 444 in production, split the chmod into a dev-only step and have
# `desktop:build` run `jlink:clean` first to force a fresh build.
- cmd: chmod -R u+w runtime/jre
platforms: [linux, darwin]
- cmd: powershell -NoProfile -Command "Get-ChildItem -Recurse runtime/jre | ForEach-Object { $_.IsReadOnly = $false }"
platforms: [windows]
status:
- test -d src-tauri/runtime/jre
- test -f runtime/jre/release
jlink:clean:
desc: "Remove JLink runtime and bundled JARs"
dir: editor/src-tauri
cmds:
- rm -rf src-tauri/libs src-tauri/runtime
- rm -rf libs runtime
# macOS-only. Replaces jlink:runtime's single-arch JRE with a universal
# (arm64 + x86_64) one for the universal Tauri shell. Runs the x86_64
@@ -117,6 +172,7 @@ tasks:
desc: "Create universal (arm64+x86_64) JRE for the macOS Tauri build"
deps: [jlink:jar]
platforms: [darwin]
dir: editor
env:
JLINK_MODULES: "{{.JLINK_MODULES}}"
OUTPUT_DIR: src-tauri/runtime/jre
+5 -5
View File
@@ -3,14 +3,14 @@ version: '3'
tasks:
install:
desc: "Install Playwright browsers"
dir: frontend
dir: frontend/editor
deps: [ ':frontend:install' ]
cmds:
- npx playwright install {{.CLI_ARGS}} --with-deps
stubbed:
desc: "Run stubbed E2E tests"
dir: frontend
dir: frontend/editor
deps: [ ':frontend:prepare' ]
cmds:
- npx playwright test --project=stubbed {{.CLI_ARGS}}
@@ -72,7 +72,7 @@ tasks:
live:runner:
internal: true
deps: [ ':frontend:prepare' ]
dir: frontend
dir: frontend/editor
vars:
BASE_DIR: '{{.ROOT_DIR}}/.test-state/playwright'
cmds:
@@ -141,14 +141,14 @@ tasks:
up first with:
task e2e:oauth:up
task e2e:saml:up
dir: frontend
dir: frontend/editor
deps: [ ':frontend:prepare' ]
cmds:
- npx playwright test --project=enterprise {{.CLI_ARGS}}
cross-browser:
desc: "Run stubbed E2E tests on Chromium, Firefox, and WebKit"
dir: frontend
dir: frontend/editor
deps: [ ':frontend:prepare' ]
cmds:
- npx playwright test --project=stubbed --project=stubbed-firefox --project=stubbed-webkit {{.CLI_ARGS}}
+28 -23
View File
@@ -1,5 +1,10 @@
version: '3'
# Tasks operate from the workspace root (frontend/). Editor commands pass
# `editor` as the vite project root (positional after `build` / before the
# mode flag) or use `--project editor/...` for tsc — so the editor lives
# under frontend/editor/ without each task needing a cd.
tasks:
install:
desc: "Install dependencies"
@@ -21,19 +26,19 @@ tasks:
vars:
MODE: '{{.MODE | default ""}}'
cmds:
- npx tsx scripts/setup-env.ts{{if .MODE}} --{{.MODE}}{{end}}
- npx tsx editor/scripts/setup-env.mts{{if .MODE}} --{{.MODE}}{{end}}
sources:
- scripts/setup-env.ts
- editor/scripts/setup-env.mts
generates:
- .env.local
- .env{{if .MODE}}.{{.MODE}}{{end}}.local
- editor/.env.local
- editor/.env{{if .MODE}}.{{.MODE}}{{end}}.local
prepare:icons:
internal: true
run: once
deps: [install]
cmds:
- node scripts/generate-icons.js
- node editor/scripts/generate-icons.js
prepare:
desc: "Set up dev environment"
@@ -60,7 +65,7 @@ tasks:
env:
BACKEND_URL: '{{.BACKEND_URL}}'
cmds:
- npx vite --mode {{.MODE}} --port {{.PORT}}{{if .OPEN}} --open{{end}}
- npx vite editor --mode {{.MODE}} --port {{.PORT}}{{if .OPEN}} --open{{end}}
dev:
desc: "Start frontend dev server"
@@ -115,19 +120,19 @@ tasks:
desc: "Production build (default mode)"
deps: [prepare]
cmds:
- npx vite build
- npx vite build editor
build:core:
desc: "Build for core mode"
deps: [prepare]
cmds:
- npx vite build --mode core
- npx vite build editor --mode core
build:proprietary:
desc: "Build for proprietary mode"
deps: [prepare]
cmds:
- npx vite build --mode proprietary
- npx vite build editor --mode proprietary
build:saas:
desc: "Build for SaaS mode"
@@ -135,7 +140,7 @@ tasks:
- task: prepare
vars: { MODE: saas }
cmds:
- npx vite build --mode saas
- npx vite build editor --mode saas
build:desktop:
desc: "Build for desktop mode"
@@ -143,13 +148,13 @@ tasks:
- task: prepare
vars: { MODE: desktop }
cmds:
- npx vite build --mode desktop
- npx vite build editor --mode desktop
build:prototypes:
desc: "Build for prototypes mode"
deps: [prepare]
cmds:
- npx vite build --mode prototypes
- npx vite build editor --mode prototypes
# ============================================================
# Code quality
@@ -160,7 +165,7 @@ tasks:
deps: [install]
cmds:
- npx eslint --max-warnings=0
- npx dpdm src --circular --no-warning --no-tree --exit-code circular:1
- npx dpdm editor/src --circular --no-warning --no-tree --exit-code circular:1
lint:fix:
desc: "Auto-fix lint issues"
@@ -195,13 +200,13 @@ tasks:
desc: "Typecheck core build variant"
deps: [prepare]
cmds:
- npx tsc --noEmit --project src/core/tsconfig.json
- npx tsc --noEmit --project editor/src/core/tsconfig.json
typecheck:proprietary:
desc: "Typecheck proprietary build variant"
deps: [prepare]
cmds:
- npx tsc --noEmit --project src/proprietary/tsconfig.json
- npx tsc --noEmit --project editor/src/proprietary/tsconfig.json
typecheck:saas:
desc: "Typecheck SaaS build variant"
@@ -209,7 +214,7 @@ tasks:
- task: prepare
vars: { MODE: saas }
cmds:
- npx tsc --noEmit --project src/saas/tsconfig.json
- npx tsc --noEmit --project editor/src/saas/tsconfig.json
typecheck:desktop:
desc: "Typecheck desktop build variant"
@@ -217,19 +222,19 @@ tasks:
- task: prepare
vars: { MODE: desktop }
cmds:
- npx tsc --noEmit --project src/desktop/tsconfig.json
- npx tsc --noEmit --project editor/src/desktop/tsconfig.json
typecheck:scripts:
desc: "Typecheck scripts"
deps: [prepare]
cmds:
- npx tsc --noEmit --project scripts/tsconfig.json
- npx tsc --noEmit --project editor/scripts/tsconfig.json
typecheck:prototypes:
desc: "Typecheck prototypes build variant"
deps: [prepare]
cmds:
- npx tsc --noEmit --project src/prototypes/tsconfig.json
- npx tsc --noEmit --project editor/src/prototypes/tsconfig.json
typecheck:all:
desc: "Typecheck all build variants"
@@ -270,19 +275,19 @@ tasks:
desc: "Run tests"
deps: [install]
cmds:
- npx vitest run
- npx vitest run --root editor
test:watch:
desc: "Run tests in watch mode"
deps: [install]
cmds:
- npx vitest --watch
- npx vitest --watch --root editor
test:coverage:
desc: "Run tests with coverage"
deps: [install]
cmds:
- npx vitest --coverage
- npx vitest --coverage --root editor
# ============================================================
# Code Generation
@@ -292,4 +297,4 @@ tasks:
desc: "Generate frontend license report"
deps: [install]
cmds:
- node scripts/generate-licenses.js
- node editor/scripts/generate-licenses.js
+6 -6
View File
@@ -10,14 +10,14 @@ When adding tools, follow this systematic approach using the established pattern
Create these files in the correct directories:
```
frontend/src/hooks/tools/[toolName]/
frontend/editor/src/hooks/tools/[toolName]/
├── use[ToolName]Parameters.ts # Parameter definitions and validation
└── use[ToolName]Operation.ts # Tool operation logic using useToolOperation
frontend/src/components/tools/[toolName]/
frontend/editor/src/components/tools/[toolName]/
└── [ToolName]Settings.tsx # Settings UI component (if needed)
frontend/src/tools/
frontend/editor/src/tools/
└── [ToolName].tsx # Main tool component
```
@@ -128,7 +128,7 @@ export default [ToolName] as ToolComponent;
## 3. Register Tool in System
Update these files to register your new tool:
**Tool Registry** (`frontend/src/data/useTranslatedToolRegistry.tsx`):
**Tool Registry** (`frontend/editor/src/data/useTranslatedToolRegistry.tsx`):
1. Add imports at the top:
```typescript
import [ToolName] from "../tools/[ToolName]";
@@ -155,7 +155,7 @@ import [ToolName]Settings from "../components/tools/[toolName]/[ToolName]Setting
## 4. Add Tooltips (Optional but Recommended)
Create user-friendly tooltips to help non-technical users understand your tool. **Use simple, clear language - avoid technical jargon:**
**Tooltip Hook** (`frontend/src/components/tooltips/use[ToolName]Tips.ts`):
**Tooltip Hook** (`frontend/editor/src/components/tooltips/use[ToolName]Tips.ts`):
```typescript
import { useTranslation } from 'react-i18next';
import { TooltipContent } from '../../types/tips';
@@ -202,7 +202,7 @@ const [ToolName] = (props: BaseToolProps) => {
## 5. Add Translations
Update translation files. **Important: Only update `en-GB` files** - other languages are handled separately.
**File to update:** `frontend/public/locales/en-GB/translation.toml`
**File to update:** `frontend/editor/public/locales/en-GB/translation.toml`
**Required Translation Keys**:
```toml
+39 -16
View File
@@ -139,9 +139,9 @@ The project structure is defined in `engine/pyproject.toml`. Any new dependencie
#### Environment Variables
- All `VITE_*` variables must be declared in the appropriate committed env file:
- `frontend/.env` — core, proprietary, and shared vars
- `frontend/.env.saas` — SaaS-only vars (layered on top of `.env` in SaaS mode)
- `frontend/.env.desktop` — desktop (Tauri)-only vars (layered on top of `.env` in desktop mode)
- `frontend/editor/.env` — core, proprietary, and shared vars
- `frontend/editor/.env.saas` — SaaS-only vars (layered on top of `.env` in SaaS mode)
- `frontend/editor/.env.desktop` — desktop (Tauri)-only vars (layered on top of `.env` in desktop mode)
- These files are committed to Git and must not contain private keys
- Local overrides (API keys, machine-specific settings) go in uncommitted sibling `.env.local` / `.env.saas.local` / `.env.desktop.local` files — Vite automatically layers them on top
- Never use `|| 'hardcoded-fallback'` inline — put defaults in the committed env files
@@ -152,7 +152,7 @@ The project structure is defined in `engine/pyproject.toml`. Any new dependencie
#### Import Paths - CRITICAL
**ALWAYS use `@app/*` for imports.** Do not use `@core/*` or `@proprietary/*` unless explicitly wrapping/extending a lower layer implementation.
For a broader explanation of the frontend layering and override architecture, see [frontend/DeveloperGuide.md](frontend/DeveloperGuide.md).
For a broader explanation of the frontend layering and override architecture, see [frontend/editor/DeveloperGuide.md](frontend/editor/DeveloperGuide.md).
```typescript
// ✅ CORRECT - Use @app/* for all imports
@@ -243,7 +243,7 @@ Frontend designed for **stateful document processing**:
- No file reloading between tools - performance critical for large PDFs (up to 100GB+)
#### FileContext - Central State Management
**Location**: `frontend/src/core/contexts/FileContext.tsx`
**Location**: `frontend/editor/src/core/contexts/FileContext.tsx`
- **Active files**: Currently loaded PDFs and their variants
- **Tool navigation**: Current mode (viewer/pageEditor/fileEditor/toolName)
- **Memory management**: PDF document cleanup, blob URL lifecycle, Web Worker management
@@ -268,7 +268,7 @@ Without cleanup: browser crashes with memory leaks.
**Architecture**: Modular hook-based system with clear separation of concerns:
- **useToolOperation** (`frontend/src/core/hooks/tools/shared/useToolOperation.ts`): Main orchestrator hook
- **useToolOperation** (`frontend/editor/src/core/hooks/tools/shared/useToolOperation.ts`): Main orchestrator hook
- Coordinates all tool operations with consistent interface
- Integrates with FileContext for operation tracking
- Handles validation, error handling, and UI state management
@@ -356,7 +356,7 @@ return useToolOperation({
### Frontend Directory Structure
The frontend is organized with a clear separation of concerns:
- **`frontend/src/core/`**: Main application code (shared, production-ready components)
- **`frontend/editor/src/core/`**: Main application code (shared, production-ready components)
- **`core/components/`**: React components organized by feature
- `core/components/tools/`: Individual PDF tool implementations
- `core/components/viewer/`: PDF viewer components
@@ -374,17 +374,17 @@ The frontend is organized with a clear separation of concerns:
- **`core/data/`**: Static data (tool taxonomy, etc.)
- **`core/services/`**: Business logic services (PDF processing, storage, etc.)
- **`frontend/src/desktop/`**: Desktop-specific (Tauri) code
- **`frontend/src/proprietary/`**: Proprietary/licensed features
- **`frontend/src-tauri/`**: Tauri (Rust) native desktop application code
- **`frontend/public/`**: Static assets served directly
- **`frontend/editor/src/desktop/`**: Desktop-specific (Tauri) code
- **`frontend/editor/src/proprietary/`**: Proprietary/licensed features
- **`frontend/editor/src-tauri/`**: Tauri (Rust) native desktop application code
- **`frontend/editor/public/`**: Static assets served directly
- `public/locales/`: Translation JSON files
### Component Architecture
- **Static Assets**: CSS, JS, and resources in `src/main/resources/static/` (legacy) + `frontend/public/` (modern)
- **Static Assets**: CSS, JS, and resources in `src/main/resources/static/` (legacy) + `frontend/editor/public/` (modern)
- **Internationalization**:
- Backend: `messages_*.properties` files
- Frontend: JSON files in `frontend/public/locales/` (converted from .properties)
- Frontend: JSON files in `frontend/editor/public/locales/` (converted from .properties)
- Conversion Script: `scripts/convert_properties_to_json.py`
### Configuration Modes
@@ -409,7 +409,7 @@ The frontend is organized with a clear separation of concerns:
4. **Code Style**: Spotless enforces Google Java Format automatically (`task backend:format`)
5. **Translations**:
- Backend: Use helper scripts in `/scripts` for multi-language updates
- Frontend: Update JSON files in `frontend/public/locales/` or use conversion script
- Frontend: Update JSON files in `frontend/editor/public/locales/` or use conversion script
6. **Documentation**: API docs auto-generated and available at `/swagger-ui/index.html`
## Frontend Architecture Status
@@ -427,11 +427,11 @@ The frontend is organized with a clear separation of concerns:
## Translation Rules
- **CRITICAL**: Always update translations in `en-GB` only, never `en-US`
- Translation files are located in `frontend/public/locales/`
- Translation files are located in `frontend/editor/public/locales/`
## Important Notes
- **Java Version**: Minimum JDK 21, supports and recommends JDK 25
- **Java Version**: Requires JDK 25.
- **Lombok**: Used extensively - ensure IDE plugin is installed
- **File Persistence**:
- **Backend**: Designed to be stateless - files are processed in memory/temp locations only
@@ -458,3 +458,26 @@ The frontend is organized with a clear separation of concerns:
- Confirm approach before making structural changes
- Request guidance on preferences (cross-platform vs specific tools, etc.)
- Verify understanding of requirements before proceeding
## Stack reality check (don't trust LLM training data) <!-- bleeding-edge-stack-note -->
This codebase is on bleeding-edge versions of its core JVM stack: **Spring Boot 4.0.6**,
**Jackson 3 (`tools.jackson`)**, **JDK 21/25 source/target with JDK 25 toolchain**.
All three are *post*-2024 releases and your training corpus is overwhelmingly Spring Boot 2/3 and
Jackson 2 patterns — those patterns will compile, run differently, or hallucinate APIs that no
longer exist.
Before writing or editing Spring / Jackson / JDK code:
1. Open an existing module in `app/core/` or `app/common/` and grep for the actual imports being
used — `import tools.jackson...` not `import com.fasterxml.jackson...`, and the new
`org.springframework.boot` 4.x package layout.
2. If you're not sure whether an API exists in this stack version, **check the source on disk
first** (the dependency JARs are downloaded under `~/.gradle/caches/modules-2/`).
3. Do not silently downgrade a Spring Boot 4 pattern to a Spring Boot 3 equivalent. If something
doesn't work, surface it to the human — don't guess.
Same goes for Jackson 3's API surface (renamed `ObjectMapper` builder methods, new
`tools.jackson.databind` namespace) and JDK 25 preview features. Ground your code in this repo's
actual imports, not what worked three years ago.
+24 -22
View File
@@ -11,7 +11,7 @@ This guide focuses on developing for Stirling 2.0, including both the React fron
**Stirling 2.0** is built using:
**Backend:**
- Spring Boot (Java 21+, JDK 25 recommended)
- Spring Boot (requires JDK 25)
- PDFBox for core PDF operations
- LibreOffice for document conversions
- qpdf for PDF optimization
@@ -45,7 +45,7 @@ This guide focuses on developing for Stirling 2.0, including both the React fron
- [Task](https://taskfile.dev/installation/) — unified command runner (recommended)
- Docker
- Git
- Java JDK 21 or later (JDK 25 recommended)
- Java JDK 25
- Node.js 18+ and npm (required for frontend development)
- Gradle 7.0 or later (Included within the repo)
- [uv](https://docs.astral.sh/uv/) — Python package manager (required for engine development)
@@ -61,7 +61,7 @@ This guide focuses on developing for Stirling 2.0, including both the React fron
cd Stirling-PDF
```
2. Install Docker and JDK 21 (or JDK 25 recommended) if not already installed.
2. Install Docker and JDK 25 if not already installed.
3. Install a recommended Java IDE such as Eclipse, IntelliJ, or VSCode
1. Only VSCode
@@ -128,24 +128,26 @@ See [the frontend README](frontend/README.md#tauri) for detailed build instructi
Stirling-PDF/
├── .github/ # GitHub-specific files (workflows, issue templates)
├── configs/ # Configuration files used by stirling at runtime (generated at runtime)
├── frontend/ # React SPA frontend (Stirling 2.0)
│ ├── src/
│ │ ├── components/ # React components
│ │ ├── tools/ # Tool-specific React components
│ │ ├── hooks/ # Custom React hooks
│ │ ├── services/ # API and utility services
│ │ ├── types/ # TypeScript type definitions
│ │ └── utils/ # Utility functions
├── src-tauri/ # Tauri desktop app configuration
│ │ ├── src/ # Rust backend code
│ │ ├── libs/ # JAR files (generated by build scripts)
│ │ ├── runtime/ # Bundled JRE (generated by build scripts)
│ │ ├── Cargo.toml # Rust dependencies
│ │ └── tauri.conf.json # Tauri configuration
├── public/
│ │ ── locales/ # Internationalization files (JSON)
├── package.json # Frontend dependencies
│ └── vite.config.ts # Vite configuration
├── frontend/ # Frontend workspace (Stirling 2.0)
│ ├── editor/ # PDF editor app (the original React SPA)
│ │ ├── src/
│ │ │ ├── components/ # React components
│ │ ├── tools/ # Tool-specific React components
│ │ │ ├── hooks/ # Custom React hooks
│ │ │ ├── services/ # API and utility services
│ │ │ ├── types/ # TypeScript type definitions
│ │ └── utils/ # Utility functions
│ │ ├── src-tauri/ # Tauri desktop app configuration
│ │ │ ├── src/ # Rust backend code
│ │ │ ├── libs/ # JAR files (generated by build scripts)
│ │ │ ├── runtime/ # Bundled JRE (generated by build scripts)
│ │ │ ├── Cargo.toml # Rust dependencies
│ │ └── tauri.conf.json # Tauri configuration
│ │ ── public/
│ │ └── locales/ # Internationalization files (JSON)
└── vite.config.ts # Vite configuration
│ ├── package.json # Shared workspace dependencies
│ └── eslint.config.mjs # Shared lint config
├── customFiles/ # Custom static files and templates (generated at runtime used to replace existing files)
├── docs/ # Documentation files
├── exampleYmlFiles/ # Example YAML configuration files
@@ -445,7 +447,7 @@ For Stirling 2.0, new features are built as React components:
1. **Create the React Component:**
```typescript
// frontend/src/tools/NewTool.tsx
// frontend/editor/src/tools/NewTool.tsx
import { useState } from 'react';
import { Button, FileInput, Container } from '@mantine/core';
+6
View File
@@ -6,8 +6,14 @@ Portions of this software are licensed as follows:
* All content that resides under the "app/proprietary/" directory of this repository,
if that directory exists, is licensed under the license defined in "app/proprietary/LICENSE".
* All content that resides under the "app/saas/" directory of this repository,
if that directory exists, is licensed under the license defined in "app/saas/LICENSE".
* All content that resides under the "engine/" directory of this repository,
if that directory exists, is licensed under the license defined in "engine/LICENSE".
* "scripts/pymupdf_convert.py", if that file exists, is licensed under the GNU Affero
General Public License v3.0 (or later) as declared in its file header. It is a separate
program invoked as an OS subprocess; its license does not extend to other content in
this repository.
* All content that resides under the "frontend/src/proprietary/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/src/proprietary/LICENSE".
* All content that resides under the "frontend/src/desktop/" directory of this repository,
+21 -4
View File
@@ -3,7 +3,7 @@ version: '3'
output: prefixed
vars:
FIND_FREE_PORT_PS: powershell -NoProfile -File scripts\find-free-port.ps1 -Preferred
FIND_FREE_PORT_PS: powershell -NoProfile -File scripts/find-free-port.ps1
FIND_FREE_PORT_SH: bash scripts/find-free-port.sh
includes:
@@ -45,7 +45,7 @@ tasks:
desc: "Start backend + frontend concurrently on free ports"
vars:
PORTS:
sh: '{{if eq OS "windows"}}{{.FIND_FREE_PORT_PS}} 8080,5173{{else}}{{.FIND_FREE_PORT_SH}} 8080 5173{{end}}'
sh: '{{if eq OS "windows"}}{{.FIND_FREE_PORT_PS}} 8080 5173{{else}}{{.FIND_FREE_PORT_SH}} 8080 5173{{end}}'
BACKEND_PORT: '{{index (splitList "\n" .PORTS) 0}}'
FRONTEND_PORT: '{{index (splitList "\n" .PORTS) 1}}'
deps:
@@ -58,11 +58,28 @@ tasks:
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
OPEN: "true"
dev:saas:
desc: "Start SaaS backend + frontend concurrently on free ports"
vars:
PORTS:
sh: '{{if eq OS "windows"}}{{.FIND_FREE_PORT_PS}} 8080 5173{{else}}{{.FIND_FREE_PORT_SH}} 8080 5173{{end}}'
BACKEND_PORT: '{{index (splitList "\n" .PORTS) 0}}'
FRONTEND_PORT: '{{index (splitList "\n" .PORTS) 1}}'
deps:
- task: backend:dev:saas
vars:
PORT: '{{.BACKEND_PORT}}'
- task: frontend:dev:saas
vars:
PORT: '{{.FRONTEND_PORT}}'
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
OPEN: "true"
dev:all:
desc: "Start backend + frontend + engine concurrently on free ports"
vars:
PORTS:
sh: '{{if eq OS "windows"}}{{.FIND_FREE_PORT_PS}} 8080,5173,5001{{else}}{{.FIND_FREE_PORT_SH}} 8080 5173 5001{{end}}'
sh: '{{if eq OS "windows"}}{{.FIND_FREE_PORT_PS}} 8080 5173 5001{{else}}{{.FIND_FREE_PORT_SH}} 8080 5173 5001{{end}}'
BACKEND_PORT: '{{index (splitList "\n" .PORTS) 0}}'
FRONTEND_PORT: '{{index (splitList "\n" .PORTS) 1}}'
ENGINE_PORT: '{{index (splitList "\n" .PORTS) 2}}'
@@ -74,7 +91,7 @@ tasks:
vars:
PORT: '{{.BACKEND_PORT}}'
AIENGINE_URL: 'http://localhost:{{.ENGINE_PORT}}'
- task: frontend:dev:prototypes
- task: frontend:dev
vars:
PORT: '{{.FRONTEND_PORT}}'
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
+1 -1
View File
@@ -86,7 +86,7 @@ If you're using Tauri's built-in updater feature:
## Configuration Files
### 1. Tauri Configuration (frontend/src-tauri/tauri.conf.json)
### 1. Tauri Configuration (frontend/editor/src-tauri/tauri.conf.json)
The Windows signing configuration is already set up:
+34
View File
@@ -0,0 +1,34 @@
###############################################################################
# Stirling-PDF SaaS environment defaults.
#
# This file is committed and provides non-secret defaults loaded by
# `task backend:dev:saas`. Put real values for secrets (passwords, project
# refs, edge function secrets) in `.env.saas.local` - any variable set there
# takes precedence over what's defined here.
#
# DO NOT commit `.env.saas.local`. Only `.env.saas` is checked in.
###############################################################################
# ---------- Supabase project ----------
# Project reference (the subdomain part of <ref>.supabase.co). Required.
# Set in .env.saas.local.
SAAS_DB_PROJECT_REF=
# Edge function secret used by billing/license rollup calls. Set in .env.saas.local.
SUPABASE_EDGE_FUNCTION_SECRET=
# ---------- Database (saas profile) ----------
# Direct JDBC URL to the Supabase Postgres. Required when running the plain
# `saas` profile (i.e. without `--spring.profiles.include=dev`).
# Example: jdbc:postgresql://db.<project-ref>.supabase.co:5432/postgres
SAAS_DB_URL=
SAAS_DB_USERNAME=postgres
SAAS_DB_PASSWORD=
# ---------- Database (dev profile overrides) ----------
# Used when `--spring.profiles.include=dev` is active. The dev profile
# defaults the URL/username to the shared dev Supabase project, but the
# password must still be provided in .env.saas.local.
SAAS_DEV_DB_URL=
SAAS_DEV_DB_USERNAME=postgres
SAAS_DEV_DB_PASSWORD=
+3
View File
@@ -0,0 +1,3 @@
# Whitelist committed env defaults. `.env.saas.local` (and any other .env*)
# stays ignored via the root .gitignore.
!.env.saas
+4
View File
@@ -44,6 +44,10 @@
"moduleName": ".*",
"moduleLicense": "The MIT License"
},
{
"moduleName": ".*",
"moduleLicense": "MIT-0"
},
{
"moduleName": "com.github.jai-imageio:jai-imageio-core",
"moduleLicense": "LICENSE.txt"
+24
View File
@@ -59,4 +59,28 @@ dependencies {
exclude group: 'org.bouncycastle', module: 'bcprov-jdk15on'
exclude group: 'com.google.code.gson', module: 'gson'
}
api 'com.stirling:jpdfium:1.0.1'
// -PjpdfiumPlatforms=all|<csv of linux-x64,linux-arm64,darwin-x64,darwin-arm64,windows-x64>
def jpdfiumPlatformsProp = (project.findProperty('jpdfiumPlatforms') ?: 'all').toString().trim()
def jpdfiumAllPlatforms = ['linux-x64', 'linux-arm64', 'darwin-x64', 'darwin-arm64', 'windows-x64']
def jpdfiumPlatforms = jpdfiumPlatformsProp == 'all'
? jpdfiumAllPlatforms
: jpdfiumPlatformsProp.split(',').collect { it.trim() }.findAll { it }
def jpdfiumInvalid = jpdfiumPlatforms.findAll { !jpdfiumAllPlatforms.contains(it) }
if (jpdfiumInvalid) {
throw new GradleException("Unknown jpdfiumPlatforms value(s): ${jpdfiumInvalid.join(', ')}. " +
"Valid: ${jpdfiumAllPlatforms.join(', ')} or 'all'.")
}
logger.lifecycle("JPDFium native platforms: ${jpdfiumPlatforms.join(', ')}")
jpdfiumPlatforms.each { platform ->
runtimeOnly "com.stirling:jpdfium-natives-${platform}:1.0.1"
}
// Bucket4j (local in-process token bucket for RateLimitStore default impl)
implementation 'com.bucket4j:bucket4j_jdk17-core:8.19.0'
// ArchUnit: enforces module dependency direction (see ArchitectureTest)
testImplementation 'com.tngtech.archunit:archunit-junit5:1.4.2'
}
@@ -75,8 +75,12 @@ public @interface AutoJobPostMapping {
boolean queueable() default false;
/**
* Relative resource weight (1100) used by the scheduler to prioritise / throttle jobs. Values
* below 1 are clamped to&nbsp;1, values above 100 to&nbsp;100.
* Relative resource weight (1-100). See {@link
* stirling.software.common.enumeration.ResourceWeight} for the standard tiers.
*
* <p>The default is a sentinel ({@link Integer#MIN_VALUE}); {@code
* AutoJobPostMappingWeightTest} fails the build if any endpoint leaves it unset. Runtime
* readers clamp the value into {@code [1, 100]}.
*/
int resourceWeight() default 50;
int resourceWeight() default Integer.MIN_VALUE;
}
@@ -0,0 +1,24 @@
package stirling.software.common.cluster;
/** Health and identity facade for the active cluster backplane. */
public interface ClusterBackplane {
/** Returns {@code true} when the backplane is reachable; used for health endpoints. */
boolean isHealthy();
/** Returns {@code "inprocess"} or {@code "valkey"}. */
String backplaneType();
/** Returns this JVM's stable node id (matches {@code Cluster.resolvedNodeId()}). */
String localNodeId();
/**
* Whether this JVM should run the local {@link
* stirling.software.common.service.TaskManager#cleanupOldJobs()} loop. Distributed backplanes
* own job expiry via their own TTL, so they should override this to return {@code false}.
* Defaults to {@code true} so in-process behavior is preserved without an explicit override.
*/
default boolean shouldRunLocalCleanup() {
return true;
}
}
@@ -0,0 +1,63 @@
package stirling.software.common.cluster;
import org.springframework.context.annotation.Configuration;
import jakarta.annotation.PostConstruct;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.ApplicationProperties.Cluster;
/**
* Validates that cluster mode is internally consistent.
*
* <p>Cluster settings are bound on the central {@link ApplicationProperties} under {@code
* cluster.*}; this class reads {@link ApplicationProperties#getCluster()} and runs guards in {@link
* PostConstruct}. When {@code cluster.enabled=false} (the default) all checks are skipped so a
* single-instance install needs no new config.
*/
@Slf4j
@Configuration
@RequiredArgsConstructor
public class ClusterConfig {
private final ApplicationProperties applicationProperties;
@PostConstruct
void validate() {
Cluster cluster = applicationProperties.getCluster();
if (!cluster.isEnabled()) {
return;
}
String backplane = cluster.getBackplane();
if ("valkey".equalsIgnoreCase(backplane)) {
String url = cluster.getValkey() == null ? null : cluster.getValkey().getUrl();
if (url == null || url.isBlank()) {
throw new IllegalStateException(
"cluster.enabled=true with backplane=valkey requires"
+ " cluster.valkey.url to be set (e.g."
+ " redis://valkey:6379).");
}
} else if ("inprocess".equalsIgnoreCase(backplane)) {
// enabled+inprocess only coordinates the local JVM; cross-node lookups will 410.
log.warn(
"cluster.enabled=true with backplane=inprocess - only the local"
+ " JVM is coordinated. Cross-node lookups and the file proxy will fail."
+ " Use backplane=valkey for real multi-node deployments.");
} else {
// Fail fast on typos like "valky" so Spring doesn't later report a cryptic
// "no ClusterBackplane bean" - the operator-facing error names the bad value.
throw new IllegalStateException(
"cluster.enabled=true with unknown backplane '"
+ backplane
+ "'. Valid values: inprocess | valkey.");
}
log.info(
"Cluster mode enabled (backplane={}, nodeRole={}, nodeId={}).",
backplane,
cluster.resolvedRole(),
cluster.resolvedNodeId());
}
}
@@ -0,0 +1,12 @@
package stirling.software.common.cluster;
import java.time.Instant;
/**
* Snapshot of a peer node as recorded in the {@link InstanceRegistry}.
*
* @param internalAddress {@code host:port} the node listens on for {@code /internal/cluster/**}
* @param role one of {@code WEB}, {@code WORKER}, {@code BOTH}
*/
public record ClusterNode(
String nodeId, String internalAddress, Instant lastHeartbeat, String role) {}
@@ -0,0 +1,21 @@
package stirling.software.common.cluster;
import java.time.Duration;
import java.util.Optional;
/** Cluster-wide mutual exclusion primitive; non-reentrant by contract. */
public interface DistributedLock {
Optional<LockHandle> tryAcquire(String lockKey, Duration leaseTime);
interface LockHandle extends AutoCloseable {
void release();
boolean renew(Duration leaseTime);
@Override
default void close() {
release();
}
}
}
@@ -0,0 +1,45 @@
package stirling.software.common.cluster;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
/** Low-level storage seam for result/job files. */
public interface FileStore {
/** Stored file record. */
record Stored(String fileId, long size) {}
/** Store the given stream and return a generated file id and total bytes written. */
Stored store(InputStream in, String originalName) throws IOException;
/**
* Store the file at {@code source} and return a generated file id and total bytes written.
*
* <p>Default implementation opens {@code source} as a stream and delegates to {@link
* #store(InputStream, String)}. Local-disk implementations should override to use a direct
* file-to-file copy ({@code Files.copy(source, dest)} can use {@code sendfile(2)} on Linux),
* which avoids the two-memory-copy hit of streaming a disk-backed upload through the JVM heap.
*/
default Stored store(Path source, String originalName) throws IOException {
try (InputStream in = Files.newInputStream(source)) {
return store(in, originalName);
}
}
/** Open the stored file for streaming reads. Caller closes. */
InputStream retrieve(String fileId) throws IOException;
/** Load the stored file into a byte array. */
byte[] retrieveBytes(String fileId) throws IOException;
/** Size of the stored file in bytes. */
long size(String fileId) throws IOException;
/** Delete the stored file. Returns true if a file was removed. */
boolean delete(String fileId);
/** Whether the file id exists in the store. */
boolean exists(String fileId);
}
@@ -0,0 +1,18 @@
package stirling.software.common.cluster;
import java.time.Duration;
import java.util.Collection;
import java.util.Optional;
/** Maps {@code nodeId} to its internal cluster address, with TTL'd heartbeats. */
public interface InstanceRegistry {
/** Register or refresh this node. Idempotent so a wiped backplane self-heals on next tick. */
void register(ClusterNode node, Duration heartbeatTtl);
Optional<ClusterNode> lookup(String nodeId);
Collection<ClusterNode> activeNodes();
void deregister(String nodeId);
}
@@ -0,0 +1,24 @@
package stirling.software.common.cluster;
import java.time.Duration;
import java.util.Collection;
import java.util.Optional;
/** Cluster-visible storage for job status and result metadata, with TTL'd entries. */
public interface JobStore {
/** Persist or overwrite a job entry. {@code ttl} sets the lifetime of the entry. */
void put(JobStoreEntry entry, Duration ttl);
Optional<JobStoreEntry> get(String jobId);
void delete(String jobId);
boolean exists(String jobId);
/** Reverse lookup: which job owns this result file id? */
Optional<String> findJobIdByFileId(String fileId);
/** Snapshot of every active entry. Used by admin/stats endpoints; may be O(n). */
Collection<JobStoreEntry> all();
}
@@ -0,0 +1,30 @@
package stirling.software.common.cluster;
import java.time.Instant;
import java.util.List;
import java.util.Map;
/**
* Cluster-visible projection of a job's status and result metadata, as persisted in {@link
* JobStore}.
*
* @param owningNodeId the node id that originally executed the job
*/
public record JobStoreEntry(
String jobId,
JobState state,
String owningNodeId,
Instant createdAt,
Instant completedAt,
String error,
List<String> fileIds,
Map<String, String> resultMeta) {
/** Lifecycle states for a job as observed by the cluster. */
public enum JobState {
PENDING,
RUNNING,
COMPLETE,
FAILED
}
}
@@ -0,0 +1,16 @@
package stirling.software.common.cluster;
import java.time.Duration;
import java.util.Optional;
/** Short-TTL namespaced key/value cache backed by the cluster backplane. */
public interface KeyValueCache {
void put(String namespace, String key, String value, Duration ttl);
Optional<String> get(String namespace, String key);
void evict(String namespace, String key);
void evictNamespace(String namespace);
}
@@ -0,0 +1,18 @@
package stirling.software.common.cluster;
import java.time.Duration;
/** Token-bucket rate limiting backed by the cluster backplane. */
public interface RateLimitStore {
/**
* Attempt to consume one token from the bucket identified by {@code bucketKey}.
*
* @param bucketKey opaque key identifying the bucket (e.g. {@code api:user:123})
* @param capacity bucket capacity
* @param refillPeriod time window over which {@code capacity} tokens refill
*/
RateLimitDecision tryConsume(String bucketKey, long capacity, Duration refillPeriod);
record RateLimitDecision(boolean allowed, long remainingTokens, long nanosToWaitForRefill) {}
}
@@ -0,0 +1,7 @@
package stirling.software.common.cluster;
/** Records one increment per sticky-session miss (a 410 Gone for a job owned by another node). */
@FunctionalInterface
public interface StickyMissRecorder {
void recordStickyMiss();
}
@@ -0,0 +1,31 @@
package stirling.software.common.cluster.inprocess;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.cluster.ClusterBackplane;
import stirling.software.common.model.ApplicationProperties;
@Slf4j
public class InProcessClusterBackplane implements ClusterBackplane {
private final ApplicationProperties applicationProperties;
public InProcessClusterBackplane(ApplicationProperties applicationProperties) {
this.applicationProperties = applicationProperties;
}
@Override
public boolean isHealthy() {
return true;
}
@Override
public String backplaneType() {
return "inprocess";
}
@Override
public String localNodeId() {
return applicationProperties.getCluster().resolvedNodeId();
}
}
@@ -0,0 +1,65 @@
package stirling.software.common.cluster.inprocess;
import org.springframework.boot.autoconfigure.condition.ConditionalOnExpression;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.cluster.ClusterBackplane;
import stirling.software.common.cluster.DistributedLock;
import stirling.software.common.cluster.InstanceRegistry;
import stirling.software.common.cluster.JobStore;
import stirling.software.common.cluster.KeyValueCache;
import stirling.software.common.cluster.RateLimitStore;
import stirling.software.common.model.ApplicationProperties;
/**
* Default cluster backplane wiring: every interface gets an {@code InProcess*} bean. Active when
* cluster mode is off or {@code cluster.backplane=inprocess}.
*/
@Slf4j
@Configuration
@ConditionalOnExpression(
"!${cluster.enabled:false} ||"
+ " '${cluster.backplane:inprocess}'.equalsIgnoreCase('inprocess')")
public class InProcessClusterConfiguration {
@Bean
@ConditionalOnMissingBean
public ClusterBackplane clusterBackplane(ApplicationProperties applicationProperties) {
log.info("Cluster backplane: in-process (single node)");
return new InProcessClusterBackplane(applicationProperties);
}
@Bean
@ConditionalOnMissingBean
public JobStore jobStore() {
return new InProcessJobStore();
}
@Bean
@ConditionalOnMissingBean
public RateLimitStore rateLimitStore() {
return new InProcessRateLimitStore();
}
@Bean
@ConditionalOnMissingBean
public DistributedLock distributedLock() {
return new InProcessDistributedLock();
}
@Bean
@ConditionalOnMissingBean
public KeyValueCache keyValueCache() {
return new InProcessKeyValueCache();
}
@Bean
@ConditionalOnMissingBean
public InstanceRegistry instanceRegistry() {
return new InProcessInstanceRegistry();
}
}
@@ -0,0 +1,128 @@
package stirling.software.common.cluster.inprocess;
import java.time.Duration;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicLong;
import stirling.software.common.cluster.DistributedLock;
/**
* In-process {@link DistributedLock}, non-reentrant per the interface contract, with lease-expiry
* semantics that mirror a SET-NX-EX style distributed backend.
*
* <p>Each lock state carries a per-acquire {@code ownerToken} and an {@code expiryNanos}; another
* caller can take over once the lease has elapsed even if the original holder never called {@link
* LockHandle#release()}. This matters mostly for parity with the Valkey-backed implementation
* (Redis {@code SETEX} auto-expires the key); within a single JVM a crashed holder takes its lock
* state with it, but tests and code that rely on the {@code leaseTime} parameter still need it to
* be honored.
*
* <p>Expiry is lazy: an expired lock state lingers in the map until the next acquire attempt for
* the same key replaces it. Per-key cleanup also happens on explicit {@link LockHandle#release()},
* so a balanced acquire/release workload keeps the map size bounded.
*/
public class InProcessDistributedLock implements DistributedLock {
private final ConcurrentHashMap<String, LockState> locks = new ConcurrentHashMap<>();
private final AtomicLong tokenSeq = new AtomicLong();
/**
* Lease state for a single acquired lock. {@code ownerToken} prevents a former holder from
* releasing or renewing a lock now owned by someone else after lease expiry; {@code
* expiryNanos} is read/written only inside {@link ConcurrentHashMap#compute} so the bin lock
* provides the necessary happens-before guarantee.
*/
private static final class LockState {
final long ownerToken;
long expiryNanos;
LockState(long ownerToken, long expiryNanos) {
this.ownerToken = ownerToken;
this.expiryNanos = expiryNanos;
}
}
@Override
public Optional<LockHandle> tryAcquire(String lockKey, Duration leaseTime) {
long token = tokenSeq.incrementAndGet();
long nowNanos = System.nanoTime();
long expiryNanos = nowNanos + leaseTime.toNanos();
boolean[] acquired = {false};
locks.compute(
lockKey,
(k, existing) -> {
if (existing == null || existing.expiryNanos - nowNanos <= 0L) {
// No lock, or the previous lease has expired - we take it. Subtraction
// form avoids the long-overflow trap that would bite a naive
// expiryNanos <= nowNanos comparison around System.nanoTime() rollover.
acquired[0] = true;
return new LockState(token, expiryNanos);
}
return existing;
});
if (!acquired[0]) {
return Optional.empty();
}
return Optional.of(new InProcessHandle(lockKey, token));
}
private void releaseInternal(String lockKey, long token) {
locks.compute(
lockKey,
(k, existing) -> {
if (existing == null || existing.ownerToken != token) {
// Already removed, expired-and-replaced, or never ours.
return existing;
}
return null;
});
}
private boolean renewInternal(String lockKey, long token, Duration leaseTime) {
long nowNanos = System.nanoTime();
boolean[] renewed = {false};
locks.compute(
lockKey,
(k, existing) -> {
if (existing == null
|| existing.ownerToken != token
|| existing.expiryNanos - nowNanos <= 0L) {
// Lock is gone or expired; renewal is a no-op so the caller can detect it.
return existing;
}
existing.expiryNanos = nowNanos + leaseTime.toNanos();
renewed[0] = true;
return existing;
});
return renewed[0];
}
private final class InProcessHandle implements LockHandle {
private final String lockKey;
private final long token;
private boolean released;
InProcessHandle(String lockKey, long token) {
this.lockKey = lockKey;
this.token = token;
}
@Override
public synchronized void release() {
if (released) {
return;
}
released = true;
releaseInternal(lockKey, token);
}
@Override
public synchronized boolean renew(Duration leaseTime) {
if (released) {
return false;
}
return renewInternal(lockKey, token, leaseTime);
}
}
}
@@ -0,0 +1,42 @@
package stirling.software.common.cluster.inprocess;
import java.time.Duration;
import java.util.Collection;
import java.util.Collections;
import java.util.Optional;
import java.util.concurrent.atomic.AtomicReference;
import stirling.software.common.cluster.ClusterNode;
import stirling.software.common.cluster.InstanceRegistry;
public class InProcessInstanceRegistry implements InstanceRegistry {
private final AtomicReference<ClusterNode> self = new AtomicReference<>();
@Override
public void register(ClusterNode node, Duration heartbeatTtl) {
self.set(node);
}
@Override
public Optional<ClusterNode> lookup(String nodeId) {
ClusterNode current = self.get();
return current != null && current.nodeId().equals(nodeId)
? Optional.of(current)
: Optional.empty();
}
@Override
public Collection<ClusterNode> activeNodes() {
ClusterNode current = self.get();
return current == null ? Collections.emptyList() : Collections.singletonList(current);
}
@Override
public void deregister(String nodeId) {
ClusterNode current = self.get();
if (current != null && current.nodeId().equals(nodeId)) {
self.set(null);
}
}
}
@@ -0,0 +1,96 @@
package stirling.software.common.cluster.inprocess;
import java.time.Duration;
import java.time.Instant;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.cluster.JobStore;
import stirling.software.common.cluster.JobStoreEntry;
@Slf4j
public class InProcessJobStore implements JobStore {
private final ConcurrentHashMap<String, Holder> entries = new ConcurrentHashMap<>();
@Override
public void put(JobStoreEntry entry, Duration ttl) {
Instant expiry = ttl == null ? Instant.MAX : Instant.now().plus(ttl);
entries.put(entry.jobId(), new Holder(entry, expiry));
}
@Override
public Optional<JobStoreEntry> get(String jobId) {
Holder h = entries.get(jobId);
if (h == null) {
return Optional.empty();
}
if (h.isExpired()) {
entries.remove(jobId, h);
return Optional.empty();
}
return Optional.of(h.entry);
}
@Override
public void delete(String jobId) {
entries.remove(jobId);
}
@Override
public boolean exists(String jobId) {
return get(jobId).isPresent();
}
@Override
public Optional<String> findJobIdByFileId(String fileId) {
for (Map.Entry<String, Holder> e : entries.entrySet()) {
Holder h = e.getValue();
if (h.isExpired()) {
continue;
}
List<String> fileIds = h.entry.fileIds();
if (fileIds != null && fileIds.contains(fileId)) {
return Optional.of(e.getKey());
}
}
return Optional.empty();
}
@Override
public Collection<JobStoreEntry> all() {
List<JobStoreEntry> result = new ArrayList<>(entries.size());
for (Holder h : entries.values()) {
if (!h.isExpired()) {
result.add(h.entry);
}
}
return result;
}
/** Drop entries whose TTL has elapsed. Called by the {@code TaskManager} cleanup scheduler. */
public int purgeExpired() {
int removed = 0;
Instant now = Instant.now();
for (Map.Entry<String, Holder> e : entries.entrySet()) {
if (!e.getValue().expiry.equals(Instant.MAX) && e.getValue().expiry.isBefore(now)) {
if (entries.remove(e.getKey(), e.getValue())) {
removed++;
}
}
}
return removed;
}
private record Holder(JobStoreEntry entry, Instant expiry) {
boolean isExpired() {
return !expiry.equals(Instant.MAX) && expiry.isBefore(Instant.now());
}
}
}
@@ -0,0 +1,55 @@
package stirling.software.common.cluster.inprocess;
import java.time.Duration;
import java.time.Instant;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
import stirling.software.common.cluster.KeyValueCache;
public class InProcessKeyValueCache implements KeyValueCache {
private final ConcurrentHashMap<String, ConcurrentHashMap<String, Expiring>> namespaces =
new ConcurrentHashMap<>();
@Override
public void put(String namespace, String key, String value, Duration ttl) {
Instant expiry = ttl == null ? Instant.MAX : Instant.now().plus(ttl);
namespaces
.computeIfAbsent(namespace, n -> new ConcurrentHashMap<>())
.put(key, new Expiring(value, expiry));
}
@Override
public Optional<String> get(String namespace, String key) {
Map<String, Expiring> ns = namespaces.get(namespace);
if (ns == null) {
return Optional.empty();
}
Expiring e = ns.get(key);
if (e == null) {
return Optional.empty();
}
if (e.expiry.isBefore(Instant.now())) {
ns.remove(key, e);
return Optional.empty();
}
return Optional.of(e.value);
}
@Override
public void evict(String namespace, String key) {
Map<String, Expiring> ns = namespaces.get(namespace);
if (ns != null) {
ns.remove(key);
}
}
@Override
public void evictNamespace(String namespace) {
namespaces.remove(namespace);
}
private record Expiring(String value, Instant expiry) {}
}
@@ -0,0 +1,49 @@
package stirling.software.common.cluster.inprocess;
import java.time.Duration;
import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.Map;
import io.github.bucket4j.Bandwidth;
import io.github.bucket4j.Bucket;
import io.github.bucket4j.ConsumptionProbe;
import io.github.bucket4j.local.LocalBucketBuilder;
import stirling.software.common.cluster.RateLimitStore;
/** Bucket4j-backed token bucket implementation of {@link RateLimitStore}. */
public class InProcessRateLimitStore implements RateLimitStore {
/** Cap to bound memory; oldest accessed buckets are evicted. */
private static final int MAX_BUCKETS = 10_000;
private final Map<String, Bucket> buckets =
Collections.synchronizedMap(
new LinkedHashMap<String, Bucket>(256, 0.75f, true) {
@Override
protected boolean removeEldestEntry(Map.Entry<String, Bucket> eldest) {
return size() > MAX_BUCKETS;
}
});
@Override
public RateLimitDecision tryConsume(String bucketKey, long capacity, Duration refillPeriod) {
String compositeKey = bucketKey + "|" + capacity + "|" + refillPeriod.toNanos();
Bucket bucket =
buckets.computeIfAbsent(compositeKey, k -> buildBucket(capacity, refillPeriod));
ConsumptionProbe probe = bucket.tryConsumeAndReturnRemaining(1);
return new RateLimitDecision(
probe.isConsumed(),
probe.getRemainingTokens(),
probe.isConsumed() ? 0L : probe.getNanosToWaitForRefill());
}
private static Bucket buildBucket(long capacity, Duration refillPeriod) {
Bandwidth limit =
Bandwidth.builder().capacity(capacity).refillGreedy(capacity, refillPeriod).build();
LocalBucketBuilder builder = Bucket.builder();
builder.addLimit(limit);
return builder.build();
}
}
@@ -0,0 +1,128 @@
package stirling.software.common.cluster.inprocess;
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.UUID;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.cluster.FileStore;
/** Local-disk {@link FileStore} storing files under a base directory keyed by a UUID file id. */
@Slf4j
public class LocalDiskFileStore implements FileStore {
private final String baseDirPath;
public LocalDiskFileStore(String baseDirPath) {
this.baseDirPath = baseDirPath;
}
@Override
public Stored store(InputStream in, String originalName) throws IOException {
String fileId = UUID.randomUUID().toString();
Path filePath = resolve(fileId);
Files.createDirectories(filePath.getParent());
boolean success = false;
try {
long size = Files.copy(in, filePath);
success = true;
return new Stored(fileId, size);
} finally {
if (!success) {
try {
Files.deleteIfExists(filePath);
} catch (IOException cleanupEx) {
log.warn(
"Failed to clean up partial file {} after store failure",
filePath,
cleanupEx);
}
}
}
}
/**
* File-to-file copy. {@link Files#copy(Path, Path, java.nio.file.CopyOption...)} can use {@code
* sendfile(2)} on Linux for a zero-copy kernel transfer when source and destination share a
* filesystem, avoiding the streaming overhead of pulling the bytes through the JVM heap. Reads
* the source size before copying so the post-copy stat is unnecessary.
*/
@Override
public Stored store(Path source, String originalName) throws IOException {
String fileId = UUID.randomUUID().toString();
Path filePath = resolve(fileId);
Files.createDirectories(filePath.getParent());
long size = Files.size(source);
boolean success = false;
try {
Files.copy(source, filePath);
success = true;
return new Stored(fileId, size);
} finally {
if (!success) {
try {
Files.deleteIfExists(filePath);
} catch (IOException cleanupEx) {
log.warn(
"Failed to clean up partial file {} after store failure",
filePath,
cleanupEx);
}
}
}
}
@Override
public InputStream retrieve(String fileId) throws IOException {
return new BufferedInputStream(Files.newInputStream(resolve(fileId)));
}
@Override
public byte[] retrieveBytes(String fileId) throws IOException {
Path filePath = resolve(fileId);
if (!Files.exists(filePath)) {
throw new IOException("File not found with ID: " + fileId);
}
return Files.readAllBytes(filePath);
}
@Override
public long size(String fileId) throws IOException {
Path filePath = resolve(fileId);
if (!Files.exists(filePath)) {
throw new IOException("File not found with ID: " + fileId);
}
return Files.size(filePath);
}
@Override
public boolean delete(String fileId) {
try {
return Files.deleteIfExists(resolve(fileId));
} catch (IOException e) {
log.error("Error deleting file with ID: {}", fileId, e);
return false;
}
}
@Override
public boolean exists(String fileId) {
return Files.exists(resolve(fileId));
}
public Path resolve(String fileId) {
if (fileId.contains("..") || fileId.contains("/") || fileId.contains("\\")) {
throw new IllegalArgumentException("Invalid file ID");
}
Path basePath = Path.of(baseDirPath).normalize().toAbsolutePath();
Path resolvedPath = basePath.resolve(fileId).normalize();
if (!resolvedPath.startsWith(basePath)) {
throw new IllegalArgumentException("File ID resolves to an invalid path");
}
return resolvedPath;
}
}
@@ -0,0 +1,29 @@
package stirling.software.common.cluster.inprocess;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import stirling.software.common.cluster.FileStore;
/**
* Always-on wiring for the per-node local-disk {@link FileStore}. Active when {@code
* cluster.artifactStore=local} (the default; {@code matchIfMissing=true}). The S3 artifact-store
* supplies its own bean when {@code cluster.artifactStore=s3}.
*/
@Configuration
@ConditionalOnProperty(
prefix = "cluster",
name = "artifactStore",
havingValue = "local",
matchIfMissing = true)
public class LocalDiskFileStoreConfiguration {
@Bean
@ConditionalOnMissingBean
public FileStore fileStore(@Value("${stirling.tempDir:/tmp/stirling-files}") String tempDir) {
return new LocalDiskFileStore(tempDir);
}
}
@@ -80,6 +80,7 @@ public class ConfigInitializer {
YamlHelper settingsFile = new YamlHelper(settingTempPath);
migrateEnterpriseEditionToPremium(settingsFile, settingsTemplateFile);
migrateProFeaturesKeyCasing(settingsFile, settingsTemplateFile);
boolean changesMade =
settingsTemplateFile.updateValuesFromYaml(settingsFile, settingsTemplateFile);
@@ -116,31 +117,52 @@ public class ConfigInitializer {
}
if (yaml.getValueByExactKeyPath("enterpriseEdition", "SSOAutoLogin") != null) {
template.updateValue(
List.of("premium", "proFeatures", "SSOAutoLogin"),
List.of("premium", "proFeatures", "ssoAutoLogin"),
yaml.getValueByExactKeyPath("enterpriseEdition", "SSOAutoLogin"));
}
if (yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "autoUpdateMetadata")
!= null) {
template.updateValue(
List.of("premium", "proFeatures", "CustomMetadata", "autoUpdateMetadata"),
List.of("premium", "proFeatures", "customMetadata", "autoUpdateMetadata"),
yaml.getValueByExactKeyPath(
"enterpriseEdition", "CustomMetadata", "autoUpdateMetadata"));
}
if (yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "author") != null) {
template.updateValue(
List.of("premium", "proFeatures", "CustomMetadata", "author"),
List.of("premium", "proFeatures", "customMetadata", "author"),
yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "author"));
}
if (yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "creator") != null) {
template.updateValue(
List.of("premium", "proFeatures", "CustomMetadata", "creator"),
List.of("premium", "proFeatures", "customMetadata", "creator"),
yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "creator"));
}
if (yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "producer")
!= null) {
template.updateValue(
List.of("premium", "proFeatures", "CustomMetadata", "producer"),
List.of("premium", "proFeatures", "customMetadata", "producer"),
yaml.getValueByExactKeyPath("enterpriseEdition", "CustomMetadata", "producer"));
}
}
// TODO: Remove post migration
// settings.yml.template renamed the two non-camelCase proFeatures keys
// ("SSOAutoLogin" -> "ssoAutoLogin", "CustomMetadata" -> "customMetadata") so the whole
// settings pipeline is consistent camelCase. The save path (YamlHelper.updateValue) matches
// keys case-sensitively, so without this carry-forward an existing install's values written
// under the old PascalCase keys would be dropped on upgrade and reset to template defaults.
void migrateProFeaturesKeyCasing(YamlHelper yaml, YamlHelper template) {
Object ssoAutoLogin = yaml.getValueByExactKeyPath("premium", "proFeatures", "SSOAutoLogin");
if (ssoAutoLogin != null) {
template.updateValue(List.of("premium", "proFeatures", "ssoAutoLogin"), ssoAutoLogin);
}
for (String field : List.of("autoUpdateMetadata", "author", "creator", "producer")) {
Object value =
yaml.getValueByExactKeyPath("premium", "proFeatures", "CustomMetadata", field);
if (value != null) {
template.updateValue(
List.of("premium", "proFeatures", "customMetadata", field), value);
}
}
}
}
@@ -0,0 +1,22 @@
package stirling.software.common.enumeration;
/**
* Standard resource-weight tiers for {@link
* stirling.software.common.annotations.AutoJobPostMapping#resourceWeight()}.
*/
public final class ResourceWeight {
/** Lightweight: rotate, page numbers, extract pages, permissions, etc. */
public static final int SMALL_WEIGHT = 1;
/** Medium: merge, split, multi-tool batch. */
public static final int MEDIUM_WEIGHT = 3;
/** Heavy: compress, OCR (small), conversions, raster. */
public static final int LARGE_WEIGHT = 5;
/** Extra heavy: OCR on large files, full-document re-render, AI-assisted edits. */
public static final int XLARGE_WEIGHT = 10;
private ResourceWeight() {}
}
@@ -13,6 +13,7 @@ import java.util.Arrays;
import java.util.Collection;
import java.util.List;
import java.util.Locale;
import java.util.UUID;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
@@ -77,6 +78,7 @@ public class ApplicationProperties {
private PdfEditor pdfEditor = new PdfEditor();
private AiEngine aiEngine = new AiEngine();
private InternalApi internalApi = new InternalApi();
private Cluster cluster = new Cluster();
@Bean
public PropertySource<?> dynamicYamlPropertySource(ConfigurableEnvironment environment)
@@ -97,7 +99,14 @@ public class ApplicationProperties {
EncodedResource encodedResource = new EncodedResource(resource);
PropertySource<?> propertySource =
new YamlPropertySourceFactory().createPropertySource(null, encodedResource);
environment.getPropertySources().addFirst(propertySource);
boolean saasActive = Arrays.asList(environment.getActiveProfiles()).contains("saas");
if (saasActive) {
// Saas-pinned values in application-saas.properties must beat settings.yml.
environment.getPropertySources().addLast(propertySource);
} else {
environment.getPropertySources().addFirst(propertySource);
}
log.debug("Loaded properties: {}", propertySource.getSource());
@@ -247,6 +256,106 @@ public class ApplicationProperties {
private int longRunningTimeoutSeconds = 600;
}
/**
* Cluster backplane configuration. All keys live under the top-level {@code cluster.*} prefix
* (e.g. env var {@code CLUSTER_ENABLED}). The master switch is {@link #enabled} and defaults to
* off; when off the in-process backplane is wired and no other cluster keys are required.
*/
@Data
public static class Cluster {
/** Master switch. When {@code false} (default) the in-process backplane is wired. */
private boolean enabled = false;
/** Backplane implementation selector. Valid values: {@code inprocess} | {@code valkey}. */
private String backplane = "inprocess";
/**
* Transient cluster job-artifact store selector. Valid values: {@code local} | {@code s3}.
*
* <p>This is distinct from {@code storage.provider}, which selects the backend for
* persistent user-uploaded files. The two switches exist because the user-facing storage
* feature is optional ({@code storage.enabled=false} is common) but every multi-node
* cluster still needs a shared artifact store to serve cross-node downloads. Both
* implementations share credentials from {@code storage.s3.*} when set to {@code s3}.
*/
private String artifactStore = "local";
private Valkey valkey = new Valkey();
private Node node = new Node();
private transient String cachedNodeId;
public NodeRole resolvedRole() {
if (node == null || node.getRole() == null) {
return NodeRole.BOTH;
}
String value = node.getRole().trim().toUpperCase(Locale.ROOT);
try {
return NodeRole.valueOf(value);
} catch (IllegalArgumentException ex) {
return NodeRole.BOTH;
}
}
public synchronized String resolvedNodeId() {
if (node != null && node.getId() != null && !node.getId().isBlank()) {
return node.getId();
}
if (cachedNodeId == null) {
cachedNodeId = UUID.randomUUID().toString();
}
return cachedNodeId;
}
public enum NodeRole {
WEB,
WORKER,
BOTH
}
@Data
public static class Valkey {
/**
* {@code redis://host:6379} or {@code rediss://...} for TLS. Required when cluster mode
* is on and backplane is valkey.
*/
private String url = "";
private Tls tls = new Tls();
@Data
public static class Tls {
/**
* When {@code true}, skip Valkey/Redis TLS certificate verification (dev/test
* only). Leave {@code false} in production.
*/
private boolean skipCertVerification = false;
}
}
@Data
public static class Node {
/** Optional explicit node id. Blank = auto-generated UUID at startup. */
private String id = "";
/** {@code web} | {@code worker} | {@code both}. */
private String role = "both";
/**
* Internal cluster address advertised in the instance registry (host:port). Blank =
* derived at startup.
*/
private String internalAddress = "";
/** {@code http} | {@code https} - scheme used when peers call this node. */
private String scheme = "http";
/** Heartbeat publish interval for the instance registry, in milliseconds. */
private long heartbeatIntervalMs = 5000;
}
}
/**
* HTTP timeouts for loopback calls to internal Stirling API endpoints, used by the AI workflow
* executor and the pipeline processor. A bounded read timeout prevents a hung tool (e.g. an
@@ -419,6 +528,16 @@ public class ApplicationProperties {
private String provider;
private Client client = new Client();
/**
* When true, the OAuth2/OIDC login flow logs the full set of ID token and UserInfo
* claims at INFO level (and again at ERROR level if the username attribute cannot be
* resolved). Used to diagnose provider misconfiguration (for example ADFS not returning
* an {@code email} claim). WARNING: writes PII (sub, email, name) to application logs.
* Leave disabled in production; enable only while actively troubleshooting and disable
* again afterwards.
*/
private Boolean debugLogging = false;
public void setScopes(String scopes) {
List<String> scopesList =
Arrays.stream(scopes.split(",")).map(String::trim).toList();
@@ -669,6 +788,7 @@ public class ApplicationProperties {
private boolean enabled = false;
private String provider = "local";
private Local local = new Local();
private S3 s3 = new S3();
private Quotas quotas = new Quotas();
private Sharing sharing = new Sharing();
private Signing signing = new Signing();
@@ -678,6 +798,57 @@ public class ApplicationProperties {
private String basePath = InstallationPathConfig.getPath() + "storage";
}
@Data
public static class S3 {
/**
* Optional custom endpoint (e.g. {@code https://<account>.r2.cloudflarestorage.com},
* {@code https://<project>.supabase.co/storage/v1/s3}, or {@code http://localhost:9000}
* for MinIO). Blank = use AWS regional default.
*/
private String endpoint = "";
private String bucket = "";
private String region = "us-east-1";
private String accessKey = "";
private String secretKey = "";
/**
* When {@code true} use path-style URLs ({@code <endpoint>/<bucket>/<key>}) instead of
* virtual-hosted ({@code <bucket>.<endpoint>/<key>}). MinIO and most S3-compatible
* gateways require path-style; AWS S3 prefers virtual-hosted.
*/
private boolean pathStyleAccess = false;
/**
* When {@code false} (default), {@code endpoint} hostnames that resolve to private,
* loopback, or link-local addresses are rejected at startup to block SSRF attacks via
* the cloud metadata service (e.g. {@code http://169.254.169.254/}). Set to {@code
* true} to opt in for MinIO / in-cluster S3 endpoints on private networks.
*/
private boolean allowPrivateEndpoints = false;
/**
* Controls when the SDK adds an {@code x-amz-checksum-*} header on PUT/UploadPart.
* Default {@code WHEN_SUPPORTED} (the SDK default since 2.30) makes the SDK send a
* CRC32 checksum on every upload - this works on AWS S3, MinIO, current Supabase,
* Backblaze B2 (post-July-2025), and modern R2. Set to {@code WHEN_REQUIRED} to
* suppress the auto-checksum on vendors that reject unknown {@code x-amz-checksum-*}
* headers (older Backblaze B2, some R2 corner cases, GCS S3 endpoint). Invalid values
* fall back to {@code WHEN_SUPPORTED}.
*/
private String requestChecksumCalculation = "WHEN_SUPPORTED";
/**
* Controls when the SDK validates returned {@code x-amz-checksum-*} headers on GET
* responses. Default {@code WHEN_SUPPORTED}. Set to {@code WHEN_REQUIRED} if your
* vendor never returns these headers and you see false-positive checksum-mismatch
* errors. Invalid values fall back to {@code WHEN_SUPPORTED}.
*/
private String responseChecksumValidation = "WHEN_SUPPORTED";
}
@Data
public static class Sharing {
private boolean enabled = false;
@@ -1179,6 +1350,10 @@ public class ApplicationProperties {
public int getFfmpegSessionLimit() {
return ffmpegSessionLimit > 0 ? ffmpegSessionLimit : 2;
}
public int getPyMuPdfConvertSessionLimit() {
return 2;
}
}
@Data
@@ -1256,6 +1431,10 @@ public class ApplicationProperties {
public long getFfmpegTimeoutMinutes() {
return ffmpegTimeoutMinutes > 0 ? ffmpegTimeoutMinutes : 30;
}
public long getPyMuPdfConvertTimeoutMinutes() {
return 10;
}
}
}
}
@@ -0,0 +1,26 @@
package stirling.software.common.model.enumeration;
import lombok.Getter;
import lombok.RequiredArgsConstructor;
/** Team invitation status */
@Getter
@RequiredArgsConstructor
public enum InvitationStatus {
PENDING("PENDING"),
ACCEPTED("ACCEPTED"),
REJECTED("REJECTED"),
CANCELLED("CANCELLED"),
EXPIRED("EXPIRED");
private final String statusName;
public static InvitationStatus fromString(String statusName) {
for (InvitationStatus status : InvitationStatus.values()) {
if (status.getStatusName().equalsIgnoreCase(statusName)) {
return status;
}
}
throw new IllegalArgumentException("No InvitationStatus defined for name: " + statusName);
}
}
@@ -16,6 +16,9 @@ public enum Role {
// Unlimited access
USER("ROLE_USER", Integer.MAX_VALUE, Integer.MAX_VALUE, "adminUserSettings.user"),
// Paid tier; set by Stripe webhooks under saas mode.
PRO_USER("ROLE_PRO_USER", Integer.MAX_VALUE, Integer.MAX_VALUE, "adminUserSettings.proUser"),
// 40 API calls Per Day, 40 web calls
LIMITED_API_USER("ROLE_LIMITED_API_USER", 40, 40, "adminUserSettings.apiUser"),
@@ -0,0 +1,26 @@
package stirling.software.common.model.enumeration;
import lombok.Getter;
import lombok.RequiredArgsConstructor;
/**
* Team membership roles LEADER: Can invite/remove members, manage settings, view usage, manage
* billing MEMBER: Regular team member with standard access
*/
@Getter
@RequiredArgsConstructor
public enum TeamRole {
LEADER("LEADER"),
MEMBER("MEMBER");
private final String roleName;
public static TeamRole fromString(String roleName) {
for (TeamRole role : TeamRole.values()) {
if (role.getRoleName().equalsIgnoreCase(roleName)) {
return role;
}
}
throw new IllegalArgumentException("No TeamRole defined for name: " + roleName);
}
}
@@ -1,15 +1,13 @@
package stirling.software.common.service;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.UUID;
import java.io.PipedInputStream;
import java.io.PipedOutputStream;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicReference;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.Resource;
import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;
@@ -18,9 +16,11 @@ import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBo
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.cluster.FileStore;
/**
* Service for storing and retrieving files with unique file IDs. Used by the AutoJobPostMapping
* system to handle file references.
* system to handle file references. Disk I/O is delegated to the injected {@link FileStore} bean.
*/
@Service
@RequiredArgsConstructor
@@ -30,251 +30,150 @@ public class FileStorage {
/** Holds the result of a stream-to-disk store operation: the file ID and the bytes written. */
public record StoredFile(String fileId, long size) {}
@Value("${stirling.tempDir:/tmp/stirling-files}")
private String tempDirPath;
private final FileOrUploadService fileOrUploadService;
private final FileStore fileStore;
/**
* Store a file and return its unique ID
*
* @param file The file to store
* @return The unique ID assigned to the file
* @throws IOException If there is an error storing the file
*/
public String storeFile(MultipartFile file) throws IOException {
String fileId = generateFileId();
Path filePath = getFilePath(fileId);
// Ensure the directory exists
Files.createDirectories(filePath.getParent());
// Transfer the file to the storage location
file.transferTo(filePath.toFile());
log.debug("Stored file with ID: {}", fileId);
return fileId;
}
/**
* Store a byte array as a file and return its unique ID
*
* @param bytes The byte array to store
* @param originalName The original name of the file (for extension)
* @return The unique ID assigned to the file
* @throws IOException If there is an error storing the file
*/
public String storeBytes(byte[] bytes, String originalName) throws IOException {
String fileId = generateFileId();
Path filePath = getFilePath(fileId);
// Ensure the directory exists
Files.createDirectories(filePath.getParent());
// Write the bytes to the file
Files.write(filePath, bytes);
log.debug("Stored byte array with ID: {}", fileId);
return fileId;
}
/**
* Retrieve a file by its ID as a MultipartFile
*
* @param fileId The ID of the file to retrieve
* @return The file as a MultipartFile
* @throws IOException If the file doesn't exist or can't be read
*/
public MultipartFile retrieveFile(String fileId) throws IOException {
Path filePath = getFilePath(fileId);
if (!Files.exists(filePath)) {
throw new IOException("File not found with ID: " + fileId);
// Fast path: when Spring buffered the multipart to disk (typical for large uploads), the
// backing Resource exposes a real File. Hand the Path to the FileStore so it can do a
// file-to-file copy (Linux sendfile, no copy through Java heap) rather than streaming
// the bytes through an 8K buffer. Falls back to the InputStream path for in-memory
// multiparts, exotic Resource impls, and anything that does not back onto a File.
Resource res;
try {
res = file.getResource();
} catch (RuntimeException ignored) {
res = null;
}
if (res != null && res.isFile()) {
try {
FileStore.Stored stored =
fileStore.store(res.getFile().toPath(), file.getOriginalFilename());
log.debug("Stored file with ID: {} (fast path)", stored.fileId());
return stored.fileId();
} catch (IOException ex) {
// Some Resource impls advertise isFile()=true but throw on getFile(); fall through.
log.debug("Resource fast path failed, falling back to stream copy", ex);
}
}
try (InputStream in = file.getInputStream()) {
FileStore.Stored stored = fileStore.store(in, file.getOriginalFilename());
log.debug("Stored file with ID: {}", stored.fileId());
return stored.fileId();
}
}
byte[] fileData = Files.readAllBytes(filePath);
public String storeBytes(byte[] bytes, String originalName) throws IOException {
FileStore.Stored stored = fileStore.store(new ByteArrayInputStream(bytes), originalName);
log.debug("Stored byte array with ID: {}", stored.fileId());
return stored.fileId();
}
public MultipartFile retrieveFile(String fileId) throws IOException {
byte[] fileData = fileStore.retrieveBytes(fileId);
return fileOrUploadService.toMockMultipartFile(fileId, fileData);
}
/**
* Retrieve a file by its ID as a byte array
*
* @param fileId The ID of the file to retrieve
* @return The file as a byte array
* @throws IOException If the file doesn't exist or can't be read
*/
public byte[] retrieveBytes(String fileId) throws IOException {
Path filePath = getFilePath(fileId);
if (!Files.exists(filePath)) {
throw new IOException("File not found with ID: " + fileId);
}
return Files.readAllBytes(filePath);
return fileStore.retrieveBytes(fileId);
}
/**
* Retrieve a file by its ID as a streaming InputStream. The caller is responsible for closing
* the returned stream.
*
* @param fileId The ID of the file to retrieve
* @return A buffered InputStream for the file
* @throws IOException If the file doesn't exist or can't be read
*/
public InputStream retrieveInputStream(String fileId) throws IOException {
Path filePath = getFilePath(fileId);
// Let Files.newInputStream throw NoSuchFileException naturally — avoids TOCTOU race
// between exists-check and open when another thread may delete concurrently.
return new BufferedInputStream(Files.newInputStream(filePath));
return fileStore.retrieve(fileId);
}
/**
* Store data from an InputStream as a file and return its unique ID and byte count. Streams
* directly to disk without buffering the entire content in heap.
*
* @param inputStream The input stream to read from
* @param originalName The original name of the file (unused, kept for API symmetry)
* @return A {@link StoredFile} containing the file ID and the number of bytes written
* @throws IOException If there is an error storing the file
*/
public StoredFile storeInputStream(InputStream inputStream, String originalName)
throws IOException {
String fileId = generateFileId();
Path filePath = getFilePath(fileId);
Files.createDirectories(filePath.getParent());
long size = Files.copy(inputStream, filePath);
log.debug("Stored input stream with ID: {}", fileId);
return new StoredFile(fileId, size);
FileStore.Stored stored = fileStore.store(inputStream, originalName);
log.debug("Stored input stream with ID: {}", stored.fileId());
return new StoredFile(stored.fileId(), stored.size());
}
public String storeFromStreamingBody(StreamingResponseBody body, String originalName)
throws IOException {
String fileId = generateFileId();
Path filePath = getFilePath(fileId);
Files.createDirectories(filePath.getParent());
boolean success = false;
try (OutputStream os = new BufferedOutputStream(Files.newOutputStream(filePath))) {
body.writeTo(os);
success = true;
} finally {
if (!success) {
// Hold Throwable not IOException: an unchecked failure (NPE, IllegalState, OOM, etc.)
// from the body writer would otherwise close the pipe with EOF and the consumer would
// return a truncated file with no error surfaced to the caller.
AtomicReference<Throwable> bodyError = new AtomicReference<>();
try (PipedOutputStream out = new PipedOutputStream();
PipedInputStream in = new PipedInputStream(out, 8192)) {
var executor = Executors.newSingleThreadExecutor(Thread.ofVirtual().factory());
java.util.concurrent.Future<?> task = null;
try {
task =
executor.submit(
() -> {
try {
body.writeTo(out);
} catch (Throwable ex) {
bodyError.set(ex);
} finally {
try {
out.close();
} catch (IOException ignored) {
// closed on the consumer side too
}
}
});
FileStore.Stored stored = fileStore.store(in, originalName);
Throwable writerErr = bodyError.get();
if (writerErr != null) {
// Body failed mid-write: the FileStore persisted a truncated entry.
// Best-effort delete so we don't leak partial files; never let cleanup
// mask the original writer error.
try {
fileStore.delete(stored.fileId());
} catch (RuntimeException cleanupEx) {
log.warn(
"Failed to delete partial file {} after writer error: {}",
stored.fileId(),
cleanupEx.getMessage());
}
if (writerErr instanceof IOException ioe) {
throw ioe;
}
throw new IOException(
"StreamingResponseBody writer failed: " + writerErr.getMessage(),
writerErr);
}
log.debug("Stored StreamingResponseBody with ID: {}", stored.fileId());
return stored.fileId();
} finally {
// Interrupt and join the writer task: shutdown() alone returns immediately and a
// failed store leaves the writer running, leaking a thread per failed upload.
if (task != null && !task.isDone()) {
task.cancel(true);
}
executor.shutdown();
try {
Files.deleteIfExists(filePath);
} catch (IOException cleanupEx) {
log.warn(
"Failed to clean up partial file {} after store failure",
filePath,
cleanupEx);
if (!executor.awaitTermination(5, java.util.concurrent.TimeUnit.SECONDS)) {
executor.shutdownNow();
}
} catch (InterruptedException ie) {
executor.shutdownNow();
Thread.currentThread().interrupt();
}
}
}
log.debug("Stored StreamingResponseBody with ID: {}", fileId);
return fileId;
}
/**
* Persist a {@link Resource} body to disk, returning the generated file ID. Used by the async
* job pipeline to capture {@code ResponseEntity<Resource>} results produced by controllers.
*/
public String storeFromResource(Resource resource, String originalName) throws IOException {
String fileId = generateFileId();
Path filePath = getFilePath(fileId);
Files.createDirectories(filePath.getParent());
boolean success = false;
try (InputStream in = resource.getInputStream()) {
Files.copy(in, filePath);
success = true;
} finally {
if (!success) {
try {
Files.deleteIfExists(filePath);
} catch (IOException cleanupEx) {
log.warn(
"Failed to clean up partial file {} after store failure",
filePath,
cleanupEx);
}
}
FileStore.Stored stored = fileStore.store(in, originalName);
log.debug("Stored Resource with ID: {}", stored.fileId());
return stored.fileId();
}
log.debug("Stored Resource with ID: {}", fileId);
return fileId;
}
/**
* Delete a file by its ID
*
* @param fileId The ID of the file to delete
* @return true if the file was deleted, false otherwise
*/
public boolean deleteFile(String fileId) {
try {
Path filePath = getFilePath(fileId);
return Files.deleteIfExists(filePath);
} catch (IOException e) {
log.error("Error deleting file with ID: {}", fileId, e);
return false;
}
return fileStore.delete(fileId);
}
/**
* Check if a file exists by its ID
*
* @param fileId The ID of the file to check
* @return true if the file exists, false otherwise
*/
public boolean fileExists(String fileId) {
Path filePath = getFilePath(fileId);
return Files.exists(filePath);
return fileStore.exists(fileId);
}
/**
* Get the size of a file by its ID without loading the content into memory
*
* @param fileId The ID of the file
* @return The size of the file in bytes
* @throws IOException If the file doesn't exist or can't be read
*/
public long getFileSize(String fileId) throws IOException {
Path filePath = getFilePath(fileId);
if (!Files.exists(filePath)) {
throw new IOException("File not found with ID: " + fileId);
}
return Files.size(filePath);
}
/**
* Get the path for a file ID
*
* @param fileId The ID of the file
* @return The path to the file
* @throws IllegalArgumentException if fileId contains path traversal characters or resolves
* outside base directory
*/
private Path getFilePath(String fileId) {
// Validate fileId to prevent path traversal
if (fileId.contains("..") || fileId.contains("/") || fileId.contains("\\")) {
throw new IllegalArgumentException("Invalid file ID");
}
Path basePath = Path.of(tempDirPath).normalize().toAbsolutePath();
Path resolvedPath = basePath.resolve(fileId).normalize();
// Ensure resolved path is within the base directory
if (!resolvedPath.startsWith(basePath)) {
throw new IllegalArgumentException("File ID resolves to an invalid path");
}
return resolvedPath;
}
/**
* Generate a unique file ID
*
* @return A unique file ID
*/
private String generateFileId() {
return UUID.randomUUID().toString();
return fileStore.size(fileId);
}
}
@@ -0,0 +1,91 @@
package stirling.software.common.service;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import java.util.Locale;
import java.util.concurrent.TimeUnit;
import org.springframework.stereotype.Service;
import jakarta.annotation.PostConstruct;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.util.ProcessExecutor;
/**
* Converts PDFs to Markdown by invoking the {@code pymupdf-convert} CLI tool as a separate
* subprocess.
*/
@Slf4j
@Service
public class PyMuPdfConverter {
private boolean available;
@PostConstruct
void init() {
available = probe();
if (available) {
log.info("pymupdf-convert found — PyMuPDF Markdown conversion enabled.");
} else {
log.info("pymupdf-convert not found — PyMuPDF Markdown conversion disabled.");
}
}
public boolean isAvailable() {
return available;
}
private boolean probe() {
boolean isWindows =
System.getProperty("os.name").toLowerCase(Locale.ROOT).contains("windows");
List<String> cmd =
isWindows
? List.of("where", "pymupdf-convert")
: List.of("which", "pymupdf-convert");
try {
Process p = new ProcessBuilder(cmd).redirectErrorStream(true).start();
boolean done = p.waitFor(5, TimeUnit.SECONDS);
return done && p.exitValue() == 0;
} catch (Exception e) {
log.debug("pymupdf-convert availability check failed: {}", e.getMessage());
return false;
}
}
/**
* Convert a PDF to Markdown by invoking {@code pymupdf-convert} as a subprocess.
*
* @throws IOException on process failure or if the tool is not installed
*/
public String convertToMarkdown(byte[] pdfBytes, String filename) throws IOException {
String safeName =
(filename == null || filename.isBlank())
? "document.pdf"
: filename.replace("\"", "");
Path tempDir = Files.createTempDirectory("stirling-pymupdf-");
Path inputPdf = tempDir.resolve(safeName);
Path outputMd = tempDir.resolve("output.md");
try {
Files.write(inputPdf, pdfBytes);
ProcessExecutor.getInstance(ProcessExecutor.Processes.PYMUPDF_CONVERT)
.runCommandWithOutputHandling(
List.of(
"pymupdf-convert",
inputPdf.toAbsolutePath().toString(),
outputMd.toAbsolutePath().toString()));
return Files.readString(outputMd, StandardCharsets.UTF_8);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new IOException("PyMuPDF conversion interrupted", e);
} finally {
Files.deleteIfExists(inputPdf);
Files.deleteIfExists(outputMd);
Files.deleteIfExists(tempDir);
}
}
}
@@ -3,9 +3,13 @@ package stirling.software.common.service;
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.time.Duration;
import java.time.Instant;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
@@ -16,6 +20,7 @@ import java.util.concurrent.TimeUnit;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.MediaType;
import org.springframework.stereotype.Service;
@@ -26,6 +31,10 @@ import jakarta.annotation.PreDestroy;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.cluster.ClusterBackplane;
import stirling.software.common.cluster.JobStore;
import stirling.software.common.cluster.JobStoreEntry;
import stirling.software.common.cluster.JobStoreEntry.JobState;
import stirling.software.common.model.job.JobResult;
import stirling.software.common.model.job.JobStats;
import stirling.software.common.model.job.ResultFile;
@@ -40,20 +49,20 @@ public class TaskManager {
private int jobResultExpiryMinutes = 30;
private final FileStorage fileStorage;
private final JobStore jobStore;
private final ClusterBackplane clusterBackplane;
private final ScheduledExecutorService cleanupExecutor =
Executors.newSingleThreadScheduledExecutor(
Thread.ofVirtual().name("task-cleanup-", 0).factory());
/** Initialize the task manager and start the cleanup scheduler */
public TaskManager(FileStorage fileStorage) {
@Autowired
public TaskManager(
FileStorage fileStorage, JobStore jobStore, ClusterBackplane clusterBackplane) {
this.fileStorage = fileStorage;
this.jobStore = jobStore;
this.clusterBackplane = clusterBackplane;
// Schedule periodic cleanup of old job results
cleanupExecutor.scheduleAtFixedRate(
this::cleanupOldJobs,
10, // Initial delay
10, // Interval
TimeUnit.MINUTES);
cleanupExecutor.scheduleAtFixedRate(this::cleanupOldJobs, 10, 10, TimeUnit.MINUTES);
log.debug(
"Task manager initialized with job result expiry of {} minutes",
@@ -66,7 +75,9 @@ public class TaskManager {
* @param jobId The job ID
*/
public void createTask(String jobId) {
jobResults.put(jobId, JobResult.createNew(jobId));
JobResult result = JobResult.createNew(jobId);
jobResults.put(jobId, result);
writeThrough(jobId, result);
log.debug("Created task with job ID: {}", jobId);
}
@@ -79,6 +90,7 @@ public class TaskManager {
public void setResult(String jobId, Object result) {
JobResult jobResult = getOrCreateJobResult(jobId);
jobResult.completeWithResult(result);
writeThrough(jobId, jobResult);
log.debug("Set result for job ID: {}", jobId);
}
@@ -101,6 +113,7 @@ public class TaskManager {
extractZipToIndividualFiles(fileId, originalFileName);
if (!extractedFiles.isEmpty()) {
jobResult.completeWithFiles(extractedFiles);
writeThrough(jobId, jobResult);
log.debug(
"Set multiple file results for job ID: {} with {} files extracted from"
+ " ZIP",
@@ -127,6 +140,7 @@ public class TaskManager {
"Failed to get file size for job {}: {}. Using size 0.", jobId, e.getMessage());
jobResult.completeWithSingleFile(fileId, originalFileName, contentType, 0);
}
writeThrough(jobId, jobResult);
}
/**
@@ -138,6 +152,7 @@ public class TaskManager {
public void setMultipleFileResults(String jobId, List<ResultFile> resultFiles) {
JobResult jobResult = getOrCreateJobResult(jobId);
jobResult.completeWithFiles(resultFiles);
writeThrough(jobId, jobResult);
log.debug(
"Set multiple file results for job ID: {} with {} files",
jobId,
@@ -153,6 +168,7 @@ public class TaskManager {
public void setError(String jobId, String error) {
JobResult jobResult = getOrCreateJobResult(jobId);
jobResult.failWithError(error);
writeThrough(jobId, jobResult);
log.debug("Set error for job ID: {}: {}", jobId, error);
}
@@ -169,6 +185,7 @@ public class TaskManager {
// If no result or error has been set, mark it as complete with an empty result
jobResult.completeWithResult("Task completed successfully");
}
writeThrough(jobId, jobResult);
log.debug("Marked job ID: {} as complete", jobId);
}
@@ -205,6 +222,7 @@ public class TaskManager {
JobResult jobResult = jobResults.get(jobId);
if (jobResult != null) {
jobResult.addNote(note);
writeThrough(jobId, jobResult);
log.debug("Added note to job ID: {}: {}", jobId, note);
return true;
}
@@ -295,8 +313,11 @@ public class TaskManager {
return jobResults.computeIfAbsent(jobId, JobResult::createNew);
}
/** Clean up old completed job results */
/** Clean up old completed job results. No-op in cluster mode; the backplane TTL owns expiry. */
public void cleanupOldJobs() {
if (clusterBackplane != null && !clusterBackplane.shouldRunLocalCleanup()) {
return;
}
LocalDateTime expiryThreshold =
LocalDateTime.now().minus(jobResultExpiryMinutes, ChronoUnit.MINUTES);
int removedCount = 0;
@@ -315,6 +336,9 @@ public class TaskManager {
// Remove the job result
jobResults.remove(entry.getKey());
if (jobStore != null) {
jobStore.delete(entry.getKey());
}
removedCount++;
}
}
@@ -327,6 +351,53 @@ public class TaskManager {
}
}
/** Mirror the in-memory {@code JobResult} into the cluster-visible {@link JobStore}. */
private void writeThrough(String jobId, JobResult result) {
if (jobStore == null) {
return;
}
try {
jobStore.put(toEntry(jobId, result), Duration.ofMinutes(jobResultExpiryMinutes));
} catch (RuntimeException ex) {
log.warn("JobStore write-through failed for job {}: {}", jobId, ex.getMessage());
}
}
private JobStoreEntry toEntry(String jobId, JobResult result) {
JobState state;
if (result.isComplete()) {
state = result.getError() != null ? JobState.FAILED : JobState.COMPLETE;
} else {
state = JobState.PENDING;
}
Instant createdAt = toInstant(result.getCreatedAt());
Instant completedAt = toInstant(result.getCompletedAt());
List<String> fileIds = new ArrayList<>();
if (result.hasFiles()) {
for (ResultFile rf : result.getAllResultFiles()) {
fileIds.add(rf.getFileId());
}
}
Map<String, String> meta = new HashMap<>();
if (result.getNotes() != null && !result.getNotes().isEmpty()) {
meta.put("notesCount", Integer.toString(result.getNotes().size()));
}
String owningNodeId = clusterBackplane == null ? "local" : clusterBackplane.localNodeId();
return new JobStoreEntry(
jobId,
state,
owningNodeId,
createdAt,
completedAt,
result.getError(),
fileIds,
meta);
}
private Instant toInstant(LocalDateTime ldt) {
return ldt == null ? null : ldt.atZone(ZoneId.systemDefault()).toInstant();
}
/** Shutdown the cleanup executor */
@PreDestroy
public void shutdown() {
@@ -370,7 +441,7 @@ public class TaskManager {
while ((entry = zipIn.getNextEntry()) != null) {
if (!entry.isDirectory()) {
String contentType = determineContentType(entry.getName());
// storeInputStream returns the fileId and byte count no extra stat needed
// storeInputStream returns the fileId and byte count - no extra stat needed
FileStorage.StoredFile stored =
fileStorage.storeInputStream(zipIn, entry.getName());
@@ -458,7 +529,8 @@ public class TaskManager {
}
/**
* Find the job key that owns a given file ID.
* Find the job key that owns a given file ID. Checks the local in-memory map first, then falls
* back to the cluster-visible {@link JobStore}.
*
* @param fileId file identifier to look up
* @return scoped job key if found, otherwise null
@@ -474,6 +546,18 @@ public class TaskManager {
}
}
}
if (jobStore != null) {
// Propagate JobStore failures: returning null on a backplane outage would conflate
// "no such file" with "lookup unavailable" and the caller would respond 404 to a
// transient blip that should be retried. Let Spring's exception handler surface a
// 5xx so clients know to retry.
try {
return jobStore.findJobIdByFileId(fileId).orElse(null);
} catch (RuntimeException e) {
log.warn("JobStore findJobIdByFileId failed for {}: {}", fileId, e.getMessage());
throw e;
}
}
return null;
}
}
@@ -4,6 +4,7 @@ import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
@@ -30,6 +31,7 @@ import io.github.pixee.security.Filenames;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.configuration.RuntimePathConfig;
import stirling.software.common.service.PyMuPdfConverter;
import stirling.software.common.util.ProcessExecutor.ProcessExecutorResult;
@Slf4j
@@ -159,6 +161,57 @@ public class PDFToFile {
finalOut, fileName, MediaType.APPLICATION_OCTET_STREAM);
}
/**
* PDF-&gt;Markdown with optional PyMuPDF acceleration.
*
* <p>When {@code pymupdf-convert} is installed and on PATH, conversion is delegated to it as a
* subprocess. On any failure — or when the tool is absent — this transparently falls back to
* the bundled {@code pdftohtml}-based converter.
*/
public ResponseEntity<Resource> processPdfToMarkdown(
MultipartFile inputFile, PyMuPdfConverter pyMuPdfConverter)
throws IOException, InterruptedException {
if (!MediaType.APPLICATION_PDF_VALUE.equals(inputFile.getContentType())) {
return new ResponseEntity<>(HttpStatus.BAD_REQUEST);
}
if (pyMuPdfConverter != null && pyMuPdfConverter.isAvailable()) {
try {
String originalName = Filenames.toSimpleFileName(inputFile.getOriginalFilename());
String baseName = originalName;
if (originalName != null && originalName.contains(".")) {
baseName = originalName.substring(0, originalName.lastIndexOf('.'));
}
String markdown =
pyMuPdfConverter.convertToMarkdown(inputFile.getBytes(), originalName);
return buildMarkdownZipResponse(markdown, baseName);
} catch (IOException e) {
log.warn(
"PyMuPDF conversion failed; falling back to pdftohtml converter: {}",
e.getMessage());
}
}
return processPdfToMarkdown(inputFile);
}
private ResponseEntity<Resource> buildMarkdownZipResponse(String markdown, String pdfBaseName)
throws IOException {
String fileName = pdfBaseName + "ToMarkdown.zip";
TempFile finalOut = tempFileManager.createManagedTempFile(".zip");
try (OutputStream fos = Files.newOutputStream(finalOut.getPath());
ZipOutputStream zipOutputStream = new ZipOutputStream(fos)) {
ZipEntry mdEntry = new ZipEntry(pdfBaseName + ".md");
zipOutputStream.putNextEntry(mdEntry);
zipOutputStream.write(markdown.getBytes(StandardCharsets.UTF_8));
zipOutputStream.closeEntry();
} catch (Exception e) {
finalOut.close();
throw e;
}
return WebResponseUtils.fileToWebResponse(
finalOut, fileName, MediaType.APPLICATION_OCTET_STREAM);
}
/**
* Updates image references in markdown to point to the images/ folder. Matches patterns like
* ![alt](filename.png) and converts to ![alt](images/filename.png)
@@ -115,6 +115,11 @@ public class ProcessExecutor {
.getProcessExecutor()
.getSessionLimit()
.getFfmpegSessionLimit();
case PYMUPDF_CONVERT ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getPyMuPdfConvertSessionLimit();
};
long timeoutMinutes =
@@ -180,6 +185,11 @@ public class ProcessExecutor {
.getProcessExecutor()
.getTimeoutMinutes()
.getFfmpegTimeoutMinutes();
case PYMUPDF_CONVERT ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getPyMuPdfConvertTimeoutMinutes();
};
return new ProcessExecutor(
processType, semaphoreLimit, liveUpdates, timeoutMinutes);
@@ -550,7 +560,8 @@ public class ProcessExecutor {
GHOSTSCRIPT,
OCR_MY_PDF,
CFF_CONVERTER,
FFMPEG
FFMPEG,
PYMUPDF_CONVERT
}
@Setter
@@ -83,7 +83,16 @@ public class RequestUriUtils {
return false;
}
// Blocklist of backend/non-frontend paths that should still go through filters
// Blocklist of backend/non-frontend paths that should still go through filters.
//
// `/files` was historically a backend route; it is now a frontend route
// owned by HomePage / FileManagerView. Direct-nav or refresh on /files
// (or /files/<folder-uuid>) was returning the Spring auth filter's 401
// JSON instead of serving index.html, so the SPA never got a chance to
// mount and the user saw a raw error response. There are no `/files`
// backend mappings at the servlet root - the real storage endpoints
// live under `/api/v1/storage/files`, which is filtered out a few lines
// up by the `startsWith("/api/")` guard.
String[] backendOnlyPrefixes = {
"/register",
"/pipeline",
@@ -91,7 +100,6 @@ public class RequestUriUtils {
"/pdfjs-legacy",
"/fonts",
"/images",
"/files",
"/css",
"/js",
"/swagger",
@@ -172,7 +180,7 @@ public class RequestUriUtils {
"/api/v1/ui-data/footer-info") // Public footer configuration
|| trimmedUri.startsWith("/api/v1/invite/validate")
|| trimmedUri.startsWith("/api/v1/invite/accept")
// Health Endoints
// Health Endpoints
|| trimmedUri.startsWith("/actuator/health")
|| trimmedUri.startsWith("/health")
|| trimmedUri.startsWith("/healthz")
@@ -181,7 +189,7 @@ public class RequestUriUtils {
|| trimmedUri.startsWith(
"/api/v1/mobile-scanner/") // Mobile scanner endpoints (no auth)
|| trimmedUri.startsWith("/v1/api-docs")
// Workflow participant endpoints access controlled by share tokens, not login
// Workflow participant endpoints - access controlled by share tokens, not login
|| trimmedUri.startsWith("/api/v1/workflow/participant/")
// Share-link SPA bootstrap; data APIs remain protected
|| trimmedUri.matches("^/share/[^/]+/?$");
@@ -0,0 +1,72 @@
package stirling.software.common.architecture;
import static com.tngtech.archunit.lang.syntax.ArchRuleDefinition.noClasses;
import org.junit.jupiter.api.Test;
import com.tngtech.archunit.core.domain.JavaClasses;
import com.tngtech.archunit.core.importer.ClassFileImporter;
import com.tngtech.archunit.core.importer.ImportOption;
import com.tngtech.archunit.lang.ArchRule;
/**
* Module dependency-direction guardrails. Allowed direction: {@code saas → proprietary → common}
* and {@code stirling-pdf → proprietary → common}.
*/
class ArchitectureTest {
private static final JavaClasses commonClasses =
new ClassFileImporter()
.withImportOption(ImportOption.Predefined.DO_NOT_INCLUDE_JARS)
.withImportOption(ImportOption.Predefined.DO_NOT_INCLUDE_TESTS)
.importPackages("stirling.software.common");
@Test
void commonDoesNotDependOnCore() {
ArchRule rule =
noClasses()
.that()
.resideInAPackage("stirling.software.common..")
.should()
.dependOnClassesThat()
.resideInAPackage("stirling.software.SPDF..");
rule.check(commonClasses);
}
@Test
void commonDoesNotDependOnProprietary() {
ArchRule rule =
noClasses()
.that()
.resideInAPackage("stirling.software.common..")
.should()
.dependOnClassesThat()
.resideInAPackage("stirling.software.proprietary..");
rule.check(commonClasses);
}
@Test
void commonDoesNotDependOnSaas() {
ArchRule rule =
noClasses()
.that()
.resideInAPackage("stirling.software.common..")
.should()
.dependOnClassesThat()
.resideInAPackage("stirling.software.saas..");
rule.check(commonClasses);
}
@Test
void clusterInterfacesHaveNoImplementationDependencies() {
ArchRule rule =
noClasses()
.that()
.resideInAPackage("stirling.software.common.cluster..")
.should()
.dependOnClassesThat()
.resideInAnyPackage(
"stirling.software.proprietary..", "stirling.software.saas..");
rule.check(commonClasses);
}
}
@@ -0,0 +1,51 @@
package stirling.software.common.cluster;
import static org.junit.jupiter.api.Assertions.assertEquals;
import java.time.Instant;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
class BackplaneContractCompilationTest {
@Test
void jobStoreEntryRecordRoundTrips() {
Instant now = Instant.now();
JobStoreEntry entry =
new JobStoreEntry(
"job-1",
JobStoreEntry.JobState.PENDING,
"node-a",
now,
null,
null,
List.of("file-1"),
Map.of("k", "v"));
assertEquals("job-1", entry.jobId());
assertEquals(JobStoreEntry.JobState.PENDING, entry.state());
assertEquals("node-a", entry.owningNodeId());
assertEquals(now, entry.createdAt());
assertEquals(List.of("file-1"), entry.fileIds());
assertEquals("v", entry.resultMeta().get("k"));
}
@Test
void clusterNodeRecordRoundTrips() {
Instant heartbeat = Instant.now();
ClusterNode node = new ClusterNode("node-a", "10.0.0.1:8080", heartbeat, "BOTH");
assertEquals("node-a", node.nodeId());
assertEquals("10.0.0.1:8080", node.internalAddress());
assertEquals(heartbeat, node.lastHeartbeat());
assertEquals("BOTH", node.role());
}
@Test
void rateLimitDecisionRecordRoundTrips() {
RateLimitStore.RateLimitDecision d = new RateLimitStore.RateLimitDecision(true, 7, 0L);
assertEquals(true, d.allowed());
assertEquals(7, d.remainingTokens());
assertEquals(0L, d.nanosToWaitForRefill());
}
}
@@ -0,0 +1,65 @@
package stirling.software.common.cluster;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertThrows;
import java.lang.reflect.Method;
import org.junit.jupiter.api.Test;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.ApplicationProperties.Cluster;
class ClusterConfigValidationTest {
@Test
void validationPassesWhenDisabled() {
ApplicationProperties props = new ApplicationProperties();
ClusterConfig config = new ClusterConfig(props);
assertDoesNotThrow(() -> invokeValidate(config));
}
@Test
void validationFailsWhenValkeyEnabledWithoutUrl() {
ApplicationProperties props = new ApplicationProperties();
Cluster cluster = props.getCluster();
cluster.setEnabled(true);
cluster.setBackplane("valkey");
ClusterConfig config = new ClusterConfig(props);
assertThrows(IllegalStateException.class, () -> invokeValidate(config));
}
@Test
void validationPassesWhenValkeyEnabledWithUrl() {
ApplicationProperties props = new ApplicationProperties();
Cluster cluster = props.getCluster();
cluster.setEnabled(true);
cluster.setBackplane("valkey");
cluster.getValkey().setUrl("redis://localhost:6379");
ClusterConfig config = new ClusterConfig(props);
assertDoesNotThrow(() -> invokeValidate(config));
}
@Test
void validationPassesWhenInProcessEnabled() {
ApplicationProperties props = new ApplicationProperties();
Cluster cluster = props.getCluster();
cluster.setEnabled(true);
cluster.setBackplane("inprocess");
ClusterConfig config = new ClusterConfig(props);
assertDoesNotThrow(() -> invokeValidate(config));
}
private void invokeValidate(ClusterConfig config) throws Exception {
Method m = ClusterConfig.class.getDeclaredMethod("validate");
m.setAccessible(true);
try {
m.invoke(config);
} catch (java.lang.reflect.InvocationTargetException ex) {
if (ex.getCause() instanceof RuntimeException re) {
throw re;
}
throw ex;
}
}
}
@@ -0,0 +1,62 @@
package stirling.software.common.cluster;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import org.junit.jupiter.api.Test;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.ApplicationProperties.Cluster;
class ClusterPropertiesTest {
@Test
void defaultsAreDisabledAndInprocess() {
Cluster props = new ApplicationProperties().getCluster();
assertFalse(props.isEnabled());
assertEquals("inprocess", props.getBackplane());
assertEquals("local", props.getArtifactStore());
assertEquals(Cluster.NodeRole.BOTH, props.resolvedRole());
assertEquals("", props.getValkey().getUrl());
assertFalse(props.getValkey().getTls().isSkipCertVerification());
assertEquals("both", props.getNode().getRole());
assertEquals("http", props.getNode().getScheme());
assertEquals(5000L, props.getNode().getHeartbeatIntervalMs());
}
@Test
void resolvedRoleParsesCaseInsensitively() {
Cluster props = new ApplicationProperties().getCluster();
props.getNode().setRole("WEB");
assertEquals(Cluster.NodeRole.WEB, props.resolvedRole());
props.getNode().setRole("web");
assertEquals(Cluster.NodeRole.WEB, props.resolvedRole());
props.getNode().setRole("Worker");
assertEquals(Cluster.NodeRole.WORKER, props.resolvedRole());
props.getNode().setRole("garbage");
assertEquals(Cluster.NodeRole.BOTH, props.resolvedRole());
props.getNode().setRole(null);
assertEquals(Cluster.NodeRole.BOTH, props.resolvedRole());
}
@Test
void resolvedNodeIdIsStableAcrossCalls() {
Cluster props = new ApplicationProperties().getCluster();
String first = props.resolvedNodeId();
String second = props.resolvedNodeId();
assertNotNull(first);
assertEquals(first, second);
}
@Test
void resolvedNodeIdHonoursExplicitId() {
Cluster props = new ApplicationProperties().getCluster();
props.getNode().setId("abc");
assertEquals("abc", props.resolvedNodeId());
}
}
@@ -0,0 +1,90 @@
package stirling.software.common.cluster;
import static org.assertj.core.api.Assertions.assertThat;
import org.junit.jupiter.api.Test;
import org.springframework.boot.autoconfigure.context.PropertyPlaceholderAutoConfiguration;
import org.springframework.boot.test.context.runner.ApplicationContextRunner;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import stirling.software.common.cluster.inprocess.InProcessClusterConfiguration;
import stirling.software.common.model.ApplicationProperties;
/**
* Verifies the {@link InProcessClusterConfiguration} conditional wiring: in-process beans wire when
* cluster mode is off or {@code backplane=inprocess}, and are skipped when {@code
* backplane=valkey}.
*/
class InProcessConfigurationConditionalTest {
private final ApplicationContextRunner runner =
new ApplicationContextRunner()
.withConfiguration(
org.springframework.boot.autoconfigure.AutoConfigurations.of(
PropertyPlaceholderAutoConfiguration.class))
.withUserConfiguration(
TestAppPropertiesConfig.class,
ClusterConfig.class,
InProcessClusterConfiguration.class);
@Test
void inProcessBeansWireWhenClusterDisabled() {
runner.run(
context ->
assertThat(context)
.hasNotFailed()
.hasSingleBean(ClusterBackplane.class)
.hasSingleBean(JobStore.class)
.hasSingleBean(RateLimitStore.class)
.hasSingleBean(DistributedLock.class)
.hasSingleBean(KeyValueCache.class)
.hasSingleBean(InstanceRegistry.class));
}
@Test
void inProcessBeansWireWhenEnabledWithInProcessBackplane() {
runner.withPropertyValues("cluster.enabled=true", "cluster.backplane=inprocess")
.run(
context ->
assertThat(context)
.hasNotFailed()
.hasSingleBean(ClusterBackplane.class)
.hasSingleBean(JobStore.class)
.hasSingleBean(RateLimitStore.class)
.hasSingleBean(DistributedLock.class)
.hasSingleBean(KeyValueCache.class)
.hasSingleBean(InstanceRegistry.class));
}
@Test
void inProcessBeansSkippedWhenEnabledWithDistributedBackplane() {
runner.withPropertyValues(
"cluster.enabled=true",
"cluster.backplane=valkey",
"cluster.valkey.url=redis://localhost:6379")
.run(
context ->
assertThat(context)
.hasNotFailed()
.doesNotHaveBean(ClusterBackplane.class)
.doesNotHaveBean(JobStore.class)
.doesNotHaveBean(RateLimitStore.class)
.doesNotHaveBean(DistributedLock.class)
.doesNotHaveBean(KeyValueCache.class)
.doesNotHaveBean(InstanceRegistry.class));
}
/**
* Hand-rolled {@link ApplicationProperties} bean: the production class loads YAML at startup
* via a {@code @PostConstruct} hook that isn't appropriate for the slice runner, so we wire a
* defaults-only instance here.
*/
@Configuration
static class TestAppPropertiesConfig {
@Bean
ApplicationProperties applicationProperties() {
return new ApplicationProperties();
}
}
}
@@ -0,0 +1,167 @@
package stirling.software.common.cluster.inprocess;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.time.Duration;
import java.util.Optional;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import org.junit.jupiter.api.Test;
import stirling.software.common.cluster.DistributedLock;
class InProcessDistributedLockTest {
@Test
void acquireReleaseAcquire() {
DistributedLock lock = new InProcessDistributedLock();
DistributedLock.LockHandle h1 = lock.tryAcquire("k", Duration.ofSeconds(30)).orElseThrow();
h1.release();
assertTrue(lock.tryAcquire("k", Duration.ofSeconds(30)).isPresent());
}
@Test
void reentryFromSameThreadFails() {
DistributedLock lock = new InProcessDistributedLock();
DistributedLock.LockHandle h1 = lock.tryAcquire("k", Duration.ofSeconds(30)).orElseThrow();
Optional<DistributedLock.LockHandle> reentry = lock.tryAcquire("k", Duration.ofSeconds(30));
assertFalse(reentry.isPresent(), "in-process lock must be non-reentrant");
h1.release();
// After release, anyone can acquire again.
assertTrue(lock.tryAcquire("k", Duration.ofSeconds(30)).isPresent());
}
@Test
void secondAcquireFromAnotherThreadFails() throws InterruptedException {
DistributedLock lock = new InProcessDistributedLock();
DistributedLock.LockHandle h1 = lock.tryAcquire("k", Duration.ofSeconds(30)).orElseThrow();
CountDownLatch done = new CountDownLatch(1);
AtomicBoolean acquired = new AtomicBoolean(true);
Thread t =
new Thread(
() -> {
Optional<DistributedLock.LockHandle> attempt =
lock.tryAcquire("k", Duration.ofSeconds(30));
acquired.set(attempt.isPresent());
attempt.ifPresent(DistributedLock.LockHandle::release);
done.countDown();
});
t.start();
assertTrue(done.await(2, TimeUnit.SECONDS));
assertFalse(acquired.get());
h1.release();
}
@Test
void leaseExpiryAllowsTakeoverEvenWithoutRelease() throws InterruptedException {
// Acquire with a short lease, never call release, then try to acquire again after the
// lease has elapsed. Matches Redis SET-NX-EX semantics - the second caller gets the lock
// because the first lease auto-expired. 250ms lease + 350ms wait gives CI generous slack.
DistributedLock lock = new InProcessDistributedLock();
DistributedLock.LockHandle h1 = lock.tryAcquire("k", Duration.ofMillis(250)).orElseThrow();
Thread.sleep(350);
Optional<DistributedLock.LockHandle> takeover =
lock.tryAcquire("k", Duration.ofSeconds(30));
assertTrue(
takeover.isPresent(),
"expired lease must release the lock so a new caller can take over");
// Calling release() on the original handle after takeover must be a no-op (token check).
h1.release();
// The takeover holder is still the legitimate owner.
assertFalse(lock.tryAcquire("k", Duration.ofSeconds(30)).isPresent());
takeover.get().release();
}
@Test
void renewExtendsLease() throws InterruptedException {
// Acquire with a short lease, renew it before it expires, then verify the lock is still
// held past the original expiry point. 200ms initial + renew to 2s + wait 350ms.
DistributedLock lock = new InProcessDistributedLock();
DistributedLock.LockHandle h1 = lock.tryAcquire("k", Duration.ofMillis(200)).orElseThrow();
assertTrue(h1.renew(Duration.ofSeconds(2)), "renew on a held lease must succeed");
Thread.sleep(350);
assertFalse(
lock.tryAcquire("k", Duration.ofSeconds(30)).isPresent(),
"renew should have pushed expiry well past the original 200ms");
h1.release();
}
@Test
void renewAfterReleaseFails() {
DistributedLock lock = new InProcessDistributedLock();
DistributedLock.LockHandle h1 = lock.tryAcquire("k", Duration.ofSeconds(30)).orElseThrow();
h1.release();
assertFalse(h1.renew(Duration.ofSeconds(30)), "renew on a released handle must fail");
}
/**
* Concurrency stress: many threads contending on the same key with each holder respecting the
* lease (hold &lt;&lt; lease). The lock behaves as a strict mutex in this regime so asserting
* mutual exclusion is meaningful. A separate test ({@link
* #leaseExpiryAllowsTakeoverEvenWithoutRelease}) covers the takeover-across-expiry branch,
* which legitimately allows two holders momentarily and is split-brain behaviour inherent to
* any lease-based lock.
*/
@Test
void concurrentContentionPreservesMutualExclusion() throws InterruptedException {
DistributedLock lock = new InProcessDistributedLock();
int threads = 16;
int attemptsPerThread = 200;
// Lease far exceeds any plausible hold time, so the takeover branch never triggers in
// this test and the lock acts as a strict mutex.
Duration lease = Duration.ofSeconds(5);
java.util.concurrent.atomic.AtomicInteger concurrentHolders =
new java.util.concurrent.atomic.AtomicInteger();
java.util.concurrent.atomic.AtomicInteger maxConcurrent =
new java.util.concurrent.atomic.AtomicInteger();
java.util.concurrent.atomic.AtomicInteger acquires =
new java.util.concurrent.atomic.AtomicInteger();
java.util.concurrent.atomic.AtomicReference<Throwable> firstFailure =
new java.util.concurrent.atomic.AtomicReference<>();
CountDownLatch start = new CountDownLatch(1);
CountDownLatch done = new CountDownLatch(threads);
for (int i = 0; i < threads; i++) {
new Thread(
() -> {
try {
start.await();
for (int j = 0; j < attemptsPerThread; j++) {
Optional<DistributedLock.LockHandle> h =
lock.tryAcquire("hot", lease);
if (h.isPresent()) {
int now = concurrentHolders.incrementAndGet();
maxConcurrent.accumulateAndGet(now, Math::max);
acquires.incrementAndGet();
// Trivial critical section; well within lease.
concurrentHolders.decrementAndGet();
h.get().release();
}
}
} catch (Throwable t) {
firstFailure.compareAndSet(null, t);
} finally {
done.countDown();
}
},
"lock-stress-" + i)
.start();
}
start.countDown();
assertTrue(done.await(30, TimeUnit.SECONDS), "stress workers must finish in time");
org.junit.jupiter.api.Assertions.assertNull(firstFailure.get(), "no worker may throw");
org.junit.jupiter.api.Assertions.assertEquals(
1,
maxConcurrent.get(),
"mutual exclusion violated: more than one holder observed simultaneously");
assertTrue(
acquires.get() > 0,
"at least some acquires must succeed under contention (saw "
+ acquires.get()
+ ")");
}
}
@@ -0,0 +1,28 @@
package stirling.software.common.cluster.inprocess;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.time.Duration;
import java.time.Instant;
import org.junit.jupiter.api.Test;
import stirling.software.common.cluster.ClusterNode;
class InProcessInstanceRegistryTest {
@Test
void registerThenLookupAndActiveNodes() {
InProcessInstanceRegistry registry = new InProcessInstanceRegistry();
ClusterNode node = new ClusterNode("node-1", "127.0.0.1:8080", Instant.now(), "BOTH");
registry.register(node, Duration.ofSeconds(30));
assertTrue(registry.lookup("node-1").isPresent());
assertEquals("node-1", registry.lookup("node-1").get().nodeId());
assertEquals(1, registry.activeNodes().size());
registry.deregister("node-1");
assertTrue(registry.lookup("node-1").isEmpty());
}
}
@@ -0,0 +1,91 @@
package stirling.software.common.cluster.inprocess;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.time.Duration;
import java.time.Instant;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
import stirling.software.common.cluster.JobStoreEntry;
class InProcessJobStoreTest {
private final InProcessJobStore store = new InProcessJobStore();
@Test
void putGetDeleteExistsRoundTrip() {
JobStoreEntry entry = entry("job-1");
store.put(entry, Duration.ofMinutes(30));
assertTrue(store.exists("job-1"));
assertEquals(entry, store.get("job-1").orElseThrow());
store.delete("job-1");
assertFalse(store.exists("job-1"));
}
@Test
void ttlExpiry() throws InterruptedException {
store.put(entry("job-2"), Duration.ofMillis(50));
Thread.sleep(100);
assertFalse(store.get("job-2").isPresent());
}
@Test
void purgeExpiredRemovesOnlyStaleEntries() throws InterruptedException {
store.put(entry("job-fresh"), Duration.ofMinutes(30));
store.put(entry("job-stale"), Duration.ofMillis(20));
Thread.sleep(80);
int removed = store.purgeExpired();
assertEquals(1, removed);
assertTrue(store.exists("job-fresh"));
}
@Test
void findJobIdByFileIdReturnsTheRightJob() {
store.put(
new JobStoreEntry(
"job-a",
JobStoreEntry.JobState.COMPLETE,
"node-1",
Instant.now(),
Instant.now(),
null,
List.of("file-1", "file-2"),
Map.of()),
Duration.ofMinutes(30));
store.put(
new JobStoreEntry(
"job-b",
JobStoreEntry.JobState.COMPLETE,
"node-1",
Instant.now(),
Instant.now(),
null,
List.of("file-3"),
Map.of()),
Duration.ofMinutes(30));
assertEquals("job-a", store.findJobIdByFileId("file-1").orElseThrow());
assertEquals("job-b", store.findJobIdByFileId("file-3").orElseThrow());
assertFalse(store.findJobIdByFileId("missing").isPresent());
}
private JobStoreEntry entry(String id) {
return new JobStoreEntry(
id,
JobStoreEntry.JobState.PENDING,
"node-1",
Instant.now(),
null,
null,
List.of(),
Map.of());
}
}

Some files were not shown because too many files have changed in this diff Show More